CCDB, Lustre, and SQLite

Did a test where I copied a CCDB SQLite file from the group disk to the volatile disk (Lustre) and from there to a local disk on my Linux box (non-Lustre). I tried the ccdb -i command, and gave the “ls” command inside of that. It failed on Lustre as expected:

CCDB provider unable to connect to sqlite:////volatile/halld/home/marki/ccdb.sqlite. Aborting command. Exception details: (sqlite3.OperationalError) database is locked [SQL: u'SELECT "schemaVersions"."schemaVersion" AS "schemaVersions_schemaVersion", "schemaVersions".id AS "schemaVersions_id" \nFROM "schemaVersions"\n LIMIT ? OFFSET ?'] [parameters: (1, 0)]

but it succeeded from the original location on the group disk and on my local disk. I conclude that a file passing through Lustre during its lifetime does not ruin it. This is inconsistent with Elton’s experience.


CCDB Feature Discussion with Harut

In the course of discussing feature requests for  CCDB, two fundamental issues were raised. These need to addressed at high priority.

Time-Independent Inheritance

Presently, when a variation inherits assignments from a parent, those assignments can change if the parent variation changes. The inheritance chain needs to be time-stamped. One only inherits the assignments of a parent at a particular time.

This would involve changing the schema to store a timestamp with the name of the parent variation when variations are created. Note that this automatically solves Issue 27.

Copy Command

We need to have a command to “copy” assignments from one variation to another. The target variation should use the same assignments for a particular run range and constant type as the source variation.

Drop in Number of halld Group Members

Cron <gluex@jlabl1> /home/gluex/bin/

jobs run at midnight
• 10/01: 179 -> 179, no change, earliest message in Trash folder
• 11/17: 179 -> 185
• 11/25: 185 -> 186
• 12/04: 186 -> 187
• 12/05: 187 -> 175
• $CCDB_HOME/scripts/users_create/ gives 174+1 = 175 group members on 12/5
• no record before 10/01, AFAIK
• see 202 users in ccdb database

Update, 12/7:

Dmitry doesn’t know why this happened. Sent message to Helpdesk.


have build scripts use native ccdb set-up[?]

  • can’t do setup if ccdb not there
  • setup will fail if ccdb is not there, even for use
  • green field problem:
    • gluex install could download and unpack ccdb first, perhaps even build
      • take a peek at the makefile, see if this is possible
      • not easy to grab ccdb in the middle of environment setup
  • copy scripts from ccdb, check consistency?
    • would give false inconsistencies
  • Just do the copy, do not check consistency
    • problem with where script resides?
  • need a check that the environment set-up is the same between master branch of both
  • it’s a can of worms, leave things alone for now