RCDB/CCDB Ad Hoc Meeting

April 3, 2018

Present: Dmitry, Sean, Thomas, MMI

  • D. wanted to plan roadmap for future development
  • T.: Javascript API
    • Proposed in run browser context
    • D: RCDB Python API should do it
    • May want to scrap Python, run only once, supply dynamic parts via Javascript
    • Python tries to cache RCDB info, that takes a long time.
    • This is a ticked in RCDB, Thomas will review it
  • CCDB Issues on GitHub
    • D.: some have been implemented, just not on the master
    • They may be on branch 1.07
    • S: v1.06.02 reports itself as 1.07, log reporting is wrong
    • Important issues according to S.
      1. log report improvements
      2. deprecated tables
      3. blame: who put in constants
    • Mark added: 4. ancestry date control
      • D.: feature coded, control levers not there
  • High-level analysis results in a timeline
    • T.: Eugene brought this up at morning RC meeting
    • Two options
      1. put into online monitoring database (OMDB, maintained by Sean)
      2. put into RCDB
    • Time-series webpage that uses OMDB not useable at present.
    • OMDB designed for this purpose
    • If in RCDB, new features could be applied to this function
  • Development issues for RCDB, from D.
    • API for extracting info: need tutorial on new best practices
    • Improve usage of features for storing/retrieving data in/from RCDB
      • review methods in DAQ scripts
      • config files stored in RCDB: parsers available to get desired fields
      • better use of aliases where appropriate
  • New features: produce ROOT trees from RCDB (CCDB?)

 

CCDB, Lustre, and SQLite

Did a test where I copied a CCDB SQLite file from the group disk to the volatile disk (Lustre) and from there to a local disk on my Linux box (non-Lustre). I tried the ccdb -i command, and gave the “ls” command inside of that. It failed on Lustre as expected:

CCDB provider unable to connect to sqlite:////volatile/halld/home/marki/ccdb.sqlite. Aborting command. Exception details: (sqlite3.OperationalError) database is locked [SQL: u'SELECT "schemaVersions"."schemaVersion" AS "schemaVersions_schemaVersion", "schemaVersions".id AS "schemaVersions_id" \nFROM "schemaVersions"\n LIMIT ? OFFSET ?'] [parameters: (1, 0)]

but it succeeded from the original location on the group disk and on my local disk. I conclude that a file passing through Lustre during its lifetime does not ruin it. This is inconsistent with Elton’s experience.

 

CCDB Feature Discussion with Harut

In the course of discussing feature requests for  CCDB, two fundamental issues were raised. These need to addressed at high priority.

Time-Independent Inheritance

Presently, when a variation inherits assignments from a parent, those assignments can change if the parent variation changes. The inheritance chain needs to be time-stamped. One only inherits the assignments of a parent at a particular time.

This would involve changing the schema to store a timestamp with the name of the parent variation when variations are created. Note that this automatically solves Issue 27.

Copy Command

We need to have a command to “copy” assignments from one variation to another. The target variation should use the same assignments for a particular run range and constant type as the source variation.

Drop in Number of halld Group Members

Cron <gluex@jlabl1> /home/gluex/bin/ccdb_update_users.sh

jobs run at midnight
• 10/01: 179 -> 179, no change, earliest message in Trash folder
• 11/17: 179 -> 185
• 11/25: 185 -> 186
• 12/04: 186 -> 187
• 12/05: 187 -> 175
• $CCDB_HOME/scripts/users_create/group_parse.pl gives 174+1 = 175 group members on 12/5
• no record before 10/01, AFAIK
• see 202 users in ccdb database

Update, 12/7:

Dmitry doesn’t know why this happened. Sent message to Helpdesk.