unit-test

ccdb clone, jlab repo, v1-main branch, built with scons, environment set up with version set pointing to this ccdb, running test_ccdb_lib, get this error:

Error [1040]: in [bool MySQLDataProvider::Connect(MySQLConnectionInfo)] mysql_real_connect() failed:
Error 1044 (Access denied for user 'ccdb_user'@'localhost' to database 'ccdb_test')

Why is it asking for ccdb_test? That does not exist on localhost and must fail as a result.
- Because ccdb_test is hard-wired. See below.
CCDB_CONNECTION=mysql://ccdb_user@hallddb.jlab.org/ccdb. Why is this not respected?
Running in debugger, see that CATCH-supplied main is being used. CATCH_CONFIG_MAIN is defined in test.cc in src/Tests.
How does CATCH know what tests to perform if it is supplying its own main? The TEST_CASE macro takes a string and generates a test function and code to invoke it (self-registers).
In debugging session following from main down to function under test requires drilling down several layers, all with internal code (to catch) that is inscrutable. One way around that is to break in the function under test. Observation from playing with the example (see following bullet).
Added a directory to the sweg repo with the example from v2 of the Catch GitHub site.
The following commits are on the v1-main branch:

commit 2af40fd507906139899c88b6d9cf6b85d424d540
Author: Mark Ito <marki@jlab.org>
Date:   Thu Apr 15 13:56:34 2021 -0400

    Add a test to get column vector from an assignment.

commit e99e3e526125f51ba4cf5e20186a81ff7721397f
Author: Mark Ito <marki@jlab.org>
Date:   Thu Apr 15 12:08:51 2021 -0400

    Fix addition of column info. Passes unit tests.

commit 062bccd807804255ad1c764202500ca3637275d6
Author: Mark Ito <marki@jlab.org>
Date:   Wed Apr 14 19:55:14 2021 -0400

    add columns loaded flag to cache key

Here is the error from the run of test_ccdb_lib:

[marki@markdesk5 bin]$ ./test_ccdb_lib 
Error [1040]: in [bool MySQLDataProvider::Connect(MySQLConnectionInfo)] mysql_real_connect() failed:
Error 1044 (Access denied for user 'ccdb_user'@'localhost' to database 'ccdb_test')
 
Error [1040]: in [bool MySQLDataProvider::Connect(MySQLConnectionInfo)] mysql_real_connect() failed:
Error 1044 (Access denied for user 'ccdb_user'@'localhost' to database 'ccdb_test')
 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_ccdb_lib is a Catch v1.2.1 host application.
Run with -? for options

-------------------------------------------------------------------------------
CCDB/MySQLDataProvider/Connection
-------------------------------------------------------------------------------
src/Tests/test_MySQLProvider_Connection.cc:20
...............................................................................

src/Tests/test_MySQLProvider_Connection.cc:28: FAILED:
  REQUIRE( prov->Connect("mysql://ccdb_user@127.0.0.1:3306/ccdb_test") )
with expansion:
  false

Error [1040]: in [bool MySQLDataProvider::Connect(MySQLConnectionInfo)] mysql_real_connect() failed:
Error 1044 (Access denied for user 'ccdb_user'@'localhost' to database 'ccdb_test')
 
Error [1040]: in [bool MySQLDataProvider::Connect(MySQLConnectionInfo)] mysql_real_connect() failed:
Error 1044 (Access denied for user 'ccdb_user'@'localhost' to database 'ccdb_test')
 
Error [1040]: in [bool MySQLDataProvider::Connect(MySQLConnectionInfo)] mysql_real_connect() failed:
Error 1044 (Access denied for user 'ccdb_user'@'localhost' to database 'ccdb_test')
 
Error [1040]: in [bool MySQLDataProvider::Connect(MySQLConnectionInfo)] mysql_real_connect() failed:
Error 1044 (Access denied for user 'ccdb_user'@'localhost' to database 'ccdb_test')
 
===============================================================================
test cases:  23 |  22 passed | 1 failed
assertions: 239 | 238 passed | 1 failed

So the test is hard-wired to use 127.0.0.1 and the ccdb_test database.

ccdb_test database

try creating a ccdb_test database on localhost
- as root: “grant all on ccdb_test.* to ccdb_user@localhost;”
- “create database ccdb_test;”
- from command line: “mysql -u ccdb_user ccdb_test < ccdb.mysql.sql” where ccdb.mysql.sql comes from the sql directory.
Now the error is:

[marki@markdesk5 sql]$ test_ccdb_lib
Error [1060]: in [ccdb::MySQLDataProvider::QuerySelect()] mysql_query() failed:
Error 1146 (Table 'ccdb_test.directories' doesn't exist)

So that sql snippet is not meant running the test.
Try making a copy of the main ccdb database
- as ccdb_user, “drop database ccdb_test;”
- “create database ccdb_test”
- from command line “mysql -u ccdb_user ccdb_test < ccdb1.sql” where ccdb1.sql is a recent dump found in $SCR
Much better, but still no go. The error is:

[marki@markdesk5 scratch]$ test_ccdb_lib

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_ccdb_lib is a Catch v1.2.1 host application.
Run with -? for options

-------------------------------------------------------------------------------
CCDB/MySQLDataProvider/Assignments
-------------------------------------------------------------------------------
src/Tests/test_MySQLProvider_Assignments.cc:20
...............................................................................

src/Tests/test_MySQLProvider_Assignments.cc:44: FAILED:
  REQUIRE( tabeled_values[0][0] == "2.2" )
with expansion:
  "4" == "2.2"

===============================================================================
test cases:  23 |  22 passed | 1 failed
assertions: 285 | 284 passed | 1 failed

So data fetched does not match data expected.
Remember this from before. So this is for the /test/test_vars/test_table table. The possible constant sets are:

MariaDB [ccdb_test]> select * from constantSets where constantTypeId = 1;
+--------+---------------------+---------------------+------------------------------------------+----------------+
| id     | created             | modified            | vault                                    | constantTypeId |
+--------+---------------------+---------------------+------------------------------------------+----------------+
|      1 | 2012-07-30 23:48:42 | 2012-07-30 23:48:42 | 1.11|1.991211|10.002|2.001|2.9912|20.111 |              1 |
|      2 | 2012-08-30 23:48:42 | 2012-08-30 23:48:42 | 1.0|2.0|3.0|4.0|5.0|6.0                  |              1 |
|      4 | 2012-10-30 23:48:42 | 2012-10-30 23:48:42 | 2.2|2.3|2.4|2.5|2.6|2.7                  |              1 |
|      5 | 2012-11-30 23:48:43 | 2012-11-30 23:48:43 | 10|11|12|13|14|15                        |              1 |
|   4206 | 2015-04-17 18:02:44 | 2015-04-17 18:02:44 | 1|2|3|4|5|6                              |              1 |
|   7546 | 2015-08-05 11:26:32 | 2015-08-05 11:26:32 | 1.|2.|3.|2.|3.|4.                        |              1 |
|  23033 | 2017-02-07 16:24:25 | 2017-02-07 16:24:25 | 10|2|30|40|5|60                          |              1 |
|  23034 | 2017-02-07 16:26:41 | 2017-02-07 16:26:41 | 10|2|30|40|5|60                          |              1 |
|  64581 | 2017-04-19 16:30:12 | 2017-04-19 16:30:12 | 10|2|30|40|5|61                          |              1 |
|  67781 | 2017-04-24 10:59:37 | 2017-04-24 10:59:37 | 1|2|3|2|3|4                              |              1 |
|  98441 | 2018-01-30 13:03:53 | 2018-01-30 13:03:53 | 1.|1.|0.|0.|1.|2.                        |              1 |
| 153962 | 2018-10-31 17:47:16 | 2018-10-31 17:47:16 | 1.|1.|6.|4.|3.|2.                        |              1 |
| 153963 | 2018-10-31 17:48:01 | 2018-10-31 17:48:01 | 1.|1.|0.|0.|1.|2.                        |              1 |
| 209548 | 2019-04-22 08:39:44 | 2019-04-22 08:39:44 | 4|6|8|10|12|13                           |              1 |
| 209712 | 2019-04-22 09:52:52 | 2019-04-22 09:52:52 | 4|6|8|10|12|13                           |              1 |
+--------+---------------------+---------------------+------------------------------------------+----------------+
15 rows in set (0.000 sec)

so we need to use constant set 4. To do this peel back layers of history by deleting assignments until we get to the assignment on 2012-10-30 as the latest. Do this by hand on the mysql command line. For example:

MariaDB [ccdb_test]> delete from assignments where constantSetId = 209712;
Query OK, 1 row affected (0.002 sec)

After this, success:

[marki@markdesk5 ccdb]$ test_ccdb_lib
===============================================================================
All tests passed (292 assertions in 23 test cases)

• make VERBOSE=1 to get CMake-generated makefiles to show commands as they are run
• working in /scratch/ccdb2 on markdesk5
• Why does jana not build?
∘ jana patches got corrupted, probably with a git merge
• conversion from mysql to sqlite does not work on markdesk5, ccdb 1 nor ccdb 2, native OS or singularity container for centos7
∘ errors like: "Error: near line 8791585: no such table: constantSets"
∘ lots of them, for all tables

submitted pull request with minor mod to convert script ccdb 1 to 2
Submitted new issue reporting failure of unit tests for C++
To convert ccdb 1 to ccdb 2 in MariaDB:
- dump ccdb 1 database
- load the dump as ccdb2
- run the commands in ccdb/sql/update_1.00_2.00.mysql.sql on the ccdb2 database
- this is all scripted in ccdb/sql/convert_ccdb_1_to_2.sh
To convert ccdb 1 or 2 to sqlite from MariaDB (copied example from /home/gluex/bin/ccdb_sqlite_create.sh):
- $CCDB_HOME/scripts/mysql2sqlite/mysql2sqlite.sh -hhallddb -uccdb_user ccdb | sqlite3 $sqlite_file
Submitted issue on ccdb -i failure with ccdb 2 on Fedora 33.
Submitted issue on ccdb -i failure with ccdb 1.06.07 on Fedora 33.
ccdb 1 will build on Fedora 33 because it uses the on-board version of scons, based on python 2

Mark's Notebook

CCDB Unit Testing Notes

ccdb_test database

Notes on CCDB 2 Conversion

Running Unit Tests on CCDB