More on CentOS 8 Generalization

Notes

• build_scripts branches:
  • python_chooser: uses the python_chooser.py script rather than python_version.mk, currently a pending pull request
    • tested on ifarm1901 with a complete build
  • python_chooser_no_adjustment: branch off python_chooser without the .python_adjustment step in the makefiles for hdds, halld_recon, and halld_sim
    • tested on fedora32 with the pasteurize_mmi branches (see below) in /scratch/python_chooser_no_adjustment
      • here pasteurization is necessary because there is no adjustment
    • tested on ifarm1901 with default version.xml
      • here python files are not pasteurized, but unadjusted versions should work
• pasteurize_mmi branches:
  • exist for hdds, halld_recon, and halld_sim
  • do not have any *3.* scripts, depends on pasteurization
  • cannot submit pull request for these three packages for general use until python_chooser_no_adjustment branch of build_scripts is installed

To Do

• pasteurize halld_sim
  • pasteurize_mmi
• pasteurize build_scripts
• make a plan for using pasteurized versions everywhere
• confirm that the centos8 still builds under centos8
• wait until after collaboration meeting
  • merge in python_chooser pull request
  • merge in master, and then submit python_chooser_no_adjustment_centos8_fedora33 pull request
  • tag new release of build_scripts and install
  • rebase hdds, halld_sim, halld_recon pasteurize_mmi branches and hdgeant4 input_python_info_mmi branch and submit pull requests
• test a CentOS 8 build using python_chooser_no_adjustment branch of build_scripts and pasteurize_mmi branches of hdds, halld_recon, and halld_sim

Generalizing the CentOS 8 Build

• to get going on centos 7:
  • use centos8 branch for build scripts
  • use version_centos8.xml
  • build on ifarm1901
  • see what blows up: it is evio trying to use scons-3, which does not exist on centos 7. stop here.
• use python_chooser branch of build_scripts
  • build on centos 8
  • python_chooser branch of build_scripts starts with master configuration
  • must redo changes made on centos8 branch to put them on python_chooser branch, but solution known by comparing master branch with centos8 branch. solution will be different in cases where python_chooser.sh can be used.
  • work on ifarm1901 in /scratch/marki/c8_4
    • singularity shell –cleanenv –bind /scratch $DIST/gluex_centos-8.2.2004.sif
• There are three options for implementing the three versions of SCons-related files that are needed (which can be thought of as CentOS 7, Fedora 32, and CentOS 8):
  • 1. Implement three versions of the files where necessary. For example sbms2.py, sbms3a.py, and sbms3b.py. Here sbms.py is actually not in repository, need to make link to make real one that is used. Disadvantage is that any change needs to be made in three places.
  • 2. Write the scripts with C-preprocessor-like commands to keep all versions in a single file. Example of command that creates a combined file of P2 and P3 source with merge markings:

diff -e SConstruct SConstruct3 | grep c$ | sed s/c/d/ > ed_script.tmp
echo w SConstruct.gap >> ed_script.tmp
echo q >> ed_script.tmp
ed SConstruct < ed_script.tmp
diff3 -m SConstruct SConstruct.gap SConstruct3 > SConstruct.raw

• (“three options” continued)
  • 2 (continued). This does not actually work with the C-preprocessor because of changes to the spacing. This is a no-go with the indentation-has-meaning paradigm of Python. There might be a way to do this with m4, but did not get to an understanding of an m4 solution.
  • 3. Futurize the Python code. Actually pasteurize it.
• Pasteurization is the way forward!
  • pasteurize the Python 3 version of the files and give them the nominal file name, i.e., no multiple versions of the same file, i.e., no sbms2.py, no sbms3.py, only sbms.py.
  • The result of pasteurization is the same on RHEL7 and Fedora 32.
  • By-hand editing needed: os.getcwdu, which results from pasteurization, needs to be changed back to os.getcwd.