OSG notes

November 14, 2017

  • gsissh username should be gluex
  • proxy is created in /tmp:
> ls -l /tmp/x509up_u2191
-rw------- 1 marki halld-1 6241 Aug 4 13:35 /tmp/x509up_u2191
  • GOC = Grid Operations Center
  • There are GOC problem tickets and Help Desk problem tickets. They are separate systems.
Advertisements

Working with Oasis

logging into the grid submit host

ssh scosg16

check for a proxy, create one, the wrong one first, destroy it, then the right one

scosg16:marki:marki> voms-proxy-info

Couldn't find a valid proxy.

scosg16:marki:marki> voms-proxy-init
Enter GRID pass phrase for this identity:
Your identity: /DC=org/DC=opensciencegrid/O=Open Science Grid/OU=People/CN=Mark Ito 1104
key values mismatch
Function: X509_check_private_key
Creating proxy ............................................................................................................................ Done

Your proxy is valid until Sat Sep 16 02:41:55 2017
scosg16:marki:marki> voms-proxy-info
subject   : /DC=org/DC=opensciencegrid/O=Open Science Grid/OU=People/CN=Mark Ito 1104/CN=2073856599
issuer    : /DC=org/DC=opensciencegrid/O=Open Science Grid/OU=People/CN=Mark Ito 1104
identity  : /DC=org/DC=opensciencegrid/O=Open Science Grid/OU=People/CN=Mark Ito 1104
type      : RFC compliant proxy
strength  : 1024 bits
path      : /tmp/x509up_u2191
timeleft  : 11:59:50
scosg16:marki:marki> voms-proxy-destroy
scosg16:marki:marki> voms-proxy-info

Couldn't find a valid proxy.

scosg16:marki:marki> voms-proxy-init -voms Gluex
Enter GRID pass phrase for this identity:
Your identity: /DC=org/DC=opensciencegrid/O=Open Science Grid/OU=People/CN=Mark Ito 1104
key values mismatch
Function: X509_check_private_key
Creating temporary proxy ....................... Done
Contacting  gryphn.phys.uconn.edu:15014 [/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=voms/gryphn.phys.uconn.edu] "Gluex" Done
Creating proxy .................... Done

Your proxy is valid until Sat Sep 16 02:47:14 2017
scosg16:marki:marki> voms-proxy-info
subject   : /DC=org/DC=opensciencegrid/O=Open Science Grid/OU=People/CN=Mark Ito 1104/CN=4260201169
issuer    : /DC=org/DC=opensciencegrid/O=Open Science Grid/OU=People/CN=Mark Ito 1104
identity  : /DC=org/DC=opensciencegrid/O=Open Science Grid/OU=People/CN=Mark Ito 1104
type      : RFC compliant proxy
strength  : 1024 bits
path      : /tmp/x509up_u2191
timeleft  : 11:59:52

documentation accessing oasis is here: https://twiki.grid.iu.edu/bin/view/Documentation/Release3/UpdateOasis
gsissh into oasis server

scosg16:marki:marki> gsissh -o GSSAPIDelegateCredentials=yes oasis-login.opensciencegrid.org
Last login: Thu Sep 14 22:26:33 2017 from gluey.phys.uconn.edu
[0] 18:48:42 UTC [ouser.gluex@oasis-login:~]$ pwd
/home/login/ouser.gluex
  • To update the Oasis files:
    go to gluex staging area: /stage/oasis/gluex
    make the mods to the files
    Issue the command: osg-oasis-update
[ouser.gluex@oasis-login gluex]$ osg-oasis-update
Submitting job(s).
1 job(s) submitted to cluster 2536.
You can use condor_q to check on the status of your update
[ouser.gluex@oasis-login gluex]$ condor_q

-- Schedd: oasis-login.grid.iu.edu : <129.79.53.147:24122?...  ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD                2536.0   ouser.gluex     9/15 18:59   0+00:00:11 R  0   0.0  oasis-batch-worker 1 jobs; 0 completed, 0 removed, 0 idle, 1 running, 0 held, 0 suspended 

remote commands can be run:

 scosg16:marki:marki> gsissh -o GSSAPIDelegateCredentials=yes oasis-login.opensciencegrid.org hostname
oasis-login.grid.iu.edu

scp works

scosg16:marki:git_test> gsiscp -o GSSAPIDelegateCredentials=yes a.txt oasis-login.opensciencegrid.org:/stage/oasis/gluex/test/
a.txt                                                                                                      100%  127     0.1KB/s   0.1KB/s   00:00

rsync works

scosg16:marki:marki> rsync -ruvt --delete -e 'gsissh -o GSSAPIDelegateCredentials=yes' /home/marki/git_test/ oasis-login.opensciencegrid.org:/stage/oasis/gluex/test/git_test/
sending incremental file list
created directory /stage/oasis/gluex/test/git_test
./
README.md
a.tt
a.txt
aa.txt
b.txt
c.txt
d.txt
e.txt
j.txt
k.txt
.git/
.git/COMMIT_EDITMSG
.git/COMMIT_EDITMSG~
.git/FETCH_HEAD
...
.git/refs/remotes/origin/idea2
.git/refs/remotes/origin/master
.git/refs/tags/
dir/
dir/l.txt
dir/m.txt

sent 43791 bytes  received 1222 bytes  12860.86 bytes/sec
total size is 39107  speedup is 0.87

and after modifying j.txt

scosg16:marki:marki> rsync -ruvt --delete -e 'gsissh -o GSSAPIDelegateCredentials=yes' /home/marki/git_test/ oasis-login.opensciencegrid.org:/stage/oasis/gluex/test/git_test/
sending incremental file list
./
j.txt

sent 2215 bytes  received 71 bytes  304.80 bytes/sec
total size is 39128  speedup is 17.12

Possibly useful command for rsync’ing Oasis

find /group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5/ -maxdepth 5 -type d \( -name lib -o -name bin \) > /scratch/marki/dirs.txt 
rsync -ruvt --delete /group/halld/Software/ /scratch/marki/group/halld/Software/ --exclude ExternalPackages/ --exclude \*\*/.git/\*\* --exclude \*\*/Linux_CentOS6\*/ --exclude \*\*/Linux_RHEL\*/ --exclude \*\*/\*.tar.gz
awk -e '{print "mkdir -p /scratch/marki" $1 " ; rsync -ruvt --delete --exclude \\*\\*/.git/\\*\\* " $1 " /scratch/marki" $1}' < /scratch/marki/dirs.txt > e.tmp

Notes on singularity and gluex software

July 29, 2017

Things to do:

  • Figure out how to copy files to oasis
  • Figure out how to copy containers to singlularity cvmfs

Useful commands:

singularity expand centos7.img
sudo /usr/local/bin/singularity shell --writable centos7.img
/usr/local/bin/singularity shell --bind /group/halld:/group/halld centos7.img

Getting mysql shared library to be seen by container:

> eval `addpath.pl -l /home/marki/lib`
> addpath.pl -l /home/marki/lib
LD_LIBRARY_PATH=/home/marki/lib:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5/gluex_root_analysis/gluex_root_analysis-0.2^ccdb165/Linux_CentOS7-x86_64-gcc4.8.5/lib/:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5/evio/evio-4.4.6/Linux-x86_64/lib:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5/rcdb/rcdb_0.01/cpp/lib:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5/ccdb/ccdb_1.06.05/lib:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5/geant4/geant4.10.02.p02/lib64:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5/root/root-6.08.06/lib:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5/xerces-c/xerces-c-3.1.4/lib:/.singularity.d/libs
> ls /home/marki/lib
libmysqlclient.so.20
> cp /usr/lib64/mysql/libmysqlclient.so.20 /u/scratch/marki

Try to find where the non-standard library is coming from, on ifarm1401:

> repoquery -f /usr/lib64/mysql/libmysqlclient.so.20.3.2
mysql-community-libs-0:5.7.15-1.el7.x86_64
> repoquery -i mysql-community-libs-0:5.7.15-1.el7.x86_64

Name        : mysql-community-libs
Version     : 5.7.15
Release     : 1.el7
Architecture: x86_64
Size        : 9898444
Packager    : MySQL Release Engineering <mysql-build@oss.oracle.com>
Group       : Applications/Databases
URL         : http://www.mysql.com/
Repository  : mysql
Summary     : Shared libraries for MySQL database client applications
Source      : mysql-community-5.7.15-1.el7.src.rpm
Description :
This package contains the shared libraries for MySQL client
applications.

Tracking down the mysql shared library needed:

In CentOS7 Singularity container:

> mysql --version
mysql Ver 15.1 Distrib 5.5.52-MariaDB, for Linux (x86_64) using readline 5.1

On ifarm1402:

> mysql --version
mysql Ver 14.14 Distrib 5.7.15, for Linux (x86_64) using EditLine wrapper
> ldd `which hd_root` | grep mysql
libmysqlclient.so.20 => /usr/lib64/mysql/libmysqlclient.so.20 (0x00007f670896c000)

On lorentz:

> mysql --version
mysql Ver 15.1 Distrib 5.5.52-MariaDB, for Linux (x86_64) using readline 5.1
> ldd `which hd_root` | grep mysql
libmysqlclient.so.18 => /usr/lib64/mysql/libmysqlclient.so.18 (0x00007f09a0cfd000)

Special repo on ifarm:

> pushd /etc/yum.repos.d
/etc/yum.repos.d /u/scratch/marki
ifarm1402:marki:yum.repos.d> ls
core72.repo epel-testing.repo.bak mysql.repo scicomp-extras.repo
epel.repo eple.repo.bak2 salt.repo
ifarm1402:marki:yum.repos.d> cat mysql.repo
# mysql rhel7 mirror
[mysql]
name = MySQL Community
baseurl=http://sca1401/yum/centos72/mysql_c7
gpgcheck=0

GlueX and the Open Science Grid

  • at JLab
    • submit host installed: for submitting jobs to the OSG
    • SciComp did installation in consultation with OSG experts
    • log-in with CUE credentials for authorized users
    • JLab users submit jobs to the OSG with the installed software
  • at Collaborating Institutions
    • meeting was held with (different) set of OSG personnel to discuss contribute of University-based clusters to OSG infrastructure
    • makes these nodes available for general GlueX computing
    • UConn, NU already contributing
    • prospective contributions from CMU, IU, FIU, FSU