Converting build_scripts from Subversion to Git

Questions and answers for plan to conversion to use of the new git repository:

  • Where to locate the new JLab working version?
    • In /group/halld/Software/build_scripts. This is a move from SVN location under scripts. The old location is misleading now since build_scripts is no longer an SVN subdirectory of scripts.
  • Is the working version an archive or a clone?
    • It is a clone. That way one can inquire about version information.
  • Is the working version a release branch or master?
    • For now is is a temporary branch. See below.

The conversion was done at SVN revision 19207. The last change to build_scripts previoiusly was at revision 19203.

The head of the build_scripts master branch does not support getting code from Git repositories at all. It does support getting source from Subversion repositories. Before conversion of build_scripts to Git, in special versions of Makefile_hdds and Makefile_sim-recon an archive is made from the local JLab Git repositories to support the nightly builds. The build_halld.csh script did this by changing the checkout (from subversion) of build_scripts to get these special versions at revision 19034 of build scripts via svn update. Note that this only worked at JLab. Here is the relevant part of the script.

< # temp update of git-enabled makefiles
< pushd build_scripts
< svn update -r19034 Makefile_hdds Makefile_sim-recon
< popd

In git this commit this svn revision is commit 16fa07793df6bf77359dd597a5e99cf685dd2f70

On JLab, in the Software/build_scripts repository, made branch nightly_temp that uses the local-only git enabled versions of Makefile_hdds and Makefile_sim-recon. It also has a modified version of build_halld.csh that will git-archive from this branch. That repository has this branch checked-out so that the special version of build_halld.csh is out there to be used.

The nightly build is now accessing build_scripts in this location, i. e., /group/halld/Software/build_scripts.

Notes on Final Conversion from Subversion to Git

last revision:
Rev 19042 2015-07-15 17:35:06

incremental conversion did not work, sim-recon/src/plugins directory did not make it, starting over

https://github.com/JeffersonLab/hdds.git
https://github.com/JeffersonLab/sim-recon.git

  1. delete the practice repositories on GitHub
  2. create a blank repository to receive the converted repository on GitHub
  3. make a copy of the complete Subversion repository on the local disk with rsync
  4. use svn2git to do the conversion using the local-disk-resident version of the Subversion repository
  5. in the resulting git repository, delete branches and tags not related to the package, as well as all “@revision” branches and tags
  6. clone a bare copy of the repository (–bare option)
  7. push the bare copy to the GitHub repository with the –mirror option
  8. add repositories to the list of those owned by the gluex team on GitHub

The sim-recon conversion with svn2git took about 8 hours. hdds took about 20 hours

Notes on GlueX Git conversion

Notes on conversion from subversion to git for gluex.

  • What happens if a pull request has a conflict?
    • It cannot be merged on the github site. instructions on how to do it on the command line are available.
  • Should we use rebasing?
    • rebasing introduces possibilities for errors, commits are replayed on another branch, thus commits exist (in an operational sense) twice. Thus the admonition, do not rebase commits that exist outside your local repository. Pull requests generated on github satisfy this no-go situation for rebasing by construction. Rebasing an expert operation, IMHO.
  • How are we notified of pull requests
    • those interested in acting on pull should “watch” the appropriate repository. go to settings/notification to configure how you are notified. go to the repository page (after logging in) to mark the repository as being watched/not-watched/ignored by you using the pull down menu at the top of the page

Subversion to Git conversion notes

Names of repositories:

  • sim-recon
  • sim-recon-release
  • hdds
  • hdds-release

Proposed work flow:

  1. clone sim-recon
  2. create branch for development
  3. work on branch, checking in as you go
  4. push branch to github as branch of sim-recon
    1. never push to master branch at github
    2. you could do it, but don’t
    3. you need, at github
      1. an account
      2. have your account added to the gluex team
      3. previous steps do not require an account at github
  5. create a pull request on github for your new branch
  6. wait for your changes to be merged on the master at github
  7. pull your changes back on to your local master branch

admin group: Mark, David, Simon, Paul M., Justin, Sean

Conversion with author list

  • put author list in lorentz:/home/marki/.svn2git/authors
    • put script to generate: user_list_svn_git.pl [?]
    • needed to add wolin, manak, weygand, jcummngs, bellis, futch, and (no author) by hand at end of script to succeed

old log entry:

lorentz:marki:BCAL> git log DBCALShower_factory_IU.cc
commit 2dba218a8e140d53878b0be8eb5bac0031e15b3b
Author: wilevine
Date: Fri Sep 12 16:19:14 2014 +0000

Fix time calculation in DBCALShower_factory

new log entry:

lorentz:marki:BCAL> git log DBCALShower_factory_IU.cc
commit 4169bd1bd4380e7814e0a13d49068cbfbbe4740b
Author: William Levine <wilevine@jlab.org>
Date: Fri Sep 12 16:19:14 2014 +0000

Fix time calculation in DBCALShower_factory

svn2git commands

svn2git file:///local/scratch/svn2git/svnroot --no-minimize-url --trunk trunk/sim-recon --branches branches --tags tags

or replace file:... with https://halldsvn.jlab.org/repos

git clone --bare file:///local/scratch/svn2git/sim-recon

Command to get all svn authors

svn log –quiet file:///local/scratch/svn2git/svnroot | grep “^r” | awk ‘{print $3}’ | sort | uniq | perl -n -e ‘print “=======”, $_; chomp; system “grep -s $_ /home/marki/.svn2git/authors”;’

Branch and Tag Filtering on Converted Git Repository

  • git branch | grep @ | xargs git branch -d
  • git branch | grep -v sim-recon | grep -v src | grep -v master | xargs git branch -d
  • git tag | grep @ | xargs git tag -d
  • git tag | grep -v sim-recon | grep -v release | grep -v src | grep -v master | xargs git tag -d

Random Notes

  • can svn repo be kept up to date from git?
  • can git repo be kept up to date from svn?
  • if so then dual repo transition period is possible
  • at end of latest svn2git got the following:

svn2git warning: Tracking remote SVN branches is deprecated.
In a future release local branches will be created without tracking.
If you must resync your branches, run: svn2git –rebase

Subversion to Git conversion

The magic command to convert from subversion to git:

svn2git $HDSVN --no-minimize-url -v --trunk trunk/home/marki/svn_test --nobranches --tags tags/home/marki

To prepare local repository for import to GitHub:

git clone --bare file:///home/marki/git_play/svn_import_test

Create new repository on GitHub to receive the bare clone.

To do the import, from the local host:

cd svn_import_test.git/
git push --mirror https://github.com/markito3/svn_import_test.git