How to do HipPy

For the startup, this is probably the best object:
https://cms-conddb.cern.ch/cmsDbBrowser/list/Prod/tags/TrackerAlignment_StartUp2018_v2
it's the first one we uploaded to the GT this year.


Instructions for running hippy:

We work in this directory:
/afs/cern.ch/cms/CAF/CMSALCA/ALCA_TRACKERALIGN2/HipPy
The easiest way to start is to cmsenv here, to start with, even if that's not the release you eventually want to use (but later the script will make a new CMSSW for you with the proper release):
/afs/cern.ch/cms/CAF/CMSALCA/ALCA_TRACKERALIGN2/HipPy/TS1highvoltage/CMSSW_10_1_6

Then you can run this script.
makeHippyCampaign.py foldername --scram-arch (...) --cmssw (...) --merge-topic hroskes:hippy-scripts --merge-topic hroskes:common-tkal-dataset

This takes a while and makes a folder /afs/cern.ch/cms/CAF/CMSALCA/ALCA_TRACKERALIGN2/HipPy/foldername, which contains your CMSSW release and folders called Jobs/ and run/.  Jobs/ is where the jobs eventually happen.  run/ is where all the scripts are.

Inside run/ there are a few things:
DataFiles - here you have to set up a txt file with a list of data files to run on.  To get the list, you can run this script.
writedatasetfile.py -d /.../.../ALCARECO --first-run ... --last-run ... --hippy myfilelist.txt -j neventsperjob -m maxevents

For collisions, neventsperjob=10000 typically makes each iteration take about 20 minutes.  I suggest starting with that and then checking how many jobs there will be (by doing wc -l myfilelist.txt).  If it's too many you can either increase neventsperjob or set maxevents.

Then you can get rid of the COSMICS and CDCs line in data_example.lst and set the minbias line to point to your txt file.

IOV - you want a file that points to the first run of your dataset

Configurations - the only thing you have to touch here is the "common" file.  Here you set the global tag, the conditions to override (templates, maybe starting alignment), and the alignables.  To align at ladder level, uncomment the lines "TrackerP1PXBLadder,111111" and "TrackerP1PXECBlade,111111".  The 111111 refer to x y z theta_x theta_y theta_z.  It's probably better to turn off z in FPIX, so change it to 110111.

You can also change minimumNumberOfHits if you want.  That's about all we normally change.

Then, in the main folder, the last thing to change is submit_template.sh.  You just have to edit the variables that start out commented.
hpnumber is the ID number of the alignment.  It refers to the name of the folder in /afs/cern.ch/cms/CAF/CMSALCA/ALCA_TRACKERALIGN2/HipPy/alignments.  You can just increment the last number by 1.
common, lstfile, IOVfile - these are the names of the files you set up
alignmentname - this is the name of a folder in Jobs where it gets run
niterations - typically we do 10
Then the script requires you to git commit what you've done.  It's a little git repository with all these scripts.

Then finally you can start running.  This has to be in a screen session.  I usually use this script to start the session:
/afs/cern.ch/work/h/hroskes/public/forTanja/startscreen
it prints out how to get back to the cmsenv and directory and emails me which lxplus I'm on so that I can get back to it.

Inside the screen you just do ./submit_whatever.sh and it should run.