Usefull Tricks

Checking int lumi

brilcalc lumi -u /nb --begin 6469 --end 

6469 is the first run in 2018. To install have a look at https://cms-service-lumi.web.cern.ch/cms-service-lumi/brilwsdoc.html

New geometry to sqlite files

/data/vami/projects/Phase2/Telescope/CMSSW_10_0_0_pre1/src/Geometry/TrackerPhase2TestBeam/python
in file Phase2TestBeamGeometryXML_cfi.py modiy
    geomXMLFiles = cms.vstring(
        'Geometry/TrackerCommonData/data/trackerParameters.xml',
         # World volume creation and default CMS materials
then go to
/data/vami/projects/Phase2/Telescope/CMSSW_10_0_0_pre1/src/CondTools/Geometry/test
vim trackergeometrywriter.py
modify Geomerty line to
process.load('Configuration.Geometry.GeometryTrackerPhase2TestBeam_cff')


toPut = cms.VPSet(
    cms.PSet(record = cms.string('IdealGeometryRecord'),tag = cms.string('TKRECO_Geometry_Phase2Telescope')),
    cms.PSet(record = cms.string('PGeometricDetExtraRcd'),tag = cms.string('TKExtra_Geometry_Phase2Telescope')),
    cms.PSet(record = cms.string('PTrackerParametersRcd'),tag = cms.string('TKParameters_Geometry_Phase2Telescope'))

What I learned from 2D Templates

cmsrel CMSSW_10_1_0_pre2
cd CMSSW_10_1_0_pre2/src/
cmsenv
git cms-merge-topic 22458

To trigger the new CPE, uncomment last two lines in
RecoTracker/TransientTrackingRecHit/python/TTRHBuilderWithTemplate_cfi.py

To get the correct label modify
CalibTracker/SiPixelESProducers/plugins/SiPixel2DTemplateDBObjectESProducer.cc
and changed the line
std::string label = "";
to
std::string label = "denominator";

scram b

cmsDriver.py SingleMuPt10_pythia8_cfi --conditions 101X_upgrade2018_realistic_Candidate_2018_03_15_16_26_46 -n 10 --era Run2_2017 --eventcontent FEVTDEBUG --relval 25000,100 -s GEN,SIM --datatier GEN-SIM --beamspot Realistic25ns13TeVEarly2017Collision --geometry DB:Extended 

cmsDriver.py step2  --conditions 101X_upgrade2018_realistic_Candidate_2018_03_15_16_26_46 -s DIGI:pdigi_valid,L1,DIGI2RAW,HLT:@relval2017 --datatier GEN-SIM-DIGI-RAW -n 10 --geometry DB:Extended --era Run2_2017 --eventcontent FEVTDEBUGHLT --filein file:SingleMuPt10_pythia8_cfi_GEN_SIM.root

cmsDriver.py step3 --conditions 101X_upgrade2018_realistic_Candidate_2018_03_15_16_26_46 -n 10 --era Run2_2018 --eventcontent RECOSIM,MINIAODSIM,DQM --runUnscheduled -s RAW2DIGI,L1Reco,RECO,RECOSIM,EI,PAT,VALIDATION:@standardValidation+@miniAODValidation,DQM:@standardDQM+@ExtraHLT+@miniAODDQM --datatier GEN-SIM-RECO,MINIAODSIM,DQMIO --geometry DB:Extended --filein file:step2_DIGI_L1_DIGI2RAW_HLT.root --no_exec



open file step3_RAW2DIGI_L1Reco_RECO_RECOSIM_EI_PAT_VALIDATION_DQM.py
and include the line 
process.load('CalibTracker.SiPixelESProducers.SiPixel2DTemplateDBObjectESProducer_cfi')


for data
cmsDriver.py step3 --conditions 101X_dataRun2_Prompt_Candidate_2018_03_26_19_48_11 -n 10 --era Run2_2018 --eventcontent RECOSIM,MINIAODSIM,DQM --runUnscheduled -s RAW2DIGI,L1Reco,RECO,RECOSIM,EI,PAT,VALIDATION:@standardValidation+@miniAODValidation,DQM:@standardDQM+@ExtraHLT+@miniAODDQM --datatier GEN-SIM-RECO,MINIAODSIM,DQMIO --geometry DB:Extended --filein root://cms-xrd-global.cern.ch//store/data/Run2017F/ZeroBias/RAW/v1/000/305/406/00000/0049642B-86B7-E711-920F-02163E013657.root --no_exec --data


cmsDriver.py RECO -s RAW2DIGI,L1Reco,RECO --data --scenario pp --conditions 101X_dataRun2_Prompt_Candidate_2018_03_26_19_48_11 --era Run2_2017 --process NTUPLE --eventcontent RECO --datatier RECO --filein root://cms-xrd-global.cern.ch//store/data/Run2017F/ZeroBias/RAW/v1/000/305/406/00000/0049642B-86B7-E711-920F-02163E013657.root  --python_filename=run_Phase1PixelTree_Data_101X_cfg.py --runUnscheduled -n 10 --no_exec

Workflow tricks

A typical workflow for the relval for the 2017 workflow (using the phase1 pixel , also true for 2018, 2019 when the only relevant change is for hcal ) , can be found like this , let's say as example for SingleMuPt10 : 
runTheMatrix.py -n | grep 2017 | grep SingleMuPt10 
--> if you type yourself this command, you will see that the Workflow number is 10007 

Therefore you if you finally type 
runTheMatrix.py -l 10007 -ne 
--> it gives you the cmsdriver commands for this particular workflow, that are actually used in Pull request test and in relvals. 
--> you can notice typing the command above that the era actually used is --era Run2_2017 

Inserting the name of this era in the github search leads you to see the config listing the various detector eras to build a 2017 CMS era : 
please look here : 
https://github.com/cms-sw/cmssw/blob/master/Configuration/Eras/python/Era_Run2_2017_cff.py 
where you can see that for the pixel it is phase1Pixel 

So looks like what you did was good (I mean by adding the modifier) . 

For the workflow you were testing 11624 (2019 configuration) , you can try to same and find out that the era is --era Run3 , therefore you could look at : 
https://github.com/cms-sw/cmssw/blob/master/Configuration/Eras/python/Era_Run3_cff.py 
which is loading 
from Configuration.Eras.Era_Run2_2018_cff import Run2_2018 
which loading the 2017 one , so we are in the same config for the pixel as before. 


Something to think about on the pixel team side is whether we would not need to deploy eras according to the year (as hcal is doing) , so far the 2017 pixel is only different that the <2016 pixel (for obvious reasons :) ) but the same as in 2018 , 2019 - Perhaps one should think to have different reconstruction (or simulation) configurations/parameters depending on the years even for the same pixels. Maybe this is irrelevant , but in case the era mechanism will be easily to implement. 

Config files tricks

# Desciption: example how to set up many input files. After 255 files put the next files to the .extend part
import FWCore.ParameterSet.Config as cms
myfilelist = cms.untracked.vstring()
myfilelist.extend([
  'file:/data/vami/projects/pilotBlade/ppProcessing/Results/PilotMinBias_13TeV_cfi_py_RAW2DIGI_L1Reco_RECO2900Event.root',
  ])
#myfilelist.extend([  
#  ])
FileNames = myfilelist

DQM tricks

A way to get just the histograms you are interested in from the DQMGUI.

From lxplus, do:
cmsrel CMSSW_9_4_0
cd CMSSW_9_4_0/src/
cmsenv
dqm-access -w -c -f '/PixelPhase1/Phase1_MechanicalView/PXBarrel/digi_occupancy_*_PXLayer_*' -e "match('/ZeroBias1/Run2017G-PromptReco-v.*/DQMIO',dataset)" -s https://cmsweb.cern.ch/dqm/offline/data/json

dqm-access --help | less

The last command can be generalized, e.g.:

• If you want to fetch histograms from all periods, use Run2017.* instead of Run2017G (this is a plain regular expression, you can assemble the one you like)
• If you are interested in the forward too, I'd suggest making a second query (if you repeat the same command with a different histogram target, the output file in your local directory will be overwritten...)
• if you are interested only in some specific run, you could use: -e "run == XYZ and match...."

LA tree production

Pixel tree production

Setup procedure is explained here
https://github.com/BenjaminMesic/PixelTree-ProductionTool
(instructions.txt is deprecated so ignore it please)

After you finish setup, open TreeProduction.py and set these four lines
https://github.com/BenjaminMesic/PixelTree-ProductionTool/blob/master/TreeProduction.py#L16-L19
(CMSSW is not mandatory but is ok for bookkeeping.)

After you did all that just run the script. It will create bunch of scripts (in batch folder) ready to be sent on batch.
Before sending you can just open one python script and see if everything is done correctly. You
can even change number of events to e.g. 5 and run script interactively.

How does script actually creates jobs? 
There is the template script which is used for making jobs.
https://github.com/BenjaminMesic/PixelTree-ProductionTool/blob/master/templates/pixel.py
(Note that there are many different templates but only one which is called pixel.py is actually used,
other templates are used for different configurations, i.e. cosmics, VCAL, etc... Most recent are
ones which have phase1 in its name. If you want to use other template, it must be called pixel.py)

If you are going to send jobs, they will be automatically stored on EOS
https://github.com/BenjaminMesic/PixelTree-ProductionTool/blob/master/TreeProduction.py#L33
I don't know if you have permissions to store there. Anyway, you need to manually create directory
(from line 33) on EOS. In case you wish to make files on EOS, please fill the google doc what you did so
that I can follow what is going on.

Right now, sending jobs is not configurable as a variable so you need to (un)comment 
https://github.com/BenjaminMesic/PixelTree-ProductionTool/blob/master/TreeProduction.py#L235
in order to send jobs.

LA plots production

root -l -b -q  'LALayer.C("input.list",1,"outfile.root")'

input list is the txt file with the list of files with the LA trees (it can be also one root file, in that case second argument should be 0). 
Third argument is the output root file with the histograms.
The 2017 LA trees are stored in the eos directory:
/eos/cms/store/group/dpg_tracker_pixel/comm_pixel/LATrees/2017/

Linux tricks

du -hsx * | sort -rh | head -10
locate "*.root" | grep "/data/vami/backup/vami/projects/*" > rootFiles.txt

Changing paylaoads

# ----------------------------------------------------------------------
process.GlobalTag.toGet = cms.VPSet(
  cms.PSet(
	    record = cms.string("#RecordName1"),
           tag = cms.string("#TagName1"),
           connect = cms.untracked.string("frontier://FrontierProd/CMS_CONDITIONS)
          ),
  cms.PSet(
	    record = cms.string("#RecordName2"),
           tag = cms.string("#TagName2"),
           connect = cms.untracked.string("frontier://FrontierProd/CMS_CONDITIONS")
          )
)
# ----------------------------------------------------------------------

edm Tricks

edmConfigDump your_cfg.py > dumped_cfg.py

CRAB notes

config.JobType.pyCfgParams = ['globalTag=80X_mcRun2_...']
['valami=1', 'masik=2', 'harmadik=3']

AlCaDB contact notes

All DB test config

MinBias

 cmsDriver.py -s GEN,SIM,DIGI,L1,DIGI2RAW,RAW2DIGI,L1Reco,RECO --evt_type MinBias_13TeV_pythia8_TuneCUETP8M1_cfi --conditions auto:phase1_2017_realistic --era Run2_2017 --geometry DB:Extended --fileout file:GENSIMRECO_MinBias.root --python_filename=PhaseI_MinBias_cfg.py --runUnscheduled -n 10

https://github.com/jkarancs/PhaseIPixelNtuplizer/blob/1a635835a54364a467758f4550215798b1e28208/test/run_PhaseIPixelNtuplizer_MinBias_cfg.py

MuonGun

https://github.com/cms-analysis/DPGAnalysis-SiPixelTools/blob/master/PixelHitAssociator/phase1/run_PixelHitAssociator_PhaseI_MuPt10_cfg.py
Options to run this:

cmsRun phase1/run_PixelHitAssociator_PhaseI_MuPt10_cfg.py saveRECO=1 useTemplates=0 useLocalLASim=1 useLocalLA=1 useLocalGenErr=1 useLocalTemplates=1 maxEvents=10000 noMagField=1 outputFileName=gen_phase1_91X_MC0T_10k.root 
cmsRun phase1/run_PixelHitAssociator_PhaseI_MuPt10_cfg.py saveRECO=1 useTemplates=1 useLocalLASim=1 useLocalLA=1 useLocalGenErr=1 useLocalTemplates=1 maxEvents=10000 noMagField=1 outputFileName=tem_phase1_91X_MC0T_10k.root

Adat:
cmsRun phase1/run_PixelHitAssociator_PhaseI_MuPt10_cfg.py saveRECO=1 useTemplates=0 useLocalLASim=1 useLocalLA=1 useLocalGenErr=1 useLocalTemplates=1 maxEvents=10000 outputFileName=gen_phase1_91X_Data38T_10k.root
cmsRun phase1/run_PixelHitAssociator_PhaseI_MuPt10_cfg.py saveRECO=1 useTemplates=1 useLocalLASim=1 useLocalLA=1 useLocalGenErr=1 useLocalTemplates=1 maxEvents=10000 outputFileName=tem_phase1_91X_Data38T_10k.root

Template&GenErr DB resolutions

https://github.com/cms-analysis/DPGAnalysis-SiPixelTools/blob/master/PixelHitAssociator/MCResolution.C#L70-L71
https://github.com/cms-analysis/DPGAnalysis-SiPixelTools/blob/master/PixelHitAssociator/MCResolution.C#L120-L130

LA DB

Geometrical coordinates are defined in: https://github.com/cms-analysis/DPGAnalysis-SiPixelTools/blob/master/PixelDBTools/test/SiPixelLorentzAngleDBLoader.cc#L109-L126
Exceptions are to be defined from rawID: https://github.com/cms-analysis/DPGAnalysis-SiPixelTools/blob/master/PixelDBTools/test/SiPixelLorentzAngleDBLoader.cc#L96-L107

conddb notes

Downloading tags

conddb_import -f frontier://FrontierProd/CMS_CONDITIONS -i 'SiPixelGenErrorDBObject_38T_v1_offline' -c sqlite:SiPixelGenErrorDBObject_38T_v2_offline.db -t 'SiPixelGenErrorDBObject_38T_v2_offline' -b '197749' -e '200960'

Comperison of two sqlites

conddb list  SiPixelDynamicInefficiency_13TeV_50ns_v2_mc  // from FrontierProd
conddb --db /afs/cern.ch/cms/CAF/CMSALCA/ALCA_TRACKERCALIB/Pixels/PixelDB2015/6SiPixelDynamicInefficiency/siPixelDynamicInefficiency_13TeV_50ns_v2_mc.db list SiPixelDynamicInefficiency_13TeV_50ns_v2_mc // from local  file

both of them give the hash.

$ conddb dump 1fc23c8994620aa9d47d27baa9f0af4e99e76909 > 1fc23c8994620aa9d47d27baa9f0af4e99e76909.xml 
$ conddb dump 5a9e60ee4edc52188da96e384da8fb2bccee30b4 > 5a9e60ee4edc52188da96e384da8fb2bccee30b4.xml

diff 1fc23c8994620aa9d47d27baa9f0af4e99e76909.xml 5a9e60ee4edc52188da96e384da8fb2bccee30b4.xml73c73

Root notes

Standard things

htemp->GetXaxis()->SetRangeUser(-1.7,1.7);
TCanvas *c2_2 = new TCanvas("c2_2", "2_2",0,0,1355,523);

TH2D (const char *name, const char *title, 
      Int_t nbinsx, Double_t xlow, Double_t xup, 
      Int_t nbinsy, Double_t ylow, Double_t yup)
 

Custum canvas

TH2* h, std::string canname, 
      int gx = 0, int gy = 0,
      int histosize_x = 500, int histosize_y = 500,
      int mar_left = 80, int mar_right = 120,
      int mar_top = 20, int mar_bottom = 60,
      int title_align = 33, float title_y = 1.0, float title_x = 1.0,
      std::string draw="COLZ", bool norm=false, bool log=false
 

Some explanations:

• title_align could be 11, 12, 13, 31, 32, 33 (upper right) For horizontal alignment the following convention applies: 1=left adjusted, 2=centered, 3=right adjusted
  For vertical alignment the following convention applies: 1=bottom adjusted, 2=centered, 3=top adjusted
  

prelim_lat_(double xmin, double xmax, double ymin, double ymax, bool in, double scale = -1)
draw_lat_(250, 198.0, "Module X", 1, 0.04, 0.0, 21);

-- TamasVami - 2016-01-14