Optical sim/reco and overlay updates#892
Open
mvicenzi wants to merge 161 commits into
Open
Conversation
…plate It used to end when the template SPR goes under 1e-4 ADC#. Now it needs to stay within +/- 1w-4 ADC# for 20 nanoseconds. This should make it possible to include undershootings. All these parameters are currently hard-coded.
…icles that might cross cathode
Good catches on names and strings. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Trigger simulation modules now support input tags with process name [private]
Following the recommendation by Francesco Poppi.
When running on simulation, timestamp TS0 is based on time 0 rather than on the timestamp of the (simulated) trigger. Change recommended by Francesco Poppi.
That propagates to overlay workflow too.
PetrilloAtWork
changed the title
- replaced explicit link to DetectorClocksServiceStandard with lardata::DetectorClocksService - removed redundant and unnecessary art framework libraries from link lists - fixed some documentation typos
…ples It allows the creation of a raw::trigger data product with the shifted time reference, which AdjustSimForTrigger module used not to produce (newer one does).
By default changes are applied to all gate types (used to be to unknown only). In addition, maintenance changes: - added debug messages; - moved some of the code of general use to `icarusalg` (sbn::BeamTime enumerator and selector); would be more impressive if the selector mechanism supported `fhicl::Sequence`, which it does not because support of `fhicl::Atom` was enough of a hack already.
This module allows to print explicitly which data product is picked when an input tag is specified at a certain step of the execution chain. For example, it would expand a `std::vector<sim::SimPhotons>` tag "shifted" to `shifted::DetSim`, or claim no data product matches.
PetrilloAtWork
changed the base branch from
release/SBN2025A
to
migrate/SBN2025A
PetrilloAtWork
changed the base branch from
migrate/SBN2025A
to
release/SBN2025A
There was a problem hiding this comment.
Pull request overview
This PR updates ICARUS optical simulation/reconstruction and overlay workflows to support SBN2025 spring reprocessing (MC + data), including new/updated database-backed calibrations (PMT gain, PMT channel status, PMT timing), trigger/beam-gate handling changes, and a set of new/updated FHiCL “partial” configs for targeted reprocessing.
Changes:
- Introduces DB-backed PMT channel status and SPE-area (gain) calibration services, and wires them into stage0/detsim/simulation service stacks.
- Refactors trigger/beam-gate configuration and multiple trigger modules to use
art::InputTagconsistently, plus adds shifting/overlay workflow updates. - Adds/updates many reprocessing/partial FHiCLs for detsim, overlay, and stage0/stage1, and adjusts build system dependencies accordingly.
Reviewed changes
Copilot reviewed 138 out of 138 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/manageDataRunDefinitions.py | Extends SAM-definition tooling (raw/compressedraw, extra dims, suffix). |
| icaruscode/Utilities/CMakeLists.txt | Adds new DataProductReader plugin build target. |
| icaruscode/TPC/SignalProcessing/RecoWire/SimTestPulse/CMakeLists.txt | Updates DetectorClocksService linkage. |
| icaruscode/Timing/timing_icarus.fcl | Adds run-number override option for PMT timing service. |
| icaruscode/Timing/PMTTimingCorrectionsProvider.h | Adds run override + DB tag accessors. |
| icaruscode/Timing/PMTTimingCorrectionsProvider.cxx | Implements override run-number DB queries. |
| icaruscode/Timing/PMTTimingCorrections.h | Extends interface with DB tag getters. |
| icaruscode/PMT/Trigger/triggerwindowdefs_icarus.fcl | Moves trigger-window definitions include usage. |
| icaruscode/PMT/Trigger/triggeremu_settings_icarus.fcl | Adds timing delay parameters + includes window defs. |
| icaruscode/PMT/Trigger/TriggerEfficiencyPlotsBase.h | Switches TriggerGatesTag to art::InputTag + helper. |
| icaruscode/PMT/Trigger/TriggerEfficiencyPlotsBase.cxx | Builds per-threshold tags via helper with validation. |
| icaruscode/PMT/Trigger/TriggerEfficiencyPlots_module.cc | Same tag handling updates as base class. |
| icaruscode/PMT/Trigger/trigger_icarus.fcl | Updates thresholds, gates, and patterns for Run2+ tuning. |
| icaruscode/PMT/Trigger/trigger_emulation_icarus.fcl | Updates trigger emulation workflow + baseline handling. |
| icaruscode/PMT/Trigger/SlidingWindowTriggerSimulation_module.cc | Uses art::InputTag and validated tag building. |
| icaruscode/PMT/Trigger/SlidingWindowTrigger_module.cc | Uses art::InputTag and validated tag building. |
| icaruscode/PMT/Trigger/MajorityTriggerSimulation_module.cc | Uses art::InputTag and validated tag building. |
| icaruscode/PMT/Trigger/LVDSgates_module.cc | Switches optional default gate tag to art::InputTag. |
| icaruscode/PMT/Trigger/FixBeamGateInfo_module.cc | Refactors beam-type selection + adds trace logging. |
| icaruscode/PMT/Trigger/DiscriminatePMTwaveforms_module.cc | Consumes baseline “parallel data product” when configured. |
| icaruscode/PMT/Trigger/DiscriminatedAdderSignal_module.cc | Simplifies optional interval parsing + improves error text. |
| icaruscode/PMT/Trigger/CMakeLists.txt | Updates link deps and target naming to newer conventions. |
| icaruscode/PMT/Trigger/Algorithms/WindowTopologyAlg.cxx | Fixes trace message formatting. |
| icaruscode/PMT/Status/PMTChannelStatusService_service.cc | New art service for PMT channel status DB. |
| icaruscode/PMT/Status/PMTChannelStatusProvider.h | New provider interface + configuration docs. |
| icaruscode/PMT/Status/PMTChannelStatusProvider.cxx | Implements DB reads, caching, override run number. |
| icaruscode/PMT/Status/PMTChannelStatus.h | New status/voltage provider interface. |
| icaruscode/PMT/Status/pmt_channel_status_icarus.fcl | Standard FHiCL config for PMT status service. |
| icaruscode/PMT/Status/IPMTChannelStatusService.h | Wrapper service interface for provider access. |
| icaruscode/PMT/Status/CMakeLists.txt | Builds PMT status library + service plugin. |
| icaruscode/PMT/SampledWaveformFunctionTool_tool.cc | Adds BiasRatio option plumbed into pulse function. |
| icaruscode/PMT/OpReco/ICARUSOpHitFinder_module.cc | Adds channel-status filtering + optional rise-time tool. |
| icaruscode/PMT/OpReco/ICARUSFlashAssAna_module.cc | Adds per-PMT integral branch in analysis tree. |
| icaruscode/PMT/OpReco/FlashFinder/ICARUSFlashFinder_module.cc | Modernizes module, uses PE() directly, SharedProducer. |
| icaruscode/PMT/OpReco/FlashFinder/FlashFinderFMWKInterface.h | Cleans includes, adds standard types. |
| icaruscode/PMT/OpReco/FlashFinder/FlashFinderFMWKInterface.cxx | Cleans includes, adds ServiceHandle include. |
| icaruscode/PMT/OpReco/FlashFinder/CMakeLists.txt | Simplifies library deps + plugin link list. |
| icaruscode/PMT/OpReco/fcl/tpcpmtbarycentermatch_config.fcl | Unifies TriggerDelay setting. |
| icaruscode/PMT/OpReco/fcl/icarus_spe.fcl | Adds Run2/3/4 SPE defaults + documentation. |
| icaruscode/PMT/OpReco/fcl/icarus_ophitfinder.fcl | Switches to ICARUSOpHitFinder + enables DB services. |
| icaruscode/PMT/OpReco/fcl/icarus_opana_modules.fcl | Retunes MC OpHit config to new SPE defaults. |
| icaruscode/PMT/OpReco/fcl/icarus_flashfinder.fcl | Removes PECalib configuration (uses OpHit PE). |
| icaruscode/PMT/OpReco/fcl/icarus_flashcalib.fcl | Removes obsolete flash calibration tables. |
| icaruscode/PMT/OpReco/fcl/icarus_flashalgo.fcl | Retunes PE thresholds to new PE scale. |
| icaruscode/PMT/OpReco/Algorithms/OpRecoFactoryStuff.h | Allows injecting optional rise-time calculator into algos. |
| icaruscode/PMT/OpReco/Algorithms/CMakeLists.txt | Updates dependencies for OpReco algorithm library. |
| icaruscode/PMT/opdetsim_pmt_icarus.fcl | Adds Run-dependent SimPMT configurations + no-noise variants. |
| icaruscode/PMT/CopyBeamTimePMTwaveforms_module.cc | Simplifies optional interval parsing. |
| icaruscode/PMT/CMakeLists.txt | Adds Status subdir + updates module/tool link deps. |
| icaruscode/PMT/Calibration/PhotonCalibratorFromDB.h | New DB-backed IPhotonCalibrator implementation. |
| icaruscode/PMT/Calibration/PhotonCalibratorFromDB.cxx | Implements DB reads + PE conversion for SPE-area table. |
| icaruscode/PMT/Calibration/ICARUSPhotonCalibratorServiceFromDB.h | New art service wrapper for DB calibrator provider. |
| icaruscode/PMT/Calibration/ICARUSPhotonCalibratorServiceFromDB_service.cc | Implements preBeginRun cache refresh. |
| icaruscode/PMT/Calibration/fcl/pmt-calibration.fcl | Removes legacy calibration job config. |
| icaruscode/PMT/Calibration/fcl/pmt_calibration_icarus.fcl | New consolidated calibration config incl. IPhotonCalibrator. |
| icaruscode/PMT/Calibration/fcl/ophit_recalibrator_icarus.fcl | New OpHitRecalibrator defaults + timing tag config. |
| icaruscode/PMT/Calibration/fcl/decodePMT_icarus_laser.fcl | Updates include to new calibration config file. |
| icaruscode/PMT/Calibration/fcl/debug_speAreas_db.fcl | Adds a debugging stage0 job for SPE DB configs. |
| icaruscode/PMT/Calibration/fcl/bkgphotons-calibration.fcl | Updates include to new calibration config file. |
| icaruscode/PMT/Calibration/CMakeLists.txt | Builds new DB calibrator library + service plugin + new modules. |
| icaruscode/PMT/Algorithms/SampledWaveformFunction.h | Adds integral + bias constant to sampled pulse function. |
| icaruscode/PMT/Algorithms/PhotoelectronPulseFunction.h | Extends base interface with integral + bias constant. |
| icaruscode/PMT/Algorithms/FastGaussianNoiseGeneratorAlg.h | Updates documentation + increases gauss adaptor resolution. |
| icaruscode/PMT/Algorithms/DiscretePhotoelectronPulse.h | Improves pulse-termination logic API. |
| icaruscode/PMT/Algorithms/DiscretePhotoelectronPulse.cxx | Implements “close-to-baseline” termination with persistence. |
| icaruscode/PMT/Algorithms/CMakeLists.txt | Links PMT algorithms with new calibration library. |
| icaruscode/Overlays/CMakeLists.txt | Adds RebuildShiftedTrigger plugin target. |
| icaruscode/Generators/CMakeLists.txt | Adds new generator-side selection module build target. |
| icaruscode/Generators/beamspilltimings.fcl | Delegates default spill timing config to SBN shared config. |
| icaruscode/Decode/OverlayProducts_module.cc | Temporarily truncates tail filling after data waveform end. |
| icaruscode/Decode/CMakeLists.txt | Adds OverlayPMTwaveforms plugin target; updates deps. |
| icaruscode/CRT/CRTUtils/CRTHitRecoAlg.h | Exposes isData() accessor. |
| icaruscode/CRT/CRTUtils/CRTHitRecoAlg.cc | Avoids trigger-offset subtraction for simulation. |
| icaruscode/CRT/CRTUtils/CMakeLists.txt | Updates DetectorClocksService linkage. |
| icaruscode/CRT/CRTSimHitProducer_module.cc | Cleans trigger handling; validates/ignores trigger tag by mode. |
| icaruscode/CRT/CRTDetSim_module.cc | Comment typo fixes in module header docs. |
| icaruscode/CRT/CMakeLists.txt | Updates DetectorClocksService linkage. |
| icaruscode/Analysis/trigger/CMakeLists.txt | Updates DetectorClocksService linkage. |
| fcl/utilities/dump_opflashes_icarus.fcl | New utility job to dump standard OpFlash products. |
| fcl/utilities/dump_opdetwaveforms_icarus.fcl | Uses simulation DetectorClocksService config in utility. |
| fcl/SkipTesting.txt | Adds override_services_for_simulation.fcl to skip list. |
| fcl/services/services_icarus_simulation.fcl | Wires timing/status/gain services + simulation clocks override. |
| fcl/services/override_services_for_simulation.fcl | New reusable services override for simulation workflows. |
| fcl/reco/Stage1/mc/stage1_run2_icarus_MC.fcl | Includes simulation services override. |
| fcl/reco/Stage0/partial/stage0_run4_opdetonly_icarus_data_reprocessing.fcl | New Run4 optical-only data reprocessing config. |
| fcl/reco/Stage0/partial/stage0_run2_opdetonly_icarus_overlay_reprocessing.fcl | New optical-only overlay stage0 reprocessing config. |
| fcl/reco/Stage0/partial/stage0_run2_opdetonly_icarus_data_reprocessing.fcl | New Run2 optical-only data reprocessing config. |
| fcl/reco/Stage0/partial/CMakeLists.txt | Installs new partial stage0 configs. |
| fcl/reco/Stage0/mc/stage0_run2_wc_icarus_mc.fcl | Includes override + tunes optical reconstruction to Run2. |
| fcl/reco/Stage0/mc/stage0_run2_wc_icarus_mc_notriggersim.fcl | Restores non-shifted inputs for no-trigger-sim workflows. |
| fcl/reco/Stage0/mc/stage0_run2_icarus_mc.fcl | Includes override + enables fast cloning (no SelectEvents). |
| fcl/reco/Stage0/mc/stage0_run2_icarus_mc_notriggersim.fcl | Restores non-shifted inputs for no-trigger-sim workflows. |
| fcl/reco/Stage0/data/partial/decodePMT_icarus.fcl | Adds channel status service to PMT decode services. |
| fcl/reco/Stage0/CMakeLists.txt | Adds new partial stage0 configs subdirectory. |
| fcl/reco/Definitions/stage0_icarus_mc_defs.fcl | Splits corrected/uncorrected ophit path; overlay PMT sequence. |
| fcl/reco/Definitions/stage0_icarus_driver_common.fcl | Adds photon calibrator + channel status services. |
| fcl/reco/Definitions/stage0_icarus_defs.fcl | Includes new calibration + recalibrator configs. |
| fcl/reco/Definitions/enable_overlay_sp.fcl | Switches to overlay PMT path + timing correction override. |
| fcl/overlays/partial/overlay_opdetonly_waveforms_reprocessing.fcl | New overlay-only waveform reprocessing job. |
| fcl/overlays/partial/CMakeLists.txt | Installs new partial overlay configs. |
| fcl/overlays/overlay_waveforms.fcl | Replaces OverlayProducts PMT overlay with OverlayPMTwaveforms module. |
| fcl/overlays/CMakeLists.txt | Adds partial overlay configs subdirectory. |
| fcl/gen/numi/genie_icarus_numioffaxis.fcl | Switches to SBN shared NuMI GENIE base + volume/log updates. |
| fcl/gen/genie/genie_icarus_bnb.fcl | Switches to SBN shared BNB GENIE base + volume/log updates. |
| fcl/g4/PDFastSim_icarus.fcl | Updates SimulationLabel and minor formatting. |
| fcl/detsim/partial/detsim_opdetonly_icarus.fcl | New optical-only detsim job config. |
| fcl/detsim/partial/detsim_opdetonly_icarus_overlay_reprocessing_run4.fcl | New Run3/4 overlay optical detsim reprocessing config. |
| fcl/detsim/partial/CMakeLists.txt | Installs new partial detsim configs. |
| fcl/detsim/detsim_2d_icarus.fcl | Explicitly disables DB effects for non-overlay MC opdaq. |
| fcl/detsim/detsim_2d_icarus_refactored.fcl | Explicitly disables DB effects for non-overlay MC opdaq. |
| fcl/detsim/detsim_2d_icarus_refactored_yzsim.fcl | Explicitly disables DB effects for non-overlay MC opdaq. |
| fcl/detsim/detsim_2d_icarus_refactored_yzsim_Run4.fcl | Selects Run3/4 optical tune; keeps DB effects off for non-overlay. |
| fcl/detsim/detsim_2d_icarus_refactored_yzsim_overlay.fcl | Enables DB effects and no-noise for overlay opdaq. |
| fcl/detsim/detsim_2d_icarus_refactored_yzsim_overlay_Run4.fcl | Run3/4 overlay tune with DB effects + no-noise. |
| fcl/detsim/detsim_2d_icarus_refactored_yzsim_notrigger.fcl | Explicitly disables DB effects for non-overlay MC opdaq. |
| fcl/detsim/detsim_2d_icarus_refactored_yzsim_notrigger_overlay.fcl | Enables DB effects + no-noise for overlay notrigger. |
| fcl/detsim/detsim_2d_icarus_refactored_Run4.fcl | New Run3/4 optical tune config (non-overlay). |
| fcl/detsim/detsim_2d_icarus_refactored_overlay.fcl | Enables DB effects + no-noise for overlay opdaq. |
| fcl/detsim/detsim_2d_icarus_refactored_overlay_Run4.fcl | New Run3/4 overlay tune config (DB + no-noise). |
| fcl/detsim/detsim_1d_icarus.fcl | Explicitly disables DB effects for non-overlay MC opdaq. |
| fcl/detsim/commissioning/run4759like_detsim_icarus.fcl | Switches includes to pmtsimulation_icarus tuning table. |
| fcl/detsim/commissioning/run4642like_detsim_icarus.fcl | Switches includes to pmtsimulation_icarus tuning table. |
| fcl/detsim/CMakeLists.txt | Adds partial detsim configs subdirectory. |
| fcl/decoder/decoder_run2_icarus.fcl | Updates inline comments; clarifies PMT discrimination purpose. |
| fcl/caf/cafmakerjob_icarus.fcl | Includes simulation services override. |
| CMakeLists.txt | Adds sbnalg dependency to build. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+51
to
+55
| * * `DBname` (default: `"pmt_voltage_data"`): database table name. | ||
| * * `StatusTag` (default: `""`): database tag to select. | ||
| * * `DefaultStatus` (default: `0` = kOff): status for channels absent from DB. | ||
| * * `Verbose` (default: `false`): print channel statuses when loading. | ||
| * * `LogCategory` (default: `"PMTChannelStatusProvider"`). |
mvicenzi
commented
Jun 13, 2026
mvicenzi
commented
Jun 13, 2026
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: mvicenzi <64699641+mvicenzi@users.noreply.github.com> Co-authored-by: Gianluca Petrillo <petrillo.at.work@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the required updates and fhicls to enable the reprocessing of light products for the SBN2025 spring productions (both MC and data). In particular, it includes fixes to the trigger/overlays procedure, as well as model improvements such as the use of time-dependent gain and pmt status databases.
Some highlights:
shiftedtrigger referenceDetectorClocksServiceconfiguration overrides for simulationdetsimoverlaysdetsimandstage0detsimandstage0for MC-overlays/datadetsimmoduleThis PR requires three
icarus_dataupdates:pmt_speareas_data.dbtov1r2pmt_voltage_data.dbtov1r3These requirements are satisfied by
icarus_datav10_20_04.This PR depends on:
(this PR includes developements previously discussed in #876 and #785 )