First ProtoDUNE-SP data challenge: finding the gaps

Image: dqweek.com

“We are learning essential steps towards making full use of the potential of the ProtoDUNE detectors in support of the final DUNE construction, operation and science outputs,” said Ruth Pordes, referring to the first ProtoDUNE-SP data challenge, which took place last week. Many people from ProtoDUNE-SP working groups and DUNE software and computing participated, working together to test the end-to-end offline system.

The team moved 28 TB of existing data files — in parallel — from an emulated data acquisition buffer at CERN to a designated area of CERN’s distributed disk storage and also to tape at both CERN and Fermilab. The data were simultaneously fed to the ProtoDUNE prompt processing system (p3s), which supports the crucial data quality monitoring (DQM) function in the experiment. The DQM transfers automatically trigger execution of the detector characterization algorithms and generate event displays.

The data files get registered in the data catalog (see related article “The Official Data Catalog, October 2017) as being located at both CERN and Fermilab. Arrival of data at Fermilab automatically triggers production processing jobs. During the data challenge the team ran jobs through the job and data management infrastructure alone, the art/LArSoft executable with no physics algorithm workflow, and the full reconstruction chain that will be used for upcoming Monte Carlo simulations.

“In parallel we are testing with both fake and existing beam instrumentation data from CERN,” Pordes said, “moving the data to Fermilab databases, and accessing the information with the data processing applications.” The team is paying attention to the available end-to-end throughput, monitoring and debugging capabilities, as well as to the accessibility of web-based information and documentation.

“This data challenge has given us necessary information about the gaps in functionality and performance relative to what we need for data taking,” said Pordes. “We are also using it as a means to understand the operational requirements and resources that are needed for sustained running.”

This data challenge has also served to solidify the team that will be able to effectively provide and support offline computing and achieve the goals of ProtoDUNE-SP.

Plans call for another ProtoDUNE-SP data challenge as well as a joint one with ProtoDUNE-DP before commissioning the full detectors in 2018. These will include the sustained transfer of data from the front-end online data buffers, demonstrating the full throughput and performance needed for data taking, and ensuring that the two ProtoDUNE experiments can move and process data together effectively.

The team:

  • Beamline instrumentation database: Igor Mandrichenko, Jon Paley (both of Fermilab)
  • DAQ/Online: Geoff Savage (Fermilab)
  • Data movement and management: Steve Timm, Stu Fuess, Igor Mandrichenko (all of Fermilab)
  • Data quality monitoring: Maxim Potekhin (BNL)
  • Neutrino Platform and CERN IT Liaisons: Nectarios Benekos, Xavier Espinal (both of CERN)
  • Physics payloads: Robert Sulej (NCBJ/Fermilab), Dorota Stefan (CERN)
  • Production processing: Anna Mazzacane (Fermilab), Ivan Furic (University of Florida)
  • DUNE software support: Tom Junk (Fermilab)

Read more on the DUNE wiki: https://wiki.dunescience.org/wiki/Data_Challenges.