Using Binary File Format Description Languages for Documenting, Parsing, and Verifying Raw Data in TAIGA Experiment

12/04/2018
by   I. Bychkov, et al.
0

The paper is devoted to the issues of raw binary data documenting, parsing and verifying in astroparticle data lifecycle. The long-term preservation of raw data of astroparticle experiments as originally generated is essential for re-running analyses and reproducing research results. The selected high-quality raw data should have detailed documentation and accompanied by open software tools for access to them. We consider applicability of binary file format description languages to specify, parse and verify raw data of the Tunka Advanced Instrument for cosmic rays and Gamma Astronomy (TAIGA) experiment. The formal specifications are implemented for five data formats of the experiment and provide automatic generation of source code for data reading libraries in target programming languages (e.g. C++, Java, and Python). These libraries were tested on TAIGA data. They showed a good performance and help us to locate the parts with corrupted data. The format specifications can be used as metadata for exchanging of astroparticle raw data. They can also simplify software development for data aggregation from various sources for the multi-messenger analysis.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/11/2021

Integrating Structural Description of Data Format Information into Programming to Auto-generate File Reading Programs

File reading is the basis for data sharing and scientific computing. How...
research
04/10/2023

Interval Parsing Grammars for File Format Parsing

File formats specify how data is encoded for persistent storage. They ca...
research
12/15/2021

Next-generation Web Applications with WebAssembly and TruffleWasm

In modern software development, the JavaScript ecosystem of various fram...
research
09/05/2023

Parsing Fortran-77 with proprietary extensions

Far from the latest innovations in software development, many organizati...
research
11/11/2022

Anonymization of Whole Slide Images in Histopathology for Research and Education

Objective: The exchange of health-related data is subject to regional la...
research
10/11/2021

Parsing Data Formats of the Inputs and Outputs of Geographic Models with Code Analysis

Model web services provide an approach for implementing and facilitating...
research
01/05/2021

Efficient Data Management in Neutron Scattering Data Reduction Workflows at ORNL

Oak Ridge National Laboratory (ORNL) experimental neutron science facili...

Please sign up or login with your details

Forgot password? Click here to reset