LightAMR format standard and lossless compression algorithms for adaptive mesh refinement grids: RAMSES use case

by   Loic Strafella, et al.

The evolution of parallel I/O library as well as new concepts such as 'in transit' and 'in situ' visualization and analysis have been identified as key technologies to circumvent I/O bottleneck in pre-exascale applications. Nevertheless, data structure and data format can also be improved for both reducing I/O volume and improving data interoperability between data producer and data consumer. In this paper, we propose a very lightweight and purpose-specific post-processing data model for AMR meshes, called lightAMR. Based on this data model, we introduce a tree pruning algorithm that removes data redundancy from a fully threaded AMR octree. In addition, we present two lossless compression algorithms, one for the AMR grid structure description and one for AMR double/single precision physical quantity scalar fields. Then we present performance benchmarks on RAMSES simulation datasets of this new lightAMR data model and the pruning and compression algorithms. We show that our pruning algorithm can reduce the total number of cells from RAMSES AMR datasets by 10-40 RAMSES AMR grid structure can be compacted by   3 orders of magnitude and the float scalar fields can be compressed by a factor   1.2 for double precision and   1.3 - 1.5 in single precision with a compression speed of   1 GB/s.


page 1

page 2

page 3

page 4


Boosting I/O and visualization for exascale era using Hercule: test case on RAMSES

It has been clearly identified that I/O is one of the bottleneck to exte...

CityJSON: A compact and easy-to-use encoding of the CityGML data model

The international standard CityGML is both a data model and an exchange ...

SkyCell: A Space-Pruning Based Parallel Skyline Algorithm

Skyline computation is an essential database operation that has many app...

SARS-CoV-2 Coronavirus Data Compression Benchmark

This paper introduces a lossless data compression competition that bench...

Compressing Volumetric Radiance Fields to 1 MB

Approximating radiance fields with volumetric grids is one of promising ...

TRAKO: Efficient Transmission of Tractography Data for Visualization

Fiber tracking produces large tractography datasets that are tens of gig...

Please sign up or login with your details

Forgot password? Click here to reset