Kai L. Polsterer, Bernd Doser, Andreas Fehlner and Sebastian Trujillo-Gomez, Heidelberg Institute for Theoretical Studies, Heidelberg, Baden-Württemberg, Germany
Spherinator and HiPSter: Representation Learning for Unbiased Knowledge Discovery from Simulations
DOI: doi.org/10.48550/arXiv.2406.03810
Simulations are the best approximation to experimental laboratories in astrophysics and cosmology. However, the complexity, richness, and large size of their outputs severely limit the interpretability of their predictions. We describe a new, unbiased, and machine learning based approach to obtaining useful scientific insights from a broad range of simulations. The method can be used on today's largest simulations and will be essential to solve the extreme data exploration and analysis challenges posed by the Exascale era. Furthermore, this concept is so flexible, that it will also enable explorative access to observed data. Our concept is based on applying nonlinear dimensionality reduction to learn compact representations of the data in a low-dimensional space. The simulation data is projected onto this space for interactive inspection, visual interpretation, sample selection, and local analysis. We present a prototype using a rotational invariant hyperspherical variational convolutional autoencoder, utilizing a power distribution in the latent space, and trained on galaxies from IllustrisTNG simulation. Thereby, we obtain a natural Hubble tuning fork like similarity space that can be visualized interactively on the surface of a sphere by exploiting the power of HiPS tilings in Aladin Lite.
Find out more: [2406.03810] Spherinator and HiPSter: Representation Learning for Unbiased Knowledge Discovery from Simulations)
Download the article.
Bernd Doser, Kai L. Polsterer, Andreas Fehlner and Sebastian Trujillo-Gomez, Heidelberg Institute for Theoretical Studies, Heidelberg, Germany
Machine Learning Workflow for Morphological Classification of Galaxies
DOI: doi.org/10.48550/arXiv.2505.04676
As part of the EU-funded Center of Excellence SPACE (Scalable Parallel Astrophysical Codes for Exascale), seven commonly used astrophysics simulation codes are being optimized to exploit exascale computing platforms. Exascale cosmological simulations will produce large amounts of data (i.e. several petabytes) that will soon be waiting to be analyzed, with enormous potential for scientific breakthroughs. Our tool Spherinator enables the reduction of these complex data sets to a low-dimensional space using Generative Deep Learning to understand the morphological structure of simulated galaxies. A spherical latent space allows the HiPSter module to provide explorative visualization using Hierarchical Progressive Surveys (HiPS) in the Aladin software.
Here we present a machine-learning workflow covering all stages, from data collection to preprocessing, training, prediction, and final deployment. This workflow ensures full reproducibility by keeping track of the code, data, and environment. Additionally, the workflow allows for scalability in managing a large amount of data and complex pipelines. We use only open-source software and standards that align with the FAIR (Findability, Accessibility, Interoperability and Reproducibility) principles. In this way, we are able to distribute the workflow reliably and enable collaboration by sharing code, data, and results efficiently.
Find out more: [2505.04676] Machine Learning Workflow for Morphological Classification of Galaxies)
Download the article.
Fenja Schweder1,2, Sebastian Trujillo-Gomez2, and Kai Polsterer2
- 1. University of Bremen, Bremen, Germany
2. Heidelberg Institute for Theoretical Studies, Heidelberg, Germany
JAvaScript Multimodal INformation Explorer
DOI: doi.org/10.48550/arXiv.2504.21393
Astronomical data is rich in volume, information and facets. Although this offers multiple research perspectives, processing the data remains a challenge. Infrastructures for analyzing, inspecting, exploring and communicating with data are mandatory. To address this issue, we introduce Jasmine, the JAvaScript Multimodal INformation Explorer. Jasmine allows users to open different data viewer modals that show a specific data point from a set. The viewer currently supports image data, as well as point cloud objects. Users can decide on which information about the data point they like to have displayed. Point clouds are interactive and allow for zooming, tossing, and turning. Picking a data point is enabled by providing a structured view of the set, arranged by a key property. This arrangement is achieved by autoencoding.
Find out more: [2504.21393] JAvaScript Multimodal INformation Explorer)
Download the article.
Nicola Tuccari; Eva Sciacca; Fabio Vitello; Iacopo Colonnelli; Yolanda Becerra; Enric Sosa Cintero; Guillermo Marin; Milan Jaros; Lubomir Riha; Petr Strakos; Sebastian Trujillo-Gomez; Emiliano Tramontana; Robert Wissing
High Performance Visualization for Astrophysics and Cosmology
2025 33rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP), Turin, Italy, 2025, pp. 443-450.
DOI: 10.1109/PDP66500.2025.00069
Modern Astrophysics and Cosmology (A&C) projects produce immense data volumes, necessitating advanced software tools for data access, storage, and analysis. Visualization Interface for the Virtual Observatory (VisIVO) is one such tool enabling multi-dimensional data analysis and knowledge discovery across complex astrophysical datasets. Leveraging containerization and virtualization, VisIVO has been deployed on various distributed computing platforms. Additionally, Blender, an open-source 3D suite, provides robust tools for rendering and processing volumetric data, making it suitable for visualizing complex datasets. At the SPACE Center of Excellence these tools are being adapted for high-performance visualization of cosmological simulations performed with GADGET and ChaNGa on pre-exascale systems. However, implementing high-performance visualization on diverse HPC platforms presents several challenges, including hardware and software compatibility, data management, scalability, performance portability, and efficient resource allocation. This paper outlines strategies to integrate VisIVO with workflow frameworks and streaming platforms to address these challenges. Workflow frameworks enhance portability, scheduling, and reproducibility of visualization workflows on pre-exascale systems used in A&C simulations. We also discuss the use of streaming platforms to enable concurrent (i.e. in-situ) analysis and visualization of simulations, reducing the need to store full simulation data by leveraging distributed databases that stream the output data in real time. Lastly, we present an adaptation of Blender to handle large-scale particle-based astrophysical data, offering high-quality visualization with interactive exploration capabilities.
Find out more: High Performance Visualization for Astrophysics and Cosmology | IEEE Conference Publication | IEEE Xplore
Eva Sciacca, Valentina Cesare, Nicola Tuccari, Fabio Vitello, Iacopo Colonnelli, Carmelita Carbone, Emiliano Tramontana, Ugo Becciani
Toward a Portable and Reproducible High-performance Visualization for Astronomy & Cosmology
16th International Workshop on Science Gateways (IWSG2024), Toulouse, France. doi.org/10.5281/zenodo.13858498
Modern Astronomy and Cosmology (A&C) generate petabyte-scale data volumes requiring the development of a new eneration of software tools to access, store, and process them. The Visualization Interface for the Virtual Observatory (VisIVO) tool performs multi-dimensional data analysis and knowledge discovery in multivariate astrophysical datasets. Thanks to containerization and virtualization technologies, VisIVO has already been exploited on top of several distributed computing infrastructures including the European Open Science Cloud (EOSC). In this work, we outline the execution strategies designed to enhance the portability and reproducibility of the VisIVO modular applications for high performance visualization of data generated on the (pre-)exascale systems by HPC A&C simulations performed with the GADGET (GAlaxies with Dark matter and Gas) code.
Download the article.
Giuliano Taffoni, Andrea Mignone, Luca Tornatore, Eva Sciacca, Massimiliano Guarrasi, Giovanni Lapenta, Lubomir Riha, Radim Vavrik, Ondrej Vysocky, Kristian Kadlubiak, Petr Strakos, Milan Jaros, Klaus Dolag, Benoit Commercon, Luciano Rezzolla, Khalil Pierre, Georgios Doulis, Sijing Shen, Manolis Marazakis, Daniele Gregori, Elisabetta Boella, Gino Perna, Marisa Zanotti, Erwan Raffin, Kai Polsterer, Sebastian Trujillo Gomez, Guillermo Marin
Redesign of astrophysical codes for exascale computing: the SPACE experience
Proceedings Volume 13101, Software and Cyberinfrastructure for Astronomy VIII; 131010X (2024) doi.org/10.1117/12.3020361
Event: SPIE Astronomical Telescopes + Instrumentation, 2024, Yokohama, Japan
High Performance Computing based simulations are crucial in Astrophysics and Cosmology, helping scientists investigate and understand complex astrophysical phenomena. Taking advantage of Exascale computing capabilities is essential for these efforts. However, the unprecedented architectural complexity of exascale systems impacts simulation codes. The SPACE Center of Excellence aims to re-engineer key astrophysical codes to adapt to these new computational challenges by adopting innovative programming paradigms and software solutions. Through co-design activities, SPACE brings together scientists, code developers, HPC experts, hardware manufacturers, and software developers. This collaboration enhances exascale astrophysics and cosmology applications, promoting the use of exascale and post-exascale computing capabilities. Additionally, SPACE addresses high-performance data analysis for the massive data outputs from exascale simulations, using machine learning and visualization tools. The project facilitates application deployment across platforms by focusing on code repositories and data sharing, integrating European astrophysical communities around exascale computing with standardized software and data protocols. In this paper, we present the SPACE Center of Excellence and the preliminary results achieved by the project.
Find out more here.
CINECA HPC Team
HPC Annual Report 2023/2024
CINECA HPC is the main Centre for scientific computing in Italy.
It runs a large computing infrastructure and makes it available to Italian and European Researchers, as well as to important Italian Companies within a programme for supporting national industrial competitiveness.
This report presents data about usage and users, projects, events and classes offered in the field of High Performance Computing.
SPACE CoE is one of the EU projects CINECA is involved in and such engagement and information about our Centre of Excellence is reported in this document (pags. 88-91).
Download the full report (pdf format) or find out more here.
V. Berta, A. Mignone, M. Bugli, G. Mattia
“A 4th-order accurate finite volume method for ideal classical and special relativistic MHD based on pointwise reconstructions”,
Journal of Computational Physics 499 (2024) 112701
We present a novel implementation of a genuinely 4th-order accurate finite volume scheme
for multidimensional classical and special relativistic magnetohydrodynamics (MHD) based on
the constrained transport (CT) formalism.
The scheme introduces several novel aspects when
compared to its predecessors yielding a more efficient computational tool.
Among the most relevant ones, our scheme exploits pointwise to pointwise reconstructions (rather than onedimensional
finite volume ones), employs the generic upwind constrained transport averaging
and sophisticated limiting strategies that include both a discontinuity detector and an order
reduction procedure. Selected numerical benchmarks demonstrate the accuracy and robustness
of the method.
Download the article (pdf format).
Bernhard Baumschlager, Sijing Shen, and James W. Wadsley
“Spectral reconstruction for radiation hydrodynamic simulations of galaxy evolution”,
Astronomy & Astrophysics manuscript no. 10BandRT - October 27, 2023
Radiation from stars and active galactic nuclei (AGN) plays an important role in galaxy formation and evolution, and profoundly
transforms the intergalactic, circumgalactic and interstellar medium (IGM, CGM & ISM). On-the-fly radiative transfer (RT) has
started being incorporated in cosmological simulations, but the complex, evolving radiation spectra are often crudely approximated
with a small number of broad bands with piece-wise constant intensity and a fixed photo-ionisation cross-section.
Such a treatment is unable to capture the changes to the spectrum as light is absorbed while it propagates through a medium with non-zero opacity. This can lead to large errors in photo-ionisation and heating rates. In this work, we present a novel approach of discretising the radiation field in narrow bands, located at the edges of the typically used bands, in order to capture the power-law slope of the radiation field.
In combination with power-law approximations for the photo-ionisation cross-sections, this model allows us to self-consistently combine
radiation from sources with different spectra and accurately follow the ionisation states of primordial and metal species through time.
The method is implemented in Gasoline2 in connection with Trevr2, a fast reverse RT algorithm with O(Nactive log2 N) scaling.
We compare our new piece-wise power-law reconstruction to the piece-wise constant method in calculating the primordial chemistry
photo-ionisation and heating rates under an evolving UV-background (UVB) and stellar spectrum, and find that our method reduces
errors significantly, up to two orders of magnitude in the case of HeII ionisation.
We apply our new spectral reconstruction method in RT post-processing of a cosmological zoom-in simulation, MUGS2 g1536 (Keller et al. 2016), including radiation from stars and a
live UVB, and find a significant increase in total neutral hydrogen (HI) mass in the ISM and the CGM due to shielding of the UVB
and a low escape fraction of the stellar radiation. This demonstrates the importance of RT and an accurate spectral approximation in
simulating the CGM-galaxy ecosystem.
Download the article (pdf format).
M. Zanotti, A. Mignone, and M. Marazakis
“SPACE: Space Parallel Astrophysical Codes for Exascale”,
ERCIM News No. 134, July 2023, pp. 43-44
In astrophysics and cosmology (A&C), high-performance computing (HPC)-based numerical simulations are invaluable
instruments to support scientific discovery.
The efficient and effective exploitation of exascale (and beyond)
computing capabilities will be key to paving the way for scientific discovery in this scientific domain, but requires a
coordinated effort towards adapting A&C codes for space applications on exascale HPC systems.
Go to the article webpage or download the magazine (pdf format).
