Isaac Nealey
Michael E. Papka
Abstract
We present a containerized workflow demonstrating in situ analysis of simulation data rendered by a ParaView/Catalyst adapter for the generic SENSEI in situ interface, then streamed to a remote site for visualization. We use Cinema, a database approach for navigating the metadata produced in situ. We developed a web socket tool, cinema_transfer, for transferring the generated cinema databases to a remote machine while the simulation is running. We evaluate the performance of this containerized workflow and identify bottlenecks for large scale runs, in addition to testing identical containers at different sites with differing hardware and Message Passing Interface (MPI) implementations.
- 1.Ahrens, J., Jourdain, S., O’Leary, P., Patchett, J., Rogers, D.H., Petersen, M.: An image-based approach to extreme scale in situ visualization and analysis. In: SC 2014: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 424–434. IEEE (2014)Google Scholar
- 2.Ahrens, J., et al.: In situ mpas-ocean image-based visualization. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, Visualization & Data Analytics Showcase (2014)Google Scholar
- 3.Argonne Leadership Computing Facility. https://alcf.anl.gov, Accessed Mar 2022Google Scholar
- 4.Ayachit, U., et al.: Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures. In: SC 2016: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 921–932. IEEE (2016)Google Scholar
- 5.Ayachit, U., et al.: Paraview catalyst: Enabling in situ data analysis and visualization. In: Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization, pp. 25–29 (2015)Google Scholar
- 6.Cobalt HPC management suite. https://github.com/ido/cobalt, Accessed Mar 2022Google Scholar
- 7.Docker open platform for developing, shipping, and running applications. https://www.docker.com, Accessed Mar 2022Google Scholar
- 8.Docker tmpfs mounts. https://docs.docker.com/storage/tmpfs/, Accessed Mar 2022Google Scholar
- 9.Kubernetes: Production-Grade Container Orchestration. https://kubernetes.io, Accessed Mar 2022Google Scholar
- 10.LAAMPS Lennard Jones Benchmark. https://www.lammps.org/bench.html, Accessed Mar 2022Google Scholar
- 11.McMillan, S.: Making containers easier with hpc container maker (2018). https://github.com/HPCSYSPROS/Workshop18/blob/master/Making_Containers_Easier_with_HPC_Container_Maker/ws_hpcsysp103.pdfGoogle Scholar
- 12.MPICH: High Performance Portable MPI. https://www.mpich.org, Accessed Mar 2022Google Scholar
- 13.Nealey, I.: https://github.com/inealey/cinema_transfer, Accessed Mar 2022Google Scholar
- 14.Nealey, I.: https://github.com/inealey/sensei/tree/lammps, Accessed Mar 2022Google Scholar
- 15.Nealey, I., Ferrier, N., Insley, J., Mateevitsi, V.A., Papka, M.E., Rizzi, S.: Artifacts for woiv (2022). https://doi.org/10.5281/zenodo.6336286Google Scholar
- 16.OpenSSH: https://www.openssh.com, Accessed Mar 2022Google Scholar
- 17.Pacific Research Platform. https://pacificresearchplatform.org, Accessed Sept 2021Google Scholar
- 18.Singularity open source container platform. https://github.com/sylabs/singularity, Accessed Mar 2022Google Scholar
- 19.Singularity Persistent Overlays. https://sylabs.io/guides/3.5/user-guide/persistent_overlays.html, Accessed Mar 2022Google Scholar
- 20.Snyder, P.: tmpfs: a virtual memory file system. In: Proceedings of the Autumn 1990 EUUG Conference, pp. 241–248 (1990)Google Scholar
- 21.Will, M., Wofford, Q., Patchett, J., Rogers, D., Lukasczyk, J., Garth, C.: Developing and evaluating in situ visualization algorithms using containers, pp. 6–11. Association for Computing Machinery, New York (2021).
DOI:
https://doi.org/10.1145/3490138.3490141Google Scholar
Index Terms
(auto-classified)Cinema Transfer: A Containerized Visualization Workflow
Comments