The HDF Group's Call the Doctor
Hdf5-iotest & your HPC questions on Call the Doctor with Scot Breitenfeld 6/27/23
In this session of Call the Doctor, Scot Breitenfeld discusses a program called hdf5-iotest available on GitHub. The program helps explore data layout and performance optimization in HDF5. It allows users to experiment with parameters such as data dimensionality, chunking, metadata size, and MPIO operation. The program generates results and timings for different combinations of these parameters. Scot also mentions additional research additions, including support for collective metadata apis, compression filters, split I/O driver, and mpio driver. He demonstrates the effectiveness of the log-based VOL and the impact of tuning parameters with the DAOS VOL. Overall, the program aids in optimizing HDF5 performance and data organization.
You can also watch this episode.
Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions intended to help users tackle real-world HDF problems from a common cold to severe headaches and offer relief where that’s possible. As time permits, we will include how-tos, offer advice on tool usage, review your code samples, teach you survival in the documentation jungle, and discuss what’s new or just around the corner in the land of HDF.
Join us every Tuesday at 12:20 p.m. central (US/Canada.) on Zoom!
In this session of Call the Doctor, Scot Breitenfeld discusses a program called hdf5-iotest available on GitHub. The program helps explore data layout and performance optimization in HDF5. It allows users to experiment with parameters such as data dimensionality, chunking, metadata size, and MPIO operation. The program generates results and timings for different combinations of these parameters. Scot also mentions additional research additions, including support for collective metadata apis, compression filters, split I/O driver, and mpio driver. He demonstrates the effectiveness of the log-based VOL and the impact of tuning parameters with the DAOS VOL. Overall, the program aids in optimizing HDF5 performance and data organization.
You can also watch this episode.
Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions intended to help users tackle real-world HDF problems from a common cold to severe headaches and offer relief where that’s possible. As time permits, we will include how-tos, offer advice on tool usage, review your code samples, teach you survival in the documentation jungle, and discuss what’s new or just around the corner in the land of HDF.
Join us every Tuesday at 12:20 p.m. central (US/Canada.) on Zoom!