• Skip to sidebar navigation
  • Skip to content

Bitbucket

  • Projects
  • Repositories
    • View all public repositories
  • Help
    • Online help
    • Learn Git
    • Welcome to Bitbucket
    • Keyboard shortcuts
  • Log In
Elena Pourmal
  1. Elena Pourmal

hdf5_ep

HDF5
hdf5
Public
Actions
  • Clone
  • Download

Learn more about cloning repositories

You have read-only access

Navigation
  • Source
  • Commits
  • Graphs
  • Branches
  • Network
  • Latest Activities

Commits

MuQun Yang
a86779f3f79
MuQun Yang committed 8933c59694620 Jul 2004
[svn-r8906] Purpose:
Adding the first round of patches about supporting collective chunk IO in HDF5

Description:
The current HDF5 library doesn't support collective MPIO with chunk storage. When users set collective option in their data transfer with chunk storage, the library silently converted the option to INDEPENDENT and that caused trememdous performance penalty. Some application like
WRF-parallel HDF5 IO module has to use contiguous storage for this reason. However, chunking storage has its own advantage(supporting compression filters and extensible dataset), so to make collective MPIO possible inside HDF5 with chunking storage is a very important task.

This check-in make collective chunk IO possible for some special cases. The condition is as follows(either case is fine with using collective chunk IO)
1. for each process, the hyperslab selection of the file data space of each dataset is regular and it is fit in one chunk.

2. for each process, the hyperslab selection of the file data space of each dataset is single and the number of chunks for the hyperslab selection should be equal.

Solution:
Lift up the contiguous storage requirement for collective IO.
Use H5D_isstore_get_addr to get the corresponding chunk address. Then the original library routines will take care of getting the correct address to make sure that MPI FILE TYPE is built correctly for collective IO>

Platforms tested:
arabica(sol), copper(AIX), eirene(Linux)
parallel test is checked at copper.

Misc. update:

Changed files

  • Git repository management for enterprise teams powered by Atlassian Bitbucket
  • Atlassian Bitbucket v4.4.1
  • Documentation
  • Contact Support
  • Request a feature
  • About
  • Contact Atlassian
Atlassian