Author Commit Message Commit Date David Young eaeede0120a Delete the hdf5_page_offset member of shadow_defree_t, it is initialized but
never read. 05 Feb 2020 David Young 4ffc6839e1b Fix a typo in a diagnostic printf and, while I'm here, use the right format
strings. 05 Feb 2020 David Young bc27f0d1ce3 Use proper format string. 05 Feb 2020 David Young 9a8ba2a7d36 Make H5FD_vfd_swmr_dump_status() take an unsigned page number to be consistent
with other code. 04 Feb 2020 David Young 456f5c6ee61 Quiet some warnings about jumping over variable initialization,
an improper format string, etc. NFCI. 04 Feb 2020 David Young 70515b290f8 Rename the deferred-free queues. There are two of them, and the names I used
for them did not help me keep track of what they were for.
For brevity, I will call a deferred free record a "defree" in the code.
The deferred_free_queue_t becomes a lower_defree_queue_t, and each record on
the queue becomes a lower_defree_t. A lower_defree_t tracks one deferred free
on the lower VFD---that is, the one under the SWMR VFD.
The old_image_queue_t becomes a shadow_defree_queue_t, and a record there... 04 Feb 2020 David Young 3c4a34951e6 Move the bsdqueue.h #inclusion up with the other #inclusions. 04 Feb 2020 Mike Folk fbcbf769077 Update VFD_SWMR_Punch_List.md 06 Feb 2020 Mike Folk ae61a8e8261 Update VFD_SWMR_Punch_List.md 04 Feb 2020 David Young 3c659197766 NFCI: exit(EXIT_FAILURE) instead of exit(1) for portability.
Delete a comment on a closing curly brace. 03 Feb 2020 David Young 036dd539d3a It's not necessary to assert(p != NULL) if you're dereferencing p in the next
assertion. 03 Feb 2020 David Young 29ee63787b8 Numerous changes supporting a floating shadow index:
Add to the H5F_shared_t (!) a new member that tells the index in
the shadow file where the index should be written.
Allocate shadow filespace for the header and the index separately so
that the index can float. Update tests to match the expected original
location of the index.
Introduce vfd_swmr_enlarge_shadow_index(), a routine that allocates space in
the shadow file for a new index that has (up to) twice as many entries as the
old ind... 03 Feb 2020 David Young 64f1ef1fda4 Make the first argument to H5PB_vfd_swmr__update_index() and H5F_t instead of
an H5F_shared_t because the new routine that will relocate the index (which
will be in a future commit) has to pass an H5F_t to the filespace allocator. 03 Feb 2020 David Young 005bd89ad5f Use FAIL_STACK_ERROR instead of TEST_ERROR after library calls, so that the
error stack will be reported.
Use calloc instead of malloc+memset. 03 Feb 2020 David Young 57fe4a52a26 Consolidate shadow-file allocation code in H5MV.c. 30 Jan 2020 David Young 590c51b68c6 Remove some dead code and out-of-date comments. 30 Jan 2020 David Young 2e15f48d789 Repair indentation and delete some superfluous comments. NFCI. 30 Jan 2020 David Young b7f5776c531 Fix typo. NFCI. 30 Jan 2020 David Young cb66576c7ae Use fewer gratuitous casts. Use struct assignment instead of member-by-member
assignment to copy `struct timespec`. NFCI. 28 Jan 2020 David Young b34e6d2f458 Add some missing spaces. 27 Jan 2020 David Young 59e7b10a7e3 Fix spelling. NFCI. 27 Jan 2020 David Young 0fbd8654f7d Prepare the VFD SWMR reader for a "floating" shadow index by overhauling
the way that the shadow header and shadow index are loaded.
In H5FD__vfd_swmr_load_hdr_and_idx(), adopt a new protocol for reading
the shadow file:
0 If the maximum number of retries have been attempted, then exit
with an error.
1 Try to read the shadow file *header*. If successful, continue to 2.
If there is a hard failure, then return an error. If there is a failure
that may be transient, then sleep and ret... 24 Jan 2020 David Young 33ea1d2b066 Add my newest commentary. Finally reached page 14! 17 Jan 2020 David Young 2754bb2b764 Change md_pages_reserved and pb_expansion_threshold parameters to uint32_t from
int32_t. 16 Jan 2020 David Young fe7965d38f8 Don't use the bash-ism [[ ]]. Use the [ ], which is standard and
perfectly adequate in this case. 13 Jan 2020 David Young 16ca8f2c954 Prepare a bit to let the shadow index "float": drastically
simplify H5FD__vfd_swmr_index_deserialize(): reuse
h5_retry_init()/h5_retry_next() for retry loops.
Don't wait for the fstat(2) to read the correct size, because the
read(2) will return short if the file isn't long enough. (This change
should save at least one system call, always.)
Leave a bunch of comments about the changes that I will have to make so
that the shadow index will float.
NFCI: do not cast H5MM_malloc() return values... 10 Jan 2020 David Young 5fb463bb2e7 Now that we're buffering the removed pages to process at once,
it's not necessary to iterate over pass numbers any more,
so just use two loops.
While I'm in here, change a comment or formatting here and there. 10 Jan 2020 David Young 7a0a2a0b5dc H5F_vfd_swmr_reader_end_of_tick(),
compares both new and old shadow indices and calls H5PB_remove_entry()
on each entry that was in the old index but is not in the new.
Ever since H5PB_remove_entry() started removing shadow index
entries, it has been possible for H5F_vfd_swmr_reader_end_of_tick()
to walk past the end of the new shadow index or even to skip entries
in the new index. Sometimes an assertion failed when that happened.
I have restructured the code in H5F_vfd_swmr_reader_end_of_... 10 Jan 2020 David Young cdbdc33c5a5 Simplify nested logic in H5FD_vfd_swmr_get_tick_and_idx().
assert() our expectation that either a NULL index was passed, or a
pointer to the index length was passed in, too. 10 Jan 2020 David Young 671f7552ab5 Correct some irregular indentation, delete an unused line, split an
`if ((p = allocate(...)) == NULL) { }` into two statements, `p =
allocate(...); if (p == NULL) { }`, put a semicolon at the end of an
HGOTO_ERROR(), remove comments /* end if */, /* end for */ after closing
curly braces. 10 Jan 2020 David Young f19b6c4d485 Make H5FD__vfd_swmr_load_hdr_and_idx() simpler and faster: copy a
pointer to the metadata index instead of copying the index itself. Use
struct assignment instead of copying individual struct members. Lower a
staircase. 10 Jan 2020 David Young c4e19fa84bf After quick-sorting the index, go ahead and assert that there are not
duplicate entries for the same HDF5 page offset. There's an O(n) cost
to that, but it should be no more than the O(n log n) cost of the
quicksort. 10 Jan 2020 David Young d288a7b93c9 In H5F__vfd_swmr_writer__create_index(), use calloc(3) instead of
malloc(3) and a for-loop that zeroes the index. 10 Jan 2020 David Young 8d2217d2955 In H5PB_remove_entry(), do not call vfd_swmr_mdf_idx_entry_remove(),
because the H5PB__evict_entry() call should have already done that.
Instead, just assert() that the index entry is not present. 10 Jan 2020 David Young 790eccbc20b In vfd_swmr_mdf_idx_entry_remove(), give some local variables less wordy
names. While I am here, do not copy the last element of the index over
the element that's being deleted, because in the very next step I'm
shifting all elements over by one. 10 Jan 2020 David Young f4d8894158e Straggler from previous: provide uint64_decode(). 10 Jan 2020 David Young 3fd9dc7b83f Clean up the code a bit: remove some unnecessary casts and such. NFCI. 10 Jan 2020 David Young 3a911e2b39b Change the blah_blah_blah_md_header `index_length` member from
`uint64_t` to `size_t` because it describes the size of an in-core
structure as well as an on-disk one, and `size_t` is wide enough
to store the size of any in-core structure, while `uint64_t` may
be much too wide. Check that `index_length` is no more than SIZE_MAX
after we read it. 10 Jan 2020 David Young 1df69ed3a09 Delete unused constants. 10 Jan 2020 David Young 5ae8d135a54 Be brief, replace `(H5FS_section_info_t *)node` with `&node->sect_info`. 10 Jan 2020 David Young f8c5797bd62 Provide a convenience constant, H5_RETRY_ONE_SECOND. 10 Jan 2020 David Young 62e1340bdd9 Disable C++ compatibility warnings to follow `develop` on the main fork. 10 Jan 2020 David Young c09a9e90e1e Update punchlist a bit with John: add some items, update entries that are now
complete. 09 Jan 2020 David Young 0b11cfbf50b Add a new version of the RFC with some commentary by me. 08 Jan 2020 David Young 0ec3340692a Makes some updates: in-progress, needs unit testing. Insert some whitespace
where it makes things more readable. 06 Jan 2020 David Young 12ea13f745f Mention the current situation concerning raw-data flushing. 31 Dec 2019 David Young 68b012f0175 Mention that I fixed the VFD SWMR tests. 30 Dec 2019 David Young 0b5dfad751c Don't cast a malloc(3), this is C. Put curly braces around a multiline `if`
statement. Delete an unnecessary backslash line continuation. NFCI. 20 Dec 2019 David Young e024b2737c9 Assert that a shadow image is page-aligned since we assume that it is when we
compute its base address and free it. 20 Dec 2019 David Young 37fb4bda711 Delete unnecessary whitespace including invisible whitespace at the end of
lines. Wrap a couple of lines. NFCI. 20 Dec 2019 ← Prev Next → require('plugin/commitgraph/network').applyCommits([{id: 'eaeede0120a9bfefac2080f1af5ce3c062f0f0c6', href: '/users/brtnfld/repos/hdf5_msb/commits/eaeede0120a9bfefac2080f1af5ce3c062f0f0c6',parents: [{ id: '4ffc6839e1b432e996f974dc3269e1fd2a48f2c0' }]},{id: '4ffc6839e1b432e996f974dc3269e1fd2a48f2c0', href: '/users/brtnfld/repos/hdf5_msb/commits/4ffc6839e1b432e996f974dc3269e1fd2a48f2c0',parents: [{ id: 'bc27f0d1ce31c4623a7b728be6638fba0d8a9b53' }]},{id: 'bc27f0d1ce31c4623a7b728be6638fba0d8a9b53', href: '/users/brtnfld/repos/hdf5_msb/commits/bc27f0d1ce31c4623a7b728be6638fba0d8a9b53',parents: [{ id: '9a8ba2a7d36f2f91b298746e6f43956dfd34e06c' }]},{id: '9a8ba2a7d36f2f91b298746e6f43956dfd34e06c', href: '/users/brtnfld/repos/hdf5_msb/commits/9a8ba2a7d36f2f91b298746e6f43956dfd34e06c',parents: [{ id: '456f5c6ee61779a992ec8901a93e43d25c19acc3' }]},{id: '456f5c6ee61779a992ec8901a93e43d25c19acc3', href: '/users/brtnfld/repos/hdf5_msb/commits/456f5c6ee61779a992ec8901a93e43d25c19acc3',parents: [{ id: '70515b290f883c0915543fef44a0f0c2c94073f8' }]},{id: '70515b290f883c0915543fef44a0f0c2c94073f8', href: '/users/brtnfld/repos/hdf5_msb/commits/70515b290f883c0915543fef44a0f0c2c94073f8',parents: [{ id: '3c4a34951e6591bf7acd09b16dc0b3e310a2e8ab' }]},{id: '3c4a34951e6591bf7acd09b16dc0b3e310a2e8ab', href: '/users/brtnfld/repos/hdf5_msb/commits/3c4a34951e6591bf7acd09b16dc0b3e310a2e8ab',parents: [{ id: 'fbcbf7690774a93f335cf9e95063dec47e771a07' }]},{id: 'fbcbf7690774a93f335cf9e95063dec47e771a07', href: '/users/brtnfld/repos/hdf5_msb/commits/fbcbf7690774a93f335cf9e95063dec47e771a07',parents: [{ id: 'ae61a8e826114fb676934d63d5a8acc443024a56' }]},{id: 'ae61a8e826114fb676934d63d5a8acc443024a56', href: '/users/brtnfld/repos/hdf5_msb/commits/ae61a8e826114fb676934d63d5a8acc443024a56',parents: [{ id: '3c659197766ebda05915cbb8a079dcf16e91fb25' }]},{id: '3c659197766ebda05915cbb8a079dcf16e91fb25', href: '/users/brtnfld/repos/hdf5_msb/commits/3c659197766ebda05915cbb8a079dcf16e91fb25',parents: [{ id: '036dd539d3ade3fa5aebed87339b620b4c926293' }]},{id: '036dd539d3ade3fa5aebed87339b620b4c926293', href: '/users/brtnfld/repos/hdf5_msb/commits/036dd539d3ade3fa5aebed87339b620b4c926293',parents: [{ id: '29ee63787b8b75bdbf16468c9a7d9955e55eec78' }]},{id: '29ee63787b8b75bdbf16468c9a7d9955e55eec78', href: '/users/brtnfld/repos/hdf5_msb/commits/29ee63787b8b75bdbf16468c9a7d9955e55eec78',parents: [{ id: '64f1ef1fda4dddb85ff16166051c7df3777254d7' }]},{id: '64f1ef1fda4dddb85ff16166051c7df3777254d7', href: '/users/brtnfld/repos/hdf5_msb/commits/64f1ef1fda4dddb85ff16166051c7df3777254d7',parents: [{ id: '005bd89ad5fbe958e946ff4381bc9f97babc1896' }]},{id: '005bd89ad5fbe958e946ff4381bc9f97babc1896', href: '/users/brtnfld/repos/hdf5_msb/commits/005bd89ad5fbe958e946ff4381bc9f97babc1896',parents: [{ id: '57fe4a52a265edb642855a5be11d0ccd7c4b1b97' }]},{id: '57fe4a52a265edb642855a5be11d0ccd7c4b1b97', href: '/users/brtnfld/repos/hdf5_msb/commits/57fe4a52a265edb642855a5be11d0ccd7c4b1b97',parents: [{ id: '590c51b68c61e5e35097523f96d3d9b0bb43a083' }]},{id: '590c51b68c61e5e35097523f96d3d9b0bb43a083', href: '/users/brtnfld/repos/hdf5_msb/commits/590c51b68c61e5e35097523f96d3d9b0bb43a083',parents: [{ id: '2e15f48d7893bda94358cac42c4f57cf7b1a9aa9' }]},{id: '2e15f48d7893bda94358cac42c4f57cf7b1a9aa9', href: '/users/brtnfld/repos/hdf5_msb/commits/2e15f48d7893bda94358cac42c4f57cf7b1a9aa9',parents: [{ id: 'b7f5776c53191bcdbe8eb81c53e18cc05443c1ad' }]},{id: 'b7f5776c53191bcdbe8eb81c53e18cc05443c1ad', href: '/users/brtnfld/repos/hdf5_msb/commits/b7f5776c53191bcdbe8eb81c53e18cc05443c1ad',parents: [{ id: 'cb66576c7aec2058173083f4601f448e5f51cfb8' }]},{id: 'cb66576c7aec2058173083f4601f448e5f51cfb8', href: '/users/brtnfld/repos/hdf5_msb/commits/cb66576c7aec2058173083f4601f448e5f51cfb8',parents: [{ id: 'b34e6d2f458543fa8ec2bc6b01b26e49f78e3763' }]},{id: 'b34e6d2f458543fa8ec2bc6b01b26e49f78e3763', href: '/users/brtnfld/repos/hdf5_msb/commits/b34e6d2f458543fa8ec2bc6b01b26e49f78e3763',parents: [{ id: '59e7b10a7e352f786493816a86bd45764241f1aa' }]},{id: '59e7b10a7e352f786493816a86bd45764241f1aa', href: '/users/brtnfld/repos/hdf5_msb/commits/59e7b10a7e352f786493816a86bd45764241f1aa',parents: [{ id: '0fbd8654f7d5af36621cb4588936c16606bbe298' }]},{id: '0fbd8654f7d5af36621cb4588936c16606bbe298', href: '/users/brtnfld/repos/hdf5_msb/commits/0fbd8654f7d5af36621cb4588936c16606bbe298',parents: [{ id: '33ea1d2b066b1c8b26f6f2df386b5c9ced3a1dc5' }]},{id: '33ea1d2b066b1c8b26f6f2df386b5c9ced3a1dc5', href: '/users/brtnfld/repos/hdf5_msb/commits/33ea1d2b066b1c8b26f6f2df386b5c9ced3a1dc5',parents: [{ id: '2754bb2b764a22aac5492ea4ba5d023e08c8eebb' }]},{id: '2754bb2b764a22aac5492ea4ba5d023e08c8eebb', href: '/users/brtnfld/repos/hdf5_msb/commits/2754bb2b764a22aac5492ea4ba5d023e08c8eebb',parents: [{ id: 'fe7965d38f818e0c4d0154c0172703fbe71aec8d' }]},{id: 'fe7965d38f818e0c4d0154c0172703fbe71aec8d', href: '/users/brtnfld/repos/hdf5_msb/commits/fe7965d38f818e0c4d0154c0172703fbe71aec8d',parents: [{ id: '16ca8f2c95416a09e437545333b3bb9a2f48d84c' }]},{id: '16ca8f2c95416a09e437545333b3bb9a2f48d84c', href: '/users/brtnfld/repos/hdf5_msb/commits/16ca8f2c95416a09e437545333b3bb9a2f48d84c',parents: [{ id: '5fb463bb2e793e515ad91aba05d387d9c2f39637' }]},{id: '5fb463bb2e793e515ad91aba05d387d9c2f39637', href: '/users/brtnfld/repos/hdf5_msb/commits/5fb463bb2e793e515ad91aba05d387d9c2f39637',parents: [{ id: '7a0a2a0b5dc62f0341c1d3628ae9ff53dcea086d' }]},{id: '7a0a2a0b5dc62f0341c1d3628ae9ff53dcea086d', href: '/users/brtnfld/repos/hdf5_msb/commits/7a0a2a0b5dc62f0341c1d3628ae9ff53dcea086d',parents: [{ id: 'cdbdc33c5a5eb2bbca91167234c465c2b1f31ded' }]},{id: 'cdbdc33c5a5eb2bbca91167234c465c2b1f31ded', href: '/users/brtnfld/repos/hdf5_msb/commits/cdbdc33c5a5eb2bbca91167234c465c2b1f31ded',parents: [{ id: '671f7552ab56a446bb3b1227d0618e4add965316' }]},{id: '671f7552ab56a446bb3b1227d0618e4add965316', href: '/users/brtnfld/repos/hdf5_msb/commits/671f7552ab56a446bb3b1227d0618e4add965316',parents: [{ id: 'f19b6c4d485bd34cba2c5825718c4330664ad54f' }]},{id: 'f19b6c4d485bd34cba2c5825718c4330664ad54f', href: '/users/brtnfld/repos/hdf5_msb/commits/f19b6c4d485bd34cba2c5825718c4330664ad54f',parents: [{ id: 'c4e19fa84bf3d9f0e621923b2b1fbf512c85b0b3' }]},{id: 'c4e19fa84bf3d9f0e621923b2b1fbf512c85b0b3', href: '/users/brtnfld/repos/hdf5_msb/commits/c4e19fa84bf3d9f0e621923b2b1fbf512c85b0b3',parents: [{ id: 'd288a7b93c991297faece77af28dd5ff7648a875' }]},{id: 'd288a7b93c991297faece77af28dd5ff7648a875', href: '/users/brtnfld/repos/hdf5_msb/commits/d288a7b93c991297faece77af28dd5ff7648a875',parents: [{ id: '8d2217d2955015f5e881dbe4bcaa255595ab6d57' }]},{id: '8d2217d2955015f5e881dbe4bcaa255595ab6d57', href: '/users/brtnfld/repos/hdf5_msb/commits/8d2217d2955015f5e881dbe4bcaa255595ab6d57',parents: [{ id: '790eccbc20b502eb036ca33f07f2c46535a25d61' }]},{id: '790eccbc20b502eb036ca33f07f2c46535a25d61', href: '/users/brtnfld/repos/hdf5_msb/commits/790eccbc20b502eb036ca33f07f2c46535a25d61',parents: [{ id: 'f4d8894158e8624f5d89cb6903a0017e2abdc670' }]},{id: 'f4d8894158e8624f5d89cb6903a0017e2abdc670', href: '/users/brtnfld/repos/hdf5_msb/commits/f4d8894158e8624f5d89cb6903a0017e2abdc670',parents: [{ id: '3fd9dc7b83fdff1ea0ecf5d7ef5a4ad2ddfeae51' }]},{id: '3fd9dc7b83fdff1ea0ecf5d7ef5a4ad2ddfeae51', href: '/users/brtnfld/repos/hdf5_msb/commits/3fd9dc7b83fdff1ea0ecf5d7ef5a4ad2ddfeae51',parents: [{ id: '3a911e2b39bb4bf5c06dd4d0a99a847ddd87ce5e' }]},{id: '3a911e2b39bb4bf5c06dd4d0a99a847ddd87ce5e', href: '/users/brtnfld/repos/hdf5_msb/commits/3a911e2b39bb4bf5c06dd4d0a99a847ddd87ce5e',parents: [{ id: '1df69ed3a09436575b455b784b35ad52cc74eabe' }]},{id: '1df69ed3a09436575b455b784b35ad52cc74eabe', href: '/users/brtnfld/repos/hdf5_msb/commits/1df69ed3a09436575b455b784b35ad52cc74eabe',parents: [{ id: '5ae8d135a541877466115637ee8c1dd3ac5c3753' }]},{id: '5ae8d135a541877466115637ee8c1dd3ac5c3753', href: '/users/brtnfld/repos/hdf5_msb/commits/5ae8d135a541877466115637ee8c1dd3ac5c3753',parents: [{ id: 'f8c5797bd62ba38bfa72a6a34106dab911c7f6ad' }]},{id: 'f8c5797bd62ba38bfa72a6a34106dab911c7f6ad', href: '/users/brtnfld/repos/hdf5_msb/commits/f8c5797bd62ba38bfa72a6a34106dab911c7f6ad',parents: [{ id: '62e1340bdd93aa12079ec97043b2da0c3f5d711a' }]},{id: '62e1340bdd93aa12079ec97043b2da0c3f5d711a', href: '/users/brtnfld/repos/hdf5_msb/commits/62e1340bdd93aa12079ec97043b2da0c3f5d711a',parents: [{ id: 'c09a9e90e1efa921f4d7a3661b5fca08c00a98d7' }]},{id: 'c09a9e90e1efa921f4d7a3661b5fca08c00a98d7', href: '/users/brtnfld/repos/hdf5_msb/commits/c09a9e90e1efa921f4d7a3661b5fca08c00a98d7',parents: [{ id: '0b11cfbf50be337fdfab8454aadbf7fb70e45aa8' }]},{id: '0b11cfbf50be337fdfab8454aadbf7fb70e45aa8', href: '/users/brtnfld/repos/hdf5_msb/commits/0b11cfbf50be337fdfab8454aadbf7fb70e45aa8',parents: [{ id: '0ec3340692a9953f7c75bc62b3a8075c780f8830' }]},{id: '0ec3340692a9953f7c75bc62b3a8075c780f8830', href: '/users/brtnfld/repos/hdf5_msb/commits/0ec3340692a9953f7c75bc62b3a8075c780f8830',parents: [{ id: '12ea13f745ff09a76a6f6d39e6d5a37807b29830' }]},{id: '12ea13f745ff09a76a6f6d39e6d5a37807b29830', href: '/users/brtnfld/repos/hdf5_msb/commits/12ea13f745ff09a76a6f6d39e6d5a37807b29830',parents: [{ id: '68b012f0175c357d59a7d870779a765f3ab8358d' }]},{id: '68b012f0175c357d59a7d870779a765f3ab8358d', href: '/users/brtnfld/repos/hdf5_msb/commits/68b012f0175c357d59a7d870779a765f3ab8358d',parents: [{ id: '0b5dfad751cf0f13151648d1eb0548d71eb27dad' }]},{id: '0b5dfad751cf0f13151648d1eb0548d71eb27dad', href: '/users/brtnfld/repos/hdf5_msb/commits/0b5dfad751cf0f13151648d1eb0548d71eb27dad',parents: [{ id: 'e024b2737c9d6a0bd52d7f3ca503399d04fb7c77' }]},{id: 'e024b2737c9d6a0bd52d7f3ca503399d04fb7c77', href: '/users/brtnfld/repos/hdf5_msb/commits/e024b2737c9d6a0bd52d7f3ca503399d04fb7c77',parents: [{ id: '37fb4bda7118b2d71c72ad19ae6b324e76144ee9' }]},{id: '37fb4bda7118b2d71c72ad19ae6b324e76144ee9', href: '/users/brtnfld/repos/hdf5_msb/commits/37fb4bda7118b2d71c72ad19ae6b324e76144ee9',parents: [{ id: '07e6f39f4bda85a969b78c25c4adf93e00b7ab17' }]}]);
