vchoi_fork

[svn-r19622] Port of fix for the round robin parallel flush bug caused by the failure
of the H5Ocache.c code to update its image of the on disk representation
of the object header on a call to the clear callback.

This wasn't an issue as long as all flushes of the object header were
made from the same process, but if an object header is modified, and
then flushed on one process and cleared on the rest, the changes were
not be reflected in the images of the on disk representation on all
processes where the object header was cleared rather than flushed.

If one of these processes did the next flush, the changes were lost in
the on disk representation.

Fixed this by causing all dirty messages and to be written to the copy
of the on disk image maintained by the object header code on both flush
and clear.

Also added associated test code in t_mdset.c.

Also checking in some cache debug code developed while chasing this bug.

Commit tested and tested (parallel) on phoenix.