hdf5_msb

[svn-r17977] Bug fix for 1584.  H5Tdetect_class said a VL string is a string type.  But when it's in a
compound type, it says it's a VL type.  We want to tell user a VL string is a string.  But
internally we treat it as a VL type.  I added a flag as a parameter of H5T_detect_class.  
It tells whether the caller is the public function H5Tdetect_class.  I also added a 
detection for VL string in the private function for the compound case (or array or nested 
VL type).

Tested on jam and amani - I tested the same change for 1.8 with h5committest.