Opening images from a Yokogawa CQ1 with Bio-formats

Dear community

I have been struggling a bit when opening images produced by a Yokogawa CQ1 microscope.
The microscope software (CellVoyager) saves each plane as an individual tif image. These images by themselves are not calibrated.
In addition, the software creates also ome.xml files, that contain all relevant metadata. It contains also the relative file paths and some UUID. Example snippet:

 <Image ID="Image:1" Name="W26(R2C2),A1,F1">
        <AcquisitionDate>2021-05-06T14:49:49.275+02:00</AcquisitionDate>
        <ObjectiveSettings ID="Objective:0" Medium="Air" RefractiveIndex="1" />
        <StageLabel Name="W26(R2C2),A1,F1" />
        <Pixels ID="Pixels:1" DimensionOrder="XYZCT" Type="uint16" SizeX="2560" SizeY="2160" SizeZ="1" SizeC="2" SizeT="1" PhysicalSizeX="1.61915" PhysicalSizeY="1.61915" PhysicalSizeZ="47.0588226">
            <Channel ID="Channel:1:0" Name="Channel 1" SamplesPerPixel="1" IlluminationType="Epifluorescence" AcquisitionMode="SpinningDiskConfocal" ContrastMethod="Fluorescence" ExcitationWavelength="561" EmissionWavelength="617" Color="-46595841">
                <LightSourceSettings ID="LightSource:2" Attenuation="1" Wavelength="561" />
                <DetectorSettings ID="Detector:0" Binning="1x1" />
                <LightPath>
                    <DichroicRef ID="Dichroic:0" />
                    <EmissionFilterRef ID="Filter:2" />
                </LightPath>
            </Channel>
            <Channel ID="Channel:1:1" Name="Channel 2" SamplesPerPixel="1" IlluminationType="Transmitted" AcquisitionMode="SpinningDiskConfocal" ContrastMethod="Brightfield">
                <LightSourceSettings ID="LightSource:4" Attenuation="0.3" />
                <DetectorSettings ID="Detector:0" Binning="1x1" />
                <LightPath>
                    <DichroicRef ID="Dichroic:0" />
                    <EmissionFilterRef ID="Filter:4" />
                </LightPath>
            </Channel>
            <TiffData IFD="0" FirstZ="0" FirstT="0" FirstC="0" PlaneCount="1">
                <UUID FileName="Projection/W0026F0001T0001Z000C1.tif">urn:uuid:56a81b88-bca4-4c6e-a8fd-6b2b4b3f4e50</UUID>
            </TiffData>
            <TiffData IFD="0" FirstZ="0" FirstT="0" FirstC="1" PlaneCount="1">
                <UUID FileName="Projection/W0026F0001T0001Z000C2.tif">urn:uuid:9099ea8d-9858-4a28-aa58-7e170ba2f69d</UUID>
            </TiffData>
            <Plane TheZ="0" TheT="0" TheC="0" ExposureTime="0.7" PositionX="-99.6" PositionY="-14.3" PositionZ="0" />
            <Plane TheZ="0" TheT="0" TheC="1" ExposureTime="0.05" PositionX="-99.6" PositionY="-14.3" PositionZ="0" />
        </Pixels>
    </Image>

When I open the ome.xml file with Fiji (Bio-Formats), it looks promising (e.g. it recognises how many multi-channel images were acquired + an additional “calibration” image). But when I try to open any, all the images are black.

When I open one of the single tif files directly with Bio-Formats I get this UUID error (for image W0026F0001T0001Z000C1.tif):

loci.formats.FormatException: Unmatched UUID: urn:uuid:9b3414d6-cb92-47eb-898e-c3d1c06af2c6
	at loci.formats.in.OMETiffReader.initFile(OMETiffReader.java:767)
	at loci.formats.FormatReader.setId(FormatReader.java:1421)
	at loci.plugins.in.ImportProcess.initializeFile(ImportProcess.java:499)
	at loci.plugins.in.ImportProcess.execute(ImportProcess.java:142)
	at loci.plugins.in.Importer.showDialogs(Importer.java:140)
	at loci.plugins.in.Importer.run(Importer.java:76)
	at loci.plugins.LociImporter.run(LociImporter.java:78)
	at ij.IJ.runUserPlugIn(IJ.java:235)
	at ij.IJ.runPlugIn(IJ.java:198)
	at ij.Executer.runCommand(Executer.java:150)
	at ij.Executer.run(Executer.java:68)
	at java.lang.Thread.run(Thread.java:748)

Hence, I tried to checked the UUID of this single tiff file (via ImageJ’s “Show Info”):

ImageDescription: <?xml version="1.0" encoding="utf-8"?>
<OME xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openmicroscopy.org/Schemas/OME/2013-06 http://www.openmicroscopy.org/Schemas/OME/2013-06/ome.xsd" UUID="urn:uuid:56a81b88-bca4-4c6e-a8fd-6b2b4b3f4e50" xmlns="http://www.openmicroscopy.org/Schemas/OME/2013-06">
    <BinaryOnly MetadataFile="../MeasurementResultMip.ome.tif" UUID="urn:uuid:3964be0e-1119-4848-bc2e-7d65e433114b" />
</OME>
Software: CQ1 Software 1.05
--------------------------------------------
(Fiji Is Just) ImageJ 2.1.0/1.53c; Java 1.8.0_202 [64-bit]; Mac OS X 10.14.6; 310MB of 11556MB (2%)
 
Title: W0026F0001T0001Z000C1.tif
Width:  26.6667 inches (2560)
Height:  22.5 inches (2160)
Size:  11MB
Resolution:  96 pixels per inch
Pixel size: 0.0104x0.0104 inch^2
ID: -2
Bits per pixel: 16 (unsigned)
Display range: 103 - 891
No threshold
Magnification: 0.50
ScaleToFit: false
Uncalibrated
Path: 
...

According to the ome.xml file:

  • The first UUID=“urn:uuid:56a81b88-bca4-4c6e-a8fd-6b2b4b3f4e50” actually belongs to Projection/W0026F0001T0001Z000C1.tif.
  • The second UUID=“urn:uuid:3964be0e-1119-4848-bc2e-7d65e433114b” belongs to the “calibration” image MeasurementResultMip.ome.tif

Questions:

  1. Is this mis-match of the UUIDs the reason why opening the ome.xml file with Bio-Formats would result in only black images?
  2. Is there another way I can use the ome.xml file to open the images correctly?
  3. Is the only way to fix this a patch-fix for the CQ1 - CellVoyager from Yokogawa?

It would be of great advantage if I could avoid scripting the opening (e.g. with Import>Image Sequence) of these images and their calibration…

Example images and ome.xml file attached.
MeasurementResultMIP.ome.xml (838.4 KB)
W0026F0001T0001Z000C1.tif (10.6 MB)
W0026F0001T0001Z000C2.tif (10.6 MB)

Thanks for your feedback!
LoĂŻc

Hi @loic

first of all, I was unaware that Yokogawa CQ1 could produce such a representation. Overall it looks interesting even though your thread indicates there ar some implementation issues that need to be resolved to make the fileset directly usable in an environment like Fiji.

For the first issue (black planes), I think the problem comes from the fact that the extension of the file associates the file with the OME-XML format. Unlike OME-TIFF, this format (and its Bio-Formats reader) does not support binary data stored using the TIFF container but only using the legacy BinData element and I suspect this causes your planes to appear as black.

The second error indicates a mismatch of the UUIDs which prevents the fileset from being opened from a constituent TIFF. The immediate questions coming to mind are:

  • does the MeasurementResultMip.ome.tif file exist as this file should contain the metadata for the entire filesets?
  • if yes, what is the content of its metadata? in addition, what is the overlap with the content of MeasurementResultMip.ome.xml that you shared earlier?

I understand that only a subset of the data was shared in this thread probably for space reasons. Would you be able to share a complete public representation of your sample plate e.g. by uploading it to Zenodo? This way we can look at the entire structure and the possible options to reconcile the metadata.

Best,
Sebastien

1 Like

Hi @s.besson

Thanks for your reply. Basically, you resolved the problem.

I had not tried to open the MeasurementResultMip.ome.tif (which does exist) explicitly with the Bio-Formats plugin.
When I did that, it recognised that the experiment contains several images - (like when using the MeasurementResultMip.ome.xml file), but the images are opened correctly!

The metadata in the MeasurementResultMip.ome.tif seems to be the same (image paths and UUIDs), except for the first lines. See here:
calibartionImage_omeMetadata.ome.xml (770.7 KB)
to compare with:

I am sorry, I was not aware that this calibration image might contain the same metadata as the ome.xml file. Neither that the OME-XML format is only for binary data…

For me, it doesn’t seem intuitive to have to open an ome.tif “calibration” image to access the fully calibrated & metadata populated imaging data… But it does the job!

For other CQ1 users: open the MeasurementResult*.ome.tif image with Bio-Formats to access the full and calibrated data set.

For completeness I uploaded a dataset. It is not the same as mentioned in the first message (for size reasons).
DOI

Thanks a lot!

Great news. Thanks for following-up and also for uploading a complete public reference sample for future reference :100:

On the ome.tiff vs ome.xml, from a quick inspection, it feels like the two metadata blocks are indeed aiming at describing the same dataset. Given that the individual binary TIFF files contain pointers to the master OME-TIFF file i.e. the one containing the metadata, it should be possible in principle to open the entire fileset from any constituent file.

The UUID mismatch issue seems to be related with the following TIFF file.

$ grep 9b3414d6-cb92-47eb-898e-c3d1c06af2c6 calibartionImage_omeMetadata.ome.xml 
<UUID FileName="Projection/W0262F0001T0001Z000C2.tif">urn:uuid:9b3414d6-cb92-47eb-898e-c3d1c06af2c6</UUID>

Could you check the metadata in this file?

I am a bit puzzled because there are no binary TIFF files in the dataset (the projections subfolder, contains the z-projected images…)

I opened the image in question with the default ImageJ tiff opener and Show Info gives following:

ImageDescription: <?xml version="1.0" encoding="utf-8"?>
<OME xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openmicroscopy.org/Schemas/OME/2013-06 http://www.openmicroscopy.org/Schemas/OME/2013-06/ome.xsd" UUID="urn:uuid:9b3414d6-cb92-47eb-898e-c3d1c06af2c6" xmlns="http://www.openmicroscopy.org/Schemas/OME/2013-06">
    <BinaryOnly MetadataFile="../MeasurementResultMip.ome.tif" UUID="urn:uuid:3964be0e-1119-4848-bc2e-7d65e433114b" />
</OME>
Software: CQ1 Software 1.05
--------------------------------------------
(Fiji Is Just) ImageJ 2.1.0/1.53c; Java 1.8.0_172 [64-bit]; Windows 10 10.0; 186MB of 32000MB (<1%)
 
Title: W0262F0001T0001Z000C2.tif
Width:  26.6667 inches (2560)
Height:  22.5 inches (2160)
Size:  11MB
Resolution:  96 pixels per inch
Pixel size: 0.0104x0.0104 inch^2
ID: -2
Bits per pixel: 16 (unsigned)
Display range: 24 - 17092
No threshold
Magnification: 0.33
ScaleToFit: false
Uncalibrated
...

It is the same UUID you just mentioned… Does this help / make sense?

Thanks for the additional information LoĂŻc,

It is the same UUID you just mentioned… Does this help / make sense?

Not really, in the sense it does not explain where the exception you reported in your initial post comes from :slight_smile:

Assuming it will display the same symptoms, I will take a look at the complete dataset you have uploaded to Zenodo and report back.

Updating this thread following some background investigation. From a specification perspective, the representative fileset uploaded to Zenodo looks completely fine and compliant.

The only noticeable configuration is that the metadata file (MeasurementResultMip.ome.tiff) and the TIFF files are not stored in the same directory. The individual TIFF files for each field of view are stored in a Projections sub-folder and the metadata of each of these files links back to the metadata file using a relative path value in BinaryOnly.MetadataFile. Under this scenario, when a file in the Projetions sub-folder, the Bio-Formats reader currently fails to detect other files and throws a Mismatched UUID error as discussed above.

I have opened a Pull Request to fix the reader and support this use case and I expect it to be included in the next Bio-Formats release.

Hi @s.besson

Thanks for following up and the explanations.

I am lost without Bio-Formats and would not know how to inspect metadata otherwise…

Great if you manage to make Bio-Formats work directly on individual TIFF files.
I’m looking forward to the new release!