Set Bioformats metadata on existing TIFF

Hi,
I am looking for a simple way to set the image metadata according to the bioformat standard, for existing tiff images.
I know that there is the python-bioformats package, but some hint would be very welcome.

I am also interested by a solution within ImageJ possibly with Jython scripting, that would be great especially precising the packages to import.
Looking forward to your proposal…
Thanks

1 Like

Hi,

There is a script below which shows some of the basics of importing an image and reading metadata using Jython and the Bio-Formats API. From that you would additionally need to set the metadata values you want and write the image.

If you let me know what types of metadata values you are looking to set I can try and help you with the API calls required.

David Gault

1 Like

Hello David,
Thank you for your answer. I would like for instance to set the pixel size, exposure time, illumination intensity…
I found this demo script in java on the bioformat github repository :

It shows how to save an image with the metadata.
Consequently I have tried to set the pixel size for instance. So far I have something like this :

#@ File (label='Path to image') Path
from loci.formats import MetadataTools, ImageWriter, FormatTools
from ome.units import UNITS # https://static.javadoc.io/org.openmicroscopy/ome-xml/5.5.4/ome/units/UNITS.html
from ij import IJ

# Create a "metadata collection" object
Metadata = MetadataTools.createOMEXMLMetadata()
#print dir(omeMeta) # see all possible setters

# ex : setting pixel size
PixelSize = 1000 # whatever...

# convert pixel size to a length object
Length  = FormatTools.createLength(PixelSize,UNITS.MICROMETRE)

# Add pixel size to metadata
Metadata.setPixelsPhysicalSizeX(Length,0)


# Open image for which to set metadata
FilePath = Path.absolutePath
imp      = IJ.openImage(FilePath)

# Try converting to byte 
imProc = imp.getProcessor()
#imByte = imProc.convertToByte()
imByte = imProc.convertToByteProcessor()

# Save image with edited metadata
writer = ImageWriter()
writer.setMetadataRetrieve(Metadata)
NewPath = FilePath[:-4] + '-bis.tif'
writer.setId(NewPath)       # https://downloads.openmicroscopy.org/bio-formats/5.1.8/api/loci/formats/ImageWriter.html#setId(java.lang.String)
writer.saveBytes(0, imByte) # imByte must be a byte object... directly the image matrix ? 
writer.close()

But the line writer.setId(NewPath) always return a null pointer exception. I have tried to give a path pointing to an existing blank image, or to a filepath previously opened in the script with open(PathToNewFile,'w') but it did not work.
Any suggestion ?

The code you have to edit the metadata looks to be correct. Im unsure why you are seeing a NullPointerException, writing to a new file should not be an issue (modifying the file name as you have done is fine). The one thing you are missing is the copying of the pixel data. Generally speaking the steps for conversion should be as below:

  • Create and setup a reader
  • Read the metadata to be modified
  • Modify the metadata
  • Create a writer and set the modified metadata
  • Copy the pixel data from the original file to the newly modified file

I have modified the example code you provided to show a working conversion which reads in a file, modifies the pixel size and writes out the new metadata with the existing pixel data:

    # read in and display ImagePlus object(s)
    from loci.plugins import BF
    from loci.formats import ImageReader, MetadataTools, ImageWriter, FormatTools
    from ij import IJ
    from ome.units import UNITS

    file = "/Users/dgault/Documents/Sample Images/ome_tiff/time-series.ome.tif"

    # Initiate the reader
    reader = ImageReader()
    omeMeta = MetadataTools.createOMEXMLMetadata()
    reader.setMetadataStore(omeMeta)
    reader.setId(file)

    # ex : setting pixel size
    PixelSize = 1000 # whatever...

    # convert pixel size to a length object
    Length  = FormatTools.createLength(PixelSize,UNITS.MICROMETRE)

    # Add pixel size to metadata
    omeMeta.setPixelsPhysicalSizeX(Length,0)

    # Create a new writer
    writer = ImageWriter()

    # Set the edited metadata
    writer.setMetadataRetrieve(omeMeta)
    NewPath = file[:-4] + '-bis.tif'
    writer.setId(NewPath)       # https://downloads.openmicroscopy.org/bio-formats/5.1.8/api/loci/formats/ImageWriter.html#setId(java.lang.String)

    # Copy the pixel data to the new edited file
    for series in range(reader.getSeriesCount()):
      reader.setSeries(series);
      writer.setSeries(series);
      for image in range(reader.getImageCount()):
        writer.saveBytes(0, reader.openBytes(0)) # imByte must be a byte object... directly the image matrix ? 

    # Close the reader and writer
    reader.close()
    writer.close()

A useful link for the steps on converting files can be found at https://docs.openmicroscopy.org/bio-formats/5.8.2/developers/export.html

2 Likes

Works like a charm thanks !
For the new file I only changed the extension to .ome.tif , rather than the -bis.tif.
The SetID somehow recognises the extension for the encoding but actually both .tif or .ome.tif works the same. Maybe it is just a convention they use, and the extension is actually just .tifin both cases.

I also added a small print 'Done' at the end as the execution is not so instantaneous, like 1sec maybe :grin:

I was wondering if there is a more straightforward way of doing it rather than copying the pixel data to a new file. Like editing the TIFF comment of the file as shown here. In my case when executing this code the xml variable is empty since I am not starting from an ome.tif but is there any chance this could work this way ?

Maybe just an XML companion file next to the image files ? but I dont know how an image processing software would know that it has to read this particular XML and if it would survive file renaming.

I am also curious about having a multi file OME-TIFF which would be convenient since here my Zstacks or channels are splitted into different TIFF files.
Yet I have no clue how to set or get the UUID for each files…

Maybe @joshmoore could give me a hint on that :wink:

Hi @LThomas

The CommentSurgery code does the same as the tiffcomment command-line. The prerequisite, however, is that the tiff have an ImageDescription (of some form; it need not be OME-XML). If you want to see if your TIFFs have a comment, then use:

tiffcomment a.tif

To set, use:

tiffcomment -set my.xml a.tif

If not, something fancier will be needed.

Regarding the multi-file OME-TIFF, the UUIDs are the value that you place in the OME-XML (ImageDescription) of each of the individual tiffs. Use tiffcomment on the files in http://downloads.openmicroscopy.org/images/OME-TIFF/2016-06/companion/ to see how the companion file is constructed.

Hope that helps.
~Josh

My images do not have an ImageDescription tag…

However, I could set such tag using a very nice utilitary called ExifTool that allows to edit, view tags from many types of file and that can be called via the command line (or perl script).

Let say we want to set the ImageDescription tag, the command looks like :
exiftool -overwrite_original -ImageDescription="SomeMetadataString" "PathToYourTIFF"

and as a verification calling tiffcomment after that :
tiffcomment "PathToYourTIFF"
I indeed recover my metadata string :smile:

I “just” need to properly format the string to an OME-XML-like and I’m good I guess.
Exiftool actually allows to set the metadata from a file but I havent tried it yet.

I think it could be stated more clearly somewhere that the OME metadata is “just” an XML string stored in this ImageDescription tag. Preferably towards the beginning of the documentation :rofl:

I managed to generate an OME-XML string thanks to a jython macro like above.

<?xml version="1.0" encoding="UTF-8" standalone="no"?><OME xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openmicroscopy.org/Schemas/OME/2016-06 http://www.openmicroscopy.org/Schemas/OME/2016-06/ome.xsd" xmlns="http://www.openmicroscopy.org/Schemas/OME/2016-06"><Image><StageLabel X="2.053" XUnit="mm" Y="2.053" YUnit="mm" Z="2.053" ZUnit="mm"/><Pixels PhysicalSizeX="1000.0" PhysicalSizeXUnit="µm" PhysicalSizeY="1000.0" PhysicalSizeYUnit="µm"><Channel EmissionWavelength="500.0" EmissionWavelengthUnit="nm" ExcitationWavelength="300.0" ExcitationWavelengthUnit="nm"/></Pixels></Image></OME>

Then I store it in the -ImageDescription tag of the TIFF image thanks to ExifTool
And I can even recover it with tiffcomment.

Yet if I try to open it using bioformat in Fiji or Knime, the XML information does not get formatted and just appear as a string in the Description field (ex: StageLabelX screenshot below)

Hi @LThomas,

looking at your OME-XML block, two obvious next steps:

  • you are probably missing are the TiffData elements which express the mapping between the Image Plane elements and the planes stored in the IFDs of the TIFF file.

  • your Pixels element might be missing some required information such as the dimensions, the pixel type and the dimension order

In general, running the xmlvalid utility on your constructed OME-TIFF should report all outstanding issues with the OME-XML. When loading an OME-TIFF into Bio-Formats (using KNIME or Fiji), if the OME-XML is invalid, the file is read as a standard multi-page TIFF and will miss all the metadata.

Best,
Sebastien

1 Like

Hi, this is a rather old thread, but I’m also in the situation where I have a Tif file on one hand and lots of metadata on the other hand and I’m looking for a easy way to include it in the tif file in a usable manner. I Can easily put it in the Image Description, but it is not recognized in ImageJ or bioformats. On the other hand, when I look at the imagedescription of a file with recognized metadata, it does not really contain anything in the image Description (just the ImageJ version and unit, sometimes intebnsities). So is there a simple way to add a large number of user-defined metadata to a tif file ?
Thanks

How do you put them in the image description exactly ?
If it’s with bioformat, you need to use the Bioformat importer to extract the metadata. Otherwise they are just displayed as a long xml string in the Image Description field.

If it’s for use in ImageJ mostly, you can use the
imp.setProperty(key, value) in a scripting language supported by Fiji
or
setMetadata(“Info”, YourLongMetadataString) in the macro language

I’ve tried with Exiftool. And yes I get a long XML in the description, but that’s it. On the contrary when I have an image with properties it is not in the Image description, so I’m assuming that the info is stored elsewhere.
To give you an idea, I need to set around 20 different user-defined (defined by me from other pieces of information - not imported form an image) properties, on 150 to 200 K files, in 500 different sub-folders for a total amount of >10TB of image data. And it is possible that I need to do it again with new properties in the future. So I’m looking something really efficient and straightforward.

Indeed but apparently not displayed by the Show Info command.
So if you need the “Show Info” command to work, e.g. for normal users to visualize the metadata, you have to use imp.setProperty("Info", "Long metadata string")

And formatting the metadata string in a way that you want, ex: one line per field by using “\n”

If this is just for yourself, then setProperty("metadata1", "value1") and getProperty should be enough, but no visualization possible and limited to ImageJ/Fiji.

Storing in custom tiff tags would not really work, as the visualization software would not know that they exist.

Thanks Laurent,

But then using imp.setProperty I can only see the tags in Bioformats or ImageJ, right ?
That’s also not ideal: I’m using also a lot of EM-specific software that won’t be able to carry it over.
I Guess the image description remains the best/portable option for me.
Cheers,

Jean-Marc

Bio-Formats should recognise the Image Description and parse the key value pairs and include them in the metadata shown in Show Info. If they are not being displayed there would you be able to share what the Image Description currently is and maybe an example of how you were setting it?

Sounds like good news. I’m totally flexible on how I set it up. I’ve tried a simplified XML format, a list of key=values, with various separators. I see that ImageJ images often uses an unquoted period-separated list, but I’m confused on how to deal with real numbers. For instance this(not from me): ImageJ=1.52p.unit=micron.min=12681.001953125.max=26366.982421875.
How is that dealt with (ImageJ version, min, max) ?
Where can I find the specs ?

There are a couple of different types of comments that Bio-Formats will recognise. But both ImageJ comments and generic comments will be separated by a new line with = being used for each key, value pair. These will simply be read as Strings so the formatting of numbers is not important. (There are some exceptions for ImageJ comments specifically where particular metadata values are parsed and used, such as number of channels, slices, frames etc, the code which handles this parsing is at https://github.com/ome/bioformats/blob/bcf8e9d6e277b5df9fa631d339d469a4be96febf/components/formats-bsd/src/loci/formats/in/TiffReader.java#L276)

So setting the below values in an xml file and then running tiffcomment -set metadataTest.xml path/to/myFile.tiff should now show the metadata values in the Show Info window.

ImageJ=1.52p
unit=micron
min=12681.001953125
max=26366.982421875

So yes, it does something, but not what I’m expecting.
For instance with this:
ImageJ=1.52p
unit=micron
SampleDepth=24.2282775874354
WD=6.7778
min=0
max=130
DataSet=20200416_RAU
DataFile=471fc4e1-187d-42ba-9897-874f42f7802f
TextFile=Bloc2_50nm_px10nm_30Pa_001 (copy)
TileX=0
TileY=0
TileZ=0
TileSubZ=0
Vacuum=VolumeScopeHighVac
Detector=VS_DBS
DwellTime=1E-06
Energy=2000
BeamDepth=24.2283
Current=0.2
HFW=81.92
SectionThickness=50
BeamX=0.0063633826975566573
BeamY=0.0063633826975566573

everything does indeed go into the ImageDescription (if I check with exiftool for instance), and yes, unit, min and max are taken into account, even if not in the beginnign of the file: they changes the contrast and unit. Because they are standard tags. But my other tags don’t appear anywhere in Images -> Show Info. I’m not expecting that they should be processed if they don’t mean anything for ImageJ, but I simply don’t even see them anymore, even if I see them with exiftool. They don’t show up in bftools “showinf” either … and incidentally bftools “showinf” shows other tags in the image that I didn’t know where there but are also not shown in Image -> Show …

OK, Got it !

I was stupid: I was opening the Image with ImageJ File -> Open

When I import via the bio-formats plugin, everything is there. My Image Description is in the OME Metadata (not in Show Info) and the tags already present are in the Showinfo window.

Thanks a lot,

Jean-Marc

1 Like