How to create an image series OME-TIFF from python

sebi06 · September 11, 2020, 7:39am

Hi all,

inside my python code I have a loop processes 5d NumPy.Array. I managed to write every singe array incl metadata as an OME-TIFF, but it would be much more user-friendly to write let’ say one OME-TIFF with X series instead.

Has anyone a snippet etc. to get me started. Ideally I would like to avoid writing an 6D array into one OME-TIFF file with X -5d image series. In my imagination it would work like this:

open OME-TIFF file for writing
start loop
analyze 5D array
save 5D array to OME-TIFF as image series N
…

Or is a completely silly idea? Any thoughts are welcome.

cgohlke · September 11, 2020, 1:01pm

Hi Sebastian,

recent versions of tifffile can write multi-series OME-TIFF files. There’s a short example in the docstring:

Write two numpy arrays to a multi-series OME-TIFF file:

>>> data0 = numpy.random.randint(0, 255, (32, 32, 3), 'uint8')
>>> data1 = numpy.random.randint(0, 1023, (4, 256, 256), 'uint16')
>>> with TiffWriter('temp.ome.tif') as tif:
...     tif.save(data0, compress=6, photometric='rgb')
...     tif.save(data1, photometric='minisblack',
...              metadata={'axes': 'ZYX', 'SignificantBits': 10,
...                        'Plane': {'PositionZ': [0.0, 1.0, 2.0, 3.0]}})

sebi06 · September 11, 2020, 1:13pm

This is exactly what I tried, but I did not succeed. I think I am missing something here …

# -*- coding: utf-8 -*-

import sys
import os
import numpy as np
from aicsimageio import AICSImage, imread
import progressbar
import shutil
from apeer_ometiff_library import io, omexmlClass
import tifffile
from lxml import etree

filename = r"testdata\WP96_4Pos_B4-10_DAPI.czi"
savename = filename.split('.')[0] + '.ome.tiff'

# get AICSImageIO object using the python wrapper for libCZI
img = AICSImage(filename)

with tifffile.TiffWriter(savename, append=False) as tif:
    for s in progressbar.progressbar(range(img.shape[0]), redirect_stdout=True):

        # get the 5d image stack
        image5d = img.get_image_data("TZCYX", S=s)

        # write scene as OME-TIFF series
        tif.save(image5d,
                 photometric='minisblack',
                 contiguous=False,
                 metadata={'axes': 'TZCXY'})

    # close the AICSImage object at the end
    img.close()

When I open this one in Fiji it has 28 timepoints instead of 28 series …

cgohlke · September 11, 2020, 1:35pm

Looks like you are using an older version of tifffile. The latest version (2020.9.3 on PyPI) will raise an exception for your code because the axes metadata should be 'TZCYX', not 'TZCXY'. Btw, contiguous=False, is no longer needed. To open the file in Fiji, use File > Import > Bio-Formats.

sebi06 · February 24, 2021, 9:25am

Cool, now it works as expected. Thank you so much. I might have mixed up my envs …

The code that works is now this:

# -*- coding: utf-8 -*-

import sys
import os
import numpy as np
from aicsimageio import AICSImage, imread
import progressbar
import shutil
from czitools import imgfileutils as imf
import tifffile
from lxml import etree

filename = r"testdata\WP96_4Pos_B4-10_DAPI.czi"
savename = filename.split('.')[0] + '.ome.tiff'

# get the metadata
md, additional_mdczi = imf.get_metadata(filename)

# get AICSImageIO object using the python wrapper for libCZI
img = AICSImage(filename)

with tifffile.TiffWriter(savename, append=False) as tif:

    for s in progressbar.progressbar(range(img.shape[0]), redirect_stdout=True):

        # get the 5d image stack
        image5d = img.get_image_data("TZCYX", S=s)

        # write scene as OME-TIFF series
        tif.save(image5d,
                 photometric='minisblack',
                 metadata={'axes': 'TZCYX',
                           'PhysicalSizeX': np.round(md['XScale'], 3),
                           'PhysicalSizeXUnit': md['XScaleUnit'],
                           'PhysicalSizeY': np.round(md['YScale'], 3),
                           'PhysicalSizeYUnit': md['YScaleUnit'],
                           'PhysicalSizeZ': np.round(md['ZScale'], 3),
                           'PhysicalSizeZUnit': md['ZScaleUnit']
                           }
                 )

    # close the AICSImage object at the end
    img.close()

print('Done')

oldbhoy · February 22, 2021, 5:02pm

I have a related issue and was hoping to piggyback off this answer.

I’ve written some microscope control software in LabVIEW, and i’m trying to construct a 5d image stack of confocal images in ‘RZCYX’ order, but saving the images as they’re acquired rather than buffering the whole capture.

I havent done this before but thought this is a better way to do it vs trying to buffer the whole thing and save at the end, in case something catastrophic happens with the microscope while scanning (power outage, software crash, hardware failure etc.). Also seems prudent for memory management. If there’s a better approach please let me know.

The built-in LabVIEW tiff functions are pretty threadbare, so instead i’m using Tifffile to save each incoming z-stack.Test code to do this is as follows:

# imports
from pathlib import Path
import numpy as np
import tifffile as tf


def write_tiff(filepath, image, append):
    # incoming image is a list if called from LabVIEW. Convert to numpy array first

    if isinstance(image, list):
        np_image = np.array(image, dtype='uint16')
    else:
        np_image = image

    with tf.TiffWriter(filepath, append=append) as tif:

        tif.save(np_image,
                 photometric='minisblack',
                 metadata={'axes': 'TZCYX'})


if __name__ == '__main__':

    # create a random z-stack image similar to the Zeiss LSM 510 12 bit tiff
    # One region, consisting of 25 z-slices, 2 channels, x*y of 512*512 pixels
    data = np.random.randint(0, 2**12, (1, 25, 2, 512, 512), dtype='uint16')

    fpath = Path("test_images/multi_stack_img.ome.tiff")

    # create a new file and write data to it
    write_tiff(fpath, data, append=False)

    # append identically shaped region data to existing file
    regions = range(10)
    for region in regions:
        new_series = np.random.randint(0, 2**8, (1, 25, 2, 512, 512), dtype='uint16')
        write_tiff(fpath, new_series, append=True)

The images seem to save (file size is as expected) but the image is only recognised as having a single timepoint (T=1). I tried using ‘R’ instead of ‘T’ in the axes metadata but this doesn’t seem to make a difference.

How do I append regions (or even timepoints) to an existing file. Do i need to reshape the file as I go to achieve this, if that’s even possible?

Any help would be greatly appreciated

cgohlke · February 25, 2021, 10:12am

First, tifffile should not allow you to append to an existing OME-TIFF file. I will fix that in the next version.

The reason why the file is recognized as having only one timepoint is that when you append to an existing file, the OME-XML metadata in the first page is not updated.

You could manually rewrite the OME-XML with a correct version at the end of the acquisition using the tiffcomment function.

Re-opening and appending to TIFF files does not scale well since the TIFF directory has to be parsed each time. Instead, if you know the number of stacks being acquired beforehand and do not require compression or tiling, try to pre-create an empty OME-TIFF file and then directly write to the memory-mapped file:

import numpy
from tifffile import imwrite, memmap

filename = 'test_append.ome.tif'
shape = (10, 25, 2, 512, 512)
dtype = 'uint16'

# create an empty OME-TIFF file
imwrite(filename, shape=shape, dtype=dtype, metadata={'axes': 'TZCYX'})

# memory map numpy array to data in OME-TIFF file
tzcyx_stack = memmap(filename)

# write data to memory-mapped array
for t in range(shape[0]):
    tzcyx_stack[t] = numpy.random.randint(0, 2 ** 12, shape[1:], dtype=dtype)
    tzcyx_stack.flush()

oldbhoy · February 24, 2021, 9:25am

Thank you so much Christophe, that worked a charm. I feel a bit silly for not realising I could use the memmap functionality. When I originally read the documentation it didnt occur to me when I would need to use that

Working example for my specific case is as below:

# imports
from pathlib import Path
import numpy as np
import tifffile as tf


def create_tiff(filename, shape, dtype):
    # create an empty OME-TIFF file the size of the scan required
    tf.imwrite(filename, 
               shape=shape, 
               dtype=dtype,
               metadata={'axes': 'TZCYX'})


def write_memmapped_tiff(filename, data, idx, dtype):
    # incoming image is a list if called from LabVIEW. Convert to numpy array first

    if isinstance(data, list):
        np_image = np.array(data, dtype=dtype)
    else:
        np_image = data

    image_stack = tf.memmap(filename)
    image_stack[idx] = data
    image_stack.flush()


if __name__ == '__main__':

    # create a random z-stack image similar to the Zeiss LSM 510 12 bit tiff
    # One region, consisting of 25 z-slices, 2 channels, x*y of 512*512 pixels

    fpath = Path("test_images/multi_stack_img.ome.tiff")
    shape = (10, 25, 2, 512, 512)
    dtype = 'uint16'

    # create a new file and write data to it
    create_tiff(fpath, shape, dtype)

    # write slices to memory-mapped tiff file
    for region in range(shape[0]):
        write_memmapped_tiff(fpath, np.random.randint(0, 2 ** 12, shape[1:], dtype=dtype), region, dtype)

This is an excellent tiff library btw, many thanks Christoph