Enabling "Process as 3D?" in NamesAndTypes input module in CP 3.0 complains of "invalid TIFF file" with DeltaVision .dv images

cellprofiler
3d
namesandtypes

#1

Hi cellprofilers!

I’m helping my lab mate process some image stacks collected from a DeltaVision microscope which generates files ending in extension “.dv”. We don’t have any issue with opening these files in FIJI / ImageJ using Bio-formats, but am running into trouble processing a single file both with “Start Test Mode” and in “Analyze Images” using CP 3.0.0 tagged release and the today’s master git revision 3743398 on Ubuntu 16.04 LTS.

What’s interesting is there is no problem extracting the metadata or assigning names to channels; namely I see the image slices correctly tabulated when hitting “update” in the Metadata and NamesAndTypes input modules (note that for the metadata input module one must first run “Update metadata” before “update”). However when I click on “Start Test Mode”, I see an error dialog box with “Error while processing NamesAndTypes: invalid TIFF file. Do you want to stop processing?” with the corresponding traceback in the terminal:

Failed to run module NamesAndTypes
Traceback (most recent call last):
  File "/share/Pariksheet/consultations/cellprofiler-3.0/CellProfiler/cellprofiler/gui/pipelinecontroller.py", line 2887, in do_step
    self.__pipeline.run_module(module, workspace)
  File "/share/Pariksheet/consultations/cellprofiler-3.0/CellProfiler/cellprofiler/pipeline.py", line 2022, in run_module
    module.run(workspace)
  File "/share/Pariksheet/consultations/cellprofiler-3.0/CellProfiler/cellprofiler/modules/namesandtypes.py", line 1758, in run
    stack)
  File "/share/Pariksheet/consultations/cellprofiler-3.0/CellProfiler/cellprofiler/modules/namesandtypes.py", line 1810, in add_image_provider
    series, index, channel)
  File "/share/Pariksheet/consultations/cellprofiler-3.0/CellProfiler/cellprofiler/modules/namesandtypes.py", line 1848, in add_simple_image
    self.add_provider_measurements(provider, m, cpmeas.IMAGE)
  File "/share/Pariksheet/consultations/cellprofiler-3.0/CellProfiler/cellprofiler/modules/namesandtypes.py", line 1867, in add_provider_measurements
    img = m.get_image(name)
  File "/share/Pariksheet/consultations/cellprofiler-3.0/CellProfiler/cellprofiler/measurement.py", line 1620, in get_image
    image = matching_providers[0].provide_image(self)
  File "/share/Pariksheet/consultations/cellprofiler-3.0/CellProfiler/cellprofiler/modules/namesandtypes.py", line 2407, in provide_image
    image = LoadImagesImageProviderURL.provide_image(self, image_set)
  File "/share/Pariksheet/consultations/cellprofiler-3.0/CellProfiler/cellprofiler/modules/loadimages.py", line 3306, in provide_image
    return self.__provide_volume()
  File "/share/Pariksheet/consultations/cellprofiler-3.0/CellProfiler/cellprofiler/modules/loadimages.py", line 3374, in __provide_volume
    data = skimage.external.tifffile.imread(pathname)
  File "/home/omsai/.local/lib/python2.7/site-packages/skimage/external/tifffile/tifffile.py", line 1233, in imread
    with TiffFile(files, **kwargs_file) as tif:
  File "/home/omsai/.local/lib/python2.7/site-packages/skimage/external/tifffile/tifffile.py", line 1331, in __init__
    self._fromfile(pages, fastij)
  File "/home/omsai/.local/lib/python2.7/site-packages/skimage/external/tifffile/tifffile.py", line 1358, in _fromfile
    raise ValueError("invalid TIFF file")
ValueError: invalid TIFF file

When I used CP 2, I remember it would prefer single TIFF images instead of image stacks because it could not process 3D data like CP 3, and so I was initially wondering whether I should convert the .dv images to single .tif z-slices. But your CellProfiler 2.0 FAQ encourages reporting issues where CellProfiler is unable to open a Bio-format supported image.

To help you reproduce the problem, below are the project file and image stack file:

I don’t get the error if I disable “Process as 3D?” in the NamesAndTypes input module, but then the processing modules only considers the first z-slice.

I couldn’t find a similar enough issue in GitHub, or posts on this forum.

Thank you for your time!


#2

Hi,

3.0 right now only accepts TIFs as input for 3D processing, not any of the variety of proprietary formats we can use for 2D (.dv, .lsm, etc). We hope to not have that limitation in the future.

FWIW, it needs to be a single TIF per z series with separate TIFs for each channel- so in your example image you’d want to load in 2 TIFs (each with 31 planes).


#3

Cheers @bcimini! I’ll split up the .dv files by channel into .tif z series.

Edit: I tried to mark the thread as solved, but can’t remember how to do that.


#4

It was apparently disabled for the “CellProfiler Bugs” category (I assume in error).


#5

FYI, to convert the proprietary files and split the channels, I came across the excellent bfconvert utility downloadable from openmicroscopy.org at https://docs.openmicroscopy.org/bio-formats/5.7.2/users/comlinetools/ so converting and splitting was as simple as tacking on _C%c to the output filename and bio-formats automagically splits the channels and converts to .tif:

#!/bin/bash                                                                                                                                                                                   

# Download bftools for the bfconvert utility from                                                                                                                                             
# https://docs.openmicroscopy.org/bio-formats/5.7.2/users/comlinetools/                                                                                                                       
PATH=/share/apps/bftools:$PATH

in_prefix=/share/Lauren/LaurenCoreLab/AntiphophoH3S28_488_1hrblock_1hrprimary_11_14_2017
out_dir=img

mkdir -p $out_dir
for in_file in $in_prefix/*/*R3D.dv; do
    out_file=$(basename $in_file .dv)_C%c.tif
    # Fix naming for 1in1000.                                                                                                                                                                 
    out_file=${out_file/H3S28dillutions/1in1000secondary}
    bfconvert $in_file $out_dir/$out_file
done | tee convert_dv_to_tif.log