Import to omero - No IFDs found

Hello

I am facing strange problem while importing data to Omero server 5.5.1. So I have nextflow pipeline running on the farm node the following step (command.sh) executing remotely (via ssh -t) file import.sh

Pipeline step .command.sh

#!/bin/bash -euo pipefail
ssh -t omero-server '/home/omero/import.sh "/path/to/Max_intensity.ome.tif" Project user "myproject" "mydataset"'

import.sh:

bin/omero import --transfer=ln_s --parallel-upload 10 --skip checksum -T 'Project:+name:myproject/Dataset:+name:mydataset' /path/to/Max_intensity.ome.tif
2020-03-16 19:51:17,056 2073       [      main] ERROR     ome.formats.importer.cli.ErrorHandler - FILE_EXCEPTION: /path/to/Max_intensity.ome.tif
loci.formats.FormatException: No IFDs found
        at loci.formats.in.MinimalTiffReader.initFile(MinimalTiffReader.java:470) ~[formats-bsd.jar:6.1.1]
        at loci.formats.in.BaseTiffReader.initFile(BaseTiffReader.java:583) ~[formats-bsd.jar:6.1.1]
        at loci.formats.FormatReader.setId(FormatReader.java:1389) ~[formats-api.jar:6.1.1]
        at loci.formats.in.TiffDelegateReader.setId(TiffDelegateReader.java:92) ~[formats-bsd.jar:6.1.1]
        at loci.formats.ImageReader.setId(ImageReader.java:843) ~[formats-api.jar:6.1.1]
        at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:650) ~[formats-api.jar:6.1.1]
        at loci.formats.ChannelFiller.setId(ChannelFiller.java:223) ~[formats-bsd.jar:6.1.1]
        at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:650) ~[formats-api.jar:6.1.1]
        at loci.formats.ChannelSeparator.setId(ChannelSeparator.java:293) ~[formats-bsd.jar:6.1.1]
        at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:650) ~[formats-api.jar:6.1.1]
        at loci.formats.Memoizer.setId(Memoizer.java:662) ~[formats-bsd.jar:6.1.1]
        at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:650) ~[formats-api.jar:6.1.1]
        at ome.formats.importer.ImportCandidates.singleFile(ImportCandidates.java:427) [omero-blitz.jar:5.5.3]
        at ome.formats.importer.ImportCandidates.handleFile(ImportCandidates.java:576) [omero-blitz.jar:5.5.3]
        at ome.formats.importer.ImportCandidates.execute(ImportCandidates.java:384) [omero-blitz.jar:5.5.3]
        at ome.formats.importer.ImportCandidates.<init>(ImportCandidates.java:222) [omero-blitz.jar:5.5.3]
        at ome.formats.importer.ImportCandidates.<init>(ImportCandidates.java:174) [omero-blitz.jar:5.5.3]
        at ome.formats.importer.cli.CommandLineImporter.<init>(CommandLineImporter.java:148) [omero-blitz.jar:5.5.3]
        at ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:989) [omero-blitz.jar:5.5.3]

Any thought why?

If I run directly on the omero-server machine is fine. so file is deffintely not corrupted

Farm node:
openjdk version “11.0.6” 2020-01-14
OpenJDK Runtime Environment (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1)
OpenJDK 64-Bit Server VM (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1, mixed mode, sharing)

omero-server:
openjdk version “1.8.0_232”
OpenJDK Runtime Environment (build 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)

It sounds like this might be due to the cluster filesystem. A couple of things you could try are:

  • Run Bio-Formats showinf as a cluster job, and see if you get a similar error
  • Write a small Java program (without Bio-Formats) that just opens and closes the file and run it in the cluster

Thanks Simon for the reply,

Is MinimalTiffReader using any of getModified or getLastModified, this may be an issue with java reading the file attributes when the jvm is started ?

MinimalTiffReader itself doesn’t check modification times. Memoizer will check memo file modification times with lastModified(), but the stack trace indicates that a memo file is not present so I wouldn’t expect that to be the problem.

As Simon said, testing with showinf -nopix /path/to/Max_intensity.ome.tif or very simple Java code such as:

try (ImageReader reader = new ImageReader()) {
    reader.setId("/path/to/Max_intensity.ome.tif");
}

may help to pin down the problem. If you haven’t already, you might also try running the import command on the cluster with increased logging (--debug DEBUG) and/or without --parallel-upload 10 just to make sure neither of those options sheds more light.

Hi Melissa, thank you for the quick reply, I was debbuging more deeply and it looks like it is pipline problem with the way pipeline framework publish the file rather then omero. Thank you for the info I will addapt your suggestion to the existing pipeline