Bioformats won't read .czi linescans properly

I am using Bioformats 6.3.1 to read and open .czi files in Matlab R2019b, following the workflow at
https://docs.openmicroscopy.org/bio-formats/6.3.1/developers/matlab-dev.html. The .czi files contain confocal linescans which should be display as XT images.

I keep having the same problem over and over with every file I try. Bioformats reads each line in the .czi file as a plane within a series, which I guess is okay. There’s always only one series and the number of planes should depend on the length of the recording (number of lines). The problem is that Bioformats only loads planes for about 1/4 of the total number of lines in the scan, and most lines contain just zeroes – no data. I can stitch the planes into an image, and it is evident there is a lot of data missing. Here’s a link with relevant files: the original .czi file, a TIFF made with Zen 2.6 and a TIFF stiched in Matlab with the data obtained with Bioformats. https://uwmadison.box.com/s/ow5f42fjh8rl677vogaeggp12h50duza

In this case, the scan has 38825 lines but Bioformats only reads 9556 into Matlab and most. Overall, there is one line with data, followed by 4 or 5 lines with just zeroes, followed by another good line. And then, there are over 29000 lines missing… The info has to be there in the .czi file, the question is why Bioformats doesn’t access it. I actually managed to find the correct T size buried in the Metadata, but have no idea how to access pixel data.

Any ideas what’s going on?? By the way, opening these .czi files in Fiji gives the same metadata as Matlab (which is good) but the same overall issue (missing, incomplete data).

Thanks!!

Here is the Matlab code I am running:

Image = bfGetReader(id);


% Query some metadata fields (keys are format-dependent)
globalMetadata=Image.getGlobalMetadata();
OriginalMetadata = Image.getSeriesMetadata();
    javaMethod('merge', 'loci.formats.MetadataTools', ...
               globalMetadata, OriginalMetadata, 'Global ');
OmeMeta = Image.getMetadataStore();


%Incorrect info obtained by Bioformats after reading file
SizeX = OmeMeta.getPixelsSizeX(0).getValue(); % image width, pixels
SizeY = OmeMeta.getPixelsSizeY(0).getValue(); % image height, pixels
SizeT = OmeMeta.getPixelsSizeT(0).getValue(); % number of T points


%Correct info buried in Metadata.
SizeX_correct=str2double(OriginalMetadata.get('Global Information|Image|SizeX #1')); % image width, pixels
SizeY_correct=str2double(OriginalMetadata.get('Global Information|Image|SizeY #1')); % image height, pixels
SizeT_correct=str2double(OriginalMetadata.get('Global Information|Image|SizeT #1')); % CORRECT number of T points

The 38825x512 array is stored in the CZI file in 9556 segments, each containing 4x512 or 5x512 samples. Looks like Bio-Formats assumes/reads only a single line per segment, not 4 or 5…

Hi @falvara,

thanks for the bug report as well as the test file. May we make your sample file publicly available in our data repository for non-regression testing under a CC-BY license to simplify the maintenance of this reader?

Unfortunately, with everyone leaving for the holidays, it’s unlikely we will be able to get to the testing before the first full week of January. But in case you have time, a few standard questions:

  • Do you know if previous versions of Bio-Formats could open the files properly?
  • Do you have older files (of the same type of acquisition) which are not failing?
  • Were there any recent changes to the acquisition hardware or related software?

All the best,
~Josh

1 Like

Hey Josh,

Yes, no problem with sharing the images.

All the versions of Bio-Formats I’ve tried (maybe the last three) give the same result. But I actually just realized something: files recorded with Zen2.3, the older version of Zeiss’ software, work fine; however, files recorded with Zen2.6 do not – we upgraded the software in our system earlier this year and I hadn’t tried any older images until just now. I added some more files to the same folder indicating the software version.

Thanks,
Fran

1 Like

Hi,

Just out of curiosity, do those files work fine with the latest version of Zen (which is 3.1) ? There is a free version to view CZI.

Sebi (from Zeiss)

Hi Sebi,

I tried Zen 3.1. It doesn’t open any of these files as XT images, but just shows a single line at the time. I’ll need to install Zen2.6 lite again.

Interestingly, using the Create Image Subset function either in Zen2.6 or Zen3.1 creates an image with the same size as Bio-Formats! The example below is the one I talked about in my initial post: an original file of 38825 lines that is opened as having 9556 lines in Bio-Formats and Zen creates a subset of 9559 lines (as below). But again, this only happens with files recorded with Zen2.6.

Original Image
image

Creating subset of whole image:
image

Subset created:
image

Fran

As a workaround, you could use Python to convert the CZI to BigTIFF:

py -m pip install czifile tifffile
py -m czifile.czi2tif cell1.czi

The resulting cell1.czi.tif file contains the 38825x512 TX image.

When I am back in the office I will try to have a look. So initially those files were recorded with Zen 2.3 with am LSM800, correct?

Sebi, all files were recorded on an LSM800. I put the Zen version in the file name, I’ve used 2.3 and 2.6. So far the issue is with Zen2.6 files. Thanks!

Hi,

so I opened the cell1-ZEN2.6czi with ZEN 3.1 on my laptop and it looks ok too me … (38825 Frames x 51 TX image). In your case you used ZEN 3.1 lite has not all the features to show an XZ/XT view.

So at least in ZEN 3.1 everything seems to be OK. This does not answer the question, why it looks different in BioFormats.

Hi all, firstly I can certainly reproduce the behaviour in the latest Bio-Formats (dimensions showing as X=512, Y=1, T=9556). Bio-Formats wont automatically provide you with an XT view so will require some post processing, in that sense it should behave similar to Zen Lite.

Am I correct that Zen Lite also displays 512 x 1 x 9556? Or is it correctly showing 512 x 1 x 38825?

Hi guys,

ZEN blue Lite 3.1 will not display linescans correctly, because it lacks the XZ/XT viewer required to do so. But ZEN blue itself displays the file correctly.

1 Like

Hi,
I created a PR to resolve the issue in the calculateDimension method of the CZI Reader:

@falvara: Thank you for the heads up regarding CreateSubset. We filed an according bug for ZEN.

The speciality of this data is that there are multiple time Points in one SubBlock/Plane. It would expect that this also happens for XZ Scans. But XZ Scans might be pretty rare and not used for further analysis/archiving.

@cgohlke: great to see that your reader can resolve that situation correctly.

@joshmoore + @dgault: The Change seems safe to not break other CZIs. I just tested the XT CZIs that @falvara provided. As I’m still a bit unfamiliar with the BioFormats checkin/PR process and automatic test execution I hope not to have missed something.

Thanks @swg08 for the quick fix! I have included it for testing with our full repo of files in the overnight builds. Hopefully we can get the fix included for the next Bio-Formats release.

1 Like

Somehow I turned notifications off and and just came back today to read this great conversation.

@sebi06: Thanks for checking the files with ZEN Blue 3.1. I’ll continue working with 2.6 Lite to view and export XT images in my laptop. Cheers to Matt Curtis, our local Zeiss imaging specialist, for helping with this as well!

@swg08 + @joshmoore + @dgault: Glad you guys could work out a solution so quickly. This was definitely beyond my abilities. Looking forward to getting the next release of Bio-Formats! Any idea when it will come out? I got so excited I finished my Matlab code and it’s read to go. Please keep my images in your repository, the may be helpful in the future.

Thanks to all.

Happy to hear we could we help. Thanks my colleague to @swg08 for the fix and say “hi” to my friend Matt from Sebi (he will know …)

@falvara The fix will be included with Bio-Formats 6.4.0 which I am hoping to release mid February.

1 Like

Great to hear! Thank you @dgault!
@falvara: We created a bug for ZEN 3.1 and scheduled the fix for Hotfix 3 (planned to be released in end of February/mid March).

1 Like

@dgault @swg08 @sebi06

Hi guys!

I just tested Bio-Formats 6.4.0 for the issue we discussed and I have bittersweet news… Just to recap, this is Bio-Formats in MatLab following the workflow at https://docs.openmicroscopy.org/bio-formats/6.4.0/developers/matlab-dev.html, to extract the planes a create an XT image from a CZI line-scan recorded with Zen2.6.

The example CZI and reconstructed TIFF image are here: https://uwmadison.box.com/s/wacr4wcnqujse02mtx4dxs4lf7taqpb1.

I’m running this code to extract the data from the CZI file and merge the planes into an XT image.

data=bfopen(id);

seriesCount=size(data, 1);
series1=data{1, 1};
series1_planeCount=size(series1, 1);


for line=1:series1_planeCount

I(line,:)=series1{line, 1};

end

imshow(I);

On the one hand, Bio-Formats now reads the correct number of lines from the CZI file, in this case 38825 lines (series1_planeCount) – great! From my MatLab workspace:

image

However, there is still missing data. Every plane containing data is followed by two, three or four planes containing only zeroes (below). This is very evident in the reconstructed TIFF image in the shared folder and upon inspecting the image “I” in MatLab:

I think there is still a bug in bfopen when reading the CZI XT line-scan. Would you mind giving it a another look?

Thanks!
Fran

Sorry to here that it did not resolve the problem Fran. I have opened a GitHub Issue for this https://github.com/ome/bioformats/issues/3525 and we will certainly take another look at putting in place a follow up fix.