Bioformats crashing Matlab with CZI files

Usage & Issues
#1

Hi
I’m dealing with datasets around the 10Gb mark - CZI files from a Zeiss 880. Bioformats opens them fine in Matlab but trying to process after loading crashes Matlab with no errors ( terminal just says ‘killed’ ).
3d median filter is the best way to get it to break, playing around I’ve found smaller files ( 1Gb) don’t crash it until several have been run.
I’ve tried increasing the memory settings in Matlab, it happened in Ubuntu 18, 20 and Xubuntu with Matlab 2018,19 and 20 - all fresh installs with the latest Bioformats.

If I save the data from Matlab in another format, reboot and load, it’s fine, it’s a workaround but it’s also a massive pain when you’ve a lot of files to process. Using ‘clear’ to wipe variables in Matlab after loading doesn’t work.

Has anyone come across anything like this - and found a solution?

cheers
Ian

#2

Hi @Scalebar, do you have the Matlab code you are using to read the files?

#3

Hi
This is a basic script that causes the problem. If I open in ImageJ, save as Tiff, import then use the equivalent code it’s fine, and if I just save the matrices. Other memory heavy operations also trip it up.
Thanks
Ian

Newimage=bfopen(’/home/ian/Zenfiles/10xtl5.czi’);
Count=size(Newimage{1});
Count=Count(1); % roundabout way to get stack size
Channelarray1=zeros(1000,1000,Count/2); % Raw data file - Odd planes Ch1, Even planes CH2
Channelarray2=zeros(1000,1000,Count/2);
for Planenumber=1:2:Count
Framenumber=floor(Planenumber/2)+1;
Channel1=Newimage{1, 1}{Planenumber, 1};
Channel2=Newimage{1,1}{Planenumber+1,1};
Channelarray1(:,:,Framenumber)=Channel1;
Channelarray2(:,:,Framenumber)=Channel2;
end

FilteredChan1=medfilt3(Channelarray1);
FilteredChan2=medfilt3(Channelarray2);

#4

My suspicion is that the extra memory will be due to large amounts of metadata being loaded from the CZI file. If you are only opening the pixel data from the file then instead it might be more performant to use the read as in https://docs.openmicroscopy.org/bio-formats/6.5.1/developers/matlab-dev.html#reading-from-an-image-file

So instead the code might look like:

reader = bfGetReader(’/home/ian/Zenfiles/10xtl5.czi’);
Count=reader.getImageCount(); 
Channelarray1=zeros(1000,1000,Count/2); % Raw data file - Odd planes Ch1, Even planes CH2
Channelarray2=zeros(1000,1000,Count/2);
for Planenumber=1:2:Count
Framenumber=floor(Planenumber/2)+1;
Channel1 = bfGetPlane(reader, Planenumber);
Channel1 = bfGetPlane(reader, Planenumber+1);
Channelarray1(:,:,Framenumber)=Channel1;
Channelarray2(:,:,Framenumber)=Channel2;
end

FilteredChan1=medfilt3(Channelarray1);
FilteredChan2=medfilt3(Channelarray2);