I’m dealing with datasets around the 10Gb mark - CZI files from a Zeiss 880. Bioformats opens them fine in Matlab but trying to process after loading crashes Matlab with no errors ( terminal just says ‘killed’ ).
3d median filter is the best way to get it to break, playing around I’ve found smaller files ( 1Gb) don’t crash it until several have been run.
I’ve tried increasing the memory settings in Matlab, it happened in Ubuntu 18, 20 and Xubuntu with Matlab 2018,19 and 20 - all fresh installs with the latest Bioformats.
If I save the data from Matlab in another format, reboot and load, it’s fine, it’s a workaround but it’s also a massive pain when you’ve a lot of files to process. Using ‘clear’ to wipe variables in Matlab after loading doesn’t work.
Has anyone come across anything like this - and found a solution?