Not enough RAM to export Ilastik probabilities as tiff sequence

Zavi · November 28, 2018, 4:35pm

I am using Pixel Classification in Ilastik to export for segmentation in Cell Profiler. We recently began taking larger images, and I was working with an image in Ilastik that was about 500 x 9000 pixels. I am working on a laptop in Windows 10 with 64GB RAM. When I tried to export probabilities as a tiff sequence, as we usually do, I received an error that there was not enough RAM to export to the selected format (it only worked when I cropped the image to a lower size). There were no other significant programs running in the background, and only about 10% of memory was being used. However, when I tried to export the exact same file on my personal 2011 Macbook Pro with only 16GB RAM, it was able to export the entire file.

This problem remained in both Ilastik 1.3.2 and 1.3.0 after reinstallation. I was wondering if there is some issue with a Mac vs PC in terms of how RAM is allocated to Ilastik, or if there is any way to allocate more RAM to Ilastik or some other solution to this problem.

Thank you

k-dominik · November 28, 2018, 5:48pm

This is very interesting, thank you very much for the report. It has to be an issue in the tiff libraries we use, or how we use them. We’ll investigate!

smithrobertj · November 28, 2018, 6:29pm

Zavi,
How much ram is actually allocated to ImageJ? You can change it in Edit > options > memory & threads to see if that changes anything.

Bob

Zavi · November 28, 2018, 7:02pm

I am working in Ilastik. Is there a similar setting to allocate memory in Ilastik?

k-dominik · November 28, 2018, 7:34pm

For ilastik you can achieve that using a configuration file. It should be located in your userhome:

for windows: C:\Users\<YOURUSERNAME>\.ilastikrc
for mac, and linux you would put it to the equivalent locations.

if the file doesn’t exist, just create it and add the following contents:

[lazyflow]
lazyflow_total_ram=<MEMORY_IN_MB>

so say, you would want to limit ram usage to 8 gigabyte, you’d replace <MEMORY_IN_MB> with 8000.

That is not a guarantee that RAM usage doesn’t exceed this value, but it encourages it

Zavi · November 28, 2018, 8:29pm

I don’t see a file with the name .ilastikrc
To clarify, do I just create a text file with those 2 lines?

Is it possible that, for some reason, Ilastik’s RAM usage is currently limited, and I could increase by writing a larger value? For example, if the computer has 64GB of RAM, could I create that text file and write 64000 for <MEMORY_IN_MB> to actually increase the RAM allocated to Ilastik?

Zach

k-dominik · November 29, 2018, 6:59am

Hi @Zavi,

I think I have misread your first post. So you really had only one image with 500 x 9000? That should, in any case not use more that 1 GB of your RAM.

Could you do us a favor and, on your windows machine delete the log file (found at C:\Users\<YourUserName>\ilastik_log.txt), start ilastik again, do the export that results in the error, and send us the log file to team@ilastik.org?

Cheers
Dominik

Zavi · November 29, 2018, 5:12pm

Just sent it to the team. Thank you for your help, let me know if there is any other information I can provide.

Zach

Zavi · December 5, 2018, 5:24pm

Hi Dominik,

I was curious if there has been any progress on this question since I sent the log file to your team.

Thanks,
Zach

k-dominik · December 6, 2018, 5:45am

Hi @Zavi
thank you very much for sending in the logs! We started looking into it, but it is not something obvious…

k-dominik · December 6, 2018, 9:01am

Hey @Zavi, I looked into it some more and could you please give me a bit more information on the image you are using? Is it’s size really just 500x9000? Furthermore, to which data type are you exporting? float, or uint16…? Also are you reshuffeling the axis order to cyx before you export?

k-dominik · December 6, 2018, 10:40am

In any case we’ve figured out the problem (it is related to default number types that are used by numpy, the numerical library we use, which are different on windows…)
Will be fixed in the next release - thank you very much for discovering/reporting this issue!!

Zavi · December 6, 2018, 4:40pm

Hi Dominik,

For more information: the size is 511 x 9063. I tried cropping a few sizes to see what would export, and it worked once I got the width lower than about 5500 (i.e. 511 x 5500). For our export, I was using unsigned 8 bit (though I think I had the same issue attempting with unsigned 16 bit), transposing axis order to cyx, and formatting as a tiff sequence.

Glad to hear you figured out the problem, looking forward to the fix in the next release! Any sense of the timing on that release?

Thanks,
Zach

Zavi · January 14, 2019, 7:17pm

Hi Dominik,

I was wondering if you knew when this release would come out to fix this issue of exporting larger images on a PC. Thanks!
Zach

k-dominik · January 18, 2019, 12:31pm

Hi @Zavi,

sorry I got distracted away from this issue and left it in a half finished state. Will pick it up again and keep you in the loop

Cheers
Dominik

Zavi · February 11, 2019, 2:02pm

Hi Dominik,
I just got an email from you about the new stable release of Ilastik 1.3.2. I was just wondering if the new release addresses this RAM issue on PCs.
Thanks,
Zach

k-dominik · February 11, 2019, 2:15pm

Hey Zavi,

this didn’t make it, unfortunately. However, we’re releasing an update of our fiji import/export plugin that is much faster, and hence, you can just export to h5 and convert in fiji, if tiff stacks is what you need.
Will post about this update, here, too.

Cheers

Zavi · April 25, 2019, 3:46pm

Hi Dominik,

I got an email from you regarding the release of the beta version of Ilastik 1.3.3. I was curious if this fix for memory handling by Ilastik on PC was fixed in this version. If not, I was wondering if the update to Fiji you mentioned above (on February 11th) had been released either.

Thanks,
Zach

Hui-Ling_Ou · March 24, 2020, 5:11pm

Hi Dominik,

I encountered similar problem as @Zavi recently with ilastik Version 1.4.0b1 for PC when exporting probabilities in tiff sequence and came across your discussion here. Wonder whether this problem can be fixed in the next release? The resolution of my images are 1536 x 2048 and I was exporting in float and reshuffling the axis order to cyx before exporting.
(I also tried to export them in h5 and convert in tiff with suggested plugin, but the outcome looked weird when saved as tiff.)

Thanks,
Hui-Ling

k-dominik · March 26, 2020, 3:21pm

Hi @Zavi, Hi @Hui-Ling_Ou,

first of all, sorry for not following up. The particular issue that @Zavi encountered should be fixed (I tried triggering it right now with a similar setup and didn’t manage.
So this brings me to you @Hui-Ling_Ou could you maybe post the error you are getting here?
Also in what way is the exported output in h5 looking “weird”? (Maybe a screenshot of the ilastik probabilities in ilastik and then imported into fiji could help).

Cheers
Dominik