Memory leak and performance issue in Icy 2.x

Dear Icy devs,

I have 2 issues with Icy 2.x

Scenario:

  • Java / OS:

Java™ SE Runtime Environment 1.8.0_101-b13 (64 bit)
Running on Windows 10 10.0 (amd64)

  • I develop Live Mouse Tracker. In this program, I manage a continuous stream of images coming from kinect devices. So it is a specific use of Icy since I manage and create a large number of IcyBufferedImages.
  • For realtime reason, and to avoid freeze during GC, I am using a specific one, I launch Icy with this command line:

java -Xmx6G -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=60 -XX:MaxTenuringThreshold=4 -XX:MaxNewSize=384m -XX:NewSize=384m -XX:SurvivorRatio=12 -jar icy.jar -x plugins.fab.livemousetracker.LMTLauncher >log.txt 2>&1

  • My app use about 1.5GB memory all the time. With version 2.x, the memory leak is about 0.5GB per hour which lead to crash in several day experiments (and before crash, the system is lagging for a long time). (typical experiment duration is 3 days to 1 week, some users use it for 10 days).
  • Also, on the same computer, same experiment starting (so memory is not an issue at this moment). Using internal CPU monitor of Icy: 30 to 40 % CPU with 3 animals with 1.9.10 version against 60 to 70% with 2.0.3 version.

Using YourKit, I saw that there is an hashmap in your new memory cache system introduced in Icy 2.0. This links to a weakReference that is never released. I tried to de activate the cache system but this is still keeping images.

  • This is maybe due to the specific garbage collector I am using ?

Workaround:

  • I am rolling back to 1.9.10 which works fine.

It would be super cool if you could add some perf/memory tests involving continuous creation and destruction of images in your pre-test release pipe !

Best

Fabrice

Hi @Fabrice_de_Chaumont,

Thanks for reporting ! that is a really obscure problem and indeed we were able to reproduce it (not the memory leak but the fluctuant GC processing and induced lag) but having spent a bit of time on investigating it we still didn’t found the source of the problem (it seems to be related to the new cache feature but we aren’t sure yet).
Your specific usage of the application (requiring real time with specific GC settings) is indeed a bit out of the scope of the classic Icy usage and it’s why we didn’t test for that case of use. The memory leak are always a concern though and we try to avoid them (and here we couldn’t reproduce the memory leak issue). About the fluctuant GC and the extra CPU usage we will continue to investigate and get back to you when if we find something but in the meantime i’m afraid you have to stick with Icy 1.9.10 to make your application working correctly.

Cheers,

– Stephane

1 Like

Thanks for this answer !

I understand efforts on Icy are now more driven to try to handle big images, and less on performances or in real time analysis I guess.

Performances are critical for Live Mouse Tracker, and as real time application are not in the scope of Icy, and that it is not really tested in this way, maybe it’s a dangerous life for LMT to stick to Icy. I am also not using the deployment and update system of Icy because of this specific memory config that cannot be handled by Icy.

Will see :slight_smile:

Best,

Fabrice