Imagej Correct 3D drift memory leak

Hi All,
I’m using the “Correct 3D drift” plugin in FIJI as part of a large macro that loops through images. After a few cycles the memory usage is huge! I’ve tried everything to free up the memory.
My images are 2160x2160, 1ch, 52 z slices

Does anyone have any ideas?

I’ve written a simple macro to replicate the issue.

 

dir = getDirectory("Select a directory containing one or several 3d tiff series files");
files = getFileList(dir);

setBatchMode(true);
for(f=0; f<files.length; f++) {
	if(endsWith(files[f], ".tif")) {
		id = dir+files[f];
		open(id);
	
		run("Correct 3D drift", "channel=1 only=100 lowest=3 highest=15");
		//Try and tidy up...
		run("Close All");
		close("*");
		run("Collect Garbage");
		run("Reset...", "reset=[Undo Buffer]");
	}
}

Hi @JohnH,

Thanks for reporting this - consider filing an issue at the plugin’s github repo.
Looks like @Christian_Tischer is maintaining that plugin.

That being said, can you explain in a bit more detail how much memory is consumed? Besides feeling wasteful, does it cause problems (e.g. Fiji crash)?

Memory leaks can be tricky to track down, and may be especially so in this case since there’s

I for one, don’t know how these interact as far as garbage collection is concerned :-/
John

3 Likes

Dear John,

As far as I know, the Correct 3D drift plugin is correcting drift over time?!
I am just wondering because your images do not seem to have any time points…


Regarding the memory issues:

  1. Can you maybe find something helpful in this thread?
    Memory not clearing over time

  2. @oodegard Somehow above issue reminded me of the one you had, doesn’t it? Anyway, would you have some tips for John?

  3. Regarding what @bogovicj said: as long as it does not crash it may be fine… I think I heard that the Java Garbage Collector tends to only free up memory when it really needs to.

3 Likes

Hi @bogovicj,
Many thanks for your reply. The memory usage increases roughly a GB per loop of the macro. I’m using the macro as part as a high throughput pipeline and I need to restart FIJI before I get through a fraction of my data…

Hi @Christian_Tischer,
Thanks for sharing your wisdom. Ah sorry, my images also have 2 time points (2160x2160, 1ch, 52 z slices, 2 time points). They are roughly 1GB each.

-I’ve tried the trick in your link (open with bioformats importer as split time points, then concatenate to give time points for the correct 3D drift plugin), but no luck.

-FIJI hangs when it runs out of memory. See below for a plot from visual VM from the macro above for alignment of 5 images The largest jumps in ram seems to occur when alignment fails and gives a huge, unrealistic shift.

2 Likes

Naive speculation: could it be that when the alignment algorithm fails and gives a huge shift, the output image gets “sized up” to be larger to accommodate that? @Christian_Tischer Is that what the script does? If so, would it make sense to introduce a “max shift” parameter limiting how much that can occur?

P.S. to @JohnH: It makes me super happy that you used JVisualVM and posted a graph. :trophy:
P.S. to @bogovicj: It makes me super happy how helpful you are, even for projects you don’t officially maintain, and that you loop in the right people in the right situations. Thanks!
P.S. to @Christian_Tischer It makes me super happy that you maintain this script, and are so helpful here!

4 Likes

Good idea!
That is what it does, indeed:

  # Compute bounds of the new volume,
  # which accounts for all translations:
  minx, miny, minz, maxx, maxy, maxz = compute_min_max(shifts)
2 Likes

totally!

@JohnH, since you are apparently using it quite frequently, do you think we should specify such a maximal drift with one number:

Max_shift [pixels]

or should we add it for all dimensions separately?

Max_shift_x [pixels]
Max_shift_y [pixels]
Max_shift_z [pixels]

I am leaning towards the latter…

2 Likes

@ctrueden,
Good suggestion!

@Christian_Tischer
Definitely the latter. That would be really handy, thank you! Happy to help if I can, though my coding is far from perfect.
When the images do align properly, with only a few pixels shift, I’m still seeing about a GB increase in memory usage with each loop of my macro. Sing out if you have any more thoughts. Wonder if a heap dump from visual VM would show me what is using the RAM…

I think this could just be the Garbage Collector not really freeing up the memory.

I will give you something to test soon. Hopefully get around to do it today…

2 Likes

Dear @JohnH,

Could you please test below script?

  1. download: Correct_3D_drift.py.zip (8.1 KB)
  2. unzip
  3. drag and drop onto Fiji, the script will open in the script editor
  4. open an image that you would like to correct
  5. click [ Run ] in the script editor window
  6. thanks :slight_smile:
1 Like

Hi @Christian_Tischer,
Nice work!

The only issue I’ve found so far is that the drift limits only work for positive drifts. I guess take the absolute value of ‘max_shifts’ on line 170’. Otherwise, looking good. I’ll see if it makes any difference to my memory usage issues.

Thanks again.

1 Like

Thanks for noting, please try: Correct_3D_drift.py.zip (8.2 KB)

I changed the code, a bit cumbersome, because python apparently does not have a sign function: https://stackoverflow.com/questions/1986152/why-doesnt-python-have-a-sign-function

def limit_shifts_to_maximal_shifts(local_new_shift, max_shifts):
  for d in range(3):
    shift = get_Point3i(local_new_shift, d)
    if shift > max_shifts[d]:
      IJ.log("Too large drift along dimension " + str(d)
         + ":  " + str(shift)
         + "; restricting to " + str(int(max_shifts[d])))
      set_Point3i(local_new_shift, d, int(max_shifts[d]))
      continue
    if shift < -1 * max_shifts[d]:
      IJ.log("Too large drift along dimension " + str(d)
         + ":  " + str(shift)
         + "; restricting to " + str(int(-1 * max_shifts[d])))
      set_Point3i(local_new_shift, d, int(-1 * max_shifts[d]))
      continue
1 Like

@Christian_Tischer why not:

from math import copysign

def get_shift(shift,maxshift):
	if (abs(shift) > abs(maxshift)):
		shift = copysign(maxshift, shift)
	return shift


print(get_shift(1, 10))
print(get_shift(-1, 10))
print(get_shift(10, 1))
print(get_shift(-10, 1))

Edit: or even:

from math import copysign

def get_shift(shift,maxshift):
	return copysign(min(abs(shift),abs(maxshift)), shift)

print(get_shift(1, 10))
print(get_shift(-1, 10))
print(get_shift(10, 1))
print(get_shift(-10, 1))
2 Likes

Seems to be working great! Thank you :slight_smile:

2 Likes

Great! I staged a pull request for this to be merged into the official version in Fiji: https://github.com/fiji/Correct_3D_Drift/pull/7

1 Like