Too big filter size error in clij

Hi @haesleinhuepf,

I just started again testing on the 3D filters using CLIJ. I ran into a problem with the 3D median. Up to a filter size of 3 to 4 in x and y direction it works, bigger seems to be not possible. See error message below. If the error strikes in once, thereafter, the macro seems to never function anymore (not even with small radii) and throws memory allocation errors. So, I would have to restart Fiji.
It also seems to be not specifically happen after a certain radius, since I could run it for 8 in another example on the T1 head. Radii combination of 8,8,1 worked, while 8,8,2 did not.
The 3Dblur filter works even for radii of 500 (did not test anything bigger)

Is this rather:

  1. a known limitation of specific radii combinations of the current 3D median
  2. a potential problem of a specific GPU (using a GeForce GTX 1060 3G)

Macro:

run("T1 Head (2.4M, 16-bits)");
start = getTime();
original = getTitle();

run("CLIJ Macro Extensions", "cl_device=[GeForce GTX 1060 3GB]");
Ext.CLIJ_clear();
Ext.CLIJ_push(original);
Ext.CLIJ_median3DSphere(original, "CLIJ_median3DBox_destination_" + original, 6.0, 6.0, 3.0);
Ext.CLIJ_pull("CLIJ_median3DBox_destination_" + original);
Ext.CLIJ_clear();

print("ms needed: " + getTime()-start);
close(original);

Error:

java.lang.IllegalArgumentException: Error: kernels of the medianSphere filter is too big. Consider increasing MAX_ARRAY_SIZE.
	at net.haesleinhuepf.clij.kernels.Kernels.medianSphere(Kernels.java:2107)
	at net.haesleinhuepf.clij.macro.modules.Median3DSphere.executeCL(Median3DSphere.java:31)
	at net.haesleinhuepf.clij.macro.CLIJHandler.lambda$handleExtension$0(CLIJHandler.java:144)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:97)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measureForceOutput(ElapsedTime.java:45)
	at net.haesleinhuepf.clij.macro.CLIJHandler.handleExtension(CLIJHandler.java:51)
	at ij.macro.ExtensionDescriptor.dispatch(ExtensionDescriptor.java:288)
	at ij.macro.Functions.doExt(Functions.java:4787)
	at ij.macro.Functions.getStringFunction(Functions.java:276)
	at ij.macro.Interpreter.getStringTerm(Interpreter.java:1407)
	at ij.macro.Interpreter.getString(Interpreter.java:1385)
	at ij.macro.Interpreter.doStatement(Interpreter.java:329)
	at ij.macro.Interpreter.doStatements(Interpreter.java:261)
	at ij.macro.Interpreter.run(Interpreter.java:157)
	at ij.macro.Interpreter.run(Interpreter.java:91)
	at ij.macro.Interpreter.run(Interpreter.java:102)
	at ij.plugin.Macro_Runner.runMacro(Macro_Runner.java:161)
	at ij.IJ.runMacro(IJ.java:148)
	at ij.IJ.runMacro(IJ.java:137)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1108)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1104)
	at net.imagej.legacy.IJ1Helper.runMacroFriendly(IJ1Helper.java:1055)
	at net.imagej.legacy.IJ1Helper.runMacro(IJ1Helper.java:1104)
	at net.imagej.legacy.plugin.IJ1MacroEngine.eval(IJ1MacroEngine.java:147)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
1 Like

Hey @biovoxxel,

yes. That’s rather a feature than a bug :wink: I built in this error message because I didn’t find a way defining arrays with varible size in OpenCL. It’s the array which is sorted to get the median.
I know that there are more sophisticated implementations of median without that limitation, but simply didn’t have the time yet time yet to implement it.
You can workaround this by increasing MAX_ARRAY_SIZE before the first kernel call.

I’m happy to improve clijs median implementation. But how urgent is your need regarding this? Would a combination of mean and median filters with smaller radii maybe also solve the issue?

Cheers,
Robert

1 Like

Hey @haesleinhuepf,

thanks for the clarification.

Need for this is not at all urgent.
I just was puzzled by this behaviour. While a radius of 8x8x2 would need just 128 values in the array to calculate the median. Theoretically, 10x10x10 should be possible with the currently defines MAX_ARRAY_SIZE=1000, isn’t it (or am I missing something?).

But anyway, don’t waste time on it at the moment, if that is not on your priority list.

2 Likes

Radius (not diameter :wink: ) 8x8x2 means cube size 17x17x5 = 1445 voxels.

There is a discrepancy between the Java API and the macro API: Macro commands take Radii as parameter, Java commands take the kernel size. This confusing API will go away with CLIJ2.

2 Likes

:crazy_face: :blush: my bad, obvious

1 Like

Hello @haesleinhuepf,
How may I increase the MAX_ARRAY_SIZE in FIJI?

Thanks,
Rafael

Hey Rafael @deliz,

you can do this by executing this jython script in Fiji:

from net.haesleinhuepf.clij.utilities import CLKernelExecutor

CLKernelExecutor.MAX_ARRAY_SIZE = 10000

print(CLKernelExecutor.MAX_ARRAY_SIZE)

However, I strongly recommend not executing a median filter with more than 1000 elements to sort. Reason: It is extremely slow.

May I ask what you’re trying to achieve? I could for example suggest using a combination of a median filter with a small radius (e.g. 3) and a mean filter with a large radius (> 10). They might solve your problem as well but run faster by orders of magnitude.

Let me know what you think!

Cheers,
Robert

I’m trying to apply a median filter with a radius of 25-px for a background subtraction. Thanks for the suggestion!

A median filter for background subtraction is unusal. Median is more reasonable for noise removal.

Have you tried subtracting the mean filtered image instead? If yes, do the results look very different? I can also recommend the topHat filter, which includes a subtraction internally. It is very close to ImageJ “Subtract Background” method. Both should be faster than the median by orders of magnitude.

Let me know if this helps!

Cheers,
Robert

1 Like

My 10 cents… it is actually a very powerful background subtraction. It is for example recommended in the publication from Dunn et al.. That was, why I included it in the BioVoxxel Toolbox as the convoluted background subtraction. Would be another possibility to include in CLIJ2 but obviously it can be done also stepwise with by subtracting a filtered copy. Furthermore, if the median filtered image is slightly grayscale dilated before subtraction it compensates the size changes of the median filter and even removes remaining edge artefacts around objects. It normally performs better that the rolling ball function. This can also be done with a mean filter but is usually not as good in subtraction performance.

Hey @biovoxxel,

Oh wow, I wasn’t aware. Do you also use the naive median variant which sorts an array or do you have a more sophisticated version?

Also, if we subtract a median filtered image from the original, don’t we get stripes where there were edges in the original?

I trust your words - however is there any quantification of this comparison available? Would be nice to take a close look.

If you point me to the code where your median background subtraction is implemented I’d be happy to take a look and check if it’s easy to translate to OpenCL.

Thanks for the feedback!

Cheers,
Robert

Mean is not suitable because we’re trying to measure the intensity of small puncta that are really bright relative to their surroundings. Thus, the median will be less susceptible to an intensity shift when there are few bright pixels. I tried the Top Hat method, but the puncta don’t appear as prominent as the median.

Currently, I use the native median from FIJI,
run("Median...", "radius=25 stack");

Here’s a still from an image stack, which is what we’re analyzing:

We are attempting to remove the cytosolic background from an image to quantify the dynamics and intensity of diffraction-limited puncta using TrackMate. Our cytosolic background fluorescence is shifting between frames. Thanks for all the suggestions!

Hey @deliz,

nice images and an interesting application! Would you mind sharing the original image so that we can play a bit with different filters?

Thanks!

Cheers,
Robert

Hi @haesleinhuepf,

first of all, here’s the link:

I use the standard median filter, but compensate object size reductions with a small dilation. That avoids exactly the stripe artefacts you were talking about. The correction factor is Math.floor(filterRadius/10)*1.5

I did not do a quantitative evaluation, but tested on various images and compared the Subtract Background... with the median subtraction. Here is an example:

Add-on: Here is also the link to the video from the IJ conference, where this is shown in action and explained.
https://vimeo.com/140929696 it starts at time 15’45’’ and a slide is shown at 11’30’’

And another edit, I finally found the slide :grimacing:

1 Like

Hello @haesleinhuepf,
Here’s the link to the picture: Image.tif. Thanks!

Best regards,
Rafael

1 Like