CLIJ2 BETA testing - call for testers 👷

Dear friends of #clij and #GPU accelerated image processing,

I’m happy to announce that CLIJ2 BETA testing is open to the public. This is a call for testers! If we manage to break clij2 now, it will be more stable after release mid June. Thanks to everyone taking part in advance! :star_struck:

Compared to CLIJ, CLIJ2 offers 224 new operations, some improve older methods, some offer new functionalty, others are convenience methods making your life with pixels on GPUs easier :wink:

image

New concepts

Vectors, matrices, pointlists, meshes and corresponding operations

CLIJ2 has various new operations for processing data structures such as vectors and matrices, for working with graphs and meshes.

Non-square shaped pixels, a.k.a. cells

Furthermore, CLIJ2 offers operations for filtering images with non-square shaped pixels managed using the new graph-based processing operations.

ByRef

CLIJ2 operations use the by-reference concept which is well known in ImageJ macro for images and other parameters. Undefined variables can be passed over and will hold values after the operation finished. This allows simplifying code.

// CLIJ
Ext.CLIJ_centerOfMass(binary);
mx = getResult("MassX", nResults() - 1);
my = getResult("MassY", nResults() - 1);

// --------------------------
// CLIJ2
Ext.CLIJ2_getCenterOfMass(binary, mx, my, _);

Easier debugging

CLIJ2 offers various methods for easier workflow debugging such has handling arrays and strings

// push array as 2x3 image
array = newArray(1, 2, 3, 4, 5, 6);
Ext.CLIJ2_pushArray(input, array, 2, 3, 1);

// do something with it
Ext.CLIJ2_multiplyImageAndScalar(input, result, 3);

// print output image
Ext.CLIJ2_print(result);

image

Time tracing

To find out which operations take how much time, you can collect time traces:

Ext.CLIJ2_startTimeTracing();
Ext.CLIJ2_multiplyImageAndScalar(input, result, 3);
Ext.CLIJ2_stopTimeTracing();

// print out how long operations took
Ext.CLIJ2_getTimeTracing(time_traces);
print(time_traces);

image

Memory management for Java/Jython/Groovy/JavasSript developers

Developers can name images and buffers and list memory consumption

CLIJ2 clij2 = CLIJ2.getInstance();

ClearCLBuffer data = clij2.create(100, 100);
data.setName("My data");

System.out.println(clij2.reportMemory());

Output:

GPU contains 1 images.
- My data 39.0 kB [My data ClearCLBuffer [mClearCLContext=ClearCLContext [device=ClearCLDevice [mClearCLPlatform=ClearCLPlatform [name=NVIDIA CUDA], name=GeForce RTX 2060 SUPER]], mNativeType=Float, mNumberOfChannels=1, mDimensions=[100, 100], getMemAllocMode()=Best, getHostAccessType()=ReadWrite, getKernelAccessType()=ReadWrite, getBackend()=net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL@63e2203c, getPeerPointer()=net.haesleinhuepf.clij.clearcl.ClearCLPeerPointer@1efed156]] 39.0 kB
= 39.0 kB

Also the method clij2.clear(); is new compared to CLIJ, it releases all images/buffers in GPU memory.

Documentation

The new CLIJ2 website offers extensive documentation, including operations which typically follow other operations.

New cheat sheets

The cheat sheets have been updated to list new CLIJ2 operations.

Installation

Update your Fiji with #clij and clij2 update sites activated, search for “Clicy” in the Icy search bar or check the installation instructions for Matlab.

Timeline

CLIJ2 is scheduled to release mid June 2020 - in 4 weeks. With that day CLIJ will deprecate and receive maintenance and support for at least another year. CLIJ users might start having a look at the CLIJ-CLIJ2 transition guide.

If anyone finds a bug or has a question, feel free to drop it here or open a new thread.

Thanks everyone for your support! :heart: :green_heart: :blue_heart:

Happy coding!

Cheers,
Robert

18 Likes

Hi @haesleinhuepf,
First of all congratulations with the BETA release of CLIJ2. It has become a really great and very useful package!

So it is with a little pain in my heart that I have to report the first bug :wink:, following up on our discussion about watershed on the GPU. Although detectMaxima3D has been substantially improved, the watershed operation gives me the following result (original - Fiji watershed - CLIJ2 watershed)

Apparenty you got a different result when you tested it than I get on nuclei_binary.tif (198.6 KB).
Edit: it also messes up a binarized blobs.gif.

Any clue?
Thanks!
Bram

1 Like

Hey @bramvdbroek,

sorry, I guess that last-minute optimization I built in wasn’t well thought-through. My fault. I just updated clij2 on the update site and the result should now look like this:

Furthermore, it supports 3D! The 2D dataset you shared comes from a 3D data set, right? Could you try it in 3D?

Thanks!

Cheers,
Robert

3 Likes

Thanks @haesleinhuepf, for fixing this so fast. Unfortunately, though it indeed works well on that particular image, the issue still remains for other images, e.g. these two:


And yes, it works in 3D. :+1:
I’ve tried several other 3D watershed options, but with these relatively high-res images it never works really well, possibly because at low and/or high z the thresholded part becomes irregular, causing extra watershed lines, while it fails to split other nuclei:


It also seems that the watershed lines are the same for all slices, making it not really 3D. (This is by the way also true for the other 3D watershed methods.) Even MorpholibJ’s Distance Transform Watershed 3D doesn’t do great on these stacks. I usually get better results when segmenting the nuclei in 2D and count substructures in 3D. Although now I have to try StarDist in 3D…
Here’s the file if you want to play yourself: 3D_binary-crop.tif (4.0 MB)

In the mean time I’ll look for a more suitable dataset, with more slices above and below, to test this further.

Best regards,
Bram

1 Like

Hey @bramvdbroek,

that’s unfortunate. I might have a good explanation why it does in 3D what it does in 3D: The objects do have a wobbly structure. If I was a watershed algorithm, I would cut the nuclei in 3D:

Typically, the solution is blurring the image before thresholding. This solution might help here as well.

To explain the problem with the parallel implementation of the watershed a bit more in detail: Assume you have a distance map like this:

0 0 1 1 1
1 1 1 2 1
1 2 1 1 1
1 1 1 0 0

It detects two maxima and cuts the object. We now need an algorithm which starts looking from any pixel and can make a decision on which of the two maxima to keep. This is not trivial. I’m just thinking of cutting off the tips of the hills and then searching for maxima-plateaus. Not sure if this is the solution. Any idea is welcome!

Cheers,
Robert

1 Like

Hi @haesleinhuepf,

Yes, that’s what I tried to say with ‘at low and/or high z the thresholded part becomes irregular’. :slight_smile: The reason for that is that in z the boundary of the nucleus is less sharp (in theory) / much less sharp (in practice) than in x and y. (standard confocal with cells on a coverslip - far from istropic)
Smoothing will certainly help, but since there are not many slices above and below in this case, the whole thing becomes almost 2D anyway.
I’ll find a dataset where nuclei are clumped also in the z-direction. I foresee that also quite a lot of xy smoothing is needed, which will have a negative influence on the watershed results in planes where the nuclei are in focus. So we need a differential filter that apdapts smoothing based on the intensity and/or sharpness. A 3D bilateral filter, or Gaussian weighted median filter perhaps?

You probably need to add a ‘radius’ parameter then (which is fine), or do you know a smart automatic way of estimating it?
I guess would be more or less similar to internally smoothing the distance map before detecting the maxima, like I did here.

Bram

Hello @haesleinhuepf
Thank you so much for all the updates and congratulations on this relaease! Are there any updates regarding the implementation of filters such as the Tubeness one or any other ones that rely on hessian eigenvalues?

Best wishes,
Rita

1 Like

Hey Rita,

bad news, I couldn’t figure out how to translate the tubeness measurement to CLIJ. Regarding the Hessian: A collaborators project has these filters. I’ll ask them if they are willing to share. Stay tuned!

Cheers,
Robert

2 Likes

Hello!

I’ve been writing a macro that uses the CLIJx_subtractBackground3D and Ext.CLIJ2_addImages, and after updating I now get this error everytime i try to run the macro with those functions.

The error appears to be similar, here’s another macro I wrote to test it the use of these two functions separately

close("*");
run("T1 Head (2.4M, 16-bits)");
input = getTitle();
getDimensions(width, height, channels, slices, frames);

blurred = "Blurred";
blurred_mask = "blurmask";
input_mask = "inmask";
FINAL = "final image";


run("CLIJ2 Macro Extensions", "cl_device="); //initiate clij
Ext.CLIJ2_clear();
Ext.CLIJ2_push(input);
Ext.CLIJx_subtractBackground3D(input, input_noback, 0.2, 0.2, 0.2);

Ext.CLIJ2_mean3DBox(input, blurred, 3, 3, 3);
Ext.CLIJ2_thresholdHuang(blurred, blurred_mask);
Ext.CLIJ2_thresholdHuang(input, input_mask);
Ext.CLIJ2_addImages(input_mask, blurred_mask, FINAL);


Ext.CLIJ2_pull(FINAL);

And here’s the console error I get

Error when trying to create kernel add_images_weighted_3d
net.haesleinhuepf.clij.clearcl.exceptions.OpenCLException: OpenCL error: -45 -> CL_INVALID_PROGRAM_EXECUTABLE
	at net.haesleinhuepf.clij.clearcl.backend.BackendUtils.checkOpenCLErrorCode(BackendUtils.java:352)
	at net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL.lambda$getKernelPeerPointer$19(ClearCLBackendJOCL.java:601)
	at net.haesleinhuepf.clij.clearcl.backend.BackendUtils.checkExceptions(BackendUtils.java:156)
	at net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL.getKernelPeerPointer(ClearCLBackendJOCL.java:593)
	at net.haesleinhuepf.clij.clearcl.ClearCLCompiledProgram.createKernel(ClearCLCompiledProgram.java:137)
	at net.haesleinhuepf.clij.clearcl.ClearCLProgram.createKernel(ClearCLProgram.java:685)
	at net.haesleinhuepf.clij.clearcl.util.CLKernelExecutor.getKernel(CLKernelExecutor.java:353)
	at net.haesleinhuepf.clij.clearcl.util.CLKernelExecutor.enqueue(CLKernelExecutor.java:229)
	at net.haesleinhuepf.clij2.CLIJ2.lambda$executeSubsequently$0(CLIJ2.java:425)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:97)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:28)
	at net.haesleinhuepf.clij2.CLIJ2.executeSubsequently(CLIJ2.java:416)
	at net.haesleinhuepf.clij2.CLIJ2.executeSubsequently(CLIJ2.java:403)
	at net.haesleinhuepf.clij2.CLIJ2.executeSubsequently(CLIJ2.java:398)
	at net.haesleinhuepf.clij2.CLIJ2.execute(CLIJ2.java:383)
	at net.haesleinhuepf.clij2.plugins.AddImagesWeighted.addImagesWeighted(AddImagesWeighted.java:47)
	at net.haesleinhuepf.clij2.plugins.AddImages.addImages(AddImages.java:28)
	at net.haesleinhuepf.clij2.CLIJ2Ops.addImages(CLIJ2Ops.java:2294)
	at net.haesleinhuepf.clij2.plugins.AddImages.executeCL(AddImages.java:24)
	at net.haesleinhuepf.clij.macro.CLIJHandler.lambda$handleExtension$0(CLIJHandler.java:163)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:97)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:28)
	at net.haesleinhuepf.clij.macro.CLIJHandler.handleExtension(CLIJHandler.java:53)
	at ij.macro.ExtensionDescriptor.dispatch(ExtensionDescriptor.java:288)
	at ij.macro.Functions.doExt(Functions.java:4787)
	at ij.macro.Functions.getStringFunction(Functions.java:276)
	at ij.macro.Interpreter.getStringTerm(Interpreter.java:1407)
	at ij.macro.Interpreter.getString(Interpreter.java:1385)
	at ij.macro.Interpreter.doStatement(Interpreter.java:329)
	at ij.macro.Interpreter.doStatements(Interpreter.java:261)
	at ij.macro.Interpreter.run(Interpreter.java:157)
	at ij.macro.Interpreter.run(Interpreter.java:91)
	at ij.macro.Interpreter.run(Interpreter.java:102)
	at ij.plugin.Macro_Runner.runMacro(Macro_Runner.java:161)
	at ij.IJ.runMacro(IJ.java:148)
	at ij.IJ.runMacro(IJ.java:137)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1148)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1144)
	at net.imagej.legacy.IJ1Helper.runMacroFriendly(IJ1Helper.java:1095)
	at net.imagej.legacy.IJ1Helper.runMacro(IJ1Helper.java:1144)
	at net.imagej.legacy.plugin.IJ1MacroEngine.eval(IJ1MacroEngine.java:145)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
java.lang.NullPointerException
	at net.haesleinhuepf.clij2.CLIJ2.execute(CLIJ2.java:384)
	at net.haesleinhuepf.clij2.plugins.AddImagesWeighted.addImagesWeighted(AddImagesWeighted.java:47)
	at net.haesleinhuepf.clij2.plugins.AddImages.addImages(AddImages.java:28)
	at net.haesleinhuepf.clij2.CLIJ2Ops.addImages(CLIJ2Ops.java:2294)
	at net.haesleinhuepf.clij2.plugins.AddImages.executeCL(AddImages.java:24)
	at net.haesleinhuepf.clij.macro.CLIJHandler.lambda$handleExtension$0(CLIJHandler.java:163)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:97)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:28)
	at net.haesleinhuepf.clij.macro.CLIJHandler.handleExtension(CLIJHandler.java:53)
	at ij.macro.ExtensionDescriptor.dispatch(ExtensionDescriptor.java:288)
	at ij.macro.Functions.doExt(Functions.java:4787)
	at ij.macro.Functions.getStringFunction(Functions.java:276)
	at ij.macro.Interpreter.getStringTerm(Interpreter.java:1407)
	at ij.macro.Interpreter.getString(Interpreter.java:1385)
	at ij.macro.Interpreter.doStatement(Interpreter.java:329)
	at ij.macro.Interpreter.doStatements(Interpreter.java:261)
	at ij.macro.Interpreter.run(Interpreter.java:157)
	at ij.macro.Interpreter.run(Interpreter.java:91)
	at ij.macro.Interpreter.run(Interpreter.java:102)
	at ij.plugin.Macro_Runner.runMacro(Macro_Runner.java:161)
	at ij.IJ.runMacro(IJ.java:148)
	at ij.IJ.runMacro(IJ.java:137)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1148)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1144)
	at net.imagej.legacy.IJ1Helper.runMacroFriendly(IJ1Helper.java:1095)
	at net.imagej.legacy.IJ1Helper.runMacro(IJ1Helper.java:1144)
	at net.imagej.legacy.plugin.IJ1MacroEngine.eval(IJ1MacroEngine.java:145)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)


Any clue on what’s happening to get this error?

Best wishes,
Rita

Hi @Rita,

sorry for the inconvenience. I obviously introduced that bug in the last release on Monday. Thanks for reporting it! If you update your Fiji, it shoud be gone and your script should work.

Let me know if it helps!

Cheers,
Robert

1 Like

Hello @haesleinhuepf,

Lately I’m getting a lot of CLIJ2/OpenCL errors, with multiple operations actually.
For instance, the simple macro

run("CLIJ2 Macro Extensions", "cl_device=");
run("T1 Head (2.4M, 16-bits)");
image1 = getTitle();

radius = 2;

// compute 3D mean
Ext.CLIJ2_push(image1);
Ext.CLIJ2_mean3DBox(image1, image2, radius, radius, radius);
Ext.CLIJ2_pull(image2);

// compute 3D median
Ext.CLIJ2_push(image1);
Ext.CLIJ2_median3DBox(image1, image3, radius, radius, radius);
Ext.CLIJ2_pull(image3);

Ext.CLIJ2_clear();

works fine for radius <= 2, but for radius >=3 it throws these exceptions in the console:

net.haesleinhuepf.clij.clearcl.exceptions.OpenCLException: OpenCL error: -36 -> CL_INVALID_COMMAND_QUEUE
	at net.haesleinhuepf.clij.clearcl.backend.BackendUtils.checkOpenCLError(BackendUtils.java:346)
	at net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL.lambda$waitQueueToFinish$46(ClearCLBackendJOCL.java:1208)
	at net.haesleinhuepf.clij.clearcl.backend.BackendUtils.checkExceptions(BackendUtils.java:171)
	at net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL.waitQueueToFinish(ClearCLBackendJOCL.java:1207)
	at net.haesleinhuepf.clij.clearcl.ClearCLQueue.waitToFinish(ClearCLQueue.java:56)
	at net.haesleinhuepf.clij.clearcl.ClearCLKernel.lambda$run$0(ClearCLKernel.java:497)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:97)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:64)
	at net.haesleinhuepf.clij.clearcl.ClearCLKernel.run(ClearCLKernel.java:477)
	at net.haesleinhuepf.clij.clearcl.ClearCLKernel.run(ClearCLKernel.java:459)
	at net.haesleinhuepf.clij.clearcl.util.CLKernelExecutor.lambda$enqueue$1(CLKernelExecutor.java:267)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:97)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:28)
	at net.haesleinhuepf.clij.clearcl.util.CLKernelExecutor.enqueue(CLKernelExecutor.java:266)
	at net.haesleinhuepf.clij2.CLIJ2.lambda$executeSubsequently$0(CLIJ2.java:431)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:97)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:28)
	at net.haesleinhuepf.clij2.CLIJ2.executeSubsequently(CLIJ2.java:422)
	at net.haesleinhuepf.clij2.CLIJ2.executeSubsequently(CLIJ2.java:409)
	at net.haesleinhuepf.clij2.CLIJ2.executeSubsequently(CLIJ2.java:404)
	at net.haesleinhuepf.clij2.CLIJ2.execute(CLIJ2.java:389)
	at net.haesleinhuepf.clij2.plugins.Median3DBox.median3DBox(Median3DBox.java:57)
	at net.haesleinhuepf.clij2.CLIJ2Ops.median3DBox(CLIJ2Ops.java:3992)
	at net.haesleinhuepf.clij2.plugins.Median3DBox.executeCL(Median3DBox.java:37)
	at net.haesleinhuepf.clij.macro.CLIJHandler.lambda$handleExtension$0(CLIJHandler.java:163)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:97)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:28)
	at net.haesleinhuepf.clij.macro.CLIJHandler.handleExtension(CLIJHandler.java:53)
	at ij.macro.ExtensionDescriptor.dispatch(ExtensionDescriptor.java:288)
	at ij.macro.Functions.doExt(Functions.java:4787)
	at ij.macro.Functions.getStringFunction(Functions.java:276)
	at ij.macro.Interpreter.getStringTerm(Interpreter.java:1407)
	at ij.macro.Interpreter.getString(Interpreter.java:1385)
	at ij.macro.Interpreter.doStatement(Interpreter.java:329)
	at ij.macro.Interpreter.doStatements(Interpreter.java:261)
	at ij.macro.Interpreter.run(Interpreter.java:157)
	at ij.macro.Interpreter.run(Interpreter.java:91)
	at ij.macro.Interpreter.run(Interpreter.java:102)
	at ij.plugin.Macro_Runner.runMacro(Macro_Runner.java:161)
	at ij.IJ.runMacro(IJ.java:148)
	at ij.IJ.runMacro(IJ.java:137)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1148)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1144)
	at net.imagej.legacy.IJ1Helper.runMacroFriendly(IJ1Helper.java:1095)
	at net.imagej.legacy.IJ1Helper.runMacro(IJ1Helper.java:1144)
	at net.imagej.legacy.plugin.IJ1MacroEngine.eval(IJ1MacroEngine.java:145)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
net.haesleinhuepf.clij.clearcl.exceptions.OpenCLException: OpenCL error: -5 -> CL_OUT_OF_RESOURCES
	at net.haesleinhuepf.clij.clearcl.backend.BackendUtils.checkOpenCLError(BackendUtils.java:346)
	at net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL.lambda$enqueueReadFromBuffer$24(ClearCLBackendJOCL.java:791)
	at net.haesleinhuepf.clij.clearcl.backend.BackendUtils.checkExceptions(BackendUtils.java:171)
	at net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL.enqueueReadFromBuffer(ClearCLBackendJOCL.java:790)
	at net.haesleinhuepf.clij.clearcl.ClearCLBuffer.writeTo(ClearCLBuffer.java:400)
	at net.haesleinhuepf.clij.clearcl.ClearCLBuffer.writeTo(ClearCLBuffer.java:366)
	at net.haesleinhuepf.clij.converters.implementations.ClearCLBufferToImagePlusConverter.convert(ClearCLBufferToImagePlusConverter.java:61)
	at net.haesleinhuepf.clij.converters.implementations.ClearCLBufferToImagePlusConverter.convert(ClearCLBufferToImagePlusConverter.java:26)
	at net.haesleinhuepf.clij.CLIJ.convert(CLIJ.java:469)
	at net.haesleinhuepf.clij.CLIJ.show(CLIJ.java:352)
	at net.haesleinhuepf.clij.macro.CLIJHandler.pullFromGPU(CLIJHandler.java:250)
	at net.haesleinhuepf.clij2.plugins.Pull.executeCL(Pull.java:24)
	at net.haesleinhuepf.clij.macro.CLIJHandler.lambda$handleExtension$0(CLIJHandler.java:163)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:97)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:28)
	at net.haesleinhuepf.clij.macro.CLIJHandler.handleExtension(CLIJHandler.java:53)
	at ij.macro.ExtensionDescriptor.dispatch(ExtensionDescriptor.java:288)
	at ij.macro.Functions.doExt(Functions.java:4787)
	at ij.macro.Functions.getStringFunction(Functions.java:276)
	at ij.macro.Interpreter.getStringTerm(Interpreter.java:1407)
	at ij.macro.Interpreter.getString(Interpreter.java:1385)
	at ij.macro.Interpreter.doStatement(Interpreter.java:329)
	at ij.macro.Interpreter.doStatements(Interpreter.java:261)
	at ij.macro.Interpreter.run(Interpreter.java:157)
	at ij.macro.Interpreter.run(Interpreter.java:91)
	at ij.macro.Interpreter.run(Interpreter.java:102)
	at ij.plugin.Macro_Runner.runMacro(Macro_Runner.java:161)
	at ij.IJ.runMacro(IJ.java:148)
	at ij.IJ.runMacro(IJ.java:137)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1148)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1144)
	at net.imagej.legacy.IJ1Helper.runMacroFriendly(IJ1Helper.java:1095)
	at net.imagej.legacy.IJ1Helper.runMacro(IJ1Helper.java:1144)
	at net.imagej.legacy.plugin.IJ1MacroEngine.eval(IJ1MacroEngine.java:145)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Clearing 
Releasing buffer CLIJ2_mean3DBox_result3
Releasing buffer CLIJ2_median3DBox_result4
Releasing buffer t1-head.tif

After that error I have to restart Fiji to get CLIJ2 back on its feet.
I’m using an NVidia GeForce GTX 850M card with 2 GB memory, OpenCL version: 1.2.

Any clue?

Some observations that may or may not be useful:

  • It should not be a memory issue, since the image is only a few tens of MB. That said, after first downsamplingthe image it works OK with radius=3, but crashes again at radius=4.
  • I have very similar issues with Ext.CLIJ2_medianSliceBySliceBox, and possibly some other CLIJ2 commands.

Thanks for the support!
Bram

1 Like

Hey @bramvdbroek,

would you mind trying to add an Ext.CLIJ2_clear() right after initialization of the CLIJ2 extensions? explanation

Furthermore, you’re pushing image1 twice. That will technically work but might take unnecessary time.

Downsampling? The code doesn’t do downsampling. Are there maybe other scripts running as well? :upside_down_face:

Let me know if that helps!

Cheers,
Robert

1 Like

5:19, you were up early, @haesleinhuepf! Such devotion…

Unfortunately the Ext.CLIJ2_clear() statement right after initialization of the CLIJ2 extensions doesn’t help. I still get
net.haesleinhuepf.clij.clearcl.exceptions.OpenCLException: OpenCL error: -36 -> CL_INVALID_COMMAND_QUEUE, sometimes followed by
net.haesleinhuepf.clij.clearcl.exceptions.OpenCLException: OpenCL error: -5 -> CL_OUT_OF_RESOURCES..

I actually forgot to tell you that Ext.CLIJ2_mean3DBox works well, but it crashes on Ext.CLIJ2_median3DBox (or anyother CLIJ2 median command).

And you are right, the downsampling I added only later. (By the way, the console also says downsample3D is deprecated? It doesn’t look like that in the docs). Here is the complete code, without pushing image1 twice:

run("CLIJ2 Macro Extensions", "cl_device=");
Ext.CLIJ2_clear();

run("T1 Head (2.4M, 16-bits)");
image1 = getTitle();
Ext.CLIJ2_push(image1);

radius = 2;

// compute 3D mean
Ext.CLIJ2_mean3DBox(image1, image2, radius, radius, radius);
Ext.CLIJ2_pull(image2);

// compute 3D median
Ext.CLIJ2_downsample3D(image1, image3, 0.5, 0.5, 0.5);
Ext.CLIJ2_median3DBox(image3, image4, radius, radius, radius);
Ext.CLIJ2_pull(image4);

Ext.CLIJ2_clear();

Which automatically brings me to another point: In analysis workflows it often occurs that you want to execute a statement conditionally, e.g. background subtract, filter, etc, based on the user’s input.
For this particular macro, if you want to run it with and without downsampling, besides outcommenting the downsampling you also need to rename the input image in Ext.CLIJ2_median3DBox to image2, because image3 doesn’t exist. Therefore, one cannot easily have a boolean user input downsampling and a statement if(downsampling == true) Ext.CLIJ2_downsample3D(...).

I guess this is a consequence of the (otherwise great) ‘byRef Image’ method. In the past you had made some CLIJ2 commands ‘in place’, but these seem to have been phased out.

Now I was thinking, would it in principle be possible for CL(IJ) to execute commands ‘inplace’ if the destination image variable is set the same as the input image? (e.g. by recognizing that, clearing the input image memory and then renaming the output reference to the input reference.) Or is that inherently very difficult ‘under the CL hood’? It would definitely solve many issues.

Thanks again!
Bram

Ok, then let’s check your hardware :wink:

  • Could you please try to execute the workflow on your Intel HD GPU? I assume your laptop has one:
run("CLIJ2 Macro Extensions", "cl_device=HD");
run("CLIJ2 Macro Extensions", "cl_device=");
Ext.CLIJ_clInfo();

No problem, as images are variables:

Ext.CLIJ2_push(input);

if (user_input) {
	Ext.CLIJ2_gaussianBlur2D(input, output, 1, 1);
} else {
	output = input;
}

Ext.CLIJ2_pull(output);

Some “apparent inplace” methods still exist in CLIJx:

run("CLIJ2 Macro Extensions", "cl_device=");
Ext.CLIJ2_help("inplace");

Having a way for doing what you want in a more systematic may end up on the roadmap for clEsperanto as python users demand it. Eventually code will likely look like this:

image = gaussianBlur(image, 2);

Until they are generally availabe, some details need to be streamlined under the hood. In the meantime, you can implement such methods yourself if you like:

function gaussianBlur(input, sigma) {
	Ext.CLIJ2_gaussianBlur2D(input, output, sigma, sigma);
	return output;	
}

Intransparent “renaming” might confuse users. However, I see where you want to go and can promise that we’re moving in the same direction :wink:

Let me know if the driver thingy solves your GPU issue.

Cheers,
Robert

1 Like

Thanks for all your answers, @haesleinhuepf!
Indeed, you mention nice workarounds that I will certainly employ (and also feel a bit ashamed that I didn’t come up with those myself).

Now about the CLIJ issue.
I gave it a try on my work computer, which has a Quadro M2000, running on Windows 7, with updated NVIDIA drivers.
Same error!
Actually worse. The first time I tried the macro, the video card’s output to the screen stopped completely. I had to go in via remote desktop from another computer to find out that Fiji had crashed. The second time the screen was momentarily flickering before giving me net.haesleinhuepf.clij.clearcl.exceptions.OpenCLException: OpenCL error: -36 -> CL_INVALID_COMMAND_QUEUE and net.haesleinhuepf.clij.clearcl.exceptions.OpenCLException: OpenCL error: -5 -> CL_OUT_OF_RESOURCES. .

Here is the CLInfo output:

Available CL backends:
  * net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL@7dfec0a2
    Functional backend:net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL@262b7ecf
    Best backend:net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL@4b4360bc
Used CL backend: net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL@46518521
ClearCL: ClearCLBase [mClearCLBackendInterface=net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL@46518521, mPeerPointer=null]
  Number of platforms:1
  [0] NVIDIA CUDA
     Number of devices: 1
     Available devices: 
     [0] Quadro M2000 
        NumberOfComputeUnits: 6 
        Clock frequency: 1162 
        Version: 1.2 
        Extensions: cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_d3d9_sharing cl_nv_d3d10_sharing cl_khr_d3d10_sharing cl_nv_d3d11_sharing cl_nv_copy_opts cl_khr_gl_event cl_nv_create_buffer cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_kernel_attribute_nv 
        GlobalMemorySizeInBytes: 4294967296 
        LocalMemorySizeInBytes: 49152 
        MaxMemoryAllocationSizeInBytes: 1073741824 
        MaxWorkGroupSize: 1024 
        Compatible image types: [SignedNormalizedInt8, SignedNormalizedInt16, UnsignedNormalizedInt8, UnsignedNormalizedInt16, SignedInt8, SignedInt16, SignedInt32, UnsignedInt8, UnsignedInt16, UnsignedInt32, HalfFloat, Float]
Best GPU device for images: Quadro M2000
Best largest GPU device: Quadro M2000
Best CPU device: Quadro M2000

Much appreciate your support!
Bram

1 Like

When updated, which version, where downloaded? (-:

Dear @haesleinhuepf
thanks for making this available, great work.
I am already trying it out, and the test code works but then I try to run your


but then get this error:
CLIJ2 error: You are accessing a CLIJ2 plugin (CLIJ2_clear) via CLIJ macro extensions.
Please make sure you run “CLIJ2 Macro Extensions” before calling Ext.CLIJ2 plugins.
java.lang.RuntimeException: Macro canceled
at ij.Macro.abort(Macro.java:76)
at net.haesleinhuepf.clij.macro.CLIJHandler.lambda$handleExtension$0(CLIJHandler.java:156)
at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:97)
at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:28)
at net.haesleinhuepf.clij.macro.CLIJHandler.handleExtension(CLIJHandler.java:53)
at ij.macro.ExtensionDescriptor.dispatch(ExtensionDescriptor.java:255)
at ij.macro.Functions.doExt(Functions.java:4787)
at ij.macro.Functions.getStringFunction(Functions.java:276)
at ij.macro.Interpreter.getStringTerm(Interpreter.java:1407)
at ij.macro.Interpreter.getString(Interpreter.java:1385)
at ij.macro.Interpreter.doStatement(Interpreter.java:329)
at ij.macro.Interpreter.doStatements(Interpreter.java:261)
at ij.macro.Interpreter.run(Interpreter.java:157)
at ij.macro.Interpreter.run(Interpreter.java:91)
at ij.macro.Interpreter.run(Interpreter.java:102)
at ij.plugin.Macro_Runner.runMacro(Macro_Runner.java:161)
at ij.IJ.runMacro(IJ.java:148)
at ij.IJ.runMacro(IJ.java:137)
at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1148)
at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1144)
at net.imagej.legacy.IJ1Helper.runMacroFriendly(IJ1Helper.java:1095)
at net.imagej.legacy.IJ1Helper.runMacro(IJ1Helper.java:1144)
at net.imagej.legacy.plugin.IJ1MacroEngine.eval(IJ1MacroEngine.java:145)
at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

any ideas?

thanks,
Alvaro

1 Like

Hey Alvaro @acrevenna ,

I apologize for the inconencience. I need to update some example scripts apparently.

Please update this line:

run("CLIJ Macro Extensions", "cl_device=");

to this line:

run("CLIJ2 Macro Extensions", "cl_device=");

This should be it.

Thanks for your feedback! I’ll update the example online and with that, you made CLIJ better today :slight_smile:

Cheers,
Robert

Awesome, thanks! That works.

1 Like

Well, I got them from https://www.nvidia.com/download/index.aspx?lang=en-us.
The latest version is 441.66, which is lower than in the link you send, but that was for Windows 10.
I had already updated to this version about two months ago. I now did it again, with the same version. Same result…

1 Like