CLIJ2 BETA testing - call for testers 👷

Dear friends of #clij and #GPU accelerated image processing,

I’m happy to announce that CLIJ2 BETA testing is open to the public. This is a call for testers! If we manage to break clij2 now, it will be more stable after release mid June. Thanks to everyone taking part in advance! :star_struck:

Compared to CLIJ, CLIJ2 offers 224 new operations, some improve older methods, some offer new functionalty, others are convenience methods making your life with pixels on GPUs easier :wink:

image

New concepts

Vectors, matrices, pointlists, meshes and corresponding operations

CLIJ2 has various new operations for processing data structures such as vectors and matrices, for working with graphs and meshes.

Non-square shaped pixels, a.k.a. cells

Furthermore, CLIJ2 offers operations for filtering images with non-square shaped pixels managed using the new graph-based processing operations.

ByRef

CLIJ2 operations use the by-reference concept which is well known in ImageJ macro for images and other parameters. Undefined variables can be passed over and will hold values after the operation finished. This allows simplifying code.

// CLIJ
Ext.CLIJ_centerOfMass(binary);
mx = getResult("MassX", nResults() - 1);
my = getResult("MassY", nResults() - 1);

// --------------------------
// CLIJ2
Ext.CLIJ2_getCenterOfMass(binary, mx, my, _);

Easier debugging

CLIJ2 offers various methods for easier workflow debugging such has handling arrays and strings

// push array as 2x3 image
array = newArray(1, 2, 3, 4, 5, 6);
Ext.CLIJ2_pushArray(input, array, 2, 3, 1);

// do something with it
Ext.CLIJ2_multiplyImageAndScalar(input, result, 3);

// print output image
Ext.CLIJ2_print(result);

image

Time tracing

To find out which operations take how much time, you can collect time traces:

Ext.CLIJ2_startTimeTracing();
Ext.CLIJ2_multiplyImageAndScalar(input, result, 3);
Ext.CLIJ2_stopTimeTracing();

// print out how long operations took
Ext.CLIJ2_getTimeTracing(time_traces);
print(time_traces);

image

Memory management for Java/Jython/Groovy/JavasSript developers

Developers can name images and buffers and list memory consumption

CLIJ2 clij2 = CLIJ2.getInstance();

ClearCLBuffer data = clij2.create(100, 100);
data.setName("My data");

System.out.println(clij2.reportMemory());

Output:

GPU contains 1 images.
- My data 39.0 kB [My data ClearCLBuffer [mClearCLContext=ClearCLContext [device=ClearCLDevice [mClearCLPlatform=ClearCLPlatform [name=NVIDIA CUDA], name=GeForce RTX 2060 SUPER]], mNativeType=Float, mNumberOfChannels=1, mDimensions=[100, 100], getMemAllocMode()=Best, getHostAccessType()=ReadWrite, getKernelAccessType()=ReadWrite, getBackend()=net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL@63e2203c, getPeerPointer()=net.haesleinhuepf.clij.clearcl.ClearCLPeerPointer@1efed156]] 39.0 kB
= 39.0 kB

Also the method clij2.clear(); is new compared to CLIJ, it releases all images/buffers in GPU memory.

Documentation

The new CLIJ2 website offers extensive documentation, including operations which typically follow other operations.

New cheat sheets

The cheat sheets have been updated to list new CLIJ2 operations.

Installation

Update your Fiji with #clij and clij2 update sites activated, search for “Clicy” in the Icy search bar or check the installation instructions for Matlab.

Timeline

CLIJ2 is scheduled to release mid June 2020 - in 4 weeks. With that day CLIJ will deprecate and receive maintenance and support for at least another year. CLIJ users might start having a look at the CLIJ-CLIJ2 transition guide.

If anyone finds a bug or has a question, feel free to drop it here or open a new thread.

Thanks everyone for your support! :heart: :green_heart: :blue_heart:

Happy coding!

Cheers,
Robert

13 Likes

Hi @haesleinhuepf,
First of all congratulations with the BETA release of CLIJ2. It has become a really great and very useful package!

So it is with a little pain in my heart that I have to report the first bug :wink:, following up on our discussion about watershed on the GPU. Although detectMaxima3D has been substantially improved, the watershed operation gives me the following result (original - Fiji watershed - CLIJ2 watershed)

Apparenty you got a different result when you tested it than I get on nuclei_binary.tif (198.6 KB).
Edit: it also messes up a binarized blobs.gif.

Any clue?
Thanks!
Bram

1 Like

Hey @bramvdbroek,

sorry, I guess that last-minute optimization I built in wasn’t well thought-through. My fault. I just updated clij2 on the update site and the result should now look like this:

Furthermore, it supports 3D! The 2D dataset you shared comes from a 3D data set, right? Could you try it in 3D?

Thanks!

Cheers,
Robert

3 Likes

Thanks @haesleinhuepf, for fixing this so fast. Unfortunately, though it indeed works well on that particular image, the issue still remains for other images, e.g. these two:


And yes, it works in 3D. :+1:
I’ve tried several other 3D watershed options, but with these relatively high-res images it never works really well, possibly because at low and/or high z the thresholded part becomes irregular, causing extra watershed lines, while it fails to split other nuclei:


It also seems that the watershed lines are the same for all slices, making it not really 3D. (This is by the way also true for the other 3D watershed methods.) Even MorpholibJ’s Distance Transform Watershed 3D doesn’t do great on these stacks. I usually get better results when segmenting the nuclei in 2D and count substructures in 3D. Although now I have to try StarDist in 3D…
Here’s the file if you want to play yourself: 3D_binary-crop.tif (4.0 MB)

In the mean time I’ll look for a more suitable dataset, with more slices above and below, to test this further.

Best regards,
Bram

1 Like

Hey @bramvdbroek,

that’s unfortunate. I might have a good explanation why it does in 3D what it does in 3D: The objects do have a wobbly structure. If I was a watershed algorithm, I would cut the nuclei in 3D:

Typically, the solution is blurring the image before thresholding. This solution might help here as well.

To explain the problem with the parallel implementation of the watershed a bit more in detail: Assume you have a distance map like this:

0 0 1 1 1
1 1 1 2 1
1 2 1 1 1
1 1 1 0 0

It detects two maxima and cuts the object. We now need an algorithm which starts looking from any pixel and can make a decision on which of the two maxima to keep. This is not trivial. I’m just thinking of cutting off the tips of the hills and then searching for maxima-plateaus. Not sure if this is the solution. Any idea is welcome!

Cheers,
Robert

1 Like

Hi @haesleinhuepf,

Yes, that’s what I tried to say with ‘at low and/or high z the thresholded part becomes irregular’. :slight_smile: The reason for that is that in z the boundary of the nucleus is less sharp (in theory) / much less sharp (in practice) than in x and y. (standard confocal with cells on a coverslip - far from istropic)
Smoothing will certainly help, but since there are not many slices above and below in this case, the whole thing becomes almost 2D anyway.
I’ll find a dataset where nuclei are clumped also in the z-direction. I foresee that also quite a lot of xy smoothing is needed, which will have a negative influence on the watershed results in planes where the nuclei are in focus. So we need a differential filter that apdapts smoothing based on the intensity and/or sharpness. A 3D bilateral filter, or Gaussian weighted median filter perhaps?

You probably need to add a ‘radius’ parameter then (which is fine), or do you know a smart automatic way of estimating it?
I guess would be more or less similar to internally smoothing the distance map before detecting the maxima, like I did here.

Bram

Hello @haesleinhuepf
Thank you so much for all the updates and congratulations on this relaease! Are there any updates regarding the implementation of filters such as the Tubeness one or any other ones that rely on hessian eigenvalues?

Best wishes,
Rita

1 Like

Hey Rita,

bad news, I couldn’t figure out how to translate the tubeness measurement to CLIJ. Regarding the Hessian: A collaborators project has these filters. I’ll ask them if they are willing to share. Stay tuned!

Cheers,
Robert

1 Like