Suggestions on needed functions for CLIJx-assistant

Hi Robert @haesleinhuepf

Congrats on the development of CLIJx-assistant. It is a completely game changing plugin.

I want to let you know about three functions that I miss a lot on it, hoping you’ll find time and motivation to include them in future updates (I apologize if some already exists and I don’t know).

(sorting in decreasing order of personal need)

  1. the possibility to start from two or more images.
  2. the function CLIJx_visualizeOutlinesOnOriginal();
  3. the possibility to copy one time point or one channel (or simply, split them).

I’ll explain:

  1. Consider a practical example. Imagine I use stardist or ilastik for nuclei segmentation and them I wan to use the nuc mask to access info in a different channel. I can’t start clijx-assistant with both images open… The workaround is to merge both as a z-stack and split later inside clijx-assistant, but would be nice a direct way (that would also result in a cleaner clijjx-assistant generated script).
  2. CLIJx_visualizeOutlinesOnOriginal() is a great and fast way to check if a segmentation meets our standards. Inside clij assistant you don’t have that choice so I need to generate a ROI and restore it on the original image and do it again for any parameter change. CLIJx_visualizeOutlinesOnOriginal() would result in a faster way given the auto update of the assistant and most importantly it works for 3D.
  3. I think this one is self-explanatory. Of course I can use Re-order Hyperstack at beginning but it would be easier to have a function to do that.

Best
Eduardo

1 Like

Hey @econdesousa ,

thanks for the great feedback. Let’s see if I understood it right, because I think the clij-assistant has already everything you need.

You can start the assistant on two images and then combine them:

The visualizeOutlinesOnOriginal operations was made for the assistant. When using the there (category Math), it also makes use of the lookup table “Hi” which is a sibling of “HiLo”.

Does the "Duplicate image and go ahead with ImageJ solve this issue?

If you see ways for making those operations more user-friendly or accessible, please let me know!

Thanks again for the feedback!

Cheers,
Robert

Hi @haesleinhuepf

thanks for your fats reply

I think I’m not as proficient in the assistant as I thought I was… :grimacing:
I’m sorry!!! I never realized that pressing the clij button twice opens two images!!! Somehow I always assumed that once you start the assistant it was on, and you wouldn’t be able to press the button again…

Thanks… There are many commands / menus in the assistant and I had not found it.

Here, I’m afraid it won’t solve the problem better than the solution of re-order hyperstack. My idea would be a function equivalent to copySlice but for copyChannel or copyTimePoint…
Nevertheless this problem has a fast workaround… the first one was the most important and it was just me being unaware of something basic of CLIJx-assistant

Thanks
Eduardo

Yeah, I know. I also wasn’t sure in which category to put it. Check out the Fiji search in case you’re lost :wink:

I’m not sure I understand why you mention re-order hyperstack in this context. If you want to share a fully-described workflow / use-case, that would be welcome. Just one more word: the CLIJ-assistant doesn’t support time-lapses because they typically don’t fit in GPU memory anyway. That’s why CLIJ is rather focused on processing time point by time point. I’m also explaining that more in detail here:

Let me know if you have ideas for improving the user interface :slight_smile:

Cheers,
Robert

That makes sense!!!

We can get back to my previous example: Consider a 2-channel image: channel 1with DAPI and channel 2 with some cell marker.
If you want to use the nuclei as seeds to watershed the cells (using marker controlled watershed)
you need to split the channels. Now I know that I can start CLIJ assistant with both images separated from the beginning, but if you need to start with both channels merged as one multi-channel image you have to work with the multichannel until the end. The alternative that I used before your first message was to start with the multi-channel converted to a zstack image with run(“Re-order Hyperstack …”, “channels=[Slices (z)] slices=[Channels (c)] frames=[Frames (t)]”); and then inside CLIJ assistant I copied the slides. See bellow a dummy workflow:

best
Eduardo

1 Like

There is actually an “extract channel” plugin in the assistant:

However, as clij doesn’t know what channels are, the code generator drops it and replaces it with a “copy” step. Together with @Christian_Tischer , we’re discussing on github how to improve on this. This is tricky and ideas how to improve on this are very welcome.

1 Like

Hi @haesleinhuepf

Again:

The assistant has lot more functions than those I knew / know…

By following the “Suggested next steps” there is the extract channel: Great!!!

But I couldn’t found it on the search bar.
Other commands do appear… for example, the keyword “copy”

but “Extract cannel” does not.
Searching by keyword “ext” there are a lot of commands but no extract channel

and typing “extra” no more options available

Thanks
Eduardo

1 Like

Alright, I’ll fix that :slight_smile:

Regarding the channels and time-points issue: I just had a look at skimage and as far as I can see there is no support for a channel and time point logic (or am I missing something)? The same, afaik, is tru for MorpholibJ.

I know it may be very convenient for the users if there is channel and time-lapse support in-build, but once has to consider the overhead (an potential mess) that creates on the library development. My gut feeling thus would be to limit the “core CLIJ API/UI” to single channel single time point data and build channel and time-lapse logic on top of that core functionality at places where it is essential. I feel much of the time point logic (apart from tracking) could probably be captured by a generic batch analysis functionality.

But, I am not sure, one could also consider writing a library that natively works on, e.g. a 5D (XYZCT) image model.

1 Like

Me neither, because: OpenCL has built-in support for 2D and 3D images. Not for 4D and 5D. Furthermore, I hardly ever needed 5D filters or operations…

The question is how to do it in practice if people enter CLIJ with a multi-channel image and want to do, for example, a threshold. Imho, it is unlikely that the current behavior of applying the same threshold to all channels is a prevalent use case. You thus could for all operations ask the user on which channel(s) to apply the operation; that is in practice, most/all UI element would get an additional element for the channel selection.

The next question is what the output would be if, e.g., the user processes only channel #2 of an image with three input channels. One could imagine to (a) only replace channel #2 with the processed version and leave the other channels as they were and return again a three channel image or (b) return an image with only 1 channel, namely the processed one. I feel at least for operations like binarization or distance maps, where the pixel types changes (not only in terms of data type but also in terms of the biophysical interpretation), the latter option may be better. If it is however a gaussian blur, maybe some users would like to keep the channels together in one image (because, e.g. the other channels may have looked smooth enough from the start).

1 Like