Simple CLIJ macro not returning image with same dimensions

Hi Roooobert

I am trying to simply do a gaussian blur with CLIJ (2D + 2Ch + time) and modified your simple macro, but the output returns only a single 2 color image… I tried with bioformats importer and using the simple "open(path), and both won’t work.
any idea what I am doing wrong?

Also, any small CLIJ macro example for batch processing? did not find on the clij github page… just to have an idea how to handle the GPU pushing/pulling in batch

run("Close All");

// open dataset
open("D:/SCIENCE/Microscopy/0_Airy/DO15_endoTRF1 live_siNUMA/20190422_cl7_NuMA_MAX/MAXProj/20190422_CL7_siCTRL_Pos02_MAX.tif");
getDimensions(width, height, channels, slices, frames);

time = getTime();

// init GPU
run("CLIJ Macro Extensions", "cl_device=");
Ext.CLIJ_clear();

// send input image to the GPU
input = "input";
rename(input);
Ext.CLIJ_push(input);



// denoising
denoised = "denoised";
Ext.CLIJ_blur2D(input, denoised, 1, 1);




// pull the resulting image from the GPU and show it
Ext.CLIJ_pull(denoised);

1 Like

Hi Debora,

Robert will have a more accurate answer but here’s a few hints. I don’t think that the Clij functions are designed to handle multi-dimensional stacks. My impression is that they can operate on 2D images or 3D stacks. At best there are functions to selectively push parts from hyperstacks e.g. CLIJ_pushCurrentZStack. I think one workaround for you is to simply turn your hyperstack into a large z-stack (mixed channels), use the 3D blurring with z-blurring set to 0, and re-transform your stack to its original shape. Here’s an example using the mitosis.tiff sample:

//Grab a multiD image and get its dimensions
run("Mitosis (26MB, 5D stack)");
getDimensions(width, height, channels, slices, frames);
totslices = channels*slices*frames

//transform the hyperstack into one large z.stack
run("Hyperstack to Stack");
run("Properties...", "channels=1 slices=" +totslices+ " frames=1")


// init GPU
run("CLIJ Macro Extensions", "cl_device=");
Ext.CLIJ_clear();

// send input image to the GPU
input = "input";
rename(input);
Ext.CLIJ_push(input);

// blur using the 3D function AND using sigmaz = 0
denoised = "denoised";
Ext.CLIJ_blur3D(input, denoised, 5, 5,0);

// pull the resulting image from the GPU and turn it back into a hyperstack
Ext.CLIJ_pull(denoised);
run("Stack to Hyperstack...", "order=xyczt(default) channels="+channels+" slices="+slices+" frames="+frames+" display=Color");
Ext.CLIJ_pushCurrentZStack(String image);

I hope it helps.
Cheers,
Guillaume

2 Likes

Hi @Debora_KellerOlivier,

as @guiwitz introduced absolutely correctly (thanks btw!), CLIJ does not support multi-channel image data as such. There are two major reasons:

  • Many operations such as blurring, thresholding, connected components analysis don’t make much sense over channels. E.g. with blurring over channels you introduce cross-talk :ghost: Furthermore, when applying thresholding and connected components analysis over time you can get horrible false-segmentations. We discussed that earlier here.
  • Secondly, some GPUs are limited regarding memory. Thus, you may want to process images frame by frame and/or stack by stack in a loop. I actually suspect you want that, too :wink:

Let me point you to some examples which might be of your interest:

Batch processing

In the CLIJ preprint we discuss a batch-processing macro. You find it here. It’s a quite complex workflow processing stacks and counting spots.

I just wrote a simple batch-processing macro and put it here. You will see that batch-processing in CLIJ is basically the same as in ImageJ:

// go through the folder
for (i = 0; i < lengthOf(files); i++) {
	open(path_to_images + files[i]);
	rename(input);

	// push images to GPU
	Ext.CLIJ_push(input);
	
	// ...
	
	// Get results back from GPU
	Ext.CLIJ_pull(result);

	// save result
	saveAs("tif", path_to_results + files[i]);
	close();
}

Processing slices of stacks individually

For your workflow it may make sense that you push the whole stack (as @guiwitz showed) using the Hyperstack to Stack method and CLIJ_push. Then, you can copy individual slices using CLIJ_copySlice, process them as 2D images and afterwards copy them back into the stack. After you processed all images, you can pull the whole stack back and save it to disc. You could even run Stack to Hyperstack before saving to get it back in the right dimensionality. I recently programmed an example macro doing something similar for a 3-channel 2D image of my mentor.

// iterate over channels
for (c = 0; c < 3; c++) {
	Ext.CLIJ_copySlice("catRGB", "cat", c);
	// ...
	
	// apply transform
	Ext.CLIJ_applyVectorField2D("cat", "rotatedShiftX", "rotatedShiftY", "transformed");

	// put resulting 2D image in the right plane
	Ext.CLIJ_copySlice("transformed", "resultStack", i * 3 + c);
}

Last but not least a kind reminder from our workshop in Porto: Running a single operation such as blur on the GPU may not be faster than on the CPU because of the time we spend for pushing and pulling image data.

May I ask what kind of workflow you are planning to apply to your images?

Cheers,
Robert

hiya,

just reading all of this now (was buried in python!!)
the images are 900Mb from an Airy scan fast and the gaussian blur of 1 is the first step of my icy protocol.
I use channel H2B to segment and track the cells using active contour.
I use channel GFP to detect and count spots within cells

Ideally, I would like to see if I can do the preprocessing of the raw files (ie3D + time) that are huuuge in CLIJ: it would be simple max proj and the gaussian blur.
would htat be a CLIJ application?
Cheers

Debbi
PS: I have the p4000 card and 64gb RAM

Hey @Debora_KellerOlivier,

that sounds like a good project for CLIJ. If you are in experimental mood, you can try CLIJ within ICY - CLICY. In case you want to give it a try, I’d be happy to help you implementing your workflow there. All I would need would be some example data set and a protocol for analysing it in ICY. Feel free to share that in private :wink:

Cheers,
Robert

sent you an image

and the icy protocol I am using - via filesender

1 Like

Hey @Debora_KellerOlivier,

thanks for the files. I wasn’t able to open/run your protocol because of an error (I may have plugin missing…). But I think I can nevertheless give you the code snippets you need in order to process your images. There are two options for processing your data and I wasn’t sure which will finally work better/faster.

Before you start, please install clicy-0.2.0 to your ICY as described on its website. It’s basically just downloading 12 .jar files and putting them in a folder.

a) Process your sequence frame by frame

I wrote an example script for you which processes the currently open sequence in Icy frame by frame. Above you mentioned you want to do a maximum-projection, but the images you sent me didn’t contain z-slices. Thus, this example only does some blurring. Again: Executing a single operation on the GPU may not be faster than on the CPU because of the time we lose while pushing/pulling. But give it a try, on a powerful workstation, it may still be faster:


As you can see, it uses clijx.method() to call operations on the GPU (For others reading with us: clijx is the experimental successor of clij). Thus, feel free to also look around in the (yet incomplete) API reference of clijx where you can find more clix operations.

b) Process your sequence channel by channel

I also wrote an example ICY protocol which processes the first channel of your sequence as a stack. It also demonstrates how to do a maxium-projection - over time. Not sure if this makes a lot of sense :wink:

I’m new to Icy, so I struggled processing all channels with the same javascript. But I bet you know how to do this in a protocol. If not, would you mind asking your local Icy experts and let us know here how to do this?

Let me know if these scripts/protocols fit your needs. While developing them for you, I extended CLICY a bit and even fixed some bugs. So I really appreciate your efforts and feedback on trying/testing it and I would love to get feedback :wink:

Thanks!

Cheers,
Robert

Update: If you install clicy-0.3.0 you can try building protocols with CLIJx blocks. This should give you more flexibility.

2 Likes