CLIJ equivalent of 3D attribute filtering

Hi
I have been using CLIJ to process 3D stacks, and utilising the GPU has significantly sped up the image processing.

As part of my workflow, I also use 3D grayscale attribute filtering from MorpholibJ. Is there a CLIJ equivalent for this??

@haesleinhuepf

Thanks
Pradeep

2 Likes

Hi @pr4deepr,

Hope you are doing well. Yes, I also tend to use CLIJ filters instead of my legacy (not so) Fast Filters 3D. For morphology filtering, simply use Maximum filter as dilation and Minimum filter as erosion. @haesleinhuepf has implemented box and sphere neighbouring, I tend to prefer sphere neighbouring.

Best,

Thomas

THanks for that @ThomasBoudier. I am well, thanks for asking. We just entered another lockdown in Victoria. Hope you are well too.

Yes, I have ben using clij equivalent filters for morphology filtering and they work quite well. I still do use few plugins from 3D ImageJ suite, and the 3D ROI Manager is extremely handy!

With the question I asked, I am after a CLIJ equivalent of this.:
image
which is from the MorphoLibJ package…
Which filters do I use to apply a similar operation?

Hi @pr4deepr,

I am not sure, but opening is : minimum then maximum. Then (?) a labelling is applied to keep only connected components with at least the specified number of voxels. In CLIJ labelling is Labelling/Connected Components.

For details, we should ask @dlegland, he gave a webinar some weeks ago, you can have a look here.

Best,

Thomas

1 Like

Hey @pr4deepr, hey @ThomasBoudier, CC-hey @dlegland,

not sure if I understand “Opening” with an attribute correctly. According to the documentation:

Attribute filters aim at removing components of an image based on a certain size criterion, rather than on intensity.

“attribute filtering” following this goal exists in CLIJ, more flexible and hence tiny little bit more complicated. Core function is excludeLabels which takes a label map and a binary vector: E.g. 0, 0, 1, 0, 0, 1 says: “Keep background and labels 1, 3 and 4. Drop labels 2 and 5.”

Thus, if you want to remove labels from a label map according to their volume, you need to have a vector with their volume and threshold it.

This also allows you to exclude labels according to anything, for example to their “distance to the moon” as explained in the webinar. The notebook about superpixel segmentation shows an example in more detail:
https://clij.github.io/clij2-docs/md/superpixel_segmentation/

----->

If you are looking for applying Attribute filtering to greyscale images, you may have to implement the workflow suggested on the MorpholibJ website yourself. If you do so, I’m happy to help and I’d be happy to put this as a new plugin in the successor of CLIJ2. I must admit I never felt the need for such an operation:

When applied to a grayscale image, attribute opening consists in generating a series of binary images by thresholding at each distinct gray level in the image. The binary attribute opening described above is then applied independently to each binary image and the grayscale output is computed as the union of the binary results. The final output is a grayscale image whose bright structures with the attribute below a given value have disappeared.

Also: Can you share details in what context you do this?

In both 3D ImageJ Suite and in CLIJ this filter can be found as TopHat explained a bit more in detail here. Edit: That’s wrong as @dlegland nicely explains below

I feel honored to read that :slight_smile: Some legacy filters stilll need to be translated though. E.g. I’m biting out my teeth these days on parallelized implementations of Find Maxima and Seeded Watershed.

The Sphere neighboring filters are usually more suited to process biologial images because we don’t have rectangular/box shaped biological structures. I often use Box neighborhood though because it’s faster because of its separable implementation. It’s a bit crazy, but I tend to push all processing workflows below 500ms to achieve real-time user-experience.

Let me know if you need anything else!

Cheers,
Robert

1 Like

I use this on grayscale images where I want to exclude bright objects below a certain voxel size without necessarily converting it to binary. In my case, it helps get rid of relatively smaller objects which have high autofluorescence as well.

Could you try TopHat and tell us if it’s performing similarily well?

1 Like

@haesleinhuepf
To phrase it very simply, attribute filters differ from your standard morphological operation in that they don’t have a fixed structuring element. Also, you can have many different types of attributes (area, diameter etc). and you can generalize many different morphological operations (openings, closings, top-hat) to use such attributes rather than a fixed structuring element (filter kernel). While I don’t have a current example I have used area top-hat filters quite a lot in the past and have not been able to reproduce similar results with a regular top-hat filter.

3 Likes

  1. Original image
  2. Attribute filtering with 3000 voxels gives the second image
  3. Top-hat with 17,17,10 voxels. Specifying the voxels is more convenient than each specifying each dimension…

The brightness is quite low on the smaller objects in image 2 which helps with downstream operations, such as segmentation…

Example of otsu Thresholding:

1 Like

Hi there!

many questions, I will try to answer the ones I can!

opening: yes, this is the result of an erosion followed by a dilation. They can be obtained by minimum and maximum filters respectively.

attribute opening: the idea is to remove objects based on a size criterion rather than on result of erosion or dilation. For binary images, this can be obtained by:

  • applying connected components labeling
  • computing the “size” (usually area for 2D and volume for 3D, but this can be generalized to otherfeatures)
  • keep only the labels with the size >threshold
  • make it binary again.

Exactly!

The grayscale attribute opening can be obtained by repeating the process for each gray level of the image after binarization, and combining the binary results to generate a new grayscale image. More efficient algorithms exist, I do not remember how it is implemented in MorphoLibJ. I think it can be implemented using CLIJ based on the above-mentioned workflow, but this would requires some “encapsulation code”…

Top-hat are combination of original image with the result of an opening. Attribute top-hat works exactly the same but with attribute opening!

Best,
David

1 Like

Thanks for that @dlegland. When you say for each grey level, would you “threshold” for each individual grey level (if its a 16-bit image, then 65535 gray levels) and get a corresponding binary image?

1 Like

yes… I think you can avoid processing all gray levels by first checking which ones exist in the image, and processing only these ones.

2 Likes

Thanks for this.
I was trying using stack histogram in IJ macro language and couldn’t find it.
CLIJ has the option for a stack histogram and it works, but what binning value do I use, especially if I want to test each grey level? 65535 is a bit much… :slight_smile:
Perhaps my approach is wrong. Do I use this approach from an older ImageJ post to get a stack histogram using IJ macro?

@haesleinhuepf, definitely interested in getting this as a plugin in CLIJ. Apologies for the late response with this…

1 Like

Hi @pr4deepr,

I just realized that this attribute filtering may be related to iterative thresholding . This iterative thresholding will test all thresholds and extract objects with size in a defined range. Maybe worth a try.

Best,

Thomas

2 Likes

Hey @ThomasBoudier,

your link isn’t functional. Do you mean this one?
https://imagejdocu.tudor.lu/plugin/segmentation/3d_spots_segmentation/start#d_iterative_thresholding

Hey @pr4deepr,

Would you mind trying it in a macro and checking how fast it is? If it’s not to bad, I can convert it to Java and make a plugin out of it afterwards. You may want to use

It might look challenging on the first view, but I also know that you have talent and proofed earlier that you can deal with it :wink:

Let me know if you need support :slight_smile:

Cheers,
Robert

2 Likes

Hi @haesleinhuepf,

Thanks for spotting the error in the link, I corrected it.

To give more details about this iterative thresholding, it is based on a classical segmentation algorithm known as MSER. It is quite slow, especially for 16-bit images, since you do a labeling for all thresholds, and then you link the objects obtained. But I must admit I usually obtain quite good results, if the volume is properly set up.

Best,

Thomas

2 Likes

Thanks for the reply @ThomasBoudier and advice/encouragement @haesleinhuepf.
I do use iterative thresholding a fair bit and really like it. As you said, it works well if the volume or size is reasonably accurate…

With the code, I gave it a go. The logic I used based on convo with @dlegland and @haesleinhuepf :

  • Generate a histogram using CLIJ (256 bins for now). With scripting, using a Stackstatistics class will give quick access to all the grey-levels which exist, but I wanted to see if this works for now.
  • Get bins where grey levels exist
  • Threshold for each grey level
  • Connected components labeling
  • get stats for labelled and background pixels to get pixel counts using labelled objects and corresponding pixels in original image
  • Use PIXEL_COUNT column as a vector to exclude objects above, say 10000 pixels
  • Create a binary image using this label
  • Do this for each bin/grey level
  • Combine all the binary images; I’ve added them all together to generate a new grayscale image. Not sure if this is the way to go

The speed is noticeable when using higher size thresholds.
I don’t think it is an honest comparison as I am not testing all the grey-levels in CLIJ.

If I use image stack: 299x300x66, 16-bit, 11 MB
and criteria for a min pixel count of 1000
CLIJ workflow: 25.6 s
MorphoLibJ: 21 sec

min pixel count: 10000
CLIJ workflow: 24 sec
MorphoLibJ: 174 sec

I don’t think it is working quite well yet because the pixel intensity values are saturated in the resulting image compared to MorphoLibJ results. I had a look at the code in grayscale 3D attrib filtering in MorphoLibJ. Do I set all pixel values in each label to the min or mean intensity of the corresponding pixel values in the original image??

run("Clear Results");
run("CLIJ2 Macro Extensions", "cl_device=");
Ext.CLIJ_clear();

image1 = getTitle();
//push image
Ext.CLIJ2_push(image1);
// get minimum and maximum intensity in the image stack
Ext.CLIJ2_minimumOfAllPixels(image1);
Ext.CLIJ2_maximumOfAllPixels(image1);

//getting the corresponding min and max values from the results table generated above
min=getResult("Min", 0);
max=getResultString("Max", 1);
print(min,max);
run("Clear Results");

//generate histogram using CLIJ, with min,max values and bins of 256
//using this instead of all grey values for now as a test
image2 = "histogram_image";
number_of_bins = 256;
minimum_grey_value = min;
maximum_grey_value = max;
determine_min_and_max = false;
Ext.CLIJ2_histogram(image1, image2, number_of_bins, minimum_grey_value, maximum_grey_value, determine_min_and_max);
//get histogram image
Ext.CLIJ2_pull(image2);
selectWindow(image2);


//generate an array with histogram bins that have positive counts
setOption("ExpandableArrays", true);
//store pixel counts for each bin in an array
stack_histogram=newArray();
//store nonzero gray_levels in an array
grey_level_nonzero=newArray();
//bin width for histogram
bin_width=(maximum_grey_value)/(number_of_bins);
bin=0;
idx=0;
for(i=0;i<number_of_bins;i++)
{
	
	stack_histogram[i]=getValue(i,0);
	if(getValue(i,0)>0 && i>0)
	{
		grey_level_nonzero[idx]=bin;
		idx+=1;
	}
	bin+=bin_width;
}

//print("Histogram: ");
//Array.print(stack_histogram);
//print("Non zero bins ");
Array.print(grey_level_nonzero);
//print(grey_level_nonzero.length);


label="label";
thr="thresholded";
// time measurements
time = getTime();

//trying to simulate the 3D grayscale attribute filter in CLIJ
label2="label2";
label3="label3";
label4="label4";
temp_label="temp";
min_size=1000;
for(grey=0;grey<grey_level_nonzero.length;grey++)
{
	print("Grey level "+grey_level_nonzero[grey]);
	//threshold for each grey level
	Ext.CLIJ2_threshold(image1, thr, grey_level_nonzero[grey]);
	//generate connected components labeling
	Ext.CLIJ2_connectedComponentsLabelingBox(thr, label);
	//get all stats of the background and labelled pixels from which we can get pixel count, which is ~size of object of itnerest
	Ext.CLIJ2_statisticsOfBackgroundAndLabelledPixels(image1, label);
	//push the pixel_count column to an image
	Ext.CLIJ2_resultsTableColumnToImage(pixel_count, "PIXEL_COUNT");
	//use pixel_count image as a vector to filter for labels within a predefined size range
	Ext.CLIJ2_excludeLabelsWithValuesOutOfRange(pixel_count, label, label2, min_size, 1000000000);
	//convert label to binary
	Ext.CLIJ2_threshold(label2, label2, 1);

	Ext.CLIJ2_resultsTableColumnToImage(min_intensity, "MIN_INTENSITY");
	//Ensures that the binary image generated for each grey level and size range is used in the next loop
	//if loop executs first time (grey=0) stores binary image (label2) in temp_label, does not perform addition
	if(grey>0) 
	{ 
		//using add images to combine the binary images generated in each loop
		//not sure if this is what dlegland meant by combining all binaries to generate a new grayscale image
		Ext.CLIJ2_addImages(temp_label, label2, label3);
		Ext.CLIJ2_copy(label3, temp_label);

	}
	else 
	{
		Ext.CLIJ2_copy(label2, temp_label);
	}
	run("Clear Results");
	
}
	
Ext.CLIJ2_pull(label3);
Ext.CLIJ2_pull(temp_label);
print("CLIJ workflow took " + (getTime() - time) + " msec");

Ext.CLIJ_clear();

//MorphoLibJ gray scale attrib filtering 3D
selectWindow(image1);
time = getTime();
run("Gray Scale Attribute Filtering 3D", "operation=Opening attribute=Volume min=&min_size connectivity=6");
print("MorphoLibJ gray scaleattrib 3D workflow took " + (getTime() - time) + " msec");

Thanks heaps!

1 Like

Hey Pradeep @pr4deepr,

fan-tas-tic! That’s a great starting point for a Java-guy like me :star_struck:

By updating Fiji, you find a new plugin in CLIJx: CLIJx_greyLevelAtttributeFiltering. Let me know if it should have a better name. I think we could make it more specific - or the selection of the properties in the plugin more generic…

Anyway, this macro applied to blobs.gif:

// init GPU
run("CLIJ2 Macro Extensions", "cl_device=");
Ext.CLIJ2_clear();

// open example image and push it to the GPU
run("Blobs (25K)");
input = getTitle();
Ext.CLIJ_push(input);	

// define test parameters
min_sizes = newArray(10, 100, 1000);

for (i = 0; i < min_sizes.length; i++) {
	// grey level atttribute filtering
	number_of_bins = 256.0;
	minimum_pixel_count = min_sizes[i];
	Ext.CLIJx_greyLevelAtttributeFiltering(input, result, number_of_bins, minimum_pixel_count);
	Ext.CLIJ_pull(result);
	rename("Filtered with size > " + minimum_pixel_count);
}

image image image

Does that look reasonable? Would you mind testing a bit mor in detail? We also need to fill out the documentation still.

Your Java-ized code lives here and the only major difference is that I multiply the binary images in every loop with the threshold and combine them by a maximum-images-operation.

Regarding the speed: I think depending on the actual use-case one can make it pretty fast by reducing the number of bins…

Let me know what you think. Feel free to send a PR and let’s see that we get this going. It’s a pleasure working with you!

Cheers,
Robert

3 Likes

Awesome!
Thanks so much.

Is the multiplication because the output from the CLIJ binarising is 1 for foreground values?
Also, curious about the reasoning for using a maximum-images operation?

BTW, I have been writing some code using Groovy recently (@ThomasBoudier volunteered his time to teach some basics a while ago). Not an expert but I’m really enjoying it. Is this better than the IJ macro code for developing your plugins?

I will test this out on a few images and send PRs on github!

Pleasure working with you too!!

Cheers
Pradeep

1 Like

I think macro is simpler because of recorder and auto-completion. And as soon as we have clesperanto running, this won’t matter anymore :wink:

Addendum:

It’s becaus of the thresholding. You threshold objects above value 45. If you put them in your final image, they should have value 45 again :wink:

An alternative to multiiplying with threshold and maximum-images would be multiplying with bin-width and adding. But this will only work if all bins have pixels… That’s why I wrote it like that.

1 Like