[CLIJ2] Non local means CPU vs GPU results

Hi,

I wanted to test the NLM filter and compare the results between CPU and GPU (on CLIJX)

The results look similar - not the same and the original one seems to do a better job.

However what is puzzling me is why the image that shows on CLIJ2 seems to be so different,
but when it’s saved, it seems to be “okay”. Is this a bug or something?

original image
Composito.tif (843.7 KB)

original image + NLM on CPU
Composito_NLM.tif (843.7 KB)

original image + NLM on GPU
non_local_means-1226553674.tif (843.6 KB)

on my screen (GPU + CLIJ2)

Hi @Haimon_Alves ,

thanks for reaching out. First of all, CLIJx is the experimental sibling of CLIJ2 and thus, it is possible that there is a glitch in the nlm filter.

Can you share some details what operation you called with which parameter? If you can record a macro that applies the two filters you were comparing, that would be perfect. I’m asking for this, because I can’t find non-local means CPU version in my Fiji.

Thanks!

Best,
Robert

Just to be clear, I believe there might be two issues:
1 - the difference in the final results on the image
2 - the difference on what I see after using the NLM on GPU
and what it shows after I save then open the image

You can find the MACRO below:

// non local means using CPU
// you can find it on UPDATE > BIOMEDGROUP
// it will appear in plugins by itself

run(“Non-local Means Denoising”, “sigma=15 smoothing_factor=1”);

run(“CLIJ2 Macro Extensions”, “cl_device=[NVIDIA GeForce GTX 1080 Ti]”);
// non local means using GPU (CLIJX)
image1 = “Composito.tif”;
Ext.CLIJ2_push(image1);
image2 = “non_local_means1659583458”;
radius_x = 2.0;
radius_y = 2.0;
radius_z = 2.0;
sigma = 15.0;
Ext.CLIJx_nonLocalMeans(image1, image2, radius_x, radius_y, radius_z, sigma);
Ext.CLIJ2_pull(image2);

Can you create and share a screenshot of the image to show how it looks like before saving?

It’s on the original message, below “on my screen (GPU + CLIJ2)”.

Can you see it? I’ll just add again, tbs

I assume there is an issue in the clij code. In particular in the math. When I wrote it (I actually took code from some other place and adapted it), I was not 100% sure about the sigma in this line:

Furthermore, I checked the code of the CPU implementation, and see how they do it differently:

Obviously, in my equation, the math appears wrong.

So I know what to try and how to potentially fix it. But it will take 1-2 weeks until I find time for this. If anyone wants to go ahead, change the math in the cl file, test it and potentially provide a pull-request, that would be very welcome!

:slight_smile:

1 Like

And would that be the same reason why it shows the figure after
the NLM is finished as something else
that seems to be visually different than the same image that I saved then opened again?

Yes, now I see it.
The difference looks like (croped image size):

@haesleinhuepf :roll_eyes: :thinking:

See my comment about the math above :slight_smile:

Robert have you realized that there seems to be a difference between the image created by clij processing before and after saving it?
(@Haimon_Alves : Did I describe this correctly?)

I can’t reproduce this. When running this macro, the mean squared error is 0:

run("CLIJ2 Macro Extensions", "cl_device=");
Ext.CLIJ2_clear();

open("C:/Users/Rober/Downloads/Composito.tif");

// push to GPU
image1 = getTitle();
Ext.CLIJ2_push(image1);

// process image
radius_x = 2.0;
radius_y = 2.0;
radius_z = 2.0;
sigma = 15.0;
Ext.CLIJx_nonLocalMeans(image1, image2, radius_x, radius_y, radius_z, sigma);
Ext.CLIJ2_pull(image2);

// save image
saveAs("tif", "C:/Users/Rober/Downloads/temp.tif");

// load again
open("C:/Users/Rober/Downloads/temp.tif");
image3 = getTitle();

// push again
Ext.CLIJ2_push(image3);

// determine mean squared error
Ext.CLIJ2_getMeanSquaredError(image2, image3, mse);

print("MSE: ", mse);

// clean up
Ext.CLIJ2_clear();

image

Maybe I’m comparing the images in a different way?

1 Like

I can confirm your finding
=> There is no difference between the saved NML image and the NLM image directly after creating it with clij.

The screen shot you have posted is in fact different from the saved image.
But the reason for this in unclear.
Can you reproduce this effect and can you descibe how?

@phaub @haesleinhuepf

The 1st image (below) shows the original image, the NLM-image using “CPU plugin” version"
and the “GPU plugin” version. All of those images were capture as seen on my screen

The 2nd image (below) shows the NLM-image with CPU (left) and GPU (right)
as seen on my screen after saving and opening them on Windows - not using Fiji yet

The 3rd image (below) shows the NLM-image with CPU (left) and GPU (right)
as seen on my screen after saving and opening them on Fiji

I’m not sure why this happens. I run the NLM plugin, both of them, without doing anything else.
My Fiji seems to be updated. It might be a bug. Does it not happen with you?

I’m not sure if I understand. Images 2 and 3 in the first row look the same as images 1 and 2 in the third row, right? The only difference is they are not saved (1st row) and saved+reloaded (3 row). I see no issue here.

I do not know why these images are shown like this. When working with scientific images, I recommend using Fiji for displaying them and no software such as Microsoft Paint or the Windows Photo viewer. Fiji is made for scientific image analysis, the others are not.

@haesleinhuepf

I’m not sure if I understand. Images 2 and 3 in the first row look the same as images 1 and 2 in the third row, right? The only difference is they are not saved (1st row) and saved+reloaded (3 row). I see no issue here.

Yes, image 2 and 3 in the 1st row are exactly the same as image 1 and 2 in the 3rd row.

I just wanted to show you how they were presented as soon as the NLM is finished in the 1st row
without doing anything else. They were not (yet) saved at the moment.

I do not know why these images are shown like this. When working with scientific images, I recommend using Fiji for displaying them and no software such as Microsoft Paint or the Windows Photo viewer. Fiji is made for scientific image analysis, the others are not.

Although I do agree with you, the differences look very strange. I have never seen
such a HUGE difference between images that were open using Fiji or using the
basic softwares that comes with Windows. I did see some differences in other occasions
in the past, but this one specifically seemed way to different to ignore.

Another thing that caught my attention is that the image shown by Windows, albeit different
from the NLM CPU one, seems to be more similar than the one shown by Fiji just after using
the NLM GPU plugin.

So I’m not sure why this happens either, and if I’m the only one. I did delete Fiji entirely
and began from scratch, but it seems this issue doesn’t go away. Since this happened
with the GPU NLM, I figured that it might be some issue with the plugin, but it is strange
that after saving the image, it seems to change that too in some way that doesn’t happen
with the other images.

Is it possible that it might be some issue with the image itself? Because I just tried using
the NLM CPU vs GPU in two other images, and this bizarre effect did not happen. The
Windows and Fiji images seemed pretty similar.

Would you mind telling us what software you are referring to? That might make it easier to reproduce your issue.

I think this is just a visual effect caused by the Min/Max display values as can be seen in the Brightness/Contrast dialog of both images.

Use the ‘Reset’ button to change the Min/Max values to 0/255.
If you want this values to ‘stay’ in the image, saved the image again.
Or just set Min/Max to 0/255 before you save the image the first time.

The effect is created by an automatic contrast update after creating the image on the GPU. The Min/Max are stored in the image as 59/183.
When you open the image in Fiji this Min/Max values are used to display the image with an increased contrast.
Other software do not use this Min/Max information and therefore display the images always with Min/Max = 0/255.

So there is no problem or bug. Everything is ok. Sorry for not thinking about this i the first place.

1 Like

I can reproduce the visualization issue with Windows Explorer. Obviously the Explorer doesn’t take brightness/contrast settings from the meta data into account.

Running this script

run("Blobs (25K)");
saveAs("Tiff", "C:/structure/temp/blobs/blobs-1.tif");

run("Duplicate...", " ");
setMinAndMax(0, 127); // change min/max visualized grey value -> brightness / contrast
saveAs("Tiff", "C:/structure/temp/blobs/blobs-2.tif");

run("Duplicate...", " ");
setMinAndMax(128, 255);
saveAs("Tiff", "C:/structure/temp/blobs/blobs-3.tif");

Leads to these images on screen:

And these images in the Windows Explorer:

You may need to switch the Explorer to the “Large icons” view to see it.

Reopening the images in Fiji shows them correctly:

One could argue this is a bug in Windows Explorer or a bug in ImageJ. Without looking into the code of the Windows Explorer, it will be hard to fix.

Hi Robert,
the main point that causes the confusion is that after

run("Non-local Means Denoising", "sigma=15 smoothing_factor=1");

the settings of the display contrast are not changed automatically (Min/Max = 0/255),
while after

run("CLIJ2 Macro Extensions", "cl_device=[NVIDIA GeForce GTX 1080 Ti]");
// non local means using GPU (CLIJX)
image1 = "Composito.tif";
Ext.CLIJ2_push(image1);
image2 = "non_local_means1659583458";
radius_x = 2.0;
radius_y = 2.0;
radius_z = 2.0;
sigma = 15.0;
Ext.CLIJx_nonLocalMeans(image1, image2, radius_x, radius_y, radius_z, sigma);
Ext.CLIJ2_pull(image2);

the display contrast settings are automatically optimized in the function show_internal() by IJ.run(imp, "Enhance Contrast", "saturated=0.35"); .

1 Like

Ah, I see. I quess that’s necessary because otherwise some filters result in black images. E.g.:

run("Blobs (25K)");
run("CLIJ2 Macro Extensions", "cl_device=[GeForce RTX 2080 Ti]");

// logarithm
image1 = "blobs.gif";
Ext.CLIJ2_push(image1);
image2 = "logarithm-1751522073";
Ext.CLIJ2_logarithm(image1, image2);
Ext.CLIJ2_pull(image2);

setMinAndMax(0, 255);

image

So I would not change it unless someone has a good idea how to deal with this issue in a general fashion.