Imagej ops stats package - ignore NaNs

Hi everyone,

I was wondering whether there was a way to use the imagej ops stats package to find the min/max/mean/median value of an image whilst ignoring the NaNs?

Thanks,
Alessandro

2 Likes

Sorry for the delay in reply, @alessandrofelder.

It seems to work already. Here is an example:

#@ OpService ops
#@ ConvertService convertService

import ij.IJ

// Create a 32-bit image with NaNs.
imp = IJ.openImage("http://imagej.nih.gov/ij/images/blobs.gif");
IJ.run(imp, "32-bit", "");
IJ.setAutoThreshold(imp, "Default");
IJ.run(imp, "NaN Background", "");

imp.show()
IJ.run(imp, "Histogram", "bins=256 use x_min=128 x_max=248 y_max=Auto");

image = convertService.convert(imp, net.imagej.Dataset.class)
minMax = ops.stats().minMax(image)
println(minMax.getA())
println(minMax.getB())

That prints 128.0 for the min, and 248.0 for the max, while ignoring all the NaNs.

Are you seeing different or erroneous behavior?

Edit: Oh, I guess mean is failing due to NaNs if added to my example above. If you compile a list of stats ops that fail due to NaNs, we can work on fixing them!

4 Likes

Thanks for the help, @ctrueden!

Yes, mean was the problem for me.
Just figured out that median has the same problem.

Inside a test class that extend AbstractOpTest:

@Test
    public void testMedianExample(){
        final IterableInterval<? extends RealType> toyImg = getToyImg();
        float median = ops.stats().median(toyImg).getRealFloat();
        assertEquals("Median wrong", 3.0, median, 1e-12);
    }

    private IterableInterval<? extends RealType> getToyImg()
    {
        final ArrayImg<FloatType, FloatArray> floats = ArrayImgs.floats(3, 3, 3);
        floats.forEach(f -> f.setReal(Float.NaN));

        final ArrayRandomAccess<FloatType> access = floats.randomAccess();
        access.setPosition(new int[]{1,1,1});
        access.get().setReal(1.0);

        access.setPosition(new int[]{0,1,1});
        access.get().setReal(3.0);

        access.setPosition(new int[]{2,1,1});
        access.get().setReal(3.5);

        return floats;
    }

this assertion fails:


Will look if I find other stats ops now.

So, I think most of the stats ops that take an IterableInterval as an argument do not ignore NaN values. I have tested:

  • mean
  • median
  • stdDev
  • skewness
  • sum
  • sum of logs
  • sum of inverses
  • sum of squares
  • geometric mean
  • harmonic mean
  • variance
  • quantile
  • percentile
  • moments 1,2,3,4 around mean
  • kurtosis

I also think expectations will vary on what the stats op should do, right? There may be some cases where you want the result to be NaN to warn you that something has gone wrong? Maybe we just want to just have the option to explicitly exclude NaNs in our stats ops calculations?

2 Likes