ImageJ2 Histogram op is too simplistic an implementation for a coloured image?

As a first minor step in developing something useful for our project I tried getting the histogram from one of our images using the ImageJ2 operations. The histogram this operation creates is far too simplistic. This is my sample code:

Histogram1d<?> result = (Histogram1d<?>) ij.op().run(Ops.Image.Histogram.class, dataset.getImgPlus());

Here’s the image I run this code on, link to the 150MB image: https://bitbucket.org/sunsear/ihcsemiquantifier/src/master/src/test/resources/CD163%206.3p.tif

What the above code seems to do for each pixel is: increment the bin for the R value, then also for the G value and lastly for the B value. The result is a really useless flattened histogram that looks like this:

For display purposes I removed the white bin (255).

It has all kinds of holes in it that are really not supposed to be there. Doing a histogram plot for this image using ImageJ2 does a better job as it actually calculates the average of the 3 color intensities for each pixel.

I am a rookie at image manipulation so my question at this point is:

  • is this a bug, or is this actually a useful operation?

Background

Image is an IHC stained slice of a heart valve leaflet

Analysis goals

Getting the combined histogram from this color image

@Rody and @mountain_man, is this possibly also the cause for the discussion you had about a week ago about a grayscaled image? I haven’t checked the image from that discussion yet, but if it has RGB channels, the histogram will look stupid…

Indeed, ImageJ2 (and therefore ImageJ Ops) treats your RGB image as a 3-dimensional image with the dimensions (X,Y,Channel). If you compute the histogram for this image, it will sort the values from all three channels into the same 256 bins.

To reproduce that same behavior with ImageJ1 commands, you can run Image › Color › Make Composite before running Analyze › Histogram, and you’ll get the same histogram with “gaps”:

image


On the other hand, computing the histogram directly on the RGB image in ImageJ1 will show you unweighted intensity values by default:

image

You can click the RGB button multiple times until you get the R + G + B version that corresponds to the above histogram from the composite:

image

If you want to compute the unweighted intensities from the RGB values in ImageJ2, you’ll have to make a Sum projection on the Channel dimension first.

The following Groovy scripts illustrates this (note that I display the 1d histogram as a 2d image to have a simple visualization as a line image):

#@ Dataset input
#@ OpService ops

sumOp = ops.op("stats.sum", input)

projected = ops.create().img([input.dimension(0), input.dimension(1)] as long[])
ops.transform().project(projected, input, sumOp, 2)

hist = ops.image().histogram(projected)
ops.transform().addDimensionView(hist, 0, 0)
2 Likes

Ok, thanks Jan, that confirms the way I expected it to work. As i was already browsing the code for the histogram plot and the histogram op, i can bring both operations together. I can imagine the operation doing the sum by default as the plot does. Then the plot to allow “cycling” through them in the way that the 1 version does.

As I said I don’t have too much experience on image processing. To me, having the sum or average functionality as the default sounds more useful in the plot. Then when you click on that, we can cycle through it to go to R, G, B and the plus variation. Having the sum in the operation also sounds more useful.

Is that going to be helpful?

@ctrueden, we are trying to get exactly the step that Jan mentions here into Knime. The problem there is that the Groovy scripting unit is buggy/incomplete/so poorly documented that it is useless. We had decided to do this bit of code in a Java Snippet therefore. Doing this in Java however is incredibly complicated as ImageJ2 depends a lot on generics.

I got as far as this, but the code simply will not compile:

UnaryComputerOp<Dataset, Img<RealType<DoubleType>>> summed = (UnaryComputerOp<Dataset, Img<RealType<DoubleType>>>) ij.op().op(Ops.Stats.Sum.class, dataset);

Img<DoubleType> img = ij.op().create().img(new long[]{dataset.getImgPlus().getImg().dimension(0), dataset.getImgPlus().getImg().dimension(1)});

ij.op().transform().project(img, dataset, summed, 2);

Histogram1d<UnsignedByteType> result = (Histogram1d<UnsignedByteType>) ij.op().run(net.imagej.ops.Ops.Image.Histogram.class, summed);

The problem is with the type of the returned op versus the created image. Are we doing something wrong in this list of steps?

Look forward to hearing from you or anyone else who can shed some light on how to do a grayscale in Java code with ImageJ2 & ImgLib2.

2 Likes

We managed to create this step in Knime itself using math steps, not using the ImageJ manipulation. Would still be very interesting to be able to do this in ImageJ2

1 Like

Here is a working version:

import java.io.IOException;

import net.imagej.Dataset;
import net.imagej.ImageJ;
import net.imagej.ops.Ops;
import net.imagej.ops.special.computer.Computers;
import net.imagej.ops.special.computer.UnaryComputerOp;
import net.imglib2.Dimensions;
import net.imglib2.FinalDimensions;
import net.imglib2.histogram.Histogram1d;
import net.imglib2.img.Img;
import net.imglib2.type.NativeType;
import net.imglib2.type.numeric.RealType;
import net.imglib2.type.numeric.real.DoubleType;

public class RGBImageHistogram {

	public static <T extends RealType<T> & NativeType<T>> void main(final String... args) throws IOException {
		// Create the ImageJ gateway.
		ImageJ ij = new ImageJ();
		
		// Load an RGB image.
		Dataset dataset = ij.scifio().datasetIO().open("/Users/curtis/data/clown.jpg");

		// NB: Reconstitute the Dataset object's recursive generic parameter.
		//
		// The Dataset class tries to protect the user from difficult generics.
		// However, because ImgLib2's core interfaces use recursive type parameters
		// of the form "T extends Type<T>", using wildcard types results in an
		// incompatible expression: "? extends Type<?>", where the compiler does not
		// know that the two ?s must in fact be the same type.
		//
		// Because the ImageJ Ops and ImgLib2 libraries use the most specific and
		// correct types for arguments, including generics, we need to go from
		// Dataset, which is an "Img<RealType<?>>" and therefore not a matching
		// type for any of our image processing methods, to Img<T> where
		// "T extends RealType<T> & NativeType<T>". We can do it using an unchecked
		// cast, but only from a method with a T type variable declared, as above.
		@SuppressWarnings("unchecked")
		Img<T> img = (Img<T>) dataset;

		// Find a stats.sum computer op which sums into a DoubleType result.
		// While we could instead use the input type for the result e.g. via
		// "outType = img.firstElement()", this would fail spectacularly for
		// common types like UnsignedByteType, because the sum would quickly
		// overflow, whereas DoubleType is a wide-enough type to accommodate
		// not-super-large images of both integer and floating point types.
		DoubleType outType = new DoubleType();
		UnaryComputerOp<Iterable<T>, DoubleType> sumOp = Computers.unary(ij.op(), Ops.Stats.Sum.class, outType, img);

		// Project the image's third dimension using the sum op into a
		// destination DoubleType image. This assumes our input image is 3D.
		Dimensions dims = new FinalDimensions(img.dimension(0), img.dimension(1));
		Img<DoubleType> projected = ij.op().create().img(dims, outType);
		ij.op().transform().project(projected, img, sumOp, 2);

		// Compute the histogram on the projected image.
		Histogram1d<DoubleType> histogram = ij.op().image().histogram(projected);

		// Output the histogram.
		System.out.println(histogram);
		System.out.println(histogram.distributionCount());
		System.out.println(histogram.valueCount());
		for (int b=0; b<histogram.getBinCount(); b++) {
			System.out.printf("[%d] %d\n", b, histogram.frequency(b));
		}

		// Clean up.
		ij.context().dispose();
	}

}

Note also that Java generics are almost always an optional layer on top. If it gets too annoying to figure out how to do them correctly in a given scenario, you can just use raw types instead. That’s one reason the Groovy scripts end up looking quite elegant: they just ignore all the generics.

Here’s a version with raw types:

DoubleType outType = new DoubleType();
UnaryComputerOp sumOp = Computers.unary(ij.op(), Ops.Stats.Sum.class, outType, dataset);
Dimensions dims = new FinalDimensions(dataset.dimension(0), dataset.dimension(1));
Img projected = ij.op().create().img(dims, outType);
ij.op().transform().project(projected, dataset, sumOp, 2);
Histogram1d histogram = ij.op().image().histogram(projected);

It’s true that this ecosystem is complex and can be frustrating. But we’re here for you as you continue the journey. :sun_with_face:

3 Likes

I figured something like this would be up with the generics. Have built a few libraries myself and adding generics is usually only helpful if you keep it simple. The T extends Type<T> variant has given me extreme headaches in my own libs in the past. I find that sometimes I rather do things a little less precise in order to make them more usable. Exactly how the groovy code is going about it.

Anyway, thanks a lot for the help! Will definitely work this into the tryout plugin I have in development at the moment.

1 Like