Performance issues on large images with IJ2 compared to IJ1

[imagejan (Jan Eglinger)]

@mdoube

so far IJ2 seems a bit slow compared to IJ1, and my use case is for large images (> 150 GB 8-bit, so 600 GB 32-bit int ) so performance issues are real.

Can you elaborate what’s the slowness of IJ2? Pixel manipulation? Display? Opening/Saving?
I bet @tpietzsch and @ctrueden might want to know.

The ImgLib2 library (which is the part of IJ2 that actually implements the data types) is aiming at fast performance, and I would be surprised if an ImgLib2 benchmark of e.g. UnsignedIntType or UnsignedLongType would be slower than comparable image processing libraries (in particular in combination with large data and the functionality in ImgLib2 to handle volatile types etc.).

[mdoube (Michael Doube)]

@imagejan

Can you elaborate what’s the slowness of IJ2?

One example is our porting of Connectivity from IJ1 using byte[] to IJ2 using BooleanType with a concomitant increase in time to complete 1 GB from ~19s to ~42s. This may be due to limitations in the multithreading model used on the IJ2 version, with 8 threads, whereas the performance on the byte[] may be limited by the memory wall kicking in at about 16 threads on my machine. (IIRC the RandomAccessibleInterval performance was really slow reading octant neighbourhoods and the 8-thread model ameliorates it to some degree, and no-one was able to advise a faster way to do it).

[tpietzsch (Tobias Pietzsch)]

@mdoube

One example is our porting of Connectivity from IJ1 using byte[] to IJ2 using BooleanType with a concomitant increase in time to complete 1 GB from ~19s to ~42s.

Is there a discussion about this that I can read somewhere? Or a benchmark example?
Looking at the code, I’m surprised that it should be so much slower. Do you remember which BooleanType you used? One implementation of BooleanType is BitType which represents booleans as bits packed into a long. Maybe decreased performance was caused by some overhead there.

(I realize that this has nothing to do with the issue at hand, and making the IJ2 version of this one example faster will not solve the general problem. I’m just curious…)

[mdoube (Michael Doube)]

@tpietzsch

Is there a discussion about this that I can read somewhere?

Yes - check https://gitter.im/imglib/imglib2 starting Apr 24 2019 09:33 (HKT timezone I think)

edited by Jan Eglinger:

Here’s a direct link:

imglib/imglib2

@tpietzsch

Or a benchmark example?

Both ImgLib2 and IJ1 style Connectivity implementations are included in BoneJ. So just install BoneJ and have a go on a favourite binary image stack. Anyway - this gets a bit off topic for this forum thread which is about a potential IJ1 IntProcessor. So maybe we should take it elsewhere?

[imagejan (Jan Eglinger)]

@mdoube

So maybe we should take it elsewhere?

I just moved the concerned posts into a new topic. Hope that’s alright (also the title).