Is there a CellImg-like class but with cells consisting of RandomAccessibleInterval?

imglib2
imagej

#1

Instead of providing a LazyCellImg with Cell via a custom getter, I’d like to provide it with a Cell<RandomAccessibleInterval> of the correct dimensions for the cell.

I understand that I could create my own by reifying a ByteAccess that reads from an underlying RandomAccess of a RandomAccessible. I am looking here for ready-made solutions.

Thanks.


#2

Anybody with experience with LazyCellImg? I’ve failed miserably at getting it to work (all be said, some critical methods and inner public interfaces like Get lack documentation).

For this example, I take the example bat cochlea volume and use a LazyCellImg to create a montage of its slices, in a 12x10 pattern:

from ij import IJ, ImagePlus
from net.imglib2.img.cell import LazyCellImg, CellGrid, Cell
from net.imglib2.img.display.imagej import ImageJFunctions as IL

imp = IJ.openImage("https://imagej.nih.gov/ij/images/bat-cochlea-volume.zip")
img = IL.wrap(imp) # Creates PlanarImg instance with pointers to imp's slices

class SliceGet(LazyCellImg.Get):
  def __init__(self, imp, grid):
    self.imp = imp
    self.grid = grid
    self.cell_dimensions = [self.imp.getWidth(), self.imp.getHeight()]
  def get(self, index):
    # ImageStack ranges from 1 to N slices inclusive
    if index < 1 or index > self.imp.getStack().size():
      # Return blank image: a ByteAccess that always returns 255
      return Cell(self.cell_dimensions,
                  [0, 0],
                  type('ConstantValue', (ByteAccess,), {'getValue': lambda self, index: 255})())
    else:
      return Cell(self.cell_dimensions,
                  [0, 0],
                  ByteArray(self.imp.getStack().getProcessor(index).getPixels()))

n_cols = 12
n_rows = 10
cell_width = imp.getWidth()
cell_height = imp.getHeight()

grid = CellGrid([n_cols * cell_width, n_rows * cell_height],
                [cell_width, cell_height])

montage = LazyCellImg(grid, img.cursor().next().createVariable(), SliceGet(imp, grid))

IL.show(montage, "Montage")

Unfortunately the code above runs for a long time, so long I never let it finish. Interestingly, adding a print statement at the SliceGet.get method, we see that the index is called with values from 0 to 1331, which is incomprehensible, as we expect values from 0 to 119 (12x10 Cell instances). And furthermore, those values are called repeatedly, twice for every number and the same list [0…1331] over and over, presumably once per each Cell (but I never saw it finish, takes too long.)

Could someone with good knowledge of LazyCellImg please explain how the interface LazyCellImg.Get is supposed to work then? A close inspection to the CellCursor suggests that it gets the indices of the LazyCellImg.LazyCells inner class, which is a ListImg containing one Cell per index in the list. But that is not the case: the index values are anything but.

Any suggestions, @tpietzsch @hanslovsky @ctrueden @axtimwalde ?


#3

A little bot off topic, but you may find this interesting. I am afraid it may be outdated but has carried me over the bar.


#4

@albertcardona I’m not sure where the 1331 comes from. In your Get implementation, you always create Cells with min = [0,0]. This is a bug, it should be the minimum of the cell in global coordinates, i.e., cell 0 has min [0,0], cell 1 has min [1 * cell_dimensions[0], 0] etc. Maybe this throws of cursor/RA of the CellImg so that they don’t know when to change cells etc, leading to the observed 1331.

Also, I would expect this to be slow. LazyCellImg calls Get every time an accessor enters a cell. Showing the image probably scans it in flat array order, so you will call Get imp.getHeight() times for every cell. The Get implementation should do some kind of caching. If your’s is a standard scenario, maybe use something more downstream (CachedCellImg). Maybe have a look at ReadOnlyCachedCellImgFactory and https://github.com/imglib/imglib2-cache-examples/blob/f713fff36add8a4e459056a9c0e78c049b4f4e33/src/main/java/net/imglib2/cache/example03/Example03.java#L33-L62

Regarding the original question: Cell<RandomAccessibleInterval> There is NO such thing, at least not without copying. CellImgs make the assumption that data in the cells is in a single primitive array, in flat order, just like ArrayImg. So one cannot wrap an arbitrary RandomAccessibleInterval as a Cell. To do something similar, there will maybe be a View to arrange a list of RAIs in a grid (this https://github.com/imglib/imglib2/pull/179, waiting for feedback from @dietzc, @MarcelWiedenmann on that…). But that will be slower than CellImgs, and there is no lazy/cached variant (but could be).


#5

Hi Tobias,

Thanks for this, it is surprising (and I think undocumented: https://github.com/imglib/imglib2/blob/master/src/main/java/net/imglib2/img/cell/Cell.java#L65) that a Cell should be created with a min coordinate reflecting it’s position in space in the enclosing image coordinates, rather than at origin. It seems like an internal detail that leaks into a public API.

I’ve fixed the min coordinates of the cell in the script and this resolved the issue. Hurrah!

Thanks, I’ve memoized the return values of the get function.

Here is the new, working script version in full to turn an ImageJ stack (ImageStack) into a montage using imglib2:

from ij import IJ, ImagePlus
from net.imglib2.img.cell import LazyCellImg, CellGrid, Cell
from net.imglib2.img.basictypeaccess.array import ByteArray
from net.imglib2.img.basictypeaccess import ByteAccess
from net.imglib2.img.display.imagej import ImageJFunctions as IL
from java.lang import System

imp = IJ.getImage()
img = IL.wrap(imp) # Creates PlanarImg instance with pointers to imp's slices

class SliceGet(LazyCellImg.Get):
  def __init__(self, imp, grid):
    self.imp = imp
    self.grid = grid
    self.cell_dimensions = [self.imp.getWidth(), self.imp.getHeight()]
    self.cache = {}
  def get(self, index):
    cell = self.cache.get(index, None)
    if not cell:
      cell = self.makeCell(index)
      self.cache[index] = cell
    return cell
  def makeCell(self, index):
    n_cols = self.grid.imgDimension(0) / self.grid.cellDimension(0)
    x0 = (index % n_cols) * self.grid.cellDimension(0)
    y0 = (index / n_cols) * self.grid.cellDimension(1)
    index += 1 # 1-based slice indices in ij.ImageStack
    if index < 1 or index > self.imp.getStack().size():
      # Return blank image: a ByteAccess that always returns 255
      return Cell(self.cell_dimensions,
                  [x0, y0],
                  type('ConstantValue', (ByteAccess,), {'getValue': lambda self, index: 255})())
    else:
      return Cell(self.cell_dimensions,
                  [x0, y0],
                  ByteArray(self.imp.getStack().getProcessor(index).getPixels()))

n_cols = 12
n_rows = 10
cell_width = imp.getWidth()
cell_height = imp.getHeight()

grid = CellGrid([n_cols * cell_width, n_rows * cell_height],
                [cell_width, cell_height])

montage = LazyCellImg(grid, img.cursor().next().createVariable(), SliceGet(imp, grid))

IL.show(montage, "Montage")

#6

Here is another working version of the stack slice montage ImgLib2 script that can add padding between the slices in the montage. Works great, and fast enough despite using a proxy with a randomAccess on a RandomAccessibleInterval view of the stack slice (a Cursor for the proxy could work too, I’ve tested it, but would only be able to run once).

What’s nice beyond the montaging with padding between panels is that it illustrates how to use ByteAccess to produce whatever pixel data on the fly as a function of the index (position within the image), a special case of which is what I did here: to use it as a proxy into a RandomAccessibleInterval, that is, any image you want. In summary, any image can be used as source data for a Cell in a CellImg, and it is not difficult to do.

I’ve also simplified and generalized the computation of positions from the index by using the util IntervalIndexer.indexToPosition. In principle, this code could work for laying out a 4D volume into a 3D montage, but I haven’t tested it.

from ij import IJ, ImagePlus
from net.imglib2.img.cell import LazyCellImg, CellGrid, Cell
from net.imglib2.img.display.imagej import ImageJFunctions as IL
from net.imglib2.view import Views
from net.imglib2.img.basictypeaccess import ByteAccess
from jarray import zeros
from net.imglib2.util.IntervalIndexer import indexToPosition

imp = IJ.openImage("https://imagej.nih.gov/ij/images/bat-cochlea-volume.zip")
#imp = IJ.getImage()

class ProxyByteAccess(ByteAccess):
  def __init__(self, rai, grid):
    self.rai = rai
    self.ra = rai.randomAccess()
    self.d = zeros(2, 'i')
    self.dimensions = zeros(rai.numDimensions(), 'l')
    rai.dimensions(self.dimensions)
    self.position = zeros(rai.numDimensions(), 'l')
  def getValue(self, index):
    indexToPosition(index, self.dimensions, self.position)
    self.ra.setPosition(self.position)
    return self.ra.get().getByte()
  def setValue(self, index, value):
    pass

class ConstantValue(ByteAccess):
  def __init__(self, value):
    self.value = value
  def getValue(self, index):
    return self.value
  def setValue(self, index, value):
    pass

class MontageSlice(LazyCellImg.Get):
  def __init__(self, imp, cell_padding, padding_color_value, grid):
    self.stack = imp.getStack()
    self.grid = grid
    self.gridDimensions = grid.getGridDimensions()
    self.cell_padding = cell_padding
    self.t = IL.wrap(imp).randomAccess().get().createVariable()
    self.t.setReal(padding_color_value)
    self.cache = {}
    self.cell_dimensions = [self.grid.cellDimension(d) for d in xrange(self.grid.numDimensions())]
    self.position = zeros(len(self.gridDimensions), 'l')

  def get(self, index):
    s = self.cache.get(index, None)
    if not s:
      s = self.create(index)
      self.cache[index] = s
    return s

  def create(self, index):
    indexToPosition(index, self.gridDimensions, self.position)
    c = [p * self.grid.cellDimension(d) for d, p in enumerate(self.position)]
    index += 1 # 1-based slice indices in ij.ImageStack
    if index < 1 or index > self.stack.size():
      # Return blank image: a ByteAccess that always returns 255
      return Cell(self.cell_dimensions,
                  c,
                  ConstantValue(255))
    else:
      # ImageJ stack slice indices are 1-based
      img = IL.wrap(ImagePlus("", self.stack.getProcessor(index)))
      # Create extended image with the padding color value
      imgE = Views.extendValue(img, self.t.copy())
      # A view that includes the padding between slices
      minC = [-self.cell_padding for d in xrange(img.numDimensions())]
      maxC = [img.dimension(d) -1 + self.cell_padding for d in xrange(img.numDimensions())]
      imgP = Views.interval(imgE, minC, maxC)
      return Cell(self.cell_dimensions,
                  c,
                  ProxyByteAccess(imgP, self.grid))


n_cols = 10
n_rows = 12
cell_padding = 5 # pixels, effective cell dimensions are 5+width+5, 5+height+5

cell_width = imp.getWidth() + cell_padding * 2
cell_height = imp.getHeight() + cell_padding * 2

grid = CellGrid([n_cols * cell_width, n_rows * cell_height],
                [cell_width, cell_height])

# Padding color: 255 is white for 8-bit images
getter = MontageSlice(imp, cell_padding, 255, grid)

montage = LazyCellImg(grid, getter.t, getter)

IL.wrap(montage, "Montage").show()

#7

Thanks a lot for sharing, @albertcardona!

In my opinion, this is more complex that it should be, for such a “simple” task as to create a mosaic montage.

I tried doing the same using ImageJ Ops and the mosaic() method from NotebookService (which was intended for displaying image montages in Jupyter notebooks), achieving the same with a few less lines (the script is in Groovy because I’m more familiar with it, but should be easily translated into Python as well):

#@ Img input
#@ Integer (value=10) columns
#@ Integer (value=12) rows
#@ Integer (value=5) padding 
#@ NotebookService ns
#@ OpService op

#@output result

import net.imglib2.type.numeric.integer.UnsignedByteType
import net.imglib2.RandomAccessibleInterval

paddingMin = -padding
paddingMaxX = input.dimension(0) + padding
paddingMaxY = input.dimension(1) + padding

slice = 0
sliceListProvider = {
	op.run("intervalView",
	       op.run("extendValueView",
	              op.run("hyperSliceView", input, 2, slice++),
	              new UnsignedByteType(255)
	             ),
	       [paddingMin,  paddingMin ] as long[],
	       [paddingMaxX, paddingMaxY] as long[]
	      )
}


imageList = []
input.dimension(2).times { imageList << sliceListProvider() }

result = ns.mosaic([columns, rows] as int[], imageList as RandomAccessibleInterval[])

I don’t know how this compares in terms of speed. Ideally, it should be as simple as running ops.run("stacks.tile", input, sliceDim, [columns, rows], padding, value) or some such… if there was an op for that.


#8

Thank you for providing an ops example. My goal is far more complex than the minimal example that I showed. All I wanted to illustrate is the use of RandomAccessibleInterval instances as data for a Cell in a CellImg, and I accomplished that.

On the way, I also showed how to use ByteAccess to generate data for an image. While above I used a constant value, one could return anything, as a function of the index. For example, here I draw sinus curves (mountains, in 2D):

from net.imglib2.img.display.imagej import ImageJFunctions as IL
from net.imglib2.img.array import ArrayImg
from net.imglib2.img.basictypeaccess import ByteAccess
from net.imglib2.type.numeric.integer import UnsignedByteType
from math import sin, cos, pi

class Sinusoid(ByteAccess):
  def __init__(self, width, span):
    self.width = width
    self.span = span
  def getValue(self, index):
    # Obtain 2D coordinates and map into the span range, normalized into the [0, 1] range
    # (which will be used as the linearized 2*pi length of the circumference)
    x = ((index % self.width) % self.span) / self.span
    y = ((index / self.width) % self.span) / self.span
    # Start at 2, with each sin or cos adding from -1 to +1 each, then normalize and expand into 8-bit range
    return int((2 + sin(x  * 2 * pi) + cos(y * 2 * pi)) / 4.0 * 255)
  def setValue(self, index, value):
    pass

t = UnsignedByteType()
dimensions = [512, 512]
data = Sinusoid(dimensions[0], dimensions[0] / 10.0)

img = ArrayImg(data, dimensions, t.getEntitiesPerPixel())
img.setLinkedType(t.getNativeTypeFactory().createLinkedType(img))

IL.wrap(img, "sinusoid").show()

While I am sure there may be an ops for that, my point is to expose the internals of ImgLib2, their power and flexibility, not that I’d actually want to e.g. montage stack slices: there’s an ImageJ function for that.