Implementation plan for Imglib2-rois 2D

omero
imagej
imglib
roi

#41

Hello all,

I just wanted to share some of the major decisions regarding ROIs which were made at the hackathon:

  • Separate “shapes” and RealRandomAccessibles
  • A RealMask can be the result of unary/binary operations on RealMasks (ex: And)
  • RealMasks can be affine transformed (AffineRealMask)
  • Options for whether or not a RealMask should or should not contain all points on the boundary (ex: Box)
  • Should have implementations for discrete space equivalents of RealMasks when applicable (ex: rectangles)
  • Should create Rasterizable interface for RealMasks which know how to efficiently rasterize themselves, also have additional options for raster algorithms

@tpietzsch, @dietzc, @axtimwalde, @ctrueden please feel free to add anything I missed, or correct me if needed.

Regards,

Alison


#42

Just a quick follow-up on the ROIs project: the other day, @awalter17 and I generalized RealMask to be simply Mask<L extends RealLocalizable>. So now you can have Mask<RealLocalizable> for continuous/real masks, and Mask<Localizable> for discrete/integer ones. This paves the way for discrete ROIs going forward, even if we don’t implement them all now. It also allowed us to generalize all the mask tree nodes (And, Or, Xor, etc.) to not care about whether the mask is real or integer, so it is now equally possible to generate unary/binary operation trees on either kind of mask. Work progresses on the shape-rois branch—be warned that that branch is undergoing heavy rebasing still, though.


Can I get to measurements using ImageJ Ops?
Efficient Sparse Labeling of Large Image with ImgLib2
#43

@ctrueden @awalter17:
Do you have any design idea (or even example implementation) on how the ROIs will integrate with the ImgLabeling and Region w.r.t. sampling information from (possibly unrelated) Imgs?


#44

Hello @stelfrich!

I haven’t done a lot with respect to sampling, but I do have adapters which can wrap Masks as RandomAccessibles (ex:
DefaultMaskAsRandomAccessible).

I’ve been mostly focused on creating additional real space ROIs. So in order to use them to sample from an Img you’d have to wrap them was a RealRandomAccessibleRealInterval using the adapter, then Views.raster(...), then Views.interval(...), then Regions.iterable(...), and then (finally) Regions.sample(...). The framework supports creating discrete space ROIs (Mask< Localizable >) which would be easier to use for sampling.

Cheers,

Alison


#45

@tpietzsch Thank you very much for all your work on this at the hackathon! :smiley_cat:

I was looking over your changes, and I noticed that in your scheme it is not possible to retrieve the operands used to create a MaskPredicate which resulted from an or, and, etc. nor is it possible to tell which operation was performed. For example, if someone unions a rectangle and a circle there’s no way to retrieve the original circle and rectangle from the resulting MaskPredicate or figure out that the new MaskPredicate resulted from a union.

I need this functionality for my conversions in imagej-omero, though I did forget to add unit tests for this functionality on my imglib2-roi shape-rois branch :cry:. In imagej-omero if I’m given a RealMaskRealInterval to upload to OMERO, I need to check if this RealMaskRealInterval resulted from an operation between multiple MaskPredicates and if so convert the original MaskPredicates to their OMERO equivalent and then recombine them in the same way as before (but using OMERO data structures).

For example, if someone tries to upload a RealMaskRealInterval which is actually four Boxes union-ed together, then I need to be able to “unwrap” the RealMaskRealInterval into the four original Boxes and I need to know that these Boxes were union-ed.

Do you have any thoughts on how to achieve this? My thoughts are:

  1. Add four new classes (RealMaskOperationResult, RealMaskRealIntervalOperationResult, etc.), and these would have methods for retrieving operands and operation types

– OR –

  1. Add operands() and operationType() methods to MaskPredicate. Then the question is what do these methods return for things like Box, Sphere, etc
    • operands() could return null, Collections.emptyList(), Collections.singletonList(this), or …
    • operationType() could return null, an Enum, or …

Please let me know your thoughts on this. And once again thank you very much for all the work you did at the hackathon, I think the changes are a big improvement!

Cheers,

Alison


#46

@awalter17 This is already there (your option 1.)

Have a look at CompositeMaskPredicate, UnaryCompositeMaskPredicate, and BinaryCompositeMaskPredicate.

I added a demo: AnalyzeCompositeDemo.java

For

RealMaskRealInterval composite = s1.and(s2.minus(s3)).and(s3).or(s1.minus(s3.negate()));

it prints

leaf  (net.imglib2.troi.geom.real.ClosedSphere@7cd84586)
OR  (net.imglib2.troi.composite.DefaultBinaryCompositeRealMaskRealInterval@30dae81)
 +--AND  (net.imglib2.troi.composite.DefaultBinaryCompositeRealMaskRealInterval@1b2c6ec2)
 |   +--AND  (net.imglib2.troi.composite.DefaultBinaryCompositeRealMaskRealInterval@4edde6e5)
 |   |   +--leaf  (net.imglib2.troi.geom.real.ClosedSphere@7cd84586)
 |   |   +--MINUS  (net.imglib2.troi.composite.DefaultBinaryCompositeRealMaskRealInterval@70177ecd)
 |   |       +--leaf  (net.imglib2.troi.geom.real.ClosedSphere@1e80bfe8)
 |   |       +--leaf  (net.imglib2.troi.geom.real.ClosedSphere@66a29884)
 |   +--leaf  (net.imglib2.troi.geom.real.ClosedSphere@66a29884)
 +--MINUS  (net.imglib2.troi.composite.DefaultBinaryCompositeRealMaskRealInterval@4769b07b)
     +--leaf  (net.imglib2.troi.geom.real.ClosedSphere@7cd84586)
     +--NEGATE  (net.imglib2.troi.composite.DefaultUnaryCompositeRealMask@cc34f4d)
         +--leaf  (net.imglib2.troi.geom.real.ClosedSphere@66a29884)

The reason you didn’t see this is probably because CompositeMaskPredicate is never returned as a type. I was almost going to do that before https://github.com/imglib/imglib2-roi/commit/029ff8e79d40e1d55bf713433342cb349a58f0fb
MaskInterval.and could return BinaryCompositeMaskInterval, etc. However, for the leafs (and therefore for the operands of a composite) I think we should leave them as general Predicate<? super T>. Therefore there needs to be instanceof checks as you descend into nested structure anyway. So…


#47

Dear @awalter17,

I am replying here to your request for review since I haven’t really looked at the implementation in depth.

Also, as you might have guessed from my previous comment, I am more concerned about how the new ROIs can be used in practice. Did I get that right, that your motivation was the saving of ROIs to OMERO (and loading from it) within the imagej-omero project? If so, that will be super useful.

From your previous explanation, I have gathered that the conversion to/from RAI<BooleanType> is pretty straightforward (thanks for the demos btw, that was really helpful!), which is awesome. I am, however, wondering how the new Masks integrate with the “old” Labelings. Do you/@tpietzsch have an idea how that integration will look like in future?

Best,
Stefan


#48

Hello @stelfrich!

That is correct! I am working on that on the rois branch in imagej-omero.

Unfortunately, I haven’t given Labelings much thought. So, I can’t give you an answer at this moment.

Cheers,

Alison


#49

Hello all,

My work in imglib2-roi has been merged and released (see PR for details)! :smile_cat:

Thank you very much to everyone who helped with this (@tpietzsch, @ctrueden, @axtimwalde, @dietzc, @haesleinhuepf, @imagejan)!

I’ve also brought my roi-demo branch in the imglib2-roi repo up to date. So, feel free to take a look at that to get a feel for the new ROI API!

Cheers,
Alison