Draw an overlay programmatically

Hi All,

I have a paticular issue that was hoping someone could help me with. It goes as follows:

  • We have a custom java application using ImageJ2, graphically embeded within this is ImageJ’s main toolbars.
  • We are using dataset and display window (from uiService.defaultUI.createDisplayWindow()) to create an image
  • The image comes from a camera. Each time a new image is availabe, we call setPlane(0, theImage)
  • When I draw an overlay (rectangle, oval, etc…) using the standard drawing tool, the overlay persists on the image despite the frequent setPlane() calls.
  • When I draw an overlay on the same image using OverlayService.drawOverlay() the overlay is drawn but there are two issues:
    • Firstly, the overlay disappears after the first call to setPlane(). I can call drawOverlay() each time I call setPlane() to make it “persist” but that involves creating a new overlay object on each new image.
    • Secondly, the overlay that is drawn using drawOverlay() just has a black outline and none of the graphical appearance (blue outline, yellow shading etc…) of an overlay that was drawn using the drawing tool.

Is OverlayService.drawOverlay() fully implemented or is it supposed to act differently to using the overlay drawing tool?

Related post:

I would appreciate any advice on this or any possible work arounds

Thanks in advance,

Jacob

I’m not sure exactly how OverlayService::drawOverlay() is supposed to work, but please note there’s a difference between ij.gui.Overlay (the ImageJ 1.x overlay) and net.imagej.overlay.Overlay (the ImageJ2 overlay).

If you say “graphically embedded within this is ImageJ’s main toolbars”, you most likely mean the legacy UI of ImageJ 1.x, together with the preferences (appearance of ROIs etc.) of ImageJ1. These are not automatically transferred to net.imagej.overlay.Overlay objects (yet).

In general, I fear that @ctrueden’s comment from the topic you linked is still valid today:

However, since then there has been the development of a new, powerful ROI model in imglib2-roi, together with converters from and to ImageJ1 ROIs in imagej-legacy.

If in the end you’re actually using ImageJ 1.x infrastructure (ImageWindow, ImagePlus etc.) to display your images in your custom GUI, it might be the safest to also stay with ImageJ 1.x overlays for now.

On the other hand, if you’re using #bigdataviewer , #sciview or some other (ImageJ2-based) visualization UI, you might want to post a link to your source code, so some experts can give more specific advice.

2 Likes

Hi Jan,

Thank you for your detailed reply. To clarify a few questions you had:

  • Everything we are doing is using ImageJ2 infrastructure wherever possibe. I am using net.imagej.overlay.Overlay
  • See the screenshot of how we’ve embedded ImageJ. You will see we’ve also added some new buttons to one of the toolbars.

  • I was reluctant to share code at first as we are using Kotlin (100% interoperable with Java) and other technologies. However here is the code (please note it is in Kotlin, but you should be able to understand it):

Create the camera display window from a dataset:

display = displayService.createDisplayQuietly(dataset)

view = ((display as DefaultImageDisplay).activeView as DefaultDatasetView)

displayWindow = uiService.defaultUI.createDisplayWindow(display)

uiService.viewerPlugins
	.map { pluginService.createInstance(it) }
	.find { it != null && it.canView(display) && it.isCompatible(uiService.defaultUI) }
	?.let {
		GUIMain.threadService.queue {
			(displayWindow as SwingDisplayWindow).apply {
				uiService.addDisplayViewer(it)
				it.view(this, display)
				pack()
				windowPanel.add((displayWindow as SwingDisplayWindow).rootPane)
			}
		}
	}

Then later on, using a custom right click menu get the drawn and selected overlay:

activeDisplay = imageDisplayService.activeImageDisplay
selectedROI = overlayService.getActiveOverlay(activeDisplay)

In the same block of code that calls setPlane(0, theImage) we have:

newOverlay = RectangleOverlay()
newOverlay.setOrigin(200.0, 0)
newOverlay.setOrigin(200.0, 1)
newOverlay.setExtent(200.0, 0)
newOverlay.setExtent(200.0, 1)
newOverlay.name = "TestOverlay"
ds = GUIMain.overlayService.defaultSettings
newOverlay.alpha = ds.alpha
newOverlay.fillColor = ds.fillColor
newOverlay.lineColor = ds.lineColor
newOverlay.lineStyle = ds.lineStyle
newOverlay.lineWidth = ds.lineWidth
	
threadService.queue {
	overlayService.drawOverlay(newOverlay, activeDisplay, ChannelCollection(listOf(0.0)))
	activeDisplay.update()
}

I have managed to find a possible workaround to this: using Java’s Robot() object - programmatically click on the overlay tool and then programmatically click and drag over the camera image. This does work but has pitfalls such as other custom build UI elements being in the way and its a workaround I want to avoid if I can.

Thanks,

Jacob

@jrjf2 I’m afraid that your project has collided with some of ImageJ2’s rough edges.

  1. You are using the Swing UI, which was originally written to replace ImageJ’s AWT user interface. In recent years, development of the Swing UI has been shelved in favor of continuing to support the AWT interface via the imagej-legacy adapter layer.

  2. The ImageJ2 net.imagej.overlay.Overlay machinery is now deprecated in favor of the imglib2-roi API. However, the ImgLib2 ROI types have not yet been integrated into the ImageJ user interface—and no one is actively working on doing this.

Keeping in mind both of the above points: what you are trying to do should be possible fundamentally, but there may be bugs. If you have the time and energy to debug into the code and fix them, then pull requests are welcome. Alternately, if you are able to create a minimal example demonstrating the issue, we can file an issue in the imagej-ui-swing component and I can try to dig a bit myself on what might be going wrong.

1 Like

@ctrueden

Please see the attached minimal examples created in IntelliJ IDEA. “DrawingOverlayMinimal” is truly minimal, “DrawingOverlay” is almost minimal but uses some more UI code that we use in our program. Both examples can reproduce the issue. I have also attached an example .tif as a dataset that will work with the examples provided.

I think we will end up making a pull request and look at fixing this ourselves but I don’t know the time frame on this.

Thanks,

JacobDrawingOverlay.zip (74.5 KB) DrawingOverlayMinimal.zip (71.3 KB) example.tif (6.3 MB)

1 Like

Sounds great, and sorry I don’t have more time to dig in myself!