Interactive image alignment

There’s no built-in feature to do this… yet.

@MustafaElshani @Research_Associate @petebankhead

Hi there,

I am trying to open hyperstack image from imagej in qupath but having difficulty, when I save the hyperstack as a tiff file it only opens one channel however in the hyperstack image I have there are 8 channels. Anyone able to point me in the right direction please?

Thanks

Are you sure that the image you created has 8 channels and not 8 slices or time points?

Hi there, I have 8 slices. So I believe I have created an 8 slice stack not an 8 channel format.
I assume there is a way to convert slices to channels.

Yep, as mentioned above, you can adjust that in ImageJ before saving the file, Image-> Properties.

So I’m trying to use the code above, but I’m having trouble with the annotations that are copied to the second image…

So they look fine on the interactive image alignment popup window

so I took the matrix from there and changed the code above to:

/**
 * Script to transfer QuPath objects from one image to another, applying an AffineTransform to any ROIs.
 */

// SET ME! Define transformation matrix
// Get this from 'Interactive image alignment (experimental)
def matrix = [
  1.000, 0.000, -329.492
  -0.000, 1.000, -3225.091
]

// SET ME! Define image containing the original objects (must be in the current project)
def otherImageName = 'Image_23.vsi - 10x'

// SET ME! Delete existing objects
def deleteExisting = false

// SET ME! Change this if things end up in the wrong place
def createInverse = true


import qupath.lib.gui.helpers.DisplayHelpers
import qupath.lib.objects.PathCellObject
import qupath.lib.objects.PathDetectionObject
import qupath.lib.objects.PathObject
import qupath.lib.objects.PathObjects
import qupath.lib.objects.PathTileObject
import qupath.lib.roi.PathROIToolsAwt
import qupath.lib.roi.interfaces.ROI

import java.awt.geom.AffineTransform

import static qupath.lib.gui.scripting.QPEx.*

if (otherImageName == null) {
    DisplayHelpers.showErrorNotification("Transform objects", "Please specify an image name in the script!")
    return
}

// Get the project & the requested image name
def project = getProject()
def entry = project.getImageList().find {it.getImageName() == otherImageName}
if (entry == null) {
    print 'Could not find image with name ' + otherImageName
    return
}

def otherHierarchy = entry.readHierarchy()
def pathObjects = otherHierarchy.getRootObject().getChildObjects()

// Define the transformation matrix
def transform = new AffineTransform(
        matrix[0], matrix[3], matrix[1],
        matrix[4], matrix[2], matrix[5]
)
if (createInverse)
    transform = transform.createInverse()

if (deleteExisting)
    clearAllObjects()

def newObjects = []
for (pathObject in pathObjects) {
    newObjects << transformObject(pathObject, transform)
}
addObjects(newObjects)

print 'Done!'

/**
 * Transform object, recursively transforming all child objects
 *
 * @param pathObject
 * @param transform
 * @return
 */
PathObject transformObject(PathObject pathObject, AffineTransform transform) {
    // Create a new object with the converted ROI
    def roi = pathObject.getROI()
    def roi2 = transformROI(roi, transform)
    def newObject = null
    if (pathObject instanceof PathCellObject) {
        def nucleusROI = pathObject.getNucleusROI()
        if (nucleusROI == null)
            newObject = PathObjects.createCellObject(roi2, pathObject.getPathClass(), pathObject.getMeasurementList())
        else
            newObject = PathObjects.createCellObject(roi2, transformROI(nucleusROI, transform), pathObject.getPathClass(), pathObject.getMeasurementList())
    } else if (pathObject instanceof PathTileObject) {
        newObject = PathObjects.createTileObject(roi2, pathObject.getPathClass(), pathObject.getMeasurementList())
    } else if (pathObject instanceof PathDetectionObject) {
        newObject = PathObjects.createDetectionObject(roi2, pathObject.getPathClass(), pathObject.getMeasurementList())
    } else {
        newObject = PathObjects.createAnnotationObject(roi2, pathObject.getPathClass(), pathObject.getMeasurementList())
    }
    // Handle child objects
    if (pathObject.hasChildren()) {
        newObject.addPathObjects(pathObject.getChildObjects().collect({transformObject(it, transform)}))
    }
    return newObject
}

/**
 * Transform ROI (via conversion to Java AWT shape)
 *
 * @param roi
 * @param transform
 * @return
 */
ROI transformROI(ROI roi, AffineTransform transform) {
    def shape = PathROIToolsAwt.getShape(roi) // Should be able to use roi.getShape() - but there's currently a bug in it for rectangles/ellipses!
    shape2 = transform.createTransformedShape(shape)
    return PathROIToolsAwt.getShapeROI(shape2, roi.getC(), roi.getZ(), roi.getT(), 0.5)
}

When I run it - it prints Done! but the annotations on the image are not in the correct place at all - I tried the troubleshooting options above (toggling createinverse) as well as looked for the missing comma on the first line of the affine transformation matrix but I think it’s there?

This is how the annotations show up on the image - as a straight polygon on the extremity of the image…

Does anyone know what could be my problem?

Thanks,

belliveau13

Same error as above, you are missing a comma in the matrix. There may be other problems, but I would start with that.

To be fair, the matrix, as copied, is missing the comma to begin with :slight_smile: Maybe if enough people bug Pete about it he’ll add it in.

He already has… (even if he got the commit message wrong)

1 Like

So turns out when I was troubleshooting I was adding the comma in the wrong matrix and the wrong place :sweat:

However, now that I’ve done this correctly, I get the ROIs but still in the wrong place. But again, the transform matrix is from the interactive overlay alignment which previews as perfect…

PREVIEW:

OUTCOME:

Thoughts?

I toggled creatInverse and it worked! Thanks for all the help!!!

2 Likes

I’ve edited the script above for compatibility with v0.2.0-m3.

Two changes:

  • PathROIToolsAwt has become RoiTools (part of simplifying things)
  • Introduced ImagePlane to define z-stack and timepoint location (rather than supplying individual ints for both)

Hi all, I am trying to transfer an annotation (polygon) and detected cells to another image using the code provided in this thread, but am running into issues. I am using version 0.2.0-m3. Here is my code:

/**

  • Script to transfer QuPath objects from one image to another, applying an AffineTransform to any ROIs.
    */

// SET ME! Define transformation matrix
// Get this from 'Interactive image alignment (experimental)
def matrix = [
1.000, 0.001, -2.253,
-0.001, 1.000, 3.678
]

// SET ME! Define image containing the original objects (must be in the current project)
def otherImageName = ‘435.tif’

// SET ME! Delete existing objects
def deleteExisting = false

// SET ME! Change this if things end up in the wrong place
def createInverse = true

import qupath.lib.gui.helpers.DisplayHelpers
import qupath.lib.objects.PathCellObject
import qupath.lib.objects.PathDetectionObject
import qupath.lib.objects.PathObject
import qupath.lib.objects.PathObjects
import qupath.lib.objects.PathTileObject
import qupath.lib.roi.RoiTools
import qupath.lib.roi.interfaces.ROI

import java.awt.geom.AffineTransform

import qupath.lib.gui.scripting.QPEx

if (otherImageName == null) {
DisplayHelpers.showErrorNotification(“Transform objects”, “Please specify an image name in the script!”)
return
}

// Get the project & the requested image name
def project = getProject()
def entry = project.getImageList().find {it.getImageName() == otherImageName}
if (entry == null) {
print 'Could not find image with name ’ + otherImageName
return
}

def otherHierarchy = entry.readHierarchy()
def pathObjects = otherHierarchy.getRootObject().getChildObjects()

// Define the transformation matrix
def transform = new AffineTransform(
matrix[0], matrix[3], matrix[1],
matrix[4], matrix[2], matrix[5]
)
if (createInverse)
transform = transform.createInverse()

if (deleteExisting)
clearAllObjects()

def newObjects =
for (pathObject in pathObjects) {
tempObject = transformObject(pathObject, transform)
if (tempObject != null) {
tempObject.setName(pathObject.getName())
newObjects << tempObject
}}
addObjects(newObjects)

print ‘Done!’

/**

  • Transform object, recursively transforming all child objects
  • @param pathObject
  • @param transform
  • @return
    */
    PathObject transformObject(PathObject pathObject, AffineTransform transform) {
    // Create a new object with the converted ROI
    def roi = pathObject.getROI()
    def roi2 = transformROI(roi, transform)
    def newObject = null
    if (pathObject instanceof PathCellObject) {
    def nucleusROI = pathObject.getNucleusROI()
    if (nucleusROI == null)
    newObject = PathObjects.createCellObject(roi2, pathObject.getPathClass(), pathObject.getMeasurementList())
    else
    newObject = PathObjects.createCellObject(roi2, transformROI(nucleusROI, transform), pathObject.getPathClass(), pathObject.getMeasurementList())
    } else if (pathObject instanceof PathTileObject) {
    newObject = PathObjects.createTileObject(roi2, pathObject.getPathClass(), pathObject.getMeasurementList())
    } else if (pathObject instanceof PathDetectionObject) {
    newObject = PathObjects.createDetectionObject(roi2, pathObject.getPathClass(), pathObject.getMeasurementList())
    } else {
    newObject = PathObjects.createAnnotationObject(roi2, pathObject.getPathClass(), pathObject.getMeasurementList())
    }
    // Handle child objects
    if (pathObject.hasChildren()) {
    newObject.addPathObjects(pathObject.getChildObjects().collect({transformObject(it, transform)}))
    }
    return newObject
    }

/**

  • Transform ROI (via conversion to Java AWT shape)
  • @param roi
  • @param transform
  • @return
    */
    ROI transformROI(ROI roi, AffineTransform transform) {
    def shape = RoiTools.getShape(roi) // Should be able to use roi.getShape() - but there’s currently a bug in it for rectangles/ellipses!
    shape2 = transform.createTransformedShape(shape)
    return RoiTools.getShapeROI(shape2, roi.getImagePlane(), 0.5)
    }

This should be exactly the same code as has been improved on above (I did change import static qupath.lib.gui.scripting.QPEx.* to import qupath.lib.gui.scripting.QPEx, as I believe this needs to be done with the new version)

When I run the code, I get this error at line 65 (which is tempObject = transformObject(pathObject, transform)):

ERROR: Error at line 65: No signature of method: static qupath.lib.objects.PathObjects.createCellObject() is applicable for argument types: (qupath.lib.roi.PolygonROI, null, qupath.lib.measurements.DefaultMeasurementList) values: [Polygon, null, [Cell: Area: 229.0, Cell: Perimeter: 57.24557357284336, Cell: Circularity: 0.8781354788185962, Cell: Max caliper: 20.452254322661084, Cell: Min caliper: 14.0, Cell: Eccentricity: 0.7036413955578946, Cell: Channel 1 mean: 497.32758620689657, Cell: Channel 1 std dev: 103.34054792817992, Cell: Channel 1 max: 745.0, Cell: Channel 1 min: 319.0, Cytoplasm: Channel 1 mean: 431.22627737226276, Cytoplasm: Channel 1 std dev: 69.37006396862219, Cytoplasm: Channel 1 max: 625.0, Cytoplasm: Channel 1 min: 319.0]]
Possible solutions: createCellObject(qupath.lib.roi.interfaces.ROI, qupath.lib.roi.interfaces.ROI, qupath.lib.objects.classes.PathClass, qupath.lib.measurements.MeasurementList)

Any help would be greatly appreciated. Thanks!

Hi, sorry no time to look in detail but the problem appears to be that the static method requires two ROIs (one for the nucleus, one for the cell boundary) but for some reason you only have one.

Since cell objects are usually created by detecting the nucleus and expanding it, both nucleus and cell ROIs are generally present. Did you use the Cell + membrane detection command, or some other way of generating cells?

It may be that replacing

newObject = PathObjects.createCellObject(roi2, pathObject.getPathClass(), pathObject.getMeasurementList())

with

newObject = PathObjects.createCellObject(roi2, null, pathObject.getPathClass(), pathObject.getMeasurementList())

may help (I haven’t checked it…) but since the primary distinguishing feature of cell objects as opposed to general detections is that they can have a nucleus then I’m not sure if this is the solution you really want… or if instead you might wish to filter out all cell objects that are missing nuclei first.

As a note, it does help anyone trying to run your code if you format it as code. It is perfectly readable as is, but attempting to copy it into QuPath would fail. Use the formatting button shown below to make code copy and pastable.
image

println(“Hello World”)
vs
println("Hello World")

1 Like

Thanks for the quick reply. I used cell detection without the include cell nucleus feature selected. I replaced the line with your code and it worked! Thanks for your help.

Thanks for the tip, I will do this next time!

Thank you very much for this wonderful program QuPath and the amazing update!

I am new to QuPath and image analysis and do not even know how to do coding. I learned QuPath from the tutorial video. My research is also examining immune cells in tumor. I have multiple cases of IHC stained slides which are AEC multiplex stain of 5 different markers on a same slide (stain and destain repeated for each markers on the same slide).
I think the QuPath 0.2.0-m3 interactive image alignment is really great for this type of research.

I have some questions using QuPath 0.2.0-m3. I would appreciate any help.

  1. Is there a way to view all the ROI markings again after transferring the multiple ROIs to another slide? What I did was merge 4 ROIs -> transfer last ROI to another slide -> split annotations. Then I could not see the annotation but if I click the annotation tab I could see one annotation at a time upon clicking the roi list.
  2. Is it possible to annotate the overlapped image while the QuPath interactive image alignment is turned on with the original image selected for working on project?
    3.I tried the above script with the input of annotated file name for OtherImageName and matrix number from the interactive image alignment and clicked ‘run’ at the selection status of the slide I want the annotation and detection transferred to. And I got the error message as follows. What could have gone wrong ? Is there more input I should do? (I may have misinterpreted the script.)
    Line 26-29 are as follows:
    if (otherImageName == null) {
    DisplayHelpers.showErrorNotification(“Transform objects”, “Please specify an image name in the script!”)
    return
    }

ERROR: Error: startup failed:
Script3.groovy: 28: unexpected token: objects” @ line 28, column 49.
wErrorNotification(“Transform objects”,

1 error

ERROR: Script error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:311)
at org.codehaus.groovy.control.ErrorCollector.addFatalError(ErrorCollector.java:151)
at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:121)
at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:133)
at org.codehaus.groovy.control.SourceUnit.addError(SourceUnit.java:325)
at org.codehaus.groovy.antlr.AntlrParserPlugin.transformCSTIntoAST(AntlrParserPlugin.java:224)
at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:190)
at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:226)
at org.codehaus.groovy.control.CompilationUnit$1.call(CompilationUnit.java:196)
at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:965)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:647)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:623)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:600)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:390)
at groovy.lang.GroovyClassLoader.access$300(GroovyClassLoader.java:89)
at groovy.lang.GroovyClassLoader$5.provide(GroovyClassLoader.java:330)
at groovy.lang.GroovyClassLoader$5.provide(GroovyClassLoader.java:327)
at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:147)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:325)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:309)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:251)
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.getScriptClass(GroovyScriptEngineImpl.java:331)
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:153)
at qupath.lib.gui.scripting.DefaultScriptEditor.executeScript(DefaultScriptEditor.java:766)
at qupath.lib.gui.scripting.DefaultScriptEditor.executeScript(DefaultScriptEditor.java:696)
at qupath.lib.gui.scripting.DefaultScriptEditor.executeScript(DefaultScriptEditor.java:676)
at qupath.lib.gui.scripting.DefaultScriptEditor$ProjectTask.call(DefaultScriptEditor.java:1287)
at qupath.lib.gui.scripting.DefaultScriptEditor$ProjectTask.call(DefaultScriptEditor.java:1236)
at javafx.concurrent.Task$TaskCallable.call(Task.java:1425)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)

4.What I would like to do is to duplicate all of the ROI and their detections onto the new image from the best stained slide, then running something like Add cell intensity measurements to get the color and intensity information for those copied detections.

  1. In the next milestone, would it be possible to get the numbering (or some kind of cell ID) for each cells on the measurement table so that the copied detected cells and the original detected cells can be compared in the merged tables? (In that way, all the measurements of same cells of the same ROIs might be recorded for each markers if there is a certain rule to generate the meaurement table … like in what order the cells in the ROI should be listed in the table. If the cell detection is copied onto the other image of the same tissue, the detected cell number and location should be the same and only the color and intensity should be different.)
    Or if I can succeed in copying annotation and cell detection with acquisition of the color and intensity measurement in the new image, I might want to try sorting the table by the x y coordinate to have them in the same order to compare. I thought about this because I am not familiar with imageJ or cellprofiler.

Any help would be appreciated.

Thank you for reading this long message.

I am guessing for the script error that you tried to copy @bluna301’s version of the script above, which won’t work since he did not post it as code.

You also did not post your code as code, so it wouldn’t be possible to copy that out and use it either.

Sorry, in a rush today and won’t have a chance to look at the rest for a bit.

I’m glad you like it :smile:

Are the annotations perhaps just hidden? See View → Show annotations (there’s also a button on the toolbar). Alternatively, is the opacity slider in the toolbar to the left (transparent) or right (opaque)?

I’m not sure I understand 100%. The overlapped image is just there for display and you can adjust the opacity with the slider on the toolbar. If you annotate, you’ll be annotating the image underneath.

However if you are displaying the overlapped image then it should impact the Wand tool, which can be useful when annotating.

I think that the quotation marks have subtly become curly during copy & paste…

“Transform objects”

should be

"Transform objects"

I totally agree that something like this is needed, but I’m afraid this is I won’t have time to work on it myself in the near future. Including an ID and making sure it is unique would be rather complicated with QuPath’s current design.

I did add some improvements this today and yesterday to adding intensity measurements (e.g. adding measurements within the nucleus only) that may help a bit.

There are probably ways to work around this through scripting. I strongly suspect @Research_Associate will come up something :wink:

Oh right, that was already done in a couple of places, but I can’t find them right now.
I guess you wanted something like this if you are copying over the whole measurement list:

i = 1
getCellObjects().each{
it.getMeasurementList().putMeasurement("Label", i)
i++
}

Each cell should have a Measurement Label, from 1 to N after running the script. Cells appear to be ordered by XY centroid coordinates.

If the measurements aren’t copied over, this likely won’t work, though, and I haven’t played with the transfer much. Maybe look into it later.