M9 Multiplex Classifier Script Updates (cell summary measurements/visualization)

Hi all!

A few quick updates to help with the new multiplexing features added in M9. First of all, the best resource for information is still:
https://qupath.readthedocs.io/en/latest/docs/tutorials/multiplex_analysis.html

First, I have updated the cell density measurements scripts last posted for m5, now working in M9 for the Multiplex complex classifiers. The “every combination of base classes” is the full list of all existing classes within the image.


So these would be my base class measurements:

And these would be some of my full class measurements:

There were WAY too many to show.

I haven’t looked into an easy way to pull out a base list from all images within the project, maybe Pete will have some ideas there. The reason that is important is because any images that do not have at least one cell [say CD8 (Opal 540): PD1 (Opal 650): PDL1 (Opal 520)] of a particular class will simply be missing that measurement in their annotations, not have a measurement value of 0. That may end up causing some problems in Excel sheets down the line.

This gives you several options to help deal with the potentially overwhelming amount of data from the many possible classes.

As noted, the measurements created do not automatically update if you re-run the classifier. BE CAREFUL OF THIS. Also make sure to clear out old measurements (checkbox) if you are rerunning the script after adjusting the classifier enough that some full classes might disappear, if you are running it for full classes. Otherwise, classes that no longer have any cells after a second classifier run would not be over-written with 0s.

//Update to M5 measurements calculator for Multiplex analysis
// Initial script by Mike Nelson @Mike_Nelson on image.sc
// Project checking from Sara McArdle @smcardle on image.sc
// Additional information at link below.
// https://forum.image.sc/t/m9-multiplex-classifier-script-updates-cell-summary-measurements-visualization/34663

import qupath.lib.plugins.parameters.ParameterList
import qupath.lib.objects.PathCellObject
import qupath.lib.objects.PathObjectTools

imageData = getCurrentImageData()
server = imageData.getServer()
def metadata = getCurrentImageData().getServer().getOriginalMetadata()
def pixelSize = metadata.pixelCalibration.pixelWidth.value
hierarchy = getCurrentHierarchy()

separatorsForBaseClass = "[.-_,:]+" //add an extra symbol between the brackets if you need to split on a different character

boolean fullClassesB = true
boolean baseClassesB = true
boolean percentages = true
boolean densities = false
boolean removeOld = false
boolean checkProject = true

/*************************************************************************/
//////////// REMOVE BELOW THIS SECTION IF RUNNING FOR PROJECT////////////////
//EDIT THE CORRECT BOOLEAN VALUES ABOVE MANUALLY

def params = new ParameterList()
        //Yes, I am bad at dialog boxes.
        .addBooleanParameter("fullClassesB", "Measurements for full classes                                                                                                                      ", fullClassesB, "E.g. CD68:PDL1 cells would be calculated on their own")
        .addBooleanParameter("baseClassesB", "Measurements for individual base classes                                                                                                           ", baseClassesB, "Measurements show up as 'All' plus the base class name")
        .addBooleanParameter("percentages", "Percentages?                                                                                                                                        ", percentages, "")
        .addBooleanParameter("densities", "Cell densities?                                                                                                                                       ", densities, "Not recommended without pixel size metadata or some sort of annotation outline")
        .addBooleanParameter("removeOld", "Clear old measurements, can be slow                                                                                                                   ", removeOld, "Slow, but needed when the classifier changes. Otherwise classes that no longer exist will not be overwritten with 0 resulting in extra incorrect measurements")
        .addBooleanParameter("checkProject", "Use class names from entire project, can be slow!                                                                                                  ", checkProject, "Needed for projects where all images are analyzed and compared")
if (!Dialogs.showParameterDialog("Cell summary measurements: M9 VALUES DO NOT AUTOMATICALLY UPDATE", params))
    return



fullClassesB = params.getBooleanParameterValue("fullClassesB")
baseClassesB = params.getBooleanParameterValue("baseClassesB")
percentages = params.getBooleanParameterValue("percentages")
densities = params.getBooleanParameterValue("densities")
removeOld = params.getBooleanParameterValue("removeOld")
checkProject = params.getBooleanParameterValue("checkProject")


//////////// REMOVE ABOVE THIS SECTION IF RUNNING FOR PROJECT ///////////////////
/*************************************************************************/



if (removeOld){
    Set annotationMeasurements = []
    getAnnotationObjects().each{it.getMeasurementList().getMeasurementNames().each{annotationMeasurements << it}}

    List remove =[]
    annotationMeasurements.each{ if(it.contains("%") || it.contains("^")) {removeMeasurements(qupath.lib.objects.PathAnnotationObject, it);}}
}
Set baseClasses = []
Set classNames = []
if (checkProject){

    getProject().getImageList().each{
        def objs = it.readHierarchy().getDetectionObjects()
        classes = objs.collect{it?.getPathClass()?.toString()}
        classNames.addAll(classes)
    }

    classNames.each{
        it?.tokenize(separatorsForBaseClass).each{str->
           baseClasses << str.trim()
        }
    }
    println("Classifications: "+classNames)
    println("Base Classes: "+baseClasses)
}else{

    classNames.addAll(getDetectionObjects().collect{it?.getPathClass()?.toString()} as Set)


    classNames.each{
        it?.tokenize(separatorsForBaseClass).each{str->
           baseClasses << str.trim()
        }
    }
    println("Classifications: "+classNames)
    println("Base Classes: "+baseClasses)
}
//This section calculates measurements for the full classes (all combinations of base classes)

if (fullClassesB){

for (annotation in getAnnotationObjects()){
    totalCells = []
    qupath.lib.objects.PathObjectTools.getDescendantObjects(annotation,totalCells, PathCellObject)


    for (aClass in classNames){
        if (aClass){
            if (totalCells.size() > 0){
                cells = totalCells.findAll{it.getPathClass() == aClass}
                
                if (percentages) {annotation.getMeasurementList().putMeasurement(aClass.toString()+" %", cells.size()*100/totalCells.size())}
                annotationArea = annotation.getROI().getArea()
                if (densities) {annotation.getMeasurementList().putMeasurement(aClass.toString()+" cells/mm^2", cells.size()/(annotationArea*pixelSize*pixelSize/1000000))}
            } else {
                if (percentages) {annotation.getMeasurementList().putMeasurement(aClass.toString()+" %", 0)}
                if (densities) {annotation.getMeasurementList().putMeasurement(aClass.toString()+" cells/mm^2", 0)}
            }
        }
    }

}
}

//This section only calculates densities of the base class types, regardless of other class combinations.
//So all PDL1 positive cells would counted for a PDL1 sub class, even if the cells had a variety of other sub classes.

if (baseClassesB){

    for (annotation in getAnnotationObjects()){
        totalCells = []
        qupath.lib.objects.PathObjectTools.getDescendantObjects(annotation,totalCells, PathCellObject)


        for (aClass in baseClasses){

            if (totalCells.size() > 0){
                cells = totalCells.findAll{it.getPathClass().toString().contains(aClass)}
                
                if (percentages) {annotation.getMeasurementList().putMeasurement("All "+aClass+" %", cells.size()*100/totalCells.size())}
                annotationArea = annotation.getROI().getArea()
                if (densities) {annotation.getMeasurementList().putMeasurement("All "+aClass+" cells/mm^2", cells.size()/(annotationArea*pixelSize*pixelSize/1000000))}
            } else {
                if (percentages) {annotation.getMeasurementList().putMeasurement("All "+aClass+" %", 0)}
                if (densities) {annotation.getMeasurementList().putMeasurement("All "+aClass+" cells/mm^2", 0)}
            }

        }

    }
}

EDIT: Updated with Sara’s suggested script to include classes from the whole project as an additional option. This can be slow in large projects! Default checked.

Secondly, I updated the cell class visualization script to work for the subclasses now used in M9 complex classifiers.

The primary advantage here is quickly visualizing all of a particular subclass of cells within the potentially large mix of cells after your full classification. However, it does not seem to update the visible flag in the Annotations tab, so that might be a little bit weird :slight_smile:

Note that I was able to click the PDL1 box on the left to show all PDL1 classes, and hide all non-PDL1 cells.



The same effect could be achieved by going through all classes in the Annotation tab and selecting only the ones with PDL1 in the class name, but this is much quicker and less error-prone.

//Objective: A quicker way to show only certain classes and hide all others
//ANY GROUP CLASS CHECKING or UNCHECKED OVERWRITE ANY SINGLE CLASS CHANGES
//Written for 0.2.0m9
//https://forum.image.sc/t/m9-multiplex-classifier-script-updates-cell-summary-measurements-visualization/34663/2?u=mike_nelson

separatorsForBaseClass = "[.-_,:]+" //add an extra symbol between the brackets if you need to split on a different character

import javafx.application.Platform
import javafx.geometry.Insets
import javafx.scene.Scene
import javafx.geometry.Pos
import javafx.scene.control.TableView
import javafx.scene.control.CheckBox
import javafx.scene.layout.BorderPane
import javafx.scene.layout.GridPane
import javafx.scene.control.ScrollPane
import javafx.scene.layout.BorderPane
import javafx.stage.Stage
import javafx.scene.input.MouseEvent
import javafx.beans.value.ChangeListener
import qupath.lib.gui.QuPathGUI

//Find all classifications of detections

/*****************************************
If you have subcellular objects, you may want 
to change this to getCellObjects() rather than 
getDetectionObjects()
*****************************************/

def classifications = new ArrayList<>(  getDetectionObjects().collect {it?.getPathClass()} as Set)
/////////////////////////////////////////////////////////////

List<String> classNames = new ArrayList<String>()
classifications.each{
    classNames<< it.toString()
}
Set baseClasses = []
classifications.each{
    getCurrentViewer().getOverlayOptions().hiddenClassesProperty().add(it)
    it?.getName()?.tokenize(separatorsForBaseClass).each{str->
        baseClasses << str.trim()
    }
}
print baseClasses
baseList = baseClasses
//Find strings with duplicates in baseClasses
//baseList = baseClasses.countBy{it}.grep{it.value > 1}.collect{it.key}

//Set up GUI
int col = 0
int row = 0
int textFieldWidth = 120
int labelWidth = 150
def gridPane = new GridPane()
gridPane.setPadding(new Insets(10, 10, 10, 10));
gridPane.setVgap(2);
gridPane.setHgap(10);

ScrollPane scrollPane = new ScrollPane(gridPane)
scrollPane.setFitToHeight(true);
BorderPane border = new BorderPane(scrollPane)
border.setPadding(new Insets(15));

//Separately set up a checkbox for All classes
allOn = new CheckBox("All")
allOn.setId("All")
gridPane.add( allOn, 1, row++, 1,1)

row = 1
ArrayList<CheckBox> boxes = new ArrayList(classifications.size());
//Create the checkboxes for each class

for (i=0; i<classifications.size();i++){
    cb = new CheckBox(classNames[i])
    cb.setId(classNames[i].toString())
    boxes.add(cb)
    gridPane.add( cb, col, row++, 1,1)
}

//Create checkboxes for base classes, defined as some string that showed up in more than one class entry
ArrayList<CheckBox> baseBoxes = new ArrayList(baseList.size());
row = 2
for (i=0; i<baseList.size();i++){
    cb = new CheckBox(baseList[i])
    cb.setId(baseList[i])
    baseBoxes.add(cb)
    gridPane.add( cb, 1, row++, 1,1)
}
//behavior for all single class checkboxes
//I can't seem to check which checkbox is selected when they are created dynamically, so the results are updated for all classes
for (c in boxes){
    c.selectedProperty().addListener({o, oldV, newV ->
        firstCol = gridPane.getChildren().findAll{gridPane.getColumnIndex(it) == 0}
        for (n in firstCol){
            if (n.isSelected()){
                getCurrentViewer().getOverlayOptions().hiddenClassesProperty().remove(getPathClass(n.getId()))
            }else {getCurrentViewer().getOverlayOptions().hiddenClassesProperty().add(getPathClass(n.getId()))}
        }
    } as ChangeListener)
}
//behavior for base class checkboxes
//I can't easily figure out which checkbox was last checked, so this overwrites any single class checkboxes that were selected or unselected
for (c in baseBoxes){
    c.selectedProperty().addListener({o, oldV, newV ->
        //verify that we are in the second column, and the nodes are selected
        secondColSel = gridPane.getChildren().findAll{gridPane.getColumnIndex(it) == 1 && it.isSelected()}
        secondColUnSel = gridPane.getChildren().findAll{gridPane.getColumnIndex(it) == 1 && !it.isSelected()}
        for (n in secondColUnSel){
            batch = gridPane.getChildren().findAll{gridPane.getColumnIndex(it) == 0 && it.getId().contains(n.getId())}
            batch.each{
                it.setSelected(false)
                getCurrentViewer().getOverlayOptions().hiddenClassesProperty().add(getPathClass(it.getId()))
            }
        }
        for (n in secondColSel){
            
                batch = gridPane.getChildren().findAll{gridPane.getColumnIndex(it) == 0 && it.getId().contains(n.getId())}
                batch.each{
                    it.setSelected(true)
                    getCurrentViewer().getOverlayOptions().hiddenClassesProperty().remove(getPathClass(it.getId()))
                }
                
        }

        
    } as ChangeListener)
}


//Turn all on or off based on the All checkbox
allOn.selectedProperty().addListener({o, oldV, newV ->

    if (!allOn.isSelected()){
        classifications.each{
            getCurrentViewer().getOverlayOptions().hiddenClassesProperty().add(it)
        }
        gridPane.getChildren().each{
            it.setSelected(false)
        }
    }else {    
        classifications.each{
            getCurrentViewer().getOverlayOptions().hiddenClassesProperty().remove(it)
        }
        gridPane.getChildren().each{
            it.setSelected(true)
        }
    } 
}as ChangeListener)



//Some stuff that controls the dialog box showing up. I don't really understand it but it is needed.
Platform.runLater {

    def stage = new Stage()
    stage.initOwner(QuPathGUI.getInstance().getStage())
    stage.setScene(new Scene( border))
    stage.setTitle("Select classes to display")
    stage.setWidth(800);
    stage.setHeight(500);
    stage.setResizable(true);
    stage.show()

}

Finally, as part of my testing, I created a script to add measurements per sub-class to all cells. This could be useful for anyone wanting to run additional scripts where they pick out all cells that are positive for one particular marker for further processing.

The result is an additional set of measurements, like this:


Showing one cell from the above test set that was positive for both PDL1 and PD1. If I wanted to do anything to all cells that were PDL1 positive now, I could easily:
PDL1Cells = getCellObjects().findAll{measurement(it,"PDL1 (Opal 520)") > 0}

//Create one measurement per sub-class, for all classes in the image.
//https://forum.image.sc/t/m9-multiplex-classifier-script-updates-cell-summary-measurements-visualization/34663/3?u=mike_nelson
separatorsForBaseClass = "[.-_,:]+" //add an extra symbol between the brackets if you need to split on a different character

def classifications = new ArrayList<>(getDetectionObjects().collect {it?.getPathClass()} as Set)

List<String> classNames = new ArrayList<String>()

classifications.each{
    classNames<< it?.toString()
}
print classNames
Set baseClasses = []
classNames.each{
    it?.tokenize(separatorsForBaseClass).each{str->
        baseClasses << str.trim()
    }
}
//print baseClasses
//Find strings with duplicates in baseClasses


getCellObjects().each{c->
    List currentBaseClasses = []
    classes = c.getPathClass().toString()
    //print classes
    classes.tokenize(separatorsForBaseClass).each{str->
        currentBaseClasses << str.trim()
    }

    baseClasses.each{e->
        if (currentBaseClasses.contains(e)){
            c.getMeasurementList().putMeasurement(e,1)
        } else {c.getMeasurementList().putMeasurement(e,0)}
    }
}

Nice! Super useful!

I haven’t looked into an easy way to pull out a base list from all images within the project, maybe Pete will have some ideas there. The reason that is important is because any images that do not have at least one cell [say CD8 (Opal 540): PD1 (Opal 650): PDL1 (Opal 520)] of a particular class will simply be missing that measurement in their annotations, not have a measurement value of 0. That may end up causing some problems in Excel sheets down the line.

Maybe this will help:

def project = QPEx.getProject()
def fileNames = project.getImageList()
detClasses=[]
fileNames.each{
    def objs = it.readHierarchy().getDetectionObjects()
    def imageClasses=objs.collect{it.getPathClass()}
    detClasses.addAll(imageClasses)
}
def projClasses=detClasses.unique()
List<String> classNames = new ArrayList<String>()

projClasses.each{
    classNames<< it?.toString()
}
1 Like

Thanks! Updated the script to handle whole project class lists!