Importing binary masks in QuPath

Continuing the discussion here

I want to import the dataset and masks availalable here to QuPath so that I get annotations corresponding to each individual gland. In the dataset masks pixels in the background have value 0, for each new annotation the next consequtive integer value is given (e.g. first annotation has pixel values 1, the 2nd one 2, etc.).
So far I have changed the mask values to binary to that all foreground pixels are 1 and all background 0 and slightly modified the script by Pete so that masks of the same size as the image are accepted (see below). As an example I attach the mask imported to python and made binary. I also changed the file names so that there are no underscores and def classificationString = parts[-2] to def classificationString = parts[-1].
With the original mask the output is

INFO: Adding Area (AWT) (testA) to hierarchy
INFO: Result: true

With the binary mask the output is:

INFO: Adding Area (AWT) (testA) to hierarchy
INFO: Adding Area (AWT) (2) to hierarchy
INFO: Result: true

There are some annotations added but they are empty with NaN centroid coordinates and 0 area.
All the help with understanding the issue would be hugely appreciated!

QuPath Script Used:

 * Script to import binary masks & create annotations, adding them to the current object hierarchy.
 * It is assumed that each mask is stored in a PNG file in a project subdirectory called 'masks'.
 * Each file name should be of the form:
 *   [Short original image name]_[Classification name]_([downsample],[x],[y],[width],[height])-mask.png
 * Note: It's assumed that the classification is a simple name without underscores, i.e. not a 'derived' classification
 * (so 'Tumor' is ok, but 'Tumor: Positive' is not)
 * The x, y, width & height values should be in terms of coordinates for the full-resolution image.
 * By default, the image name stored in the mask filename has to match that of the current image - but this check can be turned off.
 * @author Pete Bankhead

import ij.measure.Calibration
import ij.plugin.filter.ThresholdToSelection
import ij.process.ByteProcessor
import ij.process.ImageProcessor
import qupath.imagej.objects.ROIConverterIJ
import qupath.lib.objects.PathAnnotationObject
import qupath.lib.objects.classes.PathClassFactory
import qupath.lib.scripting.QPEx

import javax.imageio.ImageIO

// Get the main QuPath data structures
def imageData = QPEx.getCurrentImageData()
def hierarchy = imageData.getHierarchy()
def server = imageData.getServer()

// Only parse files that contain the specified text; set to '' if all files should be included
// (This is used to avoid adding masks intended for a different image)
def includeText = server.getShortServerName()

// Get a list of image files, stopping early if none can be found
def pathOutput = QPEx.buildFilePath(QPEx.PROJECT_BASE_DIR, 'masks')
def dirOutput = new File(pathOutput)
if (!dirOutput.isDirectory()) {
    print dirOutput + ' is not a valid directory!'
def files = dirOutput.listFiles({f -> f.isFile() && f.getName().contains(includeText) && f.getName().endsWith('-mask.png') } as FileFilter) as List
if (files.isEmpty()) {
    print 'No mask files found in ' + dirOutput

// Create annotations for all the files
def annotations = []
files.each {
    try {
        annotations << parseAnnotation(it)
    } catch (Exception e) {
        print 'Unable to parse annotation from ' + it.getName() + ': ' + e.getLocalizedMessage()

// Add annotations to image
hierarchy.addPathObjects(annotations, false)

 * Create a new annotation from a binary image, parsing the classification & region from the file name.
 * Note: this code doesn't bother with error checking or handling potential issues with formatting/blank images.
 * If something is not quite right, it is quite likely to throw an exception.
 * @param file File containing the PNG image mask.  The image name must be formatted as above.
 * @return The PathAnnotationObject created based on the mask & file name contents.
def parseAnnotation(File file) {
    // Read the image
    def img =

    // Split the file name into parts: [Image name, Classification, Region]
    def parts = file.getName().replace('-mask.png', '').split('_')

    // Discard all but the last 2 parts - it's possible that the original name contained underscores,
    // so better to work from the end of the list and not the start
    def classificationString = parts[-1]

    // Extract region, and trim off parentheses (admittedly in a lazy way...)
    //def regionString = parts[-1].replace('(', '').replace(')', '')

    // Create a classification, if necessary
    def pathClass = null
    if (classificationString != 'None')
        pathClass = PathClassFactory.getPathClass(classificationString)

    // Parse the x, y coordinates of the region - width & height not really needed
    // (but could potentially be used to estimate the downsample value, if we didn't already have it)
    //def regionParts = regionString.split(',')
    //double downsample = regionParts[0] as double
    //int x = regionParts[1] as int
    //int y = regionParts[2] as int

    // To create the ROI, travel into ImageJ
    def bp = new ByteProcessor(img)
    bp.setThreshold(127.5, Double.MAX_VALUE, ImageProcessor.NO_LUT_UPDATE)
    def roiIJ = new ThresholdToSelection().convert(bp)

    // Convert ImageJ ROI to a QuPath ROI
    // This assumes we have a single 2D image (no z-stack, time series)
    // Currently, we need to create an ImageJ Calibration object to store the origin
    // (this might be simplified in a later version)
    def cal = new Calibration()
    cal.xOrigin = 1
    cal.yOrigin = 1
    def roi = ROIConverterIJ.convertToPathROI(roiIJ, cal, 1, -1, 0, 0)

    // Create & return the object
    return new PathAnnotationObject(roi, pathClass)


In QuPath v0.1.2 you can try this:

import qupath.imagej.objects.ROIConverterIJ
import qupath.imagej.processing.ROILabeling
import qupath.lib.objects.PathAnnotationObject
import qupath.lib.objects.classes.PathClassFactory

import javax.imageio.ImageIO

// Get the main QuPath data structures
def imageData = getCurrentImageData()
def server = imageData.getServer()

// Get the annotation file
def pathAnnotation = server.getPath()
    .substring(0, server.getPath().lastIndexOf('.bmp')) + '_anno.bmp'
def fileAnnotation = new File(pathAnnotation)

// Parse the annotations
def annotations = parseAnnotations(fileAnnotation)

// Add annotations to image

 * Create new annotations from a labelled image.
def parseAnnotations(File file) {
    // Read the image
    def imp = ij.IJ.openImage(file.getAbsolutePath())
    def bp = imp.getProcessor()
    def cal = imp.getCalibration()

    // To create the ROI, travel into ImageJ
    def roisIJ = ROILabeling.labelsToConnectedROIs(bp, bp.getStatistics().max as int)
    def rois = roisIJ.collect { ROIConverterIJ.convertToPathROI(it, cal, 1, -1, 0, 0) }

    // Create & return the objects
    return rois.collect { new PathAnnotationObject(it, null) }

If assumes you have an image open, and the mask is found in the same directory with the same naming scheme and format as in the original download (.bmp is a curious choice…).

I haven’t checked it with any milestone versions, but I suspect some adjustments would be needed.

Thank you so much Pete!
I tried it with v0.1.2 with only 2 filed in the directory - the image and the mask from the original dataset, copied the script but still get an error message (see below) because of this command

def annotations = parseAnnotations(fileAnnotation)

Did I get it right that you have tried it and it worked? And did I get the instructions right?

As always very very grateful for your time!

INFO: before parseAnnotations
ERROR: Error at line 19: Cannot invoke method getProcessor() on null object

ERROR: Script error
    at org.codehaus.groovy.runtime.NullObject.invokeMethod(
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(
    at Script5.parseAnnotations(Script5.groovy:33)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
    at java.lang.reflect.Method.invoke(
    at org.codehaus.groovy.reflection.CachedMethod.invoke(
    at groovy.lang.MetaMethod.doMethodInvoke(
    at groovy.lang.MetaClassImpl.invokeMethod(
    at groovy.lang.MetaClassImpl.invokeMethod(
    at groovy.lang.DelegatingMetaClass.invokeMethod(
    at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl$2.invokeMethod(
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(
    at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(
    at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(
    at qupath.lib.scripting.DefaultScriptEditor.executeScript(
    at qupath.lib.scripting.DefaultScriptEditor.executeScript(
    at qupath.lib.scripting.DefaultScriptEditor.executeScript(
    at qupath.lib.scripting.DefaultScriptEditor.access$400(
    at qupath.lib.scripting.DefaultScriptEditor$
    at java.util.concurrent.Executors$
    at java.util.concurrent.ThreadPoolExecutor.runWorker(
    at java.util.concurrent.ThreadPoolExecutor$

Yes, I ran the script for a project containing all the original images and it worked using QuPath v0.1.2 on Windows 10. I see the same error as you if I run it on v0.2.0-m2 – in which case a small change is needed to use def fileAnnotation = new File(new URI(pathAnnotation))

The images are just unzipped from the original download, but otherwise unchanged, i.e… the annotated image name can be derived by replacing .bmp in the original name with _anno.bmp and looking in the same folder as the original. If this isn’t the case then the error will likely also occur.

Oh my god this is great, works perfectly! Thank you so much Pete, my project is rescued!

1 Like


I don’t think there are any examples of labels that have especially complicated bits in this dataset (e.g. holes, disconnected region)… if there are, please do check the script is doing what it should and let me know if you find any trouble/weirdness.

I just scrolled though most of those and the script seems to be doing a great job :slight_smile: will let you know if I come across a problem, thanks again!

Hi everyone,

I am trying to import and export binary masks into / out of QuPath using the original scripts Pete provided, but using version 0.2.0 m.3. However, I have run into problems with both scripts:

For the export script, I get an error saying I am unable to resolve class qupath.lib.roi.PathROIToolsAwt

For the import script, I get an error saying I am unable to resolve class qupath.imagej.objects.ROIConverterIJ

Any help in these areas would be greatly appreciated. Thank you.

Yep, the scripts are changing, and there is a good chance that a script for m3 would no longer work in m4 or m5. Pete may be able to figure it out quickly, but if your scripts are working in a previous version and you don’t require any of the newer functionality for your project, I would recommend sticking with 0.1.2 or 0.1.3.

You could try searching the forum for anyone else trying to do that in m3, but M3 feels like it passed by rather quickly. Maybe that’s just me and I got busy :slight_smile:

Some things simply shifted around, and there are a few posts in the Issues on Github detailing how to work around those changes. You might find them by specifically searching for those classes.

I wrote about what is changing at

At least one of the things you’re looking for is RoiTools… and I think it is IJTools for the ImageJ conversion (although I’m not at a computer not to check).

Hello Pete,

could you please point me to a direction how to modify a code for importing binary masks into annotations?

It doesn’t work in recent versions (0.2.0-m8) because of no ROIConverterIJ class, and I don’t know the alternative in RoiTools. Thanks a lot!

Hi @Jan_Horak, the relevant replacement is in IJTools. For a bit more explanation see

1 Like

Dear Pete,
Dear @Research_Associate,

I am just integrating Cellpose in our (Python) Slidescanner image analysis workflows (it is amazing!!).
At the same time our user are by now very much used to QuPath for annotations (and we love it very much). Thus, we would love to use it for modifying / correcting Cellpose annotations (rather than using the Cellpose GUI).

Getting the new annotations out of QuPath back in Python (if needed) is already established.

To this aim, my envisioned workflow would be as follows:

We save in the /home/me/contains_everything folder all images & all annotations.
Annotations are binary files of the same size as the image.
One annotation file contains only one class of annotations.
The class is coded in the file name.
Objects in the annotation file neither touch nor overlap.
Their name of the annotation file is then e.g. Mask_Class_ImageName.

Ideally, we would run a script directly after opening QuPath 0.2.0. m8.
in which we only have to define this variable:

create_project_from_this_folder = “/home/me/contains_everything”

Then the script would autocreate a project & load all images & annotate them.

I think the bits and pieces needed to do so are already out there hidden in various posts,
and i might start combining them by trial and error.

On the other hand, if you feel that this idea is of broad interest, the function
might just be included in m9 (then we would likely just wait), or you might consider,
to feature the solution as step-by-step “Script of the day” update for m8.

(or someone might alert me that he/she has already solved the problem).

What do you suggest?

Kind regards


1 Like

Hi @Tobias, I’ve already been thinking along similar lines :slight_smile:

At least, I am thinking of the following rather generic workflow:

  • Open image in QuPath, annotate (or whatever you wish to do)
  • Export tiles of a ‘manageable’ size for Python/Fiji/CellProfiler (with masks if necesssary)
  • Run whatever analysis/segmentation code you want elsewhere
  • Import binary masks/labelled images back into QuPath, creating objects of the appropriate type/classification and scaling/translating them to be in the correct position
  • Continue in QuPath as required (e.g. to measure/classify)

There wont be time to add all of the steps to m9 (which needs to be available before this workshop…), but recent additions like the TileExporter that perform some of the steps will be more usable and documented. I agree this would be generally useful, and I’d like to reduce the need for (complex) scripts as far as possible.

I think that the workflow I describe would also cover what you need, or have I misunderstood?

One thing I’m not clear about is what you mean with ‘Cellpose annotations’: are these nuclei/cells exclusively, or something else? And would you ideally want them in QuPath as editable annotations or cell objects that may be measured and classified?


Dear @Pete,

thanks a lot & have a great time in (hopefully sunny) San Diego!

Yes, your proposed workflow would cover the most important aspects of what we need.
We typically start in a jupyter notebook that deals with all pre-processing & file conversions.
(and also soon will handle segmentation using Cellpose headless).

Thus, the add-on, “create QuPath project from folder” would be nice, but it is also not much effort
to create the QuPath project manually and to add images to it manually.

The rest is then covered by your workflow.

Our ‘Cellpose annotations’ might be anything.
It might be plain Cellpose annotations of nuclei.
Or cell masks, that have already been classified in Python (in small round green cells,
big pyramid shaped red cells, ect…)

But irregardless of what it is biologically we thought to save a binary image containing all objects of a given class. Each object we would like to “interpret” as polygon object in QuPath.

This is particular important for one application in which we segment EM images using Cellpose.
Our objects of interest (special vesicle type) are typically not detected super reliably.
Thus, we would like to use QuPath to delete wrong detection, add missed ones,
and correct imprecise ones.

Following up your comment on “manageable” sizes and numbers:

Suppose we want to import nuclei segmentation of a high-res scan of an entire lung section.
Shall we then also tile the binary images into “manageable” sizes
(and encode the place they are from in the filename)?

If, yes, how do you suggest?

Thanks a lot & Kind regards


1 Like

Thanks @Tobias !

I assumed you’d already have to do this to work with the images in Python and generate the binary images (unless you’re handling the whole slide images in Python already, or working with smaller images). In any case, this is what the TileExporter class in QuPath is intended for (documentation to come…), enabling you to define the export resolution and tile size, along with various other customizations. It also encodes the position information either in the filename or, if exporting as ImageJ TIFF, the image metadata.

The importer class remains to be written fully, but it will be able to use the same naming scheme and/or read a text file that maps the binary images to their whole slide locations. Either way, it will be something doable in Python.

However, since you want a polygon mask in the end then I suspect that you could also export from Python as GeoJSON using Shapely - thereby giving the polygons rather than binary images. GeoJSON has a mechanism to support measurements and other associated information, which QuPath can then import.

You can see how such GeoJSON looks already within QuPath v0.2.0-m8 by selecting a few objects, and running the following code:

def selected = getSelectedObjects()
println GsonTools.getInstance(true).toJson(selected)

I recommend you don’t select too many, since it may result in printing rather a lot of text - but for import without printing it may be manageable.

A bit more documentation on this to come with m9…


Dear @petebankhead,

sorry for not being clear.

Yes, i can hold images of any size in Python and have there also the segmentation as (one big) binary image that I can tile as i want.

The export QuPath to Python already works.

I just modified one of your scripts according to comments distributed in various posts.
Here it is. Works with 0.2.0 m8.

 * Script to export binary masks corresponding to all annotations of an image,
 * optionally along with extracted image regions.
 * Note: Pay attention to the 'downsample' value to control the export resolution!
 * @author Pete Bankhead

import qupath.lib.images.servers.ImageServer
import qupath.lib.objects.PathObject

import javax.imageio.ImageIO
import java.awt.Color
import java.awt.image.BufferedImage

// Get the main QuPath data structures
def imageData = getCurrentImageData()
def hierarchy = imageData.getHierarchy()
def server = imageData.getServer()

// Request all objects from the hierarchy & filter only the annotations
def annotations = hierarchy.getAnnotationObjects()

// Define downsample value for export resolution & output directory, creating directory if necessary
def downsample = 1.0
def pathOutput = buildFilePath(QPEx.PROJECT_BASE_DIR, 'masks')

// Define image export type; valid values are JPG, PNG or null (if no image region should be exported with the mask)
// Note: masks will always be exported as PNG
def imageExportType = 'PNG'

// Export each annotation
annotations.each {
    saveImageAndMask(pathOutput, server, it, downsample, imageExportType)
print 'Done!'

 * Save extracted image region & mask corresponding to an object ROI.
 * @param pathOutput Directory in which to store the output
 * @param server ImageServer for the relevant image
 * @param pathObject The object to export
 * @param downsample Downsample value for the export of both image region & mask
 * @param imageExportType Type of image (original pixels, not mask!) to export ('JPG', 'PNG' or null)
 * @return
def saveImageAndMask(String pathOutput, ImageServer server, PathObject pathObject, double downsample, String imageExportType) {
    // Extract ROI & classification name
    def roi = pathObject.getROI()
    def pathClass = pathObject.getPathClass()
    def classificationName = pathClass == null ? 'None' : pathClass.toString()
    if (roi == null) {
        print 'Warning! No ROI for object ' + pathObject + ' - cannot export corresponding region & mask'

    // Create a region from the ROI
    def region = RegionRequest.createInstance(server.getPath(), downsample, roi)

    // Create a name
    String name = String.format('%s_%s_(%.2f,%d,%d,%d,%d)',

    // Request the BufferedImage
    def img = server.readBufferedImage(region)

    // Create a mask using Java2D functionality
    // (This involves applying a transform to a graphics object, so that none needs to be applied to the ROI coordinates)
    def shape = RoiTools.getShape(roi)
    def imgMask = new BufferedImage(img.getWidth(), img.getHeight(), BufferedImage.TYPE_BYTE_GRAY)
    def g2d = imgMask.createGraphics()
    g2d.scale(1.0/downsample, 1.0/downsample)
    g2d.translate(-region.getX(), -region.getY())

    // Create filename & export
    if (imageExportType != null) {
        def fileImage = new File(pathOutput, name + '.' + imageExportType.toLowerCase())
        ImageIO.write(img, imageExportType, fileImage)
    // Export the mask
    def fileMask = new File(pathOutput, name + '-mask.png')
    ImageIO.write(imgMask, 'PNG', fileMask)


I have also the Python counterpart. But that is integrated in some of our other code and is less easy to share & currently not well documented. If anyone is interested please send me a message, then i share it and help to get it running.

What does currently not work is Python to QuPath.

The current limitation I see (for some projects) is the number of objects. If we have lets say 50 000 nuclei, i would not like to export 50 000 tiny ROIs images from Python with the position encoded in the filename (as QuPath export does), but instead 50 tiled binary images with 1000 nuclei each.

Is that feasible?

If not I could indeed use Shapely, but I feel binary images are the most universal form to transfer this information, e.g. also to ImageJ, Ilastik, ect.

Thus, it would be my preferred standard.
Do you think it is feasible / easy to implement?

Kind regards


Hi @Tobias, yes I understand the problem. And while I know the export from QuPath is currently possible (since I wrote the scripts many times), the new way is better… or at least, I think the scripts are shorter, more readable, more customizable, more maintainable and faster (with parallelization).

For example, something like this should already work:

 * Script to export image tiles (can be customized in various ways).


// Get the current image (supports 'Run for project')
def imageData = getCurrentImageData()

// Define output path (here, relative to project)
def name = GeneralTools.getNameWithoutExtension(imageData.getServer().getMetadata().getName())
def pathOutput = buildFilePath(PROJECT_BASE_DIR, 'tiles', name)

// Define output resolution in calibrated units (e.g. µm if available)
double requestedPixelSize = 5.0

// Convert output resolution to a downsample factor
double pixelSize = imageData.getServer().getPixelCalibration().getAveragedPixelSize()
double downsample = requestedPixelSize / pixelSize

// Create an exporter that requests corresponding tiles from the original & labelled image servers
new TileExporter(imageData)
    .downsample(downsample)   // Define export resolution
    .imageExtension('.tif')   // Define file extension for original pixels (often .tif, .jpg, '.png' or '.ome.tif')
    .tileSize(512)            // Define size of each tile, in pixels
    .annotatedTilesOnly(false) // If true, only export tiles if there is a (classified) annotation present
    .overlap(64)              // Define overlap, in pixel units at the export resolution
    .writeTiles(pathOutput)   // Write tiles to the specified directory

print 'Done!'

I know this isn’t the problem you want to solve now, but I think that a good ‘general’ importer in QuPath needs to have a compatible ‘exporter’. The ‘exporter’ now exists, the importer will follow when I’ve time to write it.

In the meantime, you could certainly script something to do it using a combination of QuPath + ImageJ or OpenCV if you really wanted to. I wrote blog posts describing the key parts, e.g. here and here but some of the details have changed for v0.2.0 (some info here).

I agree binary images are fairly universal, but not without problems. For example, the question of 4 or 8 connectivity, or how contours are traced (which ImageJ and OpenCV do very differently, as described here), or if/how to merge objects across tile boundaries. I expect it to be much easier to solve these for one project than to solve them more generally.