Exporting annotation to tiled images excluding pixels outside the annotation

Hi all,
I would like to have a script which is exporting an annotated region as image tiles. I first tried with Pete’s example on Tiling a whole slide image and Exporting labelled images, however I would like pixels outside the annotations to be removed (i.e. set to white or so).

So what I tried based on the mentioned scripts was to convert the image to an ImagePlus, create a mask image of identical size and use the ImageJ calculator to remove the disturbing pixels:

/**
 * Export annotation partitioned to png-tiles setting pixels outside the annotation to white
 *  07/17/2019
 */
import ij.IJ
import ij.ImagePlus
import ij.plugin.ImageCalculator
import qupath.imagej.helpers.IJTools
import qupath.lib.gui.scripting.QPEx
import qupath.lib.images.servers.ImageServer
import qupath.lib.regions.RegionRequest
import qupath.lib.roi.PathROIToolsAwt
import qupath.lib.scripting.QP
import java.awt.Color
import java.awt.image.BufferedImage

Date date=new Date()
println(date)
//set tile size
int tileHeight=1200
int tileWidth=1200
//some parameters to discard tiles
//discard tiles smaller than 100px in any dimension
int minImageDimension=100
//discard empty tiles (avg.int.=255)
boolean doNotSaveEmptyTiles=true
//Indicate low content (avg.int>245 or stdDev<10) in filename of tiles
boolean indicateAlmostEmptyTiles=true
//downsample exported tiles
double downsample=1
//set format of exported tiles
String format="png"
String ext
if (format.startsWith("."))
    ext=format.substring(1).toLowerCase()
else
    ext=format.toLowerCase()


ImageServer<BufferedImage> serverOriginal= QP.getCurrentImageData().getServer()
String path = serverOriginal.getPath()
String serverName = serverOriginal.getShortServerName()

//only the first annotation is considered
annotation =QP.getAnnotationObjects().get(0)
roi=annotation.getROI()
//get pixel data from annotation and convert to IJ ImagePlus
RegionRequest request1 =RegionRequest.createInstance(path,downsample,roi)
imp1= IJTools.convertToImagePlus(serverOriginal,request1).getImage()
//get image dimensions
width = imp1.getWidth()
height= imp1.getHeight()
//create an empty RGB image of identical dimensions
def imgMask = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB)
//draw shape of annotation into the empty image
def g2d = imgMask.createGraphics()
g2d.translate(-request1.getX(), -request1.getY())
def shape = PathROIToolsAwt.getShape(roi)
Color white = new Color(1,1,1)
g2d.setColor(white)
g2d.fill(shape)
maskImp =new ImagePlus("maskImage",imgMask)
//multiply maskImage to remove pixels outside annotation area
ImageCalculator ic = new ImageCalculator()
ImagePlus maskedImp = ic.run("Multiply create", imp1, maskImp)
//multiply by 255 and invert mask image
IJ.run(maskImp, "Multiply...", "value=255")
IJ.run(maskImp, "Invert", "")
//perform OR operation of inverted mask and masked image to set pixels outside of the annotation to white
ImagePlus fullyMaskedImp=ic.run("OR create", maskedImp, maskImp)

//extract and save tiles from the masked annotation image
counter=0
for(double y=0;y<fullyMaskedImp.getHeight();y+=tileHeight){
    int yi=(int)(y+0.5)
    int y2i=(int)Math.min((int)(y+tileHeight+0.5),fullyMaskedImp.getHeight())
    int hi =y2i-yi
    if(hi/downsample<minImageDimension){
        //println("skipping row")
        continue
    }
    for(double x=0;x<fullyMaskedImp.getWidth();x+=tileWidth){
        int xi=(int)(x+0.5)
        int x2i=(int)Math.min((int)(x+tileWidth+0.5),fullyMaskedImp.getWidth())
        int wi=x2i-xi

        if(wi/downsample<minImageDimension){
            if (y > 0)
                //println("skipping column")
            continue
        }
        fullyMaskedImp.setRoi(xi,yi,wi,hi)
        tileImp = fullyMaskedImp.crop()
        stat=tileImp.getStatistics(6)
        println(stat.stdDev)
        println(stat.mean)

        resultfoldername=QP.getCurrentImageData().getServer().getDisplayedImageName().toUpperCase().replaceAll("\\s","")
        resultfolder=QPEx.buildFilePath(QPEx.PROJECT_BASE_DIR, resultfoldername)
        String tileName=String.format("%s(d=%.2f,x=%d,y=%d,w=%d,h=%d).%s",serverName,downsample,xi,yi,wi,hi,ext)

        if (doNotSaveEmptyTiles) {
            if (stat.mean > 254.9) {
                //println("Skipping empty tile " + tileName)
                continue
            }
        }
        if (indicateAlmostEmptyTiles) {
            if (stat.mean > 245 || stat.stdDev < 10) {
                tileName = "00-lowContent_" + tileName
            }
        }
        saved = saveTile(resultfolder, tileName, tileImp)

    }
}

println("Done.")

boolean saveTile(resultfolder,tileName,tileImp){
    try {
        dirOut=new File(resultfolder)
        if (!dirOut.exists()) {
            dirOut.mkdirs()
            println("Created folder "+dirOut)
        }
        File tileFile= new File(dirOut,tileName)
        IJ.save(tileImp,tileFile.getAbsolutePath())
        //println("Written tile "+tileName+ " to "+ tileFile.getAbsolutePath())
    } catch(Exception e){
        e.printStackTrace()
        println("To many errors, aborting...")
        e.printStackTrace()
        return false
    }
    return true
}

I understand that this is no good idea as it loads the potentially huge annotation image and a mask image of the same size into the memory. Now I am considering to apply the mask on the image tiles - though I do not know yet how to get “mask tiles”…
So my question is does maybe anyone here have an idea how to solve this more elegantly?

Cheers, Peter

Can you say more about the purpose, i.e. what you’ll do with the tiles afterwards?

While it’s certainly possible to do everything in a single script, it might be easier to decouple the masking and apply it as a separate step after exporting using the scripts of mine you already mentioned.

For example, this might take a few links of code with Python/Numpy, but rather a lot more Groovy since you’ll need to contend with Java BufferedImages, Graphics2D objects and so on.

If he has the tiles, it sounds like it should be a fairly straightforward region request, followed by masking… I think? Not 100% sure and early morning isn’t always my best time :slight_smile:

This script has a section for region requests that makes a mask, and then generates colocalization values within the masked area. You could probably do something similar with the annotation tiles, rather than cells or nuclei. Though it was rough at the time getting that script to work, so if it is easier to clean up after the export, as Pete said…