Exporting polygons of cell detections objects along in CSV

Hello QuPath community.

I am using version 0.2.0-m12 of QuPath (I would rather not have to change)

We need to export the list of points in the nucleus and cell roi along with the csv of the measurements. It doesn’t matter that it becomes a wide column with a long string on it.

I know perfectly how to get such values, what I don’t know is how to add them to the object so that the qupath.lib.gui.tools.MeasurementExporter is able to export it. (I am able to import it in my version)

This is how I am exploring the properties of a cellDetection object:

import static qupath.lib.gui.scripting.QPEx.*
import com.google.gson.GsonBuilder
import qupath.lib.gui.tools.MeasurementExporter

//get cell detections
def cells = getCellObjects()

//check out the properties available for a cell
def properties = cells[0].getProperties()
for(p in properties){
    print p
}

//get the points of a nucleus
def cellps = cells[0].nucleusROI.getAllPoints()

//call gson
def gson = new GsonBuilder().setPrettyPrinting().create()

//get a json representation of these points
def jsonpoly = gson.toJson(cellps)

//print it, I want to stringify it
def polygonString = jsonpoly.toString()
print polygonString

I am looking at the nice documentation on how to export measuments via scripting, and as far as I understand, I could make the stringified json list, a measurement of a cellDetectionObject and have it be exported when I export my csv.

Can I add any arbitrary key,value pair to the cellDetectionObject so that it is exported in the csv?

Can someone tell me how can I do this?

Thank you

I am fairly certain measurements can only be numbers, not strings, characters, or other complex objects. You could make an incredibly long list of X and Y coordinates as measurements, I guess… but that would be very cumbersome. It might be better to number the cells with a single measurement and match to an exported json or csv, or a csv that has one cell or nucleus per line.

1 Like

Yes, thank you as you say

It might be better to number the cells with a single measurement and match to an exported json or csv, or a csv that has one cell or nucleus per line.

In the post you link to, it says annotation ID, but since these are not annotations but detections objects I don’t know can I do it, because they don’t have ID.

When I observe all the properties of an detectionObject (not annotation) One can see that there are strings, like the class or ROI, see image.

So I was wondering if in that same way I can add an arbitrary list of properties, for example a cell id. I need a unique id to identify the cell. If I cant add the string then I could at least add a numeric ID to my cell and then associate them in this way. How would I go about adding a unique numeric ID to a cellDetectionObject?

or is it that I have to asume that the order will be the same? so I export them separately and join them outside auming they are exported in the same way?

You could apply the ID as a measurement since it would just be a number

cellNumber = 1
getCellObjects().each{
it.getMeasurementList().putMeasurement("ID", cellNumber)
cellNumber = cellNumber+1
}
fireHierarchyUpdate()

Probably something like that

Access them using something like

getCellObjects().each{
print measurement(it,"ID")
}

In each case, “it” is the current cell in a list of all cells.


You could probably add a property if you edited the source code and rebuilt QuPath, but I wouldn’t recommend it as a first option unless you have a lot of experience with similar projects.

Well, I have not tried it, but it looks like there may be an option to set a property. Have not played with it, not sure what the results would be.
https://github.com/qupath/qupath/blob/43aad4ecda893a7eb03c30774e64da5b9547bc86/qupath-core/src/main/java/qupath/lib/images/ImageData.java#L417
I don’t think it can create a new property, but you might be able to do terrible things with it.

1 Like

Very interesting, thank you. It seems that the setProperty is only for image objects. I think that for now the only solution is to add the ID, which worked, just tried it :slight_smile: thank you.

1 Like

I’m not able to follow this discussion… so if the problem is solved, feel free to ignore the rest :slight_smile:

However, if not then I think it would help to go back to the beginning. There are a couple of complications in the original script:

  • getProperties() isn’t really a standard QuPath thing at all… rather its a somewhat obscure Groovy thing that seems a bit Python-inspired. I really don’t recommend using it.
  • there’s no need for jsonpoly.toString() – since it’s already a String

Before worrying about IDs etc. I’d rather understand exactly what you want to do and why. It sounds like you would be reinventing much of what is already possible by converting a QuPath object to GeoJSON: https://qupath.readthedocs.io/en/latest/docs/advanced/exporting_annotations.html#geojson

This will already include coordinates and measurements – and has the major benefit of being a standardized, open format with a clear specification (unlike .csv for this kind of data) – and therefore you should be able to find libraries to parse it written in lots of languages (e.g. Shapely for Python).

There’s a blog post from @choosehappy describing how GeoJSON written by QuPath can be used elsewhere (with things imported back to QuPath) here: http://www.andrewjanowczyk.com/exporting-and-re-importing-annotations-from-qupath-for-usage-in-machine-learning/

That’s fine, but if anything here doesn’t work then it will be hard to help debug it – I’d recommend moving to the latest stable version if you can.

2 Likes

That explains why the only getProperties I could find was for the image. It did seem weird.

2 Likes

Alright, thank you both. I looked at the posts on the links, on qupath docs and the post from Andrew Janowczyk and tried to do

import static qupath.lib.gui.scripting.QPEx.*
import com.google.gson.GsonBuilder
import qupath.lib.gui.tools.MeasurementExporter

def gsonBuilder = new GsonBuilder();

gsonBuilder.serializeSpecialFloatingPointValues();

def gson = gsonBuilder.setPrettyPrinting().create()
        
def cells = getCellObjects()

File file = new File('C:/myfiles/some.json')
 file.withWriter('UTF-8') {
     gson.toJson(cells,it)
 }

but it attempts to save all the children and parents in a loop and never finishes running. I have the feeling this might have to do something with them being cell objects and not annotations specifically, but I don’t really know.

I think that for now I will export the csv and json files separated and then merge them via a cell ID.

But yes, I can see my intentions are transparent :wink:

Actually just in case, what I really really need to do is to get the pixels that make up the nucleus or the cell and within them, get all the values from all the channels and do some calculations per pixels combining all markers. I am very aware that I can save binary masks but I don’t want to save millions of little masks and millions of little n channel images. I wanted the polygons so I could do this dynamically.

But yeah, thank you both a lot! You are omnipresent, solving questions very quick! hats off.

Yeah I was just using to inspect this class and play around knowing what it was made of and printing things. But yeah it’s a native groovy function. I just really like to go low level haha

I tested this with my own image and aside from removing the first import statement and changing the file location (IT permissions prevented write access to my C drive), it worked. So it doesn’t have anything to do with the cells themselves, though I am not sure how the format handles nuclei vs cytoplasms (I have not checked either). It looks like in the blog they specifically avoided having a cytoplasm, so the objects were simple nuclei.
*Also I had only 6000 cells, they describe 300,000. Not sure how many cells you are processing.

@lesolorzanov I missed that you were using

def gsonBuilder = new GsonBuilder();
gsonBuilder.serializeSpecialFloatingPointValues();
def gson = gsonBuilder.setPrettyPrinting().create()

directly. You shouldn’t; use instead the pattern I use in the documentation

boolean prettyPrint = true
def gson = GsonTools.getInstance(prettyPrint)

to get a properly-initialized Gson instance that knows about QuPath objects and ROIs. But if you have a really large number of cells, Gson may be a bad choice because the file may be very large (turning off pretty printing would help a bit).

That risks ending up manipulating private fields and avoiding all the proper checks QuPath will do for you… potentially resulting in scripts that break very easily. To see what is available, use describe instead, e.g.

def cells = getCellObjects()
println describe(cells[0])
1 Like

I was using the pattern from the documentation but there was a problem with NaNs, and it suggested to use gsonBuilder.serializeSpecialFloatingPointValues(); which is why I decided to use it.

But yeah now I did this: which is exactly what I needed, thank you again for all the information

def cells = getCellObjects()

boolean prettyPrint = true
def gson = GsonTools.getInstance(prettyPrint)
File file = new File('C:/myfiles/some2.json')
file.withWriter('UTF-8') {
     gson.toJson(cells,it)
}

Thanks a million. With this now I get every cell with all the information I need.

I have it per core in a TMA and every core has maybe tops, some 5000 cells so it’s not bad.

2 Likes

Glad it works! For reference, the builder initialised with GsonTools already handles special floating point values:

1 Like