Pixel classifier or Object Classifier ? Which one will better for tissue segmentation in multiplexed images?

Hi Everyone,

I am working on some Breast cancer TMA images with multiplexed staining of few immune markers. In qupath, which classifier would be a better choice for tissue segmentaion ? Pixel classifier or Object Classifier ?

And, with either of them, Is it possible to get the area measurement for segmented areas (tumor/stroma) in each core? I can get the total tumor positive and stromal positive cell numbers for my markers, but not the area measurement of tumor or stromal area. Which is necessary for calculating the density of the immune markers.

Pixel classifier or Thresholder are the main options for tissue segmentation in QuPath - the Object classifier requires objects to classify. You could potentially get a tissue outline using SLICs, but I generally only recommend SLICs for difficult problems - “is there tissue here” is usually not too complicated.

If you have a tissue outline, you can run a second (or third or fourth) pixel classifier inside of it to further split your segmented tissue into something like tumor or stroma. Use the Measure button to add Area measurements to the tissue annotation, or the Classify button to classify cells based on the pixel classifier results. Or you could generate new annotations for the sub-areas.

As for whether or not any of that will work well, it’s hard to say without sample images. I’m not even sure at this point if your sample is brightfield or IF (multiplexed could be either still).

1 Like

Thanks a million for the suggestions ! Sorry for not mentioning earlier, I am working with IF samples.

I have actually sort of succeeded in what I was trying to do. I used the pixel classifier to segment the core into stromal and tumor area (with area measurements). Though I am not happy with the results yet. I have to further work on it. Wondering how many area annotations would be optimal for training the classifier suficiently?

After segmentation, I ran cell detection and its detecting tumor and stromal cell separately as I wanted. Now I want to use single measurement classifier for different immune phenotype detection/classification. Some of the markers also express in tumor cells along with the immune cells. Now If I want to separately classify all the cell objects further into tumor and immune cells how can I do that?

Thanks again!

How many is usually less important than how well chosen. One large circular training area with thousands of pixels can be pretty bad, while 5-6 well chosen short lines with only a couple dozen pixels each can be very good. It is more about covering variety, edge cases, and outliers. Having 90% of your training pixels being the middle of your tissue won’t likely result in good edges.

And I always recommend people check balance classes in Advanced options if they have lopsided training groups (Pie chart in the pixel classifier).

Cell classification can follow the multiplex guide on the main doc. In case your markers are cytoplasmic I’ll also link this. If you want to subdivide already classified cells into “positive” or “negative” for a particular marker, you can do that using setCellIntensityClassifications in the Classify objects menu - otherwise you may be best off designing your classifier through scripting, or creating a measurement to store a previous classifier result (say you create a measurement called “Stroma” which is 1 if the cell was stroma, and 0 if the cell was tumor - prior to it’s current classification as an immune cell or not).

My only recommendation for downstream processing and data wrangling would be to NOT use the exact same class name for tumor cells and tumor annotations. Create a new class called TumorCell or something if you have to, or name the cell by the marker you are using.

1 Like

Thanks a ton! makes a lot of sense. But one thing to ask, from the main multiplex guide what I understood is there are two ways for cell classification. Either with thresholding or Machine learning classifier. But the instructions there are for classifying different marker expressing cells. But if I want to differentiate between tumor and non tumor/immune cells which classifier should I choose ? A separate object classifier using Nucleus/Cell area ratio ?

1 Like

If you have tumor and non tumor regions based on annotations, you don’t need any of that. You would use the Parent field, which should be Tumor or Stroma, depending on which annotation the cell is inside of.

Also, you can combine machine learning classifiers and thresholding classifiers into the composite classifiers, just be careful when you do. So you could have a set of threshold based classifiers for various immune markers and throw a Tumor vs Stroma machine learning classifier into the mix. It will just be tacked on to the end or the beginning of the class name depending on the order of the sub-classifiers.

You can also use the pixel classifier you initially created to apply tumor/stroma classifications to the cells. The pixel classifier has a “Classify” option for detections, like cells. That operates separately from object classifiers and would be overwritten by an object classifier, but it is one way to use a tissue classifier to classify cells.

1 Like

Thanks again !

Firstly, If I just manually annotate a region as tumor, in the TMA measurement output or annotation measurement the cells inside the tumor annotations does not come up as a tumor classified cells. I am not sure if you are talking about the same thing or not.

But anyways, with pixel classifier, yes I can classify cells based on annotations. But the problem is, with markers that express in both tumor and immune cells, It is hard to differentiate between a tumor and an immune cell with only thresholding or ML. Especially for intratumoral immune cells. So that is why I am trying to classify them into tumor and immune cells based on morphology before classifying them based on the marker expression.

Suppose If I want quantify total FOXP3+ immune and tumor cells inside a tumor region, my workflow will look like this -

So the main thing I was wondering is what can I use to differentiate between tumor and immune cells here.

Ah, if you need to apply the measurements as a summary to the TMA, that would take a little more work - but with your diagram it looks pretty easy.

It looks to me like you are using a simple application of the multiplex classifier I linked to above. You have two object classifiers one for tumor/immune, and one for foxp3. Also, I do not recommend using the same class multiple times like this - it is confusing both for the people you are talking to, and likely for downstream analysis. Only use any one class once. So in this case, either label the tumor annotation as a different class, or the tumor cells.

You still will not be applying the measurements to the TMA, you can’t do that by default and it would require scripting. Outputting the cell data for processing will still get you

Parent field: indicates TMA (stroma) or Tumor tissue - you can use this to stratify your data
You will probably need to rename your cells based on the TMA they are inside of, or rename your annotations based on the TMA they are inside of.

Class: some combination of Tumor/Stroma and FOX+/FOX- - Once you have two individual classifiers, the Composite classifier will generate a two part class.

So collecting your data will still require some additional coding, but that’s the basic workflow.

Alternatively - if you literally just want the immune cells and do not care about the tumor cells, why not use your Tumor pixel classifier to classify cells, then Set Cell Intensity Classifications based on the FoxP3? You would end up with tumor or unclassified cells (based on the pixel classifier) and then Positive or Negative based on the FoxP3 threshold you choose. Give that a shot first if it gets you the data you need.

No need for annotations, and the cell counts apply directly to the TMA measurement list.