Export QuPath Annotations for StarDist Training

Dear @oburri , thanks for posting this information. Such a detail normally escapes usual documentation and it can mean whether someone uses the method or not.

Would you by chance share the groovy script to call StarDist from QuPath? I can imagine many people are looking forward to try that out!
thanks again,
Alvaro

Dear @acrevenna,

Your post was not directly part of the previous topic so it has been moved to a new one.

As per the requests, here is the script we use to export annotations for Stardist

Please note that it makes use of a QuPath Extension we developed for some internal actions

To install that extension, please download the .jar file and drag and drop it into QuPath.

Tested on QuPath 0.1.3 and QuPath 0.2.0-m11

All the best

Oli

5 Likes

Hopefully you don’t mind, but I adjusted the script slightly so that it seeeeeems to work without the BIOP extension, for anyone that wants to stick to vanilla. I am new to the integration between StarDist and QuPath, and realize it currently takes a special build to actually call StarDist, so I haven’t really gone though all of the steps just yet. Using this modified script may be a terrible idea, down the line.

*Not extensively tested, but outputs masks without errors in a couple of trials.

/*
// ABOUT 
Exports Annotations for StarDist (Or other Deep Learning frameworks) 

// REQUIREMENTS
You will need to install the BIOP Extension for QuPath which contains methods needed to run this code
https://github.com/BIOP/qupath-biop-extensions/releases/tag/v2.0.0


// INPUTS
You need rectangular annotations that have classes "Training" and "Validation"
After you have placed these annotations, lock them and start drawing the objects inside

// OUTPUTS
----------
The script will export each annotation and whatever is contained within as an image-label pair
These will be placed in the folder specified by the user in the main project directory.
Inside that directory, you will find 'train' and 'test' directories that contain the images with 
class 'Training' and 'Validation', respectively. 
Inside each, you will find 'images' and 'masks' folders containing the exported image and the labels, 
respectively. The naming convention was chosen to match the one used for the StarDist DSBdataset

//PARAMETERS
------------
- channel_of_interest: You can export a single channel or all of them, currently no option for _some_channels only
- downsample: you can downsample your image in case it does not make sense for you to train on the full resolution
- export_directory: name of the directory which will contain the 'train' and 'test' subdirectories

Authors: Olivier Burri, Romain Guiet BioImaging and Optics Platform (EPFL BIOP)

Tested on QuPath 0.2.0-m11, May 6th 2020

Due to the simple nature of this code, no copyright is applicable
*/

// USER SETTINGS
def channel_of_interest = 1 // null to export all the channels 
def downsample = 1


// START OF SCRIPT

def training_regions = getAnnotationObjects().findAll { it.getPathClass() == getPathClass("Training") }

def validation_regions = getAnnotationObjects().findAll { it.getPathClass() == getPathClass("Validation") }

if (training_regions.size() > 0 ) saveRegions( training_regions, channel_of_interest, downsample, 'train')

if (validation_regions.size() > 0 ) saveRegions( validation_regions, channel_of_interest, downsample, 'test')





def saveRegions( def regions, def channel, def downsample, def type ) {
    // Randomize names
    def is_randomized = getProject().getMaskImageNames()
    getProject().setMaskImageNames(true)
    def rm = RoiManager.getRoiManager() ?: new RoiManager()
    // Get the image name
    def image_name = getProjectEntry().getImageName()
    regions.eachWithIndex{ region, region_idx ->
        println("Processing Region #"+(  region_idx + 1 ) )
        
        def file_name =  image_name+"_r"+( region_idx + 1 )  
        imageData = getCurrentImageData();
        server = imageData.getServer();
        viewer = getCurrentViewer();
        hierarchy = getCurrentHierarchy();

        //def image = GUIUtils.getImagePlus( region, downsample, false, true )
        request = RegionRequest.createInstance(imageData.getServerPath(), downsample, region.getROI())
        pathImage = null;
        pathImage = IJExtension.extractROIWithOverlay(server, region, hierarchy, request, false, viewer.getOverlayOptions());
        image = pathImage.getImage()
        println("Image received" )
        //image.show()
        // Create the Labels image
        def labels = IJ.createImage( "Labels", "16-bit black", image.getWidth(), image.getHeight() ,1 );
        rm.reset()
        
        IJ.run(image, "To ROI Manager", "")
        
        def rois = rm.getRoisAsArray() as List
        println("Creating Labels" )
        
        def label_ip = labels.getProcessor()
        def idx = 0
        rois.each{ roi ->
            if (roi.getType() == Roi.RECTANGLE) {
                println("Ignoring Rectangle")
            } else {
                label_ip.setColor( ++idx )
                label_ip.setRoi( roi )
                label_ip.fill( roi )


            }
        }
        labels.setProcessor( label_ip )
        
        //labels.show()
        
        // Split to keep only channel of interest
        def output = image
        if  ( channel != null){
            imp_chs =  ChannelSplitter.split( image )
            output = imp_chs[  channel - 1 ]
        }
        
      
        
        saveImages(output, labels, file_name, type)
                
        println( file_name + " Image and Mask Saved." )
        
        // Save some RAM
        output.close()
        labels.close()
        image.close()
    }
    
    // Return Project setup as it was before
    getProject().setMaskImageNames( is_randomized )
}

// This will save the images in the selected folder
def saveImages(def images, def labels, def name, def type) {
    def source_folder = new File ( buildFilePath( PROJECT_BASE_DIR, 'ground_truth', type, 'images' ) )
    def target_folder = new File ( buildFilePath( PROJECT_BASE_DIR, 'ground_truth', type, 'masks' ) )
    mkdirs( source_folder.getAbsolutePath() )
    mkdirs( target_folder.getAbsolutePath() )
    
    IJ.save( images , new File ( source_folder, name ).getAbsolutePath()+'.tif' )
    IJ.save( labels , new File ( target_folder, name ).getAbsolutePath()+'.tif' )

}

// Manage Imports
import qupath.lib.roi.RectangleROI
import qupath.imagej.gui.IJExtension;
import ij.IJ
import ij.gui.Roi
import ij.plugin.ChannelSplitter
import ij.plugin.frame.RoiManager
print "done"
4 Likes

Hello,

Looks good to me :slight_smile:

If you want to try another way of using StarDist in QuPath, let me know.

1 Like

Sounds good, I think that was what the above user was asking about, though. The script that called StarDist, rather than generating the training data (which is also important).

@acrevenna it’s not @oburri’s way, but there’s this: https://qupath.readthedocs.io/en/latest/docs/advanced/stardist.html

3 Likes

Just found that through the question and answer section!

Since the TensorFlow flag is set during the build, is the plan to have it default to CPU for future builds (M12?) or default to off and always require a build your own in order to use TS related features and scripts?

I can’t answer that; introducing new features has proven much easier than refining and supporting them, and I don’t want to commit to anything that increases expectation and pressure further.

For now it is what it is… but certainly will become more.

2 Likes

21 posts were split to a new topic: Exporting annotations - discussion about format

Hi @petebankhead,
I’m very curious to try StarDist in QuPath!

I followed your instructions from qupath.readthedocs
I cloned the github repo, installed OpenJDK 14 (Latest) Hotspot (default)
I managed to build :

BUILD SUCCESSFUL in 1m 8s
71 actionable tasks: 59 executed, 12 up-to-date

Found within the build folder a qupath-0.2.0-m12-SNAPSHOT.jar (I was expecting a *.exe)

Downloaded the model and modified your script accordingly, but when I try to run it, QuPath stops.
Within the build folder I found a log file mentioning :

A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffe607d45b6, pid=21960, tid=54712
...

Would you have any hints to solve the issue?

Thank you for your help,

Best,

Romain

Hi @romainGuiet, it might be this – but it sounds like it was failing for you before getting that far?

So long as you include the createPackage parameter when building, you should get an .exe. If you didn’t, then I think that’s the thing to try to address first.

Apart from that, not sure if you used tensorflow-cpu or tensorflow-gpu… I’d definitely recommend the former, at least until it is running properly.

I haven’t seen the errors myself, and I know at least a few people have got it running, but I haven’t had much time to look at the TensorFlow stuff and won’t until after v0.2.0 is ‘final’… best think of it as a very early preview, inspired by the timing of the NEUBIAS webinars :slight_smile:

1 Like

Hi @petebankhead,
thank you for your reply!

  • So long as you include the createPackage parameter when building, you should get an .exe. If you didn’t, then I think that’s the thing to try to address first.

This is solved now :smiley:, @Research_Associate pointed me to the right folder! Silly me :upside_down_face:

  • Since you mentioned an issue opened about large image, I tried to increase the Maximum memory, without any difference, QuPath still stops… Furthermore, in my case, the image is not large ( a 3 channels tif, 512x512, 8 bit)

  • not sure if you used tensorflow-cpu or tensorflow-gpu

I followed the recommendation from the readtehdoc and I used cpu for now (going one step at a time )

Best,

Romain

1 Like

Oh, I really don’t think it’s an AdoptOpenJDK problem at all – it might be a TensorFlow or JavaCPP problem, but I doubt they will be very interested either unless it can be replicated without the QuPath dependency.

Basically, although the error happens outside QuPath, it ultimately comes via some pathway that QuPath has called… and since it’s in ‘native’ code, the precise interplay of your hardware/setup can matter too.

AdoptOpenJDK isn’t the right place to report it, but I’d avoid reporting it as a bug to any of the aforementioned projects unless you can replicate the error without QuPath.

It will be really tricky to track down precisely what has gone wrong – and the reason StarDist/TensorFlow isn’t part of QuPath yet is that I really just don’t have time to support all the expected issues.

If I start encountering the problem myself then I’ll be in a better position to do something about it… but that requires finding some time to start using QuPath myself for some relevant projects :slight_smile:

Ok, had a quick look at a can replicate the error if I give a path that is anything other than a valid path to an unzipped folder containing the StarDist model.

I can add more sanity-checks to the code to try to end less catastrophically. In the meantime, can you confirm what path you used in the script?

1 Like

That was it !
Stupid me :expressionless: the folder was containing the zip file, but I didn’t think about unzipping it (as some tools use the zip file directly) while tit is written in the doc ! sorry, sorry :expressionless:

Works like a charm now!

Congrats @petebankhead and sorry for making you lose your time :upside_down_face:

It’s time for me to test on larger images :wink:

Best,

Romain

Nooo, not larger ones!
No worries, I’ll change the code before the next release so that when it fails (at least at this stage) it doesn’t always take down the entire Java Virtual Machine :slight_smile:

1 Like

Works very well on “large” image : ome.tiff , 21815 x 16244, 15000 nuclei in 40sec

Maybe you can use bold text for the “unzip models” in the documentation for people like me :sweat_smile:

Cheers,

Romain

3 Likes