GPU utilisation

Hi, there,

We have mIF czi whole slide image to analyse for batch processing.
So in order to fasten the process, we would like to utilise GPU instead of CPU.
so we have got the code copied from CLUPATH and tried as suggested.
There is an error below. Can you help us to get it fixed?
Thanks a lot


import qupath.lib.scripting.QP
import qupath.lib.regions.RegionRequest
import net.haesleinhuepf.clupath.CLUPATH
import net.haesleinhuepf.clij.CLIJ

// setup clupath
clupath = CLUPATH.getInstance();
print(clupath.getGPUName());

// get current image from QuPath
def imageData = QP.getCurrentImageData()
server = imageData.getServer();
request = RegionRequest.createInstance(server.getPath(), 1, 0, 0, server.getWidth(), server.getHeight());
image = imageData.getServer();

// push image to GPU
input = clupath.push(image).push(java.awt.image.BufferedImage);
INFO: Quadro RTX 8000
ERROR: Error at line 22: No signature of method: net.haesleinhuepf.clupath.CLUPATH.push() is applicable for argument types: (qupath.lib.images.servers.bioformats.BioFormatsImageServer) values: [Bio-Formats: BioFormatsImageServer: file:/E:/Korean%203IF/AbcamCD105changed_3IF/ADD_105_post3IF_KR40.czi[--series, 0] (ADD_105_post3IF_KR40.czi)]
Possible solutions: push(ij.ImagePlus), push(java.awt.image.BufferedImage), pull(net.haesleinhuepf.clij.clearcl.ClearCLBuffer), use([Ljava.lang.Object;), dump(), with(groovy.lang.Closure)

ERROR: Script error
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:70)
    at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:127)
    at Script12.run(Script12.groovy:23)
    at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:317)
    at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:155)
    at qupath.lib.gui.scripting.DefaultScriptEditor.executeScript(DefaultScriptEditor.java:766)
    at qupath.lib.gui.scripting.DefaultScriptEditor.executeScript(DefaultScriptEditor.java:696)
    at qupath.lib.gui.scripting.DefaultScriptEditor.executeScript(DefaultScriptEditor.java:676)
    at qupath.lib.gui.scripting.DefaultScriptEditor$2.run(DefaultScriptEditor.java:1033)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)

1 Like

Calling @haesleinhuepf:slight_smile:

1 Like

Hey @inhwa,

Jeey, #clij clupath :partying_face:

I assume you’re referring to this code example:

Could you please execute it as it’s written there? Afterwards, could you please call

print(input);

… just to see if this step was working…

Last but not least: clupath was only tested on QuPath 0.2.0-m3 . Some engineering might be necessary to update it to the current QuPath milestone. But I’m happy to do it if you agree to test it and provide feedback :slight_smile: Just let me know!

Cheers,
Robert

Hi @haesleinhuepf

We would love to test anything you can through at us we have RTX 8000 and hundreds (as a starter) of .czi large sections. We already have a QuPath script b want to push it through GPU as with CPU is taken along time. Currenltly In Hwa have got it on m3 but we can try it on other milestones

Mustafa

1 Like

It looks like the key line from Robert’s script is

image = imageData.getServer().readBufferedImage(request);

You need to get the pixels first (as a Java BufferedImage) and send them to the GPU, and not try to send a QuPath ImageServer.

A lot will depend upon what exactly your script does. If it primarily calls existing QuPath things (like cell detection) then I don’t see a way that clupath can help (since those would be very hard to accelerate on the GPU, see here)… but if you can write a clupath-specific way of doing things then it might well help.

1 Like

Hi, @haesleinhuepf
I have just done it.
the error is below.

INFO: Quadro RTX 8000
ERROR: Error at line 14: Dimensions (width=121743 height=87723) are too large

Thanks a lot

1 Like

Ok, can you tell me what code line 14 does?

Hi, Robert @haesleinhuepf,

Here is the code.


import qupath.lib.scripting.QP
import qupath.lib.regions.RegionRequest
import net.haesleinhuepf.clupath.CLUPATH
import net.haesleinhuepf.clij.CLIJ

// setup clupath
clupath = CLUPATH.getInstance();
print(clupath.getGPUName());

// get current image from QuPath
def imageData = QP.getCurrentImageData()
server = imageData.getServer();
request = RegionRequest.createInstance(server.getPath(), 1, 0, 0, server.getWidth(), server.getHeight());
image = imageData.getServer();

// push image to GPU
input = clupath.push(image).push(java.awt.image.BufferedImage);
print(input)

Line 14 is

def imageData = QP.getCurrentImageData()
1 Like

Calling @petebankhead:slight_smile:

Pretty sure it’s possible to retrieve the image tile by tile…

1 Like

Yes! But I return to this…

You can get tiles of the image and send them to clupath… but what happens next? I’m not sure what exactly you want to accelerate.

You can’t use clupath to easily speed up the things already built-in to QuPath… and if you aren’t using QuPath’s built-in processing, what processing do you want to do?

1 Like

@inhwa and @MustafaElshani may know if very detail. But just as a place-holder workflow: What about generating a binary image segmentation of cells and shipping it back tile by tile to QuPath. Do you maybe have an example groovy script which could be used as a blueprint?

Not all in one place yet, but relevant bits of the documentation are:

I’ll have a look at piecing it together quickly now…

1 Like

Awesome, thanks. I’ll dive into updating clupath to QP 0.2.2 :slight_smile:

2 Likes

In the end, I wrote a more elaborate script than I intended… to make it closer to being genuinely useful.

As such, it will take care of lots of awkward things (handling different required resolutions, tile sizes etc.) and you really just need to set the user parameters and override the last method – which requires only knowing a bit of the ImageJ1 API.

2 Likes

Awesome @petebankhead! In the meantime, I upgraded clupath to use clij2 and made it compatible with QuPath 0.2.2:

Please note that the installation instructions changed.

Furthermore, I extended the example code to use clij2 API. Thus, you can now copy code snippets from the clij2 reference and groovy or jython examples right into QuPaths script editor. The new example applys Otsu Thresholding and sends the result back to QuPath as ROI:

@inhwa @MustafaElshani Let me know if this is helpful! I planned to do this long ago and finally had the right motivation. It’s now your turn to put the pieces together :slight_smile: Let us know if you need assistance!
Also big thanks @petebankhead! Your support and the QuPath documentation are awesome :slight_smile:

Cheers,
Robert

6 Likes

Hi Guys

Thank you for all of your efforts great work with the update we are now trying to test however we do get an error, the script is exactly as the one in the git hub. Could it be the .czi image being too large ?

INFO: Quadro RTX 8000
ERROR: IllegalArgumentException at line 37: Couldn't instantiate converter found from class ij.CompositeImage to class net.haesleinhuepf.clij.clearcl.ClearCLBuffer

ERROR: net.haesleinhuepf.clij.converters.FallBackCLIJConverterService.getConverter(FallBackCLIJConverterService.java:33)
    net.haesleinhuepf.clij.CLIJ.convert(CLIJ.java:473)
    net.haesleinhuepf.clij2.CLIJ2.push(CLIJ2.java:96)
    net.haesleinhuepf.clij2.CLIJ2$push$0.call(Unknown Source)
    org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
    Script4.run(Script4.groovy:38)
    org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:317)
    org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:155)
    qupath.lib.gui.scripting.DefaultScriptEditor.executeScript(DefaultScriptEditor.java:926)
    qupath.lib.gui.scripting.DefaultScriptEditor.executeScript(DefaultScriptEditor.java:859)
    qupath.lib.gui.scripting.DefaultScriptEditor.executeScript(DefaultScriptEditor.java:782)
    qupath.lib.gui.scripting.DefaultScriptEditor$2.run(DefaultScriptEditor.java:1271)
    java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    java.base/java.lang.Thread.run(Unknown Source)


1 Like

Hey @inhwa,

it appears the image type you loaded isn’t supported by clij. Could you try some conversion to the image before pushing it to the GPU? E.g.

import ij.IJ;

IJ.run(imp, "RGB Stack", "");

Let us know if it works!

Cheers,
Robert

@inhwa and @haesleinhuepf, if QuPath is giving a CompositeImage, I guess the image is multichannel fluorescence. Conversion to an RGB stack might not be supported.

1 Like

So it be might worth trying this?

IJ.run(imp, "32-bit", "");

Thanks, both @haesleinhuepf, and @petebankhead.
The computer has been used by other stuff so we shall try it and will let you both know.
Thanks so much for your expertise.

1 Like