CLIJ deconvolution

Hi @bnorthan,
happy to help!
Thank you also for the short explanation! It is very useful to me, since I indeed do not have any formal education in signal processing :sweat_smile:
looking forward to the next awesome versions!

1 Like

@CellKai and @haesleinhuepf

I’ve pulled @haesleinhuepf’s changes into the main ops-experiments repository and I’ve written an example that performs PSF preprocessing, padding, CLIJ FFT deconvolution, and un-padding all on the CElegans image.

Let me know if you have questions about the example.

If padding is done properly we shouldn’t see the artifact in the middle slice. So, next step would be to integrate proper padding and unpadding into CLIJ and/or related scripts.

This could be done by moving some of my code to user-friendly “utilities” (which I am happy to do) and/or by implementing unpadding utilities directly on the GPU (which I’m happy to help with where I can).

Brian

2 Likes

Hi Brian,

I was wondering how to run your example - does it need to be built from an IDE or from the command line?

I tried to run it a script in Fiji but it return this error

javax.script.ScriptException: Class net.haesleinhuepf.clijx.plugins.InteractiveDeconvolveFFTTest in invalid directory:

When I comment out the first line (package net.haesleinhuepf.clijx.plugins;) it continues on a bit more and gives this error (amongst a lot of path information):

Trying to download http://repo1.maven.org/maven2//jfree/jcommon/1.0.17/jcommon-1.0.17.pom.sha1
java.io.FileNotFoundException: Could not download jcommon-1.0.17.jar

// bunch if class path stuff

12: error: package net.imagej.ops.experiments does not exist
import net.imagej.ops.experiments.ImageUtility;

13: error: package net.imagej.ops.experiments.testImages does not exist
import net.imagej.ops.experiments.testImages.Bars;

Thanks,

Chris

3 Likes

Hey @evenhuis,

can you try it from Fiji with the two update sites “clij” and “clij2” activated? This should deliver the ops-experiments.jar. Not sure if it’s the right version though.

If it’s still not working, please share the script you were executing.

Let us know if this helps!

Cheers,
Robert

1 Like

Hi @haesleinhuepf,

I’ve added clij2 through the update site.

Here’s the script - I just changed the file locations:


package net.haesleinhuepf.clijx.plugins;

import java.io.IOException;

import net.haesleinhuepf.clij.CLIJ;
import net.haesleinhuepf.clij.clearcl.ClearCLBuffer;
import net.haesleinhuepf.clij2.CLIJ2;
import net.imagej.Dataset;
import net.imagej.ImageJ;
import net.imagej.ops.Ops;
import net.imagej.ops.experiments.ImageUtility;
import net.imagej.ops.experiments.testImages.Bars;
import net.imagej.ops.experiments.testImages.DeconvolutionTestData;
import net.imglib2.FinalDimensions;
import net.imglib2.Interval;
import net.imglib2.RandomAccessibleInterval;
import net.imglib2.img.Img;
import net.imglib2.outofbounds.OutOfBoundsMirrorFactory;
import net.imglib2.type.NativeType;
import net.imglib2.type.numeric.RealType;
import net.imglib2.type.numeric.real.FloatType;
import net.imglib2.util.Intervals;
import net.imglib2.view.Views;
import net.imagej.ops.filter.pad.DefaultPadInputFFT;

public class InteractiveDeconvolveFFTTest<T extends RealType<T> & NativeType<T>> {

	final static ImageJ ij = new ImageJ();

	public static <T extends RealType<T> & NativeType<T>> void main(
		final String[] args) throws IOException
	{
		// check the library path, can be useful for debugging
		System.out.println(System.getProperty("java.library.path"));

		// launch IJ so we can interact with the inputs and outputs
		ij.launch(args);

		// test names
		Dataset testData = (Dataset) ij.io().open(
			"/Users/evenhuis/Downloads/CElegans-CY3.tif");
		Dataset psf = (Dataset) ij.io().open(
			"/Users/evenhuis/Downloads/CElegans_April_2020/PSF-CElegans-CY3.tif");

		// open the test data
		RandomAccessibleInterval<FloatType> imgF = (RandomAccessibleInterval) (ij
			.op().convert().float32((Img) testData.getImgPlus().getImg()));
		
		RandomAccessibleInterval<FloatType> psfF = (RandomAccessibleInterval) (ij
			.op().convert().float32((Img) psf.getImgPlus()));

		// crop PSF - the image will be extended using PSF size
		// if the PSF size is too large it will explode image size, memory needed and processing speed
		// so crop just small enough to capture significant signal of PSF 
		psfF = ImageUtility.cropSymmetric(psfF,
				new long[] { 64, 64, 41 }, ij.op());

		// subtract min from PSF		
		psfF = Views.zeroMin(ImageUtility.subtractMin(psfF, ij.op()));

		// normalize PSF
		psfF = Views.zeroMin(ImageUtility.normalize(psfF, ij.op()));

		// compute extended dimensions based on image and PSF dimensions
		long[] extendedSize = new long[imgF.numDimensions()];

		for (int d = 0; d < imgF.numDimensions(); d++) {
			extendedSize[d] = imgF.dimension(d) + psfF.dimension(d);
		}

		FinalDimensions extendedDimensions = new FinalDimensions(extendedSize);

		// extend image
		RandomAccessibleInterval<FloatType> extended = (RandomAccessibleInterval) ij
			.op().run(DefaultPadInputFFT.class, imgF, extendedDimensions, false,
				new OutOfBoundsMirrorFactory(OutOfBoundsMirrorFactory.Boundary.SINGLE));
 
		// show image and PSF
		ij.ui().show("img ", imgF);
		ij.ui().show("psf ", psfF);
		
		// get clij
		CLIJ2 clij2 = CLIJ2.getInstance("RTX");

		// push extended image and psf to GPU
		ClearCLBuffer inputGPU = clij2.push(extended);
		ClearCLBuffer psfGPU = clij2.push(psfF);

		// create output
		ClearCLBuffer output = clij2.create(inputGPU);

		// deconvolve
		DeconvolveFFT.deconvolveFFT(clij2, inputGPU, psfGPU, output);
		
		// get deconvolved as an RAI
		RandomAccessibleInterval deconv=clij2.pullRAI(output);
		
		// create unpadding interval
		Interval interval = Intervals.createMinMax(-extended.min(0), -extended
			.min(1), -extended.min(2), -extended.min(0) + imgF.dimension(0) -
				1, -extended.min(1) + imgF.dimension(1) - 1, -extended.min(2) +
					imgF.dimension(2) - 1);

		// create an RAI for the output... we could just use a View to unpad, but performance for slicing is slow 
		RandomAccessibleInterval outputRAI=ij.op().create().img(imgF);
		
		// copy the unpadded interval back to original size
		ij.op().run(Ops.Copy.RAI.class, outputRAI, Views.zeroMin(Views.interval(deconv,
			interval)));
		
		//clij2.show(output, "output");
		ij.ui().show("deconvolved and unpadded", outputRAI);

	}
}

As is it gives this error:

javax.script.ScriptException: Class net.haesleinhuepf.clijx.plugins.InteractiveDeconvolveFFTTest in invalid directory: /Users/evenhuis/Downloads/InteractiveDeconvolveFFTTest.java
	at org.scijava.plugins.scripting.java.JavaEngine.getMavenProject(JavaEngine.java:521)
	at org.scijava.plugins.scripting.java.JavaEngine.access$700(JavaEngine.java:96)
	at org.scijava.plugins.scripting.java.JavaEngine$Builder.initialize(JavaEngine.java:405)
	at org.scijava.plugins.scripting.java.JavaEngine$Builder.access$100(JavaEngine.java:364)
	at org.scijava.plugins.scripting.java.JavaEngine.compile(JavaEngine.java:197)
	at org.scijava.plugins.scripting.java.JavaEngine.eval(JavaEngine.java:136)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
java.lang.IllegalArgumentException: Unknown code type: null
	at org.scijava.run.RunService.run(RunService.java:63)
	at org.scijava.plugins.scripting.java.JavaEngine.eval(JavaEngine.java:137)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

If I comment out the package line I get his in the console:

java.io.IOException: SHA1 mismatch: /Users/evenhuis/.m2/repository/jfree/jcommon/1.0.17/jcommon-1.0.17.pom.sha1.new: c1 != 25 (actual SHA-1: 25b55acf64649cb2c5712c7d5da21473174ff1f1)
	at org.scijava.minimaven.BuildEnvironment.downloadAndVerify(BuildEnvironment.java:488)
	at org.scijava.minimaven.BuildEnvironment.downloadAndVerify(BuildEnvironment.java:452)
	at org.scijava.minimaven.BuildEnvironment.downloadAndVerify(BuildEnvironment.java:443)
	at org.scijava.minimaven.MavenProject.download(MavenProject.java:190)
	at org.scijava.minimaven.MavenProject.maybeDownloadAutomatically(MavenProject.java:1372)
	at org.scijava.minimaven.MavenProject.findPOM(MavenProject.java:1305)
	at org.scijava.minimaven.MavenProject.getDependencies(MavenProject.java:952)
	at org.scijava.minimaven.MavenProject.getDependencies(MavenProject.java:963)
	at org.scijava.minimaven.MavenProject.getDependencies(MavenProject.java:894)
	at org.scijava.minimaven.MavenProject.checkUpToDate(MavenProject.java:220)
	at org.scijava.minimaven.MavenProject.upToDate(MavenProject.java:206)
	at org.scijava.minimaven.MavenProject.build(MavenProject.java:481)
	at org.scijava.minimaven.MavenProject.build(MavenProject.java:463)
	at org.scijava.minimaven.MavenProject.build(MavenProject.java:446)
	at org.scijava.plugins.scripting.java.JavaEngine.compile(JavaEngine.java:209)
	at org.scijava.plugins.scripting.java.JavaEngine.eval(JavaEngine.java:136)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
java.io.IOException: Server returned HTTP response code: 501 for URL: http://repo1.maven.org/maven2//jfree/jcommon/1.0.17/jcommon-1.0.17.pom.sha1
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
	at org.scijava.minimaven.BuildEnvironment.download(BuildEnvironment.java:611)
	at org.scijava.minimaven.BuildEnvironment.downloadAndVerify(BuildEnvironment.java:462)
	at org.scijava.minimaven.BuildEnvironment.downloadAndVerify(BuildEnvironment.java:452)
	at org.scijava.minimaven.BuildEnvironment.downloadAndVerify(BuildEnvironment.java:443)
	at org.scijava.minimaven.MavenProject.download(MavenProject.java:190)
	at org.scijava.minimaven.MavenProject.maybeDownloadAutomatically(MavenProject.java:1372)
	at org.scijava.minimaven.MavenProject.findPOM(MavenProject.java:1305)
	at org.scijava.minimaven.MavenProject.getDependencies(MavenProject.java:952)
	at org.scijava.minimaven.MavenProject.getDependencies(MavenProject.java:963)
	at org.scijava.minimaven.MavenProject.getDependencies(MavenProject.java:894)
	at org.scijava.minimaven.MavenProject.checkUpToDate(MavenProject.java:220)
	at org.scijava.minimaven.MavenProject.upToDate(MavenProject.java:206)
	at org.scijava.minimaven.MavenProject.build(MavenProject.java:481)
	at org.scijava.minimaven.MavenProject.build(MavenProject.java:463)
	at org.scijava.minimaven.MavenProject.build(MavenProject.java:446)
	at org.scijava.plugins.scripting.java.JavaEngine.compile(JavaEngine.java:209)
	at org.scijava.plugins.scripting.java.JavaEngine.eval(JavaEngine.java:136)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

and this is in error log:

Skipping artifact jfree:jcommon:1.0.17 (for org.scijava.scripting.java:jfreechart:1.0.0): not found
Trying to download http://maven.imagej.net/service/local/repo_groups/public/content//jfree/jcommon/1.0.17/jcommon-1.0.17.pom.sha1
Downloading http://maven.imagej.net/service/local/repo_groups/public/content//jfree/jcommon/1.0.17/jcommon-1.0.17.pom.sha1 to /Users/evenhuis/.m2/repository/jfree/jcommon/1.0.17/jcommon-1.0.17.pom.sha1.new
Trying to download http://maven.imagej.net/service/local/repo_groups/public/content//jfree/jcommon/1.0.17/jcommon-1.0.17.pom
Downloading http://maven.imagej.net/service/local/repo_groups/public/content//jfree/jcommon/1.0.17/jcommon-1.0.17.pom to /Users/evenhuis/.m2/repository/jfree/jcommon/1.0.17/jcommon-1.0.17.pom.new
Trying to download http://repo1.maven.org/maven2//jfree/jcommon/1.0.17/jcommon-1.0.17.pom.sha1
java.io.FileNotFoundException: Could not download jcommon-1.0.17.jar
	at org.scijava.minimaven.MavenProject.download(MavenProject.java:198)
	at org.scijava.minimaven.MavenProject.maybeDownloadAutomatically(MavenProject.java:1372)
	at org.scijava.minimaven.MavenProject.findPOM(MavenProject.java:1305)
	at org.scijava.minimaven.MavenProject.getDependencies(MavenProject.java:952)
	at org.scijava.minimaven.MavenProject.getDependencies(MavenProject.java:963)
	at org.scijava.minimaven.MavenProject.getDependencies(MavenProject.java:894)
	at org.scijava.minimaven.MavenProject.checkUpToDate(MavenProject.java:220)
	at org.scijava.minimaven.MavenProject.upToDate(MavenProject.java:206)
	at org.scijava.minimaven.MavenProject.build(MavenProject.java:481)
	at org.scijava.minimaven.MavenProject.build(MavenProject.java:463)
	at org.scijava.minimaven.MavenProject.build(MavenProject.java:446)
	at org.scijava.plugins.scripting.java.JavaEngine.compile(JavaEngine.java:209)
	at org.scijava.plugins.scripting.java.JavaEngine.eval(JavaEngine.java:136)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Could not download jcommon: Could not download jcommon-1.0.17.jar
Skipping artifact com.sun:tools:1.4.2 (for org.scijava.scripting.java:ij:1.0.0): not found

Cheers,

Chris

1 Like

I see. I think the Java compiler in ImageJ/Fiji is not capable of running this advanced stuff. So either you switch to an IDE, or alternatively, you use Groovy or Jython scripting in Fiji. For example, this Jython script might work on your machine:

#@ IOService io 
#@ OpService op
#@ UIService ui

from net.haesleinhuepf.clij2 import CLIJ2;
from net.haesleinhuepf.clijx.plugins import DeconvolveFFT;


from net.imagej.ops.experiments import ImageUtility;
from net.imglib2.view import Views;
from net.imglib2 import FinalDimensions;
from net.imagej.ops.filter.pad import DefaultPadInputFFT;
from net.imglib2.outofbounds import OutOfBoundsMirrorFactory;
from net.imglib2.util import Intervals;
from net.imagej.ops import Ops;


# test names
testData = io.open("/Users/evenhuis/Downloads/CElegans-CY3.tif");
psf = io.open("/Users/evenhuis/Downloads/CElegans_April_2020/PSF-CElegans-CY3.tif");

# open the test data
imgF = op.convert().float32(testData.getImgPlus().getImg());
psfF = op.convert().float32(psf.getImgPlus());

# crop PSF - the image will be extended using PSF size
# if the PSF size is too large it will explode image size, memory needed and processing speed
# so crop just small enough to capture significant signal of PSF 
psfF = ImageUtility.cropSymmetric(psfF, [ 64, 64, 41 ], op);

# subtract min from PSF		
psfF = Views.zeroMin(ImageUtility.subtractMin(psfF, op));

# normalize PSF
psfF = Views.zeroMin(ImageUtility.normalize(psfF, op));

# compute extended dimensions based on image and PSF dimensions
extendedSize = range(0, imgF.numDimensions());

for d in range( 0, imgF.numDimensions()):
	extendedSize[d] = imgF.dimension(d) + psfF.dimension(d);


extendedDimensions = FinalDimensions(extendedSize);

# extend image
extended = op.run(DefaultPadInputFFT, imgF, extendedDimensions, false,
		OutOfBoundsMirrorFactory(OutOfBoundsMirrorFactory.Boundary.SINGLE));

# show image and PSF
ui.show("img ", imgF);
ui.show("psf ", psfF);

# get clij
clij2 = CLIJ2.getInstance("RTX");

# push extended image and psf to GPU
inputGPU = clij2.push(extended);
psfGPU = clij2.push(psfF);

# create output
output = clij2.create(inputGPU);

# deconvolve
DeconvolveFFT.deconvolveFFT(clij2, inputGPU, psfGPU, output);

# get deconvolved as an RAI
deconv = clij2.pullRAI(output);

# create unpadding interval
interval = Intervals.createMinMax(-extended.min(0), -extended
	.min(1), -extended.min(2), -extended.min(0) + imgF.dimension(0) -
		1, -extended.min(1) + imgF.dimension(1) - 1, -extended.min(2) +
			imgF.dimension(2) - 1);

# create an RAI for the output... we could just use a View to unpad, but performance for slicing is slow 
outputRAI = op.create().img(imgF);

# copy the unpadded interval back to original size
op.run(Ops.Copy.RAI, outputRAI, Views.zeroMin(Views.interval(deconv, interval)));

# clij2.show(output, "output");
ui.show("deconvolved and unpadded", outputRAI);

Could you please try?

Cheers,
Robert

2 Likes

Hi @haesleinhuepf,

the python script works! This would have been my preference anyway as I’ve always been put off by the verbosity of java (although typing would nice) and never been able work out the build process.

Cheers,

Chris

2 Likes

Hi everyone in the thread!

First of all thanks for the amazing work you are doing on implementing the GPU approach to imageJ, I have great hopes that this will allow us to speed things up and perform much more complex analyses in the near future.

Second, I’m a cell biologist… so I’ll try my best in getting my message across but excuse me if I make any mistakes in the technicalities of the topic.

I’m trying to deconvolve a series of 3D images taken in a Leica SP5 confocal microscope using ImageJ. I’ve tried the DeconvolutionLab2 plugin’s Richardson-Lucy algorithm and found out it worked best at 100 iterations for my sample. I’m happy with the result, but my computer takes approximately 2h to deconvolve a single channel of a 37 slice cropped image of 500x500 pixels, so I can’t begin to imagine how long will it take for the whole image that is 3296x3296 :upside_down_face: (The size is because I’m working with nanoparticles and we are trying to achieve as much resolution as possible with the current equipment setting)

So I’ve found out CLIJ is able to perform this kind of process taking advantage of the GPU instead of the CPU in my computer, and that might speed things up.

My computer is a MacBook Pro 16’ (2019) with Intel® Core™ i9-9980HK CPU @ 2.40GHz, Intel® UHD Graphics 630 and an AMD Radeon Pro 5500M Compute Engine with 8 GB VRAM.

I’ve been trying to test the CLIJDecon.py that was posted by @bnorthan here, with a modified version (clij = CLIJ.getInstance("VEGA"); instead of clij = CLIJ.getInstance("RTX"); to call my GPU:

#@ OpService ops
#@ UIService ui
#@ Dataset data
#@ Dataset psf

from net.imglib2.type.numeric.real import FloatType
from net.imglib2.view import Views;
from net.imagej.ops.experiments import ConvertersUtility
from net.imagej.ops.experiments.filter.deconvolve import OpenCLFFTUtility
from net.imglib2 import FinalDimensions;

from java.lang import System

# init CLIJ and GPU
from net.haesleinhuepf.clij import CLIJ;

print CLIJ.getAvailableDeviceNames();

clij = CLIJ.getInstance("VEGA");

psfF=ops.convert().float32(psf);
imgF=ops.convert().float32(data);

# transfer image to the GPU
#gpuImg= clij.push(imgF);

# now call the function that pads to a supported size and pushes to the GPU
gpuImg = OpenCLFFTUtility.padInputFFTAndPush(imgF, imgF, ops, clij);

# now call the function that pads to a supported size and pushes to the GPU 
gpuPSF = OpenCLFFTUtility.padKernelFFTAndPush(psfF, FinalDimensions(gpuImg.getDimensions()), ops, clij);

# call decon passing the image as a CLBuffer but the PSF as 
# a java RAI, (there needs to be some preprocessing done on the PSF
# and runDecon does that on the CPU in java)
start = System.currentTimeMillis();
gpuEstimate = OpenCLFFTUtility.runDecon(gpuImg, gpuPSF);
finish = System.currentTimeMillis();

print('CLIJ decon time ', (finish-start));
clij.show(gpuEstimate, "GPU Decon Result");

But I get the following error:

Started CLIJDecon.py at Tue May 26 17:40:25 CEST 2020
Traceback (most recent call last):
  File "/Users/toniaranda/Desktop/CLIJ_Python/CLIJDecon.py", line 37, in <module>
    # compute extended dimensions based on image and PSF dimensions
TypeError: runDecon(): expected 4 args; got 2

	at org.python.core.Py.TypeError(Py.java:265)
	at org.python.core.PyReflectedFunction.throwError(PyReflectedFunction.java:211)
	at org.python.core.PyReflectedFunction.throwArgCountError(PyReflectedFunction.java:264)
	at org.python.core.PyReflectedFunction.throwError(PyReflectedFunction.java:321)
	at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:169)
	at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:206)
	at org.python.core.PyObject.__call__(PyObject.java:497)
	at org.python.core.PyObject.__call__(PyObject.java:501)
	at org.python.pycode._pyx6.f$0(/Users/toniaranda/Desktop/CLIJ_Python/CLIJDecon.py:41)
	at org.python.pycode._pyx6.call_function(/Users/toniaranda/Desktop/CLIJ_Python/CLIJDecon.py)
	at org.python.core.PyTableCode.call(PyTableCode.java:171)
	at org.python.core.PyCode.call(PyCode.java:18)
	at org.python.core.Py.runCode(Py.java:1614)
	at org.python.core.__builtin__.eval(__builtin__.java:497)
	at org.python.core.__builtin__.eval(__builtin__.java:501)
	at org.python.util.PythonInterpreter.eval(PythonInterpreter.java:259)
	at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:57)
	at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:31)
	at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

It seems that the command runDecon() is expecting a higher number of arguments (4) than it has been reached (it says it just got 2) and I don’t know what to do about it or if this may be caused by some non-related dependencies.

I’ve also tried @haesleinhuepf approach using the Jython code’s modified version too hoping it would work on my machine:

#@ IOService io 
#@ OpService op
#@ UIService ui

from net.haesleinhuepf.clij2 import CLIJ2;
from net.haesleinhuepf.clijx.plugins import DeconvolveFFT;


from net.imagej.ops.experiments import ImageUtility;
from net.imglib2.view import Views;
from net.imglib2 import FinalDimensions;
from net.imagej.ops.filter.pad import DefaultPadInputFFT;
from net.imglib2.outofbounds import OutOfBoundsMirrorFactory;
from net.imglib2.util import Intervals;
from net.imagej.ops import Ops;


# test names
testData = io.open("/Users/toniaranda/Desktop/Images/CElegans-CY3.tif");
psf = io.open("/Users/toniaranda/Desktop/Images/PSF-CElegans-CY3.tif");

# open the test data
imgF = op.convert().float32(testData.getImgPlus().getImg());
psfF = op.convert().float32(psf.getImgPlus());

# crop PSF - the image will be extended using PSF size
# if the PSF size is too large it will explode image size, memory needed and processing speed
# so crop just small enough to capture significant signal of PSF 
psfF = ImageUtility.cropSymmetric(psfF, [ 64, 64, 41 ], op);

# subtract min from PSF		
psfF = Views.zeroMin(ImageUtility.subtractMin(psfF, op));

# normalize PSF
psfF = Views.zeroMin(ImageUtility.normalize(psfF, op));

# compute extended dimensions based on image and PSF dimensions
extendedSize = range(0, imgF.numDimensions());

for d in range( 0, imgF.numDimensions()):
	extendedSize[d] = imgF.dimension(d) + psfF.dimension(d);


extendedDimensions = FinalDimensions(extendedSize);

# extend image
extended = op.run(DefaultPadInputFFT, imgF, extendedDimensions, false,
		OutOfBoundsMirrorFactory(OutOfBoundsMirrorFactory.Boundary.SINGLE));

# show image and PSF
ui.show("img ", imgF);
ui.show("psf ", psfF);

# get clij
clij2 = CLIJ2.getInstance("VEGA");

# push extended image and psf to GPU
inputGPU = clij2.push(extended);
psfGPU = clij2.push(psfF);

# create output
output = clij2.create(inputGPU);

# deconvolve
DeconvolveFFT.deconvolveFFT(clij2, inputGPU, psfGPU, output);

# get deconvolved as an RAI
deconv = clij2.pullRAI(output);

# create unpadding interval
interval = Intervals.createMinMax(-extended.min(0), -extended
	.min(1), -extended.min(2), -extended.min(0) + imgF.dimension(0) -
		1, -extended.min(1) + imgF.dimension(1) - 1, -extended.min(2) +
			imgF.dimension(2) - 1);

# create an RAI for the output... we could just use a View to unpad, but performance for slicing is slow 
outputRAI = op.create().img(imgF);

# copy the unpadded interval back to original size
op.run(Ops.Copy.RAI, outputRAI, Views.zeroMin(Views.interval(deconv, interval)));

# clij2.show(output, "output");
ui.show("deconvolved and unpadded", outputRAI);

But without luck too:

Started Jython.py at Tue May 26 17:52:31 CEST 2020
Traceback (most recent call last):
  File "/Users/toniaranda/Desktop/CLIJ_Python/Jython.py", line 47, in <module>
NameError: name 'false' is not defined

	at org.python.core.Py.NameError(Py.java:290)
	at org.python.core.PyFrame.getname(PyFrame.java:257)
	at org.python.pycode._pyx9.f$0(/Users/toniaranda/Desktop/CLIJ_Python/Jython.py:83)
	at org.python.pycode._pyx9.call_function(/Users/toniaranda/Desktop/CLIJ_Python/Jython.py)
	at org.python.core.PyTableCode.call(PyTableCode.java:171)
	at org.python.core.PyCode.call(PyCode.java:18)
	at org.python.core.Py.runCode(Py.java:1614)
	at org.python.core.__builtin__.eval(__builtin__.java:497)
	at org.python.core.__builtin__.eval(__builtin__.java:501)
	at org.python.util.PythonInterpreter.eval(PythonInterpreter.java:259)
	at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:57)
	at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:31)
	at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

As it says the name ‘false’ is not defined :man_shrugging:

Any help would be appreciated, I would love to be able to test this approach with my data and if I get a good result publish it as part of my workflow on my upcoming paper :slight_smile:

Sorry for the long post and congrats on the great work :clap:
Toni

1 Like

Hey @aarandaramos,

if it says “false” is not known, can you please try it with “False” instead? Just a quick guess :wink:

Cheers,
Robert

3 Likes

Thanks Robert,
It worked as you suggested :clap: :slight_smile:
The result was higher in contrast than those previously shared by yourself @haesleinhuepf and @CellKai though… Although the middle part of the stack has a good signal to noise ratio, the top and bottom planes of the series are way more saturated with noise signal than what you both got too. Look:


Left column is the original data and PSF
Middle column is an intermediate step computed while running the script.
Right column is the computed final result.

I tried 2 times just in case… but got the same result.
This was way faster than I expected! Only a couple minutes! I’m blown away by the results in terms of speed… Will try with my data and keep you posted on how did it go.
Thanks,
Toni

This fixed the first problem that I was having (I transferred here from a thread started by @bnorthan), but I am now running into the problem that FIJI just quits while running the script, not producing any error message! I have just run the updater for FIJI and CLIJ/CLIJ2, and I have Java 8 (u251) so everything should be in its ideal state? I am running this on the following system:
OS: Mac OS X 10.13.6
GPU: Nvidia GeForce GT 755M

Looking at the error report from OSX, i find the following:
Graphics hardware encountered an error and was reset: 0x0000002b

I have tried using the getInstance command both as:

clij2 = CLIJ2.getInstance();
clij2 = CLIJ2.getInstance("GeForce");

with the same result. I have not tried it on a different machine (I don’t have easy access to another right now…)

Any thoughts?

2 Likes

Hey @CMCI,

I just checked the graphics card you’re using, it is apparently 7 years old, right? So technically I would suspect it should support OpenCL, but maybe the driver is outdated? Can you please check if the NVidia website provides recent drivers for this GPU and your operating system?

Furthermore, carefully speaking, I could imagine that GPUs which are of about that age provide similar performance like current CPUs :wink: I’m not sure about the 755M, because it was apparently an expensive one. Just suggesting: If you aim for highest speed, you may want to invest some money in a recent GPU :wink:

Let us know if a fresh driver install helps. That would also help other users with similar issues in the future. Thanks for your support!

Cheers,
Robert

1 Like

Hi Robert,

Yeah, I’m hoping when I get back to the lab to be making a new workstation with a TitanV; for the moment i’m stuck on this machine. I’ll see if the driver update helps at all.

Thanks for your hard work!

2 Likes

Hi everyone,
A follow-up from yesterday’s work. I’ve tried implementing the Jython code to my dataset, as a reference:

#@ IOService io 
#@ OpService op
#@ UIService ui

from net.haesleinhuepf.clij2 import CLIJ2;
from net.haesleinhuepf.clijx.plugins import DeconvolveFFT;


from net.imagej.ops.experiments import ImageUtility;
from net.imglib2.view import Views;
from net.imglib2 import FinalDimensions;
from net.imagej.ops.filter.pad import DefaultPadInputFFT;
from net.imglib2.outofbounds import OutOfBoundsMirrorFactory;
from net.imglib2.util import Intervals;
from net.imagej.ops import Ops;


# test names
testData = io.open("/Users/toniaranda/Desktop/Images/CH1.tif");
psf = io.open("/Users/toniaranda/Desktop/Images/ORIGINAL_PSF.tif");

# open the test data
imgF = op.convert().float32(testData.getImgPlus().getImg());
psfF = op.convert().float32(psf.getImgPlus());

# crop PSF - the image will be extended using PSF size
# if the PSF size is too large it will explode image size, memory needed and processing speed
# so crop just small enough to capture significant signal of PSF 
psfF = ImageUtility.cropSymmetric(psfF, [ 75 , 75 , 37 ], op);

# subtract min from PSF		
psfF = Views.zeroMin(ImageUtility.subtractMin(psfF, op));

# normalize PSF
psfF = Views.zeroMin(ImageUtility.normalize(psfF, op));

# compute extended dimensions based on image and PSF dimensions
extendedSize = range(0, imgF.numDimensions());

for d in range( 0, imgF.numDimensions()):
	extendedSize[d] = imgF.dimension(d) + psfF.dimension(d);


extendedDimensions = FinalDimensions(extendedSize);

# extend image
extended = op.run(DefaultPadInputFFT, imgF, extendedDimensions, False,
		OutOfBoundsMirrorFactory(OutOfBoundsMirrorFactory.Boundary.SINGLE));

# show image and PSF
ui.show("img ", imgF);
ui.show("psf ", psfF);

# get clij
clij2 = CLIJ2.getInstance("VEGA");

# push extended image and psf to GPU
inputGPU = clij2.push(extended);
psfGPU = clij2.push(psfF);

# create output
output = clij2.create(inputGPU);

# deconvolve
DeconvolveFFT.deconvolveFFT(clij2, inputGPU, psfGPU, output);

# get deconvolved as an RAI
deconv = clij2.pullRAI(output);

# create unpadding interval
interval = Intervals.createMinMax(-extended.min(0), -extended
	.min(1), -extended.min(2), -extended.min(0) + imgF.dimension(0) -
		1, -extended.min(1) + imgF.dimension(1) - 1, -extended.min(2) +
			imgF.dimension(2) - 1);

# create an RAI for the output... we could just use a View to unpad, but performance for slicing is slow 
outputRAI = op.create().img(imgF);

# copy the unpadded interval back to original size
op.run(Ops.Copy.RAI, outputRAI, Views.zeroMin(Views.interval(deconv, interval)));

# clij2.show(output, "output");
ui.show("deconvolved and unpadded", outputRAI);

This time the error I get is something different, it’s on line 58, under the inputGPU command, and the subsequent errors are:

Started Jython_ORIGINAL_SIZE.py at Wed May 27 11:49:04 CEST 2020
Traceback (most recent call last):
  File "/Users/toniaranda/Desktop/Images/Jython_ORIGINAL_SIZE.py", line 58, in <module>
    inputGPU = clij2.push(extended);
	at net.haesleinhuepf.clij.clearcl.backend.BackendUtils.checkOpenCLErrorCode(BackendUtils.java:352)
	at net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL.lambda$getBufferPeerPointer$14(ClearCLBackendJOCL.java:354)
	at net.haesleinhuepf.clij.clearcl.backend.BackendUtils.checkExceptions(BackendUtils.java:156)
	at net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL.getBufferPeerPointer(ClearCLBackendJOCL.java:331)
	at net.haesleinhuepf.clij.clearcl.ClearCLContext.createBuffer(ClearCLContext.java:295)
	at net.haesleinhuepf.clij.CLIJ.createCLBuffer(CLIJ.java:341)
	at net.haesleinhuepf.clij.converters.implementations.RandomAccessibleIntervalToClearCLBufferConverter.convert(RandomAccessibleIntervalToClearCLBufferConverter.java:40)
	at net.haesleinhuepf.clij.converters.implementations.RandomAccessibleIntervalToClearCLBufferConverter.convert(RandomAccessibleIntervalToClearCLBufferConverter.java:26)
	at net.haesleinhuepf.clij.CLIJ.convert(CLIJ.java:463)
	at net.haesleinhuepf.clij2.CLIJ2.push(CLIJ2.java:95)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
net.haesleinhuepf.clij.clearcl.exceptions.OpenCLException: net.haesleinhuepf.clij.clearcl.exceptions.OpenCLException: OpenCL error: -61 -> CL_INVALID_BUFFER_SIZE

	at org.python.core.Py.JavaError(Py.java:552)
	at org.python.core.Py.JavaError(Py.java:543)
	at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:190)
	at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:206)
	at org.python.core.PyObject.__call__(PyObject.java:497)
	at org.python.core.PyObject.__call__(PyObject.java:501)
	at org.python.core.PyMethod.__call__(PyMethod.java:141)
	at org.python.pycode._pyx7.f$0(/Users/toniaranda/Desktop/Images/Jython_ORIGINAL_SIZE.py:83)
	at org.python.pycode._pyx7.call_function(/Users/toniaranda/Desktop/Images/Jython_ORIGINAL_SIZE.py)
	at org.python.core.PyTableCode.call(PyTableCode.java:171)
	at org.python.core.PyCode.call(PyCode.java:18)
	at org.python.core.Py.runCode(Py.java:1614)
	at org.python.core.__builtin__.eval(__builtin__.java:497)
	at org.python.core.__builtin__.eval(__builtin__.java:501)
	at org.python.util.PythonInterpreter.eval(PythonInterpreter.java:259)
	at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:57)
	at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:31)
	at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: net.haesleinhuepf.clij.clearcl.exceptions.OpenCLException: OpenCL error: -61 -> CL_INVALID_BUFFER_SIZE
	at net.haesleinhuepf.clij.clearcl.backend.BackendUtils.checkOpenCLErrorCode(BackendUtils.java:352)
	at net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL.lambda$getBufferPeerPointer$14(ClearCLBackendJOCL.java:354)
	at net.haesleinhuepf.clij.clearcl.backend.BackendUtils.checkExceptions(BackendUtils.java:156)
	at net.haesleinhuepf.clij.clearcl.backend.jocl.ClearCLBackendJOCL.getBufferPeerPointer(ClearCLBackendJOCL.java:331)
	at net.haesleinhuepf.clij.clearcl.ClearCLContext.createBuffer(ClearCLContext.java:295)
	at net.haesleinhuepf.clij.CLIJ.createCLBuffer(CLIJ.java:341)
	at net.haesleinhuepf.clij.converters.implementations.RandomAccessibleIntervalToClearCLBufferConverter.convert(RandomAccessibleIntervalToClearCLBufferConverter.java:40)
	at net.haesleinhuepf.clij.converters.implementations.RandomAccessibleIntervalToClearCLBufferConverter.convert(RandomAccessibleIntervalToClearCLBufferConverter.java:26)
	at net.haesleinhuepf.clij.CLIJ.convert(CLIJ.java:463)
	at net.haesleinhuepf.clij2.CLIJ2.push(CLIJ2.java:95)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:188)
	... 24 more

I’ve uploaded the input images here so you have all the information to test with.

Also, I would like to know how tunable the deconvolution settings are using this script… I’ve tried looking for the Iteration count settings on the code but I haven’t been able to identify it and therefore adjust it to the settings I would like to run my processing at.

Any help will be much appreciated as always!
Thanks in advance,

Toni

1 Like

Hey Toni @aarandaramos,

thanks for trying and testing our experimental deconvolution code :wink:

The error message “CL_INVALID_BUFFER_SIZE” usually appears, if GPU hardware doesn’t supprt images of a given size. You can read out the size limitations of your hardware by running clInfo from the menu ImageJ on GPU (CLIJ2) > Macro Tools > CLIJ2 ClInfo. In my case thats 1 GB:

Furthermore, I just tried your script. As my GPU is limited memory-wise, I cropped your image to 108 MB (from 384 MB) in advance. I also cropped the PSF to 813kB (from 1.5 GB). The script then needs about 40 seconds to run through.

I’m now wondering if it also might also work if you limit image size a bit. Would you mind trying it on a smaller dataset? Let’s first ensure that it runs in general, before we drive it to the extreme :wink:

Some more comments about your image data: The original image appears over-exposed. The white spot on the left has quite some pixels with value 255. In an 8-bit image, this is a sign for over-exposure. Deconvolution of an image with a chopped off grey-value range might lead to strange results. Furthermore, the black rings in the deconvolved image might be an artifact resulting from a sub-optimal PSF. Furthermore, the PSF image you sent is not calibrated. I think in the current implementation, PSF and original image must have the same voxel size to make the algorithm work. Could you please check that?

Let us know if this helps!

Cheers,
Robert

2 Likes

Hi @aarandaramos

As @haesleinhuepf mentions it seems you may be running out of memory. That’s exactly what happened when I ran it on my side. I modified the script to crop both the PSF and image and it worked for me.

#@ IOService io 
#@ OpService op
#@ UIService ui

from net.haesleinhuepf.clij2 import CLIJ2;
from net.haesleinhuepf.clijx.plugins import DeconvolveFFT;

from net.imagej.ops.experiments import ImageUtility;
from net.imglib2.view import Views;
from net.imglib2 import FinalDimensions;
from net.imagej.ops.filter.pad import DefaultPadInputFFT;
from net.imglib2.outofbounds import OutOfBoundsMirrorFactory;
from net.imglib2.util import Intervals;
from net.imagej.ops import Ops;

# open the test data
testData = io.open("/home/bnorthan/Images/Deconvolution/AAR_TEST/CH1.tif");
psf = io.open("/home/bnorthan/Images/Deconvolution/AAR_TEST/ORIGINAL_PSF.tif");

# crop PSF - the image will be extended using PSF size
# if the PSF size is too large it will explode image size, memory needed and processing speed
# so crop just small enough to capture significant signal of PSF 
psf = ImageUtility.cropSymmetric(psf, [ 75 , 75 , 37 ], op);

# also crop test data or else we will run out of GPU memory
testData = ImageUtility.cropSymmetric(testData, [ 1024 , 1024 , 37 ], op);

imgF = op.convert().float32(Views.zeroMin(testData));
psfF = op.convert().float32(Views.zeroMin(psf));

# subtract min from PSF		
psfF = Views.zeroMin(ImageUtility.subtractMin(psfF, op));

# normalize PSF
psfF = Views.zeroMin(ImageUtility.normalize(psfF, op));

# compute extended dimensions based on image and PSF dimensions
extendedSize = range(0, imgF.numDimensions());

for d in range( 0, imgF.numDimensions()):
	extendedSize[d] = imgF.dimension(d) + psfF.dimension(d);


extendedDimensions = FinalDimensions(extendedSize);

# extend image
extended = op.run(DefaultPadInputFFT, imgF, extendedDimensions, False,
		OutOfBoundsMirrorFactory(OutOfBoundsMirrorFactory.Boundary.SINGLE));

# show image and PSF
ui.show("img ", imgF);
ui.show("psf ", psfF);

# get clij
clij2 = CLIJ2.getInstance("VEGA");

# push extended image and psf to GPU
inputGPU = clij2.push(extended);
psfGPU = clij2.push(psfF);

# create output
output = clij2.create(inputGPU);

# deconvolve
DeconvolveFFT.deconvolveFFT(clij2, inputGPU, psfGPU, output);

# get deconvolved as an RAI
deconv = clij2.pullRAI(output);

# create unpadding interval
interval = Intervals.createMinMax(-extended.min(0), -extended
	.min(1), -extended.min(2), -extended.min(0) + imgF.dimension(0) -
		1, -extended.min(1) + imgF.dimension(1) - 1, -extended.min(2) +
			imgF.dimension(2) - 1);

# create an RAI for the output... we could just use a View to unpad, but performance for slicing is slow 
outputRAI = op.create().img(imgF);

# copy the unpadded interval back to original size
op.run(Ops.Copy.RAI, outputRAI, Views.zeroMin(Views.interval(deconv, interval)));

# clij2.show(output, "output");
ui.show("deconvolved and unpadded", Views.zeroMin(outputRAI));
2 Likes

Have you hit size limits with other datasets @haesleinhuepf? Does CLIJ have a mechanism to process in sub-blocks? I’ve looked into imglib-cache before and in my preliminary experiments it worked well. I also looked into Zarr a bit, however did not dive in too deeply yet.

1 Like

Hey Brian @bnorthan,

some people hit these issues, yes. I usually don’t because I bought hardware according to the specs of the workflows I run on it.

Some workarounds are available though: CLIJx has pushTile and pullTile methods. An example macro shows how to use them. From Java, it should be straight-forward. Deconvolution could be a very nice use case for this strategy. We could derive the margin size for the tiling from the PSF size.

In maaany other cases, slice-by-slice processing and downsampling enabled the collaborators to solve their issues. A general solution might not be obvious.

I’ll also put @maarzt in the loop because he can tell us how life in the border-land between imglib2-cache and clij is.

Cheers,
Robert

Thanks for the quick reply, and to @bnorthan too.

I’ll let you both know about my results as I go through them.

I run the CLIJ2 CIInfo and my results is: MaxMemoryAllocationSizeInBytes: 2143289344, so that’s about 2,1 GB in my case. I will try again with this limit in mind for image size. The image size was something the technicians at the Microscopy platform where data was acquired suggested in order to use Huygens deconvolution software. It was set so that the pixel size (37,3 nm) was enough for the program to run the process. I can reduce my analyses to cell-by-cell sizes and the result would be OK for my anyway.

I wonder if there is any way to increase the MaxMemoryAllocationSizeInBytes retrieving from the GlobalMemorySizeInBytes (in my case is 68719476736 bytes) of the GPU though.

In fact I tried it yesterday, using a small dataset on a region cropped from the dataset I shared with you and got this result:


That’s why I ventured into the full image dataset this time.

I know some of the pixels on the image have values of 255, but the technician said it was OK since it was only accounting for less than the 0,01% (25417/401953792= 0,006323363657%) of the information of the dataset. I trusted that it wasn’t that crucial then, but I see your point. How this little information will affect the outcome in your opinion? Is it something I can correct in any possible way other than re-imaging?

This can be, I have 0 experience on the matter and calculated the theoretical PSF using the Diffraction PSF 3D plugin at the best of my knowledge with the data I could extract from my series.
This are the settings I used.

I will try and find a way around this, but I am not sure how to callibrate using the Diffraction PSF 3D plugin I’m using. I understand Image pixel spacing and the slice spacing are different in my dataset, and that is used to calculate voxel size on the resulting PSF dataset right? So why there would be a divergence/un-callibration to the original data?

Hi @bnorthan. I will let you know how this goes too as soon as I test it. Thanks for the quick reply.

This is what I observed when running my original (and big) dataset on Huygens. The data was chopped and processed in sections, but then stitched together at the end. It was way slower than I expected though, but maybe because it was running on CPU (not sure, as it was a machine at work). I’m sure the GPU will work wonders if uses this approach.

Thanks again for your insight, it is very valuable to me, as I’m a little lost in the process and acting at the best of my knowledge as you can tell.
Best,
Toni

1 Like

May I ask whay GPU you are using? 64 GB of graphics memory appears mind blowing… Must be a very expensive GPU… Feel free to post the whole output of clInfo as a clarification.

I’m afraid that’s a driver/hardware limitation. Usually, images of 1/4 - 1/2 times the global memory size can be allocated (not sure why this appears different on your GPU). Empirical knowledge, another example, a TITAN Xp with 12 GB memory:


This limitation anyway makes a lot of sense as the original image blocks a quarter of the memory, the PSF does, the output image as well. This sums up to 75% load and leaves some space for other stuff. One can actually conveniently work with this limitation usually.

In the regions where intensity is 255 (and in reality was higher), the deconvolution result may be misleading. If you’re interested in studying this particular regions, I would carefully interpret the images there anyway.

The current script doesn’t take voxel size into account. Thus, it assumes voxel have the same size in both images. If this assumption is wrong, the deconvolution result is wrong.

I hope this helps.

Cheers,
Robert

1 Like