CLIJ deconvolution

Hi!
Thanks for your fast reply and the work both @bnorthan and you are putting into this.
I’ve tried the Jython way modifying it to call my GPU with the following code:

Jython modified for 'Radeon' GPU
#@ OpService ops
#@ UIService ui
#@ Dataset data
#@ Dataset psf


from java.lang import System

# init CLIJ and GPU
from net.haesleinhuepf.clij import CLIJ;
from net.haesleinhuepf.clij2 import CLIJ2;
from net.haesleinhuepf.clijx.plugins import Normalize;
from net.haesleinhuepf.clijx.plugins import DeconvolveRichardsonLucyFFT;

# show installed OpenCL devices
print CLIJ.getAvailableDeviceNames();

# initialize a device with a given name
clij2 = CLIJ2.getInstance("Radeon");
clij2.clear();

print "Using GPU: " + clij2.getGPUName();

# transfer image to the GPU
gpuImg = clij2.push(data);
gpuPSF = clij2.push(psf);

# measure start time
start = System.currentTimeMillis();

# normalize PSF
gpuPSF_normalized = clij2.create(gpuPSF);
Normalize.normalize(clij2, gpuPSF, gpuPSF_normalized);

# create memory for the output image first
gpuEstimate = clij2.create(gpuImg.getDimensions(), clij2.Float);

# submit deconvolution task
num_iterations = 10;
DeconvolveRichardsonLucyFFT.deconvolveRichardsonLucyFFT(clij2, gpuImg, gpuPSF_normalized, gpuEstimate, num_iterations);

# measure end time
finish = System.currentTimeMillis();

print('CLIJ decon time ', (finish-start));
clij2.show(gpuEstimate, "GPU Decon Result");

# clean up memory
clij2.clear();

I’m getting the following error messages:

Console error
[ERROR] Error loading file
java.io.IOException: io.scif.img.ImgIOException: java.io.IOException: io.scif.FormatException: Invalid TIFF file
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:163)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:133)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:138)
	at io.scif.convert.FileToDatasetConverter.convert(FileToDatasetConverter.java:66)
	at org.scijava.convert.AbstractConvertService.convert(AbstractConvertService.java:125)
	at org.scijava.convert.AbstractDelegateConverter.convert(AbstractDelegateConverter.java:53)
	at org.scijava.convert.AbstractConvertService.convert(AbstractConvertService.java:125)
	at org.scijava.module.DefaultModuleService.load(DefaultModuleService.java:316)
	at org.scijava.module.DefaultModuleService.loadInput(DefaultModuleService.java:544)
	at org.scijava.module.DefaultModuleService.lambda$loadInputs$1(DefaultModuleService.java:346)
	at java.util.ArrayList.forEach(ArrayList.java:1257)
	at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1080)
	at org.scijava.module.DefaultModuleService.loadInputs(DefaultModuleService.java:346)
	at org.scijava.module.process.LoadInputsPreprocessor.process(LoadInputsPreprocessor.java:58)
	at org.scijava.module.ModuleRunner.preProcess(ModuleRunner.java:102)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:154)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: io.scif.img.ImgIOException: java.io.IOException: io.scif.FormatException: Invalid TIFF file
	at io.scif.img.ImgOpener.createReader(ImgOpener.java:489)
	at io.scif.img.ImgOpener.openImgs(ImgOpener.java:242)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:152)
	... 22 more
Caused by: java.io.IOException: io.scif.FormatException: Invalid TIFF file
	at io.scif.AbstractReader.setSource(AbstractReader.java:279)
	at io.scif.services.DefaultInitializeService.initializeReader(DefaultInitializeService.java:91)
	at io.scif.img.ImgOpener.createReader(ImgOpener.java:483)
	... 24 more
Caused by: io.scif.FormatException: Invalid TIFF file
	at io.scif.formats.MinimalTIFFFormat$Parser.typedParse(MinimalTIFFFormat.java:412)
	at io.scif.formats.TIFFFormat$BaseTIFFParser.typedParse(TIFFFormat.java:957)
	at io.scif.formats.TIFFFormat$BaseTIFFParser.typedParse(TIFFFormat.java:939)
	at io.scif.AbstractParser.parse(AbstractParser.java:244)
	at io.scif.AbstractParser.parse(AbstractParser.java:314)
	at io.scif.AbstractParser.parse(AbstractParser.java:53)
	at io.scif.AbstractReader.setSource(AbstractReader.java:271)
	... 26 more
[ERROR] Error loading file
java.io.IOException: io.scif.img.ImgIOException: java.io.IOException: io.scif.FormatException: Invalid TIFF file
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:163)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:133)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:138)
	at io.scif.convert.FileToDatasetConverter.convert(FileToDatasetConverter.java:66)
	at org.scijava.convert.AbstractConvertService.convert(AbstractConvertService.java:125)
	at org.scijava.convert.AbstractDelegateConverter.convert(AbstractDelegateConverter.java:53)
	at org.scijava.convert.AbstractConvertService.convert(AbstractConvertService.java:125)
	at org.scijava.module.DefaultModuleService.load(DefaultModuleService.java:316)
	at org.scijava.module.DefaultModuleService.loadInput(DefaultModuleService.java:544)
	at org.scijava.module.DefaultModuleService.lambda$loadInputs$1(DefaultModuleService.java:346)
	at java.util.ArrayList.forEach(ArrayList.java:1257)
	at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1080)
	at org.scijava.module.DefaultModuleService.loadInputs(DefaultModuleService.java:346)
	at org.scijava.module.process.LoadInputsPreprocessor.process(LoadInputsPreprocessor.java:58)
	at org.scijava.module.ModuleRunner.preProcess(ModuleRunner.java:102)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:154)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: io.scif.img.ImgIOException: java.io.IOException: io.scif.FormatException: Invalid TIFF file
	at io.scif.img.ImgOpener.createReader(ImgOpener.java:489)
	at io.scif.img.ImgOpener.openImgs(ImgOpener.java:242)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:152)
	... 22 more
Caused by: java.io.IOException: io.scif.FormatException: Invalid TIFF file
	at io.scif.AbstractReader.setSource(AbstractReader.java:279)
	at io.scif.services.DefaultInitializeService.initializeReader(DefaultInitializeService.java:91)
	at io.scif.img.ImgOpener.createReader(ImgOpener.java:483)
	... 24 more
Caused by: io.scif.FormatException: Invalid TIFF file
	at io.scif.formats.MinimalTIFFFormat$Parser.typedParse(MinimalTIFFFormat.java:412)
	at io.scif.formats.TIFFFormat$BaseTIFFParser.typedParse(TIFFFormat.java:957)
	at io.scif.formats.TIFFFormat$BaseTIFFParser.typedParse(TIFFFormat.java:939)
	at io.scif.AbstractParser.parse(AbstractParser.java:244)
	at io.scif.AbstractParser.parse(AbstractParser.java:314)
	at io.scif.AbstractParser.parse(AbstractParser.java:53)
	at io.scif.AbstractReader.setSource(AbstractReader.java:271)
	... 26 more
Error on the Script interpreter
Started 200913_Original_Jython_Radeon_GPU.py at Sun Sep 13 16:40:07 CEST 2020
Traceback (most recent call last):
  File "/Users/toniaranda/Documents/2019_2020_5th_PhD_Year/2019_2020_Experiment_data_and_analyses/1_SiO2_NPs/Process/JYTHON_CODE/200913_Original_Jython_Radeon_GPU.py", line 40, in <module>
    DeconvolveRichardsonLucyFFT.deconvolveRichardsonLucyFFT(clij2, gpuImg, gpuPSF_normalized, gpuEstimate, num_iterations);
	at net.haesleinhuepf.clijx.plugins.clij2fftWrapper.deconv3d_32f_lp(Native Method)
	at net.haesleinhuepf.clijx.plugins.DeconvolveRichardsonLucyFFT.runDecon(DeconvolveRichardsonLucyFFT.java:153)
	at net.haesleinhuepf.clijx.plugins.DeconvolveRichardsonLucyFFT.deconvolveFFT(DeconvolveRichardsonLucyFFT.java:110)
	at net.haesleinhuepf.clijx.plugins.DeconvolveRichardsonLucyFFT.deconvolveRichardsonLucyFFT(DeconvolveRichardsonLucyFFT.java:77)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
java.lang.UnsatisfiedLinkError: java.lang.UnsatisfiedLinkError: net.haesleinhuepf.clijx.plugins.clij2fftWrapper.deconv3d_32f_lp(IJJJJJJJJJJ)I

	at org.python.core.Py.JavaError(Py.java:547)
	at org.python.core.Py.JavaError(Py.java:538)
	at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:192)
	at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:208)
	at org.python.core.PyObject.__call__(PyObject.java:433)
	at org.python.core.PyObject.__call__(PyObject.java:437)
	at org.python.pycode._pyx0.f$0(/Users/toniaranda/Documents/2019_2020_5th_PhD_Year/2019_2020_Experiment_data_and_analyses/1_SiO2_NPs/Process/JYTHON_CODE/200913_Original_Jython_Radeon_GPU.py:49)
	at org.python.pycode._pyx0.call_function(/Users/toniaranda/Documents/2019_2020_5th_PhD_Year/2019_2020_Experiment_data_and_analyses/1_SiO2_NPs/Process/JYTHON_CODE/200913_Original_Jython_Radeon_GPU.py)
	at org.python.core.PyTableCode.call(PyTableCode.java:173)
	at org.python.core.PyCode.call(PyCode.java:18)
	at org.python.core.Py.runCode(Py.java:1687)
	at org.python.core.__builtin__.eval(__builtin__.java:497)
	at org.python.core.__builtin__.eval(__builtin__.java:501)
	at org.python.util.PythonInterpreter.eval(PythonInterpreter.java:255)
	at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:57)
	at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:31)
	at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:157)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.UnsatisfiedLinkError: net.haesleinhuepf.clijx.plugins.clij2fftWrapper.deconv3d_32f_lp(IJJJJJJJJJJ)I
	at net.haesleinhuepf.clijx.plugins.clij2fftWrapper.deconv3d_32f_lp(Native Method)
	at net.haesleinhuepf.clijx.plugins.DeconvolveRichardsonLucyFFT.runDecon(DeconvolveRichardsonLucyFFT.java:153)
	at net.haesleinhuepf.clijx.plugins.DeconvolveRichardsonLucyFFT.deconvolveFFT(DeconvolveRichardsonLucyFFT.java:110)
	at net.haesleinhuepf.clijx.plugins.DeconvolveRichardsonLucyFFT.deconvolveRichardsonLucyFFT(DeconvolveRichardsonLucyFFT.java:77)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:190)
	... 23 more

I also modified the macro to call the GPU, but I’m not sure it’s getting the proper input doing it like the Jython modification:

Macro modification to call for GPU
run("Close All");

open("/Users/toniaranda/Documents/2019_2020_5th_PhD_Year/2019_2020_Experiment_data_and_analyses/1_SiO2_NPs/Process/PSF/CROP/C2-CROP.tif");
run("32-bit");
image = getTitle();

open("/Users/toniaranda/Documents/2019_2020_5th_PhD_Year/2019_2020_Experiment_data_and_analyses/1_SiO2_NPs/Process/PSF/CROP/PSF_1000x1000.tif");
run("32-bit");
psf = getTitle();

run("CLIJ2 Macro Extensions", "cl_device=[Radeon]");
Ext.CLIJ2_clear();

Ext.CLIJ2_push(image);
Ext.CLIJ2_push(psf);

// ensure that PSF intensities are between 0 and 1
Ext.CLIJx_normalize(psf, normalized_psf);

// deconvolve
num_iterations = 10;
Ext.CLIJx_deconvolveRichardsonLucyFFT(image, normalized_psf, deconvolved, num_iterations);

Ext.CLIJ2_pull(deconvolved);

It gives the following Console error

Console error
(Fiji Is Just) ImageJ 2.1.0/1.53e; Java 1.8.0_202 [64-bit]; Mac OS X 10.15.6; 4838MB of 44444MB (10%)
 
java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: net.haesleinhuepf.clijx.plugins.clij2fftWrapper.deconv3d_32f_lp(IJJJJJJJJJJ)I
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:117)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:28)
	at net.haesleinhuepf.clij.macro.CLIJHandler.handleExtension(CLIJHandler.java:53)
	at ij.macro.ExtensionDescriptor.dispatch(ExtensionDescriptor.java:288)
	at ij.macro.Functions.doExt(Functions.java:5000)
	at ij.macro.Functions.getStringFunction(Functions.java:278)
	at ij.macro.Interpreter.getStringTerm(Interpreter.java:1475)
	at ij.macro.Interpreter.getString(Interpreter.java:1453)
	at ij.macro.Interpreter.doStatement(Interpreter.java:333)
	at ij.macro.Interpreter.doStatements(Interpreter.java:264)
	at ij.macro.Interpreter.run(Interpreter.java:160)
	at ij.macro.Interpreter.run(Interpreter.java:93)
	at ij.macro.Interpreter.run(Interpreter.java:104)
	at ij.plugin.Macro_Runner.runMacro(Macro_Runner.java:162)
	at ij.IJ.runMacro(IJ.java:158)
	at ij.IJ.runMacro(IJ.java:147)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1148)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1144)
	at net.imagej.legacy.IJ1Helper.runMacroFriendly(IJ1Helper.java:1095)
	at net.imagej.legacy.IJ1Helper.runMacro(IJ1Helper.java:1144)
	at net.imagej.legacy.plugin.IJ1MacroEngine.eval(IJ1MacroEngine.java:145)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:157)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.UnsatisfiedLinkError: net.haesleinhuepf.clijx.plugins.clij2fftWrapper.deconv3d_32f_lp(IJJJJJJJJJJ)I
	at net.haesleinhuepf.clijx.plugins.clij2fftWrapper.deconv3d_32f_lp(Native Method)
	at net.haesleinhuepf.clijx.plugins.DeconvolveRichardsonLucyFFT.runDecon(DeconvolveRichardsonLucyFFT.java:153)
	at net.haesleinhuepf.clijx.plugins.DeconvolveRichardsonLucyFFT.deconvolveFFT(DeconvolveRichardsonLucyFFT.java:110)
	at net.haesleinhuepf.clijx.plugins.DeconvolveRichardsonLucyFFT.deconvolveRichardsonLucyFFT(DeconvolveRichardsonLucyFFT.java:77)
	at net.haesleinhuepf.clijx.plugins.DeconvolveRichardsonLucyFFT.executeCL(DeconvolveRichardsonLucyFFT.java:42)
	at net.haesleinhuepf.clij.macro.CLIJHandler.lambda$handleExtension$0(CLIJHandler.java:163)
	at net.haesleinhuepf.clij.clearcl.util.ElapsedTime.measure(ElapsedTime.java:97)
	... 29 more

I’ve updated everything before running it
Fiji 2.1.0/1.53e
CLIJ2 version 2.1.3.0
CLIJx version 0.29.3.0

Thanks,
Toni

1 Like

Hey @aarandaramos,

thanks for testing! I just recompiled it and it’s getting worse. Give us some time to think about it :wink: We’ll keep you posted.

Cheers,
Robert

1 Like

Sure thing! :muscle:t3:
You can contact me at toni@arandaramos.com in case I can be of any help testing.
Thanks,
Toni

1 Like

H @haesleinhuepf

So it looks like we never made the Mac build scriptable. I assume at one point you did it manually. However the error you are seeing is potentially because you somehow build the native lib, and the native wrapper with different compilers (I think we saw something similar on Windows where the lib was built with 32 bit compiler and the wrapper with 64).

So we need need to fill in the MacOSX part of this script.

Then we can finish the MacOSX part of this script and use it to build everything.

Sorry about all the pain. Once we get the build scriptable on each operating system it should be much easier to rebuild and trouble shoot.

@aarandaramos just to confirm are you using MacOSX too?? If so let’s get the MacOSX figured out and go from there. If you are using a different operating system, then I guess you have a different problem, so let me know which operating system you are on and we can go from there.

Brian

1 Like

Hi Robert @haesleinhuepf,

I tried the macro script (https://github.com/clij/clij2-fft/blob/d82fd0f17476e974254370cc4d1bc7fbb905a3b8/src/main/macro/deconvolve.ijm#L18) you shared on a windows machine and had a question.

The deconvolution works but increasing the number of iterations in the macro seems to run for about the same amount of time and outputs the same image. I looked at the java code for the function and noticed the comment specifying 100 iterations. Does this mean the function is currently fixed at 100 iterations? Also, thank you and @bnorthan for all your work on this.

Ben

2 Likes

Hey @aarandaramos

I eventually managed to recompile the macos version of the plugin. Would you mind testing it? On my stone age Intel GPU it takes quite a moment :smiley:

Also big thanks to @Bem for spotting the issue of fixed number of iterations! Very well observed. Thanks for reporting!

And again big thanks to @bnorthan for remote support and debugging my limited macos-compilation skills :smiley:

Cheers,
Robert

3 Likes

Thanks @bnorthan, @Bem and @haesleinhuepf for working on it so quickly,

Yes I’m working on MacOS Catalina (10.15.6)

I’ll try the update as soon as I can and update you about the results I get :wink:
Toni