Clij is not running on Windows Server 2019 / Quadro RTX 8000

Hi,

I have problems running the Clij Plugins in Fiji on our analysis server.
When I use any Clij function, I get this error message:

(Fiji Is Just) ImageJ 2.1.0/1.53c; Java 1.8.0_172 [64-bit]; Windows Server 2016 10.0; 328MB of 785960MB (<1%)
 
net.haesleinhuepf.clij.clearcl.exceptions.ClearCLTooManyContextsException: Too many contexts have been created and not released
	at net.haesleinhuepf.clij.clearcl.ClearCLDevice.createContext(ClearCLDevice.java:221)
	at net.haesleinhuepf.clij.CLIJ.<init>(CLIJ.java:156)
	at net.haesleinhuepf.clij.CLIJ.getInstance(CLIJ.java:172)
	at net.haesleinhuepf.clij.CLIJ.getInstance(CLIJ.java:166)
	at net.haesleinhuepf.clij.macro.AbstractCLIJPlugin.run(AbstractCLIJPlugin.java:265)
	at ij.plugin.filter.PlugInFilterRunner.processOneImage(PlugInFilterRunner.java:265)
	at ij.plugin.filter.PlugInFilterRunner.<init>(PlugInFilterRunner.java:114)
	at ij.IJ.runUserPlugIn(IJ.java:237)
	at ij.IJ.runPlugIn(IJ.java:198)
	at ij.Executer.runCommand(Executer.java:150)
	at ij.Executer.run(Executer.java:68)
	at java.lang.Thread.run(Thread.java:748)

I tried it with a fresh and updated Fiji and I also updated my GPU driver. I get this error also, if I use CLIJx >> Change OpenCL Device

On my Notebook everything works fine. By the way, the CLIJx-Assistant is very cool :+1:

Any ideas?
Thanks and all the best,
Thomas

1 Like

Hi Thomas @T-Zobel ,

great to see you here! This is a known bug which has something to do with Windows Server 2019 on computers that run virtual machines accessing GPUs. Others (@bramvdbroek, and @LThomas ) solved it earlier but it was not 100%ly clear what the actual issue was. It depends a bit on the system where it happens. You can read more about potential solutions in this thread:

Let us know if this helps and/or if you have further questions!

Cheers,
Robert

3 Likes

Hi Robert @haesleinhuepf,
thanks for your fast reply. I will have a look at the thread (we are also using a Hive, so the system should be similar). And maybe it is the same reason why N2V is not working at our server with Fiji. What was actually not that bad, as I am now able to use N2V with Jupyter Notebooks. :wink:

I will try it now and report if I found a solution.

Best,
Thomas

1 Like

Hi Thomas @T-Zobel and Robert @haesleinhuepf,

That Windows Server 2019 & CLIJ thread is nice, but a bit chaotic. A very short summary:

  • On the HIVE system (Windows Server 2019) OpenCL does not work (and with it CLIJ and other tools) when the Remote Desktop Session Host role is installed. You can remove this role via the server manager (more info here). Removing it fixes the issue. However, then you cannot connect to the HIVE with remote desktop anymore, which is kind of defeating the purpose.
  • Apart from that, you need to set the maximum ImageJ memory usage below 1018400 Mb (at that time at least), or you get a different error. See here and links in that post.
  • @lhess and @LThomas somehow fixed it on their system after completely reinstalling it from scratch. I haven’t been so bold, and my CLIJ is still not functional.
  • Visual Studio 2019 may be necessary, but frankly I believe nobody really knows if this is true, and why.
  • @ialexop faced the same issue. Does it work for you now, Iohannis?

@lhess and @LThomas Did you dive any deeper into it? Any ideas for things us to test?

Best regards,
Bram

4 Likes

Hi @bramvdbroek, thanks for the concise summary.
My colleague Lord tested a lot of different configurations but unfortunately we could not find the source of the problem.
We kind of ran out of ideas and time, so we suspended further testing until we get more insight into the issue.

I think this was because the Remote Desktop Session Host that you mentioned above is not installed by default with a fresh installation.

Sorry about that, we will let you know if we ultimately come up with a solution/work around.

4 Likes

Thanks @LThomas for the update.
Ok, then we are all stuck with the same issue: Windows Server 2019 + Remote Desktop Session Host + OpenCL fails.
Perhaps we can have Microsoft look at it, if we ask nicely? :slight_smile:

Bram

1 Like

Hi Bram,
I confirm that I had the same issues on a Windows 2019 Server system (very similar to HIVE).
Uninstalling the Remote Desktop Host role fixes the problem. But then only one user at a time can do a Remote Desktop connection.
Without having the technical knowledge, I believe that the RD Host service also allocates somehow graphic cards’ resources for the remote desktops, which apparently “confuses” CLIj.

I don’t have this system in my hands anymore because I changed positions (from Netherlands to Germany). But I believe the problem remains.
However, the purpose of that server is to serve multiple desktop connections for users that can use a certain commercial GPU accelerated deconvolution software. For this reason the RD Host service has a higher priority there.
It would be great to hear though that a solution can be found for this kind of configuration.
With kind regards
Ioannis

1 Like

Hi @bramvdbroek @LThomas @ialexop ,

thank you all for your replies and @bramvdbroek for the summary. :+1:
I installed VSC 2019 yesterday again (I already had installed one version) and restarted the Hive. Unfortunately CLIJ is not running.

Removing the Remote Desktop Session Host is not an option (maybe for testing, but not as an pernament solution). Has anyone tested if it works with Matlab or Icy or…
I am just wondering, because Noise2Void is also not working in Fiji for me (there is similar thread about it), but with python/jupyter notebooks it works.

1 Like

Out of curiosity: Does py-clEsperanto work?

E.g. this jupyter notebook:

Thanks!

Runs like a charm :wink:

1 Like

I am not familiar with ICY, but I installed it and “Clicy” + “InstallClicyToolbar”. Seems to work. I can use the CLIJ2 Mean3DBox. Doesn’t that mean OpenCL is working in general and there is more of a Fiji/ImageJ specific issue with Windows Server 2019 :man_shrugging:?

1 Like

That would mean that Fiji might work as well, if you replace the Java Runtime Environment…

Hi,
so good news @haesleinhuepf @bramvdbroek @LThomas @ialexop @lhess

As Robert suggested, I replaced the JRE.
In Detail:

  • I installed the latest Java from the homepage
  • Add the System Environment Variable as described here (ImageJ FAQ, Control Panel › System and Security › System › Advanced Settings › Advanced › Environment Variables)
JAVA_HOME
C:\Program Files\Java\jdk1.8.0_172
  • I deleted, on my Fiji for Clij, the java Folder

And now I have fun with the Clij-assistant.
I will test it with some scripts next week. I will write here again if there are any problems.

All the best,
Thomas

5 Likes

Wow, thanks for letting us know @T-Zobel ! Also trying Icy was s great idea! I will print out your post, frame it and hang in the hall of fame of clij-bugfixes :smiley: … and tweet about it :grin:

… after someone confirmed that this works :slightly_smiling_face:

1 Like

Hi Robert @haesleinhuepf,

Thanks a lot :wink:

But as you wrote:

lets wait a bit. Also some functions like CLIJ2>Filter>Median 3D box … working fine and fast, even on huge stacks, loading the benchmark macro gives some errors:

[ERROR] Cannot create plugin: class='net.haesleinhuepf.clijx.morpholibj.MorphoLibJChamferDistanceMap', name='CLIJx_morphoLibJChamferDistanceMap', priority=0.0, enabled=true, pluginType=CLIJMacroPlugin
[ERROR] java.lang.NullPointerException
	at net.haesleinhuepf.clij.macro.CLIJMacroExtensionAutoCompletionPlugin.getCompletions(CLIJMacroExtensionAutoCompletionPlugin.java:34)
	at net.imagej.legacy.plugin.MacroExtensionAutoCompletionService.getCompletions(MacroExtensionAutoCompletionService.java:71)
	at net.imagej.legacy.plugin.MacroAutoCompletionProvider.addMacroExtensionAutoCompletions(MacroAutoCompletionProvider.java:176)
	at net.imagej.legacy.plugin.MacroLanguageSupportPlugin.getMacroAutoCompletionProvider(MacroLanguageSupportPlugin.java:113)
	at net.imagej.legacy.plugin.MacroLanguageSupportPlugin.getCompletionProvider(MacroLanguageSupportPlugin.java:92)
	at net.imagej.legacy.plugin.MacroLanguageSupportPlugin.install(MacroLanguageSupportPlugin.java:78)
	at org.scijava.ui.swing.script.EditorPane.setLanguage(EditorPane.java:516)
	at org.scijava.ui.swing.script.EditorPane.setLanguage(EditorPane.java:443)
	at org.scijava.ui.swing.script.EditorPane.setLanguageByFileName(EditorPane.java:431)
	at org.scijava.ui.swing.script.EditorPane.setFileName(EditorPane.java:377)
	at org.scijava.ui.swing.script.TextEditor.setEditorPaneFileName(TextEditor.java:1968)
	at org.scijava.ui.swing.script.TextEditor.open(TextEditor.java:1668)
	at net.imagej.legacy.plugin.DefaultLegacyEditor.open(DefaultLegacyEditor.java:90)
	at net.imagej.legacy.DefaultLegacyHooks.openInEditor(DefaultLegacyHooks.java:304)
	at ij.io.Opener.open(Opener.java)
	at ij.io.Opener.openAndAddToRecent(Opener.java:292)
	at ij.plugin.DragAndDrop.openFile(DragAndDrop.java:194)
	at ij.plugin.DragAndDrop.run(DragAndDrop.java:160)
	at java.lang.Thread.run(Unknown Source)
package net.haesleinhuepf.clij2.plugins, clij2_, version 0.0
[ERROR] Cannot create plugin: class='net.haesleinhuepf.clijx.morpholibj.MorphoLibJChamferDistanceMap', name='CLIJx_morphoLibJChamferDistanceMap', priority=0.0, enabled=true, pluginType=CLIJMacroPlugin
[ERROR] Module threw exception
java.lang.NullPointerException
	at net.haesleinhuepf.clij.macro.CLIJMacroPluginService.getPluginExtensionDescriptor(CLIJMacroPluginService.java:69)
	at net.haesleinhuepf.clij.macro.CLIJHandler.getExtensionFunctions(CLIJHandler.java:340)
	at ij.macro.Functions.registerExtensions(Functions.java:4940)
	at net.haesleinhuepf.clij.macro.CLIJMacroExtensions.run(CLIJMacroExtensions.java:62)
	at org.scijava.command.CommandModule.run(CommandModule.java:196)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

I have to test it next week in more detail.

Do you have MorpholibJ and the clij-assistant-extensions installed? :upside_down_face:

1 Like

MorpholibJ (IJPB-Plugins) was missing :man_facepalming:
But I still get an error (a shorter one) :wink:

[ERROR] java.lang.NullPointerException
	at net.haesleinhuepf.clij.macro.CLIJMacroExtensionAutoCompletionPlugin.getCompletions(CLIJMacroExtensionAutoCompletionPlugin.java:34)
	at net.imagej.legacy.plugin.MacroExtensionAutoCompletionService.getCompletions(MacroExtensionAutoCompletionService.java:71)
	at net.imagej.legacy.plugin.MacroAutoCompletionProvider.addMacroExtensionAutoCompletions(MacroAutoCompletionProvider.java:176)
	at net.imagej.legacy.plugin.MacroLanguageSupportPlugin.getMacroAutoCompletionProvider(MacroLanguageSupportPlugin.java:113)
	at net.imagej.legacy.plugin.MacroLanguageSupportPlugin.getCompletionProvider(MacroLanguageSupportPlugin.java:92)
	at net.imagej.legacy.plugin.MacroLanguageSupportPlugin.install(MacroLanguageSupportPlugin.java:78)
	at org.scijava.ui.swing.script.EditorPane.setLanguage(EditorPane.java:516)
	at org.scijava.ui.swing.script.EditorPane.setLanguage(EditorPane.java:443)
	at org.scijava.ui.swing.script.EditorPane.setLanguageByFileName(EditorPane.java:431)
	at org.scijava.ui.swing.script.EditorPane.setFileName(EditorPane.java:377)
	at org.scijava.ui.swing.script.TextEditor.setEditorPaneFileName(TextEditor.java:1968)
	at org.scijava.ui.swing.script.TextEditor.open(TextEditor.java:1668)
	at net.imagej.legacy.plugin.DefaultLegacyEditor.open(DefaultLegacyEditor.java:90)
	at net.imagej.legacy.DefaultLegacyHooks.openInEditor(DefaultLegacyHooks.java:304)
	at ij.io.Opener.open(Opener.java)
	at ij.io.Opener.openAndAddToRecent(Opener.java:292)
	at ij.plugin.DragAndDrop.openFile(DragAndDrop.java:194)
	at ij.plugin.DragAndDrop.run(DragAndDrop.java:160)
	at java.lang.Thread.run(Unknown Source)
[INFO] Reading available sites from https://imagej.net/
1 Like

How about the other updates sites you need to activate for the extensions?

Btw. have a great weekend if time is up! :wink:

1 Like

I already had installed the clij-assistant. Now I added all the others mentioned at your website (CLIJx-Assistant installation instructions | CLIJx-assistant)

  • clijx-assistant-extensions
  • 3D ImageJ Suite
  • BoneJ
  • IJPB-Plugins
    I guess I forgot about it when I installed it in seperate Fiji.
    And that is the result with your benchmark macro :wink: :wink: :grin: :fireworks:
CPU mean filter no 1 took 752 msec
CPU mean filter no 2 took 430 msec
CPU mean filter no 3 took 384 msec
CPU mean filter no 4 took 384 msec
CPU mean filter no 5 took 393 msec
CPU mean filter no 6 took 389 msec
CPU mean filter no 7 took 394 msec
CPU mean filter no 8 took 386 msec
CPU mean filter no 9 took 394 msec
CPU mean filter no 10 took 388 msec
Pushing one image to the GPU took 16 msec
GPU mean filter no 1 took 3256 msec
GPU mean filter no 2 took 6 msec
GPU mean filter no 3 took 6 msec
GPU mean filter no 4 took 7 msec
GPU mean filter no 5 took 7 msec
GPU mean filter no 6 took 8 msec
GPU mean filter no 7 took 8 msec
GPU mean filter no 8 took 8 msec
GPU mean filter no 9 took 8 msec
GPU mean filter no 10 took 7 msec
Pulling one image from the GPU took 57 msec

No more error messages!
Good point to finish work for today.

Thanks a lot, have a nice weekend and cheers :beers:

3 Likes

Wow well done @T-Zobel and @haesleinhuepf !!
Thanks a lot for the extensive testing, I will give it a try next week first thing :stuck_out_tongue:

This mean we should ask the Fiji maintainers if the JRE shipped with Fiji could be updated ?
I don’t mind testing a few JRE versions see from which version it starts working before we involve the maintainers.

As a side note Robert, if you rely on other update sites for some stuff you might be interested in this scijava command to check if these update sites are active, and if not to handle the error :wink:
(not the part about the hack below, but the clean scijava way in the second part of Curtis message)

3 Likes