How to overlay points onto a 3D volume

Does anybody know if there is an existing solution to overlay points (e.g. defined as a csv with x, y, & z columns) onto a 3D volume? The idea would be to overlay segmented cell centroids onto the raw data.

Ideally, I’m looking for something like clearvolume, but with an “import points” option.

Thanks!
Adam

You can do this in SciView. You can add 3D stacks as volumes, then open SMLM-type .csv files that encode x,y,z coords which will be loaded as a point cloud.

Thanks @kephale, this looks really promising! However, I’m having a few issues, mostly I think due to my lack of Java knowledge. (Sorry for the wall of text).

I downloaded a fresh version of FIJI, selected the SciView update site, and lauching SciView seemed to work fine, allthough there was an error:

Started application as PID 24683
imagej-launcher version smaller or equal to non-working version (4.0.5 vs. 4.0.5), disabling Vulkan as rendering backend. Disable check by setting 'scenery.DisableLauncherVersionCheck' system property to 'true'.
NodePublisher listening on tcp://127.0.0.1:6666
Initializing OpenGL Renderer...
Loaded Deferred Shading (Deferred Shading, with HDR postprocessing and FXAA
ClearGLWindow: GLProfile[GL4/GL4.hw]
OpenGLRenderer: 852 x 662 on TITAN RTX/PCIe/SSE2, 4.5.0 NVIDIA 418.56
Creating render framebuffer GeometryBuffer for pass Scene (852x662)
 + attachment NormalsMaterial, RGBA_Float16
 + attachment DiffuseAlbedo, RGBA_UInt8
 + attachment ZBuffer, Depth32
Creating render framebuffer AOTemp1 for pass AO (852x662)
 + attachment Occlusion, R_UInt8
Creating render framebuffer AOTemp2 for pass AOBlurV (852x662)
 + attachment Occlusion, R_UInt8
Creating render framebuffer AOBuffer for pass AOBlurH (426x331)
 + attachment Occlusion, R_UInt8
Creating render framebuffer ForwardBuffer for pass DeferredLighting (852x662)
 + attachment Color, RGBA_Float16
Creating render framebuffer HDRBuffer for pass ForwardShading (852x662)
 + attachment Color, RGBA_Float16
 + attachment Depth, Depth32
Creating render framebuffer FXAABuffer for pass HDR (852x662)
 + attachment Color, RGBA_UInt8
Creating render framebuffer Viewport for pass FXAA (852x662)
Recreating framebuffers (852x662
Creating render framebuffer GeometryBuffer for pass Scene (852x662)
 + attachment NormalsMaterial, RGBA_Float16
 + attachment DiffuseAlbedo, RGBA_UInt8
 + attachment ZBuffer, Depth32
Creating render framebuffer AOTemp1 for pass AO (852x662)
 + attachment Occlusion, R_UInt8
Creating render framebuffer AOTemp2 for pass AOBlurV (852x662)
 + attachment Occlusion, R_UInt8
Creating render framebuffer AOBuffer for pass AOBlurH (426x331)
 + attachment Occlusion, R_UInt8
Creating render framebuffer ForwardBuffer for pass DeferredLighting (852x662)
 + attachment Color, RGBA_Float16
Creating render framebuffer HDRBuffer for pass ForwardShading (852x662)
 + attachment Color, RGBA_Float16
 + attachment Depth, Depth32
Creating render framebuffer FXAABuffer for pass HDR (852x662)
 + attachment Color, RGBA_UInt8
Creating render framebuffer Viewport for pass FXAA (852x662)
Initialized 1 textures
Initialized 1 textures
Reflections took 18 ms to scan 1 urls, producing 49 keys and 241 values 
[ERROR] Could not find JAR with native libraries.
[ERROR] Could not find JAR with native libraries.
No custom key configuration found, using default keybindings.
No custom key configuration found, using default keybindings.
Recreating framebuffers (852x662
Creating render framebuffer GeometryBuffer for pass Scene (852x662)
 + attachment NormalsMaterial, RGBA_Float16
 + attachment DiffuseAlbedo, RGBA_UInt8
 + attachment ZBuffer, Depth32
Creating render framebuffer AOTemp1 for pass AO (852x662)
 + attachment Occlusion, R_UInt8
Creating render framebuffer AOTemp2 for pass AOBlurV (852x662)
 + attachment Occlusion, R_UInt8
Creating render framebuffer AOBuffer for pass AOBlurH (426x331)
 + attachment Occlusion, R_UInt8
Creating render framebuffer ForwardBuffer for pass DeferredLighting (852x662)
 + attachment Color, RGBA_Float16
Creating render framebuffer HDRBuffer for pass ForwardShading (852x662)
 + attachment Color, RGBA_Float16
 + attachment Depth, Depth32
Creating render framebuffer FXAABuffer for pass HDR (852x662)
 + attachment Color, RGBA_UInt8
Creating render framebuffer Viewport for pass FXAA (852x662)
Done initializing SciView

However, then it seems ok and the demos seem to work fine. However I had some further issues:

  • Zooming just doesn’t work at all. SciView recognises shift (for rotation) and scrolling (in the menu at the side), but shift + scroll doesn’t do anything.
  • I can’t open images > 2GB, which I seem to remember is the size of a Java array. Is this a hard limit? The error is:
[ERROR] null
java.lang.IllegalArgumentException: Negative capacity: -2031713896
	at java.nio.Buffer.<init>(Buffer.java:199)
	at java.nio.ByteBuffer.<init>(ByteBuffer.java:281)
	at java.nio.ByteBuffer.<init>(ByteBuffer.java:289)
	at java.nio.MappedByteBuffer.<init>(MappedByteBuffer.java:89)
	at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:119)
	at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
	at sc.iview.SciView.updateVolume(SciView.java:1386)
	at sc.iview.SciView.addVolume(SciView.java:1344)
	at sc.iview.SciView.addVolume(SciView.java:1265)
	at sc.iview.SciView.addVolume(SciView.java:1242)
	at sc.iview.SciView.open(SciView.java:997)
	at sc.iview.commands.file.Open.run(Open.java:67)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService$3.call(DefaultThreadService.java:238)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

I figured that maybe larger images won’t be supported in favour of BDV volumes, so I tried opening a file that worked fine in BDV and BigVolumeViewer. I wasn’t sure how to open it, so opening the .h5 gives:

BDVReader initializing /mnt/nvme/adam/sciview/export.h5
Timepoint :t00000
Setup :t00000/s00
Site :t00000/s00/0
Site :t00000/s00/1
Site :t00000/s00/2
Site :t00000/s00/3

But then nothing happens. Trying to open the .xml gives a wall of output about rendering, and a lot of printed code, then the errors:

[WARN] No binding found for tpietzsch.cache.TextureCache@5ac0e054 (PBO)
[WARN] No binding found for tpietzsch.example2.LookupTextureARGB@8732589 (Texture3D)
[ERROR] Binding for tpietzsch.example2.LookupTextureARGB@8732589 not found, despite trying deferred binding. (binding=BindingState(binding=1, uniformName=null, reallocate=false)/sampler=null)

So after nothing worked, I decided to read the instructions, and saw that at github.com/scenerygraphics/sciview/wiki it says to download the “no JRE” version of FIJI, which I did. Running ./ImageJ-linux64 gives:

Could not load Java library '/usr/lib/jvm/java-11-openjdk-amd64/lib/amd64/server/libjvm.so': /usr/lib/jvm/java-11-openjdk-amd64/lib/amd64/server/libjvm.so: cannot open shared object file: No such file or directory
Warning: falling back to System JVM
Unrecognized option: -Xincgc
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

I then changed to Java 8 with sudo update-alternatives --config java, and then FIJI loaded fine, and I repeated everything above, but the results are exactly the same.

SciView looks really promising, so I’d really like to get this working. If you have any ideas, I’d be grateful.

Thanks,
Adam

Workstation specs if helpful:

  • Ubuntu 18.04
  • Dual Xeon 6132
  • 384GB RAM
  • NVIDIA TITAN RTX
  • OpenGL version string: 4.6.0 NVIDIA 418.56
  • Java:
$ sudo update-alternatives --config java
There are 2 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                            Priority   Status
------------------------------------------------------------
  0            /usr/lib/jvm/java-11-openjdk-amd64/bin/java      1101      auto mode
  1            /usr/lib/jvm/java-11-openjdk-amd64/bin/java      1101      manual mode
* 2            /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java   1081      manual mode