Light-sheet microscopy file processing

Hey, fellow microscopy fans!

I am working with zebrafish early development and image them using Zeiss Z1. I have the embryo imaged from 4 different angles and then fuse them using BigStitcher plugin in Fiji. Now, I wonder - what are your go-to methods and most importantly, why? I wonder, if it would be better to use Zen blue, instead of Fiji, for instance. And what do you do to then work with the 3D file? What program/plugin is your go-to?

I am basically working on this with a lot of enthusiasm, but sadly have also a lot of gaps in my knowledge, so I would love to learn something from this discussion and hopefully piece some things together!

1 Like

Hi @zebrafish.fan and welcome to the forum!

In general, the choice of tool depends on what you would like to accomplish, and what you have access to.

For stitching and fusing, I think the BigStitcher is a great choice. If you are happy with the result, then I think that you do not need to change the tool.

To view the data you can use BigDataViewer in Fiji, which directly reads the h5/xml file from BigStitcher.

For image processing and analysis tasks I often used Imaris (commercial software) in the past or just stayed within Fiji if possible.
What exactly would you like to do with the 3D file?

1 Like

I am happy with the stitching in BigStither for sure, but after that I would like to crop the 3D file (there are some slices that are not nescessary) and also make some movies and snapshots - for that I need to position it in a different angle than originally imaged (the finished .TIF file is not in the direction I want it to be, if you know what I mean). I will have access to Imaris soon, but what would you use for this purpose within Fiji? As much as I know, the BigDataViewer only presents the data, but there is no way to take snapshots and videos?

Hi @zebrafish.fan

The BigStitcher also offers to crop the data, either manually or automatically based on intensity. You can set a “Bounding Box” and select it in the Image Fusion dialog Advanced Image Fusion.

In the same dialog, I would further recommend to set “Fused image” to “Save as new XML project (HDF5)”.

You can open the resulting h5/xml file with BigDataViewer but also with standard Fiji using the Bio-formats importer.

In the BigDataViewer, you can save movies using Tools > Record Movie.

When opening the image using the Bio-Formats importer, you can also create more sophisticated animations using a plugin called 3Dscript.

Of course, Imaris is also very well suited for this task.

1 Like

This very helpful!
I will play around with the BigDataViewer and BigStitcher!
I have tried to use the 3D script plugin but there is some kind of issue on our imaging computer since it gives the following error:

(Fiji Is Just) ImageJ 2.0.0-rc-69/1.52p; Java 1.8.0_172 [64-bit]; Windows 7 6.1; 7128MB of 147228MB (4%)

java.lang.RuntimeException: OpenCL error 4294967292: CL_MEM_OBJECT_ALLOCATION_FAILURE in H:\fijiplugins\3D_Animation\Release_2019-11-25\3Dscript\3D_Animation_Native\Raycaster.cpp (line 262)

I tried to do the following (https://docs.substance3d.com/spdoc/gpu-drivers-crash-with-long-computations-128745489.html) but it didn’t help – nothing changed and it is weird that the error still pops up 18s after I start the plugin even when I set it to 60s to be the cutoff time (as proposed in the tutorial in the link).

Would you know what this is all about? :slight_smile:

And all the help already is super appreciated. I think I will go far just with the tips you gave me about BigDataViewer and BigStitcher! Thank you!

Hi @zebrafish.fan,

I am happy if the info is useful to you!
I am not familiar with 3Dscript, but I can tag its author @bene.schmid, maybe he can help you out.

Also, if you did not have done so already, have a look at the example gallery. It seems that making movies of images opened in BigDataViewer is also supported, which is amazing!
Can you reproduce example 6 (“Rendering of an MRI image volume of a human head using the BigDataViewer”) on your computer?

You can open the MRI image volume in Fiji with File > Open Samples > T1 Head and even view it in BigDataViewer using Plugins > BigDataViewer > Open Current Image

Hi,
the CL_MEM_OBJECT_ALLOCATION_FAILURE exception indicates that you are running out of memory on your GPU. 3Dscript, unlike Imaris, requires that the entire (multi-channel) z-stack fits in your GPU memory.

So as a workaround, you could try to crop and downsample your data set, if this is an option for you. Another option is to use a computer with a more powerful graphics card.

2 Likes

Hi @bene.schmid,
Thank you for the quick reply!
I am very confused how I can run out of the GPU since the plugin runs on my laptop (windows 10) that has a GPU memory of 7.9 GB is opening the plugin (it is slow, yes, but it opens).
While the machine that is not able to open the plugin has a GPU of 102 GB… It is however on Windows 7… Do you think that could be the issue?

Hi @CellKai, thank you for putting me in contact with @bene.schmid. Hopefully we can figure this out!
I have not looked at the example gallery, that is awesome! Thanks! I will try to reproduce the example and let you know how this goes!

Hmmm, 3Dscript writes a log file (you’ll most likely find it in the Fiji.app folder). Can you check in there if you find a hint.
Is your bigger machine using a single GPU?

Hi @bene.schmid, I found the log file but I am not able to see anything that could point out what is wrong with it. Also, here is the full exception log error. I am assuming that the bigger machine only has one GPU. I hope it is okay that I attach the files here so that you can take a look :slight_smile: I know it is a lot of text and I am super duper grateful that you are taking a look at this!!!

exception error
(Fiji Is Just) ImageJ 2.0.0-rc-69/1.52p; Java 1.8.0_172 [64-bit]; Windows 7 6.1; 5287MB of 147228MB (3%)

java.lang.RuntimeException: OpenCL error 4294967292: CL_MEM_OBJECT_ALLOCATION_FAILURE in H:\fijiplugins\3D_Animation\Release_2019-11-25\3Dscript\3D_Animation_Native\Raycaster.cpp (line 262)

at animation3d.renderer3d.OpenCLRaycaster.setTexture16(Native Method)

at animation3d.renderer3d.OpenCLRaycaster.setImage(OpenCLRaycaster.java:156)

at animation3d.renderer3d.OpenCLRaycaster.<init>(OpenCLRaycaster.java:97)

at animation3d.renderer3d.Renderer3D.<init>(Renderer3D.java:27)

at animation3d.main.InteractiveRaycaster.run(InteractiveRaycaster.java:151)

at ij.plugin.filter.PlugInFilterRunner.processOneImage(PlugInFilterRunner.java:266)

at ij.plugin.filter.PlugInFilterRunner.<init>(PlugInFilterRunner.java:114)

at ij.IJ.runUserPlugIn(IJ.java:232)

at ij.IJ.runPlugIn(IJ.java:193)

at ij.Executer.runCommand(Executer.java:137)

at ij.Executer.run(Executer.java:66)

at java.lang.Thread.run(Thread.java:748)

log
CL_DEVICE_AVAILABLE: 1
CL_DEVICE_COMPILER_AVAILABLE: 1
CL_DEVICE_ENDIAN_LITTLE: 1
CL_DEVICE_EXTENSIONS: cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_d3d9_sharing cl_nv_d3d10_sharing cl_khr_d3d10_sharing cl_nv_d3d11_sharing cl_nv_copy_opts cl_khr_gl_event cl_nv_create_buffer cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_kernel_attribute_nv
CL_DEVICE_GLOBAL_MEM_SIZE: 4294967296
CL_DEVICE_IMAGE_SUPPORT: 1
CL_DEVICE_IMAGE_MAX_BUFFER_SIZE: 134217728
CL_DEVICE_IMAGE2D_MAX_HEIGHT: 16384
CL_DEVICE_IMAGE2D_MAX_WIDTH: 16384
CL_DEVICE_IMAGE3D_MAX_DEPTH: 4096
CL_DEVICE_IMAGE3D_MAX_HEIGHT: 4096
CL_DEVICE_IMAGE3D_MAX_WIDTH: 4096
CL_DEVICE_LOCAL_MEM_SIZE: 49152
CL_DEVICE_MAX_COMPUTE_UNITS: 5
CL_DEVICE_MAX_MEM_ALLOC_SIZE: 1073741824
CL_DEVICE_MAX_SAMPLERS: 32
CL_DEVICE_MAX_WORK_GROUP_SIZE: 1024
CL_DEVICE_MAX_WORK_ITEM_DIMENSIONS: 3
CL_DEVICE_MAX_WORK_ITEM_SIZES[0]: 1024
CL_DEVICE_MAX_WORK_ITEM_SIZES[1]: 1024
CL_DEVICE_MAX_WORK_ITEM_SIZES[2]: 64
CL_DEVICE_NAME:Quadro K2200
CL_DEVICE_PROFILE:FULL_PROFILE
CL_DEVICE_TYPE: FULL_PROFILE
CL_DEVICE_VENDOR: NVIDIA Corporation
CL_DEVICE_VERSION: OpenCL 1.2 CUDA
CL_DRIVER_VERSION: 441.66
Supported formats:
CL_R, CL_FLOAT
CL_R, CL_HALF_FLOAT
CL_R, CL_UNORM_INT8
CL_R, CL_UNORM_INT16
CL_R, CL_SNORM_INT16
CL_R, CL_SIGNED_INT8
CL_R, CL_SIGNED_INT16
CL_R, CL_SIGNED_INT32
CL_R, CL_UNSIGNED_INT8
CL_R, CL_UNSIGNED_INT16
CL_R, CL_UNSIGNED_INT32
CL_A, CL_FLOAT
CL_A, CL_HALF_FLOAT
CL_A, CL_UNORM_INT8
CL_A, CL_UNORM_INT16
CL_A, CL_SNORM_INT16
CL_A, CL_SIGNED_INT8
CL_A, CL_SIGNED_INT16
CL_A, CL_SIGNED_INT32
CL_A, CL_UNSIGNED_INT8
CL_A, CL_UNSIGNED_INT16
CL_A, CL_UNSIGNED_INT32
CL_RG, CL_FLOAT
CL_RG, CL_HALF_FLOAT
CL_RG, CL_UNORM_INT8
CL_RG, CL_UNORM_INT16
CL_RG, CL_SNORM_INT16
CL_RG, CL_SIGNED_INT8
CL_RG, CL_SIGNED_INT16
CL_RG, CL_SIGNED_INT32
CL_RG, CL_UNSIGNED_INT8
CL_RG, CL_UNSIGNED_INT16
CL_RG, CL_UNSIGNED_INT32
CL_RA, CL_FLOAT
CL_RA, CL_HALF_FLOAT
CL_RA, CL_UNORM_INT8
CL_RA, CL_UNORM_INT16
CL_RA, CL_SNORM_INT16
CL_RA, CL_SIGNED_INT8
CL_RA, CL_SIGNED_INT16
CL_RA, CL_SIGNED_INT32
CL_RA, CL_UNSIGNED_INT8
CL_RA, CL_UNSIGNED_INT16
CL_RA, CL_UNSIGNED_INT32
CL_RGBA, CL_FLOAT
CL_RGBA, CL_HALF_FLOAT
CL_RGBA, CL_UNORM_INT8
CL_RGBA, CL_UNORM_INT16
CL_RGBA, CL_SNORM_INT16
CL_RGBA, CL_SIGNED_INT8
CL_RGBA, CL_SIGNED_INT16
CL_RGBA, CL_SIGNED_INT32
CL_RGBA, CL_UNSIGNED_INT8
CL_RGBA, CL_UNSIGNED_INT16
CL_RGBA, CL_UNSIGNED_INT32
CL_BGRA, CL_UNORM_INT8
CL_BGRA, CL_SIGNED_INT8
CL_BGRA, CL_UNSIGNED_INT8
CL_ARGB, CL_UNORM_INT8
CL_ARGB, CL_SIGNED_INT8
CL_ARGB, CL_UNSIGNED_INT8
CL_INTENSITY, CL_FLOAT
CL_INTENSITY, CL_HALF_FLOAT
CL_INTENSITY, CL_UNORM_INT8
CL_INTENSITY, CL_UNORM_INT16
CL_INTENSITY, CL_SNORM_INT16
CL_LUMINANCE, CL_FLOAT
CL_LUMINANCE, CL_HALF_FLOAT
CL_LUMINANCE, CL_UNORM_INT8
CL_LUMINANCE, CL_UNORM_INT16
CL_LUMINANCE, CL_SNORM_INT16
CL_BGRA, CL_SNORM_INT8
CL_BGRA, CL_SNORM_INT16
CL_ARGB, CL_SNORM_INT8
CL_ARGB, CL_SNORM_INT16
8-bit image format supported by your OpenCL implementation
16-bit image format supported by your OpenCL implementation
Signed int8 RGBA image format supported by your OpenCL implementation

If your main aim is volume rendering and creating movies, it is also worth having a look at both napari and spimagine (both written in Python). Neither supports BDV format directly (yet) IIRC so you would have to export as tif. Spimagine has some support for keyframing and making movies and for napari @guiwitz also presented some work on movie export options.

Hi @zebrafish.fan,

I just noticed that your earlier post refers to main memory, not GPU memory. Seems I was too tired yesterday to realize…
So your main memory of the big machine is 102 GB, and the graphics card is an NVIDIA K2200 (according to the log file) which Google tells me to have 4 GB of memory. Would be interesting to know which graphics card is in your laptop…

Aha, yeah, that is correct, @bene.schmid. I am not tired, just have no clue about these things… :smiley:
So, my laptop has an integrated graphics card (intel® uhd graphics 620) which is then aiming towards that maybe the other machine has an issue with switching between GPU and CPU? Or is the 4GB not enough and we should switch out the graphics card? I also noticed something weird, the exception log is mentioning H: while we only have files on C: and D:. Could that be part of the issue?

@VolkerH, that sound interesting but I am not very fluent in python. But I will try to look into it! Is there a lot of programming involved?

The Intel GPU UHD Graphics 620 doesn’t have dedicated graphics memory but uses the main memory of your laptop (as far as I know, it’s set in the BIOS how much it may use).
What dimensions does your data set have? Is it 8-bit or 16-bit?
The H: in the error message doesn’t have anything to do with it, it just uses compile time paths for error messages, just ignore it.

@bene.schmid, Yeah, I thought that was so, and it works at least. But the laptop is otherwise just not powerful enough to handle the file…that’s why I am keen on getting it to work on the imaging computer.

The dataset was 16-bit. I tried now with 8-bit and it gave me a different exception: CL_OUT_OF_RESOURCES (the same for the rest of the error e.g. raycaster etc.)

What are the dimensions of the dataset you want to visualize? I.e. width, height, depth and number of channels?

@bene.schmid, I have attached the properties of the 2 files I have tried that work on my laptop but not on the computer in question and give the same error. Do you think it is because of the file size/ graphics card size? Should we get a different graphics card? Could it be because the computer is on windows 7? Is it because of weird file locations (different drives)?

I don’t understand what is the problem and it seems like we will never find out…

I just want to use your awesome plugin on a computer that won’t crash every time I try to turn the volume… :rofl:

Thank you for helping me though!

Hi @zebrafish.fan,

Your screenshot don’t say anything about width and height, so I cannot calculate how much memory they’ll occupy theoretically. Since your data has isotropic resolution, I’ll just assume for now width and height is roughly equal to the number of slices. Then, the left data set would require a minimum of around 4 GB, and maybe this is just at the edge of what your laptop still handles (with the help of shared memory, i.e. using system memory), while your workstation is limited to 4 GB and might fail therefore. The right data set will require something like 16 GB (again assuming width and height being equal to the number of z slices) and should therefore fail on both computers.
I’d be curious about the total available graphics memory on your laptop. You can find the numbers if you right-click on the Desktop > Screen resolution > Advanced settings > Adapters (for Windows 7; I don’t have a Windows 7 system at hand, just copied these steps from the internet). In Windows 10, you’d right-click the Desktop > Display settings > Advanced display settings > Display adapter properties.

For now, I’d suggest to downsample your data, using ImageJ’s >Image>Adjust>Size… command, or crop it if possible.

And yes, I’m convinced that all of this is due to the limited memory available on your GPU, so a graphics card with more memory would definitely help.

Best wishes