I have some light sheet data acquired using Micro-manager, all time-channel-z-y-x hyperstacks. The size of the dataset is between few hundreds of GB to a few TB. And I am trying to read out the data using tifffile in Python for further processing.
Micro-manager saves the hyperstack into many 4GB-sized tiff files. It can handle the dataset well using virtual stack and I can explore through the dataset easily.
However, when I try to read the data using tifffile in Python, I got errors:
“TiffPage 0: TiffFileError: invalid tag value offset”.
I tried a few solutions:
- Open the 4GB-sized tiff files with Fiji and save them to tiff files which are then readable by tifffile.
However reading out the tiff files gets very slow with Fiji after the first few tiff files.
- Saving the hyperstack with Micro-manager to single-page tiff files. This strategy worked for a dataset which had ~ 100k single-page tiff files. But for another dataset where I have over 1 million single-page tiff files, saving them into a single directory apparently made things very slow again.
Anyone here in the forum having any suggestion or experiences with similar problems, I would really appreciate your inputs.
Thanks for your attention,