It´s related about .ndpi file extension?

Hello everyone,
I have a .ndpi image of around 4 and 5 GB. Each image has 9 level_count (from 1.0 to 256.0) with 3 channel © each and 11 slices per channel (Z-stack).
So, it means: 1 count or series has 3 channel with 11 (Z) per slice = total 33 images.
I can visualize all this information using ImageJ or Fiji software. I can also load the image in python using the openslide-python library, using it I can see, dimensions, level_dimnesions, and etc.
My questions:

  1. My aim is to extract each count or series with their respective each Channel and each Z slices. So in my case for 1 series —3 channel ---- 11 slices per channel (33). (I want to extract 33 images per each 9 count or series). How can I achieve it using openslide library in python?
  2. I can see © channel and (Z-stack) slices notation for these images in an ImageJ software, but I couldn´t find these Z-stack or C-channel notation in python while using openslide library. So how to solve this problem?
  3. And lastly, how can I convert the openside properties in an XML format?

I have devoted lots of my time to solve this but I didn´t succeed. So any help would be appreciated.
Thank you.

It looks like OpenSlide ignores focal planes in NDPI:

1 Like

QuPath should support .ndpi z-stacks, assuming they can be read by Bio-Formats: https://qupath.github.io

There are then different ways to export regions as needed: https://qupath.readthedocs.io/en/latest/docs/advanced/exporting_images.html

Hi,
Thanks for your quick reply but I was planning to do the whole thing in python environment.
Is there any other library or package in python which can handle these kinds of things?
:slightly_smiling_face:

Hello there,
So is there any way to solve this problem or is there any other package which i can use in python to solve this problem?
:slightly_smiling_face:

You can use paquo as a Python library that interacts with QuPath:

I’m not aware of any Python-friendly library for working with .ndpi files apart from OpenSlide, which as @cgohlke points out doesn’t handle z-stacks (nor does it handle most non-RGB images).

I’ve only seen a few .ndpi stacks, and Bio-Formats could open them – but that’s already Java.

Since .ndpi is TIFF-related (albeit not a very normal TIFF), if you really want to do things in Python then you might need to work with TIFF at a lower level. I think @cgohlke is the expert on that :slight_smile:

Thanks for your comments. I will go through your suggestion and let you know, whether my problem is solved or not. I need to see what are the attributes I will get after I load the .ndpi file using Paquo with QuPath interface. Hope it works :slightly_smiling_face:

You could try tifffile together with the imagecodecs and zarr packages:

$ python -m pip install tifffile imagecodecs zarr

Tifffile can give you detailed information about NDPI files. Whether tifffile can uncompress the JPEG compressed image data depends on the size of the images. If the compressed size is larger than ~2 GB, imagecodecs cannot decompress the JPEG stream. If you are not using Windows, imagecodecs cannot decompress JPEG images with widths or lengths >65535. These are limitations of the libjpeg/libjpeg-turbo libraries used by imagecodecs.

Try:

import tifffile
import zarr

filename = 'filename.ndpi'

# print detailed information about the NDPI file
with tifffile.TiffFile(filename) as tif:
    print(tif)
    for page in tif.pages:
        print(' ', page)
        for tag in page.tags:
            print('  ', tag)
        print('   NDPI_TAGS =', page.ndpi_tags)

# separate image resolutions, slices, channels to uncompressed TIFF files
with tifffile.imread(filename, aszarr=True) as store:
    group = zarr.open(store, mode='r')
    assert isinstance(group, zarr.Group)
    for r in group.keys():
        stack = group[r]
        assert stack.ndim == 4  # ZYXC
        for z in range(stack.shape[0]):
            zslice = stack[z]
            for c in range(stack.shape[-1]):
                print('.', end='')
                image = zslice[..., c]
                tifffile.imwrite(f'_r{r}_z{z}_c{c}.tif', image)
            del zslice
1 Like

Hello there, Good morning,
I tried your code. But as you said it won´t help me if the image size is more than 2GB. In my case, the input files size is more than >4gb. I got an error whether I pass input of 189 MB or 4GB, but errors are different.

Errors: When I pass the 4GB file.
Traceback (most recent call last):
** File “/Users/yubraj/PycharmProjects/VIdeo_frame_extraction/ESR4-Codes/Temp-2.py”, line 29, in **
** with tifffile.imread(input_any1) as store:**
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/site-packages/tifffile/tifffile.py”, line 721, in imread**
** return tif.asarray(kwargs)
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/site-packages/tifffile/tifffile.py”, line 2809, in asarray**
** result = stack_pages(pages, out=out, maxworkers=maxworkers)**
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/site-packages/tifffile/tifffile.py”, line 13033, in stack_pages**
** for _ in executor.map(func, pages, range(npages)):**
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/concurrent/futures/_base.py”, line 598, in result_iterator**
** yield fs.pop().result()**
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/concurrent/futures/_base.py”, line 435, in result**
** return self.__get_result()**
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/concurrent/futures/_base.py”, line 384, in __get_result**
** raise self._exception**
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/concurrent/futures/thread.py”, line 57, in run**
** result = self.fn(self.args, self.kwargs)
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/site-packages/tifffile/tifffile.py”, line 13024, in func
*
** kwargs)
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/site-packages/tifffile/tifffile.py”, line 5648, in asarray**
** func=func, lock=lock, maxworkers=maxworkers, sort=True**
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/site-packages/tifffile/tifffile.py”, line 5512, in segments**
** yield decode(segment)**
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/site-packages/tifffile/tifffile.py”, line 5501, in decode**
** result = keyframe.decode(args, decodeargs)
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/site-packages/tifffile/tifffile.py”, line 5406, in decode
*
** shape=shape[1:3]**
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/site-packages/imagecodecs/imagecodecs.py”, line 807, in jpeg_decode**
** raise exc**
** File “/Users/yubraj/.conda/envs/Temp1/lib/python3.7/site-packages/imagecodecs/imagecodecs.py”, line 795, in jpeg_decode**
** outcolorspace=outcolorspace, shape=shape, out=out)**
** File “imagecodecs/_jpeg8.pyx”, line 318, in imagecodecs._jpeg8.jpeg8_decode**
imagecodecs._jpeg8.Jpeg8Error: Empty JPEG image (DNL not supported)

Process finished with exit code 1

Error: When I pass 189 MB input image.
Traceback (most recent call last):
** File “/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/pydevd.py”, line 1448, in _exec**
** pydev_imports.execfile(file, globals, locals) # execute the script**
** File “/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py”, line 18, in execfile**
** exec(compile(contents+"\n", file, ‘exec’), glob, loc)**
** File “/Users/yubraj/PycharmProjects/VIdeo_frame_extraction/ESR4-Codes/Temp-2.py”, line 29, in **
** with tifffile.imread(input_any2) as store:**
AttributeError: enter

System:
Macbook
Python 3.7, Pycharm

And one thing, I don´t think there´s an aszarr command in a tifffile.imread function

Still looking for help.

Tifffile is out of date. Install the latest version.

I doubt that the size of a JPEG compressed stream in your 5 GB file is more than 2 GB. If you have the chance, try the imagecodecs wheels from PyPI on Windows, which includes a patched version of libjpeg-turbo for decoding images with widths or heights >65535.

Hi there,
I tried your suggestion in window´s OS.
But now i suffered from a memory error i guess.

Line No: 29: zslice = stack[z] # Has a problem.

Is there any way to ignore this error?

You can try to add a del zslice statement after the for c in range loop. Otherwise you need more RAM, at least 32 GB.

Looks like the ndpisplit command line tool is able to split NDPI files into separate files for each level and slice. It rewrites the MCUs of the ginormous JPEG encoded strips to smaller TIFF tiles.

Hi there,
zslice = stack[z] won´t let me to another step. It through´s me an error related to memory.

You mean to say this right;
zslice = stack[z]
** for c in range(stack.shape[-1]):**
** del zslice**
but this code can´t run further line 29.

Yeah, I run this command yesterday and I received output as my choice in a Tif format (But I didn´t understand why output size was small (421.37 MB per slice) even though image Dimension was 186496 × 36608) Did I do something wrong or it just did what you have said just a minute ago.
And, I was looking for a python based splitter like your code. Although I found one python code who has used ndpiTools but it didn´t work. I think there´s something missing on that https://github.com/fepegar/ndPytools/blob/37b9166aa7e9faccf75d86df2bcafb92a96cf056/ndpitools.py)

I passed this command:
ndpisplit /Users/yubraj/Desktop/ESR4-important-folders/Dataset/Third-Sample-dataset/1/01.ndpi

That looks correct if 11 slices are about 4-5 GB. The output TIFF files are still using JPEG compression but are manageable by normal TIFF readers.

Use Python’s subprocess module.