Set axis for large number of data (images)

Hi there,

I hope I am not polluting the forum with some question already asked and I used the right wording to search for an answer.

I am discovering Ilastik (very nice!) and I have been struggling with a basic problem which I cannot believe does not have a solution. So far I have been trying the pixel classification workflow.

I try to make use of a large number of images following the axis convention yxc. Unfortunately when I import them (from h5, tiff or npy files) the software consistently (but wrongly in my case) assigns them the axes zyxc (the last dimension being 1).
It seems possible to update the axes using the “Edit shared properties” option (right click on any image in the 1. Input Data -> Raw Data tab after all images were selected and update this parameter, but somehow this fails with the error:

Incompatible dataset
---------------------------
Constraint of 'Pixel Classification' applet was violated: All input images must have the same number of channels.  Your new image has 30 channel(s), but your other images have 1 channel(s).

So:
Is it possible to precise the axis upon import of the images?
Alternatively, can one precise the axes as (for instance) h5 metadata?

OR did I miss something in the GUI and I should proceed differently?

I control the generation of the data so I could also make use of some other image format if necessary.

Cheers,
Guillaume

Hello @gpotdevin,

your question definitely doesn’t sound like anything I’ve seen around here yet. But in any case no worries :slight_smile:

I’m trying to reproduce the issue here, but did not succeed so far, so I think I might be missing some details.

So, first of all, which version of ilastik are you using?

And then for your data, how do you produce the different formats (since you’re exporting to npy I assume it’s from python). Could you just give me an example shape/datatype for one of your images, so I can produce something similar?

Cheers
Dominik

Hi Dominik,

What a fast answer!

I was using ilastik-1.3.3rc2 and I also upgraded today to ilastik-1.3.3post1 but did not see any change. The platform is windows 10.

I indeed make use of python to export the data. The various libraries I have used to generate the images were
rasterio for the tif images
h5py to create basic h5 files
numpy for the npy files.

Here is a minimal script which allowed me to reproduce my problem:


import numpy as np
import pathlib

arr = np.random.randint(0,256,size=[200,200,30])
arr2 = np.random.randint(0,256,size=[200,200,30])

export_path = pathlib.Path(r'c:\temp')

np.save((export_path/'ex1.npy').as_posix(), arr)
np.save((export_path/'ex2.npy').as_posix(), arr2)

to reproduce the problem I:

  • launched the Ilastik.exe file
  • selected Create New Project -> Pixel Classification
  • stored the project .ilp file on my local disk
  • clicked on the 1.Input Data -> Raw Data Tab -> + Add New… button -> Add separate Image option
  • selected the two exported files ex1.npy and ex2.npy from the explorer window

-> the two “data” are listed with the Axes parameter = zyxc.
shape = (200,200,30,1)

  • selected both files (shift key + mouse clicks)
  • right click -> edit shared properties
  • entered yxc in the Interpret axes as field
  • clicked the OK field.

I hope these are enough information to reproduce the problem/workflow I follow.

cheers,
Guillaume.

Hey @gpotdevin,

thanks a lot for this detailed account, I can nicely reproduce the issue. Unfortunately this is a bug :bug: :confused:

If you want a quick workaround you can do the following (I blatantly copied all of your script here):

import h5py
import json
import numpy as np
import pathlib

arr = np.random.randint(0, 256, size=[200, 200, 30])
arr2 = np.random.randint(0, 256, size=[200, 200, 30])

export_path = pathlib.Path(r"c:\temp")


def write_h5_with_axistags(filepath, data):
    """Convenience function to save data along meta data (axis descriptions)
    """
    # define axes description:
    axistags = {
        "axes": [
            {"key": "y", "typeFlags": 2},
            {"key": "x", "typeFlags": 2},
            {"key": "c", "typeFlags": 1},
        ]
    }

    with h5py.File(str(filepath), "w") as h5file:
        dataset = h5file.create_dataset("data", data=data)
        dataset.attrs["axistags"] = json.dumps(axistags)


for filepath, data in zip(
    [export_path / "ex1.h5", export_path / "ex2.h5"], [arr, arr2]
):
    write_h5_with_axistags(filepath, data)

This workaround only works with hdf5, as we understand to read meta-data there. And what I added here was adding descriptions for the axes in a format that ilastik understands…

if you have any questions, please go ahead, and please let me know if this is a viable solution.

Hi!

This is certainly a viable solution for me, actually pretty ideal.
I sort of thought there was a way to encode the information in an h5 file, but I was not sure exactly on how to do it. I had a quick look at the ilastik project files but their structure is somewhat (and as one would expect) more complex.

Actually I would encourage you to publish this somewhere, as this could well be useful for other people (along with more attributes? Warning from a bug report -> feature request).

Thanks a lot for the detailed answer, I am convinced I can make it work now.

Have a good week-end,
Guillaume

1 Like