Need to create .csv cell density file from 100 .h5 cell density files

I ran cell density on about 100 images but neglected to configure the .csv file name before I started the run. Since I also neglected to save the project before I started the run, I couldn’t kill the process and restart with the .csv file name configured, so I let it run all night. So now I have a bunch of .h5 files that contain the density numbers but in a form I can’t access as a table. Do I have to write Python code to generate the .csv file from the .h5 files? If so, where is the documentation on the .h5 file format? Has anyone else run into this problem and already written a Python script to do this?

PS: It seems rather strange that there is no default .csv file for cell density processing since one is obviously trying to get a single number per image, not a bunch of .h5 files. Indeed, I found no way to inhibit the creation of .h5 (or other image format) files since I don’t want them. Moreover, when I tried doing a batch process it didn’t seem to generate a .csv file even when configured to do so.

Hi @jabowery,

How did you run cell density? Cellprofiler?

The answers to this stackoverflow question have a few of methods for converting your h5 to csv

Do I have to write Python code to generate the .csv file from the .h5 files? If so, where is the documentation on the .h5 file format? Has anyone else run into this problem and already written a Python script to do this?

Two of the answers above use python, like this one:

import numpy as np
import h5py
np.savetxt(sys.stdout, h5py.File('foo.h5')['dataname'], '%g', ',')

Hope those ideas are helpful, let us know how it goes,
John

I generated the .h5 files with ilastik. I mislabeled the post as “imagej” rather than “ilastik”. Corrected.

1 Like

The .h5 file generated by ilastik’s default cell density workflow is just an image. That means ilastik’s cell density workflow defaults to not outputting cell density? That’s absurd. What’s going on?

Here’s an image (apparently a probability map) scaled to 255 max value. How does one turn this into a density?

Hi @jabowery,

I agree that not having a default location for the table is not very user-friendly. I guess no one has complained about this in the past, so that’s why it’s the way it is.

Seeing your data here in this thread and here I would not advice to use this workflow. It is meant for very crowded scenes of cells, rather than images where the cells are so nicely separate as in your case.
So I would suggest to first do pixel classification and then object classification. In object classification you can have classes for e.g. false detections, single cells, merger of two … and so on (judging from the image I cannot make out any large accumulations). You’ll then get a nice table with a lot of descriptors for each cell, but more importantly also with the predicted class.

1 Like

I’m pretty sure that even when I configures the .csv location, the batch mode failed to output a .csv.

You are right, in batch mode the table will only contain the number for the last dataset. You can use the headless mode to run the trained ilastik project with each of your input files and produce csvs for all of them that you’d later have to merge. But for this you’d have to be comfortable with the terminal/command line. I’d be happy to help, although I would still say that going the pixel classification + object classification should be the more accurate route to take.

On a further note, you could also try @mweigert’s startist (NEUBIAS seminar), once the imagej update site is back up again. It might work out of the box for you.

The cells aren’t always as nicely separated as the lower magnification makes it appear. At bottom is a high magnification showing chlorella cells in clusters of at least 4 due to mitosis (with cohesion and/or lack of agitation). In other images I’ve found up to 12 cells in a mitotic cluster.

Here’s what a cluster of 4 looks like at the lower magnification:
Capture

But even using the cell density workflow, with a lot of features selected (which is why it took all night to run 100 images) I found it difficult to get the cell density training to provide accurate counts.

I see. In that case you could maybe go by object size? Suppose you’d do pixel classification for foreground/background then a round of object classification for cleanup (there is probably some things you want exclude, out of focus cells maybe, too.). Then you could look at object sizes in pixels and infer the number of objects that way? (this would give you the ability to handle “unseen” numbers of accumulations, as opposed to training with fixed classes for n-number of objects in object classification).

I know this is an older thread, but I am having the same problem re: needing a .csv file which contains the cell density results for many images. I have trained the algorithm using the brushes according to the documentation, and to my eye it is getting very good results. However, now I am unable to run the rest of my images in a batch and export the density results to a .csv. I can only get .h5 files, which is a difficult format to use for a novice such as myself. It would be so great to be able to do this, because I think Ilastik cell density workflow is more accurate than the ImageJ workflow which I had been using for my project. I will try to use the link from above to convert from .h5 to .csv. In any case, I think Ilastik has a lot of potential and I will continue to work with it and learn more.

Hi @Stacy_Welker,

unfortunately there was no movement with regards to this issue, so the tips from this thread still stand. If you opt to using the headless mode and need help producing a single table from the results, we’re happy to help.

Cheers
Dominik

No worries, I’ll keep playing around with different workflows to see if I can come up with a good option for my project. Thanks to you and the team for putting in the development work!