DenoiSeg - Joint Denoising and Segmentation DL method available in Python and Fiji

Hey all,

during this NEUBIAS Academy webinar we introduced DenoiSeg, a Deep Learning tool based on Noise2Void and CSBDeep for Joint Denoising and Segmentation, credits to @mp007 @tibuch @akrull @fjug.

The method can achieve decent segmentation in cases where you have a solid amount of noisy images and only a few of them are labeled.

Here are the relevant resources:

  • for the theory, read the paper
  • for Python instructions (training and prediction), read this page
  • for Fiji instructions (training and prediction), read this page
  • video and FAQ from the NEUBIAS Academy webinar


Timelapse video from a ~7h training + prediction session

Limitations

  • 2D only for now, the 3D option will very likely follow soon

Notes on the preprocessing in Fiji

Please read the Prepare your data section in the Wiki for how to create the directories used to run the training. Here are some additional notes since the page is currently read-only and I can’t improve it:

  • You need two directories for the training (raw and labelings) where the names in the labelings directory match the source name in the raw directory, but not all raw images have to be labeled:
    img0 img4

  • … and two directories for the validation (raw and labelings), which should be a smaller set of matching raw images and labelings:
    img5 img3

  • In contrast to what I said in the webinar and what the wiki states, each labeling image of type int does not necessarily need an individual index per object, you can also just use 0 for background and 1 for foreground. You only need to use different indices for different objects if they touch each other (e.g. background=0, cell1=1, cell2=2) and if you care about distinguishing them. This way you give the training a chance to learn that there should be a border in between.

16 Likes

Hi, very interesting DenoiSeg plugin. I would like to test it via Fiji but after installing all the necessary recommended components. There was a problem after launching the DenoiSeg training command with data ‘denoiseg_mouse’:
[ERROR] java.lang.NullPointerException
at java.util.ArrayList.addAll (ArrayList.java:581)
at org.scijava.util.ClassUtils.getAnnotatedFields (ClassUtils.java:193)
at org.scijava.util.ClassUtils.getAnnotatedFields (ClassUtils.java:163)
.
.
. and many other at …
and
[ERROR] Module threw exception
java.lang.NullPointerException
at de.csbdresden.denoiseg.train.InputHandler. (InputHandler.java:95)
at de.csbdresden.denoiseg.train.DenoiSegTraining.init (DenoiSegTraining.java:167)
and some more at …
Is there any easy way to solve this, I’m not a programmer, I’m a biologist :slight_smile:
Thank you in advance for your help.
Dominik

Thanks for reminding me, that’s a bug somewhere in the core, uploaded a workaround so that this error should not be thrown any more by DenoiSeg. Sorry I had no time to test it, please update and let me know in case it’s still happening!

Thank you very much, I updated and indeed the error is gone and I went one step further. I was able to see a working woodpecker :). The woodpecker works by trying to load TensorFlow endlessly. In Console appeared: [ERROR] Module threw error
java.lang.NoSuchMethodError: net.imagej.tensorflow.TensorFlowService.loadLibrary () V
at de.csbdresden.denoiseg.train.DenoiSegTraining.init (DenoiSegTraining.java:176)
at de.csbdresden.denoiseg.command.DenoiSegTrainCommand.train (DenoiSegTrainCommand.java:145)

Do you have any idea how to deal with it? I am very impressed with this plugin and am very excited to use it for ecological research.

1 Like

Ecological research, yeah!! Would love to see the kind of data you are working on (also via DM, if you don’t want to share publicly)

The issue looks like you have an old imagej-tensorflow version installed, maybe coming from another update site. Can you please do the following:

  • Open the updater via Help > Update...
  • Click Advanced mode
  • in the search bar, type imagej-tensorflow

This is how it looks like for me:

Do you have another version / an update site other than CSBDeep where it’s coming from?

It might be the TensorFlow update site, which still ships an old version. We have not found the time to update this site as well. If you are not using the Microscope Focus Quality plugin, you can safely remove this update site, it does not need to be installed for any CSBDeep tools to work. In the future, we’ll probably merge both sites.

If an older imagej-tensorflow version is coming from an update site you want to keep, you can copy your Fiji installation or download a new one and have one update site installed on one installation and the other one on the other installation. Please let us still know about it, so we can work on fixing these incompatibilities.

Thank you very much again, I uninstalled the TensorFlow plugin and tried again. This time it worked but without a woodpecker :), but now is running, great! It will probably last until morning. I’m looking at how this works on mouse data. Meanwhile, I am preparing my own material. These are aerial photos (from a drone) of several colonies of water birds. If you would like to see and comment if I prepare my data well, I would be grateful. Let me know where I could send you my data.

1 Like

After training on mouse data (28 hours! :slight_smile: ), two windows with two models appeared. They were saved on C drive in the Temp folder. After clicking the Predict button in the model window, the selected image processed correctly, the background noise disappeared. However, there was a problem with the procedure after moving the models to another place on the hard disk.

  1. The model folder was not in zip format, it was just a normal folder so I zipped it up giving it the name: “example.bioimage.io”. I got an error when pointing to this location, I guess NullPioint or something. Currently, I can not recreate this procedure, because after the update, the predict window does not open at all.

  2. After updating when I go to: Plugins> CSBDeep> DenoiSeg> predict but also train + predict as well as: Plugins> CSBDeep> N2V> predict, train and train + predict, the following message appears: A RandomAccessibleInterval is required but none exist.
    Random

Obsolete now, everything is fine with the issues described above. It’s my fault, the first problem was that I didn’t find the location of the zipped model. When I found it, it all worked.
The second problem probably resulted from the duplication of software locations on two disks on my computer.