New nuclear segmentation tool for complex 3D datasets



Dear Community,

I would like to announce a new tool for segmenting nuclei in 3D. We tested the tool for ‘difficult situations’ such as mouse embryos and 3D cell culture. We now use this in routine in our lab and it turns out to be very useful for us.

You can read all about it on BiorXiv.

  • Source code and installation instructions are available on GitLab.
  • Video Tutorial available on PeerTube

I hope this can be useful, any comments are very welcome!

Merry Xmas!

PS: this runs as a standalone but we have plan to port this as an ImageJ plugin.


This is excellent and should provide several useful tools to the community. In 2015, I also used a ridge steerable filter approach based on Aguet’s, Jacob’s, and Unser’s algorithms to segment the nucleus using structured illumination microscopy with a focus on the details of the nuclear lamina in a single cell:

Your code contains an independent implementation of the ridge steerable filters as designed in Jacob, M. and Unser, M. (2004). Unfortunately, SteerableJ, the current plugin by François Aguet (Broad Institute) that implements that 2D algorithm has fallen out of maintenance since I think his interests have moved on to genetics ( , last SteerableJ release is from 2012 ). It thus may be useful for your planned plugin to provide access to the ridge segmentation algorithms independent of the overall nuclei segmentation. Also note that a 3D steerable filter plugin was released earlier this year, which I believe extends the Unser algorithms further: by @pam66 . There seems to be some demand for a configurable 2D steerable ridge filter implementation, Set SteerableJ parameters from a macro , distinct from the ridge detector plugin based on Steger, C., 1998 implemented by @twagner.

Regarding the biology, your work uses Lamin B1 (LB1) and the Emerin transmembrane ™ domain as a nuclear envelope (NE) marker or specifically an inner nuclear membrane marker. This should prove interesting since the Allen Cell Institute is releasing images of GFP tagged LB1 in iPS cells. While I would expect the labeling to closely follow the NE, the structure where the LB1 fluorohpore resides might be more precisely termed the “nuclear lamina” (NL), which is thought of as distinct from the NE: . I admit though that the literature often refers to the NL as part of the NE despite NL being outside the double plasma membrane structure. One caveat is that the ab16048 antibody may also label Lamin B2 in mice, and the Abcam KO validation was only done on human cells.

The Emerin TM domain is arguably a more direct marker of the NE since it lacks the LEM domain of Emerin although I think the mKate2 fluorophore may reside in a similar space as the NL. I wonder if including the LEM domain may shift more of the localization from the ER to the nucleus. Overall, I agree with the approach of segmenting the nucleus using NE/NL markers.

Thanks for sharing. I look forward to seeing this extended into an ImageJ plugin.


Hi @markkitt

Thanks very much for the detailed comment and apologies for the slow reply.

Regarding the steerable filter implementation, I adapted the code from the Icy’s feature detector which, as you may know is Francois Aguet’s ImageJ plugin ported to Icy by Ricard Delgado-Gonzalo and Zsuzsanna Püspöki.

The code was GPL3 so Nessys is also GPL3 and I retained the notice on the source code. I hope this is the correct way to do this.

As you can see the algorithm is kept independent from the segmentation procedure. For Nessys, it is called from another class which can be plugged into a pipeline.

I hope this design will facilitate the creation of an independent plugin for the steerable filter.

Thanks for the pointer to the Set SteerableJ parameters from a macro post. I’d be happy to try make the plugin macro recordable. I would need to learn how to do this though, is using script parameters the way to go?

Also I am not sure when exactly I’ll find time to work on this and if other people are interested in helping I would be very happy to get in touch.

Thanks also for the precisions about the semantic NE/NL this is helpful. Overall do you think it is fine to just use the term nuclear envelop throughout the manuscript given the resolution at which nuclei are imaged? I guess we might as well use ‘nuclear contour’ - it would not work with the ‘Nessys’ acronym though.



Hi, do you have the test image stack? I want to have a try.
I saw the video, It seems that 3d ridge/watershed may be help.
But it may be better by the level set conception.


Sure you can download the test image from our owncloud server (password is ‘nessys’ - sorry I had to set a password for some reasons)
Is the level set method scalable to large number of nuclei in large images?


I think the method below may be ok:

  1. do a gaussian blur 3d
  2. find local min point in 3d
  3. use these points as seeds, do a level set in the original image stack
  4. filter the no ball-like block

But I need to find a 3d level set lib. It seems that ITK has.


That is news to me that the source was released in the JAR file under GPLv3. That seems to contradict the impression that @ctrueden had about the open source status of SteerableJ. Maybe it would be advantageous to improve the ImageJ interface based on the GPLv3 release.


You may want to look into a Python implementation of steerable filters:


Regarding NE/NL, I would just be explicit that when you say NE, you mean NE/NL given the resolution. However, we have observed circumstances where differences are observable between the NE and NL using confocal microscopy.


Thanks for the heads up, @markkitt. It is more than an “impression”; from the SteerableJ page:

Important note: You are free to use this software for research purposes, but you should not redistribute it without our consent. In addition, we expect you to include adequate citations and acknowledgments whenever you present or publish results that are based on it.

In other words: not an open source license. This is one big reason why the BIG plugins were all removed from the Fiji distribution in favor of an update site.

IANAL, but my understanding is that the people who are taking this code and adapting it to other frameworks and then releasing that adapted code as GPL have violated the license, unless they have a statement from an authority at EPFL giving permission to adapt and release as GPL.


@ctrueden The release of Feature Detector on ICY appears to come from Biomedical Imaging Group at EPFL itself. The authors of the ICY release, Ricard Delgado-Gonzaloand Zsuzsanna Püspöki, appear to have been associated with EPFL at the time of the release. As the copyright owners, BIG/EPFL could release code under a new license if they wanted, which appears to have happened.

From the ICY Feature Detector page, clicking on See Technical Details, leads to a link to download the JAR file. That version of the SteerableJ.jar file appears to lack ImageJ plugin code, but does contain a copy of the GPLv3 license as well as accompanying source code. Therefore, since

  1. the ICY release differs from the release of SteerableJ on the page you linked in that it contains ICY bindings and not ImageJ bindings ,
  2. ICY is GPLv3,
  3. the “Important note” you posted above would be incompatible with GPLv3 and is not present on the ICY release page, and
  4. GPLv3 is irrevocable

there is now a version of SteerableJ that is released under GPLv3. @Ghiomm, Nessys, or anyone should be able to release a new ImageJ plugin under GPLv3, but IANAL.

@Ghiomm, have you contacted BIG directly to explicitly confirm the GPLv3 code release and that it is not subject to additional terms? It would be helpful for the community and your plugin to understand the specific licensing terms of their code and in turn your code.


Cool, thanks for digging into that, @markkitt!


I think the best solution is not to process locally, but do a gaussian 3d blur, then we can find the local min point with a tolerance. these point may be not very accurate, but should be in every hole, Then we can use these point as seed, like blow a balloon(with surface tension and pixel resistance), untill every balloon grow stable, then we got the result. only the growing process has some trouble, may be the ITK’s 3d level set fits. I would have a try in I have much spare time.


I agree there are many solutions to an image like this and that a watershed / level-set approach could work very well. I think it may work even better if you apply it to the ridge-enhanced filter response that Nessys uses rather than an isotropic Gaussian convolution filter.

If you look into the term “steerable filters”, you will see that the original filter design is based on the combination of convolving with a Gaussian followed by a derivative. It turns out that you combine both operations into a single convolution operation that can be interpolated in terms of filter angle. In the case of a ridge filter, even (2nd, 4th, etc.) derivatives are used.

This is the classic paper that coined the term:

The design and use of steerable filters
W.T. Freeman ; E.H. Adelson
The authors present an efficient architecture to synthesize filters of arbitrary orientations from linear combinations of basis filters, allowing one to adaptively steer a filter to any orientation, and to determine analytically the filter output as a function of orientation. Steerable filters may be designed in quadrature pairs to allow adaptive control over phase as well as orientation. The authors show how to design and steer the filters and present examples of their use in the analysis of orientation and phase, angularly adaptive filtering, edge detection, and shape from shading. One can also build a self-similar steerable pyramid representation. The same concepts can be generalized to the design of 3-D steerable filters.

The algorithm we have been specifically discussing here that is used in Nessys is based on the following paper:

Design of steerable filters for feature detection using canny-like criteria
M. Jacob ; M. Unser
We propose a general approach for the design of 2D feature detectors from a class of steerable functions based on the optimization of a Canny-like criterion. In contrast with previous computational designs, our approach is truly 2D and provides filters that have closed-form expressions. It also yields operators that have a better orientation selectivity than the classical gradient or Hessian-based detectors. We illustrate the method with the design of operators for edge and ridge detection. We present some experimental results that demonstrate the performance improvement of these new feature detectors. We propose computationally efficient local optimization algorithms for the estimation of feature orientation. We also introduce the notion of shape-adaptable feature detection and use it for the detection of image corners.

A more recent review and generalization of the concept can be found here:

A Unifying Parametric Framework for 2D Steerable Wavelet Transforms
Michael Unser and Nicolas Chenouard
Read More:

Anyways, I think we are veering off topic. Please post a new topic when you have implemented your method and then we can discuss the comparison.


Thanks for confirming that reusing this code should be fine in principle.
I have sent an email to BIG to ask for further confirmation, I’ll keep you posted.


@markkitt - I now have the confirmation from people at BIG that it is fine to redistirbute the code as long as the original paper is cited and authors are credited:

M. Jacob, M. Unser, “Design of Steerable Filters for Feature Detection Using Canny-Like Criteria,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 8, pp. 1007-1019

Source code: “The plug-in was originally implemented François Aguet for ImageJ, and was ported to Icy by Ricard Delgado-Gonzalo and Zsuzsanna Püspöki.”


From Ricard Delgado-Gonzalo via Twitter:

From Zsuzsanna Püspöki via LinkedIn:
Thank you for your message, sorry for the little delay with my reply.

Yes, feel free to use the plugin and redistribute it, with the right citations on the paper and the source code:

M. Jacob, M. Unser, “Design of Steerable Filters for Feature Detection Using Canny-Like Criteria,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 8, pp. 1007-1019

Source code: “The plug-in was originally implemented François Aguet for ImageJ, and was ported to Icy by Ricard Delgado-Gonzalo and Zsuzsanna Püspöki.”

Greetings from Switzerland,

The response from BIG and Püspöki raise some potential conflict with the GPLv3, but I think you are in the clear:

IANAL, but I recommend adding the paper citation to your


Thanks for your help in sorting this out.