Identify individual grains in a ferrous microstructure

I have a microstructure of a ferrous material, seen below. I want to identify the individual grains so that later I can assign certain material parameters to them and to perform measurements on the grains (diameter, etc).
I see two challenges in this task:

  1. Preprocess the image so that the grains separate better and image processing is easier on them.
  2. Identify each grain.

Currently, I concentrate on step 2. I searched the forum and the internet for related problems, however,

  • I would like to obtain the individual grains as in this post but I already have the grains not only the nuclei
  • this example also considers a ferrous microstructure but is satisfied with determining the phases only
  • identify a few phases (such as ferrite and bainite)
  • I think my task would be similar to this post

When only a few phases need to be identified, the Trainable Weka Segmentation plugin of FIJI worked well on another microstructure. However, the task here is to identify all grains. And in realistic microstructures, there are about 2000 grains which are clearly intractable manually.

Approaches I thought of:

  • obtain internal and boundary edges
  • perform segmentation, either
    • level-set based segmentation (Plugins -> Segmentation -> Level Set in FIJI)
    • particles (Analyze -> Analyze Particles… in FIJI)

The level-set based segmentation requires initial seeds in the image. The problem is, how to obtain the grain centres as seeds? I would like a tessellation as an output, so the grains should completely cover the region of interest without any gaps.
I am completely new to image processing (actually this task is part of a larger project), but I tried using FIJI.


The microstructure:


Thanks


P.S.: It may be possible to redo the microscopy and obtain a better quality image, but the goal remains the same: identify each grain.

I think superpixel and reg merge may work.
https://scikit-image.org/docs/stable/api/skimage.segmentation.html

4 Likes

Thanks, this is impressive! I didn’t find reg merge, did you mean RAG merging? Did you use skimage.segmentation.slic to achieve this?

  1. skimage.segmentation.quickshift
  2. skimage.future.graph.cut_threshold

the first step may over segment, so ues the second step to merge.
that is what I use to got the result upon, but you can also try slic, quickshift + rag merge, cut normalized…

unfortunately, these method has some parameters, and they are very sensitive, so you need try them many times.

So I would recomend you http://github.com/image-Py/imagepy, an python framework like imagej. It contains many method in scipy.ndimage and scikit-image, And you can extend it by plugin with any lib based on numpy.


if you want to write python code, you can use ImagePy to try the parameter, If you want to use UI tool to got the result, I can give you a imagepy macros.

1 Like

Nice tool to experiment with the parameters. Now I want to write a Python script:

from skimage import io, segmentation
original = io.imread("microstructure.png"); io.imshow(original);
segmented = segmentation.quickshift(original); io.imshow(segmented);

I didn’t give any additional input parameters to the quickshift function as I used the default values in ImagePy. However, imshow does not show the segmented image in the same way as ImagePy (seen as your screenshot). It rather shows this:
wrong

How did you show two images side-by-side in ImagePy?

Quickshift return a label image

  1. you can use skimage.color.label2rgb to render the label with rgb image by mean value.
  2. you can use skimage.graph.rag_mean_color to build the superpixel graph
  3. you can use graph.cut_threshold to merge block

A Tips, ImagePy’s all plugins has the same hierarchy as the folder, you can look all the usage here:
https://github.com/Image-Py/imagepy/blob/master/imagepy/menus/Process/Segment/graph_plgs.py

https://github.com/Image-Py/imagepy/tree/master/imagepy/menus/Process/Segment

Thank you, very useful advices. I accepted your answer.

Yes, please. I performed the segmentation but not with the same result (see below). I used 5.0 as a threshold; increasing the threshold produces too large grains and “open” regions. I would like to reproduce yours.

Sorry, I forgot to tell you, my result need a little manual opertation (just cost less than one minute)

  1. build the segment line as a graph
  2. edit it in imagepy (very friendly, just cut or add edge by mouse)

if you need a half-automatic tool, I can show you the detail.

  1. How can I build the graph? Currently, I use rag_mean_color.
  2. Can you give me the name of the tool that allows me manual cutting/adding edges?

Could you please show me some details? Thank you.

How to build and edit a skeleton network

  1. Draw in ImagePy
  2. Process > Binary > Skeleton
  3. Build Network
  4. Cut Branch


    just strock cross the edge you want to remove, when mouse up, it will be ok!
  5. Add Branch


    when mouse up, the graph is update, and auto remove the node.

How to use background

  1. Open background and skeleton image
  2. Set Background And Mix Mode
  3. Edit The Network



  4. Threshold Again And Save

Tips: Do A Median Filter Befor QuickShift

I tried the graph creation and deletion with the hand drawing. That works.

How did you fetch the boundaries in ImagePy? For me, QuickShift resulted in the segmented image, but without the boundaries. Hence, I couldn’t move forward. As I couldn’t obtain the boundaries in ImagePy, I tried to do it programatically as

bnd = segmentation.boundaries.find_boundaries(labels);

where labels is an integer matrix, output of the image segmentation (i.e. the label image) and bnd is a Boolean matrix, describing the boundaries.
I wanted to save bnd as a binary image, so that I can load it in ImagePy and perform the graph operations. However, I cannot execute

io.imsave(filename, img_as_bool(bnd, force_copy=True));

to save it as binary. By default, imsave saves it as uint8, that the graph creation does not support.

Questions:

  1. How can I obtain the boundaries in ImagePy?
  2. How can I set “Set Background And Mix Mode” as you wrote in item 2 of section “How to use background”?

you can use:

  1. Process > Filters > Sobel: {both}
  2. if it is a rgb, use Image > Type > 8-bit
  3. Image > Adjust > Threshold: {use 1}
  4. Process > Binary > Skeleton

not because it is uint8, find_boundaries’s result is not a skeleton image, So please use
Process > Binary > Skeleton, Then build graph.

It is also very simple to add an boundaries’s plugins in ImagePy (less than 10 lines), If you are intresting, you can open an new topic.

Set background and mode, you need to switch the widgets to Channels RGB Panel, Look here

It worked. Now, I want to have access to the grains (so that later I can perform some statistical studies on them). I thought to do another segmentation to obtain them from the skeleton. I expected this to be easy to do on a binary image. Since I need to have access to the labelled image as a matrix, I used the programmatic approach:

  1. Save the skeleton in ImagePy
  2. Load the image using scikit-image
  3. Perform a watershed segmentation

The watershed segmentation mostly gave correct result, as you can see in the screenshot comparison below, but there are some discrepancies. Do you know a better way to have access to the individual grains after the semi-manual approach you described so far has been completed?

First of all, I think “segment this ferrous microstructure” is a difficult work.

  1. There is not a obvious edge, so we can not use “edge ridge like method”.
  2. We can not define “How Many Class”, so the “label and deep learn, features classify such as ilastik” not works.

So I choose a “region growing and merge based method”. The result is not very exactly, but I think we choose the right direction.

skimage’s graph cut method is based on region’s mean value (whether to merge), may be if we add some texture information as an multi vector, we could get a better result. But it would be a large project, and need a high programing skill.

@jni the is the author of skimage’s RAG Merge in this forum? He may have some experience and advice in this problem.

If semi-manual is acceptable, ImagePy is a good choice. ImagePy is not as powerful as Fiji, but now it has most classical method (from scipy.ndimage and skimage). What more, it is numpy based, any numpy based function could be intergrated by plugin easily (about 10 lines). So we can write some pipeline (such as your watershed), then write these as plugin.

If you want to write some plugin (such as add find_boundary), I can help.
If you need the framework to add some new feature, let me know. (such as to add an checkbox to show or hide the front image, it is helpful when we edit the graph)

If semi-manual is acceptable

Well, I come from the numerical simulations side where things can be automated. However, I realized that I need some manual editing (opening and closing edges, as you showed before), so I will stick to ImagePy (plugins can be developed for Fiji too but I don’t know Java).

Adding new GUI features would be nice but writing plugins such as find_boundary, etc. would be even more useful so that I can perform all my semi-automatic analysis from within ImagePy (I already have the programmatic workflow in a Python script).

Should we continue the plugin development here or open a new thread? Perhaps continuing here is more logical as all the information we exchanged so far is already contained.

Yes, Fiji is very powerful, So ImagePy devote to make the development more convenient.

We should start a new thread. starting with the find_boundary?

OK. Can you please give me a link where I should start learning the plugin creation within ImagePy?