Model noisy neural network output of membrane as a single line

python
#1

I’m using EMAN 2.22 (a convolutional neural network) to extract the inner membrane of a cell from tomograms. The output from EMAN 2.22 is quite noisy however. I’m using Otsu’s threshold to get rid of the salt and pepper noise and then converting the remainder to binary 0 or 1 (because parts which are not in the membrane are sometimes brighter than those which are). The noise is quite large clusters and the membrane is not one large connected cluster either. I need to convert the membrane output to a single line model to do further analysis but to do this I need to get rid of the noise.

I’ve tried morphological transformations using scikit-learn. Orig is after applying Otsu and converting to binary.

The best I have (to try and increase the joining of the real membrane but reducing connection to clusters) is by dilating, closing and then skeletonizing. (This is the third image in this post that has regression mapped on top).

I was doing this to try to increase the size of membrane clusters and reduce noise clusters to try to do a connected component analysis (and remove smaller clusters). My membrane and noise seem to be in similarly sized clusters, however.

(I also tried to do an active contour on the whole image, given the noise is on the inner of the membrane, but I can’t seem to get it to work well.)

active%20contour

I’ve split the image into tiles and then my thought was to do a Hough transform. I’ve given up on because my lines aren’t necessarily straight and it becomes harder and harder as I reduce my tile size to know where the true membrane is. I moved onto linear regression (on the dilated, closed, skeletonized tiles) to get a line through my membrane which works quite well but is still skewed by large noise.

linear%20regression%20(dilation%20closing%20skeletonize)

a = blockshaped(orig, 256, 256)

fig, axs = plt.subplots(4,4, figsize=(10, 10), sharex="col", sharey="row")
fig.subplots_adjust(hspace = .5, wspace=.001)

axs = axs.ravel()

for i in range(len(a)):
    a[i] = dilation(a[i])
    a[i] = closing(a[i])
    a[i] = skeletonize(a[i])
    axs[i].imshow(a[i])
    i += 1

x, y = np.where(a[2]>0)
plt.scatter(x, y)
plt.show()

from numpy import polyfit
from numpy import polyval
p1 = polyfit(x, y, 1)
p2 = polyfit(x, y, 2)
p3 = polyfit(x, y, 3)
print p1
%matplotlib
plt.scatter(x, y)
xp = np.linspace(0, 256, 100)
plt.plot(xp, polyval(p1, xp), "r-")
plt.plot(xp, polyval(p2, xp), "b--")
plt.plot(xp, polyval(p3, xp), "m:")

I’ve considered gradient descent algorithms like Adam to find a better function but this seems a bit too extreme?

Another thought was to make a knn graph and collapse branches leaving the main branch but I have no idea how to do this.

I’m sure that I’m missing a really simple thing and any advice which could point me in the right direction of another technique to consider would be fantastic.

Thank you so much!

0 Likes

#2

Please post a non-processed image to get an idea of what the data looks like. No JPEGs please.

0 Likes

#3

Thank you. This is the csv file with the original pixel values.

https://drive.google.com/file/d/16XmHvbE_iaSrkGIApZdaF1o0NMuOKF6f/view?usp=sharing

How I read it in:

my_test2 = np.genfromtxt('temp_bin2x_inner_tagged.csv', delimiter=',')
val2 = filters.threshold_otsu(my_test2)
binary2 = np.where(my_test2 > val2, 1, 0)
0 Likes

#4

The problem is that the contours are non existent in several places.
You might be able to “join” some parts at the expense of resolution, for example try with a “Difference of Gaussians” (DoG e,g, sizes 17 and 25), or the LoG (Laplacian of Gaussian) filters.
Good luck!

1 Like

#5

Thanks for your help! Okay I’ll keep trying.

0 Likes

#6

You can clean this up nicely with greyvalue morphology.
I’ll illustrate this with MorpholibJ (a Fiji plugin for which you must enable an additional update site).

  1. Start with the greyvalue image
  2. Use directional filtering to close the gaps. (This is basically a number of line openings with different directions if you want to implement it in python)
  3. Use greyvalue area openings to retain only large structures (area openings are in the latest version of skimage if you want to use python)
  4. Threshold

Macro recorder

selectWindow("contour.tif");
run("Directional Filtering", "type=Max operation=Opening line=110 direction=64");
selectWindow("contour.tif");
selectWindow("contour-directional");
run("Duplicate...", " ");
run("Gray Scale Attribute Filtering", "operation=Opening attribute=Area minimum=10000 connectivity=4");
run("Auto Threshold");
setOption("ScaleConversions", true);
run("8-bit");
run("Auto Threshold", "method=Otsu white");

2 Likes

#7

Oh thank you, that’s really helpful! I’ll definitely look into those types of filters in python. Thanks!

0 Likes