Fill in ROI with interpolation of values at neighboring points

Is there a ready made way/plugin that can take a ROI and replace the values inside the ROI with interpolated values from the nearest points outside the ROI? I.e. I´m not looking to fill the ROI with a constant value.

Use case would be to remove vessels from fundus images and replace them with the local background.

Good day David,

just a question:

What would make an interpolated region more suited than the original vessel?

Please study:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4114110/

Regards

Herbie

Hey Herbie, thanks for the tip. The point of this is to compare values of on-vessel points to off-vessel points to estimate light absorption in the blood column. Interpolation of closest off-vessel points would be a well-defined, automatic way of estimating the unknown (variable along vessel) background behind the vessel.

OK David,

that makes sense and appears to be an interesting investigation!

Of course you have to respect the limits of such interpolation or, to cite the title of Gœthe’s autobiography: “Dichtung und Wahrheit”

You need to specify a null-hypothesis:
What would you suggest as interpolation model?
In 1D: A polynome, a Gauss-function, etc.
In 2D things become really difficult …

Regards

Herbie

I am not looking to reconstruct the vessel profile, just the (relatively flat) background. So I´d pick bilinear or bicubic – polynomial or Gaussian is not necessary! Cheers, David

Not sure things will be that easy …

Do you have an image with the “closest off-vessel points” in isolation, i.e. outside contours of the vessels with those values?

Regards

Herbie

I was thinking to take the vessel segmentation, dilate it, then remove the original vessel segmentation, then AND the result with the original image. That way I´d have strips on either side of vessel with the correct values…

Isn´t this a pretty generic operation though; imagine an image of a ball on the beach, draw a circle around the ball and make it “disappear”, i.e. fill the circle with interpolated “sand” values.

Thanks for your help Herbie!

David,

could you please provide such a contour image that is computed from a realistic sample image.

With the contour image, we could then try to tackle the interpolation problem.

Regards

Herbie

Since this functionality doesn’t seem to be readily available, I’m going to try to implement it generically for any ROI in any image using ImgLib2 by collecting the “just outside the ROI” points (using ( dilate(ROI) NOT ROI) ) AND image ) into a KDTree, and then use KNearestNeighborSearchOnKDTree and InverseDistanceWeightingInterpolatorFactory classes into Views.interpolate. Hope this is feasible!

David,

I’m sure you’re aware of the fact that, depending on the interpolation scheme, a more than one-pixel wide outside contour is required.

Furthermore, it appears important that what you are heading to works with realistic images.

Good luck

Herbie

Yes, I was thinking perhaps dilating the ROI 2 or 3 times… I’ll try to post some results soon. Thanks again Herbie!

@david.bragason I have an old ImageJ1 implementation of this here https://github.com/axtimwalde/mpicbg/blob/master/mpicbg/src/main/java/mpicbg/ij/integral/RemoveOutliers.java#L83-L182 in the outlier removal plugin. You can test how this looks in Fiji > Plugins > Integral images filters > Remove outliers (or so). It is cheap iterative diffusion from the borders into NaN labeled pixels. Looks great for small stuff and starts to look strange for larger regions.

2 Likes

Thanks a lot @axtimwalde - the attached images show the results of an initial test of RemoveOutliers. The vessels are mostly gone, showing an estimate of the background as if the vessels had been removed.
(The vessel segmentation is also being worked on - the smallest vessels have not been detected and are thus not affected by RemoveOutliers.)
P.S. This website wouldn’t accept uploads of the original tiff files.

I came here to ask exactly this for exactly this reason.

I’m trying RemoveOutliers, and it doesn’t appear to do anything. Visually, I see no change when I run the plugin. When I subtract the output of the plugin from the original image, and enhance contrast, there are a few pixels that changed, but these are all over the image, not inside the ROI. Am I misunderstanding how this is used?

david.bragason, did you find a solution that worked well for you?

Hey there, sorry to say that I haven’t experimented any further with retinal vessel images :frowning:

But concerning the usage of RemoveOutliers, notice that the pixels that you want replaced have to have the value “NaN”. Try the following snippet, which sets a rectangular ROI to the value NaN and then runs the plugin. Notice how the values on the border of each of the four sides of the ROI are diffused towards the center.

newImage("Untitled", "32-bit ramp", 512, 512, 1);
run("Duplicate...", " ");
makeRectangle(160, 185, 103, 65);
run("Set...", "value=NaN");
run("Remove Outliers", "block_radius_x=40 block_radius_y=40 standard_deviations=3");

To run the macro, do File > New > Script… paste the lines in there, change the language to IJ1 Macro and run.

Good luck, David