Crazy problem with macro Fit calls failing ... fiji 2.0.0-rc-69 1.52i mac osx

imagej
macro

#1

Hi all,

I’ve got a problem with Fit in a macro that I can’t figure out - I’m wondering if it’s me, or an IJ bug. I’ve been trying to figure this out for 2 hours and I’m at loss…

I’m try to find the centre of small circle in an image by getting two profiles in x and y directions across it, then by fitting a gaussian to the two profile arrays (ie in x then y directions) to get the mid-points in x and y. The data sets are almost identical.

…should be pretty easy.

However, when I run Fit.doFit in my macro to do the fitting it works fine for the first array, but then is completely wrong for the 2nd. Basically:

a) the first fit works fine
b) the second fit is completely wrong - even when passing in almost correct guess.
c) the 2nd incorrect fit reports an R^2 value that is completely incorrect
d) adding some logging info into my code to print out values - and also do a Fit.logResults - results in the output coming out of order, and the 2nd call to logResults (for the 2nd Fit.doFit) results not showing up at all in the log

The two files below are the output from Fit.plot in my script. X is the first (good) one, Y is the second (crazy nuts) one.

I don’t think it’s my code: …I’ve tried fitting the output manually in a normal plot window, and get the same error. …I’ve tried copying the Y data values (only) from the plot window to a new plot window, and still the same result. …I’ve tried making a completely different macro and copying the data values into new arrays and fitting - still fails.

So I’m at a loss. doFit and manual fit seems to work fine on one set of data and completely fail for a 2nd - almost equivalent - data set even when I’m supplying guess to doFit that is almost the exact correct parameter values.

Richard.

I’ve included the two data sets from the plots below. The first doFit Gaussian fits without a problem to (even with no guess), the 2nd completely fails - even with almost perfect guess parameters passed!

First:

X Y
297 95.645
298 95.710
299 94.871
300 94.226
301 95.548
302 85.484
303 41.710
304 -31.129
305 -95.839
306 -128.290
307 -131.032
308 -104.355
309 -43.419
310 31.129
311 80.968
312 95.258
313 95.129
314 94.355
315 93.258
316 92.484
317 93.032
318 93.548
319 93.323
320 92.000
321 91.032
322 92.548
323 95.387
324 95.226
325 92.935
326 91.935
327 91.258

Second (bad - gaussian just won’t fit it):

X Y
292 93.226
293 92.484
294 93.968
295 95.903
296 95.516
297 94.710
298 95.129
299 95.097
300 93.742
301 93.484
302 89.323
303 58.290
304 -9.677
305 -83.258
306 -124.935
307 -132.677
308 -112.355
309 -60.323
310 11.548
311 68.032
312 90.774
313 94.516
314 94.065
315 92.677
316 92.097
317 92.742
318 93.903
319 95.032
320 93.968
321 92.710
322 94.226

My code - for what it’s worth - is below:

           c = newArray(boxSP);

            for (i=0;i<boxSP;i++) c[i] = i+x0;
            Plot.create("X prof "+angle,"Y", "val", c, profX); Plot.show();
            Array.getStatistics(profX, min, max, mean, stddev);     // give fit some initial guesses
            guess = newArray( max, min, (x0+x0+boxSP)/2, 2 );
            Fit.doFit("Gaussian", c, profX, guess);
            midX = Fit.p(2);

            print("fitting X");
            print("guesses: ", guess[0],guess[1],guess[2],guess[3]);
            Fit.logResults;
            Fit.plot;

            d = newArray(boxSP);

            for (i=0;i<boxSP;i++) d[i] = i+y0;
            Plot.create("Y prof "+angle,"X", "val", d, profY); Plot.show();
            Array.getStatistics(profY, min, max, mean, stddev);     // give fit some initial guesses
            guess = newArray( max, min, (y0+y0+boxSP)/2, 2 );
            Fit.doFit("Gaussian", d, profY, guess);
            midY = Fit.p(2);

            print("fitting Y");
            print("guesses: ", guess[0],guess[1],guess[2],guess[3]);
            Fit.logResults;
            Fit.plot;

#2

…follow up

the “bad” data works (ie Gaussian fits perfectly) if I invert the Y values (y = -y).

so it looks like a flaky case in doFit’s least-squares (I presuming its doing a LS fit) - but that’s kind of not so good IMHO because it fails even when I supply almost correct values as guess parameters to doFit (ie the LS would be less with the guess parameters initially passed than with the final values it spits out!)


#3

Good day,

I just tried with Kaleidagraph and it shows the same behaviour!

The problem is with the high offset (parameter c) and the low variance (parameter d^2). You need rather exact estimates of c in order to get the desired fit.

These are the estimates

a=100;
b=-200;
c=310;
d=2;

that work for both number sets in Kaleidagraph which AFAIK uses the same algorithm as ImageJ.

In any case I doubt that there is a bug in the ImageJ-fit implementation.

Just curious, did you try another way of getting the desired center?
I very much should like to see the image.

Regards

Herbie


#4

Thanks for checking it out Herbie!

I agree it’s not exactly an easy fit. But to counter that a bit - when I tried the same data, but inverted Y, it works without any problem on all the images I’ve tested. Simply inverting Y doesn’t change the non-linearity around c and d.

I also tried supplying almost perfect parameters as a guess and it would still fail in IJ. Not just fail, but the least squares sum for the guesses is way less than that for the final values!

…anyway, for me it seems to be working fine with Y inverted (so a “normal” way up gaussian) at least.

Richard


#5

Just curious, did you try another way of getting the desired center?
I very much should like to see the image.

Would you please post?

Herbie


#6

I simply did a y=-y on all the values and it works fine!

Here is a zoomed in (and ignore the overlays) picture of what I’m trying to get a centre point of.

the red pixel is the calculated centre - the measured values is actually a lot more “centred”, but that’s the closest pixel. …I might oversample the image 2x later for improving the overlay accuracy

So far - simply by inverting y - the fit algorithm seems completely stable and accurate, whereas the wheels were falling off with an “upside down” gaussian.


#7

Thanks for the screen shot.

Why not threshold to the background, get a mask, apply it as selection to the original and measure the center of mass?

Works nicely.

Herbie


#8

I hadn’t thought of using centre of mass! …I’ll have to give it a go.

…actually probably could get a mask, then use the non masked pixels values as a weighting for the CM, rather than just the pure mask black/white CM - that way it would probably be less sensitive to mask funny-ness. …I’m not sure whether the “standard” CM takes into account pixel value or just in/out of mask/region

the algorithm I’m using actually calculates x and y profiles averaged over the green box, not simply a single “line” profile somewhere through the circle. I’d already written an “average a profile over a region” function previously, so it was easy to reuse that.

I did that because a) it’s a lot less noisy and b) I don’t know the exact location of the hole (which is why I’m trying to calculate it), so trying to use a single line profile isn’t good

So your idea definitely seems like a possibility that might be easier.

Prior to that I’d tried Hough circle fitting (which isn’t good at all for something so small), and thought about doing a translation only registration of a template hole using turboreg - but decided that was just going to make my brain hurt too much.


#9

Of course, Center of Mass does take into acount the values by definition!

Herbie


#10

Hey Herbie,

for the get threshold, then a mask, then use it as a selection and measure idea …what’s the actually process for doing that?

excuse my ignorance! …while I’ve used thresholds a bit, I’ve never used masks and I’m not sure exactly how to go about doing that.

As a concrete example I’ve got an image below, and I want to measure the x and y dimensions of the region marked


I can threshold it easily enough (below) …but I’m then not sure on what steps you take to make a mask of that, make a selection of it, and then measure the selection (and also how to get rid of the smaller external alignment marks still present in the thresholded region)


#11

As a concrete example I’ve got an image below, and I want to measure the x and y dimensions of the region marked

I see no marked region, only lines.
Please be more specific if you need help.

Furthermore, always post original raw images in TIF- or PPNG-format, not screen shots.

Regards

Herbie