Averaging a 2 dimensional plot into a 1 dimensional plot while removing outliers

[Farley]

Hi everyone,

I’m working with analysing sets of data which involves measuring collimated x-rays from an indirect dosimeter. The data is eventually readout as a two dimensional map of the piece of glass, which acts as a dosimeter, and then the beam is rotated until it is vertical and then the map is averaged into a one dimensional profile.

I’ve managed to play around with the directionality plugin for orienting the beam, but my next problem is involving how to remove any outliers from the averaging that may result from imperfections in the glass, such as scratches and bubbles. Is there some way to correct for this problem using a plugin? I’ve considered writing a macro that would compare the centre point of the FWHM of each profile against a line that would fit down the centre of the beam, but I’m not exactly sure how to go about doing this.

Any help would be tremendous. I’ve attached an example of the profile that I’m typically working with, although the widths and shape of the beams can vary by quite a bit.

Thanks!

[Bio7]

Imperfections in the glass could be corrected, e.g., by applying a filter (gaussian blur, median, FFT filter etc.) with a minimum neighbourhod to preserve the real data. This will remove outliers in the image data but also changes the original image data a bit.

It depends on your required data quality if a filter is appropriate.

[anon96376101]

Dear Farley,

after having contrast enhanced your sample image I think I better understand, what you are dealing with …

Obviously, you have rotated the glass substrate by about 22.6 deg with respect to the beam to make the latter about vertical.
The glass substrate results in the noisy rectangular (a corner seems to be broken away) background that shows heavy JPG-artifacts. In any case, you should use uncompressed images for analysis and it would be helpful if you could post a sample here in TIF- oder PNG-format of your raw image data.

Here is the vertical projection (profile) of your sample image:

Of course, the projection (profile) depends on the angular adjustment of the beam.
I guess what you call outliers results from the projection of the glass substrate and I fear it will be difficult to remove without severe impact on the beam profile.

Could you please confirm that my interpretation of the sample data is correct and provide an uncompressed sample image.

Regards

Herbie

[anon96376101]

Dear Farley,

I’ve checked the angular adjustment of your sample image and it appears perfect (better than 0.4deg deviation from the vertical):

(The orientation analysis was performed within the ROI shown in the image below. A highpass having an equivalent cutoff frequency at 4% of the Nyquist frequency was applied.)

Although the glass background isn’t high, I used a ROI that is situated within the glass substrate to avoid the influence of its borders.

Regards

Herbie

[anon96376101]

Dear Farley,

I shall try to subtract an estimate of the projected background from the profile.

However, I will be away from my computer for several hours now, so please stay tuned.

Regards

Herbie

[Farley]

Hi Herbie,

Thank you so much for this, I 've attached two uncropped, unrotated images that have different profiles due to the incident energy. Please ignore the smearing on the left side of the image, as that’s due to some alignment issues from my microscope, I just ignore any data from that portion of the image. The way the data is processed is by taking the ratio of two images and subtracting a measured background. The dosimeter is based off of photoluminescence from a doped glass material that emits two different spectra as a function of irradiation so each pixel value is tuned to the response of the material.

Since I’m only interested in the profile of the beam, I crop out everything else, and then rotate the image until it is vertical. The image is actually a txt image file originally, and the rotational is done by using a macro that measures the FWHM of the beam as if it were a Gaussian, and finding the minimum value as a function of rotational angle.

[anon96376101]

Good day Farley,

please do me a favour and try to understand that most facts that are evident for you may be not as clear for me.

1. Thanks for the raw image which are again JPG-compressed which is bad.
   Please use the PNG- or TIF-format.

2. I don’t understand what you mean by:
   "The way the data is processed is by taking the ratio of two images and subtracting a measured background."
   Has this still to be done for the two raw images?

3. Both raw images show two beams. Are both beams to be evaluated?

4. I don’t understand what you mean by:
   "[…] so each pixel value is tuned to the response of the material."
   How does this refer to the images?

For the time being, I should simply like to see the sample image that you’ve originally posted but in an uncompressed format. (Of course it makes little sense to save the JPG-image as PNG-image …)

I then would try to subtract an estimate of the projected background from the profile, if this is still of interest. Obviously, you’ve already a good method to do the rotation, so I won’t think about any alternatives.

Please report back

Herbie

[Farley]

Hi Herbie,

Sorry for the confusion. I’ve been working on this project late and I think what I was trying to say made more sense in my head than it did in writing. I’ve attached the two images again in PNG format.

The attached images have already been processed by a previous macro by the following method:

• Irradiated samples are measured by two different photomultiplier tubes simultaneously that are tuned to separate emission wavelengths. The ratio of one image to the other is taken.
• Samples that have not been irradiated are also simultaneously measured under the same conditions and the ratio is taken
• the unirradiated values are subtracted from the irradiated.

This isn’t too important to the task I am concentrating on, but I thought I would mention it to demonstrate that there have been a measured background subtraction performed on the images, which makes them a lot easier to deal with while measuring the profile.

There can be multiple beams in each image, but I am mostly concerned with evaluating the longer of the two beams, since they will be averaged along the y-axis once rotated and will give an overall better average profile.

The overall problem I am trying to solve is recognizing and removing any of the significantly different profiles along the y-axis that would influence the shape of the average beam profile. Is there a simple way of doing this?

Thanks again for all your help.

• Farley

[anon96376101]

Well Farley,

thanks for the details that help.

As mentioned before, I try to remove the background not in the image but in the profile. For this purpose I take a few sample values of the profile that one can assume to be outside the beam. These samples are fitted by a model function in order to get an estimate of the projected background, of course also within the beam profile. This estimated function can then be subtracted from the original profile. Presently, and because it is easy, I assume a Gaussian as the model function but the model can be a different function as well.

Another approach would be to measure the glass substrate before the X-ray exposure but only if its position is exactly the same as during the exposure.

And again, do you think it is possible to get the first image (the one with vertical beam) in PNG-format?

Regards

Herbie

[anon96376101]

Now Farley,

here are results obtained with the proposed approach:

The background profile and its Gauss-model

Of course, the beam is truncated in this plot.

The original profile

The corrected profile according to the model

(The model function subtracted from the original profile with negative values set to zero.)

Please note that the model also changes the values within the beam profile and I think this is what you expect. Of course, and as mentioned before, the model function need not be a Gaussian.

HTH

Herbie

[Farley]

Hi Herbie,

Thanks for all your help, I’ve attached a PNG version of the cropped and rotated beam profile. Unfortunately the problem is that there might be a non zero background around the beam, so mathematically subtracting might eliminate some of the effects we want to measure.

I’m wondering if it is possible to measure each individual profile along the y-axis and remove any lines that might influence the total profile shape, such as below

(presence of a scratch or defect)

(average profile measured)

[anon96376101]

Well Farley,

of course this is possible but you need a criterion and you didn’t specify it.

A simple approach (with implicite criterion) would be to filter the 1D-profiles or line-selections, either by linear filtering or by rank-operations such as the median-operator. Because ImageJ is for images you wil generally need to code a macro to perform such operations. It’s not difficult.

Thanks for the PNG-image, but I think I spent enough hours with the wrong approach …

Good luck

Herbie

[Farley]

Hi Herbie,

Thank you for all your help, I appreciate it a lot. Sorry if I wasn’t clear enough, it’s my first time posting to this forum.

Cheers,

Farley

[anon96376101]

Good day Farley,

a final word of caution:

If in the end you are still interested in the projection of the image onto the x-axis (average profile) and especially in the side-lobes of the beam you should carefully select the relevant part of the image before doing so.

Here is an example:

If the selection area, indicated in yellow, includes parts outside of the substrate, the side-lobes of the beam can be severely influenced, hence making quantitative analyses impossible.

To get the largest area possible, I recommend not to crop the image before the rotational adjustment as shown here for the third of your posted images:

Original image

Rotated (and contrast enhanced for display only)

Cropped (and contrast enhanced for display only) with recommended selection frame

Have success

Herbie