Bio-Formats MOV Data Gaps

bio-formats
imagej
histogram
movie

#1

Using Bio-Formats I am able to import an MOV video, select an ROI, and then extract a histogram from each channel of a frame. However, it appears as though data is missing for certain intensities that is clearly unnatural. Would this be due to compression of the video? Am I importing the video incorrectly or approaching extracting frame histogram data incorrectly?

I am having a hard time validating this data as I’m unsure of how to extract a frame from a video outside of ImageJ that won’t experience some form of compression/interpolation/data manipulation. Any recommendations?

Edit: I have read up a bit on the Codec, and it is in fact typically lossy. Could that be the explanation for the gaps in the histogram data?

Some information about the MOV file:
Codec: H264 - MPEG-4 AVC (part 10) (avc1)
Color primaries: ITU-R BT.709
Color transfer function: sRGB
Color space: ITU-R BT.601 Range

A sample of the red channel of the first frame is as follows:

|0|3018|
|---|---|
|1|0|
|2|7462|
|3|0|
|4|36132|
|5|0|
|6|134412|
|7|0|
|8|282970|
|9|341702|
|10|0|
|11|260602|
|12|156634|
|13|0|
|14|93602|
|15|34572|


#2

Good day!

Am I importing the video incorrectly or approaching extracting frame histogram data incorrectly?

How can we know?

outside of ImageJ

I don’t understand.

The short answer is:
Never use images or stacks for scientific purposes that have undergone lossy compression!
Compression artifacts depend on the kind of compression. They can of course change the image histograms and color. For instance GIF uses a color table approach which causes an extreme distortion of histograms.

This holds for images and image sequences.

We have no idea where your data comes form and why it has been compressed.

Try to get the data as uncompressed image stacks or as uncompressed movies in whatever kind of container. ImageJ can store uncompressed or lossless compressed (PNG) stacks in an AVI container.

Regards

Herbie


#3

Am I importing the video incorrectly or approaching extracting frame histogram data incorrectly?

How can we know?

Good question! What sort of information could I provide regarding how bioformats imports a MOV file to better ask my question? The import is a one line function call, below.

run("Bio-Formats", "open=[" + input + "\\" + fileName +"] autoscale color_mode=Default  rois_import=[ROI manager] view=Hyperstack stack_order=XYCZT use_virtual_stack");

outside of ImageJ

I don’t understand.

Ah yes, sorry, upon re-reading that I see how I wasn’t clear. What I mean is that I would like to validate that the histogram I am capturing from a frame is an unaltered frame from the MOV. Presumably, in order to validate that the frame is unaltered due to the script and/or bio-formats importing, I would use some other program to extract a frame and create a histogram from that image as a comparison. I don’t know of any other tools that could extract an unaltered frame. My first attempt at using VLC’s Scene video filter to extract frames produced an image that does not match the frames I am extracting from ImageJ.

The short answer is:
Never use images or stacks for scientific purposes that have undergone lossy compression!
Compression artifacts depend on the kind of compression. They can of course change the image histograms and color. For instance GIF uses a color table approach which causes an extreme distortion of histograms.

I recognize that the H264 codec is lossy and so there will be artifacts, so what I would really like to do is present these histograms as evidence for the flaws in the video capturing system we have. I am having trouble piecing together a particularly strong argument because I don’t really know what the artifacts look like from lossy compression, and don’t have the experience to definitively say/show that the frames from ImageJ are unaltered MOV data.

If you have any recommendations, suggestions for reading material, thoughts, comments, whatever! I’d love to hear it.

My very WIP code is below.
transition_histogram_extractor.zip (2.1 KB)


#4

There is not much I can contribute.

I only partly understand what you like to do and I don’t like at all ploughing through code someone else has written …

As I’ve written before, I don’t understand why you need to work with data that was compressed in a lossy way. Such data is unsuited for scientific purposes and you don’t need to justify this fact.
In short: Simply refuse to consider lossy compressed images or image sequences.

If howver your job is to generally find out about artifacts that are typically due to certain compression techniques, you may find more reports dealing with this topic than you will be willing to study. However, I doubt that most of these reports are freely available and many of them may be difficult to obtain.

What I mean is that I would like to validate that the histogram I am capturing from a frame is an unaltered frame from the MOV.

Coders and de-coders can be precisely defined or they may leave a certain degree of freedom to those who implement them. That said and especially concerning de-coders, you may get different results when using different software.

don’t have the experience to definitively say/show that the frames from ImageJ are unaltered MOV data.

Please defined exactly what yoiu mean by “unaltered MOV data”.
I fear you can’t.

If you want to be sure that your data is correctly stored and reproduced, then you need to use a transparent format and this needs to be loss-less.

Generally you won’t get very far with your endeavor, at least as far as I understand it.

Regards

Herbie


#5

Hi,

there are several reasons why you should not expect a perfect histogram:

  • As mentioned by Herbie, H264 is a lossy compression. My first guess would be that it should to smear out gaps in the histogram coming from the earlier steps of acquisition and processing, but this no necessarily true.
  • Many frame grabbers and video formats (maybe on the way before conversion to H264 use only luminance (brightness) values between 16 and 235, not the full 0-255 range. If this gets stretched to the 0-255 range, it will result in gaps in the histogram.
  • sRGB has a nonlinear transfer curve; conversion from the linear scale of the analog-to-digital converter to sRGB and conversion between color spaces can cause gaps in the histogram.

In any case, unless you have areas of well-defined brightness in your image, I don’t think that the histogram tells you much about whether the chain from acquisition to import in ImageJ is ok or not.


#6

Thank you Schmid! Your indication of the luminance not being full range pointed me in the right direction, and was in fact (as far as I can tell) partially responsible for the sRGB histogram gaps.