ArrayScan and Cellomics image conversion

What is the best way to convert ArrayScan and Cellomics CX5/7 proprietary formats to PNG or other open-source formats? Can this be automated?

Hi @KrisG , have you tried Bio-Formats, it supports the following formats: https://docs.openmicroscopy.org/bio-formats/6.1.1/supported-formats.html

You can use it through FIJI and automate with a macro (some examples) or there are command line tools to do the conversion.

Hi dgault,

I work with KrisG in NYC. Kris has asked that I convey some
details about problems that we have encountered in processing
Cellomics .C01 images with Bio-Formats. I have mostly used the
ImageJ plugin, but I believe that standalone operation exhibits
similar behavior.

We are interested in converting to .PNG format, keeping maximum
bit depth and dimensional resolution. I believe that some of the
problems that we have seen relate to the max bit depth of the C01
source images. For a start, there is a discrepancy in the listed
resolution for Cellomics CX5 sensor array vs their data on image
output depth. The sensor is spec’d at 14 bits, while the image
format specs 12 bits. So perhaps they are rounding or truncating
the lower 2 bits during their processing.

In any case, the image data is probably stored with either 12 or 14
active bits. But I suspect that their header reports that it is a
16-bit image. Our attempts to convert to 16-bit PNG files result in
extremely dark (almost black) images. This would normally occur only
if several of the most significant bits were zeroed. I have tried
several different viewers and editors on the .PNG 16-bit output, all
with the same result, so information is probably getting lost in the
PNG conversion.

These images evidently do contain some information; if they are
normalized, they yield a heavily quantized (banded) output image.
It looks to me like depth of only 3 or 4 bits (8 or 16 gray levels).

I suspect that the .C01 header lists the depth as 16 bits. But given
that the entire file (including header) is compressed with Zlib, there
are no visible clues in bit-patterns.

Without seeing the pixel data, it is difficult to make suggestions, but
it seems to me that the actual bit depth of the CX5’s .C01 images
needs to be determined in advance and assumed to be valid for all
images. Normalization by pre-scanning pixel values would correct
single valid images, but would fail in the case of images that were
actually dark. It would not always be desirable to bring them up to
the full scale (0 to 65535), as that could lend a false impression of
how they compare to correctly scanned full-scale images.

One of our people has been converting .C01 images to PNG using the
stand-alone Bio-Formats program, but their process involves initial
conversion to TIFF, then subsequent conversion to PNG. In the process,
bit depth is lost. Resulting images are 8-bit.

Please let me know if I can be of further help. I hope our tests will
help to resolve the bit-resolution and 16-bit PNG conversion issues.

Mark G.

Hi Mark, from some of the sample files I have the header does seem to be reporting 16 bits as you suspect. When exporting to png the images do appear dark but reading them with autoscaling turned on should display correctly and the pixel values appear to be identical to the original. The banding is not something I would expect to see though. Can you upload a sample image to https://www.openmicroscopy.org/qa2/qa/upload/ for testing?

Hi David, Most of the conversions that resulted in quantized images were done earlier, but I did try to normalize the dark PNG files, and that resulted in similar banded effects. It seemed that low-order bits had been truncated during processing.

I had suggested that scale factors be decided by finding out how many active bits are actually present in CX5 and ArrayScan images. It is probably either 14 (as per the sensor spec on the Cellomics webpage for the CX5), or 12 (as in the spec for their image output). Given how dark the images are, I suspect 12 bits are active, with the top 4 bits zero’d. That would account for the considerable loss of luminance. And if the lower byte was lost during the normalization process, the remaining 4 most-significant bits would account for quantization to 16 gray levels.

The problem with autoscaling individual images is that all images would be normalized to full contrast, irrespective of original luminance. For example, an image that was actually very dark would be expanded to full brightness (0 to 65535 for 16 bits). So it would not maintain a valid point of comparison with other images.

If the actual range of the microscope sensors can be determined, it would be easy to lock in a constant multiplicative factor (actually a 16-bit left shift) that would yield images with good luminance, while maintaining a valid point of comparison with other images. Just a suggestion, but it may be the easiest approach overall.

Given that the .C01 images are compressed internally, it is not possible to simply view pixel bit-patterns to determine the number of active bits. I’ve been curious about that. Perhaps your team has determined this already…?

Mark

Hi Mark,

The Bio-Formats library itself doesnt perform any processing or modification of the original pixel values or metadata, so the values and bit depth that are being read from Cellomics should be the same as those being written to the PNG. For the pixel data we decompress it with the Zlib codec at the point when frames are requested, in which case they are decompressed and then read as a stream. We do have an extra field PixelsSignificantBits but having tested it today unfortunately I don’t think it will help with your workflow. In general if the range of the sensor doesn’t correspond well with the range of the Cellomics format then that feedback could be useful for the manufacturer also.

Hi David,

It would have made sense for the original .C01 format to have been left-shifted so the top significant bit corresponded to the MSB of the stored word. I suspected that the empty MSB’s were the cause of the dark images, but I’d need to write code to verify.

This would also explain why our people have been using a two-step process, saving the original to TIFF, then converting to PNG. Unfortunately, the conversion ends up in 8-bit format. I was hoping to simplify, and preserve the original bit depth.

I did notice the extra field. I figured that the header flagged the format as ‘16 bits’ to avoid possible ambiguity with a packed, bit-shifted format (though that would not have made a lot of sense), and then saved the actual bit depth in the second field. Is that field blank in the .C01 images?

Given that you have seen quite a few of the .C01 files, can you tell if the format is actually 14 bits or 12? Given the low luminance of the converted images, it’s probably 12. That could be solved simply with four 16-bit left-shifts, if the PixelsSignificantBits field is accurate. I understand the reluctance to do that if that field is not set accurately.

Mark

PS: It appears that, as a new forum member, I’m limited to only 3 replies to this thread. So I need to add this to my previous post:

I had tried selecting multiple input images via Fiji + Bio-Formats plugin, and attempted to use the batch process function. The resulting output: Multiple copies of the first selected image. I am not sure if that is due to the plugin, or to Fiji, or to a flag or option that needs to be set.

==========================================================================

David, Not sure if this will be noticed, as it will appear above your last post. I had written a followup to your last post, but I’m still limited to three replies to a given thread (?). The forum feedback advises editing previous posts as a workaround, but of course that’s awkward. I suppose we could start a new thread, but that is still not ideal. Not sure when the three-reply limit will be lifted, but perhaps one of the moderators can weigh in.

Mark

From the files we’ve seen so far all of them have used the same 16-bit size but I have no idea if they are actually 12 or 14 bit, which is why the significant bits field is not being used at present. As far as I know the actual bit depth doesn’t appear to be stored in the metadata anywhere, but I don’t have a specification for the format to be sure on that.

For the batch processing, what were the exact steps of the workflow you used?