Problem with .dv file bit depths and processing?

Hi,

When I open newer .dv files (from a DeltaVision Elite) the brightness and contrast is automatically set to values from -32768 to 32767, despite having no negative intensity values in my image (and a max value of at most 32767, typically less). FIJI shows it to be a 16 bit image, and nothing changes if I further try to select image-> type->16 bit.
A bigger problem is when I try to subtract background from an image (e.g. rolling ball of 50), the result is completely negative pixel values. Adding a value of 32768 to the entire image brings my image up to where I would expect intensities to be, but I am concerned that this bug may be affecting underlying data and could affect results after further processing, thus I do not want to blindly compensate for it.

This problem has persisted over 3 different window-based FIJI installations, but only appears to affect newer .dv files (I tested data from May 2018 and it opened and subtracted background without issue). It affects the raw data, as well as files that have been fused or stitched in Softworx, but not files that have been deconvolved in Softworx.
As the files are quite large, they aren’t attached directly, but I have included a screenshot showing the image and B&C when it is first opened, a single slice that was background subtracted, and measurements of the intensities before and after. Sample data (and dv log files) I have uploaded to a shareable folder on box.com. I suggest looking at 20200923_HeLa_CMG_Hoechst_60x_1520_DV_01_R3D.dv as a start.

Thanks for any help in sorting this out or working around it!
Deanna

I have opened your file named “20200923_HeLa_CMG_Hoechst_60x_1520_DV_01_R3D.dv” and its first 16 bytes, read as integers, show (1024, 1024, 120, 1):

$ od -N 16 -i 20200923_HeLa_CMG_Hoechst_60x_1520_DV_01_R3D.dv
0000000        1024        1024         120           1
0000020

From the file specification, this means your image is 1024 pixels wide, 1024 pixels “tall”, 120 slices (that these 120 are two channels of 60 Z planes is elsewhere on the file), and that the pixels are of mode 1. Pixel mode 1 is 16-bit signed integer, meaning a pixel whose value is an integer between -32768 and 32767.

Seems like the problem is on the file itself (probably on the acquisition software itself), and not on BioFormats or Fiji or ImageJ. You do mention that older files do not have this issue. Was softworx upgraded recently? What softworx version are you using?

To workaround this issue on the file, you can convert your images to unsigned 16-bit integer. There is no “button” on ImageJ to do this. There’s question Converting 16-bit signed tiffs to 16-bit unsigned which has a JavaScript code to do that but it will set the minimum pixel value to zero. To keep the pixel values untouched, you can use this modified version of that code:

// This script implements a patched Plugins>Filters>Signed 16-bit
// to Unsigned command, which converts signed 16-bit 
// images and stacks to unsigned.
imp = IJ.getImage();
stack = imp.getStack();
if (stack.isVirtual())
    IJ.error("Non-virtual stack required");
cal = imp.getCalibration();
if (!cal.isSigned16Bit())
    IJ.error("Signed 16-bit image required");
cal.disableDensityCalibration();
ip = imp.getProcessor();
for (i=1; i<=stack.getSize(); i++) {
    ip = stack.getProcessor(i);
    ip.add(-32768);
}
imp.setStack(stack);

Alternatively, if you are on a linux system, you can just patch the binary file like so:

perl -p -i.bak -e 'BEGIN {$/ = undef;} substr($_, 12, 4) = pack ("i", 6);' your-bad-dv-file.dv

The above command will open the file, modify the incorrect files, and save the file keeping a backup of the file with the extension .bak. If you’re happy with not making a backup, remove the -i.bak part of the command.

It looks like I’m using Softworx 7.0; it was probably updated around a year ago, which sounds about right for when I started seeing this issue. The system effectively only collects 15 bits of data (ie my camera only records values of 0-32k), so I guess the method of handling that final bit has changed. I’ll reach out to Cytiva and see if they have any ideas about fixing this on the instrument side. In the meantime, your script will be quite useful. If I understand correctly, since it’s just a matter of the single signed bit, I should be able to rely on standard processing procedures if I simply correct the 32k offset from that bit, e.g. by using your script.
Thanks for the help!

I’m sick and tired of reporting softworx issues to GE, that never get fixed. I ended up creating a github repo to collect the issues for myself, so maybe report the issue there as well. Maybe Cytiva will be better at fixing softworx issues.

It’s complicated. It depends on the implementation of the processing step you’re doing. The thing is, your pixel values are being read correctly, the problem is only on the data type. Since the minimum value of your data type is a negative value, operations that make use of that number (I guess subtract background does) might lead to an image with negative numbers. So yes, for this case, but for many processing steps you don’t need to do anything. Then it becomes embroiled on java not having an unsigned int type.