Bfconvert invalid XML character (Unicode: 0xffff) for nd2 file

Hi,

when I try bfconvert with a nd2 file I get this error:

[Fatal Error] :1:54: An invalid XML character (Unicode: 0xffff) was found in the element content of the document.
Exception in thread “main” java.lang.RuntimeException: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 54; An invalid XML character (Unicode: 0xffff) was found in the element content of the document.
at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:263)
at ome.xml.model.StructuredAnnotations.asXMLElement(StructuredAnnotations.java:681)
at ome.xml.model.OME.asXMLElement(OME.java:931)
at ome.xml.model.OME.asXMLElement(OME.java:771)
at ome.xml.meta.AbstractOMEXMLMetadata.dumpXML(AbstractOMEXMLMetadata.java:110)
at ome.xml.meta.OMEXMLMetadataImpl.dumpXML(OMEXMLMetadataImpl.java:105)
at loci.formats.ome.OMEPyramidStore.dumpXML(OMEPyramidStore.java:81)
at loci.formats.services.OMEXMLServiceImpl.getOMEXML(OMEXMLServiceImpl.java:468)
at loci.formats.tools.ImageConverter.testConvert(ImageConverter.java:544)
at loci.formats.tools.ImageConverter.main(ImageConverter.java:1096)
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 54; An invalid XML character (Unicode: 0xffff) was found in the element content of the document.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:259)
… 9 more

My Java Version is

java version “1.8.0_251”
Java™ SE Runtime Environment (build 1.8.0_251-b08)
Java HotSpot™ Client VM (build 25.251-b08, mixed mode, sharing)

and the bio-formats version is

Version: 6.5.0
Build date: 30 April 2020
VCS revision: ed992ebc2cb07d04fcb49485540af0babd1baa05

Showing the nd2 file with bfshow works. I can also open the file with the bioformats FIJI Plugin.

I did not put a name for the XY Positions in NIS Elements, when capturing the image. Internally NIS shows the names #1, #2, #3, … for the different XY Positions. Maybe NIS uses 0xffff internally for non-named XY Positions. I think the naming is the cause of the problem since it is the only obvious difference from other nd2 files, that can be converted with bfconvert.

Since bfshow and the FIJI Plugin can display the nd2 file, I wonder why bfconvert throws the error and if it could be changed to not throw the error. I was also thinking about renaming the XY Positions in the nd2 file. But I do not know how to do this in NIS… So I am very happy about any hints, to convert this file.

Best,
Niklas

Hi Niklas, that is an odd one which I haven’t seen before. If you run showinf -omexml -nopix path/to/myFile.nd2 does that throw the same exception?

If it does would you be able to upload the file to https://www.openmicroscopy.org/qa2/qa/upload/ so I can test it further?

Hi,

thanks for your reply. The above command gives the same error:

showinf -omexml -nopix test.nd2
Checking file format [Nikon ND2]
Initializing reader
ND2Reader initializing …\test_nd2-exp29\KDV1679_chip1_channel1_3dyesEndpoint_16x.nd2
Searching for blocks
Parsing block ‘ND2 FILE SIG’ 0%
Parsing block ‘ImageMetadat’ 0%
Parsing block ‘ImageCalibra’ 0%
Parsing block ‘ImageMetadat’ 0%
Parsing block ‘ImageCalibra’ 0%
Parsing block ‘ImageMetadat’ 0%
Parsing block ‘ImageAttribu’ 0%
Parsing block ‘ImageDataSeq’ 0%
Parsing block ‘ImageDataSeq’ 0%
Parsing block ‘ImageDataSeq’ 6%
Parsing block ‘ImageDataSeq’ 13%
Parsing block ‘ImageDataSeq’ 20%
Parsing block ‘ImageDataSeq’ 26%
Parsing block ‘ImageDataSeq’ 33%
Parsing block ‘ImageDataSeq’ 40%
Parsing block ‘ImageDataSeq’ 46%
Parsing block ‘ImageDataSeq’ 53%
Parsing block ‘ImageDataSeq’ 60%
Parsing block ‘ImageDataSeq’ 66%
Parsing block ‘ImageDataSeq’ 73%
Parsing block ‘ImageDataSeq’ 79%
Parsing block ‘ImageDataSeq’ 86%
Parsing block ‘ImageDataSeq’ 93%
Parsing block ‘ImageMetadat’ 99%
Parsing block ‘CustomData|R’ 99%
Parsing block ‘ImageEventsL’ 99%
Parsing block ‘ImageTextInf’ 99%
Parsing block ‘CustomDataVa’ 99%
Parsing block ‘CustomDataVa’ 99%
Parsing block ‘CustomDataVa’ 99%
Parsing block ‘CustomDataVa’ 99%
Parsing block ‘CustomDataVa’ 99%
Parsing block ‘CustomDataVa’ 99%
Parsing block ‘CustomData|C’ 99%
Parsing block ‘CustomData|C’ 99%
Parsing block ‘CustomData|P’ 99%
Parsing block ‘CustomData|P’ 99%
Parsing block ‘CustomData|X’ 99%
Parsing block ‘CustomData|Y’ 99%
Parsing block ‘CustomData|Z’ 99%
Parsing block ‘CustomData|Z’ 99%
Parsing block ‘CustomData|Z’ 99%
Parsing block ‘CustomData|T’ 99%
Parsing block ‘CustomData|A’ 99%
Parsing block ‘CustomData|A’ 99%
Parsing block ‘CustomData|A’ 99%
Parsing block ‘CustomDataVa’ 99%
Parsing block ‘ImageAttribu’ 99%
Parsing block 'ND2 FILEMAP ’ 99%
[Fatal Error] :1:54: An invalid XML character (Unicode: 0xffff) was found in the element content of the document.
Exception in thread “main” java.lang.RuntimeException: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 54; An invalid XML character (Unicode: 0xffff) was found in the element content of the document.
at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:263)
at ome.xml.model.StructuredAnnotations.asXMLElement(StructuredAnnotations.java:681)
at ome.xml.model.OME.asXMLElement(OME.java:931)
at ome.xml.model.OME.asXMLElement(OME.java:771)
at ome.xml.meta.AbstractOMEXMLMetadata.dumpXML(AbstractOMEXMLMetadata.java:110)
at ome.xml.meta.OMEXMLMetadataImpl.dumpXML(OMEXMLMetadataImpl.java:105)
at loci.formats.ome.OMEPyramidStore.dumpXML(OMEPyramidStore.java:81)
at loci.formats.services.OMEXMLServiceImpl.getOMEXML(OMEXMLServiceImpl.java:468)
at loci.formats.FormatReader.setId(FormatReader.java:1422)
at loci.formats.DelegateReader.setId(DelegateReader.java:291)
at loci.formats.ImageReader.setId(ImageReader.java:843)
at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:650)
at loci.formats.tools.ImageInfo.testRead(ImageInfo.java:1035)
at loci.formats.tools.ImageInfo.main(ImageInfo.java:1121)
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 54; An invalid XML character (Unicode: 0xffff) was found in the element content of the document.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:259)
… 13 more

When I check “Display Metadata” and “Display OME-XML metadata” in the bioformats plugin for FIJI, everything also works fine. I will upload the file soon.

Thanks for uploading the file. I have been able to carry out some initial debugging on the file but have not yet got to the bottom of the problem. I have opened a Bio-Formats GitHub Issue to track the bug: https://github.com/ome/bioformats/issues/3590

Hi, thanks a lot! I am looking forward to a solution to the problem. If I can help in some other way, I am happy to do so.