Problems opening Olympus oir files using Bio-Formats

bio-formats
#1

I am having problems opening Olympus oir files in both bio-formats-fiji and bio-formats-command-line (next I will try bio-formats-python). These problems are occurring on Window-7, macOS Mojave 10.14.3, and macOS Sierra 10.12.6

I am happy to use any combination of bio-formats from the command-line, Fiji, or Python.

My Olympus software reports

System Info: FV31S-SW, Version: 2.3.1.163 ( Powered by H-PF Version 1.23.2.205 )

I want to drag and drop an Olympus oir file onto Fiji and have it open correctly. In general, when I do this, I do get a stack in Fiji but some of the frames are out of order and others are sometimes black. The out of order frames are super confusing and troublesome to debug/notice.

Here is my example oir file (sorry for the large download):

https://www.dropbox.com/s/zyknjduopzbkvgd/20190401__0005.oir?dl=0

The example oir file is a time series of 3000 frames acquired with a resonant scanner at ~30 frames-per-second, each image is 512x173 pixels.

Width: 512
Height: 173
Frames: 3000
Frames Per Second: 30

On dragging and dropping the test oir file onto Fiji and accepting the defaults in the Bio-Formats import dialog, I get a console window with the following text in red:

Reading pixel blocks from /Users/cudmore/Dropbox/olympus/20190401__0005.oir
[WARN] No pixel blocks for plane #999
[WARN] No pixel blocks for plane #1000
[WARN] No pixel blocks for plane #1001
[WARN] No pixel blocks for plane #1002
[WARN] No pixel blocks for plane #1003

...
...
...

[WARN] No pixel blocks for plane #2997
[WARN] No pixel blocks for plane #2998
[WARN] No pixel blocks for plane #2999

It is curious to me that the errors start at image/plane #999 ?

Trying to use the command line bftools ‘bfconvert’ function, ‘bfconvert -version’ reports:

./bfconvert -version
Version: 5.8.2
Build date: 20 April 2018
VCS revision: 07ecd9a5d239985f8ae57462c12124c2fa5ccd9b

Running bftools ‘bfconvert’ as follows:

./bfconvert 20190401__0005.oir 20190401__0005.tif

results in:

OIRReader initializing 20190401__0005.oir
Reading pixel blocks from /Users/cudmore/Dropbox/olympus/bftools/20190401__0005.oir
[Olympus OIR] -> 20190401__0005.tif [Tagged Image File Format]
	Converted 28/3000 planes (0%)
	Converted 69/3000 planes (2%)
	Converted 112/3000 planes (3%)
	Converted 153/3000 planes (5%)
	Converted 195/3000 planes (6%)
	Converted 235/3000 planes (7%)
	Converted 274/3000 planes (9%)
	Converted 313/3000 planes (10%)
	Converted 354/3000 planes (11%)
	Converted 395/3000 planes (13%)
	Converted 435/3000 planes (14%)
	Converted 478/3000 planes (15%)
	Converted 517/3000 planes (17%)
	Converted 559/3000 planes (18%)
	Converted 600/3000 planes (20%)
	Converted 642/3000 planes (21%)
	Converted 676/3000 planes (22%)
	Converted 716/3000 planes (23%)
	Converted 757/3000 planes (25%)
	Converted 799/3000 planes (26%)
	Converted 841/3000 planes (28%)
	Converted 882/3000 planes (29%)
	Converted 923/3000 planes (30%)
	Converted 966/3000 planes (32%)
No pixel blocks for plane #999
No pixel blocks for plane #1000
No pixel blocks for plane #1001

...
...
...

No pixel blocks for plane #2997
No pixel blocks for plane #2998
No pixel blocks for plane #2999
	Converted 3000/3000 planes (100%)
[done]
81.966s elapsed (0.915+23.670666ms per plane, 7302ms overhead)

When I drag and drop a oir file into Fiji on windows-7 I am also occasionally getting .log files that start with ‘replay_pid’ and ‘hs_err_pid’, for example:

replay_pid54340.log
replay_pid54352.log
replay_pid54416.log
hs_err_pid54340.log
hs_err_pid54352.log
hs_err_pid54416.log

The first part of ‘hs_err_pid54340.log’ reads:

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 2269936 bytes for Chunk::new
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (allocation.cpp:390), pid=54340, tid=54544
#
# JRE version: Java(TM) SE Runtime Environment (8.0_66-b18) (build 1.8.0_66-b18)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.66-b18 mixed mode windows-amd64 )
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#

---------------  T H R E A D  ---------------

Current thread (0x0000000052b92800):  JavaThread "C2 CompilerThread2" daemon [_thread_in_native, id=54544, stack(0x0000000054490000,0x0000000054690000)]

Stack: [0x0000000054490000,0x0000000054690000]
[error occurred during error reporting (printing stack bounds), id 0xc0000005]

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)


Current CompileTask:
C2:   6246 1377       4       java.net.URLClassLoader$1::run (5 bytes)


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x00000000567f0000 JavaThread "SciJava-64030b91-Thread-0" [_thread_blocked, id=54680, stack(0x0000000058f20000,0x0000000059120000)]
  0x0000000052b97000 JavaThread "Service Thread" daemon [_thread_blocked, id=54560, stack(0x0000000054890000,0x0000000054a90000)]
  0x0000000052b93800 JavaThread "C1 CompilerThread3" daemon [_thread_blocked, id=54548, stack(0x0000000054690000,0x0000000054890000)]
=>0x0000000052b92800 JavaThread "C2 CompilerThread2" daemon [_thread_in_native, id=54544, stack(0x0000000054490000,0x0000000054690000)]
  0x0000000052b8c800 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=54540, stack(0x0000000054290000,0x0000000054490000)]
  0x0000000052b34800 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=54536, stack(0x0000000054090000,0x0000000054290000)]
  0x0000000052b33000 JavaThread "Attach Listener" daemon [_thread_blocked, id=54532, stack(0x0000000053e90000,0x0000000054090000)]
  0x0000000052b2a000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=54528, stack(0x0000000053c90000,0x0000000053e90000)]
  0x0000000052b38000 JavaThread "Surrogate Locker Thread (Concurrent GC)" daemon [_thread_blocked, id=54524, stack(0x0000000053a90000,0x0000000053c90000)]
  0x0000000052ad0000 JavaThread "Finalizer" daemon [_thread_blocked, id=54428, stack(0x00000000537d0000,0x00000000539d0000)]
  0x0000000052ac9800 JavaThread "Reference Handler" daemon [_thread_blocked, id=54424, stack(0x00000000535d0000,0x00000000537d0000)]
  0x0000000000557800 JavaThread "main" [_thread_blocked, id=54344, stack(0x0000000000030000,0x0000000000230000)]

And the first part of ‘replay_pid54340.log’ reads:

JvmtiExport can_access_local_variables 0
JvmtiExport can_hotswap_or_post_breakpoint 0
JvmtiExport can_post_on_exceptions 0
# 515 ciObject found
ciMethod java/lang/Object <init> ()V 4097 1 228332 0 0
ciMethod java/lang/Object hashCode ()I 4097 1 512 0 -1
ciMethod java/lang/Object equals (Ljava/lang/Object;)Z 2049 1 2215 0 -1
ciMethod java/lang/Object clone ()Ljava/lang/Object; 3073 1 384 0 -1
instanceKlass sun/util/locale/provider/LocaleProviderAdapter
instanceKlass java/util/spi/LocaleServiceProvider
instanceKlass java/util/Calendar
instanceKlass java/util/TimeZone$1
instanceKlass sun/util/calendar/ZoneInfoFile$ZoneOffsetTransitionRule

...
...
...

ciMethod java/lang/String equalsIgnoreCase (Ljava/lang/String;)Z 2929 1 13849 0 0
ciMethod java/lang/String regionMatches (ZILjava/lang/String;II)Z 2113 7793 14591 0 0
ciMethod java/lang/String startsWith (Ljava/lang/String;)Z 2081 1 24666 0 -1
ciMethod java/lang/String hashCode ()I 2593 32769 1231 0 336
ciMethod java/lang/String indexOf (I)I 2121 1 83980 0 -1
ciMethod java/lang/String lastIndexOf (I)I 4097 1 21184 0 304
ciMethod java/lang/String lastIndexOf (II)I 1873 55233 2593 0 304
ciMethod java/lang/String lastIndexOfSupplementary (II)I 0 0 1 0 -1
ciMethod java/lang/String substring (II)Ljava/lang/String; 2121 1 5475 0 944
ciMethod java/lang/String concat (Ljava/lang/String;)Ljava/lang/String; 4097 1 5653 0 848
ciMethod java/lang/String replace (CC)Ljava/lang/String; 1529 100617 1292 0 1072
ciMethod java/lang/String toUpperCase (Ljava/util/Locale;)Ljava/lang/String; 529 14513 1091 0 2832
ciInstanceKlass java/lang/String 1 1 548 10 8 9 9 10 100 10 10 10 10 100 10 10 10 10 10 100 8 10 10 8 10 10 10 10 10 10 10 10 10 10 10 100 10 10 10 10 10 10 10 10 10 7 10 10 10 100 100 10 10 11 11 10 10 9 11 10 10 10 10 7 3 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 11 10 10 10 10 10 7 10 10 8 10 10 3 3 7 10 10 10 10 10 11 7 10 10 100 10 10 10 11 11 11 7 3 10 10 10 10 8 8 8 10 10 10 10 10 10 10 10 10 10 10 100 10 10 10 10 8 10 10 8 8 10 10 10 10 7 9 7 10 7 100 100 100 1 1 1 1 1 1 1 1 1 5 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 100 100 1 100 1 100 1 1 1 1 1 1 1 1 100 1 100 1 1 1 1 100 100 100 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 100 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 100 1 1 1 100 100 1 100 1 1 1 100 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 12 1 12 12 7 12 1 12 12 12 12 1 100 12 12 12 12 12 1 1 7 12 1 12 12 12 12 12 12 12 100 12 12 1 12 12 7 12 100 12 12 12 12 1 12 1 1 12 12 12 12 7 12 12 7 12 12 12 12 12 1 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 7 12 12 1 12 12 1 12 1 12 12 12 12 7 12 1 12 12 1 12 12 100 12 100 12 12 1 12 12 12 7 12 1 1 1 100 12 12 12 12 12 12 12 12 12 12 12 1 12 12 1 12 1 1 100 12 100 12 7 12 12 1 12 1 12 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
staticfield java/lang/String serialPersistentFields [Ljava/io/ObjectStreamField; 0 [Ljava/io/ObjectStreamField;
staticfield java/lang/String CASE_INSENSITIVE_ORDER Ljava/util/Comparator; java/lang/String$CaseInsensitiveComparator
ciInstanceKlass java/lang/Class 1 1 1224 9 9 10 10 10 10 9 9 9 9 7 10 10 8 10 8 8 10 10 10 10 10 10 10 10 10 8 10 8 8 10 11 10 10 10 10 10 9 10 100 10 9 7 100 8 10 10 7 10 10 7 100 10 10 10 10 9 10 7 10 100 10 10 10 9 10 10 10 10 10 7 7 10 10 10 10 10 9 10 7 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 100 8 10 10 100 10 100 11 10 10 10 10 10 10 10 8 10 10 10 8 10 10 10 8 10 8 10 10 10 10 8 10 100 10 10 10 10 100 10 100 10 10 10 10 10 10 10 10 100 10 10 10 10 10 10 10 10 10 10 10 10 10 9 10 9 7 10 9 10 7 10 9 10 10 10 10 10 10 10 8 10 10 9 10 7 9 10 10 7 10 10 10 10 9 10 9 10 10 10 10 9 9 10 9 100 10 100 10 10 11 11 11 7 11 11 9 9 7 7 10 9 9 10 10 9 7 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 8 7 10 8 8 8 8 10 10 9 9 10 7 9 7 10 7 7 10 10 10 8 10 7 10 7 10 100 8 10 7 10 10 11 10 100 10 10 8 8 10 10 9 11 100 11 9 10 10 10 9 9 10 10 10 10 10 11 11 11 11 100 11 10 10 100 11 10 10 10 11 11 7 10 10 9 9 10 10 10 10 7 9 100 100 100 100 1 1 1 1 7 1 1 1 1 1 3 1 3 1 3 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 5 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 100 1 1 1 1 1 1 1 1 1 1 1 100 100 100 1 100 1 1 1 100 1 1 1 1 100 1 1 1 1 1 1 100 100 100 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 100 1 1 1 100 1 1 1 1 1 1 1 1 1 1 100 1 1 1 1 100 1 1 1 1 1 1 1 1 1 1 100 100 100 100 100 1 1 1 1 1 1 1 1 100 1 1 1 1 1 1 100 1 1 1 1 1 1 1 100 1 1 1 1 1 1 1 1 1 1 1 100 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 100 1 1 1 1 1 1 1 1 1 1 100 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 100 1 1 100 100 1 1 1 1 1 1 1 1 1 1 100 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 100 1 1 1 1 1 1 1 1 1 1 1 1 1 100 1 1 1 1 1 1 1 1 100 1 100 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 100 1 1 100 100 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 12 12 12 12 12 12 12 12 12 12 1 12 1 12 1 1 12 12 12 12 7 12 12 12 12 1 12 1 1 12 12 7 12 7 12 12 7 12 100 12 7 12 100 12 1 12 12 1 1 1 12 12 1 12 7 12 1 1 12 12 12 12 12 1 100 12 12 12 12 12 12 12 12 7 1 1 12 12 7 12 12 12 12 7 12 1 12 12 12 12 12 12 100 12 12 12 12 12 12 7 12 12 12 1 1 12 1 12 1 12 100 12 12 12 100 12 12 1 12 12 12 1 12 12 12 1 12 1 12 12 12 12 1 12 1 12 12 12 1 12 1 12 12 12 12 12 12 12 12 1 12 12 12 12 12 12 12 12 12 12 12 12 12 12 1 12 12 1 12 12 100 12 12 12 100 12 12 12 12 1 12 12 12 12 1 12 12 12 1 12 12 7 12 7 12 12 12 12 12 12 12 12 12 12 12 12 1 1 12 7 12 12 100 12 1 12 100 12 12 1 1 12 12 12 12 12 12 1 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 1 1 7 1 1 1 1 12 12 12 12 12 1 12 1 1 1 1 12 100 1 12 1 12 1 12 1 1 1 12 7 12 12 1 12 1 1 7 12 12 12 12 1 12 12 100 12 7 12 12 12 12 12 12 12 12 12 12 7 12 12 1 1 12 100 12 12 1 100 12 12 12 12 1 12 12 12 100 12 12 100 12 12 12 1 12 1 1 1 1 1 100 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Finally, here is info on my Fiji version and Bio-Formats version:

Downloaded the most recent Fiji, ran it and updated it twice using menu ‘Help - Update’. Now selecting menu ‘Help - Update’, Fiji reports ‘ImageJ is up to date’.

Used the ‘ImageJ Updater - Manage update sites’ button and ticked ‘Bio-Formats http://sites.imagej.net/Bio-Formats/’ and the updater did not show any additional files to update.

The menu ‘Help - About Plugins - Bio-Formats Plugins’ is reporting:

Bio-Formats Plugins for ImageJ
Release: 6.0.1
Build Date: 13 March 2019
VCS revision f47f5 ... 0e1b1

The imagej updater ‘plugins/bio-formats_plugins.jar’ is reporting

plugins/bio-formats_plugins-6.0.1.jar

Release date:
18 Mar 2019

Can someone help???

0 Likes

#2

Hi @cudmore, thank you for providing a sample file and clear details of the issue. I was able to test and reproduce the problem as described using the latest Bio-Formats. This is also very similar to an issue we have seen previously in https://github.com/openmicroscopy/bioformats/issues/3242. It does look to be a bug in the OIR reader. I will try and further debug the problem to see if there is a fix we can put in place.

0 Likes

#3

Thanks for verifying you could replicate with the test file. Let me know what are the next steps in getting a fix?

0 Likes

#4

I have been able to do some more in depth debugging today and have located the section of code causing the bug. The next steps will be to put in put a code fix in place in the OIRReader and open a PR against Bio-Formats. This will then allow for testing against our repository of sample files to ensure the proposed fix doesn’t break any existing workflows. At this point it will be scheduled for a suitable release and be reviewed and tested.

Once I have the PR open I will link it in this thread.

0 Likes

#5

The PR with the proposed fix is now open: https://github.com/openmicroscopy/bioformats/pull/3349

0 Likes

Problems opening Olympus oir 'line scan' files