Connection to OMERO server lost while importing SlideBook data

We have been troubleshooting some problems with OMERO reading SlideBook files using our SlideBook6Reader.jar. We resolved an issue that prevented data from being properly imported due to the sequence of close(bool) calls, which now enables (smaller) data to import. However, when importing larger data (e.g. >500 MB) we are experiencing server timeout / connection lost messages, resulting in data import failure.

The link below includes the SlideBook6Reader.jar file, along with error logs and the java source for SlideBook6Reader.class:

I’m not sure how to best go about troubleshooting this issue. We can confirm that files that fail can be opened just fine using Fiji under both Windows and Linux, so assume it’s some interaction with the sequence of calls OMERO is making. We have a test server running so can add and review whatever log might shed some light on this.

Thanks,

Brian

3 Likes

Hi @bodenste. Welcome to image.sc!

And thanks for the logs. Looking at omeroinsight.log I see:

/tmp/slidebook $grep -E "(^2019|fully)" omeroinsight.log  | tail -n 20
2019-06-11 09:59:26,430 INFO  [      o.f.i.transfers.UploadFileTransfer] ( Thread-14) Transferring C:\Users\bode\Desktop\St Jude 488 - 2.sld...
2019-06-11 10:06:15,631 ERROR [              loci.formats.FormatHandler] ( Thread-14) close (all)
2019-06-11 10:06:15,631 ERROR [              loci.formats.FormatHandler] ( Thread-14) close (all)
2019-06-11 10:06:15,635 DEBUG [ o.o.shoola.env.data.DataServicesFactory] ( Thread-14) Connection ErrorIce.UnknownLocalException
2019-06-11 10:06:23,691 DEBUG [                   omero.gateway.Gateway] (entQueue-0) joining the session
2019-06-11 10:06:29,587 ERROR [                   omero.gateway.Gateway] (entQueue-0) Failed to join the session Glacier2.CannotCreateSessionException
    reason = "Server not fully initialized"
    reason = "Server not fully initialized"
2019-06-11 10:06:29,591 ERROR [                   omero.gateway.Gateway] (entQueue-0) Failed to close the session Glacier2.CannotCreateSessionException
    reason = "Server not fully initialized"
    reason = "Server not fully initialized"
2019-06-11 10:06:29,891 ERROR [                   omero.gateway.Gateway] (entQueue-0) Failed to create connector omero.gateway.exception.DSOutOfServiceException: Failed to create connector
    reason = "Server not fully initialized"
    reason = "Server not fully initialized"
2019-06-11 10:06:30,920 WARN  [                   omero.gateway.Gateway] ( Thread-14) Cannot re-create. Returning null connector
2019-06-11 10:06:30,921 ERROR [              loci.formats.FormatHandler] ( Thread-14) close (all)
2019-06-11 10:06:30,921 ERROR [              loci.formats.FormatHandler] ( Thread-14) close (all)
2019-06-11 10:06:30,922 WARN  [                   omero.gateway.Gateway] ( Thread-14) Cannot re-create. Returning null connector
2019-06-11 10:07:02,925 INFO  [o.o.shoola.env.rnd.PixelsServicesFactory] (nitializer) Heap memory usage: max 954728448
2019-06-11 10:07:02,978 INFO  [         o.o.shoola.env.init.TaskBarInit] (nitializer) Loaded L&F: com.sun.java.swing.plaf.windows.WindowsLookAndFeel

which for me says:

  • you started uploading
  • insight lost its connection
  • the server supported “Server not fully initialized”
  • you either restarted or managed to connect, etc.

The “not fully initialized” is worrisome because that only happens when the server is starting up. Since you were already logged in, my guess is that either (a) your server ran out of memory and crashed (usually when it runs out of member it hangs rather than crash) (b) something segfaulted and the Java process exited completely.

The log files from OMERO_DIST/var/log might have more information. In case (b), there’s likely also a core dump somewhere that may tell us more.

Hope that helps.
~Josh

1 Like