Post-processing of plates in Omero

Hi all and @s.besson ,

to not mix up different topics too much I would like to open a new one for the post-processing of plates in Omero referring to the conversation at

Yesterday I imported a new plate into Omero, which is still showing up to be in post-processing after 24 hours after import (clock thumbnails in Omero.web). So I was trying the omero fs importtime command to find out if there is any issue, but the output does not seem to help me that much:

$  omero fs importtime 78159
Using session for ansla100@localhost:4064. Idle timeout: 10 min. Current group: AG_Beller
   upload time of 2744.11s for 6338 files (0.433s/file)
    setId time of  10.91s
 metadata time of  30.26s

Looking at the server logs I do see an error in the Blitz-0.log:

2021-06-07 14:43:53,853 INFO  [] (erver-2012) Adding log:UPDATE,class ome.model.screen.WellSample,4291
2021-06-07 14:43:54,100 INFO  [                 org.perf4j.TimingLogger] (erver-2012) start[1623069806267] time[27833] tag[omero.saveImportGraph]
2021-06-07 14:43:56,033 INFO  [                 org.perf4j.TimingLogger] (erver-2012) start[1623069806255] time[29777] tag[$3.doWork]
2021-06-07 14:43:56,033 INFO  [] (erver-2012)  Rslt:    {Image=null, Pixels=(ome.model.core.Pixels:Id_95489, ome.model.core.Pixels:Id_95504, ome.model.core.Pixels:Id_95577, ... 93 more), Plate=(ome
2021-06-07 14:43:56,033 ERROR [] (erver-2012) Method interface$Work.doWork invocation took 29777
2021-06-07 14:43:56,513 INFO  [                 org.perf4j.TimingLogger] (1-thread-3) start[1623069805795] time[30717] tag[omero.import.request.metadata]
2021-06-07 14:43:56,514 INFO  [] (erver-2010)  Meth:    interface ome.api.IQuery.get
2021-06-07 14:43:56,514 INFO  [] (erver-2010)  Args:    [class ome.model.core.OriginalFile, 447411]
2021-06-07 14:43:56,516 INFO  [] (erver-2010)  cctx:    group=654
2021-06-07 14:43:56,518 INFO  [] (erver-2010)  Auth:    user=1724,group=654,event=null(User),sess=44e78552-567a-493a-8678-a13bb8151a94
2021-06-07 14:43:56,520 INFO  [                 org.perf4j.TimingLogger] (erver-2010) start[1623069836514] time[5] tag[]
2021-06-07 14:43:56,520 INFO  [] (erver-2010)  Rslt:    ome.model.core.OriginalFile:Id_447411
2021-06-07 14:43:56,521 INFO  [] (erver-2007)  Meth:    interface ome.api.IUpdate.saveObject
2021-06-07 14:43:56,521 INFO  [] (erver-2007)  Args:    [ome.model.core.OriginalFile:Id_447411]
2021-06-07 14:43:56,522 INFO  [] (erver-2007)  cctx:    group=654
2021-06-07 14:43:56,531 INFO  [] (erver-2007)  Auth:    user=1724,group=654,event=6252121(User),sess=44e78552-567a-493a-8678-a13bb8151a94
2021-06-07 14:43:56,534 INFO  [] (erver-2007) Adding log:UPDATE,class ome.model.core.OriginalFile,447411
2021-06-07 14:43:56,538 INFO  [                 org.perf4j.TimingLogger] (erver-2007) start[1623069836521] time[16] tag[]
2021-06-07 14:43:56,538 INFO  [] (erver-2007)  Rslt:    null
2021-06-07 14:43:56,538 INFO  [                 org.perf4j.TimingLogger] (1-thread-3) start[1623069805795] time[30743] tag[omero.import.request.step]
2021-06-07 14:43:56,538 INFO  [                 org.perf4j.TimingLogger] (1-thread-3) start[1623069805795] time[30743] tag[omero.request.step.0]
2021-06-07 14:43:56,538 INFO  [ o.s.b.r.ManagedImportRequestI.@43c5e131] (1-thread-3) notify step 1 of 5
2021-06-07 14:43:56,540 INFO  [] (1-thread-3) Reading IFDs
2021-06-07 14:43:56,549 INFO  [] (1-thread-3) Populating metadata

and a bit later

2021-06-07 14:45:46,328 INFO  [] (erver-2007) Adding log:UPDATE,class ome.model.core.OriginalFile,447411
2021-06-07 14:45:46,332 INFO  [                 org.perf4j.TimingLogger] (erver-2007) start[1623069946315] time[16] tag[]
2021-06-07 14:45:46,332 INFO  [] (erver-2007)  Rslt:    null
2021-06-07 14:45:46,332 INFO  [                 org.perf4j.TimingLogger] (1-thread-3) start[1623069836539] time[109793] tag[omero.import.request.step]
2021-06-07 14:45:46,343 INFO  [                 org.perf4j.TimingLogger] (1-thread-3) start[1623069836539] time[109804] tag[omero.request.step.1]
2021-06-07 14:45:46,343 INFO  [                 org.perf4j.TimingLogger] (1-thread-3) start[1623069795645] time[150698] tag[omero.request]
2021-06-07 14:45:46,345 INFO  [                 org.perf4j.TimingLogger] (1-thread-3) start[1623069795636] time[150708] tag[]
2021-06-07 14:45:46,345 INFO  [] (1-thread-3)  Excp:    omero.cmd.HandleI$Cancel: import-file-exception
2021-06-07 14:45:46,352 ERROR [] (1-thread-3) Method interface$Work.doWork invocation took 150708
2021-06-07 14:45:46,353 DEBUG [ o.s.b.r.ManagedImportRequestI.@43c5e131] (1-thread-3) Request cancelled by loci.formats.FormatException: Buffer too small (got 1045440, expected 2090880).

So the “buffer too small” message is quite obvious, but which buffer is meant here?

I assume the post-processing will not finish for this plate. Is there a possibility to restart the post-processing for the already uploaded plate or do I have to delete and re-upload it? Last time where the post-processing didn’t finish, re-uploading the plate solved the issue, but of course this is a bit annoying and I would rather solve the underlying issue.

In addition I wonder how long a typical post-processing of plates takes usually. Our hardware might not be the best, but it’s also not the worst and I don’t feel like my plates are that exotic in terms of field or channel amounts. Is there a rule of thumbs in terms of image number, size, etc.?

Any hints are much appreciated.

Thanks, Anna

Btw I tried to re-upload the plate today and got the same error message. The plate is only a control, so I could upload the data for testing if this could help to investigate the issue. Just let me know.

BR, Anna

Thanks @ahamacher,

Given the output of omero fs importtime it looks like the error is happening after the objects have been created in the database (in the metadata step) and during the pixels step when the planes are read to compute min/max values and create thumbnails.

The loci.formats.FormatException: Buffer too small (got 1045440, expected 2090880) exception indicates a mismatch between the reader metadata, which determines the expected size of the plane buffer, and the actual plane size from the binary files. We will definitely need a sample dataset to investigate further.