Fastest way to add images to OMERO

What is the fastest way to add images into OMERO? I’ve got thousands of images that are pre-computed and placed in a folder in OMERO.data folder then I run `omero import [creds] --transfer=ln [other options] path1 path2 path3
It takes about 2-4s per image and with 1000 images that translates to quite some time. I was hoping to cutting it down to a fraction of that.
Is there a faster way?
Thank you.

1 Like

Hi @paulkorir,

As often with performance optimization, it likely depends on where your bottlenecks are.

What file type are you working with and how large is each file (on average)?

Do you see the average time go down when you try multiple imports in a single command?

In terms of what’s possible from the client, the options available to you are:

  --parallel-upload COUNT               Number of file upload threads to run at the same time
  --parallel-fileset COUNT              Number of fileset candidates to import at the same time

  --checksum-algorithm [TYPE]          Alternative hashing mechanisms balancing speed & accuracy
  # or
  --skip {all,checksum,minmax,thumbnails,...}     Optional step to skip during import

To get a feeling for where the time is currently being spent, you might also take a look at omero fs importtime Fileset:ID for one of your imports.

All the best,
~Josh

1 Like

Hi Josh,

Thank you very much for your helpful response. I’ll give it a shot and let you know if it doesn’t work.

Kind regards,

2 Likes

Hi Josh,

It’s definitely faster using the --parallel-* arguments (roughly x3 faster).

Very helpful.

P

1 Like