Omero move data to cold storage

Hi all,
A platform planning to use omero is managing data in a two stage way: “hot” data is stored on a file server and “cold” data on tapes with a higher access latency. We plan to import the images in place.

In that scenario, how can I update the path (I assume at least at the dataset level) of files moved from the hot to the cold storage facility?

Does omero fs rename cover this?

Thanks!

Guillaume

Hi @glyg,

Just to be overly careful, here we should talk about Filesets. Every import creates a single Fileset which can have multiple Images. The entries of the Filesets specify where the files are to be found.

omero fs rename does something similar but of a more limited case than what you need. It’s intended to be used when a server-level configuration has changed and filesets need changing en masse.

Would changing the symlinks under /OMERO/ManagedRepository be an option?

If not, then likely you’d need to adapt the logic of fs rename but there may be other barriers. It would be something of a research project.

~J.

Thanks for the detailed answer Josh. I’m trying to get in touch with the relevant sysadmin to get more info on the underlying filesystem. As I intend to do the import in place, I guess I could change the symlink? I definitely need more info on the underlying architecture!

Best,
Guillaume

1 Like

Hi @glyg,

You’ll let us know when you find out more?

As for the symlinks, OMERO doesn’t make any assumptions beyond that the path in the database remains within /OMERO/ManagedRepository (and guarantees that only the right users access any given file within).

If the server process has permissions to follow a symlink out of that directory then that’s fine.

~Josh.

Hi Josh,
yes, I should get in touch with the relevant people soon, so I’ll keep you posted, thanks!

1 Like