Our Soft links have vanished!

Usage & Issues
#1

We have used In place import with soft linking (https://docs.openmicroscopy.org/omero/5.5.0/sysadmins/in-place-import.html#ln-s-soft-linking) to import images into our Omero instance. The original files are read only on their original file system (cephfs) and the soft links (to XFS) were successfully created upon import. We chose soft linking because hard linking would not work across the different file systems.

This morning, we noticed that all but 3 of our soft links were gone. The database appeared to still contain all of the image and related data and the image entries were ‘displayed’ on the web interface (without thumbnails).

As a near term fix, we were able to create a script to re-create the soft links using data from the database. This was successful for a small number of files however re-creating a large batch of soft links has locked up the server while it re-creates thumbnails.

What could be the reasons for this deletion of soft links? Could it be some sort of ‘flush’ functionality in Omero in response to a network or filesystem outage? We are not aware of any infrastructure feature that would do this.

#2

Hi @Michael_Dausmann. Welcome to image.sc & sorry to hear you’re having troubles.

No. OMERO treats the soft links just as it would the hard links and would only delete them at the request of an authorized user, but if that were the case, the image place holders would also be deleted.

Are you in-place importing all your data? If not, can you confirm that the non-in-place imported images are all fine? If so, have you checked any of the non-image files under the omero.data.dir? Have they also been deleted?

Do you know the time period during which the files were deleted? Do you have /var/log (dmesg) and OMERO logs for that period that you could send us?

~Josh

#3

Thanks for your Response Josh. We were honestly thinking it was an Omero ‘flush’ so this helps us to refocus our investigation. Yes, we in-place imported all our data so we can’t disambiguate this way. We did look through the logs but there was nothing at all that we could see that gave us a clue.

The only other piece of info we have gathered is that the soft links were created at our request, on an existing server, in a /temp folder because we are only trialing Omero at this point and wanted to be able to clear down the installation. It is possible that there is an old script running that clears data out of /temp to save space. Maybe leave it with us and we will keep poking around.

Thanks again for your help.

1 Like
#4

OMERO records an imported image’s files in its database. To see the IDs and filepaths for image 1234’s files:

bin/omero hql "SELECT originalFile.id, originalFile.path || originalFile.name FROM FilesetEntry WHERE fileset IN (SELECT fileset FROM Image WHERE id = 1234)"

Then to see the history of one of its files, e.g., that with ID 5678:

bin/omero hql "SELECT action FROM EventLog WHERE entityType = 'ome.model.core.OriginalFile' AND entityId = 5678 ORDER BY id"

Ordinarily that might list INSERT then some UPDATE or whatever. Only for files that OMERO intended to delete will the final entry be a DELETE.

(Use the --all option if asking about images not in your current group.)