We have used In place import with soft linking (https://docs.openmicroscopy.org/omero/5.5.0/sysadmins/in-place-import.html#ln-s-soft-linking) to import images into our Omero instance. The original files are read only on their original file system (cephfs) and the soft links (to XFS) were successfully created upon import. We chose soft linking because hard linking would not work across the different file systems.
This morning, we noticed that all but 3 of our soft links were gone. The database appeared to still contain all of the image and related data and the image entries were ‘displayed’ on the web interface (without thumbnails).
As a near term fix, we were able to create a script to re-create the soft links using data from the database. This was successful for a small number of files however re-creating a large batch of soft links has locked up the server while it re-creates thumbnails.
What could be the reasons for this deletion of soft links? Could it be some sort of ‘flush’ functionality in Omero in response to a network or filesystem outage? We are not aware of any infrastructure feature that would do this.