Delete large data from OMERO

Dear @OMETeam,

Following confirmation from the PI, we would like to delete large dataset (Tbs) of our OMERO server and were wondering what are the best ways to do that.

As documented here, the action of deleting the data and everything linked to it should be pretty forward (right click, delete). However, if we were to do that on the full dataset, won’t the deletion of everything linked to it put a big load on the server ?

What about deleting a user ? Would that delete the data as well or keep it as ‘orphaned’ in the group ?

Would you have any recommendation ? Start with some smaller chunks ?

Thanks for your help.

Best,
Laurent

1 Like

Unfortunately OMERO does not yet provide the feature of deleting users or groups. Also, there is definitely room to improve the performance of delete. It can use a fair bit of server memory so making sure that the Blitz process has plenty of memory can help, see https://docs.openmicroscopy.org/omero/5.6/sysadmins/server-performance.html#memory-configuration.

What costs delete performance is not so much the file size as the number of OMERO model objects that are being deleted. So, many small files, many planes in images, many ROIs on images, that kind of thing. In that case it can indeed help to start by deleting sequentially in smaller chunks, like by shift-selecting across some of the images, to see how long they take. Good luck!

1 Like

Hi Mark,

Thanks a lot for the info ! I’ll check for the Blitz process memory before doing anything !

About deleting sequentially in smaller chunks, what’s the best approach ? Doing it manually ? Or is there any way to use the API in order to do it automatically ? For a lot of small files, it can become quite bothersome to have to delete manually every image/projects…

Thanks again !

It is absolutely possible to use the query service or FindChildren to find all the image IDs in the relevant projects then to have a delete script loop through each one individually to delete it. Then deleting the projects in the GUI should go easily for you. Do mention if you have trouble figuring out how to do this.

Update: Better to look for and delete each Fileset rather than each image, then multi-image formats (as common with high-content screening) will be properly handled.

1 Like

Thanks a lot for the tips ! We’ll try and delete small chunks of the data and monitor the server and if anything unexpected happens, I’ll come back and ask for your help ! :wink: