Setting up OMERO in a new lab - suggestions, stories and advice welcome

omero
correlative
#1

My lab does a lot of correlative imaging, so we have a lot of images from the same samples with fluorescence & electron microscopy from multiple instruments. There’s a lot of variety & we don’t have a good way to keep track of which images are part of the same sample sub-region. We think we need some kind of database to help us manage this.

  1. Is OMERO a good choice here?
  2. What’s the experience of other groups, and do you have advice for setting a system up from scratch?
  3. Any suggestions for how best to structure the database?
  4. How can we automate or enforce consistency in this structure as much as possible?

I appreciate your thoughts

2 Likes
#2

There were a couple of nice blog posts by @erickratamero about this topic.
Maybe he can give you more pointers :slight_smile:

https://blogs.warwick.ac.uk/camdu/entry/implementing_omero_from/
https://blogs.warwick.ac.uk/camdu/entry/omero_deployment_the/

3 Likes
#3

That’s really handy to know, thanks for sharing

#4

Hi Genevieve,

we’ve had quite a lot of success with OMERO at Warwick Medical School. Setting it up took some work, but anyone who is comfortable around a Linux terminal should be able to do it. You could structure it as one dataset per sub-region; definitely the trickiest part is enforcing consistent standards for that structure. The one thing that OMERO doesn’t do is folder structures, which on one hand limits the complexity of the possible data organisation schemes, but on the other hands keeps things fairly simple and easy to get. I’d probably have a browse at IDR before anything else. If you like what you see there, then OMERO is probably a good choice for you. I’d be more than happy to give some pointers in addition to those blog posts!

3 Likes
#5

OMERO does allow images to be placed into datasets (multiple at once) which can be placed into projects but that’s only two levels of hierarchy. It also allows other structured metadata to be added, for instance at https://help.openmicroscopy.org/managing-data.html#annotating, which in the server’s database is stored more like https://docs.openmicroscopy.org/ome-model/5.6.4/developers/structured-annotations.html. You’ll see this kind of thing on the right as you explore images in the IDR.

If you are willing to push beyond what the graphical clients easily support, e.g., by writing some scripts, then you can make extensive use of the OME model to manage your data. For example, with ROI folders at https://blog.openmicroscopy.org/data-model/future-plans/2016/05/23/folders-upcoming/ the regions of interest on different images that are actually different views of the same region of the sample could share the same folder. Or, the aforementioned structured annotations can be used to represent metadata as both images and ROIs may be annotated. However, as an example of where the graphical clients currently fall short: I don’t think they would show annotations on ROIs.

Someday we’d love to get to substantially improve OMERO’s support for correlative imaging. If you could tell us more about your use case and what you’re trying to do in your workflows then we might be able to help you along and that could guide us in proposing future work.

Cheers,
Mark (OME)

2 Likes
#6

how is the management of different permissions to access/sahre the images and data?

#7

OMERO follows a UNIX-like permissions approach of allowing owners, members and others various levels of read and write access, with “annotate” being a level meaning that they can’t write data but they can annotate it. Every image is owned by some user and is in a group; users may be in many groups and images may be moved among groups. The permissions can be set from wholly private to the image owner all the way up to read-write by anybody; intermediate levels allow labs to collaborate usefully. An introduction is at https://help.openmicroscopy.org/sharing-data.html and there is more information at https://docs.openmicroscopy.org/latest/omero5.4/sysadmins/server-permissions.html.

Here in Dundee we have an OMERO production server that is used by many research groups with various levels of collaborative and private access but it also hosts published data in other (OMERO-level) public groups where the DOI links in their papers bring readers back to data on that same server.

Cheers,
Mark

3 Likes
#8

Thanks Erick! I’ll have a chat with out IT team, and might end up taking you up on that offer down the track.

I read your follow up post at https://blogs.warwick.ac.uk/camdu/entry/omero_deployment_the/ - now it’s a year later is there anything else you’d add to those comments?

#9

Thanks for those detailed comments, Mark

Everyone I’ve talked to has mentioned the limited hierarchy available, seems like that’s a pretty common annoyance. We will definitely need to see what we can do to compensate with user added metadata & tags. I expect most of our analysis will be scripted.

#10

@mtbc

Someday we’d love to get to substantially improve OMERO’s support for correlative imaging. If you could tell us more about your use case and what you’re trying to do in your workflows then we might be able to help you along and that could guide us in proposing future work.

Happy to discuss this more with you, maybe email is a better choice for an extended discussion? Let me know what you prefer.

#11

Hi @GenevieveBuckley,

I’d say keeping everything on this thread at the moment could make for good reading for someone else trying to do the same thing. I’ve added the #correlative tag so hopefully others can find it. (“correlative-microscopy” is too long. Other suggestions?)

At the moment, OMERO doesn’t provide anything out of the box, so someone else will probably have to say if there’s a better choice out there. That being said, we’d certainly look forward to both capturing what it is that you’d like to see OMERO look like with your data, as well as find workarounds that would bridge the gap for you and others until there is a complete solution.

~Josh

2 Likes
#12

I think since then the biggest thing for us was finding a way to get data automatically from the microscope computers - it lowered the entry barrier and solved our biggest problem at the time (adoption). Now we have a core of heavy users who love it and a large amount of people who use it because they don’t need to worry about data transfer, external storage or anything else.

#13

that is very interesting!

What were the challenges there?
How would a workflow then be for a user, from acquisition to analysis?

Cheers,
Christopher

#14

There were a few different challenges - we’re running multiple operating systems on our microscopes, with computer that range from brand-new to, well, relatively old - there’s everything from Windows XP and CentOS 5 to up-to-date Windows 10. We’ve looked at adapting existing tools like AutoTx or the existing CLI importer, but we had nothing that would work in every single system. We settled for having an extra Ubuntu 18.04 machine that mounts data directories on the microscope computers as SMB shares inside a local network and runs the CLI importer for individual users. It’s extremely improvised and it breaks more often than I’d like, but 90% of the time it works every time.

For the user, they just need to save their data onto their folder inside a specified “Data” directory in the microscope computer - overnight, the data is imported to their user account in OMERO and (upon a successful import) moved to an “imported” directory on the microscope computer (in case they want a local copy). From there, they just need to access the OMERO server (using their university credentials) in any way they prefer.

3 Likes
#15

Thank you!

Performing an automated import is for sure the best way to ensure proper usage.

I just wonder how useful it is to import everything directly from the acquisition.
I mean one needs to have some metadata from the experiment in order to make use of the data. Also not all images are of useful quality… So I would be afraid of drowning in useless stuff, either by missing information or by useless quality.

Do you have any feedback from the groups? How actively are they using the OMERO server or is this just a data graveyard?

#16

I don’t think the “data graveyard” issue is any worse than it was before OMERO - people who aren’t doing much with their data now are the people who would just have transferred all their acquired data to a storage server somewhere and not do much with it anyway. We’ve found that, even for the “data graveyard” users, they’re still coming back to OMERO to prepare figures for talks/publications. In addition to that, we have a good core of heavy users who actually care about their data management, and for them life is now much easier than it was. Sharing, annotating and publishing take a fraction of the time it used to. Of course, there’s no way to force anyone to use anything, so we still have users who will just acquire data, save it outside the specified Data folder and do whatever they used to anyway.

In general, I don’t think life is worse than it used to be for any kind of user, and it’s definitely much better for a certain subset. I’ll consider that a success!

2 Likes
#17

Cool! Thanks a lot for the very nice discussion :slight_smile: I would love to push such an image database at some point at my place too. Have to prepare the ground first though.

1 Like
#18

Thanks much for the information! How do you handle populating metadata? I am thinking about more study-specific data and not that which is auto-populated by the imaging software.