Setting up a quiz in OMERO

Hi,

I was wondering how we might go about setting up a quiz in in Omero. People would be instructed to add Rois to a slide scanner image highlighting various features, and the markers to asses their selections.

I would like to:

  • expose the image to users for a limited time
  • avoid creating a copy for each user
  • have user RoIs not be visible to one another
  • have the RoIs be linked to the user and visible to to markers after the quiz is over

I was wondering the technical details of how best to do this. This is probably the wrong way to approach it, but here’s a rough sketch how I though it might be done:

  • Store the images in a group that has the markers as members
  • somehow create link the images into the user’s default group (this way the RoIs won’t be visible to one another)
  • remove the link into the default group after a fixed time period, leaving the RoIs in place but not visible to the users
  • pretend the user IDs to the ROI labels
  • get the markers to review the RoIs and score the results

I’m not sure about the linking of images or the persistence of the RoIs. Any pointers or suggestions?

Thanks,

Chris

Hi Chris,

I don’t have any solutions, but I’ll add my voice here - we are also interested in solving very similar problems here (more geared towards multiple researchers annotating the same data) and I’d love to hear any thoughts on this!

best regards

Erick

Hi Chris, Erick,

I think you can achieve everything using standard group permissions except hiding ROIs from other users in a Read-Annotate group (group needs to be Read-Annotate to allow you to add ROIs to someone else’s Image).

Off the top of my head, I am thinking that a partial solution might be possible by overriding the URLs that OMERO.iviewer uses to load and save ROIs.

In OMERO.web 5.6.3 there is an option to set an app as the ‘root’ app under /. See https://github.com/ome/omero-web/pull/123
This app e.g. my_app could be used to implement a URL such as my_app/api/v0/m/rois/ which would override the existing URL /api/v0/m/rois/ if my_app was set as the root app, since Django will take the first URL that matches, see:
https://docs.djangoproject.com/en/3.0/topics/http/urls/

This would allow you to customise the behaviour of /api/v0/m/rois/ (which iviewer uses for loading ROIs) or /iviewer/persist_rois/ (for saving ROIs).

I imagine you could distinguish from users and ‘Markers’ by making the Markers into group owners.
The simplest option would be to NOT load ROIs from other users, unless current user is a ‘Marker’ (and leave the saving behaviour unchanged). This would only take a few of lines of code and users would allow users to save their ROIs without seeing other users’ ROIs. And ‘Markers’ would see everyone’s ROIs. However, users would still have permission to access the ROIs using other methods (e.g. use the Insight client).

To work around this would need a more elaborate solution. For example, the ROI save behaviour could store the ROIs in a private ‘Answers’ group (different group from the read-annotate group that the Images are in). You’d need some annotation to indicate that the images were created from ‘Image:123’ in the read-annotate group, but this couldn’t be a direct link since you can’t link between objects in different groups.
The, the ROI loading behaviour could be overridden to allow the ‘Markers’ (group owners) to retrieve the ROIs from the private group.

I don’t know how you want to store the results in OMERO? I guess you could use a similar strategy.

This approach would need some testing and validation before I’d say it’s a “solution”. It’s possible there are some blockers that haven’t occurred to me, but if you’re interested in giving this a try, we could certainly give you some pointers. As a first step, the “simple solution” would be quite easy to evaluate.

Regards,
Will.

2 Likes

Hi Will,

would this be relatively easy to put together using an external viewer (e.g. napari)? Provided users stick to the external viewer, the problem of not seeing other users’ ROIs is immediately solved if we just load pixels and then check for membership of “Markers” group before loading ROIs.

Hi Erick,

napari is certainly an option. I don’t know if you’ve seen https://gitlab.com/openmicroscopy/incubator/omero-napari

This is an “incubator” side-project (not been reviewed / tested by the OME team) but it allows you to directly open OMERO images in napari and allows you to save ROIs from napari back to OMERO. However, it doesn’t yet load ROIs from OMERO (shouldn’t be too hard to add) and it uses the OMERO CLI (omero-py) which is not yet working on Windows.
Also, this doesn’t yet support ‘Big’ (tiled) images which I think is what Chris was asking for?

But in your case this may work fine (and be relatively easy to extend & customise since it’s quite a small code base).

Regards,

Will.

I wasn’t aware of it until I tried writing code to do exactly the same thing and eventually stumbled upon it while debugging :smile: I have it currently running under WSL and I’ve Dockerized it, so there’s hope for Windows users! I have it at https://github.com/TheJacksonLaboratory/napari_omero_rois and planning on extending it further.

1 Like

Hi @erickratamero and @will-moore,

I found these papers about CyTest - Lianasa et. al, 2016 and Fassina et. al., 2017 - which extends on a lot of features we’ve been discussing and integrates the into a learning management system.

I’ve had a scan through the papers and I looks OMERO is just serving up pixels and a bulk of the logic around image browsing, RoIs manipulation and user management is being done in other custom built components.

Will, a bit of vague question, but do you think that OMERO will end up accumulating more functionality like this, would it focusing more on being a flexible repository, or some medium path?

Cheers,

Chris

Hi Chris,

Thanks for linking those papers - I’d not read them before, although I was aware that they were working in this area (e.g. https://forum.image.sc/search?q=ome%20seadragon). I think ROIs are being saved/retrieved from OMERO.

I think the OMERO.server and OMERO.web releases are unlikely to add many features specifically for “Quiz” usage. Even within the OMERO.web ecosystem, we’re moving more functionality into web ‘plugins’ (figure, iviewer etc) to allow more customised installs.

We would love to help anyone develop an OMERO.web “quiz” plugin or provide hooks to customise existing tools (e.g. iviewer) or add API endpoints for integrating 3rd-party tools (such as Moodle).
But I don’t think we have the expertise or capacity to push that effort ourselves.

Cheers,

Will