Blind annotation in OMERO

Hi everyone!

We have a project at work where a researcher wants to annotate images from certain OMERO datasets/projects blindly, that is, a user shouldn’t be able to see filenames, metadata, anything but the actual image. We came up with a few ideas and would like to hear what people think! In order of (decreasing) complexity:

  1. omero-web-blind: fork omero-web to create a separate web client that doesn’t show the sensitive data to users. It would still look exactly like omero-web and talk to the same omero-server, so annotations would be linked to the correct images by default. It involves writing/deploying a separate web client, though.

  2. Standalone Python client: write a small Python application that pulls and displays images from the desired project/dataset. User can write annotations that are linked back to the original image on the server. Problems: large images become an issue, annotations will probably be limited to free text or controlled vocabulary (trickier to have users draw ROIs, etc).

  3. “annotation” dataset: crawl through original images, check which ones haven’t been annotated and import those to a separate annotation dataset where they will be stripped of all identifying information (no metadata on key-value pairs, randomized name - no duplication of data because we in-place import everything anyway). Store the new image name as a key-value pair on the original. In addition to that, crawl through the “annotation” dataset, check which images have already been annotated, link the annotations back to the original and delete them. Not “truly” blind because someone with ill intent can always find the original from the annotation copy, but probably good enough and fairly simple to put together.

Thoughts? Are we reinventing the wheel and something like that already exists? At the moment we’re tending towards 3 because it’s fairly generic and easy to implement, and it’s a fairly small project for the moment. We’d love to hear what you have to say!

Hi Eric,

What’s the starting point? How does the researcher find the Datasets/Projects they are meant to annotate? Do you have to build that UI, or can you provide the researcher with e.g. Dataset IDs?

When you say “wants to annotate images”, do you mean add Tags, Comments, Map Annotations, Ratings, ROIs?

If you were starting with Dataset IDs, you could use the iviewer/?dataset=1 and add ROIs to all the images in that Dataset. You’d only need to hide the Image name, either via your 3) routine above, or by making a slightly modified iviewer?

Will

1 Like

My understanding is that they will need to find the dataset numbers themselves in OMERO (for the original images). “Annotation” here is pretty broad, but mainly comments and/or map annotations, plus ROIs. I suppose using iviewer/?dataset=1 would solve the ROI issue with a bit of modification, but we still have the text annotations to worry about, unfortunately.

One of the simplest solutions I could imagine is a workflow similar to @evenhuis scripts at https://code.research.uts.edu.au/MIF/OMERO-instructions/wikis/organising_data/csv/Managing-Key-values-with-csv-files.
The user could view the images in iviewer and compile their comments and Key/Value pairs into a CSV file, using only the Image ID instead of Name.
Then you could use a Python script to upload the CSV and apply annotations to the Images. This could be run via the CLI, so the user doesn’t get to see the images in the webclient (to avoid seeing other metadata etc).

How will they find the Datasets in OMERO? By name, or by some annotation? Tag etc?
Again, you could write a small Python script that could list the Images or Datasets that need annotating and then create the empty .CSV file with Image IDs and column names only.
This could even contain links to open the images/datasets in iviewer.

With OMERO 5.6 it’s a lot easier for users to install the client python libs, so they can use the omero CLI (to run scripts on the server) or to run a local Python script to read/write annotations to OMERO.
https://docs.openmicroscopy.org/omero/5.6.0/developers/Python.html

conda create -n myenv -c ome python=3.6 zeroc-ice36-python omero-py
conda activate myenv

You could even package your code as a pip-installable omero plugin “omero-blind”, so users could do:

pip install omero-blind
# e.g. list datasets that need annotating
omero blind datasets
# create a blank 'blind.csv' to fill out for a chosen Dataset
omero blind create_csv Dataset:1
# after filling it out, upload to create annotations in OMERO
omero blind upload blind.csv

Does this sound feasible? The advantage is you can create your own client without worrying about developing a UI. Let us know if you want any pointers with this (or any other) approach?

Will

1 Like

Hi Will,

thanks for the suggestions again! The idea of omero-blind and a small tweak on iviewer sound feasible. We’ll need to sit down with the researchers to understand exactly what the use case will look like and which option will probably work better given that. We appreciate the input!

As a side note, this sounds slightly similar to some of the teaching scenarios that have arisen over the years. One example is https://github.com/crs4/moodle.omero-qtypes. @lucalianas has mentioned that he doesn’t regularly come to the forums, so if you’re interested, you might also post a GitHub issue there. ~Josh

1 Like