Migrating OMERO + OMERO.figure data to IDR

Dear all; and the OMERO team (Jason, Josh, Will et al)

We are working on finalising the data for a manuscript on our screen using single molecule FISH data and endogenous protein patterns in the entire nervous system of the Drosophila larvae (a simplified explanation), that has occupied us for quite a number of years.

We have made figures of each gene of 200-300 genes on omero.figure on our OMERO server. There of the order of 1000-2000 raw 3D image stacks for different parts of the nervous system in 4 channels.

Our aim is to publish the raw data, with the figures as a guide linked to the raw data (gives it a hierarchical structure with implicitly built meaning for exploration), in IDR.

You previously suggested that we export the figures as tiffs into IDR.

Has there been any progress that will allow us to export the Figures and their connections to the raw data (with correct contrasting of images (huge amount of work that represents a rich and important tapestry of interpretation of the data. There is also a large array of associated meta-data for our manual annotations as well as limitless data associated with each gene, that we have mostly focused to particular data domains of interest to the RNA field, the central points of our screen (to study post-transcriptional regulation).

I.e. migrate the data from our OMERO server to the EBI IDR server with the existing omero-figures and raw data with all the relationships and contrastings and in place (import of the metadata annotations maybe a stumbling block too).

I hope this makes sense, even if it is very brief and lacks examples (which we would of course be happy to supply).

We would welcome suggestions and would be happy to write some scripts based on the IDR API, with your guidance.

Many thanks for your help and advice
Best regards
Ilan
Cc key people in my lab and in Micron who are working on this screen project, led by Josh Titlow, whom you guys have met. Maria is doing a PhD on ML exploration of the imaging and associated bioinformatics (you also met her). David Pinto and Ian Dobbie you know of course. Darragh you have met. Aino and Mary are experienced RNA bioinformaticians and more general computational biologists as well as wet lab scientists. Jeff is doing his PhD on a related project but also working on the screen a lot, as well as learning bioinformatics.


Prof. Ilan Davis ilan.davis@bioch.ox.ac.uk http://www.ilandavis.com
https://twitter.com/ilandavis
Department of Biochemistry, The University of Oxford, South Parks Road, OXFORD OX1 3QU, UK
Direct Office No: (44) (0)1865 613265 Office Fax: (44) (0)1865 613340
Lab manager: Dr Darragh Ennis darragh.ennis@bioch.ox.ac.uk (44) (0)1865 613271 / 613272
PA: jolanta.parkinson@bioch.ox.ac.uk (44) (0)1865 613218


1 Like

Hi Ilan-

Thanks for the message and a great suggestion (and welcome to image.sc!).

There are are several interwoven topics here, including OMERO, data export formats, data submission to IDR, types of added value in public data, and most obviously, capturing the value of the work you and your colleagues have done in your study and not a) losing it or b) forcing it all to be done again.

As you may have gleaned, the short answer is, “No, none of this is in place… yet”. But there are many steps, some partial, some planned, that we can use to get at least some of what you are proposing done.

  1. Regarding a more holistic data export from OMERO, that captures all the annotations, connected data. We need a vessel for that data-- and that is what is being discussed elsewhere on this forum. Following our general approach, that vessel cannot serve only our needs, but must have much broader utility (OME can’t and won’t develop new proprietary formats!). We have funds from CZI to develop the concept and design the format (Thanks CZI!!) and are now seeking funds from other funders for the development of the format, and the software to support it.
  2. For the moment, there will be intermediate solutions for what you propose. As you and your team know, OMERO has a rich API, and you can pull rendering settings and other annotations for your datasets and export them in some convenient format (CSV or whatever). If you contact IDR Submissions, perhaps you can agree on a way to capture the metadata you want to transfer.
  3. Re: publishing files from OMERO.figure, this is a really cool idea, but definitely a new one. Fully integrated publication of figures won’t happen in the next few weeks, in time for publication of your current datasets and paper. However, we have published figures from our own papers on our institutional OMERO (published paper and public dataset). Using Figure 1A as an example, this resource has the corresponding figure from OMERO and original image data. So perhaps we can work together to get the data, annotations etc in IDR and, at least for the moment, publish the figures from your own institutional OMERO? Not ideal or perfect, but a step along the way?
  4. Side issue-- why weren’t the data in (3) published in IDR? Because they were not reference data.

A general comment: Image data publication has taken an enormous steps in the last few years. Our work on IDR with our colleagues at EMBL-EBI, and their recently announced BioImage Archive are huge steps. Despite all the amazing work, don’t think of the wider problem of publishing these datasets as “solved”. The PDB as we now know it took a few decades to reach its current level of power and sophistication. We definitely want to push towards an analogous level of power and functionality, but that will take more time. Your comments, use and especially submissions are an important part of that journey.

Hope that helps. If you have any other questions, definitely get in touch.

Cheers,

Jason

1 Like