At our institution, we are continuously encouraging our users to work with QuPath for their analysis on both fluorescent and RGB images. And for quite some time now, we have also adopted OMERO and are trying to convince users to use it for their data management.
But we noticed that the OMERO extension in QuPath is currently handling only rendered images, meaning that RGB images stored in OMERO can be opened with QuPath, while fluorescent images are not supported.
We are wondering if the OMERO extension in QuPath has been developped from the OME team or on the QuPath side?
Is it planned to access OMERO’s raw pixel data from QuPath via the extension?
Or is it something that will be handled directly from OMERO, i.e when raw pixel data will be provided via the web in the form of zarr data?
QuPath, very earliest version by me, much much improved version by @melvingelbard
Ideally yes – but as mentioned on the other thread, as far as I know it isn’t currently possible via the web API. Therefore in the near future it will remain restricted to RGB images (even with the new features in v0.3).
I hope so! We definitely want to be able to access raw pixel values in QuPath, if that can be done without introducing too much extra complexity / more dependencies.
For now, the QuPath-OMERO extension is really for image viewing and (in v0.3 at least) annotation. It may also be ok for some kinds of analysis, particularly on brightfield histology slides, if the extra JPEG compression artifacts don’t matter much.
Happy to discuss how to make this happen now. There are two pieces. On the server-side, omero-ms-zarr will allow sharing public images now. So for a full solution, it will need to learn authentication. Having a client-side to drive that would be helpful.
Since we are heavily relying on both OMERO and QuPath, we have been inspired by @phaub’s QuPath scriptings and are now considering the idea to develop a transformed image server in QuPath to access data from OMERO using the RawPixelsStore(s) from the OMERO API, like we did in this early-stage BDV-OMERO connector.
Using the RawPixelsStores would allow us to access raw pixel data from OMERO in QuPath, thing which is currently not possible with the OMERO extension in QuPath since it relies on the OMERO web API.
Do you think it is reasonable and could be useful to start developing this, while awaiting the future stream of raw pixel data from the web API via omero-ms-zarr?
This sounds like something we should discuss next week at #ome2021! (Perhaps during your demo slot?)
In general, big thumbs up to enabling what QuPath needs. It may be more a question for @petebankhead, @melvingelbard, et al. but a heads up that if the desire is to use a web API rather than Ice, I would strongly urge that that web API just be Zarr.
It took a while to respond because we’re in the process of making some rather big structural changes that involved rewriting all our build scripts – and separating out our OMERO web extension into its own repository, in a move to become more extension-friendly overall. I was waiting to see how these would look before replying, in case the plan fell apart. But it seems to be looking ok.
Sounds good! I did look into raw pixel axis a long time ago (2018?), and I faintly recall having something moderately working. However, as @dsudar mentions, it was the dependencies that stopped me. QuPath doesn’t have a nice jar management system, and it seemed like too much of a maintenance headache for us to take on – and since our institute hasn’t quite adopted OMERO yet, there was no real incentive to prioritise it.
When I saw the web API addressed a large percentage of our needs without introducing new dependencies, and learning Zarr is on the horizon, that became our focus.
One way or another, we definitely need to be able to get raw pixel values into QuPath. It may be more of a @joshmoore / OME team question what the likely value / lifespan of a RawPixelsStore-based extension might be.
In terms of QuPath distribution, it would have to be an optional extension and that’s certainly easier if it’s limited to just one or two files. I’m not sure if a fat/shadow jar is an option, or if there’s a limit to quite how many dependencies can be squeezed in.
I think ‘transformed’ shouldn’t be necessary – that’s just if you want to do something to an existing ImageServer.
As long as you have the magical services file it should be auto-discovered when added to QuPath’s extensions directory.
The tricky bit (which I’ve discussed with @phaub a few times) is that any new servers should be JSON-serializable in a maintainable way. QuPath achieves that for its own servers with the help of the (badly-named…) ServerBuilder interface. In simple cases, you can use a default implementation that takes the name of the implementing class and corresponding URI(s), like we use with OMERO web here.
Whenever transformations are involved, the serialization and maintainability is a lot harder to achieve. Server transformations are fun, but can be difficult to support and that’s why QuPath doesn’t use them as much as it could. If you do use them, I’d suggest relying as little as possible on other QuPath classes – because if an API-breaking change occurs in a later release, it will make the entire image unreadable.
The new QuPath OMERO extension adds support for browsing OMERO servers to select images (as demo’ed by @melvingelbard in his lightning talk), and sending annotations in both directions. Requesting pixels is really not so much (i.e. just ImageServer and its associates).
It would require some changes to the current extension, but perhaps we could broaden it out to request pixels using any of the following
Web API (RGB-only)
The last two would require more code to be written, perhaps separately, but if these focus only on pixel reading and not on selecting images & adding to projects then it is probably not such a huge amount.
Do you mean for requesting pixels, or more broadly? It doesn’t feel good to be requesting rendered (and JPEG-compressed) images as we do right now, and I’d very happily get away from that. But I presume we’d still need the current web API for selecting images, sending ROIs – right?
In any case, we’d want to follow whatever you and the OME team strongly urge
Of course, we can always arrange a follow-up call.
You’re right, and thanks for bringing it up. pixel-buffer will get non-rendered image data via a web API, and as with RawPixelStore, no reason not to use it right now.
In terms of the long-livedness of these APIs, our hope would be that the dependency on Ice will go first, then custom OMERO APIs next, leaving a generic, community approved NGFF API, which at the moment is Zarr.