Hello!
I frequently use zarr datasets to test napari. @joshmoore, @will-moore, and others in the OME team have been doing super cool work serving up datasets to browse remotely. For example, if you pip install napari ome-zarr
, you can do:
napari https://s3.embassy.ebi.ac.uk/idr/zarr/v0.1/6001240.zarr
and browse that dataset in napari. You can also do:
ome-zarr download https://s3.embassy.ebi.ac.uk/idr/zarr/v0.1/6001240.zarr
to download it locally, then napari 6001240.zarr
to open it locally (much better performance).
One issue Iāve run into, though, is that the latency from the UK to Australia is a killer. So although Iāve been able to download (some of) these datasets to try things out locally, itās very hard to use napari with the remote copy, and Iāve started to work with our local research cloud to host these closer to (my) home.
In addition to the above (tiny test) data, there are a couple of other datasets that would be useful to test:
https://s3.embassy.ebi.ac.uk/idr/zarr/v0.1/9822151.zarr
(The SARS-CoV-2 EM volume from the tweet linked above; see also this forum post.)
and
https://s3.embassy.ebi.ac.uk/idr/zarr/v0.1/4007801.zarr
(2TB+ 3D+t volume.)
Again, thanks to the great work of the OME team, downloading these is easy. What Iād like to know is how to host them on local cloud infrastructure, so that I can do something like napari https://datasets.nectar.org.au/idr/zarr/v0.1/9822151.zarr
and have reasonable performance.
Iām totally naive when it comes to s3 interfaces and serving up object stores, so the more detailed the instructions, the better!
Thank you!