Today, the Chan Zuckerberg Initiative announced their first round of Essential Open Science Software projects, and we’re thrilled to have had Bio-Formats selected with an application on next generation file formats.
A part of that funding will help to support the image.sc community with their existing file format issues. There are billions of files left in proprietary formats that still need to be read and that situation will remain until they can all be converted to something common, open, and stable.
The other part of the funding, however, focuses on the design of such a next-generation file format that we’d like to discuss here. Our blog post this summer, OME’s position regarding file formats, outlined our reasoning. The feedback – likes, re-links, and personal comments – has been encouraging.
With the announcement today, it’s time to consider how best to move forward with the design and general consensus building. How does everyone have their voice heard? From vendors from #industry wanting a format that can be performantly written to users wanting something that saves them time and headaches.
Our first goal will be a design proposal like OME005 - which specified pyramidal OME-TIFFs, leading to a release last year. However, there are any number of preliminary discussions before a design proposal is ready. That discussion for OME005 mostly took place on a GitHub issue. But this next step needs to be as inclusive as possible, and image.sc seems the ideal place to do so (rather than, say, a slack that everyone needs to join).
This will be a bit of an experiment to find a way to use discourse that works for everyone. Options that we’re considering:
- An open group that everyone can join and @-mention
- A tag that everyone can follow
- A discourse chat plugin
Anyone interested in getting involved is encouraged to jump into this thread. We’d encourage a certain chattiness (think gitter or zulip).
We’ll also be looking for ways to cross-link conversations to non-image.sc groups. For example, the Zarr team had a coordinated application funded, focused on building a new spec developed in concert with the n5 devs, leading hopefully to a useful base for a future imaging format. Regular calls, etc. will be organized as necessary.
This is all a great and exciting first step on a path to more common, open file-formats that we’re very much looking forward to.