I am writing a new plugin for the new SlideBook 7 format in formats-gpl/src/loci/format/in, it’s written solely in java, will be open source. It uses so far 30 public classes (30 java files). Where should they go? In a separate jar file ? where would that jar file go?
Thanks for starting the discussion. I assume the Slidebook 7 format is a new version incompatible with the previous ones. Has the format been rolled out to production yet?
Is the new reader is now completely separate from the native reader developed by 3i a few years ago for the Slidebook 6 format? And do the two readers overlap or are they each handling a given version of the file format? Finally are you talking of a reader that is endorsed and would be actively maintained by 3i or is it a rather a personal project?
Regarding code location and integration, I think there are two generic solutions:
- either maintain the code in its own repository with a dependency on the upstream Bio-Formats API, package it into a standalone JAR with the reader. Some logic could be added in BIo-Formats to detect this reader at runtime, as the current Slidebook6Reader does
- or have the code integrated directly in the upstream codebase and have the reader shipped as part of the OME releases
There are pros and cons about each solution. The first option is probably the one that gives you the maximal flexibility in terms of development and release cycle as well as code ownership and licensing. It requires you to track and keep up with upstream Bio-Formats changes as well as a dedicated distribution mechanism like an update site for Fiji. The main advantage of the second option is to ship the reader as part of each release of the open-source Bio-Formats library. On the downside, you have no full control of the release cycle and we might need to discuss licensing as having third-party contributions in the OME codebase being BSD licensed is really the most viable way.
Hope this helps starting the discussion. If you have more details on what you intend to do, we can certainly go into more details.
Slidebook 7 format is a new version and is incompatible with the
previous ones. It is based on all the metadata being stored in a
number of yaml files, while the image,histogram, and mask data
are stored in npy binary format, one file per channel, per
timepoint, one subfolder per capture. It is now been internally
tested and I am not sure when it will be officially rolled out.
Currently I am still using the same .sld suffix for the 7 format
files, and in SlideBook I read a few characters to distinguish if
it is the old 6 format or the new 7 format.
I wonder if I implement a new SlideBook7Reader, with the proper
“isThisType” for the new format, if it will be possible to still
have also SlideBook6Reader available and if the system will pick
the right one based on the result returned from isThisType, in
other words, is it possible to have more than one plugin that
service the same file suffix *sld.
all understood, thanks for keeping us posted of the internal developments.
Yes, it is possible to have multiple readers associated with files with the same extension as this is the case for all TIFF-based formats for example.
ImageReader.getReader() API loops over the readers in the order defined in readers.txt and return the first match i.e. the first reader where
I expect inserting the entry as below would first try and detect whether the file is using the new Slidebook 7 file format and fallback to the native reader if that was not the case:
# external readers with unique file extensions + loci.formats.in.SlideBook7Reader # sld loci.formats.in.SlideBook6Reader[type=external] # sld
thank you for your answer, i was expecting it would work that
way. SlideBooKReader and SlideBook6Reader already look for a magic
number in the file, so the order will not matter between 6 and 7.
Any suggestions on the 30 class files?
without seeing the code, it’s hard to give suggestions on the best layout or code architecture. What is the scope of these 30 classes? Are there largely helpers consumed by the top-level reader?
Yes, they are helpers only used by top-level reader. Each class
represent a structure in the metadata file. They could be in the
main reader file (removing the public keyword i think), just makes
the file longer…
I have attached a couple of classes.
(Attachment CImageGroup.java is missing)
(Attachment CImageRecord70.java is missing)
java were rejected, attaching a zip this time
junk.zip (421 Bytes)
think got it right this time, previous zip file was bad
example.zip (4.46 KB)
thanks for the zip. So it looks like the source files mostly contain utility classes for accessing the container structure. While you are still developing your reader, I would not worry too much about the final location of these files as long as you compile and test everything.
For reference, some readers use internal private classes to capture some of the state - see here. Alternatively, you might want these classes to be more public API or simply allow them to be consumed standalone in a different context. In this case, having these classes maintained under a container-level package is probably be a good idea.
As they stand, the new classes use the
com.threei package namespace. If you want to retain this namespace, I suspect you probably will move towards packaging these classes as a standalone container library and distribute as a low-level permissively licensed JAR that your reader consumes.
A final note as your example classes seem depend on the
org.yaml:snakeyaml dependency. Bio-Formats currently does not have a YAML library however
snakeyaml is used elsewhere in the Fiji ecosystem. I don’t expect adding this dependency to be a majorproblem but we might need to discuss and agree on versions down the line if that is the case.
Thank you for your answer.
I was talking to my boss last night, and I think we will use the
internal private classes scheme, as we are only wanting to provide
a reader. If other users want access to the classes, they can look
at the reader to see how it’s done.
As or the package name, it was something I just threw in when I
started coding, but has no importance to us, it will be the
standard reader package name .
And for snakeyaml, yes, it is fundamental to the reader as all
the metadata files are written in yaml, and snakeyaml seems to be
the real only yaml utilities available in java. I write a pretty
basic yaml, so i suspect the version is not going to be a problem.
Thanks for the update, @nicola. Marking this topic as solved as per your previous answer. Thanks again for starting this conversation openly and feel free to keep using the public forum as you progress through development.
As a side-note, your experience and feedback might be worth sharing in the context of Next Generation File Formats for BioImaging.