Build Omero.server with own BioFormat-Version

Hi all,

I want to build my own OMERO.server version with an adapted Bio-Format version. Could you please help me to figure out how to do that with the new separate omero repositories and Gradle build system?

Susanne

The change you need to make is in omero-model's build.gradle (which is like a pom.xml). There’s a line,

api("ome:formats-gpl:6.1.1")

or similar and you can adjust that version number and name SNAPSHOT versions. The formats-gpl JAR manifest lists formats-bsd and so on down the stack. If you have the formats-gpl-6.1.1.jar (or whichever) in your local Maven repository (the correct place is probably somewhere in ~/.m2/repository/ome/formats-gpl/... as if you had done mvn install from your Bio-Formats build) then I would expect Gradle to pick it up. Let us know how you get on.

You can adjust that line to have the group and artifact ID of whichever JAR provides OMERO the Bio-Formats readers it needs.

Dear Marc,

I have changed the BioFormat version and the SNAPSHOT version in omero-model’s build.gradle and had execute gradle build in omero-model. Do I have to compile the other subprojects too? I’m not quite sure yet where my server libs are located after the build.

Best,
Susanne

From what I can easily find it does look as if our developer documentation is presently rather light on this topic. To get you going I can at least share what I do:

I have a checkout of the parent https://github.com/ome/omero-build superbuild with its various submodules and I use many of the “foreach” helper scripts from https://github.com/ome/build-infra. I set each of the submodules to some recent tag or to the head of their master or somesuch. These will typically be for SNAPSHOT versions (from omero-build check with foreach-get-version) but if not then I do,

foreach-get-version-as-snapshot | foreach-set-version 

To couple each local component build to its next, from omero-build I do,

foreach-get-version-as-snapshot >/tmp/versions
VERSION_LOG=/tmp/versions foreach-set-dependencies

Then I build each omero-* module in order (model common romio renderer server blitz gateway-java) using gradle publishToMavenLocal and typically appending -x javadoc for speed. This puts all the artifacts into my ~/.m2/repository/.

Over in the https://github.com/ome/openmicroscopy repository I substitute the version numbers at the end of etc/omero.properties with those printed in omero-build by running foreach-get-version-as-property then I run ant build or ant build-dev or whatever to build OMERO. If it complains about not being able to find some artifact then I delete that part within ~/.m2/repository/ then try again.

No doubt the above is not as shiny as it could or should be but it’s what I get by with so I hope it works for you until we get proper procedures documented or another OMERO developer chimes in here with a better idea.

I forgot to mention, these days for that last stage of building OMERO I find that I need to do so from my Python virtual environment which includes the usual OMERO prerequisites installed. Which of them the build now needs I have not yet determined.

Thanks a lot for the description!

I have to run the “old” build at the end to create the server lib packages, right?
Might it be possible that the “build-dev” run overwriting the .m2/repository libs?
Maybe a “build-server” run and a “release-zip” run also works?

I’ll try it out…

Hi Susanne,

That’s a good point. Mark’s workflow documents the OME process and is probably necessary if you also had other customizations to the Java stack e.g. server extensions that you need to compile alongside your Bio-Formats version.

I gave your suggestion a quick test from the latest v5.6.0-m1 tag of ome/openmicroscopy

sbesson@ls30630:openmicroscopy (develop) $ git diff
diff --git a/ivy.xml b/ivy.xml
index 816d310659..987eb9cc2b 100644
--- a/ivy.xml
+++ b/ivy.xml
@@ -11,6 +11,7 @@
   <dependencies>
     <dependency org="org.openmicroscopy" name="omero-blitz" rev="${versions.omero-blitz}"/>
     <dependency org="org.openmicroscopy" name="omero-gateway" rev="${versions.omero-gateway}"/>
+    <dependency org="ome" name="formats-gpl" rev="6.3.0"/>
     <!-- runtime dependencies from dsl/ivy.xml -->
     <dependency org="janino" name="janino" rev="${versions.janino}"/>
   </dependencies>

Running /build.py build-dev creates a distribution bundle which overrides the JARs as expected

sbesson@ls30630:openmicroscopy (develop) $ unzip -qc dist/lib/server/formats-api.jar META-INF/MANIFEST.MF | grep Version
Manifest-Version: 1.0
Specification-Version: 6.3
Implementation-Version: 6.3.0

In case the only component of the server you want to modify is Bio-Formats, this is probably the most effective solution indeed.

I think I got it. But to be on the safe side, I’ll summarize it again:
If I want to link a new Bio-Format version to the untouched server code I can use the ant scripts from https://github.com/ome/openmicroscopy. But if I have also server extensions I have to compile first the components in https://github.com/ome/omero-build and to create the distribution bundle I have to run the “old” ant scripts after that.

Thank’s a lot!

Best,
Susanne

1 Like

Sounds right. :+1: You do that fiddling with version numbers at the end of etc/omero.properties only in the latter case.

Sorry marc, I have to ask again for the omero-insight build, how can I change there the bio-format version?

I’d use the above workflow to introduce the different Bio-Formats into the components build and from foreach-get-version-as-property reporting the versions.omero-gateway version that’s what I’d put in omero-insight’s build.gradle, e.g., replace,

implementation("org.openmicroscopy:omero-gateway:5.5.4")

with

implementation("org.openmicroscopy:omero-gateway:5.5.5-SNAPSHOT")

However, notice that Insight’s gradle has an exclude group for that omero-gateway dependency. If you have no OMERO changes then it could be interesting to try adding another exclude line in that block,

exclude group: "ome", module: "formats-gpl"

to ignore the Bio-Formats from the Java Gateway and add to the build.gradle a direct dependency to your adjusted artifact, e.g.,

implementation("ome:formats-gpl:6.3.1-SNAPSHOT")

to see if that makes for an easier alternative bypassing the OMERO components build. Gradle’s new to me but in some quick testing the approach looks promising.

Thank’s Marc, that helps!
But I’m little confused about the version numbers.
For example, if I want to rebuild Omero.server 5.5.1, I switch to the Tag 5.5.1 in git:ome/openmicroscopy and the Tag 5.5.1 in git:ome/omero-build and I would expect, that the version numbers for omero-blitz and omero-gateway are the same in [git:ome/openmicroscopy]etc/omero.properties and in the build.gradle files of this submodules in git:ome/omero-build, but they’re not. Are the Tag numbers of git:ome/openmicroscopy and git:ome/omero-build related?

This is true, the various components’ version numbers grow increasingly decoupled from one another. This allows us to release targeted updates without needlessly having to release everything else in the software stack. There is some discussion here now about quite what the CLI bin/omero version should return given that, for example, the OMERO.server and OMERO.cli might have quite different numbers. In short: expect there to be some mismatches on this point, that the main OMERO artifacts may depend on different versions of the server, gateway, etc.; the main thing is just to make sure that the component snapshot versions match the downstream dependencies to couple your local builds together as you desire.

Also note that for component compatibility from 5.5 we switch to semantic versioning so that breaking changes are thenceforth indicated by a change in major version, the first number. So, for example, Insight 5.6.x can connect to OMERO.blitz 5.5.x but not 5.4.x or 6.0.0.

I fully understand that part. I just find it a little confusing that ome/openmicroscopy and omero-build version are not coupled because of the fact I have to run ome/openmicroscopy to bundle the server version of omero-build.
But maybe it’s because that I only perceive a small portion of the develop and build process.

ome/openmicroscopy is indeed about more than just bundling its server code dependency, e.g., it also provides the template configurations for etc/. One can imagine that, for instance, we could then use ome/openmicroscopy to release extra LUTs without also re-releasing the core server code.

The omero-build components do their own kind of bundling too in building JARs which is why Insight doesn’t need any ome/openmicroscopy artifacts.

Admittedly ome/openmicroscopy may yet shrink further as we decouple more elements from it but that’s all for the future.