Circular dependency of jars

update-site
fiji
imagej
updater

#1

@haesleinhuepf @frauzufall

When trying to upload jars to my update site (EMBL-CBA) I am sometimes getting this error message:

This occurs even when I am trying to upload an unrelated jar.

The repositories can be found here:

The weird thing is that just yesterday, for the same repositories it was working (without me changing anything consciously in between).

If anyone could help that would be great!


#2

Hey @Christian_Tischer,

unfortunately, I cannot reproduce the issue. However, I observed some version-conflicts while building and trying to upload your JARs.

While installing your JARs to my Fiji, an older version of BioFormats was installed which caused some “locally modified” error later. Try updating this dependency to a recent version:

Remove this line from all your POMs. Trust me, it causes more issues than it solves:

Your parent pom is not recent. Try updating to 23.2.0:

Last hint: Could you please try if the uploading works from an empty blank Fiji?

I hope that helps.

Cheers,
Robert


#3

Thanks!

I tried now installing the “fiji-plugin-yaml-metadata” repo with the “enforcer-rules” intact and am getting this error:

[INFO] ------------------------------------------------------------------------
[INFO] Building Fiji Plugin YAML Metadata 0.1.05
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-rules) @ fiji-plugin-yamlMetadataCreator ---
[INFO] Adding ignore: module-info
[INFO] Adding ignore: META-INF/versions/*/module-info
[INFO] Adding ignore: org.hibernate.stat.ConcurrentStatisticsImpl
[INFO] Adding ignore: org.junit.runner.Runner
[INFO] Adding ignore: module-info
[WARNING] Rule 3: org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message:
No Duplicate Classes Allowed!
- For duplicate transitive dependencies, add dependency exclusions.
- For duplications between direct dependencies, resolve or add
  ignored classes to this rule's configuration.

  Found in:
    commons-io:commons-io:jar:2.6:compile
    ch.systems.cisd:jhdf5:jar:14.12.0:compile
  Duplicate classes:
    org/apache/commons/io/FileCleaningTracker.class
    org/apache/commons/io/comparator/SizeFileComparator.class
    org/apache/commons/io/output/ProxyWriter.class
    org/apache/commons/io/input/CloseShieldInputStream.class
    org/apache/commons/io/FileCleaningTracker$Tracker.class
    org/apache/commons/io/filefilter/EmptyFileFilter.class
    org/apache/commons/io/output/ThresholdingOutputStream.class
    org/apache/commons/io/IOExceptionWithCause.class
    org/apache/commons/io/filefilter/RegexFileFilter.class
    org/apache/commons/io/comparator/PathFileComparator.class
    org/apache/commons/io/filefilter/NotFileFilter.class
    org/apache/commons/io/filefilter/NameFileFilter.class
    org/apache/commons/io/EndianUtils.class
    org/apache/commons/io/input/NullReader.class
    org/apache/commons/io/filefilter/WildcardFileFilter.class
    org/apache/commons/io/filefilter/AbstractFileFilter.class
    org/apache/commons/io/filefilter/IOFileFilter.class
    org/apache/commons/io/comparator/ReverseComparator.class
    org/apache/commons/io/input/DemuxInputStream.class
    org/apache/commons/io/filefilter/PrefixFileFilter.class
    org/apache/commons/io/IOUtils.class
    org/apache/commons/io/HexDump.class
    org/apache/commons/io/filefilter/DirectoryFileFilter.class
    org/apache/commons/io/input/ClosedInputStream.class
    org/apache/commons/io/input/NullInputStream.class
    org/apache/commons/io/input/ProxyReader.class
    org/apache/commons/io/FileDeleteStrategy$ForceFileDeleteStrategy.class
    org/apache/commons/io/input/AutoCloseInputStream.class
    org/apache/commons/io/comparator/LastModifiedFileComparator.class
    org/apache/commons/io/FileUtils.class
    org/apache/commons/io/filefilter/CanReadFileFilter.class
    org/apache/commons/io/filefilter/FalseFileFilter.class
    org/apache/commons/io/filefilter/DelegateFileFilter.class
    org/apache/commons/io/filefilter/AndFileFilter.class
    org/apache/commons/io/CopyUtils.class
    org/apache/commons/io/filefilter/SizeFileFilter.class
    org/apache/commons/io/output/CloseShieldOutputStream.class
    org/apache/commons/io/filefilter/WildcardFilter.class
    org/apache/commons/io/filefilter/FileFilterUtils.class
    org/apache/commons/io/filefilter/ConditionalFileFilter.class
    org/apache/commons/io/output/NullOutputStream.class
    org/apache/commons/io/input/CharSequenceReader.class
    org/apache/commons/io/filefilter/AgeFileFilter.class
    org/apache/commons/io/output/ClosedOutputStream.class
    org/apache/commons/io/output/DeferredFileOutputStream.class
    org/apache/commons/io/IOCase.class
    org/apache/commons/io/filefilter/FileFileFilter.class
    org/apache/commons/io/DirectoryWalker.class
    org/apache/commons/io/output/LockableFileWriter.class
    org/apache/commons/io/input/ClassLoaderObjectInputStream.class
    org/apache/commons/io/filefilter/CanWriteFileFilter.class
    org/apache/commons/io/FileDeleteStrategy.class
    org/apache/commons/io/filefilter/OrFileFilter.class
    org/apache/commons/io/FilenameUtils.class
    org/apache/commons/io/output/TeeOutputStream.class
    org/apache/commons/io/output/DemuxOutputStream.class
    org/apache/commons/io/input/ProxyInputStream.class
    org/apache/commons/io/LineIterator.class
    org/apache/commons/io/input/CountingInputStream.class
    org/apache/commons/io/filefilter/SuffixFileFilter.class
    org/apache/commons/io/output/ByteArrayOutputStream.class
    org/apache/commons/io/filefilter/TrueFileFilter.class
    org/apache/commons/io/input/TeeInputStream.class
    org/apache/commons/io/FileSystemUtils.class
    org/apache/commons/io/output/NullWriter.class
    org/apache/commons/io/comparator/DefaultFileComparator.class
    org/apache/commons/io/FileCleaningTracker$Reaper.class
    org/apache/commons/io/comparator/ExtensionFileComparator.class
    org/apache/commons/io/output/ProxyOutputStream.class
    org/apache/commons/io/DirectoryWalker$CancelException.class
    org/apache/commons/io/input/SwappedDataInputStream.class
    org/apache/commons/io/output/FileWriterWithEncoding.class
    org/apache/commons/io/FileCleaner.class
    org/apache/commons/io/filefilter/HiddenFileFilter.class
    org/apache/commons/io/comparator/NameFileComparator.class
    org/apache/commons/io/output/CountingOutputStream.class

[WARNING] Rule 8: org.scijava.maven.plugin.enforcer.RequireElements failed with message:
The following required elements have errors:
* url: element is missing

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.342 s
[INFO] Finished at: 2018-09-13T11:55:52+02:00
[INFO] Final Memory: 45M/1452M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce-rules) on project fiji-plugin-yamlMetadataCreator: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce-rules) on project fiji-plugin-yamlMetadataCreator: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed.
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
	at org.codehaus.classworlds.Launcher.main(Launcher.java:47)
Caused by: org.apache.maven.plugin.MojoExecutionException: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed.
	at org.apache.maven.plugins.enforcer.EnforceMojo.execute(EnforceMojo.java:237)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	... 21 more

Do you know what do do? I updated the repo such that you can see the pom.xml


#4

You have a dependency to bio-formats_plugins that includes a transient dependency to jhdf5 which comes as a fat JAR (or uber-JAR).

As suggested in the error message, you’ll need to add dependency exclusions. Look for example how it’s done here:

(Read here why Uber-JARs are problematic.)


#5

Maven hints you to the right solution: :wink:

This would look like this:

I changed some more stuff to compile it on my machine and sent a PR.

Can you try it again - the upload?

Cheers,
Robert


#6

Great! Thanks!!!
We figured it out here meanwhile as well :slight_smile:


#7

Does it solve the circular-dependecies issue?


#8

nope :frowning:
They now both happily install even with the enforcer rules, but still the O-dependency issue.
[EDIT]: Using a clean Fiji solved the issue.

@frauzufall What would be super awesome is to have some more automated way of uploading to the update site, without going through the Updater of a local Fiji installation. Not sure this is feasible?


#9

Maybe related questions: What to do in below cases?

33


#10

@haesleinhuepf
I still cannot get rid of the circular dependency issue.
I even removed the jars from the Fiji folder and it is still complaining…this seems to show that the circular dependency is actually detected on my update site rather than in the local Fiji installation…anything else I could still try? It is actually quite important, because i cannot upload anything to my update site anymore right now, because always this error shows up…


#11

ok. i found something weird, when checking the dependencies in the Fiji Updater UI i see below result:

It looks like Fiji things that the platynereisViewer actually has the yamlMetadataCreator and bio-formats as dependencies even though they are not specified in the pom.xml of the platynereisViewer. Any idea how that could happen?

In maven everything looks fine:

… I found that the bigdataviewer-viztools have a snakeYaml dependency, which I have an explicit dependency to another version to in the yamlMetadataCreator. Could that be the issue?

The problem here is that I need to specify a version foe snakeYaml, because I think it is not in the sci-java parent pom…


#12

@ctrueden
I found that snakeyaml-1.13 comes with Fiji, however the version is not specified in the parent pom. Initially I thus specified another version of snakeyaml (namely 1.23) in one of my pom.xml and I feel that this was creating some dependency issues. What do you think?


#13

Like this?

Dependencies are computed by the ImageJ Updater, by scanning the JAR files and doing bytecode analysis. Your Maven configuration is irrelevant to this process—the pom.xml file is not used. You can edit the result of the detection by editing the dependencies in the right-hand details panel of the Updater. Yes, that right-hand pane just appears as a big text area, and the fact that editing it actually impacts the result is bizarre. But this is how the Updater works right now.


#14

In fact, removing the circular dependencies manually by editing this text window was in the end the fix that solved the issue! Thanks for the hint!

Could you still comment on the snakeyaml-1.13 issue? Do you think it would be good to add it to the parent pom?


#15

Via search.imagej.net, I found that snakeyaml is used in a couple of projects, e.g. ui-behaviour, TrackMate3, bigcat and some projects in the openmicroscopy org, so I guess it makes sense to include it.

You can submit a PR to pom-scijava, e.g. modeled after this one:


#16

Ok. I tried: