Circular dependency of jars

Christian_Tischer · 2018-09-12 15:39:25 UTC

When trying to upload jars to my update site (EMBL-CBA) I am sometimes getting this error message:

This occurs even when I am trying to upload an unrelated jar.

The repositories can be found here:

The weird thing is that just yesterday, for the same repositories it was working (without me changing anything consciously in between).

If anyone could help that would be great!

haesleinhuepf · 2018-09-12 23:04:03 UTC

Hey @Christian_Tischer,

unfortunately, I cannot reproduce the issue. However, I observed some version-conflicts while building and trying to upload your JARs.

While installing your JARs to my Fiji, an older version of BioFormats was installed which caused some “locally modified” error later. Try updating this dependency to a recent version:

github.com

tischi/fiji-plugin-yaml-metadata/blob/master/pom.xml#L132


		</dependency>


		<dependency>
			<groupId>ome</groupId>
			<artifactId>bio-formats_plugins</artifactId>
		</dependency>


		<dependency>
			<groupId>org.yaml</groupId>
			<artifactId>snakeyaml</artifactId>
			<version>1.16</version>
		</dependency>


		<!--<dependency>-->
			<!--<groupId>com.fasterxml.jackson.dataformat</groupId>-->
			<!--<artifactId>jackson-dataformat-yaml</artifactId>-->
			<!--<version>2.9.6</version>-->
		<!--</dependency>-->


		<!--<dependency>-->
			<!--<groupId>com.fasterxml.jackson.core</groupId>-->

Remove this line from all your POMs. Trust me, it causes more issues than it solves:

github.com

tischi/fiji-plugin-yaml-metadata/blob/master/pom.xml#L75


	</issueManagement>
	<ciManagement>
		<system>None</system>
	</ciManagement>


	<properties>
		<package-name>de.embl.cba.registration</package-name>
		<license.licenseName>bsd_2</license.licenseName>
		<license.copyrightOwners>EMBL</license.copyrightOwners>
		<imagej.app.directory>/Applications/Fiji.app/</imagej.app.directory>
		<enforcer.skip>true</enforcer.skip>
	</properties>




	<repositories>
		<!-- NB: for SciJava dependencies -->
		<repository>
			<id>imagej.public</id>
			<url>https://maven.imagej.net/content/groups/public</url>
		</repository>
	</repositories>

Your parent pom is not recent. Try updating to 23.2.0:

github.com

tischi/fiji-plugin-yaml-metadata/blob/master/pom.xml#L8


<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>


	<parent>
		<groupId>org.scijava</groupId>
		<artifactId>pom-scijava</artifactId>
		<version>23.1.0</version>
	</parent>


	<artifactId>fiji-plugin-yamlMetadataCreator</artifactId>
	<version>0.1.03</version>


	<name>Fiji Plugin YAML Metadata</name>
	<description>TODO</description>
	<inceptionYear>2018</inceptionYear>
	<organization>
		<name>EMBL</name>

Last hint: Could you please try if the uploading works from an empty blank Fiji?

I hope that helps.

Cheers,
Robert

Christian_Tischer · 2018-09-13 09:56:55 UTC

Thanks!

I tried now installing the “fiji-plugin-yaml-metadata” repo with the “enforcer-rules” intact and am getting this error:

[INFO] ------------------------------------------------------------------------
[INFO] Building Fiji Plugin YAML Metadata 0.1.05
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-rules) @ fiji-plugin-yamlMetadataCreator ---
[INFO] Adding ignore: module-info
[INFO] Adding ignore: META-INF/versions/*/module-info
[INFO] Adding ignore: org.hibernate.stat.ConcurrentStatisticsImpl
[INFO] Adding ignore: org.junit.runner.Runner
[INFO] Adding ignore: module-info
[WARNING] Rule 3: org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message:
No Duplicate Classes Allowed!
- For duplicate transitive dependencies, add dependency exclusions.
- For duplications between direct dependencies, resolve or add
  ignored classes to this rule's configuration.

  Found in:
    commons-io:commons-io:jar:2.6:compile
    ch.systems.cisd:jhdf5:jar:14.12.0:compile
  Duplicate classes:
    org/apache/commons/io/FileCleaningTracker.class
    org/apache/commons/io/comparator/SizeFileComparator.class
    org/apache/commons/io/output/ProxyWriter.class
    org/apache/commons/io/input/CloseShieldInputStream.class
    org/apache/commons/io/FileCleaningTracker$Tracker.class
    org/apache/commons/io/filefilter/EmptyFileFilter.class
    org/apache/commons/io/output/ThresholdingOutputStream.class
    org/apache/commons/io/IOExceptionWithCause.class
    org/apache/commons/io/filefilter/RegexFileFilter.class
    org/apache/commons/io/comparator/PathFileComparator.class
    org/apache/commons/io/filefilter/NotFileFilter.class
    org/apache/commons/io/filefilter/NameFileFilter.class
    org/apache/commons/io/EndianUtils.class
    org/apache/commons/io/input/NullReader.class
    org/apache/commons/io/filefilter/WildcardFileFilter.class
    org/apache/commons/io/filefilter/AbstractFileFilter.class
    org/apache/commons/io/filefilter/IOFileFilter.class
    org/apache/commons/io/comparator/ReverseComparator.class
    org/apache/commons/io/input/DemuxInputStream.class
    org/apache/commons/io/filefilter/PrefixFileFilter.class
    org/apache/commons/io/IOUtils.class
    org/apache/commons/io/HexDump.class
    org/apache/commons/io/filefilter/DirectoryFileFilter.class
    org/apache/commons/io/input/ClosedInputStream.class
    org/apache/commons/io/input/NullInputStream.class
    org/apache/commons/io/input/ProxyReader.class
    org/apache/commons/io/FileDeleteStrategy$ForceFileDeleteStrategy.class
    org/apache/commons/io/input/AutoCloseInputStream.class
    org/apache/commons/io/comparator/LastModifiedFileComparator.class
    org/apache/commons/io/FileUtils.class
    org/apache/commons/io/filefilter/CanReadFileFilter.class
    org/apache/commons/io/filefilter/FalseFileFilter.class
    org/apache/commons/io/filefilter/DelegateFileFilter.class
    org/apache/commons/io/filefilter/AndFileFilter.class
    org/apache/commons/io/CopyUtils.class
    org/apache/commons/io/filefilter/SizeFileFilter.class
    org/apache/commons/io/output/CloseShieldOutputStream.class
    org/apache/commons/io/filefilter/WildcardFilter.class
    org/apache/commons/io/filefilter/FileFilterUtils.class
    org/apache/commons/io/filefilter/ConditionalFileFilter.class
    org/apache/commons/io/output/NullOutputStream.class
    org/apache/commons/io/input/CharSequenceReader.class
    org/apache/commons/io/filefilter/AgeFileFilter.class
    org/apache/commons/io/output/ClosedOutputStream.class
    org/apache/commons/io/output/DeferredFileOutputStream.class
    org/apache/commons/io/IOCase.class
    org/apache/commons/io/filefilter/FileFileFilter.class
    org/apache/commons/io/DirectoryWalker.class
    org/apache/commons/io/output/LockableFileWriter.class
    org/apache/commons/io/input/ClassLoaderObjectInputStream.class
    org/apache/commons/io/filefilter/CanWriteFileFilter.class
    org/apache/commons/io/FileDeleteStrategy.class
    org/apache/commons/io/filefilter/OrFileFilter.class
    org/apache/commons/io/FilenameUtils.class
    org/apache/commons/io/output/TeeOutputStream.class
    org/apache/commons/io/output/DemuxOutputStream.class
    org/apache/commons/io/input/ProxyInputStream.class
    org/apache/commons/io/LineIterator.class
    org/apache/commons/io/input/CountingInputStream.class
    org/apache/commons/io/filefilter/SuffixFileFilter.class
    org/apache/commons/io/output/ByteArrayOutputStream.class
    org/apache/commons/io/filefilter/TrueFileFilter.class
    org/apache/commons/io/input/TeeInputStream.class
    org/apache/commons/io/FileSystemUtils.class
    org/apache/commons/io/output/NullWriter.class
    org/apache/commons/io/comparator/DefaultFileComparator.class
    org/apache/commons/io/FileCleaningTracker$Reaper.class
    org/apache/commons/io/comparator/ExtensionFileComparator.class
    org/apache/commons/io/output/ProxyOutputStream.class
    org/apache/commons/io/DirectoryWalker$CancelException.class
    org/apache/commons/io/input/SwappedDataInputStream.class
    org/apache/commons/io/output/FileWriterWithEncoding.class
    org/apache/commons/io/FileCleaner.class
    org/apache/commons/io/filefilter/HiddenFileFilter.class
    org/apache/commons/io/comparator/NameFileComparator.class
    org/apache/commons/io/output/CountingOutputStream.class

[WARNING] Rule 8: org.scijava.maven.plugin.enforcer.RequireElements failed with message:
The following required elements have errors:
* url: element is missing

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.342 s
[INFO] Finished at: 2018-09-13T11:55:52+02:00
[INFO] Final Memory: 45M/1452M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce-rules) on project fiji-plugin-yamlMetadataCreator: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce-rules) on project fiji-plugin-yamlMetadataCreator: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed.
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
	at org.codehaus.classworlds.Launcher.main(Launcher.java:47)
Caused by: org.apache.maven.plugin.MojoExecutionException: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed.
	at org.apache.maven.plugins.enforcer.EnforceMojo.execute(EnforceMojo.java:237)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	... 21 more

Do you know what do do? I updated the repo such that you can see the pom.xml

imagejan · 2018-09-13 11:33:01 UTC

You have a dependency to bio-formats_plugins that includes a transient dependency to jhdf5 which comes as a fat JAR (or uber-JAR).

As suggested in the error message, you’ll need to add dependency exclusions. Look for example how it’s done here:

github.com

fiji/register_virtual_stack_slices/blob/a1a23df2980d9c38c44a85c19fb8af93ea47b3ff/pom.xml#L152-L162


		<dependency>
			<groupId>ome</groupId>
			<artifactId>bio-formats_plugins</artifactId>
			<exclusions>
				<exclusion>
					<!-- NB: Exclude problematic uber-JAR that bundles commons-io. -->
					<groupId>ch.systems.cisd</groupId>
					<artifactId>jhdf5</artifactId>
				</exclusion>
			</exclusions>
		</dependency>

(Read here why Uber-JARs are problematic.)

haesleinhuepf · 2018-09-13 14:37:43 UTC

Maven hints you to the right solution:

This would look like this:

I changed some more stuff to compile it on my machine and sent a PR.

Can you try it again - the upload?

Cheers,
Robert

Christian_Tischer · 2018-09-13 15:08:39 UTC

Great! Thanks!!!
We figured it out here meanwhile as well

haesleinhuepf · 2018-09-13 16:24:19 UTC

Does it solve the circular-dependecies issue?

Christian_Tischer · 2018-09-14 07:44:12 UTC

nope
They now both happily install even with the enforcer rules, but still the O-dependency issue.
[EDIT]: Using a clean Fiji solved the issue.

@frauzufall What would be super awesome is to have some more automated way of uploading to the update site, without going through the Updater of a local Fiji installation. Not sure this is feasible?

Christian_Tischer · 2018-09-14 09:24:50 UTC

Maybe related questions: What to do in below cases?

Christian_Tischer · 2018-09-17 13:09:07 UTC

@haesleinhuepf
I still cannot get rid of the circular dependency issue.
I even removed the jars from the Fiji folder and it is still complaining…this seems to show that the circular dependency is actually detected on my update site rather than in the local Fiji installation…anything else I could still try? It is actually quite important, because i cannot upload anything to my update site anymore right now, because always this error shows up…

Christian_Tischer · 2018-09-17 13:18:24 UTC

ok. i found something weird, when checking the dependencies in the Fiji Updater UI i see below result:

It looks like Fiji things that the platynereisViewer actually has the yamlMetadataCreator and bio-formats as dependencies even though they are not specified in the pom.xml of the platynereisViewer. Any idea how that could happen?

In maven everything looks fine:

… I found that the bigdataviewer-viztools have a snakeYaml dependency, which I have an explicit dependency to another version to in the yamlMetadataCreator. Could that be the issue?

The problem here is that I need to specify a version foe snakeYaml, because I think it is not in the sci-java parent pom…

Christian_Tischer · 2018-09-17 14:55:56 UTC

@ctrueden
I found that snakeyaml-1.13 comes with Fiji, however the version is not specified in the parent pom. Initially I thus specified another version of snakeyaml (namely 1.23) in one of my pom.xml and I feel that this was creating some dependency issues. What do you think?

ctrueden · 2018-09-17 18:03:21 UTC

Like this?

github.com

fiji/fiji/blob/fiji-2.0.0-pre-7/bin/upload-site-simple.sh

#!/bin/sh

# When used from an existing ImageJ.app directory, uploads
# the current ImageJ.app state to the specified ImageJ wiki
# update site.
#
# REQUIRED: WIKI_UPLOAD_PASS environment variable to be set
#           appropriately

die () {
	echo "$*" >&2
	exit 1
}

test $# -eq 2 ||
die "Usage: $0 <update-site> <webdav-user>"

update_site="$1"
webdav_user="$2"
url="http://sites.imagej.net/$update_site/"

This file has been truncated. show original

Dependencies are computed by the ImageJ Updater, by scanning the JAR files and doing bytecode analysis. Your Maven configuration is irrelevant to this process—the pom.xml file is not used. You can edit the result of the detection by editing the dependencies in the right-hand details panel of the Updater. Yes, that right-hand pane just appears as a big text area, and the fact that editing it actually impacts the result is bizarre. But this is how the Updater works right now.

Christian_Tischer · 2018-09-18 08:28:09 UTC

In fact, removing the circular dependencies manually by editing this text window was in the end the fix that solved the issue! Thanks for the hint!

Could you still comment on the snakeyaml-1.13 issue? Do you think it would be good to add it to the parent pom?

imagejan · 2018-09-18 14:49:24 UTC

Via search.imagej.net, I found that snakeyaml is used in a couple of projects, e.g. ui-behaviour, TrackMate3, bigcat and some projects in the openmicroscopy org, so I guess it makes sense to include it.

You can submit a PR to pom-scijava, e.g. modeled after this one:

Christian_Tischer · 2018-09-19 08:24:25 UTC

Ok. I tried: