HTTPS to make ImageJ/ Fiji more secure

fiji
security
imagej
https
#21

@frauzufall Some feedback from my side :wink:

I added the UpdaterBeta as update site and the updates sites were correctly replaced with https except the ones that are not on the domain sites.imagej.net but I guess this is a change to do on their side.
Downloading updates seems to work then.

However uploading a test macro to my personal update site fails.
I first notice that the URL for my update site had not been change to https so I removed it from the list, add it again via the “Add personnal update site” so that it gets a https too, but I still get a ClientProtocol Exception when trying to upload.

[INFO] Successfully retrieved OPTIONS.
[INFO] Successfully called PropFind, directory exists: .
[INFO] Successfully locked db.xml.gz.lock.
[INFO] Successfully uploaded to db.xml.gz.lock
[INFO] Successfully called PropFind, directory exists: scripts/Plugins/Acquifer/.
[INFO] Successfully unlocked db.xml.gz.lock.
[ERROR] null
org.apache.http.client.ClientProtocolException
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at net.imagej.plugins.uploaders.webdav.WebDAVUploader.runMethodOnClient(WebDAVUploader.java:517)
	at net.imagej.plugins.uploaders.webdav.WebDAVUploader.upload(WebDAVUploader.java:279)
	at net.imagej.plugins.uploaders.webdav.WebDAVUploader.upload(WebDAVUploader.java:229)
	at net.imagej.updater.FilesUploader.upload(FilesUploader.java:260)
	at net.imagej.ui.swing.updater.UpdaterFrame.upload(UpdaterFrame.java:805)
	at net.imagej.ui.swing.updater.UpdaterFrame$5$1.run(UpdaterFrame.java:311)
Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity.
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:226)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	... 8 more

2 Likes

#22

Thank you so much for testing! Only the sites from the list of available update sites got updated because the links on this list change if one requests it via HTTPS. The updater doesn’t touch the manually added sites yet. I could improve that by applying hardcoded replacements (http://sites.imagej.net/.. -> https://sites.imagej.net/..), I was avoiding that so far, but I don’t really see why.

Also thank you for reporting the error, I already spent quite some time with this one in the past -.- Some of my upload tests also fail right now which worked a few days ago… I’ll have a closer look… Could you still try uploading again after starting ImageJ in debug mode and post the output?

2 Likes

#23

Here is the debug messages. I just renamed the exe to debug. I am not sure that you get much more info. Let me know if you need a more verbose report.

Actually I was trying to upload an empty script file, that does not work.
Then I wrote a few characters in the file and the upload worked :slight_smile:

I tried with a second file and it worked too when not empty.

I guess the updater does not see the content but the weight of the file (the empty is indeed a 0 kB file).

-> When it does not work (empty text file)

[DEBUG] publish(
	context = org.scijava.Context@1a35993f
	consumed = false
	progress = -1
	maximum = -1
	status = Command finished: Update...
	warning = false,null,null), called from non-EDT Thread:null
[DEBUG] Sending request OPTIONS https://sites.imagej.net/Lthomas/ HTTP/1.1
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Response: 200 HTTP/1.1 200 OK
[DEBUG] Header: Date = Mon, 15 Apr 2019 07:20:15 GMT
[DEBUG] Header: Server = Apache/2.4.18 (Ubuntu)
[DEBUG] Header: DAV = 1,2
[DEBUG] Header: DAV = <http://apache.org/dav/propset/fs/1>
[DEBUG] Header: MS-Author-Via = DAV
[DEBUG] Header: Allow = OPTIONS,GET,HEAD,POST,DELETE,TRACE,PROPFIND,PROPPATCH,COPY,MOVE,LOCK,UNLOCK
[DEBUG] Header: Content-Length = 0
[DEBUG] Header: Keep-Alive = timeout=5, max=100
[DEBUG] Header: Connection = Keep-Alive
[DEBUG] Header: Content-Type = httpd/unix-directory
[INFO] Successfully retrieved OPTIONS.
[DEBUG] Sending request PROPFIND https://sites.imagej.net/Lthomas/ HTTP/1.1
[DEBUG] Header: Depth = 0
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Response: 207 HTTP/1.1 207 Multi-Status
[DEBUG] Header: Date = Mon, 15 Apr 2019 07:20:16 GMT
[DEBUG] Header: Server = Apache/2.4.18 (Ubuntu)
[DEBUG] Header: Content-Length = 856
[DEBUG] Header: Keep-Alive = timeout=5, max=100
[DEBUG] Header: Connection = Keep-Alive
[DEBUG] Header: Content-Type = text/xml; charset="utf-8"
[INFO] Successfully called PropFind, directory exists: .
[DEBUG] Sending request LOCK https://sites.imagej.net/Lthomas/db.xml.gz.lock HTTP/1.1
[DEBUG] Header: Timeout = Second-600
[DEBUG] Header: Depth = 0
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Response: 200 HTTP/1.1 200 OK
[DEBUG] Header: Date = Mon, 15 Apr 2019 07:20:18 GMT
[DEBUG] Header: Server = Apache/2.4.18 (Ubuntu)
[DEBUG] Header: Lock-Token = <opaquelocktoken:35260d88-f82e-43e5-81c8-3926ad70388d>
[DEBUG] Header: Vary = Accept-Encoding
[DEBUG] Header: Keep-Alive = timeout=5, max=100
[DEBUG] Header: Connection = Keep-Alive
[DEBUG] Header: Content-Type = text/xml; charset="utf-8"
[INFO] Successfully locked db.xml.gz.lock.
[INFO] Tried to obtain a lock (opaquelocktoken:35260d88-f82e-43e5-81c8-3926ad70388d):
org.apache.http.client.entity.LazyDecompressingInputStream@5e0a9263
[DEBUG] got last modified 1555312572000 = timestamp 20190415091612
[DEBUG] Sending request PUT https://sites.imagej.net/Lthomas/db.xml.gz.lock HTTP/1.1
[DEBUG] Header: If = <https://sites.imagej.net/Lthomas/db.xml.gz.lock> (<opaquelocktoken:35260d88-f82e-43e5-81c8-3926ad70388d>)
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Response: 201 HTTP/1.1 201 Created
[DEBUG] Header: Date = Mon, 15 Apr 2019 07:20:20 GMT
[DEBUG] Header: Server = Apache/2.4.18 (Ubuntu)
[DEBUG] Header: Location = https://sites.imagej.net/Lthomas/db.xml.gz.lock
[DEBUG] Header: Content-Length = 280
[DEBUG] Header: Keep-Alive = timeout=5, max=100
[DEBUG] Header: Connection = Keep-Alive
[DEBUG] Header: Content-Type = text/html; charset=ISO-8859-1
[INFO] Successfully uploaded to db.xml.gz.lock
[DEBUG] Sending request PROPFIND https://sites.imagej.net/Lthomas/scripts/Plugins/Acquifer/ HTTP/1.1
[DEBUG] Header: Depth = 0
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Response: 207 HTTP/1.1 207 Multi-Status
[DEBUG] Header: Date = Mon, 15 Apr 2019 07:20:22 GMT
[DEBUG] Header: Server = Apache/2.4.18 (Ubuntu)
[DEBUG] Header: Content-Length = 881
[DEBUG] Header: Keep-Alive = timeout=5, max=100
[DEBUG] Header: Connection = Keep-Alive
[DEBUG] Header: Content-Type = text/xml; charset="utf-8"
[INFO] Successfully called PropFind, directory exists: scripts/Plugins/Acquifer/.
[DEBUG] Sending request PUT https://sites.imagej.net/Lthomas/scripts/Plugins/Acquifer/Test_.py-20190415092018 HTTP/1.1
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Sending request UNLOCK https://sites.imagej.net/Lthomas/db.xml.gz.lock HTTP/1.1
[DEBUG] Header: Lock-Token = <opaquelocktoken:35260d88-f82e-43e5-81c8-3926ad70388d>
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Response: 204 HTTP/1.1 204 No Content
[DEBUG] Header: Date = Mon, 15 Apr 2019 07:20:23 GMT
[DEBUG] Header: Server = Apache/2.4.18 (Ubuntu)
[DEBUG] Header: Keep-Alive = timeout=5, max=99
[DEBUG] Header: Connection = Keep-Alive
[DEBUG] Header: Content-Type = application/x-gzip
[INFO] Successfully unlocked db.xml.gz.lock.
[ERROR] null
org.apache.http.client.ClientProtocolException
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at net.imagej.plugins.uploaders.webdav.WebDAVUploader.runMethodOnClient(WebDAVUploader.java:517)
	at net.imagej.plugins.uploaders.webdav.WebDAVUploader.upload(WebDAVUploader.java:279)
	at net.imagej.plugins.uploaders.webdav.WebDAVUploader.upload(WebDAVUploader.java:229)
	at net.imagej.updater.FilesUploader.upload(FilesUploader.java:260)
	at net.imagej.ui.swing.updater.UpdaterFrame.upload(UpdaterFrame.java:805)
	at net.imagej.ui.swing.updater.UpdaterFrame$5$1.run(UpdaterFrame.java:311)
Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity.
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:226)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	... 8 more

-> When it worked

[DEBUG] Sending request OPTIONS https://sites.imagej.net/Lthomas/ HTTP/1.1
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Response: 200 HTTP/1.1 200 OK
[DEBUG] Header: Date = Mon, 15 Apr 2019 07:27:45 GMT
[DEBUG] Header: Server = Apache/2.4.18 (Ubuntu)
[DEBUG] Header: DAV = 1,2
[DEBUG] Header: DAV = <http://apache.org/dav/propset/fs/1>
[DEBUG] Header: MS-Author-Via = DAV
[DEBUG] Header: Allow = OPTIONS,GET,HEAD,POST,DELETE,TRACE,PROPFIND,PROPPATCH,COPY,MOVE,LOCK,UNLOCK
[DEBUG] Header: Content-Length = 0
[DEBUG] Header: Keep-Alive = timeout=5, max=100
[DEBUG] Header: Connection = Keep-Alive
[DEBUG] Header: Content-Type = httpd/unix-directory
[INFO] Successfully retrieved OPTIONS.
[DEBUG] Sending request PROPFIND https://sites.imagej.net/Lthomas/ HTTP/1.1
[DEBUG] Header: Depth = 0
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Response: 207 HTTP/1.1 207 Multi-Status
[DEBUG] Header: Date = Mon, 15 Apr 2019 07:27:46 GMT
[DEBUG] Header: Server = Apache/2.4.18 (Ubuntu)
[DEBUG] Header: Content-Length = 856
[DEBUG] Header: Keep-Alive = timeout=5, max=100
[DEBUG] Header: Connection = Keep-Alive
[DEBUG] Header: Content-Type = text/xml; charset="utf-8"
[INFO] Successfully called PropFind, directory exists: .
[DEBUG] Sending request LOCK https://sites.imagej.net/Lthomas/db.xml.gz.lock HTTP/1.1
[DEBUG] Header: Timeout = Second-600
[DEBUG] Header: Depth = 0
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Response: 200 HTTP/1.1 200 OK
[DEBUG] Header: Date = Mon, 15 Apr 2019 07:27:47 GMT
[DEBUG] Header: Server = Apache/2.4.18 (Ubuntu)
[DEBUG] Header: Lock-Token = <opaquelocktoken:818613a3-db32-45d4-b1e9-1516e3dd8f6e>
[DEBUG] Header: Vary = Accept-Encoding
[DEBUG] Header: Keep-Alive = timeout=5, max=100
[DEBUG] Header: Connection = Keep-Alive
[DEBUG] Header: Content-Type = text/xml; charset="utf-8"
[INFO] Successfully locked db.xml.gz.lock.
[INFO] Tried to obtain a lock (opaquelocktoken:818613a3-db32-45d4-b1e9-1516e3dd8f6e):
org.apache.http.client.entity.LazyDecompressingInputStream@301f2fc9
[DEBUG] got last modified 1555312572000 = timestamp 20190415091612
[DEBUG] Sending request PUT https://sites.imagej.net/Lthomas/db.xml.gz.lock HTTP/1.1
[DEBUG] Header: If = <https://sites.imagej.net/Lthomas/db.xml.gz.lock> (<opaquelocktoken:818613a3-db32-45d4-b1e9-1516e3dd8f6e>)
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Response: 204 HTTP/1.1 204 No Content
[DEBUG] Header: Date = Mon, 15 Apr 2019 07:27:49 GMT
[DEBUG] Header: Server = Apache/2.4.18 (Ubuntu)
[DEBUG] Header: Keep-Alive = timeout=5, max=100
[DEBUG] Header: Connection = Keep-Alive
[DEBUG] Header: Content-Type = application/x-gzip
[INFO] Successfully uploaded to db.xml.gz.lock
[DEBUG] Sending request PUT https://sites.imagej.net/Lthomas/scripts/Plugins/Acquifer/Test_.py-20190415092747 HTTP/1.1
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Response: 201 HTTP/1.1 201 Created
[DEBUG] Header: Date = Mon, 15 Apr 2019 07:27:50 GMT
[DEBUG] Header: Server = Apache/2.4.18 (Ubuntu)
[DEBUG] Header: Location = https://sites.imagej.net/Lthomas/scripts/Plugins/Acquifer/Test_.py-20190415092747
[DEBUG] Header: Content-Length = 314
[DEBUG] Header: Keep-Alive = timeout=5, max=100
[DEBUG] Header: Connection = Keep-Alive
[DEBUG] Header: Content-Type = text/html; charset=ISO-8859-1
[INFO] Successfully uploaded to scripts/Plugins/Acquifer/Test_.py-20190415092747
[DEBUG] Sending request MOVE https://sites.imagej.net/Lthomas/db.xml.gz.lock HTTP/1.1
[DEBUG] Header: Destination = https://sites.imagej.net/Lthomas/db.xml.gz
[DEBUG] Header: If = <https://sites.imagej.net/Lthomas/db.xml.gz.lock> (<opaquelocktoken:818613a3-db32-45d4-b1e9-1516e3dd8f6e>)
[DEBUG] Header: User-Agent = Java/1.8.0_172
[DEBUG] Response: 204 HTTP/1.1 204 No Content
[DEBUG] Header: Date = Mon, 15 Apr 2019 07:27:51 GMT
[DEBUG] Header: Server = Apache/2.4.18 (Ubuntu)
[DEBUG] Header: Keep-Alive = timeout=5, max=99
[DEBUG] Header: Connection = Keep-Alive
[DEBUG] Header: Content-Type = application/x-gzip
[INFO] Successfully moved  db.xml.gz.lock to db.xml.gz.
[DEBUG] got last modified 1555313270000 = timestamp 20190415092750
2 Likes

#24

Awesome! I could reproduce and fix it, will also add the hardcoded replacement of http://sites.imagej.net to https://sites.imagej.net in manually added sites and upload a new version.

2 Likes