OMERO 5.5.0 CLI importer giving IceSSL exception

Dear all,

we’re currently running 5.5.0 on our test environment and 5.4.10 on production, planning to update production soon as well. We run automated imports every night from a remote computer; A couple days ago, I decided to move the CLI importer on that computer to 5.5.0 as well. We’ve been having issues with SSLHandshakeExceptions for a while, which are being bypassed by adding --skip upgrade to our import commands.

However, 5.5.0 gives me this error no matter what I do:

InternalException: Failed to connect: exception ::Ice::PluginInitializationException
{
    reason = IceSSL: unable to set ciphers using `ADH:@SECLEVEL=0':
invalid command
}

Imports using 5.4.10 still work fine, both to test and production servers. 5.5.0 fails for both servers with the same error. I’ve tried every workaround I could find (adding @SECLEVEL=0 in multiple places, with or without --skip upgrade, multiple Ice versions, multiple Java versions). The error is always the same. Current OpenSSL version is 1.1.1. No matter what I do, the result is the same: 5.5.0 gives me that error, while 5.4.10 imports just fine.

Thanks in advance for the help!

Hi

Could you give us full details of your setup including:

  • Operating System including version
  • openssl full version (output of openssl version and also the package name/version)
  • java full version (output of java -version and also the package name/version)
  • output of omero admin diagnostics
  • details of how you installed and configured OMERO.server and your client including any custom configuration?

If you’re running the client and server on seperate computers could you give us details for both, and also see if you can reproduce the error when running the client on the same computer as the server?

  1. Ubuntu 18.04
  2. OpenSSL 1.1.1 11 Sep 2018
  3. openjdk version “11.0.3” 2019-04-16, but it also occurred with openjdk version “1.8.0_212”
  4. Client-side:

================================================================================
OMERO Diagnostics (admin) 5.5.0-ice36-b121
================================================================================
        
Commands:   java -version                  11.0.3    (/usr/bin/java)
Commands:   python -V                      2.7.15    (/usr/bin/python)
Commands:   icegridnode --version          3.6.4     (/usr/bin/icegridnode)
Commands:   icegridadmin --version         3.6.4     (/usr/bin/icegridadmin)
Commands:   psql --version                 not found

Server:     icegridnode                    not started

Log dir:    /home/erick/OMERO.server/var/log 
No logs available

Environment:OMERO_HOME=(unset)             
Environment:OMERO_NODE=(unset)             
Environment:OMERO_MASTER=(unset)           
Environment:OMERO_USERDIR=(unset)          
Environment:OMERO_TMPDIR=(unset)           
Environment:PATH=/home/erick/bin:/home/erick/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 
Environment:PYTHONPATH=(unset)             
Environment:ICE_HOME=(unset)               
Environment:LD_LIBRARY_PATH=(unset)        
Environment:DYLD_LIBRARY_PATH=(unset)      

Cannot list deployed applications.
OMERO data dir:'/OMERO'                       Exists? False	Is writable? False
OMERO temp dir:'/home/erick/omero/tmp'        Exists? True	Is writable? True   (Size: 0)

also tried icegridnode (and icegridadmin) version 3.7.0 with the same results.

  1. Both test and production servers are identical in everything other than OMERO version:
Commands:   java -version                  1.8.0     (/bin/java)
Commands:   python -V                      2.7.5     (/bin/python)
Commands:   icegridnode --version          3.6.4     (/bin/icegridnode)
Commands:   icegridadmin --version         3.6.4     (/bin/icegridadmin)
Commands:   psql --version                 9.6.13    (/bin/psql)

Both have OpenSSL 1.0.2k-fips 26 Jan 2017, and both give me the same error when 5.5.0 is on the client side and work fine with 5.4.10 client-side. Confusingly, running client and server on the same computer work fine for our test (5.5.0) and give me a SSLHandshakeException with production (5.4.9), even when using --skip upgrade or --no-upgrade-check.

I’m afraid we’re still having no luck whatsoever with reproducing this issue. I have set up CentOS 7 / Ubuntu 18.04 servers and clients where the Ubuntu ones have the same OpenJDK 11 and OpenSSL that you mention in your list. You also go on to mention a different OpenSSL version that I noticed matched RHEL/CentOS 7 so that’s why I brought that into the mix. For the Ubuntu server I did need our documentation’s,

bin/omero config set omero.glacier2.IceSSL.Ciphers HIGH:ADH:@SECLEVEL=0

(I also used our corresponding OS walkthroughs for the Ice 3.6.4 installation.) I can use OMERO 5.5.0 CLI to import just fine from RHEL 7 to Ubuntu 18.04, from Ubuntu 18.04 to CentOS 7, and from Ubuntu 18.04 to Ubuntu 18.04, all without doing anything special. Sorry not to have more useful news!

Sorry, I should have clarified: we’re using Ubuntu 18.04 client-side but the servers are CentOS 7.

Well, I’m stumped on this one. We’re not doing anything out of the ordinary on either side, but I’m 95% sure this issue is client-side. For the moment we can still use 5.4.10 client-side just fine, so we’ll probably stick with that. I might try to build Ice from source and see if that makes any difference. For the future, I might just wipe the client clean and reinstall everything to see if that solves it. In any case, thanks!

In case it helps you figure the difference I gathered some information from the Ubuntu client and CentOS server for which 5.5 imports work fine. (I had to put Ice on the path on the client for “diagnostics” to work happily.)
System-Info.zip (30.3 KB)

ok, I think the issue might be with the Ice versions - I’m using Ice 3.6.4 from apt packages (that, I presume, are still the 16.04 versions), while your Ubuntu client doesn’t have those packages installed. Are you compiling Ice from source? Would you have instructions for that?

thanks again!

I used the instructions at https://docs.openmicroscopy.org/omero/5.5.0/sysadmins/unix/server-ubuntu1804-ice36.html#installing-prerequisites though as I recall I did have to do something fairly obvious with at least one of PATH or PYTHONPATH if using that suggested /opt/ location (I expect Ubuntu might expect /usr/local/?) much as described in https://forum.image.sc/t/26695.

Ah, there we go - stubborn me decided I didn’t need server install instructions for a CLI client. This seems to be sorted now, thanks again!

Thank you for the update Erick, it’s great to hear that you have this working now. I can see that it’s weird for a CLI client to be somewhat in the “server” category for installation!