Reverse proxy websocket

Hi,

I’m using caddy as my server backend, using it to perform reverse-proxy from the local loop to subdomains of the fqdn of my server.

I like it a lot for two reasons:

  • The config is less verbose than nginx
  • The https encryption is automatic and on by default (it uses letsencrypt)

I deploy omero with docker-compose.

Webclient

Connecting to the webclient works like a charm: the omero-webclient docker runs on port 4080 and is reverese-proxied to subdomain.example.com by the following Caddyfile
directive:

subdomain.example.com {
    reverse_proxy  127.0.0.1:4080
}

Allas, things do not seem so simple with websockets.

I tried various combinations of redirects but am hitting a wall now.

local connection with ws and wss

I can connect locally (on the host running the docker) through both ws and wss to the omeroserver container, with websocat or a python client:

websocat -k -v wss://localhost:4066
[INFO  websocat::lints] Auto-inserting the line mode
[INFO  websocat::stdio_threaded_peer] get_stdio_peer (threaded)
[INFO  websocat::ws_client_peer] get_ws_client_peer
[INFO  websocat::ws_client_peer] Connected to ws
IceP

(The -k switches meaning I allow insecure connections)

Distant connection

The 4066 and 4065 ports are reverse-proxied by caddy:

subdomain.example.com {
    reverse_proxy  127.0.0.1:4080
    reverse_proxy  /omero-ws 127.0.0.1:4065
    reverse_proxy  /omero-wss 127.0.0.1:4066
}

Port 4065

I have a positive response with websocat with the 4065 reverse-proxy:

websocat -v wss://subdomain.example.com/omero-ws
[INFO  websocat::lints] Auto-inserting the line mode
[INFO  websocat::stdio_threaded_peer] get_stdio_peer (threaded)
[INFO  websocat::ws_client_peer] get_ws_client_peer
[INFO  websocat::ws_client_peer] Connected to ws
IceP

The -k switch is not needed here, because as I understand it encryption is managed by caddy & letsencrypt

But when I try to connect with the python client I get:

c = omero.client('wss://subdomain.example.com/omero-ws')
session  = c.createSession(user, pwd)

I get the following traceback:

---------------------------------------------------------------------------
ProtocolException                         Traceback (most recent call last)
<ipython-input-43-07428b1f5940> in <module>
----> 1 session  = c.createSession(user, pwd)

~/miniconda3/lib/python3.7/site-packages/omero/clients.py in createSession(self, username, password)
    651                         ctx[omero.constants.IP] = self.__ip
    652                     rtr = self.getRouter(self.__ic)
--> 653                     prx = rtr.createSession(username, password, ctx)
    654 
    655                     # Create the adapter

~/miniconda3/lib/python3.7/site-packages/Glacier2_Router_ice.py in createSession(self, userId, password, _ctx)
    256         """
    257         def createSession(self, userId, password, _ctx=None):
--> 258             return _M_Glacier2.Router._op_createSession.invoke(self, ((userId, password), _ctx))
    259 
    260         """

ProtocolException: exception ::Ice::ProtocolException
{
    reason = SSL error occurred for new outgoing connection:
remote address = XX.XX.XX.XX:443
tlsv1 alert internal error: SSL alert number 80
}

Port 4066

When I try the 4066 port I get:

websocat -v  wss://subdomain.example.com/omero-wss
[INFO  websocat::lints] Auto-inserting the line mode
[INFO  websocat::stdio_threaded_peer] get_stdio_peer (threaded)
[INFO  websocat::ws_client_peer] get_ws_client_peer
websocat: WebSocketError: Received unexpected status code (502 Bad Gateway)
websocat: error running

The -k switch does not change the output.

As those network layers are a bit obscure to me, I don’t really know what to do here? Can I tell omero use the caddy certs? Can I tell IceSSL to trust the caddy certs? I tried using:

    IceSSL.TrustOnly=CN=subdmomain.example.com;CN=*.example.com

But I think it broke the local connection and did not help connecting from outside.

As always, any hints or help apreciated!

Sorry for the super long post, happy to provide more details!

Guillaume

Hi Guillaume

Sorry for the delay in replying.
Did you try omero-certificates · PyPI to see if it fixes the issue?

Cheers
Jmarie

Hi Jean-Marie, No worries, I let that one stew while I’m looking into the rest of my issues :slight_smile:

omero-certificates is ran at the container start up, what I want to test now is tell caddy to use the self-signed certificates for the */omero-wss incoming requests.

I’ll keep you updated (I should be looking into this later this week or next week)

Thanks for you help!

Guillaume