Error with OMERO.web

Hi, my name is Salim. My problem is that I keep getting an error when going to OMERO.web through browser and the error says “Internal Server Error”, I checked the nginx logs and the error.log file was empty, but the access.log had the lines about my tries to reach the web application. Does anyone know what could cause the problem?

Is this error a 404? If so it might be the same problem I am having

Hi, no, I dont see the 404 error, it is reachable for me, it just looks like this:

Maybe the first question is what is the operating system you are installing to?

it is Linux, redhat 7 on a workstation.

I am not experienced on RedHat but I would expect some page error code if nginx had failed to load the OMERO subfolder. You said the error log is providing feedback but are you sure all three, omero, omero web and nginx are loaded? In Centos one can use *-systems.status to check all but I don’t know the equivalent on RedHat, sorry

Hi @salim2203
Could you give us the following details please:

  • Details of how you setup your server, e.g. did you follow one of the OMERO walkthroughs or some other instructions? If so please give us the URL so we know exactly what you’ve done
  • Your server and web configuration, and any diagnostics. This will depend on how you’ve installed OMERO. E.g.
    • OMERO.server/bin/omero config get
    • OMERO.server/bin/omero admin diagnostics
    • OMERO.web/bin/omero config get
    • OMERO.web/bin/omero web diagnostics
  • Is there anything in the OMERO.server or OMERO.web logs?
  • Can you connect to OMERO using either the CLI or Insight?

So for the setup I followed regular omero instructions.
Here are the diagnostics:

[omero@bioimages ~]$ OMERO.server/bin/omero config get
**omero.data.dir=/images**
**omero.db.pass=**********
**omero.web.application_server=wsgi-tcp**
**omero.web.check_version=true**
**omero.web.open_with=[["Image viewer", "webgateway", {"supported_objects": ["image"], "script_url": "webclient/javascript/ome.openwith_viewer.js"}], ["omero_figure", "new_figure", {"supported_objects": ["images"], "target": "_blank", "label": "OMERO.figure"}]]**

[omero@bioimages ~]$ OMERO.server/bin/omero admin diagnostics

**================================================================================**
**OMERO Diagnostics (admin) 5.5.1-ice36-b122**
**================================================================================**

**Commands:   java -version                  1.8.0     (/usr/bin/java)**
**Commands:   python -V                      2.7.5     (/usr/bin/python)**
**Commands:   icegridnode --version          3.6.4     (/usr/bin/icegridnode)**
**Commands:   icegridadmin --version         3.6.4     (/usr/bin/icegridadmin)**
**Commands:   psql --version                 9.6.13    (/usr/bin/psql)**
**Commands:   openssl version                1.0.2     (/usr/bin/openssl)**

**Server:     icegridnode                    running**
**Server:     Blitz-0                        active (pid = 26917, enabled)**
**Server:     DropBox                        active (pid = 26951, enabled)**
**Server:     FileServer                     active (pid = 26958, enabled)**
**Server:     Indexer-0                      active (pid = 26981, enabled)**
**Server:     MonitorServer                  active (pid = 26959, enabled)**
**Server:     OMERO.Glacier2                 active (pid = 26990, enabled)**
**Server:     OMERO.IceStorm                 active (pid = 26961, enabled)**
**Server:     PixelData-0                    active (pid = 26960, enabled)**
**Server:     Processor-0                    active (pid = 26963, enabled)**
**Server:     Tables-0                       active (pid = 26994, enabled)**
**Server:     TestDropBox                    inactive (enabled)**

**Log dir:    /home/omero/OMERO.server/var/log exists**
**Log files:  Blitz-0.log                    346.0 MB      errors=3    warnings=0 **
**Log files:  Blitz-0.log.1                  524.0 MB      errors=6    warnings=29**
**Log files:  Blitz-0.log.2                  524.0 MB      errors=3    warnings=28**
**Log files:  Blitz-0.log.3                  524.0 MB      errors=1    warnings=22**
**Log files:  Blitz-0.log.4                  524.0 MB      errors=11   warnings=49**
**Log files:  Blitz-0.log.5                  524.0 MB      errors=5    warnings=35**
**Log files:  Blitz-0.log.6                  524.0 MB      errors=8    warnings=50**
**Log files:  Blitz-0.log.7                  524.0 MB      errors=2    warnings=23**
**Log files:  Blitz-0.log.8                  524.0 MB      errors=3    warnings=27**
**Log files:  Blitz-0.log.9                  524.0 MB      errors=9    warnings=73**
**Log files:  DropBox.log                    10.0 KB       errors=12   warnings=6 **
**Log files:  FileServer.log                 2.0 KB**
**Log files:  Indexer-0.log                  1.0 MB        errors=12   warnings=20**
**Log files:  MonitorServer.log              3.0 KB**
**Log files:  OMEROweb.lock                  empty**
**Log files:  OMEROweb.log                   3.0 MB        errors=1354 warnings=2 **
**Log files:  PixelData-0.log                519.0 KB      errors=12   warnings=20**
**Log files:  Processor-0.log                1.0 MB        errors=24   warnings=48**
**Log files:  Tables-0.log                   97.0 KB       errors=24   warnings=48**
**Log files:  TestDropBox.log                n/a**
**Log files:  master.err                     1.0 MB        errors=6    warnings=12**
**Log files:  master.out                     empty**
**Log files:  Total size                     5075.74 MB**


**Environment:OMERO_HOME=(unset)**
**Environment:OMERO_NODE=(unset)**
**Environment:OMERO_MASTER=(unset)**
**Environment:OMERO_USERDIR=(unset)**
**Environment:OMERO_TMPDIR=/tmp/omero**
**Environment:PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/omero/.local/bin:/home/omero/bin**
**Environment:PYTHONPATH=/home/omero/OMERO.server/lib/python**
**Environment:ICE_HOME=(unset)**
**Environment:LD_LIBRARY_PATH=(unset)**
**Environment:DYLD_LIBRARY_PATH=(unset)**

**OMERO SSL port:4064**
**OMERO TCP port:4063**
**OMERO data dir:'/images'                      Exists? True      Is writable? True**
**OMERO temp dir:'/tmp/omero'                   Exists? True      Is writable? True   (Size: 0)**

**JVM settings: Blitz-${index}                -Xmx7200m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions**
**JVM settings: Indexer-${index}              -Xmx4800m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions**
**JVM settings: PixelData-${index}            -Xmx7200m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions**
**JVM settings: Repository-${index}           -Xmx4800m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions**

Alright, now I can see how many errors I got in logs, I will look through them and then respond here again.
And yes, I can get into omero through cli and insight too.

I am having the same Internal Server Error as salim2203 is describing.

I have recently upgraded from 5.4.10 to OMERO Server 5.5.1 with OMERO.web separate, as per the latest instructions on Ubuntu 16.04 LTS. After the upgrade everything was working fine, but due to a power cut I had to restart the server machine, and the error occurred after the restart. I have tried to restart the nginx service. OMERO.server is reachable through CLI and Insight. OMERO.web is kind of reachable on hostname/omero, but it looks to be without CSS.

OMERO.server/bin/omero config get:

omero.data.dir=/tank/OMERO
omero.db.name=omero_database
omero.db.pass=********
omero.db.poolsize=100
omero.db.user=db_user_omero
omero.jvmcfg.strategy=percent

OMERO.server/bin/omero admin diagnostics

================================================================================
OMERO Diagnostics (admin) 5.5.1-ice36-b122
================================================================================

Commands:   java -version                  1.8.0     (/usr/bin/java)
Commands:   python -V                      2.7.12    (/home/omero/omerowebvenv/bin/python -- 2 others)
Commands:   icegridnode --version          3.6.5     (/usr/bin/icegridnode)
Commands:   icegridadmin --version         3.6.5     (/usr/bin/icegridadmin)
Commands:   psql --version                 10.10     (/usr/bin/psql)
Commands:   openssl version                1.0.2     (/usr/bin/openssl)

Server:     icegridnode                    running
Server:     Blitz-0                        active (pid = 5115, enabled)
Server:     DropBox                        active (pid = 5159, enabled)
Server:     FileServer                     active (pid = 5161, enabled)
Server:     Indexer-0                      active (pid = 5163, enabled)
Server:     MonitorServer                  active (pid = 5166, enabled)
Server:     OMERO.Glacier2                 active (pid = 5167, enabled)
Server:     OMERO.IceStorm                 active (pid = 5169, enabled)
Server:     PixelData-0                    active (pid = 5170, enabled)
Server:     Processor-0                    active (pid = 5171, enabled)
Server:     Tables-0                       active (pid = 5176, enabled)
Server:     TestDropBox                    inactive (enabled)

Log dir:    /home/omero/OMERO.server/var/log exists
Log files:  Blitz-0.log                    203.0 MB      errors=6    warnings=140
Log files:  Blitz-0.log.1                  524.0 MB      errors=13   warnings=48
Log files:  Blitz-0.log.2                  524.0 MB      errors=16   warnings=194
Log files:  DropBox.log                    6.0 KB        errors=0    warnings=2
Log files:  FileServer.log                 796 B
Log files:  Indexer-0.log                  426.0 KB      errors=0    warnings=6
Log files:  MonitorServer.log              3.0 KB
Log files:  PixelData-0.log                127.0 KB      errors=0    warnings=6
Log files:  Processor-0.log                1.0 MB        errors=0    warnings=1
Log files:  Tables-0.log                   4.0 KB        errors=0    warnings=1
Log files:  TestDropBox.log                n/a
Log files:  master.err                     199.0 KB      errors=2    warnings=4
Log files:  master.out                     empty
Log files:  Total size                     1254.13 MB


Environment:OMERO_HOME=(unset)
Environment:OMERO_NODE=(unset)
Environment:OMERO_MASTER=(unset)
Environment:OMERO_USERDIR=(unset)
Environment:OMERO_TMPDIR=(unset)
Environment:PATH=/home/omero/omerowebvenv/bin:/home/omero/bin:/home/omero/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
Environment:PYTHONPATH=(unset)
Environment:ICE_HOME=(unset)
Environment:LD_LIBRARY_PATH=(unset)
Environment:DYLD_LIBRARY_PATH=(unset)

OMERO SSL port:4064
OMERO TCP port:4063
OMERO data dir:'/tank/OMERO'                  Exists? True      Is writable? True
OMERO temp dir:'/home/omero/omero/tmp'        Exists? True      Is writable? True   (Size: 0)

JVM settings: Blitz-${index}                -Xmx7200m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Indexer-${index}              -Xmx4800m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: PixelData-${index}            -Xmx7200m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Repository-${index}           -Xmx4800m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions

**OMERO.py/bin/omero config get**
omero.web.application_server=wsgi-tcp
omero.web.apps=["omero_figure", "omero_iviewer", "omero_fpbioimage"]
omero.web.middleware=[{"index": 1, "class": "django.middleware.common.BrokenLinkEmailsMiddleware"}, {"index": 2, "class": "django.middleware.common.CommonMiddleware"}, {"index": 3, "class": "django.contrib.sessions.middleware.SessionMiddleware"}, {"index": 4, "class": "django.middleware.csrf.CsrfViewMiddleware"}, {"index": 5, "class": "django.contrib.messages.middleware.MessageMiddleware"}, {"index": 6, "class": "django.middleware.clickjacking.XFrameOptionsMiddleware"}, {"index": 0, "class": "whitenoise.middleware.WhiteNoiseMiddleware"}]
omero.web.open_with=[["Image viewer", "webgateway", {"supported_objects": ["image"], "script_url": "webclient/javascript/ome.openwith_viewer.js"}], ["omero_figure", "new_figure", {"supported_objects": ["images"], "target": "_blank", "label": "OMERO.figure"}], ["omero_iviewer", "omero_iviewer_index", {"supported_objects": ["images", "dataset", "well"], "script_url": "omero_iviewer/openwith.js", "label": "OMERO.iviewer"}], ["omero_fpbioimage", "fpbioimage_index", {"supported_objects": ["image"], "script_url": "fpbioimage/openwith.js", "label": "FPBioimage"}]]
omero.web.prefix=/omero
omero.web.static_url=/omero/static
omero.web.ui.top_links=[["Data", "webindex", {"title": "Browse Data via Projects, Tags etc"}], ["History", "history", {"title": "History"}], ["Help", "https://help.openmicroscopy.org/", {"target": "new", "title": "Open OMERO user guide in a new tab"}], ["Figure", "figure_index", {"target": "_blank", "title": "Open Figure in new tab"}]]
omero.web.viewer.view=omero_iviewer.views.index
omero.web.wsgi_timeout=240
omero.web.wsgi_workers=18

OMERO.py/bin/omero web disgnostics

================================================================================
OMERO Diagnostics (web) 5.5.1-ice36-b122
================================================================================

OMERO.web status... [RUNNING] (PID 7406)
Django version: 1.8.19

Log dir:    /home/omero/OMERO.py/var/log   exists
Log file :  OMEROweb.log                   10.0 MB       errors=18   warnings=1944

The OMEROweb.log contains lots of WARNINGS like this:

2019-08-28 18:31:47,045 WARNI [                          django.request] (proc.07420) get_response():170 HTTP 404 <WSGIRequest: GET '/omero/static/webgateway/js/ome.csrf.js?_5.5.1-ice36-b122'>
2019-08-28 18:31:47,045 WARNI [                 omeroweb.feedback.views] (proc.07420) handler404():198 Not Found: /omero/static/webgateway/js/ome.csrf.js
2019-08-28 18:31:47,156 WARNI [                          django.request] (proc.07420) get_response():170 Not Found: /omero/static/webclient/javascript/jquery.infieldlabel-0.1.js
2019-08-28 18:31:47,156 WARNI [                          django.request] (proc.07420) get_response():170 HTTP 404 <WSGIRequest: GET '/omero/static/webclient/javascript/jquery.infieldlabel-0.1.js'>
2019-08-28 18:31:47,157 WARNI [                 omeroweb.feedback.views] (proc.07420) handler404():198 Not Found: /omero/static/webclient/javascript/jquery.infieldlabel-0.1.js

@salim2203 Is SELinux enabled on your system? Assuming you followed https://docs.openmicroscopy.org/omero/5.5.1/sysadmins/unix/install-web/walkthrough/omeroweb-install-centos7-ice3.6.html#selinux you might need one more change: /usr/sbin/restorecon -R -v /path/to/omero/web.
This is specific to RedHat 7 (it isn’t needed on CentOS 7).

@Oftakofta can you show us your nginx error logs?

I have solved the issue by regenerating the omeroweb nginx configuration and overwriting /etc/nginx/conf.d/omeroweb.conf with the new configuration. Then i restarted the nginx service again and OMERO.web worked normally.

the nginx error.log before the config rewrite:

2019/08/29 09:12:28 [error] 8763#8763: *1821 connect() failed (111: Connection refused) while connecting to upstream, client: 130.238.46.211, server: localhost, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:4080/", host: "frankencloud.imbim.uu.se"
2019/08/29 09:12:37 [error] 8763#8763: *1821 connect() failed (111: Connection refused) while connecting to upstream, client: 130.238.46.211, server: localhost, request: "GET /omero HTTP/1.1", upstream: "http://127.0.0.1:4080/omero", host: "frankencloud.imbim.uu.se"
2019/08/29 09:12:49 [error] 8763#8763: *1821 connect() failed (111: Connection refused) while connecting to upstream, client: 130.238.46.211, server: localhost, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:4080/", host: "frankencloud.imbim.uu.se"

I honestly don’t know why it needed a config rewrite, but I’m happy that it works again.

You should always regenerate your OMERO.web nginx.conf and restart Nginx when you upgrade OMERO.