How to install OMERO.web add-on apps with Docker

I’ve followed the instructions here to install OMERO.server and OMERO.web with Docker, but am now realizing that some functionality is missing (e.g., figure export). Is it possible to amend my Docker files to include the web add-ons, like as was done here with Ansible, or will I have to install them manually with Pip? Are either Docker or Ansible the preferred “easy” way to install and manage OMERO, or are both relatively equivalent in that regard? I have minimal experience with Docker and none with Ansible.

Hi Jonathan - this discussion is related:

In particlular see @manics answer and pointers to the doc:

1 Like

Sorry, we’ve gone back and forth on including figure in the base containers (e.g. Auto import, figure, wheel, etc. by joshmoore · Pull Request #21 · ome/omero-server-docker · GitHub) versus making the dockers extendable (e.g. Add infrastructure for a server extras image by manics · Pull Request #39 · ome/omero-server-docker · GitHub). The more we add to the dockers the more they have to be updated, etc. Feedback is certainly welcome. Note that here the export functionality is needed in the server docker rather than the web docker making this all the weirder.

Yes. omero-server-docker also uses ansible internally, so you could add another playbook to your server/Dockerfile and then run it with ansible-playbook.

Installing them manually in the server/Dockerfile is certainly also an option.

Not sure how to gauge which is “easiest”. That likely depends on your familiarity, and perhaps also with what infrastructure you have. E.g. if you have bare metal then likely using ansible directly is easiest. If someone has given you a managed docker host, then docker will likely be easier.

We use ansible for most of our production systems, but we are looking to move some of them to docker.

We certainly try to keep them functionally equivalent.
~Josh

Thanks, Guillaume! I appreciate the help.

Thanks for the detailed response, Josh. I can understand with such a big project that it would be difficult to keep things flexible and lightweight while maintaining full functionality. I would love to see a “batteries included” Docker/Ansible setup that includes the figure add-ons. While the OMERO documentation is extensive, it can be a bit daunting, at times, and figuring out the best or easiest way to do something is not always obvious. Along those same lines, it would be fantastic to have some “best practices” for things like setting up user groups and data annotation, particularly with some examples of standard groups/tags/key value pairs that many will find useful.

1 Like

Understood. Until there’s a full product, you could take a look at our production playbooks, e.g. prod-playbooks/playbook.yml at 79ea70cda08e3187803606601b28964648dda2a2 · ome/prod-playbooks · GitHub, where figure is installed.

~Josh

Thanks, I didn’t realize that there were production playbooks available. I think I will redo my test server with Ansible, since it seems like the most detailed examples are done that way.

In trying to use the ansible-addons configuration, since it seems to at least have the figure export enabled, but I seem to be getting an error.

[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details
ERROR! couldn't resolve module/action 'postgresql_user'. This often indicates a misspelling, missing collection, or incorrect module path.

The error appears to be in '/home/ducatlab/.ansible/roles/ome.postgresql/tasks/databases.yml': line 22, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:


- name: postgres | create users
  ^ here

I set /etc/ansible/hosts with

[omero-addons]
127.0.0.1

and installed ome-molecule via pip.

The only change I made to the playbook file was to change the default password. Is there some prerequisite step I missed?

What version of Ansible are you using (ansible --version)?

2.6.18. There seems to be something wrong with my installation, I think.

ansible 2.6.18
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/ducatlab/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ducatlab/.local/lib/python3.6/site-packages/ansible
  executable location = /home/ducatlab/.local/bin/ansible
  python version = 3.6.9 (default, Oct  8 2020, 12:12:24) [GCC 8.4.0]
Traceback (most recent call last):
  File "/home/ducatlab/.local/bin/ansible", line 117, in <module>
    cli.parse()
  File "/home/ducatlab/.local/lib/python3.6/site-packages/ansible/cli/adhoc.py", line 76, in parse
    super(AdHocCLI, self).parse()
  File "/home/ducatlab/.local/lib/python3.6/site-packages/ansible/cli/__init__.py", line 585, in parse
    self.options, self.args = self.parser.parse_args(self.args[1:])
  File "/usr/lib/python3.6/optparse.py", line 1387, in parse_args
    stop = self._process_args(largs, rargs, values)
  File "/usr/lib/python3.6/optparse.py", line 1427, in _process_args
    self._process_long_opt(rargs, values)
  File "/usr/lib/python3.6/optparse.py", line 1501, in _process_long_opt
    option.process(opt, value, values, self)
  File "/usr/lib/python3.6/optparse.py", line 785, in process
    self.action, self.dest, opt, value, values, parser)
  File "/usr/lib/python3.6/optparse.py", line 811, in take_action
    parser.exit()
  File "/usr/lib/python3.6/optparse.py", line 1559, in exit
    sys.exit(status)
SystemExit: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ducatlab/.local/bin/ansible", line 161, in <module>
    shutil.rmtree(C.DEFAULT_LOCAL_TMP, True)
AttributeError: module 'ansible.constants' has no attribute 'DEFAULT_LOCAL_TMP'

This is on a bare metal system with Ubuntu 18.04.

Ansible 2.6 should be fine, that’s the version used for the repository CI tests that validate the addons playbook works.

Maybe try creating a new virtualenv just for Ansible?

Simon,

I created a virtual environment, reinstalled ansible and ome-ansible-molecule. I get no errors after running the ansible-galaxy install -r requirements.yml command.

I edited inventory.yml with either 127.0.0.1 or the IP of the host, but I get the same warning with both:

[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

Is there another configuration step I missed?

Edit: I was able to run the playbook by editing /etc/ansible/ansible.cfg with

[defaults]
transport = local

Could you show us the full ansible-playbook command that you’re running?

What does

ansible-playbook -i inventory.yml --list-hosts <playbook>.yml

show?

Simon,

I think something is wrong with my Ansible installation. I had previously installed Ansible with apt, but uninstalled and then reinstalled in a virtual environment as you suggested. Below is the output of the playbook command.

(ome) root@PRL-DUCAT-S1:/mnt/raid1/ansible-example-omero-addons# ansible-playbook -i inventory.yml --list-hosts playbook.yml
/opt/ansible/ome/lib/python2.7/site-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.exceptions import InvalidSignature

playbook: playbook.yml

  play #1 (omero_addons): omero_addons  TAGS: []
    pattern: [u'omero_addons']
    hosts (1):
      omero_addons

But now if I try running the playbook, it fails

PLAYBOOK: playbook.yml **********************************************************************************************************************************************************************
1 plays in playbook.yml

PLAY [omero_addons] *************************************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************************************
task path: /mnt/raid1/ansible-example-omero-addons/playbook.yml:5
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1610649699.45-186063794343094 `" && echo ansible-tmp-1610649699.45-186063794343094="` echo /root/.ansible/tmp/ansible-tmp-1610649699.45-186063794343094 `" ) && sleep 0'
Using module file /opt/ansible/ome/lib/python2.7/site-packages/ansible/modules/setup.py
<127.0.0.1> PUT /root/.ansible/tmp/ansible-local-3891D8TwPt/tmpIIXsl8 TO /root/.ansible/tmp/ansible-tmp-1610649699.45-186063794343094/setup.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1610649699.45-186063794343094/ /root/.ansible/tmp/ansible-tmp-1610649699.45-186063794343094/setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1610649699.45-186063794343094/setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1610649699.45-186063794343094/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_boDAj_/ansible_module_setup.py", line 130, in <module>
    from ..module_utils.basic import AnsibleModule
ValueError: Attempted relative import in non-package

fatal: [127.0.0.1]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_boDAj_/ansible_module_setup.py\", line 130, in <module>\n    from ..module_utils.basic import AnsibleModule\nValueError: Attempted relative import in non-package\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 1
}
        to retry, use: --limit @/mnt/raid1/ansible-example-omero-addons/playbook.retry

PLAY RECAP **********************************************************************************************************************************************************************************
127.0.0.1                  : ok=0    changed=0    unreachable=0    failed=1

I’m not very familiar with Ansible, so I apologize if I’ve missed something obvious.

Ah, ok. This is looking very much like Python 2 troubles. Do you have any options for upgrading to Python 3?

~Josh

Josh,

I think you were right about Python 2 being an issue, but it doesn’t seem to solve my problem.

TASK [Gathering Facts] ******************************************************************************************************************************************************************
task path: /mnt/raid1/ansible-example-omero-addons/playbook.yml:5
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: ducatlab
<localhost> EXEC /bin/sh -c 'echo ~ducatlab && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/ducatlab/.ansible/tmp/ansible-tmp-1610738482.7650425-69105106433365 `" && echo ansible-tmp-1610738482.7650425-69105106433365="` echo /home/ducatlab/.ansible/tmp/ansible-tmp-1610738482.7650425-69105106433365 `" ) && sleep 0'
Using module file /home/ducatlab/ansible/omero/lib/python3.6/site-packages/ansible/modules/setup.py
<localhost> PUT /home/ducatlab/.ansible/tmp/ansible-local-29678rxvpepb0/tmp67bm54qp TO /home/ducatlab/.ansible/tmp/ansible-tmp-1610738482.7650425-69105106433365/setup.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/ducatlab/.ansible/tmp/ansible-tmp-1610738482.7650425-69105106433365/ /home/ducatlab/.ansible/tmp/ansible-tmp-1610738482.7650425-69105106433365/setup.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /home/ducatlab/.ansible/tmp/ansible-tmp-1610738482.7650425-69105106433365/setup.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/ducatlab/.ansible/tmp/ansible-tmp-1610738482.7650425-69105106433365/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_7xIn1T/ansible_module_setup.py", line 130, in <module>
    from ..module_utils.basic import AnsibleModule
ValueError: Attempted relative import in non-package

fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_7xIn1T/ansible_module_setup.py\", line 130, in <module>\n    from ..module_utils.basic import AnsibleModule\nValueError: Attempted relative import in non-package\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 1
}

It still fails at ansible_module setup.py. I was previously running everything as root, but I get the same result as a user. I’ve tried uninstalling and reinstalling the ansible galaxy requirements, ansible, ome-ansible-molecule, but the error is the same.