Cellprofiler 4.0

Hello,

I am able to successfully run cellprofiler in headless mode from the terminal on my Mac, but unable to do so from my Amazon Linux EC2 instance. Please find below the sequence of steps I followed on my Mac and Linux platforms along with an attached pipeline file and sample images to reproduce the results.

Mac:
(base) j.venkataraman: conda create --name py38 python=3.8
(base) j.venkataraman: conda activate py38
(py38) j.venkataraman: pip install numpy mysql cellprofiler
(py38) j.venkataraman: java -version
openjdk version “14.0.2” 2020-07-14
OpenJDK Runtime Environment (build 14.0.2+12-46)
OpenJDK 64-Bit Server VM (build 14.0.2+12-46, mixed mode, sharing)
(py38) j.venkataraman:~ $ cellprofiler -c -r -p sample_4p0p6_color_with_mask.cppipe -i cell_mask_tiles/ -o results

This works as expected

Amazon Linux (EC2 instance)
(base) ec2-user: conda create --name py38 python=3.8
(base) ec2-user: conda activate py38
(py38) ec2-user: pip install numpy mysql cellprofiler

This errors out with the following error for wxPython

**** Could not run GTK+ test program, checking why…*

  • *** The test program failed to compile or link. See the file config.log for the*

  • *** exact error that occurred. This usually means GTK+ is incorrectly installed.*

  • configure: error:*

  • The development files for GTK+ were not found. For GTK+ 2, please*

  • ensure that pkg-config is in the path and that gtk±2.0.pc is*

  • installed. For GTK+ 1.2 please check that gtk-config is in the path,*

  • and that the version is 1.2.3 or above. Also check that the*

  • libraries returned by ‘pkg-config gtk±2.0 --libs’ or 'gtk-config*

  • –libs’ are in the LD_LIBRARY_PATH or equivalent.*

  • Error running configure*

  • ERROR: failed building wxWidgets*

  • Traceback (most recent call last):*

  • File “build.py”, line 1471, in cmd_build_wx*

  •  wxbuild.main(wxDir(), build_options)*
    
  • File “/tmp/pip-install-bzvphj97/wxpython/buildtools/build_wxwidgets.py”, line 372, in main*

  •  exitIfError(wxBuilder.configure(dir=wxRootDir, options=configure_opts),*
    
  • File “/tmp/pip-install-bzvphj97/wxpython/buildtools/build_wxwidgets.py”, line 85, in exitIfError*

  •  raise builder.BuildError(msg)*
    
  • buildtools.builder.BuildError: Error running configure*

  • Finished command: build_wx (0m4.602s)*

  • Finished command: build (0m4.602s)*

  • Command ‘"/home/ec2-user/miniconda3/envs/py38/bin/python" -u build.py build’ failed with exit code 1.*

  • ----------------------------------------*

  • ERROR: Failed building wheel for wxPython*

I am able to work around that this way after searching some online forums:

(base) ec2-user: conda create --name py38 python=3.8
(base) ec2-user: conda activate py38
(py38) ec2-user: pip install numpy mysql
(py38) ec2-user: pip install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython
(py38) ec2-user: pip install cellprofiler

This works and the installation goes through. However, when I try to run the pipeline, it crashes with the following error.

(py38) ec2-user: java -version
openjdk version “14.0.1” 2020-04-14
OpenJDK Runtime Environment (build 14.0.1+7)
OpenJDK 64-Bit Server VM (build 14.0.1+7, mixed mode, sharing)

(py38) ec2-user: cellprofiler -c -r -p sample_4p0p6_color_with_mask.cppipe -i cell_mask_tiles/ -o results

#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fe08a09e570, pid=42079, tid=42167
#
# JRE version: OpenJDK Runtime Environment (11.0.7+10) (build 11.0.7+10-LTS)
# Java VM: OpenJDK 64-Bit Server VM (11.0.7+10-LTS, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# C [ld-linux-x86-64.so.2+0xb570] _dl_relocate_object+0x320
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try “ulimit -c unlimited” before starting Java again
#
# An error report file with more information is saved as:
# /home/ec2-user/scratch/cellprofiler/hs_err_pid42079.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
Aborted

I have tried different JDK versions, but to no avail. Any help will be appreciated. The pipeline and sample images are attached for your reference. Copy the images to a folder called cell_mask_tiles and run the pipeline as shown above.

Thanks
Jagadish
sample_4p0p6_color_with_mask.cppipe (9.5 KB)

Do you have JAVA_HOME set?

Yes, JAVA_HOME is set as follows. Earlier it was pointing to the JDK11, I switched it over to JDK 14.0.2, but still get the same error.

(py38) ec2-user: echo $JAVA_HOME
/usr/java/jdk-14.0.2/
(py38_2) ec2-user: java -version
java version “14.0.2” 2020-07-14
Java™ SE Runtime Environment (build 14.0.2+12-46)
Java HotSpot™ 64-Bit Server VM (build 14.0.2+12-46, mixed mode, sharing)
(py38) ec2-user: cellprofiler -c -r -p sample_4p0p6_color_with_mask.cppipe -i cell_mask_tiles/ -o .

# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f2fccf1f560, pid=30726, tid=30763
#
# JRE version: Java™ SE Runtime Environment (14.0.2+12) (build 14.0.2+12-46)
# Java VM: Java HotSpot™ 64-Bit Server VM (14.0.2+12-46, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# C [ld-linux-x86-64.so.2+0xb560] _dl_relocate_object+0x320
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try “ulimit -c unlimited” before starting Java again
#
# An error report file with more information is saved as:
# /home/ec2-user/scratch/cellprofiler/hs_err_pid30726.log
#
# If you would like to submit a bug report, please visit:
# https://bugreport.java.com/bugreport/crash.jsp
#
Aborted

Hi @bcimini

An Update!

Since I did not need the GUI version of cellprofiler on linux, I uninstalled wxPython as follows after cellprofiler installed and now then the headless run seems to work!

(base) ec2-user: conda create --name py38 python=3.8
(base) ec2-user: conda activate py38
(py38) ec2-user: pip install numpy mysql
(py38) ec2-user: pip install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython
(py38) ec2-user: pip install cellprofiler
(py38) ec2-user: pip uninstall wxPython

The whl files for wxPython are somehow not compatible with the JDK versions available for Amazon Linux, but thankfully I don’t need it.

Thanks
Jagadish