I recently installed CellProfiler in a linux cluster environment. After reading the documentation for batch processing I am running jobs that analyse a single well’s worth of images per job. The pipeline seems to run and it writes .csv files according to the settings of the ExportToSpreadsheet module, but I see many java Caught exceptions reported to standard output while it is running.
The java exceptions are of two types. The first are reported immediately after the cellprofiler command is executed and the second type are reported for each image that is analyzed. The text of these exceptions are listed below.
My question is will these exceptions alter the data that I will get from the analysis or can I safely ignore them?
Exceptions reported for each image:
Fri Apr 22 17:14:55 2016: Image # 3, module Images # 1: 0.00 sec Fri Apr 22 17:14:55 2016: Image # 3, module Metadata # 2: 0.00 sec Warning: org.apache.xerces.parsers.SAXParser: Property 'http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized. Warning: org.apache.xerces.parsers.SAXParser: Property 'http://www.oracle.com/xml/jaxp/properties/entityExpansionLimit' is not recognized. Exception in thread "Thread-0" java.io.EOFException: Attempting to read beyond end of file. at loci.common.NIOFileHandle.readFloat(NIOFileHandle.java:350) at loci.common.RandomAccessInputStream.readFloat(RandomAccessInputStream.java:501) at loci.formats.in.PQBinReader.isThisType(PQBinReader.java:96) Caused by: java.nio.BufferUnderflowException at java.nio.Buffer.nextGetIndex(Buffer.java:506) at java.nio.HeapByteBuffer.getFloat(HeapByteBuffer.java:463) at loci.common.NIOFileHandle.readFloat(NIOFileHandle.java:348) ... 2 more Caught exception in class loci.formats.in.PQBinReader.isThisType Exception in thread "Thread-0" java.lang.IllegalArgumentException at java.nio.Buffer.position(Buffer.java:244) at loci.common.NIOFileHandle.buffer(NIOFileHandle.java:603) at loci.common.NIOFileHandle.seek(NIOFileHandle.java:291) at loci.common.RandomAccessInputStream.seek(RandomAccessInputStream.java:174) at loci.formats.tiff.TiffParser.<init>(TiffParser.java:113) at loci.formats.in.FlowSightReader.isThisType(FlowSightReader.java:101) Caught exception in class loci.formats.in.FlowSightReader.isThisType Exception in thread "Thread-0" java.io.EOFException: Attempting to read beyond end of file. at loci.common.NIOFileHandle.readFloat(NIOFileHandle.java:350) at loci.common.RandomAccessInputStream.readFloat(RandomAccessInputStream.java:501) at loci.formats.in.PQBinReader.isThisType(PQBinReader.java:96) Caused by: java.nio.BufferUnderflowException at java.nio.Buffer.nextGetIndex(Buffer.java:506) at java.nio.HeapByteBuffer.getFloat(HeapByteBuffer.java:463) at loci.common.NIOFileHandle.readFloat(NIOFileHandle.java:348) ... 2 more Caught exception in class loci.formats.in.PQBinReader.isThisType Exception in thread "Thread-0" java.lang.IllegalArgumentException at java.nio.Buffer.position(Buffer.java:244) at loci.common.NIOFileHandle.buffer(NIOFileHandle.java:603) at loci.common.NIOFileHandle.seek(NIOFileHandle.java:291) at loci.common.RandomAccessInputStream.seek(RandomAccessInputStream.java:174) at loci.formats.tiff.TiffParser.<init>(TiffParser.java:113) at loci.formats.in.FlowSightReader.isThisType(FlowSightReader.java:101) Caught exception in class loci.formats.in.FlowSightReader.isThisType Exception in thread "Thread-0" java.io.EOFException: Attempting to read beyond end of file. at loci.common.NIOFileHandle.readFloat(NIOFileHandle.java:350) at loci.common.RandomAccessInputStream.readFloat(RandomAccessInputStream.java:501) at loci.formats.in.PQBinReader.isThisType(PQBinReader.java:96) Caused by: java.nio.BufferUnderflowException at java.nio.Buffer.nextGetIndex(Buffer.java:506) at java.nio.HeapByteBuffer.getFloat(HeapByteBuffer.java:463) at loci.common.NIOFileHandle.readFloat(NIOFileHandle.java:348) ... 2 more Caught exception in class loci.formats.in.PQBinReader.isThisType Exception in thread "Thread-0" java.lang.IllegalArgumentException at java.nio.Buffer.position(Buffer.java:244) at loci.common.NIOFileHandle.buffer(NIOFileHandle.java:603) at loci.common.NIOFileHandle.seek(NIOFileHandle.java:291) at loci.common.RandomAccessInputStream.seek(RandomAccessInputStream.java:174) at loci.formats.tiff.TiffParser.<init>(TiffParser.java:113) at loci.formats.in.FlowSightReader.isThisType(FlowSightReader.java:101) Caught exception in class loci.formats.in.FlowSightReader.isThisType Exception in thread "Thread-0" java.io.EOFException: Attempting to read beyond end of file. at loci.common.NIOFileHandle.readFloat(NIOFileHandle.java:350) at loci.common.RandomAccessInputStream.readFloat(RandomAccessInputStream.java:501) at loci.formats.in.PQBinReader.isThisType(PQBinReader.java:96) Caused by: java.nio.BufferUnderflowException at java.nio.Buffer.nextGetIndex(Buffer.java:506) at java.nio.HeapByteBuffer.getFloat(HeapByteBuffer.java:463) at loci.common.NIOFileHandle.readFloat(NIOFileHandle.java:348) ... 2 more Caught exception in class loci.formats.in.PQBinReader.isThisType Exception in thread "Thread-0" java.lang.IllegalArgumentException at java.nio.Buffer.position(Buffer.java:244) at loci.common.NIOFileHandle.buffer(NIOFileHandle.java:603) at loci.common.NIOFileHandle.seek(NIOFileHandle.java:291) at loci.common.RandomAccessInputStream.seek(RandomAccessInputStream.java:174) at loci.formats.tiff.TiffParser.<init>(TiffParser.java:113) at loci.formats.in.FlowSightReader.isThisType(FlowSightReader.java:101) Caught exception in class loci.formats.in.FlowSightReader.isThisType Exception in thread "Thread-0" java.io.EOFException: Attempting to read beyond end of file. at loci.common.NIOFileHandle.readFloat(NIOFileHandle.java:350) at loci.common.RandomAccessInputStream.readFloat(RandomAccessInputStream.java:501) at loci.formats.in.PQBinReader.isThisType(PQBinReader.java:96) Caused by: java.nio.BufferUnderflowException at java.nio.Buffer.nextGetIndex(Buffer.java:506) at java.nio.HeapByteBuffer.getFloat(HeapByteBuffer.java:463) at loci.common.NIOFileHandle.readFloat(NIOFileHandle.java:348) ... 2 more Caught exception in class loci.formats.in.PQBinReader.isThisType Exception in thread "Thread-0" java.lang.IllegalArgumentException at java.nio.Buffer.position(Buffer.java:244) at loci.common.NIOFileHandle.buffer(NIOFileHandle.java:603) at loci.common.NIOFileHandle.seek(NIOFileHandle.java:291) at loci.common.RandomAccessInputStream.seek(RandomAccessInputStream.java:174) at loci.formats.tiff.TiffParser.<init>(TiffParser.java:113) at loci.formats.in.FlowSightReader.isThisType(FlowSightReader.java:101) Caught exception in class loci.formats.in.FlowSightReader.isThisType Fri Apr 22 17:14:55 2016: Image # 3, module NamesAndTypes # 3: 3.19 sec Fri Apr 22 17:14:58 2016: Image # 3, module Groups # 4: 0.00 sec Fri Apr 22 17:14:58 2016: Image # 3, module IdentifyPrimaryObjects # 5: 14.08 sec Fri Apr 22 17:15:13 2016: Image # 3, module IdentifySecondaryObjects # 6: 7.32 sec Fri Apr 22 17:15:20 2016: Image # 3, module IdentifyTertiaryObjects # 7: 2.05 sec Fri Apr 22 17:15:22 2016: Image # 3, module MeasureObjectIntensity # 10: 26.76 sec Fri Apr 22 17:15:50 2016: Image # 3, module MeasureObjectSizeShape # 15: 132.56 sec Fri Apr 22 17:18:03 2016: Image # 3, module ExportToSpreadsheet # 17: 0.00 sec Fri Apr 22 17:18:03 2016: Image # 3, module CreateBatchFiles # 18: 0.00 sec
Exceptions reported immediately after executing cellprofiler command:
> cellprofiler -p /data/hd/analysis/rs/HD42m1/Batch_data.h5 -c -r -f 10 -l 18 -o /data/hd/analysis/rs/HD42m1/2
> Plugin directory doesn't point to valid folder: /home/plugins
> java.lang.IllegalArgumentException: Cannot handle replace call to list in ij.Menus's public static synchronized java.lang.String[] getPlugins()
> at net.imagej.patcher.CodeHacker.replaceCallInMethod(CodeHacker.java:725)
> at net.imagej.patcher.CodeHacker.replaceCallInMethod(CodeHacker.java:650)
> at net.imagej.patcher.LegacyExtensions.addExtraPlugins(LegacyExtensions.java:477)
> at net.imagej.patcher.LegacyExtensions.injectHooks(LegacyExtensions.java:175)
> at net.imagej.patcher.LegacyInjector.inject(LegacyInjector.java:308)
> at net.imagej.patcher.LegacyInjector.injectHooks(LegacyInjector.java:109)
> at net.imagej.patcher.LegacyEnvironment.initialize(LegacyEnvironment.java:101)
> at net.imagej.patcher.LegacyEnvironment.applyPatches(LegacyEnvironment.java:495)
> at net.imagej.patcher.LegacyInjector.preinit(LegacyInjector.java:397)
> at net.imagej.patcher.LegacyInjector.preinit(LegacyInjector.java:376)
> at net.imagej.patcher.JavaAgent.preinit(JavaAgent.java:147)
> at net.imagej.patcher.JavaAgent.premain(JavaAgent.java:95)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
> at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)
> Caused by: java.lang.RuntimeException: java.io.IOException: invalid constant type: 18
> at javassist.CtClassType.getClassFile2(CtClassType.java:203)
> at javassist.compiler.MemberResolver.lookupMethod(MemberResolver.java:110)
> at javassist.compiler.MemberResolver.lookupMethod(MemberResolver.java:96)
> at javassist.compiler.TypeChecker.atMethodCallCore(TypeChecker.java:704)
> at javassist.compiler.TypeChecker.atCallExpr(TypeChecker.java:681)
> at javassist.compiler.JvstTypeChecker.atCallExpr(JvstTypeChecker.java:156)
> at javassist.compiler.ast.CallExpr.accept(CallExpr.java:45)
> at javassist.compiler.CodeGen.doTypeCheck(CodeGen.java:241)
> at javassist.compiler.CodeGen.atStmnt(CodeGen.java:329)
> at javassist.compiler.ast.Stmnt.accept(Stmnt.java:49)
> at javassist.compiler.CodeGen.atIfStmnt(CodeGen.java:390)
> at javassist.compiler.CodeGen.atStmnt(CodeGen.java:354)
> at javassist.compiler.ast.Stmnt.accept(Stmnt.java:49)
> at javassist.compiler.Javac.compileStmnt(Javac.java:568)
> at javassist.expr.MethodCall.replace(MethodCall.java:234)
> at net.imagej.patcher.CodeHacker$7.edit(CodeHacker.java:686)
> at javassist.expr.ExprEditor.loopBody(ExprEditor.java:191)
> at javassist.expr.ExprEditor.doit(ExprEditor.java:90)
> at javassist.CtBehavior.instrument(CtBehavior.java:618)
> at net.imagej.patcher.CodeHacker$EagerExprEditor.instrument(CodeHacker.java:1278)
> at net.imagej.patcher.CodeHacker.replaceCallInMethod(CodeHacker.java:670)
> ... 17 more
> Caused by: java.io.IOException: invalid constant type: 18
> at javassist.bytecode.ConstPool.readOne(ConstPool.java:1027)
> at javassist.bytecode.ConstPool.read(ConstPool.java:970)
> at javassist.bytecode.ConstPool.<init>(ConstPool.java:127)
> at javassist.bytecode.ClassFile.read(ClassFile.java:693)
> at javassist.bytecode.ClassFile.<init>(ClassFile.java:85)
> at javassist.CtClassType.getClassFile2(CtClassType.java:190)
> ... 37 more
> java.lang.IllegalArgumentException: Cannot handle app name in ij.ImageJ's public <init>(java.applet.Applet applet, int mode)
> at net.imagej.patcher.CodeHacker.replaceAppNameInCall(CodeHacker.java:446)
> at net.imagej.patcher.LegacyExtensions.insertAppNameHooks(LegacyExtensions.java:406)
> at net.imagej.patcher.LegacyExtensions.injectHooks(LegacyExtensions.java:291)
> at net.imagej.patcher.LegacyInjector.inject(LegacyInjector.java:308)
> at net.imagej.patcher.LegacyInjector.injectHooks(LegacyInjector.java:109)
> at net.imagej.patcher.LegacyEnvironment.initialize(LegacyEnvironment.java:101)
> at net.imagej.patcher.LegacyEnvironment.applyPatches(LegacyEnvironment.java:495)
> at net.imagej.patcher.LegacyInjector.preinit(LegacyInjector.java:397)
> at net.imagej.patcher.LegacyInjector.preinit(LegacyInjector.java:376)
> at net.imagej.patcher.JavaAgent.preinit(JavaAgent.java:147)
> at net.imagej.patcher.JavaAgent.premain(JavaAgent.java:95)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
> at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)
> Caused by: java.lang.RuntimeException: java.io.IOException: invalid constant type: 18
> at javassist.CtClassType.getClassFile2(CtClassType.java:203)
> at javassist.CtClassType.subtypeOf(CtClassType.java:303)
> at javassist.CtClassType.subtypeOf(CtClassType.java:318)
> at javassist.compiler.MemberResolver.compareSignature(MemberResolver.java:247)
> at javassist.compiler.MemberResolver.lookupMethod(MemberResolver.java:119)
> at javassist.compiler.MemberResolver.lookupMethod(MemberResolver.java:96)
> at javassist.compiler.TypeChecker.atMethodCallCore(TypeChecker.java:704)
> at javassist.compiler.TypeChecker.atCallExpr(TypeChecker.java:681)
> at javassist.compiler.JvstTypeChecker.atCallExpr(JvstTypeChecker.java:156)
> at javassist.compiler.ast.CallExpr.accept(CallExpr.java:45)
> at javassist.compiler.CodeGen.doTypeCheck(CodeGen.java:241)
> at javassist.compiler.CodeGen.atStmnt(CodeGen.java:329)
> at javassist.compiler.ast.Stmnt.accept(Stmnt.java:49)
> at javassist.compiler.Javac.compileStmnt(Javac.java:568)
> at javassist.expr.MethodCall.replace(MethodCall.java:234)
> at net.imagej.patcher.CodeHacker$4.edit(CodeHacker.java:427)
> at net.imagej.patcher.CodeHacker$4.edit(CodeHacker.java:441)
> at javassist.expr.ExprEditor.loopBody(ExprEditor.java:219)
> at javassist.expr.ExprEditor.doit(ExprEditor.java:90)
> at javassist.CtBehavior.instrument(CtBehavior.java:618)
> at net.imagej.patcher.CodeHacker$EagerExprEditor.instrument(CodeHacker.java:1278)
> at net.imagej.patcher.CodeHacker.replaceAppNameInCall(CodeHacker.java:402)
> ... 16 more
> Caused by: java.io.IOException: invalid constant type: 18
> at javassist.bytecode.ConstPool.readOne(ConstPool.java:1027)
> at javassist.bytecode.ConstPool.read(ConstPool.java:970)
> at javassist.bytecode.ConstPool.<init>(ConstPool.java:127)
> at javassist.bytecode.ClassFile.read(ClassFile.java:693)
> at javassist.bytecode.ClassFile.<init>(ClassFile.java:85)
> at javassist.CtClassType.getClassFile2(CtClassType.java:190)
> ... 37 more
> Version: 2016-04-22T21:02:41 Unknown rev. / 20160422210241
> Failed to stop Ilastik
> Pipeline saved with CellProfiler version 20140723173957
> vigra import: failed to import the vigra library. Please follow the instructions on
> "http://hci.iwr.uni-heidelberg.de/vigra/" to install vigra
> Traceback (most recent call last):
> File "/home/anaconda2/lib/python2.7/site-packages/cellprofiler/modules/classifypixels.py", line 50, in <module>
> import vigra
> ImportError: No module named vigra
> vigra import: failed to import the vigra library. Please follow the instructions on
> "http://hci.iwr.uni-heidelberg.de/vigra/" to install vigra
> Traceback (most recent call last):
> File "/home/anaconda2/lib/python2.7/site-packages/cellprofiler/modules/ilastik_pixel_classification.py", line 47, in <module>
> import vigra
> ImportError: No module named vigra
> Could not load cellprofiler.modules.exporttocellh5
> Traceback (most recent call last):
> File "/home/anaconda2/lib/python2.7/site-packages/cellprofiler/modules/__init__.py", line 294, in add_module
> m = __import__(mod, globals(), locals(), ['__all__'], 0)
> File "/home/anaconda2/lib/python2.7/site-packages/cellprofiler/modules/exporttocellh5.py", line 29, in <module>
> import cellh5
> File "/home/anaconda2/lib/python2.7/site-packages/cellh5/__init__.py", line 12, in <module>
> from cellh5 import *
> File "/home/anaconda2/lib/python2.7/site-packages/cellh5/cellh5.py", line 18, in <module>
> import matplotlib.pyplot as plt
> File "/home/anaconda2/lib/python2.7/site-packages/matplotlib/pyplot.py", line 114, in <module>
> _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
> File "/home/anaconda2/lib/python2.7/site-packages/matplotlib/backends/__init__.py", line 32, in pylab_setup
> globals(),locals(),[backend_name],0)
> File "/home/anaconda2/lib/python2.7/site-packages/matplotlib/backends/backend_wxagg.py", line 11, in <module>
> from . import wx_compat as wxc
> File "/home/anaconda2/lib/python2.7/site-packages/matplotlib/backends/wx_compat.py", line 24, in <module>
> raise ImportError(missingwx)
> ImportError: Matplotlib backend_wx and backend_wxagg require wxPython >=2.8.12
> could not load these modules: cellprofiler.modules.exporttocellh5
> Times reported are CPU times for each module, not wall-clock time