CP attempting to access files not part of the batch file

Hi,

When running a job that was part of a larger batch file I was presented with a really strange error whereby some of the CP jobs are trying to access parts of the filesystem that are not in any way associated with the input for the batch file. The input selected is everything below the default input folder:

/home/morphology/Morphology/Images/Screens_Other/10_NOP10GH_HTA2mCN_can1RPL39prtdTU=HarvardCollaboration=freezerstock/NonEssential/FlexFiles

Yet for at least one of these jobs, CP was trying to search for flex files in a completely different part of the filesystem:

Exception in thread "Thread-0" loci.formats.UnknownFormatException: Unknown file format: /home/lee/lipid_droplets/code/.svn/text-base/rename_flex.py.svn-base at loci.formats.ImageReader.getReader(ImageReader.java:172) at loci.formats.ImageReader.setId(ImageReader.java:670) ERROR:root:Failed to prepare run for module LoadImages

How CP was trying to search in /home/lee/lipid_droplets/… is beyond me. I’ve attached the full error message as well as the .cp file that was used to generate the batch file for this set of jobs. I really don’t quite know where to start debugging this issue, so any help you can provide is appreciated.

Thanks,

Lee.
2011_08_18_10415_IPANuclei_IPANucleoli_ISACells_RelateNucleoliToNuclei_SaveImages=WORKS_WithMeasurementsCondensed_ExportToDB_CreateBF.cp (19.6 KB)
strange_cp_pipeline_error.cp (2.82 KB)

Is it possible that the default input folder isn’t “/home/morphology/Morphology/Images/Screens_Other/10_NOP10GH_HTA2mCN_can1RPL39prtdTU=HarvardCollaboration=freezerstock/NonEssential/FlexFiles”. I would make sure to use the -i switch when you start CellProfiler to set the default input folder. It looks like your LoadImages root folder is the default input folder. You could argue that LoadImages shouldn’t use the default input folder when you select some directories instead of all, unfortunately, it doesn’t and insists on looking for the subfolders you check in the new default input folder if you change default input folders.

–Lee

Hi Lee,

I guess it is possible. In any event, I’ve regenerated the batch file, which I learned (thanks to Mark Bray http://forum.image.sc/t/createbatchfiles-makes-two-files/14451/1) is not the default output .h5 file produced from a pipeline terminated with the CreateBatchFiles module, and am now analyzing all directories beneath the default input folder. While some jobs are still running, there is .tiff output being produced in the default output folder, and I haven’t seen any errors like those in my head post yet.

[quote=“LeeKamentsky”]Is it possible that the default input folder isn’t “/home/morphology/Morphology/Images/Screens_Other/10_NOP10GH_HTA2mCN_can1RPL39prtdTU=HarvardCollaboration=freezerstock/NonEssential/FlexFiles”. I would make sure to use the -i switch when you start CellProfiler to set the default input folder. It looks like your LoadImages root folder is the default input folder. You could argue that LoadImages shouldn’t use the default input folder when you select some directories instead of all, unfortunately, it doesn’t and insists on looking for the subfolders you check in the new default input folder if you change default input folders.

–Lee[/quote]