Problems using png's from different sources

Hi

I am using Cellprofiler under Linux and had the problem that using my Cellprofiler results (SQlite) in CPanalyst I could not see the objects in the Classifier.
I first thought that this only happens because of operating system issues. But by using Windows in a Virtualbox environment I found out that there are problems in the png-handling. I used a sample image from the cpa_example.zip and with a minimal Cellprofiler-script and analyzed it without problems in CPanalyst.
But after opening this “good” image in IrfanView and doing a “Save as” with default saving parameters this procedure ended also in non-visible objects.

Then I got a hint using the “View full image of selected” with following error:

An error occurred in the program:
ValueError: not a valid TIFF file

Traceback (most recent call last):
File “imagetile.pyc”, line 104, in OnSelectFromPopupMenu
File “imagetools.pyc”, line 61, in ShowImage
File “imagetools.pyc”, line 52, in FetchImage
File “imagereader.pyc”, line 27, in ReadImages
File “imagereader.pyc”, line 82, in read_images_old_way
File “imagereader.pyc”, line 128, in ReadBitmap
File “imagereader.pyc”, line 246, in ReadBitmapViaTIFFfile
File “tifffile.pyc”, line 143, in init
File “tifffile.pyc”, line 162, in _fromfile

So this suggests that the routines do not recognize the resaved image as an png but as a tiff. I tried working with tiffs alone and this is no problem.
It would be great to get rid of this problem because I don’t want to double my images by copying to tiffs. Therefore I attach the two identical images with just the invisible difference in file saving.

Thank you
Erwin

PS: I would be grateful if someone finds a solution for the drag and drop problem in CPanalyst with Linux




Are you running CPA from source or from a dist. In either case, which version?

I suspect you’re running from source and that you don’t have the CellProfiler source in your python path. This is required if you wish to use the bioformats image loading library. Otherwise CPA falls back to that “load_images_old_way” method, and if THAT fails, it falls back again to a tiff reader.

quick fix might be:
checkout CellProfiler to your home directory

make sure it worked by running python and importing cellprofiler

[code]$ python

import cellprofiler
[/code]

Hi

I am using the most recent exe-versions from the download pages (Cellprofiler: r11243, CPanalyst: r11246) for windows and the svn version for Linux.

The $PYTHONPATH variable doesnt help. The outcome stays the same. As I described above the problem can be traced back to different versions of PNGs. There is no difference using the windows-exe or the source version (with and without path variable).

I hope you can repeat my results if you carry out a minimal CellProfiler-script with one and then with the other png and compare what you get with CPanalyst.

My minimal script included:
LoadImages
IdentifyPrimaryObjects
MeasureObjectIntensity
ExportToDatabase

Thanks
Erwin

Hi Erwin,
Can you send me the full output generated by CPA when you start it up on Linux?

Hi

This is the whole output from starting CPanalyst, opening properties, starting classifier, fetching 4 random examples and exiting. Doing the same operations with the original image gives the same output except the part with the exception.

Thank you again for your immediate response!
Erwin

[code]DEBUG:bioformats:JVM arguments: -Djava.class.path=/home/gaubitzer/svn/CellProfiler/bioformats/loci_tools.jar:/home/gaubitzer/svn/CellProfiler/imagej/ij.jar:/home/gaubitzer/svn/CellProfiler/imagej/imglib.jar:/home/gaubitzer/svn/CellProfiler/imagej/javacl-1.0-beta-4-shaded.jar -Dloci.bioformats.loaded=true -Xmx512m -Dplugins.dir=/home/zwimi/Onkotec/Patient_PNG/Rudolfstiftung/20110329/PNG/Cellprofiler_Debugging/plugins
DEBUG:bioformats:Java virtual machine started.
DEBUG:root:ImageReader will use LoadImages from CellProfiler.
WARNING:root:PROPERTIES WARNING (channels_per_image): No value(s) specified. CPA will assume 1 channel per image.
INFO:root:PROPERTIES: Using default image_buffer_size=1
INFO:root:PROPERTIES: Using default tile_buffer_size=1
WARNING:root:PROPERTIES WARNING (plate_id): Field is required for plate map viewer.
WARNING:root:PROPERTIES WARNING (well_id): Field is required for plate map viewer.
WARNING:root:PROPERTIES WARNING (plate_type): Field is required for plate map viewer.
WARNING:root:PROPERTIES WARNING (well_format): Field was not defined, using default format of “A01”.
INFO:root:Initialized New Fast Gentle Boosting Classifier
WARNING:root:Skipping table checking step for sqlite
INFO:root:[MainThread] Connecting to the database…
INFO:root:[MainThread] SQLite file: /home/gaubitzer/Onkotec/Data/Patient_PNG/Rudolfstiftung/20110329/PNG/Cellprofiler_Debugging/DefaultDB.db
DEBUG:root:[MainThread] SELECT ImageNumber FROM AScropped_Per_Image GROUP BY ImageNumber
DEBUG:root:[MainThread] Connected to database: /home/gaubitzer/Onkotec/Data/Patient_PNG/Rudolfstiftung/20110329/PNG/Cellprofiler_Debugging/DefaultDB.db
DEBUG:root:[MainThread] SELECT ImageNumber FROM AScropped_Per_Image GROUP BY ImageNumber
DEBUG:root:[MainThread] SELECT AScropped_Per_Object.ImageNumber, COUNT(AScropped_Per_Object.ObjectNumber) FROM AScropped_Per_Object GROUP BY AScropped_Per_Object.ImageNumber
DEBUG:root:[MainThread] SELECT AScropped_Per_Image.ImageNumber FROM AScropped_Per_Image
DEBUG:root:[MainThread] SELECT ObjectNumber FROM AScropped_Per_Object WHERE ImageNumber=1 LIMIT 3,1
DEBUG:root:[MainThread] SELECT ObjectNumber FROM AScropped_Per_Object WHERE ImageNumber=1 LIMIT 10,1
DEBUG:root:[MainThread] SELECT ObjectNumber FROM AScropped_Per_Object WHERE ImageNumber=1 LIMIT 13,1
DEBUG:root:[MainThread] SELECT ObjectNumber FROM AScropped_Per_Object WHERE ImageNumber=1 LIMIT 21,1
INFO:root:[TileLoader_Thread-3] Connecting to the database…
INFO:root:[TileLoader_Thread-3] SQLite file: /home/gaubitzer/Onkotec/Data/Patient_PNG/Rudolfstiftung/20110329/PNG/Cellprofiler_Debugging/DefaultDB.db
DEBUG:root:[TileLoader_Thread-3] SELECT ImageNumber FROM AScropped_Per_Image GROUP BY ImageNumber
DEBUG:root:[TileLoader_Thread-3] Connected to database: /home/gaubitzer/Onkotec/Data/Patient_PNG/Rudolfstiftung/20110329/PNG/Cellprofiler_Debugging/DefaultDB.db
DEBUG:root:[TileLoader_Thread-3] SELECT Membranes_Location_Center_X, Membranes_Location_Center_Y FROM AScropped_Per_Object WHERE (ImageNumber=1 AND ObjectNumber=4)
DEBUG:root:[TileLoader_Thread-3] SELECT Image_PathName_FullImageGreen, Image_FileName_FullImageGreen FROM AScropped_Per_Image WHERE ImageNumber IN (1)
INFO:root:Opening image: /home/gaubitzer/Onkotec/Data/Patient_PNG/Rudolfstiftung/20110329/PNG/Cellprofiler_Debugging/AS_savedagain.png
INFO:root:Fetched 4 random cells from whole experiment
Exception in thread TileLoader_Thread-3:
Traceback (most recent call last):
File “/usr/lib/python2.6/threading.py”, line 532, in __bootstrap_inner
self.run()
File “/home/gaubitzer/svn/CPAnalyst/src/tilecollection.py”, line 129, in run
new_data = imagetools.FetchTile(obKey)
File “/home/gaubitzer/svn/CPAnalyst/src/imagetools.py”, line 39, in FetchTile
imgs = FetchImage(imKey)
File “/home/gaubitzer/svn/CPAnalyst/src/imagetools.py”, line 52, in FetchImage
imgs = ir.ReadImages(filenames)
File “/home/gaubitzer/svn/CPAnalyst/src/imagereader.py”, line 27, in ReadImages
return self.read_images_old_way(fds)
File “/home/gaubitzer/svn/CPAnalyst/src/imagereader.py”, line 82, in read_images_old_way
channels += self.ReadBitmap(fd)
File “/home/gaubitzer/svn/CPAnalyst/src/imagereader.py”, line 128, in ReadBitmap
imdata = ReadBitmapViaTIFFfile(data)
File “/home/gaubitzer/svn/CPAnalyst/src/imagereader.py”, line 246, in ReadBitmapViaTIFFfile
im = tifffile.TIFFfile(StringIO(data))
File “/home/gaubitzer/svn/CPAnalyst/src/tifffile.py”, line 143, in init
self._fromfile()
File “/home/gaubitzer/svn/CPAnalyst/src/tifffile.py”, line 162, in _fromfile
raise ValueError(“not a valid TIFF file”)
ValueError: not a valid TIFF file

DEBUG:root:Destroying: <main.MainGUI; proxy of <Swig Object of type ‘wxFrame *’ at 0x4773c10> >
DEBUG:root:Destroying: <wx._windows.TopLevelWindow; proxy of <Swig Object of type ‘wxTopLevelWindow *’ at 0x4775530> >
[/code]

Hi again

I looked into the code and found that in imagereader.py lines 6-13 is a check for Cellprofiler LoadImages and the variable is set to true, but in line 15 this variable is set to false again without any comment.

[code]try:
from cellprofiler.modules.loadimages import LoadImagesImageProvider
use_cp_loadimages = True
logging.debug(‘ImageReader will use LoadImages from CellProfiler.’)
except:
use_cp_loadimages = False
logging.warn('ImageReader failed to import LoadImagesImageProvider from '
‘CellProfiler, will fall back on PIL and TiffFile.’)

use_cp_loadimages = False
[/code]

I removed line 15 and fetching random cells works. Please let me know what purpose this overrule had.

Maybe I can also help tracing back the drag and drop problem so I will be able to use CPanalyst without having to use Windows in a Virtualbox. Give me some hints where and what to look for!

Erwin

Ugh. This is my fault. Looks like I was debugging the fallback image reader code and accidentally committed that switch. Looks like I’ll be re-releasing.

I appreciate your offer to help debug DnD in Linux. The relevant bits of code are:

[size=150]imagetile.py[/size]
ImageTile.OnMotion
Drag events start and end here. First place you should look for problems. Note the call to CursorFromImage was removed to fix DnD in a preivous version of CPA.
ImageTileDropTarget
this class allows Tiles to accept drop events… you can probably ignore it and focus on the sortbin drop target

[size=150]sortbin.py[/size]
SortBinDropTarget
Odds are the program is crashing before OnData even gets called.
ReceiveDrop
Called by the DropTarget class methods with the source bin id and the objects to move.

Thanks!