Error in classify pixel of HighRes image

Usage & Issues
#1

I’m running CP2 r11710 on a Windows 7 Enterprise computer with 48GB RAM. After using ilastik 05.06-rc5 to define the classifier I use it in a CP pipeline(first step) While this works on low resolution images, somewhere between 3000x3000 and 8300x3000 pixels it stops working (error text below). I can investigate precisely where the border lies if it helps. The File is a LZW compressed TIF (No layers) I’ve tried both grayscale and RGB images -neither works. The errors are usually similar though they sometimes vary slightly. While I’m not sure how to guarantee it’s not a RAM error, it never seems to approach 50% used. I’m sure there is more useful information I can provide here so please let me know how I can help (I’m guessing contacting the ilastik team may be the key here but I figured since CP is spitting out the error it should be my first stop.
I love how it works on the small images but I really need it working full size before I can apply it so I’m very keen on doing anything I can to help debug this!

The error I usually get is:
################## Exception in FeatureThread ##################

Precondition violation!
convolveLine<>: kernel longer than line

None
################## Exception in FeatureThread ##################
error return without exception set
Traceback (most recent call last):
File C:/…ilastik/modules/classification\core\featureMgr.py line 208 in calcFeature
… dataMgr.py line 142 in getBlockBounds
SystemError: error return without exception set

Though it often also includes:
Fatal Python error: coval: tstate mix-up

and sometimes:
Fatal Python Error: GC object already tracked

This application has requested the runtime to terminate it in an unusual way. Please contact the application’s support team for more information.

as usual many thanks for making such a great program!

ClassifyPixel bug: AttributeError
#2

Hi,
First of all, are you using a version of Ilastik other than the one bundled with CellProfiler? If so, the classifier that is generated will probably not work with CellProfiler - the data format is specific to the version you downloaded. The bug is internal to their code and it may have something to do with your choice of features to measure in Ilastik, especially if you are using the largest of them (GigaHuge). I would contact them directly and include an example full stack trace, image and project file. I can also take a look; I have some familiarity with the code, but it would probably take some time to submit a patch to them and have them build a version for you with the patch.

Ilastik version 0.5.06 has an “automate” feature which will output a .png whose grayscale intensity is proportional to the class probability. I think that you should be able to use Ilastik outside of CellProfiler to generate these files for your images and then you can use them as inputs to CellProfiler’s Identify modules. It’s appropriate to use a manual threshold of .5 rather than relying on Otsu, etc. to select a threshold.

–Lee

Installing CP2 on RH
#3

Hi Lee,
Thanks for responding with the useful suggestions. I had also pursued this question with the Ilastik mailing list and have gotten a reply from Christoph there about it as well.

To start with the good news: You were absolutely right about “gigahuge” being the culprit, as soon as I switched the classes down a size it started working. While I had it set to gigahuge it didn’t work in ilastik directly either with a whole other set of error messages - I’ve forwarded this on Christoph from the Ilastik mailing list.

But as you suspected, once I switched two of my features to Megahuge (From gigahuge) and exported the classifier CP2’s Classify Pixel started working as well - THANKS!

I hate to follow gratitude with a question that may sound like a complaint but: how come when I process an image in ilastik’s Batch mode I get the probability of all classes, uncertainty, and prediction in about the same amount of time classify pixel generates one class? e.g. if I want to process 2 out of the 3 classes in a classifier ilastik gives me all 3 classes in 3 min where as classify pixel needs 6min to get me 2 classes - am I doing something wrong here?

In any case many thanks for the tip that let me get it working and for all your hard work and time making this great software and support community!

-Bolek

#4

Your criticism of ClassifyPixels is valid - I’m looking at the code and Ilastik’s classifier generates all of the probability maps at once, so all we have to do is add a couple of buttons to CellProfiler to let you get more than one probability map at the same time. I used ClassifyPixels recently for a 3-class problem and ran into the same situation you did - this is a no-brainer and I’ll fix it in the development branch right away.

#5

Hi Bolek,
I added the ability to ask for multiple output images in ClassifyPixels. The 64-bit trunk build (cellprofiler.org/cgi-bin/trunk_build.cgi) is built and has the fixes.

–Lee

#6

HI Lee,
I just tried it out and the new classifyPIxels works brilliantly!
Many thanks for the help and improvements.
-Bolek

#7

Hi there,
thanks to this topic I learned that the “gigahuge” features still crash my pipeline in CP 2.1!