Skip Image on Error

I’ve just started experimenting with CellProfiler. One thing I noticed is that if an error occurs when processing an image set, the options are to continue processing that image or stop processing altogether. I think an option to skip the faulty image, but continue the rest of the set, might be convenient. The attached svn patch accomplishes provides this option (it looked like that functionality was halfway in place already). The patch is against build r10891.

I tried adding this as an attachment, but kept running into errors. It’s not terribly long.

### Eclipse Workspace Patch 1.0
#P CellProfiler
Index: cellprofiler/gui/pipelinecontroller.py
===================================================================
--- cellprofiler/gui/pipelinecontroller.py	(revision 10891)
+++ cellprofiler/gui/pipelinecontroller.py	(working copy)
@@ -32,7 +32,7 @@
 from cellprofiler.gui.movieslider import EVT_TAKE_STEP
 from cellprofiler.gui.help import HELP_ON_MODULE_BUT_NONE_SELECTED
 import cellprofiler.utilities.get_revision as get_revision
-from errordialog import display_error_dialog, ED_CONTINUE, ED_STOP
+from errordialog import display_error_dialog, ED_CONTINUE, ED_STOP,ED_SKIP
 from runmultiplepipelinesdialog import RunMultplePipelinesDialog
 import cellprofiler.gui.parametersampleframe as psf
 
@@ -413,8 +413,10 @@
                                           self.__pipeline,
                                           message,
                                           event.tb)
-            if result == ED_CONTINUE:
-                event.cancel_run = False
+            #if result != ED_STOP:
+                #event.cancel_run = False
+            event.cancel_run = result == ED_STOP
+            event.skip_thisset = result == ED_SKIP
                 
         elif isinstance(event, cellprofiler.pipeline.LoadExceptionEvent):
             self.on_load_exception_event(event)
Index: cellprofiler/gui/errordialog.py
===================================================================
--- cellprofiler/gui/errordialog.py	(revision 10891)
+++ cellprofiler/gui/errordialog.py	(working copy)
@@ -22,6 +22,7 @@
 
 ED_STOP = "Stop"
 ED_CONTINUE = "Continue"
+ED_SKIP = "Skip"
 
 ERROR_URL = 'http://www.cellprofiler.org/cgi-bin/reporterror.cgi'
 
@@ -143,11 +144,15 @@
     button_sizer = wx.StdDialogButtonSizer()
     yes_button = wx.Button(dialog, wx.ID_YES, "Stop processing...")
     no_button = wx.Button(dialog, wx.ID_NO, "Continue processing...")
+    skip_button = wx.Button(dialog,wx.ID_HELP,'Skip Image, Continue Pipeline')
     button_sizer.AddButton(yes_button)
     button_sizer.AddButton(no_button)
     button_sizer.AddButton(report_button)
+    
+    button_sizer.AddButton(skip_button)
+    
     button_sizer.Realize()
-    sizer.Add(button_sizer, 0, wx.EXPAND | wx.ALL, 3)
+    sizer.Add(button_sizer, 0, wx.EXPAND | wx.ALL, 4)
     result = [None]
     #
     # Handle the "No" button being pressed
@@ -167,6 +172,15 @@
         event.Skip()
     dialog.Bind(wx.EVT_BUTTON, on_yes, yes_button)
     
+    #
+    # Handle "Skip Image" button being pressed
+    #
+    def on_skip(event):
+        result[0] = ED_SKIP
+        dialog.Close()
+        event.Skip()
+    dialog.Bind(wx.EVT_BUTTON,on_skip,skip_button)
+    
     dialog.Fit()
     dialog.ShowModal()
     return result[0]
Index: cellprofiler/pipeline.py
===================================================================
--- cellprofiler/pipeline.py	(revision 10891)
+++ cellprofiler/pipeline.py	(working copy)
@@ -1176,6 +1176,11 @@
                         self.notify_listeners(event)
                         if event.cancel_run:
                             return
+                        elif event.skip_thisset:
+                           #Skip this image, continue to others
+                            workspace.set_disposition(cpw.DISPOSITION_SKIP)
+                            should_write_measurements = False
+                            measurements = None
 
                     # Paradox: ExportToDatabase must write these columns in order 
                     #  to complete, but in order to do so, the module needs to 
@@ -1745,6 +1750,7 @@
     def __init__(self, error, module, tb = None):
         self.error     = error
         self.cancel_run = True
+        self.skip_thisset = False
         self.module    = module
         self.tb = tb

This looks good. I will incorporate it, but I was wondering whether you might be able to generate this patch in Tortoise rather than Eclipse? If it’s not straightforward, no problem; it’s short so I could do the edits manually.

Thanks!
-Mark

I don’t have as much experience with Tortoise, but it’s straightforward enough. Evidently the patch had to be broken up into 2 files, both pasted below. Let me know if this format is alright.

Part 1:

[code]Index: errordialog.py

errordialog.py (revision 10918)
+++ errordialog.py (working copy)
@@ -22,6 +22,7 @@

ED_STOP = "Stop"
ED_CONTINUE = “Continue”
+ED_SKIP = “Skip”

ERROR_URL = ‘http://www.cellprofiler.org/cgi-bin/reporterror.cgi

@@ -143,11 +144,15 @@
button_sizer = wx.StdDialogButtonSizer()
yes_button = wx.Button(dialog, wx.ID_YES, “Stop processing…”)
no_button = wx.Button(dialog, wx.ID_NO, “Continue processing…”)

  • skip_button = wx.Button(dialog,wx.ID_HELP,‘Skip Image, Continue Pipeline’)
    button_sizer.AddButton(yes_button)
    button_sizer.AddButton(no_button)
    button_sizer.AddButton(report_button)
  • button_sizer.AddButton(skip_button)
  • button_sizer.Realize()
  • sizer.Add(button_sizer, 0, wx.EXPAND | wx.ALL, 3)
  • sizer.Add(button_sizer, 0, wx.EXPAND | wx.ALL, 4)
    result = [None]

    Handle the “No” button being pressed

@@ -167,6 +172,15 @@
event.Skip()
dialog.Bind(wx.EVT_BUTTON, on_yes, yes_button)

  • Handle “Skip Image” button being pressed

  • def on_skip(event):
  •    result[0] = ED_SKIP
    
  •    dialog.Close()
    
  •    event.Skip()
    
  • dialog.Bind(wx.EVT_BUTTON,on_skip,skip_button)
  • dialog.Fit()
    dialog.ShowModal()
    return result[0]
    Index: pipelinecontroller.py
    ===================================================================
    pipelinecontroller.py (revision 10918)
    +++ pipelinecontroller.py (working copy)
    @@ -32,7 +32,7 @@
    from cellprofiler.gui.movieslider import EVT_TAKE_STEP
    from cellprofiler.gui.help import HELP_ON_MODULE_BUT_NONE_SELECTED
    import cellprofiler.utilities.get_revision as get_revision
    -from errordialog import display_error_dialog, ED_CONTINUE, ED_STOP
    +from errordialog import display_error_dialog, ED_CONTINUE, ED_STOP,ED_SKIP
    from runmultiplepipelinesdialog import RunMultplePipelinesDialog
    import cellprofiler.gui.parametersampleframe as psf

@@ -413,8 +413,10 @@
self.__pipeline,
message,
event.tb)

  •        if result == ED_CONTINUE:
    
  •            event.cancel_run = False
    
  •        #if result != ED_STOP:
    
  •            #event.cancel_run = False
    
  •        event.cancel_run = result == ED_STOP
    
  •        event.skip_thisset = result == ED_SKIP
               
       elif isinstance(event, cellprofiler.pipeline.LoadExceptionEvent):
           self.on_load_exception_event(event)
    

[/code]

Part 2:

[code]Index: pipeline.py

pipeline.py (revision 10918)
+++ pipeline.py (working copy)
@@ -1176,6 +1176,14 @@
self.notify_listeners(event)
if event.cancel_run:
return

  •                    elif event.skip_thisset:
    
  •                        #Skip
    
  •                        print 'Skipping this image, continuing to others' 
    
  •                        workspace.set_disposition(cpw.DISPOSITION_SKIP)
    
  •                        should_write_measurements = False
    
  •                        measurements = None
    
  •                        ########
    
                   # Paradox: ExportToDatabase must write these columns in order 
                   #  to complete, but in order to do so, the module needs to 
    

@@ -1745,6 +1753,7 @@
def init(self, error, module, tb = None):
self.error = error
self.cancel_run = True

  •    self.skip_thisset = False
       self.module    = module
       self.tb = tb
    

[/code]

Great, thanks! Could you post the patch for pipelinecontroller.py as well?
-Mark

It was actually in the second half of the first code fragment, but here it is anyway. Would it be easier if I sent this over the mailing list?

[code]Index: pipelinecontroller.py

pipelinecontroller.py (revision 10918)
+++ pipelinecontroller.py (working copy)
@@ -32,7 +32,7 @@
from cellprofiler.gui.movieslider import EVT_TAKE_STEP
from cellprofiler.gui.help import HELP_ON_MODULE_BUT_NONE_SELECTED
import cellprofiler.utilities.get_revision as get_revision
-from errordialog import display_error_dialog, ED_CONTINUE, ED_STOP
+from errordialog import display_error_dialog, ED_CONTINUE, ED_STOP,ED_SKIP
from runmultiplepipelinesdialog import RunMultplePipelinesDialog
import cellprofiler.gui.parametersampleframe as psf

@@ -413,8 +413,10 @@
self.__pipeline,
message,
event.tb)

  •        if result == ED_CONTINUE:
    
  •            event.cancel_run = False
    
  •        #if result != ED_STOP:
    
  •            #event.cancel_run = False
    
  •        event.cancel_run = result == ED_STOP
    
  •        event.skip_thisset = result == ED_SKIP
               
       elif isinstance(event, cellprofiler.pipeline.LoadExceptionEvent):
           self.on_load_exception_event(event)
    

[/code]

Nope, no need. I’ve added it to the code base.
-Mark

One extra note: If you’re interested, you can join a developer mailing list for CellProfiler by sending mail with “Subscribe” in the Subject field to:
cellprofiler-dev-request@broadinstitute.org.
-Mark