Ilastik: Error when saving and loading project

Hi @k-dominik @ilastik_team

I was in the middle of training a new Autocontext project (in a freshly-installed ilastik-1.3.3post3-OSX) when the following error showed up when trying to save:

TypeError: No conversion path for dtype: dtype('<U44')

and when re-loading the project in an attempt to get back to a previously saved version, this pops up:

Is there a way to fix this and rescue the project? If not, how can I avoid this in future?

Thanks in advance :pray:
Tobias

PS - Here is the relevant part from my log file:

ERROR 2020-10-27 14:46:30,879 log_exception 95530 123145368051712 Traceback (most recent call last):
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/shell/projectManager.py", line 293, in saveProject
    serializer.serializeToHdf5(self.currentProjectFile, self.currentProjectPath)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 1110, in serializeToHdf5
    ss.serialize(topGroup)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 213, in serialize
    self._serialize(group, self.name, self.slot)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 691, in _serialize
    classifier.serialize_hdf5(classifier_group)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/lazyflow/lazyflow/classifiers/parallelVigraRfLazyflowClassifier.py", line 419, in serialize_hdf5
    h5py_group.create_dataset("named_importances_keys", data=list(self._named_importances.keys()))
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/Resources/lib/python3.7/site-packages/h5py/_hl/group.py", line 136, in create_dataset
    dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/Resources/lib/python3.7/site-packages/h5py/_hl/dataset.py", line 118, in make_new_dset
    tid = h5t.py_create(dtype, logical=1)
  File "h5py/h5t.pyx", line 1630, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1652, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1713, in h5py.h5t.py_create
TypeError: No conversion path for dtype: dtype('<U44')

ERROR 2020-10-27 14:46:30,879 log_exception 95530 123145368051712 Project Save Action failed due to the exception shown above.
ERROR 2020-10-27 14:46:43,884 excepthooks 95530 123145368051712 Unhandled exception in thread: 'Thread-1047'
ERROR 2020-10-27 14:46:43,890 excepthooks 95530 123145368051712 Traceback (most recent call last):
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/excepthooks.py", line 129, in run
    run_old(*args, **kwargs)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/shell/gui/ilastikShell.py", line 1721, in saveAs
    self.projectManager.saveProjectAs(newPath)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/shell/projectManager.py", line 379, in saveProjectAs
    return self._takeSnapshotAndLoadIt(newPath)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/shell/projectManager.py", line 486, in _takeSnapshotAndLoadIt
    self.saveProjectSnapshot(newPath)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/shell/projectManager.py", line 323, in saveProjectSnapshot
    with h5py.File(snapshotPath, "w") as snapshotFile:
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/Resources/lib/python3.7/site-packages/h5py/_hl/files.py", line 394, in __init__
    swmr=swmr)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/Resources/lib/python3.7/site-packages/h5py/_hl/files.py", line 176, in make_fid
    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 105, in h5py.h5f.create
OSError: Unable to create file (unable to truncate a file which is already open)

INFO 2020-10-27 14:47:31,588 ilastik_main 9885 4473775552 Starting ilastik from "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release".
WARNING 2020-10-27 14:47:35,460 opConservationTracking 9885 4473775552 Could not find any ILP solver
WARNING 2020-10-27 14:47:35,485 opStructuredTracking 9885 4473775552 Could not find any ILP solver
WARNING 2020-10-27 14:47:35,491 structuredTrackingWorkflow 9885 4473775552 Could not find any learning solver. Tracking will use flow-based solver (DPCT). Learning for tracking will be disabled!
INFO 2020-10-27 14:47:43,139 projectManager 9885 4473775552 Opening Project: /Volumes/Seagate/Image_Analysis/Ilastik/20201027_tubulin_cytoplasm_1.ilp
WARNING 2020-10-27 14:47:43,142 warnings 9885 4473775552 dataset.py(313): H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead.
INFO 2020-10-27 14:47:43,441 acceleratesupport 9885 4473775552 No OpenGL_accelerate module loaded: No module named 'OpenGL_accelerate'
WARNING 2020-10-27 14:47:43,865 warnings 9885 4473775552 slot.py(1287): UserWarning: Deprecated use slot.operator property instead
WARNING 2020-10-27 14:47:45,229 warnings 9885 4473775552 opSimpleBlockedArrayCache.py(43): FutureWarning: Conversion of the second argument of issubdtype from `dtype` to `np.generic` is deprecated. In future, it will be treated as `np.object_ == np.dtype(dtype).type`.
WARNING 2020-10-27 14:47:45,240 warnings 9885 4473775552 opSlicedBlockedArrayCache.py(141): FutureWarning: Conversion of the second argument of issubdtype from `dtype` to `np.generic` is deprecated. In future, it will be treated as `np.object_ == np.dtype(dtype).type`.
ERROR 2020-10-27 14:47:47,315 excepthooks 9885 4473775552 Unhandled exception in thread: 'MainThread'
ERROR 2020-10-27 14:47:47,322 excepthooks 9885 4473775552 Traceback (most recent call last):
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/shell/gui/ilastikShell.py", line 655, in openFileAndCloseStartscreen
    self.openProjectFile(path)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/shell/gui/ilastikShell.py", line 1489, in openProjectFile
    self._loadProject(hdf5File, projectFilePath, workflow_class, readOnly)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/shell/gui/ilastikShell.py", line 1604, in _loadProject
    self.setSelectedAppletDrawer(appletName)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/shell/gui/ilastikShell.py", line 1133, in setSelectedAppletDrawer
    self.showCentralWidget(applet_index)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/shell/gui/ilastikShell.py", line 1150, in showCentralWidget
    centralWidget = self._applets[applet_index].getMultiLaneGui().centralWidget()
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/applets/base/singleToMultiGuiAdapter.py", line 69, in centralWidget
    if self.currentGui() is None:
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/applets/base/singleToMultiGuiAdapter.py", line 48, in currentGui
    self._guis[self._imageLaneIndex] = self.singleImageGuiFactory(self._imageLaneIndex)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/applets/base/standardApplet.py", line 143, in __createSingleLaneGui
    return self.singleLaneGuiClass(self, singleLaneOperator)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/applets/layerViewer/layerViewerGui.py", line 82, in __call__
    instance = super(LayerViewerGuiMetaclass, cls).__call__(*args, **kwargs)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/applets/pixelClassification/pixelClassificationGui.py", line 468, in __init__
    self.initFeatSelDlg()
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/applets/pixelClassification/pixelClassificationGui.py", line 526, in initFeatSelDlg
    self.featSelDlg = FeatureSelectionDialog(thisOpFeatureSelection, self, self.labelListData)
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/ilastik/ilastik/applets/pixelClassification/FeatureSelectionDialog.py", line 141, in __init__
    self.selected_features_matrix = self.opFeatureSelection.SelectionMatrix.value
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/lazyflow/lazyflow/slot.py", line 1015, in value
    temp = self[:].wait()
  File "/private/var/folders/j_/37sjdxtn79g65trlwjj6cxfm0000gn/T/AppTranslocation/976EF114-C44C-48AF-B13F-165A53EDAFE0/d/ilastik-1.3.3post3-OSX.app/Contents/ilastik-release/ilastik-meta/lazyflow/lazyflow/slot.py", line 958, in __getitem__
    raise Slot.SlotNotReadyError(slotInfoMsg)
lazyflow.slot.Slot.SlotNotReadyError: Can't get data from slot <class 'ilastik.applets.featureSelection.opFeatureSelection.OpFeatureSelection'>.SelectionMatrix yet. It isn't ready.First upstream problem slot is: Wrapped OpFeatureSelection1/Wrapped OpFeatureSelection1.SelectionMatrix []: 	{_ready : False, has_mask : None, _dirty : True}

Hi @TobiasKletter,

I’m afraid you ran into a bug :disappointed:. My guess is that you have changed the classifier (something we never really do ourselves, so should probably disable…) to Vigra ... with variable importance. So to avoid, I’d say don’t change the classifier.

I’ve opened an issue to discuss this with the team.

May I ask why you are switching classifiers?

Hi @k-dominik,

thanks for the feedback and the info!

Completely forgot that I played around with the classifier at some point :slight_smile:
I changed the classifier 1) just to test if predictions & speed improve/worsen and 2) out of curiosity for the variable importance.
Maybe the users should be prompted to select their favourite classifier beforehand …

Btw, I was able to somehow salvage an earlier version of my workflow with the “project import” functionality so not everything was lost :+1:

1 Like

phew, great that you could salvage something!

1 Like