Documentation/sharing/publishing of existing Ilastik workflow

Hi,
I would like to share an Ilastik project (Ilastik 1.3.2) together with the training data in order to make our workflow reproducible for others. I have learned how to do this in the future (create a folder that contains only training data and Ilastik project, use relative file paths/store files in project file, save as zip folder and upload to zenodo), however I keep struggling when trying this with a copy of an existing analysis with sub-optimal data structure (training data and data to be analysed in one folder, default absolute path). In this case it is about documenting what I have done (beyond sharing the export tables) and just repeating the analysis is unfortunately not an option.

  1. If I have the training data in the same folder as the Ilastik project the relative path shows but I cannot select it, “copied to project file” is the only option I get.
  2. If I then select “copied to project file” and try save the project, I get an error message: “It’s too late to change the dimensionality of your data after you’ve already started training. Delete all your labels and try again”
  3. If I ignore all the warnings and save the project file, re-opening results in an error: “Could not load project file. object of tye “NoneType” has no len()”.

Any advice?

If I can add something to the wishlist from the researcher perspective, it would even be desired to be able to rename the training data before sharing the Ilastik project (as file descriptors are commonly simplified before depositing the data but the analysis long before is run on the original file descriptors).

Thank you,
Jenny

1 Like

Hi @jostrop,

first of all I thank you for your great suggestion of having more functionality with regards to renaming files. Your whole post was already a very valuable input, as it specifies a concrete use-case :slight_smile:. We know ilastik doesn’t particularly strive with respect to that. Furthermore, there have been extensive refactorings of this part over the past years, so every version might behave slightly different…

Okay. Now. 1.3.2 is rather old :/. But I assume you use this version for a reason.

I need to do some additional setup here to get 1.3.2 up and running, and will get back to you tomorrow with more insights and a possible workaround.

Hi @k-dominik,

Thanks a lot for taking the time to take a look into my issue! I think version 1.3.2post1 was the current stable version when I started to fumble around with this type of analysis and I have not updated Ilastik since (take it as an example that biologists mostly do a lousy job when it comes to data structure and keeping software up-to-date - sorry about that).

Some sort of “make this workflow shareable”-functionality, either as export function or step-by-step guideline, would definitively be a huge benefit. For non-professional users this should include clear instructions for required data structure & what information should be transferred (description of label classes, version of Ilastik etc). Being able to re-name both training images and label classes would be a bonus to match this export with the descriptors used in publications and deposited data.

Best,
Jenny

1 Like

Hi again,

I now tested if installing Ilastik 1.3.3post3 and re-opening the project-file would do the trick: it does not, no relative path shows up and saving the training data to the project file results in the same error as with version 1.3.2, only with the addition that advanced information can be found in the log file (which was not very informative to me but happy to send it if is of any use).

Best,
Jenny

Hi @jostrop

bummer!

What should work is moving the files somewhere else, so ilastik complains that it cannot find them. Then it will spin up some very un-user-friendly cycle when you open the project. It will complain that it can’t find a certain image (the name of that image you’ll have to remember) and then in the next step asks you to locate it on your drive (there you could possibly plug the renamed variation in). This is incredibly painful if you have many images.

I have to admit that there is some software where I also don’t keep up…

I guess for us this would be a great exercise, because we can then identify what we could actually do to make it more user-friendly (besides fixing this bug with the change in dimensionality… ouch…)

Hi @k-dominik,

What should work is moving the files somewhere else, so ilastik complains that it cannot find them. Then it will spin up some very un-user-friendly cycle when you open the project. It will complain that it can’t find a certain image (the name of that image you’ll have to remember) and then in the next step asks you to locate it on your drive (there you could possibly plug the renamed variation in). This is incredibly painful if you have many images.

Thanks, I will report back next week if that workaround solves the issue. I have a bunch of training images so it will take some time.

Hi @k-dominik,

What should work is moving the files somewhere else, so ilastik complains that it cannot find them. Then it will spin up some very un-user-friendly cycle when you open the project. It will complain that it can’t find a certain image (the name of that image you’ll have to remember) and then in the next step asks you to locate it on your drive (there you could possibly plug the renamed variation in). This is incredibly painful if you have many images.

Intermediate result:

  1. Yes, it was possible to re-locate files (by selecting each individual file) as you describe. However, this allows me only to create another absolute path for the images (no relative path selection available with both Ilastik 1.3.2 and 1.3.3, and trying to save in the project resulted in the known error). Thus, even if I now manage to have training images and .ilp in one folder, it is unfortunately not sufficient to make the workflow shareable (as in a public repository) since anyone who would like to take a look at the Ilastik project, had to repeat this loop.
  2. The trick was now to re-open the pixel classification .ilp linking to absolute file path in the same folder, generated by the re-localization loop in Ilastik 1.3.2, with Ilastik 1.3.3 and suddenly I could change the image file path to relative and indeed move the folder without compromising the .ilp.
    Looks like I’m half way there, still have to do the same for a subsequent object classification workflow with additional file dependencies and play around if I get the re-naming to work. Creates a minor issue how running the analysis in Ilastik 1.3.2 but creating the share-version with Ilastik 1.3.3 should be reported correctly.

Thanks a lot for your help so far and great if this could be solved in a more user-friendly and intuitive way. :slight_smile:

Regards,
Jenny