Ilastik from python: processing more than 1 image starts retraining

I have a fully-trained 2-stage autocontext ILP, and if I open that project from python and set it loose on a single preloaded array, it works fine. However, if I give it 2 preloaded arrays at once, or one then another, then it processes the first and then retrains the forest before attempting the second. This obviously slows everything down a lot. It’s also set to readonly so the newly trained model is (hopefully) not getting persisted anyway.

Script here:

from collections import OrderedDict

import vigra
import numpy as np
import ilastik_main
from ilastik.applets.dataSelection import DatasetInfo

args = ilastik_main.parse_args([])
args.headless = True
args.readonly = True
args.project = "path/to/autocontext2.ilp"

shell = ilastik_main.main(args)

arrays = [
    vigra.taggedView(
        np.random.randint(0, 256, (100, 100, 10, 1), dtype=np.uint8),
        "xytc"
    ) for _ in range(2)
]


def two_arrays_at_once():
    role_data_dict = OrderedDict([
        ("Raw Data", [DatasetInfo(preloaded_array=a) for a in arrays])
    ])

    return shell.workflow.batchProcessingApplet.run_export(
        role_data_dict, export_to_array=True
    )


def one_then_another():
    role_data_dict1 = OrderedDict([
        ("Raw Data", [DatasetInfo(preloaded_array=arrays[0])])
    ])

    preds1 = shell.workflow.batchProcessingApplet.run_export(
        role_data_dict1, export_to_array=True
    )

    role_data_dict2 = OrderedDict([
        ("Raw Data", [DatasetInfo(preloaded_array=arrays[1])])
    ])

    preds2 = shell.workflow.batchProcessingApplet.run_export(
        role_data_dict2, export_to_array=True
    )

    return preds1 + preds2


if __name__ == '__main__':
    two_arrays_at_once()
    # one_then_another()

Hi @clbarnes,

thank you for providing the example, this was easy to reproduce. I too see the classifier being retrained, which is really not how it is supposed to work.
I dug a bit into it but didn’t get a lot wiser. Will spend some more time this afternoon on this and keep you posted

1 Like

Thanks for looking into it - did you manage to get anywhere?

Hey @clbarnes,

I’m still looking into it whenever I can spare the time. I know that it (the second classifier gets dirty) happens when the lane is removed, but still don’t understand why. …