Interactive segmentation notebook in ZeroCostDL4Mic using ImJoy/Kaibu/Cellpose

We put together a new notebook as part of the ZeroCostDL4Mic platform that uses Kaibu/ImJoy to provide interactive instance segmentation!
Providing simultaneous annotation, prediction and training → Human in the loop!
Originally developed by Wei Ouyand and the ImJoy team, and demonstrated in this F1000Research paper.

Get the notebook here!

11 Likes

Hello!

This looks very promising; thank you for all your effort and for sharing. I am getting an error shortly after starting the interactive segmentation interface:

UnboundLocalError                         Traceback (most recent call last)
<ipython-input-6-93b4020df0a9> in <module>()
     36                                object_name="cell",
     37                                scale_factor=1.0,
---> 38                                restore_test_annotation=True)
     39 

5 frames
/usr/local/lib/python3.7/dist-packages/imjoy_interactive_trainer/imjoy_plugin.py in start_interactive_segmentation(restore_test_annotation, auto_get_sample, *args, **kwargs)
    485 
    486 def start_interactive_segmentation(*args, restore_test_annotation=False, auto_get_sample=True, **kwargs):
--> 487     trainer = InteractiveTrainer.get_instance(*args, **kwargs)
    488     plugin = ImJoyPlugin(trainer, restore_test_annotation, auto_get_sample)
    489     api.export(plugin)

/usr/local/lib/python3.7/dist-packages/imjoy_interactive_trainer/interactive_trainer.py in get_instance(*args, **kwargs)
    148 
    149             InteractiveTrainer.__instance__ = None
--> 150             return InteractiveTrainer(*args, **kwargs)
    151 
    152     def __init__(

/usr/local/lib/python3.7/dist-packages/imjoy_interactive_trainer/interactive_trainer.py in __init__(self, model_config, data_dir, input_channels, folder, object_name, max_pool_length, min_object_size, scale_factor)
    186         self.min_object_size = min_object_size
    187 
--> 188         self.reload_sample_pool()
    189         # _img, _mask, _info = self.sample_pool[0]
    190 

/usr/local/lib/python3.7/dist-packages/imjoy_interactive_trainer/interactive_trainer.py in reload_sample_pool(self)
    206             self.input_channels,
    207             self.scale_factor,
--> 208             self.model.transform_labels,
    209         )
    210 

/usr/local/lib/python3.7/dist-packages/imjoy_interactive_trainer/interactive_trainer.py in load_sample_pool(data_dir, folder, input_channels, scale_factor, transform_labels)
    105 
    106         annotation_file = os.path.join(data_dir, folder, sample_name, "annotation.json")
--> 107         mask_dict = geojson_to_masks(annotation_file, mask_types=["labels"])
    108         labels = mask_dict["labels"]
    109         mask = transform_labels(np.expand_dims(labels, axis=2))

/usr/local/lib/python3.7/dist-packages/imjoy_interactive_trainer/imgseg/geojson_utils.py in geojson_to_masks(file_proc, mask_types, img_size)
    110             mask_dict = borderMasks.generate(annot_dict, mask_dict)
    111 
--> 112     return mask_dict

UnboundLocalError: local variable 'mask_dict' referenced before assignment

There does not appear to be any info on stack overflow on this error. Can you please help?

Thanks!

Hi Justin,
I would need to get more info on when that happens to help you best, but initially could you check if setting
restore_test_annotation=False
would solve your problem?

Cheers,

Romain

Hi, I think there must be some issue in your folder, more specifically, it looks like some geojson annotation file doesn’t have any annotation in it. So please try to set restore_test_annotation=False and also try to remove files callled annotation.json in your sample folder. In the meantime, I just did a quick patch to the code so it will continue in case of error. If you restart your colab and run it again, you should get the latest version. Let us know if that fix the issue.

Thank you for the suggestion. It looks like the patch worked without having to set restore_test_annotation=False.

The problem I am now having is that the “Training” tab is not showing up in Kaibu with my data loaded, but it does show up using the example dataset. Similarly, there are no folders showing under the “Samples” tab, but they do show using the example dataset.

All settings including default diameter=0 and the “default” model_type is used in both cases. When I start the interactive segmentation interface, the loading dots show for several minutes. If I click “Start Training” they disappear.

Perhaps my data does not fit the assumptions for data to use. Here is a link to the images:

https://drive.google.com/drive/folders/1_b2mleUlyAD7rBRyE5cAZ7p4yR_lrh_g?usp=sharing

Hi @smith6jt

Thanks for providing detailed description about the issue and it’s great that you provided also the dataset.

My initial investigation is that is caused by a bug from colab (asyncio timeout bug · Issue #1648 · googlecolab/colabtools · GitHub), basically when you have a long sample list (because we cut images into patches), it takes more time to load the list, but colab kills it after around 200ms. There are some tricks to fix it, but it takes more time to test. Please be patient and I will fix it when I have more time.