StarDist "Error: Images and masks should have corresponding shapes/dimensions" but they do

Hey there everyone,

I’m trying to train a StarDist Model with my own fluorescence two-channel images. I have tried to adapt the Jupyter notebook accordingly, however I always get the following error:

Traceback (most recent call last):
  File "training.py", line 156, in <module>
    model.train(X_trn, Y_trn, validation_data=(X_val,Y_val), augmenter=None,
  File ".../stardist/models/model2d.py", line 355, in train
    _data_val = StarDistData2D(*validation_data, batch_size=n_take, length=1, **data_kwargs)
  File ".../stardist/models/model2d.py", line 32, in __init__
    super().__init__(X=X, Y=Y, n_rays=n_rays, grid=grid,
  File ".../stardist/models/base.py", line 94, in __init__
    all(y.ndim==nD and x.ndim==x_ndim and x.shape[:nD]==y.shape for x,y in zip(X,Y)) or _raise(ValueError("images and masks should have corresponding shapes/dimensions"))
  File ".../csbdeep/utils/utils.py", line 90, in _raise
    raise e

Based on other advice in the forum, I printed out the shape of my training images (so far I’ve only included two images, to see if it works):

[print(x.shape, y.shape) for x,y in zip(X,Y)]
>> (506, 506, 2) (506, 506, 1)
>> (1024, 1024, 2) (1024, 1024, 1)

As far as I understood the documentation, having multiple channels in the image sets and only one channel in the label set should not be an issue for StarDist. However, to be sure, I also duplicated the label set along the third dimension, so the shapes were the following:

[print(x.shape, y.shape) for x,y in zip(X,Y)]
>> (506, 506, 2) (506, 506, 2)
>> (1024, 1024, 2) (1024, 1024, 2)

I still get the same issue though. I’ve also tried both patch_size = (256, 256) and patch_size = (256, 256, 2)

Here is my configuration:
Config2D(axes='YXC', backbone='unet', grid=(1, 1), n_channel_in=2, n_channel_out=33, n_dim=2, n_rays=32, net_conv_after_unet=128, net_input_shape=(None, None, 2), net_mask_shape=(None, None, 1), train_background_reg=0.0001, train_batch_size=2, train_checkpoint='weights_best.h5', train_checkpoint_epoch='weights_now.h5', train_checkpoint_last='weights_last.h5', train_completion_crop=32, train_dist_loss='mae', train_epochs=400, train_foreground_only=0.9, train_learning_rate=0.0003, train_loss_weights=(1, 0.2), train_n_val_patches=None, train_patch_size=(256, 256, 2), train_reduce_lr={'factor': 0.5, 'patience': 40, 'min_delta': 0}, train_shape_completion=False, train_steps_per_epoch=100, train_tensorboard=True, unet_activation='relu', unet_batch_norm=False, unet_dropout=0.0, unet_kernel_size=(3, 3), unet_last_activation='relu', unet_n_conv_per_depth=2, unet_n_depth=3, unet_n_filter_base=32, unet_pool=(2, 2), unet_prefix='', use_gpu=False)

On a side note, some of the augmentations do not work (hence I’ve turned it off for now), so maybe there is something wrong with my images? But to me, it rather looks like the image is rotated along the channel axis and has dimensions (256, 2), for example. This also makes me realise that the normalisation brings out a lot more information that I previously hadn’t detected in Fiji/QuPath and maybe I should re-do the manual annotation…


Hi @Valley ,

The problem arises because your label masks are not 2D dimensional, but have an additional channel dimension. For example the shape of the first label image should be (506,506) instead of (506,506,1). Simply remove the last (unnecessary) channel dimension from all labels and it should work, e.g. by

Y = [y[...,0] for y in Y]

That is a side effect of the masks having the erroneous extra channel. Should be fixed if you fix the dimensions.

Hope that helps!

Cheers,
M

1 Like

Silly me!

Thank you so much, it works like a charm now. Also, many thanks for making this great tool available!

Hey @mweigert ,

I do have one more error that keeps showing up:

Traceback (most recent call last):
File "training.py", line 174, in <module>
    model.optimize_thresholds(X_val, Y_val)
  File ".../stardist/models/base.py", line 612, in optimize_thresholds
    Yhat_val = [self.predict(x, **_predict_kwargs(x)) for x in X_val]
  File ".../stardist/models/base.py", line 612, in <listcomp>
    Yhat_val = [self.predict(x, **_predict_kwargs(x)) for x in X_val]
  File ".../stardist/models/base.py", line 305, in predict
    self.config.n_channel_in == x.shape[channel] or _raise(ValueError())
  File ".../csbdeep/utils/utils.py", line 90, in _raise
    raise e
ValueError

Do you have any idea what might be causing this? I checked, n_channel_in in my script is correctly identified as 2, and I don’t quite understand what the prediction function does exactly to track the error down…

So your validation images all have two channels? Could you do the following?

[print(x.shape) for x in X_val]

Sure thing :slight_smile:

I only have two images in total (so only one validation image), but here’s the output:

>> [print(x.shape for x in X_val]
(506, 506, 2)
>> [print(y.shape for y in Y_val]
(506, 506)

model.train(X_trn, Y_trn, validation_data=(X_val,Y_val), augmenter=augmenter, epochs=2, steps_per_epoch=10)

works and goes through both epochs, but the program breaks at

Y_val_pred = [model.predict_instances(x, n_tiles=model._guess_n_tiles(x), show_tile_progress=False)[0] for x in tqdm(X_val)]

1 Like

Can you please share the notebook to help us debug this?

Sure, I’ve added a txt version below. I’ve just used the Jupiter training notebook from GitHub and adjusted it for my images (deleted some of the figure generation for validation here just to keep it short).

The main difference is how we handle the picture import (see also below), so maybe this is the root of the issue?

def get_snap(filepath):
     img = AICSImage(filepath)
     return img.get_image_data("CYX", S=0, Z=0, T=0, B=0)

X = [get_snap("crop1_gamma_115_image.tif"), get_snap("gamma_108_S_image.tif")]  #image
X = [move_image_axes(x, 'CYX', 'YXC') for x in X]
Y = [get_snap("crop1_gamma_115_label.tif"), get_snap("gamma_108_S_label.tif")] #mask
Y = [move_image_axes(y, 'CYX', 'YXC') for y in Y] #reorder so that the image is in StarDist's expected order
Y = [y[...,0] for y in Y] #the masks should not have a channel

shortened_notebook.txt (5.8 KB)

Hi @Valley ,

best would be to share the notebook as html including all the intermediate output/errors - that way we could see if something got messed up along the way…

1 Like

Hey @mweigert ,

I just copied the code sections of the Jupyter Notebook to my IDE, so I don’t have it as a Notebook. Sorry about that…

Hi @Valley, we can’t help you any further then. Sorry.

Found the error: During quick_demo, the Jupyter notebook loads the pertrained example, which assumes n_channel_in = 1, not 2 as in my case. Just running the whole training solved the subsequent errors… :grin: :woman_facepalming:

Sorry for taking your time with this!

if quick_demo:
    print (
        "NOTE: This is only for a quick demonstration!\n"
        "      Please set the variable 'quick_demo = False' for proper (long) training.",
        file=sys.stderr, flush=True
    )
    model.train(X_trn, Y_trn, validation_data=(X_val,Y_val), augmenter=augmenter,
                epochs=2, steps_per_epoch=10)

    print("====> Stopping training and loading previously trained demo model from disk.", file=sys.stderr, flush=True)
    model = StarDist2D.from_pretrained('2D_demo') <------ in its config, n_channel_in = 1
else:
    model.train(X_trn, Y_trn, validation_data=(X_val,Y_val), augmenter=augmenter)
1 Like

Ah, ok. Makes sense.