Hi,

I would like to use #stardist (3D) for the segmentation of a large (290, 1024, 1024) (z, y, x) confocal microscope image, but during the prediction the #jupyter kernel of the notebook crashes without any error message.

What I did so far:

I started to used an edged detection method (LoG) to create an initial data. Due to under-segmentation, the results are not usable for the type of analysis we have in mind. So we manually curated the data set with #napari.

After one week of curating and annotating, we ended up with a fully annotated volume with the dimensions (100, 476, 714) which contains approx 1000 rod-shaped bacteria.

This volume I split into 7 sub-volumes with (100, 476, 102) px each:

- 6 for training
- 1 for validation
- 1 for later testing

Thanks to data augmentation, I succeeded to train a stardist3d network with the following configurations:

```
anisotropy=(1.6521739130434783, 1.0, 1.1875),
axes='ZYXC'
backbone='resnet'
grid=(1, 2, 2)
n_channel_in=1
n_channel_out=97
n_dim=3
n_rays=96
net_conv_after_resnet=128
net_input_shape=(None, None, None, 1)
net_mask_shape=(None, None, None, 1)
rays_json={
'name': 'Rays_GoldenSpiral',
'kwargs': {
'n': 96,
'anisotropy': (1.6521739130434783, 1.0, 1.1875)}}
resnet_activation='relu'
resnet_batch_norm=False
resnet_kernel_init='he_normal'
resnet_kernel_size=(3, 3, 3)
resnet_n_blocks=4
resnet_n_conv_per_block=3
resnet_n_filter_base=32
train_background_reg=0.0001
train_batch_size=1
train_checkpoint='weights_best.h5'
train_checkpoint_epoch='weights_now.h5'
train_checkpoint_last='weights_last.h5'
train_dist_loss='mae'
train_epochs=400
train_learning_rate=0.0003
train_loss_weights=(1, 0.2)
train_n_val_patches=None
train_patch_size=(100, 100, 100)
train_reduce_lr={'factor': 0.5, 'patience': 40, 'min_delta': 0}, train_steps_per_epoch=100
train_tensorboard=True
use_gpu=True)
```

For the training I used a single GTX 980 with 4 GBs of VRAM. This is the reason why I limited the patch size to (100, 100, 100). This was simply the first patch size which worked.

Technically I have access to GPUs with larger VRAM (and longer waiting times â€¦), but I always prefer quick iterations over perfect results during testing.

In tensorboard I got the following loss curves:

The prediction with the test image (here cropped) works quite nice. To only draw-back are over-segmented cells (i.e the marked ones).

Compared with our previous efforts to tackle our problems with a classical segmentation pipeline in MATLAB; 1.5 weeks for annotation, python coding, setup, and training is ridiculously fast.

Finally I gave the larger volume a try:

```
from __future__ import print_function, unicode_literals, absolute_import, division
import sys, os
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
from glob import glob
from tifffile import imread
from csbdeep.utils import Path, normalize
from stardist import random_label_cmap
from stardist.models import StarDist3D
np.random.seed(6)
lbl_cmap = random_label_cmap()
model = StarDist3D(None, name='stardist', basedir='models')
# contains only a single tif stack:
X = glob('largedatasets/*.tif')
X = list(map(imread, X)
X= [normalize(x, 1,99.8) for x in X]
print(X[0].shape)
# returns (290, 1024, 1024)
labels= model.predict(X[0], n_tiles=(3, 11, 11))
```

(Heavily inspired by the corresponding stardist example notebook)

Shortly after the final progress bar reaches 100%, the #jupyter kernel dies and all labels are lost â€¦

My questions so far:

- Has anyone an idea what could cause the kernel to crash?
- Has anyone seen over-segmentation like the one shown above and knows how to deal with it? (Would be awesome if we could eliminate this without post-processing or - even worst - by annotating more training data)
- (Are there other improvements possible?)

Eric