Cellfinder-napari crashes with segmentation fault on macOS when saving training data in curation

Hi @adamltyson,

Napari crashes early on when saving training data. Claims segmentation fault 11, the yml file was saved, but no cubes were saved. Some info and logs below.
This one is particularly painful as it precludes curation. Happened with 100+ annotated cubes, I failed to reproduce it with ~10 training cubes.

cheers,
Ede

cellfinder-napari 0.0.10
macOS 10.14.6 (iMac17,1, BootROM 170.0.0.0.0, 4 processors, Intel Core i7, 4 GHz, 16 GB, SMC 2.33f10)
napari 0.4.7 (napari_info attached)
crashlog attached

napari_info.txt (468 Bytes)
Crash_Save_training_data.txt (136.7 KB)

Seems to be a memory issue, as crash can be reproduced with ~25 cubes on macOS. The cubes were all selected from the same optical section (but obviously may have centres in different sections).

Something similar is happening on Ubuntu 16.04 with 128GB memory (MacPro 4,1; Intel® Xeon(R) CPU E5520 @ 2.27GHz × 16).
The error message in the terminal is
[1]+ Illegal instruction (core dumped) napari

Apport is running yet there is no core dump in /var/crash - I am a bit lost there.

napari_info.txt (491 Bytes)

Can you try again with napari 0.4.8 (just released yesterday) we might have improved some of the memory handling in napari that would help

Thanks @sofroniewn,

I did the upgrade but now something seems to be broken in the cellfinder plugin and I can’t test it as it is. I tag @adamltyson here.

When I select some points and try to “Mark as cell(s)” in the plugin, there is an error: ‘Selection’ object is not subscriptable. This prevents me to add points to and thus save a new training dataset.

A few other observations:

  • the selection tool seem to have acquired the ability to move points (a behaviour new to me and unwanted for this particular task)
  • great to have an option for highlight thickness, but the default value of 1 pixel seriously confused me until I found it.

Thanks, I really like napari and you are doing a fantastic job guys!

sorry that things are broken now, it was a pretty big release, hopefully we can get a fix from somewhere relatively soon. Thanks also for the additional observations, very helpful to hear - i’ll respond below

hmm i thought we added that back in 0.4.4 (see https://github.com/napari/napari/pull/2148), interesting that you’re only see effects now. thanks for flagging for us though, we’ll keep monitoring this - cc @jni

ok yeah, good to note I think the default used to be 2, i didn’t realize that changed. I’ve made a note here that we might want to switch it back Add highlight widget to preferences dialog by ppwadhwa · Pull Request #2435 · napari/napari · GitHub.

Hi @ederancz,

Sorry for all the issues you’re having!

I haven’t seen the crashes you’re having before, but I’ve only tested extraction of lots of cubes on two very similar machines. It should work for any number of cubes, as long as you have the RAM to store 2 x 100um of image planes. It might be a while until I can look at this, but I’ve raised an issue here.

There’s definitely a bug with napari-cellfinder with napari version 0.4.8. Will need to fix that. I’ve raised an issue here.

Would you mind testing the cube extraction on version 0.4.8 to see if anything has improved? If you can’t mark new cells as training data, you could just load an existing xml file and set it to the be training data.


the selection tool seem to have acquired the ability to move points (a behaviour new to me and unwanted for this particular task)

@sofroniewn - Would setting editable=False prevent this? I’d like to be able to toggle this behaviour.

great to have an option for highlight thickness, but the default value of 1 pixel seriously confused me until I found it.

It would be great to have an option to change this. The new thickness makes it very hard to see selected points sometimes.

Thanks for all the replies. Some refinement on this point:

The default is indeed 2, not 1, my bad. This works well on macOS. However, on Ubuntu the default is barely visible, I need to increase it to at least 5px for it to look the same as on macOS (or the same it looked on the previous version of napari). I hope this helps.

Hi @adamltyson,

Yes, that makes sense. Unfortunately, the phenotype is the same, crashes after writing the yml file on both macOS and Ubuntu, with the same error messages as before. I will try another Ubuntu machine later today (which has a much more recent CPU) and getting some more RAM for the macOS machine, should be able to test it tomorrow and give an update.
[UPDATE] I have tried on an Ubuntu 20.04 (vs 16.04), 64 GB (vs 128 GB), i7-8700K CPU (vs Xeon E5520). This configuration does not crash when saving training data (neither on 0.4.7 or 0.4.8).

Sorry, I was a bit confused myself, tried to clarify it above. It is in the preferences with a default of 2. It works fine on macOS, but weirdly I need to increase it to 5px to see it on Ubuntu.

Ah ok @ederancz, are you able to join us in set default highlight thickness to 2 by ppwadhwa · Pull Request #2746 · napari/napari · GitHub we’re discussing the default size now. It could be useful to understand a bit more about the pixel ratio on your monitor / have you provide a screenshot or two just so we understand what it looks like for you on Ubuntu, there could be other things going on that are worth us understanding there

After offline discussion with @ederancz, I’m not going to look too far into errors on very old machines, unless these start cropping up a lot (so do let me know if they occur to anyone else).

There is a problem with the plugin on napari versions >4.7 and this is tracked here.