Bounding box annotations with text

I’m trying to use napari to add bounding box annotations.
I read here https://napari.org/tutorials/applications/annotate_segmentation and found the way to create box annotations with text from labels, however I don’t know how to manually add box annotations.
I want to create the following functions

  • create bouding box annotations with text manually
  • want to name one label out of several for each box
    thank you very much for your kind help!

Hi @hiroalchem, sorry no one has gotten back to you yet on this. Maybe @kevinyamauchi has a suggestion as he wrote that tutorial. I can also take a look when I get the chance!!

1 Like

Hello @hiroalchem . I am very sorry for the late reply. I lost track of this over the holiday break.

My understanding of your post is that you would like to be able to create manual bounding box annotations and be able to control the text associated with each bounding box. As described in the example you linked above, the text for each shape is generated from the layer properties. The property values for the currently selected shape and the shape the next shape to be added are stored in shapes_layer.current_properties. Thus to either edit the text associated with the currently selected shape or the next shape to be added, we must modify the values in shapes_layer.current_properties. To be a bit more concrete:

# create a shapes layer where the text for each shape is the value of the 'class' property
shapes_layer = viewer.add_shapes(data, properties=properties, text='class')

# set the viewer so that the next shape to be added as the text 'cat'
shapes_layer.current_properties['class'] = np.array(['cat'])

I have pasted an example script to create a GUI for setting/selecting the text when making manual annotations. Please note that there have been changes to napari and magicgui recently and I tested this against napari==0.4.3 and magicgui==0.2.5. There is also a small hack I had to add (commented at the bottom of the script below) to get around a bug we currently have in creating empty layers with text (see this issue).

Apologies again for the late response. Please let me know if this isn’t what you had in mind or if you have any questions!

import napari
from magicgui.widgets import ComboBox, Container
import numpy as np
from skimage import data


# set up the annotation values and text display properties
box_annotations = ['person', 'sky', 'camera']
text_property = 'box_label'
text_color = 'green'

# create the GUI for selecting the values
def create_label_menu(shapes_layer, label_property, labels):
    """Create a label menu widget that can be added to the napari viewer dock

    Parameters:
    -----------
    shapes_layer : napari.layers.Shapes
        a napari shapes layer
    label_property : str
        the name of the shapes property to use the displayed text
    labels : List[str]
        list of the possible text labels values.

    Returns:
    --------
    label_widget : magicgui.widgets.Container
        the container widget with the label combobox
    """
    # Create the label selection menu
    label_menu = ComboBox(label='text label', choices=labels)
    label_widget = Container(widgets=[label_menu])

    def update_label_menu(event):
        """This is a callback function that updates the label menu when
        the current properties of the Shapes layer change
        """
        new_label = str(shapes_layer.current_properties[label_property][0])
        if new_label != label_menu.value:
            label_menu.value = new_label

    shapes_layer.events.current_properties.connect(update_label_menu)

    def label_changed(event):
        """This is acallback that update the current properties on the Shapes layer
        when the label menu selection changes
        """
        selected_label = event.value
        current_properties = shapes_layer.current_properties
        current_properties[label_property] = np.asarray([selected_label])
        shapes_layer.current_properties = current_properties

    label_menu.changed.connect(label_changed)

    return label_widget


with napari.gui_qt():
    viewer = napari.view_image(data.camera())
    shapes = viewer.add_shapes(properties={text_property: box_annotations})
    shapes.text = 'box_label'

    # create the label section gui
    label_widget = create_label_menu(
        shapes_layer=shapes,
        label_property=text_property,
        labels=box_annotations
    )
    # add the label selection gui to the viewer as a dock widget
    viewer.window.add_dock_widget(label_widget, area='right')

    # set the shapes layer mode to adding rectangles
    shapes.mode = 'add_rectangle'

    # this is a hack to get around a bug we currently have for creating emtpy layers with text
    # see: https://github.com/napari/napari/issues/2115
    def on_data(event):
        if shapes.text.mode == 'none':
            shapes.text = text_property
            shapes.text.color = text_color
    shapes.events.set_data.connect(on_data)