How can I receive mouse move/drag events as user drags a line shape?

I am trying to capture mouse drag events as the user moves a selectable line shape. I want this so I can do some image intensity profiles of a line shape as it is dragged around the screen. Similar to the discussion here and the example code here

I am using the mouse_drag_callbacks.append decorator as follows. Where self.shapeLayer is returned from napari.Viewer.add_shapes and self.myMouseDrag_Shape is a member function of my class

@self.shapeLayer.mouse_drag_callbacks.append
def shape_mouse_drag_callback(layer, event):
	self.myMouseDrag_Shape(layer, event)

With this, I always receive an event.type=='mouse_press' of type(event)==<class 'napari.util.misc.ReadOnlyWrapper'>

But I never receive any additional events while the mouse is still down and the user is dragging the line shape around?

Not sure if this is related. As I am dragging the line shape, I do get this warning. Presumably for each mouse move/drag:

/Users/cudmore/anaconda3/lib/python3.7/site-packages/vispy/visuals/markers.py:564: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  ('a_edgewidth', np.float32, 1)])

Here is a minimal working example I would like to get working. I would like to get mouse move events as the user drags a line shape. Any clues as to what I am doing wrong? In particular, I don’t understand the docstring with ‘“dragging” on its own line followed by yield’?

import napari

class bNapari:
	def __init__(self, title):

		self.viewer = napari.Viewer(title=title)

		#
		# shapes layer with 2x lines
		line1 = np.array([[11, 13], [111, 113]])
		line2 = np.array([[200, 200], [400, 300]])
		lines = [line1, line2]
		self.shapeLayer = self.viewer.add_shapes(lines,
			shape_type='line',
			edge_width = 5,
			edge_color = 'coral',
			face_color = 'royalblue')
		self.shapeLayer.mode = 'select'
		print('type(self.shapeLayer.data):', type(self.shapeLayer.data))
		print('self.shapeLayer.data:', self.shapeLayer.data)

		# see:
		# https://github.com/napari/napari/pull/544# make a callback for all mouse moves

		"""
		# this works fine but as discussed does not get called durnig dragging
		@self.shapeLayer.mouse_move_callbacks.append
		def shape_mouse_move_callback(viewer, event):
			self.myMouseMove_Shape(viewer, event)
		"""

		@self.shapeLayer.mouse_drag_callbacks.append
		def shape_mouse_drag_callback(layer, event):
			self.myMouseDrag_Shape(layer, event)

	'''
	# this works fine but as discussed does not get called durnig dragging
	def myMouseMove_Shape(self, layer, event):
		"""
		event is type vispy.app.canvas.MouseEvent
		see: http://api.vispy.org/en/v0.1.0-0/event.html
		"""
		print('myMouseMove_Shape() layer:', layer, 'event:', event, type(event), 'type:', event.type, 'button', event.button)
		ind_x, ind_y = np.round(layer.coordinates).astype('int')
		print('   myMouseMove_Shape()', ind_x, ind_y)
	'''
	
	def myMouseDrag_Shape(self, layer, event):
		"""
		event is type napari.util.misc.ReadOnlyWrapper
		"""

		ind_x, ind_y = np.round(layer.coordinates).astype('int')
		print('******',
			'myMouseDrag_Shape() layer:', layer,
			'event:', event, 'type(event):', type(event), 'event.type:', event.type, 'event.button', event.button)
		print('   x:', ind_x, 'y:', ind_y)

		# This is from the docstring and the discussion
		# see: https://github.com/napari/napari/pull/544
		# I don't understand the sytax where you have a string sitting along on a line, e.g. "dragging"
		'''
		"dragging"
		yield

		# on move
		while event.type == 'mouse_move':
			print(event.pos)
			yield

		# on release
		print('goodbye world ;(')
		'''
		
if __name__ == '__main__':
	from PyQt5 import QtGui, QtCore, QtWidgets
	app = QtWidgets.QApplication(sys.argv)
	mn = bNapari('drag line shape and get mouse drag position')
	sys.exit(app.exec_())

os: macOS 10.12.6
python: 3.7.3
napari: 0.2.4+5.g1f9cfed

Thanks in advance and a huge thanks for making an amazing viewer. I am dedicated to using Napari.

Robert

1 Like

I think I answered my own question. Victory!

I ended up using the shape layer events.set_data.connect. to specify a callback. The key line is:

# this is a callback for when the data of the line shape changes
self.shapeLayer.events.set_data.connect(self.lineShapeChange_callback)

Can I get the opinion of the developers if this is a good strategy? Or should I get the original idea of using the shape layer .mouse_drag_callbacks.append working?

Here is the working code:

import sys
import numpy as np

import napari

print(napari.__version__)

class bNapari:
	def __init__(self, title):

		self.viewer = napari.Viewer(title=title)

		# shapes layer with 2x lines
		line1 = np.array([[11, 13], [111, 113]])
		line2 = np.array([[200, 200], [400, 300]])
		lines = [line1, line2]
		self.shapeLayer = self.viewer.add_shapes(lines,
			shape_type='line',
			edge_width = 5,
			edge_color = 'coral',
			face_color = 'royalblue')
		self.shapeLayer.mode = 'select'

		# this is a callback for when the data of the line shape changes
		self.shapeLayer.events.set_data.connect(self.lineShapeChange_callback)

	def lineShapeChange_callback(self, event):
		print('=== lineShapeChange_callback()')
		print('   type(event)', type(event))
		print('   event.source:', event.source)
		print('   event.type:', event.type)

		print('   self.shapeLayer.selected_data:', self.shapeLayer.selected_data)
		
		# self.shapeLayer.selected_data is a list of int telling us index into
		# self.shapeLayer.data of all selected shapes
		selected_data = self.shapeLayer.selected_data
		if len(selected_data) > 0:
			index = selected_data[0] # just the first selected shape
			print('   shape at index', index, 'in self.shapeLayer.data changed and is now:', self.shapeLayer.data[index])
		
if __name__ == '__main__':
	from PyQt5 import QtGui, QtCore, QtWidgets
	app = QtWidgets.QApplication(sys.argv)
	mn = bNapari('drag line shape and get mouse drag position')
	sys.exit(app.exec_())
1 Like

Hi @cudmore!

It took me a while to figure this out, but I finally got it! :sweat_smile: So, the layer callbacks aren’t threaded, but rather they are generators. That is, until you yield, no other callbacks can run. That’s why you need to yield inside a for-loop, as shown in this test. This lets the callbacks be called repeatedly rather than just once. Here’s what I came up with, which I’ll add to the napari examples shortly:

"""
Display a 2D image and do complex processing based on mouse drags on a 
shapes layer.
"""

from skimage import data
from skimage import measure
import numpy as np
import napari


def profile_lines(image, shape_layer):
    profile_data = []
    for line in shape_layer.data:
        profile_data.append(
            measure.profile_line(image, line[0], line[1]).mean()
        )
    msg = ('profile means: ['
            + ', '.join([f'{d:.2f}' for d in profile_data])
            + ']')
    shape_layer.status = msg



with napari.gui_qt():
    np.random.seed(1)
    viewer = napari.Viewer()
    blobs = data.binary_blobs(length=512, volume_fraction=0.1, n_dim=2)
    viewer.add_image(blobs, name='blobs')
    line1 = np.array([[11, 13], [111, 113]])
    line2 = np.array([[200, 200], [400, 300]])
    lines = [line1, line2]
    shapes_layer = viewer.add_shapes(lines, shape_type='line',
            edge_width=5, edge_color='coral', face_color='royalblue')
    shapes_layer.mode = 'select'

    @shapes_layer.mouse_drag_callbacks.append
    def profile_lines_drag(layer, event):
        profile_lines(blobs, layer)
        yield
        while event.type == 'mouse_move':
            profile_lines(blobs, layer)
            yield

In general, you should never have to subclass napari or create your own class. If you can’t extend the napari classes and shapes to your own purposes without subclassing, that’s a problem and we want to hear about it!

Regarding the events mechanism: I think that’s an ingenious alternate way to do it, good job finding it! :joy: The problem with that method is that it will run with anything that alters the data, rather than just with mouse drags. That may or may not be something you want. I certainly wouldn’t say that it’s forbidden! @sofroniewn might have more comments on it.

I hope this helps and thank you for using napari and for your kind words about it!

Hu @cudmore - as @jni says going with the mouse drag event is probably best for now, as many things might cause a set_data event. I’ve been wanting to add some special events to the shapes layer like shape_move or shape_finished that get emitted when a shape is moved of finished drawing, those events would carry the index of the shape too, which might make some of the code simpler.

For now though, as @jni said - see if you can avoid subclassing napari, we definitely want to enable people to do the stuff you want to do without subclassing (which is generally harder to maintain and less easy to understand then us having a fully formed event system)