Napari Scaling Mechanics

Hello, all!

I’m trying to understand exactly how layer scaling is performed in napari. I thought maybe that it was being done just by multiplying the image array by the scale array, but this appears not to be the case: either something else is going on, or there appears to be some translation happening as well.

As an example, the following code calculates a random Bezier curve and then draws it three times in a 3d viewer – once with scaling performed by napari, once with scaling performed manually by multiplying the Bezier points with the scaling array, and once with scaling performed manually by multiplying the Bezier points with the scaling array and then adding a scalar.

import bezier
import napari
import numpy as np

mean = np.random.choice(11, size=3) * 1.
control_0, control_1 = None, None
while np.equal(mean, control_0).all() or np.equal(mean, control_1).all() or np.equal(control_0, control_1).all():
    control_0 = np.random.choice(11, size=3) * 1.
    control_1 = np.random.choice(11, size=3) * 1.

bez_nodes = np.array([mean - control_0, mean - control_0 - control_1, mean - control_1]).T
curve = bezier.Curve(bez_nodes, degree=2)
terminus = (mean - control_1).reshape(-1, 1)
terminus = curve.locate(terminus)
bez_s_vals = np.linspace(0, terminus)
bez_points = curve.evaluate_multi(bez_s_vals).T

scale = np.random.choice(11, size=3)

with napari.gui_qt():
    viewer = napari.Viewer(ndisplay=3)
    viewer.add_shapes([bez_points], edge_width=0.1, shape_type='path', blending='translucent', edge_color=['yellow'], name='napari_scaling', scale=scale)

    bez_points *= scale
    viewer.add_shapes([bez_points], edge_width=0.1, shape_type='path', blending='translucent', edge_color=['cyan'], name='manual_scaling_no_translation')

    bez_points += scale // 2
    viewer.add_shapes([bez_points], edge_width=0.1, shape_type='path', blending='translucent', edge_color=['magenta'], name='manual_scaling_with_translation')

I’ve included a screenshot of the output from this script below. As we can see, the magenta curve, which is the one that was both manually scaled and translated, fits within the yellow one, which is the one that was scaled by napari; the cyan one, which was only manually scaled without translation, does not. What’s the story here? (Please ignore black screenshot artifacts near bends of cyan and yellow curves.)

For the translation scalar, scale // 2 seems to work well for the examples that I’ve tried, but I’m not sure whether this is just a fluke.

Hi @s7b92k, what’s going on might be a bug related to how our “origin” is calculated for our Surface layer.

For the Image layer the rendering engine we use (vispy) takes the origin to be the top left corner of the top left pixel. For the Points layer the rendering engine would position the center of a point at (0, 0) at this origin. We wanted our images to have an origin at the center of the top left pixel as we think of pixels as samples from a real underlying physical space and so we needed to add an offset of scale / 2, which is half a pixel size.

Now I’ll be honest without looking further into it for Surface layers it looks from your screenshot like we are applying that scaling for Surface layers too, which is probably the wrong choice! As for a Surface we vertices that should be positioned without offset.

Remembering the code, whether to apply the offset or not is determined here by this _array_like flag https://github.com/napari/napari/blob/bf0c7d081b644c1c19488fc69df5f03460275f3e/napari/_vispy/vispy_base_layer.py#L131 which is set as False in the base object, but then True in the VispyImageLayer here https://github.com/napari/napari/blob/bf0c7d081b644c1c19488fc69df5f03460275f3e/napari/_vispy/vispy_image_layer.py#L24.

Looking at the Surface layer I don’t see that flag being set to True, and that object isn’t inheriting from VispyImageLayer so at first glance I’m a little surprised that the offset is being applied. https://github.com/napari/napari/blob/bf0c7d081b644c1c19488fc69df5f03460275f3e/napari/_vispy/vispy_surface_layer.py

We can dig into this more, but this is some extra info that should at least help try and understand what’s happening. It’s also worth noting that you should be on 0.4.0 to be trying this stuff, as we made some big improvements around this stuff since the 0.3.8 release.

More to come!

Thanks for your response!

I was using 0.3.8 when I produced the output shown above, but upgrading to 0.4.0 did in fact fix the issue (multiplicative scaling without offset now produces a curve that sits inside of the one scaled by napari).

As an aside, you mentioned the Surface layer a few times; I believe that what I was concerned about here was actually the behavior of the Shapes layer (unless Shapes uses Surface under the hood).

Thanks again!

1 Like

Great!

That’s my bad, sorry! They both use the same Mesh and transforms under-the-hood, so the same applies to both, but good catch!