And that’s the answer! The main viewer needs to be the focus for the shortcut keys to work.
I assumed the keys were active everywhere, similar to the mouse wheel - in other words, they would be “global hotkeys”. For example, if you have the Brightness & contrast dialog in focus, the scroll wheel on the mouse still zooms the image on the viewer in and out.
For the keys, an alternate approach would be if the Brightness & contrast window or the Channel viewer was in focus, the keys would continue to toggle what was shown in the viewer.
The docs mention this in the section on shortcuts:
These shortcuts only work whenever the viewer is ‘in focus’, i.e. it is the last thing that was clicked. If a shortcut doesn’t seem to work, click on the viewer once to bring it into focus, then try the shortcut again.