Micro-Manager - How to do line-scanning using "Multi-Dimensional Acquisition"

Just to add another data point, I had an inexplicable hang using Pycro-Manager when running Micro-Manager from IntelliJ. The hang was induced with the following script:

from pycromanager import Bridge
mm = bridge.get_studio()
ds = mm.data().create_ram_datastore()
dv = mm.displays().close_displays_for(ds)

Using the debug=True flag did not show anything revealing, the bridge simply stopped talking.

Updating IntelliJ to the latest version (2021.1.1) fixed it… All a bit weird, but it sure would be nice to know where to start looking in situations where the bridge simply stops communicating.

Yeah that is weird. I’ve never run into this. The only thing I can think of is to check if port 4827 is still in use by a process that hasn’t exited correctly

I even rebooted the computer to be sure of that;) Anyways, the new IntelliJ worked.

I do note that the ZMQ server now takes a long time to initialize. I don’t thin that was the case earlier. Is it doing a full introspection on the class path? We could see if it makes sense to initialize it earlier, so that the user does not have to wait for it to finish.

Bizarre. It might be something specific to ZMQ itself (or JeroMQ).

It tries to get all available packages, but not the classes themselves. I think there is likely some but of redundancy here owing to the fact that all the plugins have different classloaders (I still dont really understand this).

I think also may be executing different code when introspecting source code rather than compiled quite, so this time may vary running from source or not.

But I see no problem with initializing earlier

@henrypinkard @nicost

I have another bit of information. Running the example

import pycromanager as pm

acq = pm.Acquisition(directory=r'C:\Users\pcadmin2\Desktop', name='z-stack', debug=True, core_log_debug=True)
events = pm.multi_d_acquisition_events(z_start=0, z_end=10, z_step=0.5) # Generate the events for a single z-stack

just printed the output

connecting 4827
DEBUG, sending: {‘command’: ‘connect’, ‘debug’: True}
DEBUG, recieved: {‘type’: ‘none’, ‘version’: ‘4.0.0’}
binding 4828

and threw the error

Exception in thread Event sending:
Traceback (most recent call last):
File “C:\Program Files\Python39\lib\threading.py”, line 954, in _bootstrap_inner self.run()
File “C:\Program Files\Python39\lib\threading.py”, line 892, in run self._target(*self._args, **self._kwargs)
File “C:\Program Files\Python39\lib\site-packages\pycromanager\acquire.py”, line 106, in _event_sending_fn event_socket = bridge._connect_push(event_port)
File “C:\Program Files\Python39\lib\site-packages\pycromanager\core.py”, line 320, in _connect_push return DataSocket(
File “C:\Program Files\Python39\lib\site-packages\pycromanager\core.py”, line 30, in init self._socket.bind(“tcp://{}:{}”.format(ip_address, port))
File “C:\Users\pcadmin2\AppData\Roaming\Python\Python39\site-packages\zmq\sugar\socket.py”, line 172, in bind super().bind(addr)
File “zmq\backend\cython\socket.pyx”, line 540, in zmq.backend.cython.socket.Socket.bind
File “zmq\backend\cython\checkrc.pxd”, line 28, in zmq.backend.cython.checkrc._check_rc zmq.error.ZMQError: Address in use

Any idea why it connects to port 4827, but binds to port 4828? Is this to be expected?

Also, I’ve now been able to actually run the z-stack example in Jupyter! Just executing the stuck cell again without killing the kernel seems to force start a second acquisition. An when the kernel is killed afterwards, the stuck acquisition suddenly starts. But I’m also getting 100% CPU load from javaw.exe and it stays at 100% even after killing the kernel. According to Process Explorer, it appears to be something concerning the msvcr100.dll library (Microsoft Visual C++ Runtime 10.0):

Does this help to narrow down the problem?

Yes its expected. Multiple ports are used in order to synchronize threads across languages. 4827 is the default one for connect with Bridge (which Acquisition does behind the scenes). 4828 is where the acquisition events get sent over.

Looks like this is coming from 4828, which suggests to me that another process (maybe a pycromanager acquisition, maybe something else on your computer?) is using 4828, so that it cant.

So maybe this suggests that a previous acquisition is failing to clean up, blocking 4828, and so a new one cant get created successfully. If this is the case, then when you close micromanager and pycromanager (and making sure the processes are dead with task manager), and restart them both fresh the first acquisition you run should work. Any problems would only surface on the second or later acquisition. Is this the case?

Wait, does that mean that if I do

bridge = pm.Bridge()
core = bridge.get_core()

in one cell - for instance, in the first cell after all the import statements as a basic connection/function check - and then try to run an acquisition from another cell, that the opened bridge in the first cell will actually block the second bridge within Acquisition and therefore stall the acquisition?

But then how does one instantiate and work with multiple different MMCore modules in more complex acquisition schemes? Or is this solved by using with statements everywhere?

No, the bridge is different since the python side is the client. You can have multiple clients with the same server. So that should be fine