I'm writing a device adapter for the BDPathway 435 (and 855)

Hi everyone,

I recently “acquired” a BD Pathway 435 and am attempting to write a Micro-Manager device adapter for it.

This rabbit hole goes as far as you wish it to:

  • Filterwheels, XYZ stage, shutters, spinning disk serial commands are well understood and (hopefully soon) will be controllable from my device adapter.
  • I have many additional commands related to liquid dispensing (BD Pathway 855) but there was no liquid handling devices in MM last I checked (?).
  • I haven’t started on the autofocus yet as it uses a NI PCI-6221 and I haven’t had a chance to break out the logic analyzer. Also, my AttoVision install seems to be borked…

At this point, this is just to gauge interest, there can’t be that many BD Pathways left, but they were nice machines way back when, and who knows? With a new camera, new filters and an LED light source, they could still have a very fruitful second life (with CellProfiler for the image analysis). At least that’s what I’m planning for mine.

If Nico is happy with me sending him a device adapter, I can do that once I can reliably talk to my 435.

Cheers,
Egor

Great to see you here Egor! That sounds like fun. Of course I’ll be happy to incorporate the device adapter!

Liquid handling devices are interesting. We could make such an abstraction, but this also clearly treads on other, general, lab automation. There are other projects out there (I am not well educated about them), and it will be very important to be as compatible as possible, or, preferably, use the same code. Quite a big project. Maybe just properties for now?

Thanks Nico, I’ve been lurking here for some time :wink:

Just to give you some insights into the beast, there is a small DirectLOGIC 205 PLC which handles all the commands sent to the BDPathway from a single serial connection, so there isn’t a clear split between what the microscopy part is and what the liquid dispensing part is.

Inside the BDPathway 435:

Easy enough, I wrote a BDPathway hub device, to which I attached a bunch of state devices (like the door), wheel devices and shutters, xy and z stages.

Regarding what a “liquid handling device” would look like, It would be nice if it had some specific methods which could be called as part of the MDA scripting (or from Python for those so inclined).

Top of my head:

  • Set the plates and tips SBS dimensions / formats / number of wells and possibly keep track of liquid levels.
  • Set the number of tips available and maybe automatically move to the next tip / tipbox when depleted (the 855 does this automatically).
  • Query the current nest / well position (from where the current stage coordinates for example).
  • Add a tip, aspirate, dispense, mix, eject the tip in the bin or back into a nest / position (and make sure there isn’t a tip there already).
  • Plus a bunch of properties like aspirate dispense speeds (number of mixes?).

I’m trying to think this in terms of the two I know (the BDPathway 855 and the Cellomics Arrayscan) but I don’t know how these methods would translate to other machines. And unfortunately, I don’t have access to these two anymore…

Soldiering on :slight_smile:

Cheers,
Egor

Before I can release my adapter, I need to make the serial communication with Micro-manager a lot more robust. Not sure if it is the current USB-serial dongle I am using, but I see many flipped bits (even at 19200 baud), which make my life… exciting (I don’t get out much, COVID and all).

2020-07-27T09:27:27.534247 tid8148 [dbg,dev:COM4] SetCommand -> gtZ\r
2020-07-27T09:27:27.585246 tid8148 [dbg,dev:COM4] GetAnswer <- gtJ,-1\r

I’m querying the Z step size with ‘gtZ’ but the PLC tells me the ‘gtJ’ command is not valid? Meanwhile my adapter happily splits the response string at the comma and registers -1 as the step size. Nope, not good enough.

Let’s dig in a bit:

print(bin(ord('Z')))
0b1011010
print(bin(ord('J')))
0b1001010

Oh, that’s rather unfortunate! :confounded:

EDIT: My somewhat robustified “ExecuteCommand” now picks-up command mismatches and sends these kind of strings to the log:

2020-07-27T15:22:48.847586 tid12624 [dbg,dev:COM4] SetCommand -> tY\r
2020-07-27T15:22:48.898587 tid12624 [dbg,dev:COM4] GetAnswer <- 4Y,-1\r
2020-07-27T15:22:48.898587 tid12624 [dbg,dev:BDPathway-XYStage] ExecuteCommand-SerialMismatch: sent 'tY' received '4Y,-1'
2020-07-27T15:22:48.898587 tid12624 [dbg,dev:COM4] SetCommand -> tY\r
2020-07-27T15:22:48.975585 tid12624 [dbg,dev:COM4] GetAnswer <- tY330000\r
...
2020-07-27T15:23:41.357704 tid6008 [dbg,dev:COM4] SetCommand -> TaX550000,Y330000\r
2020-07-27T15:23:41.544703 tid6008 [dbg,dev:COM4] GetAnswer <- TaX550000,Y330\x1000,-1\r
2020-07-27T15:23:41.544703 tid6008 [dbg,dev:BDPathway-XYStage] ExecuteCommand- SerialMismatch: sent 'TaX550000,Y330000' received 'TaX550000,Y33000,-1'
2020-07-27T15:23:41.544703 tid6008 [dbg,dev:COM4] SetCommand -> TaX550000,Y330000\r
2020-07-27T15:23:41.659216 tid6008 [dbg,dev:COM4] GetAnswer <- TaH550000,Y330000,-1\r
2020-07-27T15:23:41.659216 tid6008 [dbg,dev:BDPathway-XYStage] ExecuteCommand-SerialMismatch: sent 'TaX550000,Y330000' received 'TaH550000,Y330000,-1'
2020-07-27T15:23:41.659216 tid6008 [dbg,dev:COM4] SetCommand -> TaX550000,Y330000\r
2020-07-27T15:23:42.132217 tid6008 [dbg,dev:COM4] GetAnswer <- TaX550000,Y330000\r

It tries 3 times before giving up or until the reply matches the command sent to the PLC. But! if the command is corrupted just right, it will still be accepted as valid. Worse case scenario, we could crash the lens into our plate or move the stage to the wrong position without it being obvious, or switch to the wrong position on the filterwheels, or all sorts of fun I am not thinking about right now.

I only have PL2303 based USB dongles but I’ll check a different one next time I’m in the lab and report back.

A quick update, here’s where I’m at, using

  • my BDPathway adapter
  • a webcam with OpenCVgrabber
  • a 24 well plate in the HCS site generator plugin with the two top rows selected.
  • MDA with the generated map.

stage

TODO: I really need to make sure the lens is completely stopped before acquiring my images. And of course the autofocus which I haven’t looked at at all for now.

Still, it’s progress :slight_smile:

Some welcome progress on the dongle front!

It turns out that my particular USB to serial dongle, based on a Prolific PL2303 chip (USB\VID_067B&PID_2303\6&217C136&0&4), wasn’t reliable at all. I have since swapped it for a FT232R based dongle (FTDIBUS\VID_0403+PID_6001+A800CC7GA\0000) and as far as I can tell, all (?) my corruption issues are now gone.

Combine this with receiving empty “” responses when querying the (x, y or z) position while the stage is moving, and I now have a reliable way to tell whether the stage is busy, and wait for the stage to stop before issuing further commands. MDA Acquisition seems to work well enough with the (mechanical) shutter, stage movements and Excitation/Dichroic/Emission filterwheel commands.

Also, it turns out I didn’t have to do anything special to make my BDPathway adapter compatible with MM2.0gamma. Sweet!!

Next I will try to make AttoVision not bork on the old XP machine and look at how the 6221 PCI card controls the autofocus.

Wonderful Egor! Please do keep us updated on your progress here. I enjoy reading about it!

Just added your device adapter code to the 1.4 repo. It should soon make it into 2.0. If there is anyone out there who has access to one of these, you can now start playing with it (but be careful, and don’t blame Egor or me when anything breaks).

I got round my first issue which was: How do I run AttoImage?? We may have lost our original config file, or at least the config file we are trying to launch AttoImage with (on the original XP computer) isn’t quite compatible with our BD Pathway 435. I tried to contact BD, but I doubt I will ever get a reply.

AttoImage (which controls the BDPathway) first opens a serial connection and sends the rd command to query what type of Pathway it’s dealing with. I remember our BDPathway 855 in Manchester was replying with “rdAtto_AAAS”, which interestingly is the answer AttoImage is expecting with the default (?) config file I am using. Instead, our BDPathway sends back “rdAtto_BD_EP”. Was that modified for our lab? I don’t know. However, AttoImage really doesn’t like this answer and shuts down soon after telling us about the “issue”.

My workaround is to put an “Arduino” microcontroller in the middle, which can analyse and modify any string sent back from the BDPathway to the listening PC. The idea of course is to swap the response containing “BD_EP” with one containing “AAAS” instead.

Since I didn’t really want to install any new device driver on the XP machine, I chose a microcontroller with (at least) two hardware serial ports, the STM32F103 blue pill. Also, it has plenty of oomph compared to an 8 bit ATmega, can be programmed with the Arduino IDE if one so wishes, and is very cheap.

I briefly considered stealing power from the RS232 port on the XP machine, but then I thought… Hey, the USB port delivers power but can only expose a serial connection, so why not use it?

Introducing… TCOD for “two controllers, one device”.

For now, TCOD has two different working modes which you can switch by issuing the `D command (for debug) from the USB side.

In normal mode, the blue pill redirects commands from either controllers to the device and sends the device’s answers to both controllers. Since the responses could come out of order, I wouldn’t advice trying to control a (any?) microscope that way, but who knows, as an acid test maybe? This could be very interesting! :thinking:

In debug mode, you simply spy on the traffic between the XP machine (the other controller) and the BDPathway (the device) using any terminal software, like KiTTY or PuTTY.

kitty_debug where “>” denotes PC to device, and “<” device to PC.

Copy-paste into a text file and you have a nice set of commands and responses to analyse in Jupyter:

That’s all! I’ll continue to play with my setup until I break something I guess :slight_smile:

For TCOD, I’m going to buy a nicer enclosure and some isolated RS232 <-> TTL modules because I’m not feeling confident about 3 mains powered machines connected to the same ground.

Also if I could do anything about it, I would have the blue pill expose two USB serial devices, one a “normal” serial communication device, and the other outputting debug info all the time.

If there is any interest in reverse engineering microscopes and writing device adapters for them, I’m happy to share my code and start a page on the micro-manager wiki :slight_smile: