Color support for FLIR/PtGrey Cameras?

Hi, I am hoping to use a 5MP Point Grey Color Firewire Flea2 camera with micromanager. It works, but it displays without decoding the bayer so it just displays raw pixel CCD data as greyscale.

Any possibility of getting de-bayering added in for these cameras?

-Jerry

I have the same issue, I’m using a Blackfly S USB3 Flir color camera, I can see colors using Spinview, but not with Micro-manager.
Flir support couldn’t help me with this issue.

Pinging @Elliot_Steele, so that he is aware of the demand. I vaguely remember that there is some general use debayer code somewhere in the MM code base (or maybe we talked about the desire for that)? Do you have any recollections @marktsuchida?

Yeah, uM just does not see the cameras with color as an option, it just thinks they are all monochrome. My camera works fine in FlyCap2.

Hi all,

Yes I can confirm that the uManager spinnaker drivers don’t currently support colour cameras. While there isn’t any technical reason why they couldn’t, I’ve never had a colour camera to test with and, unfortunately, I don’t see that changing any time soon

@nicost there seems to be general use Debayer code in the MMDevice code ( https://github.com/micro-manager/micro-manager/blob/9e32d4c66b1fdf3bf038994a016dc5449f108cc1/MMDevice/Debayer.h ). It looks like a couple of cameras are already using it so I assume it works. Do you think it would be possible to wrap that into an ImageProcessor device? Alternatively, someone could adapt the code into an on-the-fly image processing plugin?

I could probably lend a camera to get this working. Or if there are any other ways I can help I am all ears.

Simplest workaround so far is to use the IIDC drivers but you lose a lot of control of the camera with that. At least you get a color display though.

I have the same issue. I’m using a Blackfly S USB 3 color camera but the images are displayed only in grayscale. I can lend my camera too if that would help getting things to work.

Any help would be appreciated. Thanks.

I just bought a Blackfly S model to try out uM on our old Zeiss LSM microscope. The original camera is a model that is connected via optical fiber to a host adapter and is apparently not supported by uM. Looks like the ‘S’ models dont work with the flycap2 drivers so the spinnaker SDK world is all B&W in uM at the moment. It would be great if someone could solve this issue.

For my cam I can even set the camera to color from the device property browser. The property is ‘Blackfly S BFS-U3-27S5C-Pixel Format’ . There are many options that map to a monochrome image, such as ‘Mono’, ‘Mono16’, BayerRG8’ which all work. Only the real color modes that map to a 3 color image , such as ‘RGB8packed’ dont. The image info on the bottom of the uM main window even shows that it is a color image (1936x1464 x 3 bytes) but the image display does not appear. I checked the output in python using the spinnaker python sdk and the img.GetNDArray() returns a ‘normal’ rgb color image (3 8bit images as numpy array) for the RGB8packed format. Dont know Java at all so I am not gonna dive into the uM source…

Dont know Java at all so I am not gonna dive into the uM source…

The nice thing is that the problem is not in the Java part of the code, but in the C++ part, so if you know C++ you can help!

Specifically, the upper layers in MM expect RGB images in BGRA format, so the code in the Spinnaker device adapter should either move those RGB bytes into an BGRA buffer, or do the debayering itself (there is a debayer library included in MMDevice).

I modified the Spinnaker device adapter code so that it knows what to do when you put the camera mode to RGB8. No other color modes are supported (yet). This code will be in nightly build 2021-03-08 and later. Please note that you will need to update the Spinnaker driver to one of the 2.0 drivers (the current 2.3 driver on the FLIR website should work). Let me know of any success or failure. Tested only with a Blackfly -U3-23S6C-C.

1 Like

Hi Nico,
thanks a lot for jumping in! My C skills are a bit too rusty to get this done quickly or at all. Unfortunately the color mode does not work yet for my cam. First I found out that we need the current 2.3 spinnaker sdk. I had a version 2.0xxxx SDK and with this uM crashed when it tried to access the spinnaker driver dll. The problem with the color mode is probably related to the format names. Mine does not have a format called ‘RGB8’. Instead it has ‘BGR8’ , ‘BGRa8’ and ‘RGB8Packed’ and more. If I press the snap button with one of those set, I get a ‘Missing Pixel Type’ error message. Maybe this page is helpful for the matter: Image Format Control - BFS-U3-89S6 Version 1707.1.9.0 (flir.com)
Klaus

BGRA8 should be a perfect fit (that is what MM uses internally on little endian systems). I can add the code to make that mode work, but testing without the camera will be hard. That may need some back and forth. We will also need to add some documentation on Spinnaker - Micro-Manager so that others can discover the modes that work.

That sounds great and since the FLIR cams are quite popular this seems like a good investment! In the meantime I got the Zeiss Microscope and the Flir working together at least in mono. I still have to learn some essential things in uM but the hardware control (stages , filters etc) seem to work great!
I also made an attempt to get all this running on the old microscope PC that uses win7 X86 , but the spinnaker sdk does not install at all. Maybe it needs internet access that this PC does not have…

I found that Spinnaker installer quite a pain as well. I had to run it about five times before it finally completed (hang on uninstalling V1.2, multiple hangs on re-installing, but it finally worked…). You could contact their support.

I added code to make BGRa work. Will be in tonight’s build. If you are still awake, you can get the dll from https://valelab4.ucsf.edu/~nico/images/mmgr_dal_SpinnakerCamera.dll

I am sorry to report that this was not a success, BGRa does not work for me. There is also the problem that the ‘live mode’ cannot be started from the main windows, only by doing a snap and then switching to live. This was already so in the previous build. I am not sure if this has something to do with the driver, but this morning I entered pixel calibration values for all objectives but in the image window those are not honored. The metadata says the pixel size is 0 and the affine transform value is also all 0’s.

So, a problem with basing the fix on the spinnaker SDK is that spinnaker does not support USB 2 and Firewire cameras which leaves a lot of cameras off the list. Spinnaker supports USB3 and GigE where FlyCap supports everything else.

From the discussion above I was assuming that flycap works and spinnaker not. what a mess!

Hi @cparker

It really helps to focus on one issue at the time. I think we are discussing here RGB support in the Spinneaker device adapters. Can you open new issues for the other 2 problems (first having to press start before live mode works, i guess this is only with the Spinnaker adapter?).

As for BGRa, what “does not work”? Without access to the camera, there is little I can do to test and need your help to get this to work. A description of what happened after pressing “Snap” and “Live” will be useful. It would be best if you can generate a Trouble report (Help > Report a Problem), that will give me a lot more information.