Hello image.sc forum community,
I apologize in advance if the information I am looking for is already posted somewhere. I have been searching and reading the OME website and image.sc forum for many days, and what I have found makes me think I am going to have to build some things. While I enjoy trying to make wheels rounder than anyone else has, I would happily use established solutions here if there are any.
As the title indicates, I am trying to build an image database with bioimage informatics support for an individual lab. I already have a handbuilt system in place to deal with one of the bread and butter techniques in the lab, which generates widefield microscopy movies of a genetically encoded fluorescent protein sensor. During my PhD I taught myself computer programming and built this lab bioimage informatics system using Fiji and Matlab programs. The system I built organizes, processes, and analyzes the movies in a standardized and semi-automated manner, it has been used to support multiple publications and collaborations, and is currently being used by multiple people in the lab. With a couple of months before I leave the lab, I would really like to port this system into a better developed and supported solution that will continue to serve the lab well after I leave. I would like to make it as easy as possible for future lab members to add new sensors and experiments for widefield studies, bring confocal data into the image database, and develop new processing and analysis workflows to extract biological insight from the imaging data.
OMERO seems like a beautiful solution, and I have played around with the demo server a little and it seems like it could do the job. I have a problem here that OMERO servers seem to require significant IT support, and all the mentions of OMERO servers I have found seem to be set up and managed at an institutional or microscopy core level. My previous attempts to explore research IT support at our institution have been frustrating, and our general IT here is consistently underwhelming, so I’m not sure I would trust their support in setting up, maintaining, and/or administering an OMERO server. The shared microscopy facilities that are on campus often have trouble just training new users and keeping microscopes maintained, so I don’t think trying to add an OMERO server to their workload is viable. Besides that, one of the primary data generators for the lab is a microscope that is owned and maintained by the lab, so not associated with a microscopy core. Looking at trying to set up an OMERO server on my own, I find the prospect intimidating and potentially over my head, coming from an experimental biology background with a mostly self-taught computational mindset and programming skills. I imagine I could get through the set up eventually, but there is potential for it to take more time than I have. Additionally, I am the computational guy in the experimental biology lab, there is no replacement for me lined up, and I would hate to leave a system in place that could need detailed computational / IT troubleshooting or reconfiguration with hardware / software changes, new types of experimental data, or new analyses. From playing with the demo server it seems like OMERO would be user friendly enough for non-computational experimental biologists to pick up with a small learning curve, and the OMERO.applications seem like they would be very powerful and helpful, but the concerns in this paragraph seem to make OMERO not a viable option.
Another alternative I see is to bring the images in line with the OME data model, either saving the movies as OME-TIFF files that carry the metadata with them, or leaving the movies themselves as raw TIFF files and having the OME-XML metadata stored separately, using a lab unique identifier for each TIFF file to connect with its metadata. My current system leverages a lab GitHub repository to generate a lab unique identifier for each movie entered into the image database, and stores the acquisition, experiment, processing, and analysis metadata in text files in this lab GitHub repo. The movie files themselves are centrally stored in a RAID housed in a Synology Diskstation in the lab, which is only accessible on the lab wireless network or via hardwire. Movie files also live on individual lab and personal computers, where they are processed and analyzed, and the raw files are uploaded to AWS Glacier for permanent archiving. I see in some of the earlier OME papers that local databases were talked about, but nowadays OMERO seems to be the only option. Have I missed instances of groups using OME for local image databases, or does everyone just use OMERO?
My thought is that a halfway approach of bringing the movies in line with the OME data model, without going all the way to OMERO, would facilitate processing and analysis operations on the movies using multiple programs. Fiji / ImageJ is used extensively in the lab, Matlab is used to some degree, and I’m trying to introduce KNIME before I leave, to provide a graphical programming interface and powerful computational tools and workflows for experimental biologists who may be intimidated by the thought of working in computer code. The Fiji / ImageJ and Matlab workflows established in the lab can be extended with some effort, but this is not straightforward even for me, the person who single-handedly built the system from the ground up. Incorporating new sensors, experiments, or analyses, or sharing processing and analysis workflows with collaborators, is quite difficult, if it’s even possible at all. It seems the OME data model was developed partly to deal with these very issues, and so I am curious about ways this could be implemented and set up to maximally benefit my PI and future lab members.
I think this is enough to start this off, I am happy to provide any further information that is helpful. I look forward to thoughts or suggestions from the community about this situation, and any possible insights about how to proceed. Thanks for reading!