We have recently set up an Omero server. Everything seems to work nicely except for one thing: It seems that we can only have 2 layers of folders inside the Omero dabase, Projects and Datasets.
How can we have more folders inside folders? or is it possible to automatically tag the images with their parent folder? Since this is the structure we currently have, and files inside the folders have the same name exactly, there is no way we can use Omero if we cannot have more than 2 folder layers and if we cannot point to a higher level folder to upload. Any idea?
I am not an OMERO developer, but I am a heavy user. I understand your concern regarding OMERO “folder structure”, but I would suggest you think a little differently about data organization when using a platform like OMERO.
If you have multiple levels of directory hierarchy for your data management, it is likely that you are embedding metadata in your organizational structure. Tools like OMERO approach data management in a very different way. Instead of relying on things like filenames and folder structures to provide information about your images, you directly encode that information in the form of structured metadata (e.g., tags, key-value pairs, etc.). Under this framework, Project and Dataset are just more metadata and not necessarily “folders”. In fact, I think OMERO should not use folders as icons for Project and Dataset, because this leads to exactly the confusion you are describing.
What sort of metadata are you currently capturing in the form of a folder hierarchy? Perhaps I, or others, could provide a metadata tagging strategy that would better suit what you are trying to do.
There is a tool that can help OMERO users when it comes to tagging:
Otherwise, the question you have has been mentioned previously on the old forum:
But I think @mellertd is right. Two levels of hierarchy should be enough if you use tags or key-value pairs.
I usually tell users in my lab that datasets with many images (that you filter through tags) are better than small and numerous datasets because it’s harder to get a broad view of the images…
First, my apologies for the delay in responding to you post. That was a team mistake and we’re sorry. It’s frustrating that we dropped the ball on a new user. We try hard but we’re not perfect.
@mellertd summarised the situation and also the standard response. We are well-trained to encode metadata in folder/directory names because that’s what we access on our desktops and laptops. A database gives another level of richness and new opportunities to consider.
As an example, here’s a screenshot of a folder from my laptop, where I have the docs for a paper that’s currently sitting in bioRxiv:
That probably looks familiar and you can probably figure out what I am doing. However, there are a few richer, slightly more structured, but way more useful ways of representing the metadata I am encoding in the folder names, for example:
Document Type: Paper
Status: In Progress
This is important because:
- it’s more explicit and another person doesn’t have to guess what I am doing;
- it’s the foundation for richer, more useful searching.
OMERO is designed as a tool for sharing and collaborating on data, and alot of its functionality and design target this use case. My directory tree is probably easy to understand but things get harder if we annotate with “5 uM”. Of what? For an example with OMERO, see:
http://idr.openmicroscopy.org/webclient/?show=well-1341990 (look at the right hand panel)
It also allows us to do things like:
Here we use the K-V concept as a foundation for specific queries that exercise a biological concept.
I’ll accept this argument reduces down to something like “yes this medicine tastes bad, but it’s good for you” and violates alot of the “user is always right” arguments. We are not rejecting the point-- we have discussed this alot and concluded that we either keep the hierarchy as it is or make it infinitely deep. I will point out that I would love to do the kinds of queries on my laptop that I can do in OMERO-- and I can-- but I don’t have the incentive to annotate my papers and talks for anyone beyond me. OMERO is about organising your data, but also sharing your data. That adds a whole new level, and also brings up new topics on how to annotate large collections of images, etc. Let us know if you want to go into that next level (e.g., https://pypi.org/project/omero-metadata/).
As always, many thanks for the feedback and suggestions. They help our discussions and prioritisation. We understand if this is a full block to using OMERO in your lab and definitely appreciate that view.
Again, sorry for not responding sooner.