Mass Excel Export

Hello,

I want to export several tifs at once into a single excel sheet using BiofilmQ. This way I do not have to consolidate the data onto a single sheet later. Is this possible?

Thank you,

L

Hi L,

unfortunately this feature does not exist. However, you are not the first person to ask for this and we have already added it to our list of improvements to include in a future release. Maybe you could help us make that user friendly?

Specifically, I am wondering if you are mostly interested in combining files of global properties or local properties (or both)? What do you feel would be an intuitive way to structure these excel files? In the case of local properties, each column in the file typically corresponds to a property and each row to a cube, such that different images would either need to be added below each other (e.g. with a new columns corresponding to the file name) or as additional sheets in the same file. Do you have any personal preference between these options?

Thank you for your feedback!

Best,

Hannah

Hi Hannah!

Sorry to hi-jack this thread but I think maybe I can provide some feedback on your question.
I believe concatinating global properties for each time-frame would be beneficial. This would create a “tidy” dataframe where each row correspond to an observation and each column to a variable. It also doesn’t make much sense to me to produce multiple files with only a single row, I am not sure when that is supposed to be used.

In my case, I’m conducting screens that aren’t time-resolved, but rather, position-resolved. I am scanning different positions in the different wells of a microtiter-plate and want to calculate some statistics at each position in each well. Having these statistics in a single document would then be quite nice.

As far as I understand, the default behavior of BiofilmQ to separate images based on positions with time-steps at a single position grouped together. That is not always the case. Perhaps it is possible to be able to specify whether your experiment is time-resolved or position-resolved i.e. do you want position directories time step-groups or time directories with position-groups.

Thanks!

  • Mark

Also, do you have a preferred way to receive ideas for implementations or bug fixes?
Should I open a thread in here or would you rather want an e-mail?

Hi Mark,

thank you so much for your feedback. I can definitely see that having separate files for the global properties is not practical and having them all in one file makes further processing much easier. We will change this soon and I will reply to this post when the new version is available.

It actually doesn´t really matter for BiofilmQ if multiple images in your folder correspond to different timepoints or to different positions. The property “frame”, which counts through the images and can also be used as an axis for plotting (e.g. to display results from different positions or timepoints next to each other), can be interpreted both as timepoint or position number. When importing files you have the possibility to choose between creating a new folder for each position or placing all files in the same folder. In your case, the second option should lead to the desired outcome.

This is of course assuming that you only have one timepoint in your experiment. If you have several timepoints and several positions, it does get a bit more complicated. In that case, if you want different folders to represent different timepoints rather than different positions, you would indeed have to first import all files into the same folder and then split them manually. I can see that this may become quite tedious depending on how many timepoints you have. We didn´t really encounter this use-case during our own experiments or that of collaborators, which is why we did not include the option to split by timepoint. Do you think this would be a useful feature for you and others?

If you would like to suggest new features or improvements to BiofilmQ or if you have found that something is not working as it should, creating a post in the forum is the best way to go. This way, others can also benefit from and add to the discussion. But of course, we are also happy to help you via e-mail, if you prefer this way of contacting us.

Thanks again for helping us improve BiofilmQ and I am looking forward to hearing from you soon :grinning:

Best,

Hannah

Sounds great with the global properties changes!

I may be mistaken, but as far as I can see, importing custom .tifs automatically creates new directories depending on the position-argument. I am not prompted to choose whether to create new directories or not (though this would be a great addition!). I also looked through your tutorial on custom tif import, which also creates a subdirectory automatically.

I think a dynamic approach would circumvent the problems you mention with multiple timepoints and positions. If you allow the user to choose the parameter to group by (perhaps with a prefix, as you do with data import), you should be able to create sub-directories based on that prefix. It could perhaps be cautioned with an “advanced” mention or something like that.
Then, one could group based on anything: time, position, and possibly even what you are now designating as “label”. That allows a lot of flexibility to adjust based on any one project.

This also leads me to another point (I might have a few - look out for new posts!):
Currently, your importer seems hard-coded to look for what you have specified to be positions, channels, and times. I noticed that when I “name the channels wrong” (i.e. C0 and C1 instead of C1 and C2), BiofilmQ prompts me to change that by itself. However, if my positions are followed by letters instead of numbers, the importer doesn’t recognize that as positions (e.g. _posA1).
Again, I think dynamically looking for whatever the user has specified instead of looking for specific hard-coded values might be a more flexible option that allows users to carry out their analysis with their own specific names and numbers.
Again, from my own example, using wells as positions (A1, A2, B8 etc) is more intuitive to me, subjectively, than their corresponding numbers. And what is 1, 2, and 3 may depend on whoever is thinking about the experiments (making reading about them a bit confusing).
Allowing the user to add new fields (like experiment, temperature, solar system) would also be a great addition! Currently, everything is simply deprecated into “Label” which is a shame if you have other grouping variables than the ones BiofilmQ is expecting.

Hope this all makes sense! It is a truly amazing program you’ve created and I look forward to using it a lot more!
/Mark

Hi Mark,

thank you for the reply, I think I understand a bit better now, where you´re coming from. When mentioning the option to generate a separate folder for each position, I had the import from microscopy software files such das .nd2 or .czi in mind. In those cases, the GUI looks a bit different and the possibility is given.

In the custom tif import this possibility is in fact not given. I think it should not be too complicated to implement, so we´ll try to include it in the next update. Regarding the possibility to add and group by additional features, I think this is also something that might not be too complicated adding, but I haven´t thought it through completely just yet.

Some of your other requests might be a bit harder to implement due to the nature of how we extract information from the filename. It is quite easy, for example, to extract a number from a text following a certain pattern, because it is clear when this number ends (whenever the next non-numerical digit starts). If the text part you want to extract can consist of a mixture of numbers, letters and other sings, this become more difficult, because it is not clear which part is the positon information exactly. For example, a file named PosA1_somethingElse would be position A1, but what about Pos-A_1-SomethingElse? The same algorithm might extract “A” or “-A” as the position here, potentially leading to problems later, since we usually have multiple positions starting with “A”. I will think about this a bit more and see if I can come up with a solution that would work in all cases.

Again, thank you for your detailed feedback. It is really helpful for us to hear back from users and get some input on how to improve BiofilmQ for you. I hope you understand that it may take a while for us to implement and test features and in some cases, things might just be to complicated or time-consuming to add, but we are definitely grateful for your ideas and doing our best to improve the software :slight_smile:

Best,

Hannah

That sounds great! And don’t worry, there’s no rush.

True, there will always be some constraints on the user to provide compatible file names, and I don’t think you need to implement a function to take weird file names into account. But simply looking for the string coming before the next parameter (e.g. posA1_somethingElse) will be adequate, in my opinion. And shouldn’t be too hard to implement with grepping for variables.
A regex like “variable1.*variable2” should work, but then the user needs to be aware of the order of the variables. I can’t exactly think of a solution without having to look for the next variable, but I’m guessing you may already have that in your code (since the order of my variables didn’t seem to matter for the import).

Best,
Mark