Using metadata in table prefix of ExportToDatabase?

UPDATE: It seems to work for some fields, e.g., when exporting to MySQL / CSV instead of MySQL, it works in the output file field. Not in the table name field though. For the fields where it works, you can simply right-click the field and get the option, it then inserts a green string.

Update 2: I had a look at the source and the field does not support it.

Hello,

I would like to know how it is possible to use metadata (extracted per regex from the source image file or directory name) in the table prefix field of the ExportToDatabase module.

According to the documentation here, this is possible:

[quote]Several modules are capable of using metadata tags for various purposes. Examples include:
You have a set of experiments for which you would like to produce and save results individually for each experiment but using only one analysis run. You can use metadata tags in ExportToSpreadsheet or ExportToDatabase to save a spreadsheet for each experiment in a folder named according to the experiment.
[/quote]

We defined a metadata tag named svsID in the LoadImages mdule, but are unsure how to use it (no details are given here). We tried putting

<svsID>

and

\g<svsID>

but all of them are marked as invalid by the GUI. Ignoring this leads to SQL errors when executing the pipeline. What should I put into the text field?

I attached the pipeline.

[size=85]Background
*We are running a pipeline multiple times (for different images) and want all the results in a database at once. We are using ExportToDatabase and thus need different table prefixes for the different runs. Instead of specifying this prefix manually and setting it to a fixed string in the pipeline, we need to set it from the file name of the input image because the jobs are run by scripts on a cluster and we cannot change the pipeline between the jobs.

  • We have filenames like** 5806_tile1x1_1x1.tif** and we used the pattern ^(?P[0-9])_tile(?P[0-9])x(?P[0-9])_(?P[0-9])x(?P[0-9]*).tif to extract the 5 metadata fields we need.
    [/size]
    pipeline_odysseus.cp (8.54 KB)

Hi,

I agree that this would be a useful feature (and even had a couple of issues for it in our bug tracker a while back). However, the problem was that the table prefix needs to be defined prior to the run, while the actual metadata values are only evaluated at the beginning of the run, once the image set is collected and defined. So the needed information comes too late for it to be used in this way.

However, it should be doable by generating a view on each table for each group, using metadata to name the view; this could be done in prepare_group. So I’ll reopen the issue, and report back where when it it’s done.

Regards,
-Mark