Grouping query

Question related to object tracking - what is the semantics of grouping (example: in DB - group by site, timepoint would give (site1,(t1,t2,t3)),site2, (t1,t2,t3)) - I seem to get the opposite effect here)
I am using the example pipeline for tracking with my own data - there you split data into directories. I have split my files in this way:
./site1 contains 203 files (timepoints) of syntax t1_s1 to t203_s1
same for sites 2 to 12 (12 subdirectories)
I am trying to get CP2 to process for each site a timeseries of 203 images and track on that site. If I say:

  • individual images
  • text regex
  • analyse all subfolders
  • group by metadata
  • site
  • extract metadata from path
  • .\/]site(?P[0-9]+)
    when I do this RE, I get conflicts, for files that have t100 and t101 etc and cannot do any processing

I have also tried the option of tracking by site and timepoint, but the outcome was site1_t1, site2_t1, site3_t1, or so it appeared to be, and not the expected site1_t1, site1_t2, site1_t3, using extract metadata from ‘both’. IN this case I called CP2 with files 1-203, and it was going through files as follows: 1, 204, 407, etc.

Please tell me how to set this correctly. Also, I really do not want to split my directories but do all this in one go.

Question: What is the text regex you are using?

Another issue: CP uses lexical (i.e., alphabetical) ordering for strings so that files named ‘t1’,‘t2’,‘t3’,…‘t203’ will be processed in the order ‘t1’,‘t10’,‘t11’,…‘t19’,‘t2’,‘t20’,… To get around this, you should pad the timepoints with zeros so that ‘t1’ becomes ‘t001’, ‘t2’ becomes ‘t002’ and so on.

Hi Mark,
thanks for your hint with renaming.
I had to struggle a bit with that. I had to do two renames in my file names, and the reliance on the length of the file name was a problem. What I had to do was:
split files into directories
rename files once for sites ( eg s1 to s01)
rename files again for timepoints (eg t1 to t_001)
I have now submitted a workflow for one directory, but in general, I want to rename all files once and not to have to split into directories.
I will post once I get the first directory processed.