I have been experimenting with parallel import. I am pleased to see an increase in the import speed.
I wanted to see how it would work with a more complex directory structure.
I have some test images in directories with a structure like:
|-images |-day1 |-am |-d1im1.tif |-d1im2.tif |-day2 |-d2im1.tif |-d2im2.tif
If I import my images without the parallel option using
omero import images -T "regex:+name:^.*images/(?<Container1>.*?)"
I get what I want, which is three datasets called day1, day1/am and day2 with the appropriate images in each one. If I add the options for parallel import using
omero import images -T "regex:+name:^.*images/(?<Container1>.*?)" --parallel-fileset 2 --parallel-upload 2
I get four datasets called day1, day1, day2 and day2, with one image in each. I can sort of see why it would do this.
I tried predefining the datasets, day1, day1/am and day2, and then running the parallel import and this worked better with most images ending up in the right dataset except d1im2.tif which was in day1/am instead of day1.
I realise that parallel importing is still experimental, and I may be asking too much of it, but I wondered if anyone had experience trying something similar?