Dear Fiji friends,
I have a quite huge segmentation task at hand I wanted to ask a few questions - some very small, some very big - to better understand optimisation, doability and perfomance, when doing this with the trainable weka segmentation.
The data and goal:
µ-CT scans, 32-bit (2048x2048; x2048 slices) and a total of 45 scans - no, I will likely not manage to segment them all, but still. The goal is to segment into 3 domains - that’s why I chose TWS.
So I see how I can use the explorer to trim down from 271 attributes (that’s all 2D features using a sigma = 1-16) to get efficiency here (I do that using the Explorer and Attribute evaluation, e.g. Cfs filter). I have also converted my images to 16-bit and trained my model for such images. But even at this image type, using about 20 attributes + FastRandomForrest, classes not balanced - my segmentation takes pretty long (<300 images in 12 hours). I apply it using a macro in beanshell that I found.
The machine used:
Win10 - 64
128GB RAM - of which 98171 MB are available to Fiji
Intel® Xeon® CPU E5-1620 v4 @ 3.50 GHz
From the task manager it seems that CPU is the limiting factor! I should have the chance though to send a finalized workflow onto a cluster later.
So I wonder whether:
- this is normal compute times?
- Saving the reduced attributes model form “classify” tab in explorer is valid, or are all attributes of one feature type, with all sigmas, reinvoked in Fiji’s weka implementation? I use for instance attributes of 5-6 feature types with dedicated sigmas, e.g. just 4 of the many Gabor filters, or Max for sigma 2 and 16.
- Does a larger training set slow down the segmentation? My understanding is that it only affects the training process, not the application of the model!?
- Is 20 attributes still too much? I trained my model on 3 slices from the same stack using all Attributes. I then performed the Attr filter Cfs and still added 3 Kuwahara filters, as only they performed equally well in both information gain + correlation rank - and I want to segment sand grains from pore space!
- whether or not to use the same model for all scans or whether to train the same set of attributes, making the model, for each stack would be another question. A bigger story, I know, but in case someone has their thoughts on this to offer?
- for the sake of compute time I chose 2D over 3D, as I could not really see the superiority of 3D. Anybody got contrasting knowledge?
The only other option I see for now, is to downsample my images to lower resolution - maybe half to get 1024x1024?
Would be glad about any experienced reference or suggestion. Since this is my first segmentation experience I have no EXP and would be very glad about both reassurance or critical guidings and suggestions