Trainable Weka Batch Processing Increasing Processing Times

I am batch-processing folders with 50x 20-slice stacks to generate probability maps, using the script provided in the documentation.

As the script runs, total processing time increases with every successive slice, from 8000ms to 40000ms to 100000ms and beyond. Shouldn’t the entire stack finish processing around the same time, rather than spread-out over two minutes? Why do later slices take longer to process?

Each image is roughly 1024x1024px, selected features and parameters as shown:

Running on a 32-core AMD Threadripper 3970x, 64GB RAM, 32GB page file.

@iarganda perhaps you may be able to weigh in on this?

Hello @alanthi,

Can you specify the settings of your script? Thanks!

Thanks for the reply @iarganda !
I’m not entirely sure which settings you would like me to post.

The script used is the one at the bottom of this webpage Scripting the Trainable Weka Segmentation.

Output mode Probabilities, # of Threads is auto-detect, and appears to take advantage of all available threads (64).

I see, why don’t you use the version that splits the images into tiles so you save tons of memory? I thought you were using that script, so there are some parameters related to the number of tiles to create :slight_smile:

I have tried using the tile version, but I have not been able to get it to work on my stacks of 1000 images.

When I the parameters, 5 Tiles in X, 5 Tiles in Y, 5 Tiles in Z, it immediately throws the error:
“Error: new classifier was trained on 2D images and you are using Trainable Weka Segmentation 3D.”

With parameters 5 Tiles in X, 5 Tiles in Y, 0 Tiles in Z, it proceeds a little further, but after reading the class names, it throws the error:
“Error in applyClassifier: expected dimensions were 2 but 3 were found in the array of tiles per dimension.”

I have been trying to split my stacks into individual images, but it has proven difficult to script, as “Image Splitter” and “Stack to Images” produces a semicolon in the window title that causes errors when batch-saving the images.

If you trained your classifier using the 2D version of the plugin, you have put your images in a folder as individual 2D images (not stacks). For that the simplest way is to go to File > Save as > Image Sequence…. Then yes, you have to select 0 tiles in Z so the script understands the processing is 2D.

After saving as image sequence and processing individual 2D images with 2 tiles in Y and 2 tiles in X, it takes even longer than processing the whole image at once, 160 minutes vs 70 minutes without tiling, and produces a folder of 1000 individual processed images to concatenate back into one stack.

Since I have enough RAM to process whole 200-slice stacks, does tiling 2D images excessively fragment the task?

On a side note, what is the difference between “Get Probability” and “Apply Classifier” in the Trainable Weka GUI?

“Get Probabilty” appears to work much more quickly than “Apply Classifier”, even when performed on the same stack.

It depends on your machine’s specs, of course. In general, if you have enough multi-processing power, the tiling really helps reducing the RAM consumption while the time does not increase much.

“Get Probability” calculate the probability image of the currently open image, while “Apply Classifier” applies the current trained model to new images. For the second one, the features need to be calculated, so it makes sense it takes longer.