Viewing annotations in weka trainable segmentation

Hello,
I’m somewhat new to imagej scripting so please pardon if this is well known…I did spend time looking at examples of weka trainable segmentation scripting but could not find what I need.

We have manual annotations for images that were created in a different tool, and saved as xml for each image. The xml defines polygon coordinates and the type of segment. I want to load these annotation via script as class labels for weka trainable segmentation. I figured a script like:

import ij.IJ;
import trainableSegmentation.*;
import ij.gui.Roi;
import ij.gui.PolygonRoi;
 
imp = IJ.openImage("C:\\Users\\darshat\\Desktop\\Medico\\DAB-Separation\\14_298_1.tif" );

// create Weka Segmentation object
segmentator = new WekaSegmentation(imp);
segmentator.addExample( 0, new Roi( 10, 10, 50, 50 ), 1 );
segmentator.addExample( 0, new Roi( 100, 100, 500, 500 ), 1 );
imp.show()

should work. I would like to visually cross check the annotations are applied correctly. In the UI window, I dont see these annotations after the ‘addExample’ calls are made. Is a refresh method needed to show annotations in the UI window?

Thank you,
Darshat

I looked at what commands are issues when recording the macro for creating a trace. However rerunning the same macro later results in a null exception.

Macro:

run("Trainable Weka Segmentation", "open=C:/Users/darshat/Desktop/Medico/DAB-Separation/14_317_3.tif");
selectWindow("Trainable Weka Segmentation v3.2.34");
setTool("rectangle");
makeRectangle(361, 159, 208, 130);
call("trainableSegmentation.Weka_Segmentation.addTrace", "0", "1");

Exception:

(Fiji Is Just) ImageJ 2.1.0/1.53c; Java 1.8.0_172 [64-bit]; Windows 10 10.0; 198MB of 2668MB (7%)
 
java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at ij.macro.Functions.call(Functions.java:4515)
	at ij.macro.Functions.getStringFunction(Functions.java:276)
	at ij.macro.Interpreter.getStringTerm(Interpreter.java:1475)
	at ij.macro.Interpreter.getString(Interpreter.java:1453)
	at ij.macro.Interpreter.doStatement(Interpreter.java:333)
	at ij.macro.Interpreter.doStatements(Interpreter.java:264)
	at ij.macro.Interpreter.run(Interpreter.java:160)
	at ij.macro.Interpreter.run(Interpreter.java:93)
	at ij.macro.Interpreter.run(Interpreter.java:104)
	at ij.plugin.Macro_Runner.runMacro(Macro_Runner.java:161)
	at ij.IJ.runMacro(IJ.java:153)
	at ij.IJ.runMacro(IJ.java:142)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1148)
	at net.imagej.legacy.IJ1Helper$3.call(IJ1Helper.java:1144)
	at net.imagej.legacy.IJ1Helper.runMacroFriendly(IJ1Helper.java:1095)
	at net.imagej.legacy.IJ1Helper.runMacro(IJ1Helper.java:1144)
	at net.imagej.legacy.plugin.IJ1MacroEngine.eval(IJ1MacroEngine.java:145)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:157)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
	at trainableSegmentation.Weka_Segmentation$CustomWindow.drawExamples(Weka_Segmentation.java:1013)
	at trainableSegmentation.Weka_Segmentation.addTrace(Weka_Segmentation.java:2560)
	... 30 more

The documentation for weka_segmentation.addTrace mentions it picks the current ROI. What is going wrong?
Also weka_segmentation class that the macro reports is different from wekasegmentation that is mentioned in the tutorial here that uses AddExample instead of addTrace. Hope someone can clarify how to do this!

Thanks
Darshat

This might be caused because the Trainable Weka Segmentation plugin didn’t have time to initialize yet.

Did you try adding wait(...) statements as suggested in the documentation?

1 Like

You are right, adding wait fixed it.

So how can I do this via script, instead of macro? This is because I need to parse xml files which contain the roi data, and for that I’m using groovy.

Please have a look at this page:

The examples there are in Beanshell, but the syntax is 100% compatible with Groovy. You can even leave away all the semicolons (;) :smile:

I hope it should be straightforward, but let us know if you get stuck on the way.

1 Like

Yes I have looked at it. As my first post indicated, I was looking for visual confirmation that the right segments are being added. My problem is that after the addExample call below, the UI does not show the roi:

imp = IJ.openImage("C:\\Users\\darshat\\Desktop\\Medico\\DAB-Separation\\14_298_1.tif" );

// create Weka Segmentation object
segmentator = new WekaSegmentation(imp);
segmentator.addExample( 0, new Roi( 10, 10, 50, 50 ), 1 );
segmentator.addExample( 0, new Roi( 100, 100, 500, 500 ), 1 );
// how to see the roi added to imp?

Sorry I missed that.

You will have to replicate what the UI code does here:

Something like this should work:

import ij.WindowManager

wekaWindow = WindowManager.getWindow("Trainable Weka Segmentation v3.2.34")
wekaWindow.drawExamples()

You might (again) need to add some wait() calls or some such here, as this makes you dependent on UI events again.

Ok let me try and get back. When I looked at the github source, the customWindow class was private and not accessible to script. But if the DrawExamples call is on the ImagePlus class it should be pretty simple, missed that!

drawExamples() is indeed a protected method of the private nested class CustomWindow.

But Groovy lets you call private methods in fact, so while this approach might not be the cleanest from a programming perspective, it does work in practice. @iarganda might want to comment about possibilities to add some public API to update the image overlay.

1 Like

Ok, I did run throught this, code is below. The class that comes back is indeed the CustomWindow instance, but calling the methods on it had no impact. The UI window didnt show the annotation.

import ij.IJ;
import trainableSegmentation.*;
import ij.gui.Roi;
import ij.gui.PolygonRoi;
import ij.gui.ImageWindow;
import ij.WindowManager;
 
imp = IJ.openImage("C:\\Users\\darshat\\Desktop\\Medico\\DAB-Separation\\14_298_1.tif" );
sleep(3000);
// create Weka Segmentation object
segmentator = new WekaSegmentation(imp);
segmentator.addExample( 0, new Roi( 10, 10, 50, 50 ), 1 );
segmentator.addExample( 0, new Roi( 100, 100, 500, 500 ), 1 );


import ij.WindowManager;

wekaWindow = WindowManager.getWindow("Trainable Weka Segmentation v3.2.34");
print(wekaWindow.getClass())
sleep(3000);
wekaWindow.drawExamples();
sleep(3000);
wekaWindow.updateExampleLists();
sleep(3000);


// set in parallel in both places so we know whats getting added
//proi = new PolygonRoi(xs as int[], ys as int[], 10, Roi.POLYGON)  
//imp.setRoi(pRoi);
//segmentator.addExample(0, pRoi, 1);

Hm, and if you just call imp.updateAndDraw()? (In case you didn’t try that already.)

Yes no luck with that :frowning: Its able to read from the data structures - for example:

ws = wekaWindow.getWekaSegmentation();
print(ws.getNumOfClasses());

returns back the right number of classes, e.g. if I added one more.

1 Like

Hello @darshats and sorry for the late answer!

I believe you don’t see the new examples because you are adding them to the backend, not the GUI (WekaSegmentation vs Weka_Segmentation classes). To work with the GUI, it is maybe easier to use a macro as you were doing at some point. What was the problem then?

Thanks for the reply @iarganda. Yes I am using scripting to add the examples and classes - the annotations are output of a different tool so I use groovy script to parse and add. The ability to view in the GUI what was added programmatically would be very useful. It almost seems like it should work…

I have got past it now by spot checking a few script outputs. I do want to say weka segmentation plugin has been enormously useful! Thank you for it!