TrackMate Scripting: Automatically exporting spots in tracks, links in tracks, tracks statistics and branching analysis to CSV

I am writing a script that basically does the entire TrackMate pipeline (from detection to tracking including export). The thing is, I am now stuck at the very last step. I would like to export :

  • “Spots in tracks statistics”
  • “Links in tracks statistics”
  • “Tracks statistics”
  • “Branch analysis” (this one being the most important)

alongside the standard model XML.

While it is easy to programmatically export model as XML file (just assign Model instance to TmXmlWriter instance and write it), automatically outputting the previous four information proved to be a hard thing. The classes fiji.plugin.trackmate.action.TrackBranchAnalysis.java and TrackBranchAnalysis.java
seem to implement this routine in their execute() function but alas, these classes only open ResultTable to the user[0].

Do you know whether it is possible to completely automate the export process, i.e. to export aforementioned files without user interactions?

Thanks

[0] https://github.com/fiji/TrackMate/blob/master/src/main/java/fiji/plugin/trackmate/action/ExportStatsToIJAction.java and https://github.com/fiji/TrackMate/blob/master/src/main/java/fiji/plugin/trackmate/action/TrackBranchAnalysis.java


from fiji.plugin.trackmate import Model
from fiji.plugin.trackmate import Settings
from fiji.plugin.trackmate import TrackMate
from fiji.plugin.trackmate import SelectionModel
from fiji.plugin.trackmate import Logger
from fiji.plugin.trackmate.detection import DownsampleLogDetectorFactory
from fiji.plugin.trackmate.tracking.oldlap import LAPTrackerFactory
from fiji.plugin.trackmate.detection import DetectorKeys
from fiji.plugin.trackmate.tracking import LAPUtils
from fiji.plugin.trackmate.action import ExportStatsToIJAction
from fiji.plugin.trackmate.action import TrackBranchAnalysis
from fiji.plugin.trackmate.graph import GraphUtils
from ij.plugin import HyperStackConverter
from ij.measure import ResultsTable
from ij import IJ
import fiji.plugin.trackmate.visualization.hyperstack.HyperStackDisplayer as HyperStackDisplayer
import fiji.plugin.trackmate.features.FeatureFilter as FeatureFilter
import fiji.plugin.trackmate.features.track.TrackDurationAnalyzer as TrackDurationAnalyzer
  
import sys
from java.io import File
from fiji.plugin.trackmate.io import TmXmlWriter

imp = IJ.openImage('D:\\path\\merged.tif')

impconv = HyperStackConverter()
imp = impconv.toHyperStack(imp, 1, 1, imp.getStackSize())

imp.show()

#----------------------------
# Create the model object now
#----------------------------

model = Model()

# Send all messages to ImageJ log window.
model.setLogger(Logger.IJ_LOGGER)

#------------------------
# Prepare settings object
#------------------------
   
settings = Settings()
settings.setFrom(imp)
   
# Configure detector - We use the Strings for the keys
settings.detectorFactory = DownsampleLogDetectorFactory()
settings.detectorSettings = {
	DetectorKeys.KEY_RADIUS: 14.,
	DetectorKeys.KEY_DOWNSAMPLE_FACTOR: 4,
	DetectorKeys.KEY_THRESHOLD : 0.,
}
print(settings.detectorSettings)

# Config initial spot filters value
settings.initialSpotFilterValue = 3.5

# Configure spot filters - Classical filter on quality
#filter1 = FeatureFilter('QUALITY', 2.3, True)
#settings.addSpotFilter(filter1)

# Configure tracker - We want to allow merges and fusions
settings.trackerFactory = LAPTrackerFactory()
settings.trackerSettings = LAPUtils.getDefaultLAPSettingsMap() # almost good enough
print(LAPUtils.getDefaultLAPSettingsMap())
settings.trackerSettings['LINKING_MAX_DISTANCE'] = 15.0
settings.trackerSettings['LINKING_FEATURE_PENALTIES'] = {}
#gap closing
settings.trackerSettings['ALLOW_GAP_CLOSING'] = True
settings.trackerSettings['GAP_CLOSING_MAX_DISTANCE'] = 15.0
settings.trackerSettings['MAX_FRAME_GAP'] = 5
settings.trackerSettings['GAP_CLOSING_FEATURE_PENALTIES'] = {}
#splitting
settings.trackerSettings['ALLOW_TRACK_SPLITTING'] = True
settings.trackerSettings['SPLITTING_MAX_DISTANCE'] = 15.0
settings.trackerSettings['SPLITTING_FEATURE_PENALTIES'] = {}
#merging
settings.trackerSettings['ALLOW_TRACK_MERGING'] = False
settings.trackerSettings['MERGING_MAX_DISTANCE'] = 15.0
settings.trackerSettings['MERGING_FEATURE_PENALTIES'] = {}
#etc
#settings.trackerSettings['ALTERNATIVE_LINKING_COST_FACTOR'] = 1.05
#settings.trackerSettings['BLOCKING_VALUE'] = Infinity
#settings.trackerSettings['CUTOFF_PERCENTILE'] = 0.9			

# Configure track analyzers
# The displacement feature is provided by the TrackDurationAnalyzer.

settings.addTrackAnalyzer(TrackDurationAnalyzer())

# Configure track filters - We want to get rid of the two immobile spots

filter2 = FeatureFilter('TRACK_DISPLACEMENT', 10, True)
settings.addTrackFilter(filter2)

#-------------------
# Instantiate plugin
#-------------------

trackmate = TrackMate(model, settings)
   
#--------
# Process
#--------

ok = trackmate.checkInput()
if not ok:
sys.exit(str(trackmate.getErrorMessage()))

ok = trackmate.process()
if not ok:
sys.exit(str(trackmate.getErrorMessage()))

   
#----------------
# Display results
#----------------
 
selectionModel = SelectionModel(model)
displayer =  HyperStackDisplayer(model, selectionModel, imp)
displayer.render()
displayer.refresh()

# Echo results with the logger we set at start:
model.getLogger().log(str(model))

# export needed files for further analysis

outfile = TmXmlWriter(File('D:\\path\\model.xml'))
outfile.appendModel(model)
outfile.writeToFile()

# show dialog windows
## THIS HAS TO BE AUTOMATED

esta = ExportStatsToIJAction()
esta.execute(trackmate)

tba = TrackBranchAnalysis(selectionModel)
tba.execute(trackmate)

# echo feature model

print(trackmate.getModel().getTrackModel().echo())
2 Likes

Have you tried using the saveAs method of the generated ResultsTable?
Check here:

https://imagej.nih.gov/ij/developer/api/ij/measure/ResultsTable.html#saveAs-java.lang.String-

1 Like

The generated ResultsTable is defined within execute() and therefore is not accessible outside the function[0]. Is there any way I could export the information without calling execute() or using the classes? I need to reconstruct the tracks outside ImageJ for further analysis.

[0] https://github.com/fiji/TrackMate/blob/master/src/main/java/fiji/plugin/trackmate/action/TrackBranchAnalysis.java#L66

Good point! I have no good solution then.

I would workaround the problem using plain ImageJ hack, and grab the ResultsTable by its Window using its title, based on the ImageJ WindowManager:

https://imagej.nih.gov/ij/developer/api/ij/WindowManager.html#getWindow-java.lang.String-

Since we know the window title we might be able to get it this way. But this is very ugly.

This seems like a very ugly solution indeed (if it ever works, since we’d have try to access the java.awt.Panel instance via java.awt.Window in the runtime). This might be doable in Java, not so sure in Jython as I don’t have extensive experience in it yet.

So I figured out, this one part is the only one needed for user to be able to get tracking results:

public void execute( final TrackMate trackmate ){
	
    logger.log( "Exporting statistics.\n" );

	// Model
	final Model model = trackmate.getModel();
	final FeatureModel fm = model.getFeatureModel();

	// Export spots
	logger.log( "  - Exporting spot statistics..." );
	final Set< Integer > trackIDs = model.getTrackModel().trackIDs( true );
	final Collection< String > spotFeatures = trackmate.getModel().getFeatureModel().getSpotFeatures();

	// Create table
	final ResultsTable spotTable = new ResultsTable();

	// Parse spots to insert values as objects
	for ( final Integer trackID : trackIDs )
	{
		final Set< Spot > track = model.getTrackModel().trackSpots( trackID );
		// Sort by frame
		final List< Spot > sortedTrack = new ArrayList< Spot >( track );
		Collections.sort( sortedTrack, Spot.frameComparator );

		for ( final Spot spot : sortedTrack )
		{
			spotTable.incrementCounter();
			spotTable.addLabel( spot.getName() );
			spotTable.addValue( "ID", "" + spot.ID() );
			spotTable.addValue( "TRACK_ID", "" + trackID.intValue() );
			for ( final String feature : spotFeatures )
			{
				final Double val = spot.getFeature( feature );
				if ( null == val )
				{
					spotTable.addValue( feature, "None" );
				}
				else
				{
					if ( fm.getSpotFeatureIsInt().get( feature ).booleanValue() )
					{
						spotTable.addValue( feature, "" + val.intValue() );
					}
					else
					{
						spotTable.addValue( feature, val.doubleValue() );
					}
				}
			}
		}
	}
	logger.log( " Done.\n" );

	// Export edges
	logger.log( "  - Exporting links statistics..." );
	// Yield available edge feature
	final Collection< String > edgeFeatures = fm.getEdgeFeatures();

	// Create table
	final ResultsTable edgeTable = new ResultsTable();

	// Sort by track
	for ( final Integer trackID : trackIDs )
	{
		// Comparators
		final Comparator< DefaultWeightedEdge > edgeTimeComparator = ModelTools.featureEdgeComparator( EdgeTimeLocationAnalyzer.TIME, fm );
		final Comparator< DefaultWeightedEdge > edgeSourceSpotTimeComparator = new EdgeSourceSpotFrameComparator( model );

		final Set< DefaultWeightedEdge > track = model.getTrackModel().trackEdges( trackID );
		final List< DefaultWeightedEdge > sortedTrack = new ArrayList< DefaultWeightedEdge >( track );

		/*
		 * Sort them by frame, if the EdgeTimeLocationAnalyzer feature is
		 * declared.
		 */

		if ( model.getFeatureModel().getEdgeFeatures().contains( EdgeTimeLocationAnalyzer.KEY ) )
			Collections.sort( sortedTrack, edgeTimeComparator );
		else
			Collections.sort( sortedTrack, edgeSourceSpotTimeComparator );

		for ( final DefaultWeightedEdge edge : sortedTrack )
		{
			edgeTable.incrementCounter();
			edgeTable.addLabel( edge.toString() );
			edgeTable.addValue( "TRACK_ID", "" + trackID.intValue() );
			for ( final String feature : edgeFeatures )
			{
				final Object o = fm.getEdgeFeature( edge, feature );
				if ( o instanceof String )
				{
					continue;
				}
				final Number d = ( Number ) o;
				if ( d == null )
				{
					edgeTable.addValue( feature, "None" );
				}
				else
				{
					if ( fm.getEdgeFeatureIsInt().get( feature ).booleanValue() )
					{
						edgeTable.addValue( feature, "" + d.intValue() );
					}
					else
					{
						edgeTable.addValue( feature, d.doubleValue() );
					}

				}
			}

		}
	}
	logger.log( " Done.\n" );

	// Export tracks
	logger.log( "  - Exporting tracks statistics..." );
	// Yield available edge feature
	final Collection< String > trackFeatures = fm.getTrackFeatures();

	// Create table
	final ResultsTable trackTable = new ResultsTable();

	// Sort by track
	for ( final Integer trackID : trackIDs )
	{
		trackTable.incrementCounter();
		trackTable.addLabel( model.getTrackModel().name( trackID ) );
		for ( final String feature : trackFeatures )
		{
			final Double val = fm.getTrackFeature( trackID, feature );
			if ( null == val )
			{
				trackTable.addValue( feature, "None" );
			}
			else
			{
				if ( fm.getTrackFeatureIsInt().get( feature ).booleanValue() )
				{
					trackTable.addValue( feature, "" + val.intValue() );
				}
				else
				{
					trackTable.addValue( feature, val.doubleValue() );
				}
			}
		}
	}
	logger.log( " Done.\n" );

	// Show tables
	spotTable.show( "Spots in tracks statistics" );
	edgeTable.show( "Links in tracks statistics" );
	trackTable.show( "Track statistics" );
}

do you know how to translate this into Jython? On the other note, enabling the programmatic export would be great (by decoupling this routine from the instantiation of ResultsTable or making spotTable, edgeTable and trackTable fields of class ExportStatsToIJAction.java.

In that case I would directly play with the FeatureModel and read the features yourself. \

Check for instance this python script:

hey,
I m using track mate as well and want to do a batch processing using your script, “branch analysis” export is the only thing i really need. did you end up making a script that does that? if so, can i use it?

thanks a lot,
Menahem.