PerTrackFeatureColorGenerator - HyperStackDisplayer - TrackMate - Matlab

I have been using TrackMate in Matlab for batch-processing for few years. Because of recent errors with

displayer = HyperStackDisplayer(model, selectionModel, imp); 

I switched to ‘ImageJ-Matlab’ interface over ‘MIJ’ application. I can now display the image stack and overlay tracks (default settings), generated by TrackMate.

Now, previously I configured my displayer using the following

displayer.setDisplaySettings(TrackMateModelView.KEY_TRACK_COLORING, ...
  PerTrackFeatureColorGenerator(trackmate.getModel(), ...
  TrackDurationAnalyzer.TRACK_DURATION))

I have the following error

Error using TrackMate_MatlabScript_Mar2020 (line 235)
Java exception occurred:
java.lang.NullPointerException
	at
        fiji.plugin.trackmate.visualization.PerTrackFeatureColorGenerator.refreshColorMap(PerTrackFeatureColorGenerator.java:140)
	at fiji.plugin.trackmate.visualization.PerTrackFeatureColorGenerator.setFeature(PerTrackFeatureColorGenerator.java:76)
	at fiji.plugin.trackmate.visualization.PerTrackFeatureColorGenerator.<init>(PerTrackFeatureColorGenerator.java:55)

Following is my script:

clear;
javaaddpath ('/Users/goswamid2/Documents/Matlab/mij.jar');
addpath(genpath('/Applications/Fiji.app/scripts'));
addpath ('/Users/goswamid2/Documents/Matlab Scripts/bfmatlab');
addpath('/Users/goswamid2/Documents/Matlab Scripts/jheapcl/jheapcl');
ImageJ;
%----------------------------------
% Import Fiji and TrackMate classes
%----------------------------------
import ij.*
import loci.plugins.BF.*
import java.util.HashMap
import fiji.plugin.trackmate.*
import fiji.plugin.trackmate.settings.*
import fiji.plugin.trackmate.detection.*
import fiji.plugin.trackmate.features.*
import fiji.plugin.trackmate.features.track.*
import fiji.plugin.trackmate.features.track.edges.*
import fiji.plugin.trackmate.features.spot.*
import fiji.plugin.trackmate.tracking.*
import fiji.plugin.trackmate.tracking.sparselap.*
import fiji.plugin.trackmate.util.TMUtils.*
import fiji.plugin.trackmate.SelectionModel.*
import fiji.plugin.trackmate.visualization.hyperstack.*
import fiji.plugin.trackmate.visualization.*
import fiji.plugin.trackmate.visualization.trackscheme.*
import fiji.plugin.trackmate.visualization.PerTrackFeatureColorGenerator


uiwait(warndlg('Select Single Channel .nd2 OR OME-TIF movie (stacks) files only'));
filelist=uipickfiles('FilterSpec','/Users/goswamid2/Documents/Data/',...
    'REFilter','.nd2|.ome.tif|.tif');
num_file=numel(filelist);
[pathname, name, ext] = fileparts(filelist{1});

if num_file == 0
    warndlg('At least movie file (stack) must be selected','!!warning!!')
    return
end
filename=cell(num_file,1);
addpath(pathname);
if (exist ([pathname '/TRACKS' ],'dir') ~=7)
    mkdir([pathname '/TRACKS']);
else
    dummy=0;
end

for j=1:num_file
    [pathname, name, ext] = fileparts(filelist{j});
    new_path=[pathname '/TRACKS/' ];
    reader=bfGetReader(filelist{j});
    
%----------------------------------
% Read and display image file
%----------------------------------
imps = openImagePlus(filelist{j});
imp = imps(1);
imp.show();
omeMeta = reader.getMetadataStore();
tVal=0.02;
  
%----------------------------
% Create the model object now
%----------------------------
    
% Some of the parameters we configure below need to have
% a reference to the model at creation. So we create an
% empty model now.
model = Model();
    
% Send all messages to ImageJ log window.
model.setLogger(Logger.IJ_LOGGER)
       
%------------------------
% Prepare settings object
%------------------------
       
settings = Settings();
settings.setFrom(imp)
settings.dt=tVal;
%settings.xstart=1;
%settings.yend=999;
  
% Configure detector - We use a java map
settings.detectorFactory = LogDetectorFactory(); %use LogDetectorFactory() for spot sizes ?5 to ?20 pixels in diameter.
map = HashMap();
map.put('DO_SUBPIXEL_LOCALIZATION', true);
map.put('RADIUS', 0.5);
map.put('TARGET_CHANNEL', 1);
map.put('THRESHOLD', 30);
map.put('DO_MEDIAN_FILTERING', false);
settings.detectorSettings = map;
    
% Configure spot filters - Classical filter on quality
filter1 = FeatureFilter('QUALITY', 1.0, true);
settings.addSpotFilter(filter1)

% Configure tracker - I don't want to allow splits and fusions
settings.trackerFactory = SparseLAPTrackerFactory();% added by debanjan 
settings.trackerSettings = LAPUtils.getDefaultLAPSettingsMap(); % almost good enough
settings.trackerSettings.put('LINKING_MAX_DISTANCE', 0.5 );
settings.trackerSettings.put('GAP_CLOSING_MAX_DISTANCE', 0.5 );

toInt = @(x) javaObject('java.lang.Integer', x); % added by debanjan % java.lang.Integer(5)
settings.trackerSettings.put('MAX_FRAME_GAP', toInt(2)); %not available in SimpleLAPTracker

%-------------------
% Analyze Tracks
%-------------------

% Add the analyzers for some spot features.sett
% You need to configure TrackMate with analyzers that will generate
% the data you need.
img = rawWraps(settings.imp);
settings.addSpotAnalyzerFactory(SpotIntensityAnalyzerFactory())
settings.addSpotAnalyzerFactory(SpotContrastAndSNRAnalyzerFactory())
settings.addSpotAnalyzerFactory(SpotRadiusEstimatorFactory())
settings.addSpotAnalyzerFactory(SpotMorphologyAnalyzerFactory())

% Add analyzers for edges
settings.addEdgeAnalyzer(edges.EdgeTargetAnalyzer())
settings.addEdgeAnalyzer(edges.EdgeTimeLocationAnalyzer())
settings.addEdgeAnalyzer(edges.EdgeVelocityAnalyzer())    

% Add an analyzer for some track features, such as the track mean speed.
settings.addTrackAnalyzer(TrackSpeedStatisticsAnalyzer())
settings.addTrackAnalyzer(TrackDurationAnalyzer())
settings.addTrackAnalyzer(TrackIndexAnalyzer())
settings.addTrackAnalyzer(TrackLocationAnalyzer())
%settings.addTrackAnalyzer(TrackSpotFeatureAnalyzer())

%-------------------
% Filter Tracks 
%-------------------
% Configure track filters (- We want to get rid of the two immobile spots at 
% the bottom right of the image. Track displacement must be above 10 pixels.)
filter2 = FeatureFilter('TRACK_DURATION', 0.0, true);
settings.addTrackFilter(filter2)
% filter3 = FeatureFilter('TRACK_DISPLACEMENT', 0.0, true);
% settings.addTrackFilter(filter3)

%--------------------
% Instantiate plugin
%--------------------
    
trackmate = TrackMate(model, settings);
       
%----------
% Process
%----------
    
ok = trackmate.checkInput();
if ~ok
    display(trackmate.getErrorMessage())
end
 
ok = trackmate.process();
if ~ok
    display(trackmate.getErrorMessage())
end
       
%----------------
% Display results
%----------------
selectionModel = SelectionModel(model);
displayer = HyperStackDisplayer(model, selectionModel, imp);

%---------------------------
% Configure display settings
%---------------------------

displayer.setDisplaySettings(TrackMateModelView.KEY_SPOT_COLORING, ...
  SpotColorGeneratorPerTrackFeature(trackmate.getModel(), ...
  fiji.plugin.trackmate.Spot.QUALITY))

% displayer.setDisplaySettings(TrackMateModelView.KEY_TRACK_COLORING, ...
%   PerTrackFeatureColorGenerator(trackmate.getModel(), ...
%   TrackIndexAnalyzer.TRACK_INDEX))

 displayer.setDisplaySettings(TrackMateModelView.KEY_TRACK_COLORING, ...
   PerTrackFeatureColorGenerator(trackmate.getModel(), ...
   TrackDurationAnalyzer.TRACK_DURATION))

%----------------------
% Display the displayer
%----------------------
displayer.render()
displayer.refresh() 
    
% Echo results
display(model.toString())

%-------------------------------------------------------
% The feature model, that stores edge and track features
%-------------------------------------------------------
% fm = model.getFeatureModel();
% 
% capturer = fiji.plugin.trackmate.action.CaptureOverlayAction();
% capturer.execute(trackmate);

%---------------
% Export results
%---------------
fileout = java.io.File([new_path name '.xml']);
fiji.plugin.trackmate.action.ExportTracksToXML.export(model, settings, fileout);
reader.close();
pause(1);
imp.close();
clearvars  -except filelist num_file ;
end
%MIJ.exit();
ij.IJ.maxMemory()
ij.IJ.currentMemory()
ij.IJ.freeMemory()
ij.IJ.run("Quit","");

Another wraning shows up

Warning: replacing context classloader with incompatible classloader because ImageJ was not found.

Any help is much appreciated @imagejan @tinevez @tfalle34 @emartini

Hello @debanjan.goswami

I suspect the NPE you get when displaying results is linked to missing features. You want to force their declaration and calculation. I see in your script that you add some features manually:

% Add analyzers for edges
settings.addEdgeAnalyzer(edges.EdgeTargetAnalyzer())
settings.addEdgeAnalyzer(edges.EdgeTimeLocationAnalyzer())
settings.addEdgeAnalyzer(edges.EdgeVelocityAnalyzer())    

What about adding all of them?

I would put this in your script:
(This is Python, you will have to translate to MATLAB)


    # Add ALL the feature analyzers known to TrackMate, via
    # providers. 
    # They offer automatic analyzer detection, so all the 
    # available feature analyzers will be added. 
    # Some won't make sense on the binary image (e.g. contrast)
    # but nevermind.
     
    spotAnalyzerProvider = SpotAnalyzerProvider()
    for key in spotAnalyzerProvider.getKeys():
        print( key )
        settings.addSpotAnalyzerFactory( spotAnalyzerProvider.getFactory( key ) )
     
    edgeAnalyzerProvider = EdgeAnalyzerProvider()
    for  key in edgeAnalyzerProvider.getKeys():
        print( key )
        settings.addEdgeAnalyzer( edgeAnalyzerProvider.getFactory( key ) )
     
    trackAnalyzerProvider = TrackAnalyzerProvider()
    for key in trackAnalyzerProvider.getKeys():
        print( key )
        settings.addTrackAnalyzer( trackAnalyzerProvider.getFactory( key ) )
     
    trackmate.getModel().getLogger().log( settings.toStringFeatureAnalyzersInfo() )
    trackmate.computeSpotFeatures( True )
    trackmate.computeEdgeFeatures( True )
    trackmate.computeTrackFeatures( True )

@tinevez Thanks for the quick response.

Added ALL the feature analyzers known to TrackMate manually…

%Add analyzers for spot
settings.addSpotAnalyzerFactory(SpotIntensityAnalyzerFactory())
settings.addSpotAnalyzerFactory(SpotContrastAndSNRAnalyzerFactory())
settings.addSpotAnalyzerFactory(SpotRadiusEstimatorFactory())
settings.addSpotAnalyzerFactory(SpotMorphologyAnalyzerFactory())
settings.addSpotAnalyzerFactory(SpotContrastAnalyzerFactory())
settings.addSpotAnalyzerFactory(ManualSpotColorAnalyzerFactory())

% Add analyzers for edges
settings.addEdgeAnalyzer(edges.EdgeTargetAnalyzer())
settings.addEdgeAnalyzer(edges.EdgeTimeLocationAnalyzer())
settings.addEdgeAnalyzer(edges.EdgeVelocityAnalyzer())
%settings.addEdgeAnalyzer(edges.EdgeMeanLocationAnalyzer())
settings.addEdgeAnalyzer(ManualEdgeColorAnalyzer())

% Add an analyzer for some track features, such as the track mean speed.
settings.addTrackAnalyzer(TrackSpeedStatisticsAnalyzer())
settings.addTrackAnalyzer(TrackDurationAnalyzer())
settings.addTrackAnalyzer(TrackIndexAnalyzer())
settings.addTrackAnalyzer(TrackLocationAnalyzer())
settings.addTrackAnalyzer(TrackBranchingAnalyzer())
settings.addTrackAnalyzer(TrackSpotQualityFeatureAnalyzer())

After calling TrackMate,

trackmate = TrackMate(model, settings);

I have added

trackmate.getModel().getLogger().log(settings.toStringFeatureAnalyzersInfo());
trackmate.computeSpotFeatures(1);
trackmate.computeEdgeFeatures(1);
trackmate.computeTrackFeatures(1);

Its running fine except follwing diplay configurations

 displayer.setDisplaySettings(TrackMateModelView.KEY_TRACK_COLORING, ...
   PerTrackFeatureColorGenerator(trackmate.getModel(), ...
   TrackIndexAnalyzer.TRACK_INDEX))

displayer.setDisplaySettings(TrackMateModelView.KEY_TRACK_COLORING, ...
  PerTrackFeatureColorGenerator(trackmate.getModel(), ...
  TrackDurationAnalyzer.TRACK_DURATION))

ERROR:

Error using TrackMate_MatlabScript_Mar2020 (line 255)
Java exception occurred:
java.lang.NullPointerException
	at
        fiji.plugin.trackmate.visualization.PerTrackFeatureColorGenerator.refreshColorMap(PerTrackFeatureColorGenerator.java:140)
	at fiji.plugin.trackmate.visualization.PerTrackFeatureColorGenerator.setFeature(PerTrackFeatureColorGenerator.java:76)
	at fiji.plugin.trackmate.visualization.PerTrackFeatureColorGenerator.<init>(PerTrackFeatureColorGenerator.java:55)

@tivenez As per your suggestion, I have translated your code to Matlab. As follows:

import java.util.ArrayList

spotAnalyzerProvider = fiji.plugin.trackmate.providers.SpotAnalyzerProvider();
for i = 1:java.util.ArrayList.size(spotAnalyzerProvider.getKeys())
    settings.addSpotAnalyzerFactory...
        (spotAnalyzerProvider.getFactory...
        (spotAnalyzerProvider.getKeys().get(i-1)));
end

edgeAnalyzerProvider = fiji.plugin.trackmate.providers.EdgeAnalyzerProvider();
for i = 1:java.util.ArrayList.size(edgeAnalyzerProvider.getKeys())
    settings.addEdgeAnalyzer...
        (edgeAnalyzerProvider.getFactory...
        (edgeAnalyzerProvider.getKeys().get(i-1)));
end

trackAnalyzerProvider = fiji.plugin.trackmate.providers.TrackAnalyzerProvider();
for i = 1:java.util.ArrayList.size(trackAnalyzerProvider.getKeys())
    settings.addTrackAnalyzer...
        (trackAnalyzerProvider.getFactory...
        (trackAnalyzerProvider.getKeys().get(i-1)));
end

The output ‘Settings’ turned out as follows:

val =

Image data:
For the image named: HeLa_CLIP-MAP2K1_LP20_20ms_Cell002.nd2.
Matching file HeLa_CLIP-MAP2K1_LP20_20ms_Cell002.nd2 in folder: /Users/goswamid2/Documents/Data/2020-03-12/
Geometry:
  X =    0 -   99, dx = 0.160000
  Y =    0 -   99, dy = 0.160000
  Z =    0 -    0, dz = 1.00000
  T =    0 -  999, dt = 0.0200000

Spot detection:
Detector: fiji.plugin.trackmate.detection.LogDetectorFactory@6b9ba1a6.
Detector settings:
{TARGET_CHANNEL=1.0, THRESHOLD=30.0, RADIUS=0.5, DO_SUBPIXEL_LOCALIZATION=true, DO_MEDIAN_FILTERING=false}

Spot feature analyzers:
 - Manual spot color analyzer provides: Spot color; is manual.
 - Spot descriptive statistics provides: Mean, Median, Min, Max, Total int., Stdev.
 - Spot radius estimator provides: Diam.
 - Spot contrast and SNR provides: Constrast, SNR.
Edge feature analyzers:
 - Edge target provides: Source ID, Target ID, Cost.
 - Edge mean location provides: T, X, Y, Z.
 - Edge velocity provides: V, D.
 - Manual edge color analyzer provides: Edge color; is manual.
Track feature analyzers:
 - Branching analyzer provides: N spots, Gaps, Longest gap, Splits, Merges, Complex.
 - Track duration provides: Duration, T start, T stop, Displacement.
 - Track index provides: Index, ID.
 - Track location provides: X, Y, Z.
 - Velocity provides: Mean V, Max V, Min V, Median V, V std.
 - TRACK_SPOT_QUALITY provides: Mean Q, Max Q, Min Q, Median Q, Q std.

Initial spot filter:
Initial quality filter value: 0.0.

Spot feature filters:
Set with 1 spot feature filters:
 - QUALITY > 1.0

Particle linking:
Tracker: fiji.plugin.trackmate.tracking.sparselap.SparseLAPTrackerFactory@4430fb25.
Tracker settings:
{MAX_FRAME_GAP=2, ALTERNATIVE_LINKING_COST_FACTOR=1.05, LINKING_FEATURE_PENALTIES={}, LINKING_MAX_DISTANCE=0.5, GAP_CLOSING_MAX_DISTANCE=0.5, MERGING_FEATURE_PENALTIES={}, SPLITTING_MAX_DISTANCE=15.0, BLOCKING_VALUE=Infinity, ALLOW_GAP_CLOSING=true, ALLOW_TRACK_SPLITTING=false, ALLOW_TRACK_MERGING=false, MERGING_MAX_DISTANCE=15.0, SPLITTING_FEATURE_PENALTIES={}, CUTOFF_PERCENTILE=0.9, GAP_CLOSING_FEATURE_PENALTIES={}}

Track feature filters:
Set with 1 track feature filters:
 - TRACK_DURATION > 0.0

As per your suggestions, I have also added the following:

%--------------------
% Instantiate plugin
%--------------------
    
trackmate = TrackMate(model, settings);

%-----------------------------------
% Force calculation of all features
%-----------------------------------
trackmate.getModel().getLogger().log(settings.toStringFeatureAnalyzersInfo());
trackmate.computeSpotFeatures(1);
trackmate.computeEdgeFeatures(1);
trackmate.computeTrackFeatures(1);

The script is running fine without the ‘display configurations’ settings

displayer.setDisplaySettings(TrackMateModelView.KEY_TRACK_COLORING, ...
  PerTrackFeatureColorGenerator(trackmate.getModel(), ...
  TrackDurationAnalyzer.TRACK_DURATION))

Error log:

Error using TrackMate_MatlabScript_Mar2020 (line 272)
Java exception occurred:
java.lang.NullPointerException
	at
        fiji.plugin.trackmate.visualization.PerTrackFeatureColorGenerator.refreshColorMap(PerTrackFeatureColorGenerator.java:140)
	at fiji.plugin.trackmate.visualization.PerTrackFeatureColorGenerator.setFeature(PerTrackFeatureColorGenerator.java:76)
	at fiji.plugin.trackmate.visualization.PerTrackFeatureColorGenerator.<init>(PerTrackFeatureColorGenerator.java:55)

Whereas following display configuration works just fine

%---------------------------
% Configure display settings
%---------------------------

displayer.setDisplaySettings(TrackMateModelView.KEY_SPOT_COLORING, ...
  SpotColorGeneratorPerTrackFeature(trackmate.getModel(), ...
  fiji.plugin.trackmate.Spot.QUALITY))

Any idea why is it not working. Thanks. @imagejan

That line is:

… where apparently generator is null, which gets initialized here:

This makes me assume that there’s a problem with either 1) the SciJava Context initialization/retrieval, or with 2) SciJava OptionService retrieval, and the problem seems to be specific to the ImageJ-Matlab bridge (assuming that you’re using that, and not Miji, where it would be no surprise it doesn’t work).

Needs further digging…

@imagejan thanks for the prompt response.

Yes, I suppose. I’m using ImageJ-Matlab bridge, I think.

To confirm, my initial imports as follows

javaaddpath ('/Users/goswamid2/Documents/Matlab/mij.jar');
addpath(genpath('/Applications/Fiji.app/scripts'));
addpath ('/Users/goswamid2/Documents/Matlab Scripts/bfmatlab');
addpath('/Users/goswamid2/Documents/Matlab Scripts/jheapcl/jheapcl');
ImageJ;

%----------------------------------
% Import Fiji and TrackMate classes
%----------------------------------
import ij.*
import loci.plugins.BF.*
import java.util.HashMap
import java.util.ArrayList
import fiji.plugin.trackmate.*
import fiji.plugin.trackmate.settings.*
import fiji.plugin.trackmate.detection.*
import fiji.plugin.trackmate.features.*
import fiji.plugin.trackmate.features.track.*
import fiji.plugin.trackmate.features.track.edges.*
import fiji.plugin.trackmate.features.spot.*
import fiji.plugin.trackmate.features.manual.*
import fiji.plugin.trackmate.tracking.*
import fiji.plugin.trackmate.tracking.sparselap.*
import fiji.plugin.trackmate.util.TMUtils.*
import fiji.plugin.trackmate.SelectionModel.*
import fiji.plugin.trackmate.visualization.hyperstack.*
import fiji.plugin.trackmate.visualization.*
import fiji.plugin.trackmate.visualization.trackscheme.*
import fiji.plugin.trackmate.providers.*
import fiji.plugin.trackmate.visualization.PerTrackFeatureColorGenerator.*

I am not calling ‘Miji’ at all in the whole script.

%----------------------------------
% Read and display image file
%----------------------------------
imps = openImagePlus(filelist{j});
imp = imps(1);
imp.show();

I must mention that imps = openImagePlus(filel_path); is not working without adding mij.jar to matlab script. I dont know the reason. Command should be usable from import loci.plugins.BF.*

Forgot to mention that following display configuration works just fine

%---------------------------
% Configure display settings
%---------------------------

displayer.setDisplaySettings(TrackMateModelView.KEY_SPOT_COLORING, ...
  SpotColorGeneratorPerTrackFeature(trackmate.getModel(), ...
  fiji.plugin.trackmate.Spot.QUALITY))