This is a tricky but interesting question.
If your tracks are robust and long enough, I would turn to MSD analysis http://tinevez.github.io/msdanalyzer/ (shameless ads).
For instance by fitting the log-log plot of the MSD vs delay, you can classify your tracks as being either brownian or transported, depending on the slope of the log-log curve (~1 = brownian, ~2 = transported).
In the link I gave above you will find a tutorial about how to perform this analysis in MATLAB. We used it in this paper to prove that our NEMO particles were attached to the membrane of cells. http://jcb.rupress.org/content/204/2/231.short