Which protocol are you following? It is likely to be out of date. The documentation at bonej.org that you quote is for BoneJ1, the legacy version that is no longer maintained. Up to date documentation for BoneJ2 (the version that the ImageJ updater installs) is here: https://imagej.github.io/BoneJ
Optimise Threshold has been removed from BoneJ2 because it did little more than what is now possible in the built-in threshold tool. Make sure to click “Stack Histogram” and “Auto”, and you have the same result. As a histogram-based method, the autothresholder is sensitive to relative amounts of light and dark pixels and the results may vary as a function of specimen shape, image size and bone volume fraction (BV/TV). If your imaging conditions are consistent it may be better to stick with a single threshold value that you apply to all specimens.
It probably can’t tell that you have a cylindrical specimen in a cuboidal image. The TV value is the volume of your image, but in your case TV should be the volume of the cylindrical bone sample / cylindrical tomographic field. You could estimate TV mathematically, as πr 2h × pixel volume where it looks like r for this image is (126-2)/2 = 62 pixels and h is the number of slices in the image,and pixel volume is pixel width × height × depth (from Image > Properties). BV is the same as reported in the results table (BV is just a count of foreground pixels multiplied by pixel spacing). Then calculate BV/TV using the measured value for BV and your calculated value for TV.
At least don’t use SMI or trust its results. It doesn’t properly take account of concave curvatures, which will be common in your specimen. Ellipsoid Factor is one approach to estimate the rod/plate like nature of trabecular bone that you might like to try. SMI is not included in BoneJ2.