Recently, a lab member showed me that one of my plugins distributed through the Fiji updates site no longer worked. He ran into a run-time error caused by Fiji updating the jfreechart jar to a newer version, which had moved one of the api functions to a new location. Easily fixed, however, this problem shows how easily things get broken. It is extremely painful that updating all code through the Fiji updater can lead to problems that were easily caught by the compiler.
It seems that there are various ways of dealing with this. For starters, with every change to pom-scijava, an email could go out to all maintainers of plugins, asking to update their poms, compile their code, and push out a new version if anything broke. I can certainly think of ways of doing this that would be less work for plugin maintainers and more automated, but more work for the Fiji maintainers.