Issue Running python trained Stardist Models in FIJI

Hi all, I’m having issues with my pre-trained Stardist models in FIJI. I can run the H and E and Versatile Nuclei models that come with the Stardist plugin no problem, but get error messages when I run my own pre-trained model. I created the model using the google collab ZeroCostDLC4Mic notebook.

From the Console:
[INFO] Using native TensorFlow version: TF 1.15.0
[INFO] Loading TensorFlow model GenericNetwork_aea3be563cb56b8824f53a8c2382aaa5 from source file file:/C:/Users/elcid/AppData/Local/Temp/stardist_model_3961925036190340032.zip
[INFO] Caching TensorFlow models to C:\Users\elcid\DOCUME~1\Fiji.app\models
[INFO] Shape of input tensor: [-1, -1, -1, 1]
[INFO] Shape of output tensor: [-1, -1, -1, 33]
[INFO] Normalize …
[INFO] Dataset type: 32-bit signed float, converting to FloatType.
[INFO] Dataset dimensions: [1024, 1024]
[INFO] INPUT NODE:
[INFO] Mapping of tensor input:
[INFO] datasetAxes:[X, Y]
[INFO] nodeAxes:[(Time, -1), (Y, -1), (X, -1), (Channel, 1)]
[INFO] mapping:[2, 1, 0, 3]
[INFO] OUTPUT NODE:
[INFO] Mapping of tensor output:
[INFO] datasetAxes:[X, Y]
[INFO] nodeAxes:[(Time, -1), (Y, -1), (X, -1), (Channel, 33)]
[INFO] mapping:[2, 1, 0, 3]
[INFO] Complete input axes: [X, Y, Time, Channel]
[INFO] Tiling actions: [TILE_WITH_PADDING, TILE_WITH_PADDING, TILE_WITHOUT_PADDING, NO_TILING]
[INFO] Dividing image into 1 tile(s)…
[INFO] Size of single image tile: [1024, 1024, 1, 1]
[INFO] Final image tiling: [1, 1, 1, 1]
[INFO] Network input size: [1024, 1024, 1, 1]
[INFO] Processing tile 1…
[INFO] Network output size: [1024, 1024, 1, 33]
[INFO] POSTPROCESSING
[INFO] Output axes: [X, Y, Time, Channel]
[INFO] Merging tiles…
[INFO] Crop to original size…
[INFO] Output axes: [X, Y, Time, Channel]
[INFO] Displaying result image…
[INFO] img dim before wrapping into dataset: [1024, 1024, 33]
[INFO] CSBDeep plugin exit (took 3066 milliseconds)
[INFO] Using native TensorFlow version: TF 1.15.0
[INFO] Loading TensorFlow model GenericNetwork_291149f569eab76534b11b31b6346135 from source file file:/C:/Users/elcid/Desktop/iPSC_Stardist/40xCardios/Raw/myStardist_model_cardios_400.zip
org.tensorflow.TensorFlowException: Could not find SavedModel .pb or .pbtxt at supplied export directory path: C:\Users\elcid\DOCUME~1\Fiji.app\models\GenericNetwork_291149f569eab76534b11b31b6346135
at org.tensorflow.SavedModelBundle.load(Native Method)
at org.tensorflow.SavedModelBundle.access$000(SavedModelBundle.java:27)
at org.tensorflow.SavedModelBundle$Loader.load(SavedModelBundle.java:32)
at org.tensorflow.SavedModelBundle.load(SavedModelBundle.java:95)
at net.imagej.tensorflow.CachedModelBundle.(CachedModelBundle.java:44)
at net.imagej.tensorflow.DefaultTensorFlowService.loadCachedModel(DefaultTensorFlowService.java:135)
at de.csbdresden.csbdeep.network.model.tensorflow.TensorFlowNetwork.loadModel(TensorFlowNetwork.java:135)
at de.csbdresden.csbdeep.network.model.DefaultNetwork.loadModel(DefaultNetwork.java:48)
at de.csbdresden.csbdeep.network.DefaultModelLoader.loadNetwork(DefaultModelLoader.java:41)
at de.csbdresden.csbdeep.network.DefaultModelLoader.run(DefaultModelLoader.java:20)
at de.csbdresden.csbdeep.commands.GenericCoreNetwork.tryToPrepareInputAndNetwork(GenericCoreNetwork.java:523)
at de.csbdresden.csbdeep.commands.GenericCoreNetwork.initiateModelIfNeeded(GenericCoreNetwork.java:303)
at de.csbdresden.csbdeep.commands.GenericCoreNetwork.mainThread(GenericCoreNetwork.java:445)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
java.lang.NullPointerException
at de.csbdresden.csbdeep.network.DefaultInputValidator.checkForTooManyDimensions(DefaultInputValidator.java:32)
at de.csbdresden.csbdeep.network.DefaultInputValidator.run(DefaultInputValidator.java:18)
at de.csbdresden.csbdeep.commands.GenericCoreNetwork.tryToPrepareInputAndNetwork(GenericCoreNetwork.java:526)
at de.csbdresden.csbdeep.commands.GenericCoreNetwork.initiateModelIfNeeded(GenericCoreNetwork.java:303)
at de.csbdresden.csbdeep.commands.GenericCoreNetwork.mainThread(GenericCoreNetwork.java:445)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[INFO] CSBDeep plugin exit (took 4923 milliseconds)
[ERROR] Module threw exception
java.lang.NullPointerException
at de.csbdresden.stardist.StarDist2D.splitPrediction(StarDist2D.java:338)
at de.csbdresden.stardist.StarDist2D.run(StarDist2D.java:307)
at org.scijava.command.CommandModule.run(CommandModule.java:199)
at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Any help would be greatly appreciated.

Aidan

Do you use model.export_TF() at the end of your python script? It generates a zip file which you need to provide. Providing the folder is not enough.

Let us know and if that’s the case, can you share that zip file for further testing?

Best

[edit] changed model.exportTF() to model.export_TF()

1 Like

Hi Oburri,

Thanks for the response. When I run in the ZeroCostDLC4Mic notebook this is the code for the model export:

#@markdown ###Name of the model and path to model folder:

model_name = “Aidan7_8” #@param {type:“string”}

model_path = “/content/gdrive/My Drive/Colab Notebooks/Aidan7_8” #@param {type:“string”}

#trained_model = model_path

I’ve re-trained a model (a bad one just a quick training to generate a new zipfile to send you) and I’ve attached it here. I am able to run the model again in the ZeroCostDLC4Mic, but unable to run it in FIJI.

Also in case it helps with the diagnosis, here is the exception error from FIJI:

(Fiji Is Just) ImageJ 2.0.0-rc-69/1.52p; Java 1.8.0_172 [64-bit]; Windows 10 10.0; 236MB of 24409MB (<1%)

java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NullPointerException
at net.imagej.legacy.LegacyService.runLegacyCompatibleCommand(LegacyService.java:307)
at net.imagej.legacy.DefaultLegacyHooks.interceptRunPlugIn(DefaultLegacyHooks.java:166)
at ij.IJ.runPlugIn(IJ.java)
at ij.Executer.runCommand(Executer.java:137)
at ij.Executer.run(Executer.java:66)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at net.imagej.legacy.LegacyService.runLegacyCompatibleCommand(LegacyService.java:303)
… 5 more
Caused by: java.lang.NullPointerException
at de.csbdresden.stardist.StarDist2D.splitPrediction(StarDist2D.java:338)
at de.csbdresden.stardist.StarDist2D.run(StarDist2D.java:307)
at org.scijava.command.CommandModule.run(CommandModule.java:199)
at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
… 1 more

.Zip attachment of Stardist Model:
Aidan7_8.zip (18.5 MB)

Cheers,
Aidan

No, it wasn’t used.

This is the model folder as used in Python, which doesn’t work in Fiji.
After training, you need to call model.exportTF(), which will create an additional file called TF_SavedModel.zip to be used in Fiji.

$ unzip -l Aidan7_8.zip
Archive:  Aidan7_8.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
     1051  07-08-2020 13:43   Aidan7_8/config.json
    66587  07-08-2020 13:48   Aidan7_8/Quality Control/lossCurvePlots.png
     1937  07-08-2020 13:47   Aidan7_8/Quality Control/training_evaluation.csv
  5771528  07-08-2020 13:46   Aidan7_8/weights_best.h5
  5771528  07-08-2020 13:46   Aidan7_8/weights_last.h5
       40  07-08-2020 13:47   Aidan7_8/thresholds.json
  9275901  07-08-2020 13:44   Aidan7_8/events.out.tfevents.1594241080.e1ec627b6260
---------                     -------
 20888572                     7 files

Best,
Uwe

2 Likes

Hi Uwe,

Thanks for the information, that definitely explains it. I’m attempting to add the call model.exportTF() after the model is trained and I’m receiving a call back error:

AttributeError: ‘StarDist2D’ object has no attribute ‘exportTF’

I’ll keep trying to figure this out.

Best,
Aidan

Sorry it’s model.export_TF().

1 Like

This worked! Got an exported model that worked in FIJI.

Thanks all very much for the help!

Best,
Aidan