Custom result table size limit ? Window manager objects

Hi,
I have a piece of a macro that will go all 616 slices of a stack (32bit, 4.4GB), creates a histogram for each slice, save it as a tiff image, get the histogram’s values and counts, and add them to a custom table such that the table will have one unique Values column and 616 Count column (Count1, Count2,…).
Everything runs smoothly until slice 460 where the macro stops and I get this message at the bottom of the macro text editor:

java.lang.InternalError: The current process has used all of its system allowance of handles for Window Manager objects.

at sun.awt.windows.WToolkit.eventLoop(Native Method)
at sun.awt.windows.WToolkit.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

Here is my code (part of a much bigger macro, but this is where it stops):

//histogram of stack and slices
print("    Creating histograms for the stack and saving....");
selectWindow(stackNaN);
run("Histogram", "bins="+stackMax+" x_min=0 x_max="+stackMax+" y_max=Auto stack");
histogTitle = getTitle();
saveAs("Tiff", dir6NaN+ "Histog_Tissue_Intensity_Stack.tif");
close();

table1 = "Results";
Table.create(table1);
print("    Creating histograms for each slice of the stack and saving....");
for(slice=1; slice<=nSlices; slice++){
	selectWindow(stackNaN);
	setSlice(slice);
	run("Histogram", "bins="+stackMax+" x_min=0 x_max="+stackMax+" y_max=Auto");
	saveAs("Tiff", dir6NaN+"Histog_Tissue_Intensity_Slice"+slice+".tif");
	close();
	nBins=stackMax;
	histMin=0;
	histMax=stackMax;
	getHistogram(values, counts, nBins,histMin, histMax);
	selectWindow("Results");
	Table.setColumn("Value", values);
	Table.setColumn("Count"+slice,counts);
	Table.update;
}

I am not sure if the issue is related to the table or to something else. Can anyone tell me what I am doing wrong? Thanks in advance.

  1. I don’t understand why you have the first line within the for-loop.
    Maybe this should go outside (before) the for-loop?
    Table.setColumn("Value", values);
    Maybe try this.

  2. …and could you also try to limit the for-loop to, e.g. for(slice=1; slice<=3; slice++){
    And check that the Table in fact looks the way you want it to look?

  3. Does run("Histogram", ...) maybe produce a window? Do you run the macro with setBatchMode( true )? If so, maybe try it without setBatchMode( false ) and check whether there are now 616 Histogram windows appearing. If that’s the case maybe record yourself closing them and add this to the macro code otherwise all those open Histogram Windows may lead to the crash

Hi Christian,

Thank you for your reply.

  1. Moving Table.setColumns(“Value”, values); outside the loop didn’t change anything. I even inactivated “Table.update”. Same thing.
  2. runing a smaller number of loops gives me the table I want without any visible issues
  3. run(“histogram”,…) does produce a window that is saved as tiff and then closed righ away before the table is created. There are no extra windows opened when that macro run, only the stack it is working on and the temporary histogram window.
  4. I tracked the Memory usage and it is below 3%

When the loop reached the image 460 and try to add histogram values to a new column in the table, the macro stops and this message appears:

java.lang.InternalError: The current process has used all of its system allowance of handles for Window Manager objects.
at sun.awt.windows.WToolkit.eventLoop(Native Method)
at sun.awt.windows.WToolkit.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

I am not sure what this error message is about. It is not a regular imageJ/Fiji error message. But it happens only during the creation of the table.

Any other ideas?

Somehow I don’t think it has to do with the table, but rather that there are too many windows open. That’s why my best bet was in fact that there are many Histogram Windows opened, but I see now that you in fact have a close(); call in your code.

  1. Maybe move the Table.update call outside of the loop?
  2. Did you try adding setBatchMode( true ); to the beginning of the macro?

I tried with Table.update commented out, moved outside the loop and in both cases I still get the same error. I also run the macro with setBatchMode() true and false. Same error. It almost looks like the macro? Windows? java? is not really closing the windows, even though they don’t show on the screen

1 Like

I agree, that what it feels like. Sorry, I am also out of ideas now :frowning:

The error message displays that you are running out of OS handles.

java.lang.InternalError: The current process has used all of its system allowance of handles for Window Manager objects.

Quite similar to SWT:

Note that every OS has an amount of handles which can be used:

I know these kind of errors by using SWT for a long time which uses JNI (as well as AWT of ImageJ).

Color resources, Fonts, Buttons, etc. often responsible for an out of handle error (e.g., in a table) which have to be disposed explicitly.

For SWT see this old article (in the meantime colors and fonts with SWT don’t need to be disposed anymore):

For SWT this can be monitored with Sleak:

I can only guess that the AWT table is responsible (maybe by creating a massive amount of fonts for each entry) for this error.

So you could try to save and close the first table after an amount of slices (which hopefully frees the handles) and then create a new table with the second part of slices.

2 Likes

Interesting! I did not know this was a concern.

I think the ProcessExplorer can be used to investigate which causes the massive use of handles for Windows:

Or Process Hacker:

See:

For Linux here a some commands:

@Bio7, @Christian_Tischer

Thank you for your answers.

I have actually narrowed down the problem. It happens with the repeated creation and closing of an histogram WHILE setBatchMode is true.

I used this macro to test:

stackMax = 255;
//dir6NaN = "D:/USERS/DavidChatelet/IssuesTest_20201106/TEST/";
setBatchMode(true);

for(i=1; i<=10; i++){
	for(n=1; n<=nSlices; n++){
		print("Round "+i+" - histogram image "+n+" / "+nSlices);
		setSlice(n);
		//run("Duplicate...", "title=temp");
		run("Histogram", "bins="+stackMax+" x_min=0 x_max="+stackMax+" y_max=Auto");
		wait(500);
		//saveAs("Tiff", dir6NaN+"Histog_Tissue_Intensity_Slice"+n+".tif");
		//run("Duplicate...", "title=temp");
		//close();
		close("Histo*");
		//close("temp");
		wait(500);
	}
}
exit("DONE");

When setBatchMode(true) is commented out, the macro runs trough more than 3 rounds without issue.
When setBatchMode(true) is active, the macro runs until round 2 - slice 190ish before it stops and tells me that all allowed handles have been used.
When I duplicate the slice and create an histogram, same story.
But when I duplicate the slice and close it without creating an histogram, no errors appear, even with setBatchMode(true) activated.

Is that an ImageJ bug that can be fixed? Not sure what to do now…

I also don’t know and it may not be easy. If you just want to get on with your project I recommend using a software like R to create and save the histogram plots from your saved table; they will render much more beautiful anyway :wink:

I would file a bug report to @Wayne with a complete reproducible example (create a stack example in your macro!).
Try to use the application mentioned before to identify which not closed handles causing the error.

1 Like

That is true. I may have to do that in the end. Thank you for your help.

I will do that. Thank you for your help.

Hi @davidC,
You’re getting this error because ImageJ, in batch mode, is unexpectedly creating HistogramWindow objects. Fixing this bug will require a major rewrite of the HistogramWindow class, which will take a couple of days.

3 Likes

Hi @Wayne,
I am glad this can be fixed. I will wait for the update. Thank you.

Hi @Wayne, just wondering:

  • batchMode = true: I think @davidC wants to save the HistogramWindows as images to files, would that still work if they are not created?
  • batchMode = false: @davidC is closing the histogram window in every step of the for loop close("Histo*");, but he still gets the error; is that then another issue?

This bug is fixed in the ImageJ 1.53g17 daily build.

This test macro reproduces the bug:

  newImage("Untitled", "32-bit ramp", 428, 355, 616);
  setBatchMode(true);
  path = "/Users/wayne/Downloads/stack/";
  i1 = 10;
  for (i=1; i<=i1; i++) {
     for (n=1; n<=nSlices; n++) {
        print(i, i1, n, nSlices);
        setSlice(n);
        run("Histogram", "bins="+255+" x_min=0 x_max="+255+" y_max=Auto");
        //saveAs("tiff", path+n+".tif");
        close;
     }
  }
1 Like