Weird array behaviour. Bug?

I’ve noticed weird behaviour of the following code. The code intends to distill certain elements from an array into another array. Problem surfaced when the number of array elements was tested and found incorrect.
When debugging this code in ImageJ1, with adding the third item (ashes[2]), a fourth item "undefined" is also added to the array. This causes a wrong decision when subsequently testing for the number of array elements.

Step the code and click the ashes[] array in the debug window shwoing the array details to see a mysterious "undefined" element has been inserted. Swapping elements in the input array does not affect the bug, neither does the number (ie. > 3) of elements in the input array matter, it always seems to happen after adding the third element (array index 2). There are no gremlins in the text elements of the input array. Most likely, I overlook the obvious.

The minimal code, using variables to keep an eye on values in the Debug window:

arrayOfNames= newArray(
	"488_GSD_001_ash.tif",
	"647_GSD_001_ash.tif",
	"488_GSD_002_ash.tif",
	"647_GSD_002_ash.tif",
	"488_GSD_001_ash.tsf"
	);
	setOption("ExpandableArrays", true);
	ashes=newArray();ashIndex=0;
	for(i=0;i<arrayOfNames.length;i++){
		thisName = arrayOfNames[i];
		match =( indexOf(thisName,"_ash.tif") > -1 );
		if(match == 1){
			ashes[ashIndex]=thisName;
			ashIndex++;
		}
	}

This posting is not about a workaround (which would be testing ashIndex rather than ashes.length), it is about why this goes wrong?

Hello Eljonco -

I believe that this is a feature rather than a bug.

When growing arrays (depending on the implementation) you
have to allocate new space for the expanded array and copy
the values. This has a cost. So typical expandable-array
implementation have a growth policy that grows the array
by increasingly large chunks so as to reduce the number
times the elements are copied.

(For example, c++'s (standard library) vectors make a
distinction between the size() of a vector and its
capacity(). The former is the number of elements that
have been added to the vector (and can be indexed), while
the latter is how many elements the vector can hold before
it needs to be reallocated.)

If you run this macro:

setOption ("ExpandableArrays", true);
exArray = newArray();
for (i = 0; i < 100; i++) {
  j = i + 1;
  exArray[i] = i;
}

and watch the size of exArray in the Debug window, you will see
that when it needs more space it grows as:

1, 2, 4, 7, 11, 17, 26, 40, 61, 92, 139, …

I should note that for me, running Fiji version
“ImageJ 2.0.0-rc-69/1.52i”, when I print out exArray.length, I
get the number of elements I have added to it, and not the larger
size (“capacity”) that is displayed in the debug window. (Also,
for me, the ghost values in my numerical array display as NaN,
rather than as “undefined”, as they do in your string array.

Thanks, mm

2 Likes

Thanks @mountain_man for this excellent explanation. I had noticed my macro wasn’t running properly in FIJI (and the NaNs) so I took ik back to ImageJ. Whilst single-stepping the code, I thought I had found the source of the problem and worked it back to a minimal example.

With your explanation I had expected ImageJ to keep the memory reservation for the array out of sight of the user. Apparently not. The Mac at home indeed shows (ImageJ 1.52d) NaN, But I’m sure I did not make up the “undefined” so probably there is an other version of ImageJ running on my other machine.

Hello Eljonco -

Let me add a couple of clarifications, below:

First, no, you didn’t make up “undefined.” I wasn’t fully
clear in my earlier post. I do see ghost values of the
character string undefined when working with arrays
of strings. It’s with arrays of numbers that I see ghost
values of NaN.

Second, “memory reservation” is kept mostly out of sight,
in that arr.length returns the “correct” value, by which
I mean, using the c++ terminology from my earlier post, that
.length returns the array’s size(), not its capacity().

The only two ways I can see the “memory reservation” is in
the debug window, and by indexing into the array beyond its
size(). So, without seeing all of your code, I don’t see
where your bug is. If you loop over your expandable arrays
using arr.length, I don’t see how you could run into trouble.

This macro illustrates my two points:

setOption ("ExpandableArrays", true);
numArr = newArray();
strArr = newArray();
for (i = 0; i < 5; i++) {
  numArr[i] = i;
  strArr[i] = "str_" + i;
}
print ("numArr.length =", numArr.length, ", stdArr.length =", strArr.length);
for (i = 0; i < 7; i++)
  print ("i =", i, "numArr[i] =", numArr[i], ", strArr[i] =", strArr[i]);
// print ("numArr[7] =", numArr[7]);  // "Index (7) out of 0-6 range" error

Thanks, mm

1 Like