Dear all, (and especially @haesleinhuepf)
I started using and testing pyClesperanto after attending this amazing workshop at I2K2020.
I am wondering why there are two different functions (in Python) to push the 3D/2D array on the GPU. The function push does an extra step of transposing the array compared to push_zyx, I guess that’s for 2D images (maybe for display) but it adds some processing time.
Thanks for this explanation and thanks for the wonderful work!
Jules
Hey @Jules,
great to see you here and great question! The reason has something to do with a mismatch between what dimension-order ImageJ/Java coders expect in contrast to python/matlab coders.
As you recognized correctly, one of the two methods does an axis transposition. It becomes obvious when running this code:
import pyclesperanto_prototype as cle
from skimage.io import imread
# load data
image = imread('https://samples.fiji.sc/blobs.png')
print("Loaded image size: " + str(image.shape))
# push image to GPU memory
input = cle.push(image)
print("Image size in GPU: " + str(input.shape))
# push image to GPU memory and transpose axes
input = cle.push_zyx(image)
print("Image size in GPU after push_zyx: " + str(input.shape))
Output:
Loaded image size: (254, 256)
Image size in GPU: (256, 254)
Image size in GPU after push_zyx: (254, 256)
For a closer inspection, let’s take a look at blobs.gif as example:
In ImageJ, the x-coordinate goes from the left to the right in an image and blurring blobs in x direction results in this:
In Python, the first coordinate when dealing with arrays is the y coordinate. Thus, when you push/pull an image to/from the GPU and blur it in the first dimension, it looks different compared to ImageJ:
In order to deal with this, we introduced push_zyx and pull_zyx. It allows you to mimic ImageJ behaviour:
If you do all your processing in python anyway, it may not matter. As soon as you mix scikit-image, numpy, pyclesperanto and ImageJ, you will come to the point where you have to mix pull and pull_zyx in order to get correct results out. And I’m not super sure yet how this will look in the first stable release of pyclesperanto. It might make sense to exchange the functionality of push and push_zyx. This is obvious from an ImageJ-users perspective. However, from a different perspective, it might not make so much sense. I’m super confused whenever I start thinking about it 
We have github issue discussing this aspect, because I’m also no 100% sure how to deal with this conceptional glitch. 
Let me know if you have further questions or suggestions how to improve pyclesperanto!
Thanks!
Cheers,
Robert
Thank you for your detailed answer!
For the release it might make sense to pass the behavior of push as an extra-parameter so that only one push function exists (documentation will be clearer among the different wrapper) such as:
def push(image, order='xyz'):
if order == 'xyz':
# then do push
elif order == 'zyx':
# then do actual push_zyx
else
# raise ValueError('Order not understood, expected 'xyz' or 'zyx' but got ' + order)
Regarding explicit sigma_x, sigma_y, sigma_z, I think it is more flexible to have something with a singla argument that expects a list/numpy array:
if len(sigma)==1:
np.tile(sigma, (1, input.ndim))
else:
sigma = np.array(sigma)
I will keep playing with it and for sure come back with more questions.
Great idea. Does this array contain the sigma parameter in x-y-z or in z-y-x order?
Well I guess the sigma must be provided with the order specified by the parameter order, then you can reshape it for the actual processing in OpenCL knowing that the input shape is given by order.
Maybe I’m a bit naïve but I thought it would do the trick!
Naive might be the wrong term. It’s complicated. Another soft requirement would be: It would be nice if commands in ImageJ and Python take the same parameters and do the same. That’s what makes it complicated.
Thanks for your ideas! Continue speaking them out! In this way we can shape clesperanto into a community driven project