Help with convert to pyramidal OME-TIFF needed when using bioformats2raw

Hi all,

I tried the bioformats2raw tool successfully on my Windows 10 system directly from the command line. And now I tried to automate this using Python but get stuck. Here is the code I use

import os
from pathlib import Path
import subprocess
import shlex


def get_fpath_woext(filepath):

    # create empty string
    real_extension = ''

    # get all part of the file extension
    sufs = Path(filepath).suffixes
    for s in sufs:
        real_extension = real_extension + s

    # remove real extension from filepath
    filepath_woext = filepath.replace(real_extension, '')
    return filepath_woext


bf2raw_dir = r"bioformats2raw\bin"
bf2raw = os.path.join(bf2raw_dir, "bioformats2raw")

print("Path bioformats2raw : ", bf2raw)
print("bioformats2raw found : ", os.path.exists(bf2raw))

czifile = r"C:\Temp\input\OverViewScan_8Brains.czi"
tmpfolder = get_fpath_woext(czifile)

# change working directory
os.chdir(bf2raw_dir)

# run command
cmd = 'bioformats2raw ' + '"' + czifile + '" ' + '"' + tmpfolder + '"' + ' --resolutions 4 --compression=raw'
print('Command String : ', cmd)
os.system(cmd)
print("Working Directory : ", os.getcwd())

cmd_args = shlex.split(cmd)
print(cmd_args)
completed = subprocess.run(cmd_args)
print(completed)

and this is the respective output:

(stardist) C:\Users\m1srh\Documents\Apeer_Modules\create_pyramidal_tiff_0ec109bd-6b11-4114-b2b6-b9cfec06d028>C:/ProgramData/Anaconda3/envs/stardist/python.exe c:/Users/m1srh/Documents/Apeer_Modules/create_pyramidal_tiff_0ec109bd-6b11-4114-b2b6-b9cfec06d028/test_cmd.py 
Path bioformats2raw :  bioformats2raw\bin\bioformats2raw
bioformats2raw found :  True
Command String :  bioformats2raw "C:\Temp\input\OverViewScan_8Brains.czi" "C:\Temp\input\OverViewScan_8Brains" --resolutions 4 --compression=raw
The input line is too long.
The syntax of the command is incorrect.
Working Directory :  C:\Users\m1srh\Documents\Apeer_Modules\create_pyramidal_tiff_0ec109bd-6b11-4114-b2b6-b9cfec06d028\bioformats2raw\bin
['bioformats2raw', 'C:\\Temp\\input\\OverViewScan_8Brains.czi', 'C:\\Temp\\input\\OverViewScan_8Brains', '--resolutions', '4', '--compression=raw']
Traceback (most recent call last):
  File "c:/Users/m1srh/Documents/Apeer_Modules/create_pyramidal_tiff_0ec109bd-6b11-4114-b2b6-b9cfec06d028/test_cmd.py", line 46, in <module>
    completed = subprocess.run(cmd_args)
  File "C:\ProgramData\Anaconda3\envs\stardist\lib\subprocess.py", line 488, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\ProgramData\Anaconda3\envs\stardist\lib\subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "C:\ProgramData\Anaconda3\envs\stardist\lib\subprocess.py", line 1207, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified

Hi,

I just tried your example and it works fine for me.
I ran it up to line 40 os.system(cmd) which calls bioformats2raw and does the conversion.
However if I also include

cmd_args = shlex.split(cmd)
print(cmd_args)
completed = subprocess.run(cmd_args)
print(completed)

then I get the FileNotFoundError: [WinError 2] The system cannot find the file specified as well. So shlex.split(cmd) seems to mess something up. But why do you have to do this anyway?

Kind Regards,
Dominik

Hi @dominikl,

this is exactly the problem. It works for you so the code itself seems to be OK, but when I run it it shows those strange errors.

Out of desperation I tried the shlex approach, just to make sure. Maybe it does not work because of the same “mysterious” issue or maybe I just use it wrong.

But when I run bioformats2raw from the Windows CMD or even from with ZEN it works fine. It just does bot work when calling is via os.system, which I do not understand so far.

Thanks for keeping us posted, @sebi06. Based on the lovely results in https://twitter.com/sebisabs/status/1316419199225475077, shall we assume that this is working for you well enough at the moment?

~J

Good question…

It works from within ZEN etc.but my plan was to create an Apeer module from it as well (based on python).
This is were it fails for unknowns reasons due to this strange errors.

Sure I would be happy to solve this issue somehow but it is not supercritical.

Big :thumbsup: to having it wrappable for Apeer. Any help reproducing on our side appreciated. I know elsewhere I’ve needed to be more explicit about what in the environment gets passed to subprocesses:

~J