Bfconvert: input files larger than physical RAM

Usage & Issues
,
#1

Hello,

We routinely collect large area, multichannel images that we wish to convert into pyramidal tiffs from the proprietary formats of the microscope vendors. These are collected as multiple tiles, then stitched into a single large image. We often have files of 10s to 100s of GB in size.

For smaller files (single digit GB in size) I can call bfconvert with the BF_MAX_MEM option set to a large value, say 26 GB. For larger files, bfconvert reports that the java heap memory is insufficient (java.lang.OutOfMemoryError: Java heap space).

What can be done if the file size is larger than the physical memory of the computer? Is there a way for bfconvert to use something akin to a virtual stack (as in FIJI)?

Thanks,
Will

#2

Hi Will, what sort of dimensions are these images typically? Are the input images stored using compression? Also are you using the tiling options when doing the conversion ( bfconvert -tilex 512 -tiley 512 /path/to/input output.tiff )?

#3

Hi David.

Our average-sized files are on the order of 5000x4400 pixels and 4 colors. But we are collecting multiplexed images, so file sizes can be multiples of that. The larger ones are 15 GB or so, but we do have some tissue images of 100+ GB size. I don’t believe the input data is compressed.

For the conversion I use:
bfconvert -tilex 512 -tiley 512 -noflat -pyramid-resolutions 4 -pyramid-scale 2 -option BF_MAX_MEM 26000 /path/to/input output.ome.tiff

Thanks,
Will

#4

Are you able to open and view the same files when using virtual stack in FIJI?

#5

The smaller files (5k x 4.4k pixels, 4 channels) open in FIJI with the virtual stack. But larger ones (41k x 27k pixels, 4 channels) don’t. FIJI throws an Array Size Too Large error. I set FIJI’s memory size to 26 GB.

With the bf_max_mem set to 26 GB, I can convert the 40k x 27k pixel file to a pyramid tiff. When I try to open the pyramid tiff file, I can open each of the lower resolution tiffs in FIJI. But I can’t open the original level tiff.

#6

Can you paste the output of showinf -nopix for one of the files that is failing bfconvert?

#7

Here is the showing output for the file that initially caused trouble. The file is about 9GB in size. bfconvert was unable to process it with 16GB java heap space, but it was able to convert it with 26GB heap space.

heinzwf$ showinf -nopix /Users/heinzwf/Desktop/BigFile.nd2
Checking file format [Nikon ND2]
Initializing reader
ND2Reader initializing /Users/heinzwf/Desktop/BigFile.nd2
Searching for blocks
Parsing block ‘ND2 FILE SIG’ 0%
Parsing block ‘ImageCalibra’ 0%
Parsing block ‘ImageMetadat’ 0%
Parsing block ‘ImageDataSeq’ 0%
Parsing block ‘ImageDataSeq’ 0%
Parsing block ‘ImageTextInf’ 99%
Parsing block ‘ReportObject’ 99%
Parsing block ‘CustomData|C’ 99%
Parsing block ‘CustomData|G’ 99%
Parsing block ‘CustomDataVa’ 99%
Parsing block ‘CustomDataVa’ 99%
Parsing block ‘CustomDataVa’ 99%
Parsing block ‘CustomData|A’ 99%
Parsing block ‘CustomData|A’ 99%
Parsing block ‘CustomData|A’ 99%
Parsing block ‘ImageAttribu’ 99%
Parsing block 'ND2 FILEMAP ’ 99%
Initialization took 0.182s

Reading core metadata
filename = /Users/heinzwf/Desktop/BigFile.nd2
Series count = 1
Series #0 :
Image count = 4
RGB = false (1)
Interleaved = false
Indexed = true (false color, 16-bit LUT: 3 x 65536)
Width = 40676
Height = 27164
SizeZ = 1
SizeT = 1
SizeC = 4
Tile size = 40676 x 27164
Thumbnail size = 128 x 85
Endianness = intel (little)
Dimension order = XYCZT (certain)
Pixel type = uint16
Valid bits per pixel = 16
Metadata complete = true
Thumbnail series = false
-----
Plane #0 <=> Z 0, C 0, T 0
Plane #2 <=> Z 0, C 2, T 0
Plane #3 <=> Z 0, C 3, T 0

Reading global metadata
Average: 1
AverageToQuality: 0.0
Binning #1: 1x1
Binning #2: 1x1
Binning #3: 1x1
Binning #4: 1x1
Binning #5: 1x1
Binning #6: 1x1
Binning #7: 1x1
Binning #8: 1x1
Bottom: 0
Cam0Flip: false
Cam0Rotate180: false
Camera Name: Andor Zyla VSC-04649
Camera Settings #1:
Camera Settings #2:
Camera Settings #3:
Camera Settings #4:
Camera Type #1: Andor Zyla
Camera Type #2: Andor Zyla
Camera Type #3: Andor Zyla
Camera Type #4: Andor Zyla
Camera Type #5: Andor Zyla
Camera Type #6: Andor Zyla
Camera Type #7: Andor Zyla
Camera Type #8: Andor Zyla
CameraFamilyName: Neo
CameraName: VSC-04649
CameraOffset: 0.0
CameraUniqueName: VSC-04649
CameraUserName: Andor Zyla VSC-04649
CaptureMaxExposure: 30000.0
CaptureQuality: 75
ChannelColor: 65407
ChannelCount: 0
ChannelIsActive: true
ChannelName: Comp5
Color: 16711680
Columns: 2
Component Count #1: 1
Component Count #2: 1
Component Count #3: 1
Component Count #4: 1
Conversion Gain #1: Dual Gain 1/4
Conversion Gain #2: Dual Gain 1/4
Conversion Gain #3: Dual Gain 1/4
Conversion Gain #4: Dual Gain 1/4
Conversion Gain #5: Dual Gain 1/4
Conversion Gain #6: Dual Gain 1/4
Conversion Gain #7: Dual Gain 1/4
Conversion Gain #8: Dual Gain 1/4
Data: ??
DesiredTemperatureControl: 0
DynamicRange: 2
ElectronicShutteringMode: 0
EmWavelength: 0.0
EnableAutoExposure: true
EnableExposureEtc: true
ExWavelength: 0.0
Exposure #1: 500.0
Exposure #2: 500 ms
Exposure #3: 200 ms
Exposure #4: 500 ms
Exposure #5: 500 ms
Exposure #6: 500 ms
Exposure #7: 200 ms
Exposure #8: 500 ms
Exposure #9: 500 ms
Exposure time (text) #1: 0.5
Exposure time (text) #2: 0.5
Exposure time (text) #3: 0.5
Exposure time (text) #4: 0.5
FanSpeed: 1
FilterWaveLength: 558.0
FlexibleHeight: -1
Integrate: 1
Key: MV=0,TA=0,CH=1
Left: 0
LightPath #1: L100
LightPath #2: L100
LightPath #3: L100
LightPath #4: L100
LimitListedExposureMax: -1.0
LimitListedExposureMin: -1.0
Line:1; ExW:395; Power #1: 10.0; Off
Line:1; ExW:395; Power #2: 10.0; Off
Line:1; ExW:395; Power #3: 10.0; Off
Line:1; ExW:395; Power #4: 10.0; On
Line:1; ExW:405; Power #1: 0.0; Off
Line:1; ExW:405; Power #2: 0.0; Off
Line:1; ExW:405; Power #3: 0.0; Off
Line:1; ExW:405; Power #4: 0.0; Off
Line:2; ExW:440; Power #1: 0.0; Off
Line:2; ExW:440; Power #2: 0.0; Off
Line:2; ExW:440; Power #3: 0.0; Off
Line:2; ExW:440; Power #4: 0.0; Off
Line:2; ExW:488; Power #1: 2.0; On
Line:2; ExW:488; Power #2: 2.0; On
Line:2; ExW:488; Power #3: 2.0; On
Line:2; ExW:488; Power #4: 2.0; Off
Line:3; ExW:470; Power #1: 30.0; Off
Line:3; ExW:470; Power #2: 30.0; Off
Line:3; ExW:470; Power #3: 30.0; On
Line:3; ExW:470; Power #4: 30.0; Off
Line:3; ExW:561; Power #1: 98.8; Off
Line:3; ExW:561; Power #2: 98.8; Off
Line:3; ExW:561; Power #3: 98.8; Off
Line:3; ExW:561; Power #4: 98.8; Off
Line:4; ExW:508; Power #1: 5.0; Off
Line:4; ExW:508; Power #2: 5.0; Off
Line:4; ExW:508; Power #3: 5.0; Off
Line:4; ExW:508; Power #4: 5.0; Off
Line:4; ExW:640; Power #1: 94.7; Off
Line:4; ExW:640; Power #2: 94.7; Off
Line:4; ExW:640; Power #3: 94.7; Off
Line:4; ExW:640; Power #4: 94.7; Off
Line:5; ExW:555; Power #1: 20.0; Off
Line:5; ExW:555; Power #2: 20.0; On
Line:5; ExW:555; Power #3: 20.0; Off
Line:5; ExW:555; Power #4: 20.0; Off
Line:5; ExW:650; Power:100.0; Off Sutter Lambda 10-3, FilterChanger(Filter B) #1: 6 (Empty)
Line:5; ExW:650; Power:100.0; Off Sutter Lambda 10-3, FilterChanger(Filter B) #2: 5 (Red FF01-600/52)
Line:5; ExW:650; Power:100.0; Off Sutter Lambda 10-3, FilterChanger(Filter B) #3: 4 (GFP FF01-514/44)
Line:5; ExW:650; Power:100.0; Off Sutter Lambda 10-3, FilterChanger(Filter B) #4: 3 (DAPI FF01-432/36)
Line:6; ExW:640; Power #1: 30.0; On
Line:6; ExW:640; Power #2: 30.0; Off
Line:6; ExW:640; Power #3: 30.0; Off
Line:6; ExW:640; Power #4: 30.0; Off
LiveSpeedUp: 1
MaxBulbExposure: -1.0
MaxFPSLimit: 25.0
MaxFPSLimited: false
MaxSaturatedValue: -1
Microscope #1: Ti Microscope
Microscope #2: Ti Microscope
Microscope #3: Ti Microscope
Microscope #4: Ti Microscope
Microscope Settings #1:
Microscope Settings #2:
Microscope Settings #3:
Microscope Settings #4:
Modality #1: Widefield Fluorescence
Modality #2: Widefield Fluorescence
Modality #3: Widefield Fluorescence
Modality #4: Widefield Fluorescence
ModifiedAtJDN: 0
MultiViewEnabled: false
NIDAQ, MultiLaser(AOTF) #1:
NIDAQ, MultiLaser(AOTF) #2:
NIDAQ, MultiLaser(AOTF) #3:
NIDAQ, MultiLaser(AOTF) #4:
NIDAQ, Shutter(AOTF) #1: Closed
NIDAQ, Shutter(AOTF) #2: Closed
NIDAQ, Shutter(AOTF) #3: Closed
NIDAQ, Shutter(AOTF) #4: Closed
NIDAQ, Shutter(Dia) #1: Closed
NIDAQ, Shutter(Dia) #2: Closed
NIDAQ, Shutter(Dia) #3: Closed
NIDAQ, Shutter(Dia) #4: Closed
NIDAQ, Shutter(Relay) #1: Closed
NIDAQ, Shutter(Relay) #2: Closed
NIDAQ, Shutter(Relay) #3: Closed
NIDAQ, Shutter(Relay) #4: Closed
Name #1: DAPI Zyla MB
Name #2: Cy5
Name #3: Red
Name #4: Green Zyla Narrow
Name #5: DAPI Zyla MB
Negate: 0
Nikon Ti, FilterChanger(Turret1) #1: 2 (89000 - Sedat Quad - ET)
Nikon Ti, FilterChanger(Turret1) #2: 2 (89000 - Sedat Quad - ET)
Nikon Ti, FilterChanger(Turret1) #3: 2 (89000 - Sedat Quad - ET)
Nikon Ti, FilterChanger(Turret1) #4: 2 (89000 - Sedat Quad - ET)
Nikon Ti, FilterChanger(Turret2) #1: 3
Nikon Ti, FilterChanger(Turret2) #2: 3
Nikon Ti, FilterChanger(Turret2) #3: 3
Nikon Ti, FilterChanger(Turret2) #4: 3
Nikon Ti, Illuminator(Illuminator-DIA) #1: Remote Switch Off
Nikon Ti, Illuminator(Illuminator-DIA) #2: Off
Nikon Ti, Illuminator(Illuminator-DIA) #3: Remote Switch Off
Nikon Ti, Illuminator(Illuminator-DIA) #4: Off
Nikon Ti, Illuminator(Illuminator-DIA) #5: Remote Switch Off
Nikon Ti, Illuminator(Illuminator-DIA) #6: Off
Nikon Ti, Illuminator(Illuminator-DIA) #7: Remote Switch Off
Nikon Ti, Illuminator(Illuminator-DIA) #8: Off
Nikon Ti, Illuminator(Illuminator-DIA) Voltage #1: -1.0
Nikon Ti, Illuminator(Illuminator-DIA) Voltage #2: -1.0
Nikon Ti, Illuminator(Illuminator-DIA) Voltage #3: -1.0
Nikon Ti, Illuminator(Illuminator-DIA) Voltage #4: -1.0
Number of Picture Planes: 4
Numerical Aperture: 0.75
PFIsForTriggering: /Dev1/PFI0
PFS-S, Dish Type #1: Glass
PFS-S, Dish Type #2: Glass
PFS-S, Dish Type #3: Glass
PFS-S, Dish Type #4: Glass
PFS-S, mirror #1: Inserted
PFS-S, mirror #2: Inserted
PFS-S, mirror #3: Inserted
PFS-S, mirror #4: Inserted
PFS-S, offset #1: 8695
PFS-S, offset #2: 8695
PFS-S, offset #3: 8695
PFS-S, offset #4: 8695
PFS-S, state #1: Off
PFS-S, state #2: Off
PFS-S, state #3: Off
PFS-S, state #4: Off
Plane #1:
Plane #2:
Plane #3:
Plane #4:
QuantilPixels: 100
QuantilPromile: 0.1
QuantilRelative: true
Readout Mode #1: Rolling shutter at 16-bit
Readout Mode #2: Rolling shutter at 16-bit
Readout Mode #3: Rolling shutter at 16-bit
Readout Mode #4: Rolling shutter at 16-bit
Readout Mode #5: Rolling shutter at 16-bit
Readout Mode #6: Rolling shutter at 16-bit
Readout Mode #7: Rolling shutter at 16-bit
Readout Mode #8: Rolling shutter at 16-bit
Readout Rate #1: 540 MHz
Readout Rate #2: 540 MHz
Readout Rate #3: 540 MHz
Readout Rate #4: 540 MHz
Readout Rate #5: 540 MHz
Readout Rate #6: 540 MHz
Readout Rate #7: 540 MHz
Readout Rate #8: 540 MHz
ReadoutSpeed: 1
Refractive Index: 1
Right: 0
Rotate: 90.0
Rows: 2
SLxImageAttributes: 13
SLxImageTextInfo P< TextInfoItem_: TextInfoItem_1
Sample 1:
Sample 2:
Sample 3:
Sample 4:
Sensor Mode #1: Overlap
Sensor Mode #2: Overlap
Sensor Mode #3: Overlap
Sensor Mode #4: Overlap
Sensor Mode #5: Overlap
Sensor Mode #6: Overlap
Sensor Mode #7: Overlap
Sensor Mode #8: Overlap
SensorMode: 1
SetColorManually: true
Spectra/AuraII, MultiLaser(Spectra) #1:
Spectra/AuraII, MultiLaser(Spectra) #2:
Spectra/AuraII, MultiLaser(Spectra) #3:
Spectra/AuraII, MultiLaser(Spectra) #4:
Spectra/AuraII, Shutter(Spectra) #1: Active
Spectra/AuraII, Shutter(Spectra) #2: Active
Spectra/AuraII, Shutter(Spectra) #3: Active
Spectra/AuraII, Shutter(Spectra) #4: Active
Spurious Noise Filter #1: on
Spurious Noise Filter #2: on
Spurious Noise Filter #3: on
Spurious Noise Filter #4: on
Spurious Noise Filter #5: on
Spurious Noise Filter #6: on
Spurious Noise Filter #7: on
Spurious Noise Filter #8: on
SpuriousNoiseFilter: true
Temperature #1: -10.4C
Temperature #2: -10.4C
Temperature #3: -11.1C
Temperature #4: -11.1C
Temperature #5: -10.4C
Temperature #6: -10.4C
Temperature #7: -11.1C
Temperature #8: -11.1C
TextInfoItem_ #1: TextInfoItem_3
TextInfoItem_ #2: TextInfoItem_5Dimensions: (4)
TextInfoItem_ #3: TextInfoItem_8
TextInfoItem_1 #1: TextInfoItem_11
TextInfoItem_1 #2: TextInfoItem_13Plan Apo 20x DIC M N20R;;; <t1 ;(;J;
TextInfoItem_6Ando: Zyla VSC-04649
TextInfoItem_98/6/202: 10:33:26 AM
Top: 0
Trigger Mode #1: Internal
Trigger Mode #2: Internal
Trigger Mode #3: Internal
Trigger Mode #4: Internal
Trigger Mode #5: Internal
Trigger Mode #6: Internal
Trigger Mode #7: Internal
Trigger Mode #8: Internal
TypeID: 0
UpdateLPAutomatically: true
UseOneFormat: false
__m_runType: RLxIRect
bCalibrated: false
bCamCorrReq: false
bCanExecAsyncSampleGet: false
bPickFromGraph: false
bPoints: false
bValid: true
bZPosAbsolute: false
baOpticalPathsCorrections: CLxOpticalPathsCorrectionTablewstrSystemUUID9ddddb4ef6e74c5fa5854aee5e211104vecIdsSizematRowsL
baScanArea: dZoom?dAngle dXOffset dYOffsetdScanAreaAspect?bIsCrop
dCropZoom?dCropXOffsetdCropYOffsetdCropAspect?uiPointCoint
bottom: 0
cx: -1
cy: -1
dAcqTime: 0.0
dAngle: 0.0
dAspect: 1.0
dBinningX: -1
dBinningY: -1
dCalibPrecision: -1.0
dCalibration: 0.324377938419263
dCompressionParam: 0
dExposureTime: 500.0
dFadingCorr: 0.0
dFps: -1
dGainCustom: 1
dGainGrayScale: 1
dGainGrouped: 1
dGainTrueColor: 1
dGammaSrc: 1
dLampVoltage: 0.0
dMax: 2
dMin: 0
dObjCalibration1to1: 0.324377938419263
dObjectiveMag: -1.0
dObjectiveNA: 0.75
dObjectiveToPinholeZoom: 1.0
dOffsetCustom: 0
dOffsetGrayScal: 0
dOffsetGrouped: 0
dOffsetTrueColor: 0
dPinholeDiameter: -1.0
dProjectiveMag: 1.0
dRefractIndex: 1.0
dRefractIndex1: 1.0
dRefractIndex2: 1.0
dRelayLensZoom: 1.0
dScalingToIntensity: 0.0
dSensitivity: -1
dStgLgCT11: 1.0
dStgLgCT12: 0.0
dStgLgCT21: 0.0
dStgLgCT22: 1.0
dTValue: 0.0
dTemperK: -1.0
dTimeAbsolute: 2459068.1065600812
dTimeAxisCalibration: -1.0
dTimeMSec: -1.0
dWavelength: 0.0
dXPos: 16179.0
dYPos: -1263.7500000000002
dZAxisCalibration: -1.0
dZPos: 1777.2150414898128
dZoom: 1.0
eCompression: 2
ePictureXAxis: 0
ePictureYAxis: 1
ePixelType: 1
eRepresentation: 0
eTimeSource: 0
eType: 0
fMaxSrc: 1
fMinSrc: 0
iChannelSeriesIndex: -1
left: 0
m_CompanyString: Laboratory Imaging s.r.o.
m_CopyrightString: Copyright © 1991-2020 Laboratory Imaging, http://www.lim.cz
m_GrabberString: ANDOR Sona/Zyla/Neo
m_SWNameString: NIS-Elements AR
m_VersionString: 5.21.01 (Build 1483)
m_bEnableAutoContrast: false
m_bEnableAutoWhite: false
m_bGraphVerticalSplit: false
m_bGrayGraph: false
m_bMultiLaserLineUsed0-00: 1
m_bMultiLaserLineUsed0-01: 0
m_bMultiLaserLineUsed0-02: 0
m_bMultiLaserLineUsed0-03: 0
m_bMultiLaserLineUsed0-04: 0
m_bMultiLaserLineUsed0-05: 0
m_bMultiLaserLineUsed1-00: 0
m_bMultiLaserLineUsed1-01: 0
m_bMultiLaserLineUsed1-02: 0
m_bMultiLaserLineUsed1-03: 0
m_bMultiLaserLineUsed1-04: 0
m_bMultiLaser_LineUsedLineUsed0-00: 1
m_bMultiLaser_LineUsedLineUsed0-01: 0
m_bMultiLaser_LineUsedLineUsed0-02: 0
m_bMultiLaser_LineUsedLineUsed0-03: 0
m_bMultiLaser_LineUsedLineUsed0-04: 0
m_bMultiLaser_LineUsedLineUsed0-05: 0
m_bMultiLaser_LineUsedLineUsed1-00: 0
m_bMultiLaser_LineUsedLineUsed1-01: 0
m_bMultiLaser_LineUsedLineUsed1-02: 0
m_bMultiLaser_LineUsedLineUsed1-03: 0
m_bMultiLaser_LineUsedLineUsed1-04: 0
m_bParamsValid0: 1
m_bRatioViewEnabled: false
m_bShowAllComp: true
m_bShowSpectralGraph: true
m_bViewLut: true
m_dGraphOffset00: 0
m_dGraphOffset01: 0
m_dGraphOffset02: 0
m_dGraphOffset03: 0
m_dGraphOffset04: 0
m_dGraphZoom00: 1
m_dGraphZoom01: 1
m_dGraphZoom02: 1
m_dGraphZoom03: 1
m_dGraphZoom04: 1
m_dLightIrisIntensity0: 0.0
m_dLightNDFilter0: 0.0
m_dLightVoltage0: -1.0
m_dLogMaxX0: 55000.0
m_dLogMaxY0: 37500.0
m_dLogMinX0: -55000.0
m_dLogMinY0: -37500.0
m_dMultiLaserLinePower0-00: 10.0
m_dMultiLaserLinePower0-01: 0.0
m_dMultiLaserLinePower0-02: 30.0
m_dMultiLaserLinePower0-03: 5.0
m_dMultiLaserLinePower0-04: 20.0
m_dMultiLaserLinePower0-05: 30.0
m_dMultiLaserLinePower1-00: 0.0
m_dMultiLaserLinePower1-01: 2.0
m_dMultiLaserLinePower1-02: 98.80000000000001
m_dMultiLaserLinePower1-03: 94.70000000000002
m_dMultiLaserLinePower1-04: 100.0
m_dMultiLaser_PowerLinePower0-00: 10.0
m_dMultiLaser_PowerLinePower0-01: 0.0
m_dMultiLaser_PowerLinePower0-02: 30.0
m_dMultiLaser_PowerLinePower0-03: 5.0
m_dMultiLaser_PowerLinePower0-04: 20.0
m_dMultiLaser_PowerLinePower0-05: 30.0
m_dMultiLaser_PowerLinePower1-00: 0.0
m_dMultiLaser_PowerLinePower1-01: 2.0
m_dMultiLaser_PowerLinePower1-02: 98.80000000000001
m_dMultiLaser_PowerLinePower1-03: 94.70000000000002
m_dMultiLaser_PowerLinePower1-04: 100.0
m_dShutterAperture0: 0.0
m_dShutterAperture1: 0.0
m_dShutterAperture2: 0.0
m_dShutterAperture3: 0.0
m_dXYPositionX0: 17756.600000000002
m_dXYPositionY0: -5364.6
m_dZPosition0: 1879.2
m_dZPosition1: 102.00995851018729
m_dZoomPosition: 1.0
m_eFilterType0: 2
m_eFilterType1: 2
m_eFilterType2: 1
m_eFilterType3: 1
m_eLightPowerType0: 3
m_eLightRemoteSwitchControlType0: 2
m_eLightRemoteType0: 3
m_eLightStateType0: 3
m_eLightType0: 3
m_eNature: 0
m_ePFSDishType: 0
m_ePlacement: 6
m_eShutterMode0: 0
m_eShutterMode1: 0
m_eShutterMode2: 0
m_eShutterMode3: 0
m_eShutterType0: 0
m_eShutterType1: 0
m_eShutterType2: 1
m_eShutterType3: 2
m_eSpctType: 1
m_iFilterPosition0: 1
m_iFilterPosition1: 2
m_iFilterPosition2: 1
m_iFilterPosition3: 2
m_iFilterUse0: 1
m_iFilterUse1: 1
m_iFilterUse2: 0
m_iFilterUse3: 1
m_iGraphScale: 0
m_iGraphSelected: -1
m_iLightPathPosition: 1
m_iLightPathUse: 1
m_iLightPowerUse0: 1
m_iLightRemoteUse0: 1
m_iLightStateUse0: 1
m_iLightUse0: 1
m_iMicroscopeUse: 1
m_iMultiLaserUse0: 1
m_iMultiLaserUse1: 1
m_iMultiLaser_LineUsedUse0: 1
m_iMultiLaser_LineUsedUse1: 1
m_iMultiLaser_NDFilterUse0: 1
m_iMultiLaser_NDFilterUse1: 1
m_iMultiLaser_PowerUse0: 1
m_iMultiLaser_PowerUse1: 1
m_iMultilaserLineNDFilter0-00: -1
m_iMultilaserLineNDFilter0-01: -1
m_iMultilaserLineNDFilter0-02: -1
m_iMultilaserLineNDFilter0-03: -1
m_iMultilaserLineNDFilter0-04: -1
m_iMultilaserLineNDFilter0-05: -1
m_iMultilaserLineNDFilter1-00: -1
m_iMultilaserLineNDFilter1-01: -1
m_iMultilaserLineNDFilter1-02: -1
m_iMultilaserLineNDFilter1-03: -1
m_iMultilaserLineNDFilter1-04: -1
m_iMultilaser_NDFilterLineNDFilter0-00: -1
m_iMultilaser_NDFilterLineNDFilter0-01: -1
m_iMultilaser_NDFilterLineNDFilter0-02: -1
m_iMultilaser_NDFilterLineNDFilter0-03: -1
m_iMultilaser_NDFilterLineNDFilter0-04: -1
m_iMultilaser_NDFilterLineNDFilter0-05: -1
m_iMultilaser_NDFilterLineNDFilter1-00: -1
m_iMultilaser_NDFilterLineNDFilter1-01: -1
m_iMultilaser_NDFilterLineNDFilter1-02: -1
m_iMultilaser_NDFilterLineNDFilter1-03: -1
m_iMultilaser_NDFilterLineNDFilter1-04: -1
m_iNosPosition: 0
m_iNosUse: 1
m_iPFSDMState: 1
m_iPFSDMUse: 0
m_iPFSDishTypeUse: 0
m_iPFSOffsetPosition: 8695
m_iPFSOffsetUse: 0
m_iPFSState: 0
m_iPFSUse: 0
m_iShutterUse0: 1
m_iShutterUse1: 1
m_iShutterUse2: 1
m_iShutterUse3: 1
m_iXOrientation0: 1
m_iXYUse0: 0
m_iYOrientation0: -1
m_iZUse0: 0
m_iZUse1: 0
m_iZoomUse: 0
m_ibFilterActAsShutter0: 0
m_ibFilterActAsShutter1: 0
m_ibFilterActAsShutter2: 0
m_ibFilterActAsShutter3: 0
m_ibFilterExist0: 1
m_ibFilterExist1: 1
m_ibFilterExist2: 1
m_ibFilterExist3: 1
m_ibFilterPositionValid0: 1
m_ibFilterPositionValid1: 1
m_ibFilterPositionValid2: 1
m_ibFilterPositionValid3: 1
m_ibLightExist0: 1
m_ibLightIrisControllable0: 0
m_ibLightIrisIntensityValid0: 0
m_ibLightNDFilterControllable0: 0
m_ibLightNDFilterValid0: 0
m_ibLightPathExist: 1
m_ibLightPathPositionValid: 1
m_ibLightPowerExist0: 1
m_ibLightRemoteExist0: 1
m_ibLightRemoteSwitch0: 0
m_ibLightRemoteSwitchValid0: 1
m_ibLightSegmentsValid0: 0
m_ibLightStateControllable0: 1
m_ibLightStateExist0: 1
m_ibLightStateOn0: 0
m_ibLightStateValid0: 1
m_ibLightVoltageControllable0: 1
m_ibLightVoltageValid0: 1
m_ibMicroscopeExist: 1
m_ibMultiLaserExist0: 1
m_ibMultiLaserExist1: 1
m_ibMultiLaserValid0: 1
m_ibMultiLaserValid1: 1
m_ibMultiLaser_LineUsedExist0: 1
m_ibMultiLaser_LineUsedExist1: 1
m_ibMultiLaser_LineUsedValid0: 1
m_ibMultiLaser_LineUsedValid1: 1
m_ibMultiLaser_NDFilterExist0: 0
m_ibMultiLaser_NDFilterExist1: 0
m_ibMultiLaser_NDFilterValid0: 1
m_ibMultiLaser_NDFilterValid1: 1
m_ibMultiLaser_PowerExist0: 1
m_ibMultiLaser_PowerExist1: 1
m_ibMultiLaser_PowerValid0: 1
m_ibMultiLaser_PowerValid1: 1
m_ibNosExist: 0
m_ibNosPositionValid: 1
m_ibPFSDMExist: 1
m_ibPFSDMStateValid: 1
m_ibPFSDishTypeExist: 1
m_ibPFSExist: 1
m_ibPFSOffsetExist: 1
m_ibPFSOffsetPositionValid: 1
m_ibPFSStateValid: 1
m_ibShutterApertureValid0: 1
m_ibShutterApertureValid1: 1
m_ibShutterApertureValid2: 1
m_ibShutterApertureValid3: 1
m_ibShutterExist0: 1
m_ibShutterExist1: 1
m_ibShutterExist2: 1
m_ibShutterExist3: 1
m_ibShutterModeValid0: 1
m_ibShutterModeValid1: 1
m_ibShutterModeValid2: 1
m_ibShutterModeValid3: 1
m_ibShutterStateOpen0: 0
m_ibShutterStateOpen1: 0
m_ibShutterStateOpen2: 0
m_ibShutterStateOpen3: 0
m_ibShutterStateValid0: 1
m_ibShutterStateValid1: 1
m_ibShutterStateValid2: 1
m_ibShutterStateValid3: 1
m_ibXYExist0: 0
m_ibXYPositionValid0: 1
m_ibZExist0: 0
m_ibZExist1: 0
m_ibZPositionValid0: 1
m_ibZPositionValid1: 1
m_ibZoomExist: 1
m_ibZoomPositionValid: 1
m_sActiveShutterKey: Spectra
m_sActiveShutterKey1: Spectra
m_sActiveShutterKey2: Spectra
m_sActiveShutterKey3: Spectra
m_sActiveShutterKey4: Spectra
m_sFilterFullName0: Ti Filter Block1
m_sFilterFullName1: Ti Filter Block2
m_sFilterFullName2: Lambda 10-3 Filter A
m_sFilterFullName3: Lambda 10-3 Filter B
m_sFilterKey0: Turret1
m_sFilterKey1: Turret2
m_sFilterKey2: Filter A
m_sFilterKey3: Filter B
m_sFilterName0: 89000 - Sedat Quad - ET
m_sFilterName2: Empty
m_sFilterName3: DAPI FF01-432/36
m_sFilterPhysConnectionString2: m_bNIDAQControl=0;m_bTTLAsAnalog=0;m_iNIDAQLineCount=8;m_bTriggering=0;m_bUserUncheckedEnableTriggering=0;m_iCOMPort=5;m_iBaudRate=-1;
m_sFilterPhysConnectionString3: m_bNIDAQControl=0;m_bTTLAsAnalog=0;m_iNIDAQLineCount=8;m_bTriggering=0;m_bUserUncheckedEnableTriggering=0;m_iCOMPort=5;m_iBaudRate=-1;
m_sFilterPhysFullName0: Nikon Ti
m_sFilterPhysFullName1: Nikon Ti
m_sFilterPhysFullName2: Sutter Lambda 10-3
m_sFilterPhysFullName3: Sutter Lambda 10-3
m_sFilterShortName0: Turret1
m_sFilterShortName1: Turret2
m_sFilterShortName2: Filter A
m_sFilterShortName3: Filter B
m_sLightFullName0: Ti Illuminator-DIA
m_sLightKey0: Illuminator-DIA
m_sLightPathFullName: Ti LightPath
m_sLightPathKey: LightPath
m_sLightPathPhysFullName: Nikon Ti
m_sLightPathPositionName: L100
m_sLightPathShortName: LightPath
m_sLightPhysFullName0: Nikon Ti
m_sLightPowerFullName0: Ti Illuminator-DIA
m_sLightPowerKey0: Illuminator-DIA
m_sLightPowerPhysFullName0: Nikon Ti
m_sLightPowerShortName0: Illuminator-DIA
m_sLightRemoteFullName0: Ti Illuminator-DIA
m_sLightRemoteKey0: Illuminator-DIA
m_sLightRemotePhysFullName0: Nikon Ti
m_sLightRemoteShortName0: Illuminator-DIA
m_sLightShortName0: Illuminator-DIA
m_sLightStateFullName0: Ti Illuminator-DIA
m_sLightStateKey0: Illuminator-DIA
m_sLightStatePhysFullName0: Nikon Ti
m_sLightStateShortName0: Illuminator-DIA
m_sMicroscopeFullName: Ti Microscope
m_sMicroscopeKey: Microscope
m_sMicroscopePhysFullName: Nikon Ti
m_sMicroscopePhysShortName: Ti Microscope
m_sMicroscopeShortName: Microscope
m_sMultiLaserFullName0: Lumencor MultiLaser
m_sMultiLaserFullName1: NIDAQ AOTF Multilaser (AOTF)
m_sMultiLaserKey0: Spectra
m_sMultiLaserKey1: AOTF
m_sMultiLaserPhysConnectionString0: m_iCOMPort=4;m_uiBaudRateValue=9600;
m_sMultiLaserPhysFullName0: Lumencor Spectra/AuraII
m_sMultiLaserPhysFullName1: NIDAQ
m_sMultiLaserShortName0: Lumencor MultiLaser
m_sMultiLaserShortName1: Illumination Device (AOTF)
m_sMultiLaser_LineUsedFullName0: Lumencor MultiLaser
m_sMultiLaser_LineUsedFullName1: NIDAQ AOTF Multilaser (AOTF)
m_sMultiLaser_LineUsedKey0: Spectra
m_sMultiLaser_LineUsedKey1: AOTF
m_sMultiLaser_LineUsedPhysConnectionString0: m_iCOMPort=4;m_uiBaudRateValue=9600;
m_sMultiLaser_LineUsedPhysFullName0: Lumencor Spectra/AuraII
m_sMultiLaser_LineUsedPhysFullName1: NIDAQ
m_sMultiLaser_LineUsedShortName0: Lumencor MultiLaser
m_sMultiLaser_LineUsedShortName1: Illumination Device (AOTF)
m_sMultiLaser_NDFilterFullName0: Lumencor MultiLaser
m_sMultiLaser_NDFilterFullName1: NIDAQ AOTF Multilaser (AOTF)
m_sMultiLaser_NDFilterKey0: Spectra
m_sMultiLaser_NDFilterKey1: AOTF
m_sMultiLaser_NDFilterPhysConnectionString0: m_iCOMPort=4;m_uiBaudRateValue=9600;
m_sMultiLaser_NDFilterPhysFullName0: Lumencor Spectra/AuraII
m_sMultiLaser_NDFilterPhysFullName1: NIDAQ
m_sMultiLaser_NDFilterShortName0: Lumencor MultiLaser
m_sMultiLaser_NDFilterShortName1: Illumination Device (AOTF)
m_sMultiLaser_PowerFullName0: Lumencor MultiLaser
m_sMultiLaser_PowerFullName1: NIDAQ AOTF Multilaser (AOTF)
m_sMultiLaser_PowerKey0: Spectra
m_sMultiLaser_PowerKey1: AOTF
m_sMultiLaser_PowerPhysConnectionString0: m_iCOMPort=4;m_uiBaudRateValue=9600;
m_sMultiLaser_PowerPhysFullName0: Lumencor Spectra/AuraII
m_sMultiLaser_PowerPhysFullName1: NIDAQ
m_sMultiLaser_PowerShortName0: Lumencor MultiLaser
m_sMultiLaser_PowerShortName1: Illumination Device (AOTF)
m_sName: Custom
m_sNosFullName: Ti Nosepiece
m_sNosKey: Nosepiece
m_sNosPhysFullName: Nikon Ti
m_sNosShortName: Nosepiece
m_sPFSDMFullName: Ti PFS-S
m_sPFSDMKey: PFS-S
m_sPFSDMPhysFullName: Nikon Ti
m_sPFSDMShortName: PFS-S
m_sPFSDishTypeFullName: Ti PFS-S
m_sPFSDishTypeKey: PFS-S
m_sPFSDishTypePhysFullName: Nikon Ti
m_sPFSDishTypeShortName: PFS-S
m_sPFSFullName: Ti PFS-S
m_sPFSKey: PFS-S
m_sPFSOffsetFullName: Ti PFS-S
m_sPFSOffsetKey: PFS-S
m_sPFSOffsetPhysFullName: Nikon Ti
m_sPFSOffsetShortName: PFS-S
m_sPFSPhysFullName: Nikon Ti
m_sPFSShortName: PFS-S
m_sShutterFullName0: Lumencor Shutter
m_sShutterFullName1: NIDAQ AOTF Shutter (AOTF)
m_sShutterFullName2: NIDAQ Shutter (name: Dia)
m_sShutterFullName3: NIDAQ Shutter (name: Relay)
m_sShutterKey0: Spectra
m_sShutterKey1: AOTF
m_sShutterKey2: Dia
m_sShutterKey3: Relay
m_sShutterPhysConnectionString0: m_iCOMPort=4;m_uiBaudRateValue=9600;
m_sShutterPhysFullName0: Lumencor Spectra/AuraII
m_sShutterPhysFullName1: NIDAQ
m_sShutterPhysFullName2: NIDAQ
m_sShutterPhysFullName3: NIDAQ
m_sShutterShortName0: Lumencor Shutter
m_sShutterShortName1: Shutter (AOTF)
m_sShutterShortName2: Shutter (name: Dia)
m_sShutterShortName3: Shutter (name: Relay)
m_sUserName: Custom
m_sXYFullName0: Ti XYDrive
m_sXYKey0: XYDrive
m_sXYPhysFullName0: Nikon Ti
m_sXYShortName0: XYDrive
m_sZFullName0: Ti ZDrive
m_sZFullName1: NIDAQ Piezo Z (name: Piezo Z)
m_sZKey0: Ti Z
m_sZKey1: Piezo Z
m_sZPhysFullName0: Nikon Ti
m_sZPhysFullName1: NIDAQ
m_sZShortName0: Ti Z
m_sZShortName1: Piezo Z (name: Piezo Z)
m_sZoomFullName: Ti Zoom
m_sZoomKey: Zoom
m_sZoomPhysFullName: Nikon Ti
m_sZoomShortName: Zoom
m_uiAutoWhiteColor: 16777215
m_uiColor: 16711680
m_uiCount: 1
m_uiLightSegmentCount0: 0
m_uiLightSegmentMask0: 0
m_uiMultiLaserLineColor0-00: 0
m_uiMultiLaserLineColor0-01: 0
m_uiMultiLaserLineColor0-02: 0
m_uiMultiLaserLineColor0-03: 0
m_uiMultiLaserLineColor0-04: 0
m_uiMultiLaserLineColor0-05: 0
m_uiMultiLaserLineColor1-00: 0
m_uiMultiLaserLineColor1-01: 0
m_uiMultiLaserLineColor1-02: 0
m_uiMultiLaserLineColor1-03: 0
m_uiMultiLaserLineColor1-04: 0
m_uiMultiLaserLineWavelength0-00: 395
m_uiMultiLaserLineWavelength0-01: 440
m_uiMultiLaserLineWavelength0-02: 470
m_uiMultiLaserLineWavelength0-03: 508
m_uiMultiLaserLineWavelength0-04: 555
m_uiMultiLaserLineWavelength0-05: 640
m_uiMultiLaserLineWavelength1-00: 405
m_uiMultiLaserLineWavelength1-01: 488
m_uiMultiLaserLineWavelength1-02: 561
m_uiMultiLaserLineWavelength1-03: 640
m_uiMultiLaserLineWavelength1-04: 650
m_uiMultiLaserLines0: 6
m_uiMultiLaserLines1: 5
m_uiMultiLaser_LineUsedLineColor0-00: 0
m_uiMultiLaser_LineUsedLineColor0-01: 0
m_uiMultiLaser_LineUsedLineColor0-02: 0
m_uiMultiLaser_LineUsedLineColor0-03: 0
m_uiMultiLaser_LineUsedLineColor0-04: 0
m_uiMultiLaser_LineUsedLineColor0-05: 0
m_uiMultiLaser_LineUsedLineColor1-00: 0
m_uiMultiLaser_LineUsedLineColor1-01: 0
m_uiMultiLaser_LineUsedLineColor1-02: 0
m_uiMultiLaser_LineUsedLineColor1-03: 0
m_uiMultiLaser_LineUsedLineColor1-04: 0
m_uiMultiLaser_LineUsedLineWavelength0-00: 395
m_uiMultiLaser_LineUsedLineWavelength0-01: 440
m_uiMultiLaser_LineUsedLineWavelength0-02: 470
m_uiMultiLaser_LineUsedLineWavelength0-03: 508
m_uiMultiLaser_LineUsedLineWavelength0-04: 555
m_uiMultiLaser_LineUsedLineWavelength0-05: 640
m_uiMultiLaser_LineUsedLineWavelength1-00: 405
m_uiMultiLaser_LineUsedLineWavelength1-01: 488
m_uiMultiLaser_LineUsedLineWavelength1-02: 561
m_uiMultiLaser_LineUsedLineWavelength1-03: 640
m_uiMultiLaser_LineUsedLineWavelength1-04: 650
m_uiMultiLaser_LineUsedLines0: 6
m_uiMultiLaser_LineUsedLines1: 5
m_uiMultiLaser_NDFilterLineColor0-00: 0
m_uiMultiLaser_NDFilterLineColor0-01: 0
m_uiMultiLaser_NDFilterLineColor0-02: 0
m_uiMultiLaser_NDFilterLineColor0-03: 0
m_uiMultiLaser_NDFilterLineColor0-04: 0
m_uiMultiLaser_NDFilterLineColor0-05: 0
m_uiMultiLaser_NDFilterLineColor1-00: 0
m_uiMultiLaser_NDFilterLineColor1-01: 0
m_uiMultiLaser_NDFilterLineColor1-02: 0
m_uiMultiLaser_NDFilterLineColor1-03: 0
m_uiMultiLaser_NDFilterLineColor1-04: 0
m_uiMultiLaser_NDFilterLineWavelength0-00: 395
m_uiMultiLaser_NDFilterLineWavelength0-01: 440
m_uiMultiLaser_NDFilterLineWavelength0-02: 470
m_uiMultiLaser_NDFilterLineWavelength0-03: 508
m_uiMultiLaser_NDFilterLineWavelength0-04: 555
m_uiMultiLaser_NDFilterLineWavelength0-05: 640
m_uiMultiLaser_NDFilterLineWavelength1-00: 405
m_uiMultiLaser_NDFilterLineWavelength1-01: 488
m_uiMultiLaser_NDFilterLineWavelength1-02: 561
m_uiMultiLaser_NDFilterLineWavelength1-03: 640
m_uiMultiLaser_NDFilterLineWavelength1-04: 650
m_uiMultiLaser_NDFilterLines0: 6
m_uiMultiLaser_NDFilterLines1: 5
m_uiMultiLaser_PowerLineColor0-00: 0
m_uiMultiLaser_PowerLineColor0-01: 0
m_uiMultiLaser_PowerLineColor0-02: 0
m_uiMultiLaser_PowerLineColor0-03: 0
m_uiMultiLaser_PowerLineColor0-04: 0
m_uiMultiLaser_PowerLineColor0-05: 0
m_uiMultiLaser_PowerLineColor1-00: 0
m_uiMultiLaser_PowerLineColor1-01: 0
m_uiMultiLaser_PowerLineColor1-02: 0
m_uiMultiLaser_PowerLineColor1-03: 0
m_uiMultiLaser_PowerLineColor1-04: 0
m_uiMultiLaser_PowerLineWavelength0-00: 395
m_uiMultiLaser_PowerLineWavelength0-01: 440
m_uiMultiLaser_PowerLineWavelength0-02: 470
m_uiMultiLaser_PowerLineWavelength0-03: 508
m_uiMultiLaser_PowerLineWavelength0-04: 555
m_uiMultiLaser_PowerLineWavelength0-05: 640
m_uiMultiLaser_PowerLineWavelength1-00: 405
m_uiMultiLaser_PowerLineWavelength1-01: 488
m_uiMultiLaser_PowerLineWavelength1-02: 561
m_uiMultiLaser_PowerLineWavelength1-03: 640
m_uiMultiLaser_PowerLineWavelength1-04: 650
m_uiMultiLaser_PowerLines0: 6
m_uiMultiLaser_PowerLines1: 5
m_uiPropGroupCount: 0
right: 0
sCal.dFmax: 1
sCal.dFmin: 255
sCal.dKd: 224
sCal.dRmax: 2
sCal.dRmin: 0
sCal.dTMeasCalMax: 0
sCal.dTMeasCalMin: 0
sCal.dVisc: 1
sDescription: DAPI Zyla MB
sObjective: Plan Apo 20x DIC M N2
sOpticalConfigName: DAPI Zyla
sSpecSettings:
Camera Type: Andor Zyla
Binning: 1x1
Exposure: 500 ms
Readout Mode: Rolling shutter at 16-bit
Readout Rate: 540 MHz
Conversion Gain: Dual Gain 1/4
Spurious Noise Filter: on
Sensor Mode: Overlap
Trigger Mode: Internal
Temperature: -11.1°C
sizeObjFullChip.cx: 2048
sizeObjFullChip.cy: 2048
top: 0
uiBkgndSize: 0
uiBpc: 0
uiBpcInMemory: 16
uiBpcSignificant: 16
uiColor: 16711680
uiColorCustom: 255
uiColorGroup: 255
uiColorSpace: 4
uiComp: 0
uiCompCount: 1
uiCon20(L: -1
uiCount: 1
uiDenOffset: 0
uiDenom: 1
uiFQModeUsage: 0
uiGradient: 0
uiGradientBrightField: 0
uiGroup: 0
uiGroupCount: 3
uiHeight: 27164
uiInterpretation: 0
uiLutComponentCount: 4
uiMaxSrc: 65535
uiMinDst: 0
uiMinSrc: 0
uiModalityMask: 1
uiModeFQ: 2
uiNumOffset: 0
uiNumer: 0
uiOCTypeKey: 14
uiOpticalConfigs: 1
uiPhysicalVarCount: 0
uiRepresentation: 0
uiRow: -1
uiSampleCount: 4
uiSampleIndex: 3
uiSequenceCount: 1
uiSpectralColorMode: 0
uiTileHeight: 1
uiTileWidth: 40676
uiUsageFlags: 0
uiVirtualComponents: 4
uiWidth: 40676
uiWidthBytes: 325408
vGrabberCameraSettingsFQMode: 0
wsObjectiveCode: MRD00201
wsObjectiveName: Plan Apo 20x DIC M N2
wszFormatDesc: 16-bit - No Binning (91.9 FPS)
wszFormatText: 16-bit - No Binning
wszUniqueName: FMT 1x1 (VSC-04649) 16-bit

Reading metadata
timestamp #1: -0.001

#8

Thanks, I just noticed that you had been trying to set BF_MAX_MEM using the option flag, if you instead run export BF_MAX_MEM=26g (or set BF_MAX_MEM=26g in Windows) beforehand does that then still throw the Java heap error?

#9

I did not get the heap error when I ran export BF_MAX_MEM = 26g. I also did not get it when I used the -option setting in bfconvert.

It looks like if there is enough RAM on the computer, then the conversion will occur. But what if the input file is larger than the available RAM? Is it possible to process files larger than available RAM?

#10

It is possible but there are a number of factors involved. In most cases if the size of the file is due to a large number of planes in T or Z then you should be fine. If you have very large single planes then the tiling options is recommended and will help. Issues can start occurring with compressed data, as the decompression stage can end up resulting in far greater memory usage, this is especially seen with large planes that have been compressed as an entire plane.

#11

As far as I know the nd2 files are not compressed. But they are large area scans with multiple 16-bit channels. I have been using the -tilex and -tiley flags both set to 512. Would decreasing those values help? Also, I was rereading the documentation and came across the -BigTiff flag. With such large files, should I set this as well?

#12

The optimal tile sizes will vary depending on the particular data, but 512 is a fairly standard and sensible size to be using, I wouldn’t expect the memory performance to improve noticeably by going smaller than that. BigTiff is generally used when the total file size of the image surpasses 4GB, the converter will automatically switch to BigTiff if the size requires it so generally there is no need to force it unless you have a particular reason for doing so.

#13

Is there a functional difference between setting BF_MAX_MEM via the -option flag in bfconvert versus the command line (set BF_MAX_MEM=26g)? Is one more permanent than the other?

#14

The -option approach shouldn’t actually work, the BF_MAX_MEM flag isn’t one of the recognised options. As such I would recommend using the set or export method

#15

The export method works fine in the Mac OS. In Windows though, I run into java heap size errors when I try to convert an 800 MB file after using set BF_MAX_MEM=26g on a system with 64 GB memory. It seems like the heap size is not being changed. Can I define and set the BF_MAX_MEM variable via Windows 10’s Advanced Preferences dialog? Is this equivalent to using set? Is it a permanent fix?
Thanks.

#16

The set command will temporarily modify the variable, if you wish it to be permenant you can use setx BF_MAX_MEM=26g.

I don’t currently have access to a Windows 10 machine but it looks as though you should indeed be able to set a new environment variable from that dialog using:

Got to Control Panel 
Click System
Click Advanced system settings
Go to the Advanced Tab
Click the "Environment Variable"
#17

Hello,

I am working on a project that seems to be very similar to the one described here i.e. using bfconvert to convert a 412GB .nd2 file (384 confocal z-slices of a whole mouse brain, each slice 2048x2048 tiles stitched into a large image, 1 channel, no timelapse) to .ome.tiff files. I am running Microsoft Server 2016 with 96 GB RAM. The goal is to deconvolve, reslice to orthogonal projections, and register to Allen Brain Atlas images.

I tried “bfconvert path\to\input path\to\output_Z%z.ome.tiff” which resulted in the ‘Java Heap Space’ error described here. Entering ‘set BF_MAX_MEM=40g’ to command line returned:

The specified size exceeds the maximum representable size.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

In a separate but related issue, I was able to convert this .nd2 file to an even larger .ome.tiff file using -tilex -tiley 2048x2048. However, I am unable to split the ome.tiff file to individual ome.tiff files for each z-slice using ‘_Z%z’. The output remains in one large file with ‘Zz’ in the output name instead of the expected ‘Z+index number’ output.

My questions are:
Am I entering the argument correctly to change the BF_MEM_MAX setting?
Why is my file not splitting into individual files for each z-slice?

Any guidance or advice would be greatly appreciated.

Best,

Justin

#18

The BF_MEM_MAX sounds as though you probably using a 32bit Java JVM which would only allow 2GB of Java heap. Changing to a 64bit version will likely solve that issue.

The use of %z looks correct to me, it may be that Windows is treating the % as a special that needs to be escaped. If you use %%z does it behave correctly?

#19

Yes I do have 32bit Java; I will change to 64bit asap.

I set BF_MEM_MAX=2G and tried %%z and received:

Error occurred during initialization of VM
Could not reserve enough space for 2097152KB object heap

Perhaps I should try this with 64bit Java?

#20

I have both 32 and 64 bit Java on a shared machine that has applications that use both so I assume there is a way to force JVM in 64 bit mode? I have limited coding experience, but after a little research I see that adding ‘-d64’ will do the trick, is that right?