Workflows

BRT Workflow

Overview:

A number of parameters can be passed to the “BRT” workflow. There are two types: 1) params that are passed to the adoc file. See adoc / https://bio3d.colorado.edu/imod/doc/directives.html 2) params that are used elsewhere within the worflow, providing metadata etc, eg input_dir.

Note: objects passed between Prefect Tasks (eg FilePath objects), must be considered immutable. Updates to state made in one task will be lost and not available to the next task. Create a new object. The map function is used extensively, and preserves order.

TomoJS 3D pipeline for tomographic reconstruction.

The term “batch run tomo[gram]” is abbreviated BRT.

Processes

Generate BRT Directive File

Description:

Generate a configuration file of parameter for BRT process.

Outputs:

  • a BRT directive file with a .adoc file extension.

Alternatively, the file could be provided and this step skipped. The directive file format is documented IMOD’s batchruntomo documentation. There is specific descriptions of the directives specified in a “directives.csv” file within IMOD.

The following is a Jinja like template for key-value pairs to be configured. The {{ }} denotes where the substitution should be made. Inside the curly brackets is a suggested parameter name for the value with defaults in parenthesis:

setupset.copyarg.name={{ basename }}
setupset.copyarg.gold={{ beadSize (~10)}}
setupset.copyarg.pixel={{ pixelSize }}
comparam.prenewst.newstack.BinByFactor={{ bin (4) }}
comparam.track.beadtrack.LightBeads={{ lightBeads (0) }}
comparam.tilt.tilt.THICKNESS={{ thickness (~256) }}
runtime.AlignedStack.any.binByFactor={{ bin }}
setupset.copyarg.montage={{ montage (0) }}
setupset.datasetDirectory={{ local processing folder (POSIX convention) }}

The existing directive file is dirTemplate.adoc with empty values that can be updated as above.

BRT support multiple layers of “Template Files” so that directives can be defined for a microscope, system, and user. The other directive files can be defined as parameters it the batch directive file provided on the command line.

BatchRunTomo (BRT)

Description:

Perform computational expensive operations of processing an acquired tomography tilt-series and reconstructing a 3D volume. The process consists of numerous step to prepare, align, and perform 3d reconstruction from the tilt series acquired by the microscope.

Inputs:

  1. the original image from the microscope usually with an .mrc extension (.st possible ). The filename part before the final “.” is considered the BASENAME.

  2. BRT directive file

Primary Outputs:

  1. BASENAME_rec.mrc - the source for the reconstruction movie

  2. BASENAME_full_rec.mrc - the source for the Neuroglancer pyramid

  3. BASENAME_ali.mrc - the source for the tilt movie

Auxiliary Outputs:

  1. Other outputs such as logs and transformation files may need to be saved as well.

The process is IMOD’s batchruntomo, run with the following command line arguments:

batchruntomo -di BASENAME.adoc -gpus 1 -cpus 20

The gpus is for configuring the local GPU, and cpus configures the number of cores to use. Many temporary and log files are created during this process.

Generate tilt movie

Description:

Convert a MRC file of the aligned tilt series into a movie for easy viewing.

Inputs:

  1. BASENAME_ali.mrc - the aligned 2d slices of image stack

Outputs:

  1. Generates a tilt movie for easy viewing

  2. The key thumbnail is keyimg_BASENAME_s.jpg, the corresponding MIDDLE_I is the full size.

dimensions = (header -s ${BASENAME}_ali.mrv) # space separated list of dimensions sizes (x y z)
MIDDLE_I = floor(dimensions.z/2))

for i in dimensions.z\:
  newstack -secs {i}-{i} ALI_FILENAME WORKDIR/hedwig/BASENAME_ali{i}.mrc
newstack -float 3 WORKDIR/hedwig/BASENAME_ali*.mrc WORKDIR/hedwig/ali_BASENAME.mrc
mrc2tif -j -C 0,255 WORKDIR/hedwig/ali_BASENAME.mrc WORKDIR/hedwig/BASENAME_ali
gm convert -size 300x300 WORKDIR/hedwig/BASENAME_ali.{MIDDLE_I}.jpg -resize 300x300 -sharpen 2 -quality 70 WORKDIR/hedwig/keyimg_BASENAME_s.jpg
ffmpeg -f image2 -framerate 4 -i ${BASENAME}_ali.%03d.jpg -vcodec libx264 -pix_fmt yuv420p -s 1024,1024 tiltMov_${BASENAME}.mp4

Generate reconstruction movie

Description:

Convert a MRC file of the reconstructed 3D volume into a movie for easy viewing.

Inputs:

  1. BASENAME_rec.mrc - the reconstruction of the 3d volume ( may already be binned by some factor when compared to full).

Outputs:

  1. Generates a movie of the reconstructed 3D volume.

for i in range(2, dimensions.z-2):
  IZMIN = i-2
  IZMAX = i+2
  clip avg -2d -iz IZMIN-IZMAX  -m 1 WORKDIR/BASENAME_rec.mrc WORKDIR/hedwig/BASENAME_ave${I}.mrc
newstack -float 3 WORKDIR/hedwig/BASENAME_ave* WORKDIR/hedwig/ave_BASENAME.mrc
binvol -binning 2 WORKDIR/hedwig/ave_BASENAME.mrc WORKDIR/hedwig/avebin8_BASENAME.mrc
mrc2tif -j -C 100,255 WORKDIR/hedwig/ave_BASNAME.mrc hedwig/BASENAME_mp4
ffmpeg -f image2 -framerate 8 -i WORKDIR/hedwig/BASENAME_mp4.%04d.jpg -vcodec libx264 -pix_fmt yuv420p -s 1024,1024 WORKDIR/hedwig/keyMov_BASENAME.mp4

Generate Neuroglancer Pyramid

Descriptions:

Generates a Neuroglancer precomputed pyramid from an MRC file of a 3D volume. This does not work for tilt series, or other non-volumetric files.

Inputs:

  1. A MRC file of a 3D volume.

Outputs:

  1. A directory structure of the precomputed pyramid.

Steps:

1. Convert the MRC file to NIFTI (.nii). The Neuroglancer file format only support unsigned integer of 8 or 16 bits. When this input is a signed integer the output pixel types needs to be changed and the pixel values adjusted. The NIAID tomojs-pytools mrc2nift command-line tool can do this conversion. 2. The neuroglancer-scripts tools are used to convert the NIFTI file to the precompute format:

volume-to-precomputed-pyramid --downscaling-method=average --no-gzip --flat nifti.nii {WORKDIR}/hedwig/neuro-BASENAME

3. The default minimum and maximum values used for visualization also need to be computed from the NIFTI file. The NIAID tomojs-pytools mrc_visual_min_max performs this computation:

mrc_visual_min_max {WORKDIR}/nifti.nii --mad 5 --output-json mrc2ngpc-output.json

The cloud-volume tool may be an alternative tool for the precompute conversion.

Multi-channel Microscopy Workflow (CZI/SVS/OME-TIFF)

Overview:

This workflow is referred to externally by Hedwig as the Multi-channel pipeline. It processes multidimensional microscopy images stored in CZI (ZEISS), SVS (Aperio), and OME-TIFF file formats. Supported image types include immunofluorescence (IF) multi-channel images (e.g., z-stacks, time lapse, and multiposition experiments) as well as RGB brightfield images such as H&E-stained CZI slides. The workflow converts input files to OME-NGFF zarr format, generates neuroglancer-compatible metadata, and creates thumbnail images for visualization and downstream analysis.

Outputs: 1. OME-NGFF zarr file for each input file, including OME-XML metadata. 2. Neuroglancer metadata for each sub-image (scene/channel) in the zarr file. 3. Thumbnail JPEG image for each label sub-image.

Supported Extensions for File Formats

Format Type

Description

Extensions

CZI

ZEISS multidimensional microscopy image (IF multi-channel and RGB H&E brightfield)

czi, CZI

SVS

Aperio whole-slide image

svs, SVS

OME-TIFF

Open Microscopy Environment TIFF; supports single-file and multi-file datasets with modality interpreted from embedded OME-XML metadata

ome.tiff, OME.TIFF, ome.tif, OME.TIF

The workflow relies on OME Bio-Formats (via the bioformats2raw tool) for reading CZI, SVS, and OME-TIFF files. Visualization is performed with Neuroglancer.

  1. Input File to Zarr Conversion - Uses bioformats2raw to convert CZI, SVS, or OME-TIFF files to OME-NGFF zarr format, preserving OME-XML metadata. - Output: .zarr directory for each input file.

  2. Rechunking Zarr - Re-chunks the zarr structure so that multi-channel/RGB channels are not split between chunks, using

    zarr_rechunk from tomojs-pytools.

  3. Copy Zarr to Assets Directory - Copies the generated zarr files to the assets directory for downstream use.

  4. Generate ImageSet and Thumbnails - For each sub-image in the zarr file:

    • Determines the shader type: RGB (for H&E and other RGB images), Grayscale, or MultiChannel (for IF fluorescence images).

    • Generates neuroglancer metadata (shader, dimensions, shader parameters).

    • For label sub-images, generates a thumbnail JPEG using SimpleITK (rotated 90° for correct orientation).

    • Macro sub-images are ignored.

  5. Metadata Attachment - Attaches the OME-XML metadata file location to the zarr group for developer reference and provenance.

  6. Callback and API Notification - Sends results and metadata to the API callback URL if provided.

Summary of Input Image Types

Format Type

Description

Processing Information

CZI (IF multi-channel)

Multidimensional immunofluorescence image (z-stacks, time lapse, multiposition)

Converted to OME-NGFF zarr; neuroglancer metadata generated with MultiChannel or Grayscale shader; thumbnails created for label sub-images.

CZI (RGB H&E)

RGB brightfield whole-slide image (e.g., H&E stained tissue)

Converted to OME-NGFF zarr; neuroglancer metadata generated with RGB shader; thumbnails created for label sub-images.

SVS

Aperio whole-slide image

Converted to OME-NGFF zarr; neuroglancer metadata generated; thumbnails created for label sub-images.

OME-TIFF

Open Microscopy Environment TIFF

Converted to OME-NGFF zarr via bioformats2raw; embedded OME-XML metadata is preserved in the output zarr; ( generally does not have a label sub-image)

Note

  • Macro sub-images in the input file are ignored by the workflow.

  • Label sub-images are used for thumbnail generation; the thumbnail is rotated 90° for correct orientation.

  • OME-XML metadata is attached to the output zarr for troubleshooting and provenance.

  • The shader type (RGB, Grayscale, or MultiChannel) is determined automatically from the OME-XML metadata of each sub-image.

  • For OME-TIFF files, bioformats2raw autodetects embedded OME-XML regardless of whether it is stored inline in the TIFF header or as a companion .ome.xml sidecar file. The ome.tiff / ome.tif compound extension (case-insensitive) is used to identify OME-TIFF inputs.

  • Multi-file OME-TIFF datasets (where image planes are split across multiple TIFF files) are supported. Set single_file to the name of the first TIFF file in the series; bioformats2raw will locate and assemble the remaining files automatically via the embedded OME-XML file list.

Spatialomics Workflow

Normally the dir structure is : $lab/$pi/$project/$session/$sample

For Spatialomics this is not the case, the $sample is not really a sample, it’s grouping of ROIs, PreROIs, etc from each of the slides. These grouping directories sit one down from session, in the place a $sample normally would be.

For a single unit of work (eg 8 slides) the set up looks like

$lab/$pi/$project/$session/Pre_ROI_Selection
$lab/$pi/$project/$session/Heatmaps
$lab/$pi/$project/$session/HQ_Images
$lab/$pi/$project/$session/ROI_Images

and eg:

ls $lab/$pi/$project/$session/HQ_Images/
slide_1.czi, slide_2.czi, ...

and

ls $lab/$pi/$project/$session/Pre_ROI_Selection
slide_1_Pre_ROI_a.png, slide_2_Pre_ROI_a.png, ...

Note: Different outputs from different slides are split across different dirs. The pipeline will not parse out or otherwise link slides together in any way. Similarly there is no linking from from heatmaps to any specific slide.

Also, other directories can exist in this directory, which will be ignored.

Small 2D Workflow (EM Microscopy)

Overview:

This workflow generates a thumbnail and a key images for each input image in a directory. Specific EM file formats use enhanced resampling and filtering techniques to reduce noise while adjusting the dynamic range of the images. This workflow originally was designed to handle DM3 files from TEM or STEM microscopes but has been extended to handle other 2D image file formats, including MRC, TIFF files and color figures.

Outputs:

  1. A thumbnail image for each input image.

  • An JPEG image with a maximum size of 300 pixels in both dimension.

  • The aspect ratio is preserved and may be shortened to fit within the maximum size without padding.

  • The image may be 8-bit RGB(A) or 8-bit gray-scale.

  1. A key image for each input image.

  • Similar to the thumbnail image but with a maximum size of 1024 pixels in both dimension.

The toolbox of IMOD is used primarily to process the EM images, along with SimpleITK for additional image processing tasks.

Summary of Input Image Types

Format Type

Pixel Type

Description

Processing Information

JPEG, PNG, TIFF

8-bit RGB (palette) or RGBA

Example images include figure, TEM grid image or other illustrative images rendered into a color image.

Images should directly resized without intensity scaling or advanced filtering.

TIFF, PNG, JPEG

8-bit gray-scale

Images may be rendered EM images with captions or annotations.

Images should directly resized without intensity scaling or advanced filtering.

TIFF

16-bit (signed or unsigned) gray-scale

EM images with a high dynamic range.

Use IMOD’s newstack tool to resample, filter, and scale intensities to 8-bit.

DM4 or DM3 Images

32-bit float (after MRC conversion)

Minimally processed EM images with a high dynamic range.

Convert to MRC via dm2mrc command to 32-bit float MRC files, then use newstack to resample, filter, and scale intensities to 8-bit TIFF.

MRC (2D only)

32-bit float ( no other current sample inputs )

Expected EM images with likely high dynamic range.

Convert to 8-bit TIFF via newstack command to resample, filter, and scale intensities to 8-bit.

Large 2D RGB Workflow

Overview:

This workflow is referred to externally by Hedwig as the Flat 2D pipeline. It processes large 2D RGB images (PNG or TIFF) to produce an OME-NGFF zarr pyramid suitable for Neuroglancer visualization, along with a thumbnail and a key image for browsing. The pipeline is intended for any large 2D image that requires Neuroglancer visualization, including annotated micrographs, color composites, and other 2D figures.

Outputs:

  1. OME-NGFF zarr pyramid (neuroglancerZarr asset) for Neuroglancer visualization, including neuroglancer metadata.

  2. Thumbnail JPEG image — maximum 300 × 300 pixels, aspect ratio preserved.

  3. Key image JPEG — maximum 1024 × 1024 pixels, aspect ratio preserved.

Supported Extensions for File Formats

Format Type

Description

Extensions

PNG

Portable Network Graphics

png, PNG

TIFF

Tag Image File Format

tif, TIF, tiff, TIFF

The workflow uses ImageMagick for input normalization, OME Bio-Formats (via bioformats2raw) for zarr conversion, zarr_rechunk from tomojs-pytools for rechunking, and SimpleITK for thumbnail and key image generation. Visualization is performed with Neuroglancer.

  1. Input Normalization to TIFF - Uses ImageMagick convert to produce a normalized TIFF from the input PNG or TIFF file. - Alpha transparency is removed and replaced with a white background. - Tile geometry is set to 128 × 128 pixels for downstream processing optimization. - Output: .tiff file in the working directory.

  2. TIFF to Zarr Conversion - Uses bioformats2raw to convert the normalized TIFF to an OME-NGFF zarr pyramid. - Output: .zarr directory in the working directory.

  3. Rechunking Zarr - Re-chunks the zarr structure so that multi-channel/RGB channels are not split between chunks, using

    zarr_rechunk from tomojs-pytools.

  4. Copy Zarr to Assets Directory - Copies the rechunked zarr to the assets directory for downstream access.

  5. Generate Neuroglancer Asset - Reads the zarr from the working directory to compute metadata. - Determines the shader type (RGB, Grayscale, or MultiChannel) automatically from the image data. - Produces a neuroglancerZarr asset pointing to the 0 (full-resolution) level of the zarr in the

    assets directory, along with neuroglancer metadata (shader, dimensions, shaderParameters).

    Note

    The dimensions field is currently hardcoded to XY as the GUI does not accept XYC for this pipeline.

  6. Generate Thumbnail and Key Image - Uses SimpleITK to extract and resize the image from the working zarr. - Thumbnail: maximum 300 × 300 pixels, written as JPEG (quality 90). - Key image: maximum 1024 × 1024 pixels, written as JPEG (quality 90). - Both images are copied to the assets directory.

  7. Callback and API Notification - Sends results and metadata to the API callback URL if provided.

Summary of Input Image Types

Format Type

Description

Processing Information

PNG

Color or grayscale PNG image, possibly with alpha transparency

Normalized to TIFF (alpha removed, white background); converted to OME-NGFF zarr; neuroglancer metadata and thumbnail/key image generated.

TIFF

Color or grayscale TIFF image

Normalized to TIFF (alpha removed, white background); converted to OME-NGFF zarr; neuroglancer metadata and thumbnail/key image generated.

Note

  • All inputs pass through the TIFF normalization step regardless of format. This ensures a consistent tiled TIFF with alpha removed for reliable zarr conversion.

  • The shader type (RGB, Grayscale, or MultiChannel) is determined automatically from the image data.

  • Thumbnail and key image dimensions preserve the aspect ratio and will be at most the stated maximum size.