Edit page History

BigWarp

    TODO component: {"content"=>"sc.fiji:bigwarp\\_fiji"} Bigwarp is a tool for manual, interactive, landmark-based deformable image alignment. It uses the BigDataViewer for visualization and navigation, and uses a Thin Plate Spline implemented in Java to build a deformation from point correspondences.

    The interface enables landmark pair placement and displays the effects of the warp on-the-fly.

    Installation

    Bigwarp comes with Fiji. You can access it via *Plugins ›

    BigDataViewer ›

    Big Warp*, or by modifying this example script . If this is not visible in your installation, try updating Fiji with *Help ›

    Update Fiji.*

    Usage

    Open two images in ImageJ, one moving and the other target and navigate to *Plugins ›

    BigDataViewer ›

    Big Warp.* A dialog will appear prompting selection of the moving and target images.

    Once the two image windows and one table window open, press press\|Spacebar to enter “landmark mode”. Next, click on a point in the moving image, then click on the corresponding point in the target image. After you have a few moving-target point pairs, press press\|T to transform the moving image (you may need to re-navigate if the two image are very far apart: see the press\|Q and press\|W hotkeys below).

    Getting Help

    Press press\|F1 at any time to open a help page with a listing of navigation and editing commands. For detailed help, create a post on the image.sc forum with a bigwarp tag. Report bugs on github.

    Landmark point placement and display in the viewer

    Landmark placements is done in Landmark mode which you enter by pressing press\|Spacebar . Users place pairs of corresponding points on the moving and target images.

    The following table shows the available commands and keystrokes for landmark placement, warping.

    press|T

    Toggle between warped view and raw view or moving image.

    press|Ctrl +press|O

    Open landmarks from saved file.

    press|Ctrl +press|S

    Save current landmarks to a file.

    press|Spacebar

    Toggle Landmark mode

    <Landmark mode>+press|left-click

    Clicking while in landmark mode adds a landmark point or selects and existing landmark.

    <Landmark mode>+press|left-click +press|drag

    Clicking an existing point and dragging changes it's position.

    <Landmark mode>+press|Shift +press|left-click +press|drag

    "Move" a point. The initial click places a landmark point for the moving image. The release places a landmark point for the target image.

    <Landmark mode>+press|Ctrl +press|left-click

    "Pin" a point. Add a landmark at the same location for both moving and target images.

    <Landmark mode>+press|Ctrl +press|Shift +press|left-click

    Place a point in the "other" space. Place a moving landmark from the fixed window, and vice versa.

    press|Ctrl + press|Z

    Undo the last landmark point change.

    press|Ctrl + press|Y

    Redo the last landmark point change.

    press|V

    Toggle point visibility in the viewer.

    press|N

    Toggle point name visibility in the viewer.

    Landmark selection and editing in the table

    Some changes to landmarks can be done by interacting with the landmark table.

    press|left-click

    Select row.

    press|Ctrl +press|left-click

    Add row to selection.

    press|Shift +press|left-click

    Select range of rows.

    press|Esc

    Deselect all rows.

    press|right-click › Delete

    Deletes a landmark pair (row in the table).

    press|right-click › Delete all selected

    Deletes all selected landmark pairs (row in the table).

    Notes on point addition and landmark pair selection

    • Adding a new landmark pair selects that pair (row) in the table.
    • If a row is “missing” a moving (target) landmark point, then it must be selected in order to add that missing point by clicking in the moving (target) viewer. Then BigWarp will find the “next” row that is missing a moving (target) landmark, and select that row automatically.
    • If the selected row is not missing a landmark, the next click will add a new landmark pair.
    • If multiple rows are selected when a viewer is clicked, the result will be as though only the first row was selected.

    Selecting transformation types

    Press press\|F8 to bring up a transformation type selection window (version 4.0.0 of BigWarp supports multiple options for transformations)

       
    Example of a thin plate spline transformation Example of an affine transformation

    Table of transformation types

       
    Thin Plate Spline The smoothest deformable transformation that maps moving landmarks exactly to the corresponding target landmarks.
    Affine Linear transform with translation, rotation, independent scales and shear (12 degrees of freedom)
    Similarity Linear transform with translation, rotation, and one scale parameter (7 degrees of freedom)
    Rotation Linear transform with translation, and rotation (6 degrees of freedom)
    Translation Translation only (3 degrees of freedom)

    Bigwarp inherits many image navigation, visualization, and grouping features with BigDataViewer, the details of which can be found on the BigDataViewer page or on the help page. BigWarp specific features are documented below.

    The following table shows the available navigation commands using the mouse:

    press|Q

    Align the non-active viewer with the active viewer.

    press|W

    Align the active viewer with the non-active viewer.

    press|E

    Centers the active viewer to the nearest landmark (considers 3D when applicable).

    press|Ctrl +press|D

    Centers the active viewer to the next landmark.

    press|Ctrl +press|Shift +press|D

    Centers the active viewer to the previous landmark.

    press|R

    Resets the active viewer.

    press|U

    Show warp visualization / grid dialog.

    press|F6

    Show moving image panel Visibility & and Grouping dialog.

    press|F7

    Show target image panel Visibility & and Grouping dialog.

    press|Ctrl +press|E

    Export moving image as an ImagePlus.

    press|Ctrl +press|Shift +press|E

    Export moving image as a Virtual ImagePlus.

    Commands shared with BigDataViewer

    Mouse navigation

    press|left-click|drag

    Rotate (pan and tilt) around the point where the mouse was clicked.

    press|right-click|drag or press|middle-click|drag

    Translate in the XY-plane.

    press|mouse-wheel

    Move along the z-axis.

    press|Cmd|mouse-wheel or press|Shift|Ctrl|mouse-wheel

    Zoom in and out.

    Keyboard navigation

    press|X , press|Y , press|Z

    Select keyboard rotation axis.

    press|Left , press|Right

    Rotate clockwise or counter-clockwise around the choosen rotation axis.

    press|Up , press|Down

    Zoom in or out.

    press|, , press|.

    Move forward or backward along the Z-axis.

    press|Shift|X

    Rotate to the ZY-plane of the current source. (Look along the X-axis of the current source.)

    press|Shift|Y or press|Shift|A

    Rotate to the XZ-plane of the current source. (Look along the Y-axis of the current source.)

    press|Shift|Z

    Rotate to the XY-plane of the current source. (Look along the Z-axis of the current source.)

    press|[ or press|N

    Move to previous timepoint.

    press|] or press|M

    Move to next timepoint.

    For all navigation commands you can hold press\|Shift to rotate and browse 10x faster, or hold press\|Ctrl to rotate and browse 10x slower. For example, press\|Left rotates by 1° clockwise, while press\|Shift\|Left rotates by 10°, and press\|Ctrl\|Left rotates by 0.1°.

    Save and load landmarks

    Landmarks can be exported and imported from plain text files using the drop down menu in the landmark table panel ( *File ›

    Export (Import) landmarks.* )

    Export warped images

    Export the warped moving image by clicking *File ›

    > Export as ImagePlus* or using the press\|Ctrl +press\|E keyboard shortcut.

    The default parameters will result in the exported image having the same dimensions as the target image. Note: Take care when exporting very large data sets as they can cause out-of-memory exceptions.

    • Resolution
      • Target: The output will have the same resolution as the target image
      • Moving: The output will have the same resolution as the moving image
      • Specified: The output will have the resolution given in the fields below (in the same units as the moving and target images).
    • Field of view
      • Target: The output will have the same field of view as the target image
      • Moving: The output field of view will be the smallest bounding box that contains the warped moving images (approximated)
      • Specified (pixel): ‘’ The output field of view will be that given by the Offset and Field of View parameter fields where both are in units of pixels’’** ** Specified (physical): ‘’ The output field of view will be that given by the Offset and Field of View parameter fields where both are in the physical units of the moving and target images’’

    The warped moving image can be exported as an in-memory or virtual ImagePlus. A virtual ImagePlus is generally faster to generate but slower to browse, whereas an in-memory ImagePlus will be slower to generate but faster to browse.

    Apply transforms

    Often, it is important to apply transforms estimated with one image to other images in the same space.

    If you have moving and target images open in Fiji this script to transform the moving image into the space of the target image. You will need to provide a file containing the saved landmark point pairs.

    To manually specify the field-of-view (FOV) of the target space, use this script

    To make the scripts above appear in your Fiji Plugins menu, simply copy them into the /plugins/Scripts folder in your Fiji installation.

    Warp 2d ImageJ ROIs

    Download and install this script.

    Usage:

    1. Run the script downloadable above.
    2. Indicate which open images correspond to the moving and fixed images in bigwarp.
    3. Select the csv file that stores bigwarp landmarks.
    4. Indicate whether you want to transform an ROI from moving to target space or vice versa.
    5. Click OK.

    Note that the resulting ROI will be densely sampled. This is because an ROI with a particular shape may not have the same shape after being warped (i.e., a circle may not still be circular after being warped).

    Example: In this example, we’ve chosen the boats sample as the moving image, the bridge sample as the target image, and have added some landmarks that transform the boats image.

    To warp an ROI from the moving boats image to the fixed bridge image, we run the script with the options:

    • Fixed image: boats
    • Moving image: bridge
    • Landmark file:
    • Moving to target

    The results are shown in the “Forward example” below.

    To warp an ROI from the moving boats image to the fixed bridge image, we run the script with the options:

    • Fixed image: boats
    • Moving image: bridge
    • Landmark file:
    • Target to moving

    The results are shown in the “Inverse example” below.

       
    Forward example (click to expand) Inverse example (click to expand)

    Note, at this time ImageJ ROIs are 2D objects. We recommend using another approach for 3D regions of interest.

    Warp points in a csv file

    Download and install this script.

    Usage:

    1. Run the script download-able above.
    2. Select the csv file that stores bigwarp landmarks.
    3. Select the csv file storing the points you want to transform.
    4. Select the location for csv file you want the results to be stored in.
    5. Indicate whether you want to transform the points from moving to target space or vice versa.
    6. Click OK.

    Note the input csv must be formatted:

     x1,y1,z1
     x2,y2,z2
     ...
     xN,yN,zN

    without quotation marks, spaces, or any other characters. Csv files storing bigwarp landmarks will not work as input to this script.

    Working with large images

    The “normal” BigWarp plugin accessible through *Plugins ›

    Big Data Viewer ›

    BigWarp* works well for small- and medium-sized images but not for very large volumes (that do not fit into memory). For very large volumes, we recommend first converting the volume to bigdataviewer’s xml/hdf5 format, then using this script to run BigWarp using the xml/h5 file(s). See also this forum post.

    This script enables you to run BigWarp using images stored using N5.

    Tutorials

    Publication

    Publications using BigWarp

    1. Russell et al. “3D correlative light and electron microscopy of cultured cells using serial blockface scanning electron microscopy” J Cell Sci 130: 278-291 2017.
    2. Collinson et al. “Correlating 3D light to 3D electron microscopy for systems biology” Current Opinion in Biomedical Engineering 2017 3:49-55
    3. Lerner et al. “Mycobacterium tuberculosis replicates within necrotic human macrophages” J Cell Biol 2017
    4. Hildebrand et al. “Whole-brain serial-section electron microscopy in larval zebrafish” Nature 545:345–349 2017.
    5. Zhang et al. “A Complete Electron Microscopy Volume Of The Brain Of Adult Drosophila melanogaster” Cell 174:3 P730-743.e22, 2018.
    6. Gao et al. “Cortical column and whole-brain imaging with molecular contrast and nanoscale resolution.” Science 363 (6424) 2019.
    7. Wan et al. “Single-Cell Reconstruction of Emerging Population Activity in an Entire Developing Circuit.” Cell 179(2) 2019.
    8. Hoffman, Shtengel et al. “Correlative three-dimensional super-resolution and block-face electron microscopy of whole vitreously frozen cells.” Science 367 (6475) 2020.