ImageJ

BigWarp plugin for Fiji
Project	Fiji
URL	https://imagej.net/BigWarp
Source	on GitHub
License	GPLv2
Release	3.0.0
Date	Thu May 25 12:57:29 CDT 2017
Development status	Active
Support status	Active
Team
Founders	John Bogovic, Stephan Saalfeld
Leads	John Bogovic, Stephan Saalfeld
Developers	John Bogovic, Stephan Saalfeld
Debuggers	John Bogovic, Stephan Saalfeld
Reviewers	John Bogovic, Stephan Saalfeld
Support	John Bogovic, Stephan Saalfeld
Maintainers	John Bogovic, Stephan Saalfeld
Contributors	Curtis Rueden, Mark Hiner

Bigwarp is a tool for manual, interactive, landmark-based deformable image alignment. It uses the BigDataViewer for visualization and navigation, and uses a Thin Plate Spline implemented in Java to build a deformation from point correspondences.

The interface enables landmark pair placement and displays the effects of the warp on-the-fly.

Installation

Bigwarp comes with Fiji. You can access it via Plugins › BigDataViewer › Big Warp, or by modifying this example script. If this is not visible in your installation, try updating Fiji with Help › Update Fiji.

Usage

Open two images in ImageJ, one moving and the other target and navigate to Plugins › BigDataViewer › Big Warp. A dialog will appear prompting selection of the moving and target images.

Once the two image windows and one table window open, press Spacebar to enter "landmark mode". Next, click on a point in the moving image, then click on the corresponding point in the target image. After you have a few moving-target point pairs, press T to transform the moving image (you may need to re-navigate if the two image are very far apart: see the Q and W hotkeys below).

Getting Help

Press F1 at any time to open a help page with a listing of navigation and editing commands. For detailed help, create a post on the image.sc forum with a bigwarp tag. Report bugs on github.

Landmark point placement and display in the viewer

Landmark placements is done in Landmark mode which you enter by pressing Spacebar. Users place pairs of corresponding points on the moving and target images.

The following table shows the available commands and keystrokes for landmark placement, warping.

T	Toggle between warped view and raw view or moving image.
^ Ctrl+O	Open landmarks from saved file.
^ Ctrl+S	Save current landmarks to a file.
Spacebar	Toggle Landmark mode
<Landmark mode>+left-click	Clicking while in landmark mode adds a landmark point or selects and existing landmark.
<Landmark mode>+left-click+drag	Clicking an existing point and dragging changes it's position.
<Landmark mode>+⇧ Shift+left-click+drag	"Move" a point. The initial click places a landmark point for the moving image. The release places a landmark point for the target image.
<Landmark mode>+^ Ctrl+left-click	"Pin" a point. Add a landmark at the same location for both moving and target images.
<Landmark mode>+^ Ctrl+⇧ Shift+left-click	Place a point in the "other" space. Place a moving landmark from the fixed window, and vice versa.
^ Ctrl + Z	Undo the last landmark point change.
^ Ctrl + Y	Redo the last landmark point change.
V	Toggle point visibility in the viewer.
N	Toggle point name visibility in the viewer.

Landmark selection and editing in the table

Some changes to landmarks can be done by interacting with the landmark table.

left-click	Select row.
^ Ctrl+left-click	Add row to selection.
⇧ Shift+left-click	Select range of rows.
⎋ Esc	Deselect all rows.
right-click › Delete	Deletes a landmark pair (row in the table).
right-click › Delete all selected	Deletes all selected landmark pairs (row in the table).

Notes on point addition and landmark pair selection

• Adding a new landmark pair selects that pair (row) in the table.
• If a row is "missing" a moving (target) landmark point, then it must be selected in order to add that missing point by clicking in the moving (target) viewer. Then BigWarp will find the "next" row that is missing a moving (target) landmark, and select that row automatically.
• If the selected row is not missing a landmark, the next click will add a new landmark pair.
• If multiple rows are selected when a viewer is clicked, the result will be as though only the first row was selected.

Selecting transformation types

Press F8 to bring up a transformation type selection window (version 4.0.0 of BigWarp supports multiple options for transformations)

Example of a thin plate spline transformation	Example of an affine transformation

Table of transformation types

Thin Plate Spline	The smoothest deformable transformation that maps moving landmarks exactly to the corresponding target landmarks.
Affine	Linear transform with translation, rotation, independent scales and shear (12 degrees of freedom)
Similarity	Linear transform with translation, rotation, and one scale parameter (7 degrees of freedom)
Rotation	Linear transform with translation, and rotation (6 degrees of freedom)
Translation	Translation only (3 degrees of freedom)

Navigation and Visualization

Bigwarp inherits many image navigation, visualization, and grouping features with BigDataViewer, the details of which can be found on the BigDataViewer page or on the help page. BigWarp specific features are documented below.

The following table shows the available navigation commands using the mouse:

Q	Align the non-active viewer with the active viewer.
W	Align the active viewer with the non-active viewer.
E	Centers the active viewer to the nearest landmark (considers 3D when applicable).
^ Ctrl+D	Centers the active viewer to the next landmark.
^ Ctrl+⇧ Shift+D	Centers the active viewer to the previous landmark.
R	Resets the active viewer.
U	Show warp visualization / grid dialog.
F6	Show moving image panel Visibility & and Grouping dialog.
F7	Show target image panel Visibility & and Grouping dialog.
^ Ctrl+E	Export moving image as an ImagePlus.
^ Ctrl+⇧ Shift+E	Export moving image as a Virtual ImagePlus.

Commands shared with BigDataViewer

• Displaying multiple stacks ("sources")
• Grouping sources
• Adjusting brightness and color
• Bookmarking views (locations and orientations)

Mouse navigation

left-click+drag	Rotate (pan and tilt) around the point where the mouse was clicked.
right-click+drag or middle-click+drag	Translate in the XY-plane.
mouse-wheel	Move along the z-axis.
⌘ Cmd+mouse-wheel or ⇧ Shift+^ Ctrl+mouse-wheel	Zoom in and out.

Keyboard navigation

X, Y, Z	Select keyboard rotation axis.
←, →	Rotate clockwise or counter-clockwise around the choosen rotation axis.
↑, ↓	Zoom in or out.
,, .	Move forward or backward along the Z-axis.
⇧ Shift+X	Rotate to the ZY-plane of the current source. (Look along the X-axis of the current source.)
⇧ Shift+Y or ⇧ Shift+A	Rotate to the XZ-plane of the current source. (Look along the Y-axis of the current source.)
⇧ Shift+Z	Rotate to the XY-plane of the current source. (Look along the Z-axis of the current source.)
[ or N	Move to previous timepoint.
] or M	Move to next timepoint.

For all navigation commands you can hold ⇧ Shift to rotate and browse 10x faster, or hold ^ Ctrl to rotate and browse 10x slower. For example, ← rotates by 1° clockwise, while ⇧ Shift+← rotates by 10°, and ^ Ctrl+← rotates by 0.1°.

Save and load landmarks

Landmarks can be exported and imported from plain text files using the drop down menu in the landmark table panel ( File › Export (Import) landmarks. )

Export warped images

Export the warped moving image by clicking File › > Export as ImagePlus or using the ^ Ctrl+E keyboard shortcut.

Error creating thumbnail: Unable to save thumbnail to destination

The default parameters will result in the exported image having the same dimensions as the target image. Note: Take care when exporting very large data sets as they can cause out-of-memory exceptions.

• Resolution
  • Target: The output will have the same resolution as the target image
  • Moving: The output will have the same resolution as the moving image
  • Specified: The output will have the resolution given in the fields below (in the same units as the moving and target images).

• Field of view
  • Target: The output will have the same field of view as the target image
  • Moving: The output field of view will be the smallest bounding box that contains the warped moving images (approximated)
  • Specified (pixel): The output field of view will be that given by the Offset and Field of View parameter fields where both are in units of pixels** ** Specified (physical): The output field of view will be that given by the Offset and Field of View parameter fields where both are in the physical units of the moving and target images

The warped moving image can be exported as an in-memory or virtual ImagePlus. A virtual ImagePlus is generally faster to generate but slower to browse, whereas an in-memory ImagePlus will be slower to generate but faster to browse.

Apply transforms

Often, it is important to apply transforms estimated with one image to other images in the same space.

If you have moving and target images open in Fiji this script to transform the moving image into the space of the target image. You will need to provide a file containing the saved landmark point pairs.

To manually specify the field-of-view (FOV) of the target space, use this script

To make the scripts above appear in your Fiji Plugins menu, simply copy them into the /plugins/Scripts folder in your Fiji installation.

Warp 2d ImageJ ROIs

Download and install this script.

Usage:

1. Run the script downloadable above.
2. Indicate which open images correspond to the moving and fixed images in bigwarp.
3. Select the csv file that stores bigwarp landmarks.
4. Indicate whether you want to transform an ROI from moving to target space or vice versa.
5. Click OK.

Note that the resulting ROI will be densely sampled. This is because an ROI with a particular shape may not have the same shape after being warped (i.e., a circle may not still be circular after being warped).

Example: In this example, we've chosen the boats sample as the moving image, the bridge sample as the target image, and have added some landmarks that transform the boats image.

To warp an ROI from the moving boats image to the fixed bridge image, we run the script with the options:

• Fixed image: boats
• Moving image: bridge
• Landmark file: <our landmarks file>
• Moving to target

The results are shown in the "Forward example" below.

To warp an ROI from the moving boats image to the fixed bridge image, we run the script with the options:

• Fixed image: boats
• Moving image: bridge
• Landmark file: <our landmarks file>
• Target to moving

The results are shown in the "Inverse example" below.

Forward example (click to expand)	Inverse example (click to expand)

Note, at this time ImageJ ROIs are 2D objects. We recommend using another approach for 3D regions of interest.

Warp points in a csv file

Download and install this script.

Usage:

1. Run the script download-able above.
2. Select the csv file that stores bigwarp landmarks.
3. Select the csv file storing the points you want to transform.
4. Select the location for csv file you want the results to be stored in.
5. Indicate whether you want to transform the points from moving to target space or vice versa.
6. Click OK.

Note the input csv must be formatted:

 x1,y1,z1
 x2,y2,z2
 ...
 xN,yN,zN

without quotation marks, spaces, or any other characters. Csv files storing bigwarp landmarks will not work as input to this script.

Working with large images

The "normal" BigWarp plugin accessible through Plugins › Big Data Viewer › BigWarp works well for small- and medium-sized images but not for very large volumes (that do not fit into memory). For very large volumes, we recommend first converting the volume to bigdataviewer's xml/hdf5 format, then using this script to run BigWarp using the xml/h5 file(s). See also this forum post.

This script enables you to run BigWarp using images stored using N5.

Tutorials

• An example of a 2d warping by Nicolas Chiaruttini in response to this forum post.

Publication

• Bogovic, J. A.; Hanslovsky, P. & Wong, A. et al. (2016), "Robust registration of calcium images by learned contrast synthesis", ISBI: 1123-1126, doi:10.1109/ISBI.2016.7493463, <https://ieeexplore.ieee.org/document/7493463>

Publications using BigWarp

1. Russell et al. "3D correlative light and electron microscopy of cultured cells using serial blockface scanning electron microscopy" J Cell Sci 130: 278-291 2017.
2. Collinson et al. "Correlating 3D light to 3D electron microscopy for systems biology" Current Opinion in Biomedical Engineering 2017 3:49-55
3. Lerner et al. "Mycobacterium tuberculosis replicates within necrotic human macrophages" J Cell Biol 2017
4. Hildebrand et al. "Whole-brain serial-section electron microscopy in larval zebrafish" Nature 545:345–349 2017.
5. Zhang et al. "A Complete Electron Microscopy Volume Of The Brain Of Adult Drosophila melanogaster" Cell 174:3 P730-743.e22, 2018.
6. Gao et al. "Cortical column and whole-brain imaging with molecular contrast and nanoscale resolution." Science 363 (6424) 2019.
7. Wan et al. "Single-Cell Reconstruction of Emerging Population Activity in an Entire Developing Circuit." Cell 179(2) 2019.
8. Hoffman, Shtengel et al. "Correlative three-dimensional super-resolution and block-face electron microscopy of whole vitreously frozen cells." Science 367 (6475) 2020.