Editing object shape in TrackMate

Since version 8, TrackMate ships a new feature that allows editing object shape in 2D. The spot editor is based on Labkit components, and is made to simplify and accelerate the creation of tracking ground truth. In this tutorial we will explain how to use it to modify segmentation results directly in TrackMate

Preparing the tutorial data

Download this image and open it in Fiji. It is a movie of the early development of a C. elegans embryo, projected in 2D, for which the nuclei have been stained in fluorescence. We will generate an incorrect segmentation of these nuclei in TrackMate, and fix it with the spot editor.

In Fiji, with the image open, launch TrackMate (Plugins › Tracking › TrackMate). Click the Next button and select the Thresholding detector. In its configuration page, put a threshold of 1000 and click Next.

Click the Next button until you are in the tracker selection page and select the LAP tracker. In its configuration page,

• put 5 µm as max linking distance,
• uncheck the Track segment gap closing button,
• check the Track segment splitting button and put a max distance of 5 µm,
• uncheck everything else.

You should get the following:

Launching the spot editor

The threshold we set is too stringent, and many nuclei are improperly segmented, and some polar bodies are missing. For instance, in frame 2 the top-left nucleus looks like this:

We will use the spot editor to correct some of the mistakes there. The spot editor can be launched from the Display options panel of TrackMate:

Move to frame 2 and click the Launch spot editor button. The user interface of TrackMate is frozen and a new window appear:

The large bottom right panel displays the image overlaid with the spots. Notice that the spots have been converted to masks, and that they have the same color that in the TrackMate UI.

The top bar is made of widgets that change the brush mode (navigate, add, remove, fill, …) and the brush size. We will describe them below.

The left side bar contains (from top to bottom)

• the button to end editing,
• auto-contrast and display config panel for the spot editor,
• the list of labels currently in the editor, initially there is one label per spot,
• a button to create a new label, possibly for a new spot.

Navigating in the editor

The editor window is actually a BDV component. If you know your way around BDV you will get your bearings quickly. Otherwise here is how to navigate in the image panel.

Panning, rotating and zooming

• Pan the image with the Right Click button.
• Rotate the image with the Left Click.
• Zoom in and out towards the center of the panel with the Mouse Wheel.

Resetting the view

• ⇧ Shift + Z aligns the view with the XY plane.
• ⇧ Shift + R resets the view.

Navigating to spots

On the left side bar, you can see the list of labels currently in the editor. Shift-clicking on a label in the list will center the image view on the corresponding spot mask

Changing the display settings

The Image section in the left side bar allows you to change the display settings of the image. The auto contrast button will set the display range to the min and max pixel values in the image. The settings button will open a dialog to change the display settings of the image, such as the color table, the brightness and contrast and the visibility of channels and spots.

Editing spots

Let’s fix the top left nucleus in frame 2. Center and zoom in on it, so that it fills the panel. We see that there are two spurious detections on the side, of 1-pixel size, and that the largest spot is not large enough to cover the nucleus. Let’s first remove the spurious detections.

Deleting pixels

The fourth button on the top bar is the Delete button. Click it to make the delete brush mode active. But if you click on the spurious detections, nothing happens. This is because the delete brush only works on the pixels of the label that is currently selected in the left side bar. You need to select the label you want to edit first. To do so, you can either

• ⇧ Shift +  Left Click on the spot mask in the image panel,
• click on the Select button on the top toolbar (the sixth button on the left) then click on the spot in the image,
• or select the label in the left side bar.

Shift-click on each of the spurious detections to select them, and then click on them with the delete brush on to remove them. Notice that the corresponding labels in the left side bar are not removed, which is normal.

Painting spots

We now want to enlarge the largest spot to cover the nucleus. First shift-click on it in the image panel to make the spot label active. Then click on the Add button in the top bar which is the second button from the left. Edit the brush size to about 5 pixels, and paint over the nucleus.

Removing entire spots

You can also remove entire spots from the editor using the Remove mode, which is the third button from the left in the top bar. Again, the corresponding label of the spot to remove must be selected in the left side bar. Click on the Remove button, select the label in the left side bar and click on the spot you want to remove.

Filling holes

Another way of creating spots is to paint the border and fill the inside. After drawing the contour of the nucleus, you can use the Fill button, which is the third button from the left in the top bar. Click it, and click inside the contour to fill it.

Keyboard shortcuts changing the brush mode and size

Several keyboard shortcuts are available to speed up the editing process. First you can press F1, F2, F3, F4, F5 and F6 to switch between the Navigate, Add, Fill, Delete, Remove and Select modes, respectively. You can also use Q and , E to decrease and increase the brush size, respectively. Pressing ⇧ Shift + Q and ⇧ Shift + E will change the brush size by a larger amount.

Keyboard shortcuts for editing

You can also use the following keyboard shortcuts to directly edit in the image, without having to select a brush mode first. Put the brush in the Navigate mode (F1), and start editing with the following keys:

Key	Action
A Left Click or ␣ Space +  Left Click	add pixels
D Left Click or ␣ Space +  Right Click	delete pixels
R Left Click	remove entire spot
L Left Click	fill holes
⇧ Shift +  Left Click	select spot at mouse location
Q and E	decrease and increase the brush size
⇧ Shift + Q and ⇧ Shift + E	decrease and increase the brush size faster
␣ Space +  Mouse Wheel or A +  Mouse Wheel or D +  Mouse Wheel	change the brush size

The default key bindings are chosen to match the edition of spots in the TrackMate UI. For adding, deleting, removing and filling spots, the keys must be used as modifiers, and kept pressed while you paint.

Passing the results to TrackMate

Once you are done editing, click the Close and send to TrackMate button in the top left side bar. This will close the editor and prompt you with the following dialog:

If you click No, your changes will be discarded. If you click Yes, the modified spots will be sent to TrackMate. If you selected the Simplify contours checkbox, the contours of the spots will be smoothed and simplified before being sent to TrackMate. This relies on the same routine that for the Simplify contours option found in most of the segmentation detectors in TrackMate, and explained here. Because the spot editing is done with a mask, it is recommended to use this option to avoid having jagged contours.

Tips and tricks

Editing a subset of the image

If the image is very large and you want to only edit a region, you can first draw a ROI in the image before launching the editor. Only the spots that are fully in the bounding box of the ROI will be sent to the editor.

Editing all time frames

The spot editor normally only edits the spots in the frame that is currently displayed in Fiji. However if you press the Shift while pressing the Launch spot editor button, all frames will be sent to the editor. This respects whether you have a ROI in the image or not.

Spots can be overlapping

In TrackMate, spots are represented as polygons, as as such they can overlap. This is also the case in the spot editor. This is the reason why it is necessary to select the spot you want to edit before using the brush.

Several spots can have the same label

In TrackMate, a single spot is of a single polygon. It cannot have holes, nor cannot be made of several disconnected polygons. For this reason, when you use one label in the editor to create multiple separated masks, they will appear as multiple spots in TrackMate.

Spots are reinserted in the tracks

When you edit a spot, it will be possibly reinserted in the TrackMate model, within the track it belonged to. If the edits generate masks that are too different from the initial spots, they will be considered as new spots, and the tracks will have to be corrected in TrackScheme.

Key bindings are customizable

You can customize the key bindings of the spot editor in a dedicated dialog, displayed when you press ⌃ Ctrl + ,.

Closing the editor window

Has the same effect as clicking the Close and send to TrackMate button.

---

Jean-Yves Tinevez, August 2025