This is an archive of the old MediaWiki-based ImageJ wiki. The current website can be found at imagej.net.

Project ideas

This page contains a loose list of ideas for cool/useful projects that have some relation to Fiji

Contents

Visualization

3-way viewer for Block-Face EM image volumes based in ImgLib cells

The Imglib provides a cell container, where each cell is an arbitrary n-dimensional image block potentially paged out to a file. Consider the specific case of isotropic 3d-cells stored with lossless compression. This setup would enable efficient browsing and analysis of image volumes larger than available RAM, while observing a specific 3d ROI from XY, YZ and XZ planes. These image volumes are common in Block-Face Serial Scanning Electron Microscopy, a technique now commercialized as "Gatan 3-way view EM." See this page for theory and for pictures and examples.

Goal: create a 3-way viewer for a 3d instance of an Imglib's cell container.
Language: any supported by Fiji, preferably java.
Contact: Albert Cardona

Plugin for Mixed-File-Format MultiVirtualHyperStack viewing window

The idea is to be able to display multiple virtual hyperstack-type data sets in a single multi-color composite window. I've already arranged this using multiple QuickTime Movies or AVIs. But the ideal will be to allow overlay from mixed data of any of the many BioFormats-supported file types.

The user would be able to overlay, realign and fit the separate channels in 4 dimensions, and then synchronously browse the composite view in ZT dimensions. There should be some demand for this functionality, especially with the very large data repositories being made by many labs in different file formats. Enabling automatic spatial calibration from metadata would allow measurements analysis of all channels with regions of interest addressing all of the overlaid data.

The project would consist of

  • Writing a class extending VirtualStack.java class for each type of input data file series. Currently, these exist for FileSeriesFromList and QTVirtualStack.
  • Create a MultiVirtualHyperStack.java class that can organize multiple VirtualStack types into a single VirtualHyperStack displayed via a single ImagePlus and StackWindow. The getProcessor() method in this class must be able to sort out the file coordinates of any channel/slice/frame requested from the mixed-format virtual stack and call getProcessor() from each of the specialized VirtualStack classes for each format.
  • Create a control panel that allows adjustments of XYZT position for any single VirtualHyperStack that is a component of the mixed overlay window.

Goal: Plugin for Mixed-File-Format MultiVirtualHyperStack viewing window.
Language: Java.
Contact: Bill Mohler (wmohler@neuron.uchc.edu)

Plugin: python script for multi-stack composite image

Interactively adjustable intensity/LUT curves

In Fiji, you can adjust the dynamic range of an image by calling Image  › Adjust  › Brightness & Contrast. However, this only lets you choose a linear mapping between pixel intensity and lookup table. This project aims to provide non-linear controls, such as piecewise linear functions, gamma curves, splines, etc

Image processing plugins

Applying machine learning to the image segmentation problem

The term image segmentation describes the task where objects in an image are to be outlined, so that every pixel is connected to either a named object, or background.

Segmentation is traditionally a very difficult problem, especially in the presence of variable lighting, noise, or low contrast.

Many segmentation algorithms have been implemented in Fiji to perform image segmentation, such as Auto Threshold and Auto Local Threshold, but in practice, none of them might work, as they were designed with specific images in mind, and these expectations might not be met by your images.

Recently, a new class of segmentation algorithms has been emerging: segmentation by example. These algorithms require a set of examples from which a model is calculated which can be applied to other -- similar-looking -- images.

We will consider applications for implementations that are either as generic as possible (i.e. they apply to any images), or that try to solve a very specific problem (such as segmenting neurons in serial sections imaged with electron microscopy, or with confocal imaging.)

We have several data sets of images and their corresponding manual segmentations (for training the algorithm). See for example:

You are welcome to use any scientifically-relevant dataset of your choice, but we will give priority to biologically-oriented data sets.


Goal: Implement a number of segmentation algorithms based on machine learning.
Language: Java.
Mentor: Ignacio Arganda-Carreras, Albert Cardona


Implementing algorithms for Imglib

The new imglib supports dimension-, storage- and data type independent image processing. This library has some algorithms built-in already but there is a strong need to generically implement more general image processing algorithms, storage strategies and data types such as:

  • Interpolation (Cubic, Spline, ...)
  • Entropy Filter, Average Filter, Percentile(Min, Median, Max) Filter, ...
  • Memory Management for partial image loading
  • Color Spaces and Color Space Conversions
  • Efficient representation of non-raster images (based on 2d polygonal shapes and 3d/4d meshes)

Goal: Implement generic algorithms for image processing.
Language: Java.
Mentor: Stephan Preibisch, Stephan Saalfeld], Tobias Pietzsch, Albert Cardona

Colorizing algorithms

There are a number of publications about turning greyscale images into color images. This project is about implementing as many of them as possible.

Note: this is an ill-posed problem, as there is not enough information in the greyscale images to identify the original color. However, under certain circumstances, it is possible to estimate a best guess for the color for most or all pixels.

Goal: implement a colorizing plugin.
Language: Java.
Mentor: J. Schindelin (johannes.schindelin AT gmx.de)

Image selector/sorter

Implement an algorithm that sorts a number of images by features, such as color. Inspired by Kai-Uwe Barthel's pixolu project.

A set of more powerful painting brushes and image editing tools

Even if Fiji aims at scientific image processing rather than beautifying photographs, it might be fun to take your holiday pictures and post-process them with the image processing software you are familiar with.

Possible tools to do so would be airbrushes (allowing for transparent colors) or brushes with a certain inertia to allow calligraphic effects, etc.

Goal: implement a plugin for interactive calligraphic effects.
Language: Java.
Mentor: J. Schindelin (johannes.schindelin AT gmx.de)


Wavelet inpainting

Implement a simple inpainting method (i.e. restore missing/unwanted parts of the image marked by a ROI) using wavelets: apply the wavelet transform, and then, on each level, use a diffusion algorithm to deduce a smooth signal from the surrounding parts, and finally inverse-transform the wavelet to get the restored image.

Goal: implement a plugin for interactive calligraphic effects.
Language: Java.
Mentor: [ Albert Cardona])

Scripting

Add JMathLib (MATLAB clone) support

Quite a few algorithms are available as proof-of-concept MATLAB scripts. While it is wrong to think of pixels as little squares, and literally all MATLAB scripts to perform image processing are suffering from that assumption, it would be very nice nevertheless to be able to run the scripts without having to buy MATLAB licenses just for that purpose.

MATLAB bundles a Java runtime (and in fact, all of MATLAB's GUI is implemented in Java!) and allows the user to instantiate Java classes and call methods on them: 

import java.io.File;
f = File('/usr/local/Fiji.app/');
f.exists()

Happily, there is a MATLAB clone written in Java: JMathLib. While it is apparently not a speed demon, it should be useful to add JMathLib as a new scripting language to ImageJ, and integrate it into Fiji so that MATLAB scripts can be executed just like all other ImageJ scripts, too.

So far, we have a branch which adds rudimentary JMathLib bindings to Fiji's scripting interface and a Git-SVN mirror of the JMathLib source code repository with a special fiji branch. The idea is to work on this branch to adjust JMathLib in certain ways to support this project, and once that is done, contribute the changes back to the JMathLib project.

The proof-of-concept version of the Refresh_JMathLib_Scripts class illustrates how the JMathLib Interpreter needs to be instantiated and called.

The following issues need to be tackled in the JMathLib source code:

  • In embedded (non-standalone) mode, the FunctionManager installs a WebFunctionLoader which disallows new functions to be added. Either the WebFunctionLoader needs to be taught to allow (maybe optionally) new functions to be added, or there needs to be another mode in which JMathLib is embedded, but still allows new functions to be defined.

This issue needs to be tackled in Fiji's source code:

  • JMathLib's image toolbox does not contain much. Even the most basic functions are missing. And even if there were functions, we would have to override them, because the functions need to be done in a way so that they can use and interact with ImageJ. The best approach may be to start by implementing the functions mentioned in MATLAB's image processing toolbox' Getting Started section, by implementing .m files that call directly into ImagePlus (using the above-mentioned technique).

Goal: Integrate JMathLib as a new scripting language.
Language: Java.
Mentor: Johannes Schindelin (johannes.schindelin@gmx.de)

A Javascript Recorder

Similar to the Macro Recorder but producing Javascript instead. There is a Javascript recorder in ImageJ right now, but it is in no way integrated into the Fiji Script Editor. It also appears that the Javascript recorder is not as robust as the Macro recorder yet.

Code templates in the Script Editor

The Script Editor provides a fine way to script small plugins that do some simple tasks. If you know how.

Provide a good number of templates so that the user does not have to start from scratch. A good template will also include rather more documentation than less, so that ideally the user does not have to look up the appropriate API calls, but just modifies the well-documented code.

Make Script Editor rename a Java class automatically on Save As...

A public Java class must be compiled from a source file reflecting the class name, so it makes sense to rename the Java class when the file is saved under a new name. Teach the Script Editor to do that.


Add a Bookmark function to the Script Editor

Often, it would be very convenient to remember the current cursor position to come back to, after looking around in other parts of the file. Maybe ^ Ctrl+B (together with a menu entry), or ^ Ctrl+<digit> are good ways to implement the user interface. (The code should be similar to the Goto Line... function.

Add support for Haskell (via Jaskell) and Tcl (via Jacl)

We already have Jacl in Fiji, as it is a dependency of Batik. There is also a pure-Java implementation of the Haskell language, and both should be relatively easy to integrate into Fiji as scripting languages.

For Tcl, the Script Editor would need minimal adjustments, as RSyntaxTextArea already has support for Tcl, but for Haskell, a new TokenMaker would have to be implemented.

Add Edit  › Find in files...

We already have a mechanism to jump between compile errors and locations of a stack trace. The same mechanism could be used to present results from a search through multiple files.

Add a "REPL" (Read-Eval-Print-Loop) to the Script Editor

Detect loops after macro recording

A special form of an autocorrelation (on text) should be pretty good an indication where the user repeated things that might want to be done in a loop instead. This would help users with little background in programming to write powerful plugins through the macro recorder.

Fiji development environment/infrastructure

GUI Testing framework

We have some rudimentary GUI testing in the tests branch but it may be better to use an established GUI framework such as Jemmy or Marathon.

The idea is, in any case, to record mouse moves and keyboard presses, optionally waiting for some GUI element (such as a window) to appear, and error out if something unexpected happens -- which most likely means that something broke and needs fixing.

Goal: Provide an easy way to record and run GUI regression tests.
Language: Mainly Java
Mentor: Johannes Schindelin (johannes.schindelin@gmx.de)

Interface between R and ImageJ/Fiji

It would be nice to have a set of implemented procedures so IJ/Fiji can run statistical procedures directly from Results tables, etc).

RImageJ is an R package written by Romain Francois which uses the Java/R interface included in rJava to start an ImageJ instance inside R and send commands to that instance.

It would be nice to have the opposite direction working, to call R from Fiji. As the Java/R interface included in rJava does allow to start R from within Java, it is quite feasible.

To overcome the typical problem of loading native libraries via System.loadLibrary() needing special platform-dependent settings, we should do something like this: 

// prohibit JRI to call System.exit(1);
System.setProperty("jri.ignore.ule", "yes");
if (!Rengine.jriLoaded) {
        // not found on the library path
        System.load("/absolute/path/to/the/library");
        Rengine.jriLoaded = true;
}
Rengine re = new Rengine();

Teach the Fiji Updater to accept other sites in addition to fiji.sc

The Fiji Updater always looks for a static file containing an XML database of Fiji plugins (both current and past versions) on our website. To put new versions or new plugins there (to upload into the updater), you have to be a Fiji developer with write permission for that particular directory on our server.

In some cases, there are plugins that are either too sensitive, or too specific for a certain application, or not ready for public consumption yet, but still somebody might want to install Fiji in such a way that it automatically updates those plugins, too. Of course, there must be a different location for those plugins than the official Fiji update site, lest the general audience receive those plugins automatically.

The project is not without complications, though:

  • The XML database is saved as a file in the local Fiji directory, and it is always checked at startup whether the timestamp is newer than the timestamp of the XML database on the server. If you have multiple update sites, it should be handled in a way, where the local XML database reflects the sources of the metadata, and for uploading, a temporary XML database must be constructed for one particular upload site.
  • There may be conflicts between plugins that are official Fiji plugins, but also available from a secondary site. This has to be coped with (it is not clear what the best strategy should be: take the official Fiji version over the secondary site? let the user choose?)
  • With a new site, you need to be able to upload plugins to that site, too. There needs to be a very good way to prevent confusion, lest the plugin is uploaded to the wrong site.
  • To determine whether a developer can upload new plugins (because there are new versions), the Fiji Updater scans the complete plugins directory, along with a few other places where macros, 3rd party libraries, or the Fiji launcher might hide. The Fiji Updater needs to learn not to offer these plugins for upload to a secondary site, but only the non-Fiji ones.
  • It is unlikely that our current Fiji Updater can start a database from scratch. This has to be verified, and if there is no code for that yet, it has to be implemented.
  • Cross-site dependencies should be handled by having hints in the XML database as to what other site is supposed to have the newest dependency.

Integrate JGit into Fiji

An important part of Fiji's success is the ease with which developers can collaborate through the use of Git.

There exists a pure Java implementation of Git called JGit, which already provides a large part of Git's functionality.

It would be nice to have it integrated into Fiji so that the Script Editor can give the developers an even smoother developing experience.

Make the Fiji Updater more intelligent about restarting

Only when there are updates outside plugins/ is it necessary to do a full restart; otherwise, a simple "Update Menus" will do the trick.

Further, after the message "You need to restart Fiji" (or the Update Menus), there is no reason for the Updater to stay open.

And finally, if a restart is required, the user could be asked whether a restart should be attempted, and a JNI-provided function could be called with a list of open images (if there are unsaved images, they should be saved temporarily into temporary files) and result tables, which then re-executes Fiji appropriately.

Make the Object Inspector more useful

There is a tool in Fiji that lets you sift through all open frames and inspect the corresponding objects (and their fields, in a recursive fashion).

However, there is no connection to the script editor yet (where you could open the corresponding sources for a given class). This might be pretty handy.

Additionally, there could be a mode where you open the hierarchy of objects starting with the object the cursor hovers over, updated dynamically. This could be even more useful if there was a mode to show only the listeners of the objects, so that you can easily determine what code is responsible, say, to handle the click on a specific OK button.

User interface improvements

Add a meta-plugin to run other plugins with ranges of parameters

Many plugins take parameters, and it might not be obvious what the optimal values are. So it would be nice to have a plugin that can call another plugin with a range of values.

The idea is to let the meta-plugin run the plugin which asks for the user input in the normal way. But then, the meta-plugin will analyze what parameters were specified, and ask for increments and stop criteria of the numeric ones. With these data, it will run the plugin in a loop.

The result could be presented as a stack of the result images.

Another (more hacky) possibility of getting the range parameters is to intercept the dialog before it is shown, and extend the numeric input fields by increment and stop entries.

Suggested by Quentin de Robillard.

Integrate ImageFlow into Fiji

ImageFlow.png

ImageFlow provides a graphical way to construct macros. Every action is represented by a node which the user can connect with lines to define a workflow.

ImageFlow has its own Git repository (our mirror).

The following issues need to be resolved:

  • At the moment, it is not a true plugin, but wants to start its own ImageJ instance
  • it only targets the macro language, while we want to target all the scripting languages supported by Fiji
  • it searches for its .xml files outside of the .jar file, which makes it cumbersome to ship with the Fiji updater.

Provide Help on menu item

Many commands available from menu items are actually plugins, so they are documented nicely on the Fiji Wiki. There is a beginning of a plugin that changes the cursor to an arrow with a question mark, and (temporarily) the way the menu items are handled: instead of running the corresponding command, the corresponding documentation on the Fiji Wiki is opened in a web browser.

The code needs to be improved, though, to handle special menu items correctly, such as the recent files or the windows menu.

Some menu items are actually provided by the core of ImageJ, and it may be the best to open the appropriate place on the ImageJ website instead of the Fiji Wiki. In the alternative, the ImageJ documentation should be replicated on the Fiji Wiki under the appropriate page titles.

The user should also be informed that hitting the ⎋ Esc key gets her out of this mode.

And finally, the Fiji Wiki needs some love to reflect the exact titles of the menu items, most probably by adding appropriate redirects.

Add a clever Save As plugin

For now, File  › Save As always saves the result as a .tiff file, even if the user specified a file name ending in, say, .png.

Stephan Preibisch suggests: Add a plugin that determines from a set of extensions which plugin to call to the respective writer plugin.

For extra brownie points, do not hardcode the extension/plugin mapping (like HandleExtraFileTypes), but make it configurable via one or more file.

Miscellaneous

Alpha shapes / concave hull / other Graph Theory algorithms

Fiji already contains a Delaunay_Voronoi plugin. The purpose of this project is to implement more graph algorithms. Most likely, this will involve designing a common framework for graph theory as applied to two- or higher-dimensional graphs.

Support for storing ROIs in TIFF tag fields

Fiji can save images as TIFF files and ROIs into custom .roi files. Provide a way to store the ROIs inside custom tags in the TIFF file so ROIs and images can be saved together.

Cross platform webcam support

Supporting image recording from webcams might provide a cheap way to make videomicroscope/telescope units (possibly using the Distortion Correction plugin to overcome low-quality CCD chips and lenses).

One way to achieve that would be by using the Free Java Media Framework.

A unique/common segmentation interface

I have collected near 15 new histogram segmentation methods that would be better put under a single interface together with others already available.

Note: this is more or less implemented in the Auto_Threshold and Auto_Local_Threshold plugins.--Gabriel Landini 14:47, 29 November 2009 (CET)

Virtual microscope-like image viewer

HSB/Lab painting modes

Integrate Micro-Manager into Fiji

This project requires a bit of knowledge in compiling C++ code on Linux, MacOSX and Windows. The idea is to make a recipe that other people can use to compile new releases of Micro-Manager, as well as integrate it into the Fiji project for a smooth user experience. To ensure that support for Micro-Manager is not broken inadvertently, you shall add regression tests, too.

Goal: Provide an easy way to compile and ship Micro-Manager with Fiji.
Language: Java, C++, shell
Mentor: Johannes Schindelin (johannes.schindelin@gmx.de)

Other resources

There is a wish list on the ImageJ Documentation Wiki.

Retrieved from "http://imagej.net/index.php?title=Project_ideas&oldid=41308"