ImageJ

2011-03-16

Tobias and Saalfeld found it a great idea to create this discussion page that, if ever possible, will be filled with the daily conceptual discussions and results. That way, all interested people can contribute and keep track on what we're trying to tackle next.

We discussed the View concept:

Summary is that View<T> would be an interface that can return its target T, the data it is generated from:

public interface View<T> {
    public T getTarget();
}

A View is an […]Accessible that provides possibly transformed access to a subset of this target data.

Since we could not see any reasonable application for the bare View interface, we will not implement it but use it in informal speech about virtualized access. We will also use Target to refer to the underlying data.

Examples are IterableIntervalSubset, arbitrary Neighborhoods, RandomAccessibleView (formerly called View), RealRandomAccessibleView, TransformedRandomAccessibleView, StructuredElement, HyperShpere, SuperInterval…

We discussed the View concept (again) and Transforms:

We agreed that for Views that give transformed access it is more natural to specify the transform with respect to the underlying data. That is, the transform is applied to a coordinate in the data system to obtain a coordinate in the view system.

In matrix notation, a transform would look like y = Tx. Here, it is natural to refer to x as the source vector and to y as the target vector.

Applied to Views, we will therefore completely reverse our previous opinion and refer to the underlying data as the Source.

The View<T> interface would look like

public interface View<T> {
    public T getSource();
}

For Transforms, we will adapt and simplify the CoordinateTransform and related interfaces from Fiji's mpicbg submodule. There will be an integer and a real version as for Positionable and Localizable. Transform and RealTransform can specify the number of dimensions of its source and target domain. They look like

public interface Transform {
    public int numSourceDimensions();
    public int numTargetDimensions();
    apply(long[] source, long[] target);
    apply(Localizable source, Positionable target);
}

public interface RealTransform {
    public int numSourceDimensions();
    public int numTargetDimensions();
    apply(double[] source, double[] target);
    apply(RealLocalizable source, RealPositionable target);
}

The apply methods transfer source coordinates into target coordinates.

There will be an invertible version for each of these interfaces

public interface IvertibleTransform extends Transform {
    applyInverse(long[] source, long[] target);
    applyInverse(Positionable source, Localizable target);
    InvertibleTransform inverse();
}

public interface IvertibleRealTransform extends RealTransform {
    applyInverse(double[] source, source[] target);
    applyInverse(RealPositionable source, RealLocalizable target);
    InvertibleRealTransform inverse();
}

Note that target is transferred into source in that case.

We have extensively discussed the fact that for rendering a mapped image, an inverse transformation is required whereas one prefers to define transformations in the forward manner. Nevertheless, a view (renderer) should use the forward defined instance of a Transform to create its result with changes applied to that transform having a direct effect on the result (no creation of a fresh `inverse'). Other than in mpicbg, we will achieve this by implementing Renderers for forward transformations (those transformations that can be specified in one direction only will be defined as forward transformations). For invertible transformations, the Renderer will use a final Inverter that is a forward transformations that uses the inverse apply methods of an invertible transformation for its apply method.

How to proceed with the imglib2 namespace?

We think that it would be a great idea to be able to run both imglib and imglib2 together. This would relieve us from the need to port all the legacy imglib code into imglib2. Also, a shorter package hierarchy would be nice. With org.imglib2, we would match the Maven convention and the URL is still not registered...

Why not just call it "org.imglib" then? Wouldn't clash with "mpicbg.imglib" of imglib1.

Because imglib.org is registered by somebody else already. Stephan Saalfeld 14:38, 24 March 2011 (CET)

2011-03-21

Efficient access for RandomAccessibleViews

I've been thinking all day about a reasonable concept for concatenation and simplification of Transforms for RandomAccessibleViews. No satisfying result so far, so at least I want to start writing down what it might look like in the end and what the problems are.

Simplifying View Transformations

The idea is that a view should always give you a RandomAccess that is as efficient as possible. When views are layered on top of each other it is often possible to combine and simplify their transformations.

Assume that we have a 90 degree rotated view A of an image I. and a 90 degree rotated view B of that view A. A RandomAccess on B could simply rotate by 90 degree and wrap a RandomAccess on A. The RandomAccess on A would in turn simply rotate by 90 degree and wrap a RandomAccess on I.

However, this would not a good idea for a hierarchy of many views. Instead, when we request a RandomAccess from B then we would like one that directly wraps a RandomAccess on I and rotates coordinates by 180 degree.

Transformable (leek)

Coincidentally, I was playing around with an idea for ROIs that might be applicable here. You'd like to be able to take a ROI in one space, apply a transform and get a ROI in the transformed space by transforming key internal coordinates. So that lead me to think of making ROIs implement "Transformable" if they wanted to. The some transformation could be applied to a transform itself - have a transform implement transformable so that its internal matrix could get re-jiggered to operate in the new space.

Here's the interface that I was planning on implementing:

package mpicbg.imglib.transform;
/**
 * @author leek
 *
 * A class is transformable if it can produce a copy of
 * itself in the transformed space using the supplied transform.
 * 
 * Note that a class may require either a Transform or an InvertibleTransform
 * depending on whether the strategy is to transform coordinates in the
 * source space into the destination space or to generate the object in
 * the destination space by sampling invert-transformed points in the
 * source space.
 * 
 */
public interface Transformable<O,T extends Transform> {
   /**
    * Generate a copy of the object in the transformed space.
    * @param t the transform that maps points in the source space to those
    *          in the destination space.
    * @return a copy built to operate similarly in the transformed space.
    */
   public O transform(final T t);
}

You'd only want to implement Transformable in cases where the object has internal state which allows the transformed object to operate more efficiently than transformation of the object's inputs, followed by application of the object's function. For ROIs, the savings are clear - cost to transform a handful of vertices in a polygon versus cost of back-transforming millions of coordinates into the original space.

Now it obviously can be made to work for ROIs and it can be made to work to combine two linear transformations (dot product of matrices, right?). But other things are more complex and require more thought. At the end of it, each output is a function of the inputs and the trick is to composite a function that performs the operation. For a transform T1 to be transformable by transform T2, it seems that you need to combine and reduce an equation on the inputs of T1 for each of the outputs of T2. It's a difficult enough problem that you might not want to make a transform generally transformable - you might want to have compositors with special knowledge regarding which classes can be combined to yield a new one and how it's done.

Out-Of-Bounds Handling

In imglib2, out-of-bounds access is handled by ExtendedRandomAccessibleInterval If you have a RandomAccessibleInterval you can wrap it into an ExtendedRandomAccessibleInterval which extends to infinity. Like so:

F interval; // where F extends RandomAccessibleInterval< T >
OutOfBoundsFactory< T, F > factory = new OutOfBoundsMirrorFactory< T, F >( OutOfBoundsMirrorFactory.Boundary.SINGLE );
RandomAccessible< T > extended = new ExtendedRandomAccessibleInterval< T >( randomAccessible, factory );

ExtendedRandomAccessibleInterval is also a RandomAccessibleView. It might be inserted at any point in a view hierarchy. Here is an example:

Img< FloatType > img = LOCI.openLOCIFloatType(...);
RandomAccessibleView< FloatType > view1 = Views.extend( img );	
RandomAccessibleIntervalView< FloatType > view2 = Views.superIntervalView( view1, new long[] {-20, -20}, new long[] {157, 157} );		
RandomAccessibleView< FloatType >         view3 = Views.extend( view2 );	
RandomAccessibleIntervalView< FloatType > view4 = Views.superIntervalView( view3, new long[] {-100, -100}, new long[] {357, 357} );

The original img looks like this: