Ticket #1693 (closed feature: fixed)

Lee, can you enhance this ticket description or file tickets underneath it (as its a feature ticket). Perhaps you could email Kevin to see if he has any input.

CellProfiler has a basic level of functionality and compatibility with ImageJ2:

• Currently building against 2.0.0-beta6
• Maven integration
  • CellProfiler has a POM for its Java dependencies
  • Jars are found using mvn dependency:build-classpath
  • CellProfiler developer version uses mvn to fetch remote dependencies
• ImageJ scripting
  • CP lets user select the script engine
  • Scripts are loaded and stored with the pipeline
• ImageJ plugins
  • CP uses the module discovery system to find modules
    • Constructs menu for plugin selection from the module's menu path
    • Constructs CellProfiler settings to mirror a module's widgets
    • Detects input and output images in the module and constructs appropriate image settings to transfer images back and forth

From my perspective, there are a couple rough patches:

• We create a bitmap ROI for input images that have, in CellProfiler terminology, "masks" and we try and coalesce the ROIs in a display into a mask for output images. I haven't tested this in a while.
• If I recall right, in the early phase of ImageJ 2.0 development, modules would have Datasets or DatasetViews as input and output parameters and I believe now they have Displays. I do support all three as input or output parameters, but I should check to make sure I'm properly handling all of the parameter types that IJ2 supports.
• I would like to run IJ2 truly headless (with awt.headless=true). Ideally, I'd like to have a headless profile for IJ2 in a POM so that I can fetch the classpath from that POM when starting the JVM with awt.headless=true.
• Support for IJ1 - Users can reference IJ1 plugins through scripts, but they will probably not appear on the command menu. I think our users want ImageJ mostly to run specific plugins and these are still mostly IJ1 plugins.
• Support for images in scripts. It would be useful if users could bind CellProfiler input images to variables in their scripts and if they could have CellProfiler collect output images from the script context after it was run. As a fallback, users can import a CellProfiler image into ImageJ, find it using their script and then import the currently active image from ImageJ to CellProfiler.
• Measurements / Segmentation - CellProfiler produces per-image and per-particle measurements and a segmentation of an image into particles/objects is a first-class entity in CellProfiler. We should determine how to map measurements and segmentations to their ImageJ 2 analogs and determine a timeline for carrying out the work.

Thanks for the summary, Lee.

We create a bitmap ROI for input images that have, in CellProfiler terminology, "masks" and we try and coalesce the ROIs in a display into a mask for output images.

Unfortunately, ROI integration will remain tricky for a while longer, until we really stabilize how ROIs work in ImgLib2 and ImageJ2. This was the topic of intense discussion at the recent ImgLib2 hackathon but there is still much design work to be done.

If I recall right, in the early phase of ImageJ 2.0 development, modules would have Datasets or DatasetViews as input and output parameters and I believe now they have Displays. I do support all three as input or output parameters, but I should check to make sure I'm properly handling all of the parameter types that IJ2 supports.

Yes, all three (Dataset, DatasetView and ImageDisplay) will probably need to be supported for the foreseeable future. There are various commands that use each, and third-party commands could certainly output any of those. If there are convenience methods we could add at the Java level to make this easier for you, let me know.

I would also add that the class hierarchy may be changing a bit before 2.0.0 final, but there will still probably be three layers that map conceptually to those above. So whatever logic you have to handle them is very likely to remain relevant.

I would like to run IJ2 truly headless (with awt.headless=true).

Hopefully, using -Dawt.headless=true works now. If not, let us know so we can investigate.

Ideally, I'd like to have a headless profile for IJ2 in a POM so that I can fetch the classpath from that POM when starting the JVM with awt.headless=true.

Good idea! This is done in 1df9630f. You can now run:

cd app && mvn -Djava.awt.headless=true dependency:list

To resolve the list of dependencies sans Swing or any other UI.

Relatedly: note that for each discovered plugin, you can query myPluginInfo.getAnnotation().headless() to find out whether it explicitly condones being executed in a headless environment. (Unfortunately, we have not propagated this attribute into ModuleInfo.isHeadless() yet.) All the Swing-specific commands, for example, do not.

Support for IJ1 - Users can reference IJ1 plugins through scripts, but they will probably not appear on the command menu. I think our users want ImageJ mostly to run specific plugins and these are still mostly IJ1 plugins.

Yes, that's a tricky thing. Porting the plugins to IJ2 commands is the robust way to go, like you did for Tubeness. But it's a lot of work and is a long road. Still, some IJ1 plugins may very well work headless, especially if we expand our usage of Javassist when running headless to do things like override how GenericDialog works so it's not a java.awt.Dialog.

Support for images in scripts. It would be useful if users could bind CellProfiler input images to variables in their scripts and if they could have CellProfiler collect output images from the script context after it was run. As a fallback, users can import a CellProfiler image into ImageJ, find it using their script and then import the currently active image from ImageJ to CellProfiler.

I don't know enough about how the CP-IJ integration currently works to comment much on this issue...

Measurements / Segmentation - CellProfiler produces per-image and per-particle measurements and a segmentation of an image into particles/objects is a first-class entity in CellProfiler. We should determine how to map measurements and segmentations to their ImageJ 2 analogs and determine a timeline for carrying out the work.

Yep. That is very relevant to the long-standing and dreaded ticket #660. As for a timeline, I fear that ImageJ2 development will be a bit slower over the summer, due to various project commitments that we have. But that ticket is one of my top priorities as far as ImageJ2 is concerned.

In conclusion, it is not clear to me that there are any concrete action items to come out of this discussion beyond those of which we are already aware. Lee, if there is anything specific you need for the CP-IJ integration, file a new ticket for it, or just let me know and we can discuss. But for now, I am closing this ticket. Thanks again!