# Debugging Exercises

 This page presents exercises for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

Debugging is the art of determining the cause and/or location of a problem. The purpose of this guide is to provide developers practical, hands-on experience using a variety of debugging techniques to identify problems in code.

# Requirements

As ImageJ is built using the SciJava principles of project management, this guide assumes a basic familiarity with these topics and tools, especially:

Finally, you should read the Debugging in Eclipse section, at least, of Lars Vogel’s Eclipse tutorial. Note that there is significant overlap between the topics covered between these tutorials. Vogel’s guide does a fantastic job of laying out the debugging interface and tools in Eclipse, while this guide focuses on providing hands-on practice and explaining the reasoning for when to use these tools.

If you find yourself confused when this tutorial asks you to do something in Eclipse (e.g. “start debugging”, “step into”) it’s almost certainly covered in Vogel’s guide.

# What not to do: print statements

For many developers, the first tool in their debugging toolbox is the print statement. Print statements are easy to lean on as a safety crutch: you don’t need any special knowledge to use them, and they often work to answer common questions (e.g. “why is this variable null here?”, “how many elements are in my array here?”).

However, there are critical drawbacks to trying to debug via print statement:

• They are slow. If you realize you need to move or add a print statement, you need to recompile your code and re-launch your application.
• They are part of the code. Adding print statements changes line numbers, causes git to pick up modifications to the source code, and can even affect performance and/or behavior.
• They are limited. Even the most basic breakpoint and expression evaluation in Eclipse debug mode gives you vastly more power and flexibility over print statements.

Learning to use debugging tools is, understandably, a burden: it’s “one more thing” to learn as a developer. But if you want to develop ImageJ plugins, you will almost certainly run into cases where debugging is necessary. So you may as well start familiarizing yourself with the tools now, gaining skills and perspectives that will serve you well throughout your career.

# Using this guide

The goal of these exercises is not to solve the problems, but to build up your toolbox of troubleshooting techniques and develop your intuition for when to apply each technique. To keep exercises simple and focused, most do not explicitly use ImageJ. But once you learn how to debug an external Java application, you will have the knowledge to apply any of these techniques to a rich, and complex, application like ImageJ.

Because this project is intended to help new developers practice troubleshooting skills, you may find these examples contrived - indeed, they are. Exercises are kept simple and focused to allow practice of targeted techniques. If you have complete knowledge and understanding of the code there isn’t really a need for troubleshooting: it is trivial to see why something is behaving incorrectly. Thus the source of these exercises is divided into hidden and visible packages. Users are strongly recommended to only inspect and set breakpoints from the visible classes. From a development point of view, consider the hidden package a 3rd-party library that you may not have control over, or access to the source code.

Changing the source code to actually fix the bugs is outside the scope of this guide, but motivated users are of course welcome to do so for practice.

If at any time you need to revert changes to the imagej-troubleshooting repository, you can always do so via the command:

git reset --hard origin/master


# Exercises

## Exercise 1: Stack Traces and Breakpoints

Goals

• Interpret a stack trace
• Practice setting breakpoints in Eclipse
• Use the Variables window to inspect variable values
• Use the navigation commands to execute code in Debug mode

Breakpoints are a fundamental tool of debugging. They provide a way to instruct Java to stop code execution when a certain line of code is encountered, providing a chance to explore actively running code.

To get started in this exercise, open up the source file - E1BasicBreakpoints - and run it to get an idea of what’s going on. We should see a simple stack trace:

Stack traces are a common starting point for debugging, as they are typically automatically produced when something goes wrong that the program was not prepared to handle. Java programs are executed in Last In, First Out order; that is, starting with the main method, as methods are called they are added to the top of the stack, the method at the top is what’s currently running, and when a method completes it is removed from the stack, returning the program to the next method in line. When an exception occurs, a stack trace is printed, showing the order that methods have been queued, with the top of the stack being the location of the exception (and thus a likely place to start looking for problems!).

So looking back at the stack trace we got, we can see what went wrong (tried to use a null object) and where it happened (the line number at the top of the stack), but we don’t know why the object was null at that point - which would be the actual root cause of the exception.

To investigate further, try to complete the following debugging steps:

1. Set a breakpoint in the main method, before makeAThing is called
2. Debug the file as a Java application
3. When the breakpoint is encountered, step in to the makeAThing method
4. step over the line constructing a new Object
5. step out of the makeAThing method
6. In the Variables window, look at the value of the Object variable
7. ‘resume’ execution until the program completes
 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

Takeaway

• Stack traces are helpful in identifying starting points for debugging
• The Debug view allows line-by-line execution of code and inspection of variable values to help us pinpoint errors

## Exercise 2: Expressions

Goals

• Set breakpoints on exception creation
• Use the Expressions window to evaluate code at runtime

Although breakpoints allow us a chance to peek inside running code, many times when debugging you’ll find that you’re missing a piece of information not provided by the current code - perhaps you’d like to call a utility method, or construct a new object temporarily. All of these operations are Java expressions - statements in Java syntax resolving to a single value (essentially - one line of code). Instead of making these changes in your program’s code and recompiling, in Debug mode we can dynamically evaluate any expression on-demand, without changing the source.

Start by opening the E2EffectiveExpressions source and running it. Like the previous exercise, we have a stack trace to start from:

Try setting a breakpoint on the conditional line:

if (index < 0 || index >= list.size()) {

 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

Since we are only interested in the processElementAtIndex method when a problem actually occurs, let’s try something different:

1. From the Breakpoints window in the Debug perspective, delete the old breakpoint
2. Now use the Add Java Exception Breakpoint button to add a breakpoint to IllegalArgumentException
3. Debug the program. When it stops, inspect the Variables window.

At this point, we know there is a problem accessing the 99999th element of the list, but the variables window doesn’t tell us exactly what the problem is. We can manually expand and explore the list variable - but given its size that could be cumbersome.

Instead, let’s use expressions. If it’s not already visible, open the Window > Show View > Expressions view.

In this view, we can add any number of Java expressions to evaluate. We can call methods from, and on, any variables and classes visible from the current scope.

Warning: if you evaluate expressions that change the state of variables, for example List.add, then those changes will persist. Sometimes you do legitimately need to alter the state of variables while debugging (for example, to force the conditions in which a bug appears). Just be aware that you could also be introducing problematic behavior when evaluating expressions.

To complete this exercise:

1. Write an expression that calls the size() method of our list.
2. Write a 2nd expression that evaluates to the value of the index variable
 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

Takeaways

• Setting breakpoints on exceptions avoids unnecessary breakpoint hits (and can be useful when we aren’t sure where to set the breakpoint)
• The Expressions window of the Debug view allows us to evaluate arbitrary Java expressions without modifying our source code

## Exercise 3: Conditional breakpoints

Goals

• Create a breakpoint that triggers after a specified number of hits
• Create a breakpoint that triggers when a certain condition is true

Breakpoints trigger every time the corresponding line would be executed, which may be undesirable for repeated code blocks. It may be enough to carefully consider the breakpoint placement - on an exception, or within a conditional block. But when these options aren’t available, we can make our breakpoints more powerful by triggering only when there’s something of interest to see.

Start by opening the E3ConditionalCrisis source and running it. This time our console output looks a bit different:

In addition to the exception stack trace, the program itself appears to have found an invalid object, causing the processing to go unfinished. Although we could set a breakpoint on the exception, as we did in exercise 2, the exception is actually happening after the more interesting part of the program - the loop. As we learned in exercise 2, breakpoints in code that is called repeatedly are annoying, so let’s see what we can find by attaching conditions to our breakpoint.

First set a breakpoint on the line after the everythingIsOK assignment:

everythingIsOK = ObjectAnalyzer.processElementAtIndex(myArray, i);
i++;


Then try the following:

1. Open the Breakpoints window
2. Right-click our breakpoint and select Breakpoint Properties…
3. Check the Hit Count box and set it to the object number printed in the error message.
4. Try debugging
 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

Using count-based conditional breakpoints can be very useful if the error is deterministic. In this case we need to try something different. We know the everythingIsOK flag reflects the integrity of the object at a given index - so what we really want to use here is a breakpoint that stops in the loop when the everythingIsOK flag is set to false. Fortunately, breakpoints have an optional “Conditional” flag - where we can enter any Java statement that resolves to a boolean value. Try it out:

1. Open the Breakpoints window again
2. Open the properties of our breakpoint
3. Uncheck the Hit Count box
4. Check the Conditional box, and “Suspend when ‘true’”
5. Enter the condition we want to check
6. Try debugging again

Were you able to get the breakpoint to stop in the loop only when a problem is encountered?

 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

Takeaways

• Setting a hit count on a breakpoint is useful if problematic code is called multiple times
• If problems appear randomly, using a conditional expressions on the breakpoint can help

## Exercise 4: ImageJ plugins

Goals

• Start a debugging session in Eclipse that connects to a running ImageJ instance

Exercises 1-3 are abstract, self-contained programs. The E4RemoteResearch class, on the other hand, is an actual ImageJ plugin. Although plugin developers typically set up tests to run their plugin within ImageJ, it is impossible to test all possible runtime environments: users could have a variety of update sites enabled, custom plugins installed, etc… and each modification is an opportunity for dependency skew and clashes with your plugin. When problems do arise, it is invaluable to have the ability to debug a running copy of ImageJ itself.

E4RemoteResearch does still have a main method to demonstrate the expected output for this plugin - in this case, simply printing the concrete implementation of the ConsoleService. Run the class in Eclipse and you should see some simple output in the console:

I found our console service! Look: class org.scijava.console.DefaultConsoleService


Next, we want to run this plugin in ImageJ and see what happens:

1. On the command line, run mvn clean install from the imagej-troubleshooting directory, to build a .jar
2. Copy the produced jar (e.g. target/imagej-troubleshooting-0.1.0-SNAPSHOT.jar) to the ImageJ.app/jars directory of your ImageJ installation
3. Start ImageJ

Note that the menu path of the plugin is specified in the class’s annotation:

@Plugin(type = Command.class,menuPath = "Plugins>Troubleshooting>E4 - Print ConsoleService")


So, you can now run the E4 - Print ConsoleService command either via the menus or command finder. You should get an exception:

In order to connect Eclipse to ImageJ, we need to close our running instance and launch ImageJ from the command line, which allows us to set the debug flag, e.g.:

ImageJ.app/ImageJ-linux64 --debugger=8000


Note: On Windows we need to add the console flag; see this issue for more information.

ImageJ.app/ImageJ-linux64 --debugger=8000 --console


This will start up ImageJ in a mode that’s able to communicate with Eclipse. Next we need to connect Eclipse to the running ImageJ instance:

1. Right-click the E4RemoteResearch source file in the Package Explorer
2. Select Debug As > Debug Configurations...
3. Scroll down the list of configurations (or search) until you find Remote Java Application
4. Double-click Remote Java Application to create the debug configuration
5. Rename the configuration to E4RemoteResearch-remote to distinguish it. If necessary you can update the port to match what was specified when you started ImageJ.
6. Click the Debug button to start a remote debugging session

At this point ImageJ and Eclipse should be communicating. It’s important to understand that the information flow goes from ImageJ to Eclipse: ImageJ says “I am at line number X in class Y” and if Eclipse looks in the source files it knows about, and stops if it finds a breakpoint at that location.

However, when Eclipse looks at its source files, it’s not looking at the actual classes that were used to launch the remote Application. In fact, the classes Eclipse looks in depend entirely on the classpath of the project that was used to start the remote debugging session. So when you are remote debugging, there are two best practices to follow:

• Launch the remote session from the project you’re interested in (if you want to debug a class in scijava-common, don’t start the session from the imagej-legacy project)
• Make sure the source in Eclipse matches what’s in the remote application (an easy way to guarantee this is to build the .jar and copy it to the application)

Since we already followed these best practices, we can now finally debug our plugin:

1. In Eclipse, set a breakpoint on the line where the ConsoleService> is being cast to DefualtConsoleService
2. In ImageJ, run the E4 - Print ConsoleService command
3. In Eclipse, when the breakpoint is hit, inspect the value of the consoleService field
 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

Takeaways

• Launching ImageJ from the command line allows us to add useful flags and collect debugging information
• Attaching Eclipse to a running ImageJ lets us debug plugins in a “production” environment

## Exercise 5: Git history

Goals

• Practice using git bisect to locate historical breakages

Debugging code directly via the techniques we’ve discussed thus far is not always practical. If the code is excessively complex, or the problem subtle, it may not be practical to try and step through code execution - you may not even be sure where to start. Keeping code well-structured and well-documented can help here, but we have another resource to draw on: the history of changes to our code, via git commits - assuming appropriate development practices are used. Identifying the breaking change gives us more information to use in our diagnosis (and in some cases, can be fixed with a simple git revert]).

So, the first thing we’ll do is run the E5HistoricalHysteria class and verify that it fails. For this exercise we have some additional information: this class used to run successfully, and a tag was made at that point.

To find what tags are available, on the command line we can run:

$hinerm@Nyarlathotep ~/code/imagej/imagej-troubleshooting (master) git fetch --tags$ hinerm@Nyarlathotep ~/code/imagej/imagej-troubleshooting (master)
git tag -l
e5-good-maths


The purpose of the bisect tool is to search our commit history to find the breaking commit. Although you could search one by one through the commit history, this would take linear time in proportion to the number of commits to search. Bisecting performs a binary search, speeding up the process significantly. It also allows much of the process to be automated.

To use the bisect tool we just need to know two commits: one where the test worked, and one where the test failed. In this case, we know the e5-good-maths tag worked, and our current state is broken, so we can start the bisect:

$hinerm@Nyarlathotep ~/code/imagej/imagej-troubleshooting (master) git bisect start master e5-good-maths Bisecting: 9 revisions left to test after this (roughly 3 steps) [d2e45589ca3671c93f772d4310fed9653ca1569b] E5: Zach likes the maths!$ hinerm@Nyarlathotep ~/code/imagej/imagej-troubleshooting ((d2e4558...)|BISECTING)


In each step of the bisect git will automatically move you to a new commit to be tested. “Testing” simply involves trying to reproduce the error and letting git know if this commit is good or bad. While bisecting, the whole local repository is in a special “BISECTING” mode - so that git can remember your answers for each commit. If you ever make a mistake (marking a commit incorrectly) you can abort the process via git bisect reset.

Now that you’re bisecting, follow these steps until you’ve identified the failing commit:

1. In Eclipse, refresh the imagej-troubleshooting project, to ensure that the current commit is what’s being tested
2. Run the E5HistoricalHysteria class
3. On the command line, use either git bisect bad or git bisect good depending on if an exception was thrown or not

After marking the last commit good or bad, bisect will print out the first bad commit. You can use git bisect reset to finish bisecting.

 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

Takeaways

• Use all resources available when debugging - even the git history can be useful (assuming it’s well-maintained!)

## Exercise 6: Print stack trace

Goals

• Identify problematic code when no feedback is given

When debugging we’re trying to identify why a program isn’t behaving as expected. Often this starts in response to an unhandled Java exception, which comes with a helpful stack trace to point us in the right direction. Unfortunately, there are also times when no information as given - such as when the JVM hangs (gets stuck) or crashes without warning. In this exercise we’ll look at another way to extract information from our application: by forcing a stack trace to be printed.

As we did in exercise 4, the first thing to do is build the imagej-troubleshooting .jar and install it in your ImageJ.app/jars directory. Then you can start up ImageJ and run the command for this exercise:

Plugins > Troubleshooting > E6 - Start Looping


After running this command, you should notice ImageJ sitting around for a few seconds… and then close unexpectedly. As this crash doesn’t give us any leads to follow, the next thing we should do is look at the code:

NotALoop.dontLoopTwice();

NotALoop.dontLoopThrice();

NotALoop.dontLoopForever();

NotALoop.loopForever();


We see four methods are being called. What do they do..? No idea! But we know one of them (at least) is bad. So let’s do what we can to learn what’s happening in our application up until the crash.

To investigate further, close ImageJ (if it’s running) and launch it again from the command line, e.g.:

ImageJ.app/ImageJ-linux64


We actually don’t need any extra flags this time, as this technique isn’t specific to ImageJ. When you run a program from the command line, your console is directly tied to the running instance:

In this state, we can still send signals to the running application (for example - ctrl\|c to kill the app).

When running a Java application, we can use ctrl\|\\ to print a stack trace. Note: this shortcut may vary based on your OS and keyboard: see the print stack trace instructions for more information.

With this knowledge:

1. Run the E6 - Start Looping command from ImageJ
2. Before ImageJ crashes, switch back to the terminal and use ctrl\|\\ to print a stack trace
3. Because we want to guess what the last method to run is, keep taking stack traces until ImageJ crashes
4. Look back through the console text and find the last method to be executed

Hint: raw stack dumps like this are not the easiest to read. Stack traces for all the threads in the JVM are printed, as well as additional information we’re not interested in. Look for the the sections of stack traces sorted by thread, like you would see in an exception message, and find the E6SleuthingSilence class. Whatever follows that entry is at the top of the stack, and thus what was being processed on that thread when you took the stack trace.

 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

Takeaways

• Manually acquiring stack traces is useful when we don’t have anything else to go on (e.g. if our program crashes without warning, or becomes unresponsive)

## Exercise 7: Out of memory

Goals

• Learn how to acquire heap dumps
• Analyze a heap dump for potential problems

Memory problems are a different kind of beast. Java is constantly in the process of reclaiming objects that are no longer in use (garbage collection). If something mistakenly holds onto an object when it shouldn’t you have a memory leak (for example, a user closes an image, but an array of the pixel data is preserved). Depending on the size of the leak, it might not even be encountered (e.g., only after opening 10,000 images).

Memory errors can be especially tricky to reproduce because, once memory is used up, any object creation can trigger an OutOfMemoryError. So the resulting stack trace may be misleading. Instead, what we want to do is examine the Java heap space (where object instances are stored) and figure out what went wrong.

First things first, run E7InvestigateImpressions and see what happens. You should see it print array names for a while, and then eventually hit an OutOfMemoryError.

Looking at a snippet of the code:

for (int i = 0; i < 100; i++) {
System.out.println(ObjectMaker.getFloatArray(size));
System.out.println(ObjectMaker.getLongArray(size));
System.out.println(ObjectMaker.getDoubleArray(size));
}


We see that objects are being created, but we aren’t storing any references to them. They should just be printed and discarded. So presumably one of these methods is doing something nasty, and it’s up to us to figure out which by analyzing the heap space. In particular, we want to analyze the heap space at the time of the error. jvisualvm is used to attach to running Java programs; so we will need to set a breakpoint on the OutOfMemoryError itself - similar to what we did in Exercise 2: Expressions - and debug the E7 program.

Once the OutOfMemoryError is encountered our breakpoint will trigger. To acquire the heap dump:

1. Open jvisualvm
2. From the list of local applications, right-click on net.imagej.trouble.visible.E7InvestigateImpressions and select the “Heap Dump” option.
3. The “Summary” view is open by default; switch to the “Classes” view of the heap dump
4. Sort by Size
 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

So now we know what’s taking up all of our memory - but we don’t actually know why. Although jvisualvm does have the tools to investigate further, the Memory Analyzer plugin for eclipse has several nice conveniences for further heap dump exploration. Let’s take a look:

1. With the heap dump selected in the Applications list of jvisualvm, use File > Save As... to save the .hprof file.
2. Back in Eclipse, open the “Memory Analysis” perspective (Window > Perspective > Open Perspective > Other... > Memory Analysis if you’ve never opened this perspective before)
3. File > Open Heap Dump... and select the file you created in step 1
4. Under the Actions column of the heap dump Overview, click on Histogram - which opens in a new tab
5. Filter the classes of the histogram based on the problematic class you identified in jvisualvm
6. Right-click the row for the problematic class and select the “Merge Shorted Paths to GC Roots > exclude weak/soft references” option.

There is a huge amount of information that can be browsed in the heap dump. Most of the options available via the right-click context menu are short-cuts for SQL queries. When we look at a “path to the GC root”, we’re looking at a chain of references between objects that’s keeping the selected class alive. Because we’re investigating memory leaks we can typically exclude weak and soft references, as these should be released before an OutOfMemoryError occurs.

 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

Tip: In this exercise we acquired the heap dump manually via jvisualvm. When you right-click on an application you can also set heap dumps to be acquired automatically when OutOfMemoryErrors occur. Heap dumps can also be acquired directly through the Eclipise MAT plugin, in the Memory Analysis perspective.

Takeaways

• To identify subtle problems, like memory leaks, it helps to learn how to use external tools - like jvisualvm

## Exercise 8: Profiling

Goals

• Identify performance bottlenecks

In the other debugging exercises, we look at programs failing due to errors. Another common concern is program performance - does your program run in a reasonable time frame? This often can not be deduced by simply looking at the code. So the first step in fixing performance is identifying the location of the slowdown.

Exercise 8 in pretty straightforward with the main function calling two functions - doStuff() and doMoreStuff(). Both functions are called one after the other for two minutes. Our goal is to find which function is slower than the other. One solution a programmer may think of is to keep track of time taken by comparing values returned by System.currentTimeMillis, but this method does not scale up for large program and requires modifications to the code. It is much more efficient and useful to use a profiler to monitor time spent in methods - which is part of the jvisualvm troubleshooting tool.

Steps for exercise

1. Insert a breakpoint on the while statement in main method
2. Launch JvisualVM. All running Java applications are shown in the in Applications Tab (left margin) of JvisualVM. Double Click to select E8PerceivingPerformance. Go to the profiler tab and click the CPU option.
3. Click on the settings checkbox on upper right corner of Jvisualvm. In CPU settings , make sure the class being profiled is net.imagej.trouble.** instead of net.imagej.trouble.visible.**, or the profiler won’t be looking in the right place
4. Switch to Eclipse and resume the execution of code
5. Wait for the stipulated time , twiddle your thumbs. Or you can switch to JvisualVM and see how much time is taken by each of function in real time.
6. Example output
 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

 This page has approaches for ““software developers”” to use for debugging ImageJ. If you are a ““user”” looking to troubleshoot issues, see the Troubleshooting page.

# Launching ImageJ in debug mode

To debug problems with ImageJ, it is often helpful to launch it in debug mode. See the Troubleshooting page for instructions.

# Debugging plugins in an IDE (Netbeans, IntelliJ, Eclipse, etc)

To debug a plugin in an IDE, you most likely need to provide a main() method. To make things easier, we provide helper methods in fiji-lib in the class fiji.Debug to run plugins, and to load images and run filter plugins:

import fiji.Debug;

...

public void main(String[] args) {
}


You need to replace the first argument by a valid path to a sample image and the second argument by the name of your plugin (typically the class name with underscores replaced by spaces).

If your plugin is not a filter plugin, i.e. if it does not require an image to run, simply use the Debug.run(plugin, parameters) method.

## Attaching to ImageJ instances

Sometimes, we need to debug things directly in ImageJ, for example because there might be issues with the plugin discovery (ImageJ wants to find the plugins in */plugins/*, and often we want to bundle them as *.jar* files, both of which are incompatible with Eclipse debugging). JDWP (*Java Debug Wire Protocol*) to the rescue!

After starting the Java Virtual Machine in a special mode, debuggers (such as Eclipse’s built-in one) can attach to it. To start ImageJ in said mode, you need to pass the *–debugger=* option:

ImageJ.app/ImageJ-linux64 --debugger=8000


In Eclipse (or whatever JDWP-enabled debugger) select the correct project so that the source code can be found, mark the break-points where you want execution to halt (e.g. to inspect variables’ values), and after clicking on Run>Debug Configurations… right-click on the Remote Java Application item in the left-hand side list and select New. Now you only need to make sure that the port matches the value that you specified (in the example above, 8000, Eclipse’s default port number).

If you require more control over the ImageJ side – such as picking a semi-random port if port 8000 is already in use – you can also use the -agentlib:jdwp=… Java option directly (*–debugger=* is just a shortcut for convenience):

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=server=y,suspend=y,transport=dt_socket,address=localhost:8000 --


(the marker separates the Java options – if any – from the ImageJ options). Once started that way, ImageJ will wait for the debugger to be attached, after printing a message such as:

Listening for transport dt_socket at address: 46317

Note: calling imagej -agentlib:jdwp=help – will print nice usage information with documentation of other JDWP options.

## Attach ImageJ to a waiting Eclipse

Instead of making ImageJ the debugging server, when debugging startup events and headless operations it is easier to make ImageJ the client and Eclipse (or equivalent) the server.

In this case you start the debugging session first, e.g. in Eclipse debug configurations you specify “Standard (Socket Listen)” as the connection type. Then, simply start ImageJ without the “server=y” flag to connect and debug:

ImageJ.app/ImageJ-linux64 -agentlib:jdwp=suspend=y,transport=dt_socket,address=localhost:8000 --


# Monitoring system calls

## Linux

On Linux, you should call ImageJ using the strace command:

strace -Ffo syscall.log ./imagej <args>


## MacOSX

Use the dtruss wrapper around dtrace to monitor system calls:

dtruss ./imagej <args>


## Windows

To monitor all kinds of aspects of processes on Windows, use Sysinternal’s Process Monitor.

# Debugging shared (dynamic) library issues

## Linux

Set the LD_DEBUG environment variable before launching ImageJ:

LD_DEBUG=1 ./imagej <args>


## MacOSX

Set the DYLD_PRINT_APIS environment variable before launching ImageJ:

DYLD_PRINT_APIS=1 ./imagej <args>


## Windows

Often, dynamic library issues are connected to a dependent .dll file missing. Download depends.exe and load the .dll file you suspect is missing a dependency.

# Debugging JVM hangs

When the Java VM hangs, the reason might be a dead-lock. Try taking a stack trace. If you have trouble, you can try one of the following advanced techniques:

1. You can use the jstack command (you don’t need to run ImageJ from the command line in this case). This requires that you first find the PID (process ID) of ImageJ. You can do so by running: jps

from the command line to print a list of running Java processes. If you’re not sure which PID is ImageJ’s, you can close ImageJ, run jps, open ImageJ and run jps again. Whichever PID is present in the second run but not the first is ImageJ’s. Then, to acquire a stack trace, just run:

jstack <ImageJ's PID>

2. For GUI-based debugging, can also attach to the ImageJ PID using the jvisualvm program that you can find in java///bin/. Here you can simply press a big *Thread Dump* button to view the stack trace. MacOSX users, please note that Apple decided that the VisualVM tool should no longer be shipped with the Java Development Kit; you will have to download it [from here](http://visualvm.java.net/download.html).

Regardless of which method you use to acquire the stack trace, to debug you will want to acquire multiple stack traces over time and compare. If all the stack traces are in the same method execution, then that’s the source of the deadlock (or slowdown).

# Debugging memory leaks

Sometimes, memory is not released properly, leading to OutOfMemoryExceptions.

One way to find out what is happening is to use jvisualvm (see #Debugging JVM hangs) to connect to the ImageJ process, click on Heap Dump in the Monitor tab, in said tab select the sub-tab Classes and sort by size. Double-clicking on the top user should get you to a detailed list of Instances where you can expand the tree of references to find out what is holding a reference still.

# Debugging hard JVM crashes

When you have found an issue that crashes the JVM, and you can repeat that crash reliably, there are a number of options to find out what is going on.

## Using gdb

Typically when you debug a program that crashes, you start it in a debugger, to inspect the stack trace and the variables at the time of the crash. However, there are substantial problems with gdb when starting the Java VM; either gdb gets confused by segmentation faults (used by the JVM to handle NullPointerExceptions in an efficient manner), or it gets confused by the threading system – unless you compile gdb yourself.

But there is a very easy method to use gdb to inspect serious errors such as segmentation faults or trap signals nevertheless:

./imagej -XX:OnError="gdb - %p" --


## Using lldb

On newer OS X versions, gdb has been replaced with lldb. For those familiar with gdb already, there is an LLDB to GDB Command Map cheat sheet which may be useful.

## Using the *hs_err_pid.log* files

The Java virtual machine (JVM) frequently leaves files of the format *hs_err_pid.log* in the current working directory after a crash. Such a file starts with a preamble similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f3dc887dd8b, pid=12116, tid=139899447723792
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x86d8b]  memcpy+0x15b
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


followed by thread dumps and other useful information including the command-line arguments passed to the JVM.

The most important part is the line after the line # Problematic frame: because it usually gives you an idea in which component the crash was triggered.

## Out of memory error

If the specific exception you’re receiving (or you suspect) is an OutOfMemoryError, there are JVM flags that can be enabled when running ImageJ to help pinpoint the problem:

./imagej -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/desired/path/


The first option:

-XX:+HeapDumpOnOutOfMemoryError


tells the JVM to create a heap dump (.hprof file) if an OutOfMemoryError is thrown. This is basically a snapshot of the JVM state when memory ran out.

The second option:

-XX:HeapDumpPath=/desired/path/


is not required, but convenient for controlling where the resulting .hprof file is written. Note that these heap dumps are named by PID, and thus are not easily human distinguishable.

After acquiring a heap dump, you can analyze it yourself, e.g. with a memory analyzer, or post on [Mailing Lists|imagej-devel] with a brief explanation of your problem.

# Debugging Java code with jdb

## How to attach the Java debugger jdb to a running ImageJ process

This requires two separate processes, ImageJ itself and the debugger. You can do this either in one shell, backgrounding the first process or in two shells, this is recommended. In the two shells do the following:

Shell 1
In the first shell, start ImageJ with special parameters to open a port (8000 in this case) to which jdb can connect afterwards:

./imagej -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y --


(Tested with Java 1.5.0, ymmv)

Shell 2
In the second shell, tell jdb to attach to that port:

jdb -attach 8000


## This is an ultra quick start to jdb, the default Java debugger

Hopefully you are a little familiar with gdb, since jdb resembles it lightly.

Notable differences:

• a breakpoint is set with “stop in ." or ":". Just remember that the class must be fully specified, i.e. .<subpackages...>.
• no tab completion
• no shortcuts; you have to write “run”, not “r” to run the program
• no listing files before the class was loaded
• much easier method to specify the location of the source: “use "
• “until” is “step”, “step” is “stepi”

Okay, so here you go, a little demonstration:

(If you attach jdb to a running ImageJ process, you have to use the line from the previous section instead.)

$jdb -classpath ij.jar ij.ImageJ > stop in ij.ImageJ.main Deferring breakpoint ij.ImageJ.main. It will be set after the class is loaded. > run run ij.ImageJ Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint ij.ImageJ.main Breakpoint hit: "thread=main", ij.ImageJ.main(), line=466 bci=0 main[1] use . main[1] list 462 //prefs.put(IJ_HEIGHT, Integer.toString(size.height)); 463 } 464 465 public static void main(String args[]) { 466 => if (System.getProperty("java.version").substring(0,3).compareTo("1.4")<0) { 467 javax.swing.JOptionPane.showMessageDialog(null,"ImageJ "+VERSION+" requires Java 1.4.1 or later."); 468 System.exit(0); 469 } 470 boolean noGUI = false; 471 arguments = args; main[1] print args[0] java.lang.IndexOutOfBoundsException: Invalid array range: 0 to 0  args[0] = null main[1] print args.length  args.length = 0 main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=470 bci=28 470 boolean noGUI = false; main[1] step > Step completed: "thread=main", ij.ImageJ.main(), line=471 bci=30 471 arguments = args; main[1] set noGUI = true  noGUI = true = true main[1] cont > The application exited # Inspecting serialized objects If you have a file with a serialized object, you can use this Beanshell in the Script Editor to open a tree view of the object (double-click to open/close the branches of the view): import fiji.debugging.Object_Inspector; import ij.io.OpenDialog; import java.io.FileInputStream; import java.io.ObjectInputStream; dialog = new OpenDialog("Classifier", null); if (dialog.getDirectory() != null) { path = dialog.getDirectory() + "/" + dialog.getFileName(); in = new FileInputStream(path); in = new ObjectInputStream(in); object = in.readObject(); in.close(); Object_Inspector.openFrame("classifier", object); }  # Debugging Swing (Event Dispatch Thread) issues Swing does not allow us to call all the methods on all UI objects from wherever we want. Some things, such as setVisible(true) or pack() need to be called on the Event Dispatch Thread (AKA EDT). See Sun’s detailed explanation as to why this is the case. There are a couple of ways to test for such EDT violations, see this blog post by Alexander Potochkin (current versions of debug.jar can be found here). # Debugging Java3D issues When Java3D does not work, the first order of business is to use Plugins › Utilities › Debugging › Test Java3D . If this shows a rotating cube, but the 3D Viewer does not work, please click on Help › Java3D Properties… in the 3D Viewer’s menu bar. ## Command line debugging If this information is not enough to solve the trouble, or if Test Java3D did not work, then you need to call ImageJ from the command line to find out more. From the command line, you have several options to show more or less information about Java3D. ### Windows & Linux Please find the *ImageJ-* executable in the ImageJ.app/ directory (on 32-bit Windows, that would be *ImageJ-win32.exe*. Make a copy in the same directory and rename that to *debug* (on Windows: *debug.exe*). Simply double-click that. On Windows, you will see a console window popping up; to copy information for pasting somewhere else, please right-click the upper-left window icon, select Properties…, activate the Quick Edit mode. Then mark the text in question by dragging the mouse with the left mouse button pressed, and copy it to the clipboard by right-clicking. On Linux, the output will be written to the file .xsession-errors in the home directory. ### MacOSX On MacOSX, you need to remember that any application is just a directory with a special layout. So you can call ImageJ like this from the Terminal (which you will find in the Finder by clicking on Go>Utilities. Example command line: cd /Applications/ImageJ.app/Contents/MacOS/ cp ImageJ-macosx debug ./debug  ## Show Java3D debug messages ./imagej -Dj3d.debug=true --  (Of course, you need to substitute the ./imagej executable name with the appropriate name for your platform.) Note: do not forget the trailing ; without them, ImageJ mistakes the first option for an ImageJ option rather than a Java one. Note, too: on Windows, you must not forget to pass the –console option (this can be anywhere on the command line). ## Windows-specific stuff On Windows, you can choose between OpenGL and Direct3D by passing -Dj3d.rend=ogl or -Dj3d.rend=d3d, respectively. Further, some setups require enough RAM to be reserved, so you might need to pass an option like –mem=1200m (make sure that you have enough RAM free before starting ImageJ that way, though!). If it turns out that memory was the issue, you can make the setting permanent by clicking ImageJ’s Edit › Options › Memory & Threads… menu entry. ## More Java 3D properties You can control quite a few things in Java 3D through setting Java properties. Remember, you can set properties using a command line like this: ./imagej -D<property-name>=<property-value> --  where you substitute ** and ** appropriately. You can have more than one such option, but make sure that they are appearing before the *--* on the command line, otherwise ImageJ will mistake them for ImageJ options. This list of Java 3D properties was salvaged from the now-defunct j3d.org website: Property Values Java 3D version Explanation j3d.rend "ogl" or "d3d" 1.3.2 Win32-only. Specifies which underlying rendering API should be used (thus allowing both Direct3D and OpenGL native DLLs to be installed on a singe machine. (default value "ogl") j3d.deviceSampleTime A time in millseconds 1.1 The sample time for non-blocking input devices (default value is 5ms). j3d.disablecompile N/A 1.2 If set turns off the ability to internally .compile() the scenegraph. j3d.docompaction true or false 1.3 Default true. Controls whether or not objects are removed from the render cache when they haven't been visibile for a while. If it is disabled, they stay in the render cache from when they are first visible until they are removed from the scene graph. j3d.forceReleaseView true or false 1.3.2 Default false. If this flag is set to true, the view is released after the Canvas3D is removed from the view. Can be used if you have memory leaks after disposing Canvas3D. Note: Setting this flag as true disables the bug fix 4267395 in View deactivate() j3d.implicitAntialiasing true or false 1.3 Default false. By default, full scene antialiasing is disabled if a multisampling pixel format (or visual) is chosen. To honor a display drivers multisample antialiasing setting (e.g. force scene antialiasing), set the implicitAntialiasing property to true. This causes Java3D to ignore its own scene antialias settings, letting the driver implicitly implement the feature j3d.optimizeForSpace true or false 1.3 Default true Java3d only builds display list for by-copy geometry. Set to false will cause Java3d to build display list for by-ref geometry and infrequently changing geometry using more space, but having greater speed. j3d.renderLock true or false 1.3 JDK requires getting the JAWT_DrawingSurfaceInfo and lock the surface before Java3D render on the canvas. (see comment on jdk/include/jawt.h in the SDK) Default false causes Java3D to lock the surface before rendering and unlock it afterwards for onScreen rendering in the Renderer thread. For OffScreen rendering and for front/back buffer swapping the lock will not acquired. Setting the value to true will force Java3D lock the surface using the AWT mechanism before swap() and for offScreen rendering. This may be useful for some driver/OS to workaround problem. But in general the default should work. j3d.threadLimit An integer 1.2 Controls how many threads may run in parallel regardless of how many cpu's the system has. Setting it to "1" will make the system act like a traditional OpenGL render loop. The default value is the number of CPUs in your machine + 1. j3d.transparentOffScreen true or false 1.3.2 Default false. If this flag is set to true the background of the off screen canvas is set to transparent. j3d.usePbuffer true or false 1.3.2 Default true. If this flag is set to false pbuffer will not be use for off screen rendering. j3d.viewFrustumCulling true or false 1.3.2 Default true. If this flag is set to false, the renderer view frustum culling is turned off. Java 3D uses a 2 pass view culling. The first pass is a loose view culling of the spatial tree, and the second pass is a tight view frustum culling in the renderer before sending the geometry down to the low level graphics API. This property is to control the renderer view frustum culling, and it will not affect the first pass view culling. javax.media.j3d.compileStats N/A ?? Output scenegraph compilation statistics javax.media.j3d.compileVerbose N/A ?? Output verbose message when compiling scenegraph OpenGL Only j3d.backgroundtexture true or false 1.3 Prior to Java3D 1.3 OGL version of Java3D used glDrawPixels() to render background, which is known to be very slow under Windows since most window driver did not accelerate the function. To workaround this performance problem current release uses textures for the backgrond under windows by default (glDrawPixels() is used as default under Solaris). Setting this flag to false will force Java3D fall back to use glDrawPixels() instead of texture when drawing background texture in case it provide better performance under some drivers. j3d.disableSeparateSpecular true or false 1.2 Default true enables the use of specular highlights in textures when using OpenGL 1.2. j3d.disableXinerama true or false 1.3 Solaris version only. Allows major performance boost when using dual screen environments with the X11 Xinerama extension enabled. To disable this feature you need JDK1.4. Detailed information in the release notes. j3d.displaylist true or false 1.2 Default true to use display lists (an OpenGL performance enhancing feature). False to disable for debugging. j3d.g2ddrawpixel true or false 1.1 If false, this will use texture mapping instead of glDrawPixel to flush the graphics2D to the screen. glDrawPixel is not accelerated on some older video cards (windows). j3d.sharedctx true or false 1.2 Default true for Solaris and false for windows. Shared contexts are used in OpenGL for DisplayLists and Texture Objects to improve performance. However some drivers have bugs causing weird rendering artifacts. This can be used to disable their use to see if this is the problem. j3d.sharedstereozbuffer true or false 1.2 Some framebuffers only have one Z buffer and share this between the left and right eyes. This may be the reason why they don't have quad buffer but can still support stereo by setting this flag to true. j3d.usecombiners true or false 1.3 Default false, uses the standard OpenGL all environment options. When set to true, it will make use of the Nvidia register combiner extensions to OpenGL for for Texture combine modes such as COMBINE_INTERPOLATE, COMBINE_DOT3. (ie GL_NV_register_combiners instead of standard OpenGL call glTexEnvi(GL_TEXTURE_ENV, ...)). It can be use in case like Dot3 texture when the driver does not support OpenGL extension GL_ARB_texture_env_dot3/GL_EXT_texture_env_dot3 but it supports the GL_NV_register_combiners extension instead. DirectX only j3d.d3ddevice "emulation" or "hardware" or "tnlhardware" or "reference" 1.2 Forces the software to use a particular mode for the underlying graphics accelaration. The reference option is only available if you have the Direct3D SDK installed (very unlikely). j3d.d3ddriver idx 1.2 For cards like Voodoo that run fullscreen 3D only. idx is the order DirectX enumerates its driver using DirectDrawEnumerateEx(). This number starts at 1. This will force Java3D to use the driver specified by the user (may fail if the driver is not compatible with display). The driver number and details can be found by using the j3d.debug property. For a typical setup of a 3D only card attach to a graphics card in a single monitor system, use idx=2. This will automatically toggle to fullscreen hardware accelerated mode since if the 3D card support 3D only. j3d.debug true or false 1.1 Prints out startup and running information. Useful for finding out information about the underlying hardware setup. j3d.fullscreen PREFERRED or REQUIRED or UNNECESSARY 1.2 Option to force Java3D to run in fullscreen mode for video cards that will only use hardware accelaration when using fullscreen (non-windowed) mode, like the older Voodoo series. j3d.vertexbuffer true or false 1.2 false to turn off the use of vertex buffers (a D3D performance enhancing feature equivalent to OpenGL display lists). Some drivers have implementation problems so it might be worth turning this off if you get crashes. Utility Classes j3d.audiodevice A quote string containing a class name 1.3.2 SimpleUniverse utility classes. Takes the name of a concrete subclass of com.sun.j3d.audioengines.AudioEngine3DL2 that will be constructed by Viewer.createAudioDevice(). The default value is null, which means that audio is disabled by default for applications that call Viewer.createAudioDevice(). j3d.configURL Unknown 1.3.1 Found in the ConfiguredUniverse class. Functionality unknown currently. j3d.io.ImageCompression "None" or "GZIP" or "JPEG" 1.3.1 Found in the scenegraph I/O package. Functionality unknown currently. j3d.stereo PREFERRED or REQUIRED 1.1 Only used by SimpleUniverse. If you roll your own VirtualUniverse, this property is not used. Controls whether you want Java3D to definitely create stereo mode capable canvases or not sun.java2d.d3d true or false ?? Default true. Enable Direct3D in Java 2D (not Java 3D, actually). sun.java2d.ddoffscreen true or false ?? Default true. Enable DirectDraw and Direct3D by Java 2D for off screen images, such as the Swing back buffer (not Java 3D, actually). sun.java2d.noddraw true or false ?? Default false. Completely disable DirectDraw and Direct3D by Java 2D (not Java 3D, actually). This avoids any problems associated with use of these APIs and their respective drivers. # Interactive debugging using a shared Terminal session For users running Linux and MacOSX computers (or on Windows, Cygwin with an openssh server), one can use an SSH tunnel for a debugging session shared between a user and a developer. All that is needed is a shared account on a public SSH server. The user should execute this command: ssh -R 2222:127.0.0.1:22 -t$ACCOUNT@$SSHSERVER screen  Once connected, the command ssh -p 2222$LOCALACCOUNT@127.0.0.1


will open a connection back to the local machine.

The developer should then execute this command:

ssh -t $ACCOUNT@$SSHSERVER 'screen -x'


Since this provides a shared GNU screen session, both the user and the developer can execute commands and see the output. It is even quite common to use the terminal window as sort of a private chat room by typing out what you have to say, ending the line with a Ctrl +C (lest it get executed as a command).

After the debugging party is over, the user can log out securely by hitting Ctrl +D to log out from the local machine (since the user typed in their password in the GNU screen session herself, there is no way for the developer to log back in without the user’s explicit consent). Another Ctrl +D will terminate the GNU screen session, and yet another Ctrl +D will log out from the shared account on the SSH server.

Takeaways

• Profiling tools allow quick and precise identification of performance bottlenecks.

Goals

• Observe the effects of debugging in multithreaded environments

Developing codes may often involve using multiple threads instead of sequential execution of statements. New threads may be spawned anytime when required. Essentially, threads should be used to run independent processes but if threads work on the same process there is no guarantee of execution of statements in different threads in a particular order.

E9 exercise on multiple threads focuses on this issue and also highlight an additional property of breakpoint that can be helpful in debugging program (Stop Virtual Machine).

Even though no code changes, sometimes debugging affects code execution. The Exercise 9 creates two parallel processes which are interdependent on each other. Each process throws up an error if the other process introduces a delay more than one second. So pausing for more than one second ,while debugging the program throws up an exception. So the act of debugging introduces errors in this case.

Steps for the exercise

• Run E9 with no breakpoint. Note that it works fine
• Now set a breakpoint in the “getName” method of either the Even or Odd LonelyRunnable
• Set the breakpoints property to “Suspend thread”
• Debug E9. After hitting the breakpoint, wait for a few seconds. Eventually you should see that a different thread is paused and “Expecting Exception”.
• resume the threads and see the stack trace.
• Now modify the breakpoint and tell it to suspend VM instead of just the thread.
• Debug again. When you hit a breakpoint, make sure you resume the VM and not just the thread.

Takeaways

• The act of debugging can change the way our code executes. Find the right approach for the situation.
• In case of multithreaded programming, set breakpoints to stop the virtual machine at the breakpoint to freeze the state of all threads.

# What next?

At the start of this guide we mentioned that the goal of debugging isn’t explicitly to fix the problem. So a natural question that follows is - once we’ve successfully identified the cause of an issue (or at least narrowed things down), what are the next steps to take?

The answer depends on the debugging developer’s skill and responsibilities, and whether or not they have identified a fix (a fix is often easy to come up with once the problem is identified).

If a fix is identified:

• If you are a maintainer of the component you can make the fix directly, or use a topic branch if the fix needs discussion first.
• If you do not have commit rights to the repository, you can contribute the fix via pull request

If a fix is unclear:

Even if you can’t contribute a fix, if you went through the effort of debugging - at the very least you should identify the problem, steps you took to debug, and potential fix(es) via a bug report so that your effort is not lost.