Part 2: Review and Expand on Basic Stack Techniques3

ImageJ Memory Management

Stacks are image windows that contain multiple images, or slices. You can picture the images in your computer's memory as strung one after the other in one long stream of data. Stacks containing large images, large numbers of slices, or both can occupy a lot of your computer's memory.

Not only does ImageJ have to keep track of the image data, but it maintains an invisible chunk of memory the same size as each open image called the Undo Buffer. This memory holds the image data from just before the current process so you can choose Edit > Undo and get back to where you were in the previous step. (You can only go back one step with Undo. If you need to go back any farther, the only option is to close the image without saving and re-open it, or choose File > Revert.) These are some of the reasons why it's good to have lots of memory (RAM - chips that is, not drives) in your computer.

Remember that your computer needs memory for lots of other things (like the operating system), so be realistic about your expectations. Unlike most applications you use, ImageJ doesn't just take whatever memory it needs to work—it sets aside memory space when ImageJ launches. You can change this amount, up to a maximum of 1700MB (1.7GB), if available. (If you need more, install and use the 64-bit version of ImageJ.) The default memory allocated for ImageJ is usually around 400–640MB, which should be plenty for most uses. Just remember that you have to TELL ImageJ to use more memory—adding more RAM alone won't make any difference!

Here are some practices that can help you make the most of ImageJ within the memory limits of your computer:

  • Choose Edit > Options > Memory and Threads to open the Memory dialog box. ImageJ defaults to something like 400MB of memory. Remember that this is memory, not hard drive capacity. Most computers today ship with at least 1GB of RAM installed, but you may have an older computer. Click the appropriate option below to find out how much memory is installed in your computer: This gives you an idea what you have to work with.
  • To adjust ImageJ's memory allocation, choose Edit > Options > Memory and Threads. If necessary, increase the Maximum memory setting. The Keep multiple undo buffers option allows you to "Redo" (that is, to Undo the previous Undo command). The Run garbage collector on status bar click option helps recover memory after closing images. It's okay to turn on both of these options.
    ImageJ Memory dialog box

  • How do you know how much memory is available for images when you're using ImageJ? When you click the status bar in ImageJ, it displays the ImageJ and Java version numbers, and the amount and percentage of ImageJ's memory allocation that is in use.
    Status Bar memory check

  • As ImageJ closes images, the memory they occupied isn't always released back to the application. You can force a process called "Garbage Collection" that makes this unused memory available again by clicking again on the ImageJ status bar.
    Here's an example:

    Status bar after closing images but before garbage collection (1st click)

    Garbage Collection before

    Status bar after garbage collection (2nd click)

    Garbage Collection after

  • There is even a useful little memory window that you can use to keep track of images and memory. To open this window, choose Plugins > Utilities > Monitor Memory. The window shows the amount and percentage of memory in use by ImageJ on the left, the number of open image windows on the right, and a live, rolling graph of memory in use across the bottom.
    Memory Monitor

You don't need to worry about memory issues most of the time—ImageJ usually "just works". However, if you run into memory issues while working with stacks or big images, now you have a clue what you can do about it, or at least you'll better understand the nature of the problem. It may even save you some money by keeping you from buying more memory for your computer, as you better understand how to make ImageJ use what you do have!

Stack Tools

ImageJ was designed as a general-purpose tool, but customizable to meet the user's needs. To meet these goals, the developers built in an architecture of macros and plugins (think of these as modular mini-programs) to automate tasks and to add new features to the basic tool. That's why ImageJ has found its way into such diverse fields as astronomy and molecular biology.

You'll notice that the ImageJ toolbar contains 8 buttons to the right of the color picker tool. Some of them may even be blank. These buttons are available to the user. When ImageJ launches, it automatically loads a standard set of macro tools onto the ImageJ toolbar. There are additional macro toolsets available, and one was designed specifically for working with stacks.

  • In ImageJ, click the Switch to alternate macro tool sets button - the red >> button at the right end of the ImageJ tool bar, and choose Stack Tools from the pop-up menu.
    Stack Tools buttons

    The 8 tool buttons on the right end of the tool bar are replaced with stack-related tools. These buttons provide basic controls for working with stacks.
  • Roll over the tools to display their functions in the status bar. To save time, the first button, labeled Stk, provides a menu of stack functions that duplicates most of what you'll find under the Image > Stacks submenu.

    Stack Tools menu

Stack Basics

In Part 1, you dug deeper into digital images—what they are, how they are stored, and different ways to open them in ImageJ. You also examined the characteristics such as memory and storage requirements of different image types and file formats. However, your exploration was limited to windows containing single images. Part 2 will extend these concepts to windows that contain multiple images, or stacks.

At the bottom of the Stk menu button on the toolbar is a sample stack to open. (It's also listed under the sample images menu you explored in Part 1.)

  • Click the Stk button on the ImageJ toolbar and choose MRI Stack (528K).
    • Remember that a stack is a collection of multiple images in a single window. This stack shows a series of MRI (Magnetic Resonance Imaging) "slices" through a person's head.

  • Use the new stack buttons on the toolbar to step forward (> Next Slice), backward (> Last Slice), return to the beginning (
  • Mouse over the Animate button and find out how to change the animation speed without having to go through menus.
  • Play the stack back at different speeds, including the highest and lowest speed indicated in the dialog box. As the animation is running, ImageJ will report the speed in frames per second on the status bar. Note: The speed indicator will not read accurately for frame rates lower than 1 fps. The maximum animation speed depends on the speed of your computer, so your maximum fps may vary, up to 1000 fps.
  • When you're finished experimenting with speed, return the speed setting to a normal value between 10 and 30 frames per second. (FYI - Theatrical movies play at 24 fps and television and web videos play back at around 30 fps.)
  • Try out the Loop Back and Forth option in the Animation Options dialog box. What does it do? When might you want to use this setting?
  • How many slices are in this stack? How big, memory-wise, is each slice? Use this information to calculate the memory required for the entire stack.
  • How does your calculated size compare to the actual size of the stack? (Note: the stack window is too narrow to show the memory size. Either magnify the window or drag to make the window wider until you can read the size, or you can read the size of the MRI Stack at the bottom of the Window menu.) If you did not calculate the correct size, check your math.
  • What would happen to the memory requirement if you added four more slices to the stack?
  • Check your result by adding four blank slices to the stack using the + button on the toolbar, then check the size of the stack and compare it to your calculation.
  • Delete the four slices you added by clicking the Delete Slice – button on the toolbar. Note that the memory size has returned to its original value.
  • Click the First Slice button to go to the beginning of the stack. Now add one slice. Where is the slice added—after the first slice or before it?
  • Delete the blank slice you added.
  • Still on the first slice of the stack, Option-click (Mac) or Alt-click (PC) the Add Slice + button. Where is the slice added now?
  • Close the MRI Stack.

Types of Stack Data – Spatial

Sometimes the slices of a stack are literally slices through a volume of space. An example of this is MRI (magnetic resonance imaging), which uses strong magnetic fields and radio waves to produce images that appear as slices through the body. Different types of tissue have different brightnesses in the image. Using ImageJ, we can stack and animate these slices to visualize tissues and organs in great detail.

  • Choose File > Open Samples > T1 Head (2.4M, 16-bits).
    • MRI Stack with stack counter
  • Step through the stack using the Next Slice and Previous Slice buttons. This stack is a more detailed series of MRI slices through a man's head, beginning from his left shoulder and ending at his right shoulder. The slices are spaced about 5 millimeters apart.
  • Animate the stack at a speed of about 30 fps. (If the animation does not play all 129 slices, check the settings in the Animation Options dialog box.)
  • Choose Image > Lookup Tables and select one of ImageJ's built-in color tables. Using color lookup tables sometimes makes it easier to see details in the images.

Reslicing

ImageJ can use the information in these images to create entirely new views of the data.

  • Click the Stk button and choose Reslice. In the Reslice dialog box, use the default settings and click the OK button.
  • What are the four dimensions of these stacks?
  • Animate the resulting stack. The Reslice function generates new views of the object by re-arranging the pixel values and interpolating between them.
  • Close the Reslice stack.
  • Experiment with the different reslicing options to create new views of the head. Varying just the Start at, Flip, and Rotate options, there are 16 different orientations, but you don't have to try them all.

Reslicing isn't limited to orthogonal (at 90 degrees to each other) planes. You can use the line selection tool to define the plane you want to reslice at, as well as the number of slices parallel to that plane. Imagine using this technique to study the shape and extent of a tumor.

  • Using the straight line selection tool Straight Line selection tool , make a diagonal selection across the image window.
  • Choose the Reslice command, leave the Slice count at 1, and click OK.
  • Activate the original stack and Reslice along this plane again, this time using 10 for the slice count. This will produce a 10 slice stack of images 1.5mm apart centered on the plane defined by your line selection.
  • Close the Reslice windows but leave the original stack window open.

    • Z Projection

    One problem with stacks is that it is sometimes difficult to visualize all of the data in the different slices at one time. A technique called Z projection can create a single image by statistically combining data for each pixel along the Z-axis (slices) of the image. For example, the Average Intensity method calculates the average of the pixel values at 0,0 in each slice and uses the result as the value of pixel 0,0 in the new image. Max Intensity uses the highest pixel value at 0,0 in the stack as the 0,0 value in the output image, and so on.

    • Choose Z Project and click OK. This produces an Average Intensity Z Projection image. Try out each of the different Z projection types.
      • Think about each output image and see if the result makes sense to you. Do you understand why the Min Intensity image is totally black?

    • Keep the T1-Head stack open.

    A specific use of Z projection is with a technique called confocal microscopy. Microscopes have very shallow depth of field—that is, they only focus at one distance at a time. Because of this, an object with depth can't all be in focus at one time. In confocal microscopy, the user takes a series of images where the microscope is focused at precise planes through the subject—a cell, for example—keeping the image precisely registered (aligned) in each image. The images are then loaded into IamgeJ as a stack and Z-projected using the Max Intensity projection. This works because features appear brightest when they are sharply focused because the light does not spread out.

    • Choose File > Open Samples > Confocal Series (2.2MB) to open the confocal stack. This is a special type of stack called a hyperstack. A hyperstack contains either 4 or 5 dimensions. The extra dimensions can be time (T) and channel (C). This hyperstack includes a red and a green channel in the channels dimension.
    • Use the window sliders to step through the c (channels) and z (slices) of the stack.
    • Choose Image > Color > Channels Tool to open the Channels tool panel. Use the check boxes in the Channels window to turn the red and green channels on and off. The red and green colors are from fluorescent dyes used to stain different structures in the sample.
    • Using the Z (slice) slider, you can get a sense of the overall sample, but it's difficult to visualize the whole thing. Enter Z axis projection and the Max Intensity projection.
    • Choose Z Project from the Stacks menu and set the projection type to Max Intensity. You can now see the entire sample in focus.

    Orthogonal Views

    Imagine taking these powerful processing techniques and raising them to the next level.

    • Activate the t1-head stack and choose Image > Stacks > Orthogonal Views. The three orthogonal views are all linked together. The label of each view shows the axes that define the view, and the yellow lines define the planes of the views.
    • To change the views, drag the yellow lines and the slice slider at the bottom of the main stack window.
      • These images have been density calibrated for length (distance). Imagine how this tool could assist doctors in providing precise locations for biopsies, radiation therapy, and surgery.

    • Close all of the open image windows.

    3-D Projection

    Another technique called 3D projection can do some impressive visualization from stacks.

  • Re-open the T-1 Head stack but do not add a color lookup table.
  • Choose 3D Project in the Stacks menu.
  • In the 3D Projection dialog box, use these exact settings (for now) and click OK.
    3D Projection dialog box

  • Animate the projected stack.
  • If you have time, explore the effect of changing settings in the 3D Projection dialog box. When you finish examining these stacks, close all of the stack windows.

    • Types of Stack Data – Temporal

    An obvious use for stacks in ImageJ is examining and animating a series of images that show change over time.

    Sea surface temperature stack

    1. Download the following SST time series stack and open it in ImageJ. SST_Stack (TIFF 4MB May8 10)
    2. This stack represents global sea surface temperatures from September through December of 2008.
    3. Use the right and left arrow keys to flip forward and backward through the stack.

    Chlorophyll stack

    1. Download the following Chlorophyll time series stack and open it in ImageJ. Chlorophyll_Stack (TIFF 4MB May8 10)
    2. This stack represents global chlorophyll concentrations from September through December of 2008.
    3. Use the right and left arrow keys to flip forward and backward through the stack.

    Concatenating stacks

    Concatenation is a big word for adding or stringing things together in a sequence.

    1. Open both the SST and Chlorophyll time series stacks.
    2. Activate each stack window and choose Image > Type > RGB Color to convert both stacks to RGB Color format. This is necessary because the stacks are 8-bit color and do not have the same color lookup tables.
    3. Choose Image > Stacks > Tools > Concatenate... to create a new stack that shows the SST and Chlorophyll images one after the other.
    4. Use the right and left arrow keys to flip forward and backward through the stack.
    5. Close the concatenated stack.

    What are some uses for this technique?

    Combining stacks

    Here's one last trick you can try...

    1. Open both the SST and Chlorophyll time series stacks.
    2. Activate each stack window and choose Image > Type > RGB Color to convert both stacks to RGB Color format.
    3. Choose Image > Stacks > Tools > Combine... to create a new stack that shows the SST and Chlorophyll images together in the same window. (Try this both with and without checking the Combine Vertically option.)
    4. Use the right and left arrow keys to flip forward and backward through the stack.

    What are some uses for this technique?

    Types of Stack Data – Spectral

    Creating Color

    All types of color imaging, from film, to print media, to modern digital cameras reproduce color by gathering brightness data through different colored filters. In most cases, the filters used are Red, Green, and Blue (RGB).

    • Color film has the filters built right into the film, with layers of light-sensitive emulsions recording the scene in Red, Green, and Blue light. The picture is then reproduced on photographic paper or transparency film containing layers of colored dyes. (The dyes used are Cyan, Magenta, and Yellow - the opposites of Red, Green, and Yellow. In combinations, they produce all the colors we see. For example, Cyan and Magenta combine to make Blue.
    • Modern inkjet printers produce colored pictures using microscopic dots of cyan, magenta, and yellow ink, with black ink added to produce deeper blacks and better contrast.
    • If you look at your computer monitor with a magnifying glass, you will see that each pixel on the screen is made up of a set of red, green, and blue bars or dots.
    • The sensors in digital cameras have the red, green, and blue filters coated right on the surface of each light detector cell in an alternating pattern called a Bayer pattern.
    • Scientific satellites often carry multispectral instruments that produce images in different wavelength bands. On LANDSAT satellites, the Multispectral Scanner (MSS) instrument records 4 bands and the Thematic Mapper (TM) records 7 bands, including visible and infrared wavelengths.
    • Newer instruments carried by satellites or aircraft can record images with hundreds of bands. This technology is called hyperspectral imaging. Examples are NASA/JPL's AVIRIS instrument (224 channels) and NASA/Goddard's EO-1/Hyperion instrument (220 channels).

    Working with multispectral data in ImageJ

    In this section, you will learn how satellites and aircraft use a single black and white camera with many colored filters to capture data that can be used to re-create both natural and false color images.

    • Download and unzip the LANDSAT Image Archive.zip file into the Week 3 directory or folder you created on your computer.
      • LANDSAT Image Archive.zip (Zip Archive 4MB Jan19 10)
    • Choose File > Import > Image Sequence... to import the seven LANDSAT images, representing the seven Thematic Mapper bands into ImageJ as a stack.
    • Move forward and backward through the stack. The label of each slice lists the TM band it represents. These images show the area around the community of Green River, south of Tucson, Arizona. Do you see signs of mining activity, agriculture, and recreation in the images?
    • After you have explored the images, close the stack.

    You are going to use these images to re-create a "true color" version of the scene by combining three bands that represent what is seen in red, green, and blue wavelengths.

    • Open, in order, the Band 3 (red), Band 2 (green), and Band 1 (blue) images and stack them. (Click the Stacks Menu button and choose Images to Stack.)
    • Choose Image > Color > Make Composite.
    • In the Make Composite dialog box, set the Display Mode to Composite and click OK. The resulting image - called a 321 composite, since it assigned bands 3, 2, and 1 to the red, green, and blue channels - is a true color image that approximates what the scene would look like to your "eyes in the sky".
    • Use this technique to produce a false color composite image, assigning different bands to the red, green, and blue color channels. (Hint: 432 images are often used to highlight vegetation in red - try it!)
    • Create your own false color image of this scene and save it to your Week 3 directory. Choose Image > Image Type > RGB Color and save the resulting image as a JPEG file. Use the channel assignments and your initials in your file name.

    Your Assignment: Create or Locate a Stack and Apply Appropriate Analysis Techniques

    1. Either open and explore one of the sample stacks or create one of your own. Apply an appropriate stack technique for the type of data you are exploring.
    2. When you are finished, create a montage and post it to the Part 2: Share and Discuss Page. Then describe as much as you can about what you analyzed.
    3. Engage in an online discussion, sharing your ideas about using stacked image data in your teaching.

    Source

    1Adapted from Earth Exploration Toolbook chapter instructions under Creative Commons license Attribution-NonCommercial-ShareAlike 1.0.
    2Adapted from Eyes in the Sky II online course materials, Copyright 2010, TERC. All rights reserved.
    3New material developed for Earth Analysis Techniques, Copyright 2011, TERC. All rights reserved.