FluoroSim Module

Table of Contents

Introduction

FluoroSim is a Microscope Simulator module that enables fluorescence microscope simulation. From the model objects in the scene, FluoroSim computes the image that would be expected in a fluorescence microscope. Different microscopes can by simulated by selecting different point-spread functions (PSFs) that characterize the optics of the microscopes.

Image Formation Model

A fluorescence microscope can be modeled as a linear system. As such, an image I is formed by the convolution of the true signal S from the fluorescing molecules in the specimens with the point-spread function H that characterizes the optics of the microscope. Mathematically, image formation is modeled by the equation:

I = S ⊗ H

where ⊗ denotes the convolution operator.

Fluorophore model control panel.
Fluorophore model control panel. In FluoroSim, fluorophores are represented by 3D Cartesian points. These points are generated by sampling the model object geometry's surface or the volume within the model object. Most model objects have one or more properties called fluorophore models that control aspects of the fluorophore sampling. There are two kinds of fluorophore models, surface models and volume models. In both kinds of models, the sampling density can be adjusted to match the density in a specimen. Furthermore, the color channel to which the convolved fluorophores are rendered can be selected, which is useful for generating simulated multi-channel fluorescence images.

Noise Model

Noise in fluorescence images comes from several sources in the microscope. Shot noise arises from fluctuations in the number of photons emitted from the fluorescence molecules (fluorophores) in the specimen and detected by the CCD elements in the camera. Background noise comes from stray photons in the system. Finally, read-out noise comes from the CCD electronics.

The various noise sources follow different probability distributions. Taken together, their effects can be approximated by a Gaussian distribution. The fluorescence microscope simulator models noise according to a Gaussian distribution, adding the noise to the result of the convolution of the fluorophores with the PSF. The final image formation model, then, is:

I = S ⊗ H + N

where N is the modeled noise.

Focus Controls

Focus controls
Focus controls. FluoroSim generates fluorescence images one focal plane at a time. The focal plane position along z can be set in the Focus section of the FluoroSim control panel by adjusting either the vertical slider bar, entering a number into the text box to the right of the slider bar, or by clicking on one of the arrows on the right side of the text box. The units for the focal plane position shown in the control panel are microns.

FluoroSim focus settings visualization
FluoroSim focus settings visualization Two checkboxes control how the focal plane is shown in the Model View Display Window, if it is displayed at all. The option Show Volume shows a bounding box around the simulation region whose vertical extent matches the vertical extent of the currently-selected PSF. This representation shows what parts of the model objects in the scene actually contribute to the image formation; parts outside this bounding box do not contribute to the image because they are too far from the focal plane.

Focus control settings
Focus control settings including minimum and maximum values of the z-location of the focal plane and the inter-plane spacing. The range of the focal positions and the inter-plane spacing can be set in a separate Focus Control Settings window that pops up after clicking the Settings button in the control panel (pictured above and to the right).

Point-Spread Function

Point-spread function controls
Point-spread function controls. The FluoroSim module comes with several pre-installed point-spread functions. These PSFs are listed in a pop-up menu in this part of the control panel. They are calculated PSFs from several microscope configurations. To see the microscope properties for the pre-installed PSFs, click on the Properties button.

The Exposure text field contains a scaling factor for the values in the PSF. Because the microscope simulator models a linear system, linear changes to this exposure setting will result in a linear scaling of the overall brightness of the simulated images.

The pre-installed PSFs are stored in a directory where the Microscope Simulator is installed. In default installations, this is located at C:\NSRG\bin\Microscope Simulator 1.3.0\psf. To make custom PSFs available in FluoroSim, you will need to place them in this directory. The PSF storage format is 16-bit unsigned shorts. An XML file describing the PSF must also be added to the directory; an XML file from one of the pre-installed PSFs can serve as a template for an XML file for a custom PSF.

Simulator Settings

Simulator settings
Simulator settings. FluoroSim has several controls for changing simulator settings.

Resolution
Sets the pixel resolution of generated fluorescence images. Generated images are square, so only a single text field is provided.
Method
Selects the specific convolution algorithm for generating images. We recommend using the default algorithm.
Add Gaussian Noise
This checkbox controls whether Gaussian noise is added to the simulator output. This option is available only on recent graphics cards.
Std. Dev.
Standard deviation of the Gaussian noise added to the simulation when the Add Gaussian Noise checkbox is checked. This option is available only on recent graphics cards.
Mean
Mean of the Gaussian noise added to the simulation when the Add Gaussian Noise checkbox is checked. This option is available only on recent graphics cards.
Show Fluorescence
Clicking this button makes the Fluorescence Image window open or brings it to the top if it is already open.

Processing Simulated Stacks

Processing simulated stacks
Processing options for simulated stacks. While FluoroSim interactively generates simulated fluorescence images, it can also generate z-stacks of fluorescence images. This section of the control panel provides controls for generating and processing simulated stacks.

The Export Stack button allows saving a stack as a series of files. The pop-up menu under Compare simulated stack to lists fluorescence stacks that have been imported into FluoroSim. Simulated stacks generated in FluoroSim can be compared visually to the stack selected in this menu by selecting the appropriate visualization mode icon (see the manual section on Visualization Icons). Clicking the button labeled Copy Stack Settings sets relevant simulation parameters to those from the stack selected in the pop-up menu above. In particular, it sets the focal plane settings to generate a simulated stack with the same number of slices, the same interslice spacing, the same pixel dimensions, and the same pixel size as the selected stack.

FluoroSim's comparison visualization
FluoroSim's comparison visualization mode. The Iso value parameter controls the comparison visualization showing differences between an imported and simulated stack. The comparison visualization consists of two isosurfaces, one from the experimental image stack and the other from the simulated image stack. Each isosurface is created with the same iso value. The resulting two surfaces are factored into two parts: 1) an interior portion consisting of a boundary around the intersection of the volumes defined by the surfaces, and 2) an exterior portion consisting of a boundary around the union of the volumes defined by the surfaces. The interior surface is displayed as solid while the exterior surface is displayed as wireframe. Each portion is split into red and blue portions. The red portion corresponds to the isosurface from the simulated image while the blue portion corresponds to the isosurface from the real image.

Model Optimization

Model optimization controls
Model optimization controls. Model optimization is a new feature in the FluoroSim module. This feature is intended to find the parameters of model objects that best explain an image from an experiment. The optimization routines search for parameter settings that minimize the difference between simulated and experimental images. Currently, two optimization routines are provided. One routine performs an exhaustive search through the parameter space at discrete points in grid in the n-dimensional parameter space. The other routine is an implementation of the Nelder-Mead downhill simplex derivative-free optimization routine.

A pop-up menu lists the available fluorescence optimizers. By clicking the Optimizer Settings button, various settings for the selected optimizer can be adjusted.

Image Comparison Metric

Each optimizer seeks to minimize a function that measures the mismatch of the simulated and experimental images. The match between simulated and imported fluorescence stack is calculated by the selected Comparison Metric. Three comparison metrics are currently implemented:

Exhaustive Optimization

Exhaustive fluorescence optimizer settings
Exhaustive fluorescence microscope optimizer settings. The exhaustive fluorescence optimizer searches for optimal model object parameters by computing an image comparison metric at points in a regular grid in the n-dimensional parameter space and finding the set of parameters that minimize the comparison metric. The grid is defined at regular intervals in each parameter. This method is very inefficient and does not produce accurate results, but it can be used to find a good starting point close to a global minimum for a more efficient optimization algorithm. The figure to the right shows a control panel for setting the exhaustive optimization routine options. In the upper left part of this window is the Model Object List. All model objects in the simulation appear in this list. When a particular model object is selected, the parameters of the model object that can be optimized appear in the Object Optimization Parameters table at the bottom half of the window. The columns in this table are as follows:

Property
The name of the property that can be optimized.
Use
Checkbox indicating whether the parameter should be included in optimization.
Min Value
Minimum value the parameter can take.
Max Value
Maximum value the parameter can take.
Intervals
The number of intervals between the minimum and maximum parameter values that should be considered by the exhaustive optimizer. The number of values of this parameter that the optimizer will evaluate is one plus the number of intervals.

Optimizing the properties Surface Fluors and Volume Fluors adjusts the density of the fluorophore models.

Controls in the upper right of the window change how the optimizer operates. The Number of Trials indicates how many times a simulated stack is generated for a particular set of model object parameters. The parameters that produce the best match from these trials are saved along with the quality of the match. How many parameter sets are stored is controlled by the value in the text field labeled Keep Best.

Nelder-Mead Optimization

Nelder-Mead fluorescence optimizer settings
Nelder-Mead fluorescence microscope optimizer settings. The Nelder-Mead optimizer uses a so-called direct search minimization method that uses only comparison metric values (no derivatives). A set of (n+1) points in the parameter space are kept and sorted according to the value of the image comparison metric. In each iteration of the algorithm, a better candidate set of parameters is sought by moving away from the point with the worst image comparison metric value.

The settings for the optimizer are simpler than the settings for the exhaustive optimizer:

Property
The name of the property that can be optimized.
Use
Checkbox indicating whether the parameter should be included in optimization.
Starting Value
Starting value that the parameter should take.

Controls in the upper right of the panel can be used to set parameters for the Nelder-Mead optimizer. The Report Best option indicates how many of the best parameter sets and their corresponding image comparison metric values should be saved to a file. Various coefficients control how the Nelder-Mead optimization routine functions. The default settings are recommended in Nelder and Mead's original paper (Nelder and Mead 1965), and seem to work well in practice. The Reflection coefficient should always be greater than 1, the Expansion coefficient should always be greater than the reflection coefficient, and both the Contraction and Shrinkage coefficients must be between 0 and 1.

Exploring the Image Comparison Metric

The Explore Obj. Func. button runs an exploration of the image comparison metric in the same way that the exhaustive optimizer operates. It uses the settings from the exhaustive optimizer to define the n-dimensional grid searched in the exploration. Each parameter set from points in the grid and corresponding image comparison metric is saved to a MATLAB file that can be loaded to analyze the shape of the comparison metric. This feature is useful when trying to understand why a particular optimization may fail.

Running the Optimizer

After setting the optimization parameters, clicking on the Optimize button will start the search for optimal model parameters. When optimization is complete, the model objects will be set to the parameters that produce the best match between simulated and imported fluorescence stack. In addition, a file save dialog box will appear, requesting the location and name of the comma-delimited file that will contain the results of the optimization.

References

J.A. Nelder and R. Mead (1965), A simplex method for function minimization, The Computer Journal, Vol. 7, pp. 308-313.

Publications Resulting from FluoroSim

Cory W. Quammen, Alvin C. Richardson, Julian Haase, Benjamin D. Harrison, Russell M. Taylor II, and Kerry S. Bloom. FluoroSim: A Visual Problem-Solving Environment for Fluorescence Microscopy. To appear in Proceedings of the Eurographics Workshop in Visual Computing for Biomedicine, Delft, the Netherlands, Oct. 6–7, 2008.