The Overlap pipeline in MOPEX (on the command line, the script is overlap.pl) performs background matching between overlapping frames. An additive correction is calculated for each image in the input stack in order to bring them to a common background level. This is often the first step in processing Spitzer imaging data.
4.1 Overview
To load an Overlap pipeline, either start from a MOPEX template namelist (File > New Overlap Pipeline), load an existing Overlap namelist file (File > Read Name List) or load an empty flow and insert an Overlap pipeline from there (File > New Empty MOPEX Pipeline, then go to Insert Overlap at the top of the flow window and choose Empty Pipeline). MOPEX will load an Overlap flow to which you can add and subtract modules, and modify the input parameters. For information on running Overlap on the command line, see Chapter 9.
Overlap begins with a text list of input images (for Spitzer data, these are usually the Level 1 BCDs). This is loaded in Image Stack File, in the Initial Setup module. Uncertainty and mask files are loaded in Optional Input & Mask Files. The final output is a table of overlap correction factorsand a stack of offset-corrected images. If desired, a QuickLook mosaic can be created, incorporating the corrections.
Optionally, the user can run an additional preprocessing step in order to detect and mask all bright objects in the input images that might adversely affect the process of background matching.
If the user has no uncertainty files, Overlap can run a module called S/N Estimator, which estimates the uncertainties from the gain and read noise. The user should carefully read §4.3.3 to do this correctly. For Spitzer data, the pipeline uncertainties are generally preferred.
Overlap finds the additive offsets for each frame so that the pixel differences in the overlap regions (after interpolation to a common grid) are minimized. The metric that is minimized is the sum of the uncertainty-weighted squared differences between overlapping pixels in all pairs of interpolated input images, however, this does not determine the overall level that the frames are corrected to.
In the GUI, MOPEX determines the overall level after finding the offsets, by forcing the sum of the offsets of all the images to add up to 0 (see §8.5 for a full description of the algorithm). The offsets are analyzed before applying the above condition. The outliers are found using the input parameters in the Overlap Correction module: BOTTOM_THRESHOLD and TOP_THRESHOLD in units of sigma, and MIN_IMG_NUM, the minimum number of images needed to detect outliers. This method should give good results for large numbers of frames without strong systematic deviations.
There is also a second method available on the command line, triggered by setting the parameter WEIGHT in the Overlap Correction module. In this method (nominally WEIGHT = 1), MOPEX minimizes the squared offsets as part of the original minimization. The user can experiment by sliding WEIGHT up or down, giving more or less weight to the squared offsets (WEIGHT = 0 corresponds to the default method employed by the GUI). In principle, this method should allow correction for offsets even for a small number of frames with a varying background, but the range of effectiveness has not been thoroughly tested.
The methods are described in §8.5, with more information in the online document http://irsa.ipac.caltech.edu/data/SPITZER/docs/files/spitzer/background_matching.pdf, and the first is given in more detail in Makovoz et al. 2005, PASP 117, 274-280.
Overlap has a fault in that it finds additive corrections to all frames in one swoop. For a large number of frames, this leads to a big matrix inversion, requiring a big chunk of memory. If the memory limit is reached, MOPEX fails, usually with either a MALLOC error or no error at all. If you believe you are encountering this problem (usually seen with datasets of a few thousand input frames) then you can try using the Contributed Software IDL routine called afrl_bcd_overlap (http://irsa.ipac.caltech.edu/data/SPITZER/docs/dataanalysistools/tools/contributed/irac/afrlbcdoverlap/)
With version 10.5.0 there is a new option which inserts a sparse matrix solver to increase the limit on the number of files. In the GUI this is in the Overlap Correction Settings. In command-line namelists it is set in the Overlap Correction block: USE_SPARSE_FOR_MANY_FILES = 1,. You should be able to do at least 10,000 images in most cases.
Other common problems encountered with Overlap are:
Inclusion of wildly discrepant frames, e.g. MIPS-70 or MIPS-160 stim flashes, or IRAC frames affected by “droop” from very bright sources, will give the algorithm trouble, and may cause Overlap to fail with the error message "UNSUCCESSFUL_SOLUTION". Make sure you have removed all of the stim flash frames, and checked for any other large background-level discrepancies before running MOPEX.
Systematic offsets with heavy overlap, e.g. at the start of MIPS-24 scan legs, will not be corrected.
Background matching does not perform well in cases where data are divided evenly into two background levels. An example of this would be AORs taken in different epochs such that the zodiacal background has changed. In such cases, background can be brought into closer agreement by subtracting the estimated zodiacal background (check the Apply Zodiacal Subtraction box in the Overlap Settings module). The zodiacal background estimate is taken from the ZODI_EST keyword in the IRAC and MIPS FITS headers.
4.2 Overlap Processing Stages
The main processing stages of the Overlap pipeline are as follows:
Bright Object Masking (optional)
A preprocessing step can be run to mask bright objects that can bias the background levels of the images in which they are present. Two modules are run: MedFilter and Detect. The former produces background-subtracted images, which are then input into Detect. The latter detects bright objects in the background-subtracted images and creates a set of mask images, one per input image, in which pixels corresponding to the detected objects (or radhits) are set to positive values. The background-subtracted images created by MedFilter are only used for bright object detection, not for the actual overlap correction.
Interpolation
Before background matching, the input images are interpolated onto the output grid defined by the Fiducial Image Frame (FIF) table. If no table exists, it needs to be created by the Fiducial Image Frame module. If bright object masking is selected, masked pixels are not used in the interpolation.
Background Matching
Once the images have been interpolated to a common grid, the metric to be minimized is the combined, uncertainty weighted difference between the overlapping parts of each pair of input images. It is minimized simultaneously with respect to the constant offsets in question. More information on the algorithm used can be found in §8.5: Background Matching Algorithm.
Quicklook Mosaic
It is possible to coadd the interpolated, corrected frames into a QuickLook mosaic. This is a quick-and-dirty way to examine the results of overlap correction, and will most likely produce an ugly-looking mosaic with a lot of masked areas, where Bright Object Masking has been applied. Run the Mosaic pipeline to create a science-quality mosaic.
