MOPEX was developed at the Spitzer Science Center to process Spitzer Level 1 Basic Calibrated Data (BCD) from the point at which the users download the files from the archive through to final point source extraction. While it is primarily designed to take Spitzer data as input, it is a general-purpose tool that can be applied to any images in standard FITS format (see §3.1 for MOPEX input image requirements).
2.1 How MOPEX Works
The MOPEX software package is a set of Perl wrapper scripts that call C++ and C modules. For the GUI, the Perl scripts were translated into Java. All of the Perl scripts can be found in the MOPEX installation directory, under /<mopex_dir>/bin/.
These scripts are incorporated into the GUI as pipeline flows, and each pipeline calls a series of individual modules in sequence. Depending on the options you wish to use when processing your data, you will need to include different combinations of modules in a pipeline. For example, if you want to use one of the outlier rejection algorithms to remove the cosmic rays from your final mosaic, you need to set up the Mosaic flow to include the outlier rejection modules. Setups that you choose for a pipeline can be saved as a parameter file called a “namelist”. This namelist can be read back into the GUI at a later date. Default namelists are also provided.
2.2 How to run MOPEX in the GUI
When you first start the MOPEX GUI, you are presented with a blank screen. To begin using MOPEX, you must initiate a pipeline from the File Menu. You can choose to initiate a pipeline in three different ways - you can read in a previously-saved parameter file (called a “namelist”; File > Read Namelist), you can start from a template of default modules and parameters (e.g. File > New Mosaic Pipeline), or you can choose to begin with a blank pipeline and add modules one at a time by hand (File >New Empty Mopex Pipeline). First time users are advised to begin with the template pipelines. Each module has a help page that explains what it does, what the inputs mean, and what outputs can be expected. The help page is accessed by the "?" button, or by reading the module descriptions contained in this manual.
Usually you will start by loading a template namelist as your starting point. We will assume for the moment that we have IRAC channel 1 data that we wish to background-match and mosaic together. Background matching is performed by the Overlap pipeline, and mosaicking is carried out by Mosaic. One key feature of the GUI is that it can understand that we want to use the output of the first as input to the second.
Start by loading a template Overlap namelist by going to File > New Overlap Pipeline and selecting “Overlap, IRAC ch1” from the drop-down menu. Click OK, and the template Overlap pipeline is loaded. The current pipeline is shown in the main window, with clickable buttons to add input and output files, and input fields to change the settings for each included module. Off to the side is a separate window, listing all of the available modules for the Overlap pipeline. The ones that have already been included are greyed-out, while the ones that are currently excluded are in black. Modules can be added to the flow by clicking on them in this secondary window, and removed from the flow by clicking on the cross next to them in the main pipeline window. For assistance with any module, you can click on the question mark to pull up the integrated help system.
Once you have added the input files (see Chapter 3 for input file types and formats) and set up the Overlap pipeline, you need to add the Mosaic pipeline onto the end. To do this, click on the “Insert Mosaic…” button near the top of the pipeline window. Do not go through the File menu - this will start a whole new Mosaic flow instead of appending it to the bottom of the Overlap pipeline. By inserting the Mosaic flow, MOPEX will use the output of the Overlap pipeline (i.e. the background-corrected images) as the input into the Mosaic pipeline.
After setting up the Mosaic pipeline, you are ready to run MOPEX. There are three ways to do this:
Module-by-module: click on the icon with the black arrow and the blue rectangles on the left side of each module. This runs the reduction to the end of that module and then pauses.
By Pipeline: click on the icon with the black arrow and the blue rectangles near the top of each pipeline. This will run through the associated pipeline (e.g. Overlap) and pause at the bottom.
Run the whole flow: Click on the green “play” arrow at the top left-hand corner of the MOPEX window. This will run through all of the pipelines to the end.
Once you have started the flow running, you can pause at the end of the current module by pressing the black “pause” button in the top left-hand corner of the MOPEX window, or you can stop the reduction immediately by pressing the red “stop” button next to it. If you want to rewind back up to a previous module, click the blue curved arrow icon next to the module you wish to repeat. This will rewind the flow to that point so that you can, e.g., change parameters in that one module and re-run from that point onwards.
For instructions on how to run MOPEX on the command line, see Chapter 9.
***NEW*** Note that with version 18.5.0 one can use multiprocessing to speed up MOPEX tasks. The tasks Overlap, Mosaic, Apex (Multi) and Apex User List (Multi) now allow multiprocessing of some steps. In the GUI, this is set in Initial Setup, Multi-Processing Mode. The options are "on", "off", and "manual". The default is "on" -- this grabs 3/4 of the available processors. Setting "manual" lets the user set the number of processors used. You should see speed-ups of at least 2x.
If using command-line, add these lines at the top of your namelist:
do_multiprocess = on | off | manual default = off
ncpu_multiprocess = 1 default = 1
If "manual" is set, set ncpu_multiprocess to the number of processors to use.
2.2.1 Encountering Error Messages in MOPEX
If MOPEX encounters any problems during the reduction, it will stop and open an error message window. If the explanation in the error message in unclear, you can check the log file found in ~/.spot/ for further information. If you are still stuck and have checked the manual, the MOPEX Bug List and the FAQ, please contact the Spitzer Helpdesk for assistance.
2.3 Basic Processing Steps in MOPEX
Here we outline the steps in a typical MOPEX task: combining a stack of FITS images into a mosaic and doing point-source photometry, then looking at the residuals. Spitzer Data Analysis Cookbook: Recipe 7 shows how to link all these pipelines together.
Data Input
MOPEX reads in the data frames, the associated uncertainty frames, and the associated bad pixel masks from user-created lists in ASCII format.
Fiducial Image Frame (FIF) Generation
A unified grid coordinate system is generated, which defines a common grid onto which the input images will be projected. The spatial boundaries of the FIF include all of the input images.
Background Matching
The Overlap pipeline performs background matching between overlapping input frames (see Chapter 4). An additive correction is calculated for each image in the input stack in order to bring them to a common background level (not a zero-background level - see Median Filtering below), so avoiding "patchy" mosaics. This process does not affect the photometry of the detected sources.
Image Interpolation
The image interpolation module projects the input images onto the FIF and interpolates the input pixel values to the output array. Users can define the pixel size of the output array. This step corrects for the optical distortion in the input images.
Outlier Detection and RMask Generation
A primary feature of MOPEX is outlier rejection (see §8.2). Four methods are available to use both spatial and temporal information to identify and mask outliers (e.g. cosmic rays). The combined outlier information can be stored in a single outlier mask -- the RMask -- which is then used by MOPEX to mask outliers when creating the mosaic.
Mosaic Creation
The interpolated images are averaged to produce a co-added mosaic image (see Chapter 5). There are several weighting schemes available.
Median Filtering
Mosaic images can be median-filtered to produce a background-subtracted image.
Point Source Extraction
The APEX pipeline performs multi-frame (APEX Multiframe) and single-frame (APEX Single Frame) point source extraction (see Chapter 6). These scripts include non-linear matched filtering for point-source detection, image segmentation for separating blends, point-source fitting for position and flux estimation, and aperture photometry. Point sources are fit with a Point Response Function (PRF; see §8.7).
Point Source Subtraction
The APEX_QA pipeline performs multi-frame and single-frame point source subtraction (it can also insert sources on to a frame).
A reduction such as this will result in the following output files: a single mosaicked image of all the input images; a coverage mosaic showing how many frames were combined to produce each pixel in the output mosaic; an uncertainty mosaic showing the uncertainties at each point in the mosaic, and a table of extracted flux densities for every point source, and a mosaic of the residuals after point-source subtraction. These are just the most commonly-used output products - you have the option to request the generation of others.
2.3.1 Additional Functionality
While the reduction process described above includes the most commonly used processing stages, there are many more functions available in MOPEX to carry out a more specialized reduction. See both §2.4 and Appendix A for more details.
2.3.2 Expected Input
MOPEX only requires a list of FITS images for basic mosaicking, but the additional files listed below will generally prove helpful:
A set of FITS data images;
Associated uncertainty images;
Associated status masks, flagging bad pixels in each individual data frame (in Spitzer data, these are called DCE Status Masks, where DCE = Data Collection Event);
Relevant masks of permanent detector artifacts (in Spitzer data, these are called PMasks).
For Spitzer data, all of the masks and calibration files can be downloaded from the Spitzer archive with your data. Permanently-damaged-pixel masks (PMasks) and PRF files are also stored in the mopex/cal/ directory that comes with your MOPEX installation. Template namelists are accessible through the File menu in the GUI and from the subdirectory mopex/cdf/. See Chapter 3 for more information about the format of MOPEX input files.
MOPEX is set up for input data in units of surface brightness. It can recognize two units in the FITS header keyword BUNIT: MJy/sr or microJy/arcsec^2 (BUNIT is a text string and there are alternate ways of writing these units, but MOPEX will recognize several variants). If it recognizes the units, output mosaic will be in those units, and extracted source flux densities will be in microJy. If it doesn’t recognize the units, all data will be treated as dimensionless “counts”.
2.4 Introducing the Major Pipelines in MOPEX
The major pipelines available in MOPEX are as follows:
Overlap: Performs additive background matching of the input BCD images to bring them to a common background level and thereby avoid “patchy” mosaics.
Mosaic: Interpolates the input images onto a common grid, carries out outlier detection and masking, co-adds the images together and stitches them all into a single mosaic.
APEX Multiframe: Carries out point-source extraction on a stack of images. Optionally builds a mosaic along the way, and in any case, does detections on the mosaic, extractions on the stack. A PRF function (or an array-position dependent map) for the image stack is typically provided, and is required for point-source fitting photometry.
APEX Single Frame: Carries out point source extraction on a single image, e.g. a mosaic image. A PRF function (or an array-position dependent map) for the single image typically provided, and is required for point-source fitting photometry.
APEX User List Allows users to give MOPEX a list of sources to extract, rather than allowing MOPEX to do the extraction on all detected sources in the field. This is available in two flavors - Multiframe and Single Frame.
APEX QA: Creates residual images for both APEX Multiframe and APEX Single Frame by subtracting the detected point sources from the input images. This pipeline is invaluable for checking the results of PRF-fitting.
PRF Estimate: Allows user to generate their own PRF from their dataset. Warning: Only for use with well-sampled data that does not display any intra-pixel variability. For this reason, this script should not be used with IRAC channel 1 and 2 data.
2.4.1 Ancillary Scripts
While the seven major scripts above are the most commonly used, there are a few more that are available on the command-line. A full list of all of the MOPEX scripts can be found in Appendix A, but the most commonly used ones are:
flatfield.pl: Creates a “flat-field” image by taking the median of the image stack. Optionally divides the stack by the flat-field. Allows bright objects to be detected and masked beforehand. See the file <mopex_dir>/readme/README_flatfield for documentation.
