MOPEX update 10/10/12 (v18.5.6)

1. Apex QA fixed to use (possibly) overlap-corrected files when inserted into a GUI flow.

2. Bug in fiducial_image_frame fixed for non-RHEL Linux 64bit.

3. Auto-updates will no longer overwrite "mopex-script-env.csh".

MOPEX update 06/08/12 (v18.5.4)

1. HighPrecision keyword added to Apex 1frame GUI in Aperture block. This is required for large mosaics when the aperture table is needed. All namelists and templates updated.

2. Bug in writing some tables under Linux corrected.

MOPEX release 03/15/12 (v18.5)

The following changes were made:

1. 64-bit Capability + Mac Lion - Previous binaries were 32-bit only for Linux and Macs, and the Mac versions were Universal binaries (PPC + Intel). Mac Lion no longer allows PPC binaries. Most users should be able to use the new 64-bit Linux and Mac versions; 32-bit versions are still available for now. See the Download Page.

2. Multiprocessing Speed-up - The Mopex scripts Overlap, Mosaic, Apex (Multi) and Apex User List (Multi) now allow multiprocessing of some steps. In the GUI, this is set in Initial Setup. The options are "on", "off", and "manual". The default is "on" -- this grabs 3/4 of the available processors. Setting "manual" lets the user set the number of processors used. You should see speed-ups of at least 2x.

If using command-line, add these lines at the top of your namelist:

do_multiprocess = on | off | manual default = off

ncpu_multiprocess = 1 default = 1

3. Overlap Improvements - Overlap can fail for large numbers of images. An option was added to use a sparse matrix solver to increase this limit. In the GUI, this is the Overlap Correction Settings. In command-line namelists, it is set in the Overlap Correction block:

&COMPUTEOVERLAPCORRIN
USE_SPARSE_FOR_MANY_FILES = 1,
&END

4. Simple Mosaic Photometry - Some users have expressed a desire to do "quick-and-dirty" photometry on a mosaic: Detect on the mosaic then do Aperture photometry on the fixed Detect positions. This is not the most accurate method as the Detect positions are just centroids, not best-fit centers. But it's easily done in MOPEX. A new sample namelist was added to the ./cdf directory to show this for a 0.6 arcsec pixel IRAC1 mosaic: apex_1frame_I1_mosaic_aponly_fixed.nl. The Aperture photometry for 2, 3, 5-pixel native BCD pixel radii (so 4, 6, 10 mosaic pixels) will be in *aperture.tbl. Note RA and Dec are the last columns.

5. Blend size limit - If a Detect blend has too many sources, Sourcestimate will fail. Default Apex and Apex_1frame namelists now run Select Detect on the raw Detect table using a limit of 600. Bigger blends are dropped -- none of the sources will appear in the Extract table. Users doing just Detection can turn off the Select Detect step.

6. Mosaic masks - Improvements were made in the script that allows making of a mosaic 'OR-mask', and optionally a mask cube. See http://irsa.ipac.caltech.edu/data/SPITZER/docs/files/spitzer/mosaic_mask.pdf. An example namelist is in "cdf/mosaic_mask_I1_example.nl" (command-line only).

MOPEX release 02/15/11 (v18.4.9)

The following changes were made to MOPEX for the latest release.

1. Numerous corrections to GUI flows including: allowing Apex QA (multi- and single-frame) flows to be inserted after APEX (multi- and single-frame) for point source subtraction.

2. GUI templates and new example namelists for GUI only (*_gui.nl) now automatically load files from the cal/ dir as needed. New default IRAC Pmasks are in place. These are the "ormasks" (see cal/pmasks.README_052810). Users may still wish to reset the IRAC Pmask for the date of the observation.

3. Binaries were not back-compatible with Mac Intel Tiger - fixed.

4. Some Perl scripts were stuck at older versions (a problem for command-line only) - fixed.

5. Bug fix in Source Estimate to allow valid sources on edges of tiles to get a non-negative SNR.

6. IRAC Array Position-Dependent Correction Mosaic can be made for IRAC Warm Mission data. See the Mosaic Settings window for the GUI implementation. The Default Array Correction files are cal/ch1_photcorr_rj_warm.fits and cal/ch2_photcorr_rj_warm.fits.

MOPEX release 11/29/10 (v18.4.0)

The following changes were made to MOPEX for the latest release. This version is updated from the February 2010 version (v18.3.3).

1. IRAC mosaics with 0.6" pixels are now the default. Default Mosaic, Overlap, and Apex (including Apex QA and User List) namelists have been changed to use 0.6" pixels for IRAC mosaics, like the PBCD products (was 1.22" in Mopex namelists). The new namelists have the same names, and will come up as the GUI defaults. The ones with the old pixel scale have "122" in the filename. Parameters have been updated for improved photometry.

One detail: standard Apex 1frame (including Apex QA 1frame) namelists remain at 1.22, i.e. they assume you are doing fitting on a single BCD frame, not a mosaic, for which we don't have PRFs. But the User List "aponly" namelists migrate to 0.6" pixels, since they assume you are going to do aperture photometry for your input sources on a mosaic.

A second detail: the most common form of point source detection involves running a point-source kernel (in the form of a PRF) over the mosaic. With 1.22" mosaics, the BCD PRF was used, as it had the right scale. New approximate 0.6" mosaic PRFs have been built for this purpose. They should not be used for PRF-fitting photometry on the mosaics. They have names like, e.g. cal/IRAC1_06_mosaic_4x_detect_kernel.fits. They are now in the
default namelists.

2. There is now a new MOPEX Detect Parameter - Detect_Min_Peak_Fraction. Around bright objects, MOPEX Detect will often pick out bumps in the PSF, as well as noise peaks, as detections. There is now an adjustable parameter, Detect_Min_Peak_Fraction, that can limit this: a candidate sub-peak must carry a minimum relative height above the current threshold. The height is given as a fraction of the "parent" peak height. It is roughly similar to the parameter DEBLEND_MINCONT in SExtractor (Bertin and Arnouts AASuppl 117, 393, 1996). Allowed values are between 0.0 - 1.0. If not set, it is 0.0.

For IRAC, using the point-source probablility image to detect (the default), a starting value might be around 1.0e-05. This has been set in the default namelists. Note that in using it, there is always going to be some trade-off between eliminating false detections and dropping potentially real objects near bright sources.

3. There is now a new GUI tool to create image lists automatically. In the GUI flows requiring image lists, there is a button at the top in Initial Setup called "Create Image Stack File". In that window, clicking on the question mark will bring up a help file that will explain how to make image lists, mask lists, etc.

4. The GUI procedure for using APEX on previously-made mosaics has been simplified. When running APEX (multi-frame), it is better to first run Mosaic separately. Although basic mosaic commands are available within APEX, it does not have the full outlier rejection capabilities available in Mosaic.

In command-line: Run mosaic.pl with your chosen outlier rejection(s). In the namelist file, set keep_coadded_tiles = 1. Run apex.pl with the same input lists, pixel sizes, etc. to the same output directory, with no mosaic commands. Example namelists are given in the cdf directory with "_have_mosaic" in them.

In the GUI: Run a Mosaic flow to a given output directory; the switch Keep Coadded Tiles (in Mosaic Coadd settings) should be on. From the top of the Mosaic flow, Insert an APEX flow from a file. Use a "have_mosaic" namelist.

5. Point source addition was added to GUI Apex_QA (was already in command-line). In the GUI flows for APEX QA Single Frame and APEX QA, note that there are now options in the Setting window for creating either Residual images (subtract point sources) or Point Source images (add point sources).

6. New IRAC PMasks are in the cal directory. See the file cal/pmasks.README_052810 and the IRAC webpage. The old ones can be found in irac_pmasks_20mar08.tar.

7. New IRS PeakUp imaging PRFs are in the cal directory. See the file cal/README_091010.

8. A new table of PRF normalization radius values and corresponding correction factors is now available for all instruments/modes. For more precise PSF-fitting photometry, correction factors need to be applied. See Appendix C in the MOPEX User's Guide.

9. Added additional Detect by-products in GUI (was already in command-line). Some additional products described in the MOPEX User's Guide are now also available by request in the GUI Detect module.

10. Clarified options for doing aperture photometry on fixed positions in User List mode (GUI).

11. Added independent pixel scale option to Overlap (GUI).

MOPEX release 02/03/10 (v18.3.3)

This version was released as a patch to correct a bug which prevented the creation of array location dependence mosaics.

MOPEX release 09/25/09 (v18.3.1)

The following changes were made to MOPEX for the latest release. This version is updated from the January 2009 version (v18.2.2).

1. New namelists
Previous namelist examples are now in ./cdf/cdf_pre_aug09. The namelist examples in ./cdf have been brought up to date. These should come up as the default templates in the GUI. Notable changes are improved namelists for IRAC mosaicing, and new namelists for aperture-only photometry at fixed positions. Each namelist has a marked section showing where to edit the main input files (for command-line).

Apex (multi-frame) example namelists for IRAC have been split into 2 types: those for which a mosaic and FIF are created, and those for which the mosaic and FIF have already been created by mosaic.pl using the same output directory. The latter are labeled, e.g. apex_I1_have_mosaic.nl. It's usually better to make the mosaic first.

The Apex_1frame MIPS24 example namelist (apex_1frame_M1.nl) should give reasonable results for mosaics in regions free of bright sources and complex backgrounds. Around bright sources, the current best option is the 2-step "ring-subtraction" method described in the Cookbook entry: "MIPS24 Source Extraction: General Steps". Complex backgrounds are still under study, but one approach is described in the MIPSGAL entry in the Cookbook.

In general, one does PRF-fitting for IRAC on the image stack (Apex multi-frame), but for MIPS and PUI on the mosaic (Apex 1frame). These are the cases we have good PRFs for. But namelists are often also there for the other case.

Default MIPS aperture sizes have been updated to match those on the Spitzer Wep page. Remember to multiply aperture fluxes by the appropriate aperture corrections.

NOTE: the Mopex example namelists assume IRAC mosaics will have 1.22" pixels. The PBCD mosaics that come with Spitzer data now have 0.6" pixels. For MIPS, the Mopex namelists assume the same pixel sizes as the PBCD mosaics, 2.45", 4.0", and 8.0" for 24, 70, and 160 microns respectively. The IRS Peak Up imaging ones are also the same: 1.8" pixels.

2. New MIPS Ge PRF Normalizations
The current PRFs for MIPS 70 and 160 mosaics require a correction factor to match the new aperture-corrected photometric calibration. Using the X-FLS data, the correction factor for Apex PRF-fitted fluxes was found to be 1.2 for both MIPS 70 and MIPS 160. Multiply PRF-fitted fluxes by these if using the current PRFs:

mips70_prf_mosaic_4.0_4x.fits
mips160_prf_mosaic_8.0_4x.fits

No Normalization_Radius is used in Apex's Sourcestimate block for these corrections.

3. Change to Apex User List, new Fixed Aperture only option
The Apex User List scripts no longer write columns into the input list file. Aperture photometry on fixed input positions was simplified with customized namelists, e.g. apex_user_list_1frame_M1_aponly_fixed.nl. See in the Mopex release directory readme/README_apex_user_list_082009.

4. IRAC Array Correction mosaics
There is a new option in Mopex's mosaic.pl to create an IRAC Array Correction mosaic along with the data mosaic. See in the current Mopex release directory readme/README_arraycorr_082009.

5. Zodiacal light subtraction in Overlap
There is a new option in Mopex's overlap.pl to subtract the SSC's estimate of the zodiacal background from the images first. See in the current Mopex release directory readme/README_zod_subtract_082009.

6. Fix to flatfield.pl when making flats for each Scan leg
A minor bug in Mopex's flatfield.pl when using scan_leg = 1 (use a different flat for each leg) was corrected. This was preventing command-line setting of the PMask and RMask lists, and the Output directory.

Also a new feature was added when scan_leg = 1: save complete lists of new files created, not just lists for each scan leg. These files are called:

(Output_dir)/(Medfilter_dir)/median_ImageList.txt
(Output_dir)/(Medfilter_dir)/medsub_ImageList.txt
(Output_dir)/(Detect_dir)/detect_ImageList.txt
(Output_dir)/(Correct_dir)/correct_ImageList.txt

MOPEX release 01/16/09 (v18.2.2)

The following changes were made to MOPEX for the latest release. This version is updated from the December 2008 version (v18.2.0).

1. Bug in Overlap input fixed.
There was a bug in the input to Overlap that was preventing users from using their own FIF.tbl file as input. This has now been fixed.

MOPEX release 12/05/08 (v18.2.0)

The following changes were made to MOPEX for the latest release. This version is updated from the August 2008 version 18.1.5.

1. New faster background estimation option in Medfilter based on SExtractor
This is described in a README file in the new Mopex distribution: ./readme/README_Sbkg_111008.

2. "Geometry"-selected images now used in subsequent modules
The Mosaic Geometry module (mosaic_geom) creates new image lists (plus uncertainties, masks) that overlap with a user-supplied (or edited) FIF table. Previously this was just a stand-alone process that one could call from the mosaic script or GUI. Now it has been integrated into the command-line mosaic.pl and the GUI. That is, after it has run, the remaining modules will use the "geometry"-selected lists. Fiducial_image_frame should not be run in this case. mosaic_geom has also been moved to run earlier in the flow.

3. Memory allocation problem for "median" and "absolute_minimum" mosaics during "mosaic_outlier".
This has been fixed

4. Error in new Apex switch "Use_Photerr_for_SNR" on MOPEX Bug List.
This has been fixed

5. Error in Apex switch "use_data_unc_for_fitted_SNR" on MOPEX Bug List.
This has been fixed

MOPEX release 08/21/08 (v18.1.5)

The following changes were made to MOPEX for the latest release. Updated from the Feb 08 version.

1. New IRAC PRFs
New IRAC PRFs are now available, and are a significant improvement over those previously released by the SSC. For true point sources, we are able to obtain agreement between fitted and aperture flux measurements at better than the 1% level. See IRAC PRF Fitting. Sample IRAC APEX namelists in the ./cdf directory contain an example of the new PRF file names. The PRFs are strictly valid only for IRAC BCDs.

2. APEX QA included in the GUI
The APEX QA pipeline has been included in the GUI. It is used for subtracting points sources from the input images to check the quality of the PRF fitting.

3. PRF Estimate included in the GUI
The PRF Estimate pipeline has been included in the GUI. It allows users to create PRFs from their own data. Not for use with IRAC data or any other undersampled data.

4. New SNR option
A new option for point-source fitted SNR is available ("Photerr"). This uses errors in a fixed box around each source. For most purposes, adopting the data uncertainties as the errors gives acceptable results. Set the following switch in the Apex namelist:

use_data_unc_for_fitted_SNR = 1

and the following parameter in the "sourcestimate" block:

Use_Photerr_for_SNR = 1,

See the file README_new_snr_option_081108.pdf for more information (this file can also be found in /readme).

5. New Apex User List option (still experimental)
There is a beta-testing version of modules designed to let the user input a list of source positions to be point-source fitted by APEX. See the file REAME_apex_user_list_081108.txt for more information (this file can also be found in /readme/).

6. Final mosaics use CD matrix
To keep up with the current FITS standard, the final mosaics produced by mosaic_combine have the "CD matrix" form of the WCS. However, some intermediate products will still have the old "CDELT" style, because that is what the modules are using internally.

7. Different paths now allowed for PRF files
Other directory paths are now allowed for PRF and MOSAIC_PRF files, in addition to ./cal, which was previously enforced by the command-line Perl scripts. The default backup names "PRF.fits" and "Mosaic_PRF.fits" are no longer searched for.

8. Flux input to prf_estimate turned off
There is a problem in prf_estimate.pl when the input list of sources has a "flux" column. Use of this flux has been turned off. It will now always estimate fluxes from the cropped images themselves.

9. Mosaic Reinterpolate on a list of different-sized arrays
There was a problem with memory allocation with different-sized arrays in mosaic_reint. This has been fixed.

10. Linux roundoff errors
Linux versions are now being compiled with gcc as 32-bit versions with the SSE floating-point rules. The latter is to turn off the default extended precision of gcc, which was giving numerical differences in some pieces of code compared to Solaris 10 and Intel Macs.

11. flatfield.pl problem on Macs
This was due to unrecognized NaNs and has been fixed.

12. Table column overflows by sourcestimate
This usually showed up as an error from aperture. This has been fixed.

13. Incorrect WCS in absolute_minimum and median mosaics
These have been fixed

14. Weight by exposure time option added to the Mosaic Coadder module in the GUI.

15. RMask bug that prevented Bottom Threshold from being larger than Top Threshold has been fixed.

16. Fitting Area parameters in the Source Estimate module now take precedence over the presence of the Fit Radius module.

17. Fixed the bug that prevented the RMasks from being properly reset when the list of input images is changed in the GUI.

18. The module Mosaic Geom can now be run in the GUI without including the FIF module.

19. The GUI will now display a subset of the output files for most modules.

20. A bug preventing the delete_intermediate_files option from working in the GUI has been fixed.

21. A bug affecting the creation of moving object mosaics has been fixed. Users can now choose the input BCD that they wish to set as the stationary reference frame.

22. A glossary of keywords has been included in the GUI help.

23. Some minor fixes have been made to the filenames in namelists exported from the GUI.

MOPEX Release 02/21/08 (v16.3.7)

The following changes were made to the modules and scripts used by both the command-line and GUI versions of MOPEX. Updated from the June 2007 version.

1. Uncertainties will now be calculated for aperture photometry, using data uncertainties. No additional input required.
Aperture uncertainties were added to the Mopex aperture module. It uses the data uncertainties (or "gain-estimated" ones, if enabled). The aperture flux uncertainty (sigma) is simply the sum over the aperture area:

 sigma = sqrt(Sum(sigma(i)^2))

The bad value and the value if missing is "-9.99". Any bad uncertainty value in the aperture will lead to a -9.99. Uncertainty in background subtraction is not included.

The columns are called "ap_unc1", "ap_unc2", etc. in the aperture.tbl and extract_raw.tbl. The Apex "select" script can select on them in creating the final extract.tbl.

2. Option to use data uncertainties in APEX's SNR
There is a new switch for command-line namelists apex.pl and apex_1frame.pl and in the GUI:

 use_data_unc_for_fitted_SNR = 1.

The switch tells MOPEX to use the data uncertainties for calculating the signal-to-noise ratio given in the final extract table. Default is 0, and it will use the current gaussnoise noise estimate (i.e. basically the background RMS). This is also setable on the command line of apex.pl and apex_1frame.pl, as option "w". If set, it will switch to the data uncertainties (or the "gain-estimated" ones if enabled). See the Apex User's Guide for how the Apex SNR is defined.

This step is part of our re-evaluation of Apex's signal-to-noise estimates. Most users will not need this switch at the present time.

3. Improved mosaics with Drizzle interpolation using RMasks. No additional inout required.
Users can use the Drizzle interpolation option (INTERP_METHOD = 2) in Mosaic Interpolate and use their choice of outlier rejection. The interpolation is run twice - the first time through runs a linear interpolation scheme, which is then passed to the outlier rejection algorithm to create the RMasks. The interpolation is then re-run with the Drizzle algorithm, masking out the bad pixels flagged in the RMasks. Users should not run Mosaic Reinterpolation when using Drizzle.

4. Bug Fixes:
Overlap (bad indexing in determining clusters) - Major.
The Overlap version of mopex_030106 was not affected. A bug was present in the version released with the GUI which may in some cases lead to improper overlap corrections. It has been fixed in this version.

Sourcestimate (typo in X,Y pixel ratio) - Minor.
Only important if your pixels were very non-square.

See the MOPEX Bug List for a full list of fixes of previously reported bugs

MOPEX Release 06/15/07

The following changes were made to the modules and scripts which both the commandline and GUI MOPEX use (updating them from the 1 March 2006 version). Please see the MOPEX GUI Release Notes (PDF, 198 KB) for more information.

* Environment settings for command-line MOPEX are now in mopex-script-env.csh instead of mopex.csh.

* The module "detect" was modified to provide higher precision in the x,y positions in the output detection table.

* The Perl module "mosaic.pl" was modified to allow use of existing Rmasks.

* The module "sed_fif" was modified to generate SED mosaics for special cases when PA_SLT ~= 0 or 180. The computation of ROT_SLT was also revised for all cases.

* A bug was fixed in the "aperture" module. It gave potentially incorrect aperture fluxes when distortion was present, for example in BCD data.

* Use of the Galactic and Ecliptic coordinate systems was implemented in addition to the Equatorial system Mopex has used so far. Also all of the projections implemented in the WCS library are now available for use in Mopex. The coordinate system and projection type of the mosaic are specified in the following namelist block:

&FIDUCIALIMAGEFRAMEIN
...

Projection_Type = "ARC",

Coordinate_System = "Galactic",

&END

The default value of the projection type is "TAN" and of the coordinate system "J2000". The possible options for the coordinate system are "J2000", "Galactic", and "Ecliptic". The options for the projecton type are:

LIN, AZP, TAN, SIN, STG, ARC, ZPN, ZEA, AIR, CYP, CAR, MER, CEA, COP, COD, COE, COO, BON, PCO, GLS, PAR, AIT, MOL, CSC, QSC, TSC, NCP, DSS, PLT

*A new switch "use_extract_table_for_aperture" has been added to control the input of the aperture photometry module. The default value if 1, i.e. the extract table is used as input for the module aperture. If the switch is set to 0, then the detection table is used as input for the module aperture. It is especially useful for extended objects when point source fitting is not applicable and only aperture photometry measurements can be performed.