S19.2 (Warm Mission Only; processing started on 2015 June 15)
1. Recalibration of IRAC Flux Densities in Channels 1 and 2.
The absolute photometric calibration was updated using the same method that was used for the final cryogenic IRAC calibration. The uncertainty in the absolute photometric calibration is dominated by the uncertainty in the flux density of the fundamental calibrators, Vega and Sirius, and is ≈ 2% in both channels.
2. Linearity Solution Improved in Channel 1.
The linearity solution was adjusted in channel 1 only. The change in flux from the old to new linearity solution is < 1%. The solution in channel 2 did not need to be changed.
3. Pmasks Updated
The quasi-static bad pixel masks were updated.
4. New Flat-fields Derived
Newly derived flat-fields are now in use with final precisions of 0.17% and 0.09% in channel 1 (3.6 µm) and channel 2 (4.5 µm), respectively.
5. Header Information Updated
Keywords for PCRS peak-up observations were added to FITS headers, including the R.A., Dec., and Proper Motion of the star used for the peak-up.
6. Pointing Refinement Catalog Updated
Pointing refinement in the IRAC pipeline utilizes the 2MASS catalog. The positional information of stars in the 2MASS catalog has now been improved with US Naval Observatory’s astrometric catalog UCAC4 proper motions, as 22% of 2MASS sources have non-zero proper motions.
7. Read Noise Values Updated to Observed Warm Campaign Values
Previously, the cryogenic read noise values were used in the (C)BCD pixel value uncertainty calculations. The read noises have now been updated to measured warm mission read noise values.
8. Distortion Correction Updated
The distortion correction has now been updated from a third order polynomial solution to a fifth order polynomial solution, allowing an improvement of relative astrometric accuracy from 125 milliarcseconds to 30 milliarcseconds in both channels.
S19.1 (Warm Mission Only; 2015 August 29)
1. Imask Updates
Turning on the residual image (“latent”) bit (10) in the imask now uses both residual images created within the current AOR and residual images left by preceding observations. The latter are found with algorithms that track the time decay of residual images.
2. Saturation Correction and Column Pull-down Updates
Updated saturation and column pull-down thresholds for artifact correction.
S18.25 (Cryogenic Mission Final Processing; 2012 March 15)
1. Recalibration of IRAC Fluxes in Channels 3 and 4.
While checking the calibration of Spitzer instruments by comparing the IRAC calibration to the HST calibration database, an error was identified in the S18.18 IRAC processing. In updating the flux conversions (FLUXCONV), erroneous color corrections were used for the calibration stars. The color corrections used were incorrect for the 5.8 and 8.0 µm channels. The S18.18 flux values were multiplied by 1.000, 1.000, 0.968, and 0.973 for the 3.6, 4.5, 5.8, and 8.0 µm channels, respectively. The headers were also updated to indicate pipeline version S18.24 and revised flux conversion values.
S18.18 (Warm Mission Only; 2010 May 27)
1. Recalibration of IRAC Flux Densities in Channels 1 and 2.
The absolute photometric calibration has been updated using the first year of warm campaign calibration stars.
2. Added Header Keywords
BMJD_OBS (barycentric modified Julian Date) and AORHDR (AOR requested to be observed in HDR mode) were added to (C)BCD FITS header.
3. New Pmasks and Flat-fields Created Using only Warm Mission Data.
S18.18 (2010 May 27)
1. Saturation Correction in Pipeline
Saturation is now corrected before artifacts. Artifact correction for saturated sources is now possible. The criteria for selecting sources for correction were changed. The source selection is now frame time dependent. Bit 13 in imask is changed to bit 4 after correction has been performed.
2. Imask Updates
Imask header is now formatted in a way similar to the BCD header and includes information about the various bits that may have been set.
3. ABADDATA Now Often Corrected
Frames with ABADDATA set in raw file headers are now corrected. This problem is due to shifting the data by one pixel after an extra word was read into the data. If only one such instance occurs in a frame it is now corrected. Frames with multiple instances are not corrected. The BCDs will include a header keyword BADPIX. If BADPIX = T, then another header keyword ZEROPIX gives the number of bad pixels. If ZEROPIX = 1, then header keyword ZPIXPOS gives the position of the pixel that was fixed and header keyword BADFILL gives the value (in DN) of the fixed pixel.
4. New Added Header Keywords
Barycentric Julian Date calculated with SCLK precision is now included in header keyword BMJD_OBS. Also, a new header keyword AORHDR has been added. This keyword is true if the entire AOR in which the frame was taken (not just the frame itself, such as the first frame in every AOR) was taken in the high dynamic range (HDR) mode.
S18.14 (2010 December 10)
1. Saturation Correction Update
Only sources in the 2MASS Point Source Catalog are now corrected (extended sources, such as the nuclei of bright galaxies are not corrected). Bit 13 in imasks is now flipped to zero after the saturation has been corrected.
2. Mosaic of Imasks
Mosaics of all the imasks for a given frame time in a given AOR are now produced by the pipeline and placed in the PBCD directory (mmsk files).
3. Dmasks No Longer Archived
We no longer provide dmasks as they contain misleading and incomplete information. The imasks are more robust, include flagging of various artifacts that are not present in dmasks, and make full use of the saturation correction made by the pipeline.
4. Higher Accuracy Pointing Refinement
Pointing refinement is now done with the help of the 100x sampled PRFs, leading to a more accurate pointing solution.
S18.5 (2008 December 1)
1. A New Saturated Source Fitter
The pipeline now attempts to systematically find all saturated point sources in the images and fit them using an appropriate PSF that is matched to the unsaturated wings of the source. The new module replaces the saturated point source with an unsaturated point source that has the correct flux density of the point source. Please note that the new module does not successfully fit super-saturated point sources or point sources that are too close to the edge of the field of view for proper fitting. Only the CBCD files are saturation corrected (not the BCD files). The saturated pixels that have been replaced are identified within the bimsk (imask) files (bit 13).
2. A New Muxstripe Remover
Muxstriping is now fit and corrected in a new pipeline module. The noise of the affected pixel area is compared to the unaffected pixels in the frame and a deviation is removed, without changing the background level or the flux in the pixels. Occasionally this correction fails and the muxstriping is unchanged. Only the CBCD files include this correction, not the BCD files.
3. Pmask Bits Now Included As Part of Imasks
See the imask bit definition in Section 7.1 of the IRAC Instrument Handbook for the definition of the various bits.
4. Mosaic Mask Files Now Available
The PBCD mosaics created in the pipeline now have associated mask files. The imasks associated with the CBCDs that were used for the creation of the mosaic have been combined to create the mask file, so the bit values can be deduced from the imask bit definition table.
5. Improved First-Frame Correction
More appropriate skydarks are now used in the pipeline, producing an improved first-frame effect correction.
S18.0 (2008 June 12)
1. Muxbleed Correction Update
The correction of the muxbleed effect in the BCD frames was updated. After extensive testing a new functional form and scaling law was developed for muxbleed correction in channels 1 and 2. The new functional form and scaling law correct muxbleed better than before.
2. MOPEX Now Using the bimsk.fits Files
In creating the final mosaic image, *maic.fits, the MOPEX pipeline now uses the *bimsk.fits files as input, instead of the *dmsk.fits files. The *bimsk.fits files have more relevant information useful for flagging when building mosaics and analyzing the images.
3. Header Information Updated
WCS CD matrix keywords were added to the Post-BCD file headers. The CDELT1, CDELT2, and CROTA2 keywords have been preserved but were placed in comments to avoid any confusion when handled by astronomical software. The following keywords were added to the BCD, CBCD, and Post-BCD image files: PXSCAL1, and PXSCAL2, and PA. The keywords are the pixel scale along axis 1 and axis 2 in arcseconds/pixel and the position angle of axis 2 (East of North) in degrees. Keywords containing additional information on the AOR mapping parameters have been added in a separate section of the header.
S17.0 (2007 December 5)
1. Artifact Mitigation Within the Pipeline
Artifact-mitigated images from the BCD pipeline and their associated uncertainty images (*cbcd.fits and *cbunc.fits) are now available in the archive. These images include corrections for column pull-down and banding, induced by bright sources in the images. The corrections are empirical fits to the BCDs and may not always improve the data quality. The standard BCD files (*bcd.fits) remain available in the archive. The mosaics (post-BCD products) are now created from the *cbcd.fits images.
2. Muxbleed Correction Updated Again
The muxbleed correction has been revised to include a better empirical fit.
3. Two-Dimensional Subarray Images
A two-dimensional image is now generated for each subarray BCD cube. Each pixel in the 2D image (*sub2d.fits) is a robust (outlier-rejected) mean of the 64 samples of the *bcd.fits cube. Two-dimensional masks, uncertainty images, and coverage maps are now also provided.
4. Artifacts Now Flagged In Subarray Images
The subarray imasks (*_bimsk.fits) now include masking for muxbleed, column pull-down and banding, induced by bright sources. The updated masks can be used to mitigate bright source artifacts the same way as with the full array data.
5. Darkdrift Values Written Out in the Subarray Header
The pipeline darkdrift module reduces a “jailbar” bias effect in the IRAC images. The values used for the reduction within the pipeline are included in the header of the full array data, and now in the header of the subarray data for all the planes. This allows the user to remove the correction, if desired.
6. Mosaic Header Updated
We have added more information to the mosaic header.
S16.0 (2007 April 9)
1. Labdark Change for 100 Second HDR Data
Within 100 second AORs, the channel 4 observations are split into two 50s frames. The second 50 second frame received an incorrect non-HDR 50 second labdark instead of an HDR labdark. This was a minor problem, and has now been corrected.
2. Muxbleed Correction Updated
The module that detects and corrects the muxbleed caused by bright sources has been updated. It now performs a more consistent and better correction than previously.
3. Artifacts Flagged Within the Pipeline
The imasks (*_bimsk.fits) now include masking for muxbleed, column pull-down, and banding induced by bright sources on images. The updated masks can be used with existing contributed software to mitigate bright source artifacts, and will be used in future versions of the IRAC pipeline after mitigation algorithms have been implemented. In general, observers should not flag artifacts in mosaicking unless they have observations at various roll angles.
4. Pixel Linearization
The handling of bad and saturated pixels has been changed – they are in most cases now left with their original values, as opposed to being set equal to NaN. The method of flagging saturation in BCD mask files was changed, and now more accurately reflects the presence of saturation.
S15.0 (2006 November 28)
1. Ghost Images And Scattered Light Flagged Within Pipeline
Pipeline versions of the ghost image and scattered light detection algorithms have been integrated into the IRAC pipeline. The modules use the location of bright sources upon the array (ghost image) or just outside the array, as found in 2MASS catalogs (scattered light) to predict possible optical ghosts and scattered light locations, and flag these pixels within the imask. The imask is an ancillary data product now available through the archive. IRAC observers should use the imask instead of the dmask when making mosaics etc.
2. Incorrect Group Ids in Header to Be Fixed
A bug that caused a small percentage of BCDs (< 0.1%) to have an unreadable header and which were therefore not pipeline-processed, has been fixed. This should significantly decrease the number of missed BCDs in large mapping programs.
S14.0 (2006 May 3)
1. Darkdrift Module Changes
As mentioned below, in S13 the darkdrift module was applied only to channel 3 data. This module is used to adjust the bias level in the four readouts in an array, thereby removing vertical striping in the data, the so-called "jailbar effect." After S13 reprocessing of IRAC data it was found that the jailbar effect can be triggered in channels 1, 2 and 4 as well. Therefore, the darkdrift module will again be applied to all four channels, and all the IRAC data will be reprocessed with pipeline version S14. We have released a "jailbar corrector,” which may be used to correct for the jailbar effect. It produces similar results to the darkdrift corrector module in the IRAC pipeline.
1. “Superboresight” Pointing Refinement (S13.2 And Thereafter)
Previous versions of the pipeline performed pointing refinement on each IRAC channel separately. The refinement was performed by matching detected point sources to 2MASS stars and registering the astrometry to minimize the positional offset between matches. In most cases, the refinement of channels 3 and 4 is less accurate as the number of stars detected in an individual frame is less than in channels 1 and 2. Superboresight refinement corrects the astrometry for all the four channels by simultaneously using appropriately weighted matches from all the four channels and the known orientations of the FPAs. This method can dramatically improve the pointing accuracy for channels 3 and 4, and it removes any positional offsets between channels. The superboresight pointing is inserted into the CRVAL1 and CRVAL2 keywords in the header, while the basic (less accurate) pointing refinement remains in RARFND and DECRFND header keywords, and the original boresight pointing solution is placed in new header keywords ORIG_RA and ORIG_DEC.
2. First-Frame Effect
The interval between frames (INTRFDLY) is now maintained in a database, instead of the pipeline reading the previous image in an AOR to process the current image. This streamlines operations and handling of missing images. It is also placed in the header as a keyword.
3. Linearity Correction
New linearity corrections have been calculated from on-orbit tests and small changes were made to channel 3 full array and all channel subarray data. The effect is roughly 2% at half-well, and 8% at 90% full-well in channel 3. The other channels are within specifications and the linearity corrections were not changed for them.
4. Darkdrift Module Changes
The darkdrift or jailbar effect (see Section 5.1.12) is corrected in the pipeline software by applying a constant offset per readout channel (arranged in columns), derived from the median of those columns such that their arithmetic mean is zero. In other words, all readout channels are adjusted to a common additive bias level. In in-orbit tests, the mean offset and correction was found to be negligible, except in channel 3 data. Therefore, in S13 reprocessing, the darkdrift correction was only applied to channel 3 data. The derived correction values for each channel are located in the header in the keywords DRICORR1, DRICORR2, DRICORR3, and DRICORR4. The overall background term determined is DRIBKGND.
5. Distortion Files
The subarray distortion files were found to be derived from an incorrect place on the full array and have now been updated with correct ones. This should only make a small, but noticeable difference in pixel sizes when measuring relative separations in the subarray.
6. Superskyflat
A new superskyflat has been derived from the first two years of flat-field data on IRAC and will be used as the flat-field for all reprocessing and in future campaigns. Uncertainties in the pixel-to-pixel responsivity calibration are only 0.5%, 0.2%, 0.2%, and 0.05% for channels 1 - 4, respectively.
7. Flux Conversion
The flux conversion has been updated to reflect the derivation described in the IRAC calibration paper by Reach et al. (2005). The previously used numbers were from a nearly complete phase of this derivation, but different by 3% in channel 4.
8. Added Keywords
1. Median brightness of calibration skydark (SKYDKMED)
2. More precise start time of observation (SCLK_OBS).
S12.0 (2005 May 9)
1. New Stellar Mode
New observing mode, “Stellar Mode” has multiple full array short time exposures within channel 1 and 2 and at the same time has a longer integration in channels 3 and 4. This allows for brighter objects to be observed in the longer wavelength channels to higher signal-to-noise without saturating in the shorter wavelength observations. Available frame times are 0.4/2 seconds, 2x2/12 seconds, and 2x12 seconds/30 seconds. The first number(s) refer to channels 1 and 2, the last number to channels 3 and 4.
2. Median Subtracted Skydark Value
The median value of the frames used to create the skydark subtracted from the data was placed in the header of the BCD: keyword SKYDKMED.
3. Labdark Name to Headers
The name of the labdark subtracted from the data was placed in the header: keyword LBDRKFLE.
4. Exact Time of Observation to Headers
The time of the observation (SCLK_OBS) is now computed using telemetry only to allow for more exact timing. This keyword will be placed in the database and header. Further S13 changes will include calculating the first-frame correction from this more exact timing.
5. Pointing Keyword Additions
Keywords PTGDIFFX, PTGDIFFY were inserted to refer to the pointing differences in actual pixels along the X and Y axis.
S11.0 (2004 November 30)
1. Fix to EQUINOX Keyword
The EQUINOX header keyword for BCDs has been fixed.
2. New Delay Header Keywords
Other changes to the header include the addition of the First Frame Delay and Immediate Delay (FFDLAY & IMMDLAY) times, calculated from the first-frame correction.
3. CHECKSUM Changes
Previously, a DCE with a non-zero CHECKSUM from MIPL was not allowed to process through the pipeline. In S11, the CHECKSUM will now be reported within the header and the DCE processed.
4. First Frame Correction for HDR Observations
The first-frame effect correction has been fixed for the high dynamic-range observations. The only remaining bug is for the intermediate frame times (12 seconds) when used as part of an HDR frameset. This effect will not be noticeable except as a slight background DC-level offset from frame to frame in the 12 second data as part of 100 second or 200 second HDR framesets.
5. Super Skyflat Creation
After studying last year's flat-fields and finding no noticeable change from campaign to campaign, a superskyflat has been made from last year's flat observations. A subarray flat has been extracted from this superskyflat, and both have been loaded into the pipeline.
6. Overlap Correction Added to Pipeline
Overlap correction is now applied in the post-BCD pipeline.
7. Mosaic Image Keyword Additions
The mosaic image headers have been populated with more keywords.
S10.5 (2004 June 2)
1. ffcorr Delay Time Update
Updated the ffcorr module to use the correct delay time between frames for full array non-HDR frames. The HDR frames will be fixed in S11.
S10.0 (2004 May 7)
1. New Linearity Mode
New linearity model in channel 4 (full and sub). Change from quadratic to cubic (actually updated in S9.5.2).
2. ffcorrOutput Changes
Module ffcorr set to output only one plane for interpolated correction image rather than all planes.
3. New FITS Keyword
Create and populate new FITS header keyword (DS_IDENT).
4. Readnoise Update
Update to read noise in initial noise image.
5. NaN-Value Consequences in Uncertainty Images
If BCD pixel = NaN, uncertainty pixel = 0.
6. New Skydark Keyword
Keyword from dark ensemble placed in BCD header (SKYDRKZB; skydark zodiacal background estimate added into header of science BCD).
S9.5 (2004 March 1)
1. New Fields in Caldata Tables
Addition of two fields, hdrmode and numrepeats, to caldata tables. Requires a backfill script to transform and migrate current fallbacks and metadata to new tables. The HDRMODE field is in current use. The NUMREPEATS field is to facilitate use of the external repeat number in future calibration activities.
2. Flux Conversion
In S9.5 the flux conversion will be delivered in an IPAC table, for example:
char Comment Calibration data file for dntoflux module. char INSTRUME = 'IRAC' int CHNLNUM = 4 char fluxconv = 'Conversion factor in MJy/sr per DN/s' char fluxconvunc = 'Uncertainty in fluxconv' |fluxconv |fluxconvunc | |float |float | 0.195 0.020
3. HDR Skydark Change
HDR skydarks are now different from non-HDR skydarks. The skydark processing branch of the pipeline is now aware of channel 4 repeats and the pipelines fetch skydarks for the correct repeat. This is possible due to new fields in the caldata tables.
4. Scattered Light Removal Added
Scattered light removal module (“slremove”) added to science pipeline and calibration preprocessing.
5. fpgen Used in Product Header
Calibration ensemble pipelines now use “fpgen” to clean up the product header.
6. Subarray Flat Creation
New pipeline to create subarray flat-fields from full array flat-fields.
7. Latent Ensemble Additions
Latent ensemble creates new request median and request average images.
8. New Keywords to Mosaic Headers
New keywords to be added to the mosaic header: AOT_TYPE, AORLABEL, FOVID, FOVNAME, PRIMEARR, OBJECT, PAONUM, CAMPAIGN.
S9.1 (2004 January 22)
1. Pointing Reconstruction
Less than 0.1% of the DCEs may not have pointing reconstruction applied to the data. BCDs with USEDBPHF=F indicate that the Boresight Pointing History File was not used, and the R.A. and Dec. in the headers for these cases are based on pre-observation predictions which can be off by 5 - 50 arcseconds. Do not use such data if pointing is important.
S9.0 (2004 January 14)
1. Annealing in Channel 1 and 4
Anneals: Both channels 1 and 4 will continue to be annealed simultaneously, every 12 hours (after each downlink) to reduce persistent images.
S8.9 (2003 December 8)
1. Impact of Long Term Residual Images
Some AORs have been affected by long term residual images from previous observations. For the most part, observers have sufficiently dithered so that the impact is minimal, on processed and co-added data.
2. Noise in Data vs. Predictions
Note that the noise in the images and the sensitivity to point sources are not equal to our pre-launch predictions (e.g., as available from our website until December 19, or in the Observer's Manual versions before 4.0), although they are close. New sensitivity numbers are available in the revised Observer's Manual (version 4.0), which was available at our website starting ≈ December 19, 2003. For reference, the ratio of the new point source detection threshold to the pre-launch advertised value, for low background observations in 30 second frames, is 0.69, 0.75, 1.60, and 1.31 in channels 1, 2, 3, and 4, respectively. The apparent modest decrease in sensitivity in channels 3 and 4 is under investigation.
3. Persistent Images in Channel 1
When a bright source (K=13 mag or brighter) is stared at for a long time, for example, during a downlink, it will leave a persistent image in channel 1 that decays very slowly (persists for several hours or more). A persistent image mitigation strategy involving annealing the array after downlinks has been put in place for nominal operations. These anneals will erase the persistent images from the array, but do not protect against persistent images from bright object observations that can accumulate on the array before the next downlink. Science impact: left unmitigated, you will have extra, spurious sources in your image. These sources have a PSF that is wider than the actual true source PSF. Dithering helps to get rid of these spurious sources.
4. Persistent Images in Channel 4
These are different in nature from the channel 1 persistent images. A bright source leaves a persistent image that can last for more than a week and even through IRAC power cycles. These images keep building up on the array. However, the amplitude of the persistent images is rather low. Annealing has been found to erase also the channel 4 persistent images. Therefore, we will anneal both channels 1 and 4 simultaneously, every 12 hours (after each downlink), to erase persistent images. Again, dithering helps to get rid of these spurious images.
5. Diffuse Stray Light
All IRAC images contain a stray light pattern, resembling a “butterfly” in channels 1 and 2, and a “tic-tac-toe” board in channels 3 and 4. These artifacts are due to zodiacal light scattered onto the arrays, possibly reflected from a hole in the FPA covers above the channel 1 and 2 arrays, and from reflective surfaces outside the edges of channel 3 and 4 arrays. The stray light scales with zodiacal light, which is the light source for our flat-fields, so the stray pattern contaminates the flats. As a result, the flat-fields will aesthetically remove the stray light rather well from images but will induce systematic errors of approximately 5% in flux calibration for point sources that fall in the peak stray light location. Dithering will mitigate this effect, because it is unlikely that a dithered observation will keep a source within the stray light lobes. Diffuse stray light will be removed from both the flat-fields and the science frames in a future version of the pipeline.
6. Stray Light from Point Source
Spot allows you to overlay stray light boxes on any image; if a bright star is placed in those boxes during an observation, a scattered light patch will appear on the array. We have found three more such boxes during testing, in channels 1 and 2. The new stray light boxes are included in Spot now and are also shown in the new Observer's Manual. Channels 3 and 4 have less stray light, and the stray light inducing regions are not the same as the ones we guessed (by analogy to channels 1 and 2) from the lab tests, so the channel 3 and 4 boxes were removed from Spot. In channels 3 and 4 the stray light arises when a star lands on a thin region just outside the array (the same region that causes the “tic-tac-toe” pattern from diffuse stray light in flat-fields). A redundant observing strategy will help eliminate stray light problems. Observers covering fields with bright sources should inspect the individual images; this is required if the depth of coverage is less than three, to identify spurious spots and rays that could be mistaken for real astronomical objects.
7. Dark Spots on Pick-Up Mirror
Dark spots on pick-up mirror. There is contamination on the mirror, which causes a dark spot about 10 pixels wide in channels 2 and 4. This is a 15% effect. Flat-fields completely correct for this feature in the data.
8. Muxbleed
We have a correction algorithm, but the coefficients need fine-tuning. Furthermore, for bright sources, muxbleed does not scale linearly with source brightness, so even a sophisticated algorithm cannot accurately remove it. Some experiments at fitting the muxbleed for bright sources indicate that the decay pattern is always the same, and only the amplitude appears to be variable.
9. Banding and Column Pull-down
Banding and column pull-down. A bright source on the array will cause its column to be pulled down by a small amount. An algorithm to cosmetically correct the images for column pull-down has been developed and is being tested. This appears to be an additive effect. An analogous effect for an extremely bright source is that the entire image appears to have a different DC level from the preceding and following images. The physical origin of these effects and the probably related (and already known) banding effect is not yet understood. This work is in progress.
10. Mosaics
Mosaics produced by the online pipeline for HDR mode data incorrectly weight the short and long frame times. For long exposures (> 12 seconds), data are effectively taken in HDR mode, and hence the pipeline produced mosaics will not be very useful.
