IV. 2MASS Data Processing

There are two steps in Quality Assurance (hereafter "QA"). The first step is referred to as "24hr QA" and is a quick check of the data within 24hrs after the data tape reaches IPAC. The second step is referred to as "Nightly Science QA" and is the comprehensive check of the data performed after the night is fully processed. Both of these steps are described in more detail below.

i. 24hr QA

The 24hr QA report is generated after a night's data tape has reached IPAC. In this quick processing, only the darks, responsivities, and calibration scans are generally run. The 24hr QA report checks that the telescope pointing, scanning, and focus are within tolerance. The report also checks that the intrument is behaving as expected by checking that the darks and responsivities are within nominal limits. The 24hr QA report further determines whether with default processing the night is photometric or not. This pre-processing allows us to see if any overrides will be needed in full production processing, and if the night appears to be a disaster photometrically, we can triage the night at this step and not schedule it for full production processing. The observatory can also be immediately notified that these tiles need to be re-observed.

ii. Nightly Science QA

The Nightly Science QA report shows the quality parameters generated by each subsystem in the data processing pipeline. In this processing, all of the data are run with the exception of any scans triaged at the 24hr step. The report generates an overall review of the night; scan-by-scan quality reports on telescope tracking and stepping, background/seeing/focus, astrometry, photometricity, and extended source processing; and a report on the preliminary quality scores for each scan. Diagnostic images and plots are also generated.

The quality reviewer uses the preliminary scores generated above as a baseline, then checks the quality reports along with the diagnostic images and plots to assign final scores to each scan. The final scores are then justified in a report which is mailed to UMass. UMass reviews the report, agrees with the recommendation or requests slight changes to the scoring, and then IPAC generates transaction files which (a) notify the observatory of the scores for each scan so it can prioritize any re-scanning that needs to be done and (b) which notify the database team at IPAC of scans that have sufficient quality to load into the working database.

Scans are given quality scores from 0 (scans that need immediate re-scanning) to 10 (scans that meet all level-1 specifications). The recipe used by the reviewers to determine the quality of each individual scan is as follows: In general, each scan is scored using a base quality number multiplied by two individual quality factors (score = base * photometric factor * sensitivity factor), as detailed below.

• The "base" score is 10 for optimally dithered scans (those northern nights taken after 970606n) or 5 for those few northern nights taken on 970606n or earlier. For all data from the southern facility, the base score is 10.
• The "photometric factor" ranges from 0.0 to 1.0 (rounded to the nearest tenth) computed from mult1*mult2*mult3:
• "Mult1" is 1.0 unless the number of cal sets in the photometric interval is less than six (under the old calibration strategy implemented before 971011n, where two cal fields were observed every two hours) or less than five (under the new calibration strategy implemented for 971011n and after, where a single cal field was observed every hour). For these exceptions, the photometric quality of the science scans is downgraded since there is less information about the photometric zero-point stability for the interval. Specifically for these exceptions, multi1 is computed from sqrt((number of cal sets)/6) for the old strategy or sqrt((number of cal sets)/5) for the new. It should be noted that the new strategy is used for all data from the southern facility.

  There is one special case under the new strategy -- that of a one-hour block comprised of two cal sets only -- where multi1 is set to 0.3. Such two-cal intervals are used only if all other photometric factors receive perfect grades.

• "Mult2" is based on the photometric dispersions of stars seen in the six consecutive scans of a calibration set. Multi2 is 1.0 unless this dispersion is larger than 0.04 mag. In this case, the dispersion needed to pass the cal sets is used to compute the downgrade using mult2 = 2 - ((dispersion used)/0.04 mag), and this downgrade is applied to all scans in the photometric interval. It follows then that calibration sets with dispersions exceeding 0.08 mag are considered non-photometric (multi2 = 0).
• "Mult3" is computed from photometric repeatability in the overlap regions between scans. For a contiguous block of science scans, mult3 is 1.0 if the peak-to-peak scatter for the averaged differences of all scan pairs is below 0.050 mag. For larger scatters, mult3 is downgraded: mult3 = 0.7 if the scatter is 0.050-0.075 mag, 0.4 if the scatter is 0.075-0.100 mag, or 0.0 if the scatter exceeds 0.100 mag (considered non-photometric).
• The "sensitivity factor" ranges from 0.0 to 1.0 (rounded to the nearest tenth). The sensitivity in a scan is a function of both the seeing shape and the background level, and it has been determined empirically that the sensitivity score in a scan can be computed per band using the formula shape*background^0.29. Values of this sensitivity score reflect the probability that the sensitivity is sufficient for the data to meet level-1 specifications. Specifically, sensitivity scores above the 75% threshold receive a sensitivity factor of 1.0, those with 50-75% receive 0.8, 25-50% receive 0.5, <25% receive 0.3 except for those that have 0% chance (those are given a factor of 0.1 as discussed below).

QUALITY = 0: First priority for re-scanning

• Photometric non-repeatability of science scans:
• Peak-to-peak scatter of 0.10 mag or greater in overlaps for a science block
• Photometric non-repeatability of cal scans:
• Dispersion in photometry of calibration field stars > 0.08 mag
• Large scatter in zero-point determination:
• Dispersion in photometric zero-point fit > 0.04 mag
• Uncertain photometric zero-point:
• Less than two valid calibration intervals
• RA slippage during scan:
• Double images produced (first seen on 970616n)
• Quadrant jumps:
• Connector problems causing severe ADC offsets (first seen on 970714n; corrected on 970925n)

QUALITY = 1: Second priority for re-scanning

• Severely degraded sensitivity:
• Data with a photometric factor greater than 0.0 but with a 0% chance of meeting level-1 sensitivity specs
• Severely elongated images:
• Images of point sources with minor to major axis ratios less than 0.81 in any band

QUALITY = 2: Third priority for re-scanning

• Galaxy extractions compromised:
• Rapidly varying seeing conditions which cause problems with star/galaxy classifications