2MASS Final Catalog Scan Selection Criteria

Summary

The net result of scan selection based on final processing quality, net sensitivity ranking, and iteration to maximize sky coverage is that the all-sky release will be drawn from 59,731 scans. The list of scans comprising the all-sky release is given here.

Scan List for Final All-Sky Release

I. Useful Links Concerning Catalog Scan Selection

Tile Selection for 2MASS Second Incremental Data Release
Scan Overlap Summary (L. Cambresy and J. Rho)
Tiles With Q<5 in Version 3 Processing (R. Hurt 2/28/02)

II. Scan Selection Criteria for 2MASS Final Catalog Generation

Preparation
1. Final scan quality scoring in place (QASIT)
2. Position reconstruction quality noted
3. General accounting
  - Summary of all scans of all survey and gap-fill scans
  - Validate that Working DB tables contain all final processed data
  - Decide how to handle Tiles with Q<5
  - Scan overlap statistics compiled
  - All "gaps" and identified
First-Pass Scan Selection
1. For each tile in full sky coverage plus gap-fill tiles, select scan of tile with the highest default score
2. If there are multiple scans with identical default score, rank by considering relative quality and second order quality factors:
  - Primary sensitivity metrics:
    - Zero point offset (dm = ZP)
    - Seeing and Background (dm = -0.2*(PSP-PSP₀))
  - Secondary quality indicators such as seeing stability (e.g. untracked seeing), background variations (e.g. airglow and banding), zero point stability, image aspect ratio, etc
  - Position reconstruction quality
  - Observation-time contiguity with adjacent scans
  - Observatory preference (e.g. CTIO vs. Mt. Hopkins)
  Issues: What are relative weights of different factors?
  Should secondary factors be allowed to override default quality score?
Sky Coverage Iteration
1. Determine Sky Coverage of preliminary full scan set using coverage and gap tables
2. If any scan pair fails minimum overlap or gap tests, select alternative scan that has same default quality score and satisfies overlap, or next lowest score (above minimum score threshold)
3. Iterate a-b until have full-sky coverage

III. Accounting From Final Processing Results

70712 Scans Processed Receiving Q>0

59975 Unique Tiles (including Gap-Fill Tiles)

Figure 1 - Histogram of Best-Quality (Q_best) for each unique Tile (including Gap-Fill Tiles)

59574 have Q_best>5
401 have Q_best < 5 (364 Survey and 37 Gap-Fill Tiles)

Figure 2 - Histogram of number of scans achieving same Q_best per Tile

2214 Tiles have >1 scan with same Q_best
Maximum number of scans for a Tile having same Q_best = 23 (2 Gap-Fill Tiles)

IV. Scan Selection Proposal

Final quality score contianed in SIT file is primary discriminant for multiply-observed tiles

Generate list of all tiles showing best quality achieved (Q_best), number of scans having Q_best, and date, hemisphere and scan numbers of all scans having same Q_best

For all tiles having >1 scan with same Q_best, determine the Net Relative Sensitivity, dS, for each scan. The Net Relative Sensitivity is measured magnitudes relative to the first scan and is defined as follows:

for i^th scan of a Tile: dS_i,b = [ZP(b)_i - ZP(b)₁] - 0.2[PSP(b)_i - PSP(b)₁]

where b is the band: J,H,K_s, ZP is the photometric zero point offset and PSP is the photometric sensitivity parameter.

For first scan in multiply observed set, dS_1,J=dS_1,H=dS_1,K=0

For scan i, dS is the weighted sum of dS_b over the three bands. Default is w_J=w_H=w_K=1.0, but perhaps weight K_s higher?

Because the scale of the PSP values is a function of detector array, the values are normalized to the PSP scales of the original northern data. The offsets between original north, new northern H-band, and southern, taken from John Gizis' memo on sensitivity scoring, are:

north (new H): PSP' = PSP + 1.8
south (J): PSP' = PSP - 0.4
south (H): PSP' = PSP + 1.2
south (K_s): PSP' = PSP + 0.2

Scan with maximum value of dS is selected for Catalog inclusion

Override max(dS) selection if:

Scan has untracked seeing length > 0, or
Scan has large astrometric reconstruction residuals, or
Scan has entry in coadd anomaly field

Revert to scan with next largest value of dS

If all scans have override condition, take scan with minimum total untracked seeing length

Note - these overrides are small in number and can be checked by hand and judged individually. See below.

V. Example - The Polar1 Catalog Generation Test Field

List of Tiles in +84deg dec band, with Q_best, number of times scanned, number of scans with Q_best and list of all date, hemisphere, scans with Q=Q_best

18 Tiles scan >1 scan with same Q_best

Net Relative Sensitivity selections for 18 multiply-observed NCP Tiles

Scans selected for inclusion are indicated with "X" in "select" column
There are no untracked seeing or coadd anomaly overrides
Astrometric override information is not currently available
Two selected scans have coadd background noise flags set, but that is not proposed as an override for the selection

Overlap Summary (L. Cambresy)

There are no gaps in the Polar1 scan selection.
35 scans have overlaps smaller than 50''
16 scans have overlaps smaller than 30''
2 scans have overlaps smaller than 10''
(overlap values are for the southern edge of the scan)

the 3 smallest overlaps are:

29733   19''  29734
29698    5''  29699
29581    3''  29582

None of the tile pairs with small overlaps have alternative scans with the same quality, so the sensitivity-based selection is not iterated.

VI. Net Relative Sensitivity Ranking Overrides

Primary ranking and selection of scans with multiple observation at the same best quality is done according to the net relative sensitivity (NRS). Possibly override NRS ranking for scans with untracked seeing, anomalies on Atlas Images (bugs, residual meteor trails, etc) that can trigger false sources, or large astrometric reconstruction problems. How often would the NRS be overridden?

The following three tables list the tiles for which the NRS ranking would be overridden for the three category of problems.

Untracked Seeing - 2 scans
Image Anomalies - 7 scans
Astrometric Reconstruction Anomalies - 2 scans

The format of each file is as follows:

Atile - Adjusted tile number (north: Atile = tile; south Atile = tile - 200000)
Qb - Best quality score achieved for this tile
No - Number of observations of this tile at any quality in latest SIT file
Nb - Number of scans of this tile with quality = Qb
date - UT observation date of scan with highest NRS
h - hemisphere of scan with highest NRS
scn - Scan number of observation with highest NRS
NRS - Net Relative Sensitivity for this scan (larger is better). Recall this is the sum over three bands, so the mean relative sensitivity per band is 1/3 of this number.
OVR - Override string. 3-digit code showing if override to NRS is possible. First digit corresponds to untracked seeing override, second digit to image anomaly override, and third digit corresponds to astrometric reconstruction override. "0" means no override. "1" means there is a problem the NRS ranking for this scan should be ignored.
date, h, scn, NRS, OVR... - Parameters for all other scans of this tile having same Qb, presented in order of NRS ranking.

Untracked Seeing

Untracked seeing was handled primarily in nightly quality assurance, and most scans with it were downgraded. There are only two instances of where untracked seeing might override the NRS value. In the first case, tile 13074, the alternative scan has and effective sensitivity an average of 0.2 mags worse in each band, and has a potential image anomaly. In the second case, the alternative has a mean effective sensitivity 0.17 mags worse in each band. It is recommended that neither of these overrides be accepted.

Image Anomalies

All seven cases of possible NRS overrides due to image anomalies are caused by satellite or residual meteor trails left in the scans. In five of these, there is only a very short or no discernable residual trail on the Atlas Image. In two cases, tiles 101123 and 110095, there is a relatively bright trail on the Atlas Image. In both of these latter cases, the mean relative sensitivities per band of the alternative images are only ~2% lower than the highest NRS ranked scans.

It is recommended that the overrides for only the two tiles with the brightest residual image artifacts be allowed. The remaining five tiles should use the scans with the highest NRS ranking.

Astrometric Residuals

Of the 33 scans on Howard's list of problem scans, 21 are the only scans of tiles, so must be included in the release. 10 are scans of tiles for which there are higher quality scans available, so are not factors in overrides. Four scans have the same best quality as one other scan in their respective tiles, so are subject to driving overrides. Two of these are the lower NRS-ranked scans, so do not change the ranking. There are only two tiles with Nb>1 and where scans with astrometric reconstruction anomalies were the top-NRS-ranked scans. The alternative scan for first case, tile 101358, has only a small mean sensitivity deficit (~5%), and it is recommended that the alternative scan be used.

For the second case of astrometric error override, the alternative scan also has a large astrometric reconstruction error. Howard McCallon states that these scans are "equally bad," so there is no benefit for taking the less sensitive scan.

VII. Sky Coverage Iteration

The final adjustements to the scan selection consider when there are more than one scan of a Tile (or gap-fill Tile) available with the same score, and selection of one can improve the sky-coverage of the all-sky release. This is discussed in detail in the link above.

The end-result of the sky-coverage iteration is to settle on a list of 59,731 scans to be included in the final all-sky release. The list of scans comprising the all-sky release is given here.

Scan List for Final All-Sky Release

The columns in this table are as follows:

Atile - Adjusted tile number (north: Atile = tile; south Atile = tile - 2000 00)
Qb - Best quality score achieved for this tile
No - Number of observations of this tile at any quality in latest SIT file
Nb - Number of scans of this tile with quality = Qb
date - UT observation date of scan with highest NRS
h - hemisphere of scan with highest NRS
scn - Scan number of observation with highest NRS

Last Updated: 29 July 2002
R. Cutri - IPAC