I. Scope of Data Release
1. Tile Superset (all data processed and quality reviewed as of 8 June 1999)
North: 970607n-981214n and 990202n-990521n
South: 980319n-990220s
2. Location on Sky North and South
II. Minimum Tile Quality Characteristics
1. Photometric QF >=0.5
2. Sensitivity QF >=0.5
3. Net Quality Score >=5 (effectively supercedes 1 and 2)
4. Average Seeing Limit sh < 1.25 (all bands)
5. Image Elongation Limits
Old Moment Ratio Algorithm 0.8<=moment ratio<=1.17 (all bands)
North: <=980202n
South: <=980417s
New Aspect Ratio Algorithm I2min/I2maj > 0.81 (all bands)
North: >=980203n
South: >=980418s
6. Untracked Seeing Limits j_score<=0.9 AND h_score<=1.3 AND k_score<=1.1
7. Background-Fit Residuals 5-sigma H-Background Structure Score > 3 (log n(Ks<14) < 3.4)
8. Tile Continguity Requirement Ncont >= 3
Preliminary Accounting of Tiles Available for Fall/Winter 99 Incremental Data Release
| Requirements | No. Scans - North | No. Scans - South | No. Scans - Total |
| Total In OPS DB | 16,293 | 13,690 | 29,983 |
| >970607 UT | 16,179 | 13,690 | 29,869 |
| Quality > 5 | 15,498 | 13,602 | 29,100 |
| Average Seeing Shape < 1.25 | 15,402 | 13,587 | 28,989 |
| 0.80 < Mean Moment Ratio < 1.10 (all bands) | (3918) | (939) | (4857) |
| I2min/I2max > 0.81 (all bands) | (10,890) | (12,648) | (23,538) |
| Total Image Elongation | 14,808 | 13,587 | 28,395 |
| J_seetrack < 0.9 AND H_seetrack < 1.3 AND Ks_seetrack < 1.1 | 14,123 | 13,368 | 27,491 |
| Background Fit Residuals < 1.20 DN (H-band only, all coadds, log n(Ks<14) < 3.1)) | 12,168 | 13,059 | 25,227 |
| Remove Duplicate Tiles | TBD | TBD | TBD |
| >3 Contiguous Tiles | TBD | TBD | TBD |
New Accounting of Tiles Available for Fall/Winter 99 Incremental Data Release - Using Untracked Seeing Scans, and New BG-Residual Scoring Thresholds
| Requirements | No. Scans - North | No. Scans - South | No. Scans - Total |
| Total In OPS DB | 16,288 | 13,690 | 29,978 |
| >970607 UT | 16,174 | 13,690 | 29,864 |
| Quality > 5 | 15,493 | 13,602 | 29,095 |
| Average Seeing Shape < 1.25 | 15,397 | 13,587 | 28,984 |
| 0.80 < Mean Moment Ratio < 1.10 (all bands) | (3915) | (939) | (4854) |
| I2min/I2max > 0.81 (all bands) | (10,888) | (12,648) | (23,536) |
| Total Image Elongation | 14,803 | 13,587 | 28,393 |
| J_seetrack < 0.9 AND H_seetrack < 1.3 AND Ks_seetrack < 1.1 | - | - | - |
| 5-sigma H-Background Structure Score > 3 (log n(Ks<14) < 3.4) | 14,736 | 13,499 | 28,235 |
| Remove Polar Dec Bands | 14,625 | 13,495 | 28,120 |
| Remove Duplicate Tiles | 14,595 | 13,410 | 28,007 |
| Remove Misc. Test Tiles | 14,586 | 13,410 | 27,996 |
| >3 Contiguous Tiles | - | - | 27,537 |
| Remove Problem Tiles | - | - | 27,493 |
Fall 99 Release Sky Coverage Map (galactic coordinate aitoff projection)
Fall 99 Release Sky Coverage Map (equatorial aitoff projection)
Candidate List of Date/Scan/Hemisphere/Tiles in Fall 99 Release
III. Duplicate Tile Observations
There are a number of tiles that are candidates for inclusion in the next release that have more than one observation available in the database. Of the 28,122 tiles that satisfy the basic quality requirements for the release, 107 were observed more than once. Most were observed twice, but there are a pair of southern tiles (315824 and 315825) that had 6 and 5 acceptable quality observations, respectively. These two tiles are in the core of Abell 3558 and were intentionally observed repetitively during the first week of southern operations. A number of tiles were also observed from both hemispheres, 000024-000031/200024-200031, 000900-000907/200900-200907 and 101123-101129/301123-301129.
All of the north/south observed tiles are global validation tiles. One of the northern multiply observed tiles (6480) is also global validation fields.
The following tables provide a comparison of a number of relevant parameters from each observation of the multiply observed tiles that could be used to select one. Selection could be based on scan quality (although most are 10's), nightly zero points (best transmission), seeing, image shape quality (not too relevant for the south), noise, etc...
See Section VI. for the Duplicate Tile Selection Rules
IV. Issues
- Distribution of number of coadds/scan that exceed H-band background
residual thresholds
(Click Here)
indicates that some scans may be excluded for reasons other than
airglow. Do we need an additional check to avoid removing these tiles?
H-BG residual threshold being analyzed to account for confusion noise and effects of very bright stars and galaxies - Do we need to have a selection threshold based on the horizontal
image banding? How will we deal with scans that were processed before
the banding diagnostic was in place? Is this a version 3.0 concern?
Ignore J-banding for next data release
V. Untracked Seeing Proposal
Proposal for dealing with scans containing untracked seeing:
Rae, Roc, and I have discussed the options for excising
data affected by untracked seeing. The seetrack score
has been attached to extended source records since
Fall, which means that it is a straightforward exercise
to eliminate extended sources from the catalog on the
basis of this score, provided the data was processed
since the Fall. For earlier data the options remain
under investigation. Seetracker scores may be recovered
from the archive and applied to the corresponding extended
source records or scans identified as having untracked seeing
may be rerun through GALWORKS in order to attach the
appropriate scores to the records. The current plan
is to apply the seetracker cut to the XSC as the last step
in extended source catalog generation providing the time
to evaluate these alternatives.
If you find any of these actions proposed below
unacceptable from a scientific perspective please
let me know the reasons.
================================================================
Do not exclude entire scans from the release database in which
there are only a few instances untracked seeing
Instead...
1) Excise all extended sources tagged with seetrack scores
in excess of the release threshold.
Untracked seeing can significantly affect extended source
reliability. The induced spatial incompleteness of 0.2%
of the entire sky is tiny (note that we are recovering
almost 5% of the sky as a result of this action).
2) Maintain a database of rejected areas.
These boundaries will correspond to the declination of the
first and last sources rejected from a period of untracked
seeing, and the right ascension width of the tile. If
a single source is rejected the boundary will be
one arcminute wide in declination about the source.
Those requiring detailed spatial completeness maps
will have that information available.
3) Include all point sources in the offending region. (i.e. take
no action at all for point sources).
Flagging/identifying affected point sources is costly.
The PSF mismatch will produce larger point source uncertainties
(and possibly biases) for 0.2% of the sky. Individually these
data may (or may not) lie outside the bounds of the level 1
specifications, but the specifications apply to the RMS of the
entire survey and this component is tiny. Since the PSF
used for source extraction does not match the source,
the PROPHOT extractions should naturally
have inflated uncertainties (this will be checked).
4) Include the coadd(s) containing the offending region
in the Image Atlas database. No flagging or other action necessary.
There is no reason to remove the image itself. Those who were
unhappy that their extended source did not appear in the database
will be relieved to find it is still there in the image.
5) Make no modification to the "CoordSearch" tool.
Occasionally the CoordSearch utility will say that data
exists at a given coordinate however extended sources will
have been deleted from that region. This will occur for
about 0.2% of the sky area. Frustrations from the
other 99.8% of the sky are likely to overwhelm this minor
problem.
VI. Duplicate Scan Selection
R. Cutri - 6 July 1999
1. I have devised a simple scoring routine to select among multiple
scans of the same tile for inclusion in the release.
a. First, if one scan has a higher net quality score than
other possible scans, it is selected.
b. In the case of identical quality score (as is almost always
the case), for each of a number of parameters that affect the
sensitivity of a scan, assign the scan with the "best" value
of each paramater one point. These parameter include:
i. [jhk] zero points - Best is the largest value (most positive)
ii. [jhk] seeing shape - Best is the smallest value (best seeing)
iii [jhk] moment ratio - Best is value closest to unity
(most symmetric - issue is different
algorithms)
iv. [jhk] backgrounds - Best is the lowest value
v. h-band background residual - Best is lowest value
vi. [jhk] point source noise - Best is lowest value
In the case of equivalent parameter values in all scans of a tile,
no point is awarded. As an example, here are the values for
the two scans of tile 26006 from Mt. Hopkins:
|tile | date |h|scn| ra | dec |q | jzp | hzp | kzp | j_sh | h_sh | k_sh |j_2m |h_2m |k_2m | jbg | hbg | kbg | jres| hres| kes |jnoise |hnoise | knoise|
026006 990228 n 76 174.009034 62.928244 10 0.0344 0.0406 0.0439 0.967 0.975 1.004 0.979 0.994 0.916 80.7 425.1 357.4 0.52 0.97 0.95 -1.4425 -1.2925 -1.1875
026006 990301 n 70 174.008272 62.927157 10 0.0537 0.0357 0.0410 0.970 0.982 1.024 0.923 0.944 0.966 76.8 325.8 392.2 0.52 0.89 0.99 -1.4575 -1.3075 -1.1425
The scan taken on 990227n is given "points" for having larger
h and k zero points, better, j, h and k seeing, more symmetric
j and h images, lower k background and lower k point source
noise, for a total of 9. The scan taken on 990301n is
given points for larger j zero point, more symmetric k images,
lower j and h backgrounds, lower h-band residual, and lower
j and h point source noise, for a total of 7. The first
of the two scans would be selected.
c. In the case where blocks of scans were repeated on different
nights, give weight to including the blocks from the same
night. This happens for the scans observed from both
observatories, as well as for blocks taken just from CTIO.
Usually, the data from one night is clearly better, but
when the data are similar, the scores can toggle between
scans from the different nights. In this case, see which
night has the most "winners" and select all the scans from
that night - except if any individual scans are clearly
worse.
This simple scoring scheme probably overweights some parameters and
underweights others. For example, the photometric sensitivity (in magnitudes)
is proportional to the atmospheric transparency (zero points). However, the
sensitivity is proportional to 0.2*PSP, where PSP is the photometric
sensitivity parameter (PSP = sh*BG^+0.29). In addition, the point source
noise implicitly contains contributions of other parameters such as
backgrounds and seeing.
In terms of a relative scoring scale, though, it does not seem unwarranted to
assign equal weights to all of the parameters we know can impact point and
extended source photometry.
VII. Revised Selection Criterion for Residual Airglow Background Structure
GALWORKS attempts to fit and subtract the low frequency sky background structure before performing photometry on extended sources in the coadded images. Severe OH airglow emission will cause structure in the background of higher frequencies than GALWORKS can adequately fit, leaving a residual background noise that quickly causes the extended source photometry to fail sensitivity specifications outlined in the Level 1 Science Requirements.
For the Spring 1999 release, we utilized the post-fit RMS residual in the H-band sky background as a proxy for detecting background structure that would contaminate the extended source photometry. A scan was rejected from the release if the H-band BG residual exceeded 1.2 DN, and the scan's density was less than 103.1 deg-2 (Ks < 14 mags). This threshold was derived from an early analysis.
Tom Jarrett has carried out a new analysis of the noise characteristics of 2MASS Atlas Images (coadds), using empirical data from several hundred thousand images as well as a theoretical description of the expected background noise. Tom's analysis yields a much better expression of the expected background noise, including confusion noise, in the Images as a function of source density and absolute background level. In the course of the analysis, it also became clear that the noise character of the cameras changed slightly when the GATIR electronics were replaced by the Leach electronics, and also following one of the northern camera thermal cycles, so the scoring function is also a function of calendar date. This noise model(s) provides a method of measuring how much the noise in the BG-fit subtracted images exceeds the predicted noise, and thus how much residual structure due to airglow remains. A "background structure" score is computed for every scan that is the number of coadds per scan that have residual noise levels that exceed 3 and 5 times the predicted noise in each band.
High BG residual "scores" can also result from the very bright stars that corrupt BG measurements, real astrophysical structure on the sky, and very high density star fields. Bright stars and large, diffuse structures such as large galaxies are easy to distinguish from airglow because they produce large BG-residual scores in all three bands, while airglow affects primarily the H-band. Bright stars generally affect only 1 or 2 coadds per scan.
For the next 2MASS data release, we have identified all scans that have a density of 103.4 deg-2 (Ks < 14) and have more than three H-band coadds with noise levels in excess of the 5-sigma predicted noise levels. Any of these coadds that also showed BG-residual scores >1 in J or Ks was visually examined for the presence of real structure or bright galaxies and stars. This visual examination identified M31 and M82, and a number of bright stars as producing the high scores; these scans are of course retained in the release.
The revised BG-residual scoring scheme identifies only 115 scans, north
and south, that will be rejected from the next 2MASS data release.
Compare this with the nearly 2200 tiles that would have been rejected
using the old BG-residual thresholding.
The 115 scans being rejected for the next data release are the most egregious
airglow offenders and should be scheduled for reobservation soon.
The new BG-residual scoring system will be implemented in the
nightly QA review soon.
VIII. Additional Tiles Removed From Release
A number of additional tiles have been removed from the candidate list for the next 2MASS data release for a number of reasons. These are summarized in the attached file.
R. Cutri - IPAC
Last Update - 16 February 2000 12:30