Protocamera Data Processing Status

To: 2MASS Science Team					6 June 1995
From: Roc Cutri
RE: Protocamera Data Processing Status

1. Observing Run Executive Summary

As most of you are probably aware, we were blessed with 9 nights of
photometric weather during the Protocamera run at the KPNO 50" telescope.
During those nights, over 480 sq.deg. of sky were scanned, divided 
roughly equally between J, H and Ks measurements to produce 
~40 Gbytes of data.  Approximately 340 sq.deg. were covered in long,
6 degree, survey-type scans, and 146 sq.deg. were observed during
calibration measurements.  A summary of the targets observed is 
provided in the 2MASS Project Information Web page at UMASS 
(http://pegasus.astro.umass.edu/2MASS/).  As Mike Skrutskie has 
already reported, virtually all of the highest priority 
observations were accomplished successfully, as well as most of the second 
and third priority measurements.  Observations were also carried out on 
an additional 9 non-photometric nights, and another ~30 Gb of data were 
collected in various regions, including the galactic center, a number 
of star formation regions, bright galaxies and clusters and other 
miscellaneous fields.


2. Preliminary Processing and Pipeline Preparation

Preliminary processing of selected data has been carried out over the
past several weeks to tune and prepare the operational (ops) pipeline.
Data from the nights of 4-22 and 4-25 were used for the preparation.
As a result of the analysis of earlier Protocamera data and 
recommendations by the Science Team, several refinements have been 
incorporated into the operational processor to improve photometric
accuracy and processing efficiency.  These include:

   a. Evaluation of a seeing estimation parameter (PFRAC) that measures 
      the fraction of encircled energy within fixed diameters.

   b. Use of a seeing-parameterized grid of psf's, rather than a single psf, 
      as was the case for the 6/94 data processing, or scan-by-scan psf
      generation.  Currently, the grid contains 3 psf's per band.

   c. Hybrid aperture/KAMPHOT photometry is derived for intermediate
      brightness point sources.

   d. Inclusion of photometry using a series of fixed circular apertures
      for extended sources.

  
During preliminary pipeline testing, a number of unusual features were
identified in the data and in the pipeline.  These are summarized below, 
and also given are the suspected origin of these abnormalities and the
preliminary solutions that have been incorporated into the operational
pipeline.  

Known Abnormalities in Data and Processing
------------------------------------------

a. Systematic offsets between in-scan positions of R1 and R2-R1 sources

   i. Suspected cause - Telescope/mechanical. Secondary has not yet
       settled before R1.
   ii. Ops solution - Measure offsets for large numbers of objects and 
       apply correction to source positions in frame coordinates

b. POSREC start-up errors in very high source density and/or high extinction
   regions.  This affects less than 10% of the current scans.
   
   i. Suspected causes:
      - In several cases initial telescope position reported incorrectly
      - POSREC jumpstart uses optical catalog (GSC) - bears little
        resemblance to near IR star fields in these regions.
   ii. Ops solution: These scans will not be position reconstructed at this 
       time, although photometry is extracted.  If IR-bright sources
       with reasonable astrometry can be identified in these scans, 
       POSREC will be re-initialized and rerun.  Longer term solution will 
       be to incorporate more rigorous position search and matching 
       techniques developed by H. McCallon for position reconstruction of 
       IRTS data.  Test application of these methods have successfully 
       reconstructed data in one high density field.

c. Systematic offset between aperture photometry and KAMPHOT psf-fit
   photometry in periods of poor seeing.

   i. Suspected cause: Seeing disk overfilling aperture causing 
      underestimation of point source brightness in aperture photometry.
      (We had a wealth of nights with moderate to poor seeing!).
   ii.Ops solution: Revert to KAMPHOT photometric measurement when
      seeing parameter drops below a fixed value (PFRAC < 0.4).

d. J-band ghost still present.
 
   i. Suspected cause: Optics.
   ii. Ops solution: Incorporate ghost parameters in persistence cleaning.


3. Pipeline Processing

After a few minor start-up delays, operational pipeline processing of the
April/May 2MASS Protocamera data was started on June 3rd.  
The status of processing for sets of scans from each night can be followed 
by referring to the 2MASS Science Team Information WWW page 
(/data/2MASS/docs/2mass.html). Select the Protocamera 
Processing Status line; this is a restricted access page so please refer 
to Mike Skrutskie's memo of May 24, 1995 for password information.

Each night has a separate entry on the status page.  Clicking on a night
will bring up a table containing all scans taken on that night,
and the status of each processing phase for that night:

TAPE INPUT: Raw data successfully extracted from tape

POSREC: Position reconstruction successful

MERGE: R1, and R2-R1 sources consolidated, aperture and KAMPHOT photometry
consolidated and persistence, ghosts and other artifacts removed. 

GAL: Extended source processing complete

DATA POSTED: Processed source lists and coadded images available
on line

PATH: Location of processed data 


All Science Team members are encouraged to review the processed data files
and provide feedback.  Data will be held on "lugosi", the primary
production computer, for approximately 1 week, and then will be
spooled to the archive server.  This will be reflected in the PATH 
entry in the status table. 

For each night, the data are arranged in separate sub-directories for
each scan (i.e. /l5/prod/95-04-22/s016 for scan 16 from the night
of April 22), and within each scan directory are sub-directories 
containing image, point and extended sources, along with ancillary
information that is retained for analysis and development purposes.
Of particular interest will be the following files (examples are
given for scan 16 on 95-04-22):

.../snnn/image/*.Z  - Compressed coadded FITS image files (512x1024) rebinned
                      to 1"/pixel scale. 
                      (i.e. /l5/prod/95-04-22/s016/image/ki0300009.fits.Z)

.../snnn/src/*.p.ph - Position reconstructed point source aperture photometry.
                      (i.e. /l5/prod/95-04-22/s016/src/k016.p.ph)

.../snnn/src/*.p.ps - Position reconstructed point source KAMPHOT photometry.
                      (i.e. /l5/prod/95-04-22/s016/src/k016.p.ps)

.../snnn/src/*.p.ps_s - Position reconstructed point source merged and cleaned
                      photometry.
                      (i.e. /l5/prod/95-04-22/s016/src/k016.p.ps_s)

.../snnn/extended/*.gal - Position reconstructed extended source photometry.
                      Includes selection of fixed apertures.
                      (i.e. /l5/prod/95-04-22/s016/extended/k016.gals)

Source data files have the general form bnnn... where "b" is the band (jhk),
and "nnn" is the scan.  If a scan failed any phase of the processing, such
as POSREC, some of these files may be missing.  A summary of "bad" scans is
provided at the end of each night's status file.  Users are strongly
recommended to review that file since sometimes only a few of the
scans within a set may have had problems.  Such sets are noted with
an asterisk.


4. Calibration

Currently, the photometric data in the ../src subdirectories have only
been corrected for approximate photometric zero points.  These
zero-points are likely accurate to about 10%.  More accurate
zero-point and extinction corrections will be evaluated for each night
and posted at a later time.


5. Processing Priority

As outlined in the Protocamera Run Processing Plan, data from the
Photometric nights will be processed first.  Data from non-photometric
nights will be processed following the completion of the photometric
data.  On each night, calibration scans are processed first, followed by
survey and test scans.