GOODS HST ACS Documentation

TABLE OF CONTENTS
 
 1.0 Generals

 2.0 What is being released
     2.1 Description of the files
     2.2 File nomenclature
     2.3 The Science Images
     2.4 The Weight Map Images
     2.5 File Size and Data Set Size

3.0 Data Reduction and Calibration
    3.1 Superbias
    3.2 SUperdarks
    3.3 Flat fields

4.0 Astrometry
    4.1 Geometrical Distortion Coefficients
    4.2 Velocity Aberration Correction
    4.3 The GOODS Astrometrical Solution
      4.3.1 Fundamental solution for the F850LP image
      4.3.2 Derived astrometric solution for F435W, F606W, and F775W
      
5.0 Drizzling
    5.1 The Multidrizzle Algorithm
    5.2 Cosmic Ray Rejection

6.0 Known Issues and Problems

    6.1 Missing Data
    6.2 Increased background in some of the Epoch 1 B-band images
    6.3 Astrometry and Rejection of Cosmic Ray and of Other Blemishes
    6.4 Photometry 

7.0 Updates and Future Releases

    7.1 Updates to version v1.0
    7.2 Version v2.0



1.0 Generals

On Friday, August 29 2003, at 5pm EST, the GOODS Teams has released the
version v1.0 of the reduced, calibrated, stacked and mosaiced images acquired
with HST and ACS as part of the GOODS ACS Treasury program.

Version v1.0 is a significant improvement upon the previous v0.5 release of
GOODS reduced HST/ACS images. Not only the released images are comprised of the
mosaiced stack of all the data acquired during the five epochs of GOODS
observation. They have also been re-processed using the best calibration and
reference files available to the Team at this time. Improved geometrical
distortion coefficients have been used together with corrections for the
velocity aberration distortion, and improved rejection of cosmic rays and 
other blemishes has been performed.


2.0 What is being released

The version v1.0 release consists of the full, multi-epoch stacked mosaics of
the GOODS ACS data in each passband and in both fields of the survey. 

No source catalogs or other higher-level data products are being released at
this time.

     2.1 Description of the files

The data set is organized as a series of images in each of the four GOODS
passbands (F435W, F606W, F775W, and F850LP) and in each of the GOODS fields
(HDF-N and CDF-S). We refer to these images as "sections".

Each field has been divided into sections for the following reasons. To
optimize PSF sampling, the original, pipeline-processed 4Kx4K ACS images, which
have a scale of 0.05 arcsec per pixel, have been drizzled onto a very large
virtual grid with a scale of 0.03 arcsec/pixel. Such grid would be 
40,000 x 40,000 pixels in size in the HDF-N, and 32,000 x 40,000 pixels in
the CDF-S. It would be too large to exist as an individual file (it would
exceed the 2 GB limit for file size of 32-bit computers). It would also be
highly impractical to work with and to manage, in particular during transfer
operations over the Internet. The grid, therefore has been divided into
sections.  

Each section is an image 8,192 x 8,192 pixels in size. A total of 17
sections cover the HDF-N, and a total of 18 sections cover the CDF-S. 

Sections are contiguous, namely if all adjacent sections were stitched together
with no gaps and in the proper order, with the last line or row of a given 
section abutting the first line or row of an adjacent section, then it would
be possible to reconstruct the original grid (if such a large file could be
created).

Sections are labeled according to their position in the field (see Figures
displayed in the files h_cdfs_v1.0sects_plt.jpg and h_hdfn_v1.0sects_plt.jpg)
by a two-digit number. The first digit of this number represent the "X"
position of the section in the field, the second digit the "Y" position, with
X and Y increasing towards to the right and upward, respectively. Thus, the
section located in the lower left corner is called "section11". The one in the
upper right corner is called "section45" in the CDF-S, and "section55" in the
HDF-N.

To clarify the relationship between the version v0.5 GOODS data release and 
the current v1.0 release, there is no one-to-one correspondence between v1.0 
sections and v0.5 tiles. A section covers about 44% more area than a tile, and 
it contains a number of different tiles. Sections include a varying number of 
tiles, and one tile can contribute to a number of contiguous sections.  For 
more information on v0.5 tile structure see: 
ftp://archive.stsci.edu/pub/hlsp/goods/v05/h_goods_v05_rdm.txt.

     2.2 File nomenclature

The name of the files relative to the sections contain information the field
(whether the CDF-S or the HDF-N), the passband, the data release version (v1.0
in this case), the section, and the image type (whether the science image or
whether a weight map). For example, the FITS file containing the F850LP science
data relative to section32 in the CDF-S field is called 

           h_sz_sect11_v1.0_drz_img.fits,

where the prefix "h" and subfix "img" are adopted to satisfies the general 
nomenclature rules of the STScI data archive, "sz" specifies that the section
is relative 
to the GOODS southern field (s) and is an F850LP image (z, or z band),
"sect11" specifies the location of the section in the field, "v1.0" specifies
the data release, and "drz" specifies that the file contains an image (drz
stands for drizzled). The file containing the corresponding weight map is
called 

           h_sz_sect11_v1.0_wht_img.fits,

where "wht" specifies its nature of weight map image.

     2.3 The Science Images

The pixel values of the science images report the flux count rate calibrated
in electron/second. 

The zero points to convert the count rate into an AB magnitude for the four
GOODS passbands are the following:

    Z0_F435W  = 25.65288
    Z0_F606W  = 26.49341
    Z0_F775W  = 25.64053
    Z0_F850LP = 24.84315

For the details of the photometric calibration of the ACS camera we refer to
the Instrument Handbook, which is available on line at the URL

http://www.stsci.edu/hst/acs/documents/handbooks/cycle12/cover.html

Also useful is the Instrument Data Handbook, also available on line at 

http://www.stsci.edu/hst/acs/documents/handbooks/DataHandbookv1/


     2.4 The Weight Map Images

The weight maps are images which are produced as part of the data reduction
process, and which give a measure of the background + instrumental nominal
noise per unit area (pixel) in the science data.  Because the GOODS images are
mosaics consisting of different numbers of overlapping images at each place on
the sky, the total exposure time varies as a function of position.  In
addition, pixels have been masked for a variety of reasons, including
rejection of cosmic rays, masking of satellite trails and reflection ghosts,
etc.  Finally, the sky background varied somewhat from exposure to exposure.
All of these effects contribute to variable "depth" across the image mosaics.

During the data reduction, a noise model was used to calculate the expected
noise per pixel at the background level from the combination of sky background
(modulated by the flat field), readout noise, and dark current.  The effects
of Poisson shot noise due to signal from objects in the image were not
included in this noise model.  This noise model was then used to construct
weight maps equal to the expected inverse variance per pixel.  These weight
maps were combined with masks that excluded (i.e., set to zero weight) pixels
for various reasons outlined above, and were then used to weight the
combination of images in the drizzling process.

The resulting output weight map should be equal to the expected inverse
variance (i.e., 1/RMS^2) per pixel.  The interpolations introduced by
drizzling the images (shifting, rotating, correcting distortion, and
subsampling pixels onto a finer grid) result in correlations between pixels in
the drizzled science images.  Therefore, the apparent RMS background noise
that one measures in the image is smaller than that given by the (inverse)
weight maps, because the apparent RMS is suppressed by the effects of these
correlations.  The weight maps are normalized to show the expected noise per
pixel that the images would have in the absence of these correlations. Or, put
another way, the sum of the variances (inverse weight values) over some
aperture larger than the correlation scale (a few pixels) should accurately
reflect the measurement uncertainty due to the background + instrument noise.
(We note again that no attempt is made to include Poisson uncertainty due to
signal from objects.)  For a more detailed discussion of weight map
conventions and noise correlation in drizzling, please see Casertano et
al. 2000, AJ, 120, 2747, especially Section 3.5 and Appendix A.

The scaling of the weight maps has been validated by comparing their values to
the measured image noise, after a correction for the measured autocorrelation
of background pixels.  This test indicates that the weight maps correctly
predict the image noise to an accuracy of about 5% (perhaps systematically
underestimating the noise by about that much, although this needs to be
checked more carefully with additional tests). Any updated information will be
posted on the GOODS web pages as necessary.


     2.3 File Size and Data Set Size

The files relative to each section, either the science or the weight map
images, are ~268.4 MB in size (8192 x 8192 pixels, each with a 32 depth), and 
there are images in each of the four GOODS passbands. Thus, the CDF-S data set
requires 2x268.4x18x4 = 38.655 GB, the HDF-N requires 2x268.4x17x4 = 36.507 GB. 
The full data set requires 38.655 + 36.507 = 75.162 GB. 


3.0 Data Reduction and Calibration

Raw data read from the ACS CCD array are processed by the ACS pipeline
(CALACS), which provides the basic reduction steps of dark and bias
subtractions and flat-fielding. It also provides data quality files that flag
known hot pixels, bad column and other cosmetic defects. These basically
reduced images consist of the individual exposures, or "dithers", taken in
each band for each ACS pointings (tiles).

For the v1.0 data release, all raw data have been recalibrated using the best
reference files available at this time. This has resulted in new .flt files
produced for use in the post-pipeline reduction (multidrizzling). We will
replace the previously reduced .flt files with the new one immediately after
the release of the v1.0 data.

    3.1 The Bias Reference Frame

The bias reference frame is the combination of seven frames acquired with
daily cadence. The bias pedestal is removed from the science frames using the
7-day bias frame closest in time to observations.

    3.2 The Dark Reference Frame

The dark reference frame is relative to a period of two weeks, and is a hybrid
composed of a base dark frame that accounts for the removal of the dark
pedestal pixel by pixel, and of a component of hot pixels added daily. The 
base dark frame is the combination of 56 frame, acquired with a cadence of 4
frames per day (thus, it include 2 weeks worth of dark frames). Unfortunately,
hot pixels in the ACS CCDs vary with time, as a result of radiation damage and
annealing operations. Therefore, the base dark frame is then "sprinkled" at a
daily cadence with hot pixel updates from the corresponding set of 4 dark
frames. The dark pedestal is removed from the  science frames using the
bi-weekly frame closest in time to observations with an up-to-date content of
hot pixels. 

    3.3 Flat fields

The flat field frames in each of the F435W, F606W, F775, and F850LP passbands
contain about 100,000 counts per pixel (p-flats). These frames also include
large-scale flat-fields (l-flats), obtained through dithered observations of
stellar fields. The flat field frames have been tested to be accurate to about
1%.

4.0 Astrometry

    4.1 Geometrical Distortion Coefficients

The geometric distortion applied to the individual input images uses the most
recent (July 2003) values of the IDCTAB distortion coefficients released by
the ACS group at STScI.  The geometric distortion of ACS was determined on the
basis of multiple observations of a star field in 47 Tucanae, including
observations at multiple roll angles to determine the presence of skew terms
in the solution.  Unlike for Version 0.5, the residual skew is found to be
very small.

Currently a single geometric solution is available, independent of wavelength;
wavelength-dependent terms are believed to be <~ 0.1 pixels over the full
extent of the detector.

    4.2 Velocity Aberration Correction

As of July 2003, the geometric solution for ACS includes a small correction
for the effect of differential velocity aberration.  Each image header
includes a parameter called VAFACTOR which represents the correction to the
pixel scale in that exposure due to the heliocentric velocity of HST projected
onto the line of sight of the telescope.  The VAFACTOR term is taken into
account when drizzling individual images onto the mosaic, and should require
no additional action.  As verification, we attempted an astrometric solution
in which the pixel scale was allowed to vary from image to image, and we found
no residual correlation (at the 1.e-5 level) between the derived scale and the
value of VAFACTOR.  We did find a small scale variation, which we interpret as
related to the limitation of the reference image; see Section 4.3.1 for more
details.

     4.3 The GOODS Astrometrical Solution

         4.3.1 Fundamental solution for the F850LP image
          
The astrometric solution obtained for the GOODS Version 1 images is based on
the F850LP data and was obtained through several steps. The preliminary phase
consisted of cross-identifying sources among ACS images, as follows.  First, a
reference ground-based image was selected for each hemisphere: for the North
it was a Subaru R-band image, and for the South a composite image based on
deep WFI data.  Second, the reference image was astrometrically matched to
stellar positions in the GSC2 catalog, allowing for a second order distortion.
Third, the F850LP ACS images were combined into "tiles", with one tile for
each epoch and each pointing; a tile consists of four individual exposures and
is rather clean, containing only a small number of residual cosmic rays and
other imperfections.  Fourth, each tile was registered to the reference
ground-based image in order to cross-identify sources; the match between
ground-based and ACS images had a typical noise of 0.1".

With the initial cross-identification complete, the global astrometric
solution was obtained by a multi-parameter chi-squared minimization for each
hemisphere.  Each tile was allowed three free parameters, namely the RA and
DEC of the reference point and the position angle.  In addition, the scale and
two skew terms were allowed to vary, but were forced to be the same for all
tiles in each hemisphere.  For each cross-identified source, the quantity to
be minimized included the difference between the ground based and the ACS
position, with nominal error 0.1", and the difference between each ACS
measurement and their average, with nominal error 0.005".  Therefore the
optimal solution matches ACS-to-ACS measurements with a very stiff penalty,
and matches ACS-to-ground positions with a softer penalty.  This approach
allowed us to take advantage of the relative stiffness of ACS to constrain the
overall distortion of the solution, while retaining the global scale and
orientation constraints from the ground-based image.  Allowing the scale to
vary from image to image would introduce a small position-dependent gradient
in the pixel scale, which we interpret as a residual effect of the
imperfection of the ground-based solution.  The final solution had a clipped
RMS deviation of 0.006-0.007" in each coordinate in the ACS-to-ACS difference,
and of 0.12" in each coordinate in the ACS-to-ground difference.

We are aware that slight improvements are possible in the procedure we used,
in particular in the quality of the reference ground-based image; such
improvements will be applied for Version 2.

Once the astrometric solution was obtained for each tile, the information was
back-propagated to the constituent images and each image was drizzled onto the
final grid as described in Section 5.

         4.3.2 Derived astrometric solution for F435W, F606W, and F775W
      
The images in the other bands were aligned to the F850LP solution by matching
source positions to the average position of the corresponding source in
F850LP, as determined from the fundamental astrometric solution.  The typical
clipped rms residual (F850LP average to other filters) was 0.007" to 0.010" in
each coordinate; we noticed a significant non-Gaussian tail especially in
F435W, presumably due to morphological differences between images in different
bands.

5.0 Drizzling

  This processing is done in two independent phases. In the first phase, images
of the same tile taken in the same filters are identified, sky-subtracted and
drizzled onto a common pixel grid with the same scale as the input images
(0.05 arcsec/pixel). Cosmic rays and deviant pixels are identified during this
process and flagged in mask files specifically created for this purpose. 
Information included in post-pipeline masks (which have also been drizzled onto
the same grid) is included in the new masks at this time. 

During the second phase, the images and the mask files are blotted back to the
original positions, drizzled again onto a common astrometric grid with scale
0.03 arcsec/pixel, and stacked together. During this process corrections for
the ACS geometrical distortion are applied, cosmic rays flagged during the
previous processing block are masked out from the stack, and additional,
low-level cosmic rays and defects are identified and masked, too.

    5.1 The Multidrizzle Algorithm

MultiDrizzle (Koekemoer et al. 2002, HST Calibration Workshop) is a single
Pyraf script which automatically performs the following steps on a list of
input flatfielded ACS images (flt.fits files):
  - bad pixel identification
  - sky subtraction
  - drizzle each input image onto separate, registered output images
  - create a median from these images
  - transform ("blot") the median back to the frame of each input image
  - create a derivative of the median image, and compare with the original
    image to obtain a cosmic ray mask
  - refine the shifts by cross-comparing the images
  - run drizzle a second time, to combine the images with the cosmic ray
    masks into a final, combined image.
This technique is sufficiently flexible to enable adjustment by means of
a range of parameters, but is also sufficiently automated that it is
capable of bein run in a pipeline such as that which is used by GOODS.
  
    5.2 Cosmic Ray Rejection

Initially a set of shallow, single-exposure-depth mosaics are created which
are then combined to create a clean "median" mosaic, using signal-to-noise
thresholds to reject cosmic rays and any remaining bad pixels. This technique
is extremely robust at producing a clean median mosaic, with the primary
requirement being accurate astrometric alignment between epochs. The relevant
pieces of this median mosaic are then transformed back to the frame of each
of the original input flt.fits files, using "wblot" which is essentially a
WCS-based version of "blot".

Next, the standard dither package tasks of "deriv" and "driz_cr" are used
to compare this blotted image and its derivative image with the original
input flt.fits file, and generate a cosmic ray mask based on the comparison.
Finally, all the flt.fits files, together with their newly created cosmic
ray masks, are drizzled using "wdrizzle" onto a single output mosaic, which
has units of countrate in electrons/second in each pixel. 


6.0 Known Issues and Problems

    6.1 Missing Data

A small number of individual exposures are missing from the final multi-epoch 
HDF-N mosaics. The areas where exposures are missing can be easily identified
by displaying and inspecting the weight maps. In one case, one tile (i.e. an
area with the footprint of the ACS WFC field of view) in the V (2 exposures),
i (2 exposures and z (4 exposures) bands was completely lost during epoch 5
due to a failed guide star acquisition. The GOODS Team plans to request 2
orbits to re-observe the missed tile and produce a new stacked mosaic. We will
post a note in the GOODS Web page and release an update to the data when this
will be done. 

In another instance a few individual exposures have not been included in the
final z-band stack in the HDF-N, because using them in the multidrizzle
pipeline simply makes it crash. It is not understood why this happens. The
images look just fine, and no anomalies are obviously observed. Very likely,
there is a discrepancy in some of the header parameters that has not been
identified. Once this problem is corrected, we will post a note in the
GOODS Web page and release an update to the data.

    6.2 Increased background in some of the Epoch-1 B-band images

Some B-band tiles acquired during epoch 1 in the HDF-N are affected by a
relatively large background scattered light due to an error in the scheduling
of the observations. This allowed the limb angle to become smaller than about
25 degree, which resulted in an increased Earth scattered light. Such tiles
have been re-observed at later epochs. The old and new B-band images have been
co-added in the mosaic with optimal weighting to maximize signal-to-noise
ratio. The affected tiles (and therefore, the affected sections or part of
sections) can be identified by inspection of the weight maps. Upon visual
inspection, this scattered light appears rather uniform. No special attention
or actions are required when working with these images.

    6.3 Astrometry and Rejection of Cosmic Ray and of Other Blemishes

Inspection of the weight maps sometimes reveals that residual mis-alignments
due to scatter in the astrometric solutions cause rejections of the brightest
pixels in bright, unresolved sources by the cosmic-ray rejection algorithm. 
Only a handful of bright sources are affected. The Team will refine the 
astrometric solution in the following weeks. We also plan to release a list 
of stars where we do find evidence of this problem. In no case the rejection 
problem has been observed in diffuse sources.

    6.4 Photometry 

One known problem is that the photometry of a very small number of bright
sources in the mosaics is systematically biased toward low fluxes because of
small astrometrical errors, as mentioned above.

Another known problem is that the photometry of bright stars in the
multi-epoch stacked mosaic is systematically off typically by ~1 percent and
in some case by up to ~2 percent, the sense being that the magnitude appears
fainter. No such effect is observed in single-epoch stacked images. We
discovered that this effect is due to the rejection of the "rotated"
diffraction spikes by the cosmic-ray rejection algorithm. This problem seems
to affect only bright (z<~20) unresolved sources, namely those for which the
energy fraction contained in the diffraction spikes is the highest. It effect
becomes progressively less noticeable at fainter flux, because fluctuations in
the background become comparable to the spikes. It also becomes less
significant if small photometric apertures are used (e.g. less than ~1% for
0.5 arcsec diameter apertures). This is a limitation, albeit a minor one, of
the "multi-epoch" approach to the observations.

    6.5 PSF Issues

During a small number of exposures fine guidance was lost, and the
observations were completed in gyro mode. Some of the images affected show a
small trailing effect (about 1 pixel). In some application (e.g. weak
lensing), this might be a concern. The GOODS Team will post a list of the
affected areas as soon as the list is complete.

7.0 Updates and Future Releases

    7.1 Updates to version v1.0

We will announce updates to the v1.0 release data set on the GOODS website
together with the instructions on how to download them.

    7.2 Version v2.0

A version v2.0 is planned for near the end of 2003.  Details of this release 
will be made available when they are finalized.