GOODS HST ACS Documentation TABLE OF CONTENTS 1.0 Generals 2.0 What is being released 2.1 Description of the files 2.2 File nomenclature 2.3 The Science Images 2.4 The Weight Map Images 2.5 File Size and Data Set Size 3.0 Data Reduction and Calibration 3.1 Superbias 3.2 SUperdarks 3.3 Flat fields 4.0 Astrometry 4.1 Geometrical Distortion Coefficients 4.2 Velocity Aberration Correction 4.3 The GOODS Astrometrical Solution 4.3.1 Fundamental solution for the F850LP image 4.3.2 Derived astrometric solution for F435W, F606W, and F775W 5.0 Drizzling 5.1 The Multidrizzle Algorithm 5.2 Cosmic Ray Rejection 6.0 Known Issues and Problems 6.1 Missing Data 6.2 Increased background in some of the Epoch 1 B-band images 6.3 Astrometry and Rejection of Cosmic Ray and of Other Blemishes 6.4 Photometry 7.0 Updates and Future Releases 7.1 Updates to version v1.0 7.2 Version v2.0 1.0 Generals On Friday, August 29 2003, at 5pm EST, the GOODS Teams has released the version v1.0 of the reduced, calibrated, stacked and mosaiced images acquired with HST and ACS as part of the GOODS ACS Treasury program. Version v1.0 is a significant improvement upon the previous v0.5 release of GOODS reduced HST/ACS images. Not only the released images are comprised of the mosaiced stack of all the data acquired during the five epochs of GOODS observation. They have also been re-processed using the best calibration and reference files available to the Team at this time. Improved geometrical distortion coefficients have been used together with corrections for the velocity aberration distortion, and improved rejection of cosmic rays and other blemishes has been performed. 2.0 What is being released The version v1.0 release consists of the full, multi-epoch stacked mosaics of the GOODS ACS data in each passband and in both fields of the survey. No source catalogs or other higher-level data products are being released at this time. 2.1 Description of the files The data set is organized as a series of images in each of the four GOODS passbands (F435W, F606W, F775W, and F850LP) and in each of the GOODS fields (HDF-N and CDF-S). We refer to these images as "sections". Each field has been divided into sections for the following reasons. To optimize PSF sampling, the original, pipeline-processed 4Kx4K ACS images, which have a scale of 0.05 arcsec per pixel, have been drizzled onto a very large virtual grid with a scale of 0.03 arcsec/pixel. Such grid would be 40,000 x 40,000 pixels in size in the HDF-N, and 32,000 x 40,000 pixels in the CDF-S. It would be too large to exist as an individual file (it would exceed the 2 GB limit for file size of 32-bit computers). It would also be highly impractical to work with and to manage, in particular during transfer operations over the Internet. The grid, therefore has been divided into sections. Each section is an image 8,192 x 8,192 pixels in size. A total of 17 sections cover the HDF-N, and a total of 18 sections cover the CDF-S. Sections are contiguous, namely if all adjacent sections were stitched together with no gaps and in the proper order, with the last line or row of a given section abutting the first line or row of an adjacent section, then it would be possible to reconstruct the original grid (if such a large file could be created). Sections are labeled according to their position in the field (see Figures displayed in the files h_cdfs_v1.0sects_plt.jpg and h_hdfn_v1.0sects_plt.jpg) by a two-digit number. The first digit of this number represent the "X" position of the section in the field, the second digit the "Y" position, with X and Y increasing towards to the right and upward, respectively. Thus, the section located in the lower left corner is called "section11". The one in the upper right corner is called "section45" in the CDF-S, and "section55" in the HDF-N. To clarify the relationship between the version v0.5 GOODS data release and the current v1.0 release, there is no one-to-one correspondence between v1.0 sections and v0.5 tiles. A section covers about 44% more area than a tile, and it contains a number of different tiles. Sections include a varying number of tiles, and one tile can contribute to a number of contiguous sections. For more information on v0.5 tile structure see: ftp://archive.stsci.edu/pub/hlsp/goods/v05/h_goods_v05_rdm.txt. 2.2 File nomenclature The name of the files relative to the sections contain information the field (whether the CDF-S or the HDF-N), the passband, the data release version (v1.0 in this case), the section, and the image type (whether the science image or whether a weight map). For example, the FITS file containing the F850LP science data relative to section32 in the CDF-S field is called h_sz_sect11_v1.0_drz_img.fits, where the prefix "h" and subfix "img" are adopted to satisfies the general nomenclature rules of the STScI data archive, "sz" specifies that the section is relative to the GOODS southern field (s) and is an F850LP image (z, or z band), "sect11" specifies the location of the section in the field, "v1.0" specifies the data release, and "drz" specifies that the file contains an image (drz stands for drizzled). The file containing the corresponding weight map is called h_sz_sect11_v1.0_wht_img.fits, where "wht" specifies its nature of weight map image. 2.3 The Science Images The pixel values of the science images report the flux count rate calibrated in electron/second. The zero points to convert the count rate into an AB magnitude for the four GOODS passbands are the following: Z0_F435W = 25.65288 Z0_F606W = 26.49341 Z0_F775W = 25.64053 Z0_F850LP = 24.84315 For the details of the photometric calibration of the ACS camera we refer to the Instrument Handbook, which is available on line at the URL http://www.stsci.edu/hst/acs/documents/handbooks/cycle12/cover.html Also useful is the Instrument Data Handbook, also available on line at http://www.stsci.edu/hst/acs/documents/handbooks/DataHandbookv1/ 2.4 The Weight Map Images The weight maps are images which are produced as part of the data reduction process, and which give a measure of the background + instrumental nominal noise per unit area (pixel) in the science data. Because the GOODS images are mosaics consisting of different numbers of overlapping images at each place on the sky, the total exposure time varies as a function of position. In addition, pixels have been masked for a variety of reasons, including rejection of cosmic rays, masking of satellite trails and reflection ghosts, etc. Finally, the sky background varied somewhat from exposure to exposure. All of these effects contribute to variable "depth" across the image mosaics. During the data reduction, a noise model was used to calculate the expected noise per pixel at the background level from the combination of sky background (modulated by the flat field), readout noise, and dark current. The effects of Poisson shot noise due to signal from objects in the image were not included in this noise model. This noise model was then used to construct weight maps equal to the expected inverse variance per pixel. These weight maps were combined with masks that excluded (i.e., set to zero weight) pixels for various reasons outlined above, and were then used to weight the combination of images in the drizzling process. The resulting output weight map should be equal to the expected inverse variance (i.e., 1/RMS^2) per pixel. The interpolations introduced by drizzling the images (shifting, rotating, correcting distortion, and subsampling pixels onto a finer grid) result in correlations between pixels in the drizzled science images. Therefore, the apparent RMS background noise that one measures in the image is smaller than that given by the (inverse) weight maps, because the apparent RMS is suppressed by the effects of these correlations. The weight maps are normalized to show the expected noise per pixel that the images would have in the absence of these correlations. Or, put another way, the sum of the variances (inverse weight values) over some aperture larger than the correlation scale (a few pixels) should accurately reflect the measurement uncertainty due to the background + instrument noise. (We note again that no attempt is made to include Poisson uncertainty due to signal from objects.) For a more detailed discussion of weight map conventions and noise correlation in drizzling, please see Casertano et al. 2000, AJ, 120, 2747, especially Section 3.5 and Appendix A. The scaling of the weight maps has been validated by comparing their values to the measured image noise, after a correction for the measured autocorrelation of background pixels. This test indicates that the weight maps correctly predict the image noise to an accuracy of about 5% (perhaps systematically underestimating the noise by about that much, although this needs to be checked more carefully with additional tests). Any updated information will be posted on the GOODS web pages as necessary. 2.3 File Size and Data Set Size The files relative to each section, either the science or the weight map images, are ~268.4 MB in size (8192 x 8192 pixels, each with a 32 depth), and there are images in each of the four GOODS passbands. Thus, the CDF-S data set requires 2x268.4x18x4 = 38.655 GB, the HDF-N requires 2x268.4x17x4 = 36.507 GB. The full data set requires 38.655 + 36.507 = 75.162 GB. 3.0 Data Reduction and Calibration Raw data read from the ACS CCD array are processed by the ACS pipeline (CALACS), which provides the basic reduction steps of dark and bias subtractions and flat-fielding. It also provides data quality files that flag known hot pixels, bad column and other cosmetic defects. These basically reduced images consist of the individual exposures, or "dithers", taken in each band for each ACS pointings (tiles). For the v1.0 data release, all raw data have been recalibrated using the best reference files available at this time. This has resulted in new .flt files produced for use in the post-pipeline reduction (multidrizzling). We will replace the previously reduced .flt files with the new one immediately after the release of the v1.0 data. 3.1 The Bias Reference Frame The bias reference frame is the combination of seven frames acquired with daily cadence. The bias pedestal is removed from the science frames using the 7-day bias frame closest in time to observations. 3.2 The Dark Reference Frame The dark reference frame is relative to a period of two weeks, and is a hybrid composed of a base dark frame that accounts for the removal of the dark pedestal pixel by pixel, and of a component of hot pixels added daily. The base dark frame is the combination of 56 frame, acquired with a cadence of 4 frames per day (thus, it include 2 weeks worth of dark frames). Unfortunately, hot pixels in the ACS CCDs vary with time, as a result of radiation damage and annealing operations. Therefore, the base dark frame is then "sprinkled" at a daily cadence with hot pixel updates from the corresponding set of 4 dark frames. The dark pedestal is removed from the science frames using the bi-weekly frame closest in time to observations with an up-to-date content of hot pixels. 3.3 Flat fields The flat field frames in each of the F435W, F606W, F775, and F850LP passbands contain about 100,000 counts per pixel (p-flats). These frames also include large-scale flat-fields (l-flats), obtained through dithered observations of stellar fields. The flat field frames have been tested to be accurate to about 1%. 4.0 Astrometry 4.1 Geometrical Distortion Coefficients The geometric distortion applied to the individual input images uses the most recent (July 2003) values of the IDCTAB distortion coefficients released by the ACS group at STScI. The geometric distortion of ACS was determined on the basis of multiple observations of a star field in 47 Tucanae, including observations at multiple roll angles to determine the presence of skew terms in the solution. Unlike for Version 0.5, the residual skew is found to be very small. Currently a single geometric solution is available, independent of wavelength; wavelength-dependent terms are believed to be <~ 0.1 pixels over the full extent of the detector. 4.2 Velocity Aberration Correction As of July 2003, the geometric solution for ACS includes a small correction for the effect of differential velocity aberration. Each image header includes a parameter called VAFACTOR which represents the correction to the pixel scale in that exposure due to the heliocentric velocity of HST projected onto the line of sight of the telescope. The VAFACTOR term is taken into account when drizzling individual images onto the mosaic, and should require no additional action. As verification, we attempted an astrometric solution in which the pixel scale was allowed to vary from image to image, and we found no residual correlation (at the 1.e-5 level) between the derived scale and the value of VAFACTOR. We did find a small scale variation, which we interpret as related to the limitation of the reference image; see Section 4.3.1 for more details. 4.3 The GOODS Astrometrical Solution 4.3.1 Fundamental solution for the F850LP image The astrometric solution obtained for the GOODS Version 1 images is based on the F850LP data and was obtained through several steps. The preliminary phase consisted of cross-identifying sources among ACS images, as follows. First, a reference ground-based image was selected for each hemisphere: for the North it was a Subaru R-band image, and for the South a composite image based on deep WFI data. Second, the reference image was astrometrically matched to stellar positions in the GSC2 catalog, allowing for a second order distortion. Third, the F850LP ACS images were combined into "tiles", with one tile for each epoch and each pointing; a tile consists of four individual exposures and is rather clean, containing only a small number of residual cosmic rays and other imperfections. Fourth, each tile was registered to the reference ground-based image in order to cross-identify sources; the match between ground-based and ACS images had a typical noise of 0.1". With the initial cross-identification complete, the global astrometric solution was obtained by a multi-parameter chi-squared minimization for each hemisphere. Each tile was allowed three free parameters, namely the RA and DEC of the reference point and the position angle. In addition, the scale and two skew terms were allowed to vary, but were forced to be the same for all tiles in each hemisphere. For each cross-identified source, the quantity to be minimized included the difference between the ground based and the ACS position, with nominal error 0.1", and the difference between each ACS measurement and their average, with nominal error 0.005". Therefore the optimal solution matches ACS-to-ACS measurements with a very stiff penalty, and matches ACS-to-ground positions with a softer penalty. This approach allowed us to take advantage of the relative stiffness of ACS to constrain the overall distortion of the solution, while retaining the global scale and orientation constraints from the ground-based image. Allowing the scale to vary from image to image would introduce a small position-dependent gradient in the pixel scale, which we interpret as a residual effect of the imperfection of the ground-based solution. The final solution had a clipped RMS deviation of 0.006-0.007" in each coordinate in the ACS-to-ACS difference, and of 0.12" in each coordinate in the ACS-to-ground difference. We are aware that slight improvements are possible in the procedure we used, in particular in the quality of the reference ground-based image; such improvements will be applied for Version 2. Once the astrometric solution was obtained for each tile, the information was back-propagated to the constituent images and each image was drizzled onto the final grid as described in Section 5. 4.3.2 Derived astrometric solution for F435W, F606W, and F775W The images in the other bands were aligned to the F850LP solution by matching source positions to the average position of the corresponding source in F850LP, as determined from the fundamental astrometric solution. The typical clipped rms residual (F850LP average to other filters) was 0.007" to 0.010" in each coordinate; we noticed a significant non-Gaussian tail especially in F435W, presumably due to morphological differences between images in different bands. 5.0 Drizzling This processing is done in two independent phases. In the first phase, images of the same tile taken in the same filters are identified, sky-subtracted and drizzled onto a common pixel grid with the same scale as the input images (0.05 arcsec/pixel). Cosmic rays and deviant pixels are identified during this process and flagged in mask files specifically created for this purpose. Information included in post-pipeline masks (which have also been drizzled onto the same grid) is included in the new masks at this time. During the second phase, the images and the mask files are blotted back to the original positions, drizzled again onto a common astrometric grid with scale 0.03 arcsec/pixel, and stacked together. During this process corrections for the ACS geometrical distortion are applied, cosmic rays flagged during the previous processing block are masked out from the stack, and additional, low-level cosmic rays and defects are identified and masked, too. 5.1 The Multidrizzle Algorithm MultiDrizzle (Koekemoer et al. 2002, HST Calibration Workshop) is a single Pyraf script which automatically performs the following steps on a list of input flatfielded ACS images (flt.fits files): - bad pixel identification - sky subtraction - drizzle each input image onto separate, registered output images - create a median from these images - transform ("blot") the median back to the frame of each input image - create a derivative of the median image, and compare with the original image to obtain a cosmic ray mask - refine the shifts by cross-comparing the images - run drizzle a second time, to combine the images with the cosmic ray masks into a final, combined image. This technique is sufficiently flexible to enable adjustment by means of a range of parameters, but is also sufficiently automated that it is capable of bein run in a pipeline such as that which is used by GOODS. 5.2 Cosmic Ray Rejection Initially a set of shallow, single-exposure-depth mosaics are created which are then combined to create a clean "median" mosaic, using signal-to-noise thresholds to reject cosmic rays and any remaining bad pixels. This technique is extremely robust at producing a clean median mosaic, with the primary requirement being accurate astrometric alignment between epochs. The relevant pieces of this median mosaic are then transformed back to the frame of each of the original input flt.fits files, using "wblot" which is essentially a WCS-based version of "blot". Next, the standard dither package tasks of "deriv" and "driz_cr" are used to compare this blotted image and its derivative image with the original input flt.fits file, and generate a cosmic ray mask based on the comparison. Finally, all the flt.fits files, together with their newly created cosmic ray masks, are drizzled using "wdrizzle" onto a single output mosaic, which has units of countrate in electrons/second in each pixel. 6.0 Known Issues and Problems 6.1 Missing Data A small number of individual exposures are missing from the final multi-epoch HDF-N mosaics. The areas where exposures are missing can be easily identified by displaying and inspecting the weight maps. In one case, one tile (i.e. an area with the footprint of the ACS WFC field of view) in the V (2 exposures), i (2 exposures and z (4 exposures) bands was completely lost during epoch 5 due to a failed guide star acquisition. The GOODS Team plans to request 2 orbits to re-observe the missed tile and produce a new stacked mosaic. We will post a note in the GOODS Web page and release an update to the data when this will be done. In another instance a few individual exposures have not been included in the final z-band stack in the HDF-N, because using them in the multidrizzle pipeline simply makes it crash. It is not understood why this happens. The images look just fine, and no anomalies are obviously observed. Very likely, there is a discrepancy in some of the header parameters that has not been identified. Once this problem is corrected, we will post a note in the GOODS Web page and release an update to the data. 6.2 Increased background in some of the Epoch-1 B-band images Some B-band tiles acquired during epoch 1 in the HDF-N are affected by a relatively large background scattered light due to an error in the scheduling of the observations. This allowed the limb angle to become smaller than about 25 degree, which resulted in an increased Earth scattered light. Such tiles have been re-observed at later epochs. The old and new B-band images have been co-added in the mosaic with optimal weighting to maximize signal-to-noise ratio. The affected tiles (and therefore, the affected sections or part of sections) can be identified by inspection of the weight maps. Upon visual inspection, this scattered light appears rather uniform. No special attention or actions are required when working with these images. 6.3 Astrometry and Rejection of Cosmic Ray and of Other Blemishes Inspection of the weight maps sometimes reveals that residual mis-alignments due to scatter in the astrometric solutions cause rejections of the brightest pixels in bright, unresolved sources by the cosmic-ray rejection algorithm. Only a handful of bright sources are affected. The Team will refine the astrometric solution in the following weeks. We also plan to release a list of stars where we do find evidence of this problem. In no case the rejection problem has been observed in diffuse sources. 6.4 Photometry One known problem is that the photometry of a very small number of bright sources in the mosaics is systematically biased toward low fluxes because of small astrometrical errors, as mentioned above. Another known problem is that the photometry of bright stars in the multi-epoch stacked mosaic is systematically off typically by ~1 percent and in some case by up to ~2 percent, the sense being that the magnitude appears fainter. No such effect is observed in single-epoch stacked images. We discovered that this effect is due to the rejection of the "rotated" diffraction spikes by the cosmic-ray rejection algorithm. This problem seems to affect only bright (z<~20) unresolved sources, namely those for which the energy fraction contained in the diffraction spikes is the highest. It effect becomes progressively less noticeable at fainter flux, because fluctuations in the background become comparable to the spikes. It also becomes less significant if small photometric apertures are used (e.g. less than ~1% for 0.5 arcsec diameter apertures). This is a limitation, albeit a minor one, of the "multi-epoch" approach to the observations. 6.5 PSF Issues During a small number of exposures fine guidance was lost, and the observations were completed in gyro mode. Some of the images affected show a small trailing effect (about 1 pixel). In some application (e.g. weak lensing), this might be a concern. The GOODS Team will post a list of the affected areas as soon as the list is complete. 7.0 Updates and Future Releases 7.1 Updates to version v1.0 We will announce updates to the v1.0 release data set on the GOODS website together with the instructions on how to download them. 7.2 Version v2.0 A version v2.0 is planned for near the end of 2003. Details of this release will be made available when they are finalized.