TABLE OF CONTENTS 1. OVERVIEW 2. USAGE 3. INPUT 4. NAMELIST 5. ENVINROMENTAL VARIABLES 6. MODULES 7. OUTPUT 1. OVERVIEW. Perl script hdr_mask.pl is used to mask saturated pixels in the HDR (high dynamic range) IRAC observations. Such observations have BCD's of virtually the same field taken at 2 (or 3) integration times. The pixels in the core of bright sources are very often saturated in the long integration time BCD's. These pixels are masked based on the comparison with the corresponding pixels in the short integration time BCD's. For each long integration time BCD a mask is created in which the saturated pixels are set to 1. Optionally, the values of the masked pixels in the long integration time BCD's can be replaced with the values of the corresponding pixels in the short integration time BCD's. As a preprocessing step one can run script HDR_Split.pl to split the list of input images into the lists of images for each integration time present in the data. The syntax for this script is very simple: HDR_Split.pl ImageList.txt. The output is two (three) lists of files named Short_ImageList.txt, Long_ImageList.txt (Medium_ImageList.txt). Normally, the BCD's and the uncertainty images have the HDR keywords in the their headers. The masks do not have those keywords. One can use script copy_HDR_keywords.pl to copy the HDR keywords from the headers of the BCD's to the headers of the corresponding masks: copy_HDR_keywords.pl ImageList.txt MaskList.txt You don't need this to run hdr_mask.pl, rather to run other scripts, such as overlap.pl, mosaic.pl. Perl script hdr_mask.pl runs all the modules involved in the image correction. The steps in running hdr_mask.pl are as follows 1. Compute the median-subtracted images of the input long integration time BCD's. 2. Subtract the long integration time BCD's from the short integration time BCD's 3. Detect positive outliers in the median-subtracted long integration time BCD's. 4. Detect positive outliers in the difference images. 5. Match the long integration time detections with the difference detection in order to weed out the radhits detected in the difference images. 4. Saved the matched detection in the saturation masks. 5. Replace the saturated pixels in the long integration time BCD's with the corresponding pixels in the short integration time BCD's. 2. USAGE Assuming that mopex has been installed, run hdr_mask.pl -n where the namelist should be in the subdirectory specified by the SIRTF_CDF environmental variable (cdf by default). 3. INPUT The input for pointing_refine.pl: Input File Default Name Set in Namelist Set on Commandline Required ------------------------------------------------------------------------------------------------ namelist for hdr_mask.nl N/A -n y hdr_mask.pl List of long integration time - IMAGE_STACK_LONG_FILE_NAME -L y input images List of short integration time - IMAGE_STACK_SHORT_FILE_NAME -S y input images permamently damaged - PMASK_FILE_NAME -M n pixels mask image Only permamently damaged pixels masks (Pmask) can be used in the processing. If the Pmask name is not set it will not be used. The fatal bit pattern are set in the portion of the namelist pertaining the script itself, Pmask_Fatal_BitPattern is for Pmasks. 4. NAMELIST The namelist file contains the namelists for all modules and some input parameters for the script itself. It is expected to be found in the directory specified by enironment variable SIRTF_CDF(see below). Command line options take precedence over namelist options. Here is an excerpt from a namelist file that pertains to script hdr_mask.pl run_medfilter = 1 detect_long = 1 run_subtract = 1 detect_difference = 1 run_hdr_mask = 1 delete_intermediate_files = 0 DETECT_DIR = Detect SUBTRACT_DIR = Diff HDR_FIX_DIR = Fix OUTPUT_DIR = HDR_Mask IMAGE_STACK_LONG_FILE_NAME = Long_Cosmetic_ImageList.txt IMAGE_STACK_SHORT_FILE_NAME = Short_Cosmetic_ImageList.txt PMASK_FILE_NAME = /stage/ssc-pipe/davidm/IRAC/cal/IRAC.1_pmask_flip.fits Mask_Fatal_BitPattern = 32767 The script executes only the modules that are set to 1. Module Name Namelist setting ______________________________________________________________________________________________ medfilter run_medfilter (on the long integraton time BCD's) detect detect_long (on the long integraton time BCD's) imlistarith run_subtract (subract long integraton time BCD's from the short integration time BCD's) detect detect_difference (on the difference images) hdr_mask run_hdr_mask All the results will be written in OUTPUT_DIR, which can be specified as a relative path, as above, or as an absolute path, e.g. /ssc/pipe/davidm/band4. Switch NICE =1 is used to run all the modules called by the script with "nice +19". If delete_intermediate_files switch is set all the median subracted image, the detection maps, and the difference images will be deleted. 5. ENVIRONMENTAL VARIABLES The following environment variables are used in the script. SIRTF_BIN, SIRTF_CAL, SIRTF_ANC, SIRTF_CDF. SIRTF_CDF specifies the location of the namelist file, by default it's ./cdf. Also the log files for each module are written into that directory SIRTF_CAL specifies the location of the PRF image used by apex_1frame.pl, by default it's ./cal. SIRTF_BIN specifies the location of the executables, it has no useful default, you would have to set it to wherever you copy the executables to. SIRTF_ANC specifies the location of the error header files included in the package. The default is not useful. Also make sure that the place where you copy the libraries is included in your LD_LIBRARY_PATH. Make sure PERL_PATH is set to point to the location where your perl is installed. 6. MODULES Here is a short description of the modules. Module medfilter performs estimation of the background of the input images and outputs background images and background subtracted images. The background is estimated by computing a median of the pixels in the sliding window with the size Window_X and Window_Y, with N_Outliers_Per_Window number of highest pixels being excluded from the window. The parameters are set in the namelist: &MEDFILTER Window_X = 21, Window_Y = 21, N_Outliers_Per_Window = 100, &END Module detect detects pixels in the median subtracted long integration time BCD's and in the difference images above the threshold set in the namelist. The threshold is in terms of the number of standard deviations in the image. The product of this module is a set of mask fits files, masking the bright pixels in the input images. &DETECT Detection_Max_Area = 10000, Detection_Min_Area = 2, Detection_Threshold = 20, Threshold_Type = 'simple', Neighbor_Type = 'sides_and_corners', &END Module imlistarith subtracts the long integration times BCD's from the short integration time BCD's. Module hdr_mask matches the pixels detected in long integration time BCD's with the pixels detected in the difference images. If a pixel is detected in both the program assumes that it is a saturated pixel in the core of a bright point source. If it is only detected in the difference image, the program assumes that it is a radhit in the short integration time BCD and ignores it. The saturated pixels are masked in the mask file named _hdrmask.fits. It also replaces the saturated pixels in the long integration time with the values in the corresponding short integration time BCD's and saves the products under the name _hdrfix.fits. 7. OUTPUT Results of intermediate processing steps are written in subdirectories of the output directory specified in the namelist (OUTPUT_DIR). However, if these directories are given with a full path, they are not treated as subdirectories of OUTPUT_DIR. Namelist Keyword Default Value Module(s) ______________________________________________________________________________________________ MEDFILTER_DIR Medfilter medfilter SUBTRACT_DIR Diff imlistarith DETECT_DIR Detect detect HDR_FIX_DIR HDR_Fix hdr_mask Written by David Makovoz Send questions to the IRSA helpdesk