TABLE OF CONTENTS
1. OVERVIEW
2. USAGE
3. INPUT
4. NAMELIST
5. ENVINROMENTAL VARIABLES
6. MODULES
7. OUTPUT
1. OVERVIEW.
Perl script hdr_mask.pl is used to mask saturated pixels in the HDR
(high dynamic range) IRAC observations. Such observations have BCD's of virtually the same field
taken at 2 (or 3) integration times. The pixels in the core of bright sources are very often
saturated in the long integration time BCD's. These pixels are masked
based on the comparison with the corresponding pixels in the short integration time BCD's.
For each long integration time BCD a mask is created in which the saturated pixels are set to 1.
Optionally, the values of the masked pixels in the long integration time BCD's can be
replaced with the values of the corresponding pixels in the short integration time BCD's.
As a preprocessing step one can run script HDR_Split.pl to split the list of input images
into the lists of images for each integration time present in the data. The syntax
for this script is very simple:
HDR_Split.pl ImageList.txt.
The output is two (three) lists of files named Short_ImageList.txt, Long_ImageList.txt
(Medium_ImageList.txt). Normally, the BCD's and the uncertainty images have the HDR
keywords in the their headers. The masks do not have those keywords. One can use script
copy_HDR_keywords.pl to copy the HDR keywords from the headers of the BCD's to the headers
of the corresponding masks:
copy_HDR_keywords.pl ImageList.txt MaskList.txt
You don't need this to run hdr_mask.pl, rather to run other scripts, such as
overlap.pl, mosaic.pl.
Perl script hdr_mask.pl runs all the modules involved in the image correction.
The steps in running hdr_mask.pl are as follows
1. Compute the median-subtracted images of the input long integration time BCD's.
2. Subtract the long integration time BCD's from the short integration time BCD's
3. Detect positive outliers in the median-subtracted long integration time BCD's.
4. Detect positive outliers in the difference images.
5. Match the long integration time detections with the difference detection
in order to weed out the radhits detected in the difference images.
4. Saved the matched detection in the saturation masks.
5. Replace the saturated pixels in the long integration time BCD's with the corresponding
pixels in the short integration time BCD's.
2. USAGE
Assuming that mopex has been installed, run
hdr_mask.pl -n <namelist_filename>
where the namelist <namelist_filename> should be in the subdirectory specified
by the SIRTF_CDF environmental variable (cdf by default).
3. INPUT
The input for pointing_refine.pl:
Input File Default Name Set in Namelist Set on Commandline Required
------------------------------------------------------------------------------------------------
namelist for hdr_mask.nl N/A -n y
hdr_mask.pl
List of long
integration time - IMAGE_STACK_LONG_FILE_NAME -L y
input images
List of short
integration time - IMAGE_STACK_SHORT_FILE_NAME -S y
input images
permamently damaged - PMASK_FILE_NAME -M n
pixels mask image
Only permamently damaged pixels masks (Pmask) can be used in the processing.
If the Pmask name is not set it will not be used.
The fatal bit pattern are set in the portion of the namelist pertaining the script itself,
Pmask_Fatal_BitPattern is for Pmasks.
4. NAMELIST
The namelist file contains the namelists for all modules and some
input parameters for the script itself. It is expected to be found in the directory
specified by enironment variable SIRTF_CDF(see below).
Command line options take precedence over namelist options.
Here is an excerpt from a namelist file that pertains to script hdr_mask.pl
run_medfilter = 1
detect_long = 1
run_subtract = 1
detect_difference = 1
run_hdr_mask = 1
delete_intermediate_files = 0
DETECT_DIR = Detect
SUBTRACT_DIR = Diff
HDR_FIX_DIR = Fix
OUTPUT_DIR = HDR_Mask
IMAGE_STACK_LONG_FILE_NAME = Long_Cosmetic_ImageList.txt
IMAGE_STACK_SHORT_FILE_NAME = Short_Cosmetic_ImageList.txt
PMASK_FILE_NAME = /stage/ssc-pipe/davidm/IRAC/cal/IRAC.1_pmask_flip.fits
Mask_Fatal_BitPattern = 32767
The script executes only the modules that are set to 1.
Module Name Namelist setting
______________________________________________________________________________________________
medfilter run_medfilter
(on the long integraton time BCD's)
detect detect_long
(on the long integraton time BCD's)
imlistarith run_subtract
(subract long integraton time BCD's from the short integration time BCD's)
detect detect_difference
(on the difference images)
hdr_mask run_hdr_mask
All the results will be written in OUTPUT_DIR, which can be
specified as a relative path, as above, or as an absolute path,
e.g. /ssc/pipe/davidm/band4.
Switch NICE =1 is used to run all the modules called by the script with "nice +19".
If delete_intermediate_files switch is set all the median subracted image, the detection maps,
and the difference images will be deleted.
5. ENVIRONMENTAL VARIABLES
The following environment variables are used in the script.
SIRTF_BIN, SIRTF_CAL, SIRTF_ANC, SIRTF_CDF.
SIRTF_CDF specifies the location of the namelist file, by default it's ./cdf.
Also the log files for each module are written into that directory
SIRTF_CAL specifies the location of the PRF image used by apex_1frame.pl,
by default it's ./cal.
SIRTF_BIN specifies the location of the executables, it has no useful
default, you would have to set it to wherever you copy the executables to.
SIRTF_ANC specifies the location of the error header files included in the package.
The default is not useful.
Also make sure that the place where you copy the libraries is included in your
LD_LIBRARY_PATH.
Make sure PERL_PATH is set to point to the location where your perl is installed.
6. MODULES
Here is a short description of the modules.
Module medfilter performs estimation of the background of the input images and outputs
background images and background subtracted images. The background is estimated by computing a
median of the pixels in the sliding window with the size Window_X and Window_Y,
with N_Outliers_Per_Window number of highest pixels being excluded from the window.
The parameters are set in the namelist:
&MEDFILTER
Window_X = 21,
Window_Y = 21,
N_Outliers_Per_Window = 100,
&END
Module detect detects pixels in the median subtracted long integration time BCD's
and in the difference images
above the threshold set in the namelist. The threshold is in terms of
the number of standard deviations in the image. The product of this module
is a set of mask fits files, masking the bright pixels in the input images.
&DETECT
Detection_Max_Area = 10000,
Detection_Min_Area = 2,
Detection_Threshold = 20,
Threshold_Type = 'simple',
Neighbor_Type = 'sides_and_corners',
&END
Module imlistarith subtracts the long integration times BCD's from the
short integration time BCD's.
Module hdr_mask matches the pixels detected in long integration time BCD's
with the pixels detected in the difference images. If a pixel is detected
in both the program assumes that it is a saturated pixel in the core of a
bright point source. If it is only detected in the difference image, the
program assumes that it is a radhit in the short integration time BCD
and ignores it. The saturated pixels are masked in the mask file named
<BCD_name>_hdrmask.fits. It also replaces the saturated pixels
in the long integration time with the values in the corresponding
short integration time BCD's and saves the products under the name
<BCD_name>_hdrfix.fits.
7. OUTPUT
Results of intermediate processing steps are written in subdirectories
of the output directory specified in the namelist (OUTPUT_DIR).
However, if these directories are given with a full path, they are not treated
as subdirectories of OUTPUT_DIR.
Namelist Keyword Default Value Module(s)
______________________________________________________________________________________________
MEDFILTER_DIR Medfilter medfilter
SUBTRACT_DIR Diff imlistarith
DETECT_DIR Detect detect
HDR_FIX_DIR HDR_Fix hdr_mask
Written by David Makovoz
Send questions to the IRSA helpdesk