TABLE OF CONTENTS 1. OVERVIEW 2. DATA 3. USAGE 4. INPUT 5. NAMELIST 6. ENVINROMENTAL VARIABLES 7. MODULES 8. OUTPUT 1. OVERVIEW. Perl script cosmetic.pl performs cosmetic fixes of 2 artifacts present in IRAC images: mux bleed and column pulldown. See the IRAC Instrument Handbook for the comments on these effects: http://irsa.ipac.caltech.edu/data/SPITZER/docs/irac/iracinstrumenthandbook/ Perl script cosmetic.pl runs all the modules involved in the image correction. First, a background subtracted images should be produced. This is an optional step, that can be skipped, if a set of background subtracted images has been produced elsewhere. Then the pixels affected by the mux bleed of column pulldown effects are detected and the result is saved in the detection masks. The third step is replacement of the masked pixels with the user specified kind of replacement of a number of neighboring non-masked pixels. 2. DATA 4 images suffering from the above illness have been included. They are in data/IRAC_Art and listed in data/IRAC_Art.tst file. 3. USAGE Assuming that mopex has been installed run cosmetic.pl -n where the namelist should be in the subdirectory specified by the SIRTF_CDF environmental variable (cdf by default). To process the sample data included in the package use the default namelist cdf/cosmetic.nl and run cosmetic.pl The results will be saved in ouput_cosmetic/Correct subdirectory. 4. INPUT The input for cosmetic.pl: Input File Default Name Set in Namelist Set on Commandline Required ------------------------------------------------------------------------------------------------ namelist for cosmetic.nl N/A -n y cosmetic.pl List of image_stack.txt IMAGE_STACK_FILE_NAME -I y input images List of background - MEDIAN_SUBTRACTED_IMAGE_LIST -m n subtracted images permamently damaged - PMASK_FILE_NAME -M n pixels mask image list of DCE status - DCE_STATUS_MASK_LIST -d n mask images (Dmask) Quality mask images can be used in the processing. Two kinds of mask images can be used: permamently damaged pixels masks (Pmask) and DCE status masks (Dmask). Normally there will be a single Pmask for a set of input images. If the Pmask name is not set it will not be used. If the Dmask List name is not set it will not be used. The fatal bit pattern are now set in the portion of the namelist pertaining the script itself, DCE_Status_Fatal_BitPattern is for Dmasks, Pmask_Fatal_BitPattern is for Pmasks. 5. NAMELIST The namelist file contains the namelists for all modules and some input parameters for the script itself. It is expected to be found in the directory specified by enironment variable SIRTF_CDF(see below). Command line options take precedence over namelist options. Here is an excerpt from a namelist file that pertains to script cosmetic.pl run_medfilter = 1 detect_artifacts = 1 correct_artifacts = 1 IMAGE_STACK_FILE_NAME = /mydata/IRAC/0003861504/ch1/ImageList.txt MEDIAN_SUBTRACTED_IMAGE_LIST = /mydata/IRAC/0003861504/ch1/MinBackList.txt OUTPUT_DIR = output_correct NICE = 1 verbose = 1 delete_intermediate_files = 1 The script executes only the modules that are set to 1. Module Name Namelist setting ______________________________________________________________________________________________ medfilter run_medfilter detect_instrart detect_artifacts repbadpix correct_artifacts If MEDIAN_SUBTRACTED_IMAGE_LIST is given then run_medfilter setting will be ignored. If MEDIAN_SUBTRACTED_IMAGE_LIST is not given then run_medfilter should be set to 1. All the results will be written in OUTPUT_DIR, which can be specified as a relative path, as above, or as an absolute path, e.g. /ssc/pipe/davidm/band4. Switch NICE =1 is used to run all the modules called by the script with "nice +19". If delete_intermediate_files switch is set all the background subtracted images produced by medfitler and detection masks produced by detect_instrart will be deleted. 6. ENVIRONMENTAL VARIABLES The following environment variables are used in the script. SIRTF_BIN, SIRTF_CAL, SIRTF_ANC, SIRTF_CDF. SIRTF_CDF specifies the location of the namelist file, by default it's ./cdf. Also the log files for each module are written into that directory SIRTF_CAL specifies the location of the PRF image used by apex_1frame.pl, by default it's ./cal. SIRTF_BIN specifies the location of the executables, it has no useful for you default, you would have to set it to wherever you copy the executables to. SIRTF_ANC specifies the location of the error header files included in the package. The default is not useful. Also make sure that the place where you copy the libraries is included in your LD_LIBRARY_PATH. Make sure PERL_PATH is set to point to the location where your perl is installed. 7. MODULES Here is a short description of the modules. Module medfilter performs background estimation of input images and outputs background subtracted images. The background is estimated by computing a median of the pixels in the sliding window with the size Window_X and Window_Y, with N_Outliers_Per_Window number of highest pixels being excluded from the window. The parameters are set in the namelist: &MEDFILTER Window_X = 21, Window_Y = 21, N_Outliers_Per_Window = 100, &END Module detect_instrart detects pixels affected by the mux bleed and column pulldown effects. Detection is performed separately for mux bleed and column pulldown. In order to perform column pulldown detection Do_ColumnPullup should be set to 1 in the namelist. First all the pixels below the global_mean minus ColumnPullup_Threshold times the global_sigma are detected. If in a particular column the portion of detected pixels is greater then ColumnPullup_MinLine_Portion, then all the detected pixels are confirmed as column pulldown affected and bit 0 is set in the detection mask for the corresponding pixel. In order to perform mux bleed detection Do_MuxBleed should be set to 1 in in the namelist. The algorithm is applied to each quarter-row, i.e. every fourth pixel in a row starting with the first, second, third and forth pixels. The first necessary condition for the pixels to be marked is presence of at least one pixel with the value > Min_MuxBleed_Flux given in the units of the image. If such a pixel is present, then all the pixels with the following range are found in the quarter row: global_mean + MuxBleed_Bottom_Threshold * global_sigma < pixel_value < global_mean + MuxBleed_Top_Threshold * global_sigma. If the portion of such pixels in the quarter row is greater then MaxBleed_MinLine_Portion then all these pixels are confirmed as mux bleed affected and bit 1 is set in the detection mask for the corresponding pixel. &DETECT_INSTRART Do_ColumnPullup = 1, # ColumnPullup_Threshold is the number of sigma's below the mean, # to find pixels-candidates for column pullup, ColumnPullup_Threshold = 2, # ColumnPullup_MinLine_Portion is the minimum portion of a column, # with the pixels < threshold to be subjected to column pullup, ColumnPullup_MinLine_Portion = 0.25, Do_MuxBleed = 1, # Min_MuxBleed_Flux is the minimum flux in the image units, # to flag a row that contains such a pixel, Min_MuxBleed_Flux = 50, # MaxBleed_MinLine_Portion is the minimum portion of a quarter-row, # with pixels > bottom_threshold and < top_threshold, # to be declared mux bleed pixels, MuxBleed_MinLine_Portion = 0.1, #MuxBleed_Bottom_Threshold is the number of sigma's above the mean, # defines bottom_threshold, MuxBleed_Bottom_Threshold = 2, #MuxBleed_Top_Threshold is the number of sigma's above the mean, # defines top_threshold, MuxBleed_Top_Threshold = 40, &END Module repbadpix replaces pixels in the input images masked in the input detection masks with some function of the user specified number of the neighboring pixels. There are three choices of replacement. The default choice is 'random_replacement'. For this the program randomly picks one of the Min_Number_Pixels pixels. The other two choices are the median or mean of the user specified numnber of the neighboring pixels. The choice of replacement and the number of pixels are set in the namelist: &REPBADPIX # the choices are 'random_replacemnt, 'median_replacement' and 'mean_replacement', Replacement_Mode = 'random_replacement', Min_Number_Pixels = 20, &END 8. OUTPUT Results of intermediate processing steps are written in subdirectories of the output directory specified in the namelist (OUTPUT_DIR). However, if these directories are given with a full path, they are not treated as subdirectories of OUTPUT_DIR. Namelist Keyword Default Value Module(s) ______________________________________________________________________________________________ MEDFILTER_DIR Medfilter medfilter DETECT_DIR Detect detect_instrart CORRECT_DIR Correct repbadpix David Makovoz DISCLAIMER: The software in package cosmetic.pl performs cosmetic fixes only. It has not been approved or disapproved by either IRAC IT or IST teams.