Scanpi User's Guide

Overview of Scanpi

The Scan Processing and Integration tool (Scanpi, sometimes referred to as ADDSCAN for historical reasons) performs 1-dimensional (in-scan) averaging of IRAS data. The IRAS raw scan data, known as the Calibrated Reconstructed Detector Data (CRDD), are stored in files merged with the IRAS pointing (boresight) information. This collection of IRAS survey data is known as the Level 1 Archive.

Scanpi averages the IRAS scans with various weighting schemes. It is used to obtain the fluxes of extended, confused, or faint sources, and to estimate local upper limits. The sensitivity gain is comparable to that obtained in the IRAS Faint Source Survey -- about a factor of 2-5 over the IRAS Point Source Catalog (PSC), depending on the local noise and number of scans crossing the target position. Scanpi is also useful for diagnosing source extent.

The "classic" Scanpi was upgraded and released (as version 6.0) to the public in September 2007. This User's Guide and associated documentation were updated to reflect the changes. There is a list of differences between the classic and new Scanpi. Parameter settings for Scanpi are described below; default values for the classic Scanpi are listed in red. This User's Guide contains more (and more up-to-date) information than is in the IRAS Explanatory Supplement.

Scanpi is also available using a Program Interface, where the user can access Scanpi at IRSA through HTTP program interfaces, rather than a web form.

Input Parameters

The start page allows users to enter a "Single Location" coordinate/object name or a list of sources or positions via "Upload Table of Sources".

For a list, several types of tables are acceptable, see Table Upload Help. One simple example is IPAC Table Format, an easy-to-read ASCII column-aligned table (example). You can check to see if your table is acceptable by running it through IRSA's Table Reformat and Validation tool.

(For backwards compatibility, "lon" and "lat" column names are also accepted as positional input coordinate column names, rather than "ra" and "dec". The table can have extra columns; all columns in the input table are copied into the results. If a "name" column is provided, the value is used in the results as the source name.)

If a table is input, the user can set an additional parameter under "Show Upload Table Results Summary using one of 4 Coadd Scans", either (1) Detector-Weighted Mean Coadd, (2) Mean Coadd, (3) Median Coadd, or (4) Noise-Weighted Mean. The default is the Median Coadd. This is just the type of coadd for which results are shown on the multi-source summary page, which summarizes results for each source. Results for all 4 types of coadds are available via the Result Link.

The start page also lists the processing parameters. Given below are the default values for both the new (in black) and classic (in red) versions of Scanpi.

Fitting Range Parameters for the 4 IRAS bands are described below. All parameters are +/- distances in arcminutes of the source center. A Fitting Range figure of all the ranges is available.

Processing

At the start of Scanpi processing, all scans passing within approximately the "Maximum Allowed Distance From Detector Center" are extracted from the IRAS orbit files. This allows the processing to narrow down the scans which were "close" to the source, from millions to a few hundred. "Maximum Allowed Distance From Detector Center" is set to 2.2 arcmin [for classic and new Scanpi]. The precision of the orbit files is about 0.5 arcmin, and therefore this first Scanpi parameter is just a coarse check for scans within the vicinity of the source. More precise calculations are performed once this subset of scans is read and scans within 1.7 arcminutes [the same default as classic Scanpi] ("Cross Scan Distance") of the target position are definitively identified. Then the data are interpolated using cubic splines and resampled at 10 samples per arcminute in all bands (the detectors were actually under-sampled at about 4, 4, 2, and 1 samples per arcminute at 12, 25, 60 and 100 microns). The resampling is primarily to facilitate alignment of the scans.

A baseline is fit to each individual scan. The residual noise away from the source is computed using the region given by "Source Exclusion Range" (defaulting to +/-2, +/-2 +/-4 and +/-6 arcminutes [classic Scanpi: +/-2, +/-2 +/-2.5 and +/-4 arcminutes] at 12, 25, 60, 100 microns). A point-source template is fit to each scan.

The scans are averaged four ways: a mean with noisy detectors half-weighted (which is called scan #999), a straight mean (scan #1001), a median (scan #1002) and a noise-weighted mean (scan #1003). These averaged scans are analyzed to find the peak, determine if a believable signal exists, find the zero-crossings, compute the integrated flux densities, fit a point-source template, determine point-source template flux density, and compute the full width at quarter and half maxima.

Output for a Single Source

The first "Scanpi Results" page shows, for each of the four IRAS bands, scan profile plots with overlays and shortened summary tables for the coadds. The plots are of the non-baseline-subtracted coadd scans with any successful template fits indicated. [classic Scanpi: coaddition scan #999 was plotted with the baseline, while the other three scans were plotted with the baseline subtracted]. The table column definitions are given below.

From this page, the user can click on "View, Select and Rerun", in order to view the profile plots and full summary tables for both the individual scans and the coadds. Links to scan tracks are also available at the top. It is also possible from this page to deselect scans or reset fitting parameters and rerun the processing.

In the "View, Select and Rerun" page, there is one full summary table per band; one line for each individual scan and one for each of the 4 types of coadd scans (averaged scan tabular information is presented in blue table cells). The table includes widths and flux density estimators. The flux density estimators include the integral between the zero crossings, the integral between fixed points, an estimate based on the peak flux density, and the flux density from the point source template as fit.

The full summary table contains the following columns:

Profile plots are made for each individual scan and for the four averaged/coadd scans.

In the Scan Profile Plots, the horizontal axis is the distance from the target position in the in-scan direction in arcminutes; the vertical axis is flux density or flux, indicated in either Jy or W/m**2. [classic Scanpi: Note that the scans may be flipped (+/- of zero on the x-axis) compared to the results of classic Scanpi.]

In the individual Scan Profile Plots, the data points are in black and the template fit is in red. The flux density and template fit have been background subtracted; columns plotted from the data in the 'Table' file are "in_scan" vs "flux" and "TMPFIT".

In the Coadd Scan Profile Plots, the data points (including the background flux) are in black, background fit (before subtraction) is in red and the template fit (including the background level) is in cyan. The flux and template fit have not been background subtracted; columns plotted from the data in the 'Table' file are "in_scan" vs "flux_wbg", "YFIT" and "TMPFIT_wbg".

[classic Scanpi: The plot for SCAN 999 showed the interpolated and resampled data with the baseline left in and the baseline fit indicated. The other three averaged scans (SCANs 1001, 1002 and 1003) have the baseline fit subtracted but indicate the signal range and the background range. All 4 types of plots (and the individual scans, if requested) show the template fit, if a template was successfully fit to the data.]

Within the Summary Tables, clicking on the "table" link gives you he individual scan or coadd data files; the files are in IPAC Table Format. The individual scan data files contain more header keywords, which are carried over from the IRAS Level 1 data files; most of these header keys are not relevant for coadded scans and are therefore dropped, except the band number, coadd scan number and sigma of the background. The tables have the following columns:

• in_scan - distance from the source center in arcminutes
• flux - flux density in Jy (background subtracted)
• flux_wbg - flux density in Jy (non-background subtracted flux)
• YFIT - flux density in Jy of the fit of the background level
• TMPFIT - integrated flux density of source as determined by the fit to template, after background subtraction
• TMPFIT_wbg - same as TMPFIT without background subtraction.

The flux densities presented in the Summary Results Tables for columns "peak", "fnu_t", "fnu_z" and "amp" have had corrections applied as follows:

• Units Conversion to Jy: Fluxes were converted from W/m^2 (Level 1) to Jy. The band passes are (13.48, 5.16, 2.58 and 1.0) x 10^12 Hz.
• Point Source Correction: The raw flux densities (listed in the data files) were multiplied by the AC to DC correction factor. The IRAS detectors had a dwell-time dependent responsivity, such that the calibration for very extended sources is different from the calibration for point sources. The raw flux densities are DC calibrated (i.e. for very extended sources) and must be corrected for point sources. The correction factors used are listed at the top of the Scanpi tabular output and are about 1.308, 1.277, 1.060 and 0.98 at 12, 25, 60 and 100 microns.
• Detector Widths Correction: In addition, the integrated flux densities ("fnu_z" and "fnu_t") were normalized by the integrated point source response which is 0.82, 0.844, 1.437, and 3.234 at 12, 25, 60 and 100 microns.

Note, flux densities in the tabular data download links have not had any correction applied other than unit conversion.

From the "View, Select, and "Rerun" page, one can also request Scan Tracks, illustrating the geometry of the included scans. (All scans are averaged as if they were parallel. Scans which are nearly opposite in direction compared to the north-most scan are reversed before averaging.) Click on the "Scan Tracks" link(s) to request an on-the-fly plot of the scan tracks. The background reference image can be either IRAS ISSA (bands 1, 2, 3, and 4) or DSS2 (R band). Please note that the ISSA images are in B1950 while DSS are in J2000. Scan tracks on ISSA backgrounds take approximately 3 seconds to generate. The scan tracks on DSS backgrounds can take over 30 seconds to generate since the FITS images must be requested from the Multimission Archive at STScI (MAST).

The scan tracks plots are in the usual astronomical convention, with North up and East to the left. Each scan is labeled by scan number at its start in red. Scans selected for coaddition are in cyan; scans deselected (in a rerun) are in orange; the sky grid is in blue; the user input source location is given by magenta cross-hairs next to the word "Source".

[classic Scanpi: Scan tracks are plotted using RA offset from source center and DEC offset from source center, without a background image.]

On the start page, selecting "Show Plots of Intermediate Steps" will create an additional web page of plots of each major Scanpi processing step, starting from the raw scan data and ending with point source template fitting. The "Scanpi Results" page will have a link at the top of the page, "Plots of Intermediate Steps". Clicking on this link opens a table of plots. Clicking on any of the plots brings up a larger copy of that plot. The scans are in the same order as on the "View, Select, and Rerun" page, with single scans at the start of the table, and the coadd scans at the end.

The following describes each of the plots on the Intermediate Steps page:

The plots are labeled by the type of fit being displayed, the band number and either the individual scan name or the coadd name/number. Individual scan names are characterized in the Scanpi file description README; in brief, the names take this format:

b#_sop_obs_det_suffix
b#_###_###_##_suffix

"b" followed by the band number, followed by the sop (Satellite Operating Plan) number, followed by the observation number, followed by the detector number, followed by a suffix which is unique for every processing step.

The "View, Select, and Rerun" page also allows the user to rerun the coadditions (for any of the bands), using any number of the individual scans, by deselecting the ones not needed. The ranges for the source and background fitting can also be adjusted when rerunning Scanpi.

Deselecting all the scans for one band is interpreted as a request not to perform coaddition; rerunning with that condition will not produce any of the coadd scan results. Deselecting any of the bands using the "Rereun band #" checkbox will exclude results for that band from the summary pages in the subsequent run; you can re-select "Rerun band #" in the subsequent run in order to bring back the processing for that band if needed. (Note that deselecting any of the bands on the start page, using "Process IRAS Band", excludes that band from all results and subsequent reruns.

Once done with the selection/deselection of scans, click on the "Rerun Scanpi" button at the bottom. The form can also be reset to the previous settings should one need to go back (using the "Reset Form" button at the bottom.) Lastly, the "Start Over" button will return the user to the Scanpi front page which has the search form.

Multiple Sources & Bulk Download

When running Scanpi on multiple sources, the fitted results are summarized in a milti-source table. The multi-source summary table contains the same information in the input table, plus position information for each source, as well as a summary of Scanpi results. The columns in this table are as follows: the first two columns are the "ra" and "dec" (or "lon"/"lat") positions of each source (decimal degrees), then there is a copy of the columns in the user input table, followed by a "ResultLink" which points to a web page of Scanpi results for one source; the next column is "SuccessStatus" where 'YES' indicates that the processing was successful and 'NO' when there was an error; the next column is the user requested "Band", followed by the Scanpi fit results "sigma", "snr", "peak", "fnu_t", "miss", "amp" and "corr_coeff".

The user can click on each "ResultLink" to view the Scanpi summary page for each requested source. Alternatively, the user can use a script created by IRSA, for bulk download of all or selected results. The script uses Wget to download all the Scanpi results per source. You should download the script to your machine. The scripts contain one download command per input source. You can edit the script to comment out any sources which you do not want to download. The commands are structured to perform the download recursively for each source (directory of data; named using the position of each source). Once you are finished running the download script, you will see one directory for every source you requested; in those directories is a README file which describes all results files and subdirectories.

Interpretation of Results

[classic Scanpi: The entire section below pertains to the classic version of Scanpi and has not been updated with analysis using the new version. The section should be in red, but has been left in black since these issues are still relevant to the new version of Scanpi. Note that the new version uses a different calculation for "sigma" (see above); therefore, some of the sigma cutoff values listed below may not apply. ]

Scanpi provides several flux density estimators. While it is clear that integrated flux estimators are better for extended sources, it is NOT the case that the template amplitude is always the best choice for point source fluxes. Some rules of thumb for choosing flux estimators are as follows:

For very strong point sources (> 20 Jy), the peak amplitude is a good estimator. In this case, the template amplitude is affected by hysteresis, which will put "wings" on the source, which may elevate or tilt the template relative to the large-scale baseline.

For moderate point sources (1 to 20 Jy), the best estimators are the template amplitude, peak amplitude, or fnu(t). fnu(t) is obtained by integration within the signal range specified by the user. These three estimators should be in fairly good agreement for moderately strong point sources. Under some circumstances, the template fit may be corrupted by background emission or noise, rendering it less valuable as a flux estimator.

For weak point sources (less than 1 Jy at 12, 25, and 60 microns and less than 2 Jy at 100 microns, SNR of a few), the best estimator is the template amplitude. Note that the cubic spline fitting can cause some "ringing" which tends to enhance weak point sources, pushing their amplitude up by as much as 20% and their half-maximum width down. This is one of the main reasons for preferring the template amplitude for weak sources.

For bright extended sources, fluxes are best estimated by fnu(Z), the integrated signal between the zero-crossings. However, if the SNR is less than about 10, it becomes difficult to identify reliable zero crossings, and fnu(t) may be preferable.

To decide if a source is extended (in-scan), one of two tests can be used: the integrated flux is substantially larger than the peak flux by several sigma, or the width of the signal is significantly larger than the expected width for a point source (Fullmer and Helou, 1987). The second test is best carried out for the 25% width rather than the 50% width if the SNR is sufficiently high (>20).

There is no general rule as to which of the averaging methods produces the best flux estimates. The noise-weighted mean (1003) should minimize the noise, but is more vulnerable to miscalibration. What is most important is that the results from 1001, 1002, and 1003 should agree within about 1 sigma or so. If they disagree by more than this, the individual scans should be examined more closely for peculiarities. Beyond that, the median is probably the most consistently "good" estimator of the three, chiefly because of the non-Gaussian nature of noise in the IRAS data.

References and Additional Links

Scanpi is also available using a Program Interface, where the user can access Scanpi at IRSA through HTTP program interfaces, rather than a web form.

More information about the IRAS Level 1 archive can be found at Level 1 Archive.

There is a list of differences between the classic and new Scanpi.

References

Beichman, C.A., Neugebauer, G., Habing, H.J., Clegg, P.E., and Chester, T.J. (1988) "IRAS Catalogs and Atlases: Explanatory Supplement". (Washington, DC: GPO)

Helou, G. et al, (1988), Ap. J. Suppl., 68,151