ISSA Explanatory Supplement
APPENDIX D
Global Destriping
D.1 Introduction
Global destriping for ISSA was accomplished using a BasketWeave DeStriping (BWDS) algorithm (Emerson and Gräve 1988). This algorithm assumes that each detector of the same band sees the same intensity, after removal of the zodiacal emission, when pointed to the same location on the sky at any time during the IRAS mission. Due to the redundancy in the sky coverage, a typical detector scan path crossed hundreds of other detector scan paths occurring at different times. An intensity difference history for each detector scan was generated and fitted with an nth order polynomial. Each scan was adjusted by a portion of the difference between the original scan and the fit. This process was repeated until differences were minimized.There were a number of difficulties involved in implementing this approach. The contribution of the zodiacal emission to the total intensity at any location on the sky varied throughout the mission. Imperfections in the removal of the zodiacal emission by use of a physical model developed by J. Good at IPAC left residual zodiacal emission, particularly near the ecliptic plane. This effect was most troublesome at 12 and 25 µm. Residual hysteresis remained after the hysteresis removal effort (see §III.A.2). The remaining hysteresis resulted in intensity discrepancies near the Galactic plane, noticeable at 60 and 100 µm. Finally, the detector-to-detector gain discrepancy attributable to uncertainties in individual detector AC-to-DC responsivity scale factors was also noticeable at 60 and 100 µm in the Galactic plane.
D.2 Database Generation
|
|
|
Figure D.1 Distribution of boresight intercepts
for the 1.2 million focal plane crossings in the IRAS mission,
plotted in ecliptic coordinates. larger largest |
|
|
|
Figure D.2 Histogram of boresight
crossing counts per fractional scan segment. larger largest |
One can determine how the crossing frequency varied through a typical observation by combining the focal plane crossing data from Figure D.1 with information about the way the survey scans covered the sky. This information is presented in the form of a histogram in Figure D.2. Each scan length was normalized to 1.0 and then divided into an equal number of fractional segments. A bin represents the cumulative number of crossings within that fractional segment for all scans. As expected, there are many more crossings near the end points of the observation (typically near one of the ecliptic poles) than the middle (typically near the ecliptic plane). An even distribution of points would be easier for fitting. Nothing could be done to increase crossings near the ecliptic plane, but a thinning of crossings near the poles was possible. To avoid aliasing, the selection process had to be random. However, no reduction in the crossing density near the ecliptic plane was allowed. To accomplish these two objectives, each scan was divided into 30-second time intervals and as many as four separate focal plane crossings were randomly selected from each of the time intervals.
All detector crossings within the same band were
considered separately. This had a major impact on the database size.
Only operative detectors 3/4 size or larger
(Main Supplement,
§II.C.4)
were used by
the global algorithm.
The detectors listed in Table D.1 below show that the number used varied with
wavelength over a range from 11 to 14.
| Wavelength (µm) | Dectectors | Total |
|---|---|---|
| 12 | 23 24 25 27 28 29 30 48 49 50 51 52 53 54 | 14 |
| 25 | 16 18 19 21 22 40 41 42 43 44 45 | 11 |
| 60 | 08 09 10 12 13 14 15 32 33 34 35 37 38 | 13 |
| 100 | 01 02 03 04 05 06 07 56 57 58 59 60 61 | 13 |
|
|
|
Figure D.3 Proliferation of detector intercepts
-- 100 µm. This figure illustrates the geometry of a
focal plane
crossing for 100 µm detectors. A and B represent crossing scan
tracks. The point where the path laid down by detector 5, scan A crosses
the path laid down by detector 61, scan B
is circled as a sample detector crossing point.
For the 13 detectors used at 100 µm there are 169 detector
crossing pairs. The proliferation factor varies with the square
of the number of detectors used per band:
196, 121, 169 and 169 at 12, 25, 60 and 100 µm, respectively. larger largest |
The proliferation of detector crossings is illustrated in Figure D.3. Geometry of a sample 100 µm focal plane crossing is shown. Let "A" refer to the earlier crossing and "B" refer to the later. The point where the middle of the swath laid down by detector 5, scan A crosses the middle of the swath laid down by detector 61 sometime later during scan B is circled as a sample detector crossing point. The 13 detectors used at 100 µm result in 169 detector crossing pairs. The proliferation factor varies with the square of the number of detectors used per band; 196, 121, 169 and 169 at 12, 25, 60 and 100 µm, respectively. Note that as the angle between scans A and B decreases, the detector crossing grid becomes more elongated and detectors on opposite sides of the focal plane cross further from the central point.
The database for each band had one record per node (boresight crossing point). Each record included an n× n matrix of intensities, where n equals the number of detectors used for that band. In order to minimize the file size, each detector intensity was encoded into a two-byte integer. The encoding scheme determined a unique scale factor and bias for each of the two focal plane scans crossing through the node. These were chosen to preserve as much intensity information as possible. Only those nodes selected as above made it into the database. For each selected node all detector information, boresight crossing times, scan angles and position data were included. Everything needed to reconstruct the position, time and intensities at each detector crossing was saved.
In order to compute the intensities at a node, the crossing geometry of two scans was used to compute the differences from the boresight crossing times for every pair of crossing detectors. These times were used to interpolate intensities out of the time-ordered detector data. Detector data are phased to adjust for the nominal time differences between when various rows of detectors cross the same spatial point. The phasing had to be removed before the interpolation could be performed. The intensity at the required time was linearly interpolated from the detector data, provided that the quality flags on both sides of the required time indicated good values.
To minimize database size, all necessary information was recorded
once per node along with a pointer to the location of the next node.
Since node selections were performed independently on each scan,
some nodes were used only once.
This file structure proved economical but too slow in accessing nodes
that were nonsequential. To improve performance, a separate record
was entered for each pass
through the selected node. If a node was selected on
both focal plane scan A and scan B, the node data would be
recorded twice. This allowed the file to be sorted so that access was
sequential.
Since some nodes were selected only once, the disk requirements
did not quite double.
|
|
|
Figure D.4(a) Intensity residuals (HCON-1 and
HCON-2) at 12 µm in ecliptic cylindrical coordinates. larger largest |
As mentioned earlier,
the BWDS database contained detector data that had the zodiacal
emission removed using a physical model derived at IPAC by J. Good
(§III.C.2 and
Appendix G). Figure D.4(a)
shows a low-resolution intensity
map of the entire HCON-1 and HCON-2 sky at 12 µm after removal of
the zodiacal dust using the J. Good model. Note the sharp change in overall
intensity near ecliptic longitude of 60° and a less
pronounced change near 240°. The intensity difference
across the discontinuity has a maximum value of about 2.0
MJy sr-1,
roughly 7% of the local intensity before
removal of the zodiacal component. Similar percentage
errors are found at 25 µm.
The beginning
of the descending leg occurs at 60°,
the ascending leg of the
HCON-1 and HCON-2 survey occurs at 240°.
Six months later the descending leg had progressed to 240°
and the ascending leg to 60° point. Thus
the data on the right side of each discontinuity were
taken six months later than that on the left.
|
|
|
Figure D.4(b) Intensity residuals (HCON-3)
at 12 µm in ecliptic cylindrical coordinates.
These residual intensity images were made from the
IRAS Zodiacal History File (ZOHF) Version 3.1
after removing emission due to the zodiacal dust cloud.
A zodiacal dust cloud model developed
at IPAC by J. Good was used to derive the offset. larger largest |
Although these discontinuities reflect imperfect modeling of the
zodiacal dust cloud, the residual errors are a small
percentage of the total zodiacal emission. Similar discontinuities
are not seen in the HCON-3 sky
(Figure D.4(b)) because the telescope ran out of liquid helium
before the sky coverage came back to the starting point.
D.3 Database Clean-Up
As originally set up, the BWDS database contained a small fraction of anomalous data that could cause problems in downstream processing. These anomalies arise from a myriad of sources, but they can be characterized into three classes. Class I consists of those anomalies introduced into the datastream prior to generation of the BWDS database. This is the largest class of anomalies. Class II is composed of those anomalies introduced during generation of the BWDS database. Finally, Class III consists of those time periods when the global fits are unsatisfactory. Once identified, anomalies of Class I and Class III were included in the Problem Scan Summary File (PSSF). This file was then used to set use flags in the BWDS database prior to the difference fits. It was also used by downstream processors. Class I anomalies are discussed in further detail in Appendix C and Class III anomalies in §D.5.The one error type known to belong to Class II was inadvertently introduced while encoding detector intensities for inclusion in the BWDS database. For a given band, the range from the lowest to the highest intensity recorded on the detectors of the first scan (Scan A) while passing through a given node (node n) was divided into 65,535 parts. Each intensity was assigned a two-byte integer between -32767 and +32767, with -32768 reserved to indicate unreliable data. An appropriate scale factor and bias were determined to allow later decoding. Another scale/bias set was determined in the same manner for the second scan (Scan B) passing through node n, and the intensities were encoded as before. This process was repeated for each node. The problem arose when the difference between the lowest and highest intensities became excessively large, driving down the resolution with which all the intensities could be stored. One extremely bright point in a node would degrade the resolution of all the other intensities from that scan/node.
Since intensity differences large enough to cause obvious difficulties were rare, this problem escaped notice during testing of the database generation software. Once the problem was understood, it became apparent that the nodes with degraded resolution could be quickly identified by their excessively large values of scale factor and bias. Given that the affected nodes were rare, easily identified and would be rather difficult to regenerate, it was decided to drop them from the database. In order not to disturb the existing database indexing, this was accomplished by resetting use flags rather than actually removing the records.
D.4 Intensity Difference Fits
Given the size of the BWDS database, there was only enough disk space to hold one of the four wavelength bands at a time. The intensity difference fits were performed starting with 12 µm and proceeding through 25, 60 and 100 µm. All four bands were fitted using nth order orthogonal polynomials, but the fit technique varied to some extent with band for two reasons. First, each wavelength band is unique and required some tailoring of the approach to achieve the best results. Second, the various bands were done in sequence and more experience could be brought to bear as time went on. The whole process was very time consuming; it was not feasible to reprocess earlier wavelengths.
Before discussing the approach used to fit each band, assumptions and
approximations are discussed. As mentioned earlier, the BWDS
algorithm is based on the premise that any detector from
the same band sees the same intensity when pointed to the
same spot in the sky at any time during the mission after removal of
the zodiacal emission. Ideally, each detector should:
1) be centered on the same spot
2) be oriented the same way
3) have the same size and shape
4) have the same responsivity as a function of position
5) see the sky unchanged between observations.
The requirement that two detectors share a common center point at the time of differencing can be met. Pointing reconstruction uncertainties are quite small relative to the size of the detectors. Given accurately known time histories of boresight direction and rotation angle, along with knowledge of the relative focal plane position of each detector, it is possible to precisely determine the time and sky position of each detector crossing. The requirement that they be oriented the same way can only be met near the ecliptic plane and grows increasingly worse moving toward the poles. The size and shape match can usually be met except when a 3/4 size detector crosses a full size. The matched responsivity requirement will never be exactly met except when the same detector crosses itself at a later time.
Due to the large number of intensity differences involved in a typical fit, along with the randomness of detector mix and orientations, the first four requirements are not critical. For any given difference point, errors resulting from the above problems are as likely to be positive as negative. The order of the fit is low compared to the number of differences, resulting in an increase in the dispersion rather than a shift in the mean. For the same reason, point sources do not have to be removed. If a point source is observed by one detector and not the other it will indeed throw off that difference, but the next point source is just as likely to throw it off in the opposite direction. The net effect on the mean, given a large number of differences, is thus negligible. The requirement that the sky remain unchanged between observations cannot be met so easily. The effects of asteroids and variable stars can be discounted using the arguments outlined in the previous paragraph. Time variations in the residual zodiacal foreground, however, affect extended areas of the sky in a systematic, slowly changing way. If the zodiacal emission is not completely removed prior to fitting, resulting intensities will be affected by the residual emission. Therefore, coverages of the same area made at widely separated times will have different average intensities. The BWDS algorithm performs several iterations converging to a solution that forces all coverages to their mean background. An area covered at three widely separated times will therefore have a different background intensity than an adjacent area covered only twice.
D.4.a Fits at 12 µm
The order of the orthogonal polynomial fit for each detector-scan was based on the number of intensity difference points available after questionable points had been removed. The order selected would follow low-frequency detector errors while not introducing or removing sub-field (< 12.5°)-sized structure. The relationship between the number of difference points available (N) and the order of fit (O) for 12 µm is given in TableD.2. Any scans where N was less than five were not fit by this algorithm but were fit by the local destriper (§III.C.3.b).
| Difference Points | Order of Fit |
|---|---|
| 0 - 4 | No fit |
| 5 - 50 | 0 |
| 51 - 150 | 1 |
| 151 - 350 | 2 |
| 351 - 750 | 3 |
| 751 - 1500 | 4 |
| 1501 - 2250 | 5 |
| 2251 - 3000 | 6 |
| >=3001 | 7 |
Fitting the intensity difference history assumes that the detector errors are time-varying bias errors. However, a certain amount of gain error can be removed as well. As the fraction of the intensity difference attributable to gain error increases, it becomes more difficult to derive a fit (of a given order) that works for both the high- and low-intensity regions of the scan.
Differences due to gain errors will naturally be greater in high intensity regions. If every point is weighted equally, these regions will dominate the fit. To compensate for this effect at 12 µm, each difference point was inverse-intensity weighted. The weighting intensity was the larger of the two intensities being differenced. Weights were not allowed to exceed 25 times the average.
As each intensity difference was computed, the best estimate of the truth was taken to be halfway between the two. It was actually the difference between the intensity readout from the current scan and the best estimate of the truth that was loaded into the difference history to be fitted. The algorithm corrected the intensities after all the scans had been fitted. This was a better approach than trying to make corrections to crossing scans as they became available. The latter course would make the results dependent on the order of processing and would make restarting in midstream difficult.
Observations beginning or ending near the ecliptic plane were difficult to fit due to the paucity of detector crossings near the plane. A large percentage of these scans were actually continued on the other side of the ecliptic plane under a different observation number. These observations were broken near the ecliptic plane due to avoidance maneuvers. These included avoidance of Jupiter, the Moon and the South Atlantic Anomaly (SAA). The broken scans were knit together for purposes of the BWDS fit. Because of time lost during the avoidance maneuver, time adjustments had to be made. When a fit on part A of a broken scan was desired, the part B times were adjusted and vice versa. The additional difference points that were not part of the scan were referred to as "ghost" points.
The 12 µm fits were done a total of four times, with all updates being made after each iteration. In order to prevent spurious differences from unduly affecting the fit, differences with magnitudes greater than an input threshold value were not used.
The threshold used for the first iteration was 0.5× 10-6 Wm-2sr-1. For subsequent iterations the threshold was tightened to 0.33× 10-6 Wm-2sr-1. The number and distribution of rejects were carefully monitored. Difference histories for scans with excessive rejects were plotted and manually inspected (§D.5).
D.4.b Fits at 25 µm
The 25 µm BWDS fit started with the same procedure outlined for 12 µm. The first four iterations used the 12 µm criteria to select the order of fit and to determine the best estimate from which to compute the intensity difference.Inverse intensity weighting was also used. Rejection thresholds were raised to 0.75× 10-6 Wm-2sr-1 for the first iteration and 0.5× 10-6 Wm-2sr-1 for subsequent iterations.
|
|
|
Figure D.5(a) Intensity differences along
a single detector-scan track with a sixth order fit. larger largest |
The 25 µm data differed from 12 µm in one significant way. The magnitudes of the intensity differences near the ecliptic plane were much more pronounced at 25 µm. This was probably due to gain errors being driven by the higher zodiacal foreground intensity at 25 µm. The steep rise near the ecliptic was difficult to fit, which threw off the fit in the low-intensity regions near the poles. The problem is illustrated in Figure D.5(a), which shows the intensity difference history for a detector in one scan with a sixth-order fit superimposed. Intensity differences were computed by differencing the intensities along the desired detector-scan path with the intensities of crossing detectors. To improve the fit, the algorithm was modified to allow fits as high as 12th order. This improved the fit but was not totally satisfactory. Finally, the northern and southern ecliptic hemispheres were fitted separately and then knitted together. To facilitate the knitting, an overlap region at the ecliptic plane 20° wide was considered to be part of both hemispheres. Fits were limited to 10th order, with the relationship between the number of difference points available (N) and the order of fit (O) established, shown in Table D.3.
| Difference Points | Order of Fit |
|---|---|
| 0 - 4 | No fit |
| 5 - 50 | 0 |
| 51 - 150 | 1 |
| 151 - 350 | 2 |
| 351 - 600 | 3 |
| 601 - 850 | 4 |
| 851 - 1100 | 5 |
| 1101 - 1350 | 6 |
| 1351 - 1600 | 7 |
| 1601 - 1850 | 8 |
| 1851 - 2100 | 9 |
| >=2101 | 10 |
The northern solution was used in the north and the southern solution in the south. Near the ecliptic plane the northern and southern components of each scan were knitted together by linearly changing the weighting used to combine the two solutions over the effective overlap interval. Thus on an ascending scan the southern solution would be weighted 1.0 and the northern weighted 0.0 at the beginning of the overlap interval, 0.5 and 0.5 at the midpoint and 0.0 and 1.0 at the end, respectively.
|
|
|
Figure D.5(b) Intensity differences along
a single detector-scan track with a fit
derived with the dual-hemisphere-with-overlap algorithm. larger largest |
The overlap interval for a given scan is defined to extend from the earliest to the latest detector crossing contained within the 20° overlap region. It should be noted that for any given scan the effective overlap could be less than 20° if there was a sparsity of crossing points. Fits were never extrapolated before the first or after the last detector crossing. The dual-hemispheres-with-overlap approach was used in the fifth iteration of the 25 µm parameters and proved to work well, as seen in Figure D.5(b). Because this approach fit the high- and low-intensity regions equally well, equal weighting was used.
The difference histories for the fifth iteration were computed assuming the best estimate of truth to be the crossing scan intensity. Thus a full step rather than a half step was taken, allowing the final iteration to be more effective. This effectiveness was particularly important since the fifth iteration also had to remove fitting errors caused by anomalous "ghost" points inadvertently introduced in iterations one through four. The use of "ghost" points was dropped from the fifth iteration at 25 µm as well as for all iterations at 60 and 100 µm.
A number of factors were monitored as indicators of scans for
which the dual-hemisphere approach might be inappropriate.
If any one of
these indicators exceeded a given threshold, a full-scan fit was
used for that scan. Full-scan fits used the same crossing count
vs. fit order detailed in Table D.3. Any one of the following
conditions would trigger a full-scan fit for the
affected detector-scan.
1) Insufficient points for southern hemisphere fit
2) Insufficient points for northern hemisphere fit
3) All southern hemisphere points contained in overlap
4) All northern hemisphere points contained in overlap
5) Effective overlap less than 100 seconds of time
6) Difference between north and south solutions greater than
0.125× 10-6 Wm-2sr-1
at midpoint or 0.25× 10-6
Wm-2sr-1
at either end
of effective overlap
7) Ratio of north/south intensity difference over effective
overlap time greater than 0.5× 10-9
Wm-2sr-1 s-1 at
midpoint or 1.0 × 10-9 Wm-2sr-1
s-1 at either end.
As at 12 µm, the 25 µm fits were carefully monitored (§D.5). All 25 µm scans for which "ghosts" points were used prior to iteration five were manually checked with intensity difference plots. This verified that iteration five had successfully removed the adverse effects resulting from the anomalous "ghost" points.
D.4.c Fits at 60 µm
The 60 µm fits used the same order-of-fit scheme outlined for iteration five at 25 µm (Table D.3). The fraction of the difference,
, between the path intensity and the crossing
intensity taken to be the best estimate of the truth varied
with iteration. Increasing with each iteration
provided a more rapid convergence. Only three iterations were
needed for 60 µm; was set at 0.5, 0.75, and
1.0. Reject thresholds were set at 0.75× 10-6,
0.5× 10-6 and 0.5× 10-6
Wm-2sr-1.
Intensity difference increases near the ecliptic plane were not significant for the 60 µm fits. However, the problem was significant near the Galactic plane. The cause was gain error, driven by the higher Galactic plane intensity at 60 µm. A modified version of the dual- hemisphere approach using Galactic hemispheres was considered and rejected.
The intensity difference increases at the Galactic plane could only be handled by fitting the gain errors directly. This option was investigated (see Appendix E) but, since applying a gain correction would compromise the point source calibration, it was not used. The gain fitting investigation showed that the increased intensity differences near the Galactic plane are due to two effects: an error in the DC response of each detector and residual hysteresis. Both gain errors would have to be addressed to get good fits near the Galactic plane. No attempt was made to fit this region accurately at 60 µm. The inverse intensity weighting coupled with the higher order polynomials allowed for good fits everywhere except for within 1° to 2° of the Galactic plane.
D.4.d Fits at 100 µm
The 100 µm fit procedure was similar to the 60 µm procedure with the exception that the magnitudes of the intensity differences near the Galactic plane at 100 µm were much greater. They were so large that even fits up to tenth order caused fitting problems in the low intensity regions. Inverse-intensity-squared weighting resulted in considerable improvement. Inverse-intensity-cubed worked even better and was adopted for 100 µm. Letting Ip represent the path intensity and Ix the crossing intensity, the weighting factor (W) is defined as follows:Imax = MAX(|Ip|,|Ix|)
Ibar = 2.5 × 10-7 W m-2 sr-1
W = 1/(Imax/Ibar)3
W < .01 -> W = .01
W > 10. -> W = 10
The 100 µm fit was done in three iterations with the
order-of-fit table, rejection thresholds and settings the same as
at 60 µm. All the comments regarding gain errors and
residual hysteresis made about the 60 µm fit are even more
applicable to the 100 µm. As with the 60 µm fits, the
100 µm fits should be considered questionable within 1° to
2° of the Galactic plane.
D.5 Monitoring Algorithm performance was carefully monitored throughout the fitting process. The global RMS values of intensity differences as a function of iteration number are tabulated in Table D.4 for each wavelength. The RMS is reduced with each iteration but the amount of reduction is less each time. The RMS value after the last iteration at each wavelength is an extrapolated rather than a measured value. This is because the considerable computer time required to obtain the difference statistics was not significantly less than required to do another iteration.
| Wavelength (µm) | Iteration | RMS Difference (MJy sr-1) | % Reduction |
|---|---|---|---|
| 12 | 0 | 0.616 | -- |
| 1 | 0.410 | 33.4 | |
| 2 | 0.334 | 45.8 | |
| 3 | 0.306 | 50.3 | |
| 4 | [0.296] | [51.9] | |
| 25 | 0 | 1.178 | -- |
| 1 | 0.717 | 39.1 | |
| 2 | 0.541 | 54.1 | |
| 3 | 0.476 | 59.6 | |
| 4 | 0.435 | 63.0 | |
| 5 | [0.404] | [65.7] | |
| 60 | 0 | 0.672 | -- |
| 1 | 0.437 | 35.0 | |
| 2 | 0.295 | 56.1 | |
| 3 | [0.242] | [64.0] | |
| 100 | 0 | 1.785 | -- |
| 1 | 1.117 | 37.4 | |
| 2 | 0.900 | 49.6 | |
| 3 | [0.811] | [54.6] |
Intensity difference plots, some of which have already been
shown, provided good visibility on a scan-by-scan basis.
Comprehensive checking using plots alone was not feasible due to the
prohibitive number of detector/scan combinations.
Instead, a program was written that
computed a set of parameters that served as indicators
of possible fitting problems. The following parameters proved
most useful:
1) variance of fit for detector-scan
2) absolute value of fit at each extreme point
3) absolute value of fit slope midway between each
extreme point pair as well as at each end point
4) absolute value of 2nd derivative of fit at each extreme
point
5) number of points rejected in detector-scan.
Histograms were generated for each parameter and detector-scan with extreme parameter values written into the Problem Scan Summary File (PSSF). Detector-scans identified as anomalous from other sources (see Appendix C) were written into the PSSF as well. The file was then used to establish namelists for making large numbers of intensity difference plots. The plots were manually inspected and the scans actually causing difficulties were identified. For the identified scans, problem type, severity and affected time interval were determined and loaded back into the PSSF.
|
|
|
Figure D.6 Plot in ecliptic coordinates of points
rejected by the global
destriper for detector 42 (25 µm). larger largest |
During the fitting process a file of rejected intensity differences was generated. Using this file, all-sky maps like the one in Figure D.6 were plotted to show the global position of rejects. It was found that rejects were concentrated in high-intensity regions. The Galactic plane is prominent in all four bands; exceptionally bright areas such as found in Orion, Cygnus and Ophiuchus also show up. The rejects scattered over the sky are associated with bright point sources. The long strings of rejects are due to problem scans which had to be corrected or eliminated.
Problems identified from the intensity difference plots were
divided into different types based on
characteristic signatures.
The types can be grouped into three
categories, which map as follows into two of the three broad
anomaly classes previously discussed (§D.3):
1) problems introduced into the data stream prior to
BWDS processing that could distort the BWDS fits (Class IA)
2) problems introduced into the data stream prior to
BWDS processing which would not distort the BWDS fits (Class IB)
3) time intervals identified when the global BWDS fits
should not be used for downstream processing (Class III).
Class II anomalies were eliminated prior to fitting. Residual effects from incompletely removed Class II anomalies were apparently small; none were identified on the intensity difference plots checked. It is possible that some of the scattering attributed to other causes may have resulted from incompletely removed Class IIs. Class IA anomalies which could be identified beforehand were also eliminated prior to fitting. Pre-fit identification of these anomalies came from the original images as well as the latest calibration and pointing reconstruction processing.
|
|
|
Figure D.7 Intensity differences along
a detector-scan track illustrating a class IA anomaly. larger largest |
Figure D.7 illustrates a serious Class IA anomaly. This
particular anomaly affects all
detectors in all bands and is believed to be the result of a
paint flake passing in front of the focal plane. If not removed,
it would not only distort the fit (solid line in figure) for the
observation in which it occurs but could also adversely affect
the crossing scan fits. This type of anomaly, once identified,
was flagged for non-use not only in the BWDS database but for all
downstream processors as well.
|
|
|
Figure D.8 Intensity differences along
a detector-scan track illustrating a class IB anomaly. larger largest |
Figure D.8 illustrates a typical Class IB anomaly. It is caused
by incompletely removed hysteresis as the affected scan crosses a
very bright point source. This type of anomaly sometimes shows
up as a double spike with positive and negative components on
either side of the point source crossing. Since the crossing
scans may also be affected by the point source, the signature on
the intensity difference plots depends on the mix of ascending
and descending scans. These anomalies were not removed from the
BWDS database because they have too high a spatial frequency to
appreciably affect the fit. It should be remembered by the user,
however, that intensities in areas immediately surrounding bright
sources are suspect.
A typical Class III anomaly was shown in Figure D.5(a). Had it not been possible to improve the fit with a combination of higher order and dual-hemisphere techniques, the time periods where the fit differed appreciably from the intensity differences would have gone into the PSSF as Class III anomalies. As illustrated by the figure, there is a tendency for the polynomial fits occasionally to flair off near the ends when heavily stressed elsewhere. To help locate potential problems of this type, fits having an extreme point within a plate width of an end point (start or finish) were identified and the intensity difference between the two saved. Time intervals where the magnitude of intensity difference exceeded 14.0 MJy sr-1 or the end point slope exceeded 0.2 MJy sr-1 were flagged for local fit only.
|
|
|
Figure D.9 Intensity differences along
a detector-scan track illustrating a class III anomaly. larger largest |
Another type of Class III anomaly is illustrated in Figure D.9. This type of problem can occur when there is a large internal gap in crossing times within a detector-scan. Since there are no crossing points to constrain the fit within the gap, it is possible for large excursions to occur within that region. Whether it happens depends not only on the size of the gap, but also on how stressed the fit is outside the gap.
An automated approach was developed to help identify detector-scans with the potential for this type of problem. First, the longest gap in each detector-scan was identified. Second, those detector-scans whose polynomial fits contained an extreme point within the gap period were marked. Next, the polynomial fit was evaluated at each end of the gap to provide the end points needed for a linear interpolation across the gap. Differences between the actual fit values at the marked extreme points and the linearly interpolated values were computed.
Those gap times with difference-from-linear magnitudes greater than 1.0 MJy sr-1 were flagged for local destriper only. The one exception to this was in the dual-hemisphere overlap region at 25 µm where the near-linear assumption does not apply.
|
|
|
Figure D.10 Scan segments that have close extreme
point pairs for 100 µm fits plotted in ecliptic coordinates. larger largest |
In order to maintain relative photometry across a plate, the frequency of the fits had to remain low. The intent was that it should not be possible to introduce artifacts or to remove real structure smaller than 12.5°. Order-of-fit criteria were selected so that extreme points would be separated by more than a field width. There is, of course, nothing to prevent an individual polynomial fit from having rapid fluctuations over a portion of the scan. The real question is whether the individual scans involved combine in such a way as to introduce artifacts or remove real structure. In order to address this question, those fits with extreme point pairs closer than a plate width having intensity differences greater than 1.0 MJy sr-1 were identified. The global positions of the scan legs between the identified extreme point pairs are plotted for the 100 µm fits in Figure D.10.
In areas with many identified close extreme point pairs, the global BWDS contribution to the image was subtracted out and manually inspected for adverse effects.
As has been previously described, every effort was made to identify fit problems through the method of automated checking of indicator parameters coupled with manual inspection of indicated scans. However, some problem fits will have slipped through the screen. We can be confident that the individual problem fits slipping through the BWDS monitoring were not severe in nature. The greater danger is that a number of small fitting errors slipped through which were systematically wrong in the same direction and sky position. By far the most likely place for this to happen is at the scan end points, which occur near the ecliptic poles. To guard against these dangers, images of every plate were checked manually (§III.D). In areas where there are many scan ends, the global BWDS contribution to the image was subtracted out and manually inspected for such effects.