Data Pipelines

SOFIA's data processing pipelines are now available to the public. This public release will enable our user community to better understand SOFIA data reduction, to customize their own data reduction, and to provide feedback to the pipeline team. The pipeline software, sofia_redux , is available through SOFIA's GitHub repository .

The EXES, FORCAST, FIFI-LS, FLITECAM, and HAWC+ pipelines are included in the current pipeline release.

• Tutorials
• ATRAN FITS Files
• Current Pipeline Versions
• Pipeline Users Manuals
• FAQs

Tutorials

Below are step-by-step tutorials for running sofia_redux for FIFI-LS data, FORCAST Imaging and Spectroscopy data, and FLITECAM Imaging and Spectroscopy data, and HAWC+ Imaging and Polarimetry data. These tutorials include instructions for how to download the example data from IRSA and notes for each step of the pipeline. For instructions on how to install sofia_redux, please follow the link to the GitHub repository , or to the pipeline documentation system.

• FIFI-LS Tutorial: M82
• FORCAST Imaging Tutorial: Europa
• FORCAST Spectroscopy Tutorial: NGC 7009, The Saturn Nebula
• FLITECAM Imaging Tutorial: NGC 2023
• FLITECAM Spectroscopy Tutorial: BD +30 3639
• HAWC+ Imaging Tutorial: NGC 2273
• HAWC+ Polarimetry Tutorial: NGC 2071

back to the top

ATRAN FITS Files

For optimal telluric correction, FORCAST and FIFI-LS spectroscopic reductions require a library of FITS files, containing model atmospheric transmission spectra, derived from Steve Lord's ATRAN model .

Two versions of the model libraries are available for each instrument, except FLITECAM. ATRAN files parameterized by water vapor are not available for FLITECAM.

The EXES pipeline does not use ATRAN models for telluric correction, but it does attach a reference atmospheric model at a matching altitude and zenith angle to output spectral products, if available. The models used are derived from the Planetary Spectrum Generator (PSG) .

• EXES:
  • Models using a water vapor value based on a standard atmosphere, flight altitude, and telescope elevation ( psg_exes_standard.tgz , 5.4 GB; checksum )
• FORCAST:
  • Models using a water vapor value based on a standard atmosphere, flight altitude, and telescope elevation ( atran_forcast_standard.tgz , 531.3 MB; checksum )
  • Models that are calculated for every combination over a range of zenith water vapor values, telescope elevations, and flight altitudes ( atran_forcast_wv.tgz , 37.5 GB; checksum ). These files will be used if the 'Optimize ATRAN correction' option is selected when running the pipeline.
• FIFI-LS:
  • Models using a water vapor value based on a standard atmosphere, flight altitude, and telescope elevation ( atran_fifi-ls_standard.tgz , 144 MB; checksum )
  • Models that are calculated for every combination over a range of zenith water vapor values, telescope elevations, and flight altitudes ( atran_fifi-ls_wv.tgz , 3.3 GB; checksum ). These files will be used if the 'Use WV values' option is selected when running the pipeline.
• FLITECAM: Models using a water vapor value based on a standard atmosphere, flight altitude, and telescope elevation ( atran_flitecam_standard.tgz , 875.2 MB; checksum )

To verify that you've successfully downloaded any of the ATRAN model files, you can run an md5 checksum on the file and compare the results to the contents of the checksum text files provided above. After downloading and unpacking a library, its location can be provided to the pipeline as an optional parameter in the telluric correction step.

back to the top

Current Pipeline Versions

Here are the current versions of the Data Reduction Pipelines:

When searching for and retrieving reduced data from the SOFIA Science Archive at IRSA , please check the version of the pipeline used to generate the data products. If the pipeline version does not agree with the entry given above, please contact the Help Desk to inquire if the data require re-processing or are in the process of being re-reduced. Not all revisions to the pipelines result in the need to re-process data. If re-processing is underway, new versions of the data products will be available shortly. The status of the data products for any specific set of observations can be provided upon request.

back to the top

Pipeline Users Manuals

Below is a table containing links to the User Manuals for all versions of the FIFI-LS, FORCAST, FLITECAM, and HAWC+ pipelines.

back to the top

FAQs

1. Why doesn't the data I reduced match what is in the archive at IRSA?

If the data are not from one of the most recent flight series, it's likely that the data were processed with an older version of the pipeline. To check this, compare the value stored in the PIPEVERS keyword in the primary FITS header in your locally processed file to the one at IRSA (to see that value at IRSA without downloading the file you can click on the 'Show FITS Header' icon in the data tab). It's also possible that the data at IRSA were processed using different options than the data you processed yourself. The list of options used to process the data can be found in the HISTORY keyword in the primary FITS header.

2. What version of the pipeline was used to process the data at IRSA?

The version of the pipeline used to process data is stored in the PIPEVERS keyword in the primary FITS header.

back to the top