How to use the Gator program interface

Background Information

The purpose of this document is to demonstrate how to script catalog queries in IRSA.

Gator is IRSA's Catalog Search engine; extensive documentation can be found here. Gator can be used from IRSA's web interface or through the program interface.

The program interface involves constructing URL queries of the form Root_URL?parameter1=value1&parameter2=value2& etc. The parameter=value pairs specify such things as the desired search target, search type, catalog, and output format in the URL. The Gator program interface root URL is https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query. ASCII output format is outfmt=1. The search target is specified by objstr=object_name, the type of search is spatial=Cone, search radius is radius=radius in arcseconds, and the catalog to be searched is catalog=catalog_name. Catalog names can be found here, in the catname column. Parameters are separated by & symbols. For more information on parameter options and additional parameters not used in these examples, see the Gator program interface documentation.

A commonly used program for automating queries is wget. For download instructions, see the IRSA wget help document. Another program, curl, is used to automate table upload queries, and can be downloaded here. wget is used by typing the command wget followed by the query URL in quotation marks, followed by any wget options such as the name of the desired output file. curl handles the parameter=value pairs differently, as part of the curl command itself rather than in the query URL. These parameter=value pairs are each preceded by a -F, and followed by the program interface root URL in quotation marks.

Common use cases include searching for a single target in multiple catalogs, setting the query output to include only the columns you want, and searching for a list of targets in one or more catalogs. All of these tasks can be done using the Gator user interface, however it can often be more convenient to run them via the program interface instead.


Examples

1. Searching multiple catalogs for a single source

In this example, we will script queries to obtain multi-wavelength data on 3C273 from the USNO-B catalog, 2MASS Point Source Catalog, and IRAS Point Source Catalog.


wget "https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query?outfmt=1&objstr=3c273&spatial=Cone&radius=1&catalog=usno_b1" -O usno_out.tbl

wget "https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query?outfmt=1&objstr=3c273&spatial=Cone&radius=2&catalog=fp_psc" -O 2mass_out.tbl

wget "https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query?outfmt=1&objstr=3c273&spatial=Cone&radius=30&catalog=iraspsc" -O iras_out.tbl

These commands return output tables containing all sources within 1, 2, and 30 arcsec of 3C273 in USNO-B, 2MASS PSC, and IRAS PSC, respectively.

2. Searching by coordinates

If your source does not have a name that can be resolved by NED or SIMBAD, you would instead do a coordinate search. The following demonstrates how to do this in decimal degrees, sexagesimal, and galactic coordinates for USNO-B, 2MASS PSC, and IRAS PSC respectively:

wget "https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query?outfmt=1&objstr=187.27792+2.05239&spatial=Cone&radius=1&catalog=usno_b1" -O usno_out.tbl

wget "https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query?outfmt=1&objstr=12h+29m+06.7s+02d+03m+09s&spatial=Cone&radius=2&catalog=fp_psc" -O 2mass_out.tbl

wget "https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query?outfmt=1&objstr=289.9508894+64.3599708+ga&spatial=Cone&radius=30&catalog=iraspsc" -O iras_out.tbl

3. Restricting search output

If you know precisely what information you plan to use from a given catalog, you can specify that only those columns be returned. For example, to obtain just the coordinates and basic photometric information for 3C273 from these three catalogs, run these commands:

wget "https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query?outfmt=1&objstr=3c273&spatial=Cone&radius=1&catalog=usno_b1&selcols=ra,dec,b1_mag,r1_mag,b2_mag,r2_mag,i_mag" -O usno_out.tbl

wget "https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query?outfmt=1&objstr=3c273&spatial=Cone&radius=2&catalog=fp_psc&selcols=ra,dec,j_m,h_m,k_m" -O 2mass_out.tbl

wget "https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query?outfmt=1&objstr=3c273&spatial=Cone&radius=30&catalog=iraspsc&selcols=ra,dec,fnu_12,fnu_25,fnu_60,fnu_100" -O iras_out.tbl

To find the names of the columns in a catalog, either look at the Gator web interface for that catalog, or the catalog documentation.

4. Searching a list of sources in a single catalog

In some cases you may prefer that all output be written to a single file, for example when querying the same catalog at multiple positions. This can easily be done using curl:

curl -F filename=@input.tbl -F outfmt=1 -F spatial=Upload -F uradius=2 -F uradunits=arcsec -F catalog=fp_psc "https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query" > output.file

Note that the input table must be in IPAC Table Format. For more information, please see the Table Upload Guide. As in the wget examples above, other parameters from the Gator program interface documentation (such as selcols) can be specified. For example, to restrict the output to only ra, dec, and the JHK magnitudes, add a -F selcols=ra,dec,j_m,h_m,k_m option to the curl command.

To include SQL constraints in a table upload program interface query, use the constraints field as in the following example:

curl -F filename=@input.tbl \
-F spatial=Upload \
-F uradius=1 \
-F uradunits=arcsec \
-F catalog=fp_psc \
-F outfmt=1 \
-F constraints=k_snr\>100 \
"https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query"


Note that some of the existing Gator program interface documentation is misleading, in this case you must escape special characters like > rather than using html encodings (%3E for example). However if you wish to do single target scripted queries using either wget or curl, you do need to use html encodings.

If you would prefer to use wget, create a file containing the Gator program interface URLs and run:

wget -i input.file -O output.file

An example input file searching for 3C273 and Arp220 in the 2MASS PSC is given here. Gator will return the results for each line in input.file with standard header information, then the header and results for the next line. You may wish to strip out the header rows for each search result in the output file in order to make it more easily readable. Using curl avoids this issue and is recommended.