ZTF API Queries
Overview
IRSA provides access to the ZTF image archive through an application program interface (API). Search and retrieval parameters are specified using HTTP URLs. The output is a table of image metadata satisfying the search constraints. This service is intended to be accessed by scripts or other software.
ZTF Image Metadata: This page describes ZTF metadata, as well as how to use these metadata to construct data product URLs for use in scripts or in web browsers.
Access Control: This section provides instructions and examples for accessing proprietary ZTF data.
ZTF Lightcurve Data: This page describes how ZTF lightcurves (photometry over time) can be accessed via APIs.
Base URL
A query is specified by encoding search parameters into a URL. ZTF query URLs start with one of five prefixes, depending on the category of data desired:
| ZTF image type | query URL prefix |
|---|---|
| raw | https://irsa.ipac.caltech.edu/ibe/search/ztf/products/raw? |
| science | https://irsa.ipac.caltech.edu/ibe/search/ztf/products/sci? |
| calibration | https://irsa.ipac.caltech.edu/ibe/search/ztf/products/cal? |
| reference | https://irsa.ipac.caltech.edu/ibe/search/ztf/products/ref? |
| deep reference | https://irsa.ipac.caltech.edu/ibe/search/ztf/products/deep? |
Query Parameters
- FORMAT=METADATA
-
To see the columns of a particular image metadata table, append the FORMAT parameter with a value equal to METADATA to the base URL.
Examples:
Return a list of metadata for ZTF Science Exposures:
https://irsa.ipac.caltech.edu/ibe/search/ztf/products/sci?FORMAT=METADATA&ct=html
- POS=ra,dec
-
The POS parameter value must consist of an ICRS right ascension and declination in decimal degrees. It identifies the point which returned images must contain, or the center of the search region. The coordinate values must be separated by a comma; no embedded whitespace is allowed.
POS is required in the absence of where.
Examples:
- SIZE=width[,height]
-
The SIZE parameter consists of one or two (comma separated) values in decimal degrees. The first SIZE value is taken to be the full-width of the search region along the east axis at POS, and the second is taken to be the full-height along the north axis. Taken together, POS and SIZE define a convex spherical polygon on the sky with great circle edges - the search region. During a query, this region is compared against the convex spherical polygons formed by connecting the 4 corners of each image in a data-set to determine which images should be returned.
If only one size value is specified, it is used as both the full-width and full-height. Negative sizes are illegal, and a width and height of zero indicate that the search region is a point.
SIZE is an optional parameter. If unspecified, the default search region width and height is 0 (i.e. the search region is just the point POS). Note also that if INTERSECT=CENTER (see below), SIZE is ignored.
Examples:
- INTERSECT=value
-
The INTERSECT parameter specifies a spatial predicate that specifies how the search region S and the spatial extents of a candidate image X must be related. Four values are allowed:
- COVERS: X must completely contain S. Equivalent to CENTER and OVERLAPS if S is a point.
- ENCLOSED: S must completely contain X. If S is a point, the query will always return an empty image table.
- CENTER: X must contain the center of S. If S is a point, this is equivalent to COVERS and OVERLAPS.
- OVERLAPS: The intersection of S and X is non-empty. If S is a point, this is equivalent to CENTER and COVERS.
The INTERSECT parameter is optional; if unspecified, it defaults to OVERLAPS.
The spatial extent of an image is approximated by connecting its corners with great circles to form a spherical convex polygon. The search region as defined by this service is also a spherical convex polygon (or a point). Spatial predicate evaluation is performed on pairs of spherical convex polygons without recourse to any performance motivated short-cuts. Nevertheless, the implementation uses floating point arithmetic and results are therefore not guaranteed to be mathematically exact. Note also that the spatial extent of an image with a non-gnomonic projection (or large distortion terms) may not be well approximated by the technique employed. However, the images currently being served have gnomonic or orthographic projections, are of relatively small area and have small distortion terms - hence, we believe our approximations are accurate enough to be useful.
Examples:
Tips and troubleshooting: Think carefully about the definitions of the terms above. "Covers" will likely not return anything except for when using small radii. "Enclosed" will likely not return anything except for when using large radii. "Center" and "Overlaps" will likely return results in most cases.
- COLUMNS=name1,name2,....
-
The columns parameter value must be set to a comma separated list of the column names that should be output. If this argument is omitted, a table specific default column-set is used.
Examples:
- WHERE=SQL
-
The where parameter can be set to a SQL WHERE clause, with some restrictions. Notably, function calls and sub-queries are not supported. You can use AND, OR, NOT, IN, BETWEEN, LIKE, IS, the usual arithmetic and comparison operators, and literal values.
Note that the where parameter is required in the absence of POS (a spatial constraint).
WHERE clauses should be URL encoded.
Examples:
- MCEN
-
The mcen parameter indicates that only the most centered image/image set (with respect to POS) should be returned, rather than all images/image sets containing POS. If the SIZE parameter is specified and non-zero, the mcen parameter is ignored (except when INTERSECT=CENTER).
Note that mcen is not required to have a value. If one is nevertheless specified, it is ignored.
Examples:
- CT
The query output format is controlled by the ct parameter, which may always be specified, regardless of the query type. The following values are recognized:
ct=html HTML ct=ipac_table IPAC ASCII table format ct=csv Comma-separated-value format ct=tsv Tab-separated-value format Examples:
Table uploads
It is also possible to search for multiple positions in one query (rather than issuing a series of spatial queries). This can be accomplished by setting the value of the POS parameter to the contents of an IPAC ASCII table file containing search positions. The query must be submitted via an HTTP POST rather than a GET, and the table file must contain two columns named ra and dec such that each row corresponds to a single POS value. Any other columns in the table are ignored. Here's an example IPAC ASCII table:
|ra |dec |
|double |double |
358.3 25.6
19.5 39.5
270.0 -30.3
And here is the UNIX shell command that demonstrate how to query for images overlapping these positions using curl:
curl -o out.tbl -F INTERSECT=CENTER -F "POS=@ztf_example.tbl" "https://irsa.ipac.caltech.edu/ibe/search/ztf/products/sci"
Retrieving ZTF Data Products
Once you have the metadata for a data product that you wish to retrieve, you can construct the URL to that data product using the file path patterns described on the ZTF Metadata Page.
Cutouts
Once you have identified the path to a ZTF image, you can create a cutout by appending center and size parameters.
Examples:
Here is the path (URL) to a full-sized ZTF science image:
https://irsa.ipac.caltech.edu/ibe/data/ztf/products/sci/2018/0411/467847/ztf_20180411467847_000535_zr_c11_o_q3_sciimg.fits
To get a 50 arcsec square cutout centered at RA = 255.57691 deg., Dec = 12.28378 deg. on the above science image, use the following URL:
Access Control (only for non-public data)
ZTF currently contains both proprietary and public data. Access to proprietary data requires a password. General instructions for using the IRSA API's with a password are provided here. But the following examples should suffice for ZTF.
Examples:
- Use curl to access proprietary data:
curl --user myemail@astronomy.edu:mypassword -o out.tbl "https://irsa.ipac.caltech.edu/ibe/search/ztf/products/sci?POS=358.3,25.6" - Use wget to access proprietary data:
wget --auth-no-challenge --user=myemail@astronomy.edu --password=mypassword -O out.tbl "https://irsa.ipac.caltech.edu/ibe/search/ztf/products/sci?POS=358.3,25.6" - Use curl with cookies and $USER and $PASS variables to access proprietary data:
curl -c cookies.txt "https://irsa.ipac.caltech.edu/account/signon/login.do?josso_cmd=login&josso_username=$USER&josso_password=$PASS"
curl -b cookies.txt -o out.tbl "https://irsa.ipac.caltech.edu/ibe/search/ztf/products/sci?POS=358.3,25.6"
curl -c cookies.txt "https://irsa.ipac.caltech.edu/account/signon/logout.do" - Use wget with cookies and $USER and $PASS variables to access proprietary data:
wget --save-cookies=cookies.txt -O /dev/null "https://irsa.ipac.caltech.edu/account/signon/login.do?josso_cmd=login&josso_username=$USER&josso_password=$PASS"
wget --load-cookies=cookies.txt -O out.tbl "https://irsa.ipac.caltech.edu/ibe/search/ztf/products/sci?POS=358.3,25.6"
wget --save-cookies=cookies.txt -O /dev/null "https://irsa.ipac.caltech.edu/account/signon/logout.do"
NOTE: Currently the cookie file issued using the last two examples above will be valid for 1 week, after which a new cookie file must be obtained. If requests are issued via a browser session that the user has logged into, note that login sessions will expire after 2 hours of inactivity, up to a maximum of 1 week.
Getting Help
If you've run into a problem, think you've found a bug, or simply have questions, please contact IRSA User Support.