Using Tables with IRSA Services
Many IRSA services offer the option to upload a table and conduct multiple searches (Table-Upload or Multi-Object Search) rather than entering them one by one into the web form. The input table is a plain text file, describing the search targets or positions. Currently, different IRSA services have different requirements for the exact format of this table. All IRSA services except the Spitzer Heritage Archive will accept IPAC Table Format with ra and dec specified in J2000 decimal degrees and labeled in the header row. However, it may be convenient to use a different text format. This help page describes the IRSA services' requirements for table formats and includes sample tables.
The Table Reformat and Validation service will check the format of an upload table for compatibility with IRSA. Additionally, it will convert from other formats to the standard IPAC Table Format with J2000 decimal degrees.
Note: Not all services accept all formats. For example, Catalog Search only accepts tables in the standard IPAC Table Format using J2000 decimal degrees and column names of ra and dec (lower case). Each service has specific table documentation within its help pages. (Links are provided in the Best Practices section of this document.) The Spitzer Heritage Archive currently only accepts space delimited files.
Go Directly To A Section:
- Requirements for Supported Table Formats
- Supported Coordinate Systems
- Best Practices for Successful Table Upload
- Troubleshooting Tables
- Contacting IRSA's Help Desk
Requirements for Supported Table Formats
IRSA services accept some combination of the following ASCII table formats:
- IPAC Table Format (Also known as IPAC ASCII Column-Aligned Format)
- Comma-Separated Values (CSV) or Comma-Delimited
- Tab-Delimited
- Space-Delimited
- A Simple List of Objects (such as astronomical source names or coordinates)
Each format and its respective requirements are detailed below.
1. IPAC Table Format (a.k.a IPAC ASCII Column-Aligned Format).
More information is available in the IRSA documentation (Download example.)
IPAC Table Format requires the following structure:
- Data elements (or "cell"s) in each row are delineated by alignment with ascii character-columns.
- Header information for each data-column is provided in two or more header rows, identified by vertical bars ( | ) at the column boundaries.
- Up to four header rows are allowed, defining: Column name (required), data type (required), units, and null value.
- Each data cell must fit within the character-columns for that data-column as defined by the bars in the header.
- No data is allowed under the vertical bars in the header.
- Data cells must be aligned using spaces and not tabs (which are not permitted; see Best Practices).
- Comments and keywords may be defined before the header in rows beginning with a forward slash (\).
-
IRSA Services that accept this format:
- Atlas
- Catalog Search -- requires coordinates in decimal degrees (Equatorial J2000), and compliance with the DBMS constraints.
- Cutouts
- Dust Extinction
- Finder Chart
- 2MASS Extended Sources
- 2MASS Images (Batch Image Service)
- Scanpi (for IRAS data)
- SWAS Spectrum Server
Example of a basic IPAC Table Format file:
| object | ra | dec |
| char | double | double |
M56 289.147941100 30.184500500
ic4710 277.158208330 66.982277780
hoix 49.383333330 69.045833330
tol89 210.339833330 -33.063777780
ngc 4552 188.915863750 12.556341390
M82 148.969687500 69.679383330
mrk33 158.132833330 54.401027780
ngc1097 41.579375000 -30.274888890
arp 22 179.874583330 -19.297222220
M 65 169.733166670 13.092222220
Note: In the above example, notice the vertical lines ( | ) between object, ra, and dec indicate the data-column boundaries. Additionally, the data are aligned to clearly identify the data-columns and do not span across multiple columns.
IPAC Table Format also has the flexibility of allowing the placement of additional information such as comments and keywords above the header rows. To learn more about this capability and how to use multiple header rows, read the Keyword and Comment Lines and Column Headers section of the IPAC table documentation.
2. Comma-Separated Values (CSV) or Comma-Delimited Format (Download example.)
This is a format commonly exported from Microsoft Excel as a .csv file. In this format, each data cell is separated by a comma.
Requirements for CSV tables include:
- The first line must contain the names of each column.
- The columns in the header row are separated by commas ( , ).
- The data cells in the table are separated by commas ( , ).
- The number of cells containing data is equal to or less than the number of columns.
- Blank data rows are treated as blank data records, so omit them to reduce processing time.
IRSA Services that accept this format:
- Atlas
- Cutouts
- Dust Extinction
- Finder Chart
- 2MASS Images (Batch Image Service)
- Scanpi (for IRAS data)
- SWAS Spectrum Server
Example of Comma-Separated Values (CSV) Format (also known as Comma Delimited):
object,ra,dec M56,289.147941100,30.184500500 ic4710,277.158208330,-66.982277780 hoix,149.383333330,69.045833330 tol89,210.339833330,-33.063777780 ngc 4552,188.915863750,12.556341390 M82,148.969687500,69.679383330 mrk33,158.132833330,54.401027780 ngc1097,41.579375000,-30.274888890 arp 22,179.874583330,-19.297222220 M 65,169.733166670,13.092222220
3. Tab-Delimited Format (Download example.)
This is a format commonly exported from Microsoft Excel as a .txt file by selecting Tab-Delimited when saving the file. In this format, each data cell is separated by a tab character. However, tab characters are often hard for some text editors to interpret. In addition, tabs are not allowed within data cells, only between them.
Requirements for tab-delimited tables are virtually identical to those for comma-separated and include:
- The first line must contain the names of each column.
- The columns in the header row are separated by tabs.
- The data provided in the table are separated by tabs.
- The number of cells containing data is equal to or less than the number of columns.
- Blank data rows are treated as blank data records, so omit them to reduce processing time.
IRSA Services that accept this format:
- Atlas
- Cutouts
- Dust Extinction
- Finder Chart
- 2MASS Images (Batch Image Service)
- Scanpi (for IRAS data)
- SWAS Spectrum Server
Example of Tab-Delimited Format:
object ra dec M56 289.147941100 30.184500500 ic4710 277.158208330 -66.982277780 hoix 149.383333330 69.045833330 tol89 210.339833330 -33.063777780 ngc 4552 188.915863750 12.556341390 M82 148.969687500 69.679383330 mrk33 158.132833330 54.401027780 ngc1097 41.579375000 -30.274888890 arp 22 179.874583330 -19.297222220 M 65 169.733166670 13.092222220
In the above example, the columns are aligned using tab characters between each data cell. Note the data-columns may not line up exactly, even if tabs are applied consistently (as in row #5). Resist the temptation to add additional tabs in order to make the data align correctly; this will only create extra and empty columns, and your table upload will fail. Spaces are allowed.
4. Space-Delimited Format (Download example.) Only used by the Spitzer Heritage Archive.
In this format each cell is separated by a space. Therefore, if there is a space in your object name (e.g., "NGC 1001" versus NGC1001) or position ("34 23 45.45" versus 34d23m45.45s), you need to put quotes around the target name or its position. Since this format is only used by the Spitzer Heritage Archive, there are several additional requirements:
- The first line must specify the coordinate system.
- The second line must specify the epoch.
- The third line must specify the name resolver.
- No header is required.
- Lines need not have the same number of columns.
- Comment lines may be included with # as the first character.
Example of Space-Delimited Format:
COORD_SYSTEM: Equatorial # Equatorial, Galactic, or Ecliptic - default is Equatorial EQUINOX: J2000 # B1950, J2000, or blank for Galactic - default is J2000 NAME-RESOLVER: NED # NED or Simbad - default is Simbad #Name RA/LON DEC/LAT PM-RA PM-DEC EPOCH "NGC 001" 12h34m23.45s 34d23m56.2s 2.3 3.4 1950.3 NGC2222 23.56d 34.456d 2.3 3.4 1950.3 NGC3333 23.56h 34.456d 2.3 3.4 1950.3 NGC4444 "12 34 12.23" "34 23 45.45" m31 legacy "17 18 00" "59 30 00" m32 m33 simbad NGC6946 NGC5194 ngc2992
5. Simple List of Object Names / Coordinates with No Header (Download example.)
Some IRSA services will accept a simple list of objects with no header. In this case, there are no complex formatting rules to remember. Objects can be listed by coordinates, or by name -- as recognized by NED and SIMBAD. Resolving the names with NED or SIMBAD may take a few seconds per row.
Requirements for this format are minimal:
- Each line has only one item.
- The entire line is treated as an object name or coordinate string.
IRSA Services that accept this format:
- Atlas
- Cutouts
- Dust Extinction (requires an |objname| header row; see Example 2 in the service documentation)
- 2MASS Images (Batch Image Service)
- Scanpi (for IRAS data)
- SWAS Spectrum Server
Example:
M56 ic4710 hoix 3 23 45.6 -12 34 56.78 ngc 4552 M82 3h 22m 15.6s -13d 52m 11.17s ngc1097 331.2 -6.94 ga M 65
Supported Coordinate Systems
IRSA services strive to be flexible enough to accept most common coordinate representations (see below for a list of exceptions). Coordinates can be entered as decimal degrees or sexagesimal; in Galactic, Equatorial or Ecliptic (with or without Equinox); as multiple columns or a single column coordinate string; or (if a single string) as an object name to be resolved via astronomical name resolvers.
The first step in processing an uploaded table is to identify the best column(s) to use as coordinates. When any of the pairings of right ascension and declination listed in the table below are given, they are interpreted as shown in the Result column.
Note: This list, which is extensible, is based on data formats used by IRSA's data providers. In all examples, input column names are not case-sensitive.
| Right Ascension | Declination | Result |
| ra | dec | Equatorial * |
| cra | cdec | Equatorial * |
| ra2000 | dec2000 | Equatorial J2000 |
| ra2000 | de2000 | Equatorial J2000 |
| _raj2000 | _dej2000 | Equatorial J2000 |
| raj2000 | dej2000 | Equatorial J2000 |
| raj2000 | decj2000 | Equatorial J2000 |
| ra1950 | dec1950 | Equatorial B1950 |
| ra1950 | de1950 | Equatorial B1950 |
| rab1950 | deb1950 | Equatorial B1950 |
| rab1950 | decb1950 | Equatorial B1950 |
| elon | elat | Ecliptic J2000 |
| elon2000 | elat2000 | Ecliptic J2000 |
| elon1950 | elat1950 | Ecliptic B1950 |
| glon | glat | Galactic |
| l | b | Galactic |
| lon | lat ** | |
| starlon | starlat ** |
* The first two examples default to J2000, unless an Equinox column is included.
** The last two default to Equatorial J2000 unless extra Equinox ("equinox" or "epoch") and Coordinate-system ("coord sys" or "sys" or "csys") columns are explicitly given.
If no complete set of coordinate columns is found, the table is checked for a variety of possible columns that might contain coordinate strings or object names: "object," "source," "objname," "objstr," "locstr," "location," "star," "galaxy," or "name."
The next step is to parse the coordinates or look up the object by name. Again, the processing allows for a fairly wide variety of inputs. Object name lookup includes checking with NED and SIMBAD.
Coordinates can be decimal degrees or sexagesimal in various forms:
| Longitude | Latitude |
| 3h23m45.6s | -37d15m41s |
| 3 23 45.6 | -37 15 41 |
| 3h 23m 45.6s | -37d 15m 41s |
| 3 23' 45.6" | -37 15' 41" |
| 50d 56m 24s | -37d 15m 41s |
| 50.94000 | -37.2614 |
The final step in the processing is to output a column-aligned ASCII table with the same information content as the user's input, plus, if not already included, columns named "ra" and "dec" containing the right ascension and declination in decimal degrees J2000. This output can be copied from the IRSA web site and pasted into a text file for future uploading.
Best Practices for Successful Table Upload
Each table format has its own set of rules, as described above. Here are some general best practices for all IRSA tables:
- Read the table upload rules for the applicable service(s). Some IRSA services, such as the Gator catalog search, have stricter rules regarding table formats and coordinate systems. Service-specific table help documentation is located on the following pages:
- Gator
- Dust Extinction
- Atlas
- Cutouts
- Finder Chart
- 2MASS Images Batch Image Service
- 2MASS Extended Sources
- Scanpi
- Run your table through the Table Reformat and Validation service to convert your data to IPAC Table Format, especially if you intend to upload to Gator.
- Columns should not contain "non-printable" ASCII characters. Printable characters are things like "a" and "?." Non-printable characters are sometimes inserted by text editing programs for their own purposes, and include symbols like ESCAPE (^[), FORMFEED (^L) and DELETE (^H). TAB characters are special in several ways and need to be treated differently in different contexts, as described below.
- Be very careful when using tabs. A tab (^I) is a single character used in most editors to define a display action ("jump to the next tab stop"). In aligned data, such as those in an IPAC ASCII Column-Aligned format, tabs can make the data look aligned (in your text editor) when in fact the characters don't line up and are therefore illegal. In tab-delimited tables, a single tab character is the separator between columns, and in this context any extra tabs will inflate the column count. It is best to remove all tabs from formats other than tab-delimited and to make sure there are no extra (non-column-separator) tabs there.
- Don't mix delimiters , such as using a mixture of tabs and commas ( , ).
- Don't mix coordinate systems. Using multiple types of coordinate representations in a single table may not work with some services, such at Catalog Search and Dust Extinction.
- Use an ASCII text editing program, such as vi, emacs, Notepad in Windows, or TextEdit on Macs, that prevents extraneous (and problematic) characters from being inserted.
- Omit spaces in table file names. Most services (including the Table Reformat and Validation service) will not read a file that has spaces in its name. For example, a file called "Spitzer Point Sources.txt" should be re-named "SpitzerPointSources.txt" or "Spitzer_Point_Sources.txt."
- Do not provide comment text (or anything else) at the top of the table unless you are using IPAC Table Format.
- The number of cells containing data must not exceed the number of column names. However, the number of cells can be fewer than the columns of data (the line will be padded with trailing blank cells).
- IRSA accepts a broad range of formats and coordinate systems, such as sexagesimal and Galactic. Tables run through the Table Reformat and Validation service will convert the data to RA and Dec decimal degrees in IPAC Table Format. For more information, see Sexagesimal and Galactic Coordinates. Note that the Catalog Search service requires that ra and dec columns have lower case names.
Troubleshooting Tables
Error Message: IRSA could not recognize this table format.
Reason: The table is in a binary file format, such as .doc, .rtf or .xls.
In order for a table to be read by IRSA's services, it must be in a plain text ASCII file. If you are using Microsoft Word, be aware documents may appear to be in plain text, but can still contain hidden formatting if saved in any file format other than plain text (.txt).
Solution: For best results, create the file in a plain text editor such as vi, emacs, Notepad in Windows or TextEdit in Mac. If your data is in a Microsoft Excel spreadsheet, you can save the file as a .csv file.
Error Message: Records in the table do not contain uniform number of columns.
Reason: Too many data columns, too few column names.
If you are using IPAC ASCII Column-Aligned, or tab- or comma-delimited formats, the number of cells containing data must not exceed the number of column names.
In the following example, there are three named columns: A, B, and C. However, there is a row with four cells of data:
A,B,C 5,2, 7,6,9,11
In the above example, the "11" is orphan data because it is not aligned with column A, B, or C.
If you are using IPAC ASCII Column-Aligned format, placing data under the vertical bars in the header will result in both this error message and the "failed to parse" error message, below.
Solution: Add additional column names in the header row to identify each data cell, and make sure that data entries are properly aligned with the correct header columns.
Error Message: We cannot identify columns to use as coordinates or a column to use as object names/locations.
Reason: The column names do not sufficiently describe an object name, or the locations on the sky using ra, dec, glon, glat, etc.
Solution: Clean up the column names so they are recognizable by IRSA services. See Sexagesimal and Galactic Coordinates for more information.
Example:
| jcgRA | jcgDec | size | 150.3814 2.3606 60.0 150.2794 2.1560 60.0 149.8873 2.0789 60.0 150.2323 1.9599 60.0 150.5407 2.5196 60.0 149.9343 2.4426 60.0
In the above example, there is extraneous text appended to RA and Dec, rendering the column name unrecognizable.
Error Message: Failed to parse the table.
Three common conditions may result in this error message.
Reason #1: When using IPAC Table Format, data are located beneath the vertical bars in the header.
Solution: Align the data with the columns in the header. (See IPAC Table Format)
Reason #2: Space is used as data delimiter, but also exists within the data, making it uninterpretable.
Solution: If you are using the tab-delimited format, carefully check your data for extra tabs that the system is interpreting as new columns. Be aware that tabs are not visually obvious when viewed on screen or in print; it's best to use a text editor that will allow you to view the file's underlying formatting. (See Best Practices For Successful Table Upload)
Reason: #3 Table contains non-printable ASCII characters.
A non-printable ASCII character in a table, such as ^[ (escape) or ^\ (file separator), can create problems during the upload process. Tabs are one of the most common causes of failed uploads because they are considered characters, even if they don't appear on print-outs, or even on-screen. When used in a table, the tab character is translated into its ASCII counterpart (^I), which is not interpretable during table upload.
Solution: Remove the characters from the file. Cleaning up tabs is done differently, based on your operating system and text editor:
- On Max OSX, Linux and Sun OS, use the Unix command cat to view a file's non-printable characters:
cat -tev filename
Here are two examples:
Example 1: cat -tev ipac_good_table.tbl
| lon | lat|$ | double| double|$ 237.140482 -28.797019 $ 237.3 -28.0 $
Example 2: cat -tev tab_delim_table.txt
object^Ira^Idec$ M56^I289.147941100^I30.184500500 $ ic4710^I277.158208330^I-66.982277780 $ hoix^I149.383333330^I69.045833330 $
This will bring your file to the screen with all hidden characters, such as tabs (^I) and end of lines ( $ ) indicated. Tabs may seem like spaces and can easily be found using this command, and fixed. Additional blank space at the end of each line causes parser errors; if the end of line ( $ ) characters do not line up, then delete the blank space until they do.
- In Microsoft Word on Windows machines, view all tabs in a document by toggling on formatting marks (click the toolbar button that looks like a paragraph symbol, similar to the pi symbol). Remove the arrows embedded within the table, and then save the file as "text only." Here is a screenshot example:
Error: Search results return objects at coordinates substantially different than the intended input coordinates.
Reason: The table is using IPAC ASCII Column-Aligned format, and the columns are misaligned.
Solution: Make sure the table data are contained within the boundaries set by the vertical lines in the column header rows.
In the following examples, a table with misaligned data is reformatted by the Table Reformat and Validation service into IPAC Table Format:
Misaligned data columns:
| ra | dec | name| 12 24 36 45 29 27 m1 6 45 22 5 12 54 m2
Reformatted data columns:
| ra_u | dec_u | name | ra | dec | |char |char |char |double |double | 12 24 36 45 29 27 m1 186.1500000 +0.7580556 6 45 22 5 12 54 m2 101.3416667 +0.0866667
In the above example, the first set of coordinates is interpreted as 12h24m00s, 00d45m29s instead of the intended 12h24m36s, 45d29m27s.
Contacting IRSA's Help Desk
If you having a specific issue with tables that is not addressed in this document, contact the IRSA Help Desk by submitting a User Support Ticket.
 |
 |
 |
 |
IRSA Charter :: IRSA Acknowledgement :: Privacy/Security Policy
JavaScript must be enabled in order for the site to display properly.