Python and TAP Access to KOA Data

KOA provides the capability to discover and access level 0 data through calls to a dedicated Python client, PyKOA, and through widely-used IVOA Table Access Protocol (TAP) clients via nexsciTAP, NExScI's Python based implementation of a TAP-compliant server. PyKOA supports access to public and protected KOA data, while the TAP clients support access to public data only.

The database columns names for use in queries

Information regarding the database columns available for PyKOA and TAP queries can be found in the following links:

PyKOA: A Python Interface to KOA Data

PyKOA offers provides access to public raw science and calibration files acquired with any of the KOA instruments. Keck Observatory PIs may use the KOA credentials already assigned to them with PyKOA to access their protected data.

PyKOA also enables a rich variety of searches, including cone, box, polygon, or all-sky spatial searches; temporal searches; searches on program infortmation; and complex searches on multiple attributes, through queries structured in the IVOA Astronomical Query Language (ADQL).

Functionality For End Users

PyKOA supports the following methods for accessing KOA science and calibration data:

  • Query by date or date range (with examples for each file format).
  • Query by position.
  • Query by object.
  • Query by program information.
  • Query by by combinations of the above search criteria.
  • Download raw science and associated calibration files, or a subset of data, corresponding to a collection of metadata.
  • General metadata queries in the IVOA Astronomical Data Query Language (ADQL).
    (note: not all of the ADQL specifications have been implemented yet)
  • Queries for protected data (available to Keck PIs only).

PyKOA writes the output metadata data to an output file, in the IPAC ASCII, VOTable, CSV or TSV data formats.

Installation

PyKOA can be installed from PyPI:

pip install --upgrade pykoa

Requirements

Requires Python 3.6 (or above), plus table read and write functions from Astropy. We have tested with Astropy 4.0.1. We recommend using the Anaconda Python distribution.

Jupyter Notebooks

These Jupyter Notebooks provide examples of the methods supported by PyKOA, and examples of how Keck PIs may login and access their protected data.

Instrument Notebook HTML version
ESI PyKOA_ESI_introduction.ipynb PyKOA_ESI_introduction.html
DEIMOS PyKOA_DEIMOS_introduction.ipynb PyKOA_DEIMOS_introduction.html
HIRES PyKOA_HIRES_introduction.ipynb PyKOA_HIRES_introduction.html
KCWI PyKOA_KCWI_introduction.ipynb PyKOA_KCWI_introduction.html
LRIS PyKOA_LRIS_introduction.ipynb PyKOA_LRIS_introduction.html
LWS PyKOA_LWS_NIRC_introduction.ipynb PyKOA_LWS_NIRC_introduction.html
MOSFIRE PyKOA_MOSFIRE_introduction.ipynb PyKOA_MOSFIRE_introduction.html
NIRC PyKOA_LWS_NIRC_introduction.ipynb PyKOA_LWS_NIRC_introduction.html
NIRC2 PyKOA_NIRC2_introduction.ipynb PyKOA_NIRC2_introduction.html
NIRES PyKOA_NIRES_introduction.ipynb PyKOA_NIRES_introduction.html
NIRSPEC PyKOA_NIRSPEC_introduction.ipynb PyKOA_NIRSPEC_introduction.html
OSIRIS PyKOA_OSIRIS_introduction.ipynb PyKOA_OSIRIS_introduction.html
Download all PyKOA Notebook examples


TAP Clients

KOA supports synchronous and asychronous queries queries to four widely used IVOA Table Access Protocol (TAP) clients: TAPPlus (or TAP+), PyVO, TOPCAT and IRSA Viewer. Protected (user-proprietary) data at KOA are inaccessible through these clients, as they do not support the security cookies required to access private data at KOA. PIs wishing to access their protected data through a Python API should use PyKOA.

PyVO

PyVO is a Python package that provides access to remote data and VO-compliant services.

Jupyter Notebooks

These Jupyter Notebooks provide examples of the methods supported by PyVO.

Instrument Mode Notebook HTML version
ESI Async PyVO_ESI_Asynchronous.ipynb PyVO_ESI_Asynchronous.html
Sync PyVO_ESI_Synchronous.ipynb PyVO_ESI_Synchronous.html
DEIMOS Async PyVO_DEIMOS_Asynchronous.ipynb PyVO_DEIMOS_Asynchronous.html
Sync PyVO_DEIMOS_Synchronous.ipynb PyVO_DEIMOS_Synchronous.html
KCWI Async PyVO_KCWI_Asynchronous.ipynb PyVO_KCWI_Asynchronous.html
Sync PyVO_KCWI_Synchronous.ipynb PyVO_KCWI_Synchronous.html
LRIS Async PyVO_LRIS_Asynchronous.ipynb PyVO_LRIS_Asynchronous.html
Sync PyVO_LRIS_Synchronous.ipynb PyVO_LRIS_Synchronous.html
LWS Async PyVO_LWS_NIRC_Asynchronous.ipynb PyVO_LWS_NIRC_Asynchronous.html
Sync PyVO_LWS_NIRC_Synchronous.ipynb PyVO_LWS_NIRC_Synchronous.html
HIRES Async PyVO_HIRES_Asynchronous.ipynb PyVO_HIRES_Asynchronous.html
Sync PyVO_HIRES_Synchronous.ipynb PyVO_HIRES_Synchronous.html
MOSFIRE Async PyVO_MOSFIRE_Asynchronous.ipynb PyVO_MOSFIRE_Asynchronous.html
Sync PyVO_MOSFIRE_Synchronous.ipynb PyVO_MOSFIRE_Synchronous.html
NIRC Async PyVO_LWS_NIRC_Asynchronous.ipynb PyVO_LWS_NIRC_Asynchronous.html
Sync PyVO_LWS_NIRC_Synchronous.ipynb PyVO_LWS_NIRC_Synchronous.html
NIRC2 Async PyVO_NIRC2_Asynchronous.ipynb PyVO_NIRC2_Asynchronous.html
Sync PyVO_NIRC2_Synchronous.ipynb PyVO_NIRC2_Synchronous.html
NIRES Async PyVO_NIRES_Asynchronous.ipynb PyVO_NIRES_Asynchronous.html
Sync PyVO_NIRES_Synchronous.ipynb PyVO_NIRES_Synchronous.html
NIRSPEC Async PyVO_NIRSPEC_Asynchronous.ipynb PyVO_NIRSPEC_Asynchronous.html
Sync PyVO_NIRSPEC_Synchronous.ipynb PyVO_NIRSPEC_Synchronous.html
OSIRIS Async PyVO_OSIRIS_Asynchronous.ipynb PyVO_OSIRIS_Asynchronous.html
Sync PyVO_OSIRIS_Synchronous.ipynb PyVO_OSIRIS_Synchronous.html
Download all PyVO Notebook examples


TAPPlus (TAP+)

Part of the Python Astroquery package, TAPPlus is a general Python API for accessing TAP services, and is maintained at the ESAC Space Data Centre (ESDC). In synchronous mode, the package imposes a hard-coded limit on the return of 2000 records.

Jupyter Notebooks

These Jupyter Notebook provide examples of the methods supported by TAPPlus.

Instrument Mode Notebook HTML version
ESI Async TAP+_ESI_Asynchronous.ipynb TAP+_ESI_Asynchronous.html
Sync TAP+_ESI_Synchronous.ipynb TAP+_ESI_Synchronous.html
DEIMOS Async TAP+_DEIMOS_Asynchronous.ipynb TAP+_DEIMOS_Asynchronous.html
Sync TAP+_DEIMOS_Synchronous.ipynb TAP+_DEIMOS_Synchronous.html
KCWI Async TAP+_KCWI_Asynchronous.ipynb TAP+_KCWI_Asynchronous.html
Sync TAP+_KCWI_Synchronous.ipynb TAP+_KCWI_Synchronous.html
LRIS Async TAP+_LRIS_Asynchronous.ipynb TAP+_LRIS_Asynchronous.html
Sync TAP+_LRIS_Synchronous.ipynb TAP+_LRIS_Synchronous.html
LWS Async TAP+_LWS_NIRC_Asynchronous.ipynb TAP+_LWS_NIRC_Asynchronous.html
Sync TAP+_LWS_NIRC_Synchronous.ipynb TAP+_LWS_NIRC_Synchronous.html
HIRES Async TAP+_HIRES_Asynchronous.ipynb TAP+_HIRES_Asynchronous.html
Sync TAP+_HIRES_Synchronous.ipynb TAP+_HIRES_Synchronous.html
MOSFIRE Async TAP+_MOSFIRE_Asynchronous.ipynb TAP+_MOSFIRE_Asynchronous.html
Sync TAP+_MOSFIRE_Synchronous.ipynb TAP+_MOSFIRE_Synchronous.html
NIRC Async TAP+_LWS_NIRC_Asynchronous.ipynb TAP+_LWS_NIRC_Asynchronous.html
Sync TAP+_LWS_NIRC_Synchronous.ipynb TAP+_LWS_NIRC_Synchronous.html
NIRC2 Async TAP+_NIRC2_Asynchronous.ipynb TAP+_NIRC2_Asynchronous.html
Sync TAP+_NIRC2_Synchronous.ipynb TAP+_NIRC2_Synchronous.html
NIRES Async TAP+_NIRES_Asynchronous.ipynb TAP+_NIRES_Asynchronous.html
Sync TAP+_NIRES_Synchronous.ipynb TAP+_NIRES_Synchronous.html
NIRSPEC Async TAP+_NIRSPEC_Asynchronous.ipynb TAP+_NIRSPEC_Asynchronous.html
Sync TAP+_NIRSPEC_Synchronous.ipynb TAP+_NIRSPEC_Synchronous.html
OSIRIS Async TAP+_OSIRIS_Asynchronous.ipynb TAP+_OSIRIS_Asynchronous.html
Sync TAP+_OSIRIS_Synchronous.ipynb TAP+_OSIRIS_Synchronous.html
Download all TAP+ Notebook examples


TOPCAT

TOPCAT is an interactive graphical viewer and editor for tabular data. It provides most of the capabilities astronomers need for analysis and manipulation of source catalogues and other tables. It is a stand-alone Java application that uses Virtual Observatory (VO) standards to access and integrate distributed data.

The screenshots below show how to query for HIRES data in TOPCAT; substitute your favorite instrument for HIRES.

  1. Download the jar file from the TOPCAT web site and open the application. Select "Load Table" from the file menu and then choose "Select From":

    load_table

  2. At the bottom of the window, enter the URL of the TAP service (https://koa.ipac.caltech.edu/TAP) and click "Use Service"

    load_table

  3. In the Use Service tab, enter the SQL query in the window near the bottom and hit "Run Query":

    load_table

  4. When the query has run, you will see a window that reports the status of the query:

    load_table

  5. Double click on a table in Table List on the left side of the window to visualize the table:

    load_table

  6. Now you are ready to analyze the results in TOPCAT.

    Note: If you get a "metadata not found" message when you run the query, Select "VO-> Table Service Access Protocol" from the TOPCAT main menu, then Select "TAP -> Metadata Acquisition -> TAP_SCHEMA-C"


IRSA Viewer

IRSA Viewer is a general purpose tool that offers discovery of and subsquent interaction with images, catalogs and time series data. To access KOA data:

  • On the main IRSA Viewer page, Select the "Catalogs" tab
  • Select the "VO TAP Search" tab
  • In "Select TAP Service", select "KOA https://koa.ipac.caltech.edu/TAP/"
  • In "Select Query Type", select "Single Table"
  • In "Select Table", select Table Collection (Schema): koa_tap, Table: select the KOA instrument table
  • Under "Enter Constraints", enter your search constraints and then select "Search"
In this example, we will do a spatial search for DEIMOS observations of NGC3109 from the VO TAP Search form.

Select KOA TAP service:

select KOA tap service

Select koa_deimos table:

select DEIMOS table

Under "Enter Constraints," select Spatial, and enter NGC3109 in "Coordinates or Object Name". NED will resolve the position of this galaxy. Under "Radius", select degrees and then enter "1". Then click "Search" at the bottom left of the page.

select DEIMOS table

The return page will include a table of metadata observations at the bottom, a 2MASS image centered on the returned observations, and a scatter plot of the positions of the observations. Visit the return page to learn about functions for interacting with the data.

enter search criteria