Python and TAP Access to KOA Data

KOA provides a Python client, PyKOA, to discover and access raw (level 0) data for all instruments and quick-look extracted (level 1) files for five instruments for which an automated pipe pipeline available: HIRES, NIRSPEC, NIRC2, OSIRIS, and LWS. The reduced data are intended as a guide to the science content of the data are and not intended for scientific analysis. PyKOA supports access to public and protected data. Note that level 1 data created by pipelines developed under the WMKO Data Services Initiative (DSI) are not yet available through the Python client.

Calls may also be made through widely-used IVOA Table Access Protocol (TAP) clients via nexsciTAP, NExScI's Python based implementation of a TAP-compliant server. The TAP clients support access to public data only.

• PyKOA: A Python Interface to KOA Data
• TAP Clients

The database columns names for use in queries

Information regarding the database columns available for PyKOA and TAP queries can be found in the following links:

• DEIMOS Columns
• ESI Columns
• GUIDER Columns (PyKOA n/a)
• HIRES Columns
• KCWI Columns
• KPF Columns
• LRIS Columns
• LWS Columns
• MOSFIRE Columns
• NIRC Columns
• NIRC2 Columns
• NIRES Columns
• NIRSPEC Columns
• OSIRIS Columns
• Download all instrument keywords CSV files

PyKOA: A Python Interface to KOA Data

PyKOA offers provides access to public raw science and calibration files acquired with any of the KOA instruments. Keck Observatory PIs may use the KOA credentials already assigned to them with PyKOA to access their protected data.

PyKOA also enables a rich variety of searches, including cone, box, polygon, or all-sky spatial searches; temporal searches; searches on program infortmation; and complex searches on multiple attributes, through queries structured in the IVOA Astronomical Query Language (ADQL).

Functionality For End Users

PyKOA supports the following methods for accessing KOA science and calibration data:

• Query by date or date range (with examples for each file format).
• Query by position.
• Query by object.
• Query by program information.
• Query by by combinations of the above search criteria.
• Download raw science and associated calibration files, or a subset of data, corresponding to a collection of metadata.
• General metadata queries in the IVOA Astronomical Data Query Language (ADQL).
  (note: not all of the ADQL specifications have been implemented yet)
• Queries for protected data (available to Keck PIs only).
• Queries for a moving object searches for observations of an asteroid, comet, planet, or natural satellite by NAIF ID (SPK-ID), or by manually entering the orbital elements. (as of version 1.7.2)

PyKOA writes the output metadata data to an output file, in the IPAC ASCII, VOTable, CSV or TSV data formats.

Installation

PyKOA can be installed from PyPI:

pip install --upgrade pykoa

Requirements

Requires Python 3.6 (or above), plus table read and write functions from Astropy. We have tested with Astropy 4.0.1. We recommend using the Anaconda Python distribution.

Jupyter Notebooks

These Jupyter Notebooks provide examples of the methods supported by PyKOA, and examples of how Keck PIs may login and access their protected data.

Instrument	Notebook	HTML version
All instruments (except KPF) Moving Object Search Service	PyKOA_MOSS_introduction.ipynb Display helper module: mossdisputil.py	PyKOA_MOSS_introduction.html
ESI	PyKOA_ESI_introduction.ipynb	PyKOA_ESI_introduction.html
DEIMOS	PyKOA_DEIMOS_introduction.ipynb	PyKOA_DEIMOS_introduction.html
HIRES	PyKOA_HIRES_introduction.ipynb	PyKOA_HIRES_introduction.html
KCWI	PyKOA_KCWI_introduction.ipynb	PyKOA_KCWI_introduction.html
KPF	PyKOA_KPF_introduction.ipynb	PyKOA_KPF_introduction.html
LRIS	PyKOA_LRIS_introduction.ipynb	PyKOA_LRIS_introduction.html
LWS	PyKOA_LWS_NIRC_introduction.ipynb	PyKOA_LWS_NIRC_introduction.html
MOSFIRE	PyKOA_MOSFIRE_introduction.ipynb	PyKOA_MOSFIRE_introduction.html
NIRC	PyKOA_LWS_NIRC_introduction.ipynb	PyKOA_LWS_NIRC_introduction.html
NIRC2	PyKOA_NIRC2_introduction.ipynb	PyKOA_NIRC2_introduction.html
NIRES	PyKOA_NIRES_introduction.ipynb	PyKOA_NIRES_introduction.html
NIRSPEC	PyKOA_NIRSPEC_introduction.ipynb	PyKOA_NIRSPEC_introduction.html
OSIRIS	PyKOA_OSIRIS_introduction.ipynb	PyKOA_OSIRIS_introduction.html
Download all PyKOA Notebook examples

TAP Clients

KOA supports synchronous and asychronous queries queries to three widely used IVOA Table Access Protocol (TAP) clients: PyVO, TOPCAT and IRSA Viewer. Protected (user-proprietary) data at KOA are inaccessible through these clients, as they do not support the security cookies required to access private data at KOA. PIs wishing to access their protected data through a Python API should use PyKOA. (The TAP+ client operating under astroquery has been deprecated and KOA queries to it are no longer supported).

• PyVO
• TOPCAT
• IRSA Viewer

PyVO

PyVO is a Python package that provides access to remote data and VO-compliant services.

Jupyter Notebooks

These Jupyter Notebooks provide examples of the methods supported by PyVO.

Instrument	Mode	Notebook	HTML version
ESI	Async	PyVO_ESI_Asynchronous.ipynb	PyVO_ESI_Asynchronous.html
	Sync	PyVO_ESI_Synchronous.ipynb	PyVO_ESI_Synchronous.html
DEIMOS	Async	PyVO_DEIMOS_Asynchronous.ipynb	PyVO_DEIMOS_Asynchronous.html
	Sync	PyVO_DEIMOS_Synchronous.ipynb	PyVO_DEIMOS_Synchronous.html
KCWI	Async	PyVO_KCWI_Asynchronous.ipynb	PyVO_KCWI_Asynchronous.html
	Sync	PyVO_KCWI_Synchronous.ipynb	PyVO_KCWI_Synchronous.html
KPF	Async	PyVO_KPF_Asynchronous.ipynb	PyVO_KPF_Asynchronous.html
	Sync	PyVO_KPF_Synchronous.ipynb	PyVO_KPF_Synchronous.html
LRIS	Async	PyVO_LRIS_Asynchronous.ipynb	PyVO_LRIS_Asynchronous.html
	Sync	PyVO_LRIS_Synchronous.ipynb	PyVO_LRIS_Synchronous.html
LWS	Async	PyVO_LWS_NIRC_Asynchronous.ipynb	PyVO_LWS_NIRC_Asynchronous.html
	Sync	PyVO_LWS_NIRC_Synchronous.ipynb	PyVO_LWS_NIRC_Synchronous.html
HIRES	Async	PyVO_HIRES_Asynchronous.ipynb	PyVO_HIRES_Asynchronous.html
	Sync	PyVO_HIRES_Synchronous.ipynb	PyVO_HIRES_Synchronous.html
MOSFIRE	Async	PyVO_MOSFIRE_Asynchronous.ipynb	PyVO_MOSFIRE_Asynchronous.html
	Sync	PyVO_MOSFIRE_Synchronous.ipynb	PyVO_MOSFIRE_Synchronous.html
NIRC	Async	PyVO_LWS_NIRC_Asynchronous.ipynb	PyVO_LWS_NIRC_Asynchronous.html
	Sync	PyVO_LWS_NIRC_Synchronous.ipynb	PyVO_LWS_NIRC_Synchronous.html
NIRC2	Async	PyVO_NIRC2_Asynchronous.ipynb	PyVO_NIRC2_Asynchronous.html
	Sync	PyVO_NIRC2_Synchronous.ipynb	PyVO_NIRC2_Synchronous.html
NIRES	Async	PyVO_NIRES_Asynchronous.ipynb	PyVO_NIRES_Asynchronous.html
	Sync	PyVO_NIRES_Synchronous.ipynb	PyVO_NIRES_Synchronous.html
NIRSPEC	Async	PyVO_NIRSPEC_Asynchronous.ipynb	PyVO_NIRSPEC_Asynchronous.html
	Sync	PyVO_NIRSPEC_Synchronous.ipynb	PyVO_NIRSPEC_Synchronous.html
OSIRIS	Async	PyVO_OSIRIS_Asynchronous.ipynb	PyVO_OSIRIS_Asynchronous.html
	Sync	PyVO_OSIRIS_Synchronous.ipynb	PyVO_OSIRIS_Synchronous.html
Download all PyVO Notebook examples

TOPCAT

TOPCAT is an interactive graphical viewer and editor for tabular data. It provides most of the capabilities astronomers need for analysis and manipulation of source catalogues and other tables. It is a stand-alone Java application that uses Virtual Observatory (VO) standards to access and integrate distributed data.

The screenshots below show how to query for HIRES data in TOPCAT; substitute your favorite instrument for HIRES.

1. Download the jar file from the TOPCAT web site and open the application. Select "Load Table" from the file menu and then choose "Select From":
   
   
   
   
2. At the bottom of the window, enter the URL of the TAP service (https://koa.ipac.caltech.edu/TAP) and click "Use Service"
   
   
   
   
3. In the Use Service tab, enter the SQL query in the window near the bottom and hit "Run Query":
   
   
   
   
4. When the query has run, you will see a window that reports the status of the query:
   
   
   
   
5. Double click on a table in Table List on the left side of the window to visualize the table:
   
   
   
   
6. Now you are ready to analyze the results in TOPCAT.
   
   Note: If you get a "metadata not found" message when you run the query, Select "VO-> Table Service Access Protocol" from the TOPCAT main menu, then Select "TAP -> Metadata Acquisition -> TAP_SCHEMA-C"

IRSA Viewer

IRSA Viewer is a general purpose tool that offers discovery of and subsquent interaction with images, catalogs and time series data. To access KOA data:

• On the main IRSA Viewer page, Select the "Catalogs" tab
• Select the "VO TAP Search" tab
• In "Select TAP Service", select "KOA https://koa.ipac.caltech.edu/TAP/"
• In "Select Query Type", select "Single Table"
• In "Select Table", select Table Collection (Schema): koa_tap, Table: select the KOA instrument table
• Under "Enter Constraints", enter your search constraints and then select "Search"

In this example, we will do a spatial search for DEIMOS observations of NGC3109 from the VO TAP Search form.

Select KOA TAP service:



Select koa_deimos table:



Under "Enter Constraints," select Spatial, and enter NGC3109 in "Coordinates or Object Name". NED will resolve the position of this galaxy. Under "Radius", select degrees and then enter "1". Then click "Search" at the bottom left of the page.



The return page will include a table of metadata observations at the bottom, a 2MASS image centered on the returned observations, and a scatter plot of the positions of the observations. Visit the return page to learn about functions for interacting with the data.