PyKOA offers access to public raw science and calibration files acquired at the W. M. Keck Observatory Archive, and for Keck Observatory PIs, secure access to their protected data with the KOA credentials assigned to them. PyKOA also supports queries to the nexsciTAP Table Access Protocol (TAP) server https://github.com/Caltech-IPAC/nexsciTAP. The PyKOA client thus enables a rich variety of searches, including cone, box, polygon, or all-sky spatial searches; temporal searches; searches on program information; and complex searches on multiple attributes.
This Jupyter Notebook provides examples of how to discover and access raw science and calibration data acquired with the DEep Imaging Multi-Object Spectrograph (ESI) instrument with the methods supported by PyKOA, and examples of how Keck PIs may access their protected data.
PyKOA can be installed from PyPI:
$ pip install --upgrade pykoa
Requires Python 3.6 (or above), plus table read and write functions from Astropy. We have tested with Astropy 4.0.1. We recommend using the Anaconda Python distribution.
PyKOA supports methods for discovering and downloading public and private data archived at KOA. It writes the output metadata data to an output file, in the IPAC ASCII, VOTable, CSV or TSV data formats. A dedicated method enables downloads of data discovered through queries.
This tutorial illustrates methods for discovering and accessing public and private raw science and calibration files for ESI:
The number of records returned by each query may differ from the number returned in this Notebook because new data are made public daily.
import sys
import io
import os
from pykoa.koa import Koa
from astropy.table import Table,Column
help(Koa)
try:
os.mkdir('./output')
except:
print(" Directory exists already", flush=True)
Koa.query_date ('esi', \
'2015-09-02', \
'./output/ESI_date.tbl', overwrite=True, format='ipac' )
rec = Table.read ('./output/ESI_date.tbl',format='ipac')
print (rec)
Koa.query_datetime ('esi', \
'2015-09-01 00:00:00/2015-09-30 23:59:59', \
'./output/ESI_datetime.tbl', overwrite=True, format='ipac' )
rec = Table.read ('./output/ESI_datetime.tbl',format='ipac')
print (rec)
Koa.query_datetime ('esi', \
'2015-09-01 00:00:00/2015-09-30 23:59:59', \
'./output/ESI_datetime.vot', overwrite=True, format='votable')
rec = Table.read ('./output/ESI_datetime.vot',format='votable')
print (rec)
Koa.query_datetime ('esi', \
'2015-09-01 00:00:00/2015-09-30 23:59:59', \
'./output/ESI_datetime.csv', overwrite=True, format='csv')
rec = Table.read ('./output/ESI_datetime.csv',format='csv')
print (rec)
Koa.query_datetime ('esi', \
'2015-09-01 00:00:00/2015-09-30 23:59:59', \
'./output/ESI_datetime.tsv', overwrite=True, format='tsv')
rec = Table.read ('./output/ESI_datetime.tsv',format='ascii.fast_tab')
print (rec)
Koa.query_position ('esi', \
'circle 255.03913 -11.82214 1.0', \
'./output/position_search.tbl', overwrite=True )
rec = Table.read ('./output/position_search.tbl', format='ascii.ipac')
print (rec)
Koa.query_object ('esi', \
'M32', \
'./output/M32.tbl', overwrite=True)
rec = Table.read ('./output/M32.tbl', format='ascii.ipac')
print (rec)
query ="select koaid, filehand, progid from koa_esi where (progid = 'K197E') "
Koa.query_adql (query, \
'./output/program_info.tbl', overwrite=True, \
format='ipac')
rec = Table.read('./output/program_info.tbl', format='ascii.ipac')
print (rec)
param = dict()
param['instrument'] = 'esi'
param['datetime'] = '2014-07-24 00:00:00/2014-07-24 23:59:59'
param['pos'] = 'circle 255.03913 -11.82214 1.0'
Koa.query_criteria (param, \
'./output/parameters.tbl', overwrite=True)
rec = Table.read ('./output/parameters.tbl', format='ascii.ipac')
print (rec)
query = "select koaid, object, koaimtyp, frameno, ra, dec, \
to_char(date_obs,'YYYY-MM-DD') as date_obs, ut \
elaptime, waveblue, wavered, filter, apmsknam, slmsknam, obsmode, \
progid, proginst, progpi, progtitl, semester, ofname, filehand \
from koa_esi where \
(contains(point('J2000',ra ,dec ), circle('J2000', 23.48 ,30.60, 1))=1) \
order by utdatetime"
Koa.query_adql (query, \
'./output/adql_cone_UTorder.tbl', overwrite=True, \
format='ipac')
rec = Table.read ('./output/adql_cone_UTorder.tbl', format='ascii.ipac')
print (rec)
query = "select koaid from koa_esi where \
(contains(point('J2000',ra ,dec ), box('J2000', 23.48 ,30.60, 1, 1))=1) "
Koa.query_adql (query, \
'./output/adql_radec.tbl',overwrite=True, \
format='ipac')
rec = Table.read ('./output/adql_radec.tbl', format='ascii.ipac')
print (rec)
query = "select top 10 koaid, ra ,dec, utdatetime from koa_esi \
where (contains(point('J2000',ra ,dec), \
box('J2000', 23.48 ,30.60, 1, 1)) =1) order by utdatetime desc "
Koa.query_adql (query, \
'./output/adql_radec.tbl',overwrite=True, \
format='ipac')
rec = Table.read ('./output/adql_radec.tbl', format='ascii.ipac')
print (rec)
query = "select count(koaid) from koa_esi \
where (contains(point('J2000',ra,dec), box('J2000', 23.48 ,30.60, 1, 1 ))=1) "
Koa.query_adql (query, \
'./output/adql_count.tbl',overwrite=True, \
format='ipac')
rec = Table.read ('./output/adql_count.tbl', format='ascii.ipac')
print (rec)
query = "select koaid, filehand, ra, dec from koa_esi\
where contains(point('icrs', ra, dec), \
polygon('icrs',209.80225,53.34894,209.80225,55.34894,211.80225,54.34894)) = 1"
Koa.query_adql (query, \
'./output/polygon.tbl',overwrite=True, \
format='ipac')
rec = Table.read ('./output/polygon.tbl', format='ascii.ipac')
print (rec)
Koa.download ('./output/parameters.tbl', \
'ipac', \
'./dnload_dir1', \
start_row=1, \
end_row=3)
Koa.download ('./output/parameters.tbl', \
'ipac', \
'./dnload_dir2')
Koa.download ('./output/parameters.tbl',
'ipac', \
'dnload_dir3', \
start_row=0, \
end_row=2)
Koa.download ('./output/parameters.tbl',
'ipac', \
'dnload_dir4', \
start_row=0, \
end_row=2, \
calibfile=1)
The next query shows how a PI can login with their KOA credentials, assigned when the data were acquired, and access their protected data. The example is a query for public data to show the syntax. Please login with your KOA supplied credentials to access your private data. While logged in, you can access all public data as well. Koa.login creates the cookie file at login.
Koa.login ('./tapcookie.txt')
Include the cookiepath in your query to access your data, as in this example:
Koa.query_datetime ('esi', \
'2014-07-24 00:00:00/2014-07-24 23:59:59', \
'./output/ESI_login.tbl', overwrite=True, format='ipac', \
cookiepath='./tapcookie.txt' )
rec = Table.read ('./output/ESI_login.tbl',format='ipac')
print (rec)
Please acknowledge the use of KOA by including this text in your publications: "This research has made use of the Keck Observatory Archive (KOA), which is operated by the W. M. Keck Observatory and the NASA Exoplanet Science Institute (NExScI), under contract with the National Aeronautics and Space Administration."
Please also acknowledge the PI(s) of datasets that have been obtained through KOA, and please contact the KOA Help Desk if you publish archival data:
The Keck Observatory Archive (KOA) is a collaboration between the NASA Exoplanet Science Institute (NExScI) and the W. M. Keck Observatory (WMKO). NExScI is sponsored by NASA's Exoplanet Exploration Program, and operated by the California Institute of Technology in coordination with the Jet Propulsion Laboratory (JPL).
Need help? Submit your questions to the KOA Help Desk