PyKOA offers access to public raw science and calibration files acquired with the OSIRIS integral field spectrograph, and quick-look (level 1) reduced files derived from them. The quick-look files are intended for visualization and are not for scientific analysis. See https://koa.ipac.caltech.edu/UserGuide/OSIRIS/reduced_data_os.html for a description of the reduction. For Keck Observatory PIs, PyKOA offers secure access to their protected data with the KOA credentials assigned to them.
Queries can be launched through several dedicated methods, or through the IVOA Astronomical Data Query Langage (ADQL); ADQL queries are underpinned by nexsciTAP, a NExScI Python-based server that implements the IVOA Table Access Protocol (TAP). PyKOA enables a rich variety of searches, including cone, box, polygon, or all-sky spatial searches; temporal searches; searches on program infotmation; and complex searches on multiple attributes.
This Jupyter Notebook provides examples of the methods supported by PyKOA, and examples of how Keck PIs may access their protected data.
PyKOA can be installed from PyPI:
$ pip install --upgrade pykoa
Requires Python 3.6 (or above), plus table read and write functions from Astropy. We have tested with Astropy 4.0.1. We recommend using the Anaconda Python distribution.
PyKOA supports methods for discovering and downloading public and private data archived at KOA. It writes the output metadata data to an output file, in IPAC ASCII, VOTable, CSV or TSV data formats.
This Tutorial illustrates methods for discovering and accessing public and private raw science and calibration files for HIRES:
General, complex metadata queries using the IVOA Astronomical Data Query Langage (ADQL).
Download raw science and associated calibration files, level 1 files or a subset of data, corresponding to a collection of metadata.
Queries for protected data (available to Keck PIs only).
The number of records returned by each query may differ from the number returned in this Notebook because new data are made public daily.
import sys
import io
import os
from pykoa.koa import Koa
from astropy.table import Table,Column
help(Koa)
Help on Archive in module pykoa.koa.core object: class Archive(builtins.object) | Archive(**kwargs) | | The 'Archive' class provides functions for accessing data stored in the | Keck Observatory Archive (KOA). Queries are performed via the nexsciTAP | server. | | Keck PIs can use the KOA credentials assigned to them when data were | acquired (given at login) to search for their proprietary data. | | Example: | -------- | | import os | import sys | | from pykoa.koa import Koa | | Koa.query_datetime ('hires', '2018-03-16 00:00:00/2018-03-18 00:00:00', outpath= './meta.xml', format='ipac') | | Methods defined here: | | __init__(self, **kwargs) | 'init' method initializes the class with optional debugfile flag. | | Optional inputs: | ---------------- | debugfile: a file path for the debug output | | download(self, metapath, format, outdir, **kwargs) | 'download' method allows download of FITS files shown in the | metadata file. The same method has the option of download the | associated calibration files and level 1 files. | | *** Requirement: | | To download FITS files, the following two columns: | instrume and filehand must be included in the metadata file. | | To download associated calibrated files, the following three | columns: instrume, koaid, and filehand must be included in the | metadata file. | | | Required input: | ----- | metapath (string): a full path metadata table obtained from running | query methods | | format (string): metadata table's format: ipac, votable, csv, or tsv. | | outdir (string): the directory for depositing the returned files | | At least one of the following three optional parameters: | lev1file, lev1file, and calibfile should be set to 1.: | | | Optional input: | ---------------- | cookiepath (string): cookie file path for downloading the proprietary | KOA data; | | start_row (integer): default is start_row = 0; | | end_row (integer): default is end_row = nrows - 1 where nrows is the | number of rows in the metadata file; | | lev0file (integer): 1/0; | 1: download level0 files (i.e. raw data) in 'lev0' directory; | 0: do not download level0 files. | default is 1. | | calibfile (integer): 1/0; | 1: download calibration files; | 0: do not download calibration files. | default is 0. | | lev1file (integer): 1/0; | 1: download level1 files in 'lev1' directory; | 0: do not download level1 files. | default is 0. | | calibdir (integer): 1/0; | 1: put calibration files in their own directory named 'calib'; | 0: put the calibration files in the 'lev0' directory with other | raw, science files. | default is 1. | | login(self, cookiepath, **kwargs) | 'login' method validates a user has a valid KOA account; it takes two | 'keyword' arguments: userid and password. If the inputs are not | provided in the keyword, the login method prompts for inputs. | | Required input: | --------------- | cookiepath (string): a file path provided by the user to save | returned cookie (in login method) or to serve | as input parameter for the subsequent koa | query and download methods. | | Keyword input: | --------------- | userid (string): a valid user id assigned by KOA; | | password (string): a valid password in the KOA's user table; | | | Calling synopsis: | | koa.login (cookiepath, userid='xxxx', password='xxxxxx'), or | | koa.login (cookiepath): and the program will prompt for | userid and password | | print_data(self) | | query_adql(self, query, outpath, **kwargs) | 'query_adql' method receives a qualified ADQL query string from | the user input. | | Required Inputs: | --------------- | query (string): an ADQL query | | outpath (string): the output filename of the returned metadata table | | Optional inputs: | ---------------- | cookiepath (string): cookie file path for query the proprietary | KOA data. | | format (string): output format: votable, ipac, csv, or tsv | (default: ipac) | | maxrec (integer): maximum records to be returned | default: -1 or not specified will return all requested records | | query_criteria(self, param, outpath, **kwargs) | 'query_criteria' method allows searches of KOA by multiple | parameters specified in a python dictionary (param). | | param: a dictionary containing a list of acceptable parameters: | | instrument (required): HIRES | | datetime (string): a datetime range string in the format of | datetime1/datetime2, '/' being the separator between first | and second datetime valaues where datetime format is | 'yyyy-mm-dd hh:mm:ss' | | date (string): a date range string in the format of | date1/date2, '/' being the separator between first | and second date valaues where date format is 'yyyy-mm-dd' | | pos (string): a position string in the format of | | 1. circle ra dec radius; | | 2. polygon ra1 dec1 ra2 dec2 ra3 dec3 ra4 dec4; | | 3. box ra dec width height; | | all RA Dec in decimal degree J2000 coordinate. | | target (string): target name used in the project, this will be | searched against the database -- not SIMBAD or NED. | | outpath (string): file path for the returned metadata table | | Optional parameters: | -------------------- | cookiepath (string): cookie file path obtained via login method, only | required for querying the proprietary KOA data. | | format (string): output table format -- votable, ipac, csv, or tsv; | default: ipac | | maxrec (integer): maximum records to be returned | default: -1 or not specified will return all requested records | | query_date(self, instrument, date, outpath, **kwargs) | 'query_date' method searches KOA by 'date_obs' range | | Required Inputs: | --------------- | instrument (string): HIRES | | date (string): a date_obs string in the format of | date1/date2 where '/' separates the two date values` | of format 'yyyy-mm-dd' | | the following inputs are acceptable: | | date1/: will search data with date later than (>=) | date1 | | /date2: will search data with date earlier than (<=) | date2 | | date1: will search data with date equal to (=) date1 | | outpath (string): a full output filepath of the returned metadata | table | | e.g. | instrument = 'hires', | date = '2018-03-16/2018-03-18' | | e.g. | instrument = 'hires', | date = '2018-03-16/' | | e.g. | instrument = 'hires', | date = '/2018-03-18' | | e.g. | instrument = 'hires', | date = '2018-03-16' | | Optional inputs: | ---------------- | cookiepath (string): cookie file path for querying the proprietary | KOA data | | format (string): Output format: votable, ipac, csv, or tsv | (default: ipac) | | maxrec (integer): maximum records to be returned | default: -1 or not specified will return all requested records | | query_datetime(self, instrument, datetime, outpath, **kwargs) | 'query_datetime' method searches KOA by 'datetime' range | | Required Inputs: | --------------- | instrument (string): HIRES | | datetime (string): a datetime string in the format of | datetime1/datetime2 where '/' separates the two datetime values | of format 'yyyy-mm-dd hh:mm:ss' | | the following inputs are acceptable: | | datetime1/: will search data with datetime later than (>=) | datetime1 | | /datetime2: will search data with datetime earlier than (<=) | datetime2 | | datetime1: will search data with datetime equal to (=) datetime1 | | outpath (string): a full output filepath of the returned metadata | table | | e.g. | instrument = 'hires', | datetime = '2018-03-16 06:10:55/2018-03-18 00:00:00' | | e.g. | instrument = 'hires', | datetime = '2018-03-16 06:10:55/' | | e.g. | instrument = 'hires', | datetime = '/2018-03-18 00:00:00' | | e.g. | instrument = 'hires', | datetime = '2018-03-16 06:10:55' | | Optional inputs: | ---------------- | cookiepath (string): cookie file path for query the proprietary | KOA data | | format (string): Output format: votable, ipac, csv, or tsv | (default: ipac) | | maxrec (integer): maximum records to be returned | default: -1 or not specified will return all requested records | | query_object(self, instrument, object, outpath, **kwargs) | 'query_object' method searches KOA by 'object name' | | Required Inputs: | --------------- | | instrument: HIRES | | object (string): an object name resolvable by SIMBAD, NED, and | ExoPlanet's name_resolve; | | This method resolves the object name into coordiates to be used as the | center of the circle position search with default radius of 0.5 deg. | | e.g. | instrument = 'hires', | object = 'WD 1145+017' | | Optional Input: | --------------- | cookiepath (string): cookie file path for query the proprietary | KOA data. | | format (string): Output format: votable, ipac, csv, tsv (default: ipac) | | radius (float): search radius in deg (default = 0.5 deg) | | maxrec (integer): maximum records to be returned | default: -1 or not specified will return all requested records | | query_position(self, instrument, pos, outpath, **kwargs) | 'query_position' method searches KOA by 'position' | | Required Inputs: | --------------- | | instrument (string): HIRES | | pos (string): a position string in the format of | | 1. circle ra dec radius; | | 2. polygon ra1 dec1 ra2 dec2 ra3 dec3 ra4 dec4; | | 3. box ra dec width height; | | All RA Dec in decimal degree J2000 coordinate. | | e.g. | instrument = 'hires', | pos = 'circle 230.0 45.0 0.5' | | outpath (string): a full filepath for the returned metadata table | | Optional Input: | --------------- | cookiepath (string): cookie file path for querying the proprietary | KOA data. | | format (string): votable, ipac, csv, tsv (default: ipac) | | maxrec (integer): maximum records to be returned | default: -1 or not specified will return all requested records | | ---------------------------------------------------------------------- | Data descriptors defined here: | | __dict__ | dictionary for instance variables (if defined) | | __weakref__ | list of weak references to the object (if defined) | | ---------------------------------------------------------------------- | Data and other attributes defined here: | | astropytbl = None | | content_type = '' | | debug = 0 | | debugfname = './koa.debug' | | format = 'ipac' | | maxrec = -1 | | msg = '' | | ncaliblist = 0 | | ndnloaded = 0 | | ndnloaded_calib = 0 | | ndnloaded_lev1 = 0 | | nlev1list = 0 | | outdir = '' | | outpath = '' | | parampath = '' | | propflag = 1 | | query = '' | | status = '' | | tap = None
try:
os.mkdir('./outputOS')
except:
print(" Directory exists already", flush=True)
Koa.query_date ('osiris', \
'2019-05-29', \
'./outputOS/OSIRIS_date.tbl', overwrite=True, format='ipac' )
rec = Table.read ('./outputOS/OSIRIS_date.tbl',format='ipac')
print (rec)
submitting request... Result downloaded to file [./outputOS/OSIRIS_date.tbl] koaid instrume targname ... filter instr scale ... ---------------------- -------- ------------ ... ------- ----- ----- OS.20190529.02699.fits OSIRIS horizon lock ... Kn3 spec 0.02 OS.20190529.02800.fits OSIRIS horizon lock ... Kn3 spec 0.02 OI.20190529.03140.fits OSIRIS -- ... BrGamma imag 0.02 OI.20190529.03275.fits OSIRIS -- ... BrGamma imag 0.02 OS.20190529.03883.fits OSIRIS unknown ... Kn3 spec 0.02 OS.20190529.03951.fits OSIRIS unknown ... Kn3 spec 0.02 OS.20190529.04055.fits OSIRIS unknown ... Kn3 spec 0.02 OS.20190529.04118.fits OSIRIS unknown ... Kn3 spec 0.02 OS.20190529.04427.fits OSIRIS unknown ... Kn3 spec 0.02 OS.20190529.04531.fits OSIRIS unknown ... Kn3 spec 0.02 ... ... ... ... ... ... ... OI.20190529.53145.fits OSIRIS Sky Flat ... Kp-sAnn imag 0.02 OI.20190529.53217.fits OSIRIS Sky Flat ... Kp-sAnn imag 0.02 OI.20190529.53302.fits OSIRIS Sky Flat ... Kp-sAnn imag 0.02 OI.20190529.53373.fits OSIRIS Sky Flat ... Kp-sAnn imag 0.02 OI.20190529.53458.fits OSIRIS Sky Flat ... Kp-sAnn imag 0.02 OI.20190529.53529.fits OSIRIS Sky Flat ... Kp-sAnn imag 0.02 OI.20190529.53613.fits OSIRIS Sky Flat ... Kp-sAnn imag 0.02 OI.20190529.53685.fits OSIRIS Sky Flat ... Kp-sAnn imag 0.02 OI.20190529.53770.fits OSIRIS Sky Flat ... Kp-sAnn imag 0.02 OI.20190529.53850.fits OSIRIS Sky Flat ... Kp-sAnn imag 0.02 Length = 229 rows
Koa.query_datetime ('osiris', \
'2019-05-29 00:00:00/2019-05-31 23:59:59', \
'./outputOS/OSIRIS_datetime.tbl', overwrite=True, format='ipac')
rec = Table.read ('./outputOS/OSIRIS_datetime.tbl',format='ipac')
print (rec)
submitting request... Result downloaded to file [./outputOS/OSIRIS_datetime.tbl] koaid instrume targname object ... filter instr scale ... ---------------------- -------- ------------ -------- ... ------- ----- ----- OS.20190529.02699.fits OSIRIS horizon lock <none> ... Kn3 spec 0.02 OS.20190529.02800.fits OSIRIS horizon lock <none> ... Kn3 spec 0.02 OI.20190529.03140.fits OSIRIS -- <none> ... BrGamma imag 0.02 OI.20190529.03275.fits OSIRIS -- <none> ... BrGamma imag 0.02 OS.20190529.03883.fits OSIRIS unknown <none> ... Kn3 spec 0.02 OS.20190529.03951.fits OSIRIS unknown <none> ... Kn3 spec 0.02 OS.20190529.04055.fits OSIRIS unknown <none> ... Kn3 spec 0.02 OS.20190529.04118.fits OSIRIS unknown <none> ... Kn3 spec 0.02 OS.20190529.04427.fits OSIRIS unknown <none> ... Kn3 spec 0.02 OS.20190529.04531.fits OSIRIS unknown <none> ... Kn3 spec 0.02 ... ... ... ... ... ... ... ... OI.20190531.51610.fits OSIRIS tt288 KB190842 ... Kp imag 0.02 OI.20190531.51700.fits OSIRIS tt288 KB190842 ... Kp imag 0.02 OI.20190531.51772.fits OSIRIS tt288 KB190842 ... Kp imag 0.02 OI.20190531.51842.fits OSIRIS tt288 KB190842 ... Kp imag 0.02 OI.20190531.52132.fits OSIRIS tt205 OB161067 ... Kp imag 0.02 OI.20190531.52204.fits OSIRIS tt205 OB161067 ... Kp imag 0.02 OI.20190531.52276.fits OSIRIS tt205 OB161067 ... Kp imag 0.02 OI.20190531.52361.fits OSIRIS tt205 OB161067 ... Kp imag 0.02 OI.20190531.52434.fits OSIRIS tt205 OB161067 ... Kp imag 0.02 OI.20190531.52514.fits OSIRIS tt205 OB161067 ... Kp imag 0.02 Length = 770 rows
Koa.query_datetime ('osiris', \
'2019-05-29 00:00:00/2019-05-31 23:59:59', \
'./outputOS/osiris_datetime.vot', overwrite=True, format='votable')
submitting request... Result downloaded to file [./outputOS/osiris_datetime.vot]
Koa.query_datetime ('osiris', \
'2019-05-29 00:00:00/2019-05-31 23:59:59', \
'./outputOS/osiris_datetime.csv', overwrite=True, format='csv' )
submitting request... Result downloaded to file [./outputOS/osiris_datetime.csv]
Koa.query_datetime ('osiris', \
'2019-05-29 00:00:00/2019-05-31 23:59:59', \
'./outputOS/osiris_datetime.tsv', overwrite=True, format='tsv' )
submitting request... Result downloaded to file [./outputOS/osiris_datetime.tsv]
Koa.query_position ('osiris', \
'circle 270.4 -27.2 1.0', \
'./outputOS/position_search.tbl', overwrite=True)
rec = Table.read ('./outputOS/position_search.tbl', format='ascii.ipac')
print (rec)
submitting request... Result downloaded to file [./outputOS/position_search.tbl] koaid instrume targname object ... coadds filter instr scale ... ---------------------- -------- -------- -------- ... ------ ------ ----- ----- OS.20160813.24913.fits OSIRIS SGR2016 v5668Str ... 1 Kbb spec 0.05 OS.20160813.24983.fits OSIRIS SGR2016 v5668Str ... 1 Kbb spec 0.05 OS.20160813.25147.fits OSIRIS SGR2016 v5668Str ... 1 Hbb spec 0.05 OS.20160813.25218.fits OSIRIS SGR2016 v5668Str ... 1 Hbb spec 0.05 OS.20160813.25335.fits OSIRIS SGR2016 v5668Str ... 1 Jbb spec 0.05 OS.20160813.25407.fits OSIRIS SGR2016 v5668Str ... 1 Jbb spec 0.05 OI.20190421.48934.fits OSIRIS tt038 OB190017 ... 1 Kp imag 0.02 OI.20190421.49094.fits OSIRIS tt038 OB190017 ... 1 Kp imag 0.02 OI.20190421.49287.fits OSIRIS tt038 OB190017 ... 2 Kp imag 0.02 OI.20190421.49344.fits OSIRIS tt038 OB190017 ... 2 Kp imag 0.02 ... ... ... ... ... ... ... ... ... OI.20200903.22578.fits OSIRIS tt053 OB190017 ... 1 Kp imag 0.02 OI.20200903.22630.fits OSIRIS tt053 OB190017 ... 1 Kp imag 0.02 OI.20200903.22701.fits OSIRIS tt053 OB190017 ... 1 Kp imag 0.02 OI.20200903.22753.fits OSIRIS tt053 OB190017 ... 1 Kp imag 0.02 OI.20200903.22804.fits OSIRIS tt053 OB190017 ... 1 Kp imag 0.02 OI.20200903.22864.fits OSIRIS tt053 OB190017 ... 1 Kp imag 0.02 OI.20200903.22916.fits OSIRIS tt053 OB190017 ... 1 Kp imag 0.02 OI.20200903.22986.fits OSIRIS tt053 OB190017 ... 1 Kp imag 0.02 OI.20200903.23037.fits OSIRIS tt053 OB190017 ... 1 Kp imag 0.02 OI.20200903.23090.fits OSIRIS tt053 OB190017 ... 1 Kp imag 0.02 Length = 291 rows
query = "select koaid from koa_osiris where \
(contains(point('J2000',ra ,dec), box('J2000', 270 , 14.3, 2, 2))=1) "
Koa.query_adql (query, './outputOS/adql_radec.tbl',overwrite=True, \
format='ipac')
rec = Table.read ('./outputOS/adql_radec.tbl', format='ascii.ipac')
print (rec)
submitting request... Result downloaded to file [./outputOS/adql_radec.tbl] koaid ---------------------- OI.20190529.38043.fits OI.20190529.38089.fits OI.20190529.38286.fits OI.20190529.38381.fits OI.20190529.38444.fits OI.20190529.38523.fits OI.20190529.38583.fits OI.20190529.38632.fits OI.20190529.38667.fits OI.20190529.38778.fits ... OS.20190527.37733.fits OS.20200712.30281.fits OS.20200712.30317.fits OS.20200712.30391.fits OS.20200712.30424.fits OS.20200810.19189.fits OS.20200810.19288.fits OS.20200810.20110.fits OS.20200811.19032.fits OS.20200811.19148.fits OS.20200811.19970.fits Length = 64 rows
query = "select koaid, filehand, ra, dec from koa_osiris\
where contains(point('icrs', ra, dec), \
polygon('icrs',209.80225,53.34894,209.80225,55.34894,211.80225,54.34894)) = 1"
Koa.query_adql(query, './outputOS/polygon_os.tbl', overwrite=True, format='ipac')
rec = Table.read ('./outputOS/polygon_os.tbl', format='ascii.ipac')
print (rec)
submitting request... Result downloaded to file [./outputOS/polygon_os.tbl] koaid ... dec ---------------------- ... -------- OS.20160416.49503.fits ... 54.41594 OS.20160416.49605.fits ... 54.41594 OS.20160416.49631.fits ... 54.41594 OS.20160416.49658.fits ... 54.41594 OS.20160416.49683.fits ... 54.41594 OS.20160416.49709.fits ... 54.41594 OS.20160416.49744.fits ... 54.41594 OS.20160416.49769.fits ... 54.41594 OS.20160416.49796.fits ... 54.41594 OS.20160416.49826.fits ... 54.41594 ... ... ... OS.20170509.43779.fits ... 54.23825 OS.20170509.43806.fits ... 54.23825 OS.20170509.43836.fits ... 54.23825 OS.20170509.43864.fits ... 54.23825 OS.20170509.43890.fits ... 54.23784 OS.20170509.43920.fits ... 54.23784 OS.20170509.43948.fits ... 54.23825 OS.20170509.43974.fits ... 54.23825 OS.20170509.44001.fits ... 54.23825 OS.20170509.44027.fits ... 54.23825 OS.20170509.44083.fits ... 54.23825 Length = 215 rows
Koa.query_object ('osiris', \
'3C273', './outputOS/3C273.tbl', overwrite=True, format='ipac')
rec = Table.read ('./outputOS/3C273.tbl', format='ascii.ipac')
print (rec)
object name resolved: ra= 187.27791594, dec=2.05238823 submitting request... Result downloaded to file [./outputOS/3C273.tbl] koaid instrume targname object ... coadds filter instr scale ... ---------------------- -------- -------- ------ ... ------ ------ ----- ----- OS.20160321.36849.fits OSIRIS 3C 273 <none> ... 1 Hn3 spec 0.1 OS.20160321.36913.fits OSIRIS 3C 273 <none> ... 1 Hn3 spec 0.1 OS.20160321.38283.fits OSIRIS 3C 273 -- ... 1 Hn3 spec 0.1 OS.20160321.38907.fits OSIRIS 3C 273 <none> ... 1 Hn3 spec 0.1 OS.20160321.39420.fits OSIRIS 3C 273 <none> ... 1 Hn3 spec 0.1 OS.20160321.40046.fits OSIRIS 3C 273 <none> ... 1 Hn3 spec 0.1
query ="select koaid, filehand, progid from koa_osiris where (progid = 'N021') "
Koa.query_adql (query, \
'./outputOS/program_info.tbl', overwrite=True, format='ipac')
rec = Table.read('./outputOS/program_info.tbl', format='ascii.ipac')
print (rec)
submitting request... Result downloaded to file [./outputOS/program_info.tbl] koaid ... progid ---------------------- ... ------ OI.20190529.09103.fits ... N021 OI.20190529.09810.fits ... N021 OI.20190529.10083.fits ... N021 OI.20190529.10304.fits ... N021 OI.20190529.10373.fits ... N021 OI.20190529.10484.fits ... N021 OI.20190529.10526.fits ... N021 OI.20190529.10568.fits ... N021 OI.20190529.10609.fits ... N021 OI.20190529.10650.fits ... N021 ... ... ... OI.20190531.51540.fits ... N021 OI.20190531.51610.fits ... N021 OI.20190531.51700.fits ... N021 OI.20190531.51772.fits ... N021 OI.20190531.51842.fits ... N021 OI.20190531.52132.fits ... N021 OI.20190531.52204.fits ... N021 OI.20190531.52276.fits ... N021 OI.20190531.52361.fits ... N021 OI.20190531.52434.fits ... N021 OI.20190531.52514.fits ... N021 Length = 549 rows
param = dict()
param['instrument'] = 'osiris'
param['datetime'] = '2016-08-13 00:00:00/2016-08-13 23:59:59'
param['pos'] = 'circle 270.4 -27.2 1.0'
Koa.query_criteria (param, \
'./outputOS/parameters.tbl', overwrite=True)
rec = Table.read('./outputOS/parameters.tbl', format='ipac')
print (rec)
submitting request... Result downloaded to file [./outputOS/parameters.tbl] koaid instrume targname object ... coadds filter instr scale ... ---------------------- -------- -------- -------- ... ------ ------ ----- ----- OS.20160813.24913.fits OSIRIS SGR2016 v5668Str ... 1 Kbb spec 0.05 OS.20160813.24983.fits OSIRIS SGR2016 v5668Str ... 1 Kbb spec 0.05 OS.20160813.25147.fits OSIRIS SGR2016 v5668Str ... 1 Hbb spec 0.05 OS.20160813.25218.fits OSIRIS SGR2016 v5668Str ... 1 Hbb spec 0.05 OS.20160813.25335.fits OSIRIS SGR2016 v5668Str ... 1 Jbb spec 0.05 OS.20160813.25407.fits OSIRIS SGR2016 v5668Str ... 1 Jbb spec 0.05
query = "select koaid, object, koaimtyp, frameno, airmass, ra dec, \
to_char(date_obs,'YYYY-MM-DD') as date_obs, ha, az, gain, imagmode, \
filter,elaptime, \
progid, proginst, progpi, progtitl, semester, ofname, filehand \
from koa_osiris \
where (contains(point('J2000',ra ,dec), circle('J2000', 23.48 ,30.60, 2))=1) \
order by utdatetime"
Koa.query_adql(query,'./outputOS/adql_cone.tbl', overwrite=True, format='ipac')
rec = Table.read ('./outputOS/adql_cone.tbl', format='ascii.ipac')
print (rec)
submitting request... Result downloaded to file [./outputOS/adql_cone.tbl] koaid ... ... ---------------------- ... OS.20111003.26297.fits ... OS.20111003.26363.fits ... OS.20111003.26769.fits ... OS.20161105.26394.fits ... OS.20161105.26477.fits ... OS.20161105.26558.fits ... OS.20171102.28489.fits ... OS.20171102.28587.fits ... OS.20171102.29827.fits ... OS.20171102.29922.fits ... ... ... OS.20191107.21784.fits ... OS.20200902.45082.fits ... OS.20200902.45110.fits ... OS.20200902.45133.fits ... OS.20200902.45154.fits ... OS.20200902.45180.fits ... OS.20200902.45203.fits ... OS.20200902.45226.fits ... OS.20200902.45252.fits ... OS.20200902.45276.fits ... Length = 86 rows
query = "select top 10 koaid, ra ,dec, utdatetime from koa_osiris \
where (contains(point('J2000',ra ,dec), \
box('J2000', 270 , 14.3, 2, 2))) =1) order by utdatetime desc "
Koa.query_adql (query, \
'./outputOS/adql_radec.tbl',overwrite=True, \
format='ipac')
rec = Table.read ('./outputOS/adql_radec.tbl', format='ascii.ipac')
print (rec)
submitting request... Result downloaded to file [./outputOS/adql_radec.tbl] koaid ra dec utdatetime ---------------------- --------- ------- -------------------------- OS.20170813.21294.fits 269.91293 14.3221 2017-08-13 05:54:54.430000 OS.20170813.21261.fits 269.91281 14.3221 2017-08-13 05:54:21.080000 OS.20170813.19982.fits 269.91293 14.3221 2017-08-13 05:33:02.050000 OS.20170813.19944.fits 269.91281 14.3221 2017-08-13 05:32:24.600000 OS.20170813.19416.fits 269.91293 14.3221 2017-08-13 05:23:36.610000 OS.20170813.19383.fits 269.91281 14.3221 2017-08-13 05:23:03.610000 OS.20170813.19208.fits 269.91281 14.3221 2017-08-13 05:20:08.510000 OS.20170812.20259.fits 269.91293 14.3221 2017-08-12 05:37:39.690000 OS.20170812.19339.fits 269.91293 14.3221 2017-08-12 05:22:19.090000 OS.20170812.19285.fits 269.91281 14.3221 2017-08-12 05:21:25.540000
query = "select count(koaid) from koa_osiris \
where (contains(point('J2000',ra,dec), box('J2000', 270 , 14.3, 2, 2)))=1) "
Koa.query_adql (query, \
'./outputOS/adql_count.tbl',overwrite=True, \
format='ipac')
rec = Table.read ('./outputOS/adql_count.tbl', format='ascii.ipac')
print (rec)
submitting request... Result downloaded to file [./outputOS/adql_count.tbl] count(koaid) ------------ 64
Koa.download ('./outputOS/parameters.tbl', \
'ipac', \
'./dnload_dir1', \
start_row=1, \
end_row=3)
Start downloading 3 koaid data you requested; please check your outdir: ./dnload_dir1 for progress .... A total of 3 new lev0 FITS files downloaded.
Koa.download ('./outputOS/parameters.tbl', \
'ipac', \
'./dnload_dir1')
Start downloading 6 koaid data you requested; please check your outdir: ./dnload_dir1 for progress .... A total of 3 new lev0 FITS files downloaded.
Koa.query_date ('osiris', '2019-10-09', \
'./outputOS/OSIRIS_date1009.tbl', overwrite=True, format='ipac' )
rec = Table.read ('./outputOS/OSIRIS_date1009.tbl',format='ipac')
print (rec)
submitting request... Result downloaded to file [./outputOS/OSIRIS_date1009.tbl] koaid instrume targname object ... coadds filter instr scale ... ---------------------- -------- -------- ------- ... ------ ------ ----- ----- OS.20191009.09500.fits OSIRIS unknown HD21875 ... 1 Drk spec 0.035 OS.20191009.09527.fits OSIRIS unknown HD21875 ... 1 Drk spec 0.035 OS.20191009.09555.fits OSIRIS unknown HD21875 ... 1 Drk spec 0.035 OS.20191009.09582.fits OSIRIS unknown HD21875 ... 1 Drk spec 0.035 OS.20191009.09608.fits OSIRIS unknown HD21875 ... 1 Drk spec 0.035 OS.20191009.09635.fits OSIRIS unknown HD21875 ... 1 Drk spec 0.035 OS.20191009.09684.fits OSIRIS unknown HD21875 ... 1 Drk spec 0.035 OS.20191009.09730.fits OSIRIS unknown HD21875 ... 1 Drk spec 0.035 OS.20191009.09778.fits OSIRIS unknown HD21875 ... 1 Drk spec 0.035 OS.20191009.09825.fits OSIRIS unknown HD21875 ... 1 Drk spec 0.035 ... ... ... ... ... ... ... ... ... OI.20191009.54475.fits OSIRIS tt015 <none> ... 1 Kp imag 0.02 OS.20191009.54745.fits OSIRIS tt015 <none> ... 1 Kn3 spec 0.05 OI.20191009.54750.fits OSIRIS tt015 <none> ... 1 Kp imag 0.02 OI.20191009.54897.fits OSIRIS tt015 <none> ... 1 Kp imag 0.02 OI.20191009.55035.fits OSIRIS tt015 <none> ... 1 Kp imag 0.02 OI.20191009.55173.fits OSIRIS tt015 <none> ... 1 Kp imag 0.02 OI.20191009.55310.fits OSIRIS tt015 <none> ... 1 Kp imag 0.02 OI.20191009.55448.fits OSIRIS tt015 <none> ... 1 Kp imag 0.02 OS.20191009.55889.fits OSIRIS hd 27016 <none> ... 1 Kn3 spec 0.05 OS.20191009.55933.fits OSIRIS hd 27016 <none> ... 1 Kn3 spec 0.05 Length = 66 rows
Koa.download ('./outputOS/OSIRIS_date1009.tbl', \
'ipac', \
'dnload_dir_osiris_calib1', \
start_row=35, \
end_row=38, \
lev1file=1 )
Start downloading 4 koaid data you requested; please check your outdir: dnload_dir_osiris_calib1 for progress .... No level 1 data found for koaid: [OI.20191009.51811.fits] No level 1 data found for koaid: [OI.20191009.51956.fits] A total of 4 new lev0 FITS files downloaded. 4 new lev1 list downloaded. 8 new lev1 files downloaded.
Koa.download ('./outputOS/OSIRIS_date1009.tbl', \
'ipac', \
'dnload_dir_osiris_calib1', \
start_row=35, \
end_row=40, \
lev1file=1, \
calibfile=1, \
calibdir=1 )
Start downloading 6 koaid data you requested; please check your outdir: dnload_dir_osiris_calib1 for progress .... No level 1 data found for koaid: [OI.20191009.51811.fits] No level 1 data found for koaid: [OI.20191009.51956.fits] No level 1 data found for koaid: [OI.20191009.52093.fits] No level 1 data found for koaid: [OI.20191009.52230.fits] A total of 2 new lev0 FITS files downloaded. 2 new lev1 list downloaded. 0 new lev1 files downloaded. 6 new calibration list downloaded. 48 new calibration FITS files downloaded.
Koa.download ('./outputOS/OSIRIS_date1009.tbl', \
'ipac', \
'dnload_dir_osiris_calib0', \
start_row=35, \
end_row=38, \
lev1file=1, \
calibfile=1, \
calibdir=0 )
Start downloading 4 koaid data you requested; please check your outdir: dnload_dir_osiris_calib0 for progress .... No level 1 data found for koaid: [OI.20191009.51811.fits] No level 1 data found for koaid: [OI.20191009.51956.fits] A total of 4 new lev0 FITS files downloaded. 4 new lev1 list downloaded. 8 new lev1 files downloaded. 4 new calibration list downloaded. 48 new calibration FITS files downloaded.
import json
import pandas as pd
filepath = './dnload_dir_osiris_calib1/calib/OI.20191009.51811.caliblist.json'
fp = None
jsondata = None
try:
with open (filepath) as fp:
jsondata = json.load (fp)
fp.close()
except Exception as e:
print (f'read input jsonfile error: {str(e):s}')
sys.exit()
status = ''
nrec = -1
tbldata = None
try:
status = jsondata['status']
except Exception as e:
print ('Exception extracting keyword "status"')
try:
nrec = int(jsondata['nrec'])
except Exception as e:
print ('Exception extracting keyword "nrec"')
try:
tbldata = jsondata['table']
except Exception as e:
print ('Exception extracting keyword "table"')
print (f'status= {status:s}')
print (f'nrec= {nrec:d}')
df = pd.json_normalize (tbldata)
print (df)
status= ok nrec= 10 koaid ofname instrument targname object \ 0 OI.20191008.11975.fits i191008_a003042.fits OSIRIS unknown <none> 1 OI.20191008.12128.fits i191008_a003043.fits OSIRIS unknown <none> 2 OI.20191008.12281.fits i191008_a003044.fits OSIRIS unknown <none> 3 OI.20191008.12434.fits i191008_a003045.fits OSIRIS unknown <none> 4 OI.20191008.12900.fits i191008_a003048.fits OSIRIS unknown <none> 5 OI.20191008.12747.fits i191008_a003047.fits OSIRIS unknown <none> 6 OI.20191008.13053.fits i191008_a003049.fits OSIRIS unknown <none> 7 OI.20191008.13206.fits i191008_a003050.fits OSIRIS unknown <none> 8 OI.20191008.13360.fits i191008_a003051.fits OSIRIS unknown <none> 9 OI.20191008.12594.fits i191008_a003046.fits OSIRIS unknown <none> koaimtyp frameno instr filter coadds ... ut elaptime progid \ 0 dark 42 imag Drk 1 ... 03:19:35.60 119.49700 N129 1 dark 43 imag Drk 1 ... 03:22:08.60 119.49700 N129 2 dark 44 imag Drk 1 ... 03:24:41.09 119.49700 N129 3 dark 45 imag Drk 1 ... 03:27:14.60 119.49700 N129 4 dark 48 imag Drk 1 ... 03:35:00.23 119.49700 N129 5 dark 47 imag Drk 1 ... 03:32:27.74 119.49700 N129 6 dark 49 imag Drk 1 ... 03:37:33.74 119.49700 N129 7 dark 50 imag Drk 1 ... 03:40:06.74 119.49700 N129 8 dark 51 imag Drk 1 ... 03:42:40.76 119.49700 N129 9 dark 46 imag Drk 1 ... 03:29:54.74 119.49700 N129 proginst progpi progtitl \ 0 NASA Eisenhardt Mapping Feedback and Star Formation in a Hyper... 1 NASA Eisenhardt Mapping Feedback and Star Formation in a Hyper... 2 NASA Eisenhardt Mapping Feedback and Star Formation in a Hyper... 3 NASA Eisenhardt Mapping Feedback and Star Formation in a Hyper... 4 NASA Eisenhardt Mapping Feedback and Star Formation in a Hyper... 5 NASA Eisenhardt Mapping Feedback and Star Formation in a Hyper... 6 NASA Eisenhardt Mapping Feedback and Star Formation in a Hyper... 7 NASA Eisenhardt Mapping Feedback and Star Formation in a Hyper... 8 NASA Eisenhardt Mapping Feedback and Star Formation in a Hyper... 9 NASA Eisenhardt Mapping Feedback and Star Formation in a Hyper... filehand semid propint \ 0 /koadata23/OSIRIS/20191008/lev0/OI.20191008.11... 2019B_N129 18 1 /koadata23/OSIRIS/20191008/lev0/OI.20191008.12... 2019B_N129 18 2 /koadata23/OSIRIS/20191008/lev0/OI.20191008.12... 2019B_N129 18 3 /koadata23/OSIRIS/20191008/lev0/OI.20191008.12... 2019B_N129 18 4 /koadata23/OSIRIS/20191008/lev0/OI.20191008.12... 2019B_N129 18 5 /koadata23/OSIRIS/20191008/lev0/OI.20191008.12... 2019B_N129 18 6 /koadata23/OSIRIS/20191008/lev0/OI.20191008.13... 2019B_N129 18 7 /koadata23/OSIRIS/20191008/lev0/OI.20191008.13... 2019B_N129 18 8 /koadata23/OSIRIS/20191008/lev0/OI.20191008.13... 2019B_N129 18 9 /koadata23/OSIRIS/20191008/lev0/OI.20191008.12... 2019B_N129 18 diff_date 0 1 1 1 2 1 3 1 4 1 5 1 6 1 7 1 8 1 9 1 [10 rows x 27 columns]
The next query shows how a PI can login with their KOA credentials, assigned when the data were acquired, and access their protected data. The example is a query for public data to show the syntax. Please login with your KOA supplied credentials to access your private data. While logged in, you can access all public data as well. Koa.login creates the cookie file at login.
Koa.login ('./tapcookie.txt')
Userid: tapdemo Password: ········ Successfully login as tapdemo
Include the cookiepath in your query to access your data, as in this example:
Koa.query_datetime ('osiris', \
'2019-05-28 00:00:00/2019-05-31 23:59:59', \
'./outputOS/osiris_login.tbl', overwrite=True, format='ipac', \
cookiepath='./tapcookie.txt' )
rec = Table.read ('./outputOS/osiris_login.tbl',format='ipac')
print (rec)
submitting request... Result downloaded to file [./outputOS/osiris_login.tbl] koaid instrume targname object ... filter instr scale ... ---------------------- -------- ------------ -------- ... ------ ----- ----- OS.20190528.01189.fits OSIRIS horizon lock <none> ... Kn3 spec 0.02 OS.20190528.01319.fits OSIRIS horizon lock <none> ... Kn3 spec 0.02 OS.20190528.02504.fits OSIRIS -- <none> ... Kn3 spec 0.02 OS.20190528.02692.fits OSIRIS unknown <none> ... Kn3 spec 0.02 OS.20190528.02790.fits OSIRIS unknown <none> ... Kn3 spec 0.02 OS.20190528.03606.fits OSIRIS unknown <none> ... Kn3 spec 0.02 OS.20190528.04933.fits OSIRIS unknown dark ... Kbb spec 0.05 OS.20190528.06168.fits OSIRIS unknown dark ... Moving spec 0.05 OS.20190528.06858.fits OSIRIS unknown dark ... Drk spec 0.05 OS.20190528.07194.fits OSIRIS unknown dark ... Drk spec 0.05 ... ... ... ... ... ... ... ... OI.20190531.51610.fits OSIRIS tt288 KB190842 ... Kp imag 0.02 OI.20190531.51700.fits OSIRIS tt288 KB190842 ... Kp imag 0.02 OI.20190531.51772.fits OSIRIS tt288 KB190842 ... Kp imag 0.02 OI.20190531.51842.fits OSIRIS tt288 KB190842 ... Kp imag 0.02 OI.20190531.52132.fits OSIRIS tt205 OB161067 ... Kp imag 0.02 OI.20190531.52204.fits OSIRIS tt205 OB161067 ... Kp imag 0.02 OI.20190531.52276.fits OSIRIS tt205 OB161067 ... Kp imag 0.02 OI.20190531.52361.fits OSIRIS tt205 OB161067 ... Kp imag 0.02 OI.20190531.52434.fits OSIRIS tt205 OB161067 ... Kp imag 0.02 OI.20190531.52514.fits OSIRIS tt205 OB161067 ... Kp imag 0.02 Length = 969 rows
Please acknowledge the use of KOA by including this text in your publications: "This research has made use of the Keck Observatory Archive (KOA), which is operated by the W. M. Keck Observatory and the NASA Exoplanet Science Institute (NExScI), under contract with the National Aeronautics and Space Administration."
Please also acknowledge the PI(s) of datasets that have been obtained through KOA, and please contact the KOA Help Desk if you publish archival data:
The Keck Observatory Archive (KOA) is a collaboration between the NASA Exoplanet Science Institute (NExScI) and the W. M. Keck Observatory (WMKO). NExScI is sponsored by NASA's Exoplanet Exploration Program, and operated by the California Institute of Technology in coordination with the Jet Propulsion Laboratory (JPL).
Need help? Submit your questions to the KOA Help Desk