Keck Observatory Archive (KOA) Python Client - Tutorial: Access to LWS Raw and Reduced Data and NIRC Raw Data

July 2022 - PyKOA 1.7

PyKOA offers access to public data acquired at the W. M. Keck Observatory Archive, and for Keck Observatory PIs, secure access to their protected data with the KOA credentials assigned to them.

Queries can be launched through several dedicated methods, or through the IVOA Astronomical Data Query Langage (ADQL); ADQL queries are underpinned by nexsciTAP, a NExScI Python-based server that implements the IVOA Table Access Protocol (TAP). PyKOA enables a rich variety of searches, including cone, box, polygon, or all-sky spatial searches; temporal searches; searches on program infotmation; and complex searches on multiple attributes.

This Jupyter Notebook provides examples of how to discover and access raw science and calibration data and quick-look reduced data acquired with the Long Wavelength Spectrometer (LWS) or raw science and calibration data acquired with the Near Infrared Camera (NIRC). The quick-look reduced LWS data are intended for visualization and not for scientific analysis. See https://koa.ipac.caltech.edu/UserGuide/LWS/reduced_data.html for details of the reduction

Installation

PyKOA can be installed from PyPI:

$ pip install --upgrade pykoa

Requirements

Requires Python 3.6 (or above), plus table read and write functions from Astropy. We have tested with Astropy 4.0.1. We recommend using the Anaconda Python distribution.

Overview of this Tutorial

PyKOA supports methods for discovering and downloading public and private data archived at KOA. It writes the output metadata data to an output file, in IPAC ASCII, VOTable, CSV or TSV data formats.

This Tutorial illustrates methods for discovering and accessing public and private raw science and calibration files for HIRES:

• Query by date or date range (with examples for each file format).
• Query by position (cone, box, polygon searches)
• Query by object.
• Query by program information.
• Query by combinations of the above methods.

• General, complex metadata queries using the IVOA Astronomical Data Query Langage (ADQL).

• Download raw science and associated calibration files, level 1 files or a subset of data, corresponding to a collection of metadata.

• Queries for protected data (available to Keck PIs only).

Set up

print("import sys")
print("import io")
print("import os")
print("from pykoa.koa import Koa") 
print("from astropy.table import Table,Column")
print(" ")
print('─' * 10)
print(" ")

import sys
import io
import os
from pykoa.koa import Koa 
from astropy.table import Table,Column

import sys
import io
import os
from pykoa.koa import Koa
from astropy.table import Table,Column
 
──────────
 

---

View the help file

help(Koa)

Help on Archive in module pykoa.koa.core object:

class Archive(builtins.object)
 |  Archive(**kwargs)
 |  
 |  The 'Archive' class provides functions for accessing data stored in the 
 |  Keck Observatory Archive (KOA). Queries are performed via the nexsciTAP
 |  server.
 |  
 |  Keck PIs can use the KOA credentials assigned to them when data were 
 |  acquired (given at login) to search for their proprietary data.
 |  
 |  Example:
 |  --------
 |  
 |  import os
 |  import sys 
 |  
 |  from pykoa.koa import Koa 
 |  
 |  Koa.query_datetime ('hires',         '2018-03-16 00:00:00/2018-03-18 00:00:00',         outpath= './meta.xml',         format='ipac')
 |  
 |  Methods defined here:
 |  
 |  __init__(self, **kwargs)
 |      'init' method initializes the class with optional debugfile flag.
 |      
 |      Optional inputs:
 |      ----------------
 |      debugfile: a file path for the debug output
 |  
 |  download(self, metapath, format, outdir, **kwargs)
 |      'download' method allows download of FITS files shown in the 
 |      metadata file. The same method has the option of download the
 |      associated calibration files and level 1 files. 
 |      
 |      *** Requirement: 
 |          
 |          To download FITS files, the following two columns: 
 |          instrume and filehand must be included in the metadata file.
 |      
 |          To download associated calibrated files, the following three 
 |          columns: instrume, koaid, and filehand must be included in the 
 |          metadata file.
 |      
 |      
 |      Required input:
 |      -----
 |      metapath (string): a full path metadata table obtained from running
 |                query methods    
 |      
 |      format (string):   metadata table's format: ipac, votable, csv, or tsv.
 |      
 |      outdir (string):   the directory for depositing the returned files      
 |      
 |      At least one of the following three optional parameters:
 |      lev1file, lev1file, and calibfile should be set to 1.:
 |      
 |      
 |      Optional input:
 |      ----------------
 |      cookiepath (string): cookie file path for downloading the proprietary 
 |                           KOA data;
 |      
 |      start_row (integer): default is start_row = 0;
 |      
 |      end_row (integer): default is end_row = nrows - 1 where nrows is the 
 |                         number of rows in the metadata file;
 |      
 |      lev0file (integer): 1/0; 
 |          1: download level0 files (i.e. raw data) in 'lev0' directory;
 |          0: do not download level0 files.
 |          default is 1.
 |      
 |      calibfile (integer): 1/0;
 |          1: download calibration files;
 |          0: do not download calibration files.
 |          default is 0.
 |      
 |      lev1file (integer): 1/0;
 |          1: download level1 files in 'lev1' directory;
 |          0: do not download level1 files.
 |          default is 0.
 |      
 |      calibdir (integer): 1/0; 
 |          1: put calibration files in their own directory named 'calib'; 
 |          0: put the calibration files in the 'lev0' directory with other 
 |              raw, science files.
 |          default is 1.
 |  
 |  login(self, cookiepath, **kwargs)
 |      'login' method validates a user has a valid KOA account; it takes two
 |      'keyword' arguments: userid and password. If the inputs are not 
 |      provided in the keyword, the login method prompts for inputs.
 |      
 |      Required input:
 |      ---------------     
 |      cookiepath (string): a file path provided by the user to save 
 |               returned cookie (in login method) or to serve
 |               as input parameter for the subsequent koa 
 |               query and download methods.
 |      
 |      Keyword input:
 |      ---------------     
 |      userid     (string): a valid user id assigned by KOA;
 |      
 |      password   (string): a valid password in the KOA's user table; 
 |      
 |      
 |      Calling synopsis: 
 |      
 |      koa.login (cookiepath, userid='xxxx', password='xxxxxx'), or
 |      
 |      koa.login (cookiepath): and the program will prompt for 
 |                               userid and password
 |  
 |  print_data(self)
 |  
 |  query_adql(self, query, outpath, **kwargs)
 |      'query_adql' method receives a qualified ADQL query string from
 |      the user input.
 |      
 |      Required Inputs:
 |      ---------------    
 |          query (string):  an ADQL query
 |      
 |          outpath (string): the output filename of the returned metadata table
 |      
 |      Optional inputs:
 |      ----------------
 |          cookiepath (string): cookie file path for query the proprietary 
 |                               KOA data.
 |      
 |          format (string):  output format: votable, ipac, csv, or tsv 
 |                   (default: ipac)
 |      
 |          maxrec (integer):  maximum records to be returned 
 |               default: -1 or not specified will return all requested records
 |  
 |  query_criteria(self, param, outpath, **kwargs)
 |      'query_criteria' method allows searches of KOA by multiple
 |      parameters specified in a python dictionary (param).
 |      
 |      param: a dictionary containing a list of acceptable parameters:
 |      
 |          instrument (required): HIRES
 |      
 |          datetime (string): a datetime range string in the format of 
 |              datetime1/datetime2, '/' being the separator between first
 |              and second datetime valaues where datetime format is 
 |              'yyyy-mm-dd hh:mm:ss'
 |          
 |          date (string): a date range string in the format of 
 |              date1/date2, '/' being the separator between first
 |              and second date valaues where date format is 'yyyy-mm-dd'
 |          
 |          pos (string): a position string in the format of 
 |      
 |              1.  circle ra dec radius;
 |      
 |              2.  polygon ra1 dec1 ra2 dec2 ra3 dec3 ra4 dec4;
 |      
 |              3.  box ra dec width height;
 |      
 |              all RA Dec in decimal degree J2000 coordinate.
 |           
 |          target (string): target name used in the project, this will be 
 |              searched against the database -- not SIMBAD or NED.
 |      
 |      outpath (string): file path for the returned metadata table 
 |      
 |      Optional parameters:
 |      --------------------
 |      cookiepath (string): cookie file path obtained via login method, only
 |                           required for querying the proprietary KOA data.
 |      
 |      format (string): output table format -- votable, ipac, csv, or tsv;
 |          default: ipac
 |          
 |      maxrec (integer):  maximum records to be returned 
 |               default: -1 or not specified will return all requested records
 |  
 |  query_date(self, instrument, date, outpath, **kwargs)
 |      'query_date' method searches KOA by 'date_obs' range
 |      
 |      Required Inputs:
 |      ---------------    
 |      instrument (string): HIRES
 |      
 |      date (string): a date_obs string in the format of 
 |          date1/date2 where '/' separates the two date values` 
 |          of format 'yyyy-mm-dd'
 |      
 |          the following inputs are acceptable:
 |      
 |          date1/: will search data with date later than (>=) 
 |                      date1
 |          
 |          /date2: will search data with date earlier than (<=)
 |                      date2
 |      
 |          date1: will search data with date equal to (=) date1
 |      
 |      outpath (string): a full output filepath of the returned metadata 
 |          table
 |      
 |      e.g. 
 |          instrument = 'hires',
 |          date = '2018-03-16/2018-03-18' 
 |      
 |      e.g. 
 |          instrument = 'hires',
 |          date = '2018-03-16/' 
 |      
 |      e.g. 
 |          instrument = 'hires',
 |          date = '/2018-03-18' 
 |      
 |      e.g. 
 |          instrument = 'hires',
 |          date = '2018-03-16' 
 |      
 |      Optional inputs:
 |      ----------------
 |      cookiepath (string): cookie file path for querying the proprietary 
 |                           KOA data
 |      
 |      format (string):  Output format: votable, ipac, csv, or tsv 
 |                        (default: ipac)
 |      
 |      maxrec (integer):  maximum records to be returned 
 |               default: -1 or not specified will return all requested records
 |  
 |  query_datetime(self, instrument, datetime, outpath, **kwargs)
 |      'query_datetime' method searches KOA by 'datetime' range
 |      
 |      Required Inputs:
 |      ---------------    
 |      instrument (string): HIRES
 |      
 |      datetime (string): a datetime string in the format of 
 |          datetime1/datetime2 where '/' separates the two datetime values
 |          of format 'yyyy-mm-dd hh:mm:ss'
 |      
 |          the following inputs are acceptable:
 |      
 |          datetime1/: will search data with datetime later than (>=) 
 |                      datetime1
 |          
 |          /datetime2: will search data with datetime earlier than (<=)
 |                      datetime2
 |      
 |          datetime1: will search data with datetime equal to (=) datetime1
 |      
 |      outpath (string): a full output filepath of the returned metadata 
 |          table
 |      
 |      e.g. 
 |          instrument = 'hires',
 |          datetime = '2018-03-16 06:10:55/2018-03-18 00:00:00' 
 |      
 |      e.g. 
 |          instrument = 'hires',
 |          datetime = '2018-03-16 06:10:55/' 
 |      
 |      e.g. 
 |          instrument = 'hires',
 |          datetime = '/2018-03-18 00:00:00' 
 |      
 |      e.g. 
 |          instrument = 'hires',
 |          datetime = '2018-03-16 06:10:55' 
 |      
 |      Optional inputs:
 |      ----------------
 |      cookiepath (string): cookie file path for query the proprietary 
 |                           KOA data
 |      
 |      format (string):  Output format: votable, ipac, csv, or tsv 
 |                        (default: ipac)
 |      
 |      maxrec (integer):  maximum records to be returned 
 |               default: -1 or not specified will return all requested records
 |  
 |  query_object(self, instrument, object, outpath, **kwargs)
 |      'query_object' method searches KOA by 'object name' 
 |      
 |      Required Inputs:
 |      ---------------    
 |      
 |      instrument: HIRES
 |      
 |      object (string): an object name resolvable by SIMBAD, NED, and
 |          ExoPlanet's name_resolve; 
 |      
 |      This method resolves the object name into coordiates to be used as the
 |      center of the circle position search with default radius of 0.5 deg.
 |      
 |      e.g. 
 |          instrument = 'hires',
 |          object = 'WD 1145+017'
 |      
 |      Optional Input:
 |      ---------------    
 |      cookiepath (string): cookie file path for query the proprietary 
 |                           KOA data.
 |      
 |      format (string):  Output format: votable, ipac, csv, tsv (default: ipac)
 |      
 |      radius (float): search radius in deg (default = 0.5 deg)
 |      
 |      maxrec (integer):  maximum records to be returned 
 |               default: -1 or not specified will return all requested records
 |  
 |  query_position(self, instrument, pos, outpath, **kwargs)
 |      'query_position' method searches KOA by 'position' 
 |      
 |      Required Inputs:
 |      ---------------    
 |      
 |      instrument (string): HIRES
 |      
 |      pos (string): a position string in the format of 
 |      
 |      1.  circle ra dec radius;
 |      
 |      2.  polygon ra1 dec1 ra2 dec2 ra3 dec3 ra4 dec4;
 |      
 |      3.  box ra dec width height;
 |      
 |      All RA Dec in decimal degree J2000 coordinate.
 |           
 |      e.g. 
 |          instrument = 'hires',
 |          pos = 'circle 230.0 45.0 0.5'
 |      
 |      outpath (string): a full filepath for the returned metadata table
 |      
 |      Optional Input:
 |      ---------------    
 |      cookiepath (string): cookie file path for querying the proprietary 
 |                           KOA data.
 |      
 |      format (string): votable, ipac, csv, tsv  (default: ipac)
 |      
 |      maxrec (integer):  maximum records to be returned 
 |               default: -1 or not specified will return all requested records
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |  
 |  astropytbl = None
 |  
 |  content_type = ''
 |  
 |  debug = 0
 |  
 |  debugfname = './koa.debug'
 |  
 |  format = 'ipac'
 |  
 |  maxrec = -1
 |  
 |  msg = ''
 |  
 |  ncaliblist = 0
 |  
 |  ndnloaded = 0
 |  
 |  ndnloaded_calib = 0
 |  
 |  ndnloaded_lev1 = 0
 |  
 |  nlev1list = 0
 |  
 |  outdir = ''
 |  
 |  outpath = ''
 |  
 |  parampath = ''
 |  
 |  propflag = 1
 |  
 |  query = ''
 |  
 |  status = ''
 |  
 |  tap = None

Define instrument

Uncomment the instrument field below to select either LWS or NIRC

# Uncomment below for LWS
instrument = 'LWS'

# Uncomment for NIRC
# instrument = 'NIRC'

Define search parameters

if (instrument == 'LWS'):
    instr = 'lws'
    koatable = 'koa_lws'    
    outdir = './outputLW/'
    date = '1998-12-03'
    propdate = '1999-09-24'  # LWS has no proprietary data
    daterange = '1998-12-03/1998-12-27'
    datetimerange = '1998-12-03 00:00:00/2005-02-20 23:59:59'
    program = 'A078LS'
    pos_target = 'Aldebaran'
    targname = 'HR1457'
    circle_pos = '68.98 16.51 1.0'
    box_pos = '68.98 16.51 0.5 0.5'
    polygon_pos = '67.98,15.51,67.98,17.51,70.98,22.03'
    query_keys = "koaid, object, koaimtyp, frameno, ra, dec, to_char(date_obs,'YYYY-MM-DD') as date_obs, elaptime, progid, proginst,  progpi, progtitl, semester, ofname, filehand"

if (instrument == 'NIRC'):
    instr = 'nirc'
    koatable = 'koa_nirc'
    outdir = './outputN1/'
    date = '1995-03-16'
    propdate = '2005-11-12'  # NIRC prop date for internal testing - will reset for UserGuide 
    daterange = '1995-03-16/1995-04-01'
    datetimerange = '1995-03-16 00:00:00/1995-04-01 23:59:59'
    program = 'U41N'
    circle_pos = '162.07 12.63 1.0'
    box_pos = '162.07 12.63 0.5 0.5'
    polygon_pos = '161.07,11.63,161.07,13.63,164.07,12.63'
    pos_target = 'NGC 3384'
    targname = 'NGC3384'
    query_keys = "koaid, object, koaimtyp, frameno, ra, dec, to_char(date_obs,'YYYY-MM-DD') as date_obs, elaptime, progid, proginst,  progpi, progtitl, semester, ofname, filehand"

Create output directory

print("try:")
print("\tos.mkdir('" + outdir + "')")
print("except:")
print("\tprint (\" " + instrument + " Directory exists already\", flush=True)")
print(" ")
print('─' * 10)
print(" ")

try:
    os.mkdir(outdir)
except:
    print(instrument + " Directory exists already", flush=True)

try:
	os.mkdir('./outputLW/')
except:
	print (" LWS Directory exists already", flush=True)
 
──────────
 

Anonymous access

Query by date and date range

Query by single date; output in IPAC ASCII format (default)

print("Koa.query_date ( '" + instrument + "','" +  date +"', \\")
print("\t'" + outdir + instrument + "_date.tbl', overwrite=True, format='ipac')")
print("rec = Table.read ('" + outdir + instrument + "_date.tbl',format='ipac')")
print("print (rec)")
print(" ")
print('─' * 10)
print(" ")

Koa.query_date ( instr, date, \
            outdir + instr + "_date.tbl", overwrite=True, format='ipac')
rec = Table.read (outdir + instr + "_date.tbl",format='ipac')
print (rec)

Koa.query_date ( 'LWS','1998-12-03', \
	'./outputLW/LWS_date.tbl', overwrite=True, format='ipac')
rec = Table.read ('./outputLW/LWS_date.tbl',format='ipac')
print (rec)
 
──────────
 
submitting request...
Result downloaded to file [./outputLW/lws_date.tbl]
        koaid          instrume  targname ...   semid   propint
---------------------- -------- --------- ... --------- -------
LW.19981203.08419.fits      LWS        -- ... 1998b_eng      18
LW.19981203.13742.fits      LWS        -- ... 1998b_eng      18
LW.19981203.18286.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.19367.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.19491.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.20423.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.20446.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.20471.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.20495.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.20517.fits      LWS   betaPeg ... 1998b_eng      18
                   ...      ...       ... ...       ...     ...
LW.19981203.56388.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.56623.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.56888.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.57381.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.57719.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.58397.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.58630.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.59336.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.59487.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.59572.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.59695.fits      LWS IRC+10216 ... 1998b_eng      18
Length = 63 rows

You can also query by date and time range and request the output in VOTable, CSV and TSV formats:

print("Koa.query_date ('" + instr + "', \\")
print("\t'" + daterange +"', \\")
print("\t'" + outdir + instr + "_daterange.vot', overwrite=True, format='votable' )")

Koa.query_date (instr, \
    daterange, \
    outdir + instr + "_daterange.vot", overwrite=True, format='votable' )

Koa.query_date ('lws', \
	'1998-12-03/1998-12-27', \
	'./outputLW/lws_daterange.vot', overwrite=True, format='votable' )
submitting request...
Result downloaded to file [./outputLW/lws_daterange.vot]

print("Koa.query_datetime ('" + instr + "', \\")
print("\t'" +   datetimerange + "', \\")
print("\t'" + outdir  + instr + "_datetime.csv', overwrite=True, format='csv')")
print("rec = Table.read ('" + outdir + instr + "_datetime.csv',format='csv')")

Koa.query_datetime (instr, \
    datetimerange, \
    outdir + instr + "_datetime.csv", overwrite=True, format='csv' )

Koa.query_datetime ('lws', \
	'1998-12-03 00:00:00/2005-02-20 23:59:59', \
	'./outputLW/lws_datetime.csv', overwrite=True, format='csv')
rec = Table.read ('./outputLW/lws_datetime.csv',format='csv')
submitting request...
Result downloaded to file [./outputLW/lws_datetime.csv]

print("Koa.query_date ('" + instr + "', \\")
print("\t'" + date + "', \\")
print("\t'" + outdir + instr + "_date.tsv', overwrite=True, format='tsv')")
print("rec = Table.read ('"+ outdir + instr + "_date.tsv',format='ascii.fast_tab')")

Koa.query_date (instr, \
    date, \
    outdir + instr + "_date.tsv", overwrite=True, format='tsv' )

Koa.query_date ('lws', \
	'1998-12-03', \
	'./outputLW/lws_date.tsv', overwrite=True, format='tsv')
rec = Table.read ('./outputLW/lws_date.tsv',format='ascii.fast_tab')
submitting request...
Result downloaded to file [./outputLW/lws_date.tsv]

Query by position - cone search

print("Koa.query_position ('" + instr + "', \\")
print("\t'" + "circle " + circle_pos + "', \\")
print("\t'" + outdir + instr + "_position_search.tbl', overwrite=True )")
print("rec = Table.read ('" + outdir + instr + "_position_search.tbl', format='ascii.ipac' )")
print("print (rec)")
print(" ")
print('─' * 10)
print(" ") 
    
Koa.query_position (instr , \
    "circle " + circle_pos , \
    outdir + instr + "_position_search.tbl", overwrite=True )
rec = Table.read (outdir + instr + "_position_search.tbl", format='ascii.ipac' )
print (rec)

Koa.query_position ('lws', \
	'circle 68.98 16.51 1.0', \
	'./outputLW/lws_position_search.tbl', overwrite=True )
rec = Table.read ('./outputLW/lws_position_search.tbl', format='ascii.ipac' )
print (rec)
 
──────────
 
submitting request...
Result downloaded to file [./outputLW/lws_position_search.tbl]
        koaid          instrume targname ...    semid     propint
---------------------- -------- -------- ... ------------ -------
LW.19981203.36079.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.36927.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.37399.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.37619.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.38858.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.39808.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.39910.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.40332.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.40706.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.40997.fits      LWS alphaTau ...    1998b_eng      18
                   ...      ...      ... ...          ...     ...
LW.20041222.36791.fits      LWS HD_29139 ...  2004b_k4lsn      18
LW.20050219.26018.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050219.26119.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050219.26167.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050219.26268.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050219.26334.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050219.26436.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050220.22673.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050220.22843.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050220.29357.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050220.29528.fits      LWS  Alp Tau ... 2005a_a078ls      18
Length = 518 rows

Query by position - box search

print("query = \"select koaid, filehand, ra, dec from " + koatable + " where contains(point('icrs', ra, dec), polygon('icrs'," + polygon_pos + ")) = 1\"")
print("Koa.query_adql(query, '" + outdir + instr + "_polygon.tbl', overwrite=True, format='ipac' )")
print("rec = Table.read ('" + outdir + instr + "_polygon.tbl', format='ascii.ipac')")
print("print (rec)")
print(" ")
print('─' * 10)
print(" ")

query = "select koaid, filehand, ra, dec from " + koatable + " where contains(point('icrs', ra, dec), polygon('icrs'," + polygon_pos + ")) = 1"
Koa.query_adql(query, outdir + instr + "_polygon.tbl", overwrite=True, format='ipac' )
rec = Table.read (outdir + instr + "_polygon.tbl", format='ascii.ipac')
print (rec)

query = "select koaid, filehand, ra, dec from koa_lws where contains(point('icrs', ra, dec), polygon('icrs',67.98,15.51,67.98,17.51,70.98,22.03)) = 1"
Koa.query_adql(query, './outputLW/lws_polygon.tbl', overwrite=True, format='ipac' )
rec = Table.read ('./outputLW/lws_polygon.tbl', format='ascii.ipac')
print (rec)
 
──────────
 
submitting request...
Result downloaded to file [./outputLW/lws_polygon.tbl]
        koaid          ...   dec   
---------------------- ... --------
LW.20021113.44213.fits ... 17.86405
LW.20021113.44828.fits ... 17.86406
LW.20030220.22189.fits ...   17.525
LW.20030220.23158.fits ...   17.525
LW.20010312.19037.fits ...   17.528
LW.19990823.52297.fits ...   17.528
LW.19990823.52460.fits ...   17.528
LW.19990823.53149.fits ...   17.528
LW.19990823.53945.fits ...   17.528
LW.19990823.54734.fits ...   17.528
                   ... ...      ...
LW.19991028.50618.fits ... 17.52735
LW.19991028.50744.fits ... 17.52735
LW.19991123.44243.fits ...   17.862
LW.19991123.44614.fits ...   17.862
LW.19991122.40111.fits ...    17.53
LW.19991122.41310.fits ...   17.527
LW.19991122.38915.fits ...   17.528
LW.19991122.39504.fits ...    17.53
LW.19991122.40493.fits ...   17.527
LW.19991122.40932.fits ...   17.527
LW.19991122.41699.fits ...   17.527
Length = 41 rows

Query by position - polygon search

print("query = \"select koaid from " + koatable + " where (contains(point('J2000', ra, dec), box ('J2000'," + box_pos + "))=1)\"")
print("Koa.query_adql(query, '" + outdir + instr + "_adql_radec.tbl', overwrite=True, format = 'ipac')")
print("rec = Table.read('" + outdir + instr + "_adql_radec.tbl', format='ascii.ipac')")
print("print(rec)")
print(" ")
print('─' * 10)
print(" ") 

query = "select koaid from " + koatable + " where (contains(point('J2000', ra, dec), box ('J2000'," + box_pos + "))=1)"
Koa.query_adql(query, outdir + instr + "_adql_radec.tbl", overwrite=True, format = 'ipac')
rec = Table.read(outdir + instr + "_adql_radec.tbl", format='ascii.ipac')
print(rec)

query = "select koaid from koa_lws where (contains(point('J2000', ra, dec), box ('J2000',68.98 16.51 0.5 0.5))=1)"
Koa.query_adql(query, './outputLW/lws_adql_radec.tbl', overwrite=True, format = 'ipac')
rec = Table.read('./outputLW/lws_adql_radec.tbl', format='ascii.ipac')
print(rec)
 
──────────
 
submitting request...
Result downloaded to file [./outputLW/lws_adql_radec.tbl]
        koaid         
----------------------
LW.20050219.26119.fits
LW.20050219.26167.fits
LW.20050219.26268.fits
LW.20050219.26436.fits
LW.20050219.26018.fits
LW.20050219.26334.fits
LW.20050220.22673.fits
LW.20050220.29357.fits
LW.20050220.29528.fits
LW.20050220.22843.fits
                   ...
LW.19991123.35589.fits
LW.19990902.50388.fits
LW.19990902.50306.fits
LW.19990902.50775.fits
LW.19990902.54480.fits
LW.19990902.54570.fits
LW.19990902.54954.fits
LW.19990902.56523.fits
LW.19991122.42812.fits
LW.19991122.42980.fits
LW.19991122.47190.fits
Length = 518 rows

Query by resolvable object name

print("Koa.query_object ('" + instr + "', \\")
print("\t'" + pos_target + "', '"  + outdir + instr + "_object.tbl', radius=1.0, overwrite=True, format='ipac')")
print("rec = Table.read ('" + outdir + instr + "_object.tbl', format='ascii.ipac')")
print("print (rec)")
print(" ")
print('─' * 10)
print(" ") 

Koa.query_object (instr, \
    pos_target, outdir + instr + "_object.tbl", radius=1.0, overwrite=True, format='ipac')
rec = Table.read (outdir + instr + "_object.tbl", format='ascii.ipac')
print (rec)
print(" ")
print('─' * 10)
print(" ") 

Koa.query_object ('lws', \
	'Aldebaran', './outputLW/lws_object.tbl', radius=1.0, overwrite=True, format='ipac')
rec = Table.read ('./outputLW/lws_object.tbl', format='ascii.ipac')
print (rec)
 
──────────
 
object name resolved: ra= 68.980174, dec=+16.509201
submitting request...
Result downloaded to file [./outputLW/lws_object.tbl]
        koaid          instrume targname ...    semid     propint
---------------------- -------- -------- ... ------------ -------
LW.19981203.36079.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.36927.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.37399.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.37619.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.38858.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.39808.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.39910.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.40332.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.40706.fits      LWS alphaTau ...    1998b_eng      18
LW.19981203.40997.fits      LWS alphaTau ...    1998b_eng      18
                   ...      ...      ... ...          ...     ...
LW.20041222.36791.fits      LWS HD_29139 ...  2004b_k4lsn      18
LW.20050219.26018.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050219.26119.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050219.26167.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050219.26268.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050219.26334.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050219.26436.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050220.22673.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050220.22843.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050220.29357.fits      LWS  Alp Tau ... 2005a_a078ls      18
LW.20050220.29528.fits      LWS  Alp Tau ... 2005a_a078ls      18
Length = 518 rows
 
──────────
 

Query for program information

print("query = \"select koaid, filehand, progid from " + koatable + " where (upper(progid) = '" + program + "' )\"")
print("Koa.query_adql (query, '" + outdir + instr + "_program_info.tbl', overwrite=True, format='ipac' )")
print("rec = Table.read('" + outdir + instr + "_program_info.tbl', format='ascii.ipac')")
print("print (rec)")
print(" ")
print('─' * 10)
print(" ") 

query ="select koaid, filehand, progid from " + koatable + " where (upper(progid) = '" + program + "')" 
Koa.query_adql (query, outdir + instr + "_program_info.tbl", overwrite=True, format='ipac' )
rec = Table.read(outdir + instr + "_program_info.tbl", format='ascii.ipac')
print (rec)
    

query = "select koaid, filehand, progid from koa_lws where (upper(progid) = 'A078LS' )"
Koa.query_adql (query, './outputLW/lws_program_info.tbl', overwrite=True, format='ipac' )
rec = Table.read('./outputLW/lws_program_info.tbl', format='ascii.ipac')
print (rec)
 
──────────
 
submitting request...
Result downloaded to file [./outputLW/lws_program_info.tbl]
        koaid          ... progid
---------------------- ... ------
LW.20050219.22427.fits ... A078LS
LW.20050219.22529.fits ... A078LS
LW.20050219.23025.fits ... A078LS
LW.20050219.23128.fits ... A078LS
LW.20050219.23332.fits ... A078LS
LW.20050219.23433.fits ... A078LS
LW.20050219.23865.fits ... A078LS
LW.20050219.24609.fits ... A078LS
LW.20050219.24710.fits ... A078LS
LW.20050219.24926.fits ... A078LS
                   ... ...    ...
LW.20050220.42462.fits ... A078LS
LW.20050220.43116.fits ... A078LS
LW.20050220.43477.fits ... A078LS
LW.20050220.45889.fits ... A078LS
LW.20050220.49828.fits ... A078LS
LW.20050220.50336.fits ... A078LS
LW.20050220.50700.fits ... A078LS
LW.20050220.51124.fits ... A078LS
LW.20050220.51861.fits ... A078LS
LW.20050220.52308.fits ... A078LS
LW.20050220.53887.fits ... A078LS
Length = 338 rows

Queries by combination of the above methods

Query by instrument, date, and target

print("param = dict()")
print("param['instrument'] = '" + instr + "'")
print("param['datetime'] = '"+ datetimerange + "'")
print("param['target'] = '" + targname  + "'")
print("Koa.query_criteria (param, \\")
print("\t'" + outdir + instr + "_parameters.tbl', overwrite=True)")
print("rec = Table.read('" + outdir + instr + "_parameters.tbl', format='ipac')")
print("print (rec)")
print(" ")
print('─' * 10)
print(" ") 

param = dict()
param['instrument'] = instr
param['datetime'] = datetimerange
param['target'] = targname
Koa.query_criteria (param, \
    outdir + instr + "_parameters.tbl", overwrite=True )
rec = Table.read(outdir + instr + "_parameters.tbl", format='ipac')
print (rec)

param = dict()
param['instrument'] = 'lws'
param['datetime'] = '1998-12-03 00:00:00/2005-02-20 23:59:59'
param['target'] = 'HR1457'
Koa.query_criteria (param, \
	'./outputLW/lws_parameters.tbl', overwrite=True)
rec = Table.read('./outputLW/lws_parameters.tbl', format='ipac')
print (rec)
 
──────────
 
submitting request...
Result downloaded to file [./outputLW/lws_parameters.tbl]
        koaid          instrume targname ...    semid    propint
---------------------- -------- -------- ... ----------- -------
LW.19991220.37020.fits      LWS   HR1457 ... 1999b_u62ls      18
LW.19991220.37435.fits      LWS   HR1457 ... 1999b_u62ls      18
LW.19991221.38323.fits      LWS   HR1457 ... 1999b_u62ls      18
LW.19991221.38703.fits      LWS   HR1457 ... 1999b_u62ls      18

General Metadata Queries With the Astronomical Data Query Langage (ADQL) Method.

A query made with the ADQL method enables you to make general and complex queries against the archive. If you wish to download data discovered via the ADQL query made against KOA, you must include explicitly include the koaid, instrument , and filehandle in the query.

---

Spatial cone search query with column selection; order by UT Time.

print("query = \"select " + query_keys + " from " + koatable + " where (contains(point('J2000',ra ,dec), circle('J2000', " + circle_pos + "))=1) order by utdatetime\"")
print("Koa.query_adql(query,'" + outdir + instr + "_adql_cone.tbl', overwrite=True, format='ipac')")
print("rec = Table.read ('" + outdir + instr + "_adql_cone.tbl', format='ascii.ipac')")
print("print(rec)")
print(" ")
print('─' * 10)
print(" ") 


query =  "select " + query_keys + " from " + koatable + \
    " where (contains(point('J2000',ra ,dec), circle('J2000', " + circle_pos + "))=1) order by utdatetime"
Koa.query_adql(query, outdir + instr + "_adql_cone.tbl", overwrite=True, format='ipac')
rec = Table.read (outdir + instr + "_adql_cone.tbl", format='ascii.ipac')
print(rec)

query = "select koaid, object, koaimtyp, frameno, ra, dec, to_char(date_obs,'YYYY-MM-DD') as date_obs, elaptime, progid, proginst,  progpi, progtitl, semester, ofname, filehand from koa_lws where (contains(point('J2000',ra ,dec), circle('J2000', 68.98 16.51 1.0))=1) order by utdatetime"
Koa.query_adql(query,'./outputLW/lws_adql_cone.tbl', overwrite=True, format='ipac')
rec = Table.read ('./outputLW/lws_adql_cone.tbl', format='ascii.ipac')
print(rec)
 
──────────
 
submitting request...
Result downloaded to file [./outputLW/lws_adql_cone.tbl]
        koaid          ...                       filehand                     
---------------------- ... ---------------------------------------------------
LW.19981203.36079.fits ... /koadata31/LWS/19981203/lev0/LW.19981203.36079.fits
LW.19981203.36927.fits ... /koadata31/LWS/19981203/lev0/LW.19981203.36927.fits
LW.19981203.37399.fits ... /koadata31/LWS/19981203/lev0/LW.19981203.37399.fits
LW.19981203.37619.fits ... /koadata31/LWS/19981203/lev0/LW.19981203.37619.fits
LW.19981203.38858.fits ... /koadata31/LWS/19981203/lev0/LW.19981203.38858.fits
LW.19981203.39808.fits ... /koadata31/LWS/19981203/lev0/LW.19981203.39808.fits
LW.19981203.39910.fits ... /koadata31/LWS/19981203/lev0/LW.19981203.39910.fits
LW.19981203.40332.fits ... /koadata31/LWS/19981203/lev0/LW.19981203.40332.fits
LW.19981203.40706.fits ... /koadata31/LWS/19981203/lev0/LW.19981203.40706.fits
LW.19981203.40997.fits ... /koadata31/LWS/19981203/lev0/LW.19981203.40997.fits
                   ... ...                                                 ...
LW.20041222.36791.fits ... /koadata31/LWS/20041222/lev0/LW.20041222.36791.fits
LW.20050219.26018.fits ... /koadata31/LWS/20050219/lev0/LW.20050219.26018.fits
LW.20050219.26119.fits ... /koadata31/LWS/20050219/lev0/LW.20050219.26119.fits
LW.20050219.26167.fits ... /koadata31/LWS/20050219/lev0/LW.20050219.26167.fits
LW.20050219.26268.fits ... /koadata31/LWS/20050219/lev0/LW.20050219.26268.fits
LW.20050219.26334.fits ... /koadata31/LWS/20050219/lev0/LW.20050219.26334.fits
LW.20050219.26436.fits ... /koadata31/LWS/20050219/lev0/LW.20050219.26436.fits
LW.20050220.22673.fits ... /koadata31/LWS/20050220/lev0/LW.20050220.22673.fits
LW.20050220.22843.fits ... /koadata31/LWS/20050220/lev0/LW.20050220.22843.fits
LW.20050220.29357.fits ... /koadata31/LWS/20050220/lev0/LW.20050220.29357.fits
LW.20050220.29528.fits ... /koadata31/LWS/20050220/lev0/LW.20050220.29528.fits
Length = 518 rows

Select top 10 records in a spatial box search with column selection; order results in descending UT time.

print("query = \"select top 10 koaid, ra ,dec, utdatetime from " + koatable + " where (contains(point('J2000',ra ,dec), box ('J2000', " + box_pos + "))) =1) order by utdatetime desc \"")
print("Koa.query_adql (query, '" + outdir + instr + "_adql_radec_top10.tbl',overwrite=True, format='ipac')")
print("rec = Table.read ('" + outdir + instr + "_adql_radec_top10.tbl', format='ascii.ipac')")
print("print (rec)")
print(" ")
print('─' * 10)
print(" ")

query =  "select top 10 koaid, ra ,dec, utdatetime from " + koatable + " where (contains(point('J2000',ra ,dec), box ('J2000', " + box_pos + "))) =1) order by utdatetime desc "
Koa.query_adql (query, \
    outdir + instr + "_adql_radec_top10.tbl",overwrite=True, format='ipac') 
rec = Table.read (outdir + instr + "_adql_radec_top10.tbl", format='ascii.ipac')
print (rec)

query = "select top 10 koaid, ra ,dec, utdatetime from koa_lws where (contains(point('J2000',ra ,dec), box ('J2000', 68.98 16.51 0.5 0.5))) =1) order by utdatetime desc "
Koa.query_adql (query, './outputLW/lws_adql_radec_top10.tbl',overwrite=True, format='ipac')
rec = Table.read ('./outputLW/lws_adql_radec_top10.tbl', format='ascii.ipac')
print (rec)
 
──────────
 
submitting request...
Result downloaded to file [./outputLW/lws_adql_radec_top10.tbl]
        koaid            ra    dec           utdatetime        
---------------------- ------ ------ --------------------------
LW.20050220.29528.fits 68.976 16.509 2005-02-20 08:12:08.230000
LW.20050220.29357.fits  68.98  16.51 2005-02-20 08:09:17.880000
LW.20050220.22843.fits 68.976 16.508 2005-02-20 06:20:43.950000
LW.20050220.22673.fits  68.98  16.51 2005-02-20 06:17:53.950000
LW.20050219.26436.fits 68.978 16.508 2005-02-19 07:20:36.580000
LW.20050219.26334.fits 68.982 16.509 2005-02-19 07:18:54.330000
LW.20050219.26268.fits 68.979 16.508 2005-02-19 07:17:48.580000
LW.20050219.26167.fits 68.982 16.509 2005-02-19 07:16:07.930000
LW.20050219.26119.fits 68.979 16.508 2005-02-19 07:15:19.180000
LW.20050219.26018.fits  68.98  16.51 2005-02-19 07:13:38.380000

Count records returned in a box search

print("query = \"select count(koaid) from " + koatable + " where (contains(point('J2000',ra ,dec), box ('J2000', " + box_pos + "))) =1)\"")
print("Koa.query_adql (query, '" + outdir + instr + "_adql_count.tbl',overwrite=True, format='ipac')")
print("rec = Table.read ('" + outdir + instr + "_adql_count.tbl', format='ascii.ipac')")
print("print (rec)")
print(" ")
print('─' * 10)
print(" ")

query =  "select count(koaid) from " + koatable + " where (contains(point('J2000',ra ,dec), box ('J2000', " + box_pos + "))) =1)"
Koa.query_adql (query, outdir + instr + "_adql_count.tbl",overwrite=True, format='ipac') 
rec = Table.read (outdir + instr + "_adql_count.tbl", format='ascii.ipac')
print (rec)

query = "select count(koaid) from koa_lws where (contains(point('J2000',ra ,dec), box ('J2000', 68.98 16.51 0.5 0.5))) =1)"
Koa.query_adql (query, './outputLW/lws_adql_count.tbl',overwrite=True, format='ipac')
rec = Table.read ('./outputLW/lws_adql_count.tbl', format='ascii.ipac')
print (rec)
 
──────────
 
submitting request...
Result downloaded to file [./outputLW/lws_adql_count.tbl]
count(koaid)
------------
         518

Download Data

Download a subset of results from the "query by multiple parameters: instrument, date, and position" example above

print("Koa.download ('" + outdir + instr + "_parameters.tbl', \\")
print("\t'ipac', \\")
print("\t'./dnload_dir1', \\")
print("\tstart_row=1, \\")
print("\tend_row=3 )")
print(" ")
print('─' * 10)
print(" ")

Koa.download (outdir + instr + "_parameters.tbl", \
    'ipac', \
    './dnload_dir1', \
    start_row=1, \
    end_row=3 )

Koa.download ('./outputLW/lws_parameters.tbl', \
	'ipac', \
	'./dnload_dir1', \
	start_row=1, \
	end_row=3 )
 
──────────
 
Start downloading 3 koaid data you requested;
please check your outdir: ./dnload_dir1 for  progress ....

A total of 3 new lev0 FITS files downloaded.

Download the full data set from the "query by multiple parameters: instrument, date, and position" example above

The three files already download to this directory in the previous example will be skipped and won't be overwritten.

print("Koa.download ('" + outdir + instr + "_parameters.tbl', \\")
print("\t'ipac', \\")
print("\t'./dnload_dir1')")
print(" ")
print('─' * 10)
print(" ")

Koa.download (outdir + instr + "_parameters.tbl", \
    'ipac', \
    './dnload_dir2' )

Koa.download ('./outputLW/lws_parameters.tbl', \
	'ipac', \
	'./dnload_dir1')
 
──────────
 
Start downloading 4 koaid data you requested;
please check your outdir: ./dnload_dir2 for  progress ....

A total of 4 new lev0 FITS files downloaded.

Download LWS Level 1 files and Calibration Files

PyKOA does not support queries for level 1 data. Instead, level 1 data may be downloaded by adding the "lev1file=1" in the download method.

In this example, we download the data referenced in the metadata table returned by this query to recover data on UT Date 19981203:

First set up the output directory ...

try:
    os.mkdir('./outputLW')
except:
    print(instrument + " Directory exists already", flush=True)

LWS Directory exists already

Run the query ...

Koa.query_date ('lws','1998-12-03', \
    './outputLW/LWS_date19981203.tbl', overwrite=True, format='ipac' )

rec = Table.read ('./outputLW/LWS_date19981203.tbl',format='ipac')
print (rec)

submitting request...
Result downloaded to file [./outputLW/LWS_date19981203.tbl]
        koaid          instrume  targname ...   semid   propint
---------------------- -------- --------- ... --------- -------
LW.19981203.08419.fits      LWS        -- ... 1998b_eng      18
LW.19981203.13742.fits      LWS        -- ... 1998b_eng      18
LW.19981203.18286.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.19367.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.19491.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.20423.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.20446.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.20471.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.20495.fits      LWS   betaPeg ... 1998b_eng      18
LW.19981203.20517.fits      LWS   betaPeg ... 1998b_eng      18
                   ...      ...       ... ...       ...     ...
LW.19981203.56388.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.56623.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.56888.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.57381.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.57719.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.58397.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.58630.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.59336.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.59487.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.59572.fits      LWS IRC+10216 ... 1998b_eng      18
LW.19981203.59695.fits      LWS IRC+10216 ... 1998b_eng      18
Length = 63 rows

Including the parameter "lev1file=1" in the download request below returns the raw data and level 1 data corresponding rows 1 to 25 of the metadata table returned in the above query. The data are placed in the directory 'dnload_dir_LWS_calib1'

Koa.download ('./outputLW/LWS_date19981203.tbl', \
    'ipac',  \
    'dnload_dir_LWS_calib1', \
    start_row=1, \
    end_row=25, \
    lev1file=1 )
    

Start downloading 25 koaid data you requested;
please check your outdir: dnload_dir_LWS_calib1 for  progress ....

A total of 25 new lev0 FITS files downloaded.
25 new lev1 list downloaded.
50 new lev1 files downloaded.

Calibration files may be downloaded by adding the keyword "calibfile" to the download request.

The value of calibdir specifies the directory to which the calibration files are downloaded. In the example below, calibdir=1 (default) places calibration files in the subdirectory 'calib' under 'dnload_dir_LWS_calib1'

Choose the records to be downloaded with the start_row and end_row parameters; omit them to download all data

Koa.download ('./outputLW/LWS_date19981203.tbl', \
    'ipac',  \
    'dnload_dir_LWS_calib1', \
    start_row=1, \
    end_row=30, \
    lev1file=1, \
    calibfile=1, \
    calibdir=1 )
    

Start downloading 30 koaid data you requested;
please check your outdir: dnload_dir_LWS_calib1 for  progress ....
No associated calibration list for LW.19981203.13742.fits

A total of 5 new lev0 FITS files downloaded.
5 new lev1 list downloaded.
10 new lev1 files downloaded.
29 new calibration list downloaded.
2 new calibration FITS files downloaded.

The next example uses calibdir=0 to place the calibration files in the ‘lev0’ subdirectory under 'dnload_dir_LWS_calib0'.

Koa.download ('./outputLW/LWS_date19981203.tbl', \
    'ipac',  \
    'dnload_dir_LWS_calib0', \
    start_row=1, \
    end_row=30, \
    lev1file=1, \
    calibfile=1, \
    calibdir=0  )
    

Start downloading 30 koaid data you requested;
please check your outdir: dnload_dir_LWS_calib0 for  progress ....
No associated calibration list for LW.19981203.13742.fits

A total of 30 new lev0 FITS files downloaded.
30 new lev1 list downloaded.
60 new lev1 files downloaded.
29 new calibration list downloaded.
1 new calibration FITS files downloaded.

Lists of calibration files are downloaded in IPAC table format (.tbl) and as JSON files (.json).

The JSON files can be read as Pandas data frames with the code below. The example formats the JSON file 'LW.19981203.19367.caliblist.json' in the 'dnload_dir_LWS_calib1/calib' directory

import json

import pandas as pd

filepath = './dnload_dir_LWS_calib1/calib/LW.19981203.19367.caliblist.json'

fp = None
jsondata = None
try:
    with open (filepath) as fp:
    
        jsondata = json.load (fp)
    fp.close()

except Exception as e:

    print (f'read input jsonfile error: {str(e):s}')
    sys.exit()

status = ''
nrec = -1
tbldata = None

try:
    status = jsondata['status']

except Exception as e:
    print ('Exception extracting keyword "status"')
  

try:
    nrec = int(jsondata['nrec'])

except Exception as e:
    print ('Exception extracting keyword "nrec"')
  

try:
    tbldata = jsondata['table']

except Exception as e:
    print ('Exception extracting keyword "table"')
    
    
print (f'status= {status:s}')
print (f'nrec= {nrec:d}')

df = pd.json_normalize (tbldata)      
print (df)

status= ok
nrec= 2
                    koaid        ofname instrument targname   koaimtyp  \
0  LW.19981203.13742.fits  lws0002.fits        LWS     null  undefined   
1  LW.19981203.08419.fits  lws0001.fits        LWS     null  undefined   

  frameno                                     ra  \
0       2  0.01383000000000000027922109069322687   
1       1    15.84556000000000075544903666013852   

                                     dec    date_obs           ut  \
0  0.00600999999999999971744824023289766  1998-12-03  03:49:02.14   
1    70.26828000000000429281499236822128  1998-12-03  02:20:19.49   

       elaptime progid proginst       progpi             progtitl  \
0  0.0000000000    ENG     KECK  Engineering  NIRC backup program   
1  0.0000000000    ENG     KECK  Engineering  NIRC backup program   

                                            filehand      semid propint  \
0  /koadata31/LWS/19981203/lev0/LW.19981203.13742...  1998B_ENG      18   
1  /koadata31/LWS/19981203/lev0/LW.19981203.08419...  1998B_ENG      18   

  diff_date  
0         0  
1         0  

Login access

The next query shows how a PI can login with their KOA credentials, assigned when the data were acquired, and access their protected data. The example is a query for public data to show the syntax. Please login with your KOA supplied credentials to access your private data. While logged in, you can access all public data as well. Koa.login creates the cookie file at login.

Note: if files have been downloaded already, they will not be downloaded again and overwritten.

print ("Koa.login ('./tapcookie.txt')")
print(" ")
print('─' * 10)
print(" ")

Koa.login ('./tapcookie.txt')

Koa.login ('./tapcookie.txt')
 
──────────
 
Userid: tapdemo
Password: ········
Successfully login as tapdemo

Include the cookiepath in your query to access your data, as in this example:

print("Koa.query_date ('" + instr + "', \\")
print("\t'" + propdate + "', \\")
print("\t'" + outdir + instr + "_login.tbl', overwrite=True, format='ipac', \\")
print("\t cookiepath='./tapcookie.txt' )")
print("rec = Table.read ('" + outdir + instr + "_login.tbl',format='ipac')")
print("print (rec)")
print(" ")
print('─' * 10)
print(" ")


Koa.query_date (instr, \
    propdate, \
    outdir + instr + "_login.tbl", overwrite=True, format='ipac', \
    cookiepath='./tapcookie.txt' )
rec = Table.read (outdir + instr + "_login.tbl",format='ipac')
print (rec)

                    

Koa.query_date ('lws', \
	'1999-09-24', \
	'./outputLW/lws_login.tbl', overwrite=True, format='ipac', \
	 cookiepath='./tapcookie.txt' )
rec = Table.read ('./outputLW/lws_login.tbl',format='ipac')
print (rec)
 
──────────
 
submitting request...
Result downloaded to file [./outputLW/lws_login.tbl]
        koaid          instrume   targname   ...   semid    propint
---------------------- -------- ------------ ... ---------- -------
LW.19990924.11178.fits      LWS           -- ... 1999b_u46n     204
LW.19990924.11381.fits      LWS           -- ... 1999b_u46n     204
LW.19990924.11486.fits      LWS           -- ... 1999b_u46n     204
LW.19990924.11569.fits      LWS           -- ... 1999b_u46n     204
LW.19990924.11601.fits      LWS           -- ... 1999b_u46n     204
LW.19990924.11903.fits      LWS           -- ... 1999b_u46n     204
LW.19990924.12126.fits      LWS           -- ... 1999b_u46n     204
LW.19990924.12151.fits      LWS           -- ... 1999b_u46n     204
LW.19990924.12176.fits      LWS           -- ... 1999b_u46n     204
LW.19990924.12199.fits      LWS           -- ... 1999b_u46n     204
                   ...      ...          ... ...        ...     ...
LW.19990924.57148.fits      LWS       hr1708 ... 1999b_u46n     204
LW.19990924.57407.fits      LWS        flats ... 1999b_u46n     204
LW.19990924.57556.fits      LWS        flats ... 1999b_u46n     204
LW.19990924.57710.fits      LWS        flats ... 1999b_u46n     204
LW.19990924.57876.fits      LWS           -- ... 1999b_u46n     204
LW.19990924.58003.fits      LWS           -- ... 1999b_u46n     204
LW.19990924.58213.fits      LWS fcass STEP 1 ... 1999b_u46n     204
LW.19990924.58295.fits      LWS fcass STEP 2 ... 1999b_u46n     204
LW.19990924.58372.fits      LWS fcass STEP 2 ... 1999b_u46n     204
LW.19990924.58467.fits      LWS fcass STEP 3 ... 1999b_u46n     204
LW.19990924.58577.fits      LWS fcass STEP 3 ... 1999b_u46n     204
Length = 148 rows

---

Visit KOA at https://koa.ipac.caltech.edu

Please acknowledge the use of KOA by including this text in your publications: "This research has made use of the Keck Observatory Archive (KOA), which is operated by the W. M. Keck Observatory and the NASA Exoplanet Science Institute (NExScI), under contract with the National Aeronautics and Space Administration."

Please also acknowledge the PI(s) of datasets that have been obtained through KOA, and please contact the KOA Help Desk if you publish archival data:

KOA Help Desk

The Keck Observatory Archive (KOA) is a collaboration between the NASA Exoplanet Science Institute (NExScI) and the W. M. Keck Observatory (WMKO). NExScI is sponsored by NASA's Exoplanet Exploration Program, and operated by the California Institute of Technology in coordination with the Jet Propulsion Laboratory (JPL).

Need help? Submit your questions to the KOA Help Desk