HIRES Extracted Spectra Products

• Directory Structure
• File Types
• Binary FITS Table Descriptions
  • Flux Tables
  • Trace Tables
  • Profile Tables
  • Arclamp Identification Tables
• FITS Keywords Added to KOA Tables

This document describes the files produced by the Keck Observatory Archive's automated reduction pipeline. The products include extracted, wavelength-calibrated spectra for each order from the raw image; signal-to-noise estimates; profile traces across the spectra and the trace of the spectrum's center vs. column; and additional files generated by the MAKEE data reduction software. Many of these additional products can be used as diagnostics to check the quality of the spectral extraction.

Disclaimer: The extracted spectra available from KOA are intended to be used as a browse product only. The extracted spectra are produced by a fully automated pipeline, and may not be suitable for publishable quality science. KOA recommends that you reduce the raw images into extracted spectra yourself, paying attention to the specific kind of data that you are reducing (point source vs. extended source, the types and quantities of calibration frames, etc.).

Directory Structure

When the user chooses to include the extracted spectra, the resulting packages will contain two subdirectories: "raw" containing the raw data files and "extracted" containing the extracted spectra and related products when available. Within the "extracted" subdirectory are additional directories, shown below.

Note: if no extracted products of a particular type are available for any of the KOAIDs selected for download, that directory will not be present in your download package. (Please see "Why Aren't Extracted Spectra Available" for more information.)

File Types

binaryfits

There are four different binary FITS tables inside the binaryfits subdirectory: flux, trace, profiles, and arcids. These are discussed in detail below. Within the binaryfits subdirectory, all files are organized by CCD number.

lists

The "lists" subdirectory contains lists of raw files used to generate the master calibration frames for the reduction:

KOAID.caliblist	All calibration files associated with this science file
KOAID.flatlamp.list	Flats used to create the master flat-field frame
KOAID.bias.list	Biases used to create the master bias frame

Note that not all calibration files are necessarily used in the master frames. Only the first ten flat-field or bias frames are used to create the master. Therefore, the "caliblist" file may contain files that were not actually used in the MAKEE reduction.

The other two files list the frames actually used in KOA's automated reduction.

logs

All log files created by MAKEE are stored in the "logs" subdirectory, organized by CCD number.

makee

Inside the "makee" directory are subdirectories for each CCD, which contain wavelength calibration IDs and fit residuals ("makee/CCDN/arcs" subdirectory); MAKEE-format FITS output files ("makee/CCDN/fits" subdirectory); and measured centroids tracing the run of the spectrum across each CCD ("makee/CCDN/trace subdirectory).

previews

The "previews" subdirectory contains both PostScript and GIF plots of output products such as the flux, signal-to-noise ratio, and trace. The GIF files are produced from the PostScript plots, and are the same GIFs available from the preview pages in the HIRES search interface.

tbl

The "tbl" subdirectory contains columnar ASCII output corresponding to the binary FITS tables delivered in the "binaryfits" subdirectory. These are readable by a wide variety of tools, including spreadsheets such as Microsoft Excel or Apple's Numbers, as well as any text editors.

hdr

The FITS headers for each order are saved in the "binaryfits/ccdN/hdr" subdirectory, with filenames "KOAID_C_NN_hdr.txt" (ASCII format, with line breaks) and "KOAID_C_NN_hdr.fits" (FITS format, no line breaks).

qa

Quality Assessment (QA) grades for each CCD are stored in ASCII tables in the qa subdirectory, with filenames "KOAID_C_qa.txt." Each row contains the order number and its grade (Pass, Fail, or Unknown). These grades are also encoded in the EXQUAL FITS header keyword for each output FITS file.

Binary FITS Table Descriptions

The naming scheme for binary FITS tables is KOAID_C_NN_type.fits. KOAID is the standard file identifier for the archive; C is the CCD number (1, 2 or 3 for mosaic data; 0 for single-chip data); NN is the index order number; and type is either 'flux', 'trace', 'profile', or 'arcids'. Each of these file types is described below.

Flux Tables

• FITS format: binaryfits/ccdN/flux/KOAID_C_NN_flux.fits
• ASCII format: tbl/ccdN/flux/KOAID_C_NN_flux.tbl

col	Column number, where '0' is the first column in the internal image. (In the MAKEE program, the raw HIRES CCD image is clipped and rotated if necessary so that orders run blue-to-red and left-to-right, with the bluest order closest to row=0.)
row	Row number in the clipped (and possibly rotated) image.
raw_col	Column number in the raw image (first column is defined as zero).
raw_row	Row number in the raw image (first row is defined as zero).
wave	Wavelength in Angstroms (vacuum, heliocentric, and possibly shifted to match the wavelength zero-point of the night-sky lines).
Flux	Relative flux (in counts) after division by the (normalized) flat-field image.
Error	One-sigma error of the relative flux (Flux)
Background	Background level (in counts), scaled to match total background in region of object extraction.
Sig_to_Noise	Signal-to-noise ratio per column (one-sigma).
Flat	Level of the flat field (not normalized). The normalization scale factor is given by the 'FLATSCAL' header keyword.
Arc_Lamp	Arc lamp spectrum in counts per pixel.
Sum_Flux	Same as 'Flux', but using a simple summation rather than an optimal extraction.

Trace Tables:

• FITS format: binaryfits/ccdN/trace/KOAID_C_NN_trace.fits
• ASCII format: tbl/ccdN/trace/KOAID_C_NN_trace.tbl

col	Column number in the clipped, possibly rotated image. Only every fifth column is shown in this table.
row	Row number in the clipped, possibly rotated image.
raw_col	Column number in the raw image (first column is defined as zero).
raw_row	Row number in the raw image (first row is defined as zero).
centroid	Centroid of row value
fit	Polynomial fit to centroid of row value
cent_fit	Centroid minus fit (residual)
cent_linear	Centroid minus a straight line passing through end points of fit (plotted in the file 'previews/ccdN/KOAID_N_profiles.ps).
fit_linear	Fit minus a straight line passing through end points of fit (plotted in the file 'previews/ccdN/KOAID_N_profiles.ps).
weight	Weight of this point (usually either 1.0 or 0.0).

Profile Tables:

• FITS format: binaryfits/ccdN/profile/KOAID_C_NN_profile.fits
• ASCII format: tbl/ccdN/profile/KOAID_C_NN_profile.tbl

relative_pixel	Spatial-direction pixel value relative to the trace. The centroid of the profile will be zero only if the object profile falls at the same point on the detector as the trace object.
median_flux	Median flux per pixel in approximate counts (for the entire exposure). This value is a median of all data values at a given pixel position relative to the trace position. There is no background subtraction. The values are corrected by the mean of this order in the flat field to yield counts. Notice the profile is sampled at about 10 points per pixel.
smooth_flux	The smoothed version of the profile which is used for the spectra extraction (plotted in previews/ccdN/profile/KOAID_N_profiles.ps).

Arclamp Identification Tables:

• FITS format: binaryfits/ccdN/arcids/KOAID_C_NN_arcids.fits
• ASCII format: tbl/ccdN/arcids/KOAID_C_NN_arcids.tbl

col_cen	Column centroid of arclamp emission line.
row	Row number in the clipped, rotated image (center of order).
raw_col	Column number in the raw image (first column is defined as zero).
raw_row	Row number in the raw image (first row is defined as zero).
wave_exp	Expected wavelength position of arc line (in Angstroms and in air [15 deg / 76 cm]).
wave_fit	Wavelength of the centroid of this line in polynomial fit (in Angstroms and in air [15 deg / 76 cm]).
wave_obj	Wavelength position of this line in the object wavelength scale (which is vacuum, heliocentric and possibly shifted relative to night sky emission lines).
wave_res	Difference between 'wave_fit' and 'wave_exp'.
col_res	Difference between 'col_cen' and polynomial fit.
weight	Weight used in polynomial fit.
peak	Peak counts of this arc line.
disp	Dispersion (Angstroms/pixel) at this position.

FITS Keywords Added to KOA Tables

The following is an example of the header keywords added to KOA tables by MAKEE. Parenthetical remarks add some description beyond the in-line FITS comments.

REJ_FRAC= 0.50527 / rejection fraction (actual/expected)

(Ratio of actual masked pixels to those expected based on exposure time and size of background region.)

REJ_ADJ = 1.00000 / adjustment to rejection threshold

(Adjustment applied to rejection threshold to avoid excessive pixel masking sometimes caused by unusual object profiles. If this number is larger than 1.0, it could indicate problems with the extraction.)

REJ_SKY = 411 / masked pixels in sky region

(Total number of rejected pixels in sky, usually between 300-600 for a one hour exposure.)

REJ_OBJ = 196 / masked pixels in object region

(Total number of rejected pixels in the object region.)

INDX_ORD= 16 / index of echelle order (internal)

(Internal order number, where 1 is the bluest order.)

ECHL_ORD= 72 / true echelle order number

(Actual cross-disperser order number, based on HIRES simulator.)

TRAC_OFF= 2.33737 / centroid relative to trace (pixels)

(Centroid of primary object relative to position of fit to the trace 'star'.)

TRAC_RMS= 0.01279 / trace RMS (pixels)

TRC_FIT1= ' 2.027382562e+03 5.000000000e+00 9.346586692e+02 2.370441452e-0'
TRC_FIT2= ' 1.173676048e-06 1.970342744e-11 -7.192673190e-15 0.000000000e+0'

(Trace fit parameters. The first number is an 'x offset value'. The second number is the polynomial order. The third through eigth number are the polynomial coffecients. The row value at a given column position for this echelle order is given by: (row value) = coef(0) + ( coef(1)*(column - x offset) ) + ( coef(2)*(column - x offset)^2 ) + ...)

OBJ_FIT1= ' 2.027382562e+03 5.000000000e+00 9.369960353e+02 2.370441452e-0'
OBJ_FIT2= ' 1.173676048e-06 1.970342744e-11 -7.192673190e-15 0.000000000e+0'

(Trace fit parameters shifted to position of primary object.)

BK1O_OFF= -11.20737 / backgrnd limit 1 (outside) rel. to trace (px)
BK1I_OFF= -3.60941 / backgrnd limit 1 (inside) rel. to trace (px)
BK2I_OFF= 8.28415 / backgrnd limit 2 (inside) rel. to trace (px)
BK2O_OFF= 15.11612 / backgrnd limit 2 (outside) rel. to trace (px)

(The four values above show the background window on either side of the object relative to the position of the trace "star".)

HOBJW_PX= 4.94677 / half object width (pixels)
OBJW_PX = 9.89354 / object width (pixels)

(Size of extraction window for primary object in pixels.)

OBJW_AS = 2.35466 / object width (arcseconds)

(Size of extraction window for primary object in arcseconds.)

WAVEFIT1= ' 4.909384192E+03 2.312743750E-02 -2.196307411E-07 -6.408911426E-1'
WAVEFIT2= ' 1.267038720E-14 -2.684015663E-18 2.238792894E-22 0.000000000E+0'

(Polynomial wavelength scale fit coefficients for the object spectrum, the first value is coef(0), etc. The wavelength scale is given by: wavelength(Ang) = coef(0) + (coef(1) * column) + (coef(2) * column^2) + ...)

PFWHM_PX= 3.71222 / profile FWHM in pixels
PFWHM_AS= 0.88351 / profile FWHM in arcsec

(Rough estimate of the object full-width-half-maximum based on the peak and area of the object profile.)

PSKEW = -0.39550 / profile Skew

(Defined: Sum of {y_i * ((x_i - x_mean)/x_rms)^3} / Sum of {y_i}.)

BCKGND_1= 1.93545 / background 1 in counts per pixel
BCKGND_2= 1.76999 / background 2 in counts per pixel

(Background level on either side of object.)

FRACFLUX= 0.99041 / fractional flux extracted from profile

(Rough estimate of fraction of object extracted, usually about 98%.)

MDN_S2N = 26.86371 / median signal-to-noise per column
MIN_WAVE= 4909.38419 / minimum wavelength (usable data)
MAX_WAVE= 4996.75971 / maximum wavelength (usable data)
MIN_COL = 0 / minimum column (usable data)
MAX_COL = 4055 / maximum column (usable data)

(Range of usable spectrum for this order in Angstroms and column numbers.)

AVE_DSP = 0.02155 / average dispersion (Angstroms/pixel)

CEN_COL = 2027.50000 / central column for this order
CEN_ROW = 936.99882 / image row at central column for this order

(Center of usable range of spectrum.)

MID_COL = 2027.50000 / image middle column for this order
MID_ROW = 936.99882 / image row at middle column for this order

(Middle of spectrum based on the middle of the original full image. 'MID_COL' is the same for all orders of a given CCD.)

WFIT_RMS= 0.06630 / wavelength fit RMS
WFIT_HI = 0.17220 / wavelength fit high residual
WFIT_NUM= 33 / number of arclines used in wavelength fit
WFIT_NRJ= 0 / number of arclines rejected in wavelength fit

(Wavelength fit quality values for this order.)

ARC_FIT1= ' 4.908466241E+03 2.312340571E-02 -2.195937110E-07 -6.407760192E-1'
ARC_FIT2= ' 1.266803167E-14 -2.683514698E-18 2.238374908E-22 0.000000000E+0'

(Polynomial wavelength scale fit coeffecients for the arclamp spectrum. This is similar to the 'WAVEFIT1,2' cards, except the wavelengths are in air [15 deg/76 cm] and not heliocentric or night-sky corrected.)

MK_GAIN = 0.45455 / MAKEE gain used (inverse of e-/DN)

(The inverse of electrons per digital number card 'EPERDN'.)