Data Set Definitions and Calibration Association

• HIRES Archive Data Set Definitions and Calibration Association
• KCWI Calibration Association
• LRIS Calibration Association
• LRIS Calibration Association
• MOSFIRE Calibration Association
• NIRES Calibration Association
• NIRSPEC Calibration Association (for spectroscopic modes only)
• OSIRIS Calibration Association
• Calibration Association for Other Instruments (DEIMOS, ESI, KPF, LWS, and NIRC)

HIRES Archive Data Set Definitions and Calibration Association

Searches for science files can also return calibration files associated with them by selecting the option Also show calibration files that are appropriate for data reduction (see Figure 1). In this mode, KOA returns science files matching the input criteria and calibration files taken with similar instrument and CCD configurations as the science files. The calibration types are identified in the FITS headers by the KOAIMTYP keyword (see the description of calibration types).

A calibration file is associated with a science observation by comparing those keywords that define both the instrument and CCD configurations (see Table 1). A pair of science and calibration files is said to be associated if these keyword values match, within the specified tolerances (see Table 2). For bias and darks, only the CCD configuration keywords must match; for the rest of the calibration file types, both the instrument and CCD configurations must match.

The user interface returns all calibration files associated with the science observation on the night of the observation. If fewer than 5 flats, 1 bias, 1 dark or 1 arclamp are returned, the user interface will continue searching on adjacent nights until at least these numbers of each type are found (see justification for these numbers). It will not search beyond the original night for trace or focus calibration files. Specifically, it searches in +/- one day intervals, up to +/- 3 days on either side of the science observation. If the minimum number of each calibration file type is not found in the +/- 3 day interval, the user interface will return whatever matching calibration files are available in that period.

Single CCD calibration files (data taken prior to August 2004) are only associated with single CCD science files; and 3-CCD calibration files are only associated with 3-CCD science files.

Figure 1: Choosing to return science files as well as associated calibration files:

¹SLITWID and SLITWIDT are ignored when searching for associated calibration files of type "trace."

KCWI Calibration Assocation

The KCWI Configuration Manager lets observers create different configuration states based on the observations they would like to acquire. Calibration files taken to support the reduction of science files share the same configuration name (FITS keyword: STATENAM) and ID (FITS keyword: STATEID) as those science files.

When calibration association is requested at KOA, all calibration files taken within +/-3 days and having the same STATEID as the science files are returned.

LRIS Calibration Association

Association of a science file with calibration files is made by comparing certain keywords that define the LRIS instrument and CCD configurations (see Tables 3-5). A pair of science and calibration files is said to be associated if these keyword values are identical or matched within the specified tolerances, and they were taken within a day of each other.

The set of keywords that need to be matched is slightly different for dark and non-dark calibration files. For calibration files other than 'dark' and 'bias', both instrument and CCD configurations must match (Tables 3-5). Note that Table 3 is for LRIS BLUE (KOAIDs starting with LB.), and Table 4 is for LRIS RED (KOAIDs starting with LR.). For 'dark' and 'bias' files, only the CCD configuration (Table 5) need to match. The basic set of calibration files consists of at least 3 flat (KOAIMTYP=flatlamp), and one arc lamp (KOAIMTYP=arclamp). A true dark with the dark filter in (KOAIMTYP=dark) or bias (KOAIMTYP=bias) may be present, but is not necessary.

MOSFIRE Calibration Association

Association of a science file with calibration files is made by comparing certain keywords that define the MOSFIRE instrument and CCD configurations (see Tables 6-8). A pair of science and calibration files is said to be associated if these keyword values are identical or matched within the specified tolerances, and they were taken within an appropriate time range.

The set of keywords that need to match is different for spectroscopy and imaging modes (Tables 6 and 7), except for dark calibrations. Dark calibration frames taken in imaging mode can be used for spectroscopic science frames and vice versa, as long as the specified keywords match (Table 8).

The user interface returns all calibration files associated with the science observation and taken within 24 hours of the science frame. If no files of a given calibration type (arclamp, dark, flatlamp, flatlampoff) are found within that time frame, the user interface will continue searching on adjacent nights until at least one of each type (except 'dark') is found (see justification for these numbers). It searches in +/- one day intervals, up to +/- 3 days on either side of the science observation. If the minimum number of each calibration file type is not found in the +/- 3 day interval, the user interface will return whatever matching calibration files are available in that period.

NIRC2 Calibration Association

Association of a science file with calibration files is made by comparing certain keywords that define the NIRC2 configurations (see Tables 9-11). A pair of science and calibration files is said to be associated if these keyword values are identical or matched within the specified tolerances.

Both the instrument and detector configurations (Tables 9 & 10) need to be matched. The set of keywords that need to be matched is slightly different for imaging (Table 9) or spectroscopy (Table 10). The basic set of calibration files consists of at least one flat (KOAIMTYP = flatlamp), one flat with lamp off (KOAIMTYP = flatlampoff), and in the case of spectroscopy, one arc lamp (KOAIMTYPE = arclamp). Usually, an arclamp file represents Ne, Ar, Kr, Xe taken individually or in combination. A true dark with the shutter closed (KOAIMTYP = dark) may be present, but is not necessary. For this type of calibration file, only the exposure and detector configuration keywords (Table 11) need to be matched. The calibration files can be distinguished from each other and from the science files by their KOAIMTYP header keyword. A science file always has KOAIMTYP = object.

Calibration files for NIRC2 are searched for ±2 days surrounding the observation date of the science file to be associated.

NIRES Calibration Association

Because NIRES has no user-movable parts, the association of a science file with calibration files is straightforward. For each type of science file (image or spectrum), every calibration file of that same data type observed within +/-1 day will be returned when users request automatic calibration association.

NIRSPEC Calibration Association (for spectroscopic modes only)

Searches for science files can also return calibration files associated with them by selecting the option Also show appropriate calibration files that are appropriate for data reduction (see Figure 1). In this mode, KOA returns science files matching the search criteria along with calibration files that match the instrument configuration of those science files (the detector configuration for NIRSPEC cannot be changed for normal observing, so those keywords are not checked).

The basic set of calibration files consists of at least one flat (KOAIMTYP=flatlamp), one flat with lamp off (KOAIMTYP=flatlampoff), and one arc lamp (KOAIMTYP=arclamp). Usually, an arclamp file represents Ne, Ar, Kr, Xe taken individually or in combination. A true dark with the BLANK filter in the filter wheel (KOAIMTYP=dark) may be present, but is not necessary. The calibration files can be distinguished from each other and from the science files by their KOAIMTYP header keyword. A science file always has KOAIMTYP=object. For further discussion of modes, see: NIRSPEC Modes and Instrument Properties and NIRSPEC Instrument and Data Format.

In general, dark subtraction is not necessary for NIRSPEC. The dark current is very low and unless a very large number of darks are averaged, the operation of subtracting the dark image might cause a degradation of the result (due to the intrinsic noise in the master dark frame).

For scientific images, any nod pattern would automatically subtract the dark current.

For flat fields, the exposure times are too short to require dark subtraction.

If dark subtraction is considered for a specific application, note that the dark should be subtracted from the flat field images only. As a consequence, the correct darks to use are those matching the exposure parameters of the flat fields (exposure time, coadds, etc..), not the those matching the exposure parameters of the scientific images.

To allow maximum flexibility, our calibration association algorithm returns all the darks obtained during the night, so that users can make their own choice in using them for data reduction.

Association of a science file with non-dark calibration files is made by comparing certain keywords that define the NIRSPEC instrument configuration (see Table 12). A pair of science and calibration files is said to be associated if these keyword values are identical or matched within the specified tolerances.

Calibration files are associated with science files only if they are taken within +/-1 day of each other.

OSIRIS Calibration Association (for darks and flats)

Association of a science file with calibration files is made by comparing certain keywords that define the OSIRIS configurations (see Tables 13-15). A pair of science and calibration files is said to be associated if these keyword values are identical or matched within the specified tolerances.

The set of keywords that need to be matched is slightly different for dark and non-dark calibration files. For spectroscopic mode (i.e., KOAIDs beginning with OS), calibration files consist of only "dark" frames, while for imaging mode (KOAIDs beginning with OI), calibrations include darks, flats and flatlampoffs. The basic set of calibration files for imaging consists of at least one flat (KOAIMTYP=flatlamp), one flat with lamp off (KOAIMTYP=flatlampoff) (Table 13). A true dark with the dark filter in (KOAIMTYP=dark) may be present, but is not necessary (Tables 14-15).

Only calibration files taken within ±1 day surrounding the observation date of the science file will be searched for assocation.

Note: The calibration association tables returned by KOA currently do not include rectification matrices, needed to reduce OSIRIS spectroscopic data. For details on how to download the appropriate rectification matrices and use them to reduce your data, please see OSIRIS Reduced Data.

Calibration Association for Other instruments

The calibration association for DEIMOS, ESI, KPF, LWS, and NIRC is greatly simplified, matching only the date of the science files. Users can identify the proper associated calibration files themselves, using the FITS header keywords.

DEIMOS data have an additional product not needed by the other instruments. DEIMOS uses a Flexure Compensation System (FCS) to measure and compensate for real-time flexure in the instrument. These FCS reference files have the prefix "DF" and, like the DEIMOS calibration files, are associated with science files by simply matching observation dates. The KOAID of the FCS file used during an observation can be found in the science file FITS header keyword FCSKOAID. A single FCS file can be used for several science frames. It is important to note that these files are not needed for data calibration or analysis. They are only needed if you plan to re-observe a target and want to precisely reproduce the flexure characteristics of the previous observation. See more information about the DEIMOS FCS