Real Time Ingestion

April 19, 2022:

KOA now ingests raw (level 0) DEIMOS, ESI, HIRES, KCWI, and MOSFIRE observations in near-real time, generally within 5 minutes after acquisition. In the coming months, real-time ingestion will be implemented for all active instruments. The end-of-night ingestion model now used will be decommissioned. The KOA data access policy remains unchanged, so most users accessing public data will not notice this change.

Quicklook (level 1) and science-ready (level 2) reduced products are being computed for DEIMOS by the DEIMOS DRP and KCWI by the KCWI DRP. These files take advantage of the real-time ingestion infrastructure and are ingested into KOA upon completion by the pipeline.

The W.M. Keck Observatory has instituted a policy change in which calibration files that do not contain any proprietary target information (eg. darks, flats, bias frames) are assigned to engineering (PROGID=ENG) and are available for immediate public release. This change impacts archive searches based only on the Program ID. The calibration files will be made accessible for each instrument as real-time ingestion is deployed.