HDU index table

The HDU index table is stored in a FITS file as a BINTABLE HDU:

  • Suggested filename: hdu-index.fits.gz
  • Suggested HDU name: HDU_INDEX

The HDU index table can be used to locate HDUs. E.g. for a given OBS_ID and (HDU_TYPE and / or HDU_CLASS), the HDU can be located via the information in the FILE_DIR, FILE_NAME and HDU_NAME columns. The path listed in FILE_DIR has to be relative to the location of the index file.

BASE_DIR

By default, file locations should be relative to the location of this HDU index file, i.e. the total file path is PATH_INDEX_TABLE / FILE_DIR / FILE_NAME. Tools are expected to support relative file paths in POSIX notation like FILE_DIR = "../../data/" as well as absolute file path like FILE_DIR = "/data/cta".

To allow for some additional flexibility, an optional header keyword BASE_DIR can be used. If it is given, the file path is BASE_DIR / FILE_DIR / FILE_NAME, i.e. the location of the HDU index table becomes irrelevant.

Columns

Column Name Description Data type Required?
OBS_ID Observation ID (a.k.a. run number) int yes
HDU_TYPE HDU type (see below) string yes
HDU_CLASS HDU class (see below) string yes
FILE_DIR Directory of file (rel. to this file) string yes
FILE_NAME Name of file string yes
HDU_NAME Name of HDU in file string yes
SIZE File size (bytes) int no
MTIME Modification time double no
MD5 Checksum string no

Mandatory Header keywords

As explained in HDU classes, the following header keyword should be used to declare the type of HDU (this HDU itself, the HDU index table):

HDU_TYPE and HDU_CLASS

The HDU_TYPE and HDU_CLASS can be used to select the HDU of interest.

The difference is that HDU_TYPE corresponds generally to e.g. PSF, whereas HDU_CLASS corresponds to a specific PSF format. Declaring HDU_CLASS here means that tools loading these files don’t have to do guesswork to infer the format on load.

Valid HDU_TYPE values:

  • events - Event list
  • gti - Good time interval
  • aeff - Effective area
  • psf - Point spread function
  • edisp - Energy dispersion
  • bkg - Background

Valid HDU_CLASS values:

  • events - see format spec: EVENTS
  • gti - see format spec: GTI
  • aeff_2d - see format spec: AEFF_2D
  • edisp_2d - see format spec: EDISP_2D
  • psf_table - see format spec: PSF_TABLE
  • psf_3gauss - see format spec: PSF_3GAUSS
  • psf_king - see format spec: PSF_KING
  • bkg_2d - see format spec: BKG_2D
  • bkg_3d - see format spec: BKG_3D

Relation to HDUCLAS

At HDU classes and throughout this spec, HDUCLAS header keys are defined as a declarative HDU classification scheme. This appears similar to this HDU index table, but in reality is different and incompatible!

Here in the index table, we have HDU_CLASS and HDU_TYPE. In HDU classes, there is HDUCLASS which is always “GADF” and then there is a hierarchical HDUCLAS1, HDUCLAS2, HDUCLAS3 and HDUCLAS4 that corresponds to the information in HDU_CLASS and HDU_TYPE here. Also the values are different: here we have lower-case and use e.g. HDU_CLASS="aeff", in HDU classes we use upper-case and e.g. HDUCLAS2="EFF_AREA"

One reason for these inconsistencies is that the spec for this HDU index table was written first (in 2015), and then HDU classes was introduced later (in 2017). Another reason is that here, we tried to be simple (flat, not hierarchical classification), and for HDU classes we tried to follow an existing standard.

Given that these index tables have been in use in the past years, and that we expect them to be replaced soon by a completely different way to locate and link IACT data HDUs, we decided to keep this format as-is, instead of trying to align it with HDU classes and create some form of hierarchical index table.