# Content of the directory

## Integrity check

If you want to check the data integrity and the completeness of the directories
and sub-directories, please run the following bash script:

```bash
./integrity_check.sh
```

or any other tool to check that all files listed in `checksum.sha256` exist,
and no additional file was added. The file `checksum.sha256` includes the name
each file intented to be in the current directory and each file legitimately in
the sub-directories.

## EGBS data explanation

Most of these file contents are documented in the "Euclid: Quick Data Release (Q2) -- The Euclid Galactic Bulge Survey" (Beaulieu et al., 2026).

In the following filenames:

* {obsid} is the observation Id, it's either 067068, 067070 or 067071.
  067068 corresponds to the first 16 exposures 400s-PSF calibration field
  acquired between 2025-03-23T07:56:00 and 2025-03-23T10:06:20.
  067070 corresponds to the 144 exposures 400s scientific field.
  067071 corresponds to the second 16 exposures 400s-PSF calibration field
  acquired between 2025-03-24T08:58:00 and 2025-03-24T11:08:20.

  {obsid}=067070 corresponds to the science field, whereas {obsid}=067068 and{obsid}=067071 were used for calibration to derive a PSF model that is provided in the PSFex_model folder. 

* {dither} is the dither number, it's a sequential number between 00 and 15.

* {expnum} is the exposure sequential number inside the dither, it's always 1.

## Files in these directories are:

* EUC_VIS_SWL-DET-{obsid}-{dither}-{expnum}__*.fits
  VIS calibrated frame that includes the science image, the map of flags and
  the noise map. See below how to split all these images in different files.

* EUC_VIS_SWL-BKG-{obsid}-{dither}-{expnum}__*.fits
  background image of the corresponding calibrated frame, generated by GnuAstro
  NoiseChisel.

* EUC_VIS_SWL-WGT-{obsid}-{dither}-{expnum}__*.fits
  VIS calibrated exposure weight map. It's the master PRNU image with all
  invalid pixels set to 0.

* EUC_VIS_SWL-CAT-{obsid}-{dither}-{expnum}__*.xml
  metadata associated to the VIS calibrated frame catalogue stored in the EAS database.

* EUC_VIS_SWL-CAT-{obsid}-{dither}-{expnum}__*.fits
  calibrated frame catalogue

## Concerning the EUC_VIS_SWL-DET* data :

Each field was observed 16 times at 4 slightly different positions, each observed 4 times.  
It is possible to determine the field of observation based on the dither number. As illustrated in the image below: 
- 0:15 -> field 1
- 16:31 -> field 2
- 32:47 -> field 3
- 48:63 -> field 4
- 64:79 -> field 5
- 80:95 -> field 6
- 96:111 -> field 7
- 112:127 -> field 8
- 128:143 -> field 9

<img src="images/EGBS.Footprint_with_fields.png" width="600">

From the OU-VIS team.

Each field (a focal-plane array) is divided into 144 quadrants (36 CCDs, each made of 4 quadrants), as shown in the image below:

<img src="images/quadrants.png" width="400">

From Euclid Quick Data Release (Q1) : VIS processing and data products (Euclid Collaboration: H. J. McCracken et al. 2025).

The data was processed by the VIS ground-segment team at IAP. 

## A note about the PSFEx_model directory

The file `EuclidRedmine_32876_EGBS_400s-PSF_PSFEx_model.pdf` explains how the PSF was processed by the VIS Team and what are each file is. The PSF file was generated with PSFEx. The resulting PSF is a 21x21 array that can be calculated for different options using `EuclidExtractor.py`.

## How to split the VIS calibrated frames in different files?

This step is required to be able to work with these data.

### Using command line

The code `splitsci.py` can be run from command line to split DET FITS files into 
separate SCI/FLG/RMS extension files.

```bash
python splitsci.py [-h] [-j JOBS] [-v] [--debug] files [files ...]

positional arguments:
  files                 Input FITS files to process

options:
  -h, --help            show this help message and exit
  -j JOBS, --jobs JOBS  Number of parallel processes (default: 4)
  -v, --verbose         Verbose output
  --debug               Show HDU structure (implies -j 1)
```

### Example

Example:

```bash
python splitsci.py data/EUC_VIS_SWL-DET-067070-99-1__20250621T162513.608134Z.fits
```

### Convention used for created files

* EUC_VIS_SWL-DET-{obsid}-{dither}-{expnum}__*.SCI.fits
  science image of the calibrated frame

* EUC_VIS_SWL-DET-{obsid}-{dither}-{expnum}__*.FLG.fits
  flag map of the calibrated frame

* EUC_VIS_SWL-DET-{obsid}-{dither}-{expnum}__*.RMS.fits
  noise map of the calibrated frame. It contains the quadratic sum of the image
  photon noise (square root of the science image) and the readout noise.

## EuclidExtractor Tool to work with the data

This version from 2026-02-23 was written by Manon Gilles (Exoplanet Team, IAP, France), with inputs from Henry Joy McCracken, Clotilde Laigle, the help of the VIS team, and the Exoplanet Team of IAP. 

Please see the Jupyter Notebook EuclidExtractor/Euclid_data_extraction.ipynb, written by Manon Gilles, for more details on usage and fonctions.

## Calibration source catalogues 

Please note: The calibration source catalogues are intended for calibration purposes and are not suitable for statistical analyses. For further details, please refer to the Q2 paper.