Getting Started with the KPF DRP

This section contains topics about getting started with using the KPF-Pipeline module.

First, download and install Docker Desktop if you don’t already have it.

Then clone the repository and navigate into it:

git clone https://github.com/Keck-DataReductionPipelines/KPF-Pipeline.git
cd KPF-Pipeline

Warning

Refer to Installing from Develop or Feature Branches for setting up other branches

Define the KPFPIPE_DATA environment variable and point it to a location where you want to store the input and ouput files. If you would like to work in Jupyter notebooks then also define the KPFPIPE_PORT environment variable to assign a port to use for the notebook server.

Build the package into a docker container and launch an interactive bash shell:

make docker

Install the package once the container launches:

make init

To run the pipeline, use the following command:

kpf

This will prompt the use case message, which should be:

usage: kpf [-h] -r recipe -c config_file
kpf: error: the following arguments are required: recipe, config_file

The two mandatory input arguments to kpf are: a recipe file and a config configuration file. The recipe file is expected to be a .recipe script, while the config file is expected to be a .cfg file. See Running a Simple Module for a basic example and Example of processing data from a single night to process a night of data.

To start a notebook server run the following and follow the on-screen instructions:

make notebook

Finally, some of the recipes interact with a PostgresSQL database, which is the so-called pipeline operations database. Refer to Installing/Configuring the Pipeline Operations Database for instructions on how to initially set up the pipeline operations database.