Rebuilding the TSDB
This page contains instructions for rebuilding the Time Series Database (TSDB). Note that all data in the database will be lost (and reingested)! In Postgres, it can only be rebuilt using an administrator role (timeseriesdba). Add the password in the code below.
[2]:
from database.modules.utils.tsdb import TSDB
[3]:
credentials = {"TSDBUSER": 'timeseriesdba', "TSDBPASS": 'ADD_PASSWORD_HERE'}
myDB = TSDB(backend='psql', credentials=credentials)
INFO: Starting KPF_TSDB
INFO: Jupyter Notebook environment detected.
INFO: Base data directory: /data/L0
INFO: Backend: psql
INFO: Using TSDBUSER from credentials dictionary.
INFO: Using TSDBPASS from credentials dictionary.
INFO: PSQL server: 127.0.0.1
INFO: PSQL username: timeseriesdba
INFO: PSQL user role: admin
INFO: Metadata table exists.
INFO: Metadata table read.
INFO: Data tables exist.
Drop the tables.
[4]:
myDB.drop_tables()
INFO: Dropped table: tsdb_base
INFO: Dropped table: tsdb_l0
INFO: Dropped table: tsdb_2d
INFO: Dropped table: tsdb_2d_flux
INFO: Dropped table: tsdb_l1
INFO: Dropped table: tsdb_l1_medg
INFO: Dropped table: tsdb_l1_medr
INFO: Dropped table: tsdb_l1_stdg
INFO: Dropped table: tsdb_l1_stdr
INFO: Dropped table: tsdb_l2
INFO: Dropped table: tsdb_l0t
INFO: Dropped table: tsdb_l2rv
INFO: Dropped table: tsdb_l2ccf
INFO: Dropped table: tsdb_l2_bcv
INFO: Dropped table: tsdb_l2_bjd
INFO: Dropped table: tsdb_l2_ccfw
INFO: Dropped table: tsdb_l2_sci1
INFO: Dropped table: tsdb_l2_sci2
INFO: Dropped table: tsdb_l2_sci3
INFO: Dropped table: tsdb_l2_sci
INFO: Dropped table: tsdb_l2_cal
INFO: Dropped table: tsdb_l2_sky
INFO: Dropped table: tsdb_metadata
Create new tables.
[5]:
myDB = TSDB(backend='psql', credentials=credentials)
INFO: Starting KPF_TSDB
INFO: Jupyter Notebook environment detected.
INFO: Base data directory: /data/L0
INFO: Backend: psql
INFO: Using TSDBUSER from credentials dictionary.
INFO: Using TSDBPASS from credentials dictionary.
INFO: PSQL server: 127.0.0.1
INFO: PSQL username: timeseriesdba
INFO: PSQL user role: admin
INFO: Metadata table does not exist. Attempting to create.
INFO: Metadata table created correctly with indexed columns.
INFO: Metadata table read.
INFO: Data tables do not exist. Attempting to create.
INFO: Data tables and indices created successfully.
And ingest data (change dates appropriately).
[ ]:
start_date = '20240903'
end_date = '20241101'
myDB.ingest_dates_to_db(start_date, end_date)
INFO: Adding to database between 20240903 and 20241101
Show database status.
[7]:
myDB.print_db_status()
INFO: Database Table Summary:
INFO: Table Columns Rows
INFO: -------------------------------------------------------
INFO: tsdb_base 23 14745
INFO: tsdb_l0 111 14745
INFO: tsdb_2d 95 14739
INFO: tsdb_2d_flux 96 14739
INFO: tsdb_l1 126 14070
INFO: tsdb_l1_medg 106 14070
INFO: tsdb_l1_medr 97 14070
INFO: tsdb_l1_stdg 106 14070
INFO: tsdb_l1_stdr 97 14070
INFO: tsdb_l2 25 11194
INFO: tsdb_l0t 124 14745
INFO: tsdb_l2rv 29 11194
INFO: tsdb_l2ccf 7 11194
INFO: tsdb_l2_bcv 68 11194
INFO: tsdb_l2_bjd 68 11194
INFO: tsdb_l2_ccfw 68 11194
INFO: tsdb_l2_sci1 68 11194
INFO: tsdb_l2_sci2 68 11194
INFO: tsdb_l2_sci3 68 11194
INFO: tsdb_l2_sci 135 11194
INFO: tsdb_l2_cal 135 11194
INFO: tsdb_l2_sky 135 11194
INFO: Dates: 29 days from 20250801 to 20250829
INFO: Last update: 2025-08-29 22:01:58