Operations

Download & ingest statistics

Durable log of every FIRMS fetch attempt so we can verify success and prepare for hourly (or streaming) refresh. Log file: logs/download_events.jsonl

Snapshot

Raw files on disk

FileBytesModified (UTC)Age (h)Stale >1.5h

Recent events

TimeOKProductWindowBytesSourceErrorms

Hourly & streaming (roadmap)

Now: every CLI / cron download appends to the JSONL log; this page aggregates it.

Hourly:

# crontab example
0 * * * * cd /Volumes/Projects/FIRMS_Analytics && \
  .conda/bin/python scripts/scheduled_download.py --force \
  >> logs/cron_download.log 2>&1

Streaming (later): a worker emits stream_tick events into the same log schema; admin success rate and “last success by product” stay valid.