diff options
| -rw-r--r-- | doc/migrate-fetch-to-stan.md | 77 |
1 files changed, 77 insertions, 0 deletions
diff --git a/doc/migrate-fetch-to-stan.md b/doc/migrate-fetch-to-stan.md new file mode 100644 index 0000000..9973fc3 --- /dev/null +++ b/doc/migrate-fetch-to-stan.md @@ -0,0 +1,77 @@ +--- +title: "Side-quest — migrate radar fetch from cremonde to stan" +date-created: "2026-06-10" +type: "side-quest" +status: "deferred" +author: + - "st33v" + - "claude-opus-4-7" +model: + - "claude-opus-4-7" +tldr: "BOM's anonymous FTP is intended for personal/in-organisation use. Cremonde is a Singapore VPS, which is harder to read as polite. Plan: move the fetch+composite step to stan (local AU pi), have stan push finished APNGs to cremonde over WireGuard, leave cremonde as the public TLS-terminating server. No urgency — only worth doing if BOM complains or if we want to be defensive in advance." +parent-thread: "doc/radar-four-cities-and-nav.md" +parent-doc: "doc/bom-radar-spec.md" +reader-targets: + - "claude-code" + - "st33v" +tags: + - "bom" + - "radar" + - "migration" + - "wireguard" +trigger-conditions: + - "BOM contacts re. FTP usage from cremonde" + - "decide to be defensive ahead of any contact" + - "moving more BOM-fetching jobs onto the same infrastructure" +--- + +# Migrate radar fetch from cremonde to stan + +## Why + +`bomsynoptic`'s radar job currently fetches BOM FTP every 6 minutes from cremonde (SG VPS, public IP). BOM's anonymous-FTP terms permit personal / in-organisation use; they explicitly don't promise availability and they reserve the right to throttle abusive clients. A residential AU IP (stan) hitting them is easier to read as polite than an offshore VPS doing the same. + +This is **pre-emptive politeness**, not a fix for a known complaint. Don't act unless one of the trigger-conditions above hits. + +## Design (agreed in chat) + +Split fetch from serve: + +- **stan** runs `radarFetch.sh` on the systemd timer. Composites APNGs locally. +- **cremonde** keeps the nginx vhost, TLS cert, certbot renewal, and the static HTML pages. Its `/srv/www/radar/*-loop.apng` files become a passive drop target. +- After each successful composite, stan rsyncs the published APNG to cremonde over WireGuard. Small file (~90 KB), atomic rename at the destination via rsync's default behaviour. + +``` +radarFetch.sh (stan) + → /srv/www-staging/radar/idrXXX-loop.apng (local) + → rsync -a idrXXX-loop.apng cremonde:/srv/www/radar/idrXXX-loop.apng +``` + +Only the APNGs travel. The HTML pages stay on cremonde permanently — they're stable, no reason to ship them across. + +## Changes required + +- **`radarFetch.sh`**: optional final `rsync` step, gated on `REMOTE_PUBLISH` env var (so the same script works in solo-host mode and split mode). +- **`setup.sh`**: split into two paths. On cremonde: install nginx, HTML, no radar timer. On stan: install fetch script, timers, no nginx. Could be one script with a `--role=fetch|serve|both` flag, or two scripts. +- **WireGuard**: cremonde's wg interface needs to accept ssh from stan (it presumably already does, since stan is a wg peer). Use a non-interactive key for the rsync user — separate key per direction, document in `doc/`. +- **cremonde-side**: a small `rsync`-only ssh key, restricted with `command="rsync --server …"` in authorized_keys to limit blast radius if stan's key is compromised. Or use a dedicated user. +- **Monitoring**: stan's rsync silently failing leaves stale APNGs on cremonde. Manifest file? Or use the existing pattern — APNG mtime stamped via `Last-Modified` header. A 15-min-stale check would catch it. + +## Open questions + +- WG topology: is stan already a peer of cremonde, or only client→VPS? If only client→VPS, rsync from stan → cremonde already works. If peer-mesh, fine too. If neither, set it up first. +- Does stan have the disk space + cpu for 4 radars × 6-minute composites? Pi 4 / 5 should be ample. Pi Zero might not. +- BOM's other products (synoptic chart, satellite) — should they migrate too? Synoptic is once-per-6-hours and tiny; less worth migrating. Same logic: only if defensive. + +## Out of scope + +- Re-architecting to "stan fetches, stan also serves" — defeats the point of having cremonde's public TLS termination. +- Failover (cremonde fetches if stan is down) — defeats the geo-rationale. +- Pushing HTML pages from stan to cremonde — they don't change often; deploy via the existing git-pull-and-setup.sh flow on cremonde. + +## Cross-reference + +- Original spec: `doc/bom-radar-spec.md` §10 (copyright/terms-of-use flagged for st33v). +- Rollout memoir: `doc/bom-radar-rollout.md`. +- Day-2 memoir: `doc/radar-four-cities-and-nav.md` — final exchange on this thread is the seed of this side-quest. + |