diff options
| -rw-r--r-- | doc/rtfm.md | 16 | ||||
| -rwxr-xr-x | synopticChart.sh | 10 |
2 files changed, 22 insertions, 4 deletions
diff --git a/doc/rtfm.md b/doc/rtfm.md index beb1787..bea77da 100644 --- a/doc/rtfm.md +++ b/doc/rtfm.md @@ -13,7 +13,7 @@ Everything is fetched anonymously from BOM's public FTP (`ftp.bom.gov.au`), per ## Architecture -- **`synopticChart.sh`** — fetches the latest `IDY00030` MSLP analysis PDF, renders to a 1920×1080 PNG (ImageMagick `magick -density 300` + Ghostscript), publishes it as `synopticLatest.png`, and archives both the raw PDF and rendered PNG. Deployed to `/opt/synoptic/synopticChart.sh`. +- **`synopticChart.sh`** — fetches the latest `IDY00030` MSLP analysis PDF, rasterizes it to PNG via Poppler's `pdftoppm -r 300` (previously ImageMagick's Ghostscript delegate — see Current status, 2026-08-27), resizes to 1920×1080 with ImageMagick, publishes it as `synopticLatest.png`, and archives both the raw PDF and rendered PNG. Deployed to `/opt/synoptic/synopticChart.sh`. - **`synopticMorph.sh`** — builds a rolling `WINDOW_DAYS`-day morph timelapse from the archived PNGs. Content-addressed pair cache (`<chartA>__<chartB>/p_NNNN.png`) under `WORK_ROOT` (`/mnt/enclave/synoptic`, deliberately off the cramped root volume — see Current status). Each run fills missing pairs, evicts anything outside the current window, assembles a sequence with linger/dwell holds, encodes via `ffmpeg`, and publishes `morph.mp4`. Triggered by `OnSuccess=` from `synoptic.service` (not its own timer). Deployed to `/opt/synoptic/synopticMorph.sh`. - **`radarFetch.sh`** — per-city radar fetch (`RADAR_ID`, e.g. `IDR713` Sydney), pulls dynamic radar layers + static transparency overlays, alpha-composites, publishes the last `FRAME_COUNT` frames as an APNG loop. Deployed to `/opt/radar/radarFetch.sh`, one process serves all four cities via the `RADAR_ID` arg. - **`index.html` / `morph.html` / `radar.*.html`** — static frontend, served directly by nginx from `/srv/www/pestrel/` and `/srv/www/radar/`. @@ -30,7 +30,7 @@ Everything is fetched anonymously from BOM's public FTP (`ftp.bom.gov.au`), per | Archive / working data | `/var/lib/synoptic`, `/var/lib/radar` | | Morph pair-cache | `/mnt/enclave/synoptic` | | Published web root | `/srv/www/pestrel/`, `/srv/www/radar/` | -| Runtime deps | `curl`, `imagemagick`, `ghostscript`, `ffmpeg`, `nginx` | +| Runtime deps | `curl`, `imagemagick`, `poppler` (`pdftoppm`), `ffmpeg`, `nginx` | All app-owned paths (`/opt/synoptic`, `/opt/radar`, `/var/lib/synoptic`, `/var/lib/radar`, `/srv/www/pestrel`, `/srv/www/radar`, `/mnt/enclave/synoptic`) are `st33v`-owned — day-to-day script edits/deploys need no `sudo`. Only `setup.sh` itself (systemd unit installation, package install) needs root. @@ -39,6 +39,18 @@ All app-owned paths (`/opt/synoptic`, `/opt/radar`, `/var/lib/synoptic`, `/var/l - `synopticMorph.sh`: `WINDOW_DAYS` (default `3`), `MORPH_N` (23), `FPS` (12), `LINGER` (8), `DWELL` (36), `FADE_IN`/`FADE_OUT` (1/2s), `GLYPH_FONT` (DejaVu Sans), `GLYPH_SIZE` (84). - `radarFetch.sh`: `RADAR_ID` (positional arg, default `IDR713`), `FRAME_COUNT` (6), `FRAME_DELAY` (50cs), `END_PAUSE` (150cs), `TRANSPARENCY_TTL_HOURS` (24). +## Current status (2026-08-27) + +**Synoptic chart rendering only isobars over a black background, diagnosed and fixed (2026-08-27).** st33v noticed `synopticLatest.png` had lost everything except the grey pressure-contour lines — no coastlines, state borders, grid, fronts, H/L labels, wind-barb legend, or land/sea fill. + +Root cause: BOM upgraded the `IDY00030` chart-generating software on their end at **2026-08-26 00:00 UTC** (the PDF's `Producer` metadata dropped its version suffix, `...Recode Weather 7.3.7-aubom-r7` → `...Recode Weather`). The new-format PDF triggers an upstream Ghostscript defect — confirmed via isolated local reproduction, independent of the deployed host — where `gs` (invoked directly, or via ImageMagick's PDF delegate as `synopticChart.sh` did) silently drops almost all page content once rendered at roughly ≥250dpi, while still drawing isobars; the exact same file renders fully correct at ≤200dpi, and the pre-2026-08-26 PDF format renders fully correct at 300dpi. No errors or warnings are emitted either way — it just quietly renders less. Ruled out as causes: this repo's scripts (unchanged since March), and cremonde's ImageMagick/Ghostscript/Poppler package versions (`pacman.log` shows no changes anywhere near that date). + +Poppler's `pdftoppm` renders the new-format PDF completely and correctly at every tested resolution including 300dpi. Fix: `synopticChart.sh` now rasterizes via `pdftoppm -r 300`, then resizes the result with ImageMagick, rather than letting `magick -density 300` drive Ghostscript directly. `poppler` was already installed on cremonde (an existing transitive dependency), so no new packages were needed. + +Remediation on top of the code fix: the 4 charts fetched broken between 2026-08-26 00:00 UTC and 2026-08-26 18:00 UTC were regenerated from their retained raw PDFs (`archive/raw/`) with the fixed pipeline, `synopticLatest.png` was republished immediately, the 4 morph pair-cache directories that referenced those charts were evicted (their sentinel files existed but held morphs computed from the broken images), and `synopticMorph.sh` was re-run to recompute those pairs and republish `morph.mp4`. + +**Chart is now monochrome, not a regression — a separate BOM-side defect (2026-08-27).** After the above fix, st33v noticed the chart no longer has its familiar pale-blue ocean / cream-land fill, just black-on-white. Confirmed this is *not* caused by the `pdftoppm` switch: BOM's post-2026-08-26 PDF still contains the `0.945 0.945 0.863` (cream) fill operator at the same count as the old format (1086×), but is missing the ocean pale-blue and grid-grey fills, and — more tellingly — rendering it through two independent, unrelated engines (Ghostscript and Poppler) both produce the same result: 216 unique colors, every one a pure gray. The old-format PDF renders with 2345 colors including real blues/creams on both engines. Neither PDF embeds any raster images (`pdfimages -list` empty on both), so it's a pure-vector fill/path defect in BOM's new generator — the fill color is set but whatever path it's meant to paint isn't landing, almost certainly the same software regression that caused the isobars-only bug above. Nothing to fix on our end; there's no color data left to extract. Worth periodically eyeballing `synopticLatest.png` in case BOM ships another update that restores it. + ## Current status (2026-07-31) **Disk-full incident, diagnosed and resolved (2026-07-30).** Root (`/`, `/dev/root`, 9.7G ext4) hit 100% used / 0 available. Two independent causes, both on root: diff --git a/synopticChart.sh b/synopticChart.sh index 059665d..2da65c0 100755 --- a/synopticChart.sh +++ b/synopticChart.sh @@ -21,7 +21,7 @@ require_cmd() { require_cmd curl require_cmd magick -require_cmd gs +require_cmd pdftoppm # Return the datetime string for the most recently published chart slot. # BOM chart slots: 0000, 0600, 1200, 1800 UTC. @@ -54,7 +54,13 @@ if [ "$curlExit" -gt 0 ]; then exit "$curlExit" fi -magick -density 300 "${latestChart}" -resize 1920x1080 "${dateTime}.png" +# pdftoppm (Poppler), not magick's Ghostscript delegate, does the actual PDF +# rasterization: at 300dpi, gs silently drops all page content but the +# isobars on BOM's current chart-generator output (confirmed upstream +# Ghostscript bug, not fixable here) — see doc/rtfm.md. +pdftoppm -r 300 -png "${latestChart}" "${dateTime}" +magick "${dateTime}-1.png" -resize 1920x1080 "${dateTime}.png" +rm -f "${dateTime}-1.png" mv "${latestChart}" "${RAW_DIR}/" cp "${dateTime}.png" "${ARCHIVE_DIR}/" |