# NE cells — public contract (rendered from the stored contract in public vocabulary)

Public vocabulary: **KG** (5-digit code), **unit** (land unit), **register**, **structure**, **declared / observed**. Read API (tokenless, H3-keyed): see the `## NE cells` section of /api/v1/docs/llm.txt. Observed layer per cell (LiDAR, trees, structures, NDVI, terrain): sibling https://srtm-lidar-at.exe.xyz:8100/ (same H3 ids). Writes (`/obs`, `/report`) need a peer token. Attribution: © BEV, 2026 (CC BY 4.0), bearbeitet — statistics per H3 cell, no object geometry · observed layer © srtm-lidar-at (CC BY 4.0)

---

# NE cells — contract (algo `ne-cells-2`, container `NEC1`/`NEO1` fmt 1)

Status 2026-10-04: srtm-lidar-at signed off `ne-cells-2` (both 63330 digests
reproduced bit-identically on their side) and starts **v2.4** on it. The
country-wide `lu` build (all local KGs, `scripts/migrate_local/ne_all.sh`) is
running; progress and per-KG status: `GET /api/v1/ne/manifest`. Served to
srtm-lidar-at (first token) under `/api/v1/ne/…`. Review decisions from
srtm's checks of `ne-cells-0`, `ne-cells-1` (FIX 1–5) and the v2.4 requirements
(§v2.4) are all adopted.

## Coverage rule — lossy builds are not change (2026-10-08) {#coverage}
Rebuild of 75110/73008 with local bevdirect-serve v0.3.1 **and** v0.3.2 (full table:
`docs/archive/2026-10-ne-coverage-rebuild.md`): the "change" of 336 / 985 chunks between
reports minutes apart was neither BEV nor the tag. Aligned 0.02° cells are byte-identical
v0.3.1 = v0.3.2 and reproducible across hosts and days (75110 report 3 == our build,
3 659/3 659). What differs is the **request layout**: a multi-cell `/viewport` is the union
of its cells *after* compose's unit dedupe, which keeps ONE truncated copy of a unit
wider than cell + pad (`complete:false` in every cell; 13 such units in 75110, 37 in 73008),
whereas `ne_cells` unions all truncated copies → `cells_n` drops 0.6–6 %, hundreds of chunk
digests around those units shift, 96–100 % of differing chunks touch such a unit.
Landuse never differs; the v0.3.1 structure-fragment bug is a ~10-chunk effect.

Server rule (`neBestCoverage`, `api_ne_reports.go`): per (source class, algo, chunk_res,
**identical bbox**) the build with the most `cells_n` is the complete one; a report more than
`neCoverageTol` (0.1 %) below it is **lossy** → `coverage.status:"lossy"`,
`change_suspect:"coverage_lossy"`, never a baseline nor a "last" digest (skipped in
`neReportHistory`), its `since_last` diffs counted as `chunks_changed_suspect` on
`/ne/reports` and excluded from the public contrib chart. Old lines are re-classified on
read (`neMarkLossy`; a good report whose `since_last` was taken against a lossy one is
`previous_lossy`). Viewport observers (different bbox each time) have no reference — their
edge-unit diffs stay; reading rule below still applies. Land-use change never reduces
`cells_n`, so the rule cannot hide real change. The bevdirect tag is **not** the classifier:
`neBevEquivalentTags` keeps v0.2.1…v0.3.4 in one class (v0.3.4 added 2026-10-15: tile hashes only, no assembly change; before that its first reports on 23308/04104 opened a fresh baseline instead of comparing against v0.3.3 history).

Peer status after the rule (srtm reply, 2026-10-08 ~17:30Z, srtm commit `e7c3c39`): fleet rolls onto
bevdirect-serve **v0.3.3** + vtcseamless-py 0.1.1 (`cells_for`/`cell_bbox`, exact unpadded cells);
input domain = union of the aligned cells, so the report `bbox` is now a **stable cell block**
(`row.input_bbox`) instead of the KG bbox — a new bbox key for `neBestCoverage`, which is intended
(old KG-bbox lines keep their own best; chunk history is not bbox-keyed). First v0.3.3 row: 23308
at 17:11Z, cells_n 5 141 (== the v0.3.2 KG-bbox report), `coverage.status:best`, `since_last` clean,
aligned digest `7c7e0eb45c2148d0` unchanged. srtm stores `coverage{}`/`change_suspect` per row and
zeroes suspect counts on its side. 4 236/7 850 KG bboxes are narrower than 0.02° in one axis
(23308: 0.0198×0.0132°) — a sub-cell KG is a normal whole-cell build, **not** lossy; do not ask srtm
for `VCT_SKIP_SUBCELL_KG=1`. Expect `bevdirect@v0.3.3` rows fleet-wide within hours (one peer per
2 h ramp in bev mode).

**Layout rule (2026-10-08, v0.3.3 rollout):** the first cell-block report of a KG takes its
`since_last` against digests built under the old KG-bbox layout. Request layout alone moves
chunk digests (table above), so those diffs are `change_suspect:"layout_change"`
(`neLayoutSuspect`, `since_last.changed_previous_bbox_differs`; retro on read in `neMarkLossy`)
— a re-baseline, not change. Only **cell-block vs cell-block** (same bbox key), non-lossy
`since_last` diffs reach the landing-page figure. What that figure can say: a report carries one
digest per res-10 chunk, so change is **"N chunks (within ≈ X ha)"** — the count is the
measurement, the hectares bound where it lies. It cannot say m² of land-use change or which
structures changed; that would need per-layer (lu / bld) chunk digests in the report schema
(peer-side change, not planned).

Peers: build from aligned cells (`vtcseamless observe` does; own pipelines must fetch
cells, not viewports). bevdirect-serve < v0.3.3 emitted `null` for an empty layer and the
frozen `canon.py` crashes on it — fixed in bevdirect-serve **v0.3.3** (`[]`) and
vtcseamless-py **0.1.1** (normalises). BEV regenerates all tiles nightly (~00:54Z); with a
24 h tile TTL a peer can mix tile generations in one build (73008 at210, one edge column).

## Triage in 60 s — "are reports coming in / is change computed?" {#triage}
Change in m² (not only "which chunk moved"): `docs/ne-change.md` — `curl localhost:8000/api/v1/ne/stats`
(national `change{confirmed,candidate,by_day}`), `?kg=63330`; `ls data/ne_cells/*.chunks.zst` = KGs with a
chunk-version log; `journalctl … | grep '🧾 NE chunks'` = step-2 uploads. Peers need vtcseamless-py ≥ 0.2.0
(sends `chunks_ap`); older clients keep working, they just never feed the change series.
```
TOK=$(sudo grep -E '^API_TOKEN=' /etc/survey-api.env | cut -d= -f2 | cut -d, -f1)   # slot 0 = srtm, slot 1 = ops
curl -s -H "Authorization: Bearer $TOK" localhost:8000/api/v1/ne/reports | python3 -m json.tool | less
#   live        = per token×observer since process start (empty after a restart — normal)
#   on_disk     = scan of data/ne_cells/*.reports.jsonl (memo 60 s): reports, became_baseline, compared,
#                 chunks_changed (vs epoch 0), chunks_changed_since_last, kgs_with_changes
sudo journalctl -u survey-api --no-pager | grep '🧾 NE report' | tail      # one line per POST (NE routes bypass the 📨 access log)
python3 - <<'PY'                                                            # per-report table from disk
import json,glob
for f in sorted(glob.glob('data/ne_cells/*.reports.jsonl')):
    for l in open(f):
        d=json.loads(l); r=d['report']; x=d.get('result',{}); sl=x.get('since_last') or {}
        print(f[-19:-14], d['received_at'][5:16], d.get('token'), r['observer'][:20], r['source'], 'chunks',len(r.get('chunks') or {}),
              'bbox',r.get('bbox'), '|', x.get('baseline'), 'cmp',x.get('compared'), 'same',x.get('chunks_same'),
              'chg',len(x.get('chunks_changed') or []), 'new',x.get('chunks_baselined_now'), 'sl_chg',len(sl.get('changed') or []))
PY
```
Fastest answer to "are /contrib's coming in from srtm, its peers and siedler?": the
`observers[]` array of `/ne/reports` → `on_disk` (token × observer: reports, kgs,
sources, first/last_at, compared, chunks_changed_since_last). Per-day/source counts:
```
python3 - <<'PY'
import json,glob,collections; c=collections.Counter()
for f in glob.glob('data/ne_cells/*.reports.jsonl'):
    for l in open(f): d=json.loads(l); r=d['report']; c[(d['received_at'][:10],d.get('token') or '-',r['observer'][:22],r['source'])]+=1
for k,v in sorted(c.items()): print(v,*k)
PY
```
State 2026-10-16 07:10Z: 4 461 reports / 3 386 KG files, 574 reports in the last 24 h (siedler 299,
srtm 275, 51 observers), 0 rejects, 0 non-2xx. Confirmed change still 186 cells / 4 KGs. 2 513 KGs
with a chunk log (713 MB), 522 k chunks seen ≥ twice, deepest pass 7. New observer
`vtcseamless-py-qc` (siedler token, QC probe on 63330). **bevdirect v0.3.3 → v0.3.4 rollout** (siedler
100 % v0.3.4 on Oct 16, srtm fleet ~57 %) produced a candidate spike: 336 cells Oct 15, 454 Oct 16,
almost all **KG 42005: `alpine>forest` 114 914 m²** (srtm-peer-at210 v0.3.4 vs at207 v0.3.3 baseline,
same 7 817 chunks, 22 changed); similar 76017 (at220 v0.3.4, 46 chunks, 9 k m² green→structure) and
91008 (siedler v0.3.4, 33 chunks). Cross-version one-class-flips-wholesale = build difference, not land
use. **Watch rule:** if another v0.3.4 observer confirms 42005 with the identical 114 k m² alpine→forest
delta, that is the signature that v0.3.4 changed classification → treat `bevdirect@v0.3.4` as a new
source-equivalence class (baseline per `(kg, class)`, not comparable to ≤ v0.3.3), and do not read the
resulting "confirmed" burst as BEV change. Otherwise the candidates resolve on the next v0.3.4 pass.

State 2026-10-13 05:30Z: 3 161 reports / 2 545 KG files, 1 787 siedler + ~1 370 srtm (51 `srtm-peer-at*`
observers, all v0.3.3), 450–550 reports/day; 233 k chunks seen ≥ twice (deepest pass 6). Change log:
1 664 KGs, 1.56 M versions, 473 MB. **First confirmed change**: KG 32002 Apetlon, 33 chunks (compact
cluster 47.79–47.81 N / 16.86–16.89 E, six z16 tiles) — at205 Oct 8 baseline, at212 Oct 10 13:47Z new
digest (candidate), siedler Oct 10 23:27Z identical digest → confirmed (238 cells, green→road 1.7 k m²,
water↔road 1.6 k m²). Candidates awaiting a second sighting: 31036 (at241→at238, 18 chunks), 84014
(at210→at202, 7 chunks), 23308 etc. — expected, they resolve on the next pass. No rejected chunk uploads,
no non-2xx on `/contrib`. Both the candidate→confirmed path and the cross-observer confirmation are
therefore exercised end to end; whether a confirmed delta is a BEV edit or the *baseline* being the odd
(mixed-generation) build is still only decidable by a third sighting or a local rebuild.

State 2026-10-10 06:40Z: 1 866 reports / 1 633 KG files, 1 221 siedler + ~600 srtm (48 `srtm-peer-at*`
observers, all on bevdirect@v0.3.3), 300–500 reports/day steady. Change log: 746 KGs, 697 k chunk
versions; 108 k chunks confirmed via step-1 header match (104 reports) — `🧾 NE chunks … seen=0` is
normal, step 2 only carries chunks we do not hold. Confirmed change 0 cells (no BEV epoch yet);
candidate 21 cells = KG 84014, srtm-peer-at210 (Oct 8) vs at202 (Oct 9), same v0.3.3, 7 chunks,
959 m² green→structure — awaits a second sighting. Traffic: /k survey reads fell 21 k/day (Oct 5)
→ 3.2 k (Oct 9), only srtm-lidar-at's python-requests job remains (spatial/structures, export/geojson,
osm/geometry); contrib reports are now the main inbound stream.

State 2026-10-06 11:00Z: 282 reports / 271 KGs. siedler 260 (237 on Oct 6 alone, v0.3.1),
srtm-lidar-at via observer `srtm-peer-at68` 11 KGs (v0.3.0→v0.3.1), `srtm-peer-primary-test`
3× 23308. Almost everything is `baseline:this_report` — **expected**: change needs the *same*
chunks reported twice, and the fleet is still in its first pass. Only 73119 compared so far
(see below).

**Cross-observer "change" within hours is suspect, not land use.** 73119: srtm-peer-at68
(v0.3.0, 07:39Z) then siedler (v0.3.1, 09:44Z), identical bbox and 491 chunks, 5 chunks
differ (`8a1e16cc42e7fff 8a1e16cc4317fff 8a1e16cc431ffff 8a1e16cc4657fff 8a1e16cc55affff`).
BEV does not change in two hours; candidates are host-side assembly differences
(`complete:false` units at the fetched tile edge, tile fetch failures, v0.3.0 vs v0.3.1).
The baseline is per `(kg, source-equivalence class)` across observers by design, so such
diffs *are* reported; reading rule: a changed chunk counts as land-use change only when
reported by the **same observer** across BEV epochs (months), or when two observers agree
on the new digest. Follow-up open: rebuild those 5 chunks locally with both tags.

Reading the table: siedler-oesterreich = CONTRIBUTOR token, posts **viewport-sized**
reports (2–6 km bbox, 70–700 res-10 chunks, source `bevdirect@v0.3.0`, bursts
of ~20 KGs when a user browses) — not whole KGs. First viewport of a KG →
`baseline:this_report` (expected). Later overlapping viewport → `compared:true`,
`chunks_changed` (vs first-seen digest per chunk) and `since_last.changed` (vs
previous digest). Only 63330 has a `bev` section; everywhere else the baseline is
the reports themselves. `kgs_with_changes` for 63330 = 568 is historical noise
(Oct 3 untagged/v0.1.0 test reports), not change. Stored line layout:
`{received_at, token, unchanged?, report{kg,algo,epoch,source,observer,chunk_res,cells_n,digest,bbox,chunks{}}, result{…answer…}}`;
dedupe stubs (`unchanged:true`) carry no chunks. Files on disk are `{kg}.nex`
(store, `ne_store.go`) — `.nec/.obs` are converted away, `ls *.nec` is expected to be empty.
To test the comparison live without polluting a peer's history: replay a stored
report with the **ops** token (different token ⇒ no dedupe), then `head -n -1` the
line back out of the jsonl (your digest would otherwise become `since_last` for that chunk).

## FREEZE for the v2.4 run (2026-10-04 → 2027-04-30, ~200 days)
`algo ne-cells-2`, `ns_table ns-groups-1`, `lu` = `source index-2026-03`,
`epoch 2026-03`, `bev` baseline = `bevdirect@v0.2.0`. **Cell-equivalent
bevdirect tags: `v0.2.0 ≡ v0.2.1 ≡ v0.3.0 ≡ v0.3.1 ≡ v0.3.2`** (v0.2.1 only adds the 24 h tile-cache
sweeper + `tile_ttl_s` in `/health`; v0.3.0 is the same assembly moved to the
public module github.com/raffopenssh/vtcseamless, byte-identical on five
viewports; `v0.3.1` = RAM-only tile cache, download layer only; `v0.3.2` =
multi-cell `/viewport` dedupe fix, see §aligned-cells below; so a `/report` with source `bevdirect@v0.2.1`, `@v0.3.0`, `@v0.3.1` or `@v0.3.2` is compared
against our `bevdirect@v0.2.0` baseline — `neBevEquivalent` in `api_ne.go`).

### Aligned cells only (bevdirect < v0.3.2) — multi-cell viewport duplicates
`ne_cells`-comparable builds **must come from aligned 0.02° cell requests**
(`west=ix*0.02, south=iy*0.02, east=west+0.02, north=south+0.02`, `ix=floor(lon/0.02)`)
on any bevdirect-serve < v0.3.2. Reason: `compose()` in `bevdirect/service.go`
deduped structure/landuse pieces by their per-tile id (`z_x_y#i`); a structure
crossing a z16 tile edge that lies inside a neighbour cell's 0.004° pad is
seam-merged whole in the cell holding both tiles but emitted as a clipped
fragment under a different per-tile id by the cell holding only one, so a
multi-cell (unaligned) viewport carried both. Repro: Guntrams 23308, viewport
`16.1402,47.7068,16.168,47.728` → 8 fragments starting exactly at x=16.1554
(west edge of z16 tile 35709) absent from the single-cell `16.14,47.70,16.16,47.72`;
`ne_cells build` on it gives `str_n` +1 in 7 cells, digest `3ace8fba…` instead
of the per-cell `7c7e0eb4…`; 63330 viewport build `60d6d156…`/52 458 cells vs
the reference `594025647ec3557e`/52 478. Units were never affected (stable
`kg-unit number` id + `complete`); landuse neither (no seam merge, per-tile ids stable
across cells). Fixed in vtcseamless **v0.3.2** (commit `5ffbe23`): pieces carry
`complete` (bbox touches the fetched tile-set bound, as for units) and
seam-merged structures list `members`; compose dedups by member id with the
unit rule (complete copy wins, then larger area). Single-cell docs are
byte-identical apart from the 2 new fields (`ne_cells/canon.py` ignores them), so
per-cell digests are unchanged and v0.3.2 is cell-equivalent; multi-cell
viewports now equal the union of their cells (`TestComposeEqualsCellUnion`,
`BEVDIRECT_LIVE=1 … -run Live` for the real tiles). A peer that fed unaligned
viewports to `ne_cells` must upgrade and rebuild/re-report; its old reports are
flagged by `str_n` +1 along z16 tile columns.
First external report: siedler-oesterreich, 2026-10-04 10:18Z, 63330,
`bevdirect@v0.3.0`, 52 478 cells / 1163 chunks — arrived before v0.3.0 was in
the map, so it was stored as its own baseline; replayed by ops after the fix:
`identical_kg_digest:true` (`594025647ec3557e`), 1163/1163 chunks same against
our `bev` section — an independent host running the public module reproduces
the operator build bit for bit.
Rules:
- A KG's `lu` digest **never changes under the same (algo, source, epoch)**:
  `ne_all.sh` never overwrites an existing `{kg}.nec`; `python3 -m ne_cells
  build` output goes to a tmp path and is moved only when the target is absent.
- A rebuild (new input epoch, new algo) is a NEW `(algo, source, epoch)` triple,
  written as `{kg}.nec` only after the manifest's `frozen.until`, or earlier
  only with srtm's agreement — either way it shows up in the manifest
  (`lu.epoch`/`lu.algo`/`lu.digest` change, `updated_at` moves) so the director
  re-upgrades that KG.
- `ne_cells/algo.py`, `ns_groups.py`, `canon.py`, `pack.py` are frozen; a change
  bumps `ALGO` to `ne-cells-3` and is NOT deployed to `{kg}.nec` during the freeze.
The pin is served in every `/ne` and `/ne/manifest` answer as `frozen{}`. Reference implementation:
`ne_cells/` (python, **pinned shapely 2.1.2 + h3 4.5.0**); the Go server
computes nothing, it validates, stores and serves the container.

## Served 63330 (rebuild after FIX 1–3) — reproduce these digests
| section | source | epoch | digest | cells / K | build inputs |
|---|---|---|---|---|---|
| `?layers=lu` (`63330.nec`) | `index-2026-03` | 2026-03 | `1233f3e4f9d07c27` | 52 476 / 643 | `ne_cells fetch` (strict, one call per layer, bbox = input_bbox + 0.002°) — `data/ne_cells/inputs/63330.index-2026-03.{inputs,build}.json` |
| `?layers=bev` (`63330.bev.nec`) | `bevdirect@v0.2.0` | 2026-10 | `594025647ec3557e` | 52 478 / 649 | bevdirect-serve v0.2.0 `/viewport` cells x 753–757 × y 2354–2357 (0.02° grid, `x=floor(lon/0.02)`), `--input-bbox 15.0712,47.0902,15.1517,47.1553` |
| srtm's 4-cell check (755/2356 755/2357 756/2356 756/2357, `--input-bbox 15.102,47.122,15.138,47.1486`) | `bevdirect@v0.2.0` | 2026-10 | `9966afd37f026edb` | 19 772 / 155 | report vs `bev`: 385/385 chunks same |
| same, v0.1.0 tarball | `bevdirect@v0.1.0` (pass `--source`) | 2026-10 | `1bedf6778c78c65d` | 19 772 / 155 | not comparable to `bev` (different tag) |
`input_bbox` of 63330 = `15.0712,47.0902,15.1517,47.1553` (derived, see §domain).
ne-cells-1 cross-check (same inputs as srtm, before the algo bump): HEAD →
`c32ee33c8e1e0f88`, v0.1.0 → `1e1d597227353fa7` — identical to srtm's, so the
machines agree; everything that differed was input extent.

## Licence answer
Yes, intended: declared cells are a derivative of BEV Survey (CC BY 4.0) and
go into the Zenodo products under CC BY 4.0 with the fixed attribution in every
header. The **observed** layer (NEO1) is srtm's; by posting it to `/obs` srtm
agrees it is redistributed under the same CC BY 4.0 in the same product. If a
field must stay out of Zenodo, do not post it — the join key (H3 id) lets srtm
serve it from its own side.

## Endpoints (private tier, token index 0 only; all others → tier 404)
Planned public, tokenless, H3-keyed read path: `docs/ne-public-api.md` (writes stay here).
```
GET  /api/v1/ne                               endpoints, frozen pin
GET  /api/v1/ne/manifest                      all KGs: {kgs:{kg:{lu,bev,obs,updated_at,status,…}}, queue, frozen}; gzip + ETag/304
GET  /api/v1/ne/manifest?since=<generated_at> delta: kgs with updated_at ≥ since + counters, delta:true (§manifest)
GET  /api/v1/ne/{kg}                          binary container, all stored layers (NEC1 then NEO1)
GET  /api/v1/ne/{kg}?layers=lu|bev|obs        one layer (lu = index epoch, bev = bevdirect baseline); X-NE-Layers-Missing lists absent ones
GET  /api/v1/ne/{kg}?format=json              decoded (also on Accept: application/json) — debugging
POST /api/v1/ne/{kg}/obs                      body = one NEO1 section (≤ 16 MB) → stored, replaces previous;
                                                or NEC1+NEO1 = peer's bevdirect declared build + obs (§NEO1, since 2026-10-06)
POST /api/v1/ne/{kg}/report                   epoch report (digests only), answers changed chunks
GET  /api/v1/ne/{kg}/report                   reports received
GET  /api/v1/ne/draft/ne_cells.tar.gz         the reference package (also in git: ne_cells/, pyproject.toml)
```
ETag (`"ne-<sha256[:16]>"`, 304 on If-None-Match), `Cache-Control: private,
max-age=3600`. Binary is served as-is (`application/vnd.ne-cells`, already
deflated — gzip would add nothing); JSON goes through the global gzip.

### Coverage / unbuilt KG (v2.4 req 1)
`GET /ne/{kg}?layers=lu` is **whole or nothing**: a built KG answers the full
`{kg}.nec` (~265 KB median), an unbuilt one answers **404** with body
`{status, queue_position, retry_after_s, manifest}` and — when `status` is
`queued` (or `stalled`) — a `Retry-After` header = `(queue_position+1) ×
avg_seconds` of the running build (≥ 30 s). `status` values: `built`,
`queued`, `stalled` (queue file older than 1 h, Retry-After 3600), `failed`
(build error, no Retry-After; `ne_all.sh` retries them once at the end of the pass, `RETRY_FAILED=1` re-tries leftovers),
`not_in_index` (not a local KG — we hold 7 850 KG documents; see §kg-count).
Fleet pattern: ≤ 1 req/s, one GET per KG at processing start, `If-None-Match`
on re-reads; honour `Retry-After` rather than polling.

### Manifest (v2.4 req 2) — `GET /api/v1/ne/manifest`
```
{ "frozen": {algo, ns_table, lu_source, lu_epoch, from, until, bev_source, rule},
  "kgs_total": 7850, "built": n, "obs_posted": n,
  "queue": {updated_at, total, next_index, current, done, avg_seconds, failed[], finished},
  "kgs": { "63330": { "lu":  {digest, epoch, algo, source, cells_n, bytes, built_at},
                      "bev": {…} | null,  "obs": {digest, epoch, algo, source, cells_n, bytes, built_at} | null,
                      "updated_at": "<latest built_at>", "status": "built" },
           "63333": { "lu": null, "bev": null, "obs": null, "status": "queued", "queue_position": 212, "retry_after_s": 10600 },
           … every local KG … },
  "generated_at": … }
```
Built from the section headers on disk (no inflate), rescanned every 30 s
(`?refresh=1` forces), `ETag "nem-…"` over `kgs`+`queue` (not `generated_at`),
`Cache-Control: public, max-age=60`, gzip by the global middleware (~2.1 MB →
~260 KB). Detect our rebuilds by `lu.digest`/`lu.epoch` per KG, or cheaply by
the ETag. Size line: `/api/v1/debug/inflight` → `ne_manifest`. The full body
always carries `delta:false`.

**Delta mode `?since=<generated_at>`** (2026-10-06; srtm polls every 10 min and
the full ETag moves with every ~33 s build because `queue.next_index` and every
`queue_position` change, so `If-None-Match` never yielded a 304):
```
{ "delta": true, "since": "<normalised cursor>", "generated_at": …, "kgs_total": 7850, "built": n, "obs_posted": n,
  "kgs_changed": m, "kgs": { only entries with updated_at ≥ since }, "retention_s": 604800 }
```
- Cursor = `generated_at` of the previous answer (full or delta). Selection is
  `updated_at ≥ since` at 1 s resolution, deliberately **inclusive**: a file
  written in the same second as the previous `generated_at` but after that
  scan's ReadDir would otherwise be lost forever. A client therefore re-sees
  the boundary entries — merge by KG code, it is idempotent.
- `updated_at` = latest `built_at` of the KG's layers (file mtime / `.nex`
  layer header), so a KG moving queued → built, a new obs post or a rebuild all
  show up; a KG that only moved in the queue (`queue_position`) does not —
  deltas carry no `queue`/`frozen`, read the full body for those.
- Fallback to the **full** manifest (`delta:false`, replace your copy): cursor
  older than `neManifestDeltaRetention` (7 d — a delta cannot express a KG whose
  files were *removed*, and a week-old cursor selects most of a pass anyway),
  or later than our current `generated_at` (foreign/clock-skewed cursor).
  Unparseable `since` → 400.
- ETag `"nem-…-s"` over the changed set + counters + cursor (not
  `generated_at`): an unchanged 10-min poll is a 304. `Cache-Control: public,
  max-age=60`, gzip as usual. Cost per request: one pass over the in-memory
  entries outside the lock (`neManifestCache.delta`), no disk, no SQL.

### KG count (8 440 vs 7 850) {#kg-count}
Our index holds **7 850** KG documents (every KG with units in the March-2026
BEV tiles — Austria's survey has ~7 850 KGs). srtm's "8 440 parent KGs" is a
different list (probably includes dissolved/merged KG numbers of the Statistik
Austria table). Any code not among ours answers `status: not_in_index`; send us
the list and we diff it (`/api/v1/lookup?kg=…` tells whether a code exists).

## Container (`ne_cells/pack.py`)
```
section := magic[4] fmt_ver:u8 hdr_len:u32le header:JSON(canonical,ascii) body_len:u32le body:zlib(records)
file    := section+          (GET concatenates NEC1 + NEO1; a POST body is exactly one section)
```
`header.digest = sha256(inflated record block)[:16]` — compare digests, never
deflate bytes. Header fields (NEC1): `magic, fmt_ver, algo, epoch, source,
h3_res=12, k_res=10, k=5, kg, cells_n, kcells_n, groups[9], ns_table,
attribution, notice, record_lu, record_k, coord_decimals=7, input_bbox[4],
coverage_bbox[4], input_pad_deg, input_rule, inputs[], stats{}, digest`.
`inputs[]` is provenance (file, sha256[:16], rows, endpoint, ready,
bevdirect_version, fetched_at) — informative, not covered by the digest.

### LU record — 30 B, res 12 (~307 m²), one per cell touching the KG
| off | type | field | meaning |
|---|---|---|---|
| 0 | u64 | cell_id | H3 index |
| 8 | u8 | kg_share | share of CELL area covered by units of THIS KG (1/255) |
| 9 | u8 | land | share of cell that is surveyed land, any KG |
| 10 | u8 | cov | Σlu / land, capped at 1 |
| 11 | u8 | flags | bit0 `declared_unknown` (cov < 0.90 → no verdict), bit1 `normalised` (Σlu > land by ≥ 1/255, scaled down) |
| 12 | u8[9] | lu | declared mix in `groups` order: geb bau acker gruen wald wasser verkehr alpen sonst — 1/255 of CELL area |
| 21 | u8 | str_cover | structure ground cover, 1/255 |
| 22 | u16 | str_n | structure centroids in cell |
| 24 | u8 | gk | Grenzsurvey share of land |
| 25 | u8 | n_units | units touching (≤ 255) |
| 26 | u16 | edge_m_ha | unit-boundary metres per ha of land (fabric density) |
| 28 | u8 | orient_coh | 0xFF unless str_n ≥ 5; else coherence 1/255 |
| 29 | u8 | orient_axis | 0xFF unless str_n ≥ 5; else mean axis / 2 (0..89) |

### Input domain (`input_bbox`, rule `geom-intersects-input_bbox`) — ne-cells-2
Cells are KG-independent, and the record block is a function of the domain
alone, not of how much a fetch returned:
1. `input_bbox` = bbox of the KG's own units (canonical geometry) padded by
   `input_pad_deg = 0.004°`, rounded OUTWARD to 4 decimals
   (`algo.default_input_bbox`). It is written to the header; a rebuild passes
   it back with `--input-bbox` (if omitted it is re-derived and must agree).
2. An object (unit, structure, landuse piece, any KG) is INPUT iff its
   canonical geometry intersects the closed box. Objects outside are dropped
   (`stats.*_outside_domain`), so a superset fetch is harmless.
3. A cell is emitted iff it touches the KG's own units AND lies fully inside
   `input_bbox` (`stats.cells_outside_domain`; 0 for a whole-KG build).
4. K cells (res 10) are emitted only if fully inside `coverage_bbox`
   (default = `input_bbox`).
The peer's duty: supply a COMPLETE copy of every object intersecting
`input_bbox` — fetch every bevdirect 0.02° cell that intersects it (the cell
pad 0.004° makes crossing objects complete in one cell); from the index use
`ne_cells fetch` (strict, one call per layer over `input_bbox + 0.002°`, so an
object merely touching the edge does not depend on R-tree float rounding).
Verified: pad-0.004 and pad-0.008 index fetches of 63330 → the same digest.
A border cell appears in every KG it touches with its `kg_share`, identical
bytes otherwise — dedupe by cell_id. Omitted: cells with land < 0.02
(slivers) or kg_share rounding to 0.

### K record — 29 B, res 10 (~15 000 m²), only where n_units ≥ 5
`cell_id u64, n_units u16, unit_p10 u32, unit_p50 u32, unit_p90 u32, reg_n u16,
units_per_reg×10 u8 (0xFF none), str_n u16, str_p50 u16 (0xFFFF unless str_n ≥ 5)`.
Unit size = BEV attribute m² (integer, lower-median / index percentiles —
no float statistics). A res-10 cell not fully inside the input's coverage bbox
is withheld (truncated population).

### NEO1 (srtm's observed layer, `algo obs-cells-1`) — `POST /ne/{kg}/obs`
Header must carry `algo, epoch, source` (`srtm-lidar-at@2.4`), `kg, h3_res=12,
cells_n, groups` (== the 9-group order) and `digest`. The server validates
(v2.4 req 3):
- exactly one NEO1 section, ≤ 16 MB, digest = sha256(inflated block)[:16];
- the record block is `cells_n` fixed-size rows (9..64 B, first 8 B = cell id).
  **Row size is read from the header**: `record_bytes` (+ informative `fields[]`,
  echoed in the POST response and the manifest's `obs` entry); body must be exactly
  `cells_n × record_bytes`. Without `record_bytes` the size is derived as
  body/cells_n. srtm's current row is **41 B** (was 29 B in obs-cells-1) — we
  never hard-code it; rows are opaque beyond the cell id;
- the **cell set must equal our NEC1 `lu` cell set** for the KG (same ids, no
  duplicates) → otherwise **409** with `ours_n, yours_n, missing_sample,
  extra_sample, our_lu_digest`; `lu` not built yet → 409 too (post after the
  manifest says `built`).
- same `digest` as the stored obs → **200 `unchanged:true`**, nothing written;
  different → replaced atomically, **200 `replaced:true, previous_digest`**.

**NEC1 + NEO1 body (since 2026-10-06) — obs on a bev-equivalent declared layer.**
Why: a bevdirect@v0.3.1 build differs from `lu` (index-2026-03) by 1–3 border
cells on ~half the KGs (63330: 52 478 vs 52 476), so an obs computed on the
peer's own declared build would 409 against `lu` until 2027-04-30. Instead the
peer posts its NEC1 in front of (or after) the NEO1 in the same container:
- NEC1 header must have `algo == frozen.algo` (ne-cells-2) and `source` ∈
  `{frozen.bev_source} ∪ frozen.bev_equivalent_tags` (`neValidateBevNEC1`);
  anything else → 400. Record block `cells_n×30 + kcells_n×29`, no duplicate ids.
- the NEO1 cell set is compared against **that NEC1** — mismatch → 409
  `declared_layer:"bev"`. `lu` is never a 409 on this path.
- stored: NEC1 → the KG's `bev` section (`{kg}.bev.nec` → folded into `.nex`;
  replaces an operator-built one only if the digest differs, logged
  `bev_replaced`), NEO1 → `obs`. The stored NEO1 header gets
  `declared_source`/`declared_digest` of the layer it was validated against
  (also on the bare path: `index-2026-03` + lu digest). Digest-neutral — the
  section digest covers the record block only. If the posted header already
  carries `declared_*` they must agree (else 400).
- answer: `declared_layer, declared_source, declared_digest, bev{digest,cells_n,
  source,epoch}` and, when `lu` exists, `lu_diff {lu_cells_n, bev_cells_n,
  missing_n, extra_n}` (missing = in lu not in bev) — informational, surfaces
  border-cell drift. `unchanged:true` only when NEO1 digest, NEC1 digest and
  `declared_digest` are all unchanged.
- manifest (`api_ne_manifest.go`): `obs.declared_source`, `obs.declared_digest`,
  `bev.{digest,cells_n,source}`, counter `kgs_obs_on_bev` (obs whose
  declared_digest == bev.digest; also in `?since=` deltas). The peer's confirm
  step compares `obs.declared_digest` with `bev.digest` instead of lu `cells_n`.
- reads unchanged: `/ne/{kg}?layers=lu,bev,obs`; `/ne/cell/{h3}` stays
  lu-keyed (a bev-only border cell is 404 there) and joins obs by cell id.
Test: `TestNEObsOnBevDeclared`.
Rows are passed through untouched (we decode only the cell ids). The join is
`obs_str_n/obs_str_cover` vs our `str_n/str_cover` (verdict
`building_undeclared`). NEB1 (man-made objects as points with nDSM height) and
NET1 (tree apices) stay on srtm's side and are never posted.

## Determinism (why bit-identity holds)
1. **Canonical input** (`canon.py`): GeoJSON → `set_precision(1e-7)` →
   `make_valid` → polygonal parts only. **Serialisation is part of the
   contract: 7 decimals** (`coord_decimals: 7` in the header). The index
   (`coord`) and bevdirect-serve (`Options.Decimals = 7`, `/viewport` reports
   `coord_decimals`) emit 7; the 1e-7 snap only removes float noise below that.
   siedler's own `/api/viewport` serialiser emits 6 decimals — rounding at 1e-6
   is a different geometry (double rounding flips ~5 % of last digits), so it
   is NOT a valid NE input until it emits 7. Observers fetch from
   bevdirect-serve (its output is canonical).
2. **Dedupe**: units by id — a *complete* copy wins (largest, then WKB, if
   several); if only truncated copies exist (`complete:false` in every fetched
   cell, i.e. a unit wider than the 0.004° pad) their **WKB-sorted union** is
   used — "largest copy wins" lost land inside the cells where the other copy
   was the larger one (63330 full build: 52 448 → 52 478 cells). Structures
   and landuse pieces by `(ns, WKB)`.
3. **Order-free**: every union is over a WKB-sorted list; every map iteration
   is sorted; the record block is sorted by cell id.
4. **No source attributes except** unit `area_sqm` (BEV attribute, present
   in both sources), `status/rstatus`, `register`, `kg_code`, landuse `ns/code`.
   Structure area and orientation are computed from the geometry in the
   cell-local metre frame. Areas enter only as **ratios** (affine-invariant).
5. **Quantisation**: round-half-up on the same float, never banker's rounding.
6. Pinned GEOS: shapely 2.1.2 wheels bundle GEOS 3.13; a different GEOS can
   flip a last-ulp intersection. Install with `pip install ./ne_cells` (pyproject
   pins) or `pip install shapely==2.1.2 h3==4.5.0`.
Tests: `python3 tests_py/test_ne_cells.py` — shuffled/duplicated input and the
bevdirect-shaped input (split into two cells with `complete:false` copies) give
the identical record block.

## Decisions adopted from the review (ne-cells-0 → ne-cells-1)
- res 12 for lu/str_cover/obs; k≥5 fields → res 10 (K records).
- `bld_orient` only when str_n ≥ 5 (was: any str_n → 1° axis of one building).
- `str_cover` 1/255 (was 1e-4 → exact structure area recoverable).
- cov normalised server-side (tile-overlap double count → scaled, flagged);
  cov < 0.9 → `declared_unknown`; land < 0.02 omitted.
- Fixed group order; NS→group table `ns-groups-1` frozen with the algo id.
- epoch = ISO month, source ∈ {`index-2026-03`, `bevdirect`}; the header freezes the pair.
- Attribution + notice in every header; binary default, JSON variant; ETag.
- Border cells in every KG touched, with `kg_share`; stats KG-independent.
- Buildings win over landuse: March ns=48 pieces are not cut around structures,
  so `lu` of non-geb groups is `piece − geb` before measuring.

## Epochs — how new readings come back
Sources are pinned strings and never compared across each other:
`index-2026-03` (our country-wide copy) and `bevdirect@<tag>` (tile assembly;
the tag is part of the result — v0.1.0 vs v0.2.0 differ in ~25 % of cells by
the structure seam dissolve). bevdirect-serve ≥ v0.2.0 writes
`bevdirect_version` into every `/viewport` document and `ne_cells build`
derives `source` from it (mixed or missing → error; `--source bevdirect@v0.1.0`
to override for the old tarball). **All observers run one tag: `bevdirect@v0.2.0`
(git tag `bevdirect-v0.2.0` in this repo's history; the code now lives in the
public repo https://github.com/raffopenssh/vtcseamless, whose first tag `v0.3.0` is
the same assembly under the new module path — see `neBevEquivalentTags` for
which tags are treated as cell-equivalent).** index vs bevdirect inside one bbox differ by
source noise (lu up to 8/255, `declared_unknown` 1 953 vs 0) — never read that
as change: `/report` answers `identical_source:false, compared:false` when we
hold no section with the report's exact source string.
Epoch 0 for bevdirect reports = our `bev` layer (`63330.bev.nec`, built by the
operator from a local bevdirect-serve v0.2.0 over the whole domain).
```
python3 -m ne_cells build --kg 63330 --epoch 2026-10 \
    --bevdirect cell_755_2356.json --bevdirect cell_755_2357.json … \
    --input-bbox W,S,E,N -o today.nec     # bbox = interior where the fetch is complete; source from the docs
python3 -m ne_cells report today.nec --observer siedler-vm-1 \
  | curl -X POST -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
      --data-binary @- https://umfeld-at.exe.xyz/contrib/api/v1/ne/63330/report
```
The report carries ONLY digests (whole build + one per res-10 chunk) and keeps
only chunks whose **every res-12 child** lies inside the bbox (default =
`coverage_bbox`; H3 children poke out of their parent, so testing the res-10
hexagon alone was wrong). The answer: `identical_source, compared,
chunks_compared, chunks_same, chunks_changed, chunks_unknown_to_us,
our_digest, our_sources`. Change units are then confirmed by a second observer
or by the peer posting the differing chunks' records — endpoint follows once
real reports arrive. Reports: `data/ne_cells/{kg}.reports.jsonl`.
bevdirect-serve itself does not compute cells (a Go port would break
bit-identity; peers call the python package).

## Observers — who may report, how it is watched {#observers}
Since 2026-10-04 (`api_ne_reports.go`) `POST /contrib/api/v1/ne/{kg}/report` is open
to **every private token and every contributor token**, not only the first.
`/contrib/…` is the prefix peers are told about (same handlers; `/…` keeps
working as the ops alias — never mention `/k` to a contributor). siedler-oesterreich (a
**`CONTRIBUTOR_TOKEN`**, no survey access) is the first observer running it in
production; srtm's ~24 peers use the identical protocol with the first token.
Writes of the obs layer and the draft share stay first-token only. Since
2026-10-06 the obs POST also takes `NEC1+NEO1` (the peer's own bevdirect
declared build → `bev` section, obs validated against it, `lu_diff` instead of
a 409 — §NEO1); srtm switches per peer, one a day, once the manifest shows
`obs.declared_digest`.

**Peer tooling (2026-10-06): the public Python package https://github.com/raffopenssh/vtcseamless-py**
(`pip install git+…`, pointer `clients/vtcseamless-py/README.md`). `vtcseamless observe --kg K
--observer L` = resolve domain from the own units (admin bbox is only the guess) → fetch the
aligned 0.02° cells from a bevdirect-serve (`Server` downloads the v0.3.0 release binary if none
runs) → `ne_cells build` → `POST /contrib/…/report`. Ships our `ne_cells` (frozen files
byte-identical, `scripts/ne_cells_sync_check.sh`; its CLI has no `/` fetch). Verified: 63330
→ `594025647ec3557e`, 1163/1163 same vs our `bev` section. The Go engine stays the only
assembler — a Python port would be a new, incomparable source.

Rules (same for every peer — one CLI command, one endpoint, nothing else to manage):
- **Attribution by token**, not by the self-declared `observer` string: the
  stored line and the answer carry `token` (name from `CONTRIBUTOR_TOKEN_NAMES`,
  or `srtm-lidar-at`/`ops` for the survey tokens);
  `observer` is the peer's own machine label (≤ 64 chars) — srtm peers use
  `srtm-peer-N`, siedler `siedler-<host>`.
- **Baseline rule** (`baseline` in every answer): `section:bev.nec` when we
  hold a section of that source (only 63330 today) — else the baseline is
  **per chunk** (since 2026-10-05): over every earlier full report of the same
  `(source, algo, chunk_res)` for that KG, each res-10 chunk is pinned to the
  digest it was *first* reported with, by any observer. A report whose chunks
  overlap that union is compared (`baseline:"report"`, `chunks_same/changed`,
  `baseline_reports`, `baseline_chunks`, `baseline_observer` = first report);
  chunks never seen before are answered as `chunks_unknown_to_us` and become
  baseline right away (`chunks_baselined_now`); a report with no overlap at
  all is `baseline:"this_report", compared:false`. This matters because
  siedler posts **viewport-sized** reports (2–6 km, 70–700 chunks) — the first
  cut (per-report baseline) left every later viewport of the same KG as
  "unknown" forever. `bevdirect@v0.2.0 ≡ v0.2.1 ≡ v0.3.0 ≡ v0.3.1`.
  So a country-wide observer bootstraps its own baseline on the first pass and
  gets `chunks_changed` from the second pass on — no operator build needed.
- **Change over time**: `chunks_changed` is always *vs epoch 0*; every answer
  additionally carries `since_last {compared, same, changed[], previous_received_at}`
  = the report vs the **most recent** digest we hold per chunk (from earlier
  reports of the same source, any observer; also on the `section:bev.nec`
  path). A chunk that changed once and then stayed stable is in
  `chunks_changed` but not in `since_last.changed`. `unchanged:true` stubs
  carry `since_last.same = all`. Counters/`on_disk` expose
  `chunks_changed_since_last`; each POST logs one `🧾 NE report` journal line
  (NE routes bypass the `📨` access log).
- **Dedupe**: same digest+epoch as the token's previous report of that KG →
  `unchanged:true`, stored as a stub without chunks.
- **Rate**: 2 POST/s per token, burst 30 → `429` + `Retry-After` (never
  retry inside the Retry-After; the peer queues). Body ≤ 8 MB, ≤ 50 000 chunks,
  `digest`/chunk values 16 hex chars, chunk keys res-10 H3 hex.
- **Watch**: `GET /api/v1/ne/reports` (any private token) — `live` counters
  per token×observer since process start (reports, 400s, 429s, compared,
  became_baseline, unchanged, chunks_same/changed/unknown, last_note),
  `last_50` outcomes, `on_disk` scan of `*.reports.jsonl` (memo 60 s, top 50
  KGs by changed chunks). `ne_reports` line on `/api/v1/debug/inflight`.
  Per KG: `GET /api/v1/ne/{kg}/report`.
- Admin: add a report-only peer = append a token to `CONTRIBUTOR_TOKEN` and
  its name to `CONTRIBUTOR_TOKEN_NAMES` in `/etc/survey-api.env`, restart
  (`api_tiers.go` opens only `/contrib/api/v1/ne/{kg}/report`, `/ne/reports`,
  `/ne/manifest`, `/ne/{kg}` reads for it — identical under `/k` — the same 404 as no token anywhere
  else). Peers that also need the survey (srtm) keep a `API_TOKEN`,
  which reports as well. Rotate one token alone; nothing per peer on disk.

## v2.4 requirements adopted (srtm, 2026-10-04) {#v2.4}
1. **Coverage** — `ne_all.sh` builds `lu` for every local KG (sorted order,
   ~50 s/KG → ~4–5 days for 7 850; python reference, strict fetch, never a
   partial); unbuilt → 404 + `Retry-After` (§coverage). Failed KGs are listed
   in `queue.failed` and retried after the pass (automatic retry pass in `ne_all.sh`).
   Oct 2026 lesson: 163 "FAIL 1s" were `Connection refused` during API restarts
   (deploys) — `ne_cells fetch` now rides out transport errors / 5xx for up to
   ~10 min, and the FAIL log line prints the reason (`tail -n 2`, not `tail -2`).
   `FAIL: no units of KG X in (bbox)` while the KG file and `units` rows are
   fine ⇒ a persisted "built, 0 rows" `unit_geom_kg` entry (44014/44036, from a
   Sept-2026 pre-fix build); the fetcher reads via the spatial R-tree, which is
   then empty. Check `SELECT kg_code FROM unit_geom_kg WHERE unit_count=0`,
   delete those rows (+ `structure_geom_kg … structure_count=0`), then
   `./survey-server -prewarm-kgs -prewarm-kg-list A,B` (separate process, ~5 s/KG)
   and let the retry pass pick them up.
2. **Manifest** — `/ne/manifest` (§manifest).
3. **OBS** — cell-set validation, unchanged/replace semantics, header-declared row size (§NEO1).
4. **Stability** — §FREEZE; `frozen{}` in every answer.
5. **bootstrap.sh FROM_SOURCE** — version = `git describe --tags --match
   'bevdirect-v*' --always --dirty | sed s/^bevdirect-//` (same as
   `tools/package.sh`): on the tag → `v0.2.0` → source `bevdirect@v0.2.0`;
   past it → `v0.2.0-3-gabc1234`, which is correctly ≠ the pin. The clone is
   blobless-but-full-history so tags resolve; `VERSION=bevdirect-v0.2.0`
   checks the tag out. *Since the move to the public repo
   https://github.com/raffopenssh/vtcseamless (first public tag `v0.3.0`) the tags
   are plain `v*` (`--match 'v*'`, `VERSION=v0.3.0`) and the clone is
   unauthenticated; the source string stays `bevdirect@<tag>`.*
6. **`inputs[].sha256` is now a content hash** (`hash_rule: content-sorted-rows-1`):
   per layer list, rows → canonical JSON (sorted keys, compact, ascii) →
   sorted → `sha256("<layer>\n" + rows joined by "\n")[:16]`; document keys
   (`fetched_at`, `bbox`, …) excluded. `file_sha256` keeps the bytes-as-fetched.
   63330 index inputs: units `372245087a0e4916` (5 495 rows), structures
   `f1e471374056d246` (7 534), landuse `0855881267a9425f` (21 660). Informative
   only — the record digest is the contract.

## Fixes adopted from srtm's check of ne-cells-1 (2026-10-03)
1. Served 63330 was built on a partial landuse set (`/spatial/landuse`
   answered `ready:false` with 6 506 of 14 809 rows, and a KG whose edge
   polygons reach past its centroid-derived bbox was not even a candidate).
   Server: `?strict=1` → 202 + no rows when not ready; viewport KG candidates
   padded 0.01°. Builder: refuses `ready:false`/`truncated` inputs; `ne_cells fetch`.
2. Input extent pinned (§domain) — digest independent of the fetch pad.
3. bevdirect tag in `source`; v0.2.0 is the pinned tag; digests above.
4. Index epoch is not the change baseline: source-matched reports, `bev` baseline.
5. 7-decimal serialisation stated as contract.