# q4 - Quake 4 in the browser

Unofficial, non-commercial OpenQ4 browser port. The browser runs the original
single-player or multiplayer GameLib against an operator-authorized Quake 4 data
catalog cached in browser-private storage; a native OpenQ4 dedicated process is the
authoritative public multiplayer server. Production is `https://q4.suby.pl` on
gateway port 3280 and game UDP port 28004.

Read `/srv/GAME.md` before changing cross-game contracts. Deployment and recovery are
in `DEPLOY.md`; area notes live in `port/` and the data-mod pipeline in
`mods/README.md`.

## Architecture

```text
browser
  | HTTPS: shell, immutable WASM, CC0 runtime, gated retail catalogs, replay API
  | WSS /game/<port>: one binary WS frame == one UDP datagram
  v
server.js :3280 (Node ESM, dependency: ws)
  |- public/                 UI, PWA, touch/gamepad, identity, voice HUD
  |- engine/                 frozen SP and MP OpenQ4 WebAssembly builds
  |- content/assets/baseoq4/ minimal CC0 pak0/pak1 + approved CC0 map PK4s
  |- /retail/                full 23-file fallback + common/per-map SHA allowlists
  |- server/udp-bridge.js    rate-limited WS <-> loopback UDP bridge
  |- native OpenQ4 ded       authoritative DM, bots, full-state .q4sd recording
  `- replay playback pool    isolated native processes/UDP ports, join-anytime

browser OPFS
  /q4-retail/revisions/<revision>/runtime/q4base  verified full or accumulated map PK4s
  /q4-runtime/<build>/baseoq4                    shared self-authored CC0 data
  /q4-user                                       saves/config
```

The production operator has confirmed authorization for this deployment to deliver
the required retail data. The gateway exposes it only when
`Q4_RETAIL_DELIVERY=operator-authorized`. The full fallback is the exact catalog
`pak001.pk4` through `pak022.pk4` plus `zpak_english.pk4`. A second catalog repacks the
same approved source revision into one deterministic common PK4 and deterministic
per-map PK4s. Both catalogs use content-hashed URLs; the gateway binds the bundle
manifest to the active 23-file source identity and verifies every staged bundle before
serving it. The browser verifies every SHA-256 before activating a revision in OPFS.
There is no generic retail directory mount. `assets-src/` and `runtime/` remain ignored
operator state, outside Git, the public tree, generated content, and the source offer.
The native server uses the same operator data via
`runtime/native/q4base -> assets-src/gog/q4base`. Do not move that symlink or its target
below `public/`, `content/`, or `engine/`.

Public `pak0.pk4` and `pak1.pk4` are deterministic, self-authored CC0 integrity
packs; they contain configuration/license text and two small declaration-compatibility
files, not upstream or retail media. The
approved q4web arena addon is also self-authored CC0. Staging audits every public PK4
entry against the private retail corpus by path and content hash; the allowlist is
empty by default.

## Engine variants

- `singleplayer`: the GPL OpenQ4 Emscripten `MAIN_MODULE` loads the separately built
  OpenQ4-GameLibs `src/game` `SIDE_MODULE` through `GetGameAPI`.
- `multiplayer`: the GPL OpenQ4 Emscripten `MAIN_MODULE` loads the separately built
  `src/mpgame` `SIDE_MODULE`; it must retain the multiplayer wire protocol
  (`ASYNC_WRITE_TAGS=1` and EVENT_ACK) and the local bot port.
- `replay`: normally the MP artifact connecting as a spectator to a playback port.

The variants and their GameLib side modules are deliberately separate in
`engine/engine-manifest.json`. The loader verifies each side module's SHA-256 and
stages it in WasmFS before the engine calls `dlopen`. Never point both entries at an
SP build just because it links, and never fold a GameLib back into the engine module.
The native server is the protocol oracle; validate a real WASM connection after any
GameLib/network change.

OpenQ4 engine code is pinned to `975e296b80b66f63b817d16ea78b53e0685c5c55`
and GPLv3.
OpenQ4-GameLibs is pinned to `eb26e5f3a52358c58ee00bee5602575e5b057164` and remains
under the Quake 4 SDK Limited Use EULA, not GPL. Full local notices are under
`public/legal/`; `tools/stage-source-offer.sh` publishes the exact modified source as
a code-only snapshot which omits upstream media without established redistribution
rights.
The WebAssembly module boundary mirrors OpenQ4's native dynamic GameLib boundary; it
is an implementation fact, not a determination that resolves licensing questions.

## Web platform contracts

- WebGL 2 / GLSL ES 3.00 only. The renderer implementation is under the pinned
  OpenQ4 tree at `src/renderer/webgl2`; keep shader link failures fatal and tested.
- Emscripten mounts OPFS with WasmFS at `/opfs`. `fs_basepath` points to the active
  `/q4-retail/revisions/<revision>/runtime` tree containing retail `q4base`. For a
  bundled multiplayer revision this tree accumulates the shared PK4 plus each map PK4
  already used by that browser; a complete server/manual 23-file revision also remains
  mountable. `fs_cdpath` points to
  `/q4-runtime/<build>` containing only the minimal CC0 `baseoq4` packs/addon;
  `fs_savepath` points to user data.
- `Module.q4WebSocketUrl` is same-origin `ws(s)://<host>/game/<port>`. Never hardcode
  `suby.pl`, because the same build must work through `*.z6.mm.mk:11443`.
- JSON text frames on the game socket are reserved for WebRTC voice signaling. Binary
  frames are opaque idTech datagrams. PTT is `T`; voice is created only for a human
  multiplayer client, never for bots or replay spectators.
- `public/js/voicechat.js` is an absolute symlink to the canonical
  `/srv/_shared/voice/voicechat.js`. Do not replace it with a copied file or edit it
  here. `voicehud.js` is intentionally local.
- The cookie/beacon/direct-route helpers in `public/js/identity.js` must remain aligned
  with `/srv/gry/suby-identity.reference.js` for both domain families.
- `/healthz` must keep a numeric live-client field, `retailBundles:true` for a
  publishable operator-retail deployment, and
  `Cross-Origin-Resource-Policy: cross-origin`; safe restart and direct-route probing
  depend on these fields and the header.

## Content and mods

`tools/import-retail.sh` extracts the operator's GOG installer into the ignored retail
directory. `tools/stage-retail-manifest.mjs` hashes the exact 23-file English catalog
and writes the ignored full deployment manifest. `tools/stage-retail-bundles.py`
consumes the version-bound native I/O profiles in
`runtime/retail-bundle-profiles/manifest.json`, resolves PK4 override winners, and
emits disjoint deterministic common/per-map packs plus
`runtime/retail-bundle-manifest.json`. Run it normally to rebuild and with `--verify`
to re-hash and structurally validate the staged graph against the exact source catalog.

For a covered multiplayer map, `public/js/importer.js` requests
`/retail/bundles/manifest.json`, resumes missing bundle bytes with bounded HTTP Range
requests, verifies SHA-256, and retains the packs under the bundle revision in OPFS.
Starting the same map again does not fetch its PK4 payloads; switching maps downloads
only missing packs and keeps the earlier maps as a persistent browser-private cache.
A 404/410 bundle manifest or a selected map absent from its graph falls back to the
full exact 23-file server catalog. Other bundle errors fail visibly instead of silently
activating incomplete data. A complete 23-file server/manual import takes precedence;
manual import still requires pak001-pak022 plus one language zpak and never uploads
selected local files.

The active complete bundle graph covers all 23 multiplayer maps and all 62 supported
map/mode scenarios. Its common pack is 351,993,687 bytes, individual map packs are
487,469-17,636,172 bytes, and the complete retained graph is 537,679,229 bytes.
Starting a first map therefore needs 352,481,156-369,629,859 bytes instead of the
2,779,708,326-byte full retail catalog; later maps need only their missing map pack.
`retailBundles:true` proves that the staged graph is valid and active, while the
all-scenario E2E evidence proves complete menu coverage.

Retail bot navigation is a separate, server-only catalog covering exactly all 22
retail multiplayer maps. Its primary path runs the pinned local OpenQ4 compiler with
the compiler-only `retail-playerclip-aas32-v1` profile: `playerClipSolid=1` makes
Quake 4's player-clip brushes solid for AAS generation, while the temporary
`info_player_team` seed makes team-spawn-only maps discoverable. Neither compiler
override is copied into the resulting PK4. If that primary attempt fails, staging may
use a map-specific Q4Max or SABot AAS only for an exact allowlisted compiler-failure
signature and only when the source archive, inner PK4, entry bytes, map CRC, size, and
SHA-256 all match the pinned profile. The fallback copies the verified AAS bytes and
renames the entry; it does not mutate the graph. Any unrecognized failure, missing
profile, or identity mismatch aborts the complete 22-map stage.
`mp/q4xdm13` is the deterministic-source exception: staging uses its pinned SABot
entry directly because the local compiler fails nondeterministically both before and
after a complete write.

The Q4Max/SABot source archives under `build/native-aas/`, compiler logs, generated
navigation PK4, and `runtime/retail-botnav-manifest.json` are private operator state.
They are never browser content, retail routes, or members of the public source offer.
Verified navigation and supported bot behavior are different contracts: all 22 retail
maps have AAS, but `botModes` enables bots only for `DM` on the 17 retail maps that
offer DM. Team DM, CTF, Arena CTF, and Tourney remain human-only.

`node tools/ingest.mjs <map.pk4> --license ... --source ... --author ...` validates a
data-only mod, compiles missing `.proc`, `.cm`, and `.aas32`, rejects executable code
and archive attacks, emits `mods/dist/q4web_<slug>-<hash>.pk4`, and restages content.
Every multiplayer map must include valid `DewmAAS 1.08` navigation. The CC0 test map
source is `mods/maps/q4web_arena/`.

## Replays

Every native match records a finalized full-state `.q4sd`; `.part` files are never
listed. The API retains the newest 300. Watching a replay must allocate/reuse a
dedicated playback instance and return a spectator port. A second viewer joining after
playback has started must receive current state through ordinary snapshots. Downloading
the file and replaying from timestamp zero is useful for diagnostics but does not meet
the join-anytime contract.

## Build and test

```bash
tools/sync-upstream.sh
tools/import-retail.sh                       # once; proprietary output is ignored
node tools/stage-retail-manifest.mjs          # ignored, exact 23-file server catalog
python3 tools/stage-retail-bundles.py         # ignored common/per-map PK4 graph
python3 tools/stage-retail-bundles.py --verify
tools/build-native.sh --clean --jobs 4
node tools/stage-retail-botnav.mjs             # private AAS for all 22 retail MP maps
node tools/stage-retail-botnav.mjs --verify
node tools/smoke-retail-botnav.mjs
tools/build-wasm.sh --clean --jobs 4          # canonical SP/MP build/stage entrypoint
node tools/stage-web-content.mjs
npm test
npm run test:zip-cache
node vendor/openq4/tools/tests/webgl2_shader_smoke.mjs
python3 tools/test-wasm-network.py
Q4_E2E_URL=http://127.0.0.1:3280 npm run test:e2e:shell
Q4_E2E_URL=http://127.0.0.1:3280 npm run test:e2e:bundle-cache
node tools/verify-retail-bundle-cache-evidence.mjs
Q4_E2E_URL=http://127.0.0.1:3280 npm run test:e2e:bundle-maps
npm run verify:e2e:bundle-maps
```

Before release, also run the native bot/demo/playback smokes in `port/native/`, the
retail navigation smoke (`node tools/smoke-retail-botnav.mjs`), the WebGL shader
smoke, the WASM network test, and the full Playwright suite against the enabled local
retail catalogs. The navigation smoke loads all 22 retail maps. On each of the 17 DM
maps it also parses bounded `botStatus` telemetry and requires a live bot to produce
movement input (`|forward| + |right| >= 16`) and at least 64 units of same-life XY
displacement in a valid AAS area. Death, spectating, respawn, teleport state, or a
movement discontinuity above 512 units resets the sample instead of counting it as
movement. Patch `0034` puts real bot clients on the standard server respawn path while
preserving the separate fake-client replay path, so
spectator-camera motion cannot pass the gate. Non-DM maps load and validate AAS
without spawning unsupported bots. The
bundle-cache E2E must use a fresh Chromium profile and prove first-map common+map
downloads, zero repeated-map payload, map-only download
on switch, retained earlier maps, and no full-catalog request. Inspect desktop and
mobile screenshots, not only exit codes. Its schema-3 evidence is bound to the exact
bundle-manifest bytes, cache/importer gateway code, and SHA-256 of the files actually
stored in OPFS. It is an independent publication requirement alongside the schema-4
full-engine evidence.

After changing per-map profiles or bundles, run the all-map bundle E2E as well. It
requires the bundle manifest to cover the complete multiplayer menu, uses one fresh
persistent Chromium profile, and starts all 62 supported map/mode scenarios through the
HTML launcher. Only the first mode for a map may download its still-missing bundles;
every later mode for that map must fetch zero PK4 payload. Every scenario has one
absolute deadline and requires the native server to report the exact map/mode, a first
in-game snapshot, no accumulated fatal/`InvalidGUI`/missing GUI, model, image, cube,
animation, sound, material, or shader diagnostics, and a non-black central canvas
capture. It finally returns to the first scenario with zero PK4 payload, enumerates the
actual OPFS `q4base` directory, and hashes every entry under a separate bounded
deadline; extra, missing, or non-file entries fail verification. Evidence is written
to `.selfcheck/retail-bundle-maps-e2e.json` and checked by
`npm run verify:e2e:bundle-maps`.

## Operational traps

1. Restart only with `tools/safe_restart.sh`; exit 2 means leave the live service alone.
2. Caddy's admin-API route is ephemeral. `/etc/cron.d/q4-caddy-watchdog` must remain.
3. Never add vendor Git databases, retail files, native runtime, replays, or generated
   content to Git. Frozen files in `engine/` are intentionally tracked.
4. OpenQ4 startup can look healthy before a map loads. Release evidence requires an
   actual rendered retail map and an actual MP connection, not just `main()` or shader
   compilation.
5. Keep SP and MP source staging isolated. PCH reuse across them can silently compile
   the SP protocol into a file named as MP.
6. A map that renders but has no valid `.aas32` is not navigation-ready. Valid AAS
   alone does not authorize a bot mode: only the 17 retail DM maps currently advertise
   bots, and release evidence must prove real movement rather than entity presence.
7. Public shell changes are live without restart; Node/native/engine changes require a
   drained restart. Restage the source offer and content/retail manifests before
   release; `retailBundles:false` is never publishable.
