Docs/Realms/DarkerDB

DarkerDB

A community data realm for Dark and Darker, hosted as part of the KATFORGE API. All observation data lives in katforge.darkerdb.* schema alongside the other realms (stumper, lextris, gear_goblins).

The public surface is api.darkerdb.com/v2/* and identically api.katforge.com/v1/realms/darkerdb/* — same Symfony controllers, same DB, dual mount.

The complete API reference lives at API Reference → DarkerDB. Everything below describes the realm itself: how data flows, what feeds it, what the schema looks like.

What's in it

Two data sources feed the realm. The Ghost bot owns writes through a dedicated DB role; the API runs on a SELECT-only role.

Ghost bot — a single-Steam-session observer that connects to the live game server and watches packet broadcasts.

Surface	Table	Source packet
Market listings	`darkerdb.market`	`S2C_MARKETPLACE_ITEM_LIST_RES` (and register/cancel events)
Server population	`darkerdb.server_population`	`S2C_PLAYERLIST_RES` (population fields)
Trade chat	`darkerdb.trade_chat`	`S2C_CHAT_MESSAGE_NOT` on the global trade channel
Leaderboards	`darkerdb.ranking_*`	`RANKING_INFO_RES` (auto-discovers seasons/sheets) + `RANKING_RANGE_RES` (entries)
Merchant snapshots	`darkerdb.merchant_stock`, `merchant_quests`	Periodic merchant captures
Character profiles	`darkerdb.characters`, `character_levels`, `character_ranks`	Playerlist + leaderboard observation

Codex pipeline — @katforge/codex's extractor pulls structured game data out of the patch's signed .pak files. See Codex → Dark and Darker extraction.

Surface	Table	Source
Items	`darkerdb.items`	Full attribute matrices, icons, descriptions
Maps	`darkerdb.maps`	Gameplay map metadata + minimap PNGs
Loot tables	`darkerdb.loot_tables`, `loot_entries`	Drop weights, conditions, tier filters
Spawners	`darkerdb.spawners`	Where each loot table / monster spawns on each map
Monsters	`darkerdb.monsters`, `monster_abilities`	Stat blocks, abilities, behaviors, family / tier
Monster models	`darkerdb.monster_models`	Binary glTF meshes for in-browser 3D rendering

Architecture

text

                          ┌─────────────────────────────────┐
                          │       Dark and Darker           │
                          │       game server               │
                          └────────────┬────────────────────┘
                                       │ (Steam-authed single session,
                                       │  request/response correlated
                                       │  by sequence number)
                          ┌────────────▼────────────────────┐
                          │       Ghost bot                 │
                          │   realms/darkerdb.com/ghost     │
                          │                                 │
                          │   Market stream                 │
                          │   Leaderboard auto-discovery    │
                          │   Population poll               │
                          │   Trade chat subscribe          │
                          │   Merchant capture              │
                          └────────────┬────────────────────┘
                                       │ Knex (searchPath: darkerdb)
                          ┌────────────▼────────────────────┐
                          │      katforge-db (Postgres)     │
                          │      schema: darkerdb           │
                          └────────────▲────────────────────┘
                                       │ Doctrine (default EM, schema-qualified entities)
                          ┌────────────┴────────────────────┐
                          │      api.katforge.com           │
                          │   src/Realm/DarkerDB/           │
                          │                                 │
                          │   /v1/realms/darkerdb/*  (canonical)
                          │   /v2/*                  (shadow on api.darkerdb.com host)
                          └─────────────────────────────────┘

API surface at a glance

Eleven endpoint groups under /v1/realms/darkerdb — full request/response shapes are documented in the API Reference.

Group	Endpoints	Source
Health	`GET /health`	static
Population	`GET /population`, `/population/history`	bot
Items	`GET /items`, `/items/{id}`	codex
Maps	`GET /maps`, `/maps/{id}`, `/maps/{id}/image`	codex
Monsters	`GET /monsters`, `/monsters/{id}`, `/monsters/{id}/abilities`, `/monsters/{id}/loot`, `/monsters/{id}/portrait`, `/monsters/{id}/model`	codex (+ join to bot for loot)
Loot	`GET /loot-tables`, `/loot-tables/{id}`, `/spawners`, `/spawners/{id}`	codex
Merchant	`GET /merchant/stock`, `/merchant/stock/{merchant}`, `/merchant/stock/history`, `/merchant/quests`	bot
Leaderboards	`GET /leaderboards/seasons`, `…/{season}`, `…/{season}/sheets`, `…/{season}/sheets/{sheet}`, `…/periods`, `…/entries`, `…/characters/{name}/history`	bot
Leaderboards (legacy shim)	`GET /leaderboards`, `/leaderboards/{id}`	bot, mapped from old CSV ids
Trades (planned)	global trade chat queries	bot
Characters (planned)	per-character history queries	bot

How patches roll forward

When @katforge/codex's extractor picks up a new patch:

bin/protodump <build> — extracts fresh protobuf descriptors from the game binary (DungeonCrawler.exe). Output is committed to the codex repo under protos/<build>/.
The ghost bot is rebuilt and restarted; it picks up the new protos via the symlinked ghost/src/packets/ directory.
bin/console codex:bootstrap dark-and-darker --patch-version=<current> re-imports static content (maps, loot, monsters, models) into the darkerdb.* schema.
Leaderboard schema doesn't need a touch — the bot auto-discovers seasons and sheets from RANKING_INFO_RES on the next cycle.

See Codex → Dark and Darker extraction for the full operator runbook (including the protodump and gltf sub-pipelines).

← PreviousRealms