Local

The local package is the Datahoster code that runs on your own machine. You download one versioned directory, datahoster-local-<version>, and run the one command inside it, datahoster-local.js. That command parses the command line, loads your config, and hands off to one script per job: sync (push your definitions, import your records, mirror the data down) and the local browser tools. The package is in four parts: localhoster-common/, the shared toolkit every hoster carries; localhoster-tools/, the two universal browser tools; datahoster-tools/, this hoster's own Data Logs tool; and datahoster-sync/, the sync command with the definition and storage engine it drives. The directory you see here is exactly the directory in the zip. This page is the file map. Every file is listed, then a section gives its exports, or, for the command you run, its switches. The storage it mirrors has its own page.

datahoster-local-<version>/          unzips from datahoster-local-<version>.zip
  datahoster-local.js              the one command you run: parse args, load config, dispatch
  config.example.json              the config template (copy to config.json)
  localhoster-common/               the shared toolkit, identical across every hoster
    index.js                      the single require surface (loadConfig, toolsSite, the UI kit)
    net.js                        config load, bare-IP TLS, API-version
    run.js                        the tools-site frame + per-tool run state
    shell.js                      the page frame: strip, tabs, status, stylesheet
    ui.js                         the UI kit: cards, tables, charts, pagination
    format.js                     formatters: esc, numbers, bytes, dates
    io.js                         small render/IO helpers
  localhoster-tools/                the universal tools, shared across every hoster
    web-logs/
      index.js                    run(config): pull the access logs, then build the browser
      lib/
        pull.js                  mirror the access logs down
        render.js                build the Web Logs pages
    server-logs/
      index.js                    run(config): pull the stats, then build the browser
      lib/
        pull.js                  mirror the server stats down
        render.js                build the Server Logs pages
  datahoster-tools/                 this hoster's own tool
    data-logs/
      index.js                    run(config): read the defs + mirror, then build the browser
      lib/
        render.js                render the per-host Data Logs pages
        paths.js                 path-safety and naming rules
        schema.js                definition validation
        store.js                 the columnar storage engine (read the mirror)
  datahoster-sync/                  the sync command and the engine it drives
    index.js                      run(config): push defs, import records, mirror data down
    lib/
      scan.js                     read the content root into definitions and imports
      paths.js                    path-safety and naming rules
      schema.js                   definition and submission validation
      import.js                   read and validate a collection's import records
      store.js                    the columnar storage engine (verify the mirror)
      version.js                  the API/protocol version

The datahoster-local package: the entry, the shared toolkit, the universal and datahoster-specific tools, and the sync command with its engine.

Files

File	What it does
the entry
`datahoster-local.js`	The one command you run. Parses the command line, loads the config, and dispatches to each command's own script. Holds no logic itself.
`config.example.json`	The config template: your server address, its token, your data folder, and where the tools pages are built. Copy it to `config.json`.
`localhoster-common/` — the shared toolkit
`localhoster-common/index.js`	The single require surface the entry and every tool use: `loadConfig`, the `toolsSite` frame builder, `recordRun`, and the UI kit.
`localhoster-common/net.js`	Reads and validates the config into a frozen object, skips cert verification for a bare-IP endpoint, and discovers the server's API version.
`localhoster-common/run.js`	The tools-site frame: `toolsSite(config, brand)` builds the overview + tabs + clash-safe status; `recordRun` persists each tool's last run.
`localhoster-common/shell.js`	The shared page frame: the top strip, tool tabs, per-tool status banner, and stylesheet.
`localhoster-common/ui.js`	The shared UI kit a browser composes: stat cards, tables, chart/table toggles, and pagination.
`localhoster-common/format.js`	The canonical formatters: escaping, numbers, bytes, durations, and month/day labels.
`localhoster-common/io.js`	Small render/IO helpers, chiefly the footer-stripping used by the write-if-changed comparison.
`localhoster-tools/` — the universal tools
`localhoster-tools/web-logs/`	Web Logs: `index.js` is the tool (`run(config)`); `lib/pull.js` mirrors the Caddy access logs down and `lib/render.js` builds the browser.
`localhoster-tools/server-logs/`	Server Logs: `index.js` is the tool; `lib/pull.js` mirrors the server statistics down and `lib/render.js` builds the drill-down browser.
`datahoster-tools/` — this hoster's own tool
`datahoster-tools/data-logs/`	Data Logs: `index.js` is the tool; `lib/render.js` reads the definitions and the mirrored records and renders the browsable per-host map. Its `lib/paths.js`, `schema.js` and `store.js` are the tool's own copies of the definition and storage engine. Touches no API.
`datahoster-sync/` — the sync command + engine
`datahoster-sync/index.js`	The sync command (`run(config)`): push the local definitions by content hash, import the local records, and mirror the append-only columnar data down.
`datahoster-sync/lib/scan.js`	Walk the content root into the definitions to push, the collections and their schemas, and the import files per collection.
`datahoster-sync/lib/paths.js · schema.js · import.js · store.js · version.js`	The sync's own copies of the definition, import, storage and version engine (the server carries its own). Detailed on Storage and Sync.

`datahoster-local.js` — entry point

The one command you run for the server tools, and the only file with no exports. It parses the command line, loads your config into a frozen object with loadConfig, and dispatches to one script per job: sync first (so content is published before the browsers render it), then each requested browser tool. When they have run it reassembles the shared tools site with toolsSite (the overview, the tab strip, and each tool's status). It holds no logic itself: one explicit line per command, and the config it hands each script is read-only.

Switch	What it does
`--config <path>`	Required. The full absolute path to a `.json` config file. A missing path, another flag, a relative path, or a non-`.json` file is rejected.
`--sync`	Push your definitions to the server, import your records, then mirror the data down.
`--data-logs`	Build the Data Logs browser from your definitions and the mirrored data.
`--web-logs`	Mirror the access logs down and build the Web Logs browser.
`--server-logs`	Mirror the server stats down and build the Server Logs browser.
`--all`	Everything in one process: sync and all three browser tools.

`datahoster-sync/index.js`

The sync command. Its run(config) takes one pass with the frozen config from the entry. It first pushes your definitions: it scans the content root and, one host at a time, diffs the local definitions against that host's online schema. The schema is append-only, so a collection or column that is missing locally or was changed is drift: that host is left untouched and reported with the definition to restore, while the others upload or delete only their differences. It then runs the import phase, sending only the records whose source files changed, and finally mirrors the server's append-only columnar data down into <contentRoot>/_data/<host>/datahoster-data/…, fetching each file whole once and appending only the growing tail of the current chunk by Range. Nothing stays resident. The API-version prefix is pinned to the package's own lib/version.js, so a version mismatch fails fast. The full protocol is the Sync page.

Export	What it is
`run(config)`	One sync pass: push the definitions, import the records, mirror the data down, then decode-check the mirror. Returns `{ collections, hosts }`.
`push(cfg, api, scanned, log)`	Per host, diff the local definitions against the online schema; push the clean hosts (append-only: add collections/columns, freely change users/endpoints) and return the drift for any host whose collection or column is missing or changed.
`importSync(cfg, api, scanned, log)`	Load each collection's import records: validate every file first (a bad record stops the phase, naming file and field), then per collection pre-check by file time (which also deletes any import-owned record no longer in the files) and `PUT` only the records that changed.
`mirror(cfg, api, hosts, log)`	Mirror each host's append-only columnar files under `_data/<host>/datahoster-data/…`: fetch each file whole once, and append only the growing tail of the current chunk by Range.
`makeApi(cfg)`	Build the management-API client (manifest, def, import check + put, data list + file) bound to the server and admin token, with the version prefix pinned to the package's own version.

`datahoster-tools/data-logs/index.js`

The Data Logs browser generator, and this hoster's own domain tool (the analog of sitehoster's Automation Logs and gamehoster's Game Logs). Where the log browsers read the access logs, this one reads the definitions and the mirrored data files and renders a browsable, static, per-host view of everything on the data server: every collection with its columns and the endpoints that reach it, every user, and the records themselves, loaded read-only from the columnar mirror through its own lib/store.js. Each record's at timestamp buckets it in UTC, so, like the Web and Server Logs tools, it charts the records by month, then day, then hour, each a page you drill into from the one above, and lists the newest records on every period page. Pages are written with write-if-changed and orphaned pages pruned. It composes the shared shell, UI kit and formatters, so it feels like one tool with the log browsers, and it touches no API.

Export	What it is
`run(config)`	Read the definitions and the mirrored data under `contentRoot` and render the per-host Data Logs browser. Records the summary the Data Logs card shows.
`lib/render.js: build(options)`	The generator the tool drives. Takes `{ contentRoot, dataDir, outDir, ip }`; returns the summary.

`datahoster-sync/lib/scan.js`

Reads the content root into the flat set of definition files to sync. Only dotted host directories are scanned (loose files, dotfiles and _dirs are ignored); each host holds its definitions in three folders: datahoster-collections/ (each collection a folder of columns), datahoster-users/ and datahoster-endpoints/. Every file is parsed and its name-in-file is checked against its folder or file, so a bad definition is caught locally before anything is sent. It also gathers each collection's import files, skipping the datahoster-collection-import/ folder in the definition walk (its records are data, not definitions).

Export	What it is
`scan(root)`	Walk the content root into `{ defs, collections, schemas, imports, hosts }`: the definition-path-to-bytes map, the collections found, each collection's parsed schema, the import files per collection, and the host set. Validates each definition and its name-in-file.

`datahoster-sync/lib/` — the definition and storage engine

The sync command's own copies of the engine it drives (the server carries its own, so the two bundles are each self-contained). The Data Logs tool carries its own copies of paths.js, schema.js and store.js too, since it reads the same definitions and the same mirror. These files are documented in full on the Storage and Sync pages.

File	What it is
`paths.js`	The path-safety and naming rules: a definition path must resolve inside its root, a host is a dotted name, a collection/user/endpoint name is dotless. Also parses a definition path into its host, kind and name.
`schema.js`	The one validator for collections, columns, users and endpoints, and the coercion of a submitted value to its column type, so the sync rejects a malformed definition before pushing it.
`import.js`	Read a collection's import file into records, validate and coerce each against the columns, and derive each record's stable id.
`store.js`	The append-only columnar storage engine, used read-only after the mirror to confirm every collection's data actually decodes.
`version.js`	The `API_VERSION` integer that forms the `/api/vN` path prefix; today `2`. A mismatch fails fast.

`localhoster-common/net.js`

The shared API-client plumbing every hoster's downloaders sit on: read the config, skip cert verification for a bare-IP (self-signed) endpoint, and discover the server's API version so request paths follow it. Meant to be copied across the hosters unchanged. (The sync command reads its own datahoster-config-* config through loadConfig; this file also backs the log/stat pulls the tools run.)

Export	What it is
`loadConfig(path)`	Read and validate the config JSON into a frozen `{ apiBase, apiToken, insecure, contentRoot, toolsRoot }`.
`isIpHttps(apiBase)`	Whether the endpoint is HTTPS to a bare IP literal (or `[ipv6]`), hence self-signed.
`apiGet(url, opts)`	GET one API URL, skipping cert verification per request for a self-signed IP endpoint (never process-wide).
`resolvePrefix(...)`	Discover the server's API version and build the `/api/vN` path prefix the requests use.

`localhoster-common/run.js`

The shared tools-site frame. Each tool renders its own pages during its own run(config) and records its result; this assembles the surrounding site: the overview (one card per tool), the tab strip, and a "not run yet" status page for any tool that has never produced pages. It reads each tool's own state file, so running one tool never clobbers another tool's card. There is no shared mutable state: each tool owns <toolsRoot>/<key>/.state.json.

Export	What it is
`toolsSite(config, brand)`	A builder: register each tool's tab with `.add(tool)`, one line per tool, then `.render()` writes the overview, tabs and status around whatever ran.
`recordRun(outDir, summary)`	A tool records its last-run time and summary into its own output dir, read back to draw its overview card.

`localhoster-tools/web-logs/lib/render.js`

The Web Logs browser generator. It reads the Caddy JSON access logs mirrored down by lib/pull.js and renders a set of static, self-contained HTML pages so the traffic can be browsed offline. Composes the shared shell, UI kit and formatters; also runnable standalone.

Export	What it is
`build(options)`	Render the Web Logs browser. The tool passes `{ logsDir, outDir }`; run standalone it reads the log dir beside the script.

`localhoster-tools/server-logs/lib/render.js`

The Server Logs browser generator, the reader half of the shared stats facility. It reads the per-type dated day-files mirrored down by lib/pull.js and writes a drill-down site (all time to month to day to hour to minute), each page charting every metric with a chart/table toggle.

Export	What it is
`build(options)`	Render the Server Logs browser. The tool passes `{ dataDir, outDir, serverIp }`; run standalone it reads the data dir beside the script.

`localhoster-common/shell.js`

The shared shell that owns the page frame for the tools site: the fixed top strip (brand, Overview, a tab per tool, the IP), the per-tool status banner, the stylesheet, and small formatters. The same strip goes on every page; the tab set and brand are configured per project by run.js at startup, so this file carries no project identity and copies across the hosters unchanged.

Export	What it is
`layout(opts)`	Render a full framed page (top strip + body) for the Overview and any not-run tool's status page.
`toolStrip(...)`	The fixed top strip a tool browser injects at the top of its own pages, so the tabs are identical everywhere.
`statusBanner(tool)`	The per-tool banner shown when a tool has not been run.
`overviewCard(tool, state)`	One tool's card on the Overview, reflecting its last-run state.
`setTools(tools)`, `setBrand(brand)`	Configure the tab set and brand for this project (called by `run.js`); `TOOLS`, `brandName()` expose them.
`esc`, `fmtNum`, `fmtBytes`, `stripFooter`	The formatters and footer-stripper re-exported for convenience, plus `STYLESHEET` and `TOOLSTRIP_CSS`.

`localhoster-common/ui.js`

The shared UI kit for the tools browsers, the reusable builders a domain tool composes instead of hand-rolling markup. The page frame lives in shell.js and the formatters in format.js; specialised chart bodies stay per-browser, bound to each tool's data.

Export	What it is
`panel(...)`, `table(...)`	A titled panel, and a table with a built-in Show-all toggle.
`statCards(items)`	A row of stat blocks (name + value, or a custom cell).
`navButtons(prev, next, up)`	Prev / Next (with an optional Up) pagination, used on every dated page.
`viewToggle(...)`	A chart to table switch for a metric.
`SCRIPT`, `CSS`	The shared client JS (table toggle + chart switch) and the common visual furniture.

`localhoster-common/format.js`

The canonical formatters shared by the tools browsers, pure functions with no state, meant to be copied across the hoster projects unchanged.

Export	What it is
`esc(s)`	HTML-escape a value for safe interpolation.
`fmtNum(n)`	A grouped decimal number (thousands separators).
`fmtBytes(b)`	A human byte size (B, KB, MB, and so on).
`fmtDuration(ms)`	A compact duration (ms, s, or m s).
`monthLabel`, `monthLabelShort`, `dayLabel`, `MONTH_NAMES`	Month/day label helpers and the month-name table used by the dated pages.

`localhoster-common/io.js`

Small shared render/IO helpers for the tools browsers. The footer carries the page's generation time, which changes every run; stripFooter removes the whole footer before the write-if-changed comparison so a new timestamp alone never forces a rewrite.

Export	What it is
`stripFooter(html)`	Remove the footer block before comparing a freshly rendered page to the one on disk.
`rmrf(p)`	Recursively remove a path (for a clean full rebuild).
`GEN_DURATION_TOKEN`	The placeholder a browser writes for the generation time and patches with the real value last.

Local

Files

datahoster-local.js — entry point

datahoster-sync/index.js

datahoster-tools/data-logs/index.js

datahoster-sync/lib/scan.js

datahoster-sync/lib/ — the definition and storage engine

localhoster-common/net.js

localhoster-common/run.js

localhoster-tools/web-logs/lib/render.js

localhoster-tools/server-logs/lib/render.js

localhoster-common/shell.js

localhoster-common/ui.js

localhoster-common/format.js

localhoster-common/io.js