Server

Two processes run on the box: Caddy, the only public listener, and a small dependency-free Node API bound to 127.0.0.1:8788 that Caddy reverse-proxies to. The API loads each data host's synced definitions, answers the public read/write surface and the token-authed management and tools API, and samples the machine's health on a timer. There is no configuration to speak of — every path is fixed in the code or a systemd environment line, and the only per-install state is one secret. Everything on disc is namespaced under localhoster so several hoster services can run side by side on one box: this service's own files sit under /opt/localhoster/datahoster/, beside any sibling service, while the parts every service shares — the access logs, the machine-health samples, and the bare-IP certificate — live once under /srv/localhoster/ and /etc/localhoster/.

/opt/localhoster/                    every localhoster service on this box, side by side
  datahoster/                      this service (a sibling like sitehoster/ sits beside it)
    datahoster-server-<version>/   all the datahoster server code (one dir per installed version)
      datahoster-api.js            the entry systemd runs: the localhost HTTP API
      VERSION                      this package's version (matches the dir name)
      localhoster-server/          shared server toolkit, identical across every hoster
        logs-stats.js             the read-only /logs + /stats routes
        stats-recorder.js         the loop that writes the stat day-files
        stats.js                  one machine-health snapshot
      datahoster-engine/           datahoster's own storage + schema runtime
        store.js                  the columnar storage engine
        paths.js                  path-safety and naming
        schema.js                 definition + submission validation
        import.js                 import-record reading and coercion
        version.js                the /api/vN protocol version
      deploy/
        install.sh                the installer this box ran
        uninstall.sh
        start.sh
        stop.sh
        Caddyfile                 the Caddy config source
        datahoster-api.service    the systemd unit source
    datahoster-server-current       → datahoster-server-<version>/ (the unit follows this; an upgrade flips it)
    datahoster-data/               this service's data — one folder per data host
      api.example.com/
        datahoster-collections/
        datahoster-users/
        datahoster-endpoints/
        datahoster-data/
    datahoster-token               this service's bearer token (mode 600)
/srv/localhoster/                    box-wide runtime data, shared by every service
  web-logs/                       Caddy access logs (one Caddy fronts every domain)
    access.log                   the active log, one JSON line per request
    access-<timestamp>.log       rotated, immutable (keyed by mtime, not name)
  server-logs/                    machine-health samples (the box, not any one service)
    cpu-<YYYY>-<MM>-<DD>.log
    memory-<YYYY>-<MM>-<DD>.log
    disk-<YYYY>-<MM>-<DD>.log
    diskio-<YYYY>-<MM>-<DD>.log
    netio-<YYYY>-<MM>-<DD>.log
/etc/localhoster/                    box-wide localhoster config, shared
  localhoster-api-certificate.crt  the self-signed cert for the bare-IP endpoint
  localhoster-api-certificate.key  its private key (mode 640 root:caddy)
/etc/caddy/
  Caddyfile                      the Caddy config (serves every service's domains)
/etc/systemd/system/
  datahoster-api.service         one unit per service, side by side
  caddy.service                  the shared public listener (from the Caddy package)

Everything on disc: this service under /opt/localhoster/datahoster/, and the box-wide logs, stats, cert and Caddy shared under localhoster.

Every path is fixed in the code or the service unit; the installer creates all of them and nothing lives outside them.

Name	Type	Description
`/opt/localhoster/datahoster/datahoster-data/`	directory	The data mirror — one folder per data host, each holding its synced definitions and its records stored as columns — what the sync writes and the API loads and serves.
`/opt/localhoster/datahoster/datahoster-server-<version>/`	directory	The self-contained server package: the entry, the shared `localhoster-server/` toolkit, and datahoster's own `datahoster-engine/`, plus the deploy scripts.
`/opt/localhoster/datahoster/datahoster-token`	file	The one secret: the bearer token the whole API is gated on. Absent turns the API off.
`/srv/localhoster/web-logs/`	directory	Caddy's combined JSON access log, active plus size-rotated files.
`/srv/localhoster/server-logs/`	directory	The server-health samples, one dated file per statistic type.
`/etc/localhoster/`	directory	The self-signed certificate for the bare-IP endpoint.
`/etc/caddy/Caddyfile`	file	The Caddy config the installer lays down, plus a backup of any earlier one.
`/etc/systemd/system/`	directory	The service units: the Datahoster API, and Caddy's from its package.

/opt/localhoster/datahoster/datahoster-data/

The data mirror — the tree the sync pushes up and the API loads and serves. Its children are the data hosts: one directory named exactly for the hostname it serves, which the API routes a request to by its Host header. Adding a host's folder here and pointing its A record at the box is all it takes to serve it. A certificate is issued on the first request, gated by the API (see the Caddyfile). It is owned by the datahoster service user, the only tree the API writes.

Within a host, the local client's definitions are synced in wholesale — the datahoster-collections/, datahoster-users/ and datahoster-endpoints/ folders — and the records themselves live under datahoster-data/, one directory per collection, each a set of fixed-width column files aligned by row number. The tree below shows the shape; how a collection lays its columns out, the strings files for text, and the chunking, is the whole subject of the Storage page, and how the same columns are held in memory is the Memory page.

/opt/localhoster/datahoster/datahoster-data/
  api.example.com/                            one directory per data host
    datahoster-collections/                 the schema, synced from the local client
    datahoster-users/                       users, synced from the local client
    datahoster-endpoints/                   endpoints, synced from the local client
    datahoster-data/                        the stored records
      messages/                            one directory per collection
        datahoster-data-records/           id, time and author per row
        datahoster-data-deleted/           ids of deleted records
        datahoster-data-columns/           one directory per column
          name/
            datahoster-data-column-text-00000000.dat
            datahoster-data-column-text-00000000-00000000.jsonl
          email/
          message/

One host: its synced definitions, and a messages collection's records stored as columns under datahoster-data.

A read folds a collection's record history to its current rows; a write appends. Removing a host is a matter of removing its folder here on the next sync.

/opt/localhoster/datahoster/datahoster-server-<version>/

Each version installs into its own directory; the service unit points at datahoster-server-current/datahoster-api.js, a symlink to the installed datahoster-server-<version>/, so an upgrade is atomic (the installer flips the symlink) and the previous version stays on disc for rollback. The tree below is the whole self-contained package: the entry, the shared localhoster-server/ toolkit (the read-only /logs + /stats API and the health recorder), and datahoster's own datahoster-engine/ (the columnar store, path-safety, schema validation, import reading, and the protocol version), plus the deploy scripts. Every module is plain Node with no dependencies — no npm install, no node_modules, and nothing is copied alongside the package. The local sync keeps its own copy of each engine module (store, paths, schema, import, version) so the two never disagree; the server only ever requires what is here.

datahoster-server-<version>/
  datahoster-api.js            the entry systemd runs (the localhost HTTP API)
  VERSION                      this package's version (names the install dir)
  localhoster-server/          shared server toolkit, identical across every hoster
    logs-stats.js             the read-only /logs + /stats routes
    stats-recorder.js         the loop that writes the stat day-files
    stats.js                  one machine-health snapshot
  datahoster-engine/           datahoster's own storage + schema runtime
    store.js                  the columnar storage engine (the sync carries a duplicate)
    paths.js                  path-safety and naming
    schema.js                 definition + submission validation
    import.js                 import-record reading and coercion
    version.js                the /api/vN protocol version
  deploy/
    install.sh                the installer this box ran
    uninstall.sh
    start.sh
    stop.sh
    Caddyfile                 the Caddy config source
    datahoster-api.service    the systemd unit source

The self-contained server package, unrolled file by file.

Name	Type	Description
`datahoster-api.js`	file	The API server and the process systemd starts. A localhost-only HTTP listener carrying the public read/write surface routed by `Host` header, the management surface the local client uses, the read-only logs/stats routes, the version probe, and Caddy's on-demand-TLS gate. Reads the token once at startup and starts the stats recorder on `listen()`.
`localhoster-server/logs-stats.js`	file	The read-only `/logs` and `/stats` routes (plus the `/stats/now` debug snapshot), identical across every hoster and mounted under the same token.
`localhoster-server/stats-recorder.js`	file	Started by the API, it snapshots on the interval, appends a JSON line per type to the dated day-file, and prunes files past the retention window.
`localhoster-server/stats.js`	file	Takes one snapshot of the machine's health — CPU, memory, disk, and cumulative disk- and network-IO counters. Common to every hoster.
`datahoster-engine/store.js`	file	The columnar storage engine behind every read and write: it appends each insert, update and delete to the collection's columnar files and folds a collection's history to its current rows, so nothing is ever rewritten. The local sync keeps its own copy to load the mirrored files read-only for Data Logs.
`datahoster-engine/paths.js`	file	The path-safety and naming helpers, so the server and the local sync agree on which paths are legal and how a host, collection, user or endpoint is named.
`datahoster-engine/schema.js`	file	The one validator for collections, columns, users and endpoints; the server enforces it on every request and coerces each submitted value to its column type.
`datahoster-engine/import.js`	file	Reads a collection's import files into records, derives each record's id, and coerces each value's import form.
`datahoster-engine/version.js`	file	The single API/protocol version that forms the `/api/vN` path prefix; a mismatch answers `426` to an authenticated local client, or the uniform `404` otherwise.
`deploy/`	dir	The install and service scripts: `install.sh`, `uninstall.sh`, `start.sh`, `stop.sh`, the `Caddyfile`, and the `datahoster-api.service` unit.

What each module exports, so the read and write paths and the shared toolkit all line up with the code:

datahoster-api.js

The entry point the service runs; it is a process, not a library, so it exports nothing. It routes the public read/write surface by Host header to the management and endpoint handlers, mounts the shared logs/stats routes, answers Caddy's on-demand-TLS gate, and starts the stats recorder on listen(). A select answers GET and HEAD with an ETag over the rows it returns, replying 304 to a matching If-None-Match; the management surface adds the two import routes (the change pre-check and the per-record upsert).

datahoster-engine/store.js

The columnar storage engine — the on-disc chunk files of Storage and the in-memory model of Memory. The local sync carries its own copy of it, loading the mirrored files read-only to render Data Logs.

Export	What it is
`Store`	Every host on the box; `store.database(host)` returns one host's Database.
`Database`	One host's collections by name; `db.collection(name, def, config)` opens or creates one.
`Collection`	One table: `insert`, `update`, `delete` and `upsert` (append-only, versioned; upsert appends only when data changed), `read` (walk the current rows), `seqOf` (a row's version, for the select ETag), `addColumn` (backfill a new column), the import token index, and `load` (rebuild from disc).
`Slots`	A rolling series of fixed-width `.dat` chunk files, mirrored to in-memory chunks plus a growing tail.
`Strings`	A text column's `.jsonl` strings files, addressed by the row's chunk, a byte offset and a length.
`encode` · `decode`	A value to and from its fixed-width bytes for a stored type.
`storedOf` · `defaultFor`	The stored type a field type maps to; a type's default value.
`STORED_OF` · `WIDTH` · `DEFAULTS`	The field-type→stored-type map, the byte width per stored type, and the chunk-size defaults.

datahoster-engine/paths.js

Path-safety and naming, so the server and the local sync agree which paths are legal and how each definition file is located.

Export	What it is
`isHost` · `isName` · `isAllowedName`	A hostname (has a dot); a collection / column / endpoint / user name (dotless, not `api`); a single safe segment.
`parseDefPath`	Recognises a definition-file path and returns its descriptor (`collection` · `column` · `user` · `endpoint`), or null.
`defPath`	The inverse: builds the relative path for a descriptor.
`normalizeRelPath` · `resolveInRoot`	Normalise a relative path (throws on escape); resolve it inside a root, asserting containment.
`DEF_DIRS` · `DATA_DIR` · `COLLECTION_FILE` · `COLUMNS_DIR` · `OPS`	The folder names (`datahoster-collections/`, `-users/`, `-endpoints/`, `-data/`), the collection file, the columns folder, and the four operation names.

datahoster-engine/schema.js

The one validator for every definition file and the coercion of each submitted value.

Export	What it is
`parseCollection`	Validates `datahoster-collection.json` → name plus the three chunk sizes.
`parseColumn`	Validates one column file → name, type, default and max, and whether a default was set (a column with none is required in an import record).
`parseUser` · `checkUsername`	Validates a user file → its token; checks a username is 1 to 32 URL-safe characters and does not start with the reserved `datahoster` prefix.
`parseEndpoint`	Validates one endpoint operation file (`select` · `insert` · `update` · `delete`) → its collection, columns/fields, filter/sort/limit, forced values and `{public, users}` permission.
`parseDefBytes`	Dispatches on a `parseDefPath` descriptor to the right parser above.
`coerce` · `defaultFor`	Coerce a raw value to a field type (or an error); a type's default.
`permission` · `isName`	Read an endpoint's `{public, users}`; the name rule (shared with paths).
`FIELD_TYPES` · `CHAR_BYTES` · `FILTER_OPS` · `SORT_DIRS` · `OPS` · `MAX_USERNAME` · `CHUNK_DEFAULTS`	The type set (text · char8-1024 · integer · number · boolean · int128), the char byte widths, the eight filter operators, the sort directions, the operations, the username limit, and the chunk defaults.

datahoster-engine/import.js

The import record reader; the local sync carries the same code, so it validates and the server writes with one implementation. It reads a file into records (object or array), derives each record's id, and coerces each value's import form to the store's native value.

Export	What it is
`recordsOf`	One record from a top-level object, or many from a top-level array.
`idOf` · `idFromLocal` · `idFromPosition`	A record's 128-bit id: from its local `id`, or from the file name and its position.
`coerceImportValue` · `validateAndCoerce`	Coerce one value to its column's type (raw, `hex:`, `base64:`, `binary:`, or a numeric string); validate a whole record against the columns, naming a bad field.
`IMPORT_USER`	The built-in `datahoster-import` every imported record is attributed to.

datahoster-engine/version.js

Exports API_VERSION — the single integer that forms the /api/vN path prefix; a request on another version's path answers 426 (or the wall, unauthenticated).

localhoster-server/stats.js · localhoster-server/stats-recorder.js · localhoster-server/logs-stats.js

The shared machine-health tooling, identical across every hoster.

Export	What it is
`stats.js` → `snapshot`	Takes one machine-health snapshot (CPU, memory, disk, cumulative disk/network IO).
`stats-recorder.js` → `startRecorder`, `deriveLines`, `dateStamp`	The loop that snapshots on the interval and writes the dated day-files; the per-type line deriver; the day-file date stamp.
`logs-stats.js` → `create`	Builds the read-only `/logs` and `/stats` route handler mounted in the API router.

/opt/localhoster/datahoster/datahoster-token

The whole per-install configuration is one secret. The installer generates it once (printed once) and the API reads it at startup; the local sync and tools client is handed the same value out of band. Every API call is gated on it, and with no token file the API refuses to run. The token is never put in an environment variable (which leaks via /proc) — it is read from this file. To rotate it, write a new value and restart the API. It is mode 600, owned by the datahoster user.

/srv/localhoster/web-logs/

Caddy writes one combined JSON access log covering every hosted data host and the proxied API, each line tagged with its host. It rotates by size only (there is no daily rotation), so there is one active file and a trail of size-rotated ones. The API exposes this directory read-only (GET /api/v3/logs) and the local client mirrors it down for the Web Logs tool; the directory is mode 0770 group caddy, which writes it, and the datahoster user is added to the caddy group so the API can read it. The API's own operations (each insert, update, delete and definition push) go to the systemd journal, not to a file here.

/srv/localhoster/web-logs/
  access.log                    the active log, one JSON line per request
  access-<timestamp>.log       rotated and immutable; the number is its rotation time in ms
  access-<timestamp>.log

Caddy's active and rotated access logs.

Rotation is set in the Caddyfile (roll_size 64MiB, roll_keep 50, roll_keep_for 2160h). Rotated files are keyed to the local client by mtime, never by name, so filenames never cross the wire.

/srv/localhoster/server-logs/

The API process samples the machine's own health every 15 seconds and appends one JSON line per statistic type to a dated, per-type day-file. A day-file is frozen the moment the UTC date rolls over, with no rename and no lock, and files older than 60 days are pruned. The API serves these read-only (GET /api/v3/stats); the local client mirrors each past day once and keeps it, which is the data behind the Server Logs tool. The directory is mode 750, owned by the datahoster user, and is the only path besides datahoster-data/ the service may write.

/srv/localhoster/server-logs/
  cpu-<YYYY>-<MM>-<DD>.log              per-core cumulative CPU times
  memory-<YYYY>-<MM>-<DD>.log           total / used / available / free bytes
  disk-<YYYY>-<MM>-<DD>.log             usage of the filesystem holding /
  diskio-<YYYY>-<MM>-<DD>.log           cumulative bytes read / written
  netio-<YYYY>-<MM>-<DD>.log            cumulative bytes in / out
  cpu-<YYYY>-<MM>-<DD>.log              yesterday's files, immutable until the retention cutoff

One dated file per statistic, per day.

Counter fields (CPU times, disk and network bytes) are stored cumulative, so the reader diffs two samples and divides by the true elapsed time: a gap after downtime averages out instead of spiking, and a reboot's counter reset is dropped. Metrics a host cannot report (for example disk and network IO off Linux) are simply absent.

/etc/localhoster/

A self-signed certificate for the bare-IP endpoint, generated once by the installer (CN=datahoster, ten-year life). Caddy obtains real certificates on demand for the data-host domains; this one only completes the handshake for a no-SNI / bare-IP connection, selected via the Caddyfile's default_sni. It is what lets the local client reach the API by the box's IP — it skips verification for an IP endpoint, and the token is what actually authenticates it.

/etc/localhoster/
  localhoster-api-certificate.crt  self-signed cert for the bare-IP endpoint
  localhoster-api-certificate.key  its private key (mode 640 root:caddy)

The bare-IP certificate.

/etc/caddy/Caddyfile

The Caddyfile is fully static, with no environment placeholders. It terminates TLS, obtains real certs on demand (gated by a localhost call to /internal/allow on the API, so issuance can't be forced for hosts you don't serve), redirects HTTP to HTTPS, serves a self-signed cert for bare-IP connections, and reverse-proxies every other request to the localhost API at 127.0.0.1:8788, which routes it by Host header. The installer copies it from the source deploy/Caddyfile, backing up any file already there once.

/etc/caddy/
  Caddyfile                      our config, installed from deploy/Caddyfile
  Caddyfile.pre-datahoster.<ts>   a one-time backup of any earlier Caddyfile

The Caddy config, and a backup of any earlier one.

/etc/systemd/system/

Two units run at boot. Datahoster ships its own; Caddy's comes from its package (the installer requires Caddy to be installed from a package so this unit exists) and the installer just enables and restarts it. The API unit runs as the unprivileged datahoster user with NoNewPrivileges, ProtectSystem=strict and ProtectHome, and may write only /opt/localhoster/datahoster/datahoster-data and /srv/localhoster/server-logs. It has no EnvironmentFile — the process reads its one secret from /opt/localhoster/datahoster/datahoster-token and takes everything else from fixed Environment= lines (port, bind address, the data root, and the log/stats dirs). It is added to the caddy supplementary group so it can read the access logs.

/etc/systemd/system/
  datahoster-api.service         the API — runs /opt/localhoster/datahoster/datahoster-server-<version>/datahoster-api.js
  caddy.service                  the public listener, from the Caddy package

The two service units.