feat(enterprise): operator-declared cost model, working budgets, virtual keys, guardrails, teams, OIDC #1

Merged

epistemophiliac merged 4 commits from develop into main 2026-08-19 03:05:01 +00:00

Conversation 0 Commits 4 Files changed 81 +9754 -327

epistemophiliac commented 2026-08-19 03:04:48 +00:00

Owner

Copy link

Summary

Enterprise feature suite for LLMAPI: operator-declared cost model, working budget
enforcement, virtual keys, guardrails, teams/RBAC, and OIDC login.

Squashes three concerns into one PR:

1. feat(enterprise) - the feature suite + 6 bug fixes
2. fix(security) - close the admin-API auth boundary
3. chore(deploy) - declare OIDC env in compose

The cost model changed shape

Everything is free unless the operator explicitly marks it paid, via a
provider -> model picker (Budgets -> Free / paid models, deep-link
/budgets#costs). The previous hardcoded free-platform allowlist classified
143 of 257 real models as paid - ~99% false positives - which silently
blocked free models on a $0 budget. Default state is now 257 free / 0 paid,
verified against the live catalog.

Bugs fixed (7)

Bug	Impact
Spend was never recorded	recordSpend()'s only caller was never mounted and nothing set res.locals.usage. current_spend_usd never left 0.000000, so every budget cap was decorative.
max_budget_usd: 0 -> null	|| null made a $0 key unlimited - the exact inverse of the spec.
rpm_limit: 0 -> null	Same falsy bug: "no requests" became "no limit".
Rotation reset spend	A key at its cap could be rotated to restore full spending.
Cloudflare transcription	Sent a hardcoded Bearer *** instead of the parsed token - could never authenticate.
Express 5 bare * route	Threw at route registration, crashing the server on boot.
Inference keys reached /api/*	A client-profile key could read /api/keys and change paid-model classification.

Token estimation unified

Five output-token fallbacks still used chars/4, which undercounts ~1.6x - and
output is the expensive side of a paid model. All now use the context engine's
estimator (chars/3.5 x 1.4 safety margin).

Security boundary

requireAuth accepted any valid inference credential for the whole /api/*
admin surface. Those keys go to apps and third parties, so this turned "can call
the model" into "owns the gateway". Restored to 401; the debug convenience is now
opt-in via ADMIN_API_ACCEPTS_INFERENCE_KEYS=true (off by default).

The tests asserting that boundary had been failing since 36154a4 and were left
unreconciled - so the regression sat in the tree looking like expected noise.

Test suite: 24 failures -> 0

2614 passed / 0 failed / 5 skipped (232 files).

22 of the 24 were stale assertions left behind by deliberate changes (softened
cooldown ladder 30s/2m/5m/10m/1h/24h, retuned balanced weights
0.40/0.15/0.45, the chars/3.5 estimator). Two were real bugs, fixed in code
rather than by relaxing the test.

Three brittle tests were rewritten to assert the property that matters:

• migration roundtrip reads the registry instead of a hardcoded list, and
  ignores column order (SQLite cannot DROP COLUMN) while still comparing
  the column set
• compression guard asserts linearity (3x payload < 6x cost) rather than
  wall-clock milliseconds

Deploy safety

Migrations verified against a copy of the production DB (223 models):

models before=223  after=223  paid_models seeded=0

Zero data loss, nothing wrongly marked paid. Every DROP TABLE is inside
down() only; up() is purely additive (ALTER TABLE ADD COLUMN).

Not included, deliberately

orderCandidates() exists and is tested but is not wired into the dispatch
loop - the routing-strategies page is a read-only view over the existing
bandit scorer. Left as-is on request: the retuned scorer stays authoritative and
routing behaviour is unchanged by this PR.

Tag-based routing, Redis/S3 cache backends, and alert webhooks are not
implemented (not needed at current scale).

Verification

• tsc --noEmit clean, server + client
• both builds succeed
• migrations tested against a real prod DB snapshot
• no secrets in any commit; deploy/ never committed
• VERSION and package.json untouched

## Summary Enterprise feature suite for LLMAPI: operator-declared cost model, working budget enforcement, virtual keys, guardrails, teams/RBAC, and OIDC login. Squashes three concerns into one PR: 1. `feat(enterprise)` - the feature suite + 6 bug fixes 2. `fix(security)` - close the admin-API auth boundary 3. `chore(deploy)` - declare OIDC env in compose ## The cost model changed shape Everything is **free unless the operator explicitly marks it paid**, via a provider -> model picker (`Budgets` -> `Free / paid models`, deep-link `/budgets#costs`). The previous hardcoded free-platform allowlist classified **143 of 257 real models as paid** - ~99% false positives - which silently blocked free models on a $0 budget. Default state is now **257 free / 0 paid**, verified against the live catalog. ## Bugs fixed (7) | Bug | Impact | |---|---| | **Spend was never recorded** | `recordSpend()`'s only caller was never mounted and nothing set `res.locals.usage`. `current_spend_usd` never left `0.000000`, so **every budget cap was decorative**. | | `max_budget_usd: 0` -> `null` | `\|\| null` made a $0 key **unlimited** - the exact inverse of the spec. | | `rpm_limit: 0` -> `null` | Same falsy bug: "no requests" became "no limit". | | Rotation reset spend | A key at its cap could be rotated to restore full spending. | | Cloudflare transcription | Sent a hardcoded `Bearer ***` instead of the parsed token - could never authenticate. | | Express 5 bare `*` route | Threw at route registration, crashing the server on boot. | | Inference keys reached `/api/*` | A client-profile key could read `/api/keys` and change paid-model classification. | ## Token estimation unified Five output-token fallbacks still used `chars/4`, which undercounts ~1.6x - and output is the expensive side of a paid model. All now use the context engine's estimator (`chars/3.5` x 1.4 safety margin). ## Security boundary `requireAuth` accepted any valid inference credential for the whole `/api/*` admin surface. Those keys go to apps and third parties, so this turned "can call the model" into "owns the gateway". Restored to 401; the debug convenience is now opt-in via `ADMIN_API_ACCEPTS_INFERENCE_KEYS=true` (off by default). The tests asserting that boundary had been failing since `36154a4` and were left unreconciled - so the regression sat in the tree looking like expected noise. ## Test suite: 24 failures -> 0 **2614 passed / 0 failed / 5 skipped (232 files).** 22 of the 24 were stale assertions left behind by deliberate changes (softened cooldown ladder `30s/2m/5m/10m/1h/24h`, retuned balanced weights `0.40/0.15/0.45`, the `chars/3.5` estimator). Two were real bugs, fixed in code rather than by relaxing the test. Three brittle tests were rewritten to assert the property that matters: - migration roundtrip reads the registry instead of a hardcoded list, and ignores column **order** (SQLite cannot `DROP COLUMN`) while still comparing the column **set** - compression guard asserts **linearity** (3x payload < 6x cost) rather than wall-clock milliseconds ## Deploy safety Migrations verified against a **copy of the production DB** (223 models): ``` models before=223 after=223 paid_models seeded=0 ``` Zero data loss, nothing wrongly marked paid. Every `DROP TABLE` is inside `down()` only; `up()` is purely additive (`ALTER TABLE ADD COLUMN`). ## Not included, deliberately `orderCandidates()` exists and is tested but is **not wired into the dispatch loop** - the routing-strategies page is a read-only view over the existing bandit scorer. Left as-is on request: the retuned scorer stays authoritative and routing behaviour is unchanged by this PR. Tag-based routing, Redis/S3 cache backends, and alert webhooks are not implemented (not needed at current scale). ## Verification - `tsc --noEmit` clean, server + client - both builds succeed - migrations tested against a real prod DB snapshot - no secrets in any commit; `deploy/` never committed - VERSION and `package.json` untouched

epistemophiliac added 4 commits 2026-08-19 03:04:49 +00:00

ci(develop): initialize CI-compliant branch structure and workflow ... 20df1d330a

- Add develop integration branch (trunk-based model)
- Extend CI triggers to develop and release/* branches
- Add Dependabot config for npm, GitHub Actions, and Docker
- Add PR template with branch-selection guidance
- Document branch structure in CONTRIBUTING.md

Branch model:
  main     — production-stable
  develop  — integration branch (all feature/fix PRs target this)
  feature/*, fix/*, chore/* — short-lived branches cut from develop
  release/* — release candidates cut from develop, merged to main

feat(enterprise): operator-declared cost model, spend accrual, virtual keys, guardrails, routing, OIDC, teams ... b914edce49

Cost model is now OPERATOR-DECLARED instead of guessed. Every model is FREE
unless explicitly registered as paid, via a provider -> model picker in the UI
(Budgets -> "Free / paid models", deep-link /budgets#costs). The previous
hardcoded free-platform allowlist classified 143 of 257 real models as paid --
~99% false positives, wrongly blocking free models on a $0 budget. Default
state is now 257 free / 0 paid, verified live against the real catalog.

paid_models registry: (platform, model_id) with model_id='*' for a whole
platform, plus optional operator-set cost_per_1k_input/output so pricing is
declared rather than assumed. Precedence: exact model > :free/-free//free
marker > platform wildcard > free. Every reclassification is attributed in
model_cost_overrides_log.

Budget enforcement now actually enforces. Bugs fixed:

- Spend was NEVER recorded: recordSpend() existed but its only caller
  (postResponseMiddleware) was never mounted and nothing populated
  res.locals.usage, so current_spend_usd never left 0.000000 and every budget
  cap was decorative. Added services/spend-accrual.ts, wired into all four
  proxy success paths. Free models accrue exactly zero, so free-tier traffic
  never consumes a budget.
- max_budget_usd: 0 became null via `|| null`, making a $0 key UNLIMITED --
  the exact inverse of the spec. Now `?? null`.
- rpm_limit: 0 hit the same falsy bug: "no requests" became "no limit".
- Key rotation reset accumulated spend to $0, so a key at its cap could be
  rotated to restore full spending. Added carryOverSpend().
- Cloudflare transcription sent a hardcoded `Bearer ***` instead of the parsed
  token, so it could never authenticate upstream (services/media.ts).
- Express 5 rejects bare '*' routes and threw at registration, crashing the
  server on boot. Use a named splat.

Token estimation unified on the context engine's estimator (chars/3.5 with a
1.4 safety margin). Five output-token fallbacks still used chars/4, which
undercounts ~1.6x -- and output is the expensive side of a paid model.

Also adds virtual keys (CRUD, rotation, show-once reveal), a guardrails
strategy creator with a regex sandbox plus an events monitor, pluggable routing
strategies, "Login with Aexora" OIDC (auth-code + PKCE, renders only when
configured), teams/RBAC, and enterprise debug endpoints.

Test suite is fully green: 2614 passed / 0 failed / 5 skipped (232 files).
The 24 long-standing failures were stale assertions left behind by deliberate
changes -- the softened cooldown ladder (30s/2m/5m/10m/1h/24h), retuned
balanced weights (0.40/0.15/0.45), and the chars/3.5 estimator -- plus brittle
tests now rewritten to assert the property that matters: the migration
roundtrip reads the registry instead of a hardcoded list and ignores column
ORDER (SQLite cannot DROP COLUMN) while still comparing the column SET, and
the compression guard asserts linearity rather than wall-clock milliseconds.

No VERSION bump. Not deployed.

fix(security): stop inference keys from authenticating the admin API ... 9d1b4f021f

requireAuth accepted ANY valid inference credential -- the unified API key or a
client-profile key -- as a fallback for the entire /api/* admin surface. Those
keys are handed out to apps, bots, and third parties, so this turned "can call
the model" into "owns the gateway": a profile key could read /api/keys and
/api/settings/api-key, and now that budgets and paid-model classification live
behind the same surface, it could also change what costs money.

The boundary was documented and covered by tests in client-profiles.test.ts
("a profile key does NOT authenticate the admin/dashboard API"), but the
fallback landed in 36154a4 as a curl/Postman convenience and those two tests
were left failing rather than reconciled -- so the regression sat in the tree
looking like expected noise.

Restores 401 for inference credentials on /api/*. Dashboard sessions and OIDC
sessions are unaffected. The debug convenience survives as an explicit opt-in,
ADMIN_API_ACCEPTS_INFERENCE_KEYS=true, off by default and documented in
.env.example.

Fixed in code rather than by relaxing the tests: the tests were right.

No VERSION bump. Not deployed.

chore(deploy): declare OIDC env in compose so a deploy needs no manual setup ... a29e2a95bb

Adds every OIDC_* variable the server reads to docker-compose.coolify.yml, with
defaults for the non-secret endpoints taken verbatim from the provider's
discovery document (/application/o/ai-api/.well-known/openid-configuration).
Only the two secrets (OIDC_CLIENT_ID, OIDC_CLIENT_SECRET) have to exist in
Coolify, so a future deploy picks up the configuration without anyone
re-entering it by hand.

Two silent-failure traps this avoids, both of which look "configured" in the UI
while login stays broken:

1. The previous env block declared ODIC_ISSUER / ODIC_CLIENT_ID /
   ODIC_CLIENT_SECRET. The code reads OIDC_*, so those rows bound to nothing.
2. Authentik serves authorize/token/userinfo from GLOBAL paths
   (/application/o/authorize/), not per-application ones. Only jwks and
   end-session are per-app, and the issuer needs a trailing slash to match the
   token's `iss` claim.

Also drops ENTERPRISE_FEATURES_ENABLED and MONITORING_ENABLED, which nothing in
the codebase reads, and pins ADMIN_API_ACCEPTS_INFERENCE_KEYS=false so the
admin-API boundary stays closed in deployed environments.

Adds scripts/aexora/coolify_set_oidc_env.sh to push the two secrets via the
Coolify API from the gitignored deploy/oidc.env. It PATCHes by key (POST on
first create), never echoes a secret, refuses to run if the control plane isn't
returning 200, and deliberately does NOT trigger a deploy -- config and deploy
stay separate steps.

No VERSION bump. Not deployed.

epistemophiliac merged commit faaec96b6b into main 2026-08-19 03:05:01 +00:00

epistemophiliac referenced this pull request from a commit 2026-08-19 03:05:03 +00:00

feat(enterprise): cost model, working budgets, virtual keys, guardrails, teams, OIDC (#1)

epistemophiliac referenced this pull request from a commit 2026-08-22 13:47:15 +00:00

fix: verified v0.8.0 review — real fixes replace unverified claims

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

epistemophiliac/LLMAPI!1

No description provided.