Move Authentik OIDC into Streamlit with client secret.
Remove proxy forward-auth header trust; app runs authorization code flow using OIDC_CLIENT_SECRET and registers redirect URI from SERVICE_URL or OIDC_REDIRECT_URI.
This commit is contained in:
parent
7f7133f535
commit
9756809b68
6 changed files with 226 additions and 102 deletions
10
.env.example
10
.env.example
|
|
@ -1,14 +1,12 @@
|
|||
# Bugsink (Sentry-compatible DSN from bugsink.aexoradao.com project settings)
|
||||
BUGSINK_DSN=
|
||||
|
||||
# Authentik / OIDC (configured on Coolify proxy; app trusts forwarded headers)
|
||||
AUTHENTIK_ISSUER=https://auth.aexoradao.com/application/o/quant-web/
|
||||
# In-app Authentik OIDC (client secret stays in the app, not the proxy)
|
||||
OIDC_ISSUER=https://auth.aexoradao.com/application/o/quant-web/
|
||||
AUTHENTIK_OUTPOST_URL=https://auth.aexoradao.com/outpost.goauthentik.io/auth/traefik
|
||||
OIDC_CLIENT_ID=quant-web
|
||||
AUTH_USERNAME_HEADER=X-Forwarded-User
|
||||
AUTH_UID_HEADER=X-Authentik-Uid
|
||||
AUTH_EMAIL_HEADER=X-Forwarded-Email
|
||||
OIDC_CLIENT_SECRET=
|
||||
OIDC_REDIRECT_URI=
|
||||
OIDC_SCOPES=openid profile email
|
||||
AUTH_REQUIRED=true
|
||||
|
||||
# Local dev only when AUTH_REQUIRED=false
|
||||
|
|
|
|||
61
README.md
61
README.md
|
|
@ -1,80 +1,67 @@
|
|||
# QuantTrade
|
||||
|
||||
Local-first quantitative backtesting on Coolify: Streamlit UI, VectorBT engine, Parquet market data, nightly Yahoo Finance sync, Authentik OIDC via reverse proxy, and SQLite strategy persistence.
|
||||
Local-first quantitative backtesting on Coolify: Streamlit UI, VectorBT engine, Parquet market data, nightly Yahoo Finance sync, in-app Authentik OIDC, and SQLite strategy persistence.
|
||||
|
||||
## Architecture
|
||||
|
||||
| Layer | Technology |
|
||||
|-------|------------|
|
||||
| Auth | Authentik (`auth.aexoradao.com`) via reverse proxy headers |
|
||||
| Auth | Authentik OIDC inside Streamlit (`OIDC_CLIENT_SECRET` in app env) |
|
||||
| UI | Streamlit (`streamlit` service, port 8501) |
|
||||
| Engine | VectorBT + NumPy |
|
||||
| Market data | Parquet volume (`parquet-data`) |
|
||||
| Ingestion | `harvester` cron @ 17:00 America/New_York (weekdays) |
|
||||
| Strategies | SQLite on `strategy-data` volume, keyed by proxy username |
|
||||
| Strategies | SQLite on `strategy-data` volume, keyed by OIDC username |
|
||||
| Telemetry | Bugsink via `sentry-sdk` (`bugsink.aexoradao.com`) |
|
||||
|
||||
## Coolify deployment
|
||||
|
||||
1. Create a **Docker Compose** resource pointing at this repo.
|
||||
2. Assign your public domain to the **`streamlit`** service on port **8501**.
|
||||
3. Enable **Authentik forward auth** on that domain in Coolify (OIDC happens at the proxy; Streamlit never holds client secrets).
|
||||
4. Set environment variables in Coolify (first deploy extracts defaults from compose):
|
||||
3. **Do not** enable Authentik forward auth on the Coolify proxy for this app — login happens inside Streamlit.
|
||||
4. Set environment variables in Coolify:
|
||||
- `OIDC_CLIENT_SECRET` — **required**; copy from your Authentik provider
|
||||
- `OIDC_CLIENT_ID` — Authentik provider slug (default `quant-web`)
|
||||
- `OIDC_ISSUER` — Authentik issuer URL for the provider
|
||||
- `OIDC_REDIRECT_URI` — optional; defaults to `SERVICE_URL_STREAMLIT_8501`
|
||||
- `BUGSINK_DSN` — DSN from your Bugsink project
|
||||
- `AUTHENTIK_ISSUER`, `OIDC_ISSUER`, `AUTHENTIK_OUTPOST_URL`, `OIDC_CLIENT_ID` — match your Authentik application/outpost
|
||||
- `AUTH_USERNAME_HEADER` — header Coolify/Traefik forwards after login (default `X-Forwarded-User`)
|
||||
- `AUTH_REQUIRED` — keep `true` in production
|
||||
- `CORE_TICKERS` — optional comma-separated tickers
|
||||
- `DEV_USER` — only when `AUTH_REQUIRED=false` for local testing
|
||||
|
||||
**Coolify note:** if you change any default above after the first deploy, update the value manually in Coolify UI > Environment Variables. Coolify stores first-seen defaults and will not auto-refresh them from compose.
|
||||
**Coolify note:** if you change any default after the first deploy, update the value manually in Coolify UI > Environment Variables.
|
||||
|
||||
### Authentik / proxy headers
|
||||
### Authentik provider setup
|
||||
|
||||
OIDC login is handled by Authentik + Coolify reverse proxy. The container does not run an OAuth code flow; it trusts identity headers injected after forward auth.
|
||||
In Authentik, create an **OAuth2/OIDC provider** for this app:
|
||||
|
||||
After login, the proxy must forward headers such as:
|
||||
1. **Client type:** Confidential
|
||||
2. **Client ID:** `quant-web` (or match `OIDC_CLIENT_ID`)
|
||||
3. **Client secret:** paste into Coolify as `OIDC_CLIENT_SECRET`
|
||||
4. **Redirect URIs:** your public app URL, e.g. `https://quant.example.com`
|
||||
- Must match `OIDC_REDIRECT_URI` or the Coolify-generated `SERVICE_URL_STREAMLIT_8501`
|
||||
5. **Signing key:** RS256 (Authentik default)
|
||||
|
||||
- `X-Forwarded-User` (default `AUTH_USERNAME_HEADER`)
|
||||
- `X-Authentik-Uid` (`AUTH_UID_HEADER`)
|
||||
- `X-Forwarded-Email` (`AUTH_EMAIL_HEADER`)
|
||||
|
||||
The app reads them via Streamlit websocket headers (`auth.get_current_user()`). With `AUTH_REQUIRED=true`, missing headers block the UI instead of falling back to anonymous.
|
||||
|
||||
Example Traefik middleware (adjust provider labels to your stack):
|
||||
|
||||
```yaml
|
||||
# Forward auth endpoint on Authentik
|
||||
http:
|
||||
middlewares:
|
||||
authentik:
|
||||
forwardAuth:
|
||||
address: https://auth.aexoradao.com/outpost.goauthentik.io/auth/traefik
|
||||
trustForwardHeader: true
|
||||
authResponseHeaders:
|
||||
- X-authentik-username
|
||||
- X-authentik-uid
|
||||
- X-Forwarded-User
|
||||
```
|
||||
|
||||
Map `X-authentik-username` → `X-Forwarded-User` in your proxy if Streamlit only sees the latter.
|
||||
Users hit the app → Streamlit redirects to Authentik → callback with auth code → app exchanges code using `OIDC_CLIENT_SECRET` → userinfo drives strategy ownership.
|
||||
|
||||
## Services
|
||||
|
||||
- **`data-seed`** — one-shot 5-year historical download into Parquet (idempotent).
|
||||
- **`harvester`** — cron container; appends daily bars after US cash close.
|
||||
- **`streamlit`** — dashboard, backtests, save/load strategies.
|
||||
- **`streamlit`** — dashboard, OIDC login, backtests, save/load strategies.
|
||||
|
||||
## Local development
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
# Set AUTH_REQUIRED=false and DEV_USER for local testing without Authentik
|
||||
python -m venv .venv && source .venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
python sync.py --seed
|
||||
DEV_USER=you@example.com streamlit run app.py
|
||||
streamlit run app.py
|
||||
```
|
||||
|
||||
For full OIDC locally, set `OIDC_CLIENT_SECRET` and register `http://localhost:8501` as a redirect URI in Authentik.
|
||||
|
||||
## Manual sync
|
||||
|
||||
```bash
|
||||
|
|
|
|||
18
app.py
18
app.py
|
|
@ -9,7 +9,7 @@ import plotly.graph_objects as go
|
|||
import streamlit as st
|
||||
from plotly.subplots import make_subplots
|
||||
|
||||
from auth import get_current_user
|
||||
from auth import get_current_user, logout
|
||||
from backtest import load_ohlcv, run_ma_crossover
|
||||
from strategy_db import delete_strategy, init_db, list_strategies, load_strategy, save_strategy
|
||||
from telemetry import capture_exception, init_telemetry
|
||||
|
|
@ -60,17 +60,8 @@ def render_equity_chart(result) -> None:
|
|||
|
||||
def main() -> None:
|
||||
user = get_current_user()
|
||||
if user is None:
|
||||
st.error(
|
||||
"Authentication required. Access this app through Authentik-protected "
|
||||
"Coolify proxy so identity headers are forwarded."
|
||||
)
|
||||
st.caption(
|
||||
"Expected header: "
|
||||
f"`{os.environ.get('AUTH_USERNAME_HEADER', 'X-Forwarded-User')}` "
|
||||
f"from `{os.environ.get('AUTHENTIK_ISSUER', 'https://auth.aexoradao.com')}`."
|
||||
)
|
||||
st.stop()
|
||||
if not user:
|
||||
return
|
||||
|
||||
st.title("QuantTrade")
|
||||
st.caption("VectorBT backtests on local Parquet market data")
|
||||
|
|
@ -78,6 +69,9 @@ def main() -> None:
|
|||
with st.sidebar:
|
||||
st.subheader("Account")
|
||||
st.write(f"Signed in as **{user}**")
|
||||
if st.button("Logout", use_container_width=True):
|
||||
logout()
|
||||
st.rerun()
|
||||
|
||||
st.divider()
|
||||
st.subheader("Strategy")
|
||||
|
|
|
|||
229
auth.py
229
auth.py
|
|
@ -1,17 +1,23 @@
|
|||
"""Read Authentik / reverse-proxy identity headers in Streamlit."""
|
||||
"""In-app OIDC authentication against Authentik."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import os
|
||||
from typing import Mapping
|
||||
import secrets
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
DEFAULT_HEADER_CANDIDATES = (
|
||||
"X-Forwarded-User",
|
||||
"X-Authentik-Username",
|
||||
"X-Authentik-Uid",
|
||||
"Remote-User",
|
||||
"X-Forwarded-Email",
|
||||
)
|
||||
import requests
|
||||
import streamlit as st
|
||||
from authlib.integrations.requests_client import OAuth2Session
|
||||
|
||||
SESSION_USER_KEY = "oidc_username"
|
||||
SESSION_SUB_KEY = "oidc_sub"
|
||||
STATE_TTL_SECONDS = 600
|
||||
|
||||
|
||||
def _truthy(value: str | None, default: bool = False) -> bool:
|
||||
|
|
@ -24,49 +30,188 @@ def auth_required() -> bool:
|
|||
return _truthy(os.environ.get("AUTH_REQUIRED"), default=True)
|
||||
|
||||
|
||||
def header_candidates() -> tuple[str, ...]:
|
||||
primary = os.environ.get("AUTH_USERNAME_HEADER", "").strip()
|
||||
extras = [
|
||||
os.environ.get("AUTH_UID_HEADER", "").strip(),
|
||||
os.environ.get("AUTH_EMAIL_HEADER", "").strip(),
|
||||
]
|
||||
ordered: list[str] = []
|
||||
for name in (primary, *extras, *DEFAULT_HEADER_CANDIDATES):
|
||||
if name and name not in ordered:
|
||||
ordered.append(name)
|
||||
return tuple(ordered)
|
||||
def oidc_issuer() -> str:
|
||||
raw = os.environ.get("OIDC_ISSUER") or os.environ.get("AUTHENTIK_ISSUER", "")
|
||||
return raw.rstrip("/") + "/"
|
||||
|
||||
|
||||
def _normalize(value: str | None) -> str | None:
|
||||
if not value:
|
||||
return None
|
||||
cleaned = value.strip()
|
||||
return cleaned or None
|
||||
def oidc_client_id() -> str:
|
||||
return os.environ.get("OIDC_CLIENT_ID", "").strip()
|
||||
|
||||
|
||||
def username_from_headers(headers: Mapping[str, str]) -> str | None:
|
||||
lowered = {k.lower(): v for k, v in headers.items()}
|
||||
for name in header_candidates():
|
||||
value = _normalize(lowered.get(name.lower()))
|
||||
def oidc_client_secret() -> str:
|
||||
return os.environ.get("OIDC_CLIENT_SECRET", "").strip()
|
||||
|
||||
|
||||
def oidc_scopes() -> str:
|
||||
return os.environ.get("OIDC_SCOPES", "openid profile email")
|
||||
|
||||
|
||||
def redirect_uri() -> str:
|
||||
explicit = os.environ.get("OIDC_REDIRECT_URI", "").strip()
|
||||
if explicit:
|
||||
return explicit.rstrip("/")
|
||||
service_url = os.environ.get("SERVICE_URL_STREAMLIT_8501", "").strip()
|
||||
if service_url:
|
||||
return service_url.rstrip("/")
|
||||
return "http://localhost:8501"
|
||||
|
||||
|
||||
@st.cache_data(ttl=3600, show_spinner=False)
|
||||
def discovery_document(issuer: str) -> dict[str, Any]:
|
||||
url = f"{issuer.rstrip('/')}/.well-known/openid-configuration"
|
||||
response = requests.get(url, timeout=15)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
|
||||
def _oauth_client() -> OAuth2Session:
|
||||
return OAuth2Session(
|
||||
client_id=oidc_client_id(),
|
||||
client_secret=oidc_client_secret(),
|
||||
redirect_uri=redirect_uri(),
|
||||
scope=oidc_scopes(),
|
||||
)
|
||||
|
||||
|
||||
def _sign_state(payload: dict[str, Any]) -> str:
|
||||
secret = oidc_client_secret().encode()
|
||||
body = base64.urlsafe_b64encode(json.dumps(payload, sort_keys=True).encode()).decode()
|
||||
signature = hmac.new(secret, body.encode(), hashlib.sha256).hexdigest()
|
||||
return f"{body}.{signature}"
|
||||
|
||||
|
||||
def _verify_state(state: str) -> bool:
|
||||
try:
|
||||
body, signature = state.rsplit(".", 1)
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
expected = hmac.new(oidc_client_secret().encode(), body.encode(), hashlib.sha256).hexdigest()
|
||||
if not hmac.compare_digest(signature, expected):
|
||||
return False
|
||||
|
||||
payload = json.loads(base64.urlsafe_b64decode(body.encode()).decode())
|
||||
issued_at = int(payload.get("ts", 0))
|
||||
if time.time() - issued_at > STATE_TTL_SECONDS:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _create_state() -> str:
|
||||
payload = {
|
||||
"nonce": secrets.token_urlsafe(16),
|
||||
"ts": int(time.time()),
|
||||
}
|
||||
return _sign_state(payload)
|
||||
|
||||
|
||||
def _username_from_userinfo(userinfo: dict[str, Any]) -> str:
|
||||
for key in ("preferred_username", "email", "name", "sub"):
|
||||
value = userinfo.get(key)
|
||||
if value:
|
||||
return value
|
||||
return str(value)
|
||||
return "unknown"
|
||||
|
||||
|
||||
def _dev_bypass() -> str | None:
|
||||
if auth_required():
|
||||
return None
|
||||
dev_user = os.environ.get("DEV_USER", "").strip()
|
||||
return dev_user or "anonymous"
|
||||
|
||||
|
||||
def _config_error() -> str | None:
|
||||
if not auth_required():
|
||||
return None
|
||||
if not oidc_client_id():
|
||||
return "OIDC_CLIENT_ID is not set."
|
||||
if not oidc_client_secret():
|
||||
return "OIDC_CLIENT_SECRET is not set."
|
||||
if not (os.environ.get("OIDC_ISSUER") or os.environ.get("AUTHENTIK_ISSUER")):
|
||||
return "OIDC_ISSUER (or AUTHENTIK_ISSUER) is not set."
|
||||
return None
|
||||
|
||||
|
||||
def logout() -> None:
|
||||
st.session_state.pop(SESSION_USER_KEY, None)
|
||||
st.session_state.pop(SESSION_SUB_KEY, None)
|
||||
st.query_params.clear()
|
||||
|
||||
|
||||
def get_current_user() -> str | None:
|
||||
"""Return the authenticated username from proxy-injected headers."""
|
||||
try:
|
||||
from streamlit.web.server.websocket_headers import _get_websocket_headers
|
||||
"""Return username after in-app OIDC login, or stop and render login UI."""
|
||||
if st.session_state.get(SESSION_USER_KEY):
|
||||
return str(st.session_state[SESSION_USER_KEY])
|
||||
|
||||
headers = _get_websocket_headers() or {}
|
||||
user = username_from_headers(headers)
|
||||
if user:
|
||||
return user
|
||||
except Exception:
|
||||
pass
|
||||
bypass = _dev_bypass()
|
||||
if bypass:
|
||||
return bypass
|
||||
|
||||
if auth_required():
|
||||
config_error = _config_error()
|
||||
if config_error:
|
||||
st.error(config_error)
|
||||
st.caption("Set OIDC_CLIENT_ID, OIDC_CLIENT_SECRET, and OIDC_ISSUER in Coolify.")
|
||||
st.stop()
|
||||
return None
|
||||
|
||||
dev_user = os.environ.get("DEV_USER", "").strip()
|
||||
return dev_user or "anonymous"
|
||||
query = st.query_params
|
||||
code = query.get("code")
|
||||
state = query.get("state")
|
||||
oauth_error = query.get("error")
|
||||
|
||||
if oauth_error:
|
||||
st.error(f"Authentik login failed: {oauth_error}")
|
||||
st.stop()
|
||||
return None
|
||||
|
||||
try:
|
||||
metadata = discovery_document(oidc_issuer())
|
||||
except Exception as exc:
|
||||
st.error(f"Could not load OIDC discovery document: {exc}")
|
||||
st.stop()
|
||||
return None
|
||||
|
||||
client = _oauth_client()
|
||||
|
||||
if code:
|
||||
if not state or not _verify_state(state):
|
||||
st.error("Invalid or expired OAuth state. Sign in again.")
|
||||
st.query_params.clear()
|
||||
st.stop()
|
||||
return None
|
||||
|
||||
try:
|
||||
auth_response = f"{redirect_uri()}?code={code}&state={state}"
|
||||
client.fetch_token(
|
||||
metadata["token_endpoint"],
|
||||
authorization_response=auth_response,
|
||||
)
|
||||
userinfo = client.get(metadata["userinfo_endpoint"]).json()
|
||||
except Exception as exc:
|
||||
st.error(f"Token exchange failed: {exc}")
|
||||
st.caption(f"Redirect URI in use: `{redirect_uri()}`")
|
||||
st.stop()
|
||||
return None
|
||||
|
||||
st.session_state[SESSION_USER_KEY] = _username_from_userinfo(userinfo)
|
||||
st.session_state[SESSION_SUB_KEY] = str(userinfo.get("sub", ""))
|
||||
st.query_params.clear()
|
||||
st.rerun()
|
||||
return None
|
||||
|
||||
state_token = _create_state()
|
||||
auth_url, _ = client.create_authorization_url(
|
||||
metadata["authorization_endpoint"],
|
||||
state=state_token,
|
||||
)
|
||||
|
||||
st.title("QuantTrade")
|
||||
st.info("Sign in with Authentik to access backtests and saved strategies.")
|
||||
st.link_button("Login with Authentik", auth_url, type="primary")
|
||||
st.caption(
|
||||
f"Register redirect URI `{redirect_uri()}` in your Authentik OIDC provider "
|
||||
f"({oidc_client_id()})."
|
||||
)
|
||||
st.stop()
|
||||
return None
|
||||
|
|
|
|||
|
|
@ -64,13 +64,11 @@ services:
|
|||
- 'CORE_TICKERS=${CORE_TICKERS:-SPY,QQQ,AAPL,MSFT,GOOGL,AMZN,NVDA,META,IWM,TLT}'
|
||||
- 'PARQUET_DIR=/data/parquet'
|
||||
- 'STRATEGY_DB_PATH=/data/strategies/strategies.db'
|
||||
- 'AUTHENTIK_ISSUER=${AUTHENTIK_ISSUER:-https://auth.aexoradao.com/application/o/quant-web/}'
|
||||
- 'OIDC_ISSUER=${OIDC_ISSUER:-https://auth.aexoradao.com/application/o/quant-web/}'
|
||||
- 'AUTHENTIK_OUTPOST_URL=${AUTHENTIK_OUTPOST_URL:-https://auth.aexoradao.com/outpost.goauthentik.io/auth/traefik}'
|
||||
- 'OIDC_CLIENT_ID=${OIDC_CLIENT_ID:-quant-web}'
|
||||
- 'AUTH_USERNAME_HEADER=${AUTH_USERNAME_HEADER:-X-Forwarded-User}'
|
||||
- 'AUTH_UID_HEADER=${AUTH_UID_HEADER:-X-Authentik-Uid}'
|
||||
- 'AUTH_EMAIL_HEADER=${AUTH_EMAIL_HEADER:-X-Forwarded-Email}'
|
||||
- 'OIDC_CLIENT_SECRET=${OIDC_CLIENT_SECRET:-}'
|
||||
- 'OIDC_REDIRECT_URI=${OIDC_REDIRECT_URI:-}'
|
||||
- 'OIDC_SCOPES=${OIDC_SCOPES:-openid profile email}'
|
||||
- 'AUTH_REQUIRED=${AUTH_REQUIRED:-true}'
|
||||
- 'DEV_USER=${DEV_USER:-}'
|
||||
volumes:
|
||||
|
|
|
|||
|
|
@ -6,3 +6,5 @@ numpy>=1.26.0
|
|||
pyarrow>=15.0.0
|
||||
plotly>=5.18.0
|
||||
sentry-sdk>=2.0.0
|
||||
authlib>=1.3.0
|
||||
requests>=2.31.0
|
||||
|
|
|
|||
Loading…
Reference in a new issue