627b2326df
Support builtin and custom generate_signals strategies with SQLite persistence, exhaustive grid scans (VectorBT comb optimization for MA crossover), professional backtest/optimize UI, and split harvester/app requirements with BuildKit pip cache.
111 lines
4.4 KiB
Markdown
111 lines
4.4 KiB
Markdown
# QuantTrade
|
|
|
|
Local-first quantitative backtesting on Coolify: Streamlit UI, VectorBT engine, Parquet market data, nightly Yahoo Finance sync, in-app Authentik OIDC, and SQLite strategy persistence.
|
|
|
|
## Architecture
|
|
|
|
| Layer | Technology |
|
|
|-------|------------|
|
|
| Auth | Authentik OIDC inside Streamlit (`OIDC_CLIENT_SECRET` in app env) |
|
|
| UI | Streamlit (`streamlit` service, port 8501) |
|
|
| Engine | VectorBT + NumPy |
|
|
| Market data | Parquet volume (`parquet-data`) |
|
|
| Ingestion | `harvester` cron @ 17:00 America/New_York (weekdays) |
|
|
| Strategies | SQLite on `strategy-data` volume, keyed by OIDC username |
|
|
| Telemetry | Bugsink via `sentry-sdk` (`bugsink.aexoradao.com`) |
|
|
|
|
## Coolify deployment
|
|
|
|
1. Create a **Docker Compose** resource pointing at this repo.
|
|
2. Assign your public domain to the **`streamlit`** service on port **8501**.
|
|
3. **Do not** enable Authentik forward auth on the Coolify proxy for this app — login happens inside Streamlit.
|
|
4. Set environment variables in Coolify:
|
|
- `OIDC_CLIENT_SECRET` — **required**; copy from your Authentik provider
|
|
- `OIDC_CLIENT_ID` — Authentik provider slug (default `quant-web`)
|
|
- `OIDC_ISSUER` — Authentik issuer URL for the provider
|
|
- `OIDC_REDIRECT_URI` — optional; defaults to `SERVICE_URL_STREAMLIT_8501`
|
|
- `BUGSINK_DSN` — DSN from your Bugsink project
|
|
- `AUTH_REQUIRED` — keep `true` in production
|
|
- `DEV_USER` — only when `AUTH_REQUIRED=false` for local testing
|
|
|
|
**Coolify note:** if you change any default after the first deploy, update the value manually in Coolify UI > Environment Variables.
|
|
|
|
### Authentik provider setup
|
|
|
|
In Authentik, create an **OAuth2/OIDC provider** for this app:
|
|
|
|
1. **Client type:** Confidential
|
|
2. **Client ID:** `quant-web` (or match `OIDC_CLIENT_ID`)
|
|
3. **Client secret:** paste into Coolify as `OIDC_CLIENT_SECRET`
|
|
4. **Redirect URIs:** your public app URL, e.g. `https://quant.example.com`
|
|
- Must match `OIDC_REDIRECT_URI` or the Coolify-generated `SERVICE_URL_STREAMLIT_8501`
|
|
5. **Signing key:** RS256 (Authentik default)
|
|
|
|
Users hit the app → Streamlit redirects to Authentik → callback with auth code → app exchanges code using `OIDC_CLIENT_SECRET` → userinfo drives strategy ownership.
|
|
|
|
## Services
|
|
|
|
- **`data-seed`** — one-shot 5-year historical download into Parquet (idempotent).
|
|
- **`harvester`** — cron container; appends daily bars after US cash close.
|
|
- **`streamlit`** — dashboard, OIDC login, backtests, save/load strategies.
|
|
|
|
## Local development
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
# Set AUTH_REQUIRED=false and DEV_USER for local testing without Authentik
|
|
python -m venv .venv && source .venv/bin/activate
|
|
pip install -r requirements.txt
|
|
python sync.py --seed
|
|
streamlit run app.py
|
|
```
|
|
|
|
For full OIDC locally, set `OIDC_CLIENT_SECRET` and register `http://localhost:8501` as a redirect URI in Authentik.
|
|
|
|
## Research workflow
|
|
|
|
1. **Backtest** — run a single parameter set and inspect equity curve, drawdown, trades.
|
|
2. **Optimize** — exhaustive parameter scan (grid search) ranked by Sharpe, Sortino, return, or drawdown.
|
|
3. **Python** — view builtin source or author custom strategies with `generate_signals()`.
|
|
4. **Library** — save/load strategies per user (SQLite), including custom Python source code.
|
|
|
|
### Custom Python strategy contract
|
|
|
|
```python
|
|
PARAM_GRID = {"fast_window": list(range(10, 41, 5)), "slow_window": list(range(50, 151, 10))}
|
|
DEFAULT_PARAMS = {"fast_window": 20, "slow_window": 50}
|
|
|
|
def generate_signals(close, high, low, volume, **params):
|
|
# return boolean entry/exit Series aligned to close
|
|
return entries, exits
|
|
```
|
|
|
|
Builtins: `ma_crossover` (vectorized VectorBT comb scan), `rsi_reversion` (grid scan).
|
|
|
|
## Docker build speed
|
|
|
|
- Harvester image installs only `requirements-harvester.txt` (no VectorBT/Streamlit).
|
|
- App image uses BuildKit pip cache (`RUN --mount=type=cache`).
|
|
- `.dockerignore` keeps git/cache out of build context.
|
|
|
|
Enable BuildKit on Coolify/build host for cache mounts.
|
|
|
|
|
|
```bash
|
|
python sync.py --seed # full history
|
|
python sync.py --daily # append latest bars
|
|
```
|
|
|
|
## Strategy storage
|
|
|
|
SQLite path: `/data/strategies/strategies.db` (Docker volume `strategy-data`). Each row stores username, name, ticker, and JSON parameters (MA windows, cash, fees).
|
|
|
|
## Bugsink
|
|
|
|
Set `BUGSINK_DSN` to your project DSN, e.g.:
|
|
|
|
```text
|
|
http://<public-key>@bugsink.aexoradao.com/<project-id>
|
|
```
|
|
|
|
Both `app.py` and `sync.py` initialize the Sentry-compatible SDK with tracing disabled for Bugsink compatibility.
|