# Deployment Guide — cPanel Shared Hosting (Namecheap)

## Overview

The forwarder runs as a **CloudLinux Python Selector** app on shared cPanel
hosting (Namecheap `business122.web-hosting.com`), served by **Phusion
Passenger** behind LiteSpeed. There is no long-running service manager and no
open port: LiteSpeed proxies `https://quill.nx.kg/f` to Passenger, which loads
the Flask app once per worker process.

### Architecture

```
https://quill.nx.kg/f
        │
        ▼
LiteSpeed vhost (quill.nx.kg, IP 192.64.117.115)
        │  ~/quill.nx.kg/f/.htaccess   ← selector-managed:
        │    PassengerAppRoot /home/wovewbge/quill.nx.kg
        │    PassengerBaseURI /f
        │    PassengerPython  ~/virtualenv/quill.nx.kg/3.13/bin/python
        ▼
Passenger worker (max 1 — see below)
        │  ~/quill.nx.kg/passenger_wsgi.py  →  from app import app as application
        ▼
Flask app (app.py) + Telethon client in a daemon thread
        │
        ▼
MariaDB (cPanel: localhost, wovewbge_quillapp)
```

### Key facts

| Piece | Location |
|-------|----------|
| App code (= PassengerAppRoot = addon docroot) | `~/quill.nx.kg/` |
| Entry point (startup file) | `~/quill.nx.kg/passenger_wsgi.py` |
| Selector venv | `~/virtualenv/quill.nx.kg/3.13/` |
| Restart trigger | `touch ~/quill.nx.kg/tmp/restart.txt` |
| Database | `wovewbge_quillapp` on `localhost` (user `wovewbge_quilluser`) |
| Public URL | `https://quill.nx.kg/f` |
| SSH access | `ssh jnc` (port 21098, user `wovewbge`) |

Two deployment-specific tweaks are essential:

1. **`PassengerMaxPoolSize 1`** (in `~/quill.nx.kg/f/.htaccess`) — the app runs
   a stateful Telethon client thread; multiple Passenger workers would
   duplicate the Telegram session (`AUTH_KEY_DUPLICATED`).
2. **Keep-alive cron** — Passenger reaps idle workers (~5 min), which would
   silently kill the Telegram listener. A per-minute cron hits the health
   endpoint to keep one worker alive:
   ```
   * * * * * curl -sk --resolve quill.nx.kg:443:192.64.117.115 https://quill.nx.kg/f/api/health >/dev/null 2>&1
   ```

---

## Project Files

### Application Code

| File | Purpose |
|------|---------|
| `app.py` | Flask dashboard & API |
| `core.py` | Telethon client, auth flow, forwarder logic (NewMessage, Edit, Delete) |
| `db.py` | MariaDB storage wrapper with connection pooling and Fernet encryption |
| `config.py` | Environment configuration + DB-backed config overrides |
| `crypto_vault.py` | Symmetric Fernet encryption/decryption for cTrader tokens |
| `ctrader_spot_client.py` | cTrader Open API spot price client |
| `forwarding.py` | Message forwarding and price augmentation logic |
| `signal_parser.py` | Signal text parser for extracting trading symbols |
| `signal_processor.py` | Signal processing pipeline (parse → fetch price → augment → forward) |
| `llm_classifier.py` | Optional LLM-based signal classification |
| `passenger_wsgi.py` | Passenger entry point (`application`) |
| `templates/` | Dashboard UI (index.html + components/) |

### Database & Schema

| File | Purpose |
|------|---------|
| `schema.sql` | Database schema (all tables, `CREATE IF NOT EXISTS`) |
| `migrate_existing.sql` | Idempotent migration for pre-existing databases |
| `dev_generate_keys.py` | Generate `CTRADER_TOKEN_SECRET` and update `.env` |

### Deployment Scripts

All scripts live in `scripts/` and share helpers from `scripts/common.py`.

| Script | Purpose |
|--------|---------|
| `scripts/common.py` | Shared helpers: `load_env`, `ssh_run`, `ssh_upload`, `RemoteDB`, `ForwarderClient` |
| `scripts/seed_config.py` | Seed `.env` values into remote `config_vars` DB table |
| `scripts/apply_schema.py` | Apply `schema.sql` + `migrate_existing.sql` to remote MySQL |
| `scripts/selector_env.py` | Emit JSON of env vars for CloudLinux Python Selector UI |
| `scripts/remote_db.py` | CLI to inspect remote MySQL data (tables, config, pairs, signal log) |
| `scripts/cpanel_mysql.py` | Manage cPanel MySQL DBs/users via UAPI |
| `scripts/ctrader_refresh.py` | External cron: refresh cTrader OAuth tokens |
| `scripts/ctrader_consumer.py` | Fetch and decrypt cTrader tokens from forwarder |
| `scripts/le_issue_quill.py` | Let's Encrypt ACME v2 cert issuance (HTTP-01) |

### Config & Deploy

| File | Purpose |
|------|---------|
| `deploy.sh` | Full deploy: rsync → pip → schema → seed config → selector env → restart |
| `requirements.txt` | Python dependencies |
| `.env` | Local environment variables (not synced to server) |

---

## 1. Prerequisites

### 1.1 SSH Access

Ensure `~/.ssh/config` has an entry for the cPanel server:

```
Host jnc
    HostName business122.web-hosting.com
    User wovewbge
    Port 21098
    IdentityFile ~/.ssh/id_rsa
```

Verify with:
```bash
ssh jnc true && echo "SSH OK"
```

### 1.2 Local .env File

The `.env` file in the project root is the **single source of truth** for all
configuration. It contains:

- MySQL credentials (used by deploy scripts and seeded into selector env)
- Flask secret key (seeded into selector env)
- Telegram API credentials (seeded into `config_vars` DB table)
- cTrader OAuth credentials (seeded into `config_vars`, secrets encrypted)
- LLM configuration (seeded into `config_vars`)
- cPanel UAPI credentials (used by `cpanel_mysql.py` and `le_issue_quill.py`)
- cTrader token vault secret (used for Fernet encryption)

See the [Environment Variables](#2-environment-variables) section for the
complete reference.

### 1.3 Python 3.13 Virtualenv on Server

The CloudLinux Python Selector manages the virtualenv at
`~/virtualenv/quill.nx.kg/3.13/`. The `deploy.sh` script installs
requirements into this venv automatically.

---

## 2. Environment Variables

### 2.1 Variables Set in the CloudLinux Python Selector

These 5 variables must be set in the Passenger process environment (via the
selector UI or `cloudlinux-selector set --env-vars`). They are **not** stored
in the database because they are needed *before* the database connection can be
established.

| Variable | Example | Purpose |
|----------|---------|---------|
| `MYSQL_HOST` | `localhost` | Database host (always localhost on shared hosting) |
| `MYSQL_USER` | `wovewbge_quilluser` | Database user |
| `MYSQL_PASSWORD` | `••••••••` | Database password |
| `MYSQL_DB` | `wovewbge_quillapp` | Database name |
| `FLASK_SECRET_KEY` | `tg-forwarder-secret-key-2026` | Flask session signing key (also used to derive Fernet key for DB secret encryption) |

The `deploy.sh` script sets these automatically via `scripts/selector_env.py`
which reads the local `.env` and pushes them via `cloudlinux-selector set`.

### 2.2 Variables Seeded into the Database (config_vars)

All other configuration is stored in the `config_vars` MySQL table. The running
app loads them at startup via `config.patch_from_db(db)`. The
`deploy.sh` script seeds them from the local `.env` via
`scripts/seed_config.py`.

#### Plain (unencrypted) config vars

| Env Var | DB Key | Type | Purpose |
|---------|--------|------|---------|
| `TG_API_ID` | `tg_api_id` | int | Telegram API ID from my.telegram.org |
| `TG_PHONE` | `tg_phone` | str | Telegram phone number for auth |
| `WEB_PIN` | `web_pin` | str | Dashboard PIN (min 4 digits) |
| `SIGNAL_BOT_TOKEN` | `signal_bot_token` | str | Bot token for price augmentation |
| `SIGNAL_ADMIN_CHAT_ID` | `signal_admin_chat_id` | int | Admin chat ID for bot messages |
| `SIGNAL_WEBHOOK_URL` | `signal_webhook_url` | str | Outbound webhook URL for signals |
| `LLM_ENABLED` | `llm_enabled` | bool | Enable LLM signal classification |
| `LLM_PROVIDER` | `llm_provider` | str | LLM provider name |
| `LLM_BASE_URL` | `llm_base_url` | str | OpenAI-compatible API base URL |
| `LLM_MODEL` | `llm_model` | str | LLM model name |
| `LLM_MIN_CONFIDENCE` | `llm_min_confidence` | float | Minimum confidence threshold |
| `LLM_TIMEOUT` | `llm_timeout` | float | LLM request timeout (seconds) |
| `LLM_MAX_TOKENS` | `llm_max_tokens` | int | Max tokens for LLM response |
| `LLM_WEBHOOK_URL` | `llm_webhook_url` | str | Webhook for LLM classification results |
| `CTRADER_CLIENT_ID` | `ctrader_client_id` | str | cTrader OAuth app client ID |
| `CTRADER_REDIRECT_URI` | `ctrader_redirect_uri` | str | OAuth redirect URI |
| `CTRADER_ACCOUNT_ID` | `ctrader_account_id` | int | cTrader account ID |
| `CTRADER_USE_LIVE` | `ctrader_use_live` | bool | Use live trading environment |
| `CTRADER_SYMBOL` | `ctrader_symbol` | str | Default trading symbol (e.g. XAUUSD) |
| `CTRADER_SYMBOL_ID` | `ctrader_symbol_id` | int | cTrader symbol ID |
| `CTRADER_PRICE_TIMEOUT` | `ctrader_price_timeout` | float | Price fetch timeout (seconds) |
| `MAP_RETENTION_DAYS` | `map_retention_days` | int | Message map retention period |
| `MAX_TG_ACCOUNTS` | `max_tg_accounts` | int | Max Telegram accounts allowed |
| `QR_LOGIN_TIMEOUT` | `qr_login_timeout` | int | QR login timeout (seconds) |
| `PORT` | `port` | int | Local dev server port (not used in production) |

#### Secret (Fernet-encrypted) config vars

These values are encrypted at rest in the database using a Fernet key derived
from `FLASK_SECRET_KEY`. They are stored with an `enc:` prefix.

| Env Var | DB Key | Purpose |
|---------|--------|---------|
| `TG_API_HASH` | `tg_api_hash` | Telegram API hash |
| `TG_PASSWORD` | `tg_password` | Telegram 2FA password |
| `CTRADER_CLIENT_SECRET` | `ctrader_client_secret` | cTrader OAuth client secret |
| `CTRADER_ACCESS_TOKEN` | `ctrader_access_token` | cTrader API access token |
| `CTRADER_TOKEN_SECRET` | `ctrader_token_secret` | Shared secret for token vault encryption |
| `LLM_API_KEY` | `llm_api_key` | LLM provider API key |
| `LLM_WEBHOOK_SECRET` | `llm_webhook_secret` | Webhook signing secret for LLM |
| `SIGNAL_WEBHOOK_SECRET` | `signal_webhook_secret` | Webhook signing secret for signals |

### 2.3 Variables Used Only by Local Scripts

These are used by deployment/maintenance scripts and are never loaded by the
running app:

| Variable | Purpose | Used by |
|----------|---------|---------|
| `CPANEL_HOST` | cPanel hostname | `cpanel_mysql.py`, `le_issue_quill.py` |
| `CPANEL_USER` | cPanel username | `cpanel_mysql.py`, `le_issue_quill.py` |
| `CPANEL_API_TOKEN` | cPanel UAPI token | `cpanel_mysql.py`, `le_issue_quill.py` |
| `CTRADER_API_BASE` | Forwarder URL (default `https://quill.nx.kg/f`) | `ctrader_refresh.py`, `ctrader_consumer.py` |
| `SELECTOR_APP_ROOT` | Override CloudLinux selector app root | `deploy.sh` |

---

## 3. Database Setup

### 3.1 Create Database and User

Using the cPanel UAPI script:

```bash
# Create database
python3 scripts/cpanel_mysql.py create-db wovewbge_quillapp

# Create user with generated password
python3 scripts/cpanel_mysql.py create-user wovewbge_quilluser
# → prints generated password; add to .env as MYSQL_PASSWORD

# Grant ALL privileges
python3 scripts/cpanel_mysql.py grant --user wovewbge_quilluser --db wovewbge_quillapp --privileges ALL
```

Or via cPanel → MySQL Databases manually.

### 3.2 Apply Schema

The schema is applied automatically during `deploy.sh`. To apply manually:

```bash
python3 scripts/apply_schema.py --env .env --ssh-host jnc
```

This runs `schema.sql` (creates all tables with `IF NOT EXISTS`) followed by
`migrate_existing.sql` (idempotent ALTER TABLE migrations for existing
databases).

### 3.3 Database Tables

| Table | Purpose |
|-------|---------|
| `config_vars` | Key-value configuration store (encrypted secrets prefixed `enc:`) |
| `tg_accounts` | Telegram accounts (multi-account, QR login support) |
| `bot_tokens` | Bot tokens for forwarding (encrypted at rest) |
| `channel_pairs` | Source → destination channel mapping with filter options |
| `message_map` | Source message → destination message ID mapping for edit/delete sync |
| `auth_state` | Transient Telegram auth state (phone, code hash) |
| `ctrader_accounts` | cTrader account vault (encrypted OAuth tokens) |
| `signal_log` | Log of processed signals with price data |

### 3.4 Inspect Remote Database

```bash
# List all tables
python3 scripts/remote_db.py tables

# Show config vars (secrets masked)
python3 scripts/remote_db.py config

# Show raw config vars (including encrypted values)
python3 scripts/remote_db.py config-raw

# List cTrader accounts
python3 scripts/remote_db.py ctrader

# List Telegram accounts
python3 scripts/remote_db.py tg

# List channel pairs
python3 scripts/remote_db.py pairs

# Recent signal log entries
python3 scripts/remote_db.py signal-log --limit 20

# Custom query
python3 scripts/remote_db.py query "SELECT COUNT(*) FROM signal_log"
```

---

## 4. cPanel Application Configuration

### 4.1 CloudLinux Python Selector

The app is registered in the CloudLinux Python Selector with these settings:

| Setting | Value |
|---------|-------|
| Interpreter | Python 3.13 |
| App root | `/home/wovewbge/quill.nx.kg` |
| Startup file | `passenger_wsgi.py` |
| App URL | `https://quill.nx.kg/f` |

The selector creates `~/quill.nx.kg/f/.htaccess` automatically:

```apache
PassengerAppRoot /home/wovewbge/quill.nx.kg
PassengerBaseURI /f
PassengerPython /home/wovewbge/virtualenv/quill.nx.kg/3.13/bin/python
PassengerMaxPoolSize 1
```

**Do not sync over `f/.htaccess`** — `deploy.sh` excludes it.

### 4.2 Environment Variables in Selector

As described in section 2.1, five env vars must be set in the selector.
`deploy.sh` does this automatically using `scripts/selector_env.py` which
reads the local `.env` and calls:

```bash
cloudlinux-selector set --json --interpreter python \
  --user wovewbge \
  --app-root /home/wovewbge/quill.nx.kg \
  --env-vars '{"MYSQL_HOST":"localhost","MYSQL_USER":"...","MYSQL_PASSWORD":"...","MYSQL_DB":"...","FLASK_SECRET_KEY":"..."}'
```

To verify the current selector env vars:

```bash
ssh jnc "cloudlinux-selector get --json --interpreter python --user wovewbge"
```

### 4.3 Keep-Alive Cron

Passenger reaps idle workers after ~5 minutes, which kills the Telethon
listener. A per-minute cron keeps the worker alive:

```bash
# Add to crontab on the cPanel server (or via cPanel → Cron Jobs):
* * * * * curl -sk --resolve quill.nx.kg:443:192.64.117.115 https://quill.nx.kg/f/api/health >/dev/null 2>&1
```

### 4.4 TLS Certificate

AutoSSL is not enabled for this account. Certificates are issued manually
with Let's Encrypt:

```bash
python3 scripts/le_issue_quill.py
# → outputs cert.pem, chain.pem, fullchain.pem, domain.key to /tmp/le_quill/
```

Then install via cPanel → SSL/TLS → Manage SSL websites, or via UAPI:

```bash
ssh jnc "uapi SSL install_ssl domain=quill.nx.kg cert=$(cat /tmp/le_quill/cert.pem) key=$(cat /tmp/le_quill/domain.key) cabundle=$(cat /tmp/le_quill/chain.pem)"
```

The current cert expires **2026-10-31** — re-run before then.

---

## 5. DNS

The domain is delegated to external nameservers (ns1/ns2.stackryze.com), so
records are managed at that provider, **not** in cPanel:

```
quill.nx.kg.      A    192.64.117.115
www.quill.nx.kg   A    192.64.117.115
```

(The vhost IP comes from cPanel → Domain Information, or
`DomainInfo/domains_data` via UAPI.)

---

## 6. Deployment Workflow

### 6.1 Full Deploy

From the project root:

```bash
./deploy.sh -r
```

This runs the complete pipeline:

```
1. rsync  →  Upload code to ~/quill.nx.kg/ (excludes .env, .htaccess, f/, tmp/, logs/)
2. pip    →  Install requirements.txt into ~/virtualenv/quill.nx.kg/3.13/
3. schema →  Apply schema.sql + migrate_existing.sql to remote MySQL
4. seed   →  Seed .env values into config_vars DB table (secrets Fernet-encrypted)
5. env    →  Set MYSQL_* + FLASK_SECRET_KEY in CloudLinux Python Selector
6. restart → touch ~/quill.nx.kg/tmp/restart.txt (Passenger reloads)
7. health →  curl https://quill.nx.kg/f/api/health
```

### 6.2 Dry Run

Preview what would be uploaded without making changes:

```bash
./deploy.sh --dry-run
```

### 6.3 Deploy Without Restart

Upload code, install deps, apply schema, seed config — but don't restart:

```bash
./deploy.sh
```

Useful when you want to batch multiple changes and restart later.

### 6.4 Manual Steps After Deploy

If this is the **first deployment** to a fresh server:

1. **Create the database and user** (see section 3.1)
2. **Configure the CloudLinux Python Selector** in cPanel → Setup Python App
   - Select Python 3.13
   - Set app root to `quill.nx.kg`
   - Set startup file to `passenger_wsgi.py`
   - Set app URL to `/f`
3. **Add the keep-alive cron** (see section 4.3)
4. **Issue and install TLS certificate** (see section 4.4)
5. **Run `./deploy.sh -r`**

### 6.5 Configuration Changes Without Redeploy

To change a config value without a full redeploy, use the web dashboard
(Settings tab) or update the database directly:

```bash
# Update a plain config var
python3 scripts/remote_db.py query "INSERT INTO config_vars (var_key, var_value) VALUES ('web_pin', '123456') ON DUPLICATE KEY UPDATE var_value='123456'"

# Then restart Passenger
ssh jnc "touch ~/quill.nx.kg/tmp/restart.txt"
```

For secrets, use the web dashboard which handles Fernet encryption
automatically.

---

## 7. cTrader Token Refresh

cTrader OAuth tokens expire after ~30 days. The `ctrader_refresh.py` script
fetches encrypted tokens from the forwarder, refreshes them via the cTrader
OAuth endpoint, and pushes them back.

### 7.1 Manual Refresh

```bash
# Check which tokens need refresh (dry run)
python3 scripts/ctrader_refresh.py --dry-run

# Refresh tokens expiring within 1 hour (default)
python3 scripts/ctrader_refresh.py

# Refresh all tokens regardless of expiry
python3 scripts/ctrader_refresh.py --threshold 99999999
```

### 7.2 Automated Cron

Add to crontab on your local machine or a monitoring server:

```bash
# Refresh every 30 minutes
*/30 * * * * cd /path/to/project && python3 scripts/ctrader_refresh.py >> logs/refresh.log 2>&1
```

### 7.3 Decrypt and Inspect Tokens

```bash
# Print decrypted tokens (truncated for safety)
python3 scripts/ctrader_consumer.py

# Save full decrypted tokens to a file
python3 scripts/ctrader_consumer.py --output tokens.json
```

---

## 8. Scripts Reference

### scripts/common.py

Shared helpers used by all other scripts:

| Helper | Purpose |
|--------|---------|
| `load_env(path)` | Parse a `.env` file into a `dict[str, str]` |
| `ssh_run(host, *args)` | Run a command over SSH, return `CompletedProcess` |
| `ssh_upload(host, path, data, mode)` | Upload bytes to a remote path via SSH stdin |
| `ssh_check(host, timeout)` | Verify SSH connectivity or raise `SystemExit` |
| `RemoteDB` | Query cPanel MariaDB over SSH (manages `.my.cnf` temp files) |
| `ForwarderClient` | HTTP client with Flask session/CSRF management |

### Scripts Called by deploy.sh

| Script | When | What it does |
|--------|------|--------------|
| `selector_env.py` | Step 5 | Reads `.env`, emits JSON of MYSQL_* + FLASK_SECRET_KEY for `cloudlinux-selector set` |
| `apply_schema.py` | Step 3 | Uploads `.my.cnf` + SQL to server, runs `mysql` over SSH |
| `seed_config.py` | Step 4 | Uploads `.env` to temp file, runs Python script on server that imports `db` + `config` and seeds `config_vars` |

### Standalone Scripts

| Script | Usage |
|--------|-------|
| `remote_db.py` | `python3 scripts/remote_db.py <command> [options]` |
| `cpanel_mysql.py` | `python3 scripts/cpanel_mysql.py <command> [options]` |
| `ctrader_refresh.py` | `python3 scripts/ctrader_refresh.py [--dry-run] [--threshold N]` |
| `ctrader_consumer.py` | `python3 scripts/ctrader_consumer.py [--output file.json]` |
| `le_issue_quill.py` | `python3 scripts/le_issue_quill.py` |

---

## 9. Troubleshooting

### App not responding

```bash
# Check health
curl -sk https://quill.nx.kg/f/api/health

# Check if Passenger is running
ssh jnc "ps aux | grep passenger"

# Restart manually
ssh jnc "touch ~/quill.nx.kg/tmp/restart.txt"

# Check app logs
ssh jnc "tail -50 ~/quill.nx.kg/logs/app.log"
```

### Database connection errors

```bash
# Verify credentials
python3 scripts/remote_db.py tables

# Check config vars
python3 scripts/remote_db.py config

# Verify selector env vars
ssh jnc "cloudlinux-selector get --json --interpreter python --user wovewbge"
```

### Telegram listener not running

```bash
# Check if the worker is alive (keep-alive cron may have stopped)
curl -sk https://quill.nx.kg/f/api/health | python3 -m json.tool

# Restart the app
ssh jnc "touch ~/quill.nx.kg/tmp/restart.txt"
```

### Config not updating after deploy

```bash
# Re-seed config from .env
python3 scripts/seed_config.py --env .env --ssh-host jnc

# Verify
python3 scripts/remote_db.py config

# Restart
ssh jnc "touch ~/quill.nx.kg/tmp/restart.txt"
```

### TLS certificate expired

```bash
# Issue new certificate
python3 scripts/le_issue_quill.py

# Install via cPanel → SSL/TLS, or copy files and use UAPI
ssh jnc "cp /tmp/le_quill/fullchain.pem ~/ssl/ && cp /tmp/le_quill/domain.key ~/ssl/"
```

---

## 10. Security Notes

- The `.env` file is **never synced** to the server (excluded in rsync).
- Secret config values are **Fernet-encrypted** at rest in the database.
- The Fernet key is derived from `FLASK_SECRET_KEY` (SHA-256 → base64url).
- SSH commands use `BatchMode=yes` to prevent interactive prompts.
- Database credentials are passed via temp `.my.cnf` files (not shell args).
- Temp files are cleaned up after use (`/tmp/seed_config.env`, `/tmp/remote_db.cnf`, etc.).
- `remote_db.py config` masks secret values by default; use `config-raw` for full access.
- Rotating `CTRADER_TOKEN_SECRET` or `FLASK_SECRET_KEY` invalidates all encrypted data.