st-agent-worker/README.md

# st-agent-worker

Assigns Solidarity Tech agents to department members automatically.

When someone is given the Department Rep role in Solidarity Tech, an ST
automation calls this service. The service looks up that person's chapter
(campus) and department, finds every eligible member of the same
chapter/department pair, and assigns the rep as their agent.

## Rules

- Only members whose appointment type is Senate or Recall are assigned.
  Set `ELIGIBLE_APPOINTMENTS` to change this.
- Matching is on primary chapteri (UCLAFA, BFA, SCFA...) plus Department 1. 
  A rep at one campus is never assigned members at another.
- Reps are excluded from assignment. Organizer leads (OC members, organizing
  staff) assign their agents by hand in the ST web interface.
- **Fill-only**: a member who already has an agent is left alone and
  recorded in the `skips` table. No agent is ever reassigned automatically.
  See below for consequences of this.

### Known consequence of fill-only

Some departments currently have two reps. Whichever rep runs first takes 
the whole department; the second gets nobody. The reconciler logs a warning 
in both cases that can be seen in the logs and reconciled manually.

_Potential fix_ In the future, round-robin splitting could be implemented. `RepsFor` already returns all reps for a department sorted by ID, so adding it is a 
change to how the agent is picked, and wouldn't require a code restructuring.

## Architecture

Four components do the work:

- `server.go` — HTTP handler. Verifies the shared secret, reads the rep's
  ST user ID from the query string, looks up their chapter and department
  live, takes a per-department lock, returns 200, and runs the
  reconciliation in the background. The 200 is returned before any work
  happens, because ST times out and retries otherwise.
- `client.go` — Solidarity Tech API client. Holds the rate limiter and
  handles 429 responses. Nothing above it builds a URL.
- `store.go` — SQLite database handle. Four tables: `people` (cached roster),
  `runs` (locks and progress), `skips` (members left alone), `sync_state` (last
   sync timestamp).
- `reconcile.go` — the loop. Reads department members from the cache,
  checks each one's current agent live against the API, assigns or skips.

Roster data is cached because it changes more slowly, and API rate limits would make a read/write of the whole roster take two hours. Instead, agent assignments are read live on every check, so an assignment made by hand in the dashboard
during a run can be respected right away. See _Rate limits_ below.

### Locking

`runs` has a partial unique index on `(chapter_id, department)` where
status is `running`. A second webhook for the same department fails the
insert and exits. On startup, any row still marked `running` belongs to a
dead process and is marked `crashed`.

### Rate limits

The ST API allows 60 requests per 30 seconds per API key, which averages
2 per second and permits bursts. The client is configured to match this.

## Configuration

| Variable | Meaning |
| --- | --- |
| `ST_API_KEY` | Solidarity Tech API key |
| `WEBHOOK_SECRET` | Shared secret, passed as `?s=` on the webhook URL |
| `DB_PATH` | SQLite file. `/data/worker.db` in the container |
| `DRY_RUN` | Any non-empty value logs writes instead of sending them |
| `ELIGIBLE_APPOINTMENTS` | Comma-separated labels, defaults to `senate,recall` |

The ST webhook action has no support for custom headers, so the secret
travels in the query string. Do not log raw query strings.

## Endpoints

- `POST /webhook?id={user_id}&s={secret}` — triggered by the ST automation
  when someone is assigned the Department Rep role.
- `POST /debug/sync?s={secret}` — forces a roster sync. Useful after a bulk
  import rather than waiting for the hourly update.
- `GET /healthz` — returns 200 once the server is listening.

## Health checks

The startup sync runs before the server binds, and a cold start with an
empty volume pulls the full roster, which takes about 90 seconds. Give the
container a `start_period` of at least 120 seconds or it will be marked
unhealthy before it comes up.

After deploying, check:

    curl https://$HOST/healthz

and confirm the log shows a completed sync. Then:

    sqlite3 /data/worker.db "
    SELECT COUNT(*) total, SUM(is_eligible) eligible, SUM(is_rep) reps
      FROM people;"

Should give roughly 17,500 total, 14,700 eligible.

Log lines to watch for:

- `WARNING: run N: assigned nobody` — every member already had an agent.
  Expected for the second rep in a co-repped department, unexpected
  otherwise.
- `N eligible users synced with no department` — those members cannot be
  assigned to anyone. These are likely Senate or Recall duplicates, or
  people we don't have information for.

## Initial backfill

The webhook only fires when a rep is newly assigned the role, so reps who
already had it when this was deployed never trigger it. To assign their
departments, call the endpoint for each:

    sqlite3 worker.db "SELECT id FROM people WHERE is_rep = 1;" | while read id; do
      curl -s -o /dev/null -X POST "https://$HOST/webhook?id=$id&s=$WEBHOOK_SECRET"
      sleep 1
    done

Run it with `DRY_RUN` set first and count the assignments in the log.

## Not built yet

- A nightly sweep to reassign reps. Without it, a rep who steps down keeps 
  their assignments and their replacement inherits nobody. Every member added
  after their department's rep signed up also goes unassigned until
  something triggers that department again. **FIRST PRIORITY**
- Schema migrations. The schema is created with `CREATE TABLE IF NOT
  EXISTS`, which does not add columns to an existing table. Changing the
  schema currently means deleting the database and re-syncing.