Skip to content

Running the worker

The worker is a Dramatiq consumer process attached to a single experiment. It executes the planning/coding/evaluation pipeline for jobs dispatched by the scheduler.

Start

Recommended usage with uv:

uv run loreley worker --processes 4

--processes N starts one Dramatiq master and N isolated worker processes, each with one worker thread and a PID-plus-random base repository path. N=1 keeps the direct single-process path. The master propagates shutdown signals and returns non-zero when a child fails. Make SCHEDULER_MAX_UNFINISHED_JOBS >= N if you want all processes to stay supplied; Loreley warns when the configured scheduler capacity is smaller.

Minimum required settings for a functional worker are:

  • EXPERIMENT_ID
  • MAPELITES_EXPERIMENT_ROOT_COMMIT
  • WORKER_REPO_REMOTE_URL
  • WORKER_EVALUATOR_PLUGIN

You also need database and Redis connectivity (DATABASE_URL, TASKS_REDIS_URL), and a planning/coding backend binary (defaults to the Kilocode CLI via kilo on PATH). WORKER_EVOLUTION_GLOBAL_GOAL defaults to a generic improvement objective, but you will usually want to override it with a repository-specific goal.

Job lease ownership is enabled by default. Tune it with:

  • WORKER_JOB_LEASE_TTL_SECONDS
  • WORKER_JOB_HEARTBEAT_INTERVAL_SECONDS

If MAPELITES_CODE_EMBEDDING_MODEL is not a local-hash variant, or trajectory summarization is enabled, preflight also requires either OPENAI_API_KEY / LORELEY_LLM_API_KEY, or dynamic auth via OPENAI_DYNAMIC_API_KEY_PROVIDER plus OPENAI_DYNAMIC_API_KEY_TTL_SECONDS.

If you are upgrading an older schema-version-5 database, migrate it before startup:

uv run loreley db migrate

With DB_AUTO_MIGRATE=true (the default), worker startup initializes empty databases and runs the same migration under a Postgres advisory lock before marker validation. With DB_AUTO_MIGRATE=false, worker preflight and startup require uv run loreley db migrate to have completed first. For schema inspection and validation, see Database schema commands.

Options

  • --no-preflight: skip preflight validation.
  • --preflight-timeout-seconds: network timeout used for DB/Redis connectivity checks.
  • --processes, -p: number of isolated worker processes (default 1).
  • --log-level: global option (pass before the subcommand) that overrides LOG_LEVEL for this invocation.

Queue naming

The worker consumes jobs from a single experiment-scoped queue derived from EXPERIMENT_ID. The queue name is not configurable and is derived as:

"loreley.evolution.{experiment_namespace}"

EXPERIMENT_ID can be a UUID or a short slug. Slugs are mapped to a stable UUID (uuid5) and the derived experiment_namespace is stable across processes.

Logs

Logs are written to:

  • logs/{experiment_namespace}/worker/worker-YYYYMMDD-HHMMSS-pid-{pid}.log

If a worker loses its lease or dies mid-job, the scheduler will eventually reclaim that RUNNING row. Use uv run loreley status and the Job lease recovery runbook to inspect the current state and recover stuck jobs.

Exit codes

  • 0: success (clean shutdown)
  • 1: configuration, startup, or preflight error