Skip to content

v0.8.4-alpha

This alpha patch release packages the baseline auto-retry work merged after v0.8.3-alpha.

Use it if campaign baseline evaluation can fail because of temporary evaluator, worker, timeout, service, or infrastructure errors and you want the scheduler to recover without an operator force-rerun.

Highlights

  • Campaign baseline bootstrap now retries retryable failed or degraded baseline rows after a five-minute cooldown. A temporary baseline evaluator failure no longer leaves the same baseline key stuck until an operator forces a rerun.
  • Contract and configuration failures remain non-retryable. Missing, non-finite, unconfigured, or direction-conflicting primary metrics are still reused until the campaign is fixed or explicitly force-rerun.
  • Retry attempts update the existing campaign_baselines row instead of creating duplicate rows. A later successful retry clears the failure fields and restores normal dispatch or scheduling.
  • Retry cooldowns use finished_at first, then fall back to updated_at and created_at for older or partially populated rows. Naive timestamps are treated as UTC before comparison.

Upgrade Notes

No database migration required

This release does not change the Loreley database schema or instance metadata version.

Baseline retry behavior

Retryable baseline failures now run again automatically after this cooldown:

300 seconds

The retryable failure kinds are:

  • baseline_evaluation_failed
  • evaluation_missing_result
  • evaluator_error
  • infrastructure_error
  • service_unavailable
  • timeout
  • worker_timeout

Failure kinds ending in _error are also treated as retryable unless they are listed as non-retryable.

The non-retryable failure kinds are:

  • primary_metric_direction_conflict
  • primary_metric_missing
  • primary_metric_non_finite
  • primary_metric_not_configured

When a retry starts, Loreley logs the baseline key, status, failure kind, and cooldown seconds through the scheduler.baselines logger.

Fixes

  • Retryable baseline failures no longer block campaign startup indefinitely after the cooldown has elapsed.
  • Failed and degraded baseline rows are still reused during the cooldown, so a transient failure does not trigger a rerun on every scheduler tick.
  • Repeated retry failures update the same row's failure kind, failure summary, and timestamps.
  • Rows without finished_at can still respect cooldowns through updated_at or created_at fallback timestamps.

Validation

  • Local release validation: 730 passed, 4 skipped in 42.49s.
  • Focused checks:
  • uv run pytest tests/scheduler/test_baseline_bootstrap.py -q
  • git diff --check.
  • Documentation build: uv run --with mkdocs-material mkdocs build --strict.
  • Build check: uv build.
  • Add cooldown-based auto-retry for retryable campaign baseline failures.
  • Keep primary-metric contract failures non-retryable unless an operator forces a rerun.
  • Reuse the existing baseline row across retries and avoid a database schema migration.