Research idea

Repository context checks for coding-agent workflows

Day · 2026-07-13 · Software Intelligence

Repository context should be tested as an operational dependency. The practical changes are fault injection for retrieval, security-specific context gates before code completion, and explicit escalation when infrastructure evidence is missing or stale.

3 ideas

Fault injection for repository retrieval and summaries

Coding-agent maintainers need to know whether a patch survives bad repository context. ACQUIRE shows that targeted, evidence-backed repository answers improve issue resolution, while AgentCheck finds that agents often accept stale or corrupted tool output confidently. Repository retrieval therefore belongs in the fault model, alongside API and database tools.

Add replayable mutations for stale file summaries, omitted dependency edges, incorrect symbol locations, and instructions embedded in retrieved text. Record the first trajectory divergence, then test mitigations such as source citations, file-hash checks, and read-only confirmation against current code.

Run 50 resolved SWE-bench tasks with clean and mutated context. Measure Pass@1 loss, unsafe edits, and mitigation recovery. Use a pilot decision threshold: stop if mutations reduce Pass@1 by under 5 percentage points, or revise the checks if they recover under half of the failures they expose.

Security API context gate for AI-assisted pull requests

Application-security teams reviewing Java pull requests need a reliable check when developers use Copilot around SSL/TLS or OAuth APIs. In a 44-developer study, Copilot improved functional correctness without a significant gain in secure API usage, and only two participants raised security explicitly in their prompts. ACQUIRE suggests that focused questions can supply missing repository contracts before editing.

Trigger a short pre-completion questionnaire when a security API appears: which trust policy applies, where credentials originate, what validation path is required, and which repository configuration constrains the call. Supply answers with file and symbol citations, then run API-specific misuse checks before accepting the change. TerraRepair’s schema lookup and rescanning results support this combination of exact context and executable verification.

Pilot on 30 historical security-related pull requests using blinded review. Compare insecure API misuse, functional test passage, and review time. Use a pilot decision threshold: stop if misuse falls by less than 20% relative or median review time rises by more than 10%.

Freshness-aware escalation for Terraform repair

Cloud-security engineers need automated Terraform repair to stop when deployment evidence is absent or obsolete. TerraRepair attributes most escalations to missing deployment-specific context. AgentCheck shows that stale data remains difficult even after common mitigations, so a structurally valid provider response cannot be treated as current evidence.

Attach a receipt to every proposed repair with provider-schema version, dependency values, source commit, retrieval time, scanner version, and unresolved conflicts. Refuse automatic application when required fields are missing, hashes no longer match, or sources disagree. Replay captured repairs with stale schemas and changed dependency values to test whether the gate escalates before patching.

Start with 40 historical findings split between valid and deliberately stale context. Measure unsafe auto-approvals and unnecessary escalations. Use a pilot decision threshold: stop if the gate catches under 90% of stale-context cases, or revise it if more than 25% of valid cases are escalated.