Recoleta
GitHub
Source note

Does operational privacy guidance change code generated by Qwen?

Authors
sushaantmujoo
Source
Hacker News
Published
Collection
Software Intelligence
Code IntelligenceAutomated Software ProductionPrivacy By DesignPrompt EngineeringSoftware Testing
Open GitHubMarkdown source

Summary

This pilot tests whether different privacy prompts change privacy-related implementation choices in Qwen-generated FastAPI code. The W3C + Operational condition produced the strongest observed privacy features, but the small sample and lack of runtime testing limit the conclusions.

Problem

  • Code-generation models can describe privacy principles without consistently implementing backend consent validation, data minimization, restricted disclosure, or executable privacy tests.
  • This matters because generated applications may process sensitive family health data, where privacy requirements must appear in APIs, schemas, storage, logging, disclosure rules, and tests.

Approach

  • Qwen generated the same FastAPI application across four prompt conditions: Blind, Basic, W3C, and W3C + Operational.
  • The study compared generated applications, APIs, models, and tests using file extraction, syntax checks, static privacy indicators, and arm-label-blinded semantic review.
  • The W3C + Operational prompt added concrete examples, field-level rules, schemas, API expectations, and testable implementation requirements to W3C-derived principles.
  • External behavioral tests are planned; generated tests were treated as model outputs rather than independent verification.

Results

  • No quantitative metrics, dataset sizes, or baseline scores are provided in the excerpt; the sample contains a small number of runs per condition.
  • W3C + Operational produced more privacy and consent tests, including negative tests for invalid, missing, revoked, expired, or mismatched consent.
  • It produced clearer API responsibilities, richer consent records and data models, and more explicit separation of received, stored, logged, and disclosed fields.
  • It also specified outbound field restrictions and represented purpose, recipient, category, and consent status more explicitly.
  • The apparent advantage may reflect examples, specificity, prompt length, required tests, or their combination; static and semantic analysis cannot establish runtime privacy behavior or prove causality.

Link