---
source: arxiv
url: https://arxiv.org/abs/2607.02337v1
published_at: '2026-07-02T15:43:21'
authors:
- Charlotte Brandebusemeyer
- Kerim Zunic
- Thomas Zimmermann
- Tobias Schimmer
- Bert Arnrich
topics:
- developer-experience
- github-copilot
- code-intelligence
- human-ai-interaction
- mixed-methods-study
relevance_score: 0.72
run_id: materialize-outputs
language_code: en
---

# Developers' Experience with Generative AI Beyond Productivity Assessment -- Insights from an Empirical Mixed-Methods Field Study

## Summary
This paper studies how 22 SAP developers use generative AI coding tools during real work, with attention to workload, satisfaction, and interaction style rather than productivity alone.

## Problem
- Prior studies mainly measure code quality or task speed, so they miss how developers experience GitHub Copilot and similar tools during daily work.
- Organizations need to know when GenAI helps developers and when it adds review work, cognitive load, or workflow friction.
- The paper focuses on interaction types, including in-code suggestions and chat prompts, because different tasks may need different tool behavior.

## Approach
- The study followed 22 SAP employees over 4 days, with controlled sessions on the first and last day and normal work between them.
- Controlled tasks covered 6 task types: coding, debugging, code documentation, unit tests, summarization, and brainstorming.
- The researchers combined questionnaires, NASA-TLX workload ratings, screen recordings, keyboard and mouse logs, and wristband physiology; this paper focuses on questionnaires and screen recordings.
- GitHub Copilot was the controlled-session tool, with GPT-4o as the default model and ask mode as the default chat mode.
- The analysis compares no Copilot use, in-code suggestions, chat prompts, and combined use within the same task.

## Results
- The excerpt gives no quantitative effect sizes for the paper's main findings, but it reports a 22-person professional field study and about 66 hours of screen recordings.
- Participants had an average of 5 to 10 years of IT/programming experience; the cohort included 12 software developers/engineers, 8 senior software developers/engineers, 1 senior quality specialist, and 1 principal software architect.
- Developers were generally satisfied with GenAI, especially for monotonous, repetitive, and structured tasks.
- The paper claims that in-code suggestions and chat prompts each improve task efficiency and reduce perceived workload when used separately.
- Combining in-code suggestions and chat prompts within one task reduced the benefits, according to the reported findings.
- For development-heavy tasks, perceived cognitive load came from the AI interaction, while perceived productivity depended on AI output quality.

## Link
- [https://arxiv.org/abs/2607.02337v1](https://arxiv.org/abs/2607.02337v1)