按年被提到
20222022 · 1 次2026
AI 安全、伦理与社会影响›提示注入与对齐风险·arXiv:2208.09727 · cs.CR

Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants

Gustavo Sandoval, Hammond Pearce, Teo Nys, Ramesh Karri, Siddharth Garg, Brendan Dolan-Gavitt · 2022
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2022-12-26 · 最近 2023-03-01
2不同的人
2活跃月份
2讨论串
81被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Large Language Models (LLMs) such as OpenAI Codex are increasingly being used as AI-based coding assistants. Understanding the impact of these tools on developers' code is paramount, especially as recent work showed that LLMs may suggest cybersecurity vulnerabilities. We conduct a security-driven user study (N=58) to assess code written by student programmers when assisted by LLMs. Given the potential severity of low-level bugs as well as their relative frequency in real-world projects, we tasked participants with implementing a singly-linked 'shopping list' structure in C. Our results indicate that the security impact in this setting (low-level C with pointer and array manipulations) is small: AI-assisted users produce critical security bugs at a rate no greater than 10% more than the control, indicating the use of LLMs does not introduce new security risks.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论moyix · 2022-12

“This is a really nice study! It is very cool that they were able to get professional programmers to participate, this is something that is really hard to set up as an academic team. And yes, 47 participants is a small number, but apparently sufficient in this case to detect the effect (as evidenced by the p-values). It also lines up nicely with work we di…”

Study finds AI assistants help developers produce code likely to be buggy原评论 ↗
被提交为 story · 1 次