Quoting Andrew Kelley
📰 Simon Willison's Blog
Detecting LLM-assisted code submissions by identifying unique error patterns and 'digital smells' of agentic coders
Action Steps
- Analyze code submissions for characteristic LLM hallucinations
- Compare human-generated code with LLM-generated code to identify differences in error patterns
- Look for 'digital smells' in code submissions that may indicate agentic coding background
- Test code submissions for unusual dependencies or imports that could indicate LLM assistance
- Apply code review best practices to detect and prevent LLM-assisted code submissions
Who Needs to Know This
Developers and code reviewers can benefit from understanding how to identify LLM-generated code to improve the quality of code submissions and prevent potential security risks
Key Insight
💡 LLM-generated code can be identified by its unique error patterns and 'digital smells'
Share This
🚨 Detecting LLM-assisted code submissions just got easier! 🚨
Key Takeaways
Detecting LLM-assisted code submissions by identifying unique error patterns and 'digital smells' of agentic coders
Full Article
It's a common misconception that we can't tell who is using LLM and who is not. I'm sure we didn't catch 100% of LLM-assisted PRs over the past few months, but the kind of mistakes humans make are fundamentally different than LLM hallucinations, making them easy to spot. Furthermore, people who come from the world of agentic coding have a certain digital smell that is not obvious to them but is obvi
DeepCamp AI