AI RESEARCH
Autonomous LLM Agents & CTFs: A Second Look
arXiv CS.AI
•
ArXi:2605.21497v1 Announce Type: cross Large Language Model (LLM) agents are increasingly proposed to automate offensive security tasks, with recent studies reporting near human-level success rates in Capture-the-Flag (CTF) challenges. We here revisit these results, providing a second look at these claims. We engineer different agent architectures of increasing complexity and modularity on 30 web-based CTFs challenges spanning 14 vulnerability classes.