AI RESEARCH
The Cognitive Categorical Transformer: Category-Theoretic Inductive Biases for Language Modeling
arXiv CS.AI
•
ArXi:2605.28864v1 Announce Type: new The Cognitive Categorical Transformer (CCT) is a 306M-parameter architecture that augments a pretrained GPT-2 Small backbone with cognitively grounded components derived from category theory and several inspirations from cognitive science. Under a matched-step protocol (215,000 optimizer steps, matched data, matched optimizer and schedule) on WikiText-103, CCT reaches 21.27 validation perplexity, compared with 24.19 for an identically fine-tuned GPT-2 Small baseline.