AI RESEARCH
Differential syntactic and semantic encoding in LLMs
arXiv CS.AI
•
ArXi:2601.04765v4 Announce Type: replace-cross We study how syntactic and semantic information is encoded in inner layer representations of Large Language Models (LLMs), focusing on the very large DeepSeek-V3. We find that, by averaging hidden-representation vectors of sentences sharing syntactic structure or meaning, we obtain vectors that capture a significant proportion of the syntactic and semantic information contained in the representations.