Hierarchical Concept Geometry in Language Models Emerges from Word Co-occurrence

ArXi:2605.23821v1 Announce Type: cross We propose a distributional theory of how hypernymy -- the ``is-a'' relation between general and specific concepts -- is encoded geometrically in language representations. Starting from the empirically verified assumption that words closer on the WordNet hypernym graph co-occur often, we characterize theoretically the spectrum of the resulting embedding Gram matrix of word2vec embeddings.