Native Hierarchical and Compositional Representations with Subspace Embeddings

ArXi:2508.16687v2 Announce Type: replace Traditional embeddings represent datapoints as vectors, which makes similarity easy to compute but limits how well they capture hierarchies and compositionality. We propose a fundamentally different approach: representing concepts as linear subspaces. By spanning multiple dimensions, subspaces can model broader concepts with higher-dimensional regions and nest specific concepts within them.