Efficient numeracy in language models through single-token number embeddings

ArXi:2510.06824v2 Announce Type: replace To drive progress in science and engineering, large language models (LLMs) must be able to process large amounts of numerical data and solve long calculations efficiently. This is currently only possible through the use of external tools or extensive reasoning chains, either weakening the numerical representations of LLMs or limiting the length of problems they can solve.