AI RESEARCH
Mitigating Bias in Locally Constrained Decoding via Tractable Proposals
arXiv CS.CL
•
ArXi:2606.01926v1 Announce Type: new Generations from large language models often fail to conform to desired constraints such as JSON schema. Existing locally constrained decoding (LCD) approaches enforce constraints by myopically masking out next tokens, resulting in biased sampling and degradation in performance. Recent work uses sequential Monte Carlo (SMC) methods to mitigate such biases, but designing effective proposal distributions or potential functions remains a key challenge.