AI RESEARCH
Typhoon: Towards an Effective Task-Specific Masking Strategy for Pre-trained Language Models
arXiv CS.CL
•
ArXi:2303.15619v2 Announce Type: replace The choice of \emph{which} tokens to mask is a central, under-examined design decision in masked language modeling (MLM). Standard pre