TemporalWiki, updating language models 'on the fly'
How can we update LMs *whenever* we want on up-to-date knowledge with *minimal* cost? Furthermore, how can we evaluate if each update was successful or not? We introduce ✨ TemporalWiki ✨, a lifelong benchmark providing the solution.
— Joel Jang (@jang_yoel) May 3, 2022
📝 https://t.co/vkLolTRHhi
🧵 1/8 pic.twitter.com/NblID1Tofm
Abstract of linked article:
Language Models (LMs) become outdated as the world changes; they often fail to per- form tasks requiring recent factual information which was absent or different during training, a phenomenon called temporal misalignment. This is especially a challenging problem because ...