@aps6992

#NLProc Practitioner/Research Engineer @allen_ai and @SemanticScholar, Mumbaikar - Seattleite, Pop culture buff, Foodie through and through

 Seattle, WA
Joined July 2022
  • Tweets2
  • Following18
  • Followers28
Amanpreet Singh retweeted
Ai2
@allen_ai27 Nov 2023
Exciting news from @SemanticScholar! Introducing SPECTER2, the adaptable upgrade to SPECTER. Learn about its advanced features and SciRepEval, the new benchmark for scientific document embeddings. Check it out: blog.allenai.org/specter2-ad… #EMNLP2023
2
29
2
136
43,365
Training and inference with language models is expensive. ♻️Embedding recycling♻️can preserve SOTA performance while improving efficiency. Preprint with Amanpreet Singh, @soldni, Mike D’Arcy, @armancohan, and @_DougDowney: arxiv.org/abs/2207.04993, github.com/allenai/Embedding…
Pictorial representation of embedding recycling (figure 1 from our manuscript). The image is divided into two parts. At the top, a stack of documents is sent through a transformer model to perform a task. Output of the intermediate layer is shown to get cached into a database in the bottom part of the image. Then, when running a transformer model on a new task (bottom right), intermediate representations are queried out of the embedding store.

ALT Pictorial representation of embedding recycling (figure 1 from our manuscript). The image is divided into two parts. At the top, a stack of documents is sent through a transformer model to perform a task. Output of the intermediate layer is shown to get cached into a database in the bottom part of the image. Then, when running a transformer model on a new task (bottom right), intermediate representations are queried out of the embedding store.

6
15
2
45
Load more