Yu Wang (@yusidwang) profile picture
Joined June 2026
Photo by Yu Wang (@yusidwang)
We’d like to introduce our paper on long-context positional encoding, centered on a simple principle:
2
2
1
12
28,245
Empirically, TAPA adapts to 32k context with continual pretraining on less than 0.25% of pretraining tokens, and extrapolates much better to unseen 64k length than RoPE-family baselines.
1
343