RoPE Visualizer
Rotary embeddings turn a token's position into a rotation, one angle per dimension pair. YaRN stretches those angles so a model trained on a short context can read a long one. Move the scaling factor to see which frequencies it leaves alone and which it bends.
Configuration
Everything below recomputes as you change these.
What that buys you
The wavelength range is the span of positions the embedding can still tell apart.
Frequency analysis
Inverse frequency per dimension pair, before and after scaling.
Wavelength comparison
How far a position can travel before the angle repeats.
Frequency scaling ratio
Scaled over original, per dimension. Flat means untouched.
YaRN ramp
The interpolation that decides which dimensions get stretched.
Embeddings heatmap
Position against dimension, coloured by the rotation applied.
About RoPE and YaRN
Rotary Position Embedding encodes position by rotating query and key vectors, with a rotation angle that depends on both the position and the dimension pair.
YaRN extends the usable context by scaling frequency components unevenly: high-frequency dimensions, which carry local ordering, are left alone, while low-frequency ones are interpolated so distant positions stay distinguishable.
Why it matters: the model keeps its behaviour on short sequences while becoming usable on ones far longer than it was trained on.