optimization

Context Language Models Cut Compute by Managing Their Own Context

Context language models trim input during inference, reducing softmax operations by up to 30 % while keeping accuracy near baseline.

Oct 12, 20263 min readvia InfoQ