Rethinking the foundations of the transformer
How much of what a frontier model costs is actually necessary. Nothing published yet.
Cadmus: a tokenizer that reads Europe in its own words
A byte pair encoding tokenizer covering 60 languages. On the 27 European languages it leads every open-source tokenizer we benchmarked, and it still leads on English.