ModelsMistral
Mistral opened a preview of Large 4, a trillion-parameter model with weights promised
1.05Tparameters, with open weights due by the end of October
Mistral opened a public preview of Large 4 through its Studio API, at $1.36 per million input tokens. The model is 1.05 trillion parameters in a mixture of experts, with a 1M token context and a 1.6 billion parameter vision encoder. Mistral says downloadable weights follow by the end of October, and the licence has not been published. Sources disagree on how many parameters are active per token: early coverage said 49 billion, the documentation now says 52 billion. Nothing is downloadable yet, and at four-bit precision the weights would be roughly 500GB.