November 2, 2023Breakthrough1 min read

First shot in code: who said open source couldn't?

DeepSeek-Coder shipped — four sizes from 1.3B to 33B, all open-sourced from day one. The models were trained on 2 trillion tokens, mixing 87% code and 13% natural language, covering more than 80 programming languages. The 33B variant beat the then-prominent CodeLlama-34B on multiple benchmarks. What shook developers more: this wasn't the side-project of a tech giant. It came from a team that had spun out four months earlier, built entirely for writing code. GitHub stars poured in overnight, HuggingFace rankings climbed, and programmers truly felt for the first time that the labor of love behind open-source code models could come from a company with no PR roadshow.

Sources