I just shipped my first open-source project — a small thing, an animated row-deletion pattern for web apps. Nothing world-changing. But building it with an AI coding agent sitting next to me the whole time made me want to understand something I’d been taking for granted: where did these tools actually come from? What are they trained on? Who built them, and why? This is my attempt to trace that history — partly for my own understanding, partly because I think more builders should know the story of the tools they now depend on every day.

Before the agents: a foundation laid in 2017

Almost every AI coding tool in use today — Copilot, Codex, Claude, Gemini — traces back to one research paper: “Attention Is All You Need,” published by a team at Google Brain in August 2017. It introduced the Transformer architecture, which replaced the older, slower recurrent neural networks with a mechanism called “attention” — letting a model weigh the relationships between every word (or token) in a sequence simultaneously, rather than processing text one piece at a time.

This sounds abstract, but the practical effect was enormous: Transformers could be trained in parallel, at far larger scale, on far more data, than anything before them. Nearly every large language model since — GPT, BERT, Claude, Codex — is built on this same architectural foundation. The paper wasn’t about code at all; it was about machine translation. But the architecture turned out to generalize to almost any sequence of symbols, including programming languages.

The raw material: where does the “code knowledge” come from?

Before any model can write code, it has to learn from code that already exists. And overwhelmingly, that code comes from one place: public repositories on GitHub, supplemented by GitLab and a handful of other sources.

The most direct example is a dataset called The Stack, built by the BigCode Project (a collaboration between Hugging Face and ServiceNow). The Stack v1 contains over 6TB of permissively-licensed source code across 358 programming languages; The Stack v2 scaled that to over 3 billion files across 600+ languages. It exists specifically to train open, transparent code-generation models — and, notably, it lets developers request their code be removed from it, an acknowledgment of the ongoing debate around consent and public code being used this way.

OpenAI’s own Codex paper (2021) was explicit about its sourcing: the model was trained on 159 gigabytes of Python code pulled from 54 million public GitHub repositories, plus substantial amounts of JavaScript, TypeScript, Go, Ruby, C++, C#, Java, PHP, Swift, and more.

This is the uncomfortable-but-important fact at the center of this whole story: the reason today’s AI can write code fluently is that millions of developers, over nearly two decades, wrote code in public and explained it in READMEs, comments, and commit messages — not knowing, in most cases, that it would eventually become training material for machine intelligence. GitHub, with over 630 million repositories today (395 million of them public), is functionally the largest library of “how humans write and explain software” that has ever existed. GitLab contributes to this too, at meaningfully smaller scale, since GitHub remains the dominant home for public open-source work.

From autocomplete to Codex (2021)

The first real bridge between “a language model” and “a tool a developer actually uses” was OpenAI Codex, published as a research paper in July 2021 and released as an API that August. Codex was essentially GPT-3, fine-tuned specifically on code. It became the engine behind GitHub Copilot, which launched in technical preview in June 2021 and to the public in October 2021 — the first mainstream “AI pair programmer,” living directly inside your editor, suggesting completions as you typed.

This was a genuinely new category of tool. Before Codex/Copilot, code intelligence meant autocomplete based on syntax rules and static analysis — useful, but fundamentally mechanical. Copilot could suggest an entire function based on a comment describing what it should do. It felt, to a lot of developers in 2021 and 2022, like a genuine step-change.

By March 2023, the original Codex model itself was quietly retired — superseded by GPT-3.5 and GPT-4, which had absorbed vastly more code training data at greater scale and no longer needed a separately fine-tuned “code model” to be good at code.

A different lineage: why Anthropic exists at all

Claude comes from a different origin story entirely — one rooted less in “let’s build the best product” and more in a disagreement about how fast and how carefully AI should be developed.

Anthropic was founded on January 26, 2021, by siblings Dario Amodei and Daniela Amodei, along with five other researchers — all of whom had left OpenAI a few months earlier. Dario had been OpenAI’s VP of Research; Daniela had been VP of Safety and Policy. The group’s departure and the founding of Anthropic were driven by a belief that AI capability was scaling faster than the industry’s understanding of how to keep it safe and steerable. Anthropic was built, from day one, as an “AI safety company” — its research agenda (interpretability, alignment, constitutional AI) has always been positioned as core to the product, not an afterthought bolted onto a capabilities race.

Claude, Anthropic’s model family, launched publicly not long after, and the company has since scaled Claude into a full family of models (the naming conventions have shifted over time — Claude 3, Claude 4, and now the Claude 5 family, including the very model that helped write and structure this article).

The agentic turn: from suggesting code to doing the work

For a few years, both lineages — Copilot/Codex and Claude — mostly did the same basic thing: suggest code, complete a line, answer a question in a chat window. The human stayed in the driver’s seat for everything else: running commands, navigating files, testing, committing, deploying.

That changed with what’s now called agentic coding. Claude Code, Anthropic’s terminal-based coding agent, is a useful case study in how fast this shift happened. It reportedly started as one engineer’s side project during his first month at Anthropic, in September 2024. It launched as a limited research preview on February 24, 2025. Less than a year later, it had reportedly passed a $1 billion annualized revenue run rate — one of the fastest revenue ramps in software history.

What changed technically wasn’t just a bigger model — it was giving the model tools: the ability to read a whole codebase, run shell commands, execute tests, edit multiple files, and iterate on failures, all in a loop, with much less moment-to-moment human steering. The AI stopped being an autocomplete engine and started being something closer to a junior engineer you could hand a task to.

OpenAI moved the same direction with Codex — not the 2021 model anymore, but a 2025-era agentic coding product carrying the same name, repositioned entirely around autonomous task completion rather than line-by-line suggestion.

The lines blur: competitors become co-dependents

One detail that surprised me researching this: GitHub — Microsoft’s platform, and historically OpenAI’s closest coding-AI partner — now also ships Claude. Starting in October 2024, GitHub Copilot added Claude 3.5 Sonnet as a selectable model alongside OpenAI’s own, explicitly framed as “developer choice.” By February 2026, GitHub had Claude and Codex both available as coding agents inside Copilot, and in November 2025, Microsoft committed to continued Claude access across its entire Copilot family.

In other words: the platform most associated with OpenAI’s Codex now also distributes its safety-focused rival’s models, because customers wanted the choice. The AI coding landscape in 2026 isn’t really “Team OpenAI vs. Team Anthropic” — it’s a layered stack where the same underlying platforms (GitHub, GitLab) host the training data, the code, and increasingly the AI tools themselves, often from multiple competing labs at once.

Why this history matters to me right now

I didn’t write this because I think anyone’s waiting for another “history of AI” article. I wrote it because I just spent a session shipping a small open-source library with an AI agent doing a large share of the actual work — writing the backend, catching a privacy mistake I made, walking me through GitLab and GitHub settings I’d never touched, even helping me submit a pull request to a 35,000-star community project.

That whole experience only exists because of the chain this article traces: a 2017 research paper about language translation, that turned into an architecture general enough to learn code, trained on the accumulated public work of millions of developers who never expected their GitHub commits to become raw material for a machine’s understanding — built by two different companies with two different philosophies about how fast to move, now converging on the same category of tool: an agent that doesn’t just suggest code, but does the work.

I’m not a machine learning researcher. I’m someone who builds things — in software, and now, apparently, in the open-source world too. But understanding the tools I use, not just using them, feels like the right instinct for whatever comes next in this industry. This is a start.

Sources referenced in this piece