LLM의 지난 6개월, 5분 압축

생각 덩어리

November 2025 변곡점 — 왕좌가 한 달에 다섯 번 바뀐 달

Six months is a pretty convenient time period to cover, because it captures what I’ve been calling the November 2025 inflection point. November was a critical month in LLMs, especially for coding.

For one thing, the supposedly “best” model (depending mostly on vibes) changed hands five times between the three big providers.

In November it was overtaken by GPT-5.1, then Gemini 3, then GPT-5.1 Codex Max, and then Anthropic took the crown back again with Claude Opus 4.5.

I think Gemini 3 drew the best pelican out of this lot, but pelicans aren’t everything. Most practitioners will agree that Opus 4.5 held the crown for the next couple of months.

펠리컨 자전거 벤치마크 — 훈련될 리 없는 과제

As always, I’m using my Generate an SVG of a pelican riding a bicycle test to help illustrate the differences between the models.

Why this test? Because pelicans are hard to draw, bicycles are hard to draw, pelicans can’t ride bicycles ... and there’s zero chance any AI lab would train a model for such a ridiculous task.

진짜 뉴스는 코딩 에이전트 — often-work에서 mostly-work로

It took a little while for this to become clear, but the real news from November was that the coding agents got good.

OpenAI and Anthropic had spent most of 2025 running Reinforcement Learning from Verifiable Rewards to increase the quality of code written by their models, especially when paired up with their Codex and Claude Code agent harnesses.

In November the results of this work became apparent. Coding agents went from often-work to mostly-work, crossing a quality barrier where you could use them as a daily-driver to get real work done, without needing to spend most of your time fixing their stupid mistakes.

홀리데이 LLM psychosis — 야심작들의 조용한 은퇴

Over the holiday period, from December to January, a whole lot of us took advantage of the break to have a poke at these new models and coding agents and see what they could do.

They could do a lot! Some of us got a little bit over-excited. I had my own short-lived bout of a form of LLM psychosis as I started spinning up wildly ambitious projects to see how far I could push them.

That playground demo shows JavaScript code run using my micro-javascript library, in Python, running inside Pyodide, running in WebAssembly, running in JavaScript, running in a browser!

It’s pretty cool! But did anyone out there need a buggy, slow, insecure half-baked implementation of JavaScript in Python?

They did not. I have quite a few other projects from that holiday period that I have since quietly retired!

Warelay에서 OpenClaw로 — 석 달 만의 세계 정복

Also in November, this happened—the first commit to an obscure (back then) repo called “Warelay” by some guy called Pete.

In December and January it had gone through quite a few name changes ... and by February it was taking the world by storm under its final name, OpenClaw.

The amount of attention it got is pretty astonishing for a project that was less than three months old.

OpenClaw is a “personal AI assistant”, and we actually got a generic term for these, based on NanoClaw and ZeroClaw and suchlike... they’re called Claws.

Mac Mini 수족관과 Doc Ock — Claws의 은유

Mac Minis started to sell out around Silicon Valley, because people were buying them to run their Claws.

Drew Breunig joked to me that this is because they’re the new digital pets, and a Mac Mini is the perfect aquarium for your Claw.

My favourite metaphor for Claws is Alfred Molina’s Doc Ock in the 2004 movie Spider-Man 2. His claws were powered by AI, and were perfectly safe provided nothing damaged his inhibitor chip... after which they turned evil and took over.

펠리컨의 응답 — Gemini 3.1 Pro와 Jeff Dean의 동물 퍼레이드

Also in February: Gemini 3.1 Pro came out, and drew me a really good pelican riding a bicycle. Look at this! It’s even got a fish in its basket.

And then Google’s Jeff Dean tweeted this video of an animated pelican riding a bicycle, plus a frog on a penny-farthing and a giraffe driving a tiny car and an ostrich on roller skates and a turtle kickflipping a skateboard and a dachshund driving a stretch limousine.

So maybe the AI labs have been paying attention after all!

오픈 웨이트의 약진 — Gemma 4·GLM-5.1·노트북 위의 Qwen

Google released the Gemma 4 series of models, which are the most capable open weight models I’ve seen from a US company.

Also last month, Chinese AI lab GLM came out with GLM-5.1—an open weight 1.5TB monster! This is a very effective model... if you can afford the hardware to run it.

Charles on Bluesky suggested I try it with a North Virginia Opossum on an E-scooter

And it did this! I’ve tried this on other models and they don’t even come close. “Cruising the commonwealth since dusk” is perfect. It’s animated too.

The other neat Chinese open weight models in April came from Qwen. Qwen3.6-35B-A3B on my laptop drew me a better pelican than Claude Opus 4.7. That’s a 20.9GB open weights model that runs on my laptop!

여섯 달의 두 테마 — 에이전트의 성숙, 노트북 모델의 반란

(I think this mainly demonstrates that the pelican on the bicycle has firmly exceeded its limits as a useful benchmark.)

So those were the two main themes of the past six months. The coding agents got really good... and the laptop-available models, while a lot weaker than the frontier, have started wildly outperforming expectations.