The Age of Async Agents — Cognition's Walden Yan & OpenInspect's Cole Murray

생각 덩어리

2024년의 가장 AGI-pilled한 베팅 — 코드 작성이 아니라 소프트웨어 공장

In retrospect, async agents were the most AGI pilled bet you could make in 2024 - the models weren’t good enough yet to vibecode, and people didn’t trust AI enough to let it rip, nobody (including early Cognition) was sure about the form factors.

The current Age of Async Agents points to a different future focused more on agent orchestration which drives end-to-end development.

Cursor is no longer primarily about writing code. It is about helping developers build the factory that creates their software. This factory is made up of fleets of agents that they interact with as teammates: providing initial direction, equipping them with the tools to work independently, and reviewing their work.

Context Engineering의 코이너 — prompt engineering의 업그레이드

Although I think there are many people who’d used the terms in various ways beforehand, but I did find that people, both internally and externally, enjoyed the upgrade from prompt engineering or model wrapping into maybe a more thoughtful way to build agents.

2025년 12월 인플렉션 — spec-to-PR, 그리고 Devin의 7배 램프

I think we saw a shift around the December timeframe of 2025, where the models Opus 4.5 and GPT 5.2, they reached a capability where we moved away from handholding the model and being able to actually more or less autonomously drive the model.

we could pretty much go from a specification to a completed pull request, assuming the spec was good enough, with very little friction. And that paradigm alone, I think, changed a lot of how we interact with agents, and opened this world where background agents became more practical.

it grew like 7X over, the last, I think it was, two months, three months, something like that. And then you see our engineering headcount growth. It’s, gone up by, 10% or something.

We were afraid To release this. So this is Devin commit percentages on all Devin repos, was 16% in January and now 80% in March.

OpenInspect의 기원 — PM의 Slack 세션이 엔지니어에게 안 보였다

a big issue they ran into was that the sessions that were launched were specific to whoever called it via Slack. And so if a PM was the one who invoked the session and they would then go to pass context to engineering can’t see the session.

And as Ramp released their blog post, I had a lot of the pieces for this already in place, and just thought it would be funny to, see what Claude could do just purely from the blog post.

looking around, I didn’t really see anything in the open source community that, met this type of system. I think there’s a lot that run, in localhost like Superset, Conductor, and many others. But nothing that was actually running in the cloud.

회색지대의 비즈니스 — $20/seat 대신 인프라와 adoption을 판다

One, I think that I don’t want to compete for, $20 a seat. I think that is just a really difficult business. I think it’s very easy to copy the main pieces of it.

You have money being made at the sandbox layer with Daytona, E2b, many other players. You have money being made at the model layer. And you sit in this weird in-between gray area where what are you actually selling?

when we just built it off of raw VMs from cloud providers like EC2, the boot up time was so slow ... It would just be out cold for like 10 minutes because that’s just how long these systems took. They were not built for this repeated down and up usage.

a big part of what we do at Cognition as well is to just make sure that your company learns and uses and adopts these coding agents.

Harness in the box vs out of the box — 시크릿이 상자 안으로 들어가는 순간

In a background agent systems, you have a decision to make of where the agent is actually going to run. This is typically described as the harness in the box or out of the box.

Because the agent is running in that box, unless you otherwise design it, all of your secrets need to go into that box as well. And given the nature of AI, it can be unpredictable, and you could very easily end up accidentally exfilling your secrets

we will have, quote-unquote, the brain of the agent running in some type of worker, control plane. That sandbox then is going to serve as the hands where the brain is basically operating and making tool calls into that environment to manipulate it.

in my opinion, it is the better architecture of the two. It’s just, you’re taking on a bit of complexity by doing that.

뇌와 기계의 분리 — 머신에는 가장 좁게 스코프된 시크릿만

I think a lot of that, what you mentioned, is why we actually from the start built Devin to what we called separate the brain from the machine. The other thing that this allows you to do is reuse any existing infrastructure you have for dev boxes

So only put the most scoped secrets on that machine, and then the brain is fully not accessible from the machine. So you don’t have to worry about messing with the, any of the most secure parts of the brain if the user is free to do whatever they want with the machine.

Repo setup — "Bob한테 가서 시크릿 받아와"는 에이전트에게 통하지 않는다

Let’s say you have a big repository that’s changing and updating a lot with changing dependencies. How do you make sure that the working environment of the agent actually stays up to date, has all the credentials it needs to, let’s say, run the app and test it

It’s been a perennial problem since the start of the company, of how do we help people get this set up? Because not everyone just has, working cloud environments working out of the box.

A lot of teams don’t really have great developer environment setups, if any. A lot of the times it’s, “Go talk to Bob and get the secrets,” and that obviously doesn’t work when the agent needs to actually set this up.

any environment that you’ve set up that is a good experience for your developer naturally lends itself to being easy to set up for the agent. And once you figure out that local developer story, you’ve more or less solved the agent in a sandbox, environment setup.

Docker로는 부족하다 — full VM, 그리고 Android를 위한 중첩 가상화

One thing is, Docker container’s not really a true security boundary, for one. But the other is, if you are running real applications, a lot of times those applications use Docker, and then you have to think about Docker in Docker, which is, really weird.

we realized that you actually needed, full VMs to be able to do these types of things. And especially nowadays where there’s actually value in running the application and clicking around and sending you screen recordings of these things.

Another thing that we’ve actually recently added support now for, it’s in beta, is doing Android development. To do that, we needed to support, I think, nested virtualization within our machines because the VM itself is like a, is a virtualized Firecracker instance, and then you had to then run another Android emulator inside.

Testing은 computer use보다 어렵다 — 버튼 좌표가 아니라 추론 문제

I think they actually over-index on the computer use part of it because computer use in my mind is the literal, okay, you want what button you want to click. Can you emit the right coordinates to go click that button?

imagine you make a change that spans the frontend and the backend, maybe, even some other like even more deeply nested service. To actually test that change, we have to reason through ... how do you first run these applications to orchestrate with each other with the right version of the code?

And in some cases, we found that you actually, no one frontier model can actually do this full end-to-end task itself. ... We’ve seen cases where we actually had to orchestrate different frontier models together to solve this problem together.

비디오 검증 — 코드를 안 봐도 되는 "I know it works"의 머지 순간

You, after the PR is done, you click testing approved, and then it sends you back a video. What I really like is that it labels ... what it’s testing.

this is one of those like, oh, few of the AGI moments, right? ‘cause Once I look at this, I wish I can just merge inside Of Slack instead of going to GitHub ‘cause I don’t need to see the code. I know it works.

There’s a surprisingly long tail of small details that ends up making a big difference for this end metric of like how fast do you actually merge the code in.

GitHub 위의 Devin — 셀프 리뷰 루프, 그리고 "네가 틀렸어"라고 반박하는 순간

We try to take that a step further and say, “Oh, what if we actually made sure you could interact ... with ... Devin directly on GitHub?” And so we made sure that you can comment on GitHub, and Devin would actually receive those comments and address them back.

so when Devin fixes his own comments, you might be scared that, oh, maybe I’ll infinite loop. But we’ve put a lot of work into making sure it doesn’t, both by making sure that the comments are high signal, but also that the agent is thoughtful about what comments it immediately goes and tries to fix, and what comments it’s like, “Wait a second, I think you’re wrong.”

Actually, that’s one of my favorite moments is when Devin tells me that I’m wrong, when I try to get it to do something different.

MCP만으로는 안 된다 — 결국 first-party 통합으로 돌아온다

You could give your agent a Slack MCP and okay, it can post messages back to you on Slack. But we actually use Devin like a coworker in Slack, and that’s how it’s been built from the ground up.

I would love a world where you could have something that’s more expressive than MCP. That, goes both ways, not just a set of tools, but a proper system that interacts back and lets it Have the right experience with all these interfaces.

when the MCP spec starts to get too complicated, it starts to lose its original promise of Being like a simple one-step connect. Now then we have to go figure out how to support all these different variations of things and It starts to look a lot like just building the first party integrations in a lot of these cases now.

If this is something that nearly every session is going through, it probably makes sense to own it so that you can make optimizations on top of it Versus just whatever is off the shelf.

Memory는 미해결 — 생성과 리트리벌, 두 번 어려운 문제

how I’m seeing it solved across my clients is primarily through skills. I find that skills can be a good gap within that or updating Claude MD, but I think memory as a whole is a pretty unsolved problem, and it is why I’ve been hesitant to add it. ... as a whole it’s a very difficult retrieval problem.

any time you remind Devin, “Wait, no, that’s not quite the way you’re supposed to use Git” Like, we actually want Devin to say, “Hey, do you want me to actually just remember this for the future?” ... ‘Cause I find that, 95%, I think, or some crazy stat like that of the memories that Devin has are all through these auto-generated things.

Generation, you don’t want it to remember something like, if you asked one time to like, “Oh, please open as a draft PR,” you don’t want to be like, “Oh, everyone forever now should get their PRs as draft PRs.”

one big difference between two years ago and today is these agents are really good at using anything that resembles a file system natively. And so part of us are, is thinking, “Oh, should we rebuild memories to feel more like a file system that we let the agent navigate on its own?”

잠들지 않는 에이전트 — memory.md 한 장으로 영구 PM이 된다

letting an agent basically have a very similar file, a memory.md or something, and just like be your permanent PM for a specific set of issues maybe.

you want a Devin that never stops, it’s just always awake, but it has this like memory dock that it can just maintain for itself about, okay, what are like the number one priorities of what we have to fix and prioritize?

how can we actually use Devin for more than just engineering? Can we actually upstream above the engineering process and maybe it’s just Devin creating tickets, which then maybe some humans do, but then maybe other Devins do.

멀티에이전트의 실상 — 스웜은 혼돈, 실용 최적은 아직 한 마리 Devin

We’ve actually given Devin an MCP so they can just go arbitrarily message other Devins And create new Devins, et cetera. But I guess, it somehow creates, a really chaotic world in that sense. And so we’ve still found that most practical use on a day-to-day basis has been one single Devin.

You do see more multi-agent experiments today, but you can argue, are they really multi-agents, or are they just, tool calls?

the thing that gives me the most bullishness that multi-agents might actually be possible is actually what I said earlier about Devin will actually sometimes tell me I’m wrong and push back, and I think that demonstrates a level of maturity and communication today that makes a multi-agent world possible. ... They’re not just, yes men.

don’t do a swarm or anything, just have one Devin, it does its own context management. Just let it keep running for a while and give it some crazy tasks. I think we asked it to, rebuild, a Windows OS system. And it managed to do it just like, going on for long enough.

슬롭 캐넌 — "1,000개 병렬로 쏘는 버튼"은 아직 누구도 못 만들었다

there’s a slop cannon approach where you try to loosen the single agent’s, bottleneck, and I feel like that is, probably an, a very important thing to try to figure out. I don’t think anyone’s, really solved it. Because then you just have more reviewer slop on top of the agent slop To try to wrangle it all.

I feel Devin is slow sometimes Because I’m like, well, yeah, this is very readable and very sensible, but also it is slower than it could be if I just, I want a button to just say, “Just ramp this up 1,000 ... in parallel and just, see what happens,”?

Auto-merge vibe coding의 한계선 — 2주 뒤 코드베이스는 최악의 엔지니어로 회귀한다

we’ve basically tried to build entire products, true products that we knew we would eventually ship, but for now, let’s try to see if we can do it just by purely, vibe coding on top of each other, auto merge, no code review at all. And then there’s this benchmark of how many weeks can you go onto this for Before you say, “We have the trashiest code base.”

I think we found that the state-of-the-art in December was you can probably, run this for about two weeks. By the end of those two weeks, you’d find that, hey, you want to, change the color of a button. Well, it turns out this button is implemented in, 10 different places, and they, have All these different variations, and oh, you forgot one of them, and actually it’s a slightly different color in one spot.

your code base regresses to your worst engineer, because that engineer who is, very gung-ho about AI and is not auditing their code, their pattern starts cementing into the code, and now the AI is referencing their patterns.

you want to be very strict about the boundaries between modules, and it’s your job as an architect, as a CTO, whatever, to say like, “Okay, here’s the hard contract between you guys and you guys. Whatever you do inside this black box is your business. You do whatever. But between these guys, let’s be, really damn clear, and any movement must be signed off by a human or me,”

느린 Grep의 진범 — 네트워크 파일시스템, 그리고 diff에 비례하는 부팅

I think one same problem that I recurringly heard a lot of them run into was this problem of like, “Oh, Grep is really slow on our agents’ machines.” And so a lot of them, I assume because they’re using AI and they themselves don’t have, super deep infra background knowledge, say, “Okay, we’re going to go build our own custom Grep index. It’s going to be really fast,”

a lot of these virtual machines actually underlying them don’t use real file systems. They use these, network file systems where things are actually cached over the network actually in S3. So when you’re Grepping, you’re actually making network calls Every time you’re doing these things, and that’s why Grep is extremely slow on these machines.

Imagine you have, a terabyte of disk, and your agent only, wrote, a hundred lines of code on top of that disk. How long does it, say, take to, save and re-bring up that disk? ... In our system, we try to build a file system that incrementally builds on top of each other. So every time you save and bring the machine back up, you’re only doing work that is proportional to effectively the diff in the file system.

It’s, not technically agents, but it is agent infra, and when you sell an agent as a company, you sell agent plus agent infra.

AI 슬롭의 서명 — getattr 보상 해킹, 인라인 PRD, GitAI

It doesn’t want the code to fail. And so ... Even when it knows it has the attribute, it’ll call getattr ... and for a lot of my clients who have moved towards more autonomous coding, we’ve put that in as a lint rule That if you do getattr, your pull request is going to fail.

a pattern that I see a lot of across GPT models specifically is backwards compatibility, at all costs ... Where it’s doing these weird import exports so that it doesn’t have to modify, the names of where the modules were. And I’ve seen Claude 4.6 starting to do this as well.

One of the things this new model likes to do is it writes lots of comments. Not like, it’ll, comment every line, but it’ll write, paragraph, PRDs, on top of every function. ... It’s like, “Oh, here’s actually the reasoning and why we chose this approach and what the alternatives were and why we shouldn’t do those alternatives.” Still too much information, but I wonder if this actually might be directionally correct if you want systems that can self-maintain themselves in the long run.

if you run an agent, the actual prompts you send to the agent should be stored alongside the code inside the Git metadata so that future agents can reference it, maybe code review bots can reference it. And it’s ideal world where, your context for why decisions were made constantly lives aside, beside your code.

Agent-ready 코드베이스 — 회사가 오래될수록 로컬 테스트로의 이주가 크다

In the end of the day, you want your agent to be able to do the full thing, not just write the code, but actually run it and test it. And a lot of code bases were not necessarily built for this from the start.

We’ve also internally done a big shift to make a lot of our core, components of code testable as purely local dev without needing to actually, integrate with, any live services for this reason. And honestly, the older the company, the more you have to change to shift in this direction. But you can use AI to help you perform this migration nowadays.

Windsurf 2.0 — 모든 에이전트의 로컬 커맨드 센터

we just want Windsurf to just be your, local command center of all your agents, your background ones, your local ones, and you can imagine, “Oh, okay, this agent needs me to review something. I’ll pull that down, move my other agents to the background, go test it. Okay, boom, done. On to the next one,” right?

When it can test on its own and send you a video, it’s freaking magical. Sometimes there are just really difficult things ... that you do just need to, pull down locally.

I think local agents, you want them to be a bit more fast and let the user make the call on things. Actually don’t try to autonomously go test things. The background agent mode where you go start it off, I think the agent should just assume the next message I send a user should just have everything that the user needs from me ... Keep running and don’t stop until ... you have full report.

유스케이스 — SRE 첫 응답자, 그리고 Slack에서 PR을 만드는 PM

I think the easiest and most common use case I see across everyone is SRE use cases. The idea that whether we have our alerts in Slack or Datadog or wherever they’re going, we want the agent to be the first responder on that. And that doesn’t necessarily mean that the agent is actually resolving the issue, but just being able to collect that context ahead of time is huge.

in a lot of cases, the PM, if there’s just a quick bug fix, the PM is not creating an issue anymore. The PM is just prompting through Slack, and the pull request is then being created. ... I think that trend will continue, where we’re seeing, code modifications happening outside of engineering.

The SRE use case, internally we think about it as auto triage Because we just want every message that comes in, and that’s an alert, that’s a bug report, to have Devin just start triaging it before anything else.

CLIs can sometimes be, very difficult for people who aren’t technical to go and use. But an online chat interface that anyone can go and ask questions and is super intuitive and doesn’t assume you have any technical knowledge but does have access to all parts of your code base, super useful For support, for salespeople

에이전트 예산 — 엔지니어당 $1k~5k, 그리고 frontier+subfrontier 하이브리드

Common numbers that I hear are anywhere from 1,000 an engineer up to 5,000 an engineer. I have not heard anywhere in the realm of, 50,000 an engineer for a frame of reference.

we’re going to see very expensive, very smart frontier models, And we’re also going to see people who say, “What? I don’t need the frontier anymore for a lot of the work I do,”

I’m really interested in a world where you basically have hybrid frontier and subfrontier systems Where you use the subfrontier part to be really fast, really efficient, and call out to the frontier part of the system so that you can still get frontier performance for the most part.

Autonomous coding factory의 시대정신 — 채용 기준은 high-taste

There’s a real zeitgeist in the space right now I think, for companies to want to turn themselves into these autonomous coding factories.

I think people underestimate the role of, really high-taste product engineers In this space right now.

And the test is, what have you shipped end to end that is A tasteful product. ... If you’ve shipped stuff that you think is tasteful and you’re, and you’re proud of, you should, you should come talk to us.