GitHub's Plan for Agents

생각 덩어리

13년 차 COO이자 CMO — 문제는 언제나 사람이었다

Anything that integrated with GitHub, up until really twenty eighteen, I built or ran the engineering teams.

I still code. I love coding but the problem has always been, people. It’s a much harder problem to both support our own employees, a harder problem to communicate to developers and enterprise buyers what we’re building why it matters, ‘cause those are two very different messages.

앞으로 짓는 게 아니라 뒤로 도는 루프 — AI가 COO를 다시 코딩하게 했다

I think what you see there is me, really getting back to coding thanks to AI.

Helping people find the workflows of, “Okay, I need you to go through all the PRs today. I need you to go through everything that we’ve posted online. I need you to go through what we did the last three months. Go through all of my Obsidian notes for any mentions of this then go through my transcripts at work.”

I find AI in a what most of this launch here is actually, less building forward. It’s actually, a recursive loop backwards. I’m always looking at what had happened first. Go back through the week and tell me what we did, what worked, what didn’t work?

That retrospection is actually LLMs are very good at that. Like finding all the patterns, pulling them out, and then applying that retrospection to just a couple of days or just like a short period of time.

내부 롤아웃의 원칙 — 아무도 일하는 방식을 바꾸지 않게

One of the things that I was really, passionate about is like we have to do this in a way where no one has to change how they work. I don’t want to have to teach you a tool. I don’t want to have to teach you something new.

What we’ve actually ended up doing is we’ve built like a set of skills internally. We have we each have our set of skills, and we’ve just been distributing even to the non-technical folks, the CLI.

[Slack은] a purpose-built tool for us, and I think the reality is that moving off of it would be so bluntly expensive? Simply because all the tooling is, baked in with that paradigm.

But it’s just like enabling everyone with that power of it’s going to take me a week to figure this out. Instead, we’re going “Okay I built a skill. Let’s put it into a repo. We’ll all share that skill together, and then we’ll use the CLI or now the app just to run it.”

메가 스킬의 종말 — 레고 블록과 설명서

I think that we’re ending the era of these like massive, beautiful, perfect skills that are just like not any of those things.

Less about stitch a bunch of tools together and have it produce this mega output because then weeks go by, months go by, things change, and you want to tweak ... Your mega skill and you’re screwed? You can’t do that.

And so now we’re really just talking about the Legos we’re using and just letting the instruction book be something we’re all putting together.

'summarize anything'은 없다 — 좋은 요약은 직군마다 다르다

My summarize everything is very different for each one of those like Contexts. ... if I’m summarizing something for an analyst, that’s a very different thing than, probably how I’m going to summarize something for like a customer meeting or an engagement.

They’re all just slight permutations, but those permutations are really important. It’s the difference between someone reading this and going “Did AI make this?” ... Or “This makes total sense, and I would expect this when I’m giving a briefing to Gartner.”

개발자 출신 리더의 황금기 — 토요일의 에이전트 15개

I think that like the secret has always been your ability to identify patterns and solve problems, and I think that for folks that like myself that don’t code day to day anymore, that has made me successful as a developer, made me successful as a COO and now CMO.

I just find that the folks that came from a different career, went to school for something else, went off and did this random thing, and then became a software dev, or were a dev, did a random thing, came back. Learning that extra set of information, learning those extra skills, and now having the power of an AI where I can crank up fifteen agents on Saturday while my kids are doing lacrosse, That’s like really powerful.

And I think it gets me back to that feeling of like creation, and it’s very hard to replicate that in most other senses? That first time you build an app and you click it and you show someone that’s magical.

AI 티를 지우는 스킬 — CRO·CFO 앞에서 들키지 않은 프레젠테이션

And I ultimately built this entire presentation without touching any of it and I was “Okay, I’m just going to present this to our CRO, the CFO, their teams,” without mentioning I’d built it with AI. I like built a skill to make it look very much not AI driven. Just not pretty.

We did the whole thing through. It used my notes from Obsidian, it used all the context I mentioned before, the plans, and Never came up once that it was AI generated.

I can take that tool and go, “Look, I don’t want you to go build slideshows.” They’re just helping us share information with each other. If this thing can do it With a little bit of crafting from you and then we can look at it together, awesome. There’s no value in all that extra work.

역할은 사라지지 않고 바뀐다 — chief of staff의 경우

It’s sort of the discussion every time we have some sort of technology evolution is it’s not that the jobs the roles don’t all go away, they just change?

But now I need that person that is able to go and find all the different connections between humans in those discussions to help me find out, okay, I should be meeting with this group and this team, and they have an opportunity.

But now just like chiefs of staff are not opening up, letters to process, they’re doing emails. ... It’s the same thing.

임의의 Ruby 코드에서 Actions까지 — 플랫폼 격리의 역사

GitHub Services back in the old days, right? ... we had a repository that was Ruby code, and you could write any Ruby code in there, and then we would execute that On your behalf As a service.

But, that journey from that day to “Okay, we’re just going to run all this arbitrary code, and, it’ll basically be okay,” to now, no, we have, really good containerization.

With, Dev Compute, you can, run, really fast, spin up really, small VMs really quickly, so you’re doing a tool call ... So we’re using that so definitely moving that direction to protect us from every piece of code that we’re ultimately running.

npm — 더 안전하게 만드는 모든 조치가 누군가를 깨뜨린다

The big thing for the both of us was to find a way to keep NPM, which was basically powering the internet then and way more so now to some degree running.

But, it is a unique challenge in that every move that we make to make it more secure will break a lot of people. ... we make a change that makes us more secure but hurts, there’s, a snow day for developers or a really bad fire that they have to go put out.

We’re trying to push the community, forward without necessarily, doing something that is going to break the contract that’s been for 15 years or close to it or some amount of years on NPM.

Slop fork와 vendoring — 옛것이 다시 새것

In some ways, it’s all all things, old or new again? Yeah, absolutely vendoring everything. ... I do remember twenty thirteen, twenty fourteen. ... We were having actual discussions around, or at least I remember we were “Should we take this full thing?” “Why is this so big? We only need this one file.”

It’s not going to solve the fundamental problem, I don’t think, because the vulnerabilities in an agent looking at them, there’s time and time again, there’s a million different ways in which we can convince an agent that this thing is, secure or not and pull it in.

먼저 굳히지 않는다 — GitHub의 사회적 RFC

We rarely start like a process and a practice and like push it onto the community. We usually wait for the sort of like RFC process socially or literally, everyone agreeing, and then we’ll cement something in.

We’re not going to be able to fix everything for everyone in a way that everyone is going to like. So tell, help us, tell us what is working.

[Mitchell Hashimoto의 vouch 시스템에 대해] he and his community really likes this and then I’ll go talk to other maintainers and other maintainers, globally, and they’re “No, this doesn’t work for me.” And that is the tension, but also the kind of beauty of GitHub.

You can go use this project, and if it takes off and becomes the kind of mostly standard, then yeah, we probably wouldn’t enforce it but we would add it in because that’s the flow that we tend to do?

PR의 80%가 에이전트에서 올 때 — 코드화하려는 것은 결국 신뢰

And now we have to go and Figure out the next best process or what adaptations change, or what does a pull request look like when eighty percent of your PRs are just coming from your agents and not From other devs?

I think the reason why there’s not a single answer is ultimately we’re trying to codify trust. We’re trying to say “Okay, if Sean reviews this I’m going to trust it because you’re Sean or you’re the senior dev or you’re the whatever.” And right now, when we are working in a flow where an agent writes code and another agent reviews code and then Kyle goes and looks at it the trust is kind of diffuse.

It’s a social problem ultimately. It’s a it’s a human problem to review it and agree. Or you fully trust the tool and you’re imbuing that tool with full trust Which I think in some cases that absolutely exists.

Waymo 유추 — 검증 가능한 증거와 인간적 느낌, 둘 다 필요하다

I took a Waymo here, and I was on my phone and not looking around at all. There are other, self-driving, vehicles that I would not trust while, staring at the road.

I strongly believe that’s, a mixture of verifiable proof. Like how many accidents, how much data, and so on, and the human aspect of how I feel when I’m in this car, what it tells me, et cetera.

Even when it is fully verified, not only do you have to have trust from the humans on the team, you probably have to have trust from multinational, ... Multi governments around the world and regulating agencies.

스타는 수동 신호 — 게이밍 가능한 지표와 sponsors의 hard trust

Some degree of like hard trust and support, which would like for me is like sponsors is a good example of that. ... It like costs you something. To prove that I believe in your project and I trust you To some degree or I want to support you at the very least.

I think the thing that I don’t love about stars or commit counts or anything else is ultimately, even with all of the various, abuse and de-spamming and deduplication work that we do or anti-abuse work that we do, these are all, not active social signals. They’re passive ones that are ultimately gamifiable.

Because what happens? The attackers just go and go and create a multitude of accounts, and they wait Until it ages up. Needs to have a certain amount of stars. That’s how star inflation happens.

Most open source projects have that heuristic built into their heads, if not written down in the contributing guidelines. You could take that and then go apply that and then just say, “Oh, we’re not going to accept this PR.”

개발자 2억 명 — 게이트키핑은 시간 낭비

Oh, we’re over 200 million now. ... over 200 million developers now.

I think that the idea that we should be I don’t know, splitting hairs or segmenting developers in the early era of software development is, not worth our not worth the time.

So I fight very clearly on the line of, if you create code, if you have an idea and you create it into some way of, I’m, I’m going to run it and use the app right now, you may still use AI in that moment, but that’s okay. At some point you’re going to do the next thing.

They’re working on a side app that they built on the weekend for their friends or for their new idea or whatever. And that’s how you see these enormous charts going up and to the right with With stars.

코드를 숨기지 않는다 — 전등 스위치 원칙

When we put a veneer on top of something, we still always show you the code. That’s kind of like a tenant. We’re never going to, hide the code from you ever.

Anyone should be able to go and build a thing, just like I can go change a light switch in my house. I’m not going to go into the breaker box ‘cause I’ll probably kill myself? But, I can go change that light switch. Everyone should be able to go and say, “This fricking app doesn’t do what I want. I want it to work like this.”

We’re the home for all developers, and we want everyone to be able to have that feeling that we’ve had of, had an idea, I created it and holy shit here it is.

최고이자 최악의 시기 — 작년 1년치를 이제 한 달에

Oh at GitHub? It’s a hard time. Like, it’s a hard time and also, I was just with my team and I said, “This is also, the best and most exciting time that I think I can remember at GitHub.”

I was at Universe in October saying, “This was the fastest year of growth that we’ve ever had,” right? And now we’re doing more in a month than we did in a year last year.

Kind of like you name it by roughly every measure that we’re looking at, there’s some amount of sort of growth that is much bigger, and that is breaking our system in new ways, not old ways.

We’re using the same tool you’re using. We’re not using a super secret version. ... If it’s not working it’s not working for us, and that’s kind of like the promise of dogfooding for GitHub.

무엇이 부러졌나 — CPU, MySQL One, 모노레포, 커진 파이프

More tools, more agents, more PRs mean more builds, more builds mean more CPUs.

The place that we continue to have pain is in, permissioning. And so right now many of our permissioning layers sit into a database that we like internally call MySQL One, and old Hubbers will know what I’m talking about.

Repos were smaller, but there were more of them, and now we’re seeing the opposite. Repos are bigger, and there’s, not fewer of them per se ‘cause there’s new growth, but, we’re just seeing many more big repos. Big repos, big monorepos have always had, a unique performance problem.

We built everything in a way that assumed, I guess in some ways that the size of the pipe of work was going to remain the same. There’s just going to be more people coming through each of those pipes. But instead now in places where a git push was, generally a certain size for example, is now, no longer true.

수직도 수평도 아닌 대각선 — 10년 된 서비스의 규칙을 다시 쓴다

And now we’re sort of in a like diagonal, where like vertical doesn’t really work anymore. Horizontal isn’t work either because we’re all We all have some CPU or GPU constraints in the world now, and now we have to go in and like crack open services that have been running for 10 or 15 years and go, “Okay, the rules of this service have legitimately changed, and now we have to rewrite them.”

Let’s stop putting good money after bad and do it the do it the right way or the right way now.

None of this is an excuse. This is like we’re We have to do the work. We have to make it better.

다운타임의 계약 — 점진적 개선이 아니라 step change

Part of it comes from a very old GitHub ethos, which is it’s our uptime. It’s down.

Write these blogs, write the posts, get the engineers who built it after they finish the work, just tell you “Okay, this is what we did.” I think that’s the contract that we want to bring back to the community.

It’s less about, these incremental improvements where you make a small change, and you get this big output. It’s now material change That takes a bit of time, and then you see a step change in our availability.

It’s oh, we tested it at N amount of scale, and now it’s at, N cubed maybe like in some in some vectors.

Copilot의 잃어버린 시간 — fine-tuning에 베팅했는데 모델이 좋아졌다

We came out the gate with code completion, and it was super great, powerful, et cetera. And then what we initially worked on after that sort of, initial year and a half, was, going after fine-tuning.

Everyone kind of, will ask, “Well, what happened to GitHub Copilot?” there’s all this time, and I would say that we were on an era of going okay, we want to improve everyone’s results, and so let’s focus in on fine-tuning because that’ll give us these better results. And then the models got better.

And so there was this moment of both, really trying to figure out what our customers want, models, Sherlocking us a little bit, then going and saying, “Okay, what does everyone ultimately need?”

Now have, a single underlying, SDK and harness for our coding agent Copilot ultimately. The new CLI, the new desktop app, cloud agents that use the same SDK. ... How do I do that for every GitHub issue that comes in, just stick a coding agent on it just to see if it’s possible?

GitHub이 Kyle처럼 행동할 때 — ambient AI라는 미답 영역

The final thing that will let me ultimately, feel complete at GitHub is, when we have this ability for GitHub to act like Kyle wants it to act Or Shawn or whatever.

I think the most interesting thing to me in AI is actual ambient AI, not insert assistant name thing or, I’ve tried just about every pin in tool and whatever, and they don’t work the way that I’m looking for them to work because they are just trying to capture, and then they are trying to codify and then recall.

It’s as if, software development work was a single lane task, was like it only needs a developer. Once I once I write the perfect code, we’ll be done here, but that’s just never been true. It’s all the context of the other team members, what the business is doing what’s popular right now.

[Nat Friedman이 OpenClaw를 카메라에 연결한 일화에 대해] I actually would love OpenClaw to tell me to Drink water. I don’t know that I want it to be, Changing where my car goes.

OpenClaw와 OS의 재설계 — Claw를 쓰고도 잘리지 않아야 한다

OpenClaw has become the personification of, a valuable agent that understands me because it has access to all of my information, and it can use a computer.

One of the problems that we have, right, is that our agents, if you install them not on a Mac Mini or not on a hosted device, you install them on a personal device or a work device, we need better sandboxing at the OS level. I need to be able to use that Claw and not, get fired.

Operating systems need to look different than they looked five years ago because it’s not just you using them anymore. And that’s changed the whole idea. It’s not, “Okay, my Claw is going to create a user account.” Doesn’t work like that?

Because ultimately, we need more platform components. We don’t need everyone to be building the same exact, top-line product.

SaaS에서 물리학으로 — 그리고 Satya에게 물을 질문

I would say software in the last five or six years were all going to our events, and we’re kind of saying a version of the same thing. SaaS product has new SaaS thing. It’s the best SaaS thing ever. ... And so now it’s like Oh my goodness, we’re at physics. ... We’re at physics problems. And that’s exciting.

If you are in the M365 land in any way, check out WorkIQ, check out FoundryIQ. These little, oversimplifying it context engines are wild good.

The code, the code and the code asset problem is a little bit unique. But otherwise ... We’re all working ... With each other now. It’s all just context, exactly.

I think that the best question to ask is what he thinks is true in, two or three years from now. It seems like such a throwaway question. ... you can see some of the recent shifts that have been happening inside of Microsoft to kind of drive us to a place where it’s not four, five, six, seven, eight different things. It’s not a lack of context everywhere.