grill 스킬 9가지 오해 — 질문의 fidelity·scope·모델·병렬

생각 덩어리

grill은 너를 대체하지 않는다 — 답하는 사람의 실력에 기댄다

However, I sometimes hear from people using them, like Codex just ask me 200 questions, this issue here. And I kind of wince a little bit.

...it relies on the skill of the person answering the questions. The person answering the questions, in other words, you using the Grill Me skill, need to be good at planning.

...they're designed to aid you as an engineer, not replace you as an engineer.

질문의 fidelity — Shape Up에서 빌려온 렌즈

I'm taking this language from Ryan Singer's amazing book Shape Up. High fidelity questions are questions where you need a really zoomed in, really detailed, high fidelity image in order to understand it.

low-fidelity questions are questions that you don't need a high-fidelity kind of prototype or image to answer.

grillable vs ungrillable — 첫 번째 실패 모드

...the first failure mode I see with the Grill Me skills is trying to answer high-fidelity questions during a grilling session.

...there are questions that are grillable, in other words, answerable in a grilling session, and questions that are ungrillable, questions that are not answerable in a grilling session.

prototyping handoff — 느낌으로만 답할 질문은 만들어 봐야 안다

...the thing I tend to do is I tend to have a prototyping handoff.

What I do is I use the handoff skill, which I will link to below, to handoff to a prototyping session, where I will then spend another session kind of just prototyping on that question, seeing it in a higher fidelity. And then, whatever I learn from that, I will handoff back to the original grilling session, so that I can continue with the grillable questions.

scope — 너무 크면 ~120K에서 dumb zone에 빠진다

The next concept we need to understand here is scope, how large a thing you are grilling.

For those who don't know, about 120k is where I estimate most state-of-the-art models, that's where their dumb zone begins.

...you've got to keep a really keen eye on your context window in order to make sure you don't push past that because the model will start getting too strained kind of in its attention relationships and start making stupider decisions.

passive냐 active냐 — 양쪽 다 실패 모드

When I'm doing grilling, I'm always quite active, always trying to lead the conversation. And remember, it's a conversation, not an interview.

...if you're being too passive, then it's very easy for the agent to just do stupid things with the interview, like ask you 540 questions, explode the scope, ask questions about stuff that are way too low fidelity.

But it's also possible to be too active, to just keep grilling on something that is just too low fidelity when you need to actually build something to see the thing in action.

grilling의 컨텍스트는 버리는 게 아니다 — clear 후 to-PRD는 미친 짓

...one crazy thing I've seen people getting wrong is they actually clear the context first, and they create a new context window, and just run two PRD in there. This is totally crazy to me. What are you doing?

You've created this incredible session here where you've got this, you know, 100,000 tokens of really good design decisions, and you're just going to chuck it away?

Every grilling session, every decision that you make in that session is so valuable.

grilling엔 똑똑한 모델을 — parametric knowledge에 기대니까

Another thing people get wrong is that they use too dumb a model for grilling.

...when you're relying on parametric knowledge like this, you need a model with lots of parameters, and that is usually what the big frontier models have.

What most people don't know is you can actually use a kind of dumber model for implementation because most of the information you're passing there is contextual.

병렬 grilling — 두 세션을 Slack 스레드처럼

You should grill multiple sessions in parallel.

People say this is, you know, context switching, but really it's just managing two separate Slack threads at the same time. You know, it's really not that hard.

Usually, I max out at two sessions here unless one of them is doing a particularly long-running task like some research, in which case I will try three...