Poteto Transcripts

Hi everyone, I'm Lauren Tan, and I have been at Cursor for about five months. And I think something I've noticed actually, which is quite interesting, is that there are so many parallels with, you know, management skills and how to like manage agents. And that's actually a big part about what I wanted to chat with you and everybody else about today. But yeah, that's me.

Yeah, I think the big theme for me as I've been using agents to write code, and I'm sure a lot of you have had the same experience as well, is how do you trust it? You know, especially if you are an engineer that's been writing code for a very long time, you have a lot of opinions and lessons that you've learned about doing good engineering.

And when you see agents just, you know, winging it and, you know, guessing, hallucinating, you know, confidently stating that they found the smoking gun for the hundredth time, but it's actually not the real problem. You lose a lot of trust. And when you lose, when you don't have much trust in your agents, I feel like you really can't get the most out of them. And for me, the parallel is like with management.

So if I'm an engineering manager of a team, and I have a bunch of, you know, I have a team of engineers on my team, and I don't trust them, then the mode of operation I'm going to be in is going to be like micromanagement, right? I'll have to spend a lot of time looking over my reports' shoulders and checking that they're doing their work well, you know, that they're not shipping bugs to production.

And so I drew this chart because it's not a very scientific chart, but like this is how I imagine myself and my journey through using agents. So, you know, like fast forward or back forward or fast back, fast backwards, like a year or so when, you know, nobody was or not many people were using agents to code. I think you, you know, you get into this mode where you are in very heavily in the loop with one or several, like a handful of agents.

And you find yourself just constantly, you know, trying to understand what your agents are doing. And you're very, very in the loop. You're watching every single output. You are sitting there prompting. And you really can't parallelize beyond that because you don't, again, you don't have that trust, right? You can't go to a hundred agents, like spawn a hundred agents when you don't even trust the output of one agent.

So over the past five months, I feel like I've really been able to like ascend this trust curve. And now I'm at the point where I actually have, this sounds kind of scary to say this and it makes me sound like a slop artist, but I promise I'm not. But I actually have my agents now auto-merging PRs for me, which is like a wild thing to say. But like I woke up today and there were like 20 PRs landed and I just reviewed them on main, like they were already landed and they were good.

So how did I get to that point? It's basically what I wanted to talk about today.

And again, like, yeah, feel free to jump in if you have questions, Denis.

But, oh yeah, of course I got to show this, this chart where,

No, do not trust, someone requested to control my computer probably won't do that.

But yeah, so this chart, I think I'm sharing this chart not to kind of like flex, but to kind of show like the journey. Like, so you can see like the curve, like it's sort of like inversely matches the contributions I've been able to land at Cursor. So I joined five months ago and five months ago, like I said, you know, my first month I was like not very productive because I was, you know, I was learning the codebase. I didn't know what the heck was going on.

And as I got more confident in my agents, I've really been able to kind of ramp up my productivity. And again, like, yeah, like last month I shipped a thousand PRs, which is ridiculous. And then this month, we're only on the 12th. I'm already at like almost 800 PRs landed. So the velocity is definitely high and I'm sure a lot of you will definitely be questioning like how much of this code is actually good. And I think, yeah, that's definitely fair to question.

But yeah, I think if you set up your agents well, you can definitely get to a very similar level. And so I'm going to talk about how we do that.

So for me, I think I'm curious, like, I guess, Denis, your experience as well. But for me, I think the most important skill that you should have in your toolbox when you work with agents is verification. And by verification, I mean the ability for an agent to actually run the code or take CPU traces or heap snapshots or, you know, open an iOS simulator, whatever, you know, however your application is exposed to your users.

It can do the same thing and run it for real and actually test and verify it doesn't work. Because that's the thing that really closes the loop. It doesn't guarantee your agent writes good code, but it allows them to at least write correct code, which is a big, a really big step forward for being able to trust your agent. And I will, I can share one example that we have within Cursor.

Oops. Where, let me open this.

Let me just make this a full screen. There you go.

So for the, for Cursor's Agents Window, so this is actually an interesting story. But when I joined Cursor five months ago, they're actually, well, I was supposed to join a different team. I was supposed to join like the Cloud Agents team. But then since I have a lot of experience working on React and Agents Window is a React application, I was, I was asked to basically help out with the Agent Window work.

But there wasn't really a lot of like skills to help me. So I just found myself like, okay, Agents Window is going to launch in like a week. Right. We have a really tight deadline.

And there was, you know, I was just sitting there like, okay, I'm going to open up the performance, the, the Chrome DevTools and just like take a trace, look at it myself and try to make sense of this flame graph. And keep in mind, I was just like in my first week. So I had no idea what I was looking at. No idea where, you know, I mean, I had some idea, but, you know, the codebase was completely fresh to me. And I realized like my agent had no idea either.

You know, like I would take a screenshot of the trace, I would download trace, I would send it to it and it'd be like, yeah, it kind of looks like this, you know. And it would like confidently state like it's this thing. And then I'd try to fix that. And turns out that's not the actual thing.

So this was a very, very slow process. And if you've ever done any like performance work yourself or, you know, just even development with an agent where you don't have a verification skill, you are the verifier, right? You, you're the bottleneck. You, you, you tell your agent to do something and then it goes off and write some code. Then you open up your, you know, local dev build and then you start to say, oh, you know, it doesn't work.

Then you have to copy paste screen, you know, screenshots or console errors or whatever. And then your agent like slowly kind of like, you know, works with that and then tries to understand it and fix the thing. But then you're constantly just in the loop and being a bottleneck. So there's really no way to parallelize. So the Control Glass skill is like one of the first skills I built for Cursor.

And Glass, by the way, is the code name for Agents Window that we use internally, but it's just cursor, I guess.

And so this skill is, I guess, the code itself is not super interesting. Your agent can very easily make one for you, where if you're building an Electron app or a web app or even iOS applications, you can teach your agent how to use like the Chrome DevTools Protocol or through Apple has some utilities as well for running the simulator and taking traces and controlling programmatic control as well. So that's really useful.

But one thing I actually want to talk about is the, this thing.

Where is the read me? So this skill comes with this very unique feature called, or not feature, unique file called a feature map. And so the story then is like, I built this skill. And so now the agent was able to actually run the agent window and take traces and whatnot.

But it had no idea what, what the agent's window was.

So, you know, like someone would say like, oh, the left sidebar is like laggy or something like that. Or, you know, the right side, the PR tab is not working. And the agent would just be like kind of flailing around. It would spend a lot of time trying to like look up the code. And, you know, where is this feature? How do I actually get to it on the UI? Which made it basically completely useless. You know, like we would, I would run the skill locally.

And, you know, it would spawn a dev build. But then it just be churning. Like I just try to click here. It wouldn't know how to get to things. And it was just an awful experience.

So who was putting arrows on my screen?

So, yeah, this feature map has been really useful. Because it teaches the agent how to get to all of the features that you have. And in pstack, the plugin that I've made. If you search for pstack cursor on Google, you'll find it. But there is a create verification skill in that plugin. Where it actually helps you set up something like this for yourself. Including the feature map. So it will actually explore the code. And build up this initial feature map.

That tells your agent how to get to all of the different features that you have. And this is extremely powerful. Because now that you have these user reports that come in. You can actually map even like a vague report. Or even a screenshot. So we have this internally at Cursor. Where we have a Slack channel. Where, you know, lots of people giving us feedback. On the Agents Window. And Grokbot and whatnot.

And oftentimes the report is very bad. Like very low quality. Like someone will just put. Very often we get like a screenshot. And then someone just says, question mark, question mark, question mark. Like, what is this? And, you know, like without this. Your agent is like, I have no clue. Right? But with a feature map like this. It has lot more context and understanding of how to actually navigate, how to get to all of the different features.

So like, you know, example, like, I guess, like the sidebar, like, what is the sidebar? You know, like all the different sub features that are present in it. Like from the user point of view, here's how to get to it, all the different keyboard shortcuts, even like the, what do you call it, the DOM elements, or yeah, like the attributes that you use for selecting things through the CDP are all there. So again, yeah, this is like really, really powerful for agents.

Yeah so pstack is pretty interesting because uh well first of all the name is kind of goofy like the P, the P in pstack is like potato potato snack because i um so uh there is a pretty uh famous person gary tan who is the ceo of y combinator and he's come up with this plugin called Gstack, Gstack. And funnily enough, we share the last name. We have no relations. But I thought it would be funny to kind of, you know, poke fun at Gary and make pstack my version of his plugin.

But kind of just tailor it to my own set of engineering practices.

But I honestly actually never set out to build pstack. It just started with a bunch of skills, right? Like I started with that Control Glass skill. And then I started with another skill called Howl, which I also noticed through observing agents. So in the early days of me trying to climb this ladder, I was super in the loop. And I was basically nitpicking my agents to an extreme degree. I would tell it, this feature has stopped working. Here's a bug report. Why isn't it working?

And very often the agent would just like confidently state like oh it has to be this right it has to be this thing and i noticed like when i looked at the actual tool calls i noticed it wasn't actually reading the code that i thought should be affected and that made me just extremely suspicious and at that point i was just i'm not gonna i can't trust any this agent anymore because it's just it's just completely hallucinating and i think i think it's very easy to just you know like build up that distrust and not and kind of feel helpless like you know you don't know how to help your agents succeed but like again I think the the management analogy is super helpful because like imagine if you were a manager of an engineering team and you had an engineer on your team who was a really good coder no business context whatsoever you know they just you just hired them and they onboarded you know like five seconds ago uh and So how do you actually teach that person to be effective?

So how you do that is through a skill. A skill being just, you know, it's just markdown, right? But, you know, it encodes a lot of information, instructions, a lot of, you can really draw out a lot of intelligence from an agent by, well, some people on Twitter call it like, you know, pull the agent to a different latent space, which is kind of like a fancy way of I'm just saying, like, since, you know, LLMs are sort of like, they predict the next token,

When you give it some high quality tokens to begin with, then, you know, it can kind of pattern match on, like, a higher space that's, you know, smarter.

Uh Uh.

Yeah, so evals are a way to unit test your skills basically. And actually in pstack we ship, under Potato mode, there's a playbook if you search for it called Eval Playbook. And it's, it's like not, it's actually pretty, pretty rigorous the way it's done. But essentially what I do is I spawn a lot of different subagents. I have like my main coordinator agent come up with a rubric for what I want the skill to do.

And then it spawns all these subagents and it creates individual directories for them, which are cleverly named to not let the subagent know that it's being evaluated, because agents can actually tell, and when they do they change their behavior.

But it does a bunch of stuff like that to essentially test whether or not the skill I'm making or changing is actually doing what I think it does. And one of the really nice things about Cursor is that we support so many different models so you can actually eval your skill across all sorts of different models and, you know, get a sense of how well it performs across that different matrix, especially for the models that you use.

So I do this a lot. Every time I modify a skill, I will run one of these, like the Eval Playbook, and make sure that, you know, it's actually leading to a result I want. But I will say, like,

Maintaining skills is actually pretty hard uh it requires i think a lot of taste and observation so you kind of need to be very good at being a backseat driver you know i mean like if you do if you've ever done pair programming for example uh and you watch a co-worker code and you just like you could probably do this better you know you could do you know like why did you not do this right you you ask a lot of questions to your co-worker and it's kind of a similar thing here Like, you don't want to just be a passive observer of the agent.

You want to be very in the driver's seat in the initial stages when you're building up your own set of skills. You know, obviously you can use something like pstack, but if you're building your own set of skills, it's very, I think, you know, opening up all the tool calls and reading the code and reading all the agent behavior and their thinking blocks is a really great way to see where they fail.

Right like what what you know where are they being done and then you can go and build a skill for that and then with verification how you trust it is it's I think it's also a very similar iteration loop uh where you know like I actually did the same process for verifying the verification skill where I actually get um so one thing that's interesting about evals is that you can sort of hill-climb them, meaning that your eval can produce a score, right?

A score that you can get your coordinator to produce, but also you can have a judge agent of a different model to kind of cross-reference and make sure that the first model is not being biased, right? The model that's judging all of the subagents that are running the thing.

But you can also like hill-climb so meaning that you can you can use like slash loop in Cursor and you can say okay keep looping on this eval right until everything is 10 out of 10 as an example uh and i did the same the basically the same approach with the control skill and so i kind of it was very it was very hands-off actually uh so you know i uh i kind of built i built that skill that way like the cli in that skill um and over time it's gotten really good uh but yeah it was definitely not super smooth at the beginning it required a lot of iteration and And i think there's there's an a analogy here for me, which is, well, I make this analogy later in a different slide on my drawing here.

But I think of it like, you know, as an engineer now, you're sort of more like maybe a manager. Or the analogy I like is like you're like a chef in a restaurant. You're the head chef. You're not cooking all the food yourself anymore. You have a team of cooks, right? You have line cooks, you have a sous chef, you have all these different stations.

And it's your job to really design the environment. You're in charge of setting up the kitchen. You're in charge of giving tasks to different people.

Just one follow-up there to try to go one layer deeper. So are you, let's say we wanted to build an eval or a skill for something, and we wanted to kind of get better on its own, which is what I think you're suggesting.

Um i think that uh the best place to start is local because you can observe you can definitely observe what your agents are doing. So if you're building a verification skill for yourself, I would definitely start local and just have your agent bring up the application, whether it's like a CLI or a desktop app or whatever. And so you can actually observe, right? You can see how the agent is interacting with the application.

You can see it, you know, how it calls like the different APIs that allow it to interact with the application.

But for me personally, I have basically been kind of all in, mostly all in on cloud agents because they're extremely powerful.

And the really powerful thing about Cursor is the cloud agents actually, where if you spend a little bit of time setting up your environment, these control skills, these verification skills pay a huge amount of dividends because it's not just something that makes you as a single engineer better it actually levels up your whole team uh and even your whole company because uh you can actually start thinking about cloud agents and start thinking about automations that automatically do things like uh i'll i guess i kind of talk about this a bit later but i'll just kind of get into it uh where you know for example like i talk a lot about this agent we have called benny right who who uh you know takes all of the bug reports that we get and it automatically goes off in the cloud opens up a cloud uh it's you know it's desktop it runs cursor in its own computer and it uses the same control skills to interact with the application and try to reproduce the bug or the user report.

And this is so powerful because at once I can immediately, I get so much information from this automatically. Like here in this example, you can see that the Benny actually reproduced the bug, but it's already fixed on main. So it actually confirms that we fixed this problem already. And all I need to do is just release another build of Cursor.

So that's like huge information there that I didn't have to go off and sit with an agent, and spend an hour trying to figure out, is this fixed? Is this not fixed? So you gain back so much time. But everybody on my team benefits from this. Everybody in the company benefits from this.

So definitely think that using cloud agents is super powerful. But yeah, it's like a journey. You have to trust it first before you get to this point. And it goes back to what I was saying here, where, you know, it's very hard. It's almost impossible.

Yeah. So just to kind of recap so far, basically the, if we wanted to go on the journey that you've kind of gone on, it would be just start with verification, building some skills and some ways of determining that the agents are producing

At least like correct code, whether like you said, whether it's good code or not, it's maybe a separate question, but like it's technically solving the problem by looking at, you know, stack traces, looking at, you know, the actual behavior in the app.

Yeah, exactly. I think, yeah, that's why I drew this curve, right? Because this basically describes my journey of, you know, when I started, barely could use a couple of agents and I was just observing every single thing. I think there's really no shortcut for going from here to there, because this is really about your personal level of trust in agents, right? Obviously, you know, as an engineer, you don't want to just slop code into production.

So how do you actually build up that trust? takes a lot of, I guess, taste and judgment.

But, you know, like, I think plugins like pstack definitely can help you get up to speed much quicker. And so I guess it's like, if you trust me and you trust pstack, then by extension, you can maybe trust your agents. But if you don't trust me, and I definitely would not encourage people to blindly trust me,

You know if you build up your own set of skills that you can obviously you know take a look at and kind of fork it make it your own improve the skills definitely encourage that but for me it's really all about it just keeps coming back to trust you know every one of us here in this chat have a different standard for engineering and there are different things that are important for us in our codebase and uh when you are able to encode all of that into skills and you can verify that your agent is actually doing them that allows you to really kind of ascend this curve

I think there's a third part to this which i haven't talked about yet which is kind of an interesting one which is refactoring and rewriting. One of the most controversial topics in the industry, I think, is should you rewrite your app or not? Because I think engineers are very prone to this, especially when you join a company, you come in and you see the codebase and you're like, man, this is shit. Who wrote this code? It's terrible. I want to rewrite the whole thing.

It's a very common inclination. And I think a lot of, you know, before agents, and I guess arguably even now, people will definitely discourage you from rewriting stuff. But I'm actually here to make a case for why you might want to consider it.

Because I think it really depends. You know, brownfield applications, I think, are actually in a pretty good spot, especially if they're set up well already.

uh and like recently i've been talking to some people but uh you know i was just observing i just noticed this parallel which is that a lot of big tech company problems are now everybody's problems um and the big tech company problem you know like when i was working at meta like we had this giant monorepo we had like i don't know tens of thousands of engineers just you know like banging on their keyboards and shipping code and a lot of really great engineers at meta

But I'll say like you know you'll be surprised that the code quality is actually not that good and so I often joke that like you know before AI slop we had human slop and so you know I think a lot of big tech infra like like what meta has or google you know you know really big tech companies are actually designed for that, where you're sort of like, you're catering to the, you know, like, that sounds so bad to say, but like the least capable engineer on your team, right?

You build frameworks, you build conventions, you build guardrails, you know, you restrict credentials so that, you know, your intern doesn't wipe your production database.

There's, you know, if you have that level of infra already, I think your agents can actually already do a very solid job, right? Because they have, the guardrails are already in place for agents to not cause havoc, or not cause too much havoc in your codebase. And you can always add more, you know, guardrails.

But I think, like, greenfield applications, especially are, you know, like, the brand new applications are, like, the biggest risk, in my opinion, and also the greatest opportunity. because you know if you vibe code a project a prototype like we did for Grokbot you You know, Grokbot was spun up very, very, very quickly. And if you haven't heard of Grokbot, it's like a new application we just launched yesterday. It's really cool.

Lets you orchestrate your, create like individual agents that have their own identity and you can kind of orchestrate them. It's super cool. Definitely check it out.

But yeah, it was a very greenfield application like most prototypes are. So it was like vibe coded very quickly. humans were not reading the code at all. And I had this tweet recently where I said something about organic architecture.

Maybe I'll find it. But the idea is that when you have a completely vibe-coded application, you essentially have no guardrails whatsoever. So your agents,

When you give them a task they will just solve it in whatever method is the most convenient and over time you get into this uh situation where you have a codebase that is spiraling out of control because you don't understand it uh your agents understand it i guess in a way but like they've built something that is you know optimized for short for shortcuts uh and uh you you know, you will suffer, you'll have a lot of issues with that application.

So I think starting your codebase with, like, very strong constraints is very much needed. Because, like, when you have a codebase that you can trust, right, when you have guardrails that actually help you, help your agents write good code, you can get into this part of the curve where I said I woke up today and I had like 20 PRs merged by my agents and that's because I invested a lot of time over 600 PRs I calculated yesterday

When I refactored all of Grokbot to this new architecture that I've been building

Um and yeah i've gotten to a point where i i don't really look i really don't look at the code anymore and um i say that not just you know to sell you tokens but because i you know it took a lot of work to get to that point i spent a lot of tokens to get the codebase to this point where i no longer have to look at it uh but i'm very excited because you know of the potential where you know it's not just this doesn't just benefit me it benefits everyone contributing to Grokbot and it also empowers you know designers and product managers and you know people even gtm people to add features to Grokbot and i don't have to worry you know i don't have to wake up at night in in the middle of the night and worry like oh shit someone's just merged a perf regression right i have a ton of constraints and ci it's like it's actually very annoying to write code in in Grokbot, but agents absorb all of that annoyance.

I think it depends.

I think this is fine. But we have... This is not that interesting, but a fun fact is that visualization in Grokbot and in Cursor, is actually powered by Preact, which is a sort of new library that someone's built. That's really interesting. You should check it out. But that's not really that important. I think the average PR size, I actually don't know. I don't know if I want to click on these.

There's definitely no hard cap but i do encourage my agents to split up their work into multiple prs uh i do that mostly because uh i like i like the idea of the i guess maybe this is much harder to do now as in the the world of agents and you have like so many commits but I like the idea that you know the git history is a very rich source of context uh and I like the I like each PR to sort of

Atomically describe what that small piece of thing is doing which also makes it easier for me to revert changes and like figure out you know oh I shipped a bug and it's just it's here it's not in this 40,000 line PR where who knows what landed in there.

Yeah, so, well, specifically for Grokbot. So Dune is the sort of cheeky code name for the architecture that we've built for Grokbot. The CI looks pretty annoying because there's checks for everything. So like literally I have, well if you've written any React for example, you know that one of the biggest foot guns in React is useEffect. So in Dune and in Grokbot we've banned useEffect.

So dune is just the the mental model of what dune is uh you can kind of think of it as like Next.js for uh electron apps and it's designed for agents to write uh and it's like custom for you know our agent powered applications um so the CI checks are very like specific to that like you know don't use useEffect it's it's it's banned like ci will fail uh and yell at you we have like some of the more interesting ones that people might raise eyebrows is like I actually ban code comments as well uh which is very interesting um but I've noticed that 99% of the time agents just write code comments that kind of describe some historical thing that is actually totally irrelevant to the code um like it will often say like you know oh Lauren said you should never do this and it's now in a code comment like what like why what that was I didn't say that as like a durable you know global rule i just meant like your this pr sucks and you should change that part agents don't really understand us that well surprisingly uh and or they kind of assume too much and they kind of do things in like very stupid ways so like yeah we just ban everything everything you can imagine like the agents are bad at we ban uh so one example that we actually suffer a lot in the Agents Window is we have uh you know if you've used Agents Window you've definitely seen performance issues and you know we're constantly trying to fix them uh but it's like a it's a never-ending struggle because there's so many pull requests that get merged every any one of them could just regress performance or stability or reliability uh you know the Agents Window doesn't have this architecture yet i plan to do bring this learning back there and kind of refactor everything there

But it just regresses super often because there's just one example is like we have very poor isolation between processes. So like on Electron, you have a renderer thread that renders your UI but you also have like a main thread that you can run other code that doesn't need to block the renderer. But we do a poor job of separating those things and so oftentimes you just accidentally have code that gets pulled into running on the renderer thread.

And then all of a sudden you're competing with the renderer that, you know, that has a very, if you want like 60 FPS, you have to every frame that gets drawn has to be done in 16 milliseconds. So very, very small, you know, deadline for frame. If you want, you know, a very smooth product. And when you start building, bringing in, accidentally bringing in, you know, things that are like very computationally heavy, or they have a lot of IO, So then you just get into a lot of jank.

Your FPS really drops. You start losing frames. You get long tasks that take more than 16 milliseconds, and you just get this really choppy experience. So all of those patterns that we've learned basically building Electron apps, we've encoded into this framework, and it becomes like a hard failure.

So in Grokbot, we literally have a directory called electron-main, electron-renderer, And we have import CI, I guess, where we actually check the dependency graph to make sure they're not accidentally importing code from one directory to another. So that's enforced by CI. As well as Bugbot, which is our, which cursor's code review tool that runs on CI. you know, in our AGENTS.md. it's everywhere.

Like, so I, I, I, I have this thing here where I talk about like, you know, like there are multiple layers, I think, for building a good codebase. Obviously the codebase is one where if you have an architecture like this, where it's extremely strict, you know, the, the, the way to build features is very conventional. That's like the strongest, strongest level of enforcement because agents just love to copy existing patterns.

So one example of this in Grokbot is like we have these concepts called like a feature and we have entry points and transcript cards like, oh, you know, the cards that you see in the chat. These are all like nouns, I guess, in the framework. And so there's a very conventional way of creating them. And so like a feature is all in a single directory as an example.

And so all of the code that contributes to that feature lives in one directory so it's all co-located in one place makes it super easy you know agents don't have to like uh grep around and try to figure out like where all the things are it just looks at the feature and like oh okay i'm working on the onboarding feature in Grokbot uh i'm just gonna work in this directory and for 80% of the work it's mostly just very encapsulated there.

But it's designed, again, for the dumbest agent. You don't have to think.

One of the key principles I have for this framework is the shortest path is the best path.

Because that plays exactly to how agents love to write codes. They like to take shortcuts, really. They'll find the quickest way to solve the problem. So why not make that the best way to solve the problem? So I probably won't get into all the specific details.

And this framework is really more of a collection of ideas and principles rather than something that will open source. You can screenshot this, I guess, if you want, and tell your agent to build something like this for you too.

Yeah, but it's really all about the layers. You know, like the codebase is one part with features and directories and, you know, import or blocking import dependencies that shouldn't be imported. And it all enforces that and static analysis. So like there's CI checks. We have a lot of lints for bad patterns that we observe. Compiler diagnostics.

There's also rules and Bugbot which are I think like three four five are more soft right these two actually make make CI red right so that you know there's a hard constraint where the agent can't just write crappy code for rules and skills and Bugbot your agents can still forget right you You can still, or it may not always consistently apply them. So I like to layer them, but I don't like to rely on them as the only source of enforcement. Because it's very, very soft, right?

And if you only have rules and Bugbot and skills and a style guide for your code, you will, it's only a matter of time before your codebase looks like complete trash. I'm sorry to say that.

But uh i definitely recommend yeah like you know investing in you know things that can be hard enforced right and this is why you know maybe the choice of tech stack that you use is also very important um like i think for example rust is sort of making you know it's like getting super popular again uh because the compiler is so strict right the compiler enforces so many different things you know there's a borrow checker that you have to appease and if as long as you make sure your agents don't write unsafe code blocks, you can more or less feel somewhat confident that if the code compiles, it probably works and it's good.

But you'll see, it gives you that level of trust and confidence that you as a human engineer no longer need to go and check it yourself. You know, you rely on code and static analysis to actually make that a lot smoother.

And I guess the worst part, the worst place to be in is if you are stuck in code review land, where you actually enforce all of the constraints, the invariance in your codebase by literally the human person saying, you know, reading the code and like, okay, you should not do this, right? Every time you have to do that, you should consider that as a code smell, like a anti-pattern.

And you should say, okay, instead of me commenting on the PR, how do I turn this into a hard rule, right? How do I turn this into a lint rule? How do I turn this into a CI failure? Or how do I even even... categorically eliminate this problem entirely?

Sure. Yeah, I feel like that's where I am, to be honest, is what you're describing right now, which is that I don't have all of these rules. So I have some things to go do after this session. In terms of being able to scale my agents, I'm definitely on maybe a couple of parallel ones locally stage, so like two to three locally. And I'm sure most people here are on the same. So yeah, I know we only have a couple minutes left. Lauren, was there anything else that you wanted to highlight?

I think that's a really good point. I mean, like, obviously, you know, I work at an AI lab where we have unlimited tokens. So I definitely cannot say that, you know, this is something everyone should do in the exact same way that I did it. I think it's possible to get to this point without, you know, breaking the bank.

But you know if you're like an engineering leader or you know you're you have a startup that you lead um i think to me it's a question of roi um and it's like uh yes you spend a lot of money on tokens in the upfront stage you know like refactoring your codebase is going to take a lot of tokens uh adding all these things uh is going to take a bunch of tokens but if we're heading to a world where agents are writing all the code and you know you want to be very lean right you don't want to have to hire you don't want to be you don't want to become like meta right like i mean like in terms of you don't want to become a 10,000 person engineering org because i mean that's a cool problem to have but also you know you have so much overhead there's like planning you know like you it's it's uh personally i i wouldn't uh it's not super fun but um i think you want to stay very nimble right and you want to you want to be like agents are all about allowing you to do things that you couldn't do before that's really to me like the value of agents you know it's not just spending tokens on every single little thing but um to me like the thing i couldn't do before is like enforce this level of constraints in a codebase by myself right like i'm just a single person you know uh it would have taken me years to build this framework uh and do all the refactoring and test everything myself and verify you know like run imagine if there it was just me right in in pre-agent era just like running you know it would take me so long right and my salary is pretty high right like so you know the the question i think an engineering leader might have is just then you know like what is there's a trade-off of do you hire someone to do this or do you spend the tokens to set up a codebase so that even the the most naive right the dumbest agents can do a good job and when you actually get to this point like even agents that are not you know fable size do an excellent job of writing code and this pays a lot of dividends as well for me and personally where I've empowered not just myself but again like PMs, designers,

Engineers who are not familiar with Grokbot to just contribute in a way that is sustainable.

So I think yeah it's definitely like a trade-off for sure you know like nothing is like free for sure and tokens are pretty expensive uh but oh actually uh i i don't know how many of you have seen this but we actually announced Grok 4.6 today so very exciting finally out um so yeah Grok 4.6 would be like a great it's very very smart uh it's really good on the on the benchmarks uh and it's the same the tokens uh well uh hopefully i'm not saying this incorrectly but uh i believe the cost per token is the same as 4.5.

So you're actually getting more intelligence for the same cost. I think this is an area that Cursor and SpaceXAI try to really optimize for, like that Pareto frontier of cost versus intelligence. We don't necessarily want to build the biggest model ever because that is extremely expensive to run. It's really about how do you find that sweet spot, right? You don't need a giant model, but it's just super smart, right? And it's not very expensive for inference.

Um i think this is where grokbot has been actually exceedingly powerful uh where so before grokbot like you know uh obviously cursor only had cursor like we only had Agents Window we had a cli we had an ide and these are really like power user tools right like they're designed for developers so it's very very developer centric you can do knowledge work in them but it like the ui is not really optimized for that so we actually didn't really have uh well i think like a lot of people like you know gtm product like they might have used cursor uh to do their work but it definitely wasn't like a delightful experience for them um i think now with Grokbot,

It's become, Grokbot is basically, like, the cursor moment for people who are not in tech, in my opinion. Like, it's, like, it's, like, a very, very accessible way to use agents in a very comfortable, very familiar interface. It looks like iMessage, and it's very fun, too. You know, you can give your agent a fun name.

You can have, you can kind of do orchestration in a very, like, natural way where you can sort of, you know, each agent is like a person, and now you've got a team of agents working on, you have one agent per account that you manage, as an example. Or if you're a PM, you can have an agent that summarizes all the work that Lauren did last night, and then now you know what I did, right? So I think our PMs are leveraging that a lot, and they're shipping code too.

So, you know, oftentimes they will just say, oh, here's a bug, I fixed it, can you look at it? And then I'll go review it, and actually, it's just perfect.

I'm like, okay. So I think that shows that the Dune architecture is holding up, right? All the really strict constraints allow people who are not experts in engineering to contribute at a high level. So I feel like I'm already seeing that pay off a lot where designers and PMs are just able to ship features directly. And that just makes the Grokbot team super fast, right? where we can ship so quickly.

Yeah, that's awesome. We are at time. So I guess, Lauren, if folks want to support you, maybe go try out Grokbot, try out 4.6 and, you know, provide some feedback. But yeah, this was awesome. Really appreciate you taking the time. Thanks everyone for all the messages in the chat. Lots of good questions. I know we didn't get through everything, But as I kind of said at the top, way more questions than we could get through. But yeah, really, really thanks. Thanks for joining.