Our APIs Are Still Bad: Rethinking APIs for the Agent Era with Sterling Chin

Sterling

Yeah. Oh, yes. I have I relocated to the Bay Area the prompt was that I wanted to be at the center of everything, I feel like San Francisco is Florence in the middle of the Renaissance. We're in the middle of everything AI. And I've been coming to San Francisco for every three or four weeks when I was at my previous employer, Postman. When I was at Postman, I was here every three or four weeks doing stuff, working with people. And I've seen the rise, and the flourishing of what's happening here in San Francisco, specifically around AI. And so I'm like, I got to be here. And so, yeah, I took the leap. I am East Bay.

Brandon

So, yes, I'm not in San Francisco. Okay, got it.

Sterling

But I'm even further east. I'm south of Walnut Creek. So in East Bay, you have Oakland, and then there's a little Oakland Hills. I'm on the other side. So, like, I don't actually. All right, do you.

Brandon

Are you on, what's the public transportation? It's not BART. When you're on that side, what is it you take? Is it BART?

Sterling

Mm-hmm. You take BART.

Brandon

Okay, so how far are you from the BART stop to get to San Francisco? Or is it like you have to drive?

Sterling

I have to drive. It's about 15 to 20 minutes, depending on traffic, and then it's a 45-minute right into the city.

Brandon

And then you can go all in. All right, good. So this is, so for everyone that wants to move to San Francisco, this is a real-life look at what it means to live in San Francisco. This is like, because I feel like this is everyone I know except like this very, very, very successful people. They kind of live, they often live like in the city proper. And then everybody else is like, well, I'm in just, they say just what you said. It's like, well, I'm in this part. So good. Well, what, so I guess the natural question is like, and I guess maybe already alluded to it. Do you, let me ask you this way. Do you miss anything from Houston? Is there anything that you're like, man? Yeah, that thing in Houston was good.

Sterling

I miss... The... I'm not a Texas native, so to be fair, I only lived in Texas for four years. We moved there in 2021. Previously, I was in Utah. And in the four years of living in Texas, I fell in love with a few things. And it's true when they say everything is bigger in Texas. There's just, you have to be there to experience it. I miss H-E-B, which is a grocery store. You will never under, any Texan you ask, when you say H-E-B, they're like, I get it. Like no other grocery store comes close. The other thing is I miss Whataburger. I miss a good, just a good Whataburger. Like you could be like, just they're big. They're good meals, good burgers. Those are two things. The other thing I miss, I do miss about Houston is I really miss the culture. The museum culture was amazing. I get it here. You know, I get it here in the Bay Area. There's a lot of great things happening, but, I, there's, yeah, there, there's some things I really do miss.

Brandon

You know, the food take is pretty common, but like kind of saying that Houston has more culture in San Francisco, that's a little bit of a hot take. I'm good. Maybe we'll get some listener feedback on that. And I was like, I don't, I know what you're, I know what you mean. I don't think you're wrong, but I'm like, it's not what some, most people point out. So that's good. Well, what about, so, you know, you said you wanted to move to the mecca of AI, which you are. So, you know, is it, is it as advertised for like, you know, you just go to, you're not going to ATB anymore, but you go to the grocery store to get some ice cream or something. And the cashier's talking about the new frontier models. Is it that level of indoctrination? Everybody is like, yeah, we got to get on this new Claude thing, whether Uber driver or the cashier or your wait staff. What's happening out there?

Sterling

It's not to that level yet. We're definitely ahead of the curve when it comes to random conversations happening. I mean, I can go to any coffee shop and you hear every other conversation is about AI. And it doesn't matter where you're at, even in San Francisco, East Bay, anywhere in, you know, like where I live and like south of Walnut Creek, everywhere we go, it's like you can't, you're exposed to it. It is...

It is everything i kind of expected um but it's also causing, like there there's some really big pendulum swings happening it's like everything is ai ai ai ai ai and like i was at my dentist i was at my dentist talking to them and we started a whole conversation about that, um and we started talking they said oh what are you doing so i work in ai and she just went on a rant and we started like talking, it's you can't you can't help but not be exposed to it but you see kind of that pendulum it's like you've got the ai accelerationists who are like this is the greatest thing ever and then you've got the other side which is like, i don't want your ai like i don't want it i don't want any more of it like can i just do my job and can i can i just live my life so, it it's i i think it's a really good combination like i i start i'm as i'm watching the cultural swings happen. I think this is what we're seeing across the nation, across the United States, across the world, is like, it feels like the Bay Area is kind of a microcosm for, and kind of maybe the canary in the coal mine, as other things are happening outside.

Brandon

All right. Well, good. Well, that's good to see. You truly are at the Mecca. We'll see. Well, I guess we'll know the next time I go to the dentist, I'll see if the hygienist or my dentist wants to talk about AI when they're like, when I'm getting there. I have this vision of you, like, you know, with all the instruments. And they're like, well, what do you think about this new anthropic thing? So, well, listen, we had you on last time, about 18 months ago, I think, if, I don't know, something like that.

You know, people can fact check that. But at that time, we were talking a lot about MCP servers. That was, like, relatively new. It feels kind of quaint now. I mean, it feels like that's just, like, you know, like that's been with us forever, right? And, you know, your big thing, I think, was talking about how APIs really were the, you know, the way that agents and AI was going to interact with everything. So, why don't you kind of give us, like, a little take here? Because before we get into your new thing, like, sort of like, what have you been thinking about over this last 18 months? And what kind of prompted what problems have you been seeing? And what kind of prompted you to, to go out on your own and start building something new?

Sterling

Yeah. Well, I still hold on to the fact, oh, my dog's making an appearance. Everyone say hi, Penny. I've, I am, over the last 18 months, with how much I've seen about MCP, I still see a lot of major problems with how it's being used. And more importantly I see a major problem happening still with APIs and that's the space I was in 18 months ago I was at Postman for four years now I'm on my own the APIs.

The original talk that I wrote at Postman was you can't do AI without APIs and MCP is right there like every tool call, you know, are basically API calls. The problem is no one is reimagining what those APIs should look like. And so what a lot of these larger corporations are doing, and a lot of companies are just slapping an MCP onto their existing APIs and saying, great, we're going to get adoption. And not rethinking is like, hey, our APIs are externally facing APIs, like agents were like, AI was never meant to consume our APIs. And it's.

You have your API spec and you've got your docs all over here and everything is spread out across multiple platforms, multiple places. And as a human, as a software engineer, I would go out and I'd be like, yeah, I know. Like, I want to use someone's API. I'm going to go grab their API spec and maybe open API spec and I'll load it up and I'll test it. But then if I hit an endpoint, I don't know, I'm going to go to the docs. I can do all of that and that's going to take time. an agent, like any coding agent or any AI agent goes out there and you give it, you know, that and it's like, hey, I don't know how to use it. Or it's like, I still need to get an API key. Or there's still like, there's not enough information. The agent doesn't have enough information at its fingertips to be able to do its job. And we would call that as ourselves, we call that tribal knowledge, right? Like I have built up as 10 years of software engineer. I would spend 10, I know, like, if someone gives me an API, I know exactly what I need to go do to go get that API to work. An agent doesn't know that.

And I'm using knowledge that I've, you know, when I used, you know, last week I was using Twilio's API for an episode of Code TV.

And I was using Twilio's API. I'm like, I hadn't used it in three years, but I already knew some of the gotchas because I had used it in the past. That is something that I, Like when I handed it to Codex or, you know, when I handed it to my coding agents, they didn't have all of that information at their at its fingertips. And so it borked at me. It's like, I don't know what's going on. So the problem, I think still, and this is a problem space that I've been in is like our APIs are still shit, like across the board. We have to rethink the, we have to, I'm, I, my focus has been like rethinking what the internet and what the world looks like in an agent first world. Like what does the agent experience look like? Um, and that's, that leads me to where I'm at today, but yeah, I was, I was, I was struggling with that 18 months ago. I'm still like, I, I consult with, with companies. I, I work with, I talk to people all over the world, um, still, and everyone is still struggling with How do my APIs really, like, they're saying, my MCP isn't getting adoption or my MCP isn't great. It's like, yeah, but.

You're putting a Band-Aid on a wound. Like, you're just trying to cover up a whole slew of problems. Like, we, the good companies that I'm seeing that are doing this are throwing, everything out and building from the ground up. Even though they may be hitting the same databases, they may be hitting the same stuff, they're building from the ground up and rethinking their entire API strategy.

Brandon

I like it. It makes sense. So it seems like you've been frustrated, and I guess maybe that's a place to dig in a little bit. It's like, why is MCP? MCP in some ways was supposed to solve this, right? Was supposed to say, it's like, Mr. AI, like, here's my MCP server. Query, Mr. Agent, rather. Here's my MCP server. You can query this MCCP server, it'll tell you all the things it can do, and you should be off and running. So why has that vision not come true?

Sterling

We have to think about how long it takes for standards and new protocols to really mature. And MCP, like I was at AgentCon and MCPCon last week that hosted by the amazing Linux Foundation and the AIF. I was there last week in San Jose. Spent two days listening and learning from like AI leaders and MCP maintainers. My biggest takeaway at the end of the entire conference was that MCP is starting to get to a maturity level, and the agents are starting to catch up to it so that we're getting to a point where MCP really is going to be solid. I think we have to remind ourselves that MCP was, it's not even two years old.

Right? We're still, or we're coming up to two years old. Like it's still early on. And how long did it take for JavaScript or open API or any, you know, GraphQL or gRPC? How long did it take for, for a lot of these, like a lot of these standards to really flourish and become like, and, and, and mature into a point where it was going to be, it was going to be better. I saw the maintainers I saw DSP get up and David who was the creator of MCP I saw some of the maintainers get up at some of the talks I think her name was Kathleen.

I missed her name but it was fantastic they're talking about where MCP is going in the future we're going to have long running they're talking about long running MCP servers where, you can ship a tool and it's like a web socket. It's like, this tool is going to be going and running and we're waiting for that response to come back from the tool so the agent that's manning or using that MCP server can continue. These are things that you have to build on and something as I'm learning, as I'm building my own company, it's like, I want to, have these grandiose ideas, but you got to start from the very beginning, start very small, and then build on top of it. I think that's where MCP is right now.

It's I think if I'm more bullish on MCP and where the future of MCP is going now than I was a year ago. And I think it's a huge part because of... One, we have a governing body, a neutral governing body. It's not owned by anybody else, right? It's not, you know, Anthropics or it's not OpenAI. It's the AAIF and the Linux Foundation are the neutral governing body of MCP, which means not one company is going to, like, hold on to and force us into a certain type of standard. And so it opens up a lot more possibilities. And I think that's where with MCP, what I'm starting to see is like, okay, we're starting to gain more and more momentum. But it does. It honestly feels like, I talk about time dilation all the time. It feels like the last three years have been 20 years, right? We have to think like, I haven't written code. I haven't written like handwritten code in a year and a half.

And if i look at it five years ago um or right before i joined post right before i joined postman i was a tech lead and engineering manager at a company called podium, and i and that was the last time i shipped production like code to production but it was only five years ago, and in five years i've gone from you know i've had all my own side projects and i've and i've written code i've we've gone from like all of that to, an entire you know look at the world like we've gone through pandemics we've gone through pandemics wildfires all this all the crazy stuff that happened in 2020 and then now we have ai and i'm not writing code anymore and i have age i have a slew of agents, you know that are doing a whole bottle a whole load of stuff for me doing all this work for you Yeah.

Brandon

Yeah, yeah.

Sterling

So I think part of it, going back, is like MCP is a great protocol, but people are starting to really get into the understandings like, The other thing is, a lot of companies were still dropping, like, Twilio. I know this because, and I'm not calling them out because I like their API, but I was using their API, so I'm, like, this is very fresh on my mind. They have 197 APIs. Externally public APIs that I can hit. If i drop an mcp tool on if twilio and they didn't do this thankfully but if but if i was if but the overall perspective of a lot of companies is well let's drop an mcp server on our on our entire api spec, now you've got all these tools and you've got really bad names and the tools don't match the tools and what they do don't match what they're capable like it just, that is a problem and I see a lot of companies, like the companies are doing well. They're like, all right, let's consolidate our APIs. But still, it's like 197 APIs. Like, come on, let's, let's reduce those a little bit. But I think that's part of the maturity level

of MCP and where we're moving towards. We're moving in.

Brandon

You seem pretty bullish on MCP, which is good. Because I was going to say, I feel like, I don't know if there's like a big backlash, but you know, I like, I follow Cloudflare pretty closely because they have a lot of free services that end up using because I can just make all my free apps and stuff. Don't tell anyone Cloudflare that. Hopefully they're not listening. But, you know, they have done a bunch of different posts at different times around, like they had a post a while back was basically, it's like, I'm paraphrasing, but they said something like, MCP is really bloated. We just started using the API and we, whatever, reduced our token usage from like, whatever, from like by 80%. And so, you know, I feel like that's, and even in my own like little usage of it, like sometimes I'll be talking about, it's like, well, do I really want, the MCP server or do I just want an API key and just turn the agent loose? Just be like, agent, you know, here's the API key, just go read, read the docs, and then just do your thing, like, and just, you know, get it out of the way.

So I don't know, I find myself doing that more than I am, I actually need to go, you know, I actually need to look after this recording to be like, how many MCP servers I have actually configured? First, how many just API keys do I have? Or I just let the agent do its own thing. So I don't know, do you have any thoughts or any take on that?

Sterling

I do. So, It's not a one-size-fits-all, and you can't just drop MCP onto everything, right? So my company that I'm building, I have an MCP server and a CLI. They sit next to each other, and they run very different purposes, and it's very different for when an agent is going to use my company, when they're going to use it, this tool. The MCP server is still necessary to do certain things. Now, so for me, I think there's a point where if you have an autonomous agent that's going to be going and doing something and you want them to be.

You know, you can't just like, again, Twilio, going back to that side, you can't just drop 197 APIs onto a Twilio MCP server. But if I want an MCP server, if I have an MCP server, the way I built for myself or a Twilio MCP, and it's like, hey, agent. You know, maybe I'm at a dentist because we were talking about this before the show a little bit. But maybe the dentist wants to go and they have a personal agent, or a company agent and it's going to go say, hey, go send all of these text messages to a bunch of our clients and invite them to, you know, get a new cleaning or whatever, right? That is like, that's a perfect example of when an MCP server should be small and where an MCP server could be really beneficial. Is it's like very small, dedicated, but it's still a tool call. And the agent is not going to, it's not something that is programmatic, right? So like the reason why I use CLI or the reason why you do something with your agent with an API is you want something a little bit more deterministic. But when the agent has a choice, you want to give them the tools in front of them to let them make that decision. And that's why MCP is great.

But we, again, we talked earlier, like there's this pendulum swing everyone was like mcp mcp and maybe we dropped their entire api spec onto mcp and like.

Done i solved my problem and i and and i'll be honest like i was in that camp for a long time i i built that like i i would drop an entire open api spec and it's like make me an mcp server for this so my agent can run it, and i've gone to the other side and it's like we need to be very careful it's like good architecture yeah i like that, and you have to architect your apps in a way that works not just for.

Works for you and works for your agents. And it's like, why do you use TypeScript over JavaScript? Why do you use Python versus Rust? You know. The standard stack now is like, if you go into any major corporation or, I mean, hell, any company, like they don't have just one stack. A lot of them are running different, like multiple coding or multiple, you know, protocols, multiple languages.

They're running a lot of different things because you, as a good mentor of mine once taught me, you pick the right tool for the right job so MCP is the right tool for certain things CLI or APIs are another tool, that are really good for other things and you have to be able to, and that's where our perspective, like my perspective but our perspective as engineers, engineering leaders we need to be very careful when, we are trying, when all these new shinies come out, we want to like oh we're going to do all this no no we need to be very careful it's like look, we pick the right tool for the right job we make sure it's like is mcp is this really a use case for mcp and i think again going back you know i i sat down with with people that i know at the linux foundation and on and and the aif, um thursday friday last week and i was, like when when i was sitting there talking to him it's like they're all saying the exact same thing They're not, even the Linux Foundation isn't saying MCP is for everything. It's like, no, they have MCP, they have A2A, they've got, I think they had like six new projects that joined the AAIF over the last six months or something like that.

Brandon

Uh-oh, that sounds like a lot of reading. Sounds like we're going to have to do a lot of catching up if there's six more.

Sterling

Yeah, yeah.

Brandon

Okay, so I think you made a good case there, but I do, as you were talking, I was thinking to myself, so MCP is more like, it's like, because the place when I do see it used well, and you've seen these improve a lot, it's like, you know, for all like, whatever. I'm looking at Claw Desktop, and, you know, obviously OpenAI has their own thing, where they've kind of like tightened up the MCP server so they interact with like Google Docs and stuff, whereas like the first set that came out were just like, it's kind of like you said, it's like, they did everything in the APIs, but they kind of did nothing. Like, nothing really kind of worked when you try to do it. But it seems like, over time, those MCP servers have gotten better. Like, it's like anything else. To your kind of overall message here, they've sort of figured out, what are agents trying to do? And how do I make that really easy? Like, they can create a Google Doc. And now they can go back and they can edit the same Google Doc. Whereas before, it was just like, if you had to make a change, it's like, well, the Doc has to be started over. Like, everything was just very, like, I don't know, kind of weird. Right?

Sterling

Yeah.

Brandon

So, okay, so let's, so now you said it at the beginning, and so you're on a mission to make these APIs better. And you mentioned like tribal knowledge. And I think everyone listening just relates to that. It's like, if there is an API, there is tribal knowledge. It's like, oh yeah, never call it that way. Or like, I don't know, you need to add this parameter. I'm not sure what it does, but it just works, right? So there's a lot of that out there. So I know you're on a mission and you've created your company, which I'm going to attempt to say correctly Sonner and it's spelled, it's S-A-N-N-R so I don't know hopefully everyone you can look that up so so give us the elevator pitch what are you trying to accomplish and like what do you have out there with Sonner.

Sterling

Yeah, so Sonor, by the way, is Old Norse. So it's Old Norse for true. So a little bit of historical background, but I'll give you the 30-second elevator pitch of Sonor. What I'm trying to build is a verifiable API context for coding agents. And it just happens that API context is a very specific niche type of memory system for coding agents. And what we're doing is, so Sonner is, my tagline is, it's an API client built for coding agents that remembers. And the goal here is anytime we have, so we used to have this tribal knowledge. I mean, we used to, we still have it. You know, you go into any dev team and as an engineering manager, I remember spending, I hear that so-and-so was leaving, you know, or one of the people on my team was leaving And it was my job. I was like, there's a brief happy that I'm happy to see them growing and moving.

Brandon

Like, congratulations, where are you going?

Sterling

Congratulations.

Brandon

And then you're like, okay, you've got to transition.

Sterling

I've got two weeks, and he's been at the company for four years. I don't know what the hell's in his head. I've got to get him to write all of this stuff down and transfer that tribal knowledge from his brain into something. And so we'd have, you know, and you do this. Ask any engineering manager. This is not a new thing. They'd write that, you know, you have them write, you know, they're off-boarding and it's like, all right, you're going to spend the next two weeks writing. You're not writing code. You're going to write all the things that you've done, where all the skeletons are, what, you know, what are we missing, all the things that you do that no one else can do. So we do that and we have, I mean, whole companies were brought, built up on the idea that they could solve the tribal knowledge problem. Well, now as engineers, I could spin up 12 or dozens of coding agents at any given time.

Every time that session ends, everything that those agents learn in that session disappears. And so what Sonar's solving is we're taking all the things that every single session that those agents learn, and specifically around APIs and the context around it. We are storing that, and those learnings now get transferred to all subsequent sessions that are going to touch that. And so the difference what we're doing is I'm doing, I'm adding the provenance that goes with that API context. So it's the who, what, where, when and why of an API. So it's I'm not trying to replace open API spec or any of the specs. They are amazing. They're really good. And we'd still need those open standards like that. So like an open API spec. But what I do is Sonar can take your spec. And because I have an API client that lives on your machine, it's just written in Node, it saves the coding agent from running curl commands all the time. I can see the, I can ship the request, I can hit the request, get the response, track everything, grab all the hotel, grab all of the telemetry around it. I say we, the agent can, Sonar can, and then we can store all of it and say, hey, For instance, Twilio last week.

I'm gonna say this, because their docs, so I grabbed their OpenAPI spec, and I grabbed their docs, and I put that through to my agent, and I said, hey, using Sonor, go and build me an entire, give me all of the API context around it. And it found mistakes. It said, this endpoint should come back as a 400 unauthorized.

Based on the docs and based on the OpenAPI spec. And the finding was it came back as a 422 unauthorized, not a 400 unauthorized. And we're like, wait, those are different. And so now what we're doing, and again, that's where the tribal knowledge for us comes around. It's like, hey, I know that maybe. Maybe I used it. And so I could have turned around and I could tell the next engineer. It's like, oh, yeah, yeah, yeah. You know, you hit this endpoint. It's going to give you a 422. Don't worry about it. You're fine. Well, now that coding agent captures all of that learning, puts it into a structured memory system, and then that gets passed. So, and it, by the way, it lives in the code for the next coding agent. So then next coding agent, whenever that coding agent touches that API, immediately has access to all of that knowledge. And they can say, oh, well, when was this last verified? When did we last hit this? Maybe it was two hours ago. maybe it was two weeks or two months or two years from now doesn't matter,

we can now go back and we can validate it verify it we can diff it um and it's that's the problem i i get so passionate about this because this is like this is the kind of the problems that that that we're struggling that that i think we're all trying to solve for, and it's like oh like, All these companies have come up trying to solve travel knowledge. And I think I've, well, I've solved it.

Brandon

And you're on your way to the site. Okay, well, let's dive in a little bit more, because I want to make sure I fully understand this. Okay, so when you say API context in this context, which is a weird sentence, but when you're saying that, what you mean, or you tell me what you mean. But I'm here, you're talking about, so like, when I think of that, I think it's like, okay, I have an API key. I have an endpoint I'm hitting, and then I've got some specific information about the query I'm making. Maybe it's a customer ID I'm querying or something like that, right? So when you say, am I understanding that right? When you say API context, what do you mean by context in the definition of what we're talking about here?

Sterling

Yeah. So traditionally, you'd use a traditional API client, and you'd go out and you'd put the headers together and you put the request body together and you'd ship that off. And then you'd get a response and you'd get all of that information around it. But oftentimes, you're also like learning things about that API. So it's not just the request headers or, you know, the request body and the response. Sometimes you're looking at, hey, you know, when I'm doing something around this API, I want like maybe there's pagination and you don't have exactly how much, maybe you can't you know the open API spec doesn't tell you how many you can paginate how many are coming back per page and so it's all of the, extra knowledge around that API that you're not going to get from the docs or the spec and you're only going to get, from running that API and running it over and over and over again, in a safe environment.

Brandon

Okay, I get it. So as you're running it, it's like anything that's quote-unquote not documented or observed, it's like anything else you then can then make a note of. It's like, oh, pagination being one thing, or the code is slightly different, set 400, 422, or this didn't work, or this timed out. Okay, now, okay, so that makes sense to me. Now, what's, because now I do think on your, one thing I want to challenge you on a little bit is like, okay, so on like a regular coding agent, when I'm just like in Cloud Code.

And like I'm kind of like getting ready to like finish the session, like I've done something good, at least in my definition. I'll often say like, make a checkpoint here, but I'll just be like, and I'll just tell the agent, like update everything that we've done here. So like GoTouch, Cloud MD, Agents MD, Memory. I don't even, honestly, I don't even know what it does. I just tell it, like, write, like, update. I kind of do. I just say, like, everything we've done here, let's just update everything to the state. And obviously, I'm not reading any of this, but like, and many times, though, but it does. It makes little notes about your project. It's like, oh, yeah, I'll make, it's like kind of like you, but it's not an API. It's like tribal knowledge for the thing that we learned. It's like, okay, I'll make a note of the decision because sometimes at the end it'll tell you, like, I made a note that we discussed it. So we made this decision going forward. I'll do this, right? This being, I mean, this is, of course, the agent. We're not trying to make it a person, but it is a person. Right? So, okay, so we do that. So that's kind of how I think of, like, just my agent interaction.

So now, let's go back to what you were saying before. It's like, you know, you finish your stuff and all the knowledge is lost. Like, I mean, is, some of the knowledge can be saved, right? Or am I wrong? Like, how do you think, see it working?

Sterling

It's one of the reasons, so, okay, I love this, because I use, I'm going to use an analogy here, and, I'm going to kind of shit on general agent memory companies, not purposely, but, I think the general agent, like there's an idea here that there's general agent memory and it's just going to be this cloud of data, this data lake of information. And we can continue to shove information into this context and it's going to remember it all.

But that's not how we as humans remember and that's not how AI remembers. So specifically okay Brandon you and I are in the same age range I went to I was in high school in the 90s if I asked you, you know if I said tell me about a specific memory, like like I went like tell me about a dance you went to, you know you went on a you went you asked this pretty girl out on it to go to be your date on the dance what did you wear and and if i start to ask you follow up questions what did you wear, who was like you to me maybe you don't even remember her name but you're like, you know like asking specific questions around that, you're more likely to remember that information and you're gonna be able to like retrieve that faster but if i say, tell me about all your girls you ever dated. Tell me about, you know, every dance you ever went to. Or, like, if I don't ask you something specific, it's, like, it goes into this general knowledge base and it forgets. So, to me, agent memory has to be very niche and siloed. So, that's why I'm focusing on API context. API.

APIs can be tested. They can be rerun over and over again. And so I can test against what I, what was observed prior and I can testing and verify it over and over and over again. But. Because it's very niche, and I'm only focusing on that. So if I go to my coding agent and say, go look at everything, all the APIs we ever hit, if it's in this big blob of data, this big. Superfluous MD file somewhere, that's going to balloon really, really quickly. And when we look at larger scale operations, like large companies, Expedia has, you know i don't know how many tens of thousands of repos that they've got in these large organizations you can't just balloon over time, and so okay that focused so something that i've learned over the last 10 months, something that i so i built a memory system back in december called marvin um, uh marvin was just a memory system for a personal agent, That memory system had a tree-like structure to it. So I was remembering everything that was going on during that day was a current state file. And the agents had the ability to read and write and delete from. Then I had every day was a session. That session, was append only.

So if I had multiple agents working, they all could only append to it. And it was just this running tally of what they were all doing throughout that day. Then I would take five days and combine those, condense those into a singular report on what was happening that week. This tree-like structure, and this memory system here.

It helps agents retrieve that data faster than they could otherwise if you just drop it into this big thing. And I'm sorry, I mean, I see this, I've seen this on people on Twitter and on Reddit complain about this, is that they did these general agent memories, like they generalize this agent memories, like remember everything we did, but it grew so large that it was so cumbersome that the agent was overwhelmed. It wasn't context rot. We were worried about context rot for so long. It wasn't context rot. It was context overwhelm. And the agents were overwhelmed by how much context they were getting. If you focus down and say, I'm only going to touch API context, I'm only going to worry about API memory, then it's smaller, it is faster. Now, I'm using this and I'm using a structured JSON object here, but the agents can read that and get to what they need faster, and they're not spending or wasting tokens trying to read through a massive context file.

Brandon

All right, so the memory makes sense to me. I get your point around like, yeah, and I actually have the same problem with, you know, just doing multiple projects on my, it's like, yeah, the agent really only knows, it only knows the thing if it's in that project, right? If it's like outside of that project, then I have to like, oh, I have to sometimes like tell it to like, oh, I learned, I learned, I am the person telling it, like, go read this or something like those lines. So, okay, well then kind of get into like, how do you, like, I don't know, behind the scenes, like, how do you solve this? So like, I guess first it's like, what do I do to use this thing? I guess like, how do I like start using it? And then what's your secret sauce to like, as you learn all of these things, where are you saving it? And like, how do you how do you make sure that it doesn't context like overload to your earlier point?

Sterling

Yeah, so Sonor is designed as a dev tool for coding agents. It's not designed for a human. Humans are a secondary thought here. So unlike what you've seen, what I love about Sonor is the coding agent is the one that matters. And so working with a coding agent, a couple of things I've learned from this is that I had to put that context right where the agent sees. Agents are very... Confidently oblivious is is kind of the word that the words I've been using, they're so confident in their own knowledge in in the in the fact that they don't have they they're oblivious sorry they're so confident confidently oblivious to their knowledge gaps.

So if they don't know understand something they will hallucinate or they'll make stuff up and so much so to the fact that a lot of times coding agents don't ask questions. So my first foray into trying to solve this problem with API context was I built a great MCP server that I would capture, I would gather all the information on and I would put it here. And then it was an MCP server and the agent could then ask, oh, I'm hitting this endpoint, this auth endpoint or this health endpoint.

Tell me, go to the MCP server and ask the questions. And it never did. He's like oh well i already have the end point i know what it looks like i'm just gonna i'm gonna do it myself and, that didn't work and i ran a bunch of experiments on that then the later one i built an agent for it and i built my own agent i named her the librarian um fun fact i started as my first job was a librarian i worked as at a library at 16 years old, um it was fantastic which is why the librarian stuck here, and this agent was great i loved it like it when i would query it was like here's all the information you need about it we've run it all the things, but in practice coding agents like it was right there for the coding agent to go ask and the coding agent would never go ask, and so i out of that one of the things i've learned is that i had to put that right in front of the code like right in the code with the coding where the coding agents just where they are forced to accept or acknowledge that that's there.

So you have to push that information right in front of the coding agent and not expect them that they're going to pull that information for themselves. And so Sonar's context, so what we call it is we call it the Sonar stacks, the library stacks, right? So, I mean, there's a lot of themes here.

Brandon

A lot of metaphor here, Gallant.

Sterling

A lot of metaphor. So you got the Sonar stacks, and it's all based on the endpoint and the base URL. So you know you're gonna so open api or open ai for instance has multiple endpoint multiple base urls and so when i break it up into the sonar stacks it's all based on that base whatever that base url is um and then, it's then based on like the endpoint itself, and so you'll have multiple sonar stacks potentially for like open api twilio or open ai twilio like all the like a bunch of different, like every third-party, API that you're going to interact with or integrate with, you have, they're going to have their own kind of stacks. And there's three files that I have in there. One is the environments. So it's just some environment variables. So API keys, if you have a .env file somewhere on here, we will reference those. And in there, we can say, in production, here's the API key you need or here's whatever the auth or whatever you're using is if you're using staging or development, all of that information we store but we only link so it's never, none of this is ever committed and it's never pushed up then I have the evidence files.

And that evidence is, it's like, I can have a whole folder full of every run that's ever run. Anytime you've ever hit that API, it's stored and saved on your machine. It's get ignored, so it never gets pushed up. But now I can track, we can see all of the times that that endpoint has been hit. Then we have the library so the actual API context that API context is a structured JSON that, has all the information it has the API context there and it's append only so every time an agent hits something it's like oh going back to the append only that I learned through my other memory system, anytime it hits something we date it we add all the provenance to it so there's the who what where when and why, that is all stored and so then you can have an endpoint that has six seven eight notes and now we can start to see over time what's happened we can start to see api drift if if there is any um, and we can also track like is this verified like we've had this we've we've hit this three times we get every time we we get the same response back, Do we want to say this is verifiable or this is verified? Yeah, cool. So Sonor is designed for a dev tool. Now, because of that, you asked the question.

Sorry, I kind of sidetracked it because I get so excited about all these things.

How to install it. If you go to sonor.dev, S-A-N-N-R.dev, there's a simple prompt. Copy and paste a prompt into your coding agent of choice, and that will do the rest. Now your coding agent has a lightweight API client and all the Sonor stacks are built into it and you can start using it from there. There's no other setup. So as soon as I work across.

Brandon

Do I have Cloud code, do I just paste it into every session? Or do I have to put it in, a Cloud.nd file? Because I guess does every project then start creating these stacks, like its own little, yeah, I guess it's Asana stacks for each project. How does that work?

Sterling

So depending on how you want to do it, you can do it, you can install it globally if you want. So there is an NPM, you can NPM-G and install it globally. And then what we do is we look at your repo and we install it specifically, like we go per repo. And so then that context is stored. Any API in that repo is stored in that repo. So there's not a global Sonar stack there. You can do it. I personally prefer to be very careful on what I install globally. And so I always recommend go in if you're in cloud code or in your codex you go into the project directory itself and then install it then it stays at that directory at that repo level and nothing else but that's just, me it's.

Brandon

Just sort of like how you want to run your hygiene I think all of us have the same things I'm starting a new project, I'm setting a repo and then there's a set of things that you do, like you mentioned environment variables as well. Plug for another guest. Maybe we'll talk about afterwards. Varlocks, my favorite way to manage environment variables. So that's what I always tell people. It's like, don't write .env files. There's a whole solution for that. Whole other episode. Other people have already heard it. So we won't go into that. But I know what you're saying. It's like, these are the things I do for every project. I'm like, this is how I manage my environment variable. So maybe this would be something I would add. It's like, okay, as I'm starting a project, I want this project to start to accumulate the tribal knowledge for all these APIs. So I would install Sonar, and then the stacks would start to be created, right? And that's sort of the magic here. And then, I guess, is it like red? I guess, well, so when I install the prompt, I guess you'll have to tell me a little bit about what the prompt says.

So then, so like when I start, so after I got it all set up, like it's in, I guess, would it be in my Claude MD file or something that would say, okay, always like, like here's the Sonor stuff. Make sure to check all this before you start using the APIs. Is that, and it loads up in the context?

Sterling

Yeah, it's a point. So in the Agents MD file, so depending on, depending on whether you use Claude or OpenAI or whatever, there are pointers to everything. So if you're using, if the standard now is you put the pointers in your agents.md file.

Brandon

Okay. All right.

Sterling

And then it loads it all up. It'll load it all up. We have an MCP server and a CLI. Then anytime your agent is going to hit an API or, it's going to push and go through the Sonner request. So Sonner request is that API client. It's written in Node. Zero dependencies aside from Node.

Brandon

So then I guess the issue, I mean, I know you're saying it's probably going to work, but then, okay, so the developer has to trust you. It has to be like, instead of just making these APIs, I trust that Sonner's going to make the request that I want. It's going to observe and get all this information, and it's not going to mess anything up. It's not going to like, I mean, I think that would be, I don't know. Is that kind of, that's kind of the, the, the contract you're making, right?

Sterling

Yeah. The contract is like, because I have zero to, I have zero dependencies on, on anything else. So this is just straight node, like a node API client. The other thing is there's no telemetry. So I, there's, there's no, there's no phone homes. The only thing I will say is that I do have a feedback endpoint that I have that you can, that your agent can submit feedback to me immediately.

Brandon

Like this didn't work, right?

Sterling

This didn't work, but it is only at the request of the human. So there's never a phone home.

Brandon

Got it.

Sterling

And that's something that I'm very careful. I don't like phone homes. It's a good thing and a bad thing, because a bad thing is like, I know I've got users.

Brandon

It's not so much phone homes. It's more like, make it easy to call me. It's like, okay, this is messed up. so make it like here just send it over just make it easier for okay so, Really, in the end here, so it's the client, it's the API client that you mentioned multiple times in Node. And then I guess, so it's really three files, just to go back to what you said before. It's the setup and configuration file. Then you said there was a log of everything, right? The log. And then the third one is sort of like, I guess that's essentially, I don't, I guess I want to see the difference between the log and the API context. So are you doing some magic to be like, like not every, like I assume everything goes into the log. But are you doing some type of logic or be like, write this to the context file when something interesting happens? Or what's the difference between the log and the context?

Sterling

Yeah, so the log that you're gonna get is really a running, so going back to the hierarchical aspect of all of this. The log is going to be where I put all of the evidence files. So if I hit an endpoint and I have a megabyte's worth of data coming back, I don't want to feed all of that context back into the agent. So what I do is I'll reference it. So that API context stays lean and it stays small. So I'm only sharing the information in that API context layer that the agents actually care about. So it's going to be the notes. It's going to say, last time we hit this, this is what we did. If a coding agent has a question and says, oh, well, what happened? Can you give me the request headers and the response? Then I have an address. I have, that's where the evidence file is linked to. Then it says, oh, go to this evidence file. And then the agent can go back. But this is where I talk about context overwhelm. If I were to, theoretically, I could drop all of that into a context window.

Brandon

Right, but it'd be too much. It'd get confused.

Sterling

It's going to get too much. And so that's why I had to split everything. And so, the config stuff is super small. The Sonor stack, where the API context lives, is really, it's small-ish. It's about five. It ends up, depending on your APIs, it ends up being anywhere from 1,000 to 5,000 lines long. But token-wise, it's structured JSON, So it's pretty fast.

Brandon

Yeah. And it's not, I get it. That makes total sense to me. Cause it's obviously you're keeping your context window smaller and it's easy to remember. So, okay. So all of that, okay. That all makes sense. And so, so what's like your best, uh, like, you know, obviously you've spent a lot of time doing this, like, I mean, maybe just as a personal success story here. So it's like, what's your best story about like, Oh, without Sonner, the agent got confused and wasted like eight hours of time, whatever doing this. And then with Sonner, like it was told the answer immediately. Like what, like what's a real life thing that you did that you were like, Oh, Sonner like saved me.

Sterling

Yeah. I mean, so last week I was on an episode of Code TV, um, with, and it was, um, by the time this airs, that episode will be out. So it was open API or sorry, open AI and Twilio were the sponsors. So I was, I was on that one. So what I did is I grabbed all of their context and over the course, uh, and I, and I fed that in and I created my own Sonner stacks. And then I went into the went into the hackathon and just built. We had 80 different requests that went through Sonor that week or that day and, I on the side I also tracked and there were less mistakes at the notes that were being used. I was using the new decisions API and it caught a gotcha and the next agent that I was running didn't hit that gotcha. So something fun. So those are the wins but, I'm an evidence-based person, so I've run experiments on this.

I ran multiple sessions, I think hundreds of sessions, and I had session one was there was an agent, there was going to be a set of five APIs it was going to do something with, and one of those APIs, I gave it the open API spec and I gave it the docs, but one of those APIs, there was something wrong. There was an incident in one of those APIs. And it was just something that was like a different mouth, like it was expecting, I think this experiment was, it was expecting JSON versus, you know, it wouldn't accept, it's expected a JSON object. And we were passing in a, just a, just a string. And those, so they learned it in that one session. Then I opened up another session, same repo, back-to-back, and gave the next coding agent the next step in the project. And that second agent in the control, with it was no Sonar, that control hit that incident 81 times. It hit it 81 out of 81 times.

Brandon

Okay.

Sterling

So it continued to hit that problem. So with Sonner, those incidents, those persistent incidences from session one to session two was only six. And that was, and so that was part, so we can see after multiple generations of like agents using this and using that context, they're, they're having less problems down the line. So they're not having to relearn something. So from a, I, I, that's, that, that's to me is my biggest, are the biggest wins is like not having.

Brandon

Right. I can see, I'd even say it kind of slightly differently. It's just sort of like anything that you find that is used a lot is going to have some compounding benefit, right? It's going to have like some exponential benefit. If it's like, if I call this API, you know, a thousand times and like a thousand different agents or, you know, have to either relearn or something versus like I give them the answer, the agency answer, then it's like, whatever, whatever the problem was times a thousand or whatever it is, you know, it can be a lot, it can be very huge. So, all right, so that, okay, so pretty cool. All right, so I'm gonna have to definitely try this. So you mentioned, I mean, I guess it's like, I've been doing a lot of things, as listeners of the show knows, I have a whole portfolio of Apple apps I've built. I was just wondering, can I use this? Could I use this for like just the local Swift? I wonder if I should use something like this for the Swift APIs. I don't know, I'll have to see if that would work. Obviously it's gonna work

well for like your traditional endpoints that are all out there, the traditional internet stuff. So, well, what about the business side? So this sounds great, a little too good to be true, which is always a great business story. But are you trying to make money? How do you make money? How's all this going to work?

Sterling

So Sonor is open source. I'm in a private beta, a private alpha right now. Kind of private public alpha, like people can use it. I'm going to go to beta next week. Once we're in beta, we'll be open sourced. Yeah. So no, no money, no money yet.

Brandon

So everyone can use it for free.

Sterling

Everyone can use it for free. The biggest, where I do plan is when we move into the enterprise, I'm working on Sonner for Teams. So that is linking multiple repos together and being able to share a shared context across repos. So. At a larger company, so my own personal stuff, our own side projects, your front-end code and your back-end code pretty much live in a monorepo. But other companies, large corporations have, maybe you have your back-end repo here and your front-end repo over here. If you want to be able to share that context, and you want to be able to share that API context across repos. And so that's the next evolution I'm working with. I'm working with Cloudflare. I'm working with enterprises on how they can share that. So So I look at this from two perspectives. You've got the API builders, and you have the API integrators. There's two kind of, and there's like that nice little separation there. Now with coding agents, we're both using and consuming our own APIs. But if I'm integrating with someone else's API, I still need to be able to share that information to them.

And so the other aspect of that is how do I, if I build, like if I'm Stripe, for instance, and I'm building an API for external consumption... Can I create, can Sonor create an API context file that works, that would generate all of it, and then can I pass that to the consumer, their consumer agent, or the consumer's agents themselves, so they can integrate faster? So that's where I'm also looking at.

Brandon

I like to say this is like single player is free, a multiplayer, when you have like a large enterprise, and you want to share things, that's going to be something people pay for. And ideally, that's going to be valuable to a large enterprise because just as we talked about before, it's like instead of just one person getting the benefits of not making the bad API call, you can have your whole organization do it. Well, it does sound like you're creating quite the corpus. I don't know, as you're just talking there, I'm just thinking to myself, it's like, I don't know. It feels like this should go into the training data of the frontier models. They could just magically read all of what you're, let's call it the observed, because that's kind of what the internet is in some ways. The internet has a lot of this tribal knowledge that we load up, right? Because I never Google search for anything. I just ask Claude to be like, how do I do this? And then it'll tell you, it'll be like, oh yeah, it says this, but do this, right?

So it sounds like you're creating, I don't know, I understand no one would want to give up their information, but inside of all these stacks to your library metaphor, it's almost the opposite. It's not really a library. It's like you've locked it. It's not public.

I guess I would say to keep maybe an idea for you It was like, I want a library card. You know what I mean? I want to almost feel like, you know, I can think of in a perfect world it would be like, I'm going to start using this API. Before I do it, go get all the stacks. Go read all the stacks about this API so that I don't make the mistake, which is what a library is, right? It's like, before I go do it, read all the books, see what's been done. And so, I don't know. I mean, probably impossible, but I just, like, my mind is like, there's going to be a lot of good information out there. I don't know. This must have crossed your mind. Like, what do you think? of that.

Sterling

It's so auth is one of those things that that's why on single player there's, you're going to have your own ideas but when you get to multiplayer and you're dealing with like i don't you know maybe my agent shouldn't have access to the you know maybe, maybe i'm on one side of the my one side of the company and i and my agent shouldn't have the ability to go and run a drop table, on another or delete you know use a delete endpoint and so yeah there is there is like the idea of a library card that i'm working on it's less at, it's less at the single player and more at the multiplayer level, but i'm still so i'm i'm, you know i'm still actually actively you know fundraising i get it, i'm thinking about it but i'm fundraising and so yeah.

Brandon

If someone's got like pony up and give you some uh.

Sterling

If i get the funds you know then i will go in and that's like i've already got a whole plan on that and a whole set of experiments built around that but um let's go you can drive Right down the Sand Hill Road.

Brandon

You know, put on a... Actually, I was like, you can put on a jacket, but you don't even need to. They don't even want to see a jacket. Just, you know, put on your run MCP shirt and be like, okay, I'm here to raise money, people. And like, this is what's going on, so... All right. Well, as always, Sterling, I think I had like 30 questions here. I think we got through 10, but we're out of time. But that just means you have to come back. A couple of things. I will get the link from you and I'll put it in the show notes for the episode that you went on. That sounds like a good one. Also, I'll have links to Sonner and all the things we talked about. But you know what? We should start with the best question. If someone wants to give you money, if someone has been so moved by our conversation and believes that, you figured it out, how should they contact you and send you a term sheet with incredible terms? How would they find you?

Sterling

So the easiest way to find me, I'm on LinkedIn. I'm on Twitter. So LinkedIn, just look up Sterling Shin. Sterling at sonner.dev is my email address. You can find me on all the socials. and Sonner.dev will also have all the links to get a hold of me.

Brandon

Okay, well, that's a good place to find them. I'm sure, go check your website. I'm sure you make an appearance at any API conference. I see you post on LinkedIn. So if you're at an API-related conference, I don't know, just look around. Sterling's probably there. He's probably got a T-shirt on. Maybe a hat, maybe a backwards hat. I don't know.

Sterling

I always have a backwards hat.

Brandon

Yeah, you can find him. You can probably find him. Just ask him around. The guy talking about MCP and Sonner. He's probably there. All right, well, really appreciate you coming on the show. As always, Sterling, thanks for coming on.

Sterling

Appreciate it. Thanks for having me.

Brandon

And for everyone else, if this is the first time you listen to Software Defined Talk, then of course, welcome. Make sure to subscribe in the podcast player that you're listening to right now. And if you would like a sticker, just send your mailing address to stickers at softwaredefinedtalk.com. I'm happy to send you a sticker anywhere in the world. And with that, we will talk to you next time.