Product Market Fit, Secure by Design, and Betting on the Models with Daghan Altas

00:00:02 Stephen Koza
Welcome to TechPod Talks. I'm Stephen Koza, CEO at EverOps. This is the show for engineers and technical leaders in the trenches, the people building platforms, running cloud, scaling infrastructure and figuring out what's next. Every episode, I sit down with somebody who's actually doing the work to get the real story. No theory, no fluff. Just what's working right now. Let's get into it.

00:00:36 Stephen Koza
Human reviews on PRs are pretty typical. One of the oldest safety nets in software. My guest today thinks there's a better way to do it, and that those are now the bottleneck. He believes that having a well-designed system can give you better security and velocity at the same time. And he knows what he's talking about. He runs product at a code security company, and he knows how unpopular that might sound in our own industry. But let's find out.

00:01:05 Stephen Koza
Welcome to TechPod Talks. This is Stephen Koza. I'm the CEO of EverOps and we do candid conversations with leaders who are building what's next. I'm really excited to have our guest on today. Daghan Altas has spent his career going back and forth between product and sales at technical companies. I won't give you the whole resume because it's long and distinguished, but he and I know each other from our days at Cisco Meraki. He created the firewall and the SD-WAN products that eventually became the standard all the way up to Fortune 100 companies. And now he leads product at Semgrep, a code security company. Before leading product he was their CRO and ran sales. And he also has graduate degrees in microelectronics from McGill and data science from UC Berkeley. Daghan, what's up, man? Welcome to TechPod Talks.

00:01:58 Daghan Altas
Stephen, you're remarkably good at this.

00:02:01 Stephen Koza
I'm double digit episodes in. So rewind and watch one or two and you might not say that. How you doing, man?

00:02:07 Daghan Altas
Great. Excited to talk to you.

00:02:09 Stephen Koza
Yeah, likewise, man. Well, you ready to dive in?

00:02:13 Daghan Altas
Let's go.

00:02:14 Stephen Koza
Cool. Well, I know a lot about your career, but most of our listeners probably not. So let's go back to the days. You were there before me. I can't remember who left first, but suffice it to say, we both spent a good chunk of our careers there. So tell us the story, man. I think you joined as a product manager. You get the credit for the MX family of products. Tell us about it. Tell us the story.

00:02:46 Daghan Altas
I was hired in 2010, and in fact, Meraki was literally walking distance from my house. And so I didn't find them, they found me, because they had a salesperson that I worked with at Dell, and they were looking for a sales product manager. And Layla, the salesperson, she recommended me. So they talked to me. I got interviewed by Hans Robertson, who's now president of Verkada, and Sanjit Biswas, who's the CEO of Samsara. These guys are titans. And I can't believe I got the job. Hans later told me that I was interviewee number 35 or 40. So I don't know what it was about me.

00:03:30 Daghan Altas
It's just luck. I got the job in 2010. It wasn't all sunshine and butterflies. We had a lot of ups and a lot of downs, just like startups. But I stayed there until the Thanksgiving of 2018. So just eight years. It was a phenomenal ride. It was just one big rocket ship.

00:03:51 Stephen Koza
Insane. Yeah, I think I left in maybe June of '19, so maybe I followed you out the door. I can't remember my exact psyche at the moment. It was becoming a proper Cisco, and it was time to go.

00:04:05 Daghan Altas
Yeah. Yeah, I felt the same way. I was ready for something new.

00:04:09 Stephen Koza
So tell us about the MX. What was the idea behind that product? And you scaled that to quite a big business. Take us through that.

00:04:22 Daghan Altas
You know, at the time, if you were to ask me this question maybe five or six years ago, I would have a completely different answer, because back at the time, I felt like 90% of the progress or the accomplishment belonged to me and the team. Now, I realize that actually 90% of the accomplishment belonged to the concept. We just rode the coattails. The concept was very powerful, and you could have plugged in a number of products, which we did, by the way, and mostly all of them did well, except maybe one exception or so.

00:04:53 Daghan Altas
But I got hired in 2010. They were like, hey, look, we're a wireless company and our customers love us. We're growing three x year over year. And every time we put these people in a room, they always say the same thing. Yeah, but you just do wireless. We want networking more than wireless. And so my boss, Hans Robertson, by the way, the smartest person I know. It was gift work for that person, honestly. So if anybody's listening to this, one actionable thing you can do is go look at his job listings. And if there's a role there, go apply for it. Because Hans is someone you want to work for.

00:05:32 Daghan Altas
So Hans gave me the box and said, hey, here's a box. This box is a wired box, as you can see, it has wires going in and out. And so we want you to make something wired. And so that was it. I was the product manager for the wired product. And so, not knowing the market, and I came from an IT company, Dell. Well, Dell does a lot, but at the time I was on the IT side of Dell so I didn't do any networking. And so we had to just go and ask a bunch of people what it could be. I would basically say that after doing this for 20-some years in product management, the secret, like, there are a lot of books about product management, but I'd say that 99% of product management can be summarized as pick up the phone and call customers, because the pattern is there, and they'll tell you.

00:06:24 Daghan Altas
Hey, we need a faster horse, faster horse. A great product manager realizes, wait, I can build a car. So okay, that part is on you. That's the skill part. But in terms of doing the basic things right, I think what we did is, there was an enthusiastic community of customers who wanted to talk to us, and we talked to all of them.

00:06:42 Daghan Altas
And so by way of elimination, we settled on these gateway slash firewalls. We first wanted to do a router. So this is where the irony sort of happens. We wanted to do a router, then realized when we were Meraki, nothing to do with Cisco, that nobody wanted to buy a router. The router market was completely captured by Cisco. If you wanted to buy a router and you didn't buy from Cisco, you're crazy. And if you were a competitor and wanted to get into the router market, you were also crazy. And so the router thing completely flopped. But we realized people were looking for this gateway, this sort of firewall thing, and nothing in the market worked. The only really good firewall product in the market was Palo Alto Networks.

00:07:20 Daghan Altas
And their distributed, sort of remote story, which is that, hey, I have 10,000 branches now, how do I connect them and keep them secure, all that story was very broken. So we found an opening there and we just kind of walked through that. Now fast forward later, the software-defined networking era just blossomed. And with that, software-defined wide area networking, SD-WAN, started to happen. And then we pivoted back to being a router. So it's never a straight line. So we basically rode the coattails of velocity and momentum, the sales force, the go-to-market power, the brand, the simplicity. And it just worked. That place was magic.

00:08:07 Stephen Koza
Yeah. Fond memories for sure. A lot of good ones. Funny stories too. We could take it that way, but I won't. So go back to trying to figure out what to build. Was there a moment when a light bulb went off, or was there a moment when you're like, oh, you know what, this is product market fit?

00:08:55 Daghan Altas
Yeah. So I think the light bulb and the product market fit are a little different. The light bulb is probably first and product market fit is probably a year or two later, once you keep going down that light bulb path. And so for us the light bulb was when we started to hear the same repeated pattern. Honestly, I can say that one area I got better at over the years is product market fit. That's something that I've done twice now.

00:09:13 Daghan Altas
The first time you do it is sort of accidental. You stumble upon it and somebody asks you, well, how did you do it? You think you know, but you don't know. But if you do it a second time, then you realize, oh, okay, so of the ten things that I remember, four of them were true, six were wrong. And so you really hone in. I would say the thing you want to do, if you're early in the product, is that you should have zero conviction. VCs call this "strong beliefs weakly held." You shouldn't say, oh, I don't want to do this, it's bad revenue, a bad deal, bad proposition. If people want to write you a check, you should cash your first five checks without asking a question, as long as you can deliver what they're asking for.

00:09:45 Daghan Altas
But you shouldn't be super opinionated. I hear these terms, oh, we want to be disciplined, we want to be opinionated. And I'm like, about what? You don't have any data. But once you get to like five to ten transactions and a pattern starts to occur, then you say, okay. Now there's a lot of fear about the local minima, local maxima. You're like, hey, maybe we shouldn't just settle on this, maybe we should keep the search space broad so that we find the global maxima. I [unclear] that. I think that you should just follow the customers' moves.

00:10:10 Daghan Altas
And so that's what we did. And I think after customer four or five, it started to emerge. The moment was like, wait, if I'm an enterprise and I have like 200 branches, I have no idea what's going on. And my way of dealing with networking problems is that I get a phone call from the store manager. So I'm flying completely blind. So once we had that problem statement, that these people were basically reacting to the people on the ground and they had to be one step ahead of it. And sure, the industry sold them a lot of solutions for centralized management, the servers, databases and all that stuff. But none of them was simple.

00:10:52 Daghan Altas
So we created something where we wanted to have that iPhone experience, where you plug in the appliance that we basically shipped to you, you plug it into the wall, you give it internet access and that's it. And then you're in control of that business. You basically can completely control everything about that store remotely through the dashboard.

00:11:14 Daghan Altas
And so that was the moment. In terms of product market fit, my rule of thumb at the time was that if any 100K transaction happens without me knowing about it, then I know we had product market fit. So for me, product market fit was the elimination of me from the process. And once I saw the first deal of that size, the first deal where I'm like, oh wow, look at this transaction, somebody bought a bunch of these boxes without me promising features or making a roadmap presentation, hey, this fits the market. And so I distinctly remember that 100K being my mark. And I feel like from there I just took off.

00:12:00 Stephen Koza
I like that. Everybody's got a different definition of product market fit, and most of them are pretty wishy washy. But how do you know? I like how you put it, man. If that happens, you know. I remember, so by the way, I'm zero for two on trying to find product market fit. But I was always the sales leader, and I suppose you're probably in the wrong job if you're running sales and there's no product market fit. But anyways, I was in a meeting with [Sanjit] once and he said something that stuck with me, which was, when you're talking to customers and you're like, we've got these features and here's the value and blah blah blah, and then all of a sudden you start to hear them repeating the same thing back to you more than once, customer to customer to customer, that's a really important signal. Maybe not quite product market fit, but you know you're onto something.

00:12:48 Daghan Altas
Yeah. That's the moment. Exactly. That's the moment you know that okay, this is going in that direction. Then after that you've got to remove all the frictions, because the reality is that there's a million reasons why the product doesn't work for the customer.

00:13:03 Stephen Koza
Maybe I'm one for three on product market fit, if you include the service provider business we worked on together. That one worked. I'll take that one. Well, hey, let's move on. So you and I leave Meraki. You headed to Semgrep. Tell us about Semgrep. What do you guys do?

00:13:27 Daghan Altas
So the story of Semgrep for me is a time machine. One of my big gripes, personal melodrama at Meraki, was, gosh, I was employee 50. But if I could have been employee ten or earlier, then the world would be different for me personally.

00:13:42 Daghan Altas
And then I was looking around for a while. At that level, you can't just switch jobs in a couple of weeks, it takes a while. And then I came across these three founders. They're also from MIT, just like Sanjit, Hans and John Bicket. And I kind of felt the same intellectual power and humility, but also the ability to get stuff done.

00:14:07 Daghan Altas
And I kind of felt like, okay, the universe gives me a second chance at joining incredibly smart MIT co-founders, except this time I came in in the first ten. And then, quite frankly, I was not very much sold on their business idea, which was going after software security. And this is year 2018. And this is the beauty of false negatives, in the sense that if you join a company where the idea is obvious, then you're going to be competing against 50 other companies.

00:14:44 Daghan Altas
So you want to find something that is slightly contrarian, and hope that you are lucky. And did we get lucky. Cybersecurity just kind of boomed, in terms of CAGR, compound annual growth rate. I think it's one of the fastest growing sectors in security. And so yeah, they were visionaries. They saw the future. And it was just a privilege to be able to join them.

00:15:06 Daghan Altas
Now, I didn't join them to be their head of sales or anything like that. Frankly, they even had a head of product, Luke, a good friend of mine. They just needed an old guy. They were in their 20s, I was in my 40s. So I had a big running catalog of all the mistakes in business that I've done or I've seen people do. And so the board and the founders felt like they needed access to that database of mistakes so that they don't do those mistakes, or they can make new mistakes.

00:15:42 Daghan Altas
And so I kind of joined them as their old guy to help them. Well, maybe we should think about this a little bit differently, etc. But obviously I learned a lot more from them than they learned from me. But that's just how it goes. In terms of running sales for Semgrep, we had a catch-22 that I think you can relate to, which is that when you're a tiny startup with no repeatable process, no pipeline, no clear ICP, you're in this complete search space, you can't hire a great sales leader. It just doesn't work that way.

00:16:30 Daghan Altas
But unfortunately, if you don't have a good sales process and a leader, it's not easy to go iterate with the customers, to have a team that can go and test your ideas in the market. And so you have that catch-22. I think [Max] got to a term, which is that we didn't like anybody who wanted to run our sales, I don't think they were qualified to do it. And anybody qualified enough to run our sales would not be interested in running our sales.

00:16:49 Daghan Altas
And so we needed to break that catch-22. So the board asked me to basically take a swing at it. And I told them that this was the hottest seat in the house and I'd be fired in the first recession or hiccup. And they were like, well, look, we'll take care of you.

00:17:05 Daghan Altas
And so I did it. I ran it until, I think, mid-20s ARR, in terms of millions, like mid-20 million ARR from zero. It was a wild ride. I don't have the temperament to be a long-term sales leader. It aged me. Things happened, but we did it.

00:17:30 Daghan Altas
We did it. We basically got to a point where we could attract a world-class sales leader, and the rest is history.

00:17:43 Stephen Koza
Nice, man. Would you do it again if somebody asked you to run sales?

00:17:48 Daghan Altas
I would, because life's an adventure. I'll try to do it a little better this time.

00:17:53 Stephen Koza
That's funny. I thought you might say that. I mean, you're pretty good at sales. I've worked with you long enough. You've got the instinct and the EQ and all the important stuff. What did you learn running sales about product that you didn't know?

00:18:14 Daghan Altas
Sales is the hardest job in a company. And my favorite product managers are those who have empathy for salespeople. The answer doesn't always have to be yes, but there has to be a way. There has to be a sort of off-ramp, even if it's no, we're not going to do that, we're not going to get into that market. Because obviously the job isn't to just build a sweatshop, saying yes to all kinds of things.

00:18:30 Daghan Altas
But I think even if you end up at the same point, if you come there with an enormous amount of empathy for your salespeople, you are a much better product leader. And then the thing that EPD, which is engineering, product and design, the EPD organizations don't really get, is that they live in a very deterministic world. I call this the Newtonian physics.

00:18:52 Daghan Altas
They're like, well, if the customer will buy, we'll build this feature. Well guess what? Sales lives in a quantum uncertainty, the foam of deals coming in and out of nowhere. You work on a deal for four months, five months, it checks all the boxes in the process, you have a champion, you have a metric, you have value, you have access to executives and things like that. And boom, they get acquired. Boom, they have layoffs. Things happen. And so I think that engineering organizations or product organizations don't realize how much of a game the salespeople have to play. And we should have more empathy for them, for sure.

00:19:38 Stephen Koza
I like that, man. Maybe that's why we get along. Let's talk about your product for a minute and maybe where we overlap. So you're familiar with what we do, engineering consulting and helping people scale and solve their modernization and scaling challenges. And the infra and SDLC is usually part of that. So you've got a security product company, and as I understand it, what you do lives in the development workflow. What would you say engineering orgs get wrong about where security fits, and how to do it well when you're talking about software development?

00:20:14 Daghan Altas
Well, it would be quite presumptuous of me to say all security organizations get this wrong. But I can tell you one failure pattern I see in some security organizations or engineering organizations. If you think about software development, or software, computer science and computer engineering, and contrast it to some other engineering disciplines, let's say civil engineering. One example I used to give is, imagine a civil engineering company says, hey, we're going to build this bridge, and they go build this bridge. And it's a beautiful bridge, it's like the Golden Gate Bridge. And they say, before we actually let the first car pass through the bridge, we need to X-ray this bridge because we don't know how many cracks are in this bridge. We're like, wait, what?

00:20:59 Daghan Altas
You probably should make these calculations ahead of time. You should have probably made sure that this is built correct by design. And that's where, because my earlier part of engineering, before I specialized in electrical engineering undergrad, it was kind of a little bit of this, a little bit of that, mechanical engineering and civil engineering. And there's a lot of correct by design. It's often said that engineers are not the ones who build the best bridge, they're the ones who build the barely standing bridge. Right? Because you need to get the constraints and parameters right, so that it's not ten x the budget, ten x the bill of materials and all that stuff.

00:21:52 Daghan Altas
In software engineering, I think typically the tier one teams get this and they're doing a great job. Google, Facebook, tech-forward companies, the ones that you work with, I don't think they have this problem. But I see this problem in other companies, that they spend an extraordinary amount of time securing their software after it's been built, versus working on secure by design core principles. And so we find ourselves sometimes hunting vulnerabilities, hunting bugs, whereas it would have been infinitely better, not infinitely, but definitely better, if the company had invested time and energy in secure by design development principles.

00:22:28 Stephen Koza
Makes a ton of sense. I mean, it sounds pretty obvious. Why do companies not do it that way if it's just so obvious?

00:22:39 Daghan Altas
I don't want to go down the rabbit hole, but I'll just leave the technically savvy listeners with SQL injection, or injections as a category. It's been OWASP Top Ten for maybe a decade, if not longer. It should have been eradicated. It should not exist. The technique for secure design for that is just super obvious. It's trivial. And yet we still find these issues.

00:23:04 Stephen Koza
And so the solution is more cultural than technical.

00:23:07 Daghan Altas
Yeah.

00:23:08 Stephen Koza
I was going to ask. Yeah, people, not technology. Let's even talk about AI.

00:23:11 Daghan Altas
Let's go.

00:23:12 Stephen Koza
You guys probably don't do anything with AI these days, right?

00:23:15 Daghan Altas
Yeah, not at all.

00:23:18 Stephen Koza
I've heard you talk about betting on the models. And maybe before you even get there, with these advanced models that are very capable when it comes to offense and defense in cyber, and hopefully you get the same or better defense for whoever's out there trying to do it offensively, how did that change the product, your approach, what you guys do? Because if I'm an idiot, and most days I am, I'd say, well wait a minute, can I just go use Claude Fable and boom, there's my code security. What do I need Semgrep for? So maybe start there. Then I'll come back to my betting on the models question.

00:24:00 Daghan Altas
It's undeniably true that LLM models made the detection problem an easier problem to solve. When Semgrep started in 2020 or 2019, the core concept was that, hey, we know the patterns that you're trying to detect in your code, and we're going to give you a product that makes it really easy to do so. And so yeah, that problem has eroded for sure.

00:24:24 Daghan Altas
But as you know, every technology brings a benefit, but also some side effects. And so LLMs, by making detection much easier for a defender, they also made detection easier for an offender, a red team or an attacker. And in that case, your supply chain vulnerabilities, your exposed secrets, your code vulnerabilities, they all become much easier for an offender to detect and to tackle.

00:24:53 Daghan Altas
And so I think the game has shifted from, can we detect these things and put it in the backlog and have some ticketing system that triages them so that we can work on it on our merry way. That has shifted into, okay, how do we close all of this in two months? Or actually two weeks?

00:25:10 Daghan Altas
Well, I mean two days. Well, no, you know what, we've got to close it all today. And so we're on a tear for velocity right now. And so when you want to talk about velocity of the whole thing, detection is just a small piece of that. How do you close 10,000 tickets? What is the triaging process? What is the remediation process? What is the proof of work process, so that when somebody comes to you and says, hey, that malware, we're done with it, how do you know you're done? And how fast are you done?

00:25:48 Daghan Altas
And so I think the industry, ourselves included, are all shifting towards giving the defenders the velocity of defense, because attack has all the velocity right now. They're already running at, pick your metaphor, velociraptor speed. But defenders are not fast enough yet. And so really the core problem is not necessarily one of can we detect better. It's, can we close this?

00:26:05 Stephen Koza
Can you fix it.

00:26:07 Daghan Altas
Yeah, yeah. The entire loop, including reporting and proof of work. Can we close the entire loop as fast as we possibly can?

00:26:19 Stephen Koza
Okay. So, betting on the models. Tell me about that. I don't want to misquote what you've said, but it was interesting.

00:26:24 Daghan Altas
This is something that I might have heard from Sam Altman. AI has been this beast that just came out of nowhere. And if you think about it like tectonic plates shifting, this continent of AI just formed out of nowhere in the last three or four years. And so it moved everyone around, if that makes sense, budget wise, access, etc. And so it was really important to me to learn and listen as much as possible.

00:26:53 Daghan Altas
They're not going to tell you, hey, here's our three-year roadmap. They did not have a three-year roadmap, but even if they did, they kept it pretty close, because these are trillion dollar games they're playing. But I knew that every now and then they would leak information.

00:27:10 Daghan Altas
So I was religiously listening to every podcast they had, which 99.9% was a mediocre transmission of information. But every now and then there would be a big nugget. And so I think this is one of those nuggets, where it's like, wait a minute, if you are building systems that are trying to compensate for or overcome the limitations of an AI model, you're in trouble.

00:27:33 Daghan Altas
If your value proposition is something like, hey, AI does this, but it's terrible at that, so I'm going to go fix that, guess what, you're giving yourself a one year life, because the next model is going to fix that. And so the internal mathematical equation, literally there's a mathematical equation for this, is that the derivative of value over the derivative of AI has to be positive.

00:27:59 Daghan Altas
What does that mean? What that means is that for every unit change of AI, Semgrep has to get [2x] better. If not, then we're in trouble. So flatline would be, AI gets better, Semgrep doesn't change. In fact, we were on a negative line. The old product was a pure detection focused product, and as AI gets better, Semgrep actually looks less and less valuable. So we're on a negative slope.

00:28:18 Daghan Altas
And so we needed to change that slope to positive, which is that, hey, we've got to bet on the models, that the models are going to get better. So even if we try something today and, you know what, this is really not good, it's just 80% there, but not 100% there, great. That's the kind of thing we want to work on, because that's the thing where as AI gets better, the delta value that we provide the customer gets linearly, actually hopefully non-linearly, better than that.

00:29:06 Stephen Koza
You know, it's, I don't know, amazing or funny to me. You've got all these companies, and some people say, well, is that a feature or a product? And they built this thing that just becomes somebody else's feature overnight because they thought it was a good idea. And then boom, they're not nearly as valuable anymore. And then you also have the wrapper companies that just put a good skin on what the model does, and then all of a sudden Claude has a plugin or a native feature. And so it sounds like that's not what you said, but those are the types of things you consciously decided not to do because you had the foresight.

00:29:43 Daghan Altas
Yeah. I think that we also misread a number of times. Right, just to be fair, there were some markets where we didn't want to go. We said, oh, this is going to be completely taken over by AI, it's so obvious, there's no need for us to be here. And we see players there thriving even today. They're growing like 100%, 200%. So, no crystal ball here. The world is changing very rapidly. I think you just want to have a lot of humility right now when you're approaching all the changes.

00:30:13 Stephen Koza
Yeah. Smart. It's hard to have a crystal ball right now that tells you anything fast enough and accurately enough. Anyways, let's talk about some of the advancements in the models. So when Mythos came out, there was a whole bunch of PR and news about it, and they said, we're going to give this to the 50 or 100 companies and that's it, and it's too dangerous for everybody else. I think you guys did some benchmark testing on that. Give me your view on the models as they are today when it comes to cyber, what you think happens next, and then what do you guys invest in to make sure that that fancy equation you gave me earlier, which, man, I forgot all my math, I don't know what a derivative is anymore, tell me how you guys think about that stuff.

00:31:05 Daghan Altas
So I think the models are very good in precision. Precision and recall, these are two key concepts in detection, and they go way back to cancer screening. They go way back to the ROC, which is radar operating curve, things like, hey, when I see a, can I detect this airplane on the radar? These are concepts that predate this, this is just core statistics. But precision is, if the model tells you something, how many are really true. And so let's say the model says, hey, I have ten vulnerabilities here, and nine of them are true positives. So you've got 90% precision. That's great.

00:31:46 Daghan Altas
But what about the things that the model did not tell you? So imagine that you had 90 real vulnerabilities in the code that you knew, the ground truth. Let's assume that, which is a bold assumption, but let's assume that we know all the vulnerabilities in a given piece of code, and that's 90. And the model will give you nine of them. Well, now your recall is 10%. You only got nine out of 90. So the remaining 81 just stay hidden in your code. So that's the recall piece. And to this day models still struggle with recall. They're going to get better with that for sure.

00:32:07 Daghan Altas
But I think the other side of the problem, as I said, is that detection is only a small piece of closing the loop. And one of the things that we have found is that once you really build an organizational context for a customer, and that's not just understanding that code base, but understanding all code bases, how they relate to each other and how the customer's business works, how the threat models connect, once you have that much bigger picture understanding, then the models do a phenomenal job.

00:32:53 Daghan Altas
And so basically we work for the models, is a simpler way to explain that. Our job at Semgrep is to surface as much as we can so that the models can do the best job they can. And so at the heart of our detection, there is a model. And now we basically have a lot of special sauce in terms of how to model users, tools and databases and access to contexts and program analysis. But the point is that all of those are at the service of the model, designed and built for the model.

00:33:09 Stephen Koza
You know, what we see is no surprise. There's a spectrum of AI-native maturity across orgs. You've got the ones that are just bleeding edge, pushing the envelope on innovation. And you've got some that are stuck, or maybe they finally ponied up for, actually, I guess you get Microsoft Copilot if you're, I was going to say they finally bought it. They finally started using it. And they're like, hey, guess what, everybody gets AI. And then you've got everybody else kind of in between. When you talk to companies who believe they're behind or acknowledge they're behind, maybe they bought some tools, maybe they're trying to teach everybody about AI, but you and I might not consider them ahead or even keeping pace, where do they start? What's that conversation look like? If you were going to give advice to a company like that, would it be specific to code security or even more broadly, just about AI adoption?

00:34:12 Daghan Altas
I think engineering organizations have a better chance to make faster progress. I look at even our company, and I'm looking at the progress on the engineering side versus the go-to-market side. AI shines in the hands of engineers. I'll just say that. And the reason is that AI thrives with data, AI thrives with verification loops and also context. And so pooling context is a lot of data engineering.

00:34:50 Daghan Altas
In fact, I had a degree at Berkeley, and you think that you go in there and you're going to do these beautiful algorithms and a lot of inference and this. No. Like 99% of the time you're doing what they call data engineering, which is, how do you basically get the right data to the system? Because the inference part is easy. The hard part is getting the data, and clean data, ground truth data. And these are hard things for people to put together unless you're an engineer.

00:35:10 Daghan Altas
The other piece, and I was giving this advice to our HR team here, is, you read all these stories on X, like [product name unclear], how I automated this and that, and it gives you this feeling of, I should be able to launch a rocket and land it on Mars.

00:35:32 Daghan Altas
Honestly, I think that the number one thing you have to do is, literally, it's just true, just like playing basketball, volleyball, you've got to clock your time. You've got to spend at least eight hours a week solid working with AI, because the first thing you're going to do is you're going to use it like Google.

00:35:50 Daghan Altas
You're just going to ask questions, get answers and things like that. But you've got to build that muscle. I think the failure mode is people try to do something super fancy. Like, oh, I'm going to automate my, I'll give you an HR example, I'm going to automate my recruiting, my sourcing. No. That's too hard.

00:36:09 Daghan Altas
Try something smaller. And so I think the people who really gradually ratchet up their AI mileage, then you develop a taste of what's possible, and then you start to accelerate. Because once you know what's possible, it starts to compound.

00:36:29 Daghan Altas
Although compound is a word they overuse, because everybody who wants to be AI native uses that word. But yes, it starts to build on top of itself. And then you really do accelerate. But start slow and you've got to put in your time. Honestly, Stephen, you know what, I've got to take this back.

00:36:45 Daghan Altas
Let me ask you this question. Hey, Stephen, I want you to become a great Italian chef. What do you think you should do?

00:36:55 Stephen Koza
Oh, boy. Probably start to cook some stuff. Yeah, just get into the kitchen and start to cook some stuff. And probably start with an omelet.

00:37:05 Daghan Altas
Yeah, totally. I would put on some really amazing Italian music, and my wife likes to make fun of me, but man, if that's playing in the kitchen, you feel like a chef all of a sudden.

00:37:13 Stephen Koza
No, I really like that, man. I've given the same advice. Yours was a little bit more eloquent, but whenever I would give that advice, like just hack, just build something, go surf X or Reddit or watch some YouTube and play and build and hack and figure it out. I would always say that because I didn't have a better answer for them. But here, and the way you explained it, I tend to agree. I mean, sure, there's training classes and all this other stuff, but man, it's moving so fast.

00:37:44 Daghan Altas
Yeah, just do it.

00:37:46 Stephen Koza
And then probably eight hours a week is the right amount of time.

00:37:50 Daghan Altas
Just like, you know, you should devote 20% of your working hours. If you're in tech, you're not working 40 hours, let's just be honest about that. But I'd say 15 to 20% of your time should go into meaningfully making progress with AI.

00:38:02 Stephen Koza
Yeah. For sure. My approach has been nights and weekends, and working on something during the day that I think is going to move the needle. But I was talking to somebody, I think it was a client the other day, and he took a week off, PTO staycation, and he went and vibe coded something. He spent his vacation doing it. And I was like, man, I should take some vacation and do that. That sounds fun. But I think you're right. You just got to get your hands dirty.

00:38:29 Daghan Altas
I'll give you one more specific piece of advice on this, organizational advice. Don't peanut butter your talent on this. If you have, say, in an organization, five or six people who are doing it the right way, put them all in the same team. Let them rip. Because what's going to happen is it's going to create a lot of momentum and critical mass, and it's going to attract other people who want to join the team.

00:38:49 Daghan Altas
And it's also going to create that sense of what's possible, and they're going to start laying the foundations of what the company should do. So I think it's important that if you're looking at this at an organization level, if you have phenomenal people who know how to do this, or they're showing you how they're doing it, you've got to assemble them in a team.

00:39:06 Stephen Koza
Let me come back into our world for a minute, while we're talking about AI. So as we all know, AI is great at coding. It can produce infinite lines of code. And it's not all good out of the gate, there's software engineers that still have jobs that need to make that great code or a great product. But coding is not the bottleneck anymore. Now it's everything else, which is your world. It's the security and the testing and the stack and all the things. And so one, I want to know if you agree with that, which I suspect you do. But two, what's the approach for an organization that, like you and me, believes that to be true, but they're just stuck? They're like, great, we're AI native, we're using Claude Code, we accelerated our roadmap, we fixed all these bugs, but the bottleneck is what's behind it, and they haven't fixed the bottleneck. What do you do?

00:40:12 Daghan Altas
First of all, I agree with the statement. The reality is that I haven't seen that scenario that you've laid out, which is that they are so good that they were able to automate their coding practices, but they're not good enough to do what's next. I think if they're good enough to do the first part, they're actually thinking about the second part, and the second part is the verification bottleneck and how to solve that.

00:40:33 Daghan Altas
I think the greatest analogy I can think of is what's happening in driving, with Waymos and Teslas versus human drivers. Every study unequivocally shows that driverless cars are safer than human driven cars.

00:40:47 Daghan Altas
Frankly, I drive a Model Y, the new version with the new hardware with the full self-driving, and I absolutely trust that system more than I trust myself. It doesn't get distracted, it doesn't have a bad day. I may get some criticism for saying this, but yes, I absolutely think that every time I do the full self-driving, I'm actually keeping myself and my family safe, as opposed to compared to my baseline.

00:41:12 Daghan Altas
The reason why I mention that is that one of the things that I think is rapidly going to go away is PR code reviews from humans. And the criticism I hear about this is that, yeah, AI systems make that mistake. And in that criticism people compare AI to a godlike, sort of [unclear] code review.

00:41:38 Daghan Altas
People forget that you're comparing AI to a human, and a human who gets a giant pull request. Now all these pull requests are coming from an AI bot, like 9,000 lines, 25 files, who knows what. And you're asking a human, in fact, I'm going to give a talk about this, so I researched this a little bit, and they did some studies.

00:41:58 Daghan Altas
When they gave humans known bad code without telling them, hey, review this, I think humans found three out of like 50, voluntarily. So their accuracy was around 7%. When they told them specifically, hey, there's stuff here, carefully look at it, the accuracy went up to like 40%.

00:42:20 Daghan Altas
And so the benchmark you're trying to beat here is already abysmal. And now the only way I think code reviews survive is architectural threat modeling. Really, really strategic high level, like, wait, are we moving away from protobuf? Or wait, are we no longer on Kubernetes?

00:42:41 Daghan Altas
If your AI is taking your product in that direction, you want to have these architectural conversations with your AI system. Yeah, of course those should be flagged, and those exceptions should be read by a human. But the vast majority of day-to-day code should just fly through your AI system. You'll be better off.

00:43:03 Stephen Koza
Yeah. Great analogy. Hard to argue with the self-driving thing. Well, let's wrap up here, man. This has been fun, I appreciate it. Where can people track you down? How do they find you?

00:43:17 Daghan Altas
I'm on LinkedIn, and I'm on X. But LinkedIn, as you know, is the best place to find me. I'm a part of Team Semgrep, and Altas is really easy to find. I always respond to the DMs and try to help people as much as I can. DMs open.

00:43:38 Stephen Koza
Cool. Love it, man. Well done.

00:43:40 Daghan Altas
Thanks, man.

00:43:41 Stephen Koza
So, loved hearing about the product to sales journey and back. Fun chat on AI. So for our listeners, if you got something out of this one, please subscribe, like, give us a review rating wherever you found us. And we'll put Daghan's contact stuff here in the show notes, and we're on all the platforms if you want to find some more episodes. I'm Stephen Koza, and this has been another episode of TechPod Talks. Thanks, everybody.

‍