Episode 8 – A Great Time to Be a Nerd

0:05
5 seconds
Uh welcome everyone and thank you for joining us for another episode of Beckline Unfiltered. Uh we took some summer break and we’re super excited to
0:13
13 seconds
be back here today. We’re hosting uh David Seidman and we’ll let him introduce himself uh quickly before we jump right into it.
0:20
20 seconds
Great. Nice to talk with you today. I’m David Seidman. I’m the head of platform security at Plaid. Before Plaid, I was with Robin Hood, Salesforce, Google,
0:29
29 seconds
Microsoft. So, some big tech, some fintech, and currently loving it at Plaid.
0:34
34 seconds
Thank you. Thank you very much for joining, David. I enjoyed our conversations from the um you know, very very beginning, and I knew it would be
0:42
42 seconds
super fun to host you in our podcast and thank you again for joining. We had so many kind of back and forth
0:49
49 seconds
conversations about application security and vulnerabilities and everything in between. And when Mitos hit us a couple of months ago, I knew it would be
0:57
57 seconds
a great uh topic to touch on. There’s so many questions out there. Some say, you know, some naysayers say it doesn’t
1:05
1 minute, 5 seconds
change anything. Some people say it changes everything. So, we’ll start to we’ll try at least today to figure out
1:12
1 minute, 12 seconds
uh if if MTOS really changes everything and your perspective coming from all of these organizations will be super valuable for many, I’m sure. So the the
1:21
1 minute, 21 seconds
you know the first question that you know obviously comes to mind where did you feel like mythos hit you like at
1:28
1 minute, 28 seconds
what point in time and what was your initial reaction?
1:31
1 minute, 31 seconds
So my reaction to mythos specifically was finally people are waking up to what’s happening. I think if if you have
1:39
1 minute, 39 seconds
been paying close attention prior to the disclosure of mythos, the vulnerability discovery results from AI models had
1:47
1 minute, 47 seconds
improved steadily over time. And if you listen to the folks who were hands-on doing vulnerability discovery at the cutting edge, they’ve been saying well
1:56
1 minute, 56 seconds
before Mythos for months that that this is coming. Gotti Avon Heather Atkins had a piece that came out well before mythos
2:04
2 minutes, 4 seconds
saying this is on the horizon. And what we have seen is that AIs prior to mythos were already really good at finding
2:12
2 minutes, 12 seconds
vulnerabilities if you developed a sufficiently good prompting and validation harness.
2:19
2 minutes, 19 seconds
When Mythos broke and everyone freaked out, my feeling was that Mythos itself was probably not a groundshattering
2:27
2 minutes, 27 seconds
breakthrough on its own, but it was the moment where everyone in the industry woke up to the true potential that had
2:34
2 minutes, 34 seconds
I’ve already been there. So the people saying this is nothing new, they’re basically right. Like Mythos, based on what we’ve seen, it was a significant advance, but
2:43
2 minutes, 43 seconds
it wasn’t a revolutionary advance. A lot of the things you could do with Mythos, you could do with previous models, too.
2:50
2 minutes, 50 seconds
You just had to develop a sufficient testing and validation harness, and you still have to have that for mythos.
2:55
2 minutes, 55 seconds
Actually, I think that’s my understanding is that’s part of what Anthropic built as part of Project Glass Wing is not just the Mythos AI model itself, but all of the infrastructure
3:03
3 minutes, 3 seconds
around it. So to me, Mythos itself was maybe a little bit overhyped, but the broader trend that it was a part of had
3:10
3 minutes, 10 seconds
been underhyped before that. So I was happy that this problematic trend that I had seen coming was finally getting the recognition it deserved and I’m glad
3:19
3 minutes, 19 seconds
that people are now like paying attention and dealing with this problem.
3:23
3 minutes, 23 seconds
When you say the recognition that it deserves, does that mean vulnerability management as a whole or do you mean
3:30
3 minutes, 30 seconds
like something you know because people used to say oh we can ignore everything which is not critical or high or whatever and you know like everything else under that because I feel like
3:39
3 minutes, 39 seconds
there’s a lot of marketing behind mythos but there’s one truth to this myth which is now that like Mar said we can actually change things together you
3:47
3 minutes, 47 seconds
can’t ignore the lows and mediums because now they can become much stronger things.
3:54
3 minutes, 54 seconds
You know, I I think that angle is true, but I also think it’s a little bit overhyped because let’s be honest, people are ignoring criticals right now.
4:03
4 minutes, 3 seconds
People are ignoring importance. Like you go into the average company and you look at their vulnerability management dashboard and they have like dozens or
4:10
4 minutes, 10 seconds
hundreds or thousands of unpatched high vulnerabilities. A lot of orgs are pretty good about criticals, but particularly the highs, they’re not even
4:17
4 minutes, 17 seconds
patching those reliably. So yes, I agree that that mediums are also becoming more important and we’re increasingly seeing
4:24
4 minutes, 24 seconds
that a bunch of mediums do chain to a high or critical exploit. So I think it’s good that we’re paying attention to those, but I ultimately think that
4:32
4 minutes, 32 seconds
that’s not the core of the problem. Like people weren’t patching the highs, so adding a bunch of mediums doesn’t fundamentally change the the nature of
4:40
4 minutes, 40 seconds
vulnerability management. Maybe what changes the nature of vulnerability management is exploitation. And I don’t know if you shall I go down that road.
4:48
4 minutes, 48 seconds
Yeah, that’s a yeah, I think that that is like to me this is exactly the question, right? As
4:55
4 minutes, 55 seconds
someone who used, you know, Nessus 25 years ago and we’ve seen how vulnerability management and scanning
5:01
5 minutes, 1 second
evolve over the last 25 years, you know, we we got to a point we we used to have vulnerability management and then risk based vulnerability management. And
5:10
5 minutes, 10 seconds
every time we it feels like we’re trying to re rename, rethink about the same
5:17
5 minutes, 17 seconds
problem. Maybe a and and and maybe never changing in in in any like meaningful practical way what we’re actually doing.
5:26
5 minutes, 26 seconds
Maybe now things can start to actually change. So if Gartner used to say we’re never going to pet ourselves out of it,
5:35
5 minutes, 35 seconds
Is this still the case? and and what is the proper reaction to me?
5:40
5 minutes, 40 seconds
Yeah. So, I I have always disagreed with that framing. I’ve always thought that we we cannot patch our way out of a
5:49
5 minutes, 49 seconds
business problem if the business isn’t willing to cooperate. But could we as a business get the critical vulnerabilities, the high vulnerabilities in our dashboards down to approximately zero?
5:59
5 minutes, 59 seconds
Absolutely. Like if we actually prioritize to patch promptly every time, we could
6:06
6 minutes, 6 seconds
absolutely do. It’s not rocket science at all. And in fact, what we see is acrewing problems from not dealing with the patching problem over the years.
6:14
6 minutes, 14 seconds
Like what a lot of companies have is they’ll have some set of key systems.
6:18
6 minutes, 18 seconds
Maybe it’s a language runtime or a database system that is several major versions out of date and it’s coming up on its end of life date, but it isn’t
6:25
6 minutes, 25 seconds
there yet. And then they’ve got a bunch of other things that are pinned at low versions because they’re dependent on that centralized system remaining at a
6:33
6 minutes, 33 seconds
low version. You know that the CL you upgrade your Python runtime and then you’ve got to update 15 other technologies that came to it and
6:40
6 minutes, 40 seconds
somewhere in the that dependency graph is a real bug or is a conflict with your systems that requires real engineering
6:47
6 minutes, 47 seconds
work. So it’s not a problem, we don’t know how to solve this problem. It’s just engineering work.
6:52
6 minutes, 52 seconds
Like it’s not there’s no rocket science to patching your bugs. It’s hard like it’s it’s effort. It’s but it’s not complicated. I mean depends on your
7:01
7 minutes, 1 second
definition of comp. It’s the kind of problem where if you deploy sufficient engineering resources and it’s a relatively reasonable magnitude you can solve it. Sometimes they’re contractual
7:09
7 minutes, 9 seconds
issues as well. You know you have to write a check to a vendor to upgrade the version. But my feeling is that this is something that we have always known how
7:15
7 minutes, 15 seconds
to solve. And where mythos and its companions are relevant is business leaders are now acknowledging that this
7:23
7 minutes, 23 seconds
is a real problem. Like we have CEOs saying we need to solve this. We need to get we need to make mythos not be a problem for our or anymore. And in the
7:30
7 minutes, 30 seconds
past they would ask how little can we do? How much can we get away with not patching? And now they’re saying we can’t get away with it anymore.
7:37
7 minutes, 37 seconds
It’s it’s pretty fascinating that fascinating used to be like you said a board meeting like or boardroom discussion and now it’s a
7:46
7 minutes, 46 seconds
kitchen table discussion which is pretty insane. My my kids are talking about it and you know like my dad a 78-year-old physician that I’m sure has zero idea
7:55
7 minutes, 55 seconds
about this actually have an opinion which is interesting because it it like it gets this down as a natural problem
8:02
8 minutes, 2 seconds
that everyone is talking about. It brings I think it’s probably like a moment of reckoning in a way that is not like everything is coming together and yes okay we we need to deal with this head first.
8:12
8 minutes, 12 seconds
I think that what you said about you not you know like they’re not dealing with the highs or you know like forget critical let’s say they do part of it I
8:22
8 minutes, 22 seconds
think is also because they like you said they’re stuck because of previous versions that haven’t been patched for so many years but there’s there’s a side
8:30
8 minutes, 30 seconds
effect to this that they need to be able to sometimes look at the bigger picture and you know this is going to have like
8:37
8 minutes, 37 seconds
a domino effect on other things and we’ve had it in you know like a multiple you engagements that we’ve had where you know like we were told yes this is not
8:45
8 minutes, 45 seconds
solvable. So you know like maybe if you can do something about it and then you figure out that eventually it’s like solving an equation with 15 variables
8:53
8 minutes, 53 seconds
it’s very very hard because you need to everything to lock in together at the same time and then you can actually do it. There is the the you know like the
9:01
9 minutes, 1 second
width of the problem and also the depth of the problem which is everything that we call dependency hell because everything depends on everything and
9:09
9 minutes, 9 seconds
there are you know like things that contradict each other. So this whole thing is is there’s this whole this part and the other part is that in some cases
9:17
9 minutes, 17 seconds
they’re like we don’t have the capacity to do this so we’re trying to oh let’s look at is it actually problem in this
9:25
9 minutes, 25 seconds
environment. I mean, if you ask security people per se, they see something that says critical or high. They don’t care if it’s exploitable in their
9:33
9 minutes, 33 seconds
environment. They don’t care. They say, you know, like take it out of here.
9:36
9 minutes, 36 seconds
Versus engineering that needs to to do the actual effort and they will say, oh, wait, give me a second. Let’s see if it’s actually here or not. So, I I think
9:45
9 minutes, 45 seconds
this is a this is maybe the sentiment of, you know, like we’re never going to patch ourselves out of it. That of we don’t have to solve everything. That’s
9:54
9 minutes, 54 seconds
what they thought. But we figure out that over time we will have to you know pro probably solve everything.
10:02
10 minutes, 2 seconds
Yeah, I think we Yeah, I think sorry no um I just wanted to comment on you like your on that notion uh because
10:10
10 minutes, 10 seconds
I agree with you David that we can we can remediate uh and fix a lot more if
10:17
10 minutes, 17 seconds
we wanted to. And the thing is, you know, at backline we are building an autonomous remediation technology. And
10:24
10 minutes, 24 seconds
initially our premise was let’s just fix stuff and we’ve learned and and we I think we’ve successfully demonstrated
10:31
10 minutes, 31 seconds
that we can definitely fix the vast majority of vulnerabilities autonomously and and I you know I openly say that you
10:39
10 minutes, 39 seconds
know with or without backline I think as an industry will be in a place where in 3 to 5 years the vast majorities of vulnerabilities will be remediated
10:47
10 minutes, 47 seconds
autonomously. I’m I’m sure we will get there. There’s no there’s no doubt about it. To me the the interesting thing is
10:54
10 minutes, 54 seconds
that still we hear from customers and we hear from security leaders not just engineering leaders who are pushing back but also security leaders yeah we get it
11:02
11 minutes, 2 seconds
you can fix it but if it’s not exploitable if it’s not validated if it’s not this if it’s not that maybe just leave it alone and you know of
11:10
11 minutes, 10 seconds
course we we as a vendor we’ve added a lot of capabilities to support that but as a security person sometimes I’m still
11:17
11 minutes, 17 seconds
struggling with this answer because dude If you can fix it, just fix it. Why do you care if it’s exploitable or not
11:25
11 minutes, 25 seconds
exploit? Maybe it’s not reachable today, it will be reachable tomorrow. If you can fix it, fix it. But still, a lot of I think the mindset is let’s let’s
11:34
11 minutes, 34 seconds
really make sure we need to fix it before we fix it.
11:37
11 minutes, 37 seconds
It’s mindboggling the amount of effort that people spend on on actually validating when
11:44
11 minutes, 44 seconds
validating when just click a button and it’s there. Click a button and it’s there. I come down on the other side of that.
11:49
11 minutes, 49 seconds
I come down on the other side. I was actually going what I was about to say was praise for those tools that help you identify which vulnerabilities are not
11:56
11 minutes, 56 seconds
exploitable in your environment because I think there are there are cases where it’s very clear it’s not a problem like
12:04
12 minutes, 4 seconds
you might have a denial of service vulnerability in some component that is only accessible internally. If I see one
12:11
12 minutes, 11 seconds
of my laptops doing a denial of service on an internal server, I’m just going to knock that laptop off the network. Like I’m not terribly worried about it. and it’s not an a technique that attackers
12:19
12 minutes, 19 seconds
tend to use anyway. So I think that and I I also will say early in my career I worked on appsc at Microsoft and
12:27
12 minutes, 27 seconds
routinely we would find an exploitable condition that potentially could be reached three different ways but in practice two of them were not
12:35
12 minutes, 35 seconds
exploitable and so we would rate our patch based on the actual code paths not the code paths that could exist. So I
12:44
12 minutes, 44 seconds
think that I think it’s totally reasonable to not patch a system if you can demonstrate with high confidence that it is not exploitable in your
12:51
12 minutes, 51 seconds
environment. And then I think let’s say something changes in the future it becomes exploitable you need to catch that at that time and patch it at that time. And and there’s a lot of nuance
12:59
12 minutes, 59 seconds
there. You know maybe if it’s a super critical zero click remote code execution vulnerability you want to patch that everywhere it appears even if it’s not currently exposed because the
13:08
13 minutes, 8 seconds
risk if it became exposed is so imminent so immediate. But I I actually like that we’re able to conclude that many patches aren’t needed because I said earlier
13:17
13 minutes, 17 seconds
that there’s no hard problem in this space. There is actually one hard problem that I forgot about. Um it’s actually a really key problem. It’s not in the patching itself. It’s in the validation. Every time you patch a
13:25
13 minutes, 25 seconds
service, you typically have no idea if it’s going to break. You know, you have some tests. You can write more. AI is great at writing more unit tests, even some endto-end testing, but there are
13:34
13 minutes, 34 seconds
often these very esoteric, unexpected interactions between services where you go to patch something that seems very straightforward and simple and you bring
13:42
13 minutes, 42 seconds
down a public facing service. That is very scary and it’s something that we as an industry do not have an answer for.
13:48
13 minutes, 48 seconds
Every company I’ve been at has had this problem. And you can sort of brute force your way through it by just writing tons and tons and tons of testing. Or if
13:56
13 minutes, 56 seconds
you’re a very giant service, you can do a very slow roll out. That doesn’t help you in an emergency patching situation.
14:01
14 minutes, 1 second
But, you know, you’re Google, you know, you have billions of customers, you can roll something out slowly and if it causes a problem, you only affect a few thousand people. You know, you can’t you
14:09
14 minutes, 9 seconds
don’t really get the same level of coverage at a company even the size of Plaid, which is a very large and systemic company, but not in the billions yet. My feeling is that those
14:18
14 minutes, 18 seconds
tools are actually very useful because when they’re accurate, they really do take work completely off your plate.
14:23
14 minutes, 23 seconds
Like the vulnerability that is rated critical in your dashboard is not actually critical as it’s deployed in your organization. That I think is actually very helpful for defenders.
14:31
14 minutes, 31 seconds
I’ll say also the other problem that I’ve seen or get into is the accumulated dependency graph is so complicated and and requires so much simultaneous
14:40
14 minutes, 40 seconds
remediation that it exceeds the capacity of their entire engineering organization. they would have to like take all the engineers offline for six months to bring everything up to date.
14:48
14 minutes, 48 seconds
That is that’s an organizational problem, not a technical one, but it’s still Yeah, we I think we see the same. And
14:57
14 minutes, 57 seconds
effectively, you know, when when we discussed it before, the way that we see this is, you know, like we’re right now at this at a point
15:06
15 minutes, 6 seconds
where tooling that actually helps you to fix is going to basically push and fast forward the engineering process and
15:14
15 minutes, 14 seconds
triaging and engineering and everything as much as we can to the end possible.
15:20
15 minutes, 20 seconds
But eventually you want it to behave like an engineering team member, right?
15:25
15 minutes, 25 seconds
So someone needs to to review and approve this. And I think the way we see it is that we want it to be as transparent as possible and to test
15:33
15 minutes, 33 seconds
everything that we can as much as we can and to make sure that the process actually gets feedback loops from everything that happens so we can see
15:41
15 minutes, 41 seconds
the results and we can in confidence say yes this is not breaking this is safe to use.
15:48
15 minutes, 48 seconds
And I feel like this is you know like where the the reason why we are doing a lot of the you know like the validation
15:56
15 minutes, 56 seconds
and and reachability and all that stuff is because we effectively understand that we will open more fixes. These
16:03
16 minutes, 3 seconds
fixes needs to be reviewed. So we need to kind of you know like throttle that based on those findings and those you know those uh you know those triaging
16:12
16 minutes, 12 seconds
processes. But eventually, fast forward, some would say 1 year, some would say 3 years, some would say 5 years. But in
16:20
16 minutes, 20 seconds
the foreseeable future, all of that would be, you know, like would be completely autonomous. That’s that’s what I honestly believe. I don’t think
16:27
16 minutes, 27 seconds
it’s a question of if. I think it’s a question of when. And when you bring it to that point, you’re saying, okay, so maybe now we can fix everything because
16:34
16 minutes, 34 seconds
it’s no longer becoming. Because if we have the confidence that it’s not breaking, so what’s the reason not to fix it? let’s say that it’s not exploitable and it’s not reachable right
16:42
16 minutes, 42 seconds
now, but it’s out there and it’s a potential for the future. So maybe we can take out of it now and we know that it’s safe. So it kind of you know it
16:50
16 minutes, 50 seconds
brings me you know like to a different question. So how do you expect you know like mythos and its let’s say future versions or you know like other frontier
16:58
16 minutes, 58 seconds
models going to impact the world of application security and you know like vulnerability management in general.
17:05
17 minutes, 5 seconds
I I think this is a really fascinating question in in so many dimensions.
17:11
17 minutes, 11 seconds
So one thing I think is interesting is that we have not solved the problem of safe rollouts as humans. We do not have
17:18
17 minutes, 18 seconds
an existence proof that this is even possible. So I think that there is a chance we may find that modern
17:26
17 minutes, 26 seconds
large-scale services like those operated by large technology companies are sufficiently chaotic as to be unpredictable and that a a relatively
17:35
17 minutes, 35 seconds
high error rate is inevitable no matter what level of intelligence. Maybe some far future super intelligence can can handle it and maybe the far future is
17:43
17 minutes, 43 seconds
five years from now. But we do not today. You know, our humans cause security incidents, cause outages. Like
17:50
17 minutes, 50 seconds
our humans say, “We can’t test this. We don’t know whether this change is safe, and we don’t know how to figure out.” And often, even at the very largest
17:58
17 minutes, 58 seconds
companies that throw immense amounts of resources, they still have these problems. So, I don’t think we can safely conclude that even an advanced
18:05
18 minutes, 5 seconds
AI, but short of super intelligence can necessarily solve this problem. I think we can hope that it does, but I’m I’m not so confident in saying that we’ll be
18:13
18 minutes, 13 seconds
able to have safe rollouts because I don’t think we’ve proven that that we can do it period. you know there are some other directions we can go provable
18:20
18 minutes, 20 seconds
like verifiable computation and some other some other techniques that might get us that way indirectly but but I think also we’re going to have this
18:29
18 minutes, 29 seconds
force pushing us in the other direction that is already happening today which is we see just a massive increase in
18:36
18 minutes, 36 seconds
patches and in CDE coming out and defenders are already getting a huge influx of vulnerabilities in their own
18:44
18 minutes, 44 seconds
software and patches from their dependencies and third parties And it’s really straining just the the mechanics, the logistics of these operations. You
18:52
18 minutes, 52 seconds
know, bug bounty programs have increasingly long wait times to respond to people. They’re using their own AIs that are very errorprone. Some
19:01
19 minutes, 1 second
organizations have shut down their bug bounty programs entirely because they just can’t deal. And it used to be that these were mostly false positives, but
19:08
19 minutes, 8 seconds
everything I’m seeing tells me that that that validation has come through. And now organizations are dealing with an influx
19:16
19 minutes, 16 seconds
of real bugs, real patches. And I’ll say that the patch load incoming at Plaid is not overwhelming yet. And I’m not seeing
19:24
19 minutes, 24 seconds
indications from most orgs that their patching load is overwhelming yet. But there’s a pretty big lead time for a lot of these patches. You know, Mythos finds
19:32
19 minutes, 32 seconds
some vulnerability, they report it to the maintainer, takes the maintainer a couple of months to patch it, and then only then do we have to deal with it.
19:41
19 minutes, 41 seconds
So, and the slower orgs, the smaller open source orgs that are getting overwhelmed, you know, we could be looking at many months for them to work through all these vulnerabilities, get
19:49
19 minutes, 49 seconds
the patches out, deal with the second wave of vulnerabilities that will never be found based on the first ones, all the bugs in those patches. So, I think
19:57
19 minutes, 57 seconds
we are just right now starting to see a breaking wave of patches that potentially can inundate defenders, both
20:04
20 minutes, 4 seconds
patching their own software and picking up patches from third parties. I think just the sheer volume is going to be a huge issue for most orgs and the farther
20:12
20 minutes, 12 seconds
behind they’ve gotten the worse it’s going to be. Right now we’re also not seeing widespread exploitation. I think a lot of folks have been predicting that the attackers would start to use these.
20:21
20 minutes, 21 seconds
That’s a very interesting debatable point because today most attacks don’t use zero days. Most attacks don’t need to use zero days. But we’re seeing an
20:29
20 minutes, 29 seconds
increase somewhat in the rate of zero day usage. And if I’m an attacker and a zero day is literally free for the asking from Claude or from the internet
20:36
20 minutes, 36 seconds
where someone has already published it, I I have to believe that that’s going to lead to an increase in utilization. Even if most attackers don’t need zero days,
20:43
20 minutes, 43 seconds
if it’s free and easy, why would you not? So I think that may also put even more pressure on defenders to patch
20:51
20 minutes, 51 seconds
quickly at the same time as they’re getting slammed with a bunch of patches that they’re struggling to validate and a bunch of bugs in their own software. I think there’s a non-zero chance that the
20:59
20 minutes, 59 seconds
entire tech industry has to do a massive fire drill and pull like 50% of engineers off of new product development to get their security story clean. I
21:06
21 minutes, 6 seconds
don’t think it’s a 95% chance, but it’s definitely more than zero.
21:13
21 minutes, 13 seconds
Well, I think we definitely saw some evidence of of people thinking about it, right? From Crowd Strike CEO to others think about how can we build different I
21:22
21 minutes, 22 seconds
heard similar things from Red Hat. How can we build these or organized efforts to to go back and either patch or find
21:30
21 minutes, 30 seconds
different ways to to mitigate these things to help our customers just use safe software? I’m wondering now if that
21:38
21 minutes, 38 seconds
overload of patches creates well not not really new but but increases the likelihood of more supply chain attacks
21:47
21 minutes, 47 seconds
with that, right? because now within this influx maybe it’s easier to kind of add some undesired to to into these uh
21:56
21 minutes, 56 seconds
kind of undesires behavior into these efforts as well.
21:59
21 minutes, 59 seconds
Yeah. And we and we see AI assisted uh fishing campaigns, AI assisted social engineering like that that is a risk for
22:07
22 minutes, 7 seconds
all of those maintainers. So absolutely I think that is a big risk.
22:11
22 minutes, 11 seconds
Yeah. Yeah. Crazy days. I think it will uh you know as the saying goes it might it might need to get worse before it will get better. So I think this is this
22:20
22 minutes, 20 seconds
is where we are when you think like forward and and try to envision how do you think all of this will impact the application security industry?
22:28
22 minutes, 28 seconds
Yeah. Well if nothing else it’s going to be a very exciting time. I I think you know it’s a little bit hard to say
22:35
22 minutes, 35 seconds
because it’s unclear the speed at which this is going to break and also the difference in adoption speed between
22:43
22 minutes, 43 seconds
different companies is going to make a huge difference. You know plaid we were we had our own AI discovery agent vulnerability discovery agent prior to mythos and it’s been really effective.
22:53
22 minutes, 53 seconds
We found a bunch of vulnerabilities in our own software through AI and fix them of course. So, but then I talked to some friends at other companies and some of
23:01
23 minutes, 1 second
them are saying, you know, yeah, I’ve used AI sometimes. You know, it’s good at at answering questions. You know, I tried to have it write some code six months ago and it wasn’t that good. I
23:09
23 minutes, 9 seconds
don’t know. I guess I should look at it again. And you you look at the rate of people using AI. Even in software engineering, there’s I think like 20% of people, I could be wrong on this stat,
23:17
23 minutes, 17 seconds
but I think there’s a significant number of people who are not using AI at all.
23:21
23 minutes, 21 seconds
And so, I I look at them and I think, wow, that’s a sitting duck. like that is a that is an organization that is going to be exploited because it’s similar I
23:30
23 minutes, 30 seconds
don’t know folks probably a lot of the folks who would remember this firsthand are retired but when fuzzers first came out when that concept of of randomized
23:38
23 minutes, 38 seconds
mutation testing first came about and found a lot of vulnerabilities and where a a researcher pointed a fuzzer at a
23:47
23 minutes, 47 seconds
company’s product and that company had not used a fuzzer themselves it was just bug after bug after bug after bug I mean
23:53
23 minutes, 53 seconds
I I myself found a CVE in about 15 minutes just with no effort just off-the-shelf commercial fuzzer pointed
24:01
24 minutes, 1 second
it at a second tier PDF reader product and just went to lunch and came back to a zero day. So I think it’s going to be kind of like that. It’s going to be
24:09
24 minutes, 9 seconds
great for the people who are skilled with those tools both on the offensive and defensive side. It’s going to be terrible for the people who don’t use
24:16
24 minutes, 16 seconds
them. I think also the the surrounding industry of things like runtime security protection, you know, those companies
24:24
24 minutes, 24 seconds
that make targeted security patches for out of support products. There’s going to be a ton of business in those directions. We’re we’re very much
24:31
24 minutes, 31 seconds
already seeing this. So, I think it’s going to be an extremely dynamic time.
24:35
24 minutes, 35 seconds
It’s going to be great for business for everyone in this area. And I think there going to be a lot of companies that don’t get with the program. They get hit with something. It becomes a problem overnight. Maybe they don’t have a CISO.
24:44
24 minutes, 44 seconds
they finally hire someone that that person explains to the CEO why they need to write like a million dollars in checks. And so I think there’s there’s
24:53
24 minutes, 53 seconds
tremendous opportunity from a business perspective, from a technical perspective. Like we’re all still figuring this AI thing out, right? Like everything is changing monthtomonth. I
25:01
25 minutes, 1 second
mean, we just passed the one-year anniversary of Claude Code a couple months ago. Like that is just wild to me. So I think it’s very clear that that
25:10
25 minutes, 10 seconds
there are powerful new technologies developing but it’s very unclear to me exactly what course things are ultimately going to take. So you know
25:19
25 minutes, 19 seconds
it’s a great time to be a nerd. It’s a great time to be an engineer because we are innovating at the fastest pace I’ve
25:26
25 minutes, 26 seconds
seen. You know it it reminds me a little bit of when cloud first came out and SAS and that whole the whole transition to
25:33
25 minutes, 33 seconds
the internet but but like 10x that. I mean that was that was quite something but this is reminds me also of the first time I got a smartphone. I got a
25:41
25 minutes, 41 seconds
smartphone before before it was cool and just immediately knew that this technology was going to transform everything but it’s a little bit hard to say how I didn’t think of you know the
25:50
25 minutes, 50 seconds
whole app connected ecosystem and how that would interact with cloud computing. I guarantee there are things in the future of how we use AI that nobody’s thought of yet. Par par
25:57
25 minutes, 57 seconds
paradigms that we haven’t thought of yet. You know I think on top of that you’ve got all these AI specific appsite problems like prompt injection. Um, you know, there’s that’s just an unsolved
26:06
26 minutes, 6 seconds
problem right now. And so we’ve got a bunch of companies that are trying to throw guard rails around prompts that are trying to scrub in various ways, but
26:15
26 minutes, 15 seconds
nobody’s cracked this yet. And it’s really unclear whether this is a problem that’ll be solved technically next year and we’ll be all said and done. No more
26:22
26 minutes, 22 seconds
problems with prompt injection or if this is going to be like a 105 year problem. So I think it’s it’s exciting, but I think anybody who really knows where it’s going, who who thinks they
26:31
26 minutes, 31 seconds
really know where it’s going is probably wrong. Maybe maybe baby quad itself knows.
26:38
26 minutes, 38 seconds
Yeah, I I definitely agree. Um I’ve been in this industry since the late 90s, mid late 90s and and yeah, I I I never experienced anything that moves so fast.
26:49
26 minutes, 49 seconds
You just said we’re celebrating one year anniversary for like it’s it’s hard to me to believe that when we started backline there was no cloud cut like the
26:58
26 minutes, 58 seconds
thing didn’t exist and like and you feel it’s like it’s been part of your life like forever. So definitely I agree that
27:05
27 minutes, 5 seconds
I’ve never seen anything move so quickly and I also it’s hard to predict where uh where where we’ll find ourselves right
27:12
27 minutes, 12 seconds
and and and where things and when you know what’s the kind of a glass ceiling for for model improvement and which which I’m sure we’ll hit at some point
27:21
27 minutes, 21 seconds
and then what will be the next thing that will replace that. I do have one fun prediction.
27:26
27 minutes, 26 seconds
We used to think that it is it’s not possible to find all the bugs. Software will always have vulnerabilities. I
27:34
27 minutes, 34 seconds
think there’s a pretty significant chance that we will in practice find all the vulnerabilities.
27:40
27 minutes, 40 seconds
You know, I think that at some point the and I think we may even be at or close to that point. Any vulnerability that a
27:49
27 minutes, 49 seconds
human can find an AI can find. And it’s mostly a matter of your token spending.
27:56
27 minutes, 56 seconds
At that point, the the cost per vulnerability starts to rise over time.
28:01
28 minutes, 1 second
There’s you don’t really need to think much more about innovating new techniques because basically by definition, the AI can find anything that is discoverable by anyone. And so
28:10
28 minutes, 10 seconds
your rate of vulnerability discovery is purely proportional to your token spend.
28:14
28 minutes, 14 seconds
And so you as a defensive organization or as an organization trying to defend someone else, you can spend money until you stop finding vulnerabilities at a
28:23
28 minutes, 23 seconds
cost you can afford. So if you’re Microsoft and a vulnerability, it costs like let’s say $10 million for every bug you have to patch. I’m just making that
28:31
28 minutes, 31 seconds
number up. But you can keep spending tokens until you’re spending $10 million per vulnerability in your software. And that means that anyone else who wants to find those tokens basically has to spend
28:40
28 minutes, 40 seconds
$10 million per vulnerability, which means the cost of an ODA is not $10 million. And so it becomes completely driven by the economics. I’m not trying to say that you’re going to get to zero.
28:49
28 minutes, 49 seconds
And there are some weird exceptions like maybe someone comes up with a research breakthrough a better way to prompt the AI and their cost per vulnerability is less than yours. So, it’s not a perfect
28:57
28 minutes, 57 seconds
analogy, but basically once AIs are the primary way that people find bugs, you can you can set your price per
29:05
29 minutes, 5 seconds
vulnerability and whatever price you’re willing to spend, you set that same bar for the adversary. So, it becomes purely an economic question. And in practice,
29:14
29 minutes, 14 seconds
you know, if you’re Microsoft, you spend $10 million for bug, there are very, very few people who are going to drop $10 million to find an ODA in your software.
29:23
29 minutes, 23 seconds
In practice, you’re not actually at zero vulnerabilities, but in practice, you’re at economic zero, you might call it. I think that is a very I would say that is
29:31
29 minutes, 31 seconds
a likely outcome. I’m not I’m not going to speculate on time scales, but under under 10 years, I’m willing to say.
29:37
29 minutes, 37 seconds
Do you feel like it’s also true for uh for bugs in general or just for vulnerabilities? Because the the
29:45
29 minutes, 45 seconds
difference is you know like bugs is something that is application specific and vulnerability is something that is a behavior which is unwanted in a way but they’re both you know like kind of
29:53
29 minutes, 53 seconds
unwanted behavior. So at the end of the way yeah I think I think yes but the economic threshold for nonsecurity bugs
30:02
30 minutes, 2 seconds
is much much lower like nobody’s spending $10 million to remove a a UI discrepancy.
30:12
30 minutes, 12 seconds
Very cool. Yeah, I agree. Maybe closing thoughts on people and culture and how
30:19
30 minutes, 19 seconds
do you feel MTOS is maybe already impacting organizations maybe your or that there’s always some I don’t want to
30:27
30 minutes, 27 seconds
say animosity but there’s some you know challenges between engineers and security and do you think it will help
30:35
30 minutes, 35 seconds
reduce tensions and then make everybody lives easier increase tensions how do you how do you think this will play out on on the human level
30:42
30 minutes, 42 seconds
I think a lot of engineers secretly like this because they don’t like the fact that everything is out ofd. They they view that as technical debt. It’s it’s
30:49
30 minutes, 49 seconds
unclean. It’s not aesthetic. It’s not good engineering. And so I found that a lot of people love having an excuse to bring all of their stuff up to date and
30:58
30 minutes, 58 seconds
to prioritize patching over new feature development. I think it’s mostly the business people who don’t like that because it’s money spent that doesn’t
31:05
31 minutes, 5 seconds
earn revenue. I think there also are some people who believe that mythos is all hype or that the whole trend is all hype and so they don’t like being
31:13
31 minutes, 13 seconds
pressured to do work that they think is unnecessary. But generally we’ve found good cooperation. We have not found push
31:21
31 minutes, 21 seconds
back or tension from engineers other than people who just don’t believe this is a real thing or who want to see the data. They want to see the evidence a bit more before they they patch it. And
31:29
31 minutes, 29 seconds
I think that that’s pretty fair. The the other interesting thing that I’m seeing happen in this area is the the the
31:37
31 minutes, 37 seconds
product implications are real. You know, they’re like business people do not like this. And if we had to do a a much larger fire drill, that is potentially
31:45
31 minutes, 45 seconds
extremely disruptive to the business flow and you start running like companies that can’t afford to pass their vulnerabilities. If they take all that engineering or you know, you’re a
31:53
31 minutes, 53 seconds
smaller startup, you’re getting exploited by ODAS, but you pull your whole engineering team on that, you stop making revenue, you go out of business.
31:59
31 minutes, 59 seconds
So I I do think that there are some companies that are in a bad state and and I see people also struggling with the uncertainty of it all. Like we are
32:08
32 minutes, 8 seconds
now we the security team are going to folks and saying hey there’s this whole mythos thing you need to patch your vulnerabilities. They are saying show us the data and we’re saying well it’s not
32:16
32 minutes, 16 seconds
really there yet. You know we have some graphs they do look scary but you know has it happened here? No. Have we been exploited? No. You know have we seen an
32:23
32 minutes, 23 seconds
up trend in vulnerabilities? You know maybe not that much. You know is it is it like meaningful? I don’t know. So, we don’t really have the data to say this
32:32
32 minutes, 32 seconds
is a huge problem, but we also don’t want to get that data. Like, I don’t want to get to the point where I’m going to teams and saying we’ve had 17 security incidents in the past month because of this. Like, can we please
32:40
32 minutes, 40 seconds
patch now? And then now they’re saying, “Okay, great. Three to six months from now we’ll be done.” Like, so it’s it’s a very tough spot for everyone to be in because we’re trying to stay ahead of
32:49
32 minutes, 49 seconds
the problem, but we also don’t want to spend a ton of resources solving a problem that may never materialize.
32:57
32 minutes, 57 seconds
Yeah, I think I think that was always unfortunately part of our job as security professionals to make sure that
33:04
33 minutes, 4 seconds
we somehow find a way to to balance that. There are a lot of organizations I will say that on the one hand didn’t manage the risk
33:12
33 minutes, 12 seconds
on time and we had few very famous you know breaches over the last 20 years but a lot of really good resurrections
33:21
33 minutes, 21 seconds
stories from some of those breaches as well and and some of the most forwardthinking security organizations on the planet
33:28
33 minutes, 28 seconds
today I know in 10 years ago had to go through this huge disaster and then had
33:34
33 minutes, 34 seconds
to rise from it. Well, but better not avoid it for sure.
33:40
33 minutes, 40 seconds
Better avoid it. David, thank you very much for joining us today. We really appreciate your time. Really appreciate your thoughts. Thank you for sharing
33:48
33 minutes, 48 seconds
those uh with our audience. I’m sure many will find it super insightful and looking forward to our next Absolutely. Thank you for having me. It was great pleasure to be here.