Episode 10 – That Later Never Happens

Maor (00:02.571)
Yes. Awesome. Thank you everyone for joining for yet another episode of Backline Unfiltered. Really happy to have everyone here and incredibly excited for this conversation, which is going to be all about actually using AI and AI agents in production from someone who has tons of experience, both professionally and personally using AI agents. I’m super happy to welcome Kishor Patil from Best Buy.

Maor (00:31.612)
He’s here with us today and I’ll let him quickly introduce himself.

Kishor (00:35.513)
Hey, thanks Iran and thanks Mayor for having me and excited to be here today. So my name is Kishor Patel and as you said, like I’ve been with Best Buy for a long time. So for past sixteen years I’m with Lebest Py and I’ve worked across different teams like in a different role. So I’ve been with technology like for a long time and I’ve seen that change quite a bit during this time. So in my role, what we do is like I’m focused on Google Cloud Platform nowadays and

what we do is like we make Google Cloud safe and easy for people to use. So we look at the cloud services and start evaluating them from a security, from a networking, and from a feasibility standpoint. And we have like certain standards, certain goals. And once that evaluation is complete, what we do is like we we start automating those services. So all the complexities around the identity.

the networking and the security, right, is handled by the automation platform. So the end users, they don’t have to worry about like all the underlying complexity. That means that developer just comes in, just plugs and plays, and they don’t have to worry about any technical like background details. And outside my work, I’m spending a lot of my time exploring agents and that’s that’s keeping me busy. And I’m also part of a lot of communities like Google Developer Group.

The BCS open source group, IEEE, where we organize talks, workshops, and discuss about the technology. So that’s about me.

Maor (02:12.563)
Awesome. Thank you very much for joining us. We often speak with security leaders and security professionals and the conversation often tends to be a lot about, how can we use AI for security? Or why do we need to do from a security perspective to secure AI? And that often we get the opportunity to spend time with someone who’s thinking and using AI.

I would say more generally and specifically to enable business processes and business success. So this is quite cool and personally close to my heart because I’m a big tech shopper, unfortunately, or fortunately. So I’m sure this is going to be a fun one. Let me pick things off maybe with the first topic. In security, I think generally speaking with AI, we often feel that there is

Eran Leib (02:49.229)
you

Maor (03:10.082)
a large appetite to go and try things, right? It’s super easy to try. All of these technologies are really available for us as consumers and in the workplace. And we want to play around with it. We want to see what benefit we can deliver to our business from with. Why do you think organization is so quick to try and deploy agentics, say our workflows?

Kishor (03:37.734)
Yeah. So I feel there are there are a few reasons. Like first is very easy. So so remember those old days, right? You need to re like learn about the Python frameworks, the Spring Boot frameworks, right? It’s not the case now. You don’t you don’t need a big project or a big budget. A single person can build an AI agent like in in an afternoon, right?

And it is not like engineers anymore. It’s like a salesperson or like a finance person. They can just they just can go and build the agents. And and the new thing, like that wipe coding, okay, everyone is on that and they’re able to deploy the agents like very easily. So I think that’s the part like everyone is trying to explore. I think one more area is like it’s very fast for for example, the turnaround time, right? Development cycle is very quick.

So a good demo just gets created in a minute. Imagine an AI agent that reads like a a long support tickets and just gets a clear summary for in in a few seconds, right? That that’s like a that’s quick. That’s in a couple of hours you can build that. And that works. That used to take portion like a 20 minutes, right? Now with a with a agent tick flow, all this is available in like a minutes. So

So what people are doing is instead of let’s put this like let’s try this, instead of that, it it becomes a requirement immediately and people are like jumping on it. So that’s a trend I’m seeing it. And and and there’s a very important factor here. The speed is understandable, but people jump from the demo to the production like quick without realizing a lot of security factors.

And it creates a lot of loopholes. So that that’s a very important point. And also there is a third most important thing is the massive industry pressure. So companies are like afraid of like falling behind. If one company talks about a new agent and the other company asks like, okay, why don’t we have this this one? Right? So waiting feels risky to companies like

Kishor (05:52.272)
And what they feel is like, okay, let’s move faster and we we are in good hands. Even if it’s not safe, they are just moving fast. And and that that thing, right, let’s try it out, turns into a let’s just do it real quick. So I feel these are some of the reasons people are like just jumping on the board.

Eran Leib (06:12.549)
So you feel that moving things into, I’m going to call it production mode happens today much easier, even though it’s kind of counterintuitive for that to happen because it’s like risky and new and you’re like, people are not sure how you’re going to behave. And all of a sudden something that, you know, like used to be, we should wait and test it out and see how it plays out. And then all of a sudden, okay, let’s just shove it to production.

Kishor (06:36.975)
Yeah, yeah. So because of this non deterministic nature, right? with with the earlier software cycle, we used to know, okay, this is going to follow this path, A, B, C. But with this non-deterministic nature, you don’t know like what path it’s going to follow, what decisions LM gonna make. So people are just jumping and just putting in a production which is without without guardrails. So that’s kind of a little risky.

And people are realizing it now and they’re they’re working on it.

Maor (07:10.124)
It’s interesting. So I run myself. I’m not going to name names because I don’t want to. I shouldn’t name names, but I run myself. I just to tie on your point, went to eat lunch with a friend here in the Bay Area is a security leader in a large security vendor. And we came in, we had our lunch and it was clear that our friend is very

frustrated and I was like, dude, what’s going on? What’s with the long face? And he was telling us that their CEO, and we’re talking about security, right? It’s pushing all of the engineers to use AI technologies and AI everything. And basically said, if you’re not using this, we’re not using that, you’re falling behind. And he was saying, as a security leader in a security event or that they feel they don’t have enough time to react.

and respond to all of these new technologies. And it was definitely, I think, as someone who’s been in the industry, and I would say more conservative perplexing, like, dude, you need to make sure you’re not creating any danger and you’re introducing new risks. But also, as someone who lives in the world today, I understand why people want to move quick, because I think your point is exactly the point.

you’re not doing it, you’re falling behind.

Eran Leib (08:42.914)
I’m not sure by the way, what’s the ratio between let’s be productive to FOMO, right? There’s a lot of FOMO in the industry about it. we have to do this because like Kishore said, someone else is doing that like a competitor versus this is being productive. It’s not necessarily true to go with FOMO.

Kishor (09:08.377)
Yeah, exactly, exactly. So now all those security factors and concerns, right? people are like kind of ignoring because of because of the pressure. The pressure, amount of pressure, people are having it. Every second day people are deploying agents like in their production. I’ll tell you my personal experience. I was chatting with one of the agents and I wanted to update my personal data and I was chatting and I was like, okay, update this information.

It says okay, updated information. it didn’t update. Again, I went back, check my details, it’s exactly the same. I was in a loop for like twenty times, and that agent was giving me, okay, your information is updated. So obviously it’s a bad customer experience and it’s gonna ultimately hamper the organization. So I think speed is important, but I think eventually security should be prioritized and

That’s going to save a lot of things.

Maor (10:09.618)
Yeah, for sure. Yeah, I agree. think it’s obviously very interesting times. And as a consumer, I just understand why there is a feel that there is an opportunity here to move quicker because I think unlike a lot of technologies that we’ve introduced into the workplace in the past, the productivity gains

from AI are almost crystal clear. It’s very easy to envision why this is going to be great. But I think to your point, it’s not very easy to envision, okay, from how we feel it could be great to get to a point where it’s great and it’s actually working and it’s predictable and it doesn’t generate slope and so forth and so on. That journey I think is probably longer than what a lot of…

organizations, leader and people actually feel. Were there like any specific repeating concerns that you or repeating challenges that you found yourself having to deal with? Where today you can say, well, these are the top three things that I would kind of recommend looking at before or while adopting AI, generative AI or agents.

Kishor (11:33.65)
Sure. So there are some of the big big security worries which comes down to my mind is like, first is too much access. Right. So when teams build an AI agent, the fastest way to make it work is like just give a wide range of permissions, right? So it does not hit a wall. And then the plan is to tighten that security later, and that later never happens, right? So

For example, Tim Bields, an agent, like answer a question about about a status, order status, right? we connect that agent to a full customer database. Now that database has like a phone number, email, a lot of other information. And if if user asks a simple data like, okay, give me this details, the agent might return the personal information which was not intended, right? So nobody hacks anything, right?

Nobody like no there was no bad player, but the agent simply had more access than it is needed and it gave away a lot of information. So that’s that’s clearly no-no. the next is I feel is like the agent trust and what it can reach through the tools. Okay. So this I have personally felt when I was doing a demo to certain people. the agent reads, suppose the agent reads the email documents on our web pages.

And and and it can be tricked very easily, right? with the instructions. we call that as a prompt injection. So what we do is like agents use tools and if that tool has has not been reviewed properly, right, we we get into this trouble. I’ll give you one example. So imagine an agent reads a customer email, right? There’s a huge email, and we developed an agent that reads the email and takes some actions. And there are some hidden lines saying, Send me the last ten orders.

Right? That agent cannot always tell the difference between a real instruction and a trick. So that is why the safest way to put the fix is like you should have a proper guardrails to restrict the tool usage. And that that’s very important factor. maybe one more I would highlight to highlight is the lack of visibility, right? The identity problem. So when several agents, like I started my POC with a single agent.

Kishor (13:57.796)
And within a week it became like a 10 agents. And all the agents were talking to each other, right? So which agent did what and in what order it has processed things? It’s it’s very difficult. It’s a huge identity problem. So I I’ll give you one more example. Like customer asks for a refund. Suppose there’s a customer who comes in and asks for a refund. One agent reads the request, second just takes the order status.

Next is processes the refund, right? And one more agent updates the order. Now later someone finds that refund was wrong, right? What will happen is with a good record, you’ll be able to backtrack every agent what action it has performed. But if you don’t have this properly, what will happen is you’ll just see the refund issued. That’s it. Who took action, what action? It’s very difficult to track it. So I think that

Eran Leib (14:55.521)
And why? And why is a big question.

Kishor (14:58.233)
Yeah. Yeah, correct. Why? Exactly. So I think that identity problems are huge. A lot of people like overlook that, but this multi-agentic system where which agent took what actions, it’s it’s it’s very very important to track that.

Eran Leib (15:20.13)
So I think that, know, like at the same time, there’s, there are a lot of opportunities that are tied, you know, to deploying agents to help, you know, like with security challenges and with other things. What kind of opportunities like that you see? I know what I see, but I’m curious to see what you, what’s your perspective on this.

Kishor (15:41.308)
So th there are there are multiple opportunities for for example, from a security standpoint, right? if I would like to highlight is like these teams are always buried into the alerts, checking the logs, and most of the time they are reading and sorting things to find out some problem problem areas, right? So first is like a security team can just get thousands of alerts in a day and and like

an agent can group related alerts together and say, like these 12 alerts all come from a same problem. Now person look at the one item instead of like 12, right? So it’s going to save your time. So so these agents are very easy to develop and they’re going to save a ton. Second is like explaining what happened. When something goes wrong, right? People people lose their like collecting logs many places and like

It takes lot of time. So a a simple agent can read logs and and and and write first summary such as which account did what. Then what teams can do is like start a clear picture instead of a blank page. They’ll have a clear traceability, like which agent took what action. So these are the these are some of the things which they can easily develop. One more is like suppose finding access that is not needed.

So you can you can actually develop agents over time. find out the accounts or agents which are over permissive and and nobody and nobody’s like tracking them, right? So an agent can look through the environment and point out to the account which was like a orphan and for months nobody has used it. So they can just have an audit every week and pull out all the details and give it to the person who can review it, right? So

Maor (17:09.808)
Thank

Kishor (17:34.63)
That’s one more thing. and I think agent can also look at the newly deployed cloud architecture, compare it with your company policies and security baselines and exactly see what’s matching and what what needs to fix. So th there are a lot of opportunities. it’s just that we have to look for it and and we we can it it can easily be developed.

Eran Leib (17:59.221)
In many cases it almost seems like the repetitive tasks that people are not doing because they’re too tired of it can definitely be handed over to agents and just come back when you see an outlier. It’s almost like the low-hanging fruit. The big question is can the agents understand when the outlier

Maor (18:01.127)
Yeah.

Kishor (18:09.713)
Correct. Yes.

Eran Leib (18:27.578)
Is an actual outlier versus okay, that’s just no like okay and no big deal but I guess some some some alerts are fine

Maor (18:37.708)
I think traceability and reasoning, to your point before, are going to be super important. You want to know who did what, why, when, and have the ability, in maybe some cases, to even be able to roll back actions. You see a chain of events that happened and you want to say,

All of this is something that I want to be able to roll back. So traceability, accountability, reasoning, and the ability to roll back actions when needed. I’m still thinking a lot about our trust in, I would say, LLMs in general. I feel like personally that the more I use it, the more I lose that muscle of

Eran Leib (19:30.429)
you

Maor (19:34.231)
Hey, you need to be critical of this information. There could be some hallucinations here, some stuff worth double checking. And I’m wondering, from your experience, verifying results, is this something that is top of mind or we’re just gradually getting into the place where, well, it’s usually fine. it’s

Kishor (20:03.281)
Now, I think it’s a very important factor. So I think because of this non-deterministic nature, right? What path they take it for coming to a decision or taking action, it’s it cannot be predicted. So I think most of the cloud providers nowadays are coming up with different services. Like I would say like Google, right, came up with a cloud model armor which goes on top of LLM. So before you send something to LLM.

And after LLM responds back, it can sit there and it can evaluate. So there are guardrails which we can put it. and and also like grounding. Grounding is a very important part where like you understand like how these LLM models are like hallucinating, and eventually we need to have a proper grounded data and proper validation and guardrails to to

keep the L L models intact so so that it doesn’t cross as the boundaries.

Eran Leib (21:07.8)
I think pounding is key. cause I, it’s, it’s almost insane, you know, like that, we read and look at things and we’re saying, okay, this is AI. This is not you. or it’s not, it’s not realistic, because they almost fall into their own pitfall with regular, with regular, you know, like there are regular things and things that they say.

Kishor (21:10.095)
Yeah. Yeah.

Eran Leib (21:37.582)
But even when some of it is regular and it’s normal, then we don’t believe it. all of a sudden, you’re like, so grounding it in something which is more related and people can believe is pretty much crucial here.

Kishor (21:56.177)
True. Yeah, exactly.

Maor (21:57.975)
Yeah, yeah, I agree. I wonder if we’re getting to a point where it will make sense. I’m thinking how we used to like SCM, like we’ve introduced SCM when we’ve built applications, right? So we have a single source of throws to all of our code. We can see the changes. We can see what happened, who did what. We can roll back stuff if needed.

And I wonder if we will get to the point where organizations will need something like that for agentic operations, right? So you can commit, can review, you can approve, you can roll back. So there’s some like central record database of what the agents are doing in our company. And we have a central place to enforce policies on it.

Eran Leib (22:50.275)
Yeah.

Kishor (22:56.133)
Yeah, yeah, I think that traceability and and as you exactly pointed out, I think people are investing in that area, the observability and the traceability, what agent, what actions they’re taking and they’re putting alerts on it. And it’s a it’s a very hot item right now because I think people are realizing if things go wrong, it can impact if it can impact like a huge impact. So

And and these are very important factors to build like a like a a framework around that.

Maor (23:30.698)
Yeah.

Eran Leib (23:31.77)
There’s an interesting, no, I want to say like a bigger leap here that can you identify what did they do? What did the agents do because someone told them to do versus what did they do because they, you know, like it was like a second or third level, you know, like of, okay, this is what I need to do. And they’re just going ahead and doing it versus like more telling the agents do a.

Does a it might do it like in this way or in this way in that way? But it came from a direct request versus it’s doing something now You know like I have to do this and to get there I’ll do like ten different things and it’s completely you know like its own original thought so to speak so can you trace back to request versus Intent maybe or stuff like that it can be interesting to think about that as

Kishor (24:26.555)
Yeah. Yeah, yeah, true, true, exactly. I think through my personal experience, I think we have faced these issues where where are certain actions were performed and there were updates and we couldn’t find like wha what was this update. And we are bringing this observability platform like as a mandate, like what actions is we we should be able to track each and every point in the in the trace. So

I think these tracing frameworks, this observability frameworks are important factors.

Eran Leib (25:03.167)
Would that help organizations? We were talking before about moving from playing around to actually doing things. But even in Let’s Do That, there’s another level of trusting the agents to be autonomous. Do you think that observability is enough to allow that level of trust to let agents work autonomously?

Or is that still, you know, like far out?

Kishor (25:36.396)
so I still believe if there are critical flows, we should have human in the loop. for for example, if we give the complete control with the agents, right? and and if it’s impactful for the organization, it can go south very quickly. So there are critical flows where we should have human in the loop and there will should be approval process and humans should review that. But but I think things are improving.

there are agents which are very smart and with a proper grounding and all, they are taking a lot of autonomous activities and they are they’re getting built very well. But I always think that if if it’s a very critical flow, you should always have human in the loop.

Maor (26:24.07)
Yeah, I agree. I think there is a tension between the desire to do, you know, we talk about machine speed, right, decisions and all of that. And of course, there is an issue of volume, right? Because you say human in the loop, but when you say human in the loop, you automatically put a big cap on how much of that process you can actually run, right? So that tension…

be something that organization will need to bake into their thinking, right? And like, if you go back, this is kind of maybe we’re going a little bit of script, but one of the things that really, when I looked into the hugging face incident, one of the things that really kind of made me think about that…

Kishor (26:55.397)
Yeah yeah yeah yeah.

Maor (27:20.324)
thing was not just what was the agent intends and what did they did, what they did, but also the amount of work that they’ve done to try and be successful in their goals. when humans had to review triage and understand there was so much you talked earlier about, you know, looking at the logs, understanding what happened. There was so much of slop and hallucinations in the log. So it was really hard for human to understand.

what was real, what was not, what did the agent try to do and whatnot, because just the amount of events was unprecedented when there’s a human or a set of humans that operate or even when they deploy bots, it still doesn’t rise to the level of, okay, now we have AI agents, are, we can generate so much data, take so many activities that it’s really, really hard to humans to stay in the loop.

Kishor (28:18.363)
Yeah, yeah. I think yeah, yeah. I think I agree that balance is important. Like how much you need human in the loop. I think the f the example which you gave was really not like i it was something different. And I think you cannot have like a human in the loop in such scenarios where like it’ll be it’ll it’ll be not fair, like and it won’t be advisable. But yeah, you’ll have to maintain that balance.

Maor (28:48.59)
For sure. Yeah, I think there will be some places where you will say, know, yeah, some stuff are easy to think about if you need to issue a refund and it’s more than, I don’t know, $5, then human needs to review it, right? It’s an easy example to think. But then if you have a billion of 499 refunds going out a day, you’re in a problem. So it’s a…

Kishor (29:05.733)
Yeah. True.

Kishor (29:11.455)
Ha ha.

Yeah, yeah, I agree. Yeah.

Maor (29:18.303)
It’s not always super easy. So, you know, with that, maybe just one last question to you. What excites you the most? Right? You’ve been, I think, heads down with so much experience actually doing and building with AI and AI agents. And when you’re looking at the future, kind of what excites you the most about it?

Eran Leib (29:18.736)
Yeah.

Kishor (29:44.412)
Yeah, I think as Ziran mentioned, that repeated work, right? if an agent handles that first past, the engineer can spend like like the finding finding answers to the hard questions, such as like how this design and the system better first place. Like so there that repeated work can easily be achieved with the agents and it’s super, right? You you you you spend a quality time on your problems.

The second is building that framework, that core framework where you identify all the networking aspects, the security aspects, what the egress rules, what the ingress rules, right? So building that clear framework so that it can be reused is is I feel is very important. And we we are working on it and it it is it is very exciting to like see how granular we can go with that, right? With the identity aspects, with the security aspects.

So I think these are some of the things which are like very exciting. But there are problems as well. So for example, agents are granted a real access to the real system, right? Like that they’re giving like all the access. We should be always careful while granting this access. And also that trust boundaries, right? agent can always sound like very confident, but we should always think, okay,

It’s not always they’re hundred percent correct. So we should have a proper grounding rules and in necessary flows, human humans in the loop. So these are some of the important factors we should take care of. So yeah.

Maor (31:25.515)
Awesome, agree. I’m very excited for the future. I feel like we’re living in a time where you can’t predict what’s going to happen next month.

Kishor (31:38.789)
Yeah. Yeah. And and I’m optimistic. The problems are like these are solvable problems. It’s just like we have to limit the access, keep the records, and these can be solved. we will get there. It’s a new thing, so it’s a new world, it’s a new learnings.

Maor (31:56.33)
I agree. think our industry in general, you know, proved that for the most part we can get right. We’re techies, we get excited by things, we move quick and eventually I think we learn to operate things in a grounded, reasonable way. But there’s always something that will…

surprise you. I don’t know if you read about the refrigerator incident. I think it was in France. There was attack last week, think, cybersecurity attack where someone hacked controllers of refrigerators in a huge retail shop and changed the temperature. Basically,

cooked all of the food inside those refrigerators and caused many many tens of millions of dollars damage to merchandise and it was a real cyber attack. I don’t know if it was someone’s just trying to create damage maybe a competitor of that chain hired someone to create damage but I was like these are old refrigerators come on but you see there’s always

There’s always something out there. Think about technology in cars these days. There’s so many use cases that there was something about dealerships that have easy button to open the car. many times it’s like a default password that they don’t actually remove it and it stays there when they sell the car. There are a lot of cars out there today with this thing inside that’s

You can actually go and open your car, open the car with your smartphone. Just endless, endless risks.

Kishor (33:57.851)
Yeah, yeah, true.

Yeah. It’s a new world.

Maor (34:02.696)
Cool, thank you very much. Yeah, thank you very much for joining us today. We really appreciate it. It was fun.

Kishor (34:07.483)
Yeah. Thank you. Thank you.

Maor (34:12.849)
Awesome.