Coming from a security lens, skepticism about AI is the default. With last week’s Mythos debacle hardly in the rearview, the attitude is only reinforced.
I recently had the chance to sit down with Troy Hunt and pick his brain on the subject. What he had to offer was surprising, and (dare I say) refreshing.
While we can never stop asking “how is this going to hurt us,” a myopic view of the dangers of AI could blind us to the some of the genuinely useful things it can do, if we’re savvy enough to do it safely.
The trick is knowing how to tame the tiger.
Troy just might. In this interview, he walks me through what he’s using AI for at the moment, how he gets it to do his bidding, and how his team maintains control over its rogue aspects while still squeezing out the best value.
There are a lot of “AI influencers” out there right now. Tell me what you think.
The problem of AI influencers (with no security background) is one of the bigger problems with generalized AI right now, he claims, as these folks who’ve built large audiences off of success they’ve had in completely unrelated fields.
“The central problem of using AI is inherently around security right now, so this kind of behavior is especially concerning for the normies (non-technical folks) that are curious about AI.”
He says a good example he saw recently was someone talking about giving an AI model full access to their email and letting it act on their behalf. It went off, contacted an insurance company, and apparently resolved a dispute better than the guy himself could. His takeaway was, “this is incredible, look what AI can do.”
While it is impressive, it’s also a perfect example of what not to do. Handing over that level of access to AI – effectively letting it operate under your identity with full permissions – introduces a lot more risk than it does efficiency, ultimately costing you more than you gained. One big wrong AI move, and all that productivity goes down the drain.
So we absolutely should be excited about the opportunities. But there’s a line, and we can’t just pretend it isn’t there.
How are you using AI for Have I Been Pwnd? Any interesting projects?
At Have I Been Pwned, Troy says its most recent project has been using AI to replicate the work of a junior analyst and getting it to do some of my tasks, but within a tightly controlled scope and a layer of mandatory oversight.
“So right now, Bruce the Bot is handling our Zendesk tickets and managing to do a pretty good job. I’ll wake up and he’ll have about 20 of them queued up for me. Remember, this is all on an approve or deny basis, and it’s under his own identity.” Troy says He’s got a couple of API keys that let Bruce do his job: one for accessing Zendesk (as a limited user), one for accessing X (but not posting), one to query HIBP’s APIs, and so forth.
However, what they haven’t done is let Bruce run around under Troy, or anyone else’s identity. Even within his limited scope, he’s still gone rogue and made a few mistakes, but that’s to be expected: it’s no different than a human analyst new on the job. “On our side, we just train and retrain until he gets it right – and make sure he’s using UK spelling!”
When you pull back AI from a technical standpoint, what sits under the hood?
Troy says this is an interesting question, and one that he thinks can best be explained in the context of Bruce. When you peel back the layers and look at what Bruce is actually doing, it’s not altogether magical. That can be more than a little disappointing for those of us still amazed by what it can do but also freeing because you understand the logic.
Bruce (or Open Claw) creates a combination of markdown files, or what amounts to human readable files with instructions. They’re all there in a logical structure: one for tone and behavior, one for his particular set of skills, and so on.
And then for the actions we consider agentic, Bruce is just writing Python script. So if he were to be asked how to reply to an email asking for educational discounts, for instance, he just picks up the markdown file that has that information, sends it to Claude, waits for Claude to reply, then pushes the answer back out via the API for Zendesk. When it comes to other agentic tasks, like checking Zendesk every 60 seconds for new tickets, that’s just a cool job.
On the surface there’s a wow factor, but when you step back you think, “yeah, that’s actually how I as a human would probably write this.” The logic is the same; it’s just the AI doing the work.
Is there a “right way” for businesses to go about adopting AI?
Troy says there is, and it comes down to doing this with a “less is more” mindset when it comes to agentic AI; less autonomous activity from the beginning and more oversight produces the kind of solid, well-trained model most highly regulated companies are looking for.
Going back to the point I made earlier about Bruce’s boundaries as a “junior analyst,” Tryo says companies need to have this same least-privilege mindset with every AI tool that interfaces directly with their systems. They cannot (yet) be left to “do their own thing,” even if they can respond to your insurance emails better than you or your team can. At this point, the security downsides are still too risky, and models still require a good amount of analyst training before we can trust them to act like us.
Whilst this bursts the AI bubble slightly, taking it down from a genie in a bottle to a really smart intern, but that’s the level that businesses need to operate on in order to not get burned. At least with where current AI capabilities stand today.
Disclaimer: The views and opinions expressed in this article are solely those of Troy Hunt.
Your Guide to Secure AI Innovation
In this accelerated threat landscape, every security company must embrace AI not as an option, but as an operational necessity.
