CISO Series Podcast
The Difference with AI Red Teaming is We Added the Word AI

Is red teaming AI simply a more specialized version of something we already understand? Or do we need to think differently about the risks in LLMs? Do any of our old tools and methodologies still apply to AI-infused tools?

This week’s episode is hosted by David Spark, producer of CISO Series and Andy Ellis, principal of Duha. Joining them is our sponsored guest, Khush Kashyap, senior director, GRC, Vanta.

Listen to the full episode here.

Skip the Sermon

When a CISO talks to executives, they need to skip the sophistry and get down to brass tacks. Wil Klusovsky of Appalachia Technologies warned that "executives want clarity, not a TED Talk," noting that security professionals lose executive attention to smartphones and glazed-over stares by failing to focus on risk, revenue, and reputation. The curse of knowledge is the culprit here. Walking into a room, assuming everyone is already up to speed on the context that lives only in your head, is a recipe for failure. If you can spare an executive who needs to context-switch for the conversation, you have a better chance of your message resonating.

When to coach versus command

Constantly solving your team's problems creates dependency rather than capability. An easy enough principle to understand, but knowing when to step back requires reading the situation. When you can, ask, "What have you tried?" instead of immediately jumping into solution mode, suggested Elizabeth Lotardo in Harvard Business Review. Context is key. Active incidents don't allow time for teachable moments, but staying in command mode all the time prevents growth. Another approach is the 15-minute rule: team members must try to solve a problem for 15 minutes and document what they tried before asking for help, creating discipline while respecting everyone's time.

Making risk quantification useful

The risk quantification space suffers from a fundamental assumption that humans and organizations make rational decisions based on data. But when is the last time you changed someone's mind purely on logic alone? Risk quantification dominated by frameworks and simulations often become so academic that they paralyze conversations, with months or years spent debating assumptions while the business moves on. What the field needs is a shift toward using external data, such as cyber insurance claims, that reflect real-world losses. No one expects risk quantification to be perfect, but it needs to be useful. Focus on changing how executives view the world rather than achieving precision. You're not there to impress other security professionals; you're trying to get this information to resonate with the business.

Recognizing a distinct discipline

AI red teaming is fundamentally different from traditional security testing, though both remain critical for different reasons. Patrick Sullivan of A-LIGN argues that while traditional red teaming focuses on breaking into systems to test security controls, AI red teaming examines outputs. You need to use that to see if models behave fairly, safely, and as intended. Traditional red teaming is focused on if a lock can be picked. In the world of AI you need to figure out if the lock sometimes decides on its own to open for the wrong person.

Listen to the full episode on our blog or your favorite podcast app, where you can read the entire transcript. If you haven’t subscribed to the CISO Series Podcast via your favorite podcast app, please do so now.

Thanks to Dustin Sachs of the CyberRisk Collective for providing our "What's Worse" scenario.

Thanks to our podcast sponsor, Vanta

Subscribe
Subscribe to CISO Series Podcast

Please subscribe via Apple Podcasts, Spotify, YouTube Music, Amazon Music, Pocket Casts, RSS, or just type "CISO Series Podcast" into your favorite podcast app.

10-second security tip…

“Don't chase every risk at once. Focus on the few that could truly break your business and communicate them in business terms. “ - Khush Kashyap, senior director, GRC, Vanta

Listen to the full episode of “The Difference with AI Red Teaming is We Added the Word AI”

What is the Visibility That Security Teams Need?

"The thing about visibility, it isn’t about seeing everything. It’s about discovering what you’re not seeing. Maybe a third-party vendor that’s sitting there has a service account with more privileges than you thought. It’s not so much, what can you show me? It’s more like, help me discover what I don’t know I’m missing." - James Bruce, business security services director, WPP

Listen to the full episode of “What is the Visibility That Security Teams Need?”

Subscribe to our newsletters on LinkedIn!

CISO Series Newsletter - Twice every week

Cyber Security Headlines Newsletter - Every weekday

Security You Should Know Newsletter - Weekly

CISO Series Meetup in New York City!

Join us for a CISO Series meetup in New York City! On Tuesday, October 21, network with NYC security pros at Gibney’s NYC. Drinks, laughs, and cyber friends welcome!

RSVP here!

Thanks to our event sponsors, Anvilogic and ThreatLocker.

LIVE!
Cyber Security Headlines - Week in Review

Make sure you register on YouTube to join the LIVE "Week In Review" this Friday for Cyber Security Headlines with CISO Series reporter Richard Stroffolino. We do it this and every Friday at 3:30 PM ET/12:30 PM PT for a short 20-minute discussion of the week's cyber news. Our guest will be Andrew Wilder, CISO Vetcor.

Thanks to our Cyber Security Headlines sponsor, Vanta

Super Cyber Fridays!
Join us Friday for “Hacking Next Gen Data Threats”

Join us on Friday, October 17, 2025, for Super Cyber Friday: “Hacking Next Gen Data Threats.”

It all kicks off at 1 PM ET / 10 AM PT, when David Spark will be joined by Abhi Sharma, CEO and co-founder, Relyance AI, and Caleb Sima, builder, WhiteRabbit, for an hour of insightful conversation and engaging games. And at 2 PM ET / 11 AM PT, stick around for our always-popular meetup, hosted right inside the event platform.

Register now

Thanks to our Super Cyber Friday sponsor, Relyance AI

Thank you!
Thank you for supporting CISO Series and all our programming

We love all kinds of support: listening, watching, contributions, What's Worse?! scenarios, telling your friends, sharing in social media, and most of all we love our sponsors!

Everything is available at cisoseries.com.

Interested in sponsorship, contact me, David Spark.