- CISO Series Newsletter
- Posts
- I Don’t Just Guess About Effectiveness, I Make Educated Guesses!
I Don’t Just Guess About Effectiveness, I Make Educated Guesses!
CISO Series Podcast
I Don’t Just Guess About Effectiveness, I Make Educated Guesses!
In cybersecurity, we know what controls work well. Think of MFA. But beyond the basics, it's often hard to tell what is actually effective. If we don't know what's working, how do we decide what tools to invest in?
This week’s episode is hosted by David Spark, producer of CISO Series and Andy Ellis, principal of Duha. Joining us is Sara Madden, CISO, Convera.
Listen to the full episode here.
Optimizing for reality, not idealism
Building a greenfield IT infrastructure might sound great. It definitely resonated on the cybersecurity subreddit. The wish list typically includes cloud-first architecture with zero trust, containers for all the things, and mandatory hardware keys. But most of the wishlist comes down to wanting an accurate configuration management database where every team actually logs their infrastructure instead of building fiefdoms. The real challenge isn't the tech stack. It's that we build infrastructure that's reliant on the end user doing the right thing all the time. Good luck with that.
Engineering governance instead of monitoring compliance
Automation is changing the nature of compliance work. GRC is shifting from policy monitoring to AI-enabled governance engineering. As AI infuses into GRC products, the role requires staffers who can engineer these systems rather than track adherence, argued Nikhil Sarnot of Accenture. Humans must shift into areas requiring ethical reasoning. With that in mind, organizations need to rethink how they staff GRC roles. For those already working in the field, staying ahead of this sea change means developing new skills. AI system design, ethical frameworks, and complex decision architecture will soon be table stakes.
When AI finds what humans miss
AI-native security scanners are uncovering hundreds of real vulnerabilities in critical open-source software that traditional scanners overlook entirely. These tools impressed the security engineer, Joshua Rogers. They've already moved beyond pattern matching to tease out business-logic flaws and mismatches between developer intent and implementation. These AI tools keep finding wild bugs after every run. This raises an uncomfortable question for security teams: what does a red team do that truly can't be automated? While AI can likely handle most current pentesting, it also creates space for human testers to focus on sophisticated business logic analysis that requires deep contextual understanding.
The measurement problem
Even cyber insurance companies struggle to identify which security controls definitively don't work. Jeremiah Grossman, CEO of Root Evidence, pointed out that insurers know of a handful of controls that measurably reduce risk, but struggle to show what doesn't. This difficulty sounds familiar to anyone in cybersecurity and exposes how the entire industry makes security investment decisions. The problem isn't just about proving positive impact. It's about the inability to demonstrate negative results. Without clear evidence of what fails to provide value, security budgets remain vulnerable to vendor marketing rather than empirical effectiveness.
Listen to the full episode on our blog or your favorite podcast app, where you can read the entire transcript. If you haven’t subscribed to the CISO Series Podcast via your favorite podcast app, please do so now.
Thanks to Howard Holton of GigaOM for providing our "What's Worse" scenario.
Special thanks to our AI-Infused Security Operations Tip Sponsor, Anvilogic
Thanks to our podcast sponsor, ThreatLocker
Subscribe
Subscribe to CISO Series Podcast
Please subscribe via Apple Podcasts, Spotify, YouTube Music, Amazon Music, Pocket Casts, RSS, or just type "CISO Series Podcast" into your favorite podcast app.
Security You Should Know
Optimizing Access Management with Imprivata
In this episode, Chip Hughes, chief product officer at Imprivata, explains how the company addresses shared access management challenges with specialized solutions that prioritize both security and user experience. Joining him are Kathleen M., former CISO at MyCareGorithm, and Howard Holton, CEO at GigaOm.
Want to know:
Why does shared access management remain such a persistent challenge across industries?
What does Imprivata’s solution actually do versus traditional IAM tools?
How does passwordless authentication work in high-security, high-speed environments?
What authentication modalities beyond badges are organizations adopting?
How can organizations integrate access management across devices, operating systems, and applications?
What are the unique access challenges in healthcare, law enforcement, and manufacturing?
Can shared mobile devices provide enterprise-grade security while reducing hardware costs?
Check out the episode for the answers you need. Read more here.
Thanks to our podcast sponsor, Imprivata
Subscribe
Subscribe to Security You Should Know
Please subscribe via Apple Podcasts, Spotify, Amazon Music, Pocket Casts, RSS, or just type "Security You Should Know" into your favorite podcast app.
Ten-second security tip…
“Our world is driven by urgency, and this causes mistakes that create security incidents all the time. So, my tip is it's never too urgent to not fully think through. Calm down. Think before you act. “ - Sara Madden, CISO, Convera
Listen to the full episode of “I Don’t Just Guess About Effectiveness, I Make Educated Guesses!”
How Do We Measure Our Defenses Against Social Engineering Attacks?
"I wouldn’t call phishing click rate ‘meaningless,’ but it’s clearly insufficient because attackers are hitting us across multiple channels, not just email." - Bobby Ford, chief strategy and experience officer, Doppel
Listen to the full episode of “How Do We Measure Our Defenses Against Social Engineering Attacks?”
CISO Series Newsletter - Twice every week
Cyber Security Headlines Newsletter - Every weekday
Security You Should Know Newsletter - Weekly
PREVIEW: CISO Series Podcast LIVE in NYC TOMORROW
The CISO Series Podcast will be recording live at FAIRCON25 in New York City. David Spark will be joined on stage by Saket Modi, CEO of Safe Security, for a candid and entertaining conversation about the biggest challenges facing security leaders today.
The event takes place November 4–5, 2025, at The Glasshouse in New York.
Use promo code FC25CISOSERIESCODE for 75% off.
Register here.
Watch the short video filmed in Times Square for a preview, and join us for the live recording at FAIRCON25.
Thanks to our sponsor, Safe Security
LIVE!
Cyber Security Headlines - Department of Know
Our LIVE stream of The Department of Know happens every Monday at 4 PM ET / 1 PM PT with CISO Series producer Richard Stroffolino, and a panel of security pros. Each week, we bring you the cybersecurity stories that actually matter, and the conversations you’ll be having at work all week long.
Monday’s episode featured Rob Teel, field CTO, GigaOm, and Davi Ottenheimer, vp, digital trust and ethics, Inrupt. Missed it? Watch the replay on YouTube and catch up on what’s shaping the week in security.
Join us again next week, and every Monday.
Thanks to our Cyber Security Headlines sponsor, ThreatLocker
Super Cyber Fridays!
Join us Friday for “Hacking Remediation”
Join us on Friday, November 7, 2025, for Super Cyber Friday: “Hacking Remediation: An hour of critical thinking about how to take alerts from found to fixed.”
It all kicks off at 1 PM ET / 10 AM PT, when David Spark will be joined by Matt Brown, solutions architect, Endor Labs, and Joe Harrington, senior security engineer, Principal Financial Group, for an hour of insightful conversation and engaging games. And at 2 PM ET / 11 AM PT, stick around for our always-popular meetup, hosted right inside the event platform.
Thanks to our Super Cyber Friday sponsor, Endor Labs
Thank you!
Thank you for supporting CISO Series and all our programming
We love all kinds of support: listening, watching, contributions, What's Worse?! scenarios, telling your friends, sharing in social media, and most of all we love our sponsors!
Everything is available at cisoseries.com.
Interested in sponsorship, contact me, David Spark.









