The Time Our Provider Screwed Us - Paul Biggar

In October we cohosted a special Test in Production session with Honeycomb.io. Since our monthly session fell one night before Halloween, we decided to invite speakers to share their scariest stories of operational outages and other horrors. Though stories of epic failure can be hard to hear, they are invaluable learning experiences. Paul Biggar, CTO and Co-founder of Dark, former CEO and Co-founder of CircleCI, tells a horrific tale of how a massive security breach nearly ended their company. As a result of Adobe getting hacked, Mongo HQ was compromised, and as a customer, CircleCI was also affected. At one point the team decided they had to completely shut down their service, and no one had any idea how that would turn out. “So it started with a missing IAM key. Now an IAM key is an AWS identity and access management key. And those aren't things you lose. You don't leave them in your door; they're not physical keys, and so that fact that one of our AWS keys is missing is perhaps a cause for alarm.” - Paul Biggar, CTO and Co-founder of Dark Watch the video to hear how Paul’s team handled this event. Learn how they got through the incident with sage advice around transparency from the “Master of Disaster”, Jesse Robbins. If you’re interested in joining us at a future Meetup, you can sign up here: https://www.meetup.com/Test-in-Production/ #launchdarkly #honeycomb #testinproduction Subscribe to LaunchDarkly: https://www.youtube.com/c/LaunchDarkly?sub_confirmation=1 Learn more about LaunchDarkly: Website: https://launchdarkly.com/ LinkedIn: https://www.linkedin.com/company/launchdarkly Twitter: https://twitter.com/launchdarkly About LaunchDarkly: Feature management allows development teams to innovate faster by fundamentally transforming how software is delivered to customers. With the ability to gradually release new software features to any segment of users on any platform, DevOps teams can standardize safe releases at scale, accelerate their journey to the cloud and collaborate more effectively with business teams. Today, LaunchDarkly deploys peaks of 20 trillion feature flags each day, and that number continues to grow. Founded in 2014 in Oakland, California by Edith Harbaugh and John Kodumal, LaunchDarkly has been named on the Forbes Cloud 100 list, InfoWorld’s 2021 Technology of the Year list, and the Enterprise Tech 30 list.