Centralized Security in a Decentralized Org

Chris Holden, CISO, Crum & Forster

Chris Holden was interested in cybersecurity from an early age and decided to attain his Bachelor of Science degree in Cybersecurity and Information Assurance during college. After graduating, he applied for an internship with HP’s Cyber Forensics department and was hired immediately, later finding out he was the only person who applied with a degree in Cybersecurity. After holding various cybersecurity jobs, Holden began working at Crum & Forster, a leading national property, casualty and accident & health insurer with a large, diversified specialty platform.

Alright, there we go. So, alright, everybody, thank you for sticking around. So, my name is Chris Holden. I’m the CISO for Crum & Forster. We’ll go more about Crum & Forster later. So, I want to talk about a little bit today centralized security in a decentralized organization. So, basically, the way I approach or we approach cybersecurity internally and build it out for our company, who it’s highly decentralized.

So, a little bit about me. Who am I? I’m the CISO at Crum & Forster, as I mentioned. I’ve been in cybersecurity for about 10 plus years. I’ve held multiple positions all the way from forensics, incident response, to penetration testing, application security testing, and cyber program development, both internally and in consulting capacities over the years. I live in upstate New York with my wife and newborn son. And I only added this last piece because I was asked this the other day, and I didn’t have a good answer for what my hobbies were. I think the pandemic got us a little lazy and having something quick on that. So, carpentry and handyman was the only thing I could think of, because I purchased… I thought I was going to get in real estate investing a couple years ago, and now I just am a plumber and painter and everything else under the sun there.

So, Crum & Forster. So, Crum & Forster is a market leading property and casualty accident and health and specialty insurance solutions company. We’re a $3.1 billion gross premium written. We have about 3000 employees, and we’re actively hiring lots and lots of roles. We’ve grown significantly over the last couple years since I’ve been there. And we have about 37 offices across the US and a full remote work policy.

So, our offices are roughly I’d say 2500. As you can see here from this slide, we have multiple business units. And that’s going to be a central topic for what we’re going to talk about in a little bit. And my security team… and so the way I describe Crum & Forster also is 2022 will be our bicentennial year. So, we’ve been around for 200 years. IT has been in house for approximately just over 20 years. And cybersecurity started 6 months prior to me joining the company 4 years ago. So, all in all, relatively new, built on cybersecurity program, going back very legacy solutions, and now keeping up with some of the new changes.

So, as the CISO, I have 7 full-time engineers under me, 2 of which are focused on cloud and application security. And I do have 2 open positions, potentially 3 coming up. That will be for my team. So, as I mentioned earlier, we are running 6 very different and distinct corporate infrastructures. So, we are all central… our business is centralized and insurance, but we let the business units operate in a very autonomous operation to spark… from the business perspective, this is to spark creativity and competition among the different business units. So, what this means for us from a technical standpoint is that we’re all multi-cloud and we’re all… and each business unit has their own development teams, their own architecture. So, we get what we discussed earlier today is the sprawl.

So, for this, what we found over the years of adding security into the company is that we cannot push security, security needs to be pulled by the business units. So, how we’ve gone about doing this is we have elected essentially security ambassadors from each team, and they are our direct contacts for the team. We’ve centralized communications. Each business unit is assigned to a security member on the team. And we help them any direct all contact that way.

We’ve also started implementing vendor-less policies. So, our initial approach was, “We’re going to go out. We’re going to buy the security solution. And guess what?” You’re going to use that security solution. Nobody wants to be told what to do. People want to if they have input, and they can help build the program, the procedure implement the technology, they’re much more willing to adopt. Your return on investment is much quicker. And general acceptance is much easier to get past.

So, what we started doing was with a vendor-less policies, we also started offering with those policies a centralized solution. Right? So, if we had a web application firewall that we needed, we would say, “Okay, you need a web application firewall. It needs to meet these criterias. You have to work with governance and compliance on these few things every quarter, month, year, whatever. And you can use whatever solution you want. But we have an offering for you that you don’t need to go out and buy, you don’t need to go out and configure, you don’t need to go out and monitor.”

So, what we’ve found over the years is we’ve had 1 instance where somebody says, “Well, I’m going to go by my own solution that lasted 3 months,” and now they’re on boarded into our solution from there. Another big piece that drove that effectiveness is the way we started structuring our proof of concepts when we were looking for new technology to onboard. So, we started building instead of security going out and all the responsibility of testing the software, designing the software, being on security, we got the stakeholders involved from the very beginning, gave them some input.

This helped kind of on 2 different facets. So, 1, gave them ownership of the product. And 2, being in security, we’re not developers. We’re not the operations team. We’re not the business units. There’s nuances of how they work and what they need to get their job done that I might think they need this tool, but I don’t know the nuances of their day-to-day well enough to really apply that. So, getting them involved from the beginning has been an absolutely imperative task to drive delivery and timely delivery of, once these solutions were identified, to onboard them. And then I already mentioned we are providing them business freedom of choice, which ends up coming back to the security solution 9 times out of 10, or 10 times out of 10 now, I should say.

So, as I mentioned, we’re a 200-year-old company. IT was very, about 20 years old. And when I first joined was a very legacy environment. On-premise data centers, little to no cloud whatsoever. And like I said, this was about 4 years ago. That quickly and abruptly changed. And we now have the amplification for the need of secrets management. So, secrets management has always been there. It’s always been an issue. It was just different approached in a much different scale.

So, prior to going to the cloud, we still needed to maintain encryption keys for database servers and stuff on-premise, and which we often saw hard coded in credentials and configuration files, not the most secure way to go about that, potentially, but everything was on-prem. We thought, “Hey, we have the fire… we have firewalls and our internal networks trust it. Why not?” Right?

So, then we start moving abruptly to the cloud. And as I mentioned, we had the post cloud, the sprawl. Right? We have hundreds of accounts, multiple different organizational accounts, different structures, applications that depend on AWS, Azure, NGCP, all for one application. Access keys are now of developers being stored locally on desktops.

It becomes this massive, massive, uncontrollable problem, and we have no centralized way to revoke those keys in the event that something happens. As I mentioned earlier, I’ve spent a lot of time in forensics and incident response throughout my career. The response to an incident is top of mind to me. So, the revocation was a very, very important piece. Right? I want to make my life as easy as possible, when things go haywire.

So, what are our options now for secrets management? So, we identified a risk of requiring secrets management, as I mentioned, prior to any involvement in the cloud. So, about 2-ish years ago, we started looking into and formally set up a project, started going out and looking what was available to us in the secrets management world. We had recently completed a project where we had done some privileged access management, the human centric approach that we’ve discussed earlier today. And now we needed something for our machine-to-machine access that was a little bit more centralized and better than we were going.

So, what were our options when it came to the secrets management? We started looking at your traditional privilege access management companies. And these were… what we found is, yeah, they offer secrets management, but they still took a very human-centric approach. Right? They still took the human the login, check out, and the access that we’re used to from that standpoint. It was great for classic administrative users, but didn’t really scale to the level of the DevOps use cases that we really needed. So, that wasn’t really going to work for us. It wasn’t going to… we didn’t find that to be a good use of our time or investment to really move forward in any one of those directions.

So, then, what we did, and maybe a mitigating factor until we can centralized was we started looking for the cloud providers and their KMS tools, which arguably very good, however, very limited set of use cases that can be provided for those instances. They’re also squarely focused on just the API static secrets. And the management was all throughout the account level, there was very little you could do from a centralized standpoint for an entire organization, going back to my response. a nightmare. So, again, that wasn’t going to work, especially as we started going to a hybrid and multi-tenancy cloud environments for our applications that needed to… this just didn’t scale.

So, then we started looking at the open-source self-deployed. And one of my favorite sayings about open-source is, you’re pulling an open-source like, “Well, a lot of open-source, or the free software, it’s free, as long as you do not value time.”

And what I mean by that is the engineering capabilities that are required and needed to stand these solutions up, get them operational, make them secure, get the resiliency that we need, as well as the integrity that must be built into secrets management was mentioned earlier today also. Like, secrets management’s great, but if you cannot access your keys or you’re down for any avail… about a time, there’s an impact your availability, that’s also a major, major issue for secrets management.

So, we found it to be very engineering, the cost of ownership on that, from the engineering perspective and the infrastructure perspective, to be very high and something we didn’t support. As I mentioned earlier, I have 7 engineers, I don’t have time for people to be dedicated to patching and updating and stuff like that. So, then, we have Akeyless, who provides us the best of all the worlds. Right?

Modernly built for a new legacy and new and legacy use cases, the hybrid and multi-cloud support. And one of the biggest selling points for us was the truly software as a service, their software as a service nature of the Akeyless product, which means zero maintenance for me. And then as well as a strong encryption model, because that’s the next worst thing that can happen after your… first worst thing that could happen is that your keys system gets compromised. Second is that it’s not operational, and there’s no availability.

So, we found that to have a much lower cost of ownership and deployment was much quicker. So, it’s satisfied from the business perspective, those 2 criteria of a quick return on investment and an effective long-term solution that we were looking for. So, thank you.