Transcript - Building an Effective Industrial Control Systems Security Program

Welcome to a new edition of Powerful Insights and our continuing series on cyber security awareness. This series is intended to highlight ways organisations can be proactive in addressing critical security challenges. We explore how leaders can dynamically build cyber resilience while maximising value.

I’m Kevin Donahue, a senior director with Protiviti and host of Powerful Insights. In our cyber security series, I’m talking to our leaders who are in the market working with organisations to address these challenges. For this episode, I had the pleasure of speaking Protiviti associate director Justin Turner and manager Derek Dunkel-JahanTigh about security for industrial control systems, or operational technology. Both are with Protiviti’s security and privacy practice and are based in Houston. Derek, thanks for joining me today.

You’re quite welcome – happy to be on the show.

Justin, great to speak with you as well. Let me ask you the first question here. How would your parents describe what you do for a living?

Hey, Kevin. Thanks for having us, first of all. The answer to that – it’s interesting: My family, broadly, I feel like half of them think I work for Geek Squad, so I get a lot of questions about fixing printers and which surge protector or power strip is going to be the best for their home office, and the other half of them think I’m some kind of a bounty hunter/hacker for hire. So, I’ve been asked, “Hey, can you hack someone’s phone for me?” So, somewhere in the middle of the things that I actually do as a cyber security professional, maybe I don’t do a great job of articulating what it is I do for a living, but it is interesting to hear friends and family speculate about what it is that I do for a living.

That’s funny. In our cyber security podcast series we’ve been doing, that is a very common theme. Derek, what about you? What would your parents say you do for a living?

I think it’s a little bit interesting with my parents and family because a lot of my family is in cyber security, but they’re more on the sales side than the technical side, so they’d say, “Anything technical, Derek’s got the answer.” If you ask my wife, she would say that I am helping companies protect their customer’s data, whether it’s credit card, sensitive information, etc., but I think somewhere in between, trying to utilise all the technology to help companies improve their security controls and protect their customers.

That’s great. Thanks, Derek. I wanted to ask you our next question here. One of your areas of expertise is industrial control systems or operational technology security. What is that exactly, and why are those areas so important?

Operational technology is production technology, and a lot of different companies have industrial-control systems, operational technology. The best way to think about it is, these are systems that can take an input and then provide an output on the physical world. For example, in a chemical plant, your industrial-control system will take your processes. For example, you’re Coca-Cola and you are bottling Coca-Cola ­– it will take your secret process, make the Coca-Cola, bottle it, and have all the controls to help each system along the way take some sort of physical impact on the world, whether it’s making something, mixing something, etc. And for a lot of different areas, this is very important, because there can be physical harm in the world.

There have been instances of nation-state-wide industrial-control systems being targeted by nation-state adversaries to take physical harm in the world, whether that’s a pipeline bursting, whether that’s water being contaminated. The real concern is, how do we protect our control systems, operational technology, so that people don’t get harmed in the real world and we can protect the health and safety of our employees in manufacturing in plant networks?

That’s a fascinating topic and, frankly, a bit scary. Justin, what are your thoughts on this? Why is this so important? I would add, do you think companies are really addressing it given the appropriate attention that this clearly needs?

Yes, I think Derek did a nice job at articulating both what it is and why it’s important, and to summarise, we’ve talked about the health and safety and potential environmental impact, but there’s also a direct operational impact to that as well. So, I think some of my plants will struggle putting a number on something like the content of a data breach.

We have an attacker get access to our corporate or enterprise IT systems – we know that there’s a risk and we know that there’s some soft or hard dollar amount that we can attribute to that, but how do we exactly put a score on it, or how do we quantify that in terms that someone can understand? The operational impact of an industrial-control system compromise is a bit easier to understand. If you’re a pipeline company or you’re an oil and gas drilling company, you lose the ability to produce at a given well or collection of drilling sites over, let’s say, 24 hours. Well, you can say, “We know what our volume is here. We know what we produce,” and if we’ve lost the ability to remotely control this facility, then we may be able to put a dollar value on that downtime that may be a result of a cyber security attack.

I think what Derek articulated on the health and safety side is certainly important, but also not ignoring a lot of our clients that have industrial-control systems, and those risks are applicable to them. This is directly impacting their bottom line and their ability to produce and to make money.

Justin, what do you see, or what do you hear, as some of the common myths for securing these ICS environments?

I would say the most common thing, that misconception I see, is that we see a tendency to take controls or tools in technology that work on the enterprise IT network. OK, we know we’ve got a profit here. We know this particular vulnerability-management solution. We’ve been using this on our enterprise IT system. Let’s take this and uplift and shift it, and let’s layer that onto our operational-technology network.

The reality is, there are a lot of nuances with how those systems work, and it really does require a customised approach, and that’s where we help our clients realise some of those things and where we have driven value in being able to speak to “What are those unique challenges, and how do we take the same concept?” We need to be able to pass our systems to address vulnerability, but the way that we go about attacking that is very different on the IT network versus what we would talk about on the OT network.

Derek, any thoughts from you on these myths? I’m also curious to know if you have further thoughts on how you debunk them in talking with clients and companies in the market.

I think Justin did a great job of explaining why some of the IT controls have easily been ported over or addressed in the OT environment. I think another myth that I hear a lot is because a system is legacy, it is inherently insecure and will be hacked.

Now, legacy systems are very common in these control networks. You’ll have PLCs, programmable logic controllers, that do a lot of the process tree, a lot of the nuts-and-bolts logic at the environment – some of these are maybe 25 years old. They may have, someone wrote the logic to it, may have left the company 10 years ago, and you’re still utilising that type of controller to run million-dollar processes, and I think the way that I would debunk that is to say, what is necessary for the system to run, how should it run and how can we secure it from everything else?

I think it’s absolutely true that a 25-year old system is less secure compared to a modern operating system, but that doesn’t mean that that’s not securable. You can still provide appropriate granularity in controls, in network communications, data flows. Normally, whenever we hear these type of concerns, we try to change the conversation around what is expected, where are the expected assets, expected data flows, expected communications, and then you try to narrow your controls to only allow that and then monitor that for additional review and write rules to understand when things go beyond an expected baseline.

That’s fascinating regarding the data of some of these systems, but it’s great to hear that they are still securable, as you mentioned. This probably touches on the same area as Derek, but what do you see as the biggest challenge facing your clients and other companies right now in the space?

I think one of the biggest challenges right now is just the times that we’re in. Especially for a lot of our oil and gas customers, you have lower demand, and market pressures on price. So, these items that we’re talking about, some of these things you can secure with homegrown applications and through better processes, but a lot of it is, you need resources, you need tools. And the ability to make that business case and to be able to move the goal post forward has been a challenge for our customers, and I think we’re trying to help our customers in securing their critical operations so that they can have more resiliency in their automation-control systems that drive their revenue and bottom lines.

Justin, is that your view as well? I was wondering if you might have any examples in the market that you’ve seen where a company is, or companies are, addressing this challenge.

Yes. If we think of macro level or industry-agnostic level, that’s a pretty good summation, what Derek provided there. I would say part of what I’m seeing, though, is organisations that are resource constrained right now. That’s either marketing conditions in cash flow, or they have had to go through a reduction in force and they maybe don’t have as many resources to allocate their cyber security.

So, part of that is that I’m actually seeing a little bit more focus on this area. We have to be really intentional about where we’re allocating dollars and resources in this climate, and so a lot of companies are saying, “Because we’re limited in that capacity, let’s make sure that we’re securing our operational asset, because that is getting more attention at board and at audit committee levels.” I would say on the tactical level, as we work with IT and OT teams, one of the things that they really struggle with is asset management.

Asset management is a foundation for cyber security because you have to know what you have and where it is in order to be able to appropriately secure it. So, I do a lot of work in the upstream subsector of oil and gas with exploration and production companies, and oftentimes, the way that they grow is through merger-and-acquisition activity. And so they’re purchasing wells and acreage that used to belong to other companies, and so they grow by hodgepodging things together, and you may not even have a spreadsheet listing all your different operational technology assets that are connected to your network. You don’t even have a spreadsheet, and certainly not something like a robust asset-management solution.

So, that’s one of the most common things that we do. We come in and do more of a tactical assessment that the very common theme that we find is, you don’t even really know completely what’s out there, so how do we know what risks we’re facing, and then, how do we appropriately address those risks?

Thanks, Justin. Derek, I wanted to also ask you, what’s the one question you were asked most often by companies who have a need to secure their ICS environments, and how do you answer it?

I think the number one question is, “Where do I start?” and I think that can be a very difficult unsolvable problem, seemingly, for companies of larger sizes: “How do I start to tackle this problem?” You start with, “How do you map it?” One byte at a time. I think from our perspective, one thing that we’ve really talked about in our webinar and that we talk about with our clients a lot is, prioritise your risks, prioritise your locations. Not all plant locations are going to be created equal. Some tend to have higher risks than others, due to the revenue generation, the types of systems, the complexity, developer risk framework.

Map it all out and then prioritise: “Let’s tackle risk number one first. Let’s do a pilot approach.” We’ll start with plant location A and then build upon that and have, at the end, an iterative approach where each time you are implementing security controls, you’re learning from that process and taking those lessons learned into the next plant site and hopefully gaining buy-in from each of your OT counterparts as you move across this journey.

Fantastic answer, Derek. I would just add, the other component of that is, we’ve talked about with our webinar and then the title of this podcast, as well for governance. So, also, identifying who’s going to own and drive this program and who’s going to lead that risk assessment activity and get the right folks at the table to help prioritise risk and prioritise assets. So, that governance piece of it is, who’s going to own it, and how do we identify and have multiple liaisons or contacts throughout the organisation?

We’re going to need support from corporate IT, we’re going to need support from enterprise security. And in most of these environments, we’re going to need the buy-in and support of our folks that are on our automation teams or on our operational-technology teams that are operation or engineering focused, because they’re going to be able to help articulate what’s realistic and to be able to speak to what are the challenges down at the field level or at the manufacturing shop floor, and we’ll be able to have the input that’s critical in mapping out the path forward.

This has been a great conversation with both of you – so many interesting themes and issues for these companies to consider. Justin, with respect to ICS security overall, what are you most curious about right now? What are the industry trends that are really standing out to you?

I’m really watching two things: One is an increased reliance on automation – organisations that have had plants or facilities that are relying on older serial or pneumatic technologies and connections that weren’t really networked into the rest of the environment. They’re starting to become more automated, and we’ve been seeing concepts like the Internet of Things down at the center level, and you’ve got even some internet connectivity ­– at least outbound internet from these OT devices out to the internet. I’m looking to see how that trend evolves, and there’s obviously going to be some cyber security considerations there.

And then on the tools and applications landscapes, I’m interested to see if some of the bigger players through acquisition. Microsoft has expanded their ICS security capabilities, so, some of the bigger players in the tech space continue to invest more in technologies that are geared toward these environments specifically and take the challenges of OT systems in mind and the sensitivity of those systems and allow for better security monitoring and alerting of threat intelligence in OT environment. As I’m looking at maybe the next three to five years, those are some of the trends that I’m keeping an eye on.

That’s great. Thanks, Justin. Derek, I wanted your views on this as well. What’s really striking your curiosity at the moment?

I guess for starters, I’m a little bit jealous that Justin got to answer the question first, because he stole most of my good answers, but I think the only things to add onto that are lot of security technologies that are kind of popular in enterprise now. I’m seeing how they get adopted to the ICS environments, that zero trust. A lot of OT vendors are going toward more cloud models, and I’m curious how that gets situated because there’s normally, and for good reasons, been a lot of hesitation from the OT environment perspective to have any sort of connection to the outside world, and that has changed a lot with COVID and remote workforces. It’s going to continue to change with vendors taking more of a cloud-based approach.

And then, from a business and analytics perspective, I think from an ICS perspective, three to five years down the line, I think there’s going to be a lot more integration with IIOT, which is the Industrial Internet of things, and the data analytics. I think that security can be a modernisation driver for these types of products within the ICS environment and can also help in securing these technologies, but also as a technology leader, helping these plant networks get more out of their technology and better utilise their existing data.

Derek, Justin, this has been just a terrific and informative conversation. Thanks for joining me today.

Yes, absolutely. We really enjoyed it.

Yes. Thank you, Kevin. Appreciate you having us.

Thank you for listening today. For more insights from Protiviti on cyber security issues, challenges and best practices, please visit protiviti.com/security. I also invite you to subscribe to our Powerful Insights podcast series wherever you find your podcast content.