https://cisoseries.com/if-the-network-is-up-somebody-is-violating-our-acceptable-use-policy/
This week’s episode is hosted by me, David Spark (@dspark), producer of CISO Series and Andy Ellis (@csoandy), operating partner, YL Ventures. Our sponsored guest is Matt Radolec, senior director, incident response and cloud operations, Varonis.
Full transcript
[Voiceover] Ten-second security tip, go!
[Andy Ellis] There’s a tendency in cyber to overemphasize specific IOCs, and we’re seeing this a lot with Log4j and everyone racing to implement very specific regex-based detections. Intrusions will happen. Prepare to detect what happens next.
[Voiceover] It’s time to begin the CISO/Security Vendor Relationship Podcast.
[David Spark] Welcome to the CISO/Security Vendor Relationship Podcast. My name is David Spark, I am the producer of the CISO Series, and my co-host for this very episode, the warm and lovely – and he’s wearing a camouflage jacket so I can’t actually see him, he looks more like a floating head – his name is Andy Ellis. He’s the Operating Partner over at YL Ventures. Andy, thanks so much for joining us.
[Andy Ellis] Thanks for having me, David.
[David Spark] We’re available at CISOseries.com. If you haven’t spent enough time there, you should. Lots of great stuff. Can access our three-plus years, we’re closing in on four years, of content, or we’re more like three-and-a-half years of content at this point. Our sponsor for today’s episode, and they have been a superb, phenomenal sponsor of the CISO Series, and that is Varonis. Thrilled to have Varonis sponsoring again, and they’re also responsible for our guest. But before we bring on our guest, this episode is dropping on February 22nd, 2022, or 2-22-22. And you’re obsessed with these numeric palindromes, yes, Andy?
[Andy Ellis] I love dates that have cool, like, palindrome effects. Like, March 14th is pi day. What really gets my goat, though, is all the people who complain that it’s not in their preferred date format. And I just want to say if we’re talking about the Gregorian calendar, if it’s not in ISO 8601, don’t sweat it. Just have fun with the date. Don’t worry about the fact that you want to do it as 22/02/22, or whatever you think is a more relevant one. ISO 8601 or bust.
[David Spark] I want to isolate the “have fun with the date.” If you’re having fun with dates, then your essentially barrier or bar for entertainment is pretty darn low.
[Andy Ellis] It really is because there’s a lot of really fun dates. And the nice thing about a numeric calendaring system is you run into them all the time. I mean, the other calendar I work with today is the 21st of Adar I in 5782. We literally repeat the name of a month because we have leap months. This is just weird and I can’t have fun with the date numbers.
[David Spark] Well, you’ll be pleased to know that our guest today, our sponsored guest, we just found out that he had his wedding day on a palindrome day, and it was not by accident.
[Andy Ellis] Which is awesome, I love it.
[David Spark] I’m thrilled to bring him on as a repeat guest, he was actually on our other show and now he’s on CISO Vendor. It is Matt Radolec who’s a Senior Director, Incident Response and Cloud Operations, over at Varonis. Matt, thank you so much for joining us.
[Matt Radolec] Yeah. Thanks for having me here.
That’s something I would like to avoid.
3:02.771
[David Spark] Why do tabletop exercises fail? Over on Cyber Management Alliance, they have some obvious reasons as to why tabletop exercises fail, such as flimsy scenario, apathetic participants, inexperienced host, and lack of preparation. Sounds like the exact formula for failure on a tabletop game like Dungeons and Dragons. So, Andy, I’ll start with you. Think of the absolute best tabletop exercise you’ve ever been a part of, what made it so awesome?
[Andy Ellis] Oh, man. The best ones, they all had one thing in common which was Tim April, a former colleague of mine who’s an amazing incident responder, really…
[David Spark] So, essentially, your Dungeon Master, if you will?
[Andy Ellis] Basically, he was the Dungeon Master, and there were a lot of people who went into building a tabletop. I don’t want to say it was all Tim, but he did a lot of the scenario planning. And what he would do is he would take a vulnerability and risk he was actually worried about, so this wasn’t an artificial, “Oh, let’s just pretend there’s a credit card breach.” No, no. It’s, “This is exactly how the system is going to fail that results in whatever you initially detect.” But oftentimes what happened is people want to go from an outcome. They say, “Oh, let’s pretend we had a credit card breach and now work backwards to create a scenario,” but the scenario ends up not plausible. When people tried to do detection forensics in the tabletop, they’d come up with things that the planner never even thought to think about and say, “Oh. Well, what would that look like?” and so they’re making it up on the fly.
And there’s a tendency for people who are gamemasters to want to feel cool, and one of the lessons I learned writing Assassins games that we played live-action roleplaying when I was at MIT is that gamemaster coolness is inversely proportional to player coolness. So that tendency for the person running the scenario to make things up on the fly to get cooler gets in the way of the participants being cool and clever and solving the problem because it becomes about how cool is the scenario rather than how cool is the execution.
[David Spark] By the way, I’m glad you answered that because my feeling is for tabletop exercises, for panel discussions, the success or failure leans on the moderator or the person sort of running the tabletop. Matt, what do you say? From your experiences with tabletop exercises, what makes it so awesome and what is the key formula, you think?
[Matt Radolec] I think one is using a real scenario. So, we would often take, when we did these tabletop exercises, something that actually happened to a peer of ours – another firm or another organization – and we would replay their incident where all the knowledge, all the details were already in the public domain. And we would just experience, “Well, how would we have handled it? What steps would we have taken? How would we have figured it out?” And it made it very real and very predictable, because a lot of our executives that participated could go and read about it ahead of time. They could read and see things that maybe they thought, “Oh, wow. I don’t like the comment that they put in the public. I wouldn’t make the same comment.” And it gave people a safe space to do some learning who didn’t have experience with it before, and so taking real incidents, things you read about in the news, is a great way.
Another thing is we’ve also done the “Choose your own adventure” where we had some preplanned scenarios. And depending on the decisions that the tabletop participants made, a different path would unravel which allowed there to not be some of that gamemaster ambiguity, as they would say, where things would take a turn because everything was very preplanned. We had a list of menu options to choose from and we were able to get through several scenarios that way, which was a really great way to warm up our executives to the idea of even how would we handle multiple incidents at once.
[David Spark] So, for those who are starting their very first tabletop exercise, and I’ll ask for quick responses from both you Andy, and Matt. What should they prepare, because one of the complaints I hear is how kind of bored people get, what should they prepare to avoid that?
[Andy Ellis] So, I think first thing is what is the success criteria for every participant. What are they going to learn, how are they going to learn it, assuming the scenario goes as planned? And also recognize you get to fast forward. If somebody says, “I’m off to go debug some logs,” hand them an answer immediately. Don’t make people waste time because this is just a tabletop, it’s just a practice. If you want to have somebody practice grepping through logs, do that separately from your tabletop.
[David Spark] And Matt, your thoughts?
[Matt Radolec] Decouple the management track from the technical track. So, go through the entire simulation with the management track completely independent of what happens on the technical track, and don’t make them rely on them. You want to do that in reality too.
Maybe you shouldn’t have done that.
7:49.712
[David Spark] An Acceptable Use Policy is a document that nobody reads but spells out appropriate ways to use the corporate network or any corporate-sanctioned equipment. It is violated all the time, knowingly and unknowingly. So, what do you do when someone violates it? What about repeatedly? We’ve been told that scolding through more security awareness training is not appropriate. And amazingly, while there are articles that claim to give advice on how to deal with this, there’s essentially no advice. The attitude is we’ll need to figure it out ourselves. So, I get the sense that most don’t want to enforce or discipline for a violation of an AUP, Acceptable Use Policy. So, Matt, how have you dealt with this or seen others deal with this? And obviously, there’s varying degrees of violations.
[Matt Radolec] One thing is definitely partner with Human Resources, with General Counsel on these sorts of things because there’s probably a precedent. There’s probably a, “This is what we do when people violate policy,” type of a thing, it just hasn’t necessarily been applied to technology or security. But one of the things that’s worked well for me is actually a lot like a traffic cop, or even like an automated speed camera or a red light camera. If you’re able to set up policy violation notices that would send someone a notification, an email message that says, “Hello. We noticed that you moved some data to an unauthorized Sass repository. Just so you know, that’s not in accordance with the company’s policy and this is a written warning.” That alone lets people know, number one, that the watching is occurring but also it’s just a… We called it a policy reminder, it reminded them that this policy existed in case they had forgotten.
[David Spark] Andy, what about you? By the way, you were at Akamai for 20 years so I’m sure you had to deal with this many times.
[Andy Ellis] So, it’s really important to recognize that there are two kinds of policies. There’s one which is set to be enforced, those often come from the CISO. Like, “We do not want you installing malware on your devices through the following five ways.” Great. That’s a policy that you would enforce just like anything else, even if it comes up under the AUP. But there’s a lot of policies, and most AUPs fall into this category, that are typically generated by the Legal Department, and they are CYA policies, which is “cover your derrière” since we try to be clean on this show.
[David Spark] So, CYDs.
[Andy Ellis] So, CYD policies.
[Laughter]
[Andy Ellis] And you should not try to enforce those. The whole point of those policies is realistically so that if somebody sues the company after some incident, the company would say, “Hey, look. This was the employee’s fault.” It’s like the, “You don’t talk on your cellphone while driving.” Probably the single most violated policy in corporate America because people hop onto calls when they were commuting home, at least before COVID. Now, COVID – hey, look, everybody follows that policy because we don’t get into our cars. But if you the security team are trying to chase down these CYA policies, you’re basically putting yourself directly athwart the company and the lawyers don’t really have your backing. They’re not interested in enforcing the policy, they just don’t want to know the policy’s being broken, and so you don’t make any friends by noticing and trying to enforce a policy that wasn’t meant to be enforced.
[David Spark] All right. Excellent distinction. When someone did violate one of the first policies where they really have to adhere to it, how did you deal with it and did you ever have to do more sort of tougher discipline?
[Andy Ellis] So, the only time we did really tough, tough discipline was when somebody hid the fact that they had done so. Like, “You violated a policy? Great. Let’s have a conversation.” Maybe we’re going to involve your manager, your manager might be grumpy with you for bringing you to the security team’s attention. But our general attitude is if you violated the policy, let’s educate you on why this is a problem, make sure you understand, figure out was there a reason you were violating the policy. Like were you trying to get your job done and the policy was in the way? Because I can fix policy. Or was this, “Oh, you just wanted to install an IRC server on a production system?”
I literally had that incident happen. Somebody was running their own private IRC server on a production system because they’d never had access to that much bandwidth in their life. And so when we went to have a conversation with them, they deleted everything and said, “Ah, I didn’t do it.” And we’re like, “Oh, look. We have all the evidence, but if you’re willing to hide that from us when you know we’ve got the evidence, we don’t know what else you’re going to do. Have a nice day, enjoy your career somewhere else.” And usually it’s their managers who come in hard, it’s not usually the security team saying, “Oh, you have to fire that person.” Their manager opens right up with, “Oh, I can’t trust this person,” if they’re going to hide that from you.
[David Spark] Have you had something similar? That’s kind of a good distinction, Andy. People violate these all the time, mostly to just get their darn job done. But the hiding fact, that’s a trust issue.
[Matt Radolec] And what I try to do in these scenarios is use the enforcement of policy as like a entry-level insider threat program. Because if you just start looking for people that are doing things that are outside of acceptable use, you’re bound to have someone that might have bad intent, and that’s what we’re really talking about here, right? Someone that either had malicious intent, bad intent, or naive intent, which might be the case for Andy’s example.
We had one that really stands out for me where identified a person who had uploaded 27 gigs of information to an external website, and when confronted about it, said, “Oh, yeah. I was backing up Netflix DVDs.” Well, that didn’t quite make sense, right? You were using your company-provided device to rip Netflix DVDs to Dropbox? Things didn’t compute and it didn’t compute with all the other logs that we had. And that was something that went all the way to the highest levels of the organization, as it was seen that not only was this person actively trying to hide it but they were creating a different story. They were creating a narrative that matched the outcome that they wanted, not the one that we had identified.
It’s time to play “What’s Worse?”
13:49.541
[David Spark] In the spirit of your opening suggestion about Log4j, and by the way, we are recording this at the end of December although this is airing at late February, so everyone knows.
[Andy Ellis] So, we hope Log4j is over and in the past, and you’re all wondering why we’re bringing up ancient history.
[David Spark] Right.
[Andy Ellis] And not it’s been a thing you’ve been patching every week as we discover more flaws for the past two months.
[David Spark] Which also, interesting you say that, Andy, because depending on how you answer this, it may be a very different answer come late February, okay? So, I just want to put the context for everybody.
[Andy Ellis] Oh, boy.
[David Spark] Matt, you know how this game is played, correct?
[Matt Radolec] Oh, absolutely.
[David Spark] All right. So, I’m going to make Andy answer first, you answer second. I always love it when our guests disagree with our host.
[Andy Ellis] And I, of course, love it when you don’t disagree with our host.
[David Spark] There you go. So, you’re either going to make me happy or Andy happy. You pick, all right? Here we go. It comes from Nir Rothenberg who’s the CISO over at Rapyd. He has supplied many, many of What’s Worse scenarios. So, patching Log4j three times and counting in a week, which creates distrust and will slow cooperation for the next vulnerabilities. Or the other option is waiting a week for a stable patch and relying on other mitigations like your WAF or firewall rules thus retaining high buy-in. So, you could have low buy-in in the first situation, you may get high buy-in on the second situation. Andy, what’s worse?
[Andy Ellis] Oh, I think Nir’s got it backwards. I think you have low buy-in in the second situation, and I’m going to easily go with that one is worse, and I’m going to explain why. Because if you’re willing to say this first thing that came out, first version, had an RCE triggerable by input that is passed from app to app, and you’re willing to say, “Well, I’m going to wait until the patch stabilizes and that’s going to take a week or so,” you have just told everybody in the business that this patch doesn’t matter. That you’re willing to wait.
Whereas instead if you said, “Nope. We got to patch it. Oh, and look. The patch came out and it’s got a CVSS 3.8.” And then two days later, like, “Oh, no. It turns out that’s actually a 9, but still at least it’s not a 9.8.” And then the third version actually has a DoS in it, but these are getting incrementally better. Now, you should plan and communicate that. You should say, “Anytime we’re doing a zero-day patch…” And we’ve seen this with almost everything, like Heartbleed was the exact same thing, that went through three versions. If you were smart, what you did was you compiled that one with UDP support off, and therefore you didn’t have to worry about it at all.
But no, I absolutely think just waiting, because at what point do people trust that this is the stable patch? You waited a week, there’s been three revs. Everyone’s going to say, “Well, I’m going to wait for the fourth or the fifth rev. Yeah, I’ll let somebody else go first.” You’ve already made it not urgent.
[David Spark] So, doing five patches in a week – far less worse?
[Andy Ellis] Absolutely, and you telegraph it. You tell everybody, “We are in a zero-day. There’s a chance there will be revisions to this patch because it got rolled out so quickly.” If we could rip out Log4j, like there are people who did that instead, went to an alternate library. Of course, it’s a library nobody’s looked at, so maybe in a month they’re going to be panicking when everybody looks at it and discovers they have a set of vulnerabilities. But demonstrate that you can be agile, be humble, recognize you’re not going to get it right that first time. But the worst thing you can do at that point is say, “No, no. You don’t need to patch right now.”
[David Spark] Matt, agree or disagree?
[Matt Radolec] Can I go with option C?
[David Spark] What’s your option C?
[Andy Ellis] David never likes that.
[David Spark] No. You can’t have an option C. So, what’s worse? You have to pick of two bad scenarios.
[Matt Radolec] Okay. Well, I’ll do both. So, option C is policy, this comes back to policy. Having a vulnerability management policy that says, “What you’re going to do when a zero-day happens, it means that you can go with the latter.” That I will agree with Andy and say, “You can patch 10 times in a week if there are 10 patches.” Because you have established the ground rules for a scenario like this, and people know that when the time comes, that it might mean you got to apply five, six patches. So, if you’re grounded in policy and that policy is to patch as things are available based on the severity, then people are prepared. They’re not surprised by the fact that you have three updates to the first update.
[David Spark] So, you’re agreeing with Andy that the second, waiting a week for the stable patch and relying on your other mitigations like the WAF and the firewall is going to cover you? Or that’s the worst situation?
[Matt Radolec] That’s the worst of the two. And completely agreeing with Andy in a sense that you are downplaying the importance of patching at that point. You are prioritizing. Let someone else override that, let someone else say, “We have to wait.” Be the one that says, “No. We got to do this now. It’s very important, it’s 9.8.”
[David Spark] All right. You made Andy very happy. I don’t know how much nicer I’m going to be with you for the rest of the show.
[Matt Radolec] I don’t know if I’ll get invited back but, Andy, I’m there with you, I stand with you. [Laughter]
[Andy Ellis] Yeah. No, I’m really happy that you’re there with me. And look – if you’re in this situation, which many of you have been – the way to make a friend, because look, you’re not going to make any friends no matter what in Log4j, is you come out of it and you say, “Look. I know we had to do three patches in a week. What made that hard and can I help get some of those challenges out of your way?” Recognize the pain of just software development. And if there’s anything you as the CISO can do to help smooth the CI/CD pipeline, that’s how you make friends, when you say, “Look. I recognize… Wait. Is it my policy that caused a problem? Let me fix that.”
Please, enough. No, more.
19:22.633
[David Spark] Our topic is cyber resiliency. And I’m going to make this claim here. I think that was probably the biggest new theme of cybersecurity in 2021. Because most have – if they didn’t realize it before – most realize, “We’re going to get hit, how can we withstand the damage?” So, I’ll start with you, Andy. What have you heard enough about with cyber resiliency, and what would you like to hear a lot more?
[Andy Ellis] I think I’ve probably heard enough about AWS zones. I’m tired of hearing about, like, “Oh, AWS had an availability zone go down and all these companies had issues.” Either accept that you’re going to occasionally be down or get to multizone support. But I’m tired of hearing about that, I think we all know that. I don’t think I’ve heard enough about DNS and BGP. Like, it’s always DNS or BGP in many of these large-scale incidents, and I don’t think people are thinking well enough about how to be more resilient in both of those platforms.
[David Spark] All right. I throw this one to you, Matt. Same question – what have you heard enough about with resiliency? By the way, I’m going to ask both of you, would you agree that this was kind of the theme of 2021? Yes?
[Matt Radolec] Absolutely. It came out of business resiliency from COVID. Cyber resiliency was a natural derivative of the increase in cybercrime and the acceptance of these catastrophic events. They will happen, it’s going to happen. When will it happen to you is the million, 10 million, or potentially even larger $100 million question. I would say that the first thing that comes to mind for me, what’s been over talked about, is recovery, just like backup and recovery. If something bad happens, everything is recoverable. We will recover, but it might take us a lot of time.
I think that you’re ignoring the fact that how can you make the damage less, how can you not have to recover as much? Moving the idea of stopping this catastrophic event, this incident earlier in the storyline, that’s what I’m really passionate about. How can you limit the potential? How can you constantly be simulating yourself as being under attack so that you become more resilient to whatever it is? Whether it’s a Log4j exploitation attempt on something that you didn’t know about, or the next zero-day. Which if things happen like they did last year, will be in late January or early February.
[David Spark] How have you at Varonis been working with your clients in this issue? And I’m sure it’s ramped up like crazy in 2021. What are they coming to you about?
[Matt Radolec] A lot of it is specific around cybercrime. People are concerned about the exfiltration aspect of cybercrime. The ransomware aspect, a lot of organizations, they’re concerned about getting it deployed and bringing down their systems, but they’re mostly concerned about reputational loss, brand damage, associated with their data being posted online. So, we’ve been focusing a lot on making that harder. We call that reducing the blast radius of an attack. How can we limit the amount of information, systems, applications, that any one user has access to so that we’re making the attackers have to move beyond that initial entry point to get to good data?
Because every time you make an attacker’s job a little harder, you’re giving your detective controls a chance to pick them up. So, if you can’t get to a lot of data, and maybe you can’t exfiltrate a lot with that first user account that you compromised, which most likely is going to be a regular end user, and you make that attacker have to get administrative account or a service account, you’re giving yourself the opportunity to catch that act. You’re giving yourself the opportunity to identify that anomaly. And every time that you do that, as you get closer to a zero trust model, you’re making it easier to pick up on those nuances which could have identified Log4j or SolarWinds exploitation before we even knew that those zero-days existed because you focused on detecting what happens after the intrusion than just the intrusion itself.
[David Spark] What layers are we talking about here, Matt? Obviously, I’m hearing micro-segmentation. What layers are we talking to build that?
[Matt Radolec] We talk a lot about zero trust and specifically zero trust about data. So many organizations don’t focus on that data layer of security. They’ve done network security, they’ve done endpoint or server hardening, but they’ve done nothing to limit the amount of data that’s accessible to an individual user. On average, we find about that an entry-level employee has access to more than 25 million files. That just doesn’t seem like that’s what they need, and we really do drill that into a lot of the organizations that we work with. We want them to understand that it is possible to overcome that, that not everyone needs access to everything, and limiting that makes an attacker’s job of finding and exfiltrating data a lot harder.
[David Spark] I’ll throw this to you, Andy. It is easy to give access to everybody because it just makes your life easier when you’re setting somebody up. But my feeling is trying to figure out what people need access to to do their job that doesn’t slow them down when you’re like, “Oh, I didn’t plan for this and this and this,” how do you prepare for that?
[Andy Ellis] It’s a huge challenge, right? Especially with the proliferation of things like Google Drive, Office 365, where people want to be able to say, “Look. I wrote this blog post and I need it to be reviewed, but I’m not even sure who’s going to review it. So, do I need to manually add people to it? Or can I just send out a link that anybody in the company could click on, but if they don’t have it they’re not going to get to it?” Literally, I do this when I write blog posts, right? I send them out to a bunch of people but it’s an open link, anybody can go look at it. Because it’s just so painful to do anything other than that.
I think the bigger challenge that lot of companies have is they have old data that nobody is accessing but it still has that level of permission. So, it’s one thing to say, “Look. I’m actively working on this thing. I need to work with an indeterminate number of people.” But at some point, I stop working on it and so does everybody else. And you’d really like to have some better and more automated way to just sort of flag or turn those down to now somebody has to ask for permission rather than just being granted to you. Like your financial reports for the last 10 years. Somebody who walks into the company now doesn’t need to see that just because they know how to do the right search.
[David Spark] Matt, how are you managing this very thorny issue?
[Matt Radolec] We focus on restricting access to sensitive data first, so patient records, financial records, the HR records, legal, financial information, all these regulated data types or this high-risk information. If we get rid of the open access to that first, Andy can still share his blog post with a link because it probably doesn’t contain sensitive information. In fact, that classification of that is probably public.
[Andy Ellis] It will be once it gets approved.
[Matt Radolec] It will be. It’ll be public once it gets approved. But if Andy did that same thing with the secret recipe, probably not the right way to treat the secret recipe. And it is possible, right? These are things that you can do at scale. We focus on a lot of that with our clients as well as helping to extend that to other business processes. One of the things that you mentioned earlier I wanted to expand on is a lot of times it’s seen as impossible to figure this stuff out. But just think about having all the usage data. Knowing what types of people access what types of information is a great way to get that process started. You’ll learn who touches PII, who generates PII, who doesn’t. This can help you set up people for the right level of access.
There’s got to be a better way to handle this.
26:40.214
[David Spark] “Visual detection systems help analysts make sense of alerts and events flagged up by automated systems. Visual investigation techniques reveal new threats that would otherwise be missed by automated systems,” and this was said someone at the Department of International Trade The Netherlands in an article on Medium. The article outlined three different use cases for visual detection techniques and they are visual threat detection, investigation, and sharing intelligence. Matt, I’ll start with you. Where have you seen the most significant impact of visual detection techniques that just couldn’t be done before without visualizations, or at least not done simply?
[Matt Radolec] One of those is going to be in post-based anomalies with connections. So, if you think about your network and all the systems that you have, everything is interconnected. Servers are talking to other servers, workstations are talking to servers over different ports and protocols. When you can visualize all those connections like a big massive spider web, you might realize that there are hosts in your environment that have far greater connections than you ever could have thought of before.
And when you look at that in something like an Excel sheet where you see that this one host has, like, 40 ports and hundreds of different connections on it, you don’t think as much of it as when you see this cluster on a visualization and you scratch your head and go, “That’s not a domain controller or an Exchange server. That’s not a public device inside of my network that everyone should be working with. Why does this one workstation have so much traffic coming in and out of it?” And that’s been an area of great success on the investigative side for us, really trying to visualize the impact. Or once you identify a patient zero, that’s drawing that spider web between patient zero and everyone else. Visualizations is really, really key in that area.
[David Spark] Andy, what about you? What have you seen with visualizations?
[Andy Ellis] It is challenging but I think Matt hit it on the head. There’s got to be that head-scratch moment of something that doesn’t match. I’ll give an example, actually, I illustrated [Phonetic 00:28:41] a system disaster resilience. We were looking a long time ago at every system in the company and how well-prepared it was for an outage. And so they got a score from one to five. One was you have one machine and if it fails you have no idea how to rebuild it, to five is you’re hot, hot, multi-available, multiple geographic zones. And so we went to everybody in the company, we said, “Tell us about your systems. Just give us the names. Tell us what their disaster resilience looks like and then tell us who they depend on. Tell us what systems you depend on, and if that system failed how much it would impact you,” and we had a simple little calculator for it. And then basically we color-coded every system where the center of the system was from very light to very bright, so it was white to bright red based on how resilient it was. And then the outside was based on how dependent other systems were on it.
And so what we discovered is we had this system source code repository that was not at all resilient, literally it had cold backups was the best it got, but other systems relied on it in real-time. Literally, we had dynamic updates being written into source code in one place, read somewhere else. And if those updates stopped, that system was going to have a hard time, and it would affect so many downstream things. Nobody had ever told that manager and said, “Hey. Your source code repository, we need that to be up 24 by 7,” because they were thinking, “Eh, if it goes down, if it’s business hours it matters, but if it’s off hours, fine, developers can’t check in code for a couple hours. We’ve got time to get it back up.”
So, it’s that aha moment of these colors just didn’t match, and somebody could easily look at it and see that that’s a problem. And so that’s what you’ve got to figure out how to get to is how you’re going to find that aha moment without knowing it in advance. Because if you know what the aha moment is, you could just write a query, you don’t need the visual graphic for it.
[David Spark] And I think that’s what I’ve found so powerful about these visualizations, but it’s all about the setup, like how are you doing the setup. The classic thing I like to say about comedy is comedy is not about the punchlines. It’s about having a great setup.
[Andy Ellis] Right.
[David Spark] And if you have the right setup, finding the problems, like where all of a sudden it appears to you where you’re just [Inaudible 00:30:59] glancing at an image, all of a sudden, “Oh. Now I know what to go into.” So, it’s the creating the backbone, yes, Matt?
[Matt Radolec] Yeah, exactly. Building the infrastructure to be able to do it and then using it a lot. That’s how you make visualizations a part of everything that you do. Not just thinking, “Hey, I want to visualize this one thing.” It’s, “How do I build a platform or an infrastructure to be able to visualize many disparate problems?”
[Andy Ellis] Like if you want to understand how hard this is as somebody who’d coming at it from the outside, go pick up your favorite map of your local country with your roads on it, whatever. And just start looking at it. And you’re going to discover some things that are like, “That’s really weird.” Like in the US, we have what’s called the Northwest Angle, which is this tiny little piece of the US that’s actually looks like it ought to be in Canada, it’s on the north side of the Great Lakes, not connected to the US. And when you spot it, you’re like, “That’s weird,” and that will send you down a rathole of treaties all the way back to the Treaty of Paris to figure out why that’s weird, right? But if you don’t know that exists, you would never find it other than visually, and that’s the same problem. Is you got to put all the data there and then hope you just find something on inspection.
[David Spark] I’ll tell you something that I saw that was similar to that. I was watching a planetarium show that were showing these amazing satellite video and images of the world. And it was literally crossing countries, so you saw whole countries live. And it was the image of North and South Korea and how South Korea’s lit up and North Korea is dark. It was bizarre.
[Andy Ellis] Didn’t you know? North Korea permanently celebrates Earth Hour.
[David Spark] Good point.
Closing
32:39.860
[David Spark] All right. That brings us to the very end of the show. I want to thank our guest, Matt Radolec of Varonis. And I want to thank Varonis for being our phenomenal sponsor of today’s episode, and for many things on the CISO Series. Matt, I’ll let you have the last word and things I always like to ask – are you hiring? If you have any last comments. And if you’d like to make an offer or anything you would like everyone to know about Varonis before we wrap up. But first, Andy, any last thoughts?
[Andy Ellis] Yep. Well, as always, if you want to go in a very early stage startup, YL Ventures is hiring, or at least our companies are, so jobs.ylventures.com.
[David Spark] And that takes you to all the other?
[Andy Ellis] That takes you, like, we aggregated them all in one place. We actually have a Chief HR Officer to support all of our startups now.
[David Spark] There you go. Very good. So that’s if you want. But if you want to work at Varonis, you are hiring, yes, Matt?
[Matt Radolec] Oh, yes, and in nearly every department. Whether you’re a developer, account management, engineering, or even on my team, the Security Incident Response and Cloud teams. We have openings all across Varonis so definitely check out our website for that. And if I could make a plug to everybody who’s listening, everything we’ve talked about today, whether it’s zero trust on data or cyber resiliency, these are all things that your local Varonis account team can help you with and show you what datacentric cyber or the data-first approach looks like. And we’re happy to educate you on that and really show you how it’s possible, in the realm of possible, on gaining control over your data.
[David Spark] I’m going to also throw this out and tag onto what you just said, Matt. And that is I don’t think enough security leaders take advantage of the knowledge that they can learn from their existing vendors and other potential vendors. I know there’s this fear of getting trapped into some marketing funnel as well, which is understandable. But the vendors are out there to not only improve themselves but to educate others and also bring on customers as well. But they have a lot of education to offer. So, I truly say to take Matt up on his offer, where Varonis can educate you as well. Speaking of that, Matt, if someone wanted to get in contact with you, how would they go about doing that?
[Matt Radolec] They could go to varonis.com and just say, “Reach out,” or they could find me on LinkedIn, very responsive on LinkedIn, you can just search Matt Radolec on LinkedIn or send me an email at radolec@varonis.com.
[David Spark] And I will tell you on this episode, if you go to the blog post, I will link to your LinkedIn profile, so that’s the fastest way to find you as well. Thank you very much, Matt. Thank you very much to Varonis as well. Thank you, Andy. And as always, I say to our audience thank you so much for your contributions and for listening to the CISO/Security Vendor Relationship Podcast.
[Voiceover] That wraps up another episode. If you haven’t subscribed to the podcast, please do. If you’re already a subscriber, write a review. This show thrives on your input. Head over to cisoseries.com, and you’ll see plenty of ways to participate, including recording a question or comment for the show. If you’re interested in sponsoring the podcast, contact David Spark directly at david@cisoseries.com. Thank you for listening to the “CISO/Security Vendor Relationship Podcast.”