https://cisoseries.com/decommission-our-legacy-tech-or-just-shut-down-the-business/
This week’s episode is hosted by me, David Spark (@dspark), producer of CISO Series and Andy Ellis (@csoandy), operating partner, YL Ventures. Our guest is TJ Mann (@teejaymann), CISO, Children’s Mercy Kansas City.
Full transcriptI
[Voiceover] Best advice I ever got in security. Go!
[TJ Mann] Your data is your most valuable asset. Be careful who you share with your data with, and what data points do you share. Because the likelihood is that those data points are being scraped to get into your accounts from an unauthorized access perspective. And if you haven’t turned on two-factor authentication on your accounts then you should do that immediately because the old practice of using security questions to which all the answers that you’re posting on social media sites about your friend’s birthday, or the first car you owned are being used to get into your accounts, which you should protect with MFA or two-factor authentication today.
[Voiceover] It’s time to begin the CISO Series Podcast.
[David Spark] Welcome to the CISO Series Podcast. My name is David Spark. I am the producer of the CISO Series. Joining me for this very episode is Andy Ellis. He is the operating partner of YL Ventures. Andy, the sound of your voice would be something like what?
[Andy Ellis] It would sound a little bit like this. Or maybe it sounds like this.
[David Spark] No, it rarely sounds like that.
[Andy Ellis] Okay, that’s just my Barry White voice when I’m testing out my microphone.
[David Spark] No, it doesn’t… We’re available, by the way, at CISOseries.com. You can check out everything there, all our other programs as well. Our sponsor for today’s episode is CYREBRO. They are your online cyber security central command managed SOC. That would be your security operations center. More about CYREBRO… I know many of you may be using SOC or you want to use a better SOC. You might be interested in what CYREBRO has to say later in the show. Andy, I have a CRM tool that I like using that I use to send out emails, and it just gives me information as to when and if people open my emails. That’s all the information I need. But they just instituted a couple of weeks ago link tracking. So, they are converting all of my links into their links, and you’re nodding your head. What do you think has happened, Andy?
[Andy Ellis] Well, I suspect it’s a couple things. So, one is probably there are folks like me who do not load remote content. If you send me something, you don’t know if I’ve read it. Sorry. Nice try.
[David Spark] And I do know that some people have that turned off, and no big deal.
[Andy Ellis] Right. They want to be able to capture the data for you of who clicked what link. And it’s a standard practice. One of the challenges is a lot of people do link inspection, for better or worse. They read the link to decide if they want to click in. And all of a sudden when they’re getting links that go to some domain they’ve never heard of before, they’re like, “Eh, I don’t know about that one.”
[David Spark] Well, it’s worse than that. The links are being removed, flat out. Because their software programs are seeing it. Their email programs are seeing it and then stripping the links out of the emails for fear that they’re phishing emails. So, I’m sending emails that have links in them that when they arrive they’ve got no links in them.
[Andy Ellis] Oh, yeah, that’s a problem. I like to say that the security user experience has one of the best acronyms that is self-referential everywhere.
[David Spark] So, I contacted the company. I said, “I want to turn off this link tracking, but I want to keep the email tracking on.” And they go, “Well, all you can do is turn off the email tracking, which also dumps the link tracking.” I go, “I don’t want that. You just introduced this link tracking,” which I never wanted in the first place. And by the way, I can manually turn it off link by link, which is not easy. Yeah, it’s a giant pain in the butt. You can do it link, by link, by link for every single email.
[Andy Ellis] I’ve got an op ed I’m writing on this problem right now actually.
[David Spark] Well, I’ll give you some more details on the company and everything else. But I ping them, and I go, “There’s no way I’m the only customer having this problem. No conceivable way.”
[Andy Ellis] You’re probably just one of the first ones to notice and complain.
[David Spark] Yes, I’m the first one to alert them about it. So, hopefully by the time this episode airs, which will be in May, this problem will have been solved. Fingers crossed. Let’s bring on our guest today, why don’t we. I’m thrilled to have our guest today. It is the CISO for Children’s Mercy Kansas City, TJ Mann. TJ, thank you so much for joining us.
[TJ Mann] Pleasure to be on the show.
What’s the best way to handle this?
4:23.713
[David Spark] “Legacy technology is any technology that makes it difficult for organizations to change their application systems to support changing business requirements. It impedes business agility,” said Anne Thomas of Gartner in an article on CIO drive by Katie Malone. Now, legacy technology is a compatibility issue. And when a vendor no longer offers support, it becomes legacy. But in certain industries like medicine… And we have a guest in that space. Legacy technology often builds up because there’s no other choice or the cost of the hardware you’re dealing with is so incredibly expensive it just prevents you from simply jumping to another solution. So, Andy, I’ll start with you. Are we doing anything better now to deal with legacy technology, and is there anything that can be done better at the purchase point to understand how you’re going to sunset equipment and technology?
[Andy Ellis] I’m not sure I agree entirely with the definition of legacy tech as anything that gets in the way of changing business requirements because I think that means everything, to be very clear. But I think if you talk about the things that you can’t easily move away from when you want to.
[David Spark] They don’t say anything here. It just makes it difficult for organizations.
[Andy Ellis] Any technology that makes it difficult. Well, that’s basically… Email is a legacy technology there. But I think to dig into this one, it’s one of the things that I’ve started to see in the Cloud first approach is this common approach now to start at the beginning at the purchase point and say, “How do I move off of you? How do I stop using this technology?” And that’s a precondition to buy. People don’t want to be locked in. And I recognize that there are industries in which it’s hard to say, “Look, I want to be able to move off of my MRI system.” That’s a little more challenging than moving off of your CRM system for instance, David. You could say, “Hey, I’m just going to migrate to another CRM system.”
[David Spark] That happened to me. I was using a program called Business Contact Manager, which I don’t know if it legally was allowed to be called CRM. But they did call it that. And their export capability, minimal to zero.
[Andy Ellis] Right. And so that’s the problem is most people when they buy a technology aren’t yet asking the right question, which is, “How do I get off of this when you stop supporting it, when I decide I don’t like it, whatever it is? How do I export my data and take it to somewhere else?” And a good vendor will let you.
[David Spark] That’s been my number one thing. If I can’t export, if I can’t get out cleanly, I don’t want it. TJ, now I remember when we were speaking not on the show you mentioned that a lot of the legacy stuff is tied to physically big equipment like MRIs but also connected to essentially the nonprofit experience and donations as well. So, this becomes even muddier.
[TJ Mann] Oh, yeah. Oh, yeah.
[David Spark] Explain.
[TJ Mann] I will specifically…because my focus is on my health, but I’ve worked in financially services… I’ve worked industry as well. A lot of my experience comes from there, too. So, this challenge is not just in healthcare. It’s also in financial services. It’s in banks. It’s in insurance companies. They have those AS400 old systems that you probably won’t even find developers who can code right now. But like you said, this becomes even more muddier when it comes to healthcare and especially when it comes to a nonprofit healthcare system. Because a lot of our… We get a lot of philanthropy funds. I’ll give you an example. If you have a donor who donated an MRI machine which costs let’s say about a million dollars or 500,000 to a million dollars depending on the machine… And it was donated ten years ago. At that time, it was built to run on Windows XP. Guess what? Unless we find a new donor who can donate another million dollars, the decision from a risk based standpoint is, “Hey, we got to keep this stuff because we don’t have any replacements right now. Because we need to have funds, or we need to find a new donor who can donate another MRI machine that we can replace it with it.” And more so often I think it’s a joint responsibility between the vendor and the consumers. Especially when it comes to medical device security. There’s been a lot of discussion around vendor responsibility when it comes to keeping the software up to date, managing and maintaining a healthy application life cycle that would allow the technology to grow along with the business. Earlier you were talking about, hey, everything that becomes a barrier in the business is a legacy technology. The way I look at it is the business is always running at the speed of the Ferrari. So, regardless of even if you take the legacy systems issue out, there is a lot of other things that are going to be that barrier because we need to catch up with the business.
Should we lower the barrier to entry?
9:16.677
[David Spark] “We’re not in a shortage of cyber talent,” argued Matt Trevors of Amazon. On LinkedIn, Trevors said that we’re not willing to hire and train to get what we want. This argument caused a flurry of discussion on LinkedIn, and some organizations act also as education institutions where they train staff to become the employees they want. Now I’m not thinking security specific, but Pixar comes to mind. They have their own university. And also the CRM software, Zoho has a Zoho University where they train all their staff as well. So, I’m going to start with you again, Andy, on this one. What would the organization have to look like if you shifted from a recruiting heavy effort to get talent to a training focus where you were hiring more for passion and less on skills? And I’m thinking this is a good idea because the pool of under skilled or inexperienced people is much larger and much cheaper, but it would also take longer to get them up to speed. My question is would it even cost more? I don’t even know.
[Andy Ellis] It’s doable. I did a piece of this at my last job. We had a studio that took folks fresh out of college and folks who were transition staff – people who had been out of the workforce that had come back in. The company had a program for doing some lightweight retraining – folks on QA and operations. And we always took whoever was the most troublesome person in that class into the security organization.
[Crosstalk 00:10:46]
[Andy Ellis] We said, “Who’s the person who asks the question that’s a fair question, but you really wish they hadn’t opened their mouth,” I’m like, “I want them.” But we then put them in a studio. They would have peers. They’ve had jobs that were carefully managed where we said, “Okay, we want you to do a design review, and you’ll have an architect who will work with you.” And what we found first of all… It’s a fantastic program if you can build something like that. And then whenever you have an opening in your team, you’re hiring out of your own organization. And so what I think a lot of people get wrong is they’re like, “Well, I need a principal architect right now, and so I’m going to go hire one.” And I’m like, “Okay. Well, why don’t you take your senior architect who’s really angsting to be a principal architect and promote them? And then find an architect who wanted to be a senior architect and promote then. And then find a senior security researcher who wanted to be an architect…” And you do this cascade, and now you’re just hiring generally more junior staff. As for costing more, here’s what it does. It means that all of those people you already invested many years in don’t leave your organization because you promote them, and you give them opportunities. And so you’re saving money by reducing turnover.
[David Spark] Can I just also point out – every company I ever spoke to always says, “We like to hire within.” But every time I worked with a company, they do not do that. They always don’t do it, and it drives me crazy. And by the way, when you’re an employee and you see someone come from the outside in, and you didn’t get that promotion, that is a signal to you, the employee, get out.
[Andy Ellis] Right. And let’s be careful. When you say hire from within, that’s where you involve recruiting, and you do this effort. And every manager gets in the way of it because they don’t want to lose their people. What I’m talking about is carefully managed promotion paths.
[David Spark] I’m going to go to you, TJ. Are you able to train to get what you want?
[TJ Mann] Yes. And I think my approach has always been you hire for attitude, not for aptitude. And the reason being to what Andy was talking about earlier – growing staff from within is usually the most cost affective way of recruiting. And this particular question has become a lot more relevant in the last year due to COVID. There’s been a huge…with work from home… We’re based out of the Midwest, so I can tell you that companies from east coast and west coast have been just poaching talent the Midwest because they can afford it. They can still pay them less than what they were paying somebody on the west coast and have them work remotely. But the challenge then becomes how do you maintain a healthy talent pipeline that you can maintain over time. And instead of going for a national search every time… And granted there are some positions you may have to go for a national search. But instead of looking outside, how do you promote from within. Because that’s what creates motivation for employees to stay. That’s what engages employees. That’s what tells them that I’m valuable, that my skills…that the company is equally invested in me as much as I’m invested in the company to stay here.
[David Spark] That’s a good point. And let me ask both of you… So, not all organizations have the wherewithal to build training within. Have either of you used third parties to do training? And, A, how did you go look for them? Or did you get referrals? And how did those work out for you? I’ll just start with you, Andy.
[Andy Ellis] At the company level, we used an outsider to manage our retraining program. It had some mixed results. These were people who were sort of really profit focused at that moment and like, “How do I run this efficiently?” I heard some horror stories. We actually replaced the vendor based on just how the people were coming out said, “God, I really didn’t like that program.” And we’re like, “Our whole point is to make you love us.” So, we failed. For targeted skills training, we did. But we actually found that what would often work is we would send somebody to a class or training program, and they’d come back, and they’d be like, “That was awful.” And then they would write one that was better. So, in a sense it was this really useful seed. I like to talk about the red ink rule. If you want somebody to do something, handing them a blank page doesn’t work. But if you hand them a page with a really bad sketch on it, they will pull out a red pen and write it better for you.
[David Spark] People are better editors.
[Andy Ellis] Yes, people are way better editors.
[David Spark] TJ, have you hired third party organizations?
[TJ Mann] Yeah, we contract with some third party training organizations to help our staff maintain their skill set because that’s an important initiative. That’s changing every day or every month. There’s new technology that’s coming out. Gosh, Microsoft keeps renaming everything, too. So, we use third parties for technical skillset. What we do inhouse is leadership skillset and leadership training. That’s something that we’ve recently built a leadership institute at Children’s Mercy to allow and enroll all leaders in that leadership institute and give them a career path – give them a leadership career path that they want to join and they want to pursue and help them reach that level. Maybe it’s building their communication skills. Maybe it’s building their management skills. Maybe it’s building their leadership skills overall. So, we look at it as double pronged. There is a technical skillset need that can be accomplished with outside parties, but the internal growth from a leader standpoint is still something that we want to align with our mission vision values of the organization to make sure that the leaders that we’re growing are leaders that are aligned with the organization’s mission.
Sponsor – CYREBRO
16:14.632
[Steve Prentice] CYREBRO is the world’s first true SOC platform that operates and functions as a full product. Why is this important now? Nadav Arbel is the company’s founder and CEO.
[Nadav Arbel] We’re changing the way companies of any shapes and size operate day to day security, and we’re literally becoming I like to say the 0 365 of Samsung. So, we’ve taken something that has always been on prem, that has always been machines with integrators. We’ve created a product as a platform, put it in the Cloud, and said, “We want to do this in a whole different way with a whole different approach.” And this is what we believe the market is going to be 100% a couple of years from now.
[Steve Prentice] Nadav says there are two reasons that move them to this point.
[Nadav Arbel] One, the current or old way were SIM and software done on prem have become so convoluted because the security systems have become so progressed. You need to have such a wide variety of skillsets to understand and operate each one of the new security systems that the likelihood of you having all the skillsets required to operate everything has just become just unrealistic. And the second thing that’s changed is that the small, the medium, and even the large businesses who weren’t players in this market have all become somebody that have to have it. But they can’t afford the old, classic couple of million dollar cost of a SIM soft. Productizing it was the only way to do it.
[Steve Prentice] For more information visit cyrebro.io.
It’s time to play, “What’s worse?”
17:50.619
[David Spark] Are you ready to play, Andy, another round of “what’s worse?”
[Andy Ellis] I am excited to play a round of “what’s worse.” It’s my favorite game of the week.
[David Spark] What other games do you get to play during the week that are security focused?
[Andy Ellis] Oh, that are security focused… Yeah, I was going to say I play a bunch of games on my iPhone, but I don’t think those count. This is I think the only security focused game I play, so it’s my favorite security focused game.
[David Spark] Well, if you come on our Fridays for our Super Cyber Fridays, which we have renamed… It used to be called the CISO Series Video Chat. We play lots of games, and we’re changing them up. We’re trying to change them up even more.
[Andy Ellis] You do, but it runs up against a reoccurring meeting I’ve got.
[David Spark] Well, obviously that meeting needs to be cancelled.
[Andy Ellis] I’ll tell my therapist that.
[David Spark] [Laughs] There you go. [Laughs] Invite your therapist to our event. Here we go. This comes from Nir Rothenberg, who is the CISO over at Rapyd, who has given us umpteen awesome “what’s worse” scenarios. Here is the scenario – you have a very small security IT dev ops staff and can’t outsource. All right? TJ, by the way, Andy always answers first here. Here are the two scenarios. Scenarios number one – you’re Cloud native but in one AWS data center and obviously one availability zone. That’s scenario number one. Or scenario number two, you’re on premise, but someone in the past set up some regular offsite backup for all critical systems that seemed to work. So, you do have some kind of Cloud presence.
[Andy Ellis] I like that, “Seemed to work.” [Laughs]
[David Spark] [Laughs] All right. So, which one is worse?
[Andy Ellis] Oh, the on premise. This one is actually pretty easy for me.
[David Spark] And why is that?
[Andy Ellis] When AWS goes down… And I’m hoping I’m in like [Inaudible 00:19:43] even though it’s the one that I feel like it always goes down. But everybody else is having a bad day, too. And so I’m [Inaudible 00:19:50] Like at the end of the day I’m like, “Hey, I had a bad day, but how do you tell that I have a bad day when everybody else is also having a bad day?” Whereas if I go down on premise, first of all, I’ve got a small team. There’s a good chance that we haven’t really exercised recovery. This is what almost everybody does wrong. They set up backup, and then they don’t exercise recovery. I had this a long time ago. Our logs, which we did billing off of, were in fact all backed up. Except the tape machines that we used to back them up…this is how long ago it was…it took 23 hours of the day writing tapes for the backups. Well, that meant that recovering from backups only got one hour a day. So, literally we had to…when we actually had an event, and it was like, “We need to recover 30 days of backups.” And that was going to take us a year and a half at current capacity, we had to go buy a tape deck so that we could recover our backups. It didn’t work. We thought it worked. We had the backups. But it didn’t really work, and we’d never exercise it. And I’m pretty sure in the scenario I have not exercised it, so that’s worse. I would rather be in the Cloud. I’ve got reasonable backups. But when AWS is done, AWS is done, and I can’t do anything.
[David Spark] TJ, you’re agreeing with him?
[TJ Mann] Yeah, I agree. I agree with Andy. I’d rather put my resources to secure the my responsibility of the Cloud within the AWS than having them focus and build that whole DR plan and DR capability. Again, most organizations will say that, “Hey, we’re pretty good.” But the reality is everybody lacks process maturity. So, I’d rather put my eggs with Amazon, who I’m going to say that they have more money than I do. They likely have better process maturity, and they have more staff than I do who can jump on things when needed when things go awry than having things on premise and having both the capability from a DR perspective and then also securing that from a security teams perspective. So, I’m already short on resources. I’d rather give Amazon more money to do it for me. And based on how much money they have, based on their infrastructure, based on their teams, I would rather think that they would be in a better position in this kind of beautiful scenario that you gave me to protect me than myself.
[Andy Ellis] TJ actually had a really good point there that I don’t want people to lose, which is you have limited resources. And do you want them doing lower value work or higher value work? And in that on premise world, the work that they’re going to probably end up doing is not as high value as securing things in the Cloud and leveraging the Cloud to get better and using Cloud native tools than if you’ve got them trapped in 20-year-ago legacy technology.
[David Spark] We’re assuming this is 20 years old? It could be a spanking new machine.
[Andy Ellis] If you’re still building in your legacy environment today, that is not…should not be brand spanking new. If you built it today, it’s Cloud native.
[David Spark] We have a scenario – they were both built today.
[Andy Ellis] Fine, then you’re building it today with 20-year-old technology.
[Laughter]
[Andy Ellis] That’s what I’m going to say.
[TJ Mann] And like I said, I’d rather have my team focus on the Cloud security responsibilities [Inaudible 00:23:07] versus the vendor than having them focus on building a whole DR capability. And I’d rather just have them focus on making sure that we have a DR capability, and it works, and we test that with the vendor rather than having them focus on maintaining that on a day to day basis. Like you said, the tapes and everything. It’s a lot of work. And like you said, Andy, just putting that in place isn’t enough. You got to make sure it’s working. You got to make sure you test that on an annual basis at a minimum, on an annual basis. You’ve got to make sure there’s a redundancy in place from an IT standpoint that if you need to flip the switch it actually flips the switch. It actually works. So, I’d rather have all those headaches to the vendor and pay them more than have my team just secure our side of the fence.
Got a better answer than “We’re trying”?
23:51.579
[David Spark] Tal Eliyahu of Standard Chartered Bank asks, “Can you please consider doing a segment on how to send a survey to all the vendors after POC, proof of concept? I find myself wanting to be better with the RFP, request for proposal, and provide good information and environment with every POC I run, but I don’t have the feedback from the vendors once we’re done choosing the winning vendor.” So, what do you say, TJ? You’re done with the POC. You picked a winning vendor, but you want to let the others know why you didn’t choose them. A, do you do it? And B, what’s a better way to handle it that way?
[TJ Mann] Yeah, we don’t do it right now. I haven’t met a lot of people who do it. You usually find that either before or during the POC what’s going to work and what’s not going to work. And at that point, the time is limited. So, we’re focused on just, “All right, what’s working? Let’s move that forward.” But we don’t close that loop. But I think the vendors also have…they already have an idea if their product is going to work or not. At times, yes, there could be a fair competition where we could have two products that are very, very similar and equal. And I think in that scenario, there is always a differentiator that’s going to be either the business differentiator. Maybe from technical capabilities perspective, both products are the same. But maybe one product is a better product in terms of integration with the rest of the stack and the rest of the other products that we have in the environment. And it could be that kind of differentiator. But I think most vendors already know when they’re in the POC process or even before getting into the POC process what their product is lacking and what their competitor’s product has more than what they don’t.
[David Spark] All right, Andy, I’m throwing this one to you. A, have you ever closed a loop like you pick a winner? And you tell the users, “You lost.” And A, give them the reason? Or they just came to you and say, “Can you explain why we lost?” And actually gave them the time to explain it.
[Andy Ellis] Yeah, so it’s hard to do. I’ve done it. A lot of people feel like it’s like, “Oh, I don’t want to deal with rejecting somebody, and I don’t want to deal with the stress of it.” But every good vendor I’ve ever interacted with is eager. If you say no, they want to know why. They might know why, but they want to hear it from you because that lets them know for sure. And often if you said, “Hey, I would be happy to do a win/loss with you, either way. I’ll tell you why you won if you’re the ones who won. And if you’re the ones who lose, I’ll tell you why lose. Does your sales operations organizations or your marketing team have a formal program for doing it?” First of all, if the answer is yes and everybody else says no, you should consider maybe that one of those vendors is a little more professional than the others. Maybe that might weight into your decision. But I thin people are too afraid to have a conversation. Recognize that sales teams…
[David Spark] People don’t want to have confrontation… They feel it’s going to be confrontation. They don’t want to deal with it.
[Andy Ellis] But you know something? A sales team hears no all the time. It’s not confrontational to them. I see too many places where a vendor loses, and nobody tells them. All of a sudden you just ghost them, and the vendor is like, “I’m pretty sure we lost there, but you won’t even give me the dignity of a one-line email that says, ‘We decided to go with someone else.’”
[David Spark] First of all, I like to hear why I’m rejected because I want to… I’m like, “Oh, this doesn’t work. Let’s stop doing that.” And, look, you just want to make everyone better. And honestly the 15 to 20 minutes of your time… And you always have to set it up, “Are you okay hearing the rough news?” You got to always say that. Like, “Because I’m going to speak openly here about what we didn’t like.” And you’re just like, “Yeah, please feel free to say openly.” Rather than… Because you don’t want to get in a situation where they’re going, “Yeah, but we don’t do that.” Because then he goes, “Well, that’s what I heard,” kind of a thing.
[Andy Ellis] Yeah. Oh, no, in this day and age, I’ve talked to a bunch of our portfolio companies, and they do this where they’ll be like, “No, absolutely. We’ll get you on a Zoom. We’ll plug in Chorus, or Gong, or Otter, so we can record what you say, and we’re going to go share it back with our whole product team and tell them exactly why so you only have to tell us once. Because we’re really interested in learning from it.” That’s more and more becoming a standard practice.
[David Spark] Can you think of some of the portfolio companies…anything that they’ve learned? Just give one or two tidbits maybe.
[Andy Ellis] Oh, they’ll learn things…sometimes it’s just even how you did the demo. Like I’ve been sometimes shocked when someone comes back and says the reason we lost is because we didn’t tell them we did a thing. Not we didn’t do it. We never showed them the capability, and so we lost because our competitor had that capability, but we have it. So, their whole marketing team and sales organization now tries to figure out like why didn’t you share this with somebody if it was a differentiator. Why didn’t you share it? That’s probably the biggest single common thing is sales teams that don’t actually sell.
[David Spark] That’s a good one. That’s why to ask them…always ask them first what’s important to you. TJ?
[TJ Mann] Yeah, at times, Andy, just to add to that…at times vendors would bring like their super technical people on the call, and what you have on the other side is the actual strategic leaders. And they’re not able to build that bridge from what does my product from a technical capabilities perspective translate to you from a strategic business objective perspective. And that just gets lost, even if they have a really good product. I’m not here to buy a product. I’m here to build relationships. If you’re not able to tell me how you add value to my organization then it’s just another product for me, and I can buy another product from anyone. I’m looking for vendors who are willing to build relationships who want to add value to in my case Children’s Mercy. How do you add value to Children’s Mercy? And if you’re willing to do that then I’m willing to do a multiyear deal with you.
[David Spark] Quoting my other cohost, Steve Zalewski, who used to be the CISO over at Levi Strauss…he would always say, “How does your product help me sell more jeans?”
How scared should we be?
29:38.939
[David Spark] All right, so this is going to be a little bit of a game here as well, but we don’t really have a formal game around it. But this comes from an anonymous listener, and they have a question/really game. Here is the scenario. I’m going to throw it to you first, Andy. Your interview questions for security team members has been scraped, posted, and shared by the recruiters to every new candidate. How damaging is this if at all you think?
[Andy Ellis] So, if you have good questions, this is not damaging. If you’re relying on trick questions like how many people piano tuners are there in Manhattan, then of course that’s damaging to you. But I’ll tell you, one of our standard questions that a number of our technical staff would ask of anybody who came in…they said, “When you put a URL into a web browser, explain to me what happens, as deep technical level as you can go.” And here’s what’s fascinating… Because if you put this question in front of people, sometimes you get an explanation about DNS. Other times people will talk about hardware interrupts of what’s going on in your keyboard. Some people want to talk about HTML. It’s amazing how many different ways people might go deep in it. And I can tell you in advance that’s what the question is going to be, but you’re really revealing your knowledge through the question. You could prep it a little bit. And if they were all released, I’d probably want to tweak the questions a little bit for a while. But if somebody is perfectly prepared based on my questions, it’s more likely damaging to them. If they give me too perfect answers, I’m going to be like, “What’s going on here? That’s a little bit weird.”
[David Spark] What about TJ? How damaging do you think this is?
[TJ Mann] I agree with Andy. If your questions are binary then that’s more damaging. And especially if your questions are more of from technical standpoint, then that could be damaging. But the key is to have open ended questions. But if you’re looking at leadership or management level focused questions then you’re going to be open ended anyways. “Tell me how you build relationships with people. Tell me what are you going to do in the first hundred days.” So, I think it could be damaging. But if those questions are very binary, which is actually not a good practice… You shouldn’t ask binary questions in an interview anyways. You want to speak as little as possible, and you want the candidate to speak as much as possible. And you want to ask open ended questions so they can explain themselves.
[Andy Ellis] And we had a…I think rubrics I would worry more about. There were things we looked for to tell us, “Uh, maybe we don’t want to work with this candidate.” Like you have a paired interview with one man and one woman. And you have the woman asking the questions, and you’re watching to see if the answer comes back to her or if the candidate turns and looks at the man to always answer. Because that’s a red flag. And so if I told you that that’s what’s going on, that that’s why I’m doing it… Of course I just told everybody, but that’s okay. I don’t hire in that organization anymore. Like, “Oh…” Then somebody could come in and be prepared that even if they would normally disregard women, they’re going to plan not to for this 45-minute interview. That’s probably a little bit damaging. But at the same time, I’m going to tell everybody, don’t ignore the technical woman in the room. So, if you’re going to practice it for 45 minutes, hey, maybe that’s at least a good thing that, you know, “I care about it.” And if you don’t do it once you’re here, yeah, we’ll deal with it then.
[David Spark] Podcaster Rob Walch, who is actually with Libsyn, used to have a great question that he would ask at the beginning, and the way they answered it depended on how technical and how sort of geeky his guest would be. He’d ask the question of, “What was your first computer?” Now, if I say that to you, Andy, your answer is…
[Andy Ellis] Commodore 64. But the zero model. The very first one. It was that different color brown.
[David Spark] Commodore 64 you said?
[Andy Ellis] Yeah, Commodore 64. I learned on a VIC-20 in school, but the first one that was mine was the C640 model.
[David Spark] For me, the TI994A. And TJ, your first computer?
[TJ Mann] It was a Windows computer with 64 megabits of ram.
[Andy Ellis] Megabits! No, no, no. K. K. Man… Young pups.
[David Spark] So, the people don’t know that just go, “Oh no. I don’t know. It was some computer I had,” kind of a thing… And so he realized, “All right, this is what the conversation is going to be. It’s not going to be geeky enough.”
[Andy Ellis] Right. So, I’ve done questions like, “Tell me about a technology that excites you.” Very open ended. And sometimes people are just like, “What do you mean?” I’m like, “Anything. What do you think about the microwave? Is that cool? Pick a technology,” because I want to see if there’s anything that you would get excited and geek out about about technology at all.
[TJ Mann] You know another question that I love to ask is, “What ticks you off? How can I make you angry?”
[David Spark] I like that one.
[Andy Ellis] That feels like a dangerous trap question though. Like if anybody asked me that question, I’m going to try to give you a very pablum answer about what makes me angry.
[David Spark] You’ll also see how open the person is and how honest they are, too.
[TJ Mann] Yeah, and I want to hear that because I want to know if you’re a good fit in our culture or not. Because I know what kind of people, what kind of staff works in our culture, and I know how we solve problems together. And I know how we discuss problems. If a candidate comes in and says, “Well, I like to just leave the room when nobody agrees with me,” then is that a good answer? Or like, “I like to present logical reasoning for everything that I have to say and the decision making to the executives.” Or elsewhere. So, that question always gives me interesting answers.
Closing
35:17.731
[David Spark] And that brings us to the very end of this show. Thank you very much, TJ. I don’t know what you can say to me to tick me off, but I’m going to wrap this show up regardless. I’m going to let you have the very last word here. I want to thank our sponsor, CYREBRO. If you’re in the market for a SOC and you want to know what new is possible in SOCs, check them out at cyrebro.io. Thank you very much for sponsoring this very episode of the podcast. And by the way, I always ask my guests are they hiring, so make sure I’ve got an answer to that. Andy, any last words?
[Andy Ellis] I have to say that I would be very ticked off if I was asked questions in an interview about what ticks me off.
[Laughter]
[David Spark] Now everyone knows what not to say to Andy. TJ, are you hiring?
[TJ Mann] We are hiring. We’re hiring in a lot of different areas at Children’s Mercy. So, if you have expertise in data, if you have expertise in data analytics, if you have expertise on building digital products, reach out.
[David Spark] Excellent. Well, I want to thank you, TJ. I want to thank Andy as well. I want to thank our audience for all their amazing contributions and for listening to the CISO Series Podcast.
[Voiceover] That wraps up another episode. If you haven’t subscribed to the podcast, please do. We have lots more shows on our website, CISOseries.com. Please join us on Fridays for our live shows – Super Cyber Friday, our virtual meet up, and Cyber Security Headlines – Week in Review. This show thrives on your input. Go to the participate menu on our site for plenty of ways to get involved, including recording a question or a comment for the show. If you’re interested in sponsoring the podcast, contact David Spark directly at David@CISOseries.com. Thanks for listening to the CISO Series Podcast.