How to prevent ransomware, slow its spread, and respond if you get it
This week, Prasanna and Mr. Backup (W. Curtis Preston) review a series of posts made by Snorkel42, who previously appeared on this podcast in the episode called "Security expert rips Okta for their response to hack." Things were recorded out of order, so this is the episode where we discovered him on Reddit, and tried our best to distill several thousand words into about 30 mins of advice on how to protect against ransomware. We talk about how to prevent getting it in the first place, how to limit its damage if you do get it, and how to respond and restore your data once that happens. There is a ton of really good advice here, so check it out!
Here are the three posts:
https://www.reddit.com/r/sysadmin/comments/tdvbp4/security_cadence_okay_fine_lets_talk_ransomware/
https://www.reddit.com/r/SecurityCadence/comments/tedapy/security_cadence_ransomware_part_2_actions_on/
https://www.reddit.com/r/SecurityCadence/comments/tfm927/security_cadence_ransomware_part_3_the_worst_case/
Mentioned in this episode:
Interview ad
What's a TLD for our listeners?
curtis:Oh, top level domain.
curtis:That's like.com or dot ransomware.
curtis:Hi and welcome to Backup Central's Restore it All podcast.
curtis:I'm your host, W.
curtis:Curtis Preston, AKA Mr.
curtis:Backup.
curtis:And I have with me, my delayed shipment consultant, Prasanna Malaiyandi.
curtis:How's it going , Prasanna?
Prasanna Malaiyandi:I'm good.
Prasanna Malaiyandi:Curtis, wait, what's delayed.
curtis:my, my, my flooring shipment, you know,
curtis:and I, I turn to you for.
Prasanna Malaiyandi:what I thought you received one.
curtis:I did I did, but . I ordered a big shipment of flooring, and then
curtis:I ordered a much smaller shipment and I did that in two shipments because
curtis:I couldn't order all of it at once.
curtis:And then I had to order like another 10% and the second shipment I received the
curtis:second shipment like three weeks ago, I still haven't received the first shipment.
curtis:And, um, I just turned to you for, for, you know, emotional
curtis:support in this time of.
curtis:I'm not doing anything until the entire shipment comes in., it's just ridiculous.
curtis:I ordered this,
Prasanna Malaiyandi:Have you heard about supply chain issues?
Prasanna Malaiyandi:Curtis has this not.
curtis:I gave them grace because of the supply chain, but here's the thing.
curtis:This is made right up the road from me.
curtis:Well, it's more like up the road from you, but it's made in California.
curtis:It's vinyl.
curtis:The manufacturing is happening in California.
curtis:But the problem is that they've lied to me.
curtis:They lied to me before.
curtis:They told me it's in production because you know, they make several colors.
curtis:They're like, oh, that color, it was really in demand.
curtis:It's in production.
curtis:Now.
curtis:They told me that like three weeks ago, they said it's in production.
curtis:It should ship out any day now.
curtis:They're now claiming they're out of stock.
Prasanna Malaiyandi:Oh,
curtis:Right.
curtis:They're like, oh yeah, we, we, we did it was in production.
curtis:We didn't lie to you.
curtis:We just didn't make enough.
curtis:Well, why did you stop the production run before you made
curtis:enough to fulfill back orders?
curtis:I mean, I get that.
curtis:You're behind.
curtis:I get that you had a big promotion, but retooling, the production line is a pain.
curtis:Right.
curtis:So why would you retool it
Prasanna Malaiyandi:Maybe they ran out of
curtis:of color.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:Whatever.
curtis:So this is why you're here.
curtis:You're here to make me not so angry.
curtis:That's why I said you're my delayed shipment consultant.
curtis:All I know is it's not in my hot little hands and I'm not doing squat in my
curtis:garage until I get the entire shipment.
Prasanna Malaiyandi:Just think though.
Prasanna Malaiyandi:How about delayed gratification?
Prasanna Malaiyandi:Once you finally get the pallets
curtis:This is the ultimate in delayed gratification.
curtis:I've never had so much trouble spending money in my life.
curtis:Right.
curtis:I mean, and that even includes the two recent, very expensive
curtis:couches that we bought.
curtis:There were way more expensive than this.
curtis:Um, we ordered it and then they were like, it's in a ship off long beach.
curtis:If you want to see your couches go to the long beach Harbor and look out into
curtis:the water and you can see, and that was, that was promised like four weeks.
curtis:And it was more like eight, but at least there, I was like, well, I'm part of
curtis:the whole, you know, shipment problem.
curtis:And I just had to wait, but here it's just frustrating because they,
curtis:because they've miscommunicated,
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:I think that's the problem, right?
Prasanna Malaiyandi:If they had not given you any information that yeah.
Prasanna Malaiyandi:It's in production, right.
Prasanna Malaiyandi:You probably would have been fine.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:It's just shipping delays.
Prasanna Malaiyandi:That's fine.
Prasanna Malaiyandi:The fact that they told you now you're annoyed.
curtis:Hashtag
Prasanna Malaiyandi:it'll be
curtis:#firstworldproblems.
Prasanna Malaiyandi:Take a deep breath.
curtis:Yeah, good times.
curtis:Good times.
curtis:Um,
curtis:Our disclaimer, Prasanna works for Zoom.
curtis:I work for Druva and, uh, the opinions that you hear are ours.
curtis:This is not a podcast of either company.
curtis:And a rate us at ratethispodcast.com/restore, or just
curtis:click on your favorite pod catcher.
curtis:And, uh, click down to the bottom and give us some stars, or maybe even a comment.
curtis:Talk about how much you love Prasanna's beard.
curtis:I'm good with that.
curtis:And how it's so much longer and darker than mine and.
curtis:And, uh, you know, if you're, if you're curious about such things, if any of
curtis:these things, we talk about excite you either way then, uh, you know, @wcpreston
curtis:on Twitter or wcurtispreston@gmail and, uh, you'll find me.
curtis:So I see.
curtis:I sent you this, this post that I, that I saw on Reddit, which it's well,
curtis:it's actually a series of three posts from a Reddit user called snorkel42.
curtis:Don't let his, you know, snorkeling ID fool you the, the person
curtis:knows what they're talking about.
curtis:I don't know.
curtis:I don't know anything about this person.
curtis:Other than that, they, they have, they post regularly in a
curtis:subreddit called security cadence.
curtis:Um, but he also posted he or she, I don't know if I mistaken
curtis:mistakenly called the person.
curtis:He, I apologize in advance for my misogeny, so.
curtis:The, it was about ransomware and, and they are a specialist in the areas
curtis:of security and many people had asked them to post stuff about ransomware
curtis:and they had continually sort of said, I don't want to post about ransomware.
curtis:And can you imagine why that would be
Prasanna Malaiyandi:You're just sort of propagate well, it's ransomware you get
Prasanna Malaiyandi:hit with, because there were a bunch of gaps before ransomware got hit and it's
Prasanna Malaiyandi:better to address the problem rather than trying to address sort of the outcome.
curtis:Yeah.
curtis:So ransomware to this person is the symptom of a whole lot of bad things
curtis:that you were already doing or not doing.
curtis:And they've spent their career helping to make sure you do those things.
curtis:But with the, I think two things, one is that obviously the ransomware attacks are
curtis:getting to a fever pitch and then two.
curtis:There is what we talked about on the previous episode, which was this concern
curtis:about Russia and D w we did cover that.
curtis:Didn't
curtis:we?
Prasanna Malaiyandi:Yeah, we cover the Conti ransomware gang
curtis:Yeah.
curtis:Yeah.
curtis:Um, yeah, the, the, the Krebs on security post.
Prasanna Malaiyandi:Yep.
curtis:That the concern is that the level of the fever pitch that we're experiencing
curtis:might actually go through the roof.
curtis:And so they said, Hey, I'm gonna finally, I'm fine, fine.
curtis:I'll post about ransomware, but even in their post about ransomware, it
curtis:really wasn't that much about ransomware as much as it was about the things.
curtis:Well, no, that's not true.
curtis:I'll take that back.
curtis:It was, it was here is the way ransomware works.
curtis:And so I I'd say the first one, I'd say of the three series,
curtis:The first one was about here's how to prevent it.
curtis:Number one, like from getting in.
curtis:The second was here's how to prevent it from doing more damage once it's in.
curtis:And then the third one, it was okay.
curtis:All right.
curtis:You're totally screwed.
curtis:You've got to reach for your backups.
Prasanna Malaiyandi:Yeah.
curtis:that
Prasanna Malaiyandi:The one thing I would add to that , is he also was careful
Prasanna Malaiyandi:saying, I don't want to just focus on the Conti ransomware and provide you steps
Prasanna Malaiyandi:to prevent that because there are so many other ransomware flavors out there.
Prasanna Malaiyandi:If you build something for just one.
Prasanna Malaiyandi:You're not going to be protecting yourself.
Prasanna Malaiyandi:Let's take a holistic approach.
Prasanna Malaiyandi:And like you said, let's cover, how do you prevent it from getting in?
Prasanna Malaiyandi:What, how do you prevent the spread of it?
Prasanna Malaiyandi:And then how do you recover?
curtis:Yeah.
curtis:Good point.
Prasanna Malaiyandi:The first one is called initial breach, I think
Prasanna Malaiyandi:is how he titled the first article.
curtis:Right.
curtis:So the phishing basically, they're saying That That is the number
curtis:one way that you get ransomware.
Prasanna Malaiyandi:Yep.
Prasanna Malaiyandi:Someone accidentally clicking an email, opening up something,
Prasanna Malaiyandi:letting the attackers in, and they don't even know about it.
Prasanna Malaiyandi:So how do you prevent your users from clicking on malicious links?
curtis:now, now, it's interesting.
curtis:This goes, yeah.
curtis:Sorry.
curtis:This goes somewhat against what, some of the advice of one of the guests
curtis:that we had on the podcast, which was, they basically said, look, your
curtis:people are going to click on stuff, stop relying on, you know, I dunno.
curtis:I dunno if it's against, but, but he, he, de-prioritized training and, and like, uh,
curtis:phishing assessments, didn't you think.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:So.
Prasanna Malaiyandi:This author does say training can only help you so much?
Prasanna Malaiyandi:I think the couple things, the couple things though, that he did mention is,
Prasanna Malaiyandi:um, you do need some level of training, but you need to make sure people don't
Prasanna Malaiyandi:feel like they're being punished.
Prasanna Malaiyandi:When they do the wrong thing, right?
Prasanna Malaiyandi:You want that transparency.
Prasanna Malaiyandi:You want to be telling people it's okay for you to say that I clicked
Prasanna Malaiyandi:the wrong thing because then the IT team can try to evaluate what's
Prasanna Malaiyandi:going on and try to contain it.
Prasanna Malaiyandi:The sooner they know the better it is.
Prasanna Malaiyandi:But if say someone's afraid because they're going to get in trouble.
Prasanna Malaiyandi:They might be fired, right.
Prasanna Malaiyandi:It becomes taboo then no one's going to report it.
Prasanna Malaiyandi:And that's actually really bad.
curtis:Yeah.
curtis:Um, they said to prioritize rewarding over punishment.
curtis:Right?
curtis:Make it, make it known.
curtis:Like you said, that it's okay to call in.
curtis:We want you to call in, even if you messed up and, and then, and
curtis:they also said consider doing your own phishing assessments.
curtis:I read some of the comments and they talked about that.
curtis:They had a thing where you, you, you got some.
curtis:You got some, it was sort of some strikes and it was like 10 strikes.
curtis:It was like, you could click on 10 malicious emails.
curtis:And, and then it was the 10th.
curtis:When, and that they actually had a series of escalations where, you
curtis:know, it started out, Hey, you know, we really told you kind of thing.
curtis:Um, I think you can do both.
curtis:I think you can do both carrot and stick, right.
curtis:Reward and punishment where yes.
curtis:You want to reward people for calling in.
curtis:Thank you for calling accidentally clicked and then.
curtis:And then if the person clicks doesn't know, but you know, because you did a
curtis:phishing assessment, you do a series of escalating things where that
curtis:ultimately you can have a person.
curtis:And this was discussed in the comments, not necessarily that you
curtis:would fire somebody that, that keeps doing this, but you might say, okay,
curtis:this person cannot be trusted with a straight internet connection.
Prasanna Malaiyandi:Yup.
curtis:Right.
curtis:All email from this person will be monitored.
curtis:Yeah.
curtis:They can only open email that's straight from our Exchange server
curtis:or whatever stuff like that.
Prasanna Malaiyandi:So phishing was sort of one way that people get in.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:But I think once they're in whichever mechanism it is, it's like, okay,
Prasanna Malaiyandi:how do you detect that someone's in?
Prasanna Malaiyandi:And I think Curtis, this is what you're going to say, right.
Prasanna Malaiyandi:About sort of this notion of droppers.
curtis:Yeah, I actually didn't know this part.
curtis:That's I was fascinated that that basically that the actual phishing
curtis:results in a very small piece of software whose job it is to install
curtis:the actual piece of software
Prasanna Malaiyandi:Yeah.
curtis:and that he calls out a dropper.
Prasanna Malaiyandi:Yep.
curtis:Well, and so the idea is understand that that's the way it works,
curtis:that a piece of code gets dropped in, and then that piece of code executes, and
curtis:the only purpose of that piece of code is to download the other piece of code.
curtis:And so they said that you could, you could stop that.
curtis:You could say, well, you can't run arbitrary pieces of code in,
curtis:in locations that are directly accessible by the end user,
curtis:you know,
Prasanna Malaiyandi:Or you could restrict what applications are allowed
Prasanna Malaiyandi:to run on a laptop for instance,
curtis:yes,
curtis:Whitelisting, I think whitelisting is it, I think it's the, the best.
curtis:The best way to stop stuff like this.
curtis:It's also the highest touch because it means that every new
curtis:application that anybody has to install, they have to get approval.
Prasanna Malaiyandi:Yep.
Prasanna Malaiyandi:think it's a way to guarantee sort of legitimate applications have gone through
Prasanna Malaiyandi:some sort of validation process, security review, et cetera, before it's being
Prasanna Malaiyandi:allowed to be deployed in your environment
curtis:Right.
curtis:And then the next thing it talked about was that a random file
curtis:running should not be downloading files from the internet, right.
curtis:That it should only be HTTP and HTTPS is downloading from the internet.
curtis:And so.
curtis:He said with exceptions, like, you know, um, uh, SFTP for example.
curtis:So he talked about, he talked about, you know, again, accessing that also
curtis:possibly blocking bizarre TLDs right.
curtis:And unnecessary locations.
curtis:You could just simply say, listen, uh, we don't have anything to do with Russia.
curtis:Why would we download anything from Russia?
curtis:And if there is somebody in our company that needs to download stuff from Russia,
curtis:they will be, they will be accepted.
curtis:That was a very running theme I heard was lock down everything and allow exceptions.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:And, uh, it was going to bring up two things.
Prasanna Malaiyandi:One was what's a TLD for our listeners?
curtis:Oh, top level domain.
curtis:That's like.com or dot ransomware.
curtis:There is no dot
curtis:ransomware,
curtis:but.
Prasanna Malaiyandi:And was it you, or was it one of our guests who were, who
Prasanna Malaiyandi:was talking about how they worked at a company that completely locked down
Prasanna Malaiyandi:their network and the network admin would never let them do their backups
Prasanna Malaiyandi:and everything was by except.
curtis:That was me.
curtis:Yeah.
curtis:Yeah.
curtis:Uh, that was, I was a client of mine where they had internal firewalls and
curtis:that's an example of, you know, going to the extreme of, well, now you're now
curtis:you're preventing core business functions,
Prasanna Malaiyandi:Yeah,
curtis:right?
Prasanna Malaiyandi:but
curtis:they also talked about local firewalls, right.
curtis:Which is what we were just talking about, that the, and we're going to get
curtis:to that more in the next section is, so they're just looking, he's looking
curtis:for ways to stop the dropper from getting yeah, exactly.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:thought was an interesting point I'd never thought about is he does have a point
Prasanna Malaiyandi:about they block newly created domains.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Which I thought that had been dormant for a while and then are now active,
Prasanna Malaiyandi:which I thought was very interesting because it's something I had never
Prasanna Malaiyandi:thought about, but it totally makes sense.
Prasanna Malaiyandi:Usually when you get ransomware, right.
Prasanna Malaiyandi:These actors, they spin up domains and they start
Prasanna Malaiyandi:communicating, using that domain.
Prasanna Malaiyandi:So he's like, yeah, you could have a policy to just block these domains.
Prasanna Malaiyandi:So they can't actually reach back out to the command and control
Prasanna Malaiyandi:servers to be able to download from the dropper, the actual exploit.
curtis:Right.
curtis:And, and they said they weren't aware of anything.
curtis:Where that you can do this for free, but there are tools that are
curtis:available to help you do This right.
curtis:There's
Prasanna Malaiyandi:remember, uh, what are the D D.
Prasanna Malaiyandi:Uh, what were the initials?
Prasanna Malaiyandi:The DNS
curtis:DDI.
Prasanna Malaiyandi:right.
Prasanna Malaiyandi:And I think that goes to some of that as well.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Where it's like, Hey, if you have some of those controls in place, you can now
Prasanna Malaiyandi:prevent unauthorized access to domains.
Prasanna Malaiyandi:They should not be having access to.
curtis:Exactly.
curtis:And then they started talking about preventing lateral movement inside.
curtis:Think about the ways that people need to move within your organization and allow
curtis:that, but block all other movement, right.
curtis:Lateral movement between servers and I, and I think, again, going back
curtis:to that company, that was a perfect example of, they had blocked all
curtis:lateral movement between all servers and I couldn't get my job done.
curtis:They're only problem w and they should have done that.
curtis:And, you know, they were forward thinking in that regard, but you do need to allow
curtis:exceptions for things like backup, right.
curtis:That is definitely a server to server lateral movement.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:And it's also other simple things.
Prasanna Malaiyandi:Like one of them was your favorite topic, right?
Prasanna Malaiyandi:Locking down RDP and SSH.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:If it's not needed, then lock it down.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:SMB is the same way as well for vCenter, right?
Prasanna Malaiyandi:Figuring out what actually needs access and what.
Prasanna Malaiyandi:Needs to be available to the internet.
Prasanna Malaiyandi:And one of the points he made is you should just assume that
Prasanna Malaiyandi:your inner internal network is as hostile as internet access.
Prasanna Malaiyandi:Right?
Prasanna Malaiyandi:So once an exploit happens, you can't trust anything internally.
curtis:They were also, I, you know, I didn't necessarily
curtis:agree with this one here.
curtis:And that was it's time to kill monolithic file servers.
curtis:Right.
curtis:Now I don't have a problem with the file server.
curtis:It's just, I think when, when they mean monolithic file server, they're just
curtis:saying a file server where everybody in the company can access all the data.
curtis:I would agree there anybody that's doing that, you know, in a
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:Segregate the data isolate to departments that need access.
Prasanna Malaiyandi:You use ACLs, make sure the people who need access have access and
Prasanna Malaiyandi:then monitor who's accessing what.
curtis:So they made a specific example of like, you know, just because just
curtis:because accounts receivable gets attacked, something shouldn't happen to payroll.
curtis:These are, these are both finance functions, but they're separate
curtis:financial functions and they should have their own areas.
curtis:Uh, and this is another one that I harp on is about protecting
curtis:privileged credentials.
curtis:And
Prasanna Malaiyandi:don't just have your password tattooed
Prasanna Malaiyandi:on your forehead, Curtis.
curtis:They recommended implementing, uh, things like LAPS, which I had
curtis:to look up, which stands for local administrator password solution.
Prasanna Malaiyandi:Uh, setting a different random password for
Prasanna Malaiyandi:the common local admin account on every computer in the domain.
Prasanna Malaiyandi:So you don't use one password for everything.
curtis:And then MFA, I think, I think every system, you know, every, every
curtis:privileged account needs to have MFA and, you know, I'm sorry, that's a pain.
curtis:I, you know, I use it all the time, but it what is
Prasanna Malaiyandi:but wait, why do you need a privileged account?
Prasanna Malaiyandi:You should.
Prasanna Malaiyandi:Here's the thing.
Prasanna Malaiyandi:Most times you should probably not need privileged accounts, so you do not need
Prasanna Malaiyandi:to access your privileged accounts.
curtis:Agreed, but, but they have to exist.
curtis:And so you have to lock them down this way.
curtis:I think what you're saying is MFA, shouldn't be that big of a deal for you.
curtis:If you set up modern administration.
Prasanna Malaiyandi:yeah.
Prasanna Malaiyandi:And you should rarely be using that.
curtis:Right.
curtis:Right.
curtis:And then very last on the list and I would have put it first, but you know, it's
curtis:just me and that was patching your stuff.
Prasanna Malaiyandi:How many times does that come up on the podcast?
Prasanna Malaiyandi:When we talk about ransomware, you know,
curtis:Yeah, exactly.
curtis:So the next one is about.
curtis:It's like, okay, so you got some ransomware.
curtis:Let's talk about the things that they're going to try to do.
curtis:The very first thing they listed was deleting of shadow copies.
curtis:And so I, and really shadow copies are basically like he's talking
curtis:about windows shadow copies.
Prasanna Malaiyandi:Yeah, I think windows shadow copies.
Prasanna Malaiyandi:Yup.
curtis:Right.
curtis:And so there is a tool here, which I had never heard of called raccine.
curtis:And it, it stops you from deleting shadow copies.
curtis:He said it stops everybody from deleting them.
curtis:So just realize that if you've got some regular thing that regularly deletes
curtis:shadow copies, it'll break that, but it looks it's something on github.
curtis:So it's, uh, you know, it's an open source tool.
Prasanna Malaiyandi:And just reading that briefly, I think many backup
Prasanna Malaiyandi:tools when you're backing up windows applications uses shadow copy.
Prasanna Malaiyandi:So be careful if you are using that because you may not
Prasanna Malaiyandi:be able to do your backups.
curtis:Yeah, that's a good question.
curtis:I, I guess, you know, I would differentiate between shadow
curtis:copies made just for the purposes of backups and shadow copies that
curtis:are made and then left there.
curtis:I don't know if there's like a different.
curtis:I know that when you make a snapshot, you say why you're making the snapshot.
Prasanna Malaiyandi:Yeah.
curtis:Um, but agreed that this is not something that you're just
curtis:going to download and just implement,
Prasanna Malaiyandi:Yeah.
curtis:might break all your backups.
curtis:Well, what it might do is it might allow you to create that snapshot,
curtis:but then it leaves all those snapshots around and let you delete them.
curtis:and you might get an error on your backup because you can't,
curtis:it can't delete the snapshot.
Prasanna Malaiyandi:yeah.
Prasanna Malaiyandi:Or your production could run out of space and then your app dies.
curtis:And then what's the next one
Prasanna Malaiyandi:So the next one is a common theme for us.
Prasanna Malaiyandi:Uh, when we talk about ransomware, it's less about the actual encrypting of data.
Prasanna Malaiyandi:It's the fact that these ransomware actors, especially the Conti group,
Prasanna Malaiyandi:they like to exfiltrate your data and steal sensitive data, and then hold you
Prasanna Malaiyandi:hostage and be like, Hey, you want to pay?
Prasanna Malaiyandi:Then you have to pay twice once for the decryption key.
Prasanna Malaiyandi:And then once to make sure we don't publish your data.
Prasanna Malaiyandi:And then sometimes they will still go and publish your data.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:So in this post, he talks about sort of, how can you make sure
Prasanna Malaiyandi:you can detect data exfiltration?
Prasanna Malaiyandi:And he talks about everything from, if you have, if you understand network
Prasanna Malaiyandi:patterns, you could look for anomalies.
Prasanna Malaiyandi:Um, you can also look at other tools.
Prasanna Malaiyandi:To see when data is actually being read and sent.
Prasanna Malaiyandi:So there's some interesting tools that he talked about.
Prasanna Malaiyandi:One that I never thought about, which was this mechanism called, uh,
Prasanna Malaiyandi:from things called Canary tokens,
Prasanna Malaiyandi:where it basically creates a false file.
Prasanna Malaiyandi:And any time someone accesses it, it generates a token and sends it home.
Prasanna Malaiyandi:And then it'll send you an email, say, Hey, by the way,
Prasanna Malaiyandi:someone accessed this file.
Prasanna Malaiyandi:So you can sort of get notified of, Hey, someone's accessing something, which
Prasanna Malaiyandi:they probably normally never should be.
Prasanna Malaiyandi:Because most of this ransomware software and data exfiltration, it's
Prasanna Malaiyandi:just programmatically reading, like scanning folders, reading files, right.
Prasanna Malaiyandi:Trying to figure out what to send.
curtis:Right.
curtis:And they mentioned both commercial solutions and open source solutions.
curtis:Like the one you mentioned, they also mentioned something called, uh, Zeke,
curtis:which, uh, And you know, that it analyzes NetFlow, but there are commercial
curtis:tools, which we've mentioned on here.
curtis:Um, and I, and I'd like to get, I'd like to get more of those companies on here.
curtis:And their recommendation was the same as mine, which is looking
curtis:for something that uses behavioral analytics to determine what is, and
curtis:is not a normal file transfer, right.
curtis:That should be able to spot a massive, uh, exfiltration attack..
curtis:And then the response against encryption, they talked about the EDR
curtis:XDR, which is I had to look that up.
curtis:I was not in my, so this is what,
Prasanna Malaiyandi:And point D
Prasanna Malaiyandi:endpoint detection and response.
curtis:right.
curtis:Okay.
curtis:So.
curtis:The idea is that if you've got, if you've got the money to put something
curtis:on each laptop that basically looks at and stops, massive file modifications,
curtis:it would detect and stop those.
curtis:Right.
curtis:And then same thing with the, with the honeypot.
curtis:I liked the idea with the creating an entire separate file server that has,
curtis:has all the same file names, but just with junk data, watch for anybody doing
curtis:anything there and then report on.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:And the interesting thing is when he was talking about honeypots, I didn't
Prasanna Malaiyandi:know, this is, he was like, oh yeah.
Prasanna Malaiyandi:And then to make it more realistic, you, there are a couple things you can do.
Prasanna Malaiyandi:You can map those device shares to actual endpoint devices.
Prasanna Malaiyandi:So they show up there because if I'm a ransomware program and I'm just looking
Prasanna Malaiyandi:at all the devices attached, right.
Prasanna Malaiyandi:I don't know if it's real or not.
Prasanna Malaiyandi:And the question came up, Hey, how do you hide it from your end users?
Prasanna Malaiyandi:Because you don't want your end users clicking on it as well.
Prasanna Malaiyandi:And there are registry commands in Windows, so you can actually hide them.
Prasanna Malaiyandi:So your users don't actually see those drives.
Prasanna Malaiyandi:And instead he suggested you actually bookmarked.
Prasanna Malaiyandi:Shared drive letters with these honeypot shared drives because ransomware,
Prasanna Malaiyandi:uh, programs are either going to start from a and work alphabetically
Prasanna Malaiyandi:or start from Z and come backwards, to see what drives are available.
Prasanna Malaiyandi:And then they'll just start looking that way.
curtis:So, so put a honeypot at a and put a honeypot at z.
Prasanna Malaiyandi:Yup.
curtis:I like um,
Prasanna Malaiyandi:were some really interesting things that he talked about.
curtis:And we can only cover a little bit here.
curtis:I just would highly recommend anybody that's interested in this, which should
curtis:be everybody go read this thread.
curtis:It's really well-written thread
Prasanna Malaiyandi:It's like how to trick ransomware
Prasanna Malaiyandi:and how to protect yourself.
curtis:Right.
curtis:then
Prasanna Malaiyandi:jump onto the third?
Prasanna Malaiyandi:Curtis.
curtis:Yeah.
curtis:Get up on the third?
curtis:one.
Prasanna Malaiyandi:Sorry, what is the third one about by the way?
curtis:Oh, the third one well, basically it's like, well, you've been infected.
curtis:What are we going to do?
curtis:Worst case scenario you've been infected and it's spread, and now
curtis:you need to reach for your backups.
curtis:So they mentioned go to the, the, the incident response plan.
curtis:And of course that assumes that you have one, which we've said
curtis:that you need to have one, right?
curtis:We we've mentioned repeatedly that a ransomware attack is
curtis:not the same as a disaster.
curtis:There are elements that I'd say a disaster is a subset of.
curtis:Uh, typical DR response is a subset of a, of a ransomware attack response.
Prasanna Malaiyandi:Think people get confused because in the end you're
Prasanna Malaiyandi:trying to do the same things, right.
Prasanna Malaiyandi:Get your data up.
Prasanna Malaiyandi:But I think the steps and the number of people, the different types of
Prasanna Malaiyandi:people involved are significantly different between just a normal DR.
Prasanna Malaiyandi:Versus a ransomware recovery.
curtis:Well, you know, simplistically to me, the biggest difference between,
curtis:uh, responding to a ransomware attack and a disaster, it'd be the
curtis:equivalent of like, if you're doing a DR and you've had a flood step
curtis:number one is drain the data center,
curtis:right?
curtis:Get all the water out of the data center.
curtis:Well, a ransomware attack is like, you're trying to drain the data center while you
curtis:have a person standing there with a fire hose, it's filling up your datacenter.
curtis:Right?
curtis:the, that's the difference between a disaster recovery and a
curtis:ransomware recovery is that they are actively still attacking you.
curtis:And you're actively experiencing the disaster at the same time as
curtis:you're trying to recover from it.
curtis:And so they've got a good thing here on what should be
curtis:in an incident response, right?
curtis:Some things you have to have in your incident response plan
curtis:got eight things about right.
curtis:Procedures and policies and an incident firm.
curtis:Right.
curtis:You, you need, you basically get professionals, retain them now, right?
curtis:Oh, by the way, I just, I just gotta throw out a really hilarious thing from,
curtis:uh, my granddaughter Lily yesterday.
curtis:So we have a friend, a mutual friend that was in a car accident a while back.
curtis:Not, not seriously injured, but injured enough that there is a lawsuit that
curtis:our, that, that that's going on.
curtis:And Lily said, uh, she, you know, she, she mentioned that I couldn't, she couldn't
curtis:pick her up because, you know, she was with her, she was with her lawyer and
curtis:then she looks at me, we were just walking and then she's like, do I have a lawyer?
curtis:I was like, no, I don't think you have a lawyer.
curtis:You don't need a lawyer right now.
Prasanna Malaiyandi:
Speaker:But, but you're right.
Prasanna Malaiyandi:
Speaker:Most people don't even think about that.
Prasanna Malaiyandi:
Speaker:Like even in like everyday, like normal situations, it's like, if
Prasanna Malaiyandi:
Speaker:I, God forbid get arrested, right.
Prasanna Malaiyandi:
Speaker:Who am I going to call?
Prasanna Malaiyandi:
Speaker:It's like,
curtis:Right.
curtis:Yeah.
curtis:And so w what they're saying here is, you know, go, go find who you're going to hire
Prasanna Malaiyandi:who are you going to call Ghostbusters?
curtis:going to call?
curtis:And, um, you know, and they got a policy, oh, a policy.
curtis:This is interesting policy for informing partners and customers and the media.
curtis:Right?
curtis:Decision-makers right.
curtis:All of that stuff.
curtis:This should all be decided upfront.
curtis:You should be deciding that now.
curtis:I don't know how many times we can say that.
Prasanna Malaiyandi:Yep.
Prasanna Malaiyandi:And then they talk about restoring your data.
curtis:Restoring your data.
curtis:right?
curtis:And I think how they said alright, three posts in and we
curtis:can finally talk about backups.
curtis:Right.
curtis:It's interesting here.
curtis:Right?
curtis:And he talks about, you know, the typical call-out is that ransomware's
curtis:going to target your backups.
curtis:And so you need some sort of immutable backup solution.
curtis:Right.
Prasanna Malaiyandi:Um, he does also talk and I know Curtis, you're probably
Prasanna Malaiyandi:going to have concerns with this, right?
Prasanna Malaiyandi:That you don't have to be offsite to protect your backups properly.
Prasanna Malaiyandi:He mentions that you could use strict network segmentation or other
Prasanna Malaiyandi:mechanisms to ensure separation, which would protect you in the case
Prasanna Malaiyandi:of ransomware, but may not protect you from all disasters that could occur.
curtis:Agreed.
curtis:And, and, and I don't, I don't have an issue with that, right.
curtis:Obviously, you know, I'll say obviously I work at a service-based backup company.
curtis:And we see that as the easy it's easy peasy.
curtis:All our backups are off site.
curtis:I I'm not against, you know, as a backup expert, I'm not against onsite backups.
curtis:There's a lot of good reasons for an onsite copy, but I completely agree
curtis:with this person that you have to protect that onsite copy from attacks.
curtis:And there are a lot of very common backup designs, incredibly common backup designs
curtis:that do not that the default installation of those products do not protect you.
curtis:Right.
curtis:And I, and I'll, you know, I don't wanna, I don't wanna pick on our friends at
curtis:Veeam, but that's a perfect example.
curtis:The guys from Veeam came on here and they explained to you, if you listen,
curtis:if you, if you haven't seen those episodes, go back and listen to them.
curtis:Uh, about, you know, when they talked about the, the Conti ransomware attacks
curtis:and how you can configure your Veeam backups to protect against that.
curtis:My concern is that most of their customers are not listening to this podcast, by
curtis:the way, they're more than welcome.
curtis:All 700,000 Veeam customers are more than welcome to come listen to the podcast.
curtis:But if, if you just do the default installation and you don't take their
curtis:recommendations on how to further protect your data, you know, it's no different
curtis:than any of the other products, right?
curtis:So
Prasanna Malaiyandi:Read
curtis:you've got to stop doing that.
curtis:Read the manual, read the best practices.
curtis:Call Rickatron.
curtis:Rickatron'll sort, you out and.
curtis:So he talks about that.
curtis:He also talks about testing, testing, your backups.
curtis:I'm editing right now, like literally in I'm in the middle of editing
curtis:the podcast, the episode of the restore test gone horribly wrong.
Prasanna Malaiyandi:backup.
curtis:It's going to be a great episode.
curtis:The.
curtis:Yeah, Schrodinger's backup.
curtis:Exactly.
curtis:That's going to, if, yeah, if you haven't heard that episode
curtis:go back and listen to it.
curtis:It's a, it's a
Prasanna Malaiyandi:Yeah.
curtis:uh,
curtis:episode.
Prasanna Malaiyandi:article also refers to it, right?
Prasanna Malaiyandi:Yeah.
curtis:yes, he does.
curtis:Uh, did he, did he actually refer to Shrodinger's
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:HInging your company's future on a schrodinger's backup thought
Prasanna Malaiyandi:experiment is a terrible idea.
Prasanna Malaiyandi:Don't do that.
curtis:Nice.
curtis:So, and then why don't you talk about the decryption part?
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:So I guess the final part right.
Prasanna Malaiyandi:Is you've been hit with encryption, right?
Prasanna Malaiyandi:So now what do you do?
Prasanna Malaiyandi:And in most cases, it's.
Prasanna Malaiyandi:You can try to get, like, if you're lucky, there might be a free
Prasanna Malaiyandi:decryptor out there for your data.
Prasanna Malaiyandi:It's just going to take a very long time.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:And if you do pay the ransom and you have to understand that paying the ransom may
Prasanna Malaiyandi:be illegal to some of these groups, right?
Prasanna Malaiyandi:They'll give you back a decryption key.
Prasanna Malaiyandi:Hopefully it'll work.
Prasanna Malaiyandi:It's not, it's in the ransomware.
Prasanna Malaiyandi:Group's best interest not to cheat you there, but you're
Prasanna Malaiyandi:taking a risk there as well.
Prasanna Malaiyandi:And then finally, Once you've actually decrypted your data.
Prasanna Malaiyandi:You've gone back up and running.
Prasanna Malaiyandi:There's nothing that prevents them from either coming back
Prasanna Malaiyandi:and attacking you again, if you haven't fixed anything right.
Prasanna Malaiyandi:Or the next group coming back.
Prasanna Malaiyandi:Cause that's another common thing you see is one group gets in encrypts your data.
Prasanna Malaiyandi:Another group figures out a different mechanism because they
Prasanna Malaiyandi:know now that you're willing to pay.
Prasanna Malaiyandi:And so they might come after you as well.
Prasanna Malaiyandi:So even once you have your data decrypted, it's not the end of the story.
curtis:Right.
curtis:And then the there's a, there's a what's next and, and, and all of
curtis:these words, and this is a really long series of posts, which I highly
curtis:recommend you go look through.
curtis:There's one part where they typed in all caps, and this is it right when
curtis:you're done, whatever you did restore, pay the ransom, whatever it is.
curtis:It's not over, you clearly have a serious gap in your defenses.
curtis:You need to find these and fix them.
curtis:And then this is all caps and you need to understand that those gaps are bigger
curtis:than just whatever the initial breach vector was as highlighted in parts one
curtis:and two of this series, there are several opportunities to stop a ransomware
curtis:breach before it gets to this point.
curtis:So, um, there, there was some other.
curtis:It was another one that I read, uh, somebody, they said, well, if I, if
curtis:I, if I was at a company that had a highly, I think it was actually in here.
curtis:If I was at a company that a highly publicized breach does this hurt my
curtis:chances of getting a job and the author of this article didn't think so, because they
curtis:basically said you now have experience
Prasanna Malaiyandi:Yep.
Prasanna Malaiyandi:I think it was actually at the end of this article is where he wrote about that.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:He's like, yeah.
Prasanna Malaiyandi:It's something you should actually show that you've gone through this because for
Prasanna Malaiyandi:a lot of people it's just theoretical.
Prasanna Malaiyandi:They've never experienced it.
Prasanna Malaiyandi:It's like you Curtis.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:I can sit here and talk about like how to back up your data, how to restore
Prasanna Malaiyandi:your data, ideally how it should be done.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:But I've never cut my teeth in a production environment, trying to do a
Prasanna Malaiyandi:restore with people, yelling at me over my shoulder or watching over my shoulder.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:You have, and I think that's sort of the difference, right?
Prasanna Malaiyandi:Is you have that experience because trial by fire.
curtis:Yeah, I, you know, you just, you just reminded me of, and I know
curtis:I've told this story before, but not everybody's listening to every episode.
curtis:My, one of my favorite restore stories was, was back at my first big job.
curtis:And we had somebody in the NOC that was coordinating the various things that
curtis:were happening of this big restore.
curtis:And we had another guy that was in the data center that
curtis:was actually doing things and.
curtis:He was talking to the person who was on the phone in the NOC.
curtis:And he, he didn't know that he was on speaker.
curtis:And so he said, he's like, oh, so you know where you are.
curtis:I'm in the NOC.
curtis:He goes, oh, so I suppose you have Tom and Tom standing on
curtis:your left and right shoulder.
curtis:And he was referring to our boss's boss and our boss's boss's boss.
curtis:Right.
curtis:And, um, the, uh, that would be Tom Thomaides and Tom Lackey.
curtis:And they were indeed standing both on his left and right shoulder.
curtis:And they said that when he said that, oh, so you have Tom and Tom standing
curtis:on your left and right shoulder.
curtis:He said they just both took one step back.
Prasanna Malaiyandi:but it's true, right?
Prasanna Malaiyandi:It's a stressful thing everyone's watching to make sure it goes perfect.
curtis:Right.
curtis:And, um, so, you know, we wish you all the best of luck.
curtis:I continue to be concerned about our, our friends over there in the Ukraine.
curtis:And, uh, we wish them the best of luck and.
curtis:You should also be concerned about the potential ramifications that all of
curtis:that has on continued further attacks on your data center and read this
curtis:article, read every word of this article, not just this summary and, um, you
Prasanna Malaiyandi:the three
curtis:read all three parts and we'll, we'll put links to
curtis:it in the show description so that you can easily find it.
curtis:Cause finding stuff on Reddit is not necessarily easy.
curtis:So, uh, Thanks again Prasanna for your wise, uh, shipping advice
curtis:and, um, you know, a good, good commentary on article well.
Prasanna Malaiyandi:anytime Curtis and I hope I know, normally when we talk
Prasanna Malaiyandi:about ransomware, you get very depressed.
Prasanna Malaiyandi:So I, it feels like this isn't a depressing article.
Prasanna Malaiyandi:It feels like here are things you should be doing.
Prasanna Malaiyandi:So
curtis:Here are things that you should do now.
Prasanna Malaiyandi:Yeah,
curtis:Yeah, absolutely.
curtis:So, all right, well, thanks to the listeners.
curtis:Uh, you know, we'd be nothing without you remember to subscribe
Listen On
