Security expert rips Okta for their response to hack
We have none other than Snorkel42 from Reddit on the podcast today. He has 20 years experience in InfoSec, and is a prolific writer on Reddit under the handle Snorkel42. (Check out his posts here: https://www.reddit.com/user/snorkel42/). (We will not be using his given name during the recording.).
He thinks Okta managed to turn a mole hill into a mountain by incorrectly handling the hack that happened in January – that we just learned about last week. That's right, we just found out about a hack that actually happened in January!
We dive deep into what happened, what it means, and how the worst problem of all is how Okta responded to it. Our expert says he no longer trusts Okta, and gives advice to customers on what to do next.
This is a very timely episode that you will really enjoy – unless you're an Okta customer or employee.
Mentioned in this episode:
Interview ad
Snorkel42 00:00:00
can I make an admission real quick?
Snorkel42 00:00:02
I had no idea what that karma meant.
W. Curtis Preston:Yeah.
Snorkel42 00:00:06
I did not know what that number translated.
W. Curtis Preston:Hi, and welcome to Backup Central's Restore it All podcast.
W. Curtis Preston:I'm your host, W.
W. Curtis Preston:Curtis Preston.
W. Curtis Preston:AKA Mr.
W. Curtis Preston:Backup.
W. Curtis Preston:And I have with me, my fellow photo-shoot model, Prasanna Malaiyandi.
W. Curtis Preston:going, Prasanna?
Prasanna Malaiyandi:I'm good.
Prasanna Malaiyandi:That was a very interesting experience, but I had fun and
Prasanna Malaiyandi:it was good to see you too.
Prasanna Malaiyandi:What's funny is that's the first time I've been to an office in the last two years.
Prasanna Malaiyandi:It just wasn't my office.
W. Curtis Preston:Right, Yeah.
W. Curtis Preston:yeah, w it is actually, Druva's new office.
W. Curtis Preston:We moved from, we were over on California avenue in Sunnyvale,
W. Curtis Preston:and now we're over on.
W. Curtis Preston:What is it?
Prasanna Malaiyandi:mission college
W. Curtis Preston:college in Santa Clara across from the Intel museum?
W. Curtis Preston:I believe,
Prasanna Malaiyandi:yeah, it's one of Intel's campuses
W. Curtis Preston:Yeah.
W. Curtis Preston:and, so I wanted to do some photos podcast.
Prasanna Malaiyandi:Curtis tired of his face being on the homepage
Prasanna Malaiyandi:or the title art for the podcast.
Prasanna Malaiyandi:But I liked your photo.
W. Curtis Preston:what's that.
Prasanna Malaiyandi:I liked the picture of you on the podcast.
W. Curtis Preston:have you seen the new, the the bearded contemplated one?
Prasanna Malaiyandi:Yes, I have.
Prasanna Malaiyandi:That one's also good too.
Prasanna Malaiyandi:Yeah.
W. Curtis Preston:I got lots of compliments on that photo,
W. Curtis Preston:but I wanted your picture.
W. Curtis Preston:I wanted this giant mane of yours to be captured because I think one
W. Curtis Preston:day you're going to come to your senses and maybe cut it all off and
W. Curtis Preston:I want to capture It for posterity.
W. Curtis Preston:And so we had a photo shoot and, we got to, we got some, it was.
W. Curtis Preston:a lot of fun doing that,
Prasanna Malaiyandi:It was.
Prasanna Malaiyandi:Thank you, Alex, if you're listening to this, but that was awesome.
Prasanna Malaiyandi:Thank you for the help.
W. Curtis Preston:Alex did a great job and there were, just a couple
W. Curtis Preston:of dudes taking photos in an office.
W. Curtis Preston:It wasn't awkward at any time.
Prasanna Malaiyandi:No, not at all.
W. Curtis Preston:Not at all.
W. Curtis Preston:Yeah.
W. Curtis Preston:Look longingly into each other's eyes.
W. Curtis Preston:Yeah, By the time you hear this, hopefully those photos or some
W. Curtis Preston:of those photos will be up on the website, at backupcentral.com.
W. Curtis Preston:And, before I bring on our guest, I'll go ahead and throw out our usual disclaimer.
W. Curtis Preston:I mentioned, the Druva office I work for Druva.
W. Curtis Preston:Prasanna happens to work for zoom.
W. Curtis Preston:This is not a podcast of either company.
W. Curtis Preston:And the opinions that you hear are all ours.
W. Curtis Preston:Be sure to rate us at ratethispodcast.com/restore
W. Curtis Preston:and subscribe to the podcast.
W. Curtis Preston:You can either do it in your favorite pod catcher, or you can
W. Curtis Preston:go over to backupcentral.com and subscribe to our mailing list.
W. Curtis Preston:And we'll let you know, every time we come out with an episode and, we also are
W. Curtis Preston:always looking for interesting guests.
W. Curtis Preston:We have discovered a good one this week.
W. Curtis Preston:I'm very excited to bring him on.
W. Curtis Preston:And, if you're interested in the things we're interested, which is data
W. Curtis Preston:protection, disaster recovery, backup and recovery security, beer, barbecue.
Prasanna Malaiyandi:barbecue.
Prasanna Malaiyandi:Definitely.
Prasanna Malaiyandi:Yes.
Prasanna Malaiyandi:To barbecue.
W. Curtis Preston:Definitely.
W. Curtis Preston:Yes.
W. Curtis Preston:To barbecue.
W. Curtis Preston:And, then, just reach out to me @wcpreston on Twitter or wcurtispreston@gmail.com.
W. Curtis Preston:Today I wanted to talk about a little something has
W. Curtis Preston:happened in the InfoSec world.
W. Curtis Preston:Isn't that right, Prasanna?
Prasanna Malaiyandi:Yep.
Prasanna Malaiyandi:just a tiny little something, nothing big.
W. Curtis Preston:I'm flashing back to the matrix.
W. Curtis Preston:I don't remember which one.
W. Curtis Preston:I think it was the third one, the key master.
W. Curtis Preston:you got the guy with all the keys.
W. Curtis Preston:So we're talking of course, about the Okta compromise that went wide this week.
W. Curtis Preston:So this guest Prasanna,
Prasanna Malaiyandi:You're excited.
Prasanna Malaiyandi:Aren't you Curtis?
Prasanna Malaiyandi:Aren't you?
W. Curtis Preston:I, am.
W. Curtis Preston:He's shrouded in mystery.
W. Curtis Preston:we've had guests on before where we've used pseudonyms.
W. Curtis Preston:but I knew those people.
W. Curtis Preston:And so I knew their actual names.
W. Curtis Preston:And then I had to pretend not to know their names and refer to them
W. Curtis Preston:by Harry Potter and Ron Weasley.
W. Curtis Preston:And you may recall that during the.
W. Curtis Preston:Recording.
W. Curtis Preston:Occasionally I would slip up and call
Prasanna Malaiyandi:Yes.
Prasanna Malaiyandi:I remember that.
Prasanna Malaiyandi:Lots of editing for you.
Prasanna Malaiyandi:Lots of listening to it.
W. Curtis Preston:But in this case, I have no idea who this guy actually is.
W. Curtis Preston:I just know that he knows his stuff.
W. Curtis Preston:He has been in IT for 25 years in the InfoSec space for about 20.
W. Curtis Preston:He has are you ready for this?
W. Curtis Preston:He has a karma rating on Reddit of 33,000.
W. Curtis Preston:I'm excited that I've got like 600.
W. Curtis Preston:He's got 33,000, which means that at least 33,000 times someone has upvoted him.
W. Curtis Preston:That's impressive in and of itself because Reddit is a crazy place
W. Curtis Preston:where if you say things that people don't like, even if you're correct,
W. Curtis Preston:they'll vote you down anyway.
W. Curtis Preston:So he's managed to convince 33,000 people somewhere to click, on what he wrote.
W. Curtis Preston:Welcome to the podcast.
W. Curtis Preston:Snorkel42.
Snorkel42 00:05:26
Hello, can I make an admission real quick?
Snorkel42 00:05:29
I had no idea what that karma meant.
W. Curtis Preston:Yeah.
Snorkel42 00:05:33
I did not know what that number translated.
W. Curtis Preston:Every time you get an up vote, you get karma and
W. Curtis Preston:every time someone downvotes you, your karma is subtracted from.
Snorkel42 00:05:44
Okay, now I know
W. Curtis Preston:Yeah.
Snorkel42 00:05:47
me
W. Curtis Preston:I post in a number of non it, subreddits where very
W. Curtis Preston:much you can say the right thing, the correct thing, and they will
W. Curtis Preston:get mad at you and downvote you, and then you just lost a bunch of karma.
W. Curtis Preston:And since I'm, I'm a relatively, especially compared to you,
W. Curtis Preston:I'm a relatively new Redditor, would that be the right word?
W. Curtis Preston:And I'm trying to,
Prasanna Malaiyandi:Reddit poster.
W. Curtis Preston:It helps you bubble up into threads and things.
W. Curtis Preston:it gives your posts more weight.
W. Curtis Preston:So yeah.
W. Curtis Preston:The fact that you have 33,000 is a BFD.
Snorkel42 00:06:25
I'll get a t-shirt made.
W. Curtis Preston:Why don't you describe what happened at Okta.
Snorkel42 00:06:31
Yeah, I guess to start, I don't have any insider knowledge here.
Snorkel42 00:06:36
All I have is what's been announced and what little Okta has
Snorkel42 00:06:40
been able to slip through their marketing and legal teams, but
W. Curtis Preston:We're going to get
Snorkel42 00:06:45
to right.
Snorkel42 00:06:46
but basically they contract out their customer support to a third
Snorkel42 00:06:50
party located in Costa Rica.
Snorkel42 00:06:52
and sometime around the end of January.
Snorkel42 00:06:56
A attacker from the lapses group gained access to one of those support
Snorkel42 00:07:01
engineer's laptops, supposedly over RDP, which is something else to get into.
Snorkel42 00:07:05
and for about five days they had access to, that laptop and were able to
Snorkel42 00:07:10
monitor if not interact the privileged access that support engineer had.
Snorkel42 00:07:14
So Okta has a, an internal application that they call a super user,
Snorkel42 00:07:18
which by the way, what a terrible name doesn't mean that you have,
Snorkel42 00:07:21
super user God mode necessarily.
Snorkel42 00:07:24
It's just the administrative interface to.
Snorkel42 00:07:26
Um, but it doesn't necessarily mean that, Hey, I'm going to go in and reset
Snorkel42 00:07:30
everyone's password to something I know, but it did give them the ability
Snorkel42 00:07:33
to reset passwords, to send, links to customers, to go reset their passwords
Snorkel42 00:07:38
and to reset their multifactor.
Snorkel42 00:07:40
When Okta started off with, their CSO saying no customers impacted.
Snorkel42 00:07:44
We detected this back in January, or we detected an unsuccessful attempt.
Snorkel42 00:07:49
Which is the other thing to get into here, back in January and shut it down.
Snorkel42 00:07:53
And then the second blog post, they added a little bit more.
Snorkel42 00:07:56
And then the third blog posts like, 2.5% of our customers,
Snorkel42 00:07:59
about 366 people impacted,
Prasanna Malaiyandi:Yeah, and it's interesting because it's
Prasanna Malaiyandi:366 people, but it's really 366 customers, which could be of any size.
Snorkel42 00:08:09
and we certainly know from the screenshots that CloudFlare was one
Snorkel42 00:08:12
of those customers and CloudFlare CEO was fairly vocal on Twitter about his
Snorkel42 00:08:16
dissatisfaction with Okta right now.
Prasanna Malaiyandi:And CloudFlare, I thought published a pretty
Prasanna Malaiyandi:good blog as well about sort of their analysis and what they did.
W. Curtis Preston:fact, a lot of people felt that Cloudflare's analysis was
W. Curtis Preston:better than what Octa provided themselves.
Snorkel42 00:08:35
me included.
Snorkel42 00:08:36
Yeah.
Prasanna Malaiyandi:Okay.
W. Curtis Preston:Yeah.
W. Curtis Preston:and by the way, there, there were a couple of people you put on a, another
W. Curtis Preston:Reddit thread that was about this.
W. Curtis Preston:Maybe more than one.
W. Curtis Preston:I don't know, but I know I was reading one of them.
W. Curtis Preston:And more than one former employee of Okta logged in and, and replied and
W. Curtis Preston:explained basically what the super-user, which I agree it's a really bad name,
W. Curtis Preston:but, I don't know where to start, but the thing about the resetting passwords
W. Curtis Preston:and stuff, like they, it didn't appear that they had the ability to do anything,
W. Curtis Preston:to be able to access an account.
W. Curtis Preston:They could reset a password.
W. Curtis Preston:They could reset or even remove MFA, but that, but anything towards
W. Curtis Preston:that would be sent to the customer.
W. Curtis Preston:So at best, they would only be able to do that if they had access to
W. Curtis Preston:that customer in the first place.
W. Curtis Preston:So it didn't look like they would have been able to use this to
W. Curtis Preston:actually access any customers.
W. Curtis Preston:does that sound about right?
W. Curtis Preston:Yeah.
Snorkel42 00:09:44
It sounds about right to me for a few reasons.
Snorkel42 00:09:46
Look to me, the biggest issue here has been the way Octa has been communicating
Snorkel42 00:09:52
to customers and the lack of transparency.
Snorkel42 00:09:54
And the reason why it's important is I find myself in my own back of
Snorkel42 00:09:59
my mind saying, if we believe Okta.
Snorkel42 00:10:02
Every time I think about this.
Snorkel42 00:10:04
If we believe them, and the only reason I have that question for a
Snorkel42 00:10:06
company that their entire existence hinges on, if we believe Okta.
Snorkel42 00:10:11
Like they provide, maybe we should start that off of what does Okta do.
Snorkel42 00:10:15
So Okta provides authentication services and identity management to corporations,
Snorkel42 00:10:19
particularly around single sign-on And, multi-factor, and the way, you can place
Snorkel42 00:10:22
them in your infrastructure can get as far as deep as: through the Okta portal,
Snorkel42 00:10:27
I can reset your active directory, your on-prem active directory password.
Snorkel42 00:10:31
so you know, it's a company that holds a lot of keys and if you
Snorkel42 00:10:35
can't trust them, my God, how do you keep them in your enterprise?
Snorkel42 00:10:40
And so since to your point of, yeah, it doesn't look like they had that
Snorkel42 00:10:44
kind of access to really take over the accounts of Okta customers.
Snorkel42 00:10:49
And I believe that to be true, not so much because of what Okta said, but mainly
Snorkel42 00:10:53
because lapses didn't appear to use it.
Snorkel42 00:10:55
I think if they would have had that access, we would have seen a
Snorkel42 00:10:57
much bigger impact, but I don't.
W. Curtis Preston:I'm a hundred percent on board with everything that
W. Curtis Preston:you're saying about, that the biggest issue here has been Okta's response.
W. Curtis Preston:I just wanted to for anyone who's listening to this for the
W. Curtis Preston:first time, it sounds horrible.
W. Curtis Preston:And screenshots of customer data sounds horrible, but it, I do want
W. Curtis Preston:to at least say it does look based on the information that we have most of
W. Curtis Preston:which did not come from Okta, that they wouldn't have been able to actually
W. Curtis Preston:access any customer's environment.
W. Curtis Preston:They might have been able to annoy some customers, Change your
W. Curtis Preston:passwords and things like that.
W. Curtis Preston:but they.
W. Curtis Preston:I completely agree with you that from the get-go, like from the very beginning from
W. Curtis Preston:message one and all the way up to message three their verbiage is really weird,
Snorkel42 00:11:47
Yeah.
Prasanna Malaiyandi:I wondering if this has to do with any of, the new laws going
Prasanna Malaiyandi:through Congress, around data breaches, or just public perception with everything
Prasanna Malaiyandi:going on in the world right now that they just did a PR blunder, if you will.
Snorkel42 00:12:06
It could be that.
Snorkel42 00:12:08
Yeah.
Snorkel42 00:12:08
and certainly they are a publicly traded company and that has certainly been
Snorkel42 00:12:12
called out a number of forums of, they, they may be tied as to what they can say.
Snorkel42 00:12:17
That one of the things that really sticks out to me is from Cloudflare's response,
Snorkel42 00:12:22
it was very clear that they learned about this along with everyone else.
Snorkel42 00:12:25
So one day they woke up and saw screenshots on Twitter from an
Snorkel42 00:12:30
attack group of their information.
Snorkel42 00:12:32
And Okta has a publicly disclosed, privacy and security policy that - i can even name
Snorkel42 00:12:39
the sections 20 and 21- talk about when they will alert customers of a breach.
Snorkel42 00:12:44
And I think you would have to do some pretty fancy legal footwork
Snorkel42 00:12:50
to explain why CloudFlare did not know about this in January 20.
Snorkel42 00:12:53
To me that is the real big takeaway from, do I trust this company anymore?
Snorkel42 00:12:57
The fact that the customers that we do know were impacted clearly
Snorkel42 00:13:00
didn't find out until the rest of the world found out as well.
Snorkel42 00:13:03
Um, and that's an example of Okta not following their own policies
Snorkel42 00:13:06
and that's troubling to me.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:And especially because these are in legal contracts, You could be held
Prasanna Malaiyandi:liable for it and you're losing the trust of your customers, right?
Prasanna Malaiyandi:Who would trust Okta the next time something happens?
Snorkel42 00:13:17
yeah.
Snorkel42 00:13:17
And then of course the timing, all of it is very interesting in that Okta is
Snorkel42 00:13:21
saying that we couldn't respond until we got the security incident report
Snorkel42 00:13:26
from the company that this third party hired, and we just got that conveniently.
Snorkel42 00:13:30
As the screenshots got posted.
Snorkel42 00:13:31
but even then it's so you had a third party contractor to get compromised
Snorkel42 00:13:35
and what kind of access they had.
Snorkel42 00:13:37
You just sat on it for three months.
Snorkel42 00:13:39
are you telling me that Okta did not actually get involved in
Snorkel42 00:13:41
that incident response at all?
Snorkel42 00:13:44
so just a lot of things not adding up and it certainly doesn't
Snorkel42 00:13:47
paint a pretty picture for Okta.
W. Curtis Preston:yeah.
W. Curtis Preston:I forgot.
W. Curtis Preston:I w I was scrolling through the post right now, the verbiage where it says,
W. Curtis Preston:you made a comment that, that it seemed that, marketing was involved in these,
W. Curtis Preston:the statements that went out because like it did start out with even in the third
W. Curtis Preston:one, even in the third message, they referred to it as, an attempt to access.
Snorkel42 00:14:15
and unsuccessful.
W. Curtis Preston:This was not an attempt.
W. Curtis Preston:This was a hack.
W. Curtis Preston:I don't know what the proper term is.
Snorkel42 00:14:20
no, I think a hack is absolutely the right term And
Snorkel42 00:14:23
so they also have a timeline.
Snorkel42 00:14:25
And I think if you marry those blog posts, where they talk about
Snorkel42 00:14:29
unsuccessful attempts and the timeline.
Snorkel42 00:14:30
You, you see where they're getting to the unsuccessful attempt that I derive is.
Snorkel42 00:14:35
where Okta caught em.
Snorkel42 00:14:36
which honestly, we need to take a step back and really give Okta security, some
Snorkel42 00:14:40
kudos in the detection to begin with.
Snorkel42 00:14:42
So where Okta caught them was the attacker apparently attempted to add
Snorkel42 00:14:47
a second MFA token to the support engineer's account, so that they could
Snorkel42 00:14:51
start approving from what I've been told.
Snorkel42 00:14:53
Okta internally uses MFA a lot.
Snorkel42 00:14:56
So apparently Okta is huge on every step you take within their networks MFA.
Snorkel42 00:15:01
So it looks like the attacker tried to add their own MFA token.
Snorkel42 00:15:04
So while they had RDP access and the support engineer was
Snorkel42 00:15:08
away, they could start moving around and start respond to MFA.
Snorkel42 00:15:11
And Okta caught that.
Snorkel42 00:15:13
The addition of an MFA token from a weird location.
Snorkel42 00:15:16
Which is fantastic, like really great job and darn it.
Snorkel42 00:15:20
So I think what, Okta is referring to when they say, there was an unsuccessful
Snorkel42 00:15:25
attempt, was the unsuccessful attempt to add that MFA token.
Snorkel42 00:15:28
But boy is that some marketing wordplay there that say, oh Yeah.
Snorkel42 00:15:32
it's an unsuccessful attempt to take over an engineer laptop.
Snorkel42 00:15:35
no, they took it over.
Prasanna Malaiyandi:They took it over.
Prasanna Malaiyandi:They had access to your network.
Prasanna Malaiyandi:They got in, they just weren't able to.
Snorkel42 00:15:44
Yeah.
W. Curtis Preston:And for five days they could do, because they were
W. Curtis Preston:controlling that laptop via RDP, which is the remote desktop protocol, which
W. Curtis Preston:should not be exposed to the internet.
Prasanna Malaiyandi:Curtis' second favorite topic, I think.
Snorkel42 00:15:56
and so it is a really good point.
Snorkel42 00:16:00
And I think this is what's way more interesting about this.
Snorkel42 00:16:03
Is w was it exposed to the internet?
Snorkel42 00:16:05
We don't know if it wasn't then it would sure seem, the lapses actually have access
Snorkel42 00:16:13
to this third party's network first, and then managed to get RDP access to this
Snorkel42 00:16:18
contractors laptop, the screenshots show Global Protect of which Global Protect
Snorkel42 00:16:22
is a VPN product from Palo Alto Networks.
Snorkel42 00:16:25
So it was, this person worked from home.
Snorkel42 00:16:26
possibly and was their home network breached.
Snorkel42 00:16:29
So it does, from an InfoSec standpoint, certainly this screams to me.
Snorkel42 00:16:32
you should have RDP locked down on your workstations, does your laptop need
Snorkel42 00:16:36
to be able to accept RDP connections?
Snorkel42 00:16:38
Absolutely not.
Snorkel42 00:16:39
But, yeah.
Snorkel42 00:16:39
So I think that RDP side though is really a big topic because it has a lot of, fine
Snorkel42 00:16:48
read between the lines there of, what does it mean that they had RDP access?
Snorkel42 00:16:51
How did they even reach it over RDP?
Prasanna Malaiyandi:And it's something we may never really find out.
Prasanna Malaiyandi:Unless.
Snorkel42 00:16:59
Yeah.
Prasanna Malaiyandi:because noone's really going to bring that up.
Prasanna Malaiyandi:Noone's going to talk about that.
Prasanna Malaiyandi:They're just going to say, yeah, we stopped the attack
Prasanna Malaiyandi:or whatever the breach was.
Prasanna Malaiyandi:They'll focus on the Okta side.
Prasanna Malaiyandi:Everyone talks about okay, we found the issue or we saw what they were
Prasanna Malaiyandi:trying to do and we stopped it.
Prasanna Malaiyandi:End of story.
Prasanna Malaiyandi:Not necessarily.
Prasanna Malaiyandi:How do they really get in the first place?
Prasanna Malaiyandi:And what does that look like and how do we prevent that from happening.
Snorkel42 00:17:20
Yeah, and the thing to keep in mind too.
Snorkel42 00:17:22
When we think about this is a third-party contractor that specializes in 24/7
Snorkel42 00:17:27
customer support for larger enterprises.
Snorkel42 00:17:29
If lapses, which is a attack group, that's setting the world on fire Right.
Snorkel42 00:17:33
now had access to their network.
Snorkel42 00:17:36
What else was going on?
Snorkel42 00:17:38
Okta may be the tip the iceberg for them.
W. Curtis Preston:Yeah.
W. Curtis Preston:somebody, one of the commenters, they felt that the whole like throwing out the.
W. Curtis Preston:Screenshots from Okta was actually an attempt at subterfuge on the part
W. Curtis Preston:of lapses to throw away attention from the fact that the real problem
W. Curtis Preston:is the access you just mentioned.
W. Curtis Preston:Somebody said, maybe that's why they threw out all the Okta information because.
W. Curtis Preston:we're not talking about Sykes.
W. Curtis Preston:We're talking about Okta.
W. Curtis Preston:The technically the hack was actually a Sykes.
W. Curtis Preston:Okta was just the customer in this case.
Snorkel42 00:18:07
And Octa is very quick to call that out.
Snorkel42 00:18:10
They say third party, as often as they possibly can as if that's,
Snorkel42 00:18:14
oh, wash our hands of that.
Snorkel42 00:18:15
Sure, we gave that third-party access to go reset your passwords
Snorkel42 00:18:18
and MFAs, but that's their problem.
W. Curtis Preston:There were people, a handful of people in the comments
W. Curtis Preston:and there, as there always is on the internet, there were a handful of people
W. Curtis Preston:that came to Octa's defense regarding that they get thousands of attacks a day.
W. Curtis Preston:And that they're saying no customer systems were accessed.
W. Curtis Preston:and so was there really a duty to report back in January?
W. Curtis Preston:What do you think about that?
Snorkel42 00:18:50
The title of the post I had made was am I overreacting?
Snorkel42 00:18:53
cause I is probably coming through in this podcast.
Snorkel42 00:18:55
I'm still quite upset with them.
Snorkel42 00:18:57
and I agree with a lot of what the, what those folks were saying.
Snorkel42 00:19:00
yes, I'm sure Okta is attacked repeatedly.
Snorkel42 00:19:04
Now the question is what does that look like if they are attacked so often to the
Snorkel42 00:19:09
point that people have access to the super user program that it's such a non-event
Snorkel42 00:19:13
for them, like that's an everyday event
Snorkel42 00:19:15
then holy cow, right?
Snorkel42 00:19:16
Like seriously, if that's every day for Okta, then we had something
Snorkel42 00:19:19
way bigger to talk about here.
Snorkel42 00:19:20
I'm guessing that's not the case.
Snorkel42 00:19:22
I'm guessing this.
Snorkel42 00:19:23
This was a significant event for, um, and downplaying that I don't buy.
Snorkel42 00:19:28
I think this was a significant event and Okta was very happy to.
Snorkel42 00:19:32
keep it under wraps and hope that it never came out.
Snorkel42 00:19:35
and I think I, as we said at the start, yeah, I don't think anyone
Snorkel42 00:19:39
really, I don't think there was any actual breach of a customer's account.
Snorkel42 00:19:43
I think what we saw on the screenshots was pretty much all that happened,
Snorkel42 00:19:47
but I think if you were to get CloudFlare CEO on those podcasts, he
Snorkel42 00:19:51
would tell you that was significant.
Snorkel42 00:19:52
and the fact that he didn't know about it until a couple of days
Snorkel42 00:19:56
ago, when it was posted on Twitter, Was not acceptable for him.
Snorkel42 00:19:59
And I'll be really surprised if CloudFlare isn't looking at moving
Snorkel42 00:20:03
to another provider right now.
Prasanna Malaiyandi:It reminds me.
W. Curtis Preston:If the Cloudflare CEO wants to come on this podcast.
W. Curtis Preston:he's he or she is more than welcome.
Prasanna Malaiyandi:This kind of reminds me like how bad it could be
Prasanna Malaiyandi:of, I don't know if you both recalled the RSA hack that happened many years
Prasanna Malaiyandi:ago where the root key was compromised.
Prasanna Malaiyandi:Because that's almost what could have happened to Okta, except in the case of
Prasanna Malaiyandi:Okta, there is no hardware fobs, right?
Snorkel42 00:20:29
so again, I think Okta does absolutely deserve some praise here.
Snorkel42 00:20:34
despite giving the super user application a really stupid name,
Snorkel42 00:20:38
this tier two support engineer, didn't have the ability to reset the password
Snorkel42 00:20:42
to something that he would know.
Snorkel42 00:20:43
if that were a scenario, if the attacker could have gone in and made
Snorkel42 00:20:46
the password password, then this would have been a much bigger deal.
W. Curtis Preston:
Speaker:subsequently deactivating MFA.
Snorkel42 00:20:54
yeah, absolutely.
W. Curtis Preston:Change the password to what you want and
W. Curtis Preston:subsequently deactivating MFA.
W. Curtis Preston:You're in.
Snorkel42 00:21:00
Yeah.
Snorkel42 00:21:00
and if you start thinking about how many customers Okta has and what Octa
Snorkel42 00:21:03
actually does and where I'm walking into.
Snorkel42 00:21:08
If they had access to the fed ramp, if they were able to get
Snorkel42 00:21:10
into government systems that way.
Snorkel42 00:21:12
But, Zoom, for example, if they were able to get into the zoom
Snorkel42 00:21:14
Okta page, what applications would they be able to get into?
Snorkel42 00:21:17
I can tell you from my company, it's pretty much domain admin.
Snorkel42 00:21:21
you have access to everything, right?
Snorkel42 00:21:23
definitely kudos to Okta for having those controls.
Snorkel42 00:21:26
And again, I really do praise their security team for
Snorkel42 00:21:29
catching it that quickly.
Snorkel42 00:21:30
that was an excellent detection on their part, especially for
Snorkel42 00:21:32
a third party in Costa Rica.
Snorkel42 00:21:34
Having that kind of logging.
Snorkel42 00:21:35
Fantastic.
Snorkel42 00:21:36
but Yeah.
Snorkel42 00:21:36
to your point, it could have been massive.
W. Curtis Preston:Yeah, this, I do think I, I liked that even though, we
W. Curtis Preston:agree it was a weird, it's a weird name.
W. Curtis Preston:It does appear that.
W. Curtis Preston:That th that there, what they did employ the concept of least privilege, right?
W. Curtis Preston:There's a reason that they have that, that they have the ability for
W. Curtis Preston:a support person to do the password reset because sometimes customers get
W. Curtis Preston:locked out of their accounts there's no other way to do that, but they at
Snorkel42 00:22:10
I need to interrupt because I disagree.
W. Curtis Preston:you disagree?
Snorkel42 00:22:14
I do.
Snorkel42 00:22:15
so here's the thing, right?
Snorkel42 00:22:16
So they had the, these Okta support engineers, or honestly,
Snorkel42 00:22:19
these, what did we say?
Snorkel42 00:22:20
A Sykes?
Snorkel42 00:22:21
I can't remember which company came first, but they, these contractors
Snorkel42 00:22:25
have the ability to reset the passwords of their customers, of
Snorkel42 00:22:30
every user within their customers.
Snorkel42 00:22:32
And if Octa is your identity provider, that is where your accounts live.
Snorkel42 00:22:35
They are the source of truth.
Snorkel42 00:22:37
Then may.
Snorkel42 00:22:40
But if Okta is just your single sign on solution, your SAML solution or
Snorkel42 00:22:43
something along those lines for active directory, do you, if you signed up for
Snorkel42 00:22:48
Okta as a customer, do you expect that there's some third-party company and
Snorkel42 00:22:51
Costa Rica that can reset your active directory passwords of your CEO right now?
Snorkel42 00:22:56
I wouldn't.
Snorkel42 00:22:56
And I would expect them to give me access my admin access back to my dashboard if I
Snorkel42 00:23:01
happen to walk myself out at that point.
Snorkel42 00:23:03
Yeah.
Snorkel42 00:23:04
But down to the individual user level, that certainly caught me off guard.
Snorkel42 00:23:08
I did not expect that.
Prasanna Malaiyandi:Because that should go through your normal it process,
Prasanna Malaiyandi:which is owned by your company and driving it through active directory.
Prasanna Malaiyandi:That way.
Prasanna Malaiyandi:way
Snorkel42 00:23:16
And it calls out another thing that Okta publishes a document
Snorkel42 00:23:20
of their subcontractors, who they use.
Snorkel42 00:23:23
and this company is on that list and our 24/7 customer support.
Snorkel42 00:23:27
And in the notes, it says something along the lines of, they have no data centers.
Snorkel42 00:23:34
They simply have access to our Salesforce and AWS, that's it.
Snorkel42 00:23:38
So if you are doing your due diligence as a customer and doing your vendor reviews
Snorkel42 00:23:42
you're gonna look at their subcontractors.
Snorkel42 00:23:43
You see this and like, all right, I don't care if they have access to Salesforce
Snorkel42 00:23:46
and how AWS that can mean anything.
Snorkel42 00:23:49
I don't think any reasonable human can read that and go, oh, this third-party
Snorkel42 00:23:53
contractor has the ability to reset my active directory passwords.
W. Curtis Preston:Yeah, good point.
W. Curtis Preston:And so I'll take back my comment.
W. Curtis Preston:I forgot about that part.
W. Curtis Preston:and there was a comment again from the former Okta employees or people
W. Curtis Preston:claiming to be former Okta employees.
W. Curtis Preston:And what they said was the practice and the policy is that you do
W. Curtis Preston:not use this power to do that, but that power is still there.
Prasanna Malaiyandi:that's probably the mistake, right?
Prasanna Malaiyandi:That you can't trust people to have the power and not use it.
W. Curtis Preston:you, what you would do, I could see edge
W. Curtis Preston:cases where maybe that's needed.
W. Curtis Preston:I don't, I can't imagine it right now, but let's just say those edge cases
W. Curtis Preston:to me, those would be edge cases and they would require additional MFA.
W. Curtis Preston:For example, if you're going to do the thing that, we don't think
W. Curtis Preston:should normally be done, then that should require a MFA or MPA.
W. Curtis Preston:If you're going to reset, a password that deep, then it should have
W. Curtis Preston:to come from multiple people.
Snorkel42 00:24:50
Yeah.
Snorkel42 00:24:51
I just going to say, if it's an edge case, it would be an escalation.
Snorkel42 00:24:54
right?
Snorkel42 00:24:55
There's one person on a floor.
W. Curtis Preston:But, and so yeah, you're right.
W. Curtis Preston:So in one sense, they did separate, but they didn't, it sounds like you're saying
W. Curtis Preston:they could do, they could have done the least privilege concept a little better.
Snorkel42 00:25:07
Yeah.
Snorkel42 00:25:07
I think more it's and it goes back to exactly what I was railing
Snorkel42 00:25:11
against at the very beginning.
Snorkel42 00:25:13
It's a communication issue that I'm struggling with Okta right
Snorkel42 00:25:16
now, as an Okta customer, I did not know they had that capability.
Snorkel42 00:25:21
And after doing due diligence and I happen to know, but sub-process
Snorkel42 00:25:24
or document, cause I happened to have it on my computer.
Snorkel42 00:25:27
When this happened, I went, wait a minute.
Snorkel42 00:25:29
Was that even disclosed?
Snorkel42 00:25:30
And I went and looked he's huh, no, not at all.
Snorkel42 00:25:33
And the word that really stuck out to me was the simply they simply have the
Snorkel42 00:25:39
ability to access Salesforce and AWS wow.
Snorkel42 00:25:42
so how does a company.
Snorkel42 00:25:43
Who's trying to do the right thing.
Snorkel42 00:25:45
He's trying to do their due diligence, trying to make sure that they're
Snorkel42 00:25:47
onboarding vendors, that aren't going to open them up for security woes.
Snorkel42 00:25:52
What do you do in that situation?
Snorkel42 00:25:54
When a company, as big as Okta is frankly, at least being awfully
Snorkel42 00:26:01
liberal with their definitions.
W. Curtis Preston:Yeah, that's that is, I would feel much better if they
W. Curtis Preston:notified of what happened in January.
W. Curtis Preston:And they said, listen, there was this thing happen.
W. Curtis Preston:I can understand why they might not want to, but I can see, this
W. Curtis Preston:is what, this is what happened.
W. Curtis Preston:We're not sure of the extent we're studying it, et cetera, whatever,
W. Curtis Preston:but just a simple notification.
W. Curtis Preston:And that would allow people to go, just do a quick.
W. Curtis Preston:did anybody change their passwords?
W. Curtis Preston:Did anybody lose her MFA?
W. Curtis Preston:W whatever.
W. Curtis Preston:just let me go as a user go.
W. Curtis Preston:Just do a quick check that, that everything seems fine, but if you're not
W. Curtis Preston:even told that you're not going to go do it, you're not going to go do a check.
Snorkel42 00:26:48
and honestly, it could have been a great success story for
Snorkel42 00:26:51
them, as I've said several times now I have nothing but praise for
Snorkel42 00:26:54
the security team that caught it.
Snorkel42 00:26:56
And if they would've came out in a reasonable time and said, Hey,
Snorkel42 00:26:59
heads up, here's what happened.
Snorkel42 00:27:01
Here's where we detected and stopped and the controls that
Snorkel42 00:27:03
we had in place to realize it.
Snorkel42 00:27:05
And here's what we're going to do to make sure that RDP is an accessible on,
Snorkel42 00:27:08
there's, third-party support, contractors, laptops and things along those lines.
Snorkel42 00:27:12
I think people would have been, wow.
Snorkel42 00:27:14
Yeah, that could have been real bad, but kudos to Okta, they got their fingers
Snorkel42 00:27:18
on the pulse and they know what's up instead this is a PR nightmare for them.
Snorkel42 00:27:23
To what end?
Snorkel42 00:27:23
I don't know.
Snorkel42 00:27:24
I don't know what they were gaining from trying to keep this quiet and then
Snorkel42 00:27:28
doing this frankly, pathetic attempt at wordsmithing, their official message.
Prasanna Malaiyandi:and I think that's the danger because a lot of
Prasanna Malaiyandi:people in the InfoSec community, right?
Prasanna Malaiyandi:You guys know the difference.
Prasanna Malaiyandi:Like you can smell, The lack of transparency, That something is
Prasanna Malaiyandi:fishy, something doesn't sound right.
Snorkel42 00:27:45
right.
Prasanna Malaiyandi:And you lose trust in them.
Snorkel42 00:27:48
Yeah.
Snorkel42 00:27:49
And again, for a company like Okta, that's not a company you can lose
Snorkel42 00:27:51
trust in and it's a shame too.
Snorkel42 00:27:53
It's a shame to see because it would've been such an easy message
Snorkel42 00:27:56
for them to deal with properly.
W. Curtis Preston:Yeah.
W. Curtis Preston:their stock price went from 166 to 145 in the last two days.
W. Curtis Preston:it could have been it's a 10% loss could have been, could have been better.
W. Curtis Preston:and I agree with you.
W. Curtis Preston:I think that companies that are Okta customers.
W. Curtis Preston:are going to reevaluate their, their trust that they have
W. Curtis Preston:placed in this huge company.
W. Curtis Preston:And Okta's, and the thing that really is the Octa's the default, right?
W. Curtis Preston:Okta's everywhere
Snorkel42 00:28:27
yeah.
W. Curtis Preston:and, yeah, that's just.
Prasanna Malaiyandi:you have to have a good reason not to pick Okta.
Prasanna Malaiyandi:Like you won't lose your job for recommending Okta and your company.
Prasanna Malaiyandi:gotten to that stage for Okta, or it was that way for Okta.
Snorkel42 00:28:41
It's to the point where if you are implementing a new product
Snorkel42 00:28:44
and you look up their documentation of how do I implement SAML chances
Snorkel42 00:28:47
are, it's going to say, oh, if you're an Okta customer, just click here
Snorkel42 00:28:49
and they'll have those screenshots.
Snorkel42 00:28:51
And then there's the everyone else.
Prasanna Malaiyandi:Yeah.
W. Curtis Preston:Okta screenshots means something very different now.
Snorkel42 00:28:57
Right?
W. Curtis Preston:Oh, that's tough.
W. Curtis Preston:so w I can, I assume now that the 2.5% of customers have been notified.
Snorkel42 00:29:11
this again goes back to if we take Okta's word for it.
Snorkel42 00:29:16
Yes.
Snorkel42 00:29:16
that, that was the other side of this, of the blog post, right?
Snorkel42 00:29:19
The comment of.
Snorkel42 00:29:20
after we, after doing a thorough analysis over the last 24 hours and scanning, I
Snorkel42 00:29:25
think it was like 125,000 log entries.
Snorkel42 00:29:27
that's a significant number as logs go.
Snorkel42 00:29:29
we now know that 2.5% of our customers were impacted, which again, you take
Snorkel42 00:29:33
just half a step back and you think about that, so for three months you did nothing.
Snorkel42 00:29:38
Then these screenshots came out and in 24 hours you looked at what do
Snorkel42 00:29:42
you think 125,000 logs are to Okta.
Snorkel42 00:29:46
An eighth of their log.
Snorkel42 00:29:47
Yeah.
Snorkel42 00:29:48
It's nothing.
Snorkel42 00:29:48
And it's such a disingenuous comment, too., you look at it like, oh,
Snorkel42 00:29:51
so you had like skilled security engineers looking at those logs.
Snorkel42 00:29:55
No, you ran some greps on them.
Snorkel42 00:29:57
you didn't actually look at them.
W. Curtis Preston:I was going to throw a grep.
W. Curtis Preston:I was going to say they, grepped some stuff.
Snorkel42 00:30:02
like I tell you this, I'm sure the, their CTO isn't the guy
Snorkel42 00:30:06
actually or CSO, I forget what his title is, is actually writing these things.
Snorkel42 00:30:09
I'm sure it's going through all kinds of legal marketing, but man, every
Snorkel42 00:30:13
time he posts it's get the popcorn out.
Snorkel42 00:30:14
Cause he's just keeps making it worse.
W. Curtis Preston:I don't know any advice for Okta customers?
W. Curtis Preston:What is it?
Snorkel42 00:30:18
with regards to this breach, I truly don't think it's
Snorkel42 00:30:22
significant in terms of customer impact.
Snorkel42 00:30:24
always do your IR.
Snorkel42 00:30:25
Always do your due diligence.
Snorkel42 00:30:26
Go look at your logs from that timeframe.
Snorkel42 00:30:28
See if there were any strange resets.
Snorkel42 00:30:31
but I, that breach is not keeping me up at night.
Snorkel42 00:30:35
What my advice is.
Snorkel42 00:30:37
And certainly what I've been doing is contact your Okta reps and make
Snorkel42 00:30:40
a stink and try to drive the message that I'm not concerned about.
Snorkel42 00:30:45
This breach.
Snorkel42 00:30:46
I am concerned about how you handled it and the fact that you waited
Snorkel42 00:30:51
three months and for the attackers to give up the evidence for you to say
Snorkel42 00:30:54
something makes me have to question.
Snorkel42 00:30:57
What else are you sitting on?
Snorkel42 00:30:58
What other security incidents have occurred that you have
Snorkel42 00:31:01
conveniently not reported?
Snorkel42 00:31:04
And I'm not saying there aren't any, I don't know, but that's the problem that I
Snorkel42 00:31:08
don't, I no longer trust them to tell me.
Snorkel42 00:31:11
and I think that's what Okta needs to hear and, be nice to your rep, right?
Snorkel42 00:31:16
Like they're caught up in those too, but make sure you're saying, Hey,
Snorkel42 00:31:18
escalate this to your executive level.
Snorkel42 00:31:20
Cause this is just not an acceptable way for a company like Okta to be off.
Snorkel42 00:31:23
Yeah.
Snorkel42 00:31:23
that's my main comment to all Okta customers is now's the time to take
Snorkel42 00:31:28
off the gloves and raise some stink.
W. Curtis Preston:send a WTF.
Snorkel42 00:31:32
Yeah.
W. Curtis Preston:Your Octa rep for the record.
W. Curtis Preston:It's two months, not three months just throwing that out there.
Snorkel42 00:31:39
End of January.
Snorkel42 00:31:40
It's end of March.
Snorkel42 00:31:41
Okay.
Snorkel42 00:31:42
Fair enough.
W. Curtis Preston:Once I got the math, right.
Prasanna Malaiyandi:I know for once Curtis.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:Curtis math, they're not
W. Curtis Preston:always off by an order of magnitude usually or something I'm
W. Curtis Preston:like, I think that was 10% and was like,
Snorkel42 00:31:55
There's 1%.
Snorkel42 00:31:56
2, 3.
Snorkel42 00:31:57
That's how it works.
W. Curtis Preston:Exactly.
Snorkel42 00:32:02
Fair enough.
Snorkel42 00:32:02
Yeah,
W. Curtis Preston:yeah.
W. Curtis Preston:thanks for coming on to talk about the, the Okta situation.
Snorkel42 00:32:09
it was a pleasure.
Snorkel42 00:32:11
I don't know if that's the right word.
W. Curtis Preston:is never fun is it Prasanna?,
Prasanna Malaiyandi:No, it's not.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:I hope at least you had a chance to vent snorkel.
Snorkel42 00:32:21
Yeah, it was cathartic.
Snorkel42 00:32:22
Yes.
Snorkel42 00:32:23
Thank you.
W. Curtis Preston:Yeah.
W. Curtis Preston:Yeah, this is not a, one of my favorite words is the German word
W. Curtis Preston:schaudenfreude, which means taking joy in the misfortunes of others.
W. Curtis Preston:this is not that, this is anger, right?
W. Curtis Preston:This is.
W. Curtis Preston:I agree with you.
W. Curtis Preston:Like how is this, it's the whole like, oh, now you're telling us
W. Curtis Preston:after the screenshots came out and would we ever even heard of anything?
Snorkel42 00:32:47
They should have saw it coming, This is what this group does.
Snorkel42 00:32:49
They're an extortion group.
Snorkel42 00:32:50
They stole the screenshots and I would be, I'd be willing to bet money that
Snorkel42 00:32:55
they were holding that up to Okta, send us money, or we're going to disclose.
Snorkel42 00:32:59
I could not have been surprised when those screenshots finally appeared
Snorkel42 00:33:02
the whole thing.
Snorkel42 00:33:03
Just You have to wonder what they were thinking.
Prasanna Malaiyandi:It's, it sounds a little staged if you will, or
Prasanna Malaiyandi:planned, They knew it was coming.
Prasanna Malaiyandi:They weren't paying up or whatever, and
Snorkel42 00:33:13
Which you would think that they would take the other approach and
Snorkel42 00:33:17
let's be the ones to control that message.
Snorkel42 00:33:18
Then let's be the ones to disclose that this happened.
Snorkel42 00:33:20
Cause it really, if we take them at their word, it wasn't that big of a deal.
Snorkel42 00:33:27
They could have controlled that message instead.
Snorkel42 00:33:28
it's a big deal.
W. Curtis Preston:it's not the crime.
W. Curtis Preston:It's the cover up.
Prasanna Malaiyandi:Yeah.
Snorkel42 00:33:33
No.
W. Curtis Preston:same old.
W. Curtis Preston:All right.
W. Curtis Preston:thanks to our listeners and, be sure to subscribe so that you can restore it all.
Listen On
