The Role of Replication in Modern Data Protection systems
In this episode, W. Curtis Preston, aka Mr. Backup, and Prasanna Malaiyandi discuss the fundamental technology of replication in data protection systems. They explore what replication is, how it differs from other methods, and why it's not used for everything. They also delve into the differences between synchronous and asynchronous replication and why it matters. The hosts also share news of a backup company called Alcion, which recently raised funding with support from Veeam, a backup company investing in Alcion's focus on Microsoft 365 backups for SMB customers. They also discuss a report from ESG about how cloud backup has evolved. The episode provides insights into the world of data protection and highlights the importance of replication in safeguarding valuable data.
Articles discussed in this episode:
Speaker:
This week, we're talking about one of the most fundamental technologies
Speaker:
underpinning today's data protection systems and that's replication.
Speaker:
What is it?
Speaker:
How is it different than other methods?
Speaker:
And if it's so great, why don't we just use it for everything?
Speaker:
And of course what's the difference between synchronous and asynchronous.
Speaker:
And why does that matter?
Speaker:
Hi, I'm W.
Speaker:
Curtis Preston AKA Mr.
Speaker:
Backup.
Speaker:
And I've dedicated my 30 year career to helping people like you keep your data
Speaker:
safe from all that would do it harm.
Speaker:
This is the backup wrap-up.
Speaker:
Welcome to the show with me, as always, as a guy who seems to never
Speaker:
be happy until I have problems.
Speaker:
Prasanna Malayandi.
Speaker:
How's
Speaker:
I'm good.
Speaker:
I'm just trying to think what problems do you have right now?
Speaker:
Talking about, I'm trying to edit the podcast and you're giving
Speaker:
me problems about my network.
Speaker:
That's,
Speaker:
what I'm talking about.
Speaker:
yeah, it's.
Speaker:
What I do, you need someone to ask the hard questions,
Speaker:
Somebody's got to do it.
Speaker:
The network's going all right.
Speaker:
Those that have followed, I do have this new tool.
Speaker:
I have, um, the, uh, Firewalla.
Speaker:
version 125 of Curtis's home network, architecture.
Speaker:
Yeah.
Speaker:
but I now have this Firewalla that gives me a lot more
Speaker:
information than I had before.
Speaker:
And unfortunately it verified, my new ISPs, thing that I use too
Speaker:
much internet and I have to buy.
Speaker:
the unlimited package, which is sad because it's another 50 bucks a month.
Speaker:
But anyway, so let's get to the news of the week
Speaker:
I'm going to let you take the first story, uh,
Speaker:
Yeah, yeah, yeah, so the first one is an article that came out recently.
Speaker:
it is about a backup compa I guess, would you call them backup or data
Speaker:
protection company that recently, Raised around a funding, it's called Alcion.
Speaker:
I actually used to work with the founder previously at an old job.
Speaker:
and they actually raised a round of funding and it was actually led by
Speaker:
Veeam, which is interesting because Veeam, as we all know, is also a backup
Speaker:
company and they are investing in Alcion, who is primarily targeting backups
Speaker:
for Microsoft 365 for SMB customers.
Speaker:
today?
Speaker:
That's their initial target market.
Speaker:
I'd say that the biggest difference between the two companies,
Speaker:
one is a software product, the other is a SaaS offering, and
Speaker:
that makes them very different.
Speaker:
And I guess Veeam felt that it was different enough.
Speaker:
I like the quote in the article.
Speaker:
It said something along the lines of that, that from Veeam, that
Speaker:
this problem of ransomware is a big enough problem that we're going to
Speaker:
need the whole industry to solve it.
Speaker:
And so they invested in what.
Speaker:
is essentially, I would call it a partial competitor, right?
Speaker:
They do compete for the Microsoft 365 space, I also know the founder as
Speaker:
well, and, I think they've taken an interesting angle with a very AI based
Speaker:
approach, a very security based approach,
Speaker:
Yeah.
Speaker:
And one of the things that I'm actually looking forward to is seeing if this
Speaker:
is going to force other vendors to reconsider how they approach the problem.
Speaker:
And.
Speaker:
Everyone in all the vendors raise a bar when it comes to ransomware
Speaker:
protection and detection and other things like that, which we all
Speaker:
need.
Speaker:
Right?
Speaker:
Anybody that's listened to this show knows that we're concerned about that problem.
Speaker:
my second news story, it's from the enterprise strategy group, which for
Speaker:
those of you that have known ESG for a while, they were, A few years ago,
Speaker:
acquired by TechTarget, and there was a, 2023 survey of IT professionals.
Speaker:
The most interesting number that I see here is that they said 65 percent
Speaker:
of the almost 400 respondents said their organization were using some
Speaker:
sort of cloud based, backup system.
Speaker:
And the way they worded it, that could be either a cloud based
Speaker:
service or they're storing a copy of their backups in the cloud.
Speaker:
That, to me, really, that's a very different number, wouldn't you say, than
Speaker:
a few years
Speaker:
Yeah, everyone would be like, I don't trust my data in the cloud.
Speaker:
How do I get my data up into the cloud and bring it back and all the rest of that?
Speaker:
In fact, I think one of the interesting things was even just
Speaker:
the title of the article, right?
Speaker:
It's like cloud backup and disaster recovery evolved toward maturity.
Speaker:
You know, that just kind of says it all, right, versus before you're
Speaker:
fighting tooth and nail with customers, trying to persuade them, Hey, it's
Speaker:
okay to store your data in the cloud.
Speaker:
But
Speaker:
Yeah, so the not so good number was saying that half of the surveyed orgs
Speaker:
recovered 75 percent or less of their cloud data per incident on average.
Speaker:
That sounds really bad.
Speaker:
that is not, I agree with, Christoph Bertrand, which said that's just
Speaker:
not acceptable for, such a mission
Speaker:
I guess the question I would have is, it would be, we probably need
Speaker:
to compare that to what's your recovery rate of on premises backups,
Speaker:
because that's not going to be 100%.
Speaker:
and then I think it's also not fair to compare it because how
Speaker:
long has traditional backups, been around and had time to evolve
Speaker:
versus cloud backups, right?
Speaker:
It hasn't had that time to evolve and mature.
Speaker:
Fully as these other products
Speaker:
Yeah, I would say that part, having formerly worked at a cloud data protection
Speaker:
vendor, I would think that the problem that I see with this number is that they
Speaker:
do conflate people storing backups in the cloud versus people using a cloud service.
Speaker:
I would be very surprised to learn that.
Speaker:
People that are using a fully cloud based service, if they, if their recovery's
Speaker:
failed, because the other, it's really, the cloud just happens to be a part of it.
Speaker:
All of the same sort of misconfiguration problems that lead to problems
Speaker:
with traditional backup are going to bleed over into a cloud.
Speaker:
Just because you're using cloud
Speaker:
but it's the cloud.
Speaker:
It's magical.
Speaker:
Come on Curtis.
Speaker:
Yeah
Speaker:
Yeah, it is magic.
Speaker:
the one other interesting number from this survey, and there were
Speaker:
some other interesting numbers, but just for brevity purposes, this other
Speaker:
interesting number, they said 16 percent of the, respondents said that they.
Speaker:
reviewed their, their cloud data protection strategies annually.
Speaker:
that is, that's a very different number than back in the day.
Speaker:
Back in the day, that number was like five years, maybe three years.
Speaker:
Certainly not annually.
Speaker:
Backup historically has been an incredibly sticky process.
Speaker:
And, this idea that they would review it annually, I think it's a
Speaker:
I think it also makes sense though, because you like with the five
Speaker:
year, three year, five year, right?
Speaker:
You have to worry about hardware depreciation, right?
Speaker:
Upgrade cycles all the rest With cloud, right?
Speaker:
There might be new technologies, cheaper ways to do things, right?
Speaker:
Your workloads might change.
Speaker:
what you had running on premises might be migrated to the cloud.
Speaker:
now you have to reconsider it.
Speaker:
How do I back it up?
Speaker:
And so I think that's probably why looking at these things more
Speaker:
frequently, like annually, probably starting to make a lot more sense.
Speaker:
Well, that is the news of the day.
Speaker:
on this episode of the Backup to basic series, we're going to be talking
Speaker:
about replication, different kinds of replication, the good things, the bad
Speaker:
things, and what they, you know, what they have to do with backup and, and
Speaker:
I, I,
Speaker:
it a backup,
Speaker:
I, I, I I thought you were going to talk about like the replication
Speaker:
technologies that they have in like Star Trek and, Right?
Speaker:
Where it's like, create me,
Speaker:
No, that's, that's not replication.
Speaker:
That's...
Speaker:
What is that called?
Speaker:
what are those things?
Speaker:
Transporters?
Speaker:
Oh, oh, no.
Speaker:
the making, the stuff, right?
Speaker:
The food
Speaker:
Do they call them replicators?
Speaker:
Yeah.
Speaker:
I think they call them replicators.
Speaker:
We are, we are not discussing replicators like in Star Trek, although that does,
Speaker:
those do sound magical, by the way.
Speaker:
And by the way, like if you could do transporters, of course you
Speaker:
could do replicators, right?
Speaker:
Because all replicators are is storing a backup copy of the food and then
Speaker:
saying this is what it is, right?
Speaker:
Do you think that they have backups of those backups?
Speaker:
The copies?
Speaker:
Yeah, no, the copies of the food because imagine you wipe out the
Speaker:
copies and then you've basically killed off humanity because they
Speaker:
wouldn't have any way to make food.
Speaker:
I
Speaker:
Or maybe they're immutable, or maybe they're immutable copies, you know?
Speaker:
Maybe they're immutable copies.
Speaker:
I would, I would hope that no one can go in and change the
Speaker:
food before it gets replicated.
Speaker:
Um,
Speaker:
On to a more serious matter.
Speaker:
let's do a more serious matter.
Speaker:
Well, so we, we've moved on the previous episodes.
Speaker:
We talked about sort of regular kind of backups and they were, they were backups.
Speaker:
That require a restore basically is, is what we were talking about.
Speaker:
We've now moved on to types of backups where the restore
Speaker:
is essentially already done.
Speaker:
Right.
Speaker:
Um, and replication.
Speaker:
And so I'm, I'm, I'm putting these and, and by the way, you know, for those that
Speaker:
don't know what I'm talking about, we are basically working our way through.
Speaker:
This book, Modern Data Protection, which is my book, which is available,
Speaker:
you know, wherever books are sold.
Speaker:
And, um, you know, I might even put a, put a link in the show description there.
Speaker:
Um, and the idea here is what we're talking about is methods of backup.
Speaker:
And again, remember when I say backup, I I'm very broad in that term, methods
Speaker:
of backup that support instant recovery.
Speaker:
What do you want to talk about?
Speaker:
What instant recovery is?
Speaker:
Yeah, so, normally when you think about, you took a backup, you
Speaker:
need to, something happens, you need to get back your data, right?
Speaker:
If you had your data stored on tape, you gotta go fetch the tape, you gotta pull
Speaker:
the data off of tape, copy it somewhere else, and now you actually have your data
Speaker:
and you can do something with it, right?
Speaker:
With Instant Recovery, It's really, your data is already available for you.
Speaker:
You don't have to have any sort of processing that you have to do before you
Speaker:
can actually start accessing your data.
Speaker:
Yeah, I mean, it's a beautiful thing.
Speaker:
I think that maybe, I dream that sometime in the future, this is
Speaker:
the way all restores will be done.
Speaker:
And the cloud really does make this possible.
Speaker:
We could talk about that all day long, but we just need to work through
Speaker:
the different ways that, um, we're going to create an always on readily
Speaker:
available copy of the data, and the first that we're going to talk about is
Speaker:
Before you get there though, I think one important point to cover is, I
Speaker:
know you said in the future maybe everything will be like this, right?
Speaker:
I think if you had a gazillion dollars, yes, Because there
Speaker:
is a cost associated, right?
Speaker:
Which I think is important to cover, right?
Speaker:
With sort of using these various technologies that might make it more
Speaker:
difficult now Like you said, maybe the cloud helps with some of these but there
Speaker:
is still a cost associated with Keeping data on something that is instantly
Speaker:
available Versus say using a tape or some sort of offline Media where the
Speaker:
cost may not be just the cost of the storage but think about from power cooling
Speaker:
Right all those other aspects as well
Speaker:
Yeah, I was speaking in the future when these problems have all been solved.
Speaker:
Um, I think even today that there are services, there are cloud storage
Speaker:
services, that if used properly and judiciously, can provide you A ready
Speaker:
to go copy of your data pennies on the dollar that costs you, you know,
Speaker:
next to nothing to, to have and to possess and to have it ready to go.
Speaker:
You will then pay through the nose if you use them, right?
Speaker:
I'm thinking of course of like Glacier, Deep Archive, Instant Recovery.
Speaker:
Right?
Speaker:
There are services that are still storing the data on disk, but storing
Speaker:
it in such a way, I don't know, we don't, you know, neither of us have
Speaker:
ever worked at AWS or any of the cloud providers, but they're storing it in
Speaker:
such a way that they've made it so cheap.
Speaker:
To, to, to have that copy, to have it offsite, to have it in,
Speaker:
you know, in another region.
Speaker:
I just think that as that continues to happen, that we will eventually
Speaker:
get to a place where all DR copies can be held like this.
Speaker:
Can I have a dream, Prasanna?
Speaker:
Can I have a dream?
Speaker:
Can
Speaker:
You can have a dream, but I think, but yes, but before, sorry, before
Speaker:
we also continue on, I think we also need to define what we
Speaker:
mean when we say instant, right?
Speaker:
Because instant,
Speaker:
when we say instant recovery, right?
Speaker:
I think it's helpful to talk about that because instant is a spectrum, right?
Speaker:
Like you were talking about for your Glacier instant restore feature, right?
Speaker:
It's not like, Oh, I want my data.
Speaker:
It's available within the Cloud.
Speaker:
Like, snap my fingers, it's there.
Speaker:
Right?
Speaker:
It does take time, but that time may be acceptable to you, right?
Speaker:
It might be, say, minutes, not days.
Speaker:
yeah.
Speaker:
So instant is kind of like continuous.
Speaker:
It's a binary condition.
Speaker:
It's sort of like immutable.
Speaker:
It's a binary condition.
Speaker:
It's like pregnancy, right?
Speaker:
You're either pregnant or you're not.
Speaker:
And so it's either instance or it's not.
Speaker:
But it's yet another one of these terms where it's not truly binary.
Speaker:
Right?
Speaker:
It's just, we're defining instant recovery in this point is, I don't need
Speaker:
to do a restore to do the recovery.
Speaker:
The data's already been restored.
Speaker:
It might take a few minutes to make it accessible to me.
Speaker:
Yeah, sorry, and, and just for people who may not have heard our previous
Speaker:
episodes, do you want to quickly cover differences between restore and
Speaker:
recovery since you just talked about it?
Speaker:
A restore is essentially the act of bringing back the, you know,
Speaker:
the data and putting it in place.
Speaker:
The recovery is the process of sort of bringing that data online and using it.
Speaker:
And, and sometimes that recovery involves massaging other things.
Speaker:
It involves massaging the data and it involves doing other things.
Speaker:
When you're done with the recovery.
Speaker:
Everything is working again.
Speaker:
When you're done with the restore, everything is ready to be working again.
Speaker:
Uh, so depending on how good or how complicated the restore is,
Speaker:
the two may be, they, they may happen at the same time, but in a
Speaker:
complicated situation, you do the restore and then you do the recovery.
Speaker:
this case, uh, the recovery would be this instant of, um,
Speaker:
you know, like I said, just,
Speaker:
Everything's ready to go.
Speaker:
Or sorry, everything.
Speaker:
has, is available.
Speaker:
Yeah, exactly.
Speaker:
Uh, we don't have to do a restore.
Speaker:
And I would say that in a true instant recovery, the size of the, of the thing
Speaker:
being recovered should not be a factor.
Speaker:
Whereas in a restore, it very much is,
Speaker:
Oh yeah.
Speaker:
right?
Speaker:
Yeah.
Speaker:
Yeah.
Speaker:
Okay.
Speaker:
So with that in mind, we're going to talk about the first method that could be used.
Speaker:
And, and is used actually, it is used quite a bit to do an instant recovery.
Speaker:
Uh, as we often do on the show, let me take you back in the day.
Speaker:
Yeah.
Speaker:
So back in the day, there was a very sharp dividing line between.
Speaker:
Backup and recovery and disaster recovery.
Speaker:
And for backup and recovery for what we also call operational
Speaker:
recovery, you had a backup system.
Speaker:
You went to tape and, and when you did a restore, you, you came back
Speaker:
up, you came back off of tape.
Speaker:
Anyone with a tight recovery time objective would never build,
Speaker:
uh, their DR system based on tape back in the day, right?
Speaker:
Even as we move to disk based backup and recovery systems, anybody with a
Speaker:
tight RTO is not going to build their disaster recovery system built on
Speaker:
having to do a restore first, right?
Speaker:
So back in the day, if you worked at like a financial trading firm or
Speaker:
something like that, and you wanted to have a disaster recovery plan because of
Speaker:
course you wanted that because you had the OCC that was looking into you and
Speaker:
Regulation,
Speaker:
could say we're not a bank, right?
Speaker:
You used replication.
Speaker:
You had a backup and recovery system that did operational recoveries
Speaker:
and file recoveries, and maybe even recovering an entire server.
Speaker:
But if you had data that truly mattered, you used replication.
Speaker:
And, and I know you worked at a company
Speaker:
Yeah, that, was my first job out of, out of
Speaker:
college.
Speaker:
Yeah, So, let's talk about what replication is, uh, how does
Speaker:
it typically manifest itself.
Speaker:
Do you want to just...
Speaker:
Sort of define what replication is?
Speaker:
so replication is basically taking data, making a copy, and sending
Speaker:
it off to some other system, right?
Speaker:
At a high level.
Speaker:
Now, there's various nuances and combinations that other system
Speaker:
could be the exact same vendor, the same model, all the rest.
Speaker:
As the original, it could be a different vendor, right?
Speaker:
A different system.
Speaker:
Right?
Speaker:
So there are different ways you could do it.
Speaker:
There are different protocols for transferring that data.
Speaker:
You could use protocols that some of these vendors have built in
Speaker:
for communicating between them to make things more efficient.
Speaker:
Because when you have to send data across from one system to another,
Speaker:
doing it over and over, because it's not like it's a one time operation, right?
Speaker:
Depending,
Speaker:
Yeah.
Speaker:
If I could interrupt, I think that's one word missing from your definition.
Speaker:
If I could insert at the beginning of your definition, the word continuously.
Speaker:
Right.
Speaker:
Because anybody can copy the data over.
Speaker:
I think replication implies that we're continuously copying it
Speaker:
over.
Speaker:
The idea is that it, that it's happening all the time, right?
Speaker:
We're going to talk about the difference between asynchronous and synchronous.
Speaker:
So sorry for the confusion for those, but basically you have some data, you
Speaker:
know, in A and you, and you have a copy of that data in B and that copy,
Speaker:
that, that B copy is continuously, for lack of a better word, updated to
Speaker:
be the same exact thing as what's in
Speaker:
Yeah, so going to Curtis's disaster recovery scenario, right?
Speaker:
Something happens on system A, right?
Speaker:
System B has a version of what system A looked like at a point in time,
Speaker:
and you can fail over to system B, and that's how you're instantly
Speaker:
up and running, because system B
Speaker:
I think one thing we should also cover, which I want to talk about
Speaker:
earlier, I know you threw out the term recovery time objective.
Speaker:
I think another point to talk about here is also recovery point objective, right?
Speaker:
Yeah.
Speaker:
Well, go ahead.
Speaker:
You brought it up.
Speaker:
So yes, in the case of when you're doing synchronous replication, yes, there's
Speaker:
still going to be an RTO on that.
Speaker:
But that RTO, you should be able to measure in seconds to minutes, depending
Speaker:
on how long it takes you to bring up system B and make it available for
Speaker:
your applications and other things.
Speaker:
So that's kind of
Speaker:
like RTO side.
Speaker:
Do you wanna talk about the RPO, Curtis?
Speaker:
Yeah.
Speaker:
So the RPO is, is basically the amount of data that you
Speaker:
agree that you can lose, right?
Speaker:
The recovery point objective is we say we can lose an hour's worth of data.
Speaker:
We can lose a day's worth of data.
Speaker:
We can only lose five minutes worth of data.
Speaker:
This is.
Speaker:
You're setting your recovery point objective.
Speaker:
What, to what point must you recover?
Speaker:
And the fastest RPOs in the business are going to come from these types
Speaker:
of products that we're going to talk about in the next couple of episodes.
Speaker:
The first of which being.
Speaker:
Uh, replication with a, with a, you know, a replication
Speaker:
system, you could get an RPO.
Speaker:
Uh, you could meet an RPO that's as close to zero as possible,
Speaker:
actually seen zero,
Speaker:
Yeah, I'm sure you can do zero.
Speaker:
It's just that
Speaker:
so
Speaker:
I'm gonna
Speaker:
could happen,
Speaker:
So it's in a very, very unique case.
Speaker:
So I don't know if you're aware of this, there is a mode on
Speaker:
some systems called domino mode.
Speaker:
It's out there in the industry, right?
Speaker:
And basically what you're saying is, every time I accept a write
Speaker:
on system, A, I'm gonna make sure it's replicated, it hit system B.
Speaker:
It's acknowledged before system A acknowledges a client.
Speaker:
Yeah,
Speaker:
you're guaranteeing every
Speaker:
basically, it's like a two phase commit for storage.
Speaker:
A two phase commit is a term from database, um, world, which makes
Speaker:
sure that it's written both to the log and it's written to the, you
Speaker:
know, before you acknowledge that the, the transaction has happened.
Speaker:
It's High
Speaker:
latency, very expensive, but if you have those requirements, there is an option.
Speaker:
But like you said, most
Speaker:
people will...
Speaker:
So the question is what matters more to you, performance or never losing any data?
Speaker:
So let's, speaking of which.
Speaker:
Let's talk about the difference between synchronous replication
Speaker:
and asynchronous replication.
Speaker:
Um, synchronous is basically what you just described that, um, well,
Speaker:
it's not necessarily what you
Speaker:
Yeah, not fully to domino
Speaker:
we are going to, we are going to replicate every byte.
Speaker:
As it's happening, right.
Speaker:
And literally every single, you know, the moment it, a change happens here, the rep,
Speaker:
the change is going to be replicated to the, to the destination, to the target.
Speaker:
Right.
Speaker:
And, um, the, the good side of synchronous replication is you're going to have.
Speaker:
The smallest RPO is possible, right?
Speaker:
Including, I guess I, I, I, I guess I was aware of it.
Speaker:
I didn't know that that, that I didn't know the term dominant mode, but,
Speaker:
but I was aware of that, that idea, the, this is going to give you your,
Speaker:
your tightest, uh, RPO as possible.
Speaker:
The challenge with, uh, this is a little thing called the speed of light.
Speaker:
The, um, it takes, so, you know, one of the things that we.
Speaker:
That we know is that we want the recovery copy to be some
Speaker:
distance away from the primary
Speaker:
in the same rack.
Speaker:
not in the same rack, not in the same town.
Speaker:
Um, there are sadly some stories from 9 11 where there were companies
Speaker:
that had their hot site, which was a synchronously replicated copy of
Speaker:
their data in the, uh, other tower.
Speaker:
And so there were companies that ceased to exist on 9 11 because they lost both their
Speaker:
primary and their secondary copy, right?
Speaker:
You don't want to have the two copies in the same place where a disaster
Speaker:
could wipe out the entire town, a flood, a earthquake, whatever.
Speaker:
And so you want to have some distance between.
Speaker:
uh, the two.
Speaker:
Now after 9 11, the, I don't know which, um, which governmental organization,
Speaker:
but after 9 11, the, the fed, the federal government, not the fed, which
Speaker:
we say the fed or you know what I mean?
Speaker:
The federal government started to dictate that A recovery that, that a, that a
Speaker:
synchronous copy of the data needed to be stored more than it was like 250 miles.
Speaker:
And, and everybody basically came back and said, that's not really feasible.
Speaker:
It's possible, but not feasible because going 250 miles.
Speaker:
At the speed of light, every single time you replicate, every single time you
Speaker:
update a block on every piece of storage, um, basically really pulls down the, um,
Speaker:
the performance of the primary system.
Speaker:
And the other thing to also mention is when you're doing synchronous replication,
Speaker:
you want to make sure that you're...
Speaker:
Destination system, right?
Speaker:
Your replica copy is as performant as your primary copy, right?
Speaker:
Because otherwise, if it starts to lag and slow down, eventually you're going to
Speaker:
start falling further and further behind.
Speaker:
And usually these technologies, after a while, they'll just stop accepting writes.
Speaker:
And they'll kind of fall back into like an asynchronous mode and
Speaker:
have to then catch back up, right?
Speaker:
Which you don't want because that affects your RPO.
Speaker:
Yeah, you don't want.
Speaker:
Yeah.
Speaker:
So yeah, so it's got to be basically the same, if not better
Speaker:
performance on the backup system, which is crazy talk to most people.
Speaker:
Right?
Speaker:
Um, but that, but that is if you want an RPO of zero.
Speaker:
Synchronous replication is literally your only choice, right?
Speaker:
Let's talk about asynchronous replication and that is basically
Speaker:
not that we're, we're not, we're going to build essentially a buffer.
Speaker:
Uh, of writes as the, know, and we try to not let the buffer get full, right?
Speaker:
The, basically the, what it does is it disconnects the, the writes on the
Speaker:
other side with the writes on this side, we may, and we'll try to keep up
Speaker:
with the pace, but writes can continue to update and you might get a spur of
Speaker:
writes, uh, on the, on the primary side.
Speaker:
And then they fill up the buffer and then, you know, the, the buffer gets emptied.
Speaker:
And that I think works as long as essentially your, you know, your bandwidth
Speaker:
and your latency is, is sufficient.
Speaker:
Um,
Speaker:
And,
Speaker:
go ahead.
Speaker:
and I just want to chime in that I know you called it async and async
Speaker:
threw me for a while because in my mind async is different, but that's
Speaker:
just based on my background, right?
Speaker:
Um, other vendors might refer to this as like semi synchronous, right?
Speaker:
Where it is not fully synchronous.
Speaker:
Right, but
Speaker:
Well, not synchronous.
Speaker:
Asynchronous means
Speaker:
know, I know, I know, but this is just some vendors.
Speaker:
So if you are looking at vendor terminology, right, and looking at
Speaker:
their vendor technical specs, what, so here's a question for you, Curtis.
Speaker:
What are some things that a user can look at for a vendor's technical
Speaker:
specs to determine if they support async replication, like what
Speaker:
we're talking about right now?
Speaker:
Well, I mean, I don't know what other words they would use to describe
Speaker:
it,
Speaker:
I know NetApp uses semi synchronous.
Speaker:
For what you're describing.
Speaker:
whatever.
Speaker:
That's just mark.
Speaker:
That's just marketing BS.
Speaker:
Okay.
Speaker:
It's either synchronous or it's not synchronous.
Speaker:
You're either replicating everything as it's happening or you're not.
Speaker:
Okay.
Speaker:
Like, so the question is, so the question to the vendor, it's not, is, can my target
Speaker:
copy ever get behind my primary copy?
Speaker:
And the answer is yes.
Speaker:
And it's like, well, well, how much are we talking?
Speaker:
Right?
Speaker:
So the question is, because that's going
Speaker:
the, so that's
Speaker:
actually the key, yeah.
Speaker:
Yeah, exactly.
Speaker:
So if you can build up a buffer of an hour of changes, you could lose an
Speaker:
hour of changes on the, on the target end if you lost your primary, Um,
Speaker:
and.
Speaker:
what if you said like, I want, so, okay, so then I'll take back my comment.
Speaker:
In your mind though, if I said I want to have a replica copy
Speaker:
updated and my RPO is 8 hours.
Speaker:
That's still async.
Speaker:
Well, what else would you
Speaker:
Okay.
Speaker:
No, just, just checking.
Speaker:
Yeah.
Speaker:
I mean, again, you're either synchronous or you're not
Speaker:
So then, so then I take back my comments, right?
Speaker:
So if I look specifically, that's why I just wanted to make sure.
Speaker:
You stand corrected,
Speaker:
Yes.
Speaker:
So if I look at, sorry, I used to, I'm more familiar with NetApp and NetApp's
Speaker:
replication technologies, right?
Speaker:
But if I look at NetApp, right, they have synchronous, which we talked
Speaker:
about, which matches perfectly with what we were talking about.
Speaker:
They have semi sync where it allows some amount of lag.
Speaker:
I think it's up to 60 seconds.
Speaker:
Right.
Speaker:
But it does like what you said, buffering, and then they have
Speaker:
sort of their async, right?
Speaker:
Where it allows you to go beyond, I think it's from five minutes to, you can set a
Speaker:
replication every five minutes to days,
Speaker:
And those are just two different, those are just both asynchronous with different
Speaker:
settings, as far as I'm concerned.
Speaker:
Um, and, and you could also, like, if you were a net app and I don't think they do
Speaker:
it like this, but if you waited until you took a snapshot and then replicated only
Speaker:
the bits necessary for that snapshot, that's still kind of asynchronous,
Speaker:
just a technology.
Speaker:
That's a means.
Speaker:
yeah.
Speaker:
But generally here, we're not talking about snapshots and things, because
Speaker:
that's what we're going to talk about.
Speaker:
In a following episode, generally what we're talking about is two,
Speaker:
you know, block devices that are being continuously updated to be
Speaker:
exactly the same with possibly a lag.
Speaker:
If there's a lag, then it's asynchronous.
Speaker:
If there's no lag, then it's
Speaker:
Does it have to be, so I'm going to ask another question.
Speaker:
So
Speaker:
you are
Speaker:
I know I am.
Speaker:
So you said block devices.
Speaker:
Would you consider anything that also replicates a VM at a VM block level
Speaker:
Yeah.
Speaker:
Yeah.
Speaker:
I mean, in the end, it's still a block device underneath, right?
Speaker:
It's not what we're replicating.
Speaker:
It's just that, um.
Speaker:
We don't, we don't have this conversation when we're talking about object,
Speaker:
Or files.
Speaker:
We don't, we don't, well, even, even file replication, we don't really talk
Speaker:
about synchronous and asynchronous when we do file replication, I don't think.
Speaker:
I think that's always asynchronous.
Speaker:
At least I don't think so.
Speaker:
I think this is generally a block discussion, right?
Speaker:
There may be exceptions to that rule, but generally this is a block discussion.
Speaker:
Even if this is a NetApp, right?
Speaker:
Really what you're replicating underneath the NetApp is file.
Speaker:
It's a file device, but underneath you're replicating blocks.
Speaker:
Um, all that, all that discussion just from one word.
Speaker:
Um, so, you can also have a hybrid of these two things.
Speaker:
Can you think of an example of that?
Speaker:
Is
Speaker:
Yeah.
Speaker:
So take your bank example, right?
Speaker:
So you're a financial customer in New York.
Speaker:
You want to make sure that your data is synchronously
Speaker:
protected to someplace nearby.
Speaker:
So maybe you go over the river to New Jersey using synchronous
Speaker:
replication, and now you need to make sure that you're protected in case
Speaker:
something happens in the East coast.
Speaker:
And so you might use async replication and send that copy off to say, Utah.
Speaker:
that when we go through the woods?
Speaker:
Yes.
Speaker:
See what I did there?
Speaker:
Over the river and through the woods?
Speaker:
Um, yeah, so you might, and that was, I remember back in the day,
Speaker:
we had a synchronous copy that was literally like right next door.
Speaker:
I mean literally, when I say next door, I mean Like it's the next
Speaker:
rack over, um, that was, uh, like an A or asynchronous replication.
Speaker:
And then actually, will you remember Symmetrics and or,
Speaker:
or, or EMC and, and NetApp?
Speaker:
They, you'd have like three.
Speaker:
You'd have like four storage arrays, right?
Speaker:
You'd have the,
Speaker:
Oh, you can have
Speaker:
a lot more.
Speaker:
Yeah,
Speaker:
you'd have the synchronous that was right next to it, the asynchronous
Speaker:
that was down the street.
Speaker:
And then another asynchronous that was the other.
Speaker:
Yeah,
Speaker:
and sometimes there are customers who could use it for like data
Speaker:
distribution use cases, right?
Speaker:
So I need to
Speaker:
make sure that my database is available locally in different countries.
Speaker:
So I'm just going to replicate it all.
Speaker:
yeah.
Speaker:
So synchronous, both forms of replication have a lot more applications
Speaker:
than just backup and recovery.
Speaker:
Um, so I'm glad you mentioned, you know, or you VMs, there is another.
Speaker:
Not file and not VM thing that gets replicated, which is very, where
Speaker:
we still have sort of the same discussion about synchronous versus
Speaker:
asynchronous, and that is databases,
Speaker:
Application level synchronous
Speaker:
right?
Speaker:
And, and, and I'd say that in that case, basically.
Speaker:
When we, again, that can definitely have two phase commit, right?
Speaker:
So in the database.
Speaker:
You're replicating a transaction or, or something.
Speaker:
I don't have a better word.
Speaker:
I know not every database has transactions, but I
Speaker:
don't have a better word for
Speaker:
It's the atomic unit, whatever it is, or sorry, the unit of
Speaker:
Yeah, an atomic unit of a thing, you changed a thing in the database, right?
Speaker:
And you're replicating that, rather than the blocks underneath.
Speaker:
And, well, you could be replicating the blocks underneath, and that's
Speaker:
what we're talking about over here.
Speaker:
But at this point, we're talking about database level,
Speaker:
application level replication.
Speaker:
And when you do that, you have the choice of either immediate
Speaker:
consistency, or eventual consistency.
Speaker:
Which is essentially analogous to synchronous and asynchronous.
Speaker:
And, and what made me think about it was when you were talking about that,
Speaker:
there are other reasons that people replicate and, um, there are databases
Speaker:
that allow replicating data, you know, across the world and the best.
Speaker:
Example that a lot of people are used to is the DNS database, right?
Speaker:
That is an eventually consistent, replicated, um, system.
Speaker:
If you've never, if you've never made changes to DNS.
Speaker:
Let me tell you, it can take a minute.
Speaker:
I recently, literally in the last week or two, I've been making changes to
Speaker:
various websites that I own to, you know, change their, uh, DNS entries
Speaker:
and CNAME entries and things like that, and they don't all immediately happen.
Speaker:
Um,
Speaker:
And yeah,
Speaker:
but
Speaker:
they will eventually get there.
Speaker:
That is essentially the database equivalent to, um, to, uh,
Speaker:
asynchronous replication.
Speaker:
Now, let me ask you a question.
Speaker:
Prasanna,
Speaker:
mm hmm.
Speaker:
if replication is so amazing and we can have the tightest RPOs, and, and, and
Speaker:
I think we can agree that Tighter RPOs are better than looser RPOs, right?
Speaker:
An RPO of zero or a minute is way better than an RPO of 36 hours, which is what
Speaker:
we typically get with a backup system.
Speaker:
Why don't we do replication for everything?
Speaker:
And by the way, money is not the issue.
Speaker:
You knew what I was going to say, uh,
Speaker:
It's not, well, it is an issue besides money, right?
Speaker:
Because replication is
Speaker:
yeah, so the
Speaker:
don't we do that?
Speaker:
so the biggest problem with replication is for most of the replication technologies,
Speaker:
when you replicate it, you only have that one copy on your target system, right?
Speaker:
And so, If you had a user who dropped a database table and you replicated that
Speaker:
command over to your replica, guess what?
Speaker:
It gets dropped on your replica as well, and you have no ability to recover.
Speaker:
And
Speaker:
so that's one of the biggest limitations of relying on replication,
Speaker:
or at
Speaker:
And same would be true of any change on the, on the primary system, which
Speaker:
includes things like ransomware, right?
Speaker:
So if there's a, if there's a hacker goes in.
Speaker:
and deletes all your tables, or just, you know, corrupts your, your data,
Speaker:
deletes all your files, whatever.
Speaker:
It just, if a virus does, whatever, whatever, you know, if there's any
Speaker:
kind of cyber attack on the primary system, it simply makes that attack more
Speaker:
efficient,
Speaker:
And because the entire purpose of replication is to create a duplicate
Speaker:
copy on your target system, right?
Speaker:
Immediately, if you ask for an RPO of zero, you're going to get an RPO of zero,
Speaker:
now, now I will say, I will put a caveat that, wait until we talk about snapshots,
Speaker:
we will come back to this specific point,
Speaker:
Right.
Speaker:
There, There, are, there are back backup data protection based systems
Speaker:
that use replication at their core.
Speaker:
And don't have this problem.
Speaker:
This is just, just replication, right?
Speaker:
It's like, why don't we use raid for everything?
Speaker:
And then we don't need backup.
Speaker:
Well, same exact reasons, right?
Speaker:
Exactly.
Speaker:
Uh, raid does help with media failure and equipment failure.
Speaker:
And, um.
Speaker:
Site failure and power failure and, and, and, and, but it is completely defenseless
Speaker:
against cyber attacks and stupidity.
Speaker:
Or logic issues.
Speaker:
Well, I'm just, you're being nice.
Speaker:
See, this is why this is,
Speaker:
Well, I was thinking software bugs, not user logic.
Speaker:
Oh, okay.
Speaker:
Well, that's still stupidity.
Speaker:
It's just, you know, Different
Speaker:
Yes.
Speaker:
person who made the bug.
Speaker:
Um, well, this has been fun.
Speaker:
Uh, but yeah, this whole thing of, um, this idea that replication is where
Speaker:
we're continuously updating the data.
Speaker:
Now, now that you see like this idea of, of, um, you know, why we were wrestling
Speaker:
over the word continuous, right?
Speaker:
Because.
Speaker:
Is asynchronous continuous?
Speaker:
I think it's still a continuous process.
Speaker:
It's just continuous with an asterisk, continuous with, with, with a buffer.
Speaker:
Um, and I think that, well, in fact, I know that today is.
Speaker:
Way better than it was back in the day.
Speaker:
You back in the day, you had to have the same vendor on both sides.
Speaker:
It, it, it sold a lot of EMC, right?
Speaker:
You had to have the same vendor because the replication was based on the box.
Speaker:
And now there are a number of products.
Speaker:
one of which we've had on here, right?
Speaker:
We had Datacore on here, which, which does replication between, you know,
Speaker:
uh, disparate, different architectures.
Speaker:
You can have a very expensive primary system.
Speaker:
And a less expensive target system.
Speaker:
You can use it as a way to upgrade and move your technology.
Speaker:
You can move your older systems to be the target system, knowing that if you
Speaker:
need to use it in a disaster, you will suffer a performance loss, but you
Speaker:
don't have to, like you did back in the
Speaker:
day.
Speaker:
You don't, you know, when you primer, you don't have to have the same, you know,
Speaker:
and so you get, you could cost savings.
Speaker:
You don't have to do it that way, but it gives you the flexibility.
Speaker:
And I think that modern day replication systems are much more
Speaker:
forgiving of things like You know, outages and, and things like that.
Speaker:
Right.
Speaker:
Um, and, but, you know, but they still don't protect you from, you know, mistakes
Speaker:
and cyber attacks, which is why we have to talk about the alternatives, which we'll
Speaker:
do on later episodes, but, uh, so you were, you were unusually, not unusually,
Speaker:
you were excessively argumentative in this episode, but I thank you anyway.
Speaker:
that, that, I try, I basically spent all of my, almost all of my career
Speaker:
working on replication technology, so.
Speaker:
Yeah.
Speaker:
And I, I, yeah, you worked at a company that was like, it was trying
Speaker:
to say, well, there's asynchronous and there's near synchronous.
Speaker:
And I'm like, those are two words for asynchronous.
Speaker:
As I was editing this episode, I realized that.
Speaker:
I was being a little too pedantic about continuous or not continuous.
Speaker:
The thing with replication that really sets it apart from other data
Speaker:
protection technologies, is , one is that the copy is an actual copy.
Speaker:
It is a, it looks exactly like the original, unlike what we do in the
Speaker:
backup world, where we tend to put stuff in a container or something like that.
Speaker:
Um, that's one big thing.
Speaker:
That's separate separates replication from other backup technologies.
Speaker:
And then also the idea is that it is, it is incrementally updating that.
Speaker:
Whenever we do it, whether it's continuous asynchronous or even periodic.
Speaker:
I really forgot about how, like there are vendors like NetApp.
Speaker:
That actually do periodic replication.
Speaker:
There's nothing wrong with periodic replication.
Speaker:
It's still replication.
Speaker:
Um, so you've got continuous asynchronous and then periodic
Speaker:
that, what, what sets replication apart from everything else is this.
Speaker:
Whenever we do the next replication.
Speaker:
What we're doing is we're transferring the, the, the things, whether it's
Speaker:
database records or blocks, we're transferring the things that have changed
Speaker:
since the last time we replicated.
Speaker:
So I hope that.
Speaker:
Helps Muddy the waters.
Speaker:
Just a little bit.
Speaker:
You're either synchronous or you're not just like in.
Speaker:
In the following episode, we will be talking about a
Speaker:
term near continuous, which I actually coined, but, uh, a lot
Speaker:
of people think isn't a thing.
Speaker:
I don't know
Speaker:
if
Speaker:
think, they're like, you can't have...
Speaker:
Then again, they'll say that continuous is a...
Speaker:
Um, is, is, is a binary, uh, condition.
Speaker:
So anyway, well, um, I want to again, uh, say thank you to our listeners.
Speaker:
You are why we do this.
Speaker:
Remember, this is an independent podcast and the opinions that you hear are ours
Speaker:
I hope this episode has been helpful.
Speaker:
That's a wrap.
Listen On
