Backup team completes two-year project in three months
Hear the incredible story of Albert Uy, who was handed a two-year project that had already wasted 18 months. They told him he had four months to complete it. He looked at that timeline and told them there was only one way he could do, by switching horses in mid-stream. Not only did they finish it in time; they finished it early. Albert has also managed backup teams for many years, so before we cover this project story, we talk about backup has progressed over the years. We also talk about how much easier his job got once he started using Druva has his backup solution.
Mentioned in this episode:
Interview ad
This week on restored all, we've got a backup practitioner that has used
Speaker:multiple backup products, including the one where I happened to work.
Speaker:I hope you enjoy this week's episode.
W. Curtis Preston:Hi, and welcome to Backup Central's Restore it All podcast.
W. Curtis Preston:I'm a host w Curtis Preston, a k a, Mr.
W. Curtis Preston:Backup, and I have with me my colossal Chore completion
W. Curtis Preston:celebration companion Prasanna
Prasanna Malaiyandi:Woohoo,
Prasanna Malaiyandi:. W. Curtis Preston: Malaiyandi
Prasanna Malaiyandi:How does it feel, Curtis that.
W. Curtis Preston:my Lord.
W. Curtis Preston:it's, you know, uh, there are no words.
W. Curtis Preston:I, I, I will technical, you know, basically, you know, to,
W. Curtis Preston:so to be honest, I have finished the part of the project that.
W. Curtis Preston:everyone can see.
W. Curtis Preston:I'm still sitting on carpet in this one room, which is my office.
W. Curtis Preston:Um, and uh, but everything else has been completed, which means that we're
W. Curtis Preston:gonna have guests over for Christmas.
W. Curtis Preston:So that's
Prasanna Malaiyandi:And, and this is basically for the listeners who
Prasanna Malaiyandi:may not realize this is Curtis's.
Prasanna Malaiyandi:I'm going to replace all the flooring in the downstairs by myself
Prasanna Malaiyandi:and take up tile, which has been.
Prasanna Malaiyandi:Glued down with super glue, basically
W. Curtis Preston:Yeah.
Prasanna Malaiyandi:and
W. Curtis Preston:was, there was what?
W. Curtis Preston:There was carpet, there was tile, you know, large, large format ceramic
W. Curtis Preston:tile and also Pergo in one room.
W. Curtis Preston:So I took all that up and it was approximately 1500 square feet of
W. Curtis Preston:luxury vinyl tile that I laid down.
Prasanna Malaiyandi:it was like three pallets that were
Prasanna Malaiyandi:sitting in your garage, right?
W. Curtis Preston:Yeah.
W. Curtis Preston:Uh, and I'd say about, um, I don't know, about a 25% into the project I was.
W. Curtis Preston:what have I done?
W. Curtis Preston:Like, because I, I actually got hurt.
W. Curtis Preston:Like, I, I, you know, I'm, I, you know, I'm in my fifties, I'm down on my knees.
W. Curtis Preston:My, my, that, that was a mess.
W. Curtis Preston:Um, and then I got sick, so, but yeah, the really funny thing is, um, for the, for
W. Curtis Preston:the long time listeners, you know, that sometimes my granddaughter comes on here
W. Curtis Preston:and at the beginning of the project when she saw how much work it was, her comment
W. Curtis Preston:was, this is gonna take like five months.
W. Curtis Preston:and I was really insulted at the time, but, uh, it took actually a
W. Curtis Preston:little over five months due to my injury and my, and my sickness.
W. Curtis Preston:But, but yeah, at this point it looks beautiful out there and I, I, I don't
W. Curtis Preston:think the mic would pick it up, but right now, just outside my door is
W. Curtis Preston:our new, uh, Roomba that we bought,
Prasanna Malaiyandi:To keep it.
W. Curtis Preston:of that.
W. Curtis Preston:Yeah.
Prasanna Malaiyandi:it and span
W. Curtis Preston:Yeah, exactly.
W. Curtis Preston:Because the one thing I noticed with having, you know, the hard floor
W. Curtis Preston:versus the co carpet is that you see and feel every little bit of dust.
W. Curtis Preston:And so, um, that, uh, so yeah, so we, we added that, that it's, it's
W. Curtis Preston:now done, its first full vacuuming and now it's trying to do its second.
W. Curtis Preston:So it's a fun, it's a fun time.
W. Curtis Preston:At the Preston household
Prasanna Malaiyandi:Well, at least it's all done before Christmas,
W. Curtis Preston:Exactly.
W. Curtis Preston:Um,
Prasanna Malaiyandi:I'm sure your wife is quite pleased.
W. Curtis Preston:yeah, she really likes, she really likes decorating
W. Curtis Preston:the house for Christmas and we really weren't able to because of, you know,
W. Curtis Preston:the fact that yeah, it was a construction zone up until just like two days ago.
Prasanna Malaiyandi:so now here's a question, Curtis, are you gonna quit
Prasanna Malaiyandi:your day job and go do INS flooring installation as a full-time gig?
W. Curtis Preston:absolutely not.
W. Curtis Preston:There was this guy, and by the way, shout out to there, there's a website called
W. Curtis Preston:for anybody who's thinking about doing it.
W. Curtis Preston:It, it wasn't like difficult per se other than the fact that you're on
W. Curtis Preston:your hands and knees the whole time.
W. Curtis Preston:And, and one thing that was invaluable was this, there's a guy, um, who.
W. Curtis Preston:Has a site called, so that's how you do that.com . And he has a
W. Curtis Preston:service where you give him like, I think it's like a hundred bucks.
W. Curtis Preston:and then he guides you through the project.
W. Curtis Preston:You give him your floor layout and he's like, start in this room, go
W. Curtis Preston:to this room and here's the room where you're gonna work backwards.
W. Curtis Preston:And um, you know, and that was invaluable throughout the entire process.
W. Curtis Preston:Um, so shout out to him.
W. Curtis Preston:Um, yeah.
W. Curtis Preston:Thanks Joe.
W. Curtis Preston:Anyway, so let's get on to our guest who's sitting there going, what kind of
W. Curtis Preston:podcast have I come on At this point, I didn't sign up for home improvement stuff.
W. Curtis Preston:He has been in it for over 20 years and was most recently the Vice
W. Curtis Preston:President of Technology, architecture and Innovation at Maximus, a
W. Curtis Preston:government administration company.
W. Curtis Preston:If you're looking for an experienced bright IT manager, we think he'd
W. Curtis Preston:be a great addition to your team.
W. Curtis Preston:Welcome to the podcast, Albert.
W. Curtis Preston:Uy,
Albert Uy:Uh, hi Curtis.
Albert Uy:Good morning.
Albert Uy:Thank you for the introduction and I'm glad to be here.
W. Curtis Preston:uh, have you recently completed any major?
W. Curtis Preston:Do it yourself projects
Albert Uy:Yes, actually, like you're saying, I'm doing the Christmas
Albert Uy:lights and, and kind of upgraded my laundry, so did some cabinets and sink.
Albert Uy:So I'm a, I'm a diy like you, so d i
W. Curtis Preston:right.
W. Curtis Preston:All right.
W. Curtis Preston:Fellow, fellow DIY person
W. Curtis Preston:respect.
W. Curtis Preston:Um, sometimes, you know, you bite off a little more than you can chew.
W. Curtis Preston:Uh, you, you know, you're halfway through the project and you're like, why?
W. Curtis Preston:You know, why didn't I just call the guy?
Prasanna Malaiyandi:The solar project, Curtis
W. Curtis Preston:Yeah.
W. Curtis Preston:We won't talk about the solar project.
W. Curtis Preston:The solar project was, yeah.
W. Curtis Preston:Speaking of taking six months.
W. Curtis Preston:Um, yeah, but it, but so ha Is it the, the, the laundry thing?
W. Curtis Preston:Is it finished?
Albert Uy:Yeah, it's finished.
Albert Uy:Um, do you wanna hear an interesting thing I wanna share?
Albert Uy:So, you know, I, you know, I got a point.
Albert Uy:Install the cabinet and install the sink with the plumbing,
Albert Uy:and I'm like, okay, I need it.
Albert Uy:I need to get it certified, so I need a certified plumber.
Albert Uy:Right?
Albert Uy:So they say they usually charge like $90 plan trip and stuff, but when they heard,
Albert Uy:uh, I just moved into a new house, the price went from $90 to like five, $600.
Albert Uy:So I know when they hear like, oh, you just moved into a new house.
Albert Uy:And so I went and to uh, forget it, I can do it.
Albert Uy:I went and just did it and I'm, I'm pretty sure I, I would,
Albert Uy:yeah, I, I did a great job.
Albert Uy:So my wife said, Hey, thumbs up.
W. Curtis Preston:why they would
Albert Uy:Yeah.
Albert Uy:Yeah.
Albert Uy:When they hear a new house, the price just went up the roof.
Albert Uy:I don't know why.
Albert Uy:From 90, 90 to a hundred to like four, 500 dollars.
Albert Uy:Just, just to just install the plumbing, the, you know, the, the, the, yeah.
Albert Uy:So
W. Curtis Preston:Wow.
Albert Uy:crazy.
W. Curtis Preston:Yeah, it is crazy.
W. Curtis Preston:Uh, yeah, I recently redid some, you know, some stuff in the kitchen and,
W. Curtis Preston:and I, and I, I did call the guy cuz plumbing is one of those things
W. Curtis Preston:where it, it's not, Technically very hard, per se, at least much of it.
W. Curtis Preston:Some of it is actually really hard, uh, but it's, but it, but it's, it's one of
W. Curtis Preston:those things where it's not, it's like, it's not technically difficult, but you're
W. Curtis Preston:on your back and you're in these tight, tight, tight spaces and all that stuff.
W. Curtis Preston:So it's definitely in that category where I'm like, you know what?
W. Curtis Preston:And, and also the one thing about plumbing is if you get it wrong,
W. Curtis Preston:you might not know right away.
W. Curtis Preston:Right.
W. Curtis Preston:It might be just a slow leak and then you find it out way later
W. Curtis Preston:and it costs you way more money.
W. Curtis Preston:Um, we won't talk about the bathtub incident of, uh, 2000
Prasanna Malaiyandi:Well, you know another thing that reminds me though, I
Prasanna Malaiyandi:know you're talking about plumbing, but I feel the same way about cars and brakes.
Prasanna Malaiyandi:Right?
Prasanna Malaiyandi:It's like a brake job.
Prasanna Malaiyandi:It's like get someone to do that.
Prasanna Malaiyandi:And I know, Curtis, you're laughing because I know you do your own breaks, but
W. Curtis Preston:Yeah, I, that was, that was the way I felt for a long time
W. Curtis Preston:until I, until I found out it, that, that,
W. Curtis Preston:that they actually weren't, that.
W. Curtis Preston:Yeah, they're not, and, and, yeah.
W. Curtis Preston:Yeah.
W. Curtis Preston:Uh, and generally with breaks, they either work or they don't, like, it's
W. Curtis Preston:not like you, you know what I mean?
W. Curtis Preston:Anyway.
W. Curtis Preston:Yeah.
W. Curtis Preston:But I, but I, I, I respect the, the not doing it as well.
Albert Uy:And uh, and I was gonna share the trick is you have to
Albert Uy:put the right washer and clamps.
Albert Uy:And like you said, I went and put a pan after my work is done.
Albert Uy:Uh, just to make sure there's no leak.
Albert Uy:I left it for a month and there were no leak.
Albert Uy:So,
W. Curtis Preston:Nice, nice, nice.
W. Curtis Preston:Yeah, so, well, we're not talking about plumbing today.
W. Curtis Preston:We're
Prasanna Malaiyandi:Are you sure we could
W. Curtis Preston:Yeah, we could, we could do a just a DIY podcast where I
W. Curtis Preston:talk about what I've done and you talk about what you've seen on YouTube,
Prasanna Malaiyandi:Yep, exactly.
Prasanna Malaiyandi:Yep.
W. Curtis Preston:Um, and, um, maybe, maybe Albert and Albert and
W. Curtis Preston:I can, can start a d i y podcast.
W. Curtis Preston:Um,
Prasanna Malaiyandi:I was recently watching a YouTube video of a guy trying
Prasanna Malaiyandi:to clean up a 30 by 60 saltwater pool,
Albert Uy:Wow.
W. Curtis Preston:clean it out.
W. Curtis Preston:What do you
Prasanna Malaiyandi:sorry, clean it up because it was full
Prasanna Malaiyandi:of algae and everything else.
Prasanna Malaiyandi:He had just moved into a house and he had no pool experience, and he gave
Prasanna Malaiyandi:up and he just hired a pool service.
Prasanna Malaiyandi:They wanted five grand to bring it back to life, and then 1200 a month.
W. Curtis Preston:See, this is why I do stuff myself.
W. Curtis Preston:I had the same problem, five grand.
Prasanna Malaiyandi:Yep.
Prasanna Malaiyandi:Well this is a 30 by 60, Curtis.
W. Curtis Preston:Oh, but still, I mean, 30 by 60 is big,
Albert Uy:Yeah, it is
Prasanna Malaiyandi:Yeah.
W. Curtis Preston:okay.
W. Curtis Preston:One, one more.
W. Curtis Preston:So, one more.
W. Curtis Preston:So years ago when my, I have a pool, and by the way, you don't need a pool.
W. Curtis Preston:You need a friend with a pool.
W. Curtis Preston:Um, that's a life lesson to anyone.
W. Curtis Preston:Please take that lesson.
W. Curtis Preston:You don't need a boat.
W. Curtis Preston:You don't need a pool.
W. Curtis Preston:You need a.
W. Curtis Preston:with, with both of those, or, or one friend with one each.
W. Curtis Preston:Uh, my pool, my pump went down and I didn't have the money to replace it right
W. Curtis Preston:away, and by the time I replaced it, I had a swamp in the, you know, I had a, a,
W. Curtis Preston:it was disgusting and I actually, if you can believe this, and a bunch of stuff
W. Curtis Preston:had been dumped in the pool like, like leaves and twigs and all that kind of
W. Curtis Preston:stuff, I actually put on my scuba gear.
W. Curtis Preston:and went in my pool and went down and cleared out all the
W. Curtis Preston:debris in the bottom of my pool.
W. Curtis Preston:And by the way, I was completely blind cuz it was just green, right?
W. Curtis Preston:Just a hundred percent green.
W. Curtis Preston:And I'm down there with a, with a bag and I'm just, just
W. Curtis Preston:crawling around in the bottom.
W. Curtis Preston:I had myself super weighted so that I was just laying on the bottom and I'm
W. Curtis Preston:pulling out the, the twigs and the rocks and the, you know, And, and because I
W. Curtis Preston:knew that like any one of those could, could mess up the, the, the pump, right?
W. Curtis Preston:The Well, no, the, the, you know, the,
Prasanna Malaiyandi:Oh, the Impalas?
W. Curtis Preston:
Speaker:crawley thing, you know?
W. Curtis Preston:
Speaker:Yeah.
W. Curtis Preston:
Speaker:So, so I, I, I, so the, and then, and then I just dumped it.
W. Curtis Preston:
Speaker:I just dumped.
W. Curtis Preston:
Speaker:The pool, right?
W. Curtis Preston:
Speaker:Um, and instead of trying to spend hundreds and hundreds and hundreds of
W. Curtis Preston:
Speaker:dollars of chemicals trying to bring what it was back to life, uh, but that
W. Curtis Preston:
Speaker:was an expensive, just the process of dumping the pool and refilling the
W. Curtis Preston:
Speaker:pool, uh, was an incredibly expensive process because they charge you, you
W. Curtis Preston:
Speaker:know, uh, what it is, is it, it impacts both your water bill and your sewer.
W. Curtis Preston:
Speaker:um, because they're like, oh, they're, you know, cuz they, they look at how much it
W. Curtis Preston:
Speaker:impacts your sewer bill for like the next quarter because they see how much water
W. Curtis Preston:
Speaker:you put out.
W. Curtis Preston:
Speaker:Yeah.
W. Curtis Preston:
Speaker:And so, uh, so the secret is to fill it with your neighbor's hose,
Albert Uy:Good tip.
Prasanna Malaiyandi:Oh, Curtis.
W. Curtis Preston:i y tip, fill your pool with your neighbor's hose.
W. Curtis Preston:Um,
Prasanna Malaiyandi:Hmm.
W. Curtis Preston:I kid.
W. Curtis Preston:I kid.
W. Curtis Preston:So, so, Hey Albert.
W. Curtis Preston:Uh, you, you've been in it for a while, for a minute.
W. Curtis Preston:As the kids, say,
Prasanna Malaiyandi:Hmm.
W. Curtis Preston:um, and, you know, and, and I'm sure you've been in and
W. Curtis Preston:around backup systems quite a bit.
W. Curtis Preston:From your perspective, how has backup gotten.
W. Curtis Preston:Better and worse, like over, you know, you've seen, uh, if you've been in it 20,
W. Curtis Preston:you've been through the tape world, you've been through the conversion to disc world.
W. Curtis Preston:You've been into the, you know, into the, um, obviously into the cloud world.
W. Curtis Preston:Uh, what, what's that?
W. Curtis Preston:What's that been like right.
Albert Uy:I would say people are lucky to be, uh, managing backup today compared to
Albert Uy:20 plus years ago because like you said, when I first started, there were only
Albert Uy:like one cartridge or there's no robotic, and, and the tape is, can only do like
Albert Uy:megabyte versus gigabyte and all that.
Albert Uy:And then, and then you know, you have to start with on-premise and then
Albert Uy:now you got cloud and all you need to do is good internet connection.
Albert Uy:So, you know, it's almost like night and day comparison.
W. Curtis Preston:Right.
W. Curtis Preston:So talk to me about, um, so first, you know, we took, you know, you, you
W. Curtis Preston:were in that tape world and I, and I grew up in that tape world, right?
W. Curtis Preston:My, my.
W. Curtis Preston:The backup tape that I first, well, technically the first tape I did a backup
W. Curtis Preston:to was actually a nine track tape, which I don't, I don't remember what the capacity
W. Curtis Preston:of that, but it wasn't much because, um, because I remember that the, the density
W. Curtis Preston:of that tape was 120 bits per inch.
W. Curtis Preston:It was a single.
W. Curtis Preston:It was a single, um, recording stream.
W. Curtis Preston:Right.
W. Curtis Preston:Um, and then I, I have, I'd have to look up, I, yeah.
W. Curtis Preston:Anyway, so that was my first, and then I remember working with,
W. Curtis Preston:you know, eight track tapes.
W. Curtis Preston:I'm sorry.
W. Curtis Preston:Um, eight millimeter tapes, four millimeter tapes, uh,
W. Curtis Preston:then, um, dlt, then L T.
W. Curtis Preston:, right.
W. Curtis Preston:Um, grew through all of those.
W. Curtis Preston:Um, d do you remember, were things ever good with that world Because
W. Curtis Preston:I remember like, for a while they were bad because the drives
W. Curtis Preston:themselves were really unreliable.
W. Curtis Preston:Didn't the drives?
W. Curtis Preston:My memory is that the, the drives got more reliable, but then they got too fast.
W. Curtis Preston:There.
W. Curtis Preston:Was this, the shoe shining problem of the tape drive being too fast for the job?
W. Curtis Preston:Uh, how, how about.
Albert Uy:Right.
Albert Uy:Yeah.
Albert Uy:Um, well, I was gonna say, well, the key thing is when you're doing backup and
Albert Uy:disaster recovery is r p o and r t o.
Albert Uy:RPO stands for recovery point objective, and RTO stands
Albert Uy:for recovery time objective.
Albert Uy:So, like you said, um, more, uh, two decades ago because of the speed wise,
Albert Uy:um, there's three types of, uh, backing up full backup differential and incremental.
Albert Uy:Um, because of the slow speed you have to do of a lot of incremental, right?
Albert Uy:However, people don't realize when you restore, restore, that means you
Albert Uy:have to restore a lot of incrementals.
Albert Uy:Right.
Albert Uy:So as the tape device goes fast and fast, you can go less incremental
Albert Uy:and do more differential.
Albert Uy:And today with the speed, you can also do full backup.
Albert Uy:So what I'm trying to say, we use it when you restore, you always
Albert Uy:want to start restoring, using full backup and then whatever differential
Albert Uy:based on r p on R T O based on, you know, restore your differential.
Albert Uy:And if not, you're incremental.
Albert Uy:Uh, if I make sense.
W. Curtis Preston:Yeah.
W. Curtis Preston:You do.
W. Curtis Preston:Yeah.
W. Curtis Preston:Prasanna, you, you remember?
W. Curtis Preston:We just, what?
W. Curtis Preston:It was
Prasanna Malaiyandi:I think it was like two episodes ago.
Prasanna Malaiyandi:Yeah.
W. Curtis Preston:We just did an episode on backup levels and, uh, I
W. Curtis Preston:prefer the term cumulative incremental
Albert Uy:Yep, yep, yep.
W. Curtis Preston:different companies
W. Curtis Preston:use the term differential to mean
W. Curtis Preston:different things.
W. Curtis Preston:Um, but yeah, that was a, that was a huge thing back then was the,
W. Curtis Preston:the, how often we're gonna do a full, how often we're gonna do a
W. Curtis Preston:differential or cumulative, incremental.
W. Curtis Preston:And then obviously we did incrementals every day if we
W. Curtis Preston:weren't doing one of those other
Prasanna Malaiyandi:
Speaker:Wasn't there a Hanoi Tower?
W. Curtis Preston:Yeah, there was, the Tower of Hanoi thing was about
W. Curtis Preston:different levels of backup real.
W. Curtis Preston:That that was a huge thing.
W. Curtis Preston:Compare that to what we do now, which I, I would say when you look at,
W. Curtis Preston:um, and by the way, I, I, I'll throw out our usual disclaimer Prasanna,
W. Curtis Preston:and I work for different companies.
W. Curtis Preston:I work for Druva, and, uh, Prasanna works for Zoom and uh, this is
W. Curtis Preston:not a podcast of either company.
W. Curtis Preston:Uh, and the opinions that you hear are ours and sometimes they're all mine.
Prasanna Malaiyandi:They're all Curtis's.
W. Curtis Preston:I don't give Prasanna a word in edge, a word to speak.
W. Curtis Preston:Um, and, um, uh, and by the way, be sure to subscribe to this podcast
W. Curtis Preston:so that you don't miss anything.
W. Curtis Preston:And also, Yeah, just click right.
W. Curtis Preston:Just click subscribe right now so you don't forget.
W. Curtis Preston:And be sure to rate us, just, uh, scroll down to the rating thing.
W. Curtis Preston:Give us, you know, five stars and, uh, give us a comment.
W. Curtis Preston:We love, we love comments.
W. Curtis Preston:Always helpful.
W. Curtis Preston:Always.
W. Curtis Preston:It helps.
W. Curtis Preston:It helps other people find our podcast, the more comments that we have.
W. Curtis Preston:And, um, But, uh, and, and also, uh, you know, in full disclosure, uh,
W. Curtis Preston:Albert is a former Druva customer.
W. Curtis Preston:Uh, he's not, uh, you know, he's not employed right now, so he is
W. Curtis Preston:not a current Druva customer, but, uh, he was a former Druva customer.
W. Curtis Preston:So we'll be talking about Druva a little bit in this episode.
W. Curtis Preston:A little bit more than normal, I think.
W. Curtis Preston:But, um, so what I was about to say was, You know, with a product like Druva, and
W. Curtis Preston:we're not, we're not the only ones that do it, but with a product like Druva, you
W. Curtis Preston:do a, what I call a forever incremental.
W. Curtis Preston:You don't do repeated fulls, but.
W. Curtis Preston:, but the backups are stored in such a way that each backup behaves as a
W. Curtis Preston:full, from a recovery perspective.
Prasanna Malaiyandi:I know we always talk about sort of the incrementals,
Prasanna Malaiyandi:but something that I always like to bring up is when you talk applications
Prasanna Malaiyandi:is after the incrementals, then you gotta worry about all your archived
Prasanna Malaiyandi:redo logs and whatever the nomenclature, the application uses, right?
Prasanna Malaiyandi:Which just adds more and more to the time it takes to recover.
Albert Uy:And, and that's a very good point.
Albert Uy:So I said the first requirement you knew to know is the R P R T O.
Albert Uy:The second I'm a big, um, follower methodology of people, first
Albert Uy:process, and then technology.
Albert Uy:So just to share with you, I also used to work at GameStop,
Albert Uy:which is also a Druva customer.
Albert Uy:Um, I was able to reduce the 72 hours of Dr.
Albert Uy:Uh, r t o to 24 hours.
Albert Uy:and when I mean people process technology, um, because, uh, they were like siloed the
Albert Uy:service team by themselves and database.
Albert Uy:So when we were doing disaster recovery, um, the server guys have to restore the
Albert Uy:file first and then database comes in and then use their database back up.
Albert Uy:To restore the database.
Albert Uy:So, you know, there's two different methods of, you know, um, restore, right?
Albert Uy:So yeah, that's why people, you know, get together with the people first and then
Albert Uy:create the process to make the people work together and then apply the technology.
Prasanna Malaiyandi:Yeah, sometimes you people tend to forget about
Prasanna Malaiyandi:those first two parts, right?
Prasanna Malaiyandi:They're like, oh, here's an awesome technology.
Prasanna Malaiyandi:It'll solve my problems.
Prasanna Malaiyandi:But then they apply it, and then it doesn't work the way they think,
Prasanna Malaiyandi:because the people in the process part, they haven't worked through.
Prasanna Malaiyandi:And it's like, oh, the technology's bad.
Prasanna Malaiyandi:But it's like, no, the technology isn't bad.
Prasanna Malaiyandi:It's just you only did one third of the problem.
Prasanna Malaiyandi:You have to also worry about the people in the process.
Albert Uy:And, and another thing I wanna share is when I was at
Albert Uy:Maximus, um, because of the people, there were only two tier support.
Albert Uy:People requesting the restore and then the backup team restoring.
Albert Uy:But the thing is, you know, it queues up because the backup
Albert Uy:team are the only restoring.
Albert Uy:So I created a third tier, like a middle tier.
Albert Uy:Um, we gave, um, the database team power, user rights, server
Albert Uy:team power, user rights, site administrator, power, user rights.
Albert Uy:So instead of going to a backup team, the site support team can, can restore
Albert Uy:site files, database can restore database files, and server team can restore
Albert Uy:server team server files, and then application can restore application files.
Albert Uy:By doing that, the sla, um, we were able to meet SLA because it's not entirely,
Albert Uy:uh, dependent on the backup team.
Prasanna Malaiyandi:it's almost a little bit like self-service.
Albert Uy:Yep, exactly.
W. Curtis Preston:Yeah.
W. Curtis Preston:You know, you, you bring up a really, I, I think a really good,
W. Curtis Preston:another part of technology that.
W. Curtis Preston:That has happened in the last, I'm gonna say 10 years, and that is the idea
W. Curtis Preston:of role-based administration, right?
W. Curtis Preston:It is probably a little more than 10 years, but, um, what I remember
W. Curtis Preston:again, back, back in the day, uh, if you were gonna administer
W. Curtis Preston:a backup system, you were root.
W. Curtis Preston:Right.
W. Curtis Preston:You, you had root, you logged into the system as root.
W. Curtis Preston:You pulled up the, the, the ui, uh, we called it a gooey back in the day, right?
W. Curtis Preston:You pulled up the gooey and you ran that gooey as root.
W. Curtis Preston:And if you didn't root, because root was the, was the, the user ID that
W. Curtis Preston:had the power to get all the files.
W. Curtis Preston:That's why you had to run it as route.
W. Curtis Preston:Right.
W. Curtis Preston:Um, and that, but that was a horrible thing because I remember back in the
W. Curtis Preston:day, Saying, you know, I, I, I would joke about it, but I would say, you know, be
W. Curtis Preston:friendly to the backup person because with, with, in order to do backups,
W. Curtis Preston:not only did I have root on the backup adminis, the backup system, I had root
W. Curtis Preston:on every system because I needed to.
W. Curtis Preston:So I had the ability to log into every.
W. Curtis Preston:Destroy your world, right?
W. Curtis Preston:And then go to the backup system, or maybe before that, go to the backup
W. Curtis Preston:system and destroy all the backups, right?
W. Curtis Preston:So be really kind to the backup person because they have the
W. Curtis Preston:ability to destroy your entire world and leave no evidence, right?
W. Curtis Preston:Um, that's, that's what we did away.
W. Curtis Preston:With this idea of role-based administration, this idea that you
W. Curtis Preston:gave, because back then, if you wanted, you know, the Oracle DBAs
W. Curtis Preston:to, to, to do their own thing, you had to give them root, right?
W. Curtis Preston:Which I think we could all agree.
W. Curtis Preston:They don't need root, we don't need Oracle, uh, permission.
W. Curtis Preston:Right.
W. Curtis Preston:Um, and of course I had Oracle permission back then too.
W. Curtis Preston:Um, so this idea that you could give certain groups of people the ability to
W. Curtis Preston:do things, The way you described it was it was a way to make things better for them,
W. Curtis Preston:but it's also a way to make things better for you in that you gave them just, you
W. Curtis Preston:know, we, we have this concept of least privilege that we talk a lot about, right?
W. Curtis Preston:You gave them just enough power to do the job they needed to do, but
W. Curtis Preston:without giving them the ability to mess up the rest of the world.
W. Curtis Preston:Does that sound about.
Albert Uy:Exactly role based
Prasanna Malaiyandi:don't give him the keys to the Kingdom
W. Curtis Preston:Yeah.
W. Curtis Preston:Yeah.
W. Curtis Preston:So, so let's talk about that a little bit.
W. Curtis Preston:So, I know Druva has role-based administration.
W. Curtis Preston:What other things did you do with that?
W. Curtis Preston:Like how, how did you divvy up responsibilities?
Albert Uy:yeah, so role based, so we'd just like to say database
Albert Uy:team can only restore database application, application server
Albert Uy:server, uh, site administrator site.
Albert Uy:. And the reason is, um, you know, we have data centers, but
Albert Uy:we also have remote offices.
Albert Uy:Uh, because of, um, government services.
Albert Uy:What we have is what you call a tl, uh, authorized to operate.
Albert Uy:So some of the files has to be in the remote office before we can
Albert Uy:put it in the cloud or data center.
Albert Uy:So that's when the site administrator come in and they
Albert Uy:restore whatever locally to their
W. Curtis Preston:Also when when you say site administrator, you mean
W. Curtis Preston:administrator of that particular
Albert Uy:office.
Albert Uy:Yes.
W. Curtis Preston:And what about, um, I know, I don't know if you've, if you've
W. Curtis Preston:done this, but what about separating.
W. Curtis Preston:Other things within the backup system.
W. Curtis Preston:Like some, I've seen some people where they're like, we, we give this
W. Curtis Preston:person the ability just to run the backups, but then they don't have
W. Curtis Preston:any ability to say delete backups.
W. Curtis Preston:And then, you know, we have another person who can configure backups,
W. Curtis Preston:but they can't restore backups.
W. Curtis Preston:I've seen some very interesting stuff.
W. Curtis Preston:Uh, can you think of anything else from the people that run the backup system?
W. Curtis Preston:Anything that you did?
Albert Uy:So, so the backup team, we used to call them, well, the team was called
Albert Uy:e edp, enterprise Data Protection Team.
Albert Uy:So they schedule a backup, so they do all the backup based
Albert Uy:on the project requirement.
W. Curtis Preston:Mm-hmm.
Albert Uy:So in the, on the roll base, um, the roll base are able
Albert Uy:to restore, but usually it's the EDP team that schedules the backup.
W. Curtis Preston:Mm-hmm.
Albert Uy:So on the database team, um, because of the project requirement
Albert Uy:based on the r p rtl, you know how many times you do full versus incremental.
Albert Uy:So they schedule it.
Albert Uy:And then one thing we made it easy is because we went to the cloud,
Albert Uy:we also, um, uh, created a tagging.
Albert Uy:Um, you know, you can create a tag on your cloud instances.
Albert Uy:So we have a, um, a tag called backup, and then they would put a value,
Albert Uy:uh, which is a string, and then they use regex to go to the string to
Albert Uy:say, Hey, how often do we want full?
Albert Uy:Uh, you know, is this a database?
Albert Uy:Is this a file?
Albert Uy:Is this so.
W. Curtis Preston:Mm-hmm.
W. Curtis Preston:. Mm-hmm.
W. Curtis Preston:. Mm-hmm.
Albert Uy:all the,
Prasanna Malaiyandi:without having to require some two people talking together.
Albert Uy:that's correct.
Albert Uy:That's correct.
W. Curtis Preston:Did you have anything?
W. Curtis Preston:What?
W. Curtis Preston:What would you do if you had a new resource?
W. Curtis Preston:Cuz I'm assuming you used like auto discovery and things like that.
W. Curtis Preston:If you had a new resource that needed to be backed up but someone
W. Curtis Preston:forgot to put a tag, did you, did you have anything for that?
Albert Uy:Yes.
Albert Uy:So, um, we have a delivery team.
Albert Uy:Um, so we automated everything with the help of Druva.
Albert Uy:We created Lambdas on aws, so for, um, for operational readiness.
Albert Uy:So when a project goes live in two weeks, we already checked
Albert Uy:two weeks prior to going live.
Albert Uy:Those tags have values, so everything's shared.
Albert Uy:So, did I answer question Curtis?
W. Curtis Preston:that warms my heart.
W. Curtis Preston:Uh, Albert the idea that someone would wait to go live on a project,
W. Curtis Preston:they would wait to make sure backups work before that project goes live.
W. Curtis Preston:That, that warms
W. Curtis Preston:my heart.
Albert Uy:And, and, and Curtis.
Albert Uy:With GRS Health, we create what you call backup of service.
Albert Uy:So every, every instance cloud instance being created has
Albert Uy:the site value of the backup.
Albert Uy:We already know, uh, if that instance will be backed up, if that
Albert Uy:instance will be patched, and if that instance will be monitored.
Albert Uy:So those three functions are in the tags, and we have a operational
Albert Uy:readiness team that goes in and makes sure those tags are values.
Albert Uy:And then we also have, uh, uh, account owner, account owners.
Albert Uy:We have technical point of contact, uh, for those project and for those instances.
Albert Uy:So everything's labeled, everything has to be labeled.
Albert Uy:So if not, uh, um, which we partner with the security team,
Albert Uy:we have what you call compliance.
Albert Uy:So before they go live, we make sure the compliance is a hundred
Albert Uy:percent before they go live.
Albert Uy:If not, uh, if not, we create, um, incident priorities.
Albert Uy:Um, we create, um, a day, um, well, two weeks before it goes live, we
Albert Uy:create a P two, and then one day before it goes live, it becomes a P one.
Albert Uy:In other words, you need to fix it before it goes live.
Prasanna Malaiyandi:now the two weeks before.
Prasanna Malaiyandi:Are when someone tags it.
Prasanna Malaiyandi:Are you actually doing backups at that point to make sure that the entire
Prasanna Malaiyandi:process and everything works, that things can be restored because it hasn't quite
Prasanna Malaiyandi:been pushed to production yet, correct?
Albert Uy:Right, right, right.
Albert Uy:So, so we follow, um, two SLAs, a corporate SLA and a project sla.
Albert Uy:So before you, you know, when we deliver everything, follow the
Albert Uy:corporate sla, and then if the project SLA is stricter than the corporate
Albert Uy:O overrides the corporate sla.
Prasanna Malaiyandi:Hmm.
Albert Uy:So, so when we deliver, we based on those sla, so those values
Albert Uy:are filled out based on those SLAs.
Albert Uy:Make
W. Curtis Preston:that.
W. Curtis Preston:I love, yeah.
W. Curtis Preston:I, you know, when I started again, a hundred years ago, when
W. Curtis Preston:I started backup sla sch, I mean, we didn't have anything like that.
W. Curtis Preston:I
W. Curtis Preston:mean, at, I mean, at best we did have, we had, we, we didn't, I, I
W. Curtis Preston:think I was in the backup business.
W. Curtis Preston:Probably five years or so before I heard the terms RTO and RPO
W. Curtis Preston:and by the way, I agree with you, that's where it starts, right back then
W. Curtis Preston:again, you'll remember this back then.
W. Curtis Preston:We spent so much of our time talking about like backup window
W. Curtis Preston:and um, you know, we didn't, we hardly talked about restore window.
W. Curtis Preston:We talked about backup window because that was the thing we had to do every
W. Curtis Preston:night, whether or not we could fit our backups within the backup window.
W. Curtis Preston:So this idea that you basically, you agree on the SLA first.
W. Curtis Preston:and then you design backups and other things, uh, to meet those SLAs and that
W. Curtis Preston:all of this is done as a cooperation between the business and the technology
W. Curtis Preston:folks before the server goes into, or the application goes into production
W. Curtis Preston:again, that this is what Prasanna isn't just what we talk about all the
W. Curtis Preston:time, right?
W. Curtis Preston:That.
W. Curtis Preston:That con that, that, that cooperation between the business and the Yeah.
W. Curtis Preston:The stakeholders and the, and the people.
W. Curtis Preston:Uh, or the, or the technology folks.
W. Curtis Preston:There, there's another thing by the, there was another group that you talked about,
W. Curtis Preston:and again, this warms my heart as well.
W. Curtis Preston:You talked about partnering with the security folks, um, you know,
W. Curtis Preston:in this world of ransomware.
Prasanna Malaiyandi:Hmm.
W. Curtis Preston:I think that partnership is more important than ever
W. Curtis Preston:before, um, that folks on the security side need to have some basic understanding
W. Curtis Preston:of what, what the way backups work.
W. Curtis Preston:And folks on the backup side need to have a basic understanding
W. Curtis Preston:of, you know, security.
W. Curtis Preston:Well, thi this concept that you talked about, you know,
W. Curtis Preston:role-based administration, that's a security concept, right?
W. Curtis Preston:Least, uh, least privilege is a security concept.
W. Curtis Preston:. And, um, so, so let me ask you this.
W. Curtis Preston:How have you, over, over the years, how have you dealt with one of the
W. Curtis Preston:problems that, that I see in it, which is the, the, the idea that nobody wants
W. Curtis Preston:to do the backups , so nobody wants to be in charge of the backup system.
W. Curtis Preston:Uh, how, how did you deal with that before and how was that any different, uh, when,
W. Curtis Preston:you know, when you, uh, went with Druva.
Albert Uy:Right.
Albert Uy:Well, going to the cloud is very helpful and since Druva is also cloud
Albert Uy:native, it's very helpful and one advantage of working in government
Albert Uy:services were highly regulated, right?
Albert Uy:So, so we, you know, because we're highly reg regulated, security
Albert Uy:kind of dictates what we need.
Albert Uy:So actually our operational readiness is monitored by, How about compliance?
Albert Uy:We monitor all the, all the security compliance and operational compliance.
Albert Uy:So before going live, so let's say if your backup choice has, you need backup
Albert Uy:agents, make sure backup agents, you know are installed, we check on that and
Albert Uy:then make sure the tag values for the backup agent is full, you know, field.
Albert Uy:And then same with patching, and same all that.
Albert Uy:So compliance is almost like our key, like you said, security, uh, and it
Albert Uy:covers both security and operational.
Prasanna Malaiyandi:and
W. Curtis Preston:what about get, getting the person to actually
W. Curtis Preston:getting somebody to actually take the responsibility for the backup process?
W. Curtis Preston:Is that, is that less of a problem for you in that, in that,
W. Curtis Preston:world?
Albert Uy:Um, yeah, because, uh, the delivery team, so whenever any
Albert Uy:instance, um, gets, um, created, it follows the corporate sla, everything.
Albert Uy:So all the engines install, everything's automated.
Albert Uy:Uh, the, the security agents installed, the monitoring agents
Albert Uy:installed, the backup agents installed.
Albert Uy:So everything's automated.
Albert Uy:And then what we do is we use the compliance report to go in and check, make
Albert Uy:sure those, um, those agents have, he.
Albert Uy:and make sure those agents also have values on their tagging.
Albert Uy:So it's all automated and we
W. Curtis Preston:like.
Albert Uy:actually Yep.
W. Curtis Preston:It sounds like it's part of a team rather than one person
W. Curtis Preston:that's just responsible for backups.
W. Curtis Preston:It sounds like the team is responsible for backups.
W. Curtis Preston:Is that okay?
W. Curtis Preston:That, I think that's perhaps one way to solve
Prasanna Malaiyandi:Yeah, and I'm guessing that having this automated
Prasanna Malaiyandi:process also helps you avoid sort of like the shadow IT issue that we heard so much
Prasanna Malaiyandi:about like five, seven years ago, right?
Prasanna Malaiyandi:Where a department's like, Hey, I need to spin up something, but I don't want
Prasanna Malaiyandi:to go through normal it, so I'm just gonna go swipe my credit card, get an
Prasanna Malaiyandi:AWS account, and start running right.
Albert Uy:Right, right.
W. Curtis Preston:Yeah.
W. Curtis Preston:And, and, and it sounds like I, I think that automated process was
W. Curtis Preston:also enabled by the way Druva works, you know, again, before and after.
W. Curtis Preston:W you know, you could automate to a certain degree, pick
W. Curtis Preston:your favorite backup product.
W. Curtis Preston:You can automate that to a certain degree with one major caveat,
W. Curtis Preston:and that is capacity, right?
W. Curtis Preston:Um, a, a backup server or, or, you know, cluster of backup
W. Curtis Preston:servers can only back up so much.
W. Curtis Preston:It can only handle it from a, from a comp, uh, compute perspective,
W. Curtis Preston:from a throughput perspective.
W. Curtis Preston:Most importantly, from a capacity perspective, when the bits
W. Curtis Preston:are full, the bits are full.
W. Curtis Preston:And you can't just, you can automate all you want.
W. Curtis Preston:Your backup system's just gonna be like, uh, I don't know where
W. Curtis Preston:you expect me to put this stuff.
W. Curtis Preston:Uh, you know, but with, but with Druva, you can a hundred percent
W. Curtis Preston:automate it because the compute and additional capacity that you need is
W. Curtis Preston:just automatically added to the system.
W. Curtis Preston:Does that, does that seem about.
Albert Uy:Yep, that's right.
Prasanna Malaiyandi:Just following up on that sort of the automation side, I think
Prasanna Malaiyandi:one thing I'm curious about Albert is.
Prasanna Malaiyandi:Enabling of this automation, the APIs, that a lot of, because previously,
Prasanna Malaiyandi:like Curtis was talking about, right?
Prasanna Malaiyandi:You went into a gooey or a UI and you manually clicked all these
Prasanna Malaiyandi:buttons and now you have sort of this automation side of things.
Prasanna Malaiyandi:How important are APIs as you start to look at this new world, right?
Prasanna Malaiyandi:And the enabling of integration of backups into these process.
Albert Uy:Um, APIs are very important and don't forget, there's different flavors.
Albert Uy:APIs, you have public APIs and private APIs, so most time you get
Albert Uy:public APIs, but you have to see if the public APIs support role-based,
Albert Uy:blah, blah, blah, everything, right?
Albert Uy:So APIs important.
Albert Uy:One thing though I wanna add is, uh, what's more important than API is culture.
Albert Uy:Like automation.
Albert Uy:So when we start implementing automation, everybody's like, Ooh, if
Albert Uy:I automated, I'm gonna lose my job.
Albert Uy:Right.
Albert Uy:So, so, so the one thing I did was promote the culture first before, you know, you
Albert Uy:have api, but people don't wanna use it.
Albert Uy:It's useless.
Albert Uy:Right.
Prasanna Malaiyandi:Yeah.
Albert Uy:So, and how I promoted Yeah.
Albert Uy:And how I promoted it.
Albert Uy:I, um, I create, you know, sometimes it's nice to mix fun with work.
Albert Uy:So I started with automation.
Prasanna Malaiyandi:Hmm.
Albert Uy:So, and I create a process in automation games.
Albert Uy:Hey, automation games is not just for developers.
Albert Uy:So I kind of come up with a, uh, idea board.
Albert Uy:So if, if, if a project manager have an idea, uh, and then other people will see
Albert Uy:the idea and say, Hey, I wanna join you.
Albert Uy:So the developer will have a developer joining him, have a QA joining him, or.
Albert Uy:and then together as a team, they create this automation.
Albert Uy:They, and then they submit it on the automation games and
Albert Uy:then, you know, and all that.
Albert Uy:So, so kind of created automation games to promote the culture,
Albert Uy:and then people get the point, Hey, this is actually helping me.
Albert Uy:Uh, I can do, I, I can spend my time doing more important
Albert Uy:stuff than just doing backup.
Albert Uy:Right?
Albert Uy:So,
W. Curtis Preston:Right.
W. Curtis Preston:Um, yeah, I like that a lot.
W. Curtis Preston:Be because, you know, you, you really, um, I flash back to a.
W. Curtis Preston:I remember, um, you know, a hundred years ago, uh, when I worked, and
W. Curtis Preston:here comes the name again, when I worked with Stewart, um, . It's a name
W. Curtis Preston:that comes up a lot on our podcast.
W. Curtis Preston:Uh, there was a, there was a guy at, at that company.
W. Curtis Preston:That his thing was, he was the NetWare administrator, remember NetWare.
Albert Uy:Yes, no.
W. Curtis Preston:And, and he had zero interest in modernizing the
W. Curtis Preston:infrastructure because it meant that he wasn't gonna have anything else to do.
W. Curtis Preston:, right?
W. Curtis Preston:Uh, that, that problem is.
W. Curtis Preston:I think that, I think you, you really hit the nail on the head that this,
W. Curtis Preston:this problem with au the, the problem with automation is people think,
W. Curtis Preston:well, if I fully automate the world, then I'm going to be out of a job.
W. Curtis Preston:And, and I would say that what you do is you automate the stuff that
W. Curtis Preston:you, that's just the mundane, the stuff that has to happen every day.
W. Curtis Preston:One perfect example is backup.
W. Curtis Preston:So that you can then do the interesting stuff, which is like, we can talk
W. Curtis Preston:about hunting for the bad guys, right?
W. Curtis Preston:We can talk about watching the environment.
W. Curtis Preston:Something that that can be assisted by technology.
W. Curtis Preston:Right?
W. Curtis Preston:And I know, I know Druva has that as well, where it has the posture and
W. Curtis Preston:observability features and it can.
W. Curtis Preston:Look for bad things going on.
W. Curtis Preston:But I think that that's also a very human thing to do is to, number one,
W. Curtis Preston:watch for the, for the bad actors.
W. Curtis Preston:And number two, um, continually, you know, you talked about that idea
W. Curtis Preston:board continually figuring out how we can make our security better, right?
W. Curtis Preston:Uh, because in in this world, there is, there, there has been no time where I
W. Curtis Preston:think cybersecurity is more important.
Albert Uy:Yes.
Albert Uy:Uh, actually I'm glad, uh, Curtis, you touch, uh, what people don't
Albert Uy:understand is there's um, um, there's three data situation.
Albert Uy:You call it data at res.
Albert Uy:Data in motion and data in use, right?
Albert Uy:So like you said, mostly when you're doing data backup, you wanted data at rest.
Albert Uy:So one thing we're looking at is actually we're looking at
Albert Uy:Druva is data resiliency, right?
Albert Uy:So that's when security and cybersecurity comes in.
Albert Uy:So you don't just bank it out when it's a data arrest.
Albert Uy:You also have to protect it during InMotion and uh, at use, right?
Albert Uy:So,
W. Curtis Preston:Right.
Albert Uy:so that's in terms of cyber.
Albert Uy:Uh, one thing I wanna mention too, on automation, um, One trick, um, because my
Albert Uy:pre my predecessors tried automation game it, you know, coupled them and it failed.
Albert Uy:Uh, one thing I also learned is you need to make it open.
Albert Uy:You know, the guy, the person before me said, Hey, okay, automation
Albert Uy:game, everybody has to use Ansible.
Albert Uy:But the thing is not everybody knows Ansible.
Albert Uy:So when I create automation, I open it.
Albert Uy:You can use dos batch file, you can use Lambda, you can use, uh, batch,
Albert Uy:uh, you know, batch script, and.
Albert Uy:and that's how it's successful.
Albert Uy:So, and, and then what, what we needed to do is find a tool that will support
Albert Uy:all open scripting or, or programming.
Albert Uy:So,
W. Curtis Preston:Nice.
W. Curtis Preston:Uh, yeah, I like that.
W. Curtis Preston:Yeah, because not, not everybody knows.
W. Curtis Preston:I mean, I remember back um, everybody was jumping on Pearl.
W. Curtis Preston:and all I knew was, all I knew was Bhe.
W. Curtis Preston:Right.
W. Curtis Preston:Um, I was an old Bhe guy and I, I learned Pearl and I, I wrote, I remember
W. Curtis Preston:writing, uh, just to learn Pearl.
W. Curtis Preston:I remember writing a monitoring program that, that created a webpage.
W. Curtis Preston:of how net backup was running at the time.
W. Curtis Preston:Right.
W. Curtis Preston:Um, this was, uh, I actually remember the first time that, that, that, that, uh,
W. Curtis Preston:system, I dunno what to call it, program, uh, the first time I went live was
W. Curtis Preston:actually at Amazon back in, back in 1998.
W. Curtis Preston:Uh, it was a long time ago.
W. Curtis Preston:Um, I, you, you mentioned earlier about how you made Dr.
W. Curtis Preston:Faster.
W. Curtis Preston:Um, when you talk about that 24 Dr.
W. Curtis Preston:Pro, 24 hour DR process, uh, talk to me about how, how
W. Curtis Preston:does that 24 hours break down?
Albert Uy:Um,
W. Curtis Preston:what, what does it consist of?
Albert Uy:yeah, because, uh, everybody was operating in, uh, silo.
Albert Uy:So the, the, the server team, uh, used a product called, um, at that time was
Albert Uy:using a product called CommVault, and then the database team was using SQL
Albert Uy:backup, so the old database between SQL backup and then the server team
Albert Uy:will use Combo to back up the sql.
Albert Uy:So when you do diver dr, you do reverse, right?
Albert Uy:So the server team comes in and restore the combo file, and then
Albert Uy:after that, the database team has to use that file and, and, and, and
Albert Uy:restore it into a SQL database, right?
Albert Uy:So that's like a, just the process alone is 72 hours.
Albert Uy:But since they, we created the process.
Albert Uy:So instead of using SQL backup, it's just one file.
Albert Uy:So when the server team restore it, it's already the.
Albert Uy:You don't need the database team to come and restore this file into
Albert Uy:a SQL database, if that makes.
W. Curtis Preston:you, you made the actual restore process itself.
W. Curtis Preston:You, you took out steps in the restore process itself.
Albert Uy:Right, right.
Albert Uy:And then it cut costs to this.
Albert Uy:And I was talking about at that time there were no cloud yet.
Albert Uy:So when we are doing VR exercise, um, we had to fly in everybody.
Albert Uy:The file backup team, the database backup team, you're
Albert Uy:talking about a good 15 people.
Albert Uy:So, uh, 15 people, um, not doing their day-to-day job, just
Albert Uy:doing the DR for three days, so,
W. Curtis Preston:Right, right.
W. Curtis Preston:Yeah.
W. Curtis Preston:We, we did that back in the day.
W. Curtis Preston:We would, we didn't have to fly anybody in cuz we were all in the same place.
W. Curtis Preston:But, but we did it over the weekend cuz we were a bank.
W. Curtis Preston:Um, so it's not like we're gonna take the bank down for a few days.
W. Curtis Preston:We did it over the weekend.
W. Curtis Preston:. Um, and, um, we would, uh, one of the, one of the rules that we
W. Curtis Preston:had back then was that the person responsible for the backups, which
W. Curtis Preston:was me, was not to run the Dr.
W. Curtis Preston:right.
W. Curtis Preston:Um, because the idea is, it's supposed to be properly documented enough that the,
W. Curtis Preston:the other person, you know, that another person that's capable should be able
W. Curtis Preston:to follow the documentation and do it.
W. Curtis Preston:I will tell you, in the three years that I was there, not once did we
W. Curtis Preston:successfully do the , the restore from beginning to end without having
W. Curtis Preston:to come in and, and, and help.
W. Curtis Preston:I mean, it did work.
W. Curtis Preston:It's just.
W. Curtis Preston:You know, the, the process was so complicated and so convoluted that,
W. Curtis Preston:um, you can only document it to a certain point and then, right.
W. Curtis Preston:And, and so I, I, I do think that's, you know, you talked about making
W. Curtis Preston:it so much simpler with a, with a single step restore process.
W. Curtis Preston:That seems like that would be the, the biggest boon to
W. Curtis Preston:getting things restored in.
Prasanna Malaiyandi:it's interesting that sometimes people forget about that.
Prasanna Malaiyandi:Instead of looking at how do we document this process?
Prasanna Malaiyandi:It should be, how do we simplify this Right.
Prasanna Malaiyandi:and start thinking about that.
W. Curtis Preston:Yeah.
W. Curtis Preston:How can we make this process so simple that documentation is like,
W. Curtis Preston:pull up the UI and log in, or, or
Prasanna Malaiyandi:Well, well here, well here's a, here's a question for you.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Whenever you go buy a product today, how often or do you go and like open
Prasanna Malaiyandi:the instruction manual, read through the instruction manual before using it?
Prasanna Malaiyandi:Right, Curtis?
Prasanna Malaiyandi:Or you just got a Roomba, right?
Prasanna Malaiyandi:Did you actually read the instruction manual or were you
Prasanna Malaiyandi:like, oh, I'm just gonna use it?
W. Curtis Preston:Well, that was the quick start card that I did need because
W. Curtis Preston:there, there were some steps that I had to, but it was just a two-sided of a card.
W. Curtis Preston:Right.
W. Curtis Preston:Um, but you, you, you did make me think about, again, in this process we bought
W. Curtis Preston:a new, um, we bought a new dishwasher.
W. Curtis Preston:And there was this, I, I was putting in the dishwasher and I had put it in
W. Curtis Preston:essentially without, without really, I mean, I had looked at the manual a little
W. Curtis Preston:bit and, and, and I had it installed.
W. Curtis Preston:And then we wanted to, um, I wanted to level it and it was level, but I
W. Curtis Preston:wanted to raise the level of it so that it fit better in the, in the,
W. Curtis Preston:in the uh, counter and.
W. Curtis Preston:I was doing it.
W. Curtis Preston:Like, I was like, well how hard could it be?
W. Curtis Preston:You just turned the things right.
W. Curtis Preston:Um, right.
W. Curtis Preston:And my wife in the middle of this and I was getting frustrated cuz it wasn't
W. Curtis Preston:working the way I would expect it to work.
W. Curtis Preston:And my wife was like, have you looked at the manual?
W. Curtis Preston:And I got mad.
W. Curtis Preston:I was like, dang, I don't need the manual to turn some screws.
W. Curtis Preston:And it turned out I looked at the manual and I was doing it all
W. Curtis Preston:wrong because they had changed.
W. Curtis Preston:They had changed, uh, Yeah, so the manuals can be helpful, but
W. Curtis Preston:yeah, the process should be simple enough that you can do that.
W. Curtis Preston:Uh, so you know, Albert, I wanna thank you a lot for coming on.
W. Curtis Preston:Is there anything that we, that maybe you wanted to talk about that
W. Curtis Preston:we, that we haven't covered yet?
Albert Uy:If I could go back to what you said about how backup has progress.
Albert Uy:So if you remember 20 years ago, there's no cloud.
Albert Uy:So, and then most of the time you change your DR provider every two years.
Albert Uy:Right?
Albert Uy:You know, popular ones are like Sunguard and ibm.
Albert Uy:20 plus years ago, if you go with a provider, uh, you say,
Albert Uy:how many servers do you need?
Albert Uy:So 20 years ago you send this team going in and then they installed
Albert Uy:the os, they installed the app.
Albert Uy:and then 10 years later, uh, I don't know if you heard, they're
Albert Uy:searching as bare metal restoration.
Albert Uy:So you, so I did that too.
Albert Uy:So what you do is you put a bare metal orchestrator on that location.
Albert Uy:Uh, you, you, you, you say how many servers and then, you know, based on the
Albert Uy:nick, um, then you have to spare bare metal restoration to restore it, right?
Albert Uy:And then now today you got the cloud.
Albert Uy:So it's gets simpler, simpler, uh, and faster.
W. Curtis Preston:Yeah, because with, with the cloud doing Dr.
W. Curtis Preston:I, I've always thought that Dr.
W. Curtis Preston:Cuz you know, some people are like, oh, he says that cuz he works for Druva.
W. Curtis Preston:I, I've only worked for Druva for five years, I've said this for many years.
W. Curtis Preston:And that is that.
W. Curtis Preston:The Dr.
W. Curtis Preston:The, the DR is the killer app for the cloud.
W. Curtis Preston:Because what you want is you want, uh, you know, a thousand servers right now.
W. Curtis Preston:and you don't wanna pay for them until they arrive.
W. Curtis Preston:Right.
W. Curtis Preston:Until you need them.
W. Curtis Preston:And, but with, with a, with a product like Druva, you can do the
W. Curtis Preston:restore part in advance, right?
W. Curtis Preston:And then it just automatically brings the servers in when it, when it needs.
W. Curtis Preston:And, you know, you talk about a simple DR process, it's just literally a one button
W. Curtis Preston:and then it automates all of that stuff.
W. Curtis Preston:Bringing all those servers online, all of the network settings, all of that stuff.
W. Curtis Preston:. Um, so yeah, I, I, I think if, if, if you haven't experienced DR in the
W. Curtis Preston:Cloud, uh, I'd say give it a try.
W. Curtis Preston:If you're pr, if the product you have currently offers it, give that a shot.
W. Curtis Preston:If it doesn't, uh, you know, they'd be happy, happy to talk to you over at
W. Curtis Preston:Druva, uh, to see how easy Dr can be.
W. Curtis Preston:Uh, and, and I'd also say cloud, it makes it so much less expensive
W. Curtis Preston:that it can bring Dr down to.
W. Curtis Preston:The masses, if you will, right?
W. Curtis Preston:It used to be it was only the, the companies who had lots of money that
W. Curtis Preston:could afford the sun guards, right?
W. Curtis Preston:This, this is literally companies of any size should be able to
W. Curtis Preston:automate their DR environment.
Albert Uy:And, and Curtis, I would like to add has to be a
Albert Uy:hundred percent cloud native.
Albert Uy:So just to share with, that's reason why we went with Druva.
Albert Uy:So when I was at Maximus, um, they had a project to go from on-premise to cloud.
Albert Uy:. So the backup team was reporting to the engineering team.
Albert Uy:At that time, I was managing the operations team.
Albert Uy:They had two years to do it.
Albert Uy:after 18 months, they just did one remote location and, and then they gave
Albert Uy:me the team and say, Hey, finish it.
Albert Uy:And I got six months left from the 24 months.
Albert Uy:And you know, you're talking about several remote sites and big data centers.
Albert Uy:So they said, here's the solution, implement six months.
Albert Uy:I'm like, okay, you tried it for for 18 months and you couldn't do
Albert Uy:it and you're giving me six months.
Albert Uy:Fortunately I was technical and I look at the solution, I'm
Albert Uy:like, this is not cloud native.
Albert Uy:This is hybrid.
Albert Uy:And that's the reason why it's taking forever.
Albert Uy:Because when you talk about hybrid, that means including pro premise and on
Albert Uy:premise equates to you need provisioning.
Albert Uy:So, you know, I spent two months using their solutions that
Albert Uy:there's no way this is gonna work.
Albert Uy:I only have four months left and I have like so many sites and
Albert Uy:because I had Druva experience in my previous job, which is with Gamestop.
Albert Uy:I went in to the executive team and said, Hey, if you want me to get
Albert Uy:this done in four months, uh, uh, we need to go with my new solution.
Albert Uy:And they gave me a thumbs up and I was able to do it in four months.
Albert Uy:And how I did it in four months is because I know how Druva works I've spent the
Albert Uy:first month with, uh, what I call A P O V, I don't call it poc, proof of value.
W. Curtis Preston:Yep.
Albert Uy:And I had the Druva team help me.
Albert Uy:We created playbooks.
Albert Uy:So all I needed to do is I know the solution works, I just create the.
Albert Uy:A playbook for database, a playbook for application, a playbook for
Albert Uy:server, a playbook for sites, right?
Albert Uy:Spend a whole month creating the playbook and then project
Albert Uy:management team help PMO help me come in and spend the three months.
Albert Uy:And actually I told 'em I don't want to get in done in three months.
Albert Uy:I wanna get it done in two months.
Albert Uy:Because if you give them three months, they're gonna use three months.
Albert Uy:Right?
W. Curtis Preston:right,
Albert Uy:So, so I only have four months left.
Albert Uy:One month for p.
Albert Uy:Three months employment, but I told him two months and we were
Albert Uy:done in two and a half months.
Albert Uy:So two weeks before the deadline, because of expensive, on-premise, uh, license
Albert Uy:renewal, um, we just, we had time to check and make sure everything works.
Albert Uy:So we got it completed in four months.
Albert Uy:Using the solution.
Albert Uy:So just share.
Albert Uy:And I did apply the people process tools.
Albert Uy:I know the technology works.
Albert Uy:Now I have to figure out the people, the database team, the server team,
Albert Uy:the role base, like you said, in creative process, how we can make all
Albert Uy:of them work together and how everybody follow the same playbook and all sort.
Albert Uy:So
Prasanna Malaiyandi:Nice.
W. Curtis Preston:I like it.
W. Curtis Preston:I like it.
Albert Uy:Right.
Albert Uy:Just to share
W. Curtis Preston:Thanks so much for, thanks so, so much
W. Curtis Preston:for coming on the podcast
Albert Uy:uh, thank you for inviting me.
Albert Uy:Uh, pleasure meeting.
W. Curtis Preston:and Prasanna.
W. Curtis Preston:Thanks again and thanks for helping me celebrate my little.
Prasanna Malaiyandi:Woo-hoo.
Prasanna Malaiyandi:Nice to meet you, Albert.
Albert Uy:Likewise.
Albert Uy:Thank you.
Albert Uy:Happy holidays.
W. Curtis Preston:yeah, absolutely.
W. Curtis Preston:Happy holidays and thanks to the listeners.
W. Curtis Preston:Remember to subscribe so that you can restore it all.
Listen On
