Getting approval for a new backup design (Backup to Basics Series)
We say it all the time: your backup requirements must be based on business/organizational needs. If you keep that mind, not only will your organization be better off, it'll be easier to get approval for the new backup system you want. This episode of our Backup to Basics series goes right to that core idea, and we get advice directly from one of our favorite guests: Jeff Rochlin. He is now the Head of Technical Operations at the LA Studio of Framestore, a visual effects company. In his 30+ year career he has designed and implemented dozens of systems (at least one of them with Mr. Backup). He shares with us his tips on how to design the system AND how to get it approved.
If you'd like to see what Jeff and Curtis do for fun, check out their other podcast, The Things That Entertain Us.
Mentioned in this episode:
Interview ad
â W. Curtis Preston: hi, and welcome to
Speaker:I'm a host w Curtis Preston, aka Mr.
Speaker:Backup, and I have with me my software troubles consultant, Prasanna Malaiyandi.
Speaker:How's it going, Prasanna?
Prasanna Malaiyandi:I'm good.
Prasanna Malaiyandi:I'm good Curtis.
Prasanna Malaiyandi:And like software troubles.
Prasanna Malaiyandi:You should expect it.
Prasanna Malaiyandi:You've been in the industry for how long now?
Prasanna Malaiyandi:Like it
W. Curtis Preston:I know, but I, I, you know, that I generally don't
W. Curtis Preston:like, I generally, so like, I like tech and then like, I like working
W. Curtis Preston:on tech, but then I like using tech and the tech that I'm using.
W. Curtis Preston:I just want it to work.
W. Curtis Preston:I'm not TiVo anymore, but you know, when I had the TiVo, that's why I
W. Curtis Preston:opted for TiVo and not Linux box.
W. Curtis Preston:With hard drive in some software that will record my TV stream, right?
W. Curtis Preston:Um, the stuff that I actually use on a day to day basis.
W. Curtis Preston:I want to just, I want it to just work.
W. Curtis Preston:And I've broken that trend by, uh, upgrading a tool that I love, right?
W. Curtis Preston:I love descript, but I've decided to go on the bleeding edge with
W. Curtis Preston:their new version, which is amazing.
W. Curtis Preston:But it's in beta and I'm and I'm using a beta tool for the first time.
W. Curtis Preston:And I'm like, I don't know.
W. Curtis Preston:I push this button and it doesn't, it doesn't do the thing.
W. Curtis Preston:Right.
W. Curtis Preston:And then, yeah,
Prasanna Malaiyandi:But, but, but at least you have to hand it
Prasanna Malaiyandi:to the descript support folks for actually like, taking your calls and
Prasanna Malaiyandi:your chats and your emails, Right.
Prasanna Malaiyandi:And be like, Hey, let me walk you through that.
W. Curtis Preston:they're good with live chat.
W. Curtis Preston:Although there's been a handful where.
W. Curtis Preston:Where at some point they're like, you know, I think it's
W. Curtis Preston:best to transfer this to email.
W. Curtis Preston:That's when you know, you've put, you've asked like one too
W. Curtis Preston:many questions in a support chat.
W. Curtis Preston:They're like, I think maybe you should put all your questions in an email.
W. Curtis Preston:I'm like, But that's not, that's not how my brain works.
W. Curtis Preston:Like I, I get a question, I get it answered.
W. Curtis Preston:I work for a little bit a while and then I get another question, you know?
W. Curtis Preston:Um, yeah.
W. Curtis Preston:Anyway, so
Prasanna Malaiyandi:sometimes it unblocks you and then you like hit the next
Prasanna Malaiyandi:thing and you're like, Oh, what is this?
Prasanna Malaiyandi:Which is why if you're a good product manager, right, you go in front
Prasanna Malaiyandi:of customers and you're like, Hey, show me what you're doing and let me
Prasanna Malaiyandi:figure out like what your problems are, what your pain points are,
Prasanna Malaiyandi:what features I should be building?
W. Curtis Preston:Yeah.
W. Curtis Preston:And, and it is, it is kind of funny though that I now, I'm like
W. Curtis Preston:on a first name basis with like four or so of their, of their, uh,
Prasanna Malaiyandi:shout out to the descript support team
W. Curtis Preston:Yeah.
W. Curtis Preston:Shout out to for putting up with my, uh, my questions.
W. Curtis Preston:Cause it's like, it's a combination of, the new version is significantly
W. Curtis Preston:different than the old version.
W. Curtis Preston:Um, and so it's a, it's a combination between Is this the way it's
W. Curtis Preston:supposed to work now or is this the way it's supposed to work?
Prasanna Malaiyandi:Well, and
Prasanna Malaiyandi:I
W. Curtis Preston:think it's broken.
Prasanna Malaiyandi:And I think for a lot of the features, right,
Prasanna Malaiyandi:they're building the right things.
Prasanna Malaiyandi:Like just talking to you like they've optimized some of your flows and some of
Prasanna Malaiyandi:the things that you've spent so much time working on in the past, right?
Prasanna Malaiyandi:So it's going to get there, It's just, it's gonna take a little bit of time.
W. Curtis Preston:But it's been, it's been a difficult
W. Curtis Preston:couple of weeks, you know?
W. Curtis Preston:Um, anyway, let's bring on our longtime friend.
W. Curtis Preston:Uh, I, I've known him for, I don't know, for forever, and he's for, he's
W. Curtis Preston:been on the podcast multiple times.
W. Curtis Preston:You may remember him from, from such hits as how does Hollywood do backup?
W. Curtis Preston:And, uh, he is also.
W. Curtis Preston:A, uh, a co-author on my book, uh, latest book, uh, Modern Data
W. Curtis Preston:Protection, which by the way, you can get free by going to druva.com/ebook.
W. Curtis Preston:And, um, he wrote the chapter that we're gonna talk about today
W. Curtis Preston:in our Back to basic series.
W. Curtis Preston:He is now the head of Technical Operations for Frame Store, a
W. Curtis Preston:visual effects studio in Hollywood.
W. Curtis Preston:Welcome to the podcast Jeff Rochlin.
Jeff Rochlin:Hey.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:. . Woo-hoo.
Jeff Rochlin:Hi guys.
Jeff Rochlin:How are ya?
W. Curtis Preston:How's it going, Jeff?
Jeff Rochlin:It's going.
Jeff Rochlin:It's going well.
Jeff Rochlin:It's going.
W. Curtis Preston:It, it's, uh, it, it is it nice to be back?
W. Curtis Preston:Cuz I know for a while you weren't, you weren't, you, you've always
W. Curtis Preston:enjoyed sort of combining the tech side of you and the, and the, you
W. Curtis Preston:know, the entertainment side of you.
W. Curtis Preston:And for a while you weren't in that space Right now it's good to be
W. Curtis Preston:back with, you're a full person.
W. Curtis Preston:You get to be in entertainment and be in tech at the same time.
Jeff Rochlin:definitely.
Jeff Rochlin:Um, I love the creative.
Jeff Rochlin:Part of it.
Jeff Rochlin:And I love dealing with the crazy people that make creative things.
Jeff Rochlin:And so it's, that's, that's why I've spent, you know, the better part
Jeff Rochlin:of the last 30 years working at, um, uh, some kind of movie studios,
Jeff Rochlin:preferably in production and not, you know, it cause anybody, this will get
Jeff Rochlin:me in trouble, anybody can do email.
Jeff Rochlin:But, you know, dealing, dealing with the, dealing with the executive
Jeff Rochlin:producer who's trying to get his movie out on time is a different kind of,
Prasanna Malaiyandi:Yeah, I can only imagine the pressure with that
Jeff Rochlin:Um, yeah, the, the only place I've worked in the past that
Jeff Rochlin:where the pressure was worse in a different kind of way was Wall Street.
Jeff Rochlin:Working on a trading floor in Wall Street is a different piece also,
Jeff Rochlin:but
W. Curtis Preston:cuz that's where like you're, you know, you're making a
W. Curtis Preston:trade or you're trying to make a trade or you're trying to make thousands of
W. Curtis Preston:trades and suddenly the system goes down.
Jeff Rochlin:Right?
Jeff Rochlin:And it's, and it's all time critical, right?
Jeff Rochlin:Because the price is not sitting static waiting for you to close the trade.
Jeff Rochlin:If you don't buy it right now, then five minutes from now it's gonna be, you know,
Jeff Rochlin:a hundred million dollars more in cost and there goes your profit margin, right?
Jeff Rochlin:So,
W. Curtis Preston:Yeah.
W. Curtis Preston:Which is why, at least in in my IT experience, and I spent a, I spent a good
W. Curtis Preston:time in and outta Wall Street and working with other consultants that I knew that
W. Curtis Preston:were working on Wall Street, and I, I got the sense, it was the one place that I've
W. Curtis Preston:seen where it's like money is no object.
W. Curtis Preston:So they do, they do what is right.
W. Curtis Preston:To, um, to have fully, highly available systems.
W. Curtis Preston:Um, and I, I'd say Hollywood might be right behind that, right?
W. Curtis Preston:Because you've got all these super expensive actors, um, and super expensive,
W. Curtis Preston:you know, both above and below the line.
W. Curtis Preston:People see, I learned from you, I learned that
Jeff Rochlin:You did.
Jeff Rochlin:you.
Jeff Rochlin:did.
W. Curtis Preston:Uh, and, and they're all sitting there and, and
W. Curtis Preston:you know, and if, and if they can't do it because the computers are down,
Prasanna Malaiyandi:what?
Prasanna Malaiyandi:What's above and below the lo line?
Prasanna Malaiyandi:By the way,
W. Curtis Preston:oh.
Jeff Rochlin:anybody in the creative titles, actors, directors, producers,
Jeff Rochlin:writers are called Above the line.
Jeff Rochlin:Um, anybody in the skilled trades and the crew, right?
Jeff Rochlin:The, the camera, the, uh, the gaffer, who's the electrician, uh, you know,
Jeff Rochlin:those kind of got the stage hands.
Jeff Rochlin:Those are below the line.
Jeff Rochlin:Um, and the best way to describe it is, I guess if you watch the credits of a
Jeff Rochlin:movie, um, at the end, you know, they always run the, the cast gets their,
Jeff Rochlin:their individual cards and the people who wrote the movie get their cards,
Jeff Rochlin:and then it goes into that long scroll.
Jeff Rochlin:Of everybody else that's kind of above and below the line, you know, visually.
Jeff Rochlin:Um,
Prasanna Malaiyandi:Interesting.
Prasanna Malaiyandi:Yeah.
Jeff Rochlin:so Hollywood is kind of always walking that middle line between,
Jeff Rochlin:I wanna, I want perfection and I really don't want to have to pay for it.
Jeff Rochlin:You know what I mean?
Jeff Rochlin:Which is sort of what the rest of the world is doing as well.
Jeff Rochlin:Um, how can I strike the balance so that I know I'll get a hundred percent of what
Jeff Rochlin:I need and yet at the same time, not have to spend the money to, to make it happen?
Jeff Rochlin:Uh, backups are a, backups and storage in a production are a
Jeff Rochlin:perfect example of this, right?
Jeff Rochlin:Because artists iterate their work over and over and over again.
Jeff Rochlin:They create a lot of files at the end of the day that they don't need because.
Jeff Rochlin:They've done, you know, they've done 20 versions of the scene and the
Jeff Rochlin:21st is the one they're gonna use.
Jeff Rochlin:Um, and then you go back to them and say, We're out of space.
Jeff Rochlin:Go clean up.
Jeff Rochlin:And they look at you like, Well, I don't have time to clean up.
Jeff Rochlin:I'm busy making art.
Jeff Rochlin:If you want somebody to clean up, then I can't make art.
Jeff Rochlin:Right.
Jeff Rochlin:. And you're looking at 'em and going, Well, I can go spend a hundred
Jeff Rochlin:thousand dollars and buy more storage for you to put more stuff on.
Jeff Rochlin:But eventually somebody's gotta clean this up cuz you don't need that storage.
Jeff Rochlin:Um, and that's been a kind of a standard theme throughout my career in Hollywood.
Jeff Rochlin:Um, they, they are upset when it's broken.
Jeff Rochlin:They, it costs them a lot of extra money to make up for it when it's broken, right?
Jeff Rochlin:Because over time on skilled trade and union people is not cheap.
Jeff Rochlin:Um, and so we do everything we can to make sure that it doesn't break..
Prasanna Malaiyandi:And I'm guessing, like especially with Hollywood, right,
Prasanna Malaiyandi:when you're producing or creating right, uh, content, right, Those are
Prasanna Malaiyandi:like significantly large files, right?
Prasanna Malaiyandi:Uh, like you said, if you're doing like 21 versions of it, right?
Prasanna Malaiyandi:The only other industries I could really think of that are similar is
Prasanna Malaiyandi:probably things in like the, uh, genome space or the mining space, right?
Prasanna Malaiyandi:Where you just have a whole bunch of data all coming in at once, but.
Jeff Rochlin:And we're in, uh, from a, like a rendering, the, the final
Jeff Rochlin:stage of production compute process.
Jeff Rochlin:It's the same kind of technology being used, right?
Jeff Rochlin:We run supercomputing clusters on the back end to render large projects, uh,
Jeff Rochlin:just like oil and gas, use supercomputing clusters to analyze data to figure
Jeff Rochlin:out where to drill next, right?
Jeff Rochlin:Um, and as the, the beautiful picture that you get on your TV at home gets
Jeff Rochlin:better and better when it moves from two K to 4k to eight K to 16 K plugged
Jeff Rochlin:directly into the headset that's connected into your cerebral cortex so that you're
Jeff Rochlin:living in the picture , that the files get exponentially larger every time.
Jeff Rochlin:Right.
Jeff Rochlin:Um, and I think I did the math for you a long time ago where, think about a.
Jeff Rochlin:From the perspective of there are 23 frames, still pictures in every
Jeff Rochlin:second of the film, and the film averages two and a half hours.
Jeff Rochlin:Right.
Jeff Rochlin:That's a lot of, that's a lot of images,
W. Curtis Preston:Right.
W. Curtis Preston:And, and, you know, and, and I'm glad, I'm glad we sort of started with this
W. Curtis Preston:part of the discussion because what we wanted to talk about, you know, this is
W. Curtis Preston:our back to basic series and I'll take this opportunity to do our disclaimer,
W. Curtis Preston:Um, uh, Prasanna and I and Jeff, we all work for different companies.
W. Curtis Preston:I work for Druva, Prasanna works for Zoom, and Jeff works for Frame Store.
W. Curtis Preston:We're not speaking for our companies.
W. Curtis Preston:We're speaking, uh, our individual, um, uh, opinions.
W. Curtis Preston:And uh, also if you would like to rate us, we'd love you to do that.
W. Curtis Preston:We're.
W. Curtis Preston:Running our Santa Claus special . If you, if you want to see this beard get
W. Curtis Preston:longer and longer to turn into a Santa Claus beard by Christmas, then uh,
W. Curtis Preston:you know, you gotta get us a few more comments in, uh, specifically in iTunes.
W. Curtis Preston:So go to the iTunes, just scroll to the bottom, you know, click on and give us
W. Curtis Preston:a, give us a rating, give us a comment.
W. Curtis Preston:And, um, if you'd like to be a part of this conversation, please reach
W. Curtis Preston:out to me at WC preston on Twitter or w Curtis Preston on gmail.
W. Curtis Preston:And, um, so we, we've been doing this back to the basics, uh, series, Jeff.
W. Curtis Preston:And, uh, so basically I'm, I'm pulling out, you know, my favorite book, Modern
W. Curtis Preston:Modern Data Protection, which you may recall having, having a part in.
W. Curtis Preston:And, um, we already did chapter one, it was Why We back up.
W. Curtis Preston:Right, Right.
W. Curtis Preston:Risks, what are all the things that, that could do damage to data.
W. Curtis Preston:and what I really want, you know, we talk a lot about, in the backup space, we
W. Curtis Preston:talk a lot about, um, the fact that too many times, those of us in technology,
W. Curtis Preston:we focus only on the technology, and then we find ourself asking questions, kind of
W. Curtis Preston:like you were talking about the stores.
W. Curtis Preston:Well, what should my retention be?
W. Curtis Preston:Right?
W. Curtis Preston:How long should I keep my backups?
W. Curtis Preston:How long, how, how often should I do backups?
W. Curtis Preston:Right?
W. Curtis Preston:What, So, and you know, and how quickly should I backup and how
W. Curtis Preston:quickly should I be able to restore?
W. Curtis Preston:And what I'm constantly telling people is, you should not be
W. Curtis Preston:the one answering this question.
W. Curtis Preston:You should always go back to the business because the business is
W. Curtis Preston:why you are doing what you're doing.
W. Curtis Preston:And so, Chapter two is about, um, gathering and determining
W. Curtis Preston:your service levels.
W. Curtis Preston:we have to realize what our organization does, whether it's
W. Curtis Preston:a government organization or a business organization, and I think
W. Curtis Preston:you always understood that role.
Jeff Rochlin:Um, yeah.
Jeff Rochlin:Well, so one of the things that we do as workers at any particular company we
Jeff Rochlin:go to is we focus on the job we have to.
Jeff Rochlin:Learning everything we need to learn about the job and, and understanding
Jeff Rochlin:that so we could be good at it, right?
Jeff Rochlin:But very few people started a company and then sit down and say, I wanna
Jeff Rochlin:understand the product we produce.
Jeff Rochlin:Right?
Jeff Rochlin:I wanna understand what the marketing is all about.
Jeff Rochlin:I wanna understand everything this business does because I don't just see
Jeff Rochlin:myself as a worker for the company.
Jeff Rochlin:I see myself as part owner of this business, right?
Jeff Rochlin:I'm, I'm a critical cog in the wheel that makes the place produce something.
Jeff Rochlin:And so that's always been my approach to my jobs.
Jeff Rochlin:It's not good enough to just know I'm a sysadmin.
Jeff Rochlin:I have to be able to write scripts to run this Unix box.
Jeff Rochlin:Um, I wanna understand that we rent cars to people all
Jeff Rochlin:over the, all over the world.
Jeff Rochlin:And there's a process involved in how that works or.
Jeff Rochlin:When I worked in, in broadcasting, right, I took the time to understand what
Jeff Rochlin:the actual broadcast chain is, right?
Jeff Rochlin:We create an image, it goes into a bunch of systems before
Jeff Rochlin:it goes out over the air.
Jeff Rochlin:It passes through a dozen different things that layer different sub carriers on it.
Jeff Rochlin:Like your Nielsen ratings box, right?
Jeff Rochlin:There's a, there's a subcarrier running on the signal that tweaks
Jeff Rochlin:your Nielsen box that says, Hey, he's watching Channel nine, right?
Jeff Rochlin:Um, and I wanted to understand all of that right up to the, the
Jeff Rochlin:transmitter, because I'm crazy that way.
Jeff Rochlin:Um, and what you Yeah, exactly right.
Jeff Rochlin:Um, and what, what you find as you become that interested in the business
Jeff Rochlin:also is that the people who are running the business part of the business, they
Jeff Rochlin:don't know anything about technology.
Jeff Rochlin:It could be a technology company.
Jeff Rochlin:The guy who's running the business isn't thinking about
Jeff Rochlin:how the engine actually operates.
Jeff Rochlin:He just wants the engine moving fast.
W. Curtis Preston:It's kind of like, kind of like it in the earlier
W. Curtis Preston:part, you know, when I was saying I, I just want my technology to work.
W. Curtis Preston:That's the way they are with tech.
Jeff Rochlin:exactly.
Jeff Rochlin:Exactly.
Jeff Rochlin:And they, and they don't really care about the details.
Jeff Rochlin:It's not important to them.
Jeff Rochlin:What they care about is they're gonna get what they expect on the outside
Jeff Rochlin:of the business right now because it people being who we are, think
Jeff Rochlin:that the technology is the do all end, all of what makes the business
Jeff Rochlin:work, especially in big corporations.
Jeff Rochlin:IT departments tend to do it for the sake of doing it.
Jeff Rochlin:Right.
Jeff Rochlin:They make systems for everybody else to use that are complicated,
Jeff Rochlin:uh, because they work really well inside the machine, right?
Jeff Rochlin:They take advantage of faster databases, faster storage, so
Jeff Rochlin:that things on the back end, which are complicated work really well.
Jeff Rochlin:But then when you and I have to go, you know, fill out our payroll record, you get
Jeff Rochlin:this screen that looks like it's something from 1954 off of an IBM mainframe, right?
Jeff Rochlin:In a world where.
W. Curtis Preston:possibly is
Jeff Rochlin:Uh, true, but, but you're in a world where everything is,
Jeff Rochlin:um, you know, graphical interfaces.
Jeff Rochlin:There's no reason why it has to be that way, and that's why I've always
Jeff Rochlin:taken it, that you have to understand what the business is doing, and you
Jeff Rochlin:have to understand what the people in the business are expecting the
Jeff Rochlin:business to do in order to be able to deliver to them effective systems.
W. Curtis Preston:I, I've always had that similar opinion and, and it could
W. Curtis Preston:be because my very first technology job when I was at MBNA, the bank that
W. Curtis Preston:everybody of a certain level had to spend four hours a month as a customer
W. Curtis Preston:support rep talking to cardholders,
Jeff Rochlin:that's cool.
Jeff Rochlin:That's really great.
W. Curtis Preston:and, and it really gave you a sense of like,
W. Curtis Preston:this is what we were about.
W. Curtis Preston:Right?
W. Curtis Preston:We are about the cardholder out there spending money that we are lending
W. Curtis Preston:them because that's how a credit card company is judged, is the outstanding
W. Curtis Preston:balance of everybody added all up, which in that world was 35 billion.
W. Curtis Preston:Right?
W. Curtis Preston:Which is just staggering to think about it.
Jeff Rochlin:the, the old adage, the customer is always right, is absolutely
Jeff Rochlin:true all of the time, even when they're wrong and there are pains in your butt.
Jeff Rochlin:Um, because if you're not going in with that attitude, Then people are gonna
Jeff Rochlin:say, Well, especially nowadays, right?
Jeff Rochlin:When there's a big in different choices, they're gonna say, Well, screw this.
Jeff Rochlin:I'll just go somewhere else.
Jeff Rochlin:I don't need,
W. Curtis Preston:We talk about building a framework and building review
W. Curtis Preston:and advisory boards in that chapter.
W. Curtis Preston:What, what is, what is that?
Jeff Rochlin:First of all, um, think about.
Jeff Rochlin:A pilot flying an airplane, right?
Jeff Rochlin:He's got a checklist that he goes through to make sure that he's
Jeff Rochlin:checked all the important critical things about the system, right?
Jeff Rochlin:And the reason why he has it written on a checklist is because those are
Jeff Rochlin:things you don't wanna forget,
Jeff Rochlin:So that's why you build a framework with anything that's important, right?
Jeff Rochlin:You wanna make sure you capture all of the things you need to understand
Jeff Rochlin:and have them on a list that you can check off so that you don't find out
Jeff Rochlin:after you've built the building that, oh, you forgot to dig the foundation.
Jeff Rochlin:Um, and, and it just assures that, you know, it's, you build things, right?
Jeff Rochlin:So it's, it's, it's that same concept of you measure twice and you cut once, right?
Jeff Rochlin:You don't wanna have to go back and have to redo it all again afterwards.
Jeff Rochlin:Um,
Prasanna Malaiyandi:question I have for you, Jeff, is I'm guessing the
Prasanna Malaiyandi:checklist is kind of what the, like you said, the pilot uses, right?
Prasanna Malaiyandi:In a term that, in this case, it would be like the backup admin would use
Prasanna Malaiyandi:and they would have their checklist in terms of, Okay, these are the things
Prasanna Malaiyandi:I need to make sure that I cover.
Prasanna Malaiyandi:But I, and I know we'll talk about this in a minute, but like that's
Prasanna Malaiyandi:not necessarily though exactly what you're going to be like word for word.
Prasanna Malaiyandi:You're not gonna be asking like the business, okay, what is your rto, Right?
Prasanna Malaiyandi:Because I'm guessing that like what you understand, right?
Prasanna Malaiyandi:You need some translation between like what's on that checklist
Prasanna Malaiyandi:into what you're asking the
Jeff Rochlin:When you're out collecting requirements for what your system
Jeff Rochlin:needs to be able to do, you may have to translate the question, right?
Jeff Rochlin:You may not go up to the, the CEO of the company and say, What,
Jeff Rochlin:what rto are you looking for?
Jeff Rochlin:But you may go up to and say, So the building burns down tomorrow.
Jeff Rochlin:At what point does it become no longer effective for you to reopen the business.
Jeff Rochlin:At what point is it bankruptcy and we're gone versus we can
Jeff Rochlin:survive this and move forward.
Jeff Rochlin:Right?
Jeff Rochlin:And again, that's extreme, but it's translating the question and, and a
Jeff Rochlin:checklist is like collecting, uh, the, uh, the requirements for any project, right?
Jeff Rochlin:It's.
W. Curtis Preston:Right.
W. Curtis Preston:The context that I remember in the chapter was we talked about that
W. Curtis Preston:process that we, that we had at that company where you and I worked.
W. Curtis Preston:Um, that was, it was a very rigid process for developing a new product.
W. Curtis Preston:And it was, you know, there was the,
Jeff Rochlin:we were working together in aerospace and you have to remember
Jeff Rochlin:that aerospace is one of those businesses where if you miss a step
Jeff Rochlin:in the process, things go bad, right?
Jeff Rochlin:Everything has to be measured and, and has to be checked
Jeff Rochlin:off on to make sure it's safe.
Jeff Rochlin:, Um, they took a lot of that and they've translated that down into their IT
Jeff Rochlin:processes, which by the way, are the same things that they give you in ITIL right?
Jeff Rochlin:If you're an IT person, which is a 34 letter word in most certainly
Jeff Rochlin:creative companies to drop it l um, it teaches you the idea of
Jeff Rochlin:things have to be structured right?
Jeff Rochlin:There's a, there's a right way to do things and you just need
Jeff Rochlin:to try to follow the right.
Jeff Rochlin:In the case of the aerospace company, part of that process is make sure that to
Jeff Rochlin:keep you on track, check in with people all the time, and make sure that they
Jeff Rochlin:agreed that we're still on track, right?
Jeff Rochlin:I, I spend a month collecting, um, the project plan and what the
Jeff Rochlin:requirements are, and then I send it out the stakeholders that are
Jeff Rochlin:gonna need to use this later and say, Just tell me did I get it right?
Jeff Rochlin:Is this what you wanted?
Jeff Rochlin:And they come back and go, Yay.
Jeff Rochlin:Or they come back and go, No.
Jeff Rochlin:And then you.
W. Curtis Preston:Right.
Jeff Rochlin:you check in and you do the same thing, right?
Jeff Rochlin:You come back and say, Okay, we're moving along this way, and is this still okay?
Jeff Rochlin:Um, or did your plans change?
Prasanna Malaiyandi:So totally makes sense having this board that
Prasanna Malaiyandi:you go check back with, right?
Prasanna Malaiyandi:Figuring out process.
Prasanna Malaiyandi:What are some examples of, uh, requirements or
Prasanna Malaiyandi:questions you would ask Right.
Prasanna Malaiyandi:For specifically around backup, right.
Prasanna Malaiyandi:As you're going through this process.
Jeff Rochlin:yeah.
Jeff Rochlin:Um, the very first question I always ask, and it's the hardest question
Jeff Rochlin:for any business to be able to answer, is, what is acceptable downtime?
Jeff Rochlin:Right.
Jeff Rochlin:If, if, if something happens, we can't control it, what can you live with?
Jeff Rochlin:Right?
Jeff Rochlin:I realize it's always gonna be zero.
Jeff Rochlin:. But now dig down deep in your heart cuz it's gonna happen and
Jeff Rochlin:tell me what's acceptable on that.
Jeff Rochlin:Right?
Jeff Rochlin:Um, and remember that I can give you as close to zero downtime as humanly
Jeff Rochlin:possible, but it's gonna cost you a lot
Prasanna Malaiyandi:That that's, that's exactly what I was gonna ask.
Prasanna Malaiyandi:It's like, do you then tell them, Hey, by the way, you won zero downtime,
Prasanna Malaiyandi:it's gonna cost you 5 billion.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Or something like that.
Prasanna Malaiyandi:And
Jeff Rochlin:So for me, you always present it in multiple stages, right?
Jeff Rochlin:There's the, this is, this is the worst.
Jeff Rochlin:You don't want to do this.
Jeff Rochlin:This is survival mode, how much it'll cost.
Jeff Rochlin:This is.
Jeff Rochlin:This is like gold plated, the greatest it'll ever be in the world.
Jeff Rochlin:Once you get this, you'll be the happiest person ever.
Jeff Rochlin:It's gonna cost you this much.
Jeff Rochlin:And then this is probably the answer you want.
Jeff Rochlin:This is the middle, this is the price, right?
Jeff Rochlin:Um, but you always go back to the business and the business
Jeff Rochlin:has to decide how critical is it?
Jeff Rochlin:And it, and it could be for this piece of the business, right?
Jeff Rochlin:For my, for my call center, I need a hundred percent uptime because
Jeff Rochlin:worst case, they can write the order down on paper and we can key 'em in.
Jeff Rochlin:Um, whereas, you know, for the database service, if it's down for an hour,
Jeff Rochlin:it's down for two hours, maybe that's okay, as long as I can recover it to
Jeff Rochlin:within a, a certain period of time,
Prasanna Malaiyandi:And would you say that that's probably
Prasanna Malaiyandi:the most important question?
Prasanna Malaiyandi:To always start with
Jeff Rochlin:Yes.
Jeff Rochlin:I, I believe it is because, um, it drives everything else right.
W. Curtis Preston:The thing is downtime is sort of measured in two D.
W. Curtis Preston:Ways, Right?
W. Curtis Preston:So there's downtime in terms of like how long the system is down.
W. Curtis Preston:There's also downtime in terms of how much data we lost, right?
W. Curtis Preston:So we translate that into RTO and rpo, which I, I know we already did a, uh,
W. Curtis Preston:a podcast back to basics podcast on, Um, so that, that's the big thing is,
W. Curtis Preston:is making sure that whatever it is that we're getting ready to design that.
W. Curtis Preston:We're designing it based on the requirements that business
W. Curtis Preston:came, you know, they gave us
W. Curtis Preston:. Jeff Rochlin: But that's why you have
W. Curtis Preston:you go back and you feed it back to them every once in a while and say,
W. Curtis Preston:Right, this is what you asked for.
W. Curtis Preston:Correct.
W. Curtis Preston:Cause this is what we're gonna build.
W. Curtis Preston:Right.
W. Curtis Preston:We're good.
W. Curtis Preston:And you do it multiple times so that at the end of the day, if something,
W. Curtis Preston:Assuming you did everything right and the system performs the way you
W. Curtis Preston:expected to perform, and they come back and go, What the hell, man?
W. Curtis Preston:You can look at them and go, Six times you came back and said this was acceptable,
W. Curtis Preston:Yeah.
W. Curtis Preston:Yeah.
W. Curtis Preston:Six times.
W. Curtis Preston:you and I both agreed that a six hour RTO was acceptable because a two
W. Curtis Preston:hour RTO was gonna cost us 3 billion.
W. Curtis Preston:You and I agree, remember that I have it in writing here.
W. Curtis Preston:And so the, the fact that this, you know, that we met a six hour RTO is good
Jeff Rochlin:yeah,
Prasanna Malaiyandi:and it's documented.
Jeff Rochlin:yeah, yeah, exactly.
Jeff Rochlin:It's, it's, you know, there's a, there's some element of you're
Jeff Rochlin:covering your own ass when you're doing it right, But we all have to.
Jeff Rochlin:Be sure that we're delivering what the business needs.
Jeff Rochlin:And that's why you have to understand what the business does because if
Jeff Rochlin:you understand what the business does and somebody comes back and you know,
Jeff Rochlin:you're in a business where you're talking to customers constantly and
Jeff Rochlin:somebody comes back when you ask them.
Jeff Rochlin:Um, what's acceptable and they go, ah, if we're down for a day or two, that's fine.
Jeff Rochlin:You know, they're wrong.
Jeff Rochlin:You know that they're not telling you the truth.
Jeff Rochlin:You know that they're trying to give you the easy answer.
Jeff Rochlin:And if you understand what the business does, you can look back
Jeff Rochlin:at them and say, But wait a second.
Jeff Rochlin:We answer calls 24 hours a day from customers.
Jeff Rochlin:You're okay with not answering calls 24 hours a day for those customers?
Jeff Rochlin:Uh, at which point they go, Oh, no, no, I don't mean that
W. Curtis Preston:I, I did, I did work with a business once,
W. Curtis Preston:Jeff, that had a two week rto.
W. Curtis Preston:You know what, You know what it was?
W. Curtis Preston:They, they were a paper mill.
W. Curtis Preston:I.
W. Curtis Preston:And they were like, If the computer sit down for a couple of weeks, we don't care.
W. Curtis Preston:Like we're, we're gonna keep cutting down trees.
W. Curtis Preston:We're gonna keep turning it into pulp and we're gonna keep making paper.
W. Curtis Preston:If, if you can't, if you can't bill anybody, uh, what
W. Curtis Preston:They'll still need paper.
W. Curtis Preston:We'll be, that was their attitude.
W. Curtis Preston:And so for them, a two week RTO was fine, you know?
W. Curtis Preston:Uh, but yeah.
W. Curtis Preston:But it goes back to the first thing we started with it, it goes with
W. Curtis Preston:what kind of business you are.
W. Curtis Preston:Right.
W. Curtis Preston:Most businesses, you start using the word day
Jeff Rochlin:They're,
W. Curtis Preston:in your rto.
W. Curtis Preston:They're not gonna be okay with that.
Jeff Rochlin:If you're in a retail business, you know, and you're down,
Jeff Rochlin:you know, you're, you're running a big, uh, online catalog service and you've
Jeff Rochlin:got call centers all around the world taking calls from people or websites that
Jeff Rochlin:are, you know, taking, uh, selling movie tickets, let's say, which I have a certain
Jeff Rochlin:amount of experience within the past.
Jeff Rochlin:Right.
Jeff Rochlin:And that opening weekend for the next Star Wars movie is coming up and
Jeff Rochlin:your systems go down for 10 hours.
Jeff Rochlin:You.
Jeff Rochlin:Yeah, you're in trouble, right?
Jeff Rochlin:Cause you know that that money is going to go somewhere else.
Jeff Rochlin:It's not like they're gonna sit there and go, Well, I'll
Jeff Rochlin:keep hitting return until I can
W. Curtis Preston:yeah, it's not like they're gonna not
W. Curtis Preston:buy tickets for that movie.
W. Curtis Preston:Right.
W. Curtis Preston:They're gonna, they're gonna do that.
W. Curtis Preston:Um, gosh.
W. Curtis Preston:You remember when we used to stand in line, Jeff
Jeff Rochlin:Yeah, I do.
Jeff Rochlin:You know, there was, there was a certain charm to it, but
W. Curtis Preston:Yeah, I remember standing in line, like, you stood
W. Curtis Preston:in line to stand in line, right?
W. Curtis Preston:You, you got up there, there's a big line and you're like, you, you know, it's one
W. Curtis Preston:o'clock, you want the four o'clock show?
W. Curtis Preston:And they're like, Oh, the four o'clock show already sold out.
W. Curtis Preston:Okay, so I guess we're doing the seven o'clock show and uh, and
W. Curtis Preston:you bought tickets for the seven o'clock show at one o'clock, right?
W. Curtis Preston:And then you went and did something with yourself for the next six hours.
Jeff Rochlin:Yeah.
W. Curtis Preston:Anyway, Sorry, two old guys remeniscing about back in the day.
W. Curtis Preston:Um, so you, Yeah, you, you, Jeff, in the next section, in the chapter, you
W. Curtis Preston:talked about drawing up multiple designs.
W. Curtis Preston:I think that you hinted at that already about the, here's the
W. Curtis Preston:one you definitely don't want.
W. Curtis Preston:Here's the one you're probably not gonna pay for.
W. Curtis Preston:And here, you know, here's the Goldilocks zone.
Jeff Rochlin:People want to feel like they have choices.
Jeff Rochlin:It's part of just the way life is, right, And everybody wants gold plated, but
Jeff Rochlin:they don't want to pay for gold plated.
Jeff Rochlin:So the best way to show them is there aren't, you know, I'm not narrow minded.
Jeff Rochlin:There are multiple ways you can do this.
Jeff Rochlin:You can solve this problem.
Jeff Rochlin:And at different cost levels, and I'm gonna leave it to you to decide how much
Jeff Rochlin:money you wanna spend based on that.
Jeff Rochlin:Right.
Jeff Rochlin:Um, and then just from experience, uh, I know that you present three layers
Jeff Rochlin:because there, you know, it's, there's a little psychology involved in it.
Jeff Rochlin:It's like when you write a budget, there's always something in the budget
Jeff Rochlin:that you're willing to cut because you know they're gonna come back no
Jeff Rochlin:matter what, and they're gonna ask you to cut something out of the budget.
Jeff Rochlin:It's almost like a goodwill kind of thing, right?
Prasanna Malaiyandi:So far we've kind of been talking about like the requirements
Prasanna Malaiyandi:are a little bit in stone, right?
Prasanna Malaiyandi:Sort of, uh, the business comes, gives you the requirements, you
Prasanna Malaiyandi:come back with the proposals, right?
Prasanna Malaiyandi:But as we all know, like business needs and requirements change, right?
Prasanna Malaiyandi:And.
Prasanna Malaiyandi:How do you sort of account for those changes as you're like,
Prasanna Malaiyandi:so you had these three options.
Prasanna Malaiyandi:They picked the middle option, business requirements change.
Prasanna Malaiyandi:Right?
Prasanna Malaiyandi:How do you sort of incorporate that?
Prasanna Malaiyandi:Because you can't be like, Hey, by the way, we already started on our way
Prasanna Malaiyandi:and so now you have to wait 12 months before we can incorporate those in.
Jeff Rochlin:We're agile now, right?
Jeff Rochlin:We don't actually.
Jeff Rochlin:We don't build projects that we set a two-year goal and then we check
Jeff Rochlin:in at the end of year two and say, Here's what we've got to find out.
Jeff Rochlin:We got it wrong.
Jeff Rochlin:The biggest challenge in that space to me is when you're at
Jeff Rochlin:the foundation level, right?
Jeff Rochlin:It's like if I'm building a house and we've designed the house and we've
Jeff Rochlin:decided the bathroom's gonna go here, the kitchen's gonna go there, right?
Jeff Rochlin:I'm running pipes now out of the foundation to go to those places.
Jeff Rochlin:You can't come back to.
Jeff Rochlin:Three weeks later and say, I've decided to put the bathroom at the totally
Jeff Rochlin:opposite end of the house cuz I've already poured the concrete and that
Jeff Rochlin:pipe is not gonna move on its own.
Prasanna Malaiyandi:Yeah, well, Or well not move on its own.
Prasanna Malaiyandi:And I guess that's the thing, right?
Prasanna Malaiyandi:Is it's okay to go back and say, Yeah, this requires sort of like a redesign
Prasanna Malaiyandi:or a rethought re-implementation, right?
Prasanna Malaiyandi:It's gonna affect your timelines for sure, but if that's what
Prasanna Malaiyandi:the business needs, we could.
Jeff Rochlin:So I think it, I, I think the best way you deal with this kind
Jeff Rochlin:of thing is in the ordering of how your project is established, right?
Jeff Rochlin:You've gotta get the core, the solid infrastructure that isn't
Jeff Rochlin:going to change, um, done first.
Jeff Rochlin:And you have to make sure the buy off is there and you have to keep
Jeff Rochlin:going back and saying, Right, we're going to build this network.
Jeff Rochlin:We're going to buy this kind of storage, We're going to buy these kinds of tape.
Jeff Rochlin:We can go back and change out systems and how they operate.
Jeff Rochlin:We can change schedules, we can add, but if you turn around after
Jeff Rochlin:that check is written and say, I don't want to use that anymore.
Jeff Rochlin:I can't get you that money back, right?
Jeff Rochlin:You've spent it.
Jeff Rochlin:So you need to be sure your foundation is solid in what you're trying to
Jeff Rochlin:build on, and you need to know that.
Jeff Rochlin:Um, features can change, right?
Jeff Rochlin:But the core foundation of it has to be once you've started laying it down.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:And I, I guess Curtis, this would probably be your.
Prasanna Malaiyandi:Pitch that you would say.
Prasanna Malaiyandi:It's like, Hey.
Prasanna Malaiyandi:Yeah, but with the SaaS like service, you may not need to put
Prasanna Malaiyandi:all that money up front, right.
Prasanna Malaiyandi:To pay for something.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:It does give you some amount of flexibility to be like,
W. Curtis Preston:That that dynamic scaling up and down is definitely a
W. Curtis Preston:real value of a SaaS product, but like any other product you, you have to take
W. Curtis Preston:into account a little bit of the future.
W. Curtis Preston:Right.
W. Curtis Preston:And you know, the one, the one challenge, I'm gonna say, a backing up to a SaaS
W. Curtis Preston:provider is to just make sure that we've, we've calculated in the laws of physics,
Jeff Rochlin:Mm-hmm.
W. Curtis Preston:Because if you, if you've got, if you've
W. Curtis Preston:got 10, you know, petabyte and a T1 line, it's not gonna work.
W. Curtis Preston:Right?
W. Curtis Preston:So as long as we've ca love, as we're not at the upper edge of your bandwidth
W. Curtis Preston:utilization, you should be able to grow, uh, significantly on the back end without
W. Curtis Preston:having to have a change on the, on the,
Jeff Rochlin:But again, but again, that's the foundation, right?
Jeff Rochlin:That's the, that's the core infrastructure that you have to agree on up front.
Jeff Rochlin:How much data do I have?
Jeff Rochlin:How fast do I need to get it up to that damn service in the cloud?
Jeff Rochlin:And do I have enough bandwidth and capability to do that?
Jeff Rochlin:Right?
Jeff Rochlin:Um, and those are the things that become immutable in the beginning.
Jeff Rochlin:You lay 'em down.
Jeff Rochlin:You understand them, and then you can build on top of it whatever
Jeff Rochlin:you want because you have a foundation that'll support it.
Jeff Rochlin:You can't turn around though afterwards and say, Well, I didn't really have
Jeff Rochlin:that much network bandwidth, and it turns out that in order to get that much
Jeff Rochlin:network bandwidth, I have to have this, pay this guy over here to dig a tunnel
Jeff Rochlin:underneath, you know, the, the biggest avenue in town to run a fiber optics to my
Jeff Rochlin:building, and I'm not gonna pay for that.
Jeff Rochlin:So I guess we'll have to figure out how to do it without the bandwidth.
Jeff Rochlin:Right?
Jeff Rochlin:That's when it, that's when the tower collapses and none of it's ever.
W. Curtis Preston:Yeah, so I, I'd say that, So we, so
W. Curtis Preston:we, we got the requirements.
W. Curtis Preston:We've got, um, and we, we've, we've done this iterative process of going
W. Curtis Preston:back to the stakeholders to make sure we understand the requirements.
W. Curtis Preston:We've built out theoretical system.
W. Curtis Preston:The, the next phase that I think is, is really important is to make sure.
W. Curtis Preston:The product that we, that we're supposed to be buying does what it
W. Curtis Preston:is that it was supposed to do, right?
W. Curtis Preston:So we're talking about things like POCs and pilots.
Jeff Rochlin:I, I do the POC early, right.
Jeff Rochlin:I do it like before I even, I go back to, if the business comes back to me and
Jeff Rochlin:says we need to update backups, I go out and start doing the POC on the backup of
Jeff Rochlin:products before I come back and propose here's what I think the solution would be.
Jeff Rochlin:Right.
Jeff Rochlin:Um, you know, There are the sales guys and there are the sales engineers, right?
Jeff Rochlin:Don't listen to anything the sales guys have to say to you.
Jeff Rochlin:Cause the sales guys are sales guys.
Jeff Rochlin:That's what they do.
Jeff Rochlin:Let them buy you lunch and then don't worry about it.
Jeff Rochlin:Find yourself a good sales engineer who's attached to the account.
Jeff Rochlin:Connect up with them.
Jeff Rochlin:They'll give you this straight answer, and then give them your real
Jeff Rochlin:world, your real world scenario, that's kind of where I come from.
Jeff Rochlin:Right.
Jeff Rochlin:If you're gonna, if you're gonna go to the trouble of selling me a service
Jeff Rochlin:up in the cloud somewhere, then it should do everything it needs to do.
Jeff Rochlin:Right.
Jeff Rochlin:And.
Jeff Rochlin:Go on a tangent, which I know you'll cut cuz you hate when I go on tangent.
Jeff Rochlin:So I'm keeping you from doing other
W. Curtis Preston:It depends on the value of the tangent, Jeff
Jeff Rochlin:um.
Jeff Rochlin:When, when people, especially the non-technical people go off
Jeff Rochlin:and think of the cloud, right?
Jeff Rochlin:They think the cloud is a solution for everything, right?
Jeff Rochlin:I want to go store things in the cloud, so I will just copy all of my
Jeff Rochlin:data up into the cloud and things will be wonderful, and it's like, No, you
Jeff Rochlin:copy your stuff up into the cloud and all you're doing is you're handing it
Jeff Rochlin:off to somebody else's data center.
Jeff Rochlin:That doesn't mean it's properly backed up.
Jeff Rochlin:That doesn't mean it's redundantly stored in multiple locations
Jeff Rochlin:in case there's a disaster.
Jeff Rochlin:You still need a systems engineer to look at the problem and come up
Jeff Rochlin:with a solution within the context of the cloud as if it was your own.
Jeff Rochlin:But, but cloud is a marketing term, right?
Jeff Rochlin:That we sell to people who don't understand technology about a,
Jeff Rochlin:here the solution is there, baby.
Jeff Rochlin:Just buy this and, and, and your life will be better and everything will just
W. Curtis Preston:It, You know what you need, Jeff, In that
W. Curtis Preston:scenario, you, you need a Prasanna.
W. Curtis Preston:That is what Prasanna does.
W. Curtis Preston:So, you know, he's actually, he's actually, you know, I'm gonna say he's
W. Curtis Preston:more technical than I am, and when we talk about current cloud technology,
W. Curtis Preston:uh, he's actually making things happen and actually understands that stuff.
W. Curtis Preston:So you need somebody like that.
W. Curtis Preston:Like, it's not, like the cloud doesn't just like, Oh,
W. Curtis Preston:we don't need any IT people,
Prasanna Malaiyandi:Okay.
W. Curtis Preston:We just need different kinds of IT people.
Jeff Rochlin:Right.
Jeff Rochlin:But you would be surprised at the number of C level and especially
Jeff Rochlin:financial people out there who think.
Jeff Rochlin:By buying into a cloud service, they can throw away their data center and they
Jeff Rochlin:no longer need as big an IT department
W. Curtis Preston:Yeah.
Jeff Rochlin:there.
W. Curtis Preston:might, might
W. Curtis Preston:need as big, but Yeah.
Jeff Rochlin:Yeah.
Jeff Rochlin:you're gonna eliminate certain positions, right?
Jeff Rochlin:You're gonna eliminate the guys that are handling your hardware.
Jeff Rochlin:But, um, there, there is this perception and that's why everybody
Jeff Rochlin:thinks the cloud is cheaper, right?
Jeff Rochlin:In many cases it's more expensive than running your own data
Jeff Rochlin:center depending on what you're
W. Curtis Preston:It, It, definitely depends on how you implemented it.
W. Curtis Preston:Yeah.
Jeff Rochlin:Right.
Jeff Rochlin:But,
Prasanna Malaiyandi:talk to Corey Quinn or follow him on Twitter, right?
Prasanna Malaiyandi:That's his big thing.
Jeff Rochlin:yeah,
W. Curtis Preston:Yeah.
Jeff Rochlin:so anyway, that was, that was my tangent, but I thought it might
W. Curtis Preston:That was a good tangent, Jeff.
W. Curtis Preston:Um, alright, so the, so the final thing here of, of, you know, of, of,
W. Curtis Preston:in this chapter was about documenting and implementing the new system, right?
Jeff Rochlin:Yep.
Jeff Rochlin:The job, the job that nobody ever wants to have to do is documentation.
W. Curtis Preston:yeah.
W. Curtis Preston:Um, you know what,
Jeff Rochlin:You,
W. Curtis Preston:that about?
W. Curtis Preston:Right?
W. Curtis Preston:Do do I have to, do I have to document?
Jeff Rochlin:Yeah.
Jeff Rochlin:You have to.
Jeff Rochlin:Sorry.
Jeff Rochlin:You know, cuz it's like that's your proof of life, right?
Jeff Rochlin:You have, you have it down in writing, you have a plan that you're working from.
Jeff Rochlin:It's the checklist, right?
Jeff Rochlin:And it's visible to everybody.
Jeff Rochlin:You can hand a copy to somebody and say, I want you to read this, and then
Jeff Rochlin:I want you to sign on the bottom of it so that I know that you've read
Jeff Rochlin:it and you either agree or you don't.
Jeff Rochlin:And then if they take it and they just sign it and hand it back
Jeff Rochlin:to you when there's something in there that they didn't like or
Jeff Rochlin:that they didn't expect, you look at them and go, You signed for it.
Jeff Rochlin:Right?
Jeff Rochlin:Would you sign a contract without reading it?
Jeff Rochlin:Would you, You know,
Prasanna Malaiyandi:all the time.
Prasanna Malaiyandi:I agree.
Jeff Rochlin:Written.
Jeff Rochlin:The written documentation, especially the design documentation is actually kind of
Jeff Rochlin:like a contract with your customer, right?
Jeff Rochlin:Your customer being the business.
W. Curtis Preston:right.
Prasanna Malaiyandi:this design document, right?
Prasanna Malaiyandi:It, I'm guessing it also sort of restates, Hey, here are all the
Prasanna Malaiyandi:requirements that I built against, Right.
Prasanna Malaiyandi:That we had agreed to during the sort of the checklist and
Prasanna Malaiyandi:those initial discussions.
Prasanna Malaiyandi:Right?
Prasanna Malaiyandi:And these are sort of the requirements that this design satisfies.
Jeff Rochlin:Yeah.
Jeff Rochlin:When you, when you write that initial document once, once everybody has
Jeff Rochlin:blessed it and signed off on it, you may never go back to it again.
Jeff Rochlin:Right?
Jeff Rochlin:You may never have to go back to the printed document, except for the
Jeff Rochlin:fact that it's part of your completed set of things you need to have.
Jeff Rochlin:That's just there for the future.
Jeff Rochlin:Right?
Jeff Rochlin:And if you've ever been involved in any kind of construction, right on the back
Jeff Rochlin:end of the construction, there are the as-builts, right there is the actual
Jeff Rochlin:design document in the blueprints of the building as it was completed by
Jeff Rochlin:the contractors who did all the work that you stick in a box somewhere and
Jeff Rochlin:only go back to when you're trying to figure out where that damn wire that's
Jeff Rochlin:powering your wifi, uh, device is running.
Jeff Rochlin:Because that's when you need your as-builts, right?
Jeff Rochlin:You need to understand how things are wired.
Jeff Rochlin:This becomes the as-built for the project.
Jeff Rochlin:It gives you esoteric details that you really don't care about until
Jeff Rochlin:for some reason you need to know about it, and then it's there.
W. Curtis Preston:When I, when I think about the value of documentation, I'm
W. Curtis Preston:speaking in this case more operational documentation than design documentation,
W. Curtis Preston:but I go all the way back to when my oldest daughter was born, right?
W. Curtis Preston:I was in Delaware, uh, and uh, I was at the hospital in my wife's delivery room.
W. Curtis Preston:Nina was born, She's, you know, brand new baby, and I'm, I'm standing in, you
W. Curtis Preston:know, in New Father Bliss Zone, right.
W. Curtis Preston:And the phone rings in my wife's hospital room and you know, this
W. Curtis Preston:is pre-cellphone days, right?
W. Curtis Preston:And, um, I, um, I picked up the phone.
W. Curtis Preston:I'm like, Why is this phone, It's the bank on the phone, and I.
W. Curtis Preston:What, And they said, Well, well we have this restore, uh, that we need to do.
W. Curtis Preston:I'm like, I am in my wife's hospital room.
W. Curtis Preston:And they're like, Well, we don't understand abc.
W. Curtis Preston:And I said, Well,
Jeff Rochlin:Yeah.
W. Curtis Preston:did you read the documentation?
W. Curtis Preston:Is there a problem with the documentation?
W. Curtis Preston:They said, Oh, we haven't looked at it.
W. Curtis Preston:Slam.
W. Curtis Preston:I just slammed down the phone, , I.
W. Curtis Preston:The, I had good documentation and that was, you know, I, I was like, I'm good.
Jeff Rochlin:I, I regularly go to developers when they're writing software
Jeff Rochlin:that I'm involved in having to support and telling them that, um, because
Jeff Rochlin:I run operations as well, right?
Jeff Rochlin:And telling them that, um, it is in there enlightened self.
Jeff Rochlin:To write good documentation for their product because I will call them at
Jeff Rochlin:two o'clock in the morning on Saturday.
Jeff Rochlin:If it breaks and there's something broken, not only will I call you, I'm
Jeff Rochlin:gonna make sure I have your wife's cell phone number as well, and I
Jeff Rochlin:will call her if you choose not to
W. Curtis Preston:I call your mom
Jeff Rochlin:Cause if the, I would, Cause if the business is flat out, down
Jeff Rochlin:on its ass and I have the CEO of the company who's going to come looking for
Jeff Rochlin:me because I'm responsible for operations.
Jeff Rochlin:What am I gonna tell 'em?
Jeff Rochlin:Sorry, the software doesn't work.
Jeff Rochlin:We'll get back to you on Monday.
Jeff Rochlin:Right?
Jeff Rochlin:It's like, no, it doesn't work that way.
Jeff Rochlin:Um, and that's why everything, the better the documentation is, the
Jeff Rochlin:less the operations guys have to go and bother the people who wrote it.
Prasanna Malaiyandi:It just going back to the earlier thread that
Prasanna Malaiyandi:I was on, how do you ensure that cuz things will change, right?
Prasanna Malaiyandi:The system will change.
Prasanna Malaiyandi:The designs will change.
Prasanna Malaiyandi:Is that part of the process that any time like you get new
Prasanna Malaiyandi:requirements or anything else like.
Prasanna Malaiyandi:Always go back and update the documentation.
Jeff Rochlin:Yeah.
Jeff Rochlin:Should be, should be.
Jeff Rochlin:Without a doubt.
Jeff Rochlin:And if it's happening during the development process, you take it
Jeff Rochlin:back to the review boards so that again, everybody's aware of it
Jeff Rochlin:and everybody checks off on it.
Jeff Rochlin:Right.
Jeff Rochlin:That they're okay with the change.
Jeff Rochlin:That it's not a surprise to finance.
Jeff Rochlin:Right.
Jeff Rochlin:And it's not a surprise to, uh, facilities operations that you're,
Jeff Rochlin:you know, all of a sudden gonna re.
Jeff Rochlin:Uh, the, the waterlines or something like that to make it work?
W. Curtis Preston:Yeah, I, I would say it's your job as the technology side
W. Curtis Preston:of things to watch what's happening with the business side of things and
W. Curtis Preston:to see if there are changes, right?
W. Curtis Preston:If all of a sudden to, to use your world.
W. Curtis Preston:Jeff, you know, you are normally doing 10 movies at a time and you happen
W. Curtis Preston:to be in some quarterly meeting and you hear that, They're like, Listen,
W. Curtis Preston:we're gonna double production.
W. Curtis Preston:We've, you know, we've had some great sales and we're
W. Curtis Preston:gonna do 20 movies at a time.
W. Curtis Preston:You're gonna be like, um,
Jeff Rochlin:Yeah.
W. Curtis Preston:Right.
W. Curtis Preston:You know, current capacity is based on 10 movies at a time.
W. Curtis Preston:I'm gonna need twice as much
Jeff Rochlin:yeah, I guess examples in my life of that were like, um, I was working
Jeff Rochlin:at Disney at the time and, uh, I sat in a meeting and they said, Hey, we're gonna,
Jeff Rochlin:we're gonna release this one in imax.
Jeff Rochlin:You know, sitting there going, Wait, I have a system that's designed to
Jeff Rochlin:create two K frames on the back end.
Jeff Rochlin:I don't even know what IMAX is at this point.
Jeff Rochlin:Or, Hey, we're gonna start doing 3d.
Jeff Rochlin:That was like, ah, right.
Jeff Rochlin:When it comes to things like storage, right, The transition from,
Jeff Rochlin:we're gonna do two k, we're gonna.
Jeff Rochlin:4k, we wanna do these special formats.
Jeff Rochlin:Oh wait, we're gonna start doing live action visual effects.
Jeff Rochlin:That means we've gotta shoot, you know, real film back plates,
Jeff Rochlin:uh, that you're gonna put stuff on top
Jeff Rochlin:of,
W. Curtis Preston:The point here is that you need to be keeping an
W. Curtis Preston:eye to going back, sort of bringing this all back to the beginning.
W. Curtis Preston:Your organization, what is its function, right?
W. Curtis Preston:This, that idea works for businesses, it works for government, uh, right?
W. Curtis Preston:So what is the purpose of your organization and what do you, what
W. Curtis Preston:purpose do you serve within that purpose?
Jeff Rochlin:Right.
Jeff Rochlin:I can, I can promise you that the head, the, the, the chief creative
Jeff Rochlin:officer at Disney is not thinking about the difference in storage capacity
Jeff Rochlin:between a two K movie and an 8K movie.
W. Curtis Preston:Yeah, it's not, not their job
Jeff Rochlin:right.
Jeff Rochlin:But you have to be listening to him to hear that he just said eight K, and go,
Jeff Rochlin:Well, okay, I need to go figure out what that means, and then I need to go find
Jeff Rochlin:the right executive who's willing to go ask him if he's willing to spend money on
Jeff Rochlin:that or if he was just talking about it.
W. Curtis Preston:This whole thing was about gathering the requirements,
W. Curtis Preston:just making sure what your company does, making sure that you are doing
W. Curtis Preston:the things that you need to do to, to, to meet those requirements.
W. Curtis Preston:Um, and, and, and that, and that you should absolutely not be
W. Curtis Preston:determining those requirements.
W. Curtis Preston:You should not be determining retention.
W. Curtis Preston:Recovery time, objective, rpo, any of that stuff that should
W. Curtis Preston:all come back from the business.
Jeff Rochlin:Yeah, be an active participant in your company, right?
Jeff Rochlin:It's more, be more at your job than just your job.
Jeff Rochlin:Understand what your business does, Um, understand why it's doing it,
Jeff Rochlin:what makes it, you know, special.
Jeff Rochlin:in whatever it is that it's doing.
Jeff Rochlin:And when you understand all that stuff, you'll get to know the people
Jeff Rochlin:in the business better because you know, like I said, the guy in
Jeff Rochlin:customer service doesn't care about what's going on in the computer room.
Jeff Rochlin:But if you get to know the guy in customer service because you understand
Jeff Rochlin:what he's up against, all of a sudden you can build that relationship.
Jeff Rochlin:That'll make it easier to get the info you need later and then document the hell out
Jeff Rochlin:of it and make sure everybody signs off.
Jeff Rochlin:So, Accountability is spread evenly across the whole
Jeff Rochlin:organization at the end of the day.
Jeff Rochlin:Cause you don't want to be the guy that everybody points to and
Jeff Rochlin:says, Well, we're down for a week.
Jeff Rochlin:Because, you know, he decided that we could be down for a week
Jeff Rochlin:and go, No, here's the book.
Jeff Rochlin:Right.
Jeff Rochlin:And then you document it all through an inch of its life so that a,
Jeff Rochlin:you have it as a reference, right?
Jeff Rochlin:Cause other people are going to have to operate it if you don't want to
Jeff Rochlin:be the only person who gets called at two o'clock in the morning cause they
Jeff Rochlin:can't figure out how your system works.
Jeff Rochlin:Then you describe to people how your system works so they can figure it out at
Jeff Rochlin:two o'clock in the morning without you.
Jeff Rochlin:Um, and that, um, all of the parameters are covered so that you can now move
Jeff Rochlin:up to be the president of the company and the next guy coming in behind you
Jeff Rochlin:will be able to just easily step into.
Jeff Rochlin:Past role cuz he can just open the books and it's all there for
W. Curtis Preston:I like it.
W. Curtis Preston:Well, thanks Jeff.
W. Curtis Preston:Always, always a treat.
Jeff Rochlin:My pleasure guys.
Jeff Rochlin:Thanks for having me.
W. Curtis Preston:Thanks.
W. Curtis Preston:Prasanna.
W. Curtis Preston:I know between Jeff and I, you didn't get a word in edge wise in
Prasanna Malaiyandi:That's okay.
Prasanna Malaiyandi:I, I
Jeff Rochlin:He asks good
Prasanna Malaiyandi:
Speaker:to the old man, old men
Jeff Rochlin:Hello?
Prasanna Malaiyandi:By the way, if you wanna listen to the two of
Prasanna Malaiyandi:them talk more, they do have their, another podcast, right, where you
W. Curtis Preston:We do, we do
W. Curtis Preston:the, the things that entertain us Yes.
W. Curtis Preston:Is the name of our other podcast.
Jeff Rochlin:May maybe, maybe Curtis will stick a link in in the, uh, show notes.
Jeff Rochlin:Who
W. Curtis Preston:maybe, maybe he will, Maybe he will.
W. Curtis Preston:Uh, all right.
W. Curtis Preston:And thanks again to our listeners.
W. Curtis Preston:We're nothing without you.
W. Curtis Preston:Remember to subscribe so that you can restore it all.
Listen On
