8.1 Understand waterfall project management - Video Tutorials & Practice Problems

<v ->Hello.</v> We have been talking about a variety of lessons related to the basics of project management to this point. And what I wanna do now is introduce an exciting topic that has been built off of the basic ideas of project management over the years, namely agile. And agile project management has taken the business world by storm in many industries because it offers a variety of challenging, but really important advantages when managing projects. Now, in order to understand agile project management, in order to make sense of this, we have to set the stage a little bit for where we were, where we are, and then what agile is attempting to do. See, the problem is that project management in general was starting to develop a bit of a problem. You see, we understood the value of project management, and yet the evidence wasn't always supporting that advantage. So for example, we know that after years of use, failure rates were remaining very high, especially in certain industries. For example, the specific class of projects in the software and IT field had been very, very difficult to manage, and their failure rates had not improved markedly over a period of time. Now, when I say that, I have to couch this by saying that the CHAOS reports, the CHAOS organization that comes out with these reports basically evaluates the state of the field in software project management on a semi-yearly basis. And what they were doing was charting project success and performance over an extended period of time. And they were finding very disturbing information. So for example, they were discovering that the most recent annual CHAOS reports demonstrated success rates, success rates of about 29%. Now, what that meant, if we parse that the other way is what that's suggesting is 70% of software projects were either challenged, as they put it, or outright failures. So 70% challenge or failure rates is pretty disturbing. And the more disturbing issue, as you can see, is the fact that that number had remained steady since 1994. So over 25 years of evidence looking at how software projects were being run was not yielding improvement. Now, this is important because we have to understand that other research has born out this same problem. So for example, KPMG Consulting found, in their studies of IT failure rates, they also confirmed failure rates, not success, failure rates of approximately 65%. And here's the really disturbing information when we talk about that, is that when they interviewed managers after the fact and they presented this information to them, the general reaction of most of these managers was to nod their heads and say, "Yeah, that sounds about right. "Those numbers are pretty accurate," and worse, they considered them normal. So we have a serious problem that's brewing, and that is looking at how projects are being run, recognizing the value of project management, and yet when it hits the road, when the rubber hits the road, it's not always working the way we anticipate it should be working. And so, consequently, high failure rates are not impressive either for the company or for the managers involved in this. So what was going on? Well, one argument is that this problem, this failure rate particularly in software and IT projects, but it didn't end there, it's across in many other areas, this failure had a name and that name was waterfall. Now, waterfall implies the idea, as you can see here, of what we do is we attempt to create a very structured, very organized, very orderly process for developing a project. And you can see, for example, let's take a software project example. And you can see on the screen how this plays out logically, and it does make sense logically. Let's acknowledge that, that these steps seem appropriate. And so, the steps start, you can see, with getting requirements. What do people need? Then we design the system, then we implement the system, then we test the system. After testing, we fully deploy the system. And then ultimately, the last thing on the agenda there is maintenance of the system. So you can see, over time, from the start of the project to its termination, that we have a series of logical measured steps as to how that process should work. And you can see the way this is represented on the screen, why this is, in fact, called the waterfall model. Because we start in the top left with these ideas at the beginning of the project, and we gradually move from one step to the next, moving it downstream. And so, consequently, the idea of waterfall. Now, don't get me wrong. Waterfall works fine as a methodology for running projects. Waterfall has a lot of advantages, and it works fine, but only under certain conditions. So for example, the first condition is this: the requirements are well-understood, and more importantly, they can be fixed at the beginning. So the requirements are clear, mutually agreed on by all members of the project and all stakeholders, and they're fixed early on. So if we can do that, fine. Secondly, waterfall works well when the product definition is stable. We know what we're attempting to do. We know the specs involved here. And we don't anticipate any changes. So from the start of the project to the conclusion of the project, there should be no surprises. Third, waterfall works well when the technology is clearly understood. So for example, if our technology for developing this project is fairly basic, maybe it's just word processing needs, or some basic Excel spreadsheet work, or some engineering design work, schematics, etc., if we understand the technologies and we've employed them multiple times in the past, then it shouldn't be a problem because we're familiar with it. Fourth, waterfall works well when there's enough resources with enough expertise that's available when we need it. In other words, there will be no surprises downstream with getting to a point in that project testing, for example, and discovering that the testing personnel aren't available when we need them, or that we've lost them, or that something else has come up. So the resources, particularly the human resources, are knowledgeable and they're available. So that's another important point about waterfall. Finally, and this is an important point, the fifth criterion in which waterfall works well is if the project is of relatively short duration. By short duration, we could be talking in a manner of weeks, perhaps months, but certainly, not extended periods of time. We understand this intuitively, don't we? If we think about our own work experience or if we think about our own life experience, we know that one thing that is absolutely a rule of thumb is change. Technical changes, all kinds of changes are occurring all around us. And so, it would make sense that in our own organizations, the longer a project exists, the more likely there is going to be some changes occurring. So we want something of relatively short duration that allows us to feel good about lock in. So it's not gonna be too long. We can lock in the expectations, we have the resources, etc. Under those circumstances, waterfall is a great methodology because it provides logic and structure to how I develop the project. And ultimately, these are things that project managers crave and have the ability to create. But what happens? What happens when certain conditions are violated, when it doesn't work out the way we had hoped it would? So for example, what happens when your organization doesn't have those ample resources that we talked about, that everybody is multitasking, or everybody is in a state of flux. Job requirements and job specifications are changeable. People are moving in, people are moving out. So those ample resources don't exist. There's a financial downturn in our company and we have to cut budgets. You get the idea. What happens when that's not the case? What happens when the project team can't get lock-in on the technical specs? So for example, we have an idea of what's required here, but much of this is still up in the air. So we're not entirely clear on the technical aspects of this. We have a sense of it, but we can't be precise about what the expectations are. What happens if the product is subject to changing consumer tastes or needs? Well, this is a big one, isn't it? Think about long-term projects, for example, that we present to a customer, only to discover that the technology has changed in the meantime. So what we thought they needed, they don't need anymore. Their commercial expectations or their commercial situation has changed. So before they had the luxury of developing this project over a period of time, and now everything is tightened up, and so they don't have that luxury anymore. Or it's simply something that the public in general doesn't desire. So we're creating a product for a disappearing audience. What happens when the customer doesn't really know what they want? You see, I deal with this all the time, and I deal with folks like yourselves, project managers in organizations that are constantly frustrated by this fourth point, that the customer thinks they know what they want, but ultimately doesn't really know. And at the end of the day, just sort of says, "Well, you're the expert. "You figure it out. "You tell me what it is that we need here." You see, they throw that back on the project manager and expect you somehow to read their minds or to know exactly downstream, you have a crystal ball that can project in six months or whatever what they're going to be needing. You see, that's not fair, but a lot of times this happens. The customer may think they know what they want, but the more you ask them, the more questions you probe with, the more they get frustrated and they just kind of throw up their hands and say, "Well, you figure it out." What's the bottom line? The bottom line, as you can see on the screen, is an important point, and that is that waterfall, which is a classic example of what I would call a rigid methodology, waterfall or any methodology that's very rigid, is both a strength and a weakness of traditional project management. Let me say that again. Rigid methodologies are a strength of project management in the sense if we are able to employ a methodology in a particular kind of setting, we can create structure, we can create order out of chaos, we can create the realization of dreams out of somebody's thought processes, but they don't really know what they are. So rigid methodologies can be beneficial, and they can be a real benefit of project management. But at the same time, you can see that under certain conditions, they can actually be a real weakness of traditional project management. So for example, those of you who've worked in IT or work in a software setting, interacting with a customer, I have heard versions of this following exchange so many times that I've lost track of them. The user, the person at the end of the pipeline starts pushing a few buttons on this new software system you develop for them or this new application. They start clicking around on it and quickly say, "That's not what I wanted." And the developer, who was justifiably proud of what they were presenting to the user, the developer's saying, "But wait a minute. "That's what you asked us to do. "That's what you wanted." And the user said, "Yeah, but I didn't know "this is what it was gonna be." And you see, the problem here occurs time after time after time when we deal with the user in a very limited amount of time, usually early in the process, lock in specs that they think they want. But remember, they're a user. They may not have the technical expertise, so they think they'd like this. We lock that in. We employ the waterfall method to go and develop it. We present it to them triumphantly at the end of the process, only to have them push a few buttons and say, "This isn't what I wanted." "But you asked for it." "But this doesn't work. "This isn't what I need." And again and again, we have seen examples of this. And as I said, I've worked with many organizations, and it's a constant frustration on the part of their software engineers in dealing with customers. But it's understandable in many ways because it is the software engineers that are the experts. It's the users that use the system but don't entirely understand the technology. They just understand what they need, but they can't convey it well. And so, back and forth we go with this, one boulder hitting into another, crashing together, trying to make a project work when neither person really understands the other side's perspective.