As I explained in a previous post, the last year I have been involved in a renewal process of all of our application development tools. One of the first things we did when we started the program was to apply the most fundamental lean principle: eliminate waste. To lean thinking, waste is anything that does not create value for a customer. For those of you who are not familiarized with the lean principles, I recommend the “Lean Software Development: An Agile Toolkit” book. In this book, Mary and Tom Poppendieck translated the seven wastes of manufacturing identified in the Toyota Production System into the seven wastes of software development: Partially Done Work, Extra Processes, Extra Features, Task Switching, Waiting, Motion and Defects. In this entry I would like to explain some of the problems we have encountered while trying to eliminate waste and some lessons learned.

The first waste we tried to eliminate was the extra processes. In other word, we tried to eliminate paperwork that does not means adding value for our users or for our organization. In our case, this task was not related to the development of our tools, it was about eliminating extra processes that were embedded in the tools we developed which forced our users (developers) to execute some unnecessary processes. This action produced some surprises, since near the end of the development, in one of the latest functional demos, there was a crisis moment. Some developers reminded us an essential process they were following in the old tool: they defined the batch programs in a product repository. This process was removed deliberately, because it does not provide any value, so it was a surprise for us that our users asked for this. When we asked them why this process is necessary, they answered that they did not know, but it was something they used to do because someone told them that they must do this task. “Is this useful?” we replied. “No, but we must continue doing that because … we must do that”. Wow, it remembers me the monkey experiment. Obviously, and despite the laments of our users, we did not add again that process. So lesson learned: in order to eliminate waste you need to break the status quo, you need to break the corporate culture.

The second waste we tried to eliminate was the extra features, because this was one of the biggest mistakes we did in the past in other tools. Some years ago we started developing a new tool and, using the usability argument, we added a lot of features that lately nobody used. Some of the hitches you may suffer adding those features are that your code-base grows uselessly, increases maintenance costs and makes future developments more complex. This must not be a big problem if users really need these features, but why must we maintain them if they are not being used? Users, moreover, feel that the tool is more complex, so your usability argument disappears, and they reproached us that we are not focusing on what it is really important for them. Now, I am proud to say that our tools have less features, but, at least, the ones we implemented are really used. So two more lessons learned: 1) more features does not mean better tools; 2) usability does not mean more features.

I am going to stop here. I am sure most of you who have tried to eliminate waste have found these or similar problems. But I do not want to conclude this entry without explaining one of my latest lessons learned. It is not strictly related to lean thinking, it applies to software development in general, albeit it could only apply to some organizations. Sometimes, I believe it is better not to explain that you are using an “x” methodology, or to intensify your position saying that you learned those practices from whatever methodology. Sound strange, isn’t it? But I have discovered that lots of developers hate the words “methodology” and “process”, and they have adverse reactions when they hear them. I find easier to explain practices without any reference to the original methodology. Lots of times, using the common sense is better to prove the goodness of a practice. And if it does not sound good, perhaps it does not match your organization. OK, maybe I am generalizing. In every change process you will find resistance, so perhaps if it does not sound good it is because fear. So, my last lesson learned: use common sense, do not arbitrarily adopt new practices.

Recently, I have been involved in a major IT program, framed on a high demanding business strategic plan, which aims to renovate all of our core banking system. One of the program’s first steps was changing our IT organizational and governance structure, the enterprise architecture, the application development tools and the software development methodology.

Although my responsibility in this program only relies on the application development tools, as one of the PMO leaders, I was able to follow closely the rest of the items. One of the topics that I was specially interested on was the software development methodology, because among other things, before the program, it was one of my responsibilities, and, after the program, it will be, again, my responsibility. The truth is that I had high hopes for change (perhaps influenced by the “Yes, we can!” slogan), but the fact that they came to conclusion that we need another waterfall methodology, perhaps a bit stricter than the one we use today, disappointed and frustrated me.

But please, that nobody misunderstood me. I deeply respect the work and decisions of my colleagues. I have had the opportunity to explain my thoughts. I gave them some books on the topic. I tried to influence the people who had the task to define the new methodology in order to introduce more innovative methods/process/practices of software development, but, maybe, being too innovative in a very classic environment doesn’t helped me. I think that changing the software development process in a big organization requires lots of effort, education, very slow and gradual steps, …, and I do not want to miss this opportunity, I believe that now is the moment to do that, taking advantage of the whole changing program. Well, at least, now some people knows that there are more life after the waterfall model, there’s a gray scale between white and black.

As I’m a bit nonconformist, I tried to find out which were the reasons behind that decision. So I decided to ask some developers and project managers to get an exact idea of what they think/know about software development methodologies. Although most of the answers were what I expected to find, I also found some interesting frustrating observations. I want to share with you some of them:

• Most developers and project managers are unaware of the existence of another methodologies, and they do not have any interest in learning them. Methodologies and process are something bored and do not provide any value to what they are doing actually. Although they recognize there are lot of inefficiencies in the way they work, they don’t want to change it. When I ask some of them if they follow our actual methodology, their answer is NO. “Well, at least, are you following any predefined process?” again, the answer is NO.
• Most project managers feel that a plan-driven methodology provides them more control over the whole project, that their projects are more predictable. But when I ask them if their projects are on time, most of them recognizes that NO. They also doesn’t have/use information from previous projects in order to estimate or improve the next ones, every project is different.
• They see iterative / agility methodologies as a chaos, the wild west, where there isn’t any discipline. W00t? discipline and agile are not conflicting. XP, for example, requires high levels of discipline.
• Some people told me that waterfall methodologies encourages a comprehensive documentation. “Well, could you show me your last project’s documentation? no, we didn’t have time to write it. OK, doesn’t mind. Could you show me any documentation of any project you’re involved? no, we’re still working on it, you know, we’ve tigh schedules”.
• Some answers reflected the “We’re Special” syndrome: “Agile is suitable when you want to develop software for an Iphone, but not for financial applications” (paraphrased).
• Some project managers get annoyed when they must talk with their clients or stakeholders, they hate them (I believe this is a mutual feeling). Yeah, those evil people that everyday changes the requirements and doesn’t have any idea how hard is the software development process.

I also tried to analyze some projects, and, surprisingly, I discovered that most of them are short term projects (2-3 months). I expected longer projects, as corresponds to a waterfall process. So I ask myself if we are really using a waterfall methodology, or we’re using a masked iterative process?

How about you? Did you find these kind of answers in your company? Any ideas on how to address this situation?

Each time I attend a conference about agile software development, inevitably, the same question always appears: how to deal with fixed price contracts when working on agile methodologies. Usually, people did not get any satisfactory answers.

Scott Ambler, Practice Leader Agile Development within the IBM Methods group,
invariably gets the same question when working with clients or internal development teams. So he decided to take it one step further and address the question in the Dr. Dobb’s article “Agile on a Fixed Budget: Resources, schedule, and scope“:

The column first summarizes strategies for what you would do when each of these factors is allowed to vary so that you understand what trade-offs you’re making. Then it describes what I consider to be your best approach remaining to you for each combination of constraining the three factors. The main point of the article is that although it isn’t ideal to have one or more of these factors constrained, you can still take an Agile approach even when such constraints exist.

Scott approach is not the panacea, but it’s a worth read.

Another interesting read is the Arnon Rotem-Gal-Oz post “Fixed Bids and Agile Projects”, which shares a few strategies he successfully used and provides two useful links: Agile Fixed Price Projects part 1: “The Price Is Right” and Agile Fixed Price Projects part 2: “Do you want agility with that?”.

And finally, here there are some other opinions I have recollected:

• Fixed Price Contracts and Agile Delivery: Richard suggests to apply agile’s iterative approach to the proposal.
• Successfully Applying Agile to Fixed-Bid Projects: the post suggests that the customer must understand that the fixed-bid is not in their interest if the scope is also fixed.
• Fixed bids, agile projects: Oren points out several options and give some tips on how to win such projects.
• Agile development in a FDA regulated setting: this post explains why agile methodologies have a long way to go before we see them commonly used in regulated environments, in this case, FDA.
• FixedPrice: Martin Fowler points out that you can’t come up with a fixed scope when working on fixed price contracts.
• Fixed-Price Agile Projects: Peter’s approach is to fix price each iteration before it starts.

Do you have any other successful strategy or experience?

After OpenUP/Basic and XP, now there is a third agile process available through the Eclipse Process Framework: Scrum. The content is in French, but there is an English Scrum version under way.

If it is the first time you hear about the Eclipse Process Framework Project:

EPF aims at producing a customizable software process engineering framework, with exemplary process content and tools, supporting a broad variety of project types and development styles.

Besides the three process mentioned before, there is also a DSDM plugin in the review stage.

VersionOne conducted a global survey to highlight the value teams deliver from Agile development. This survey summarizes the state of Agile Development in organizations covering every industry vertical from financial services, health care, and education to video games, government, and defense. The result is their “State of Agile Development” Survey Results.

(Via InfoQ)

In This InfoQ video interview, Ivar Jacobson, one of the founders of UML, RUP, use cases, introduces his vision for a next generation development methodology that is both agile yet large like UP, by having humans collaborate with ‘Intelligent Agents’. Ivar also talks about his views on UML, MDA, AOP, and the future. Recorded at JAOO (jaoo.dk).

Trail Ridge Consulting is interested in knowing what tools your team and organization use to enable, manage and scale your agile process. You could take the online survey before October 31, 2006.

(Via InfoQ)