Of course, there is value in widespread project exposure. Gaining experience with new technologies. Experiencing the dynamics of different teams. Understanding the challenges surrounding different organizational cultures. Each project is unique, bringing its own set of experiences, and these experiences are all incredibly valuable.

But when I see that someone has jumped from one project to another, it also leaves me wondering! Have they ever stuck around long enough on any single project to see their decisions through to the end? Have they ever had to live with the decisions they’ve made? An amazing compilation of knowledge is obtained by not only participating in the early stages of development, but also in maintaining the software system after it’s been released. To name just a few:

• Dealing with the challenges of Phase 2, while also maintaining Phase 1.
• Managing multiple branches of the software system. Merging those branches back together. Creating new branches.
  
• A critical production issue arising about the same time you need to start heading out for that important customer demo.
• Attempting to change a piece of code that hasn’t been touched in 10 months.
• Watching the software system grow from nothing to a system that’s more than 500,000 lines of code. Keeping the build performant. Keeping the build working!
• Being forced to live with seemingly meaningless early design decisions that haunt you over time.
• Trying to upgrade versions of third party libraries while development is ongoing. Simply recognizing the right time to upgrade.

And there is so much more. Priorities tugging you in five different directions at one time. If you’ve had the luxury of living with a system (and the mistakes) you created, you’ll realize that there are very few, if any, decisions that shouldn’t be given conscious thought.

When an individual sticks with a project for a long time, they realize the importance of maintaining a clean design, a robust suite of unit tests, and how they package their software system. There is significant knowledge gained by sticking with a project for a long period of time. Perhaps the next time you hire a developer, it might be wise to ask them, “What’s the longest you’ve spent on any given project?” That may be more important than the number of projects they’ve been on.

Somewhere along the way, I picked up a small piece of advice that has helped serve as a valuable guide. There is a big difference between ten years of experience and one year of experience ten times. When you jump from project to project, never living with the decisions you’ve made, you have lost an opportunity to learn what works well and what does not.

For this little exercise, I’ve decided to compare sample TCO over a period of time (ie. 2 years) for a Mac and a PC. This includes the initial purchase cost, ongoing support costs, and software purchases. It does not include any personal time that you would need to devote to maintaining and managing the device, such as reinstalling the operating system, cleaning up unused applications, dealing with anti-virus software and configuration, and general troubleshooting tasks.

I’ve made my most valiant effort to present an unbiased view and have arguably given favor to the PC. It would have been easy to close the price gap much more quickly if I were to toss in a few not-so-easy to measure items, such as the likelihood that you’ll have to take your PC to a professional to have it saved. But I’m sticking with hard numbers; the stuff that’s easy to measure. Let’s get started.

The Initial Purchase

I picked two models, fairly equal in their capabilities - a Dell Inspiron 15 and a 15″ MacBook Pro. When purchasing the Dell, I had to configure it so it had specs equal to the Mac. Let’s look at purchasing and configuring the Mac first.

The MacBook Pro

The 15″ MacBook Pro we’re going to purchase comes already loaded, so we aren’t going to make many customizations to it. We’re going to go with the entry level model, which prices out at $1699. The general specs include a 2.53 GHz Intel Core 2 duo processor with 4 GB of RAM. All we’re going to do is increase the hard drive to 320 GB, which adds $50 to the price, and purchase iWork, which adds $49 (Pages, Numbers, and Keynote). For those not familiar with iWork, this is what we’ll use on the Mac instead of Microsoft Office.

Total price for the MacBook Pro: $1798.00.

The Dell Inspiron 15

Keep in mind that the Inspiron is under Dell’s “Home” category. What this generally means is that it comes pre-loaded with a whole bunch of useless trial software that expires in either 30, 60, or 90 days. After that initial trial period, you’ll have to purchase a license to continue using it. For a business, you’d probably purchase the Latitude to avoid all of this unnecessary garbage. To keep the price down, I’m going to go with the Inspiron 15. See…told you I was giving favor to the PC!

To get a somewhat even comparison, I’m going to have to customize this thing to bring it up to equal specs as that of the Mac we’re going to purchase. The initial cost of the base machine is $499. But without any further customizations, that machine doesn’t offer much horsepower. First, I’ve got to add a Core 2 duo processor clocking in at 2.53 GHz. This adds $225 to the price.  We do need the wireless -N card, which adds $40. Why would this not be standard? Odd!

Next is Microsoft Office Home and Student 2007, which adds $119. We need this if we want to create spreadsheets, documents, or presentations. I recognize I could have gone with OpenOffice, but I find that most people choose Microsoft Office, so that’s what I’m doing here. Since we’re running Windows, we need anti-virus software. With the Dell model, there’s an option to purchase a 36 month license of McAfee. I’m going to jump on that because it only adds $40 to the price.

At this point, we’re finished with the Dell. There are a few things we could have done to even out the playing field a bit more. For an additional $45, we could have upgraded to the 9-cell battery that would have given us up to 8 hours of battery life. Keep in mind the Mac has a battery that gives us about 7 hours. Additionally, the version of Microsoft Office we purchased doesn’t come with Outlook, so we’ll still need to find an e-mail client once we get our laptop. But I just couldn’t see spending another $160 to purchase the Small Business edition that comes equipped with Outlook. The Mac will have Mail.app preinstalled for us, which is a nice e-mail client with Exchange integration. Already, the cost of software licensing is a sign of things to come that’s going to factor heavily into TCO.

Total price for the Dell: $883.

The Software

While these are each good machines, to get great things done requires software. Each of us have different needs surrounding the software we use, so I’ll offer up a few samples based on what I use. It won’t take long to get a clear understanding of the significant impact this has on TCO. Choose your own software stack, and see how it adds up. Then, let me know if you’re findings are consistent with mine.

First, I want to purchase software that allows me to create some nice drawings. For the Mac, I’m going to purchase OmniGraffle, which comes in two versions, standard ($99.95) and professional ($199.95). For Windows, Visio is a good option, which also comes in two different versions, standard ($259.95) and professional ($559.95). Let’s go the cheaper route on each. This brings the total price for the Mac to $1897.95 and the price of the Dell to $1082.95. Note: If I had purchased the professional version of each, the TCO at this point would be 1997.95 and 1382.95, respectively.

Next, I want some screen capture software so I can create some short training videos for online publication. For this, we’ll use Camtasia. The Mac edition of Camtasia prices out at $99.00, and the Windows version comes in at a rather lofty $299.00. TCO for the Mac is now at $1996.95 and the Dell is at $1381.95.

See a pattern developing here yet?

Keeping this post at a reasonable length, I’m only going to make one more purchase. At some point throughout my time as owner of one of these wonderful products, it’s likely that a new version of an operating system is going to arrive. And I’m going to want to upgrade. When Snow Leopard for Mac hit the market, the cost to upgrade was a paltry $9.95. Alternatively, the price to upgrade to Windows 7 Home Premium comes in around $119.95. In fact, a full license of Snow Leopard is only $29.95. A full license of Windows 7 Home Premium? $199.99! Again, the pattern is developing. TCO for the Mac is now $2006.90, while the Dell is at $1501.90.

Additional Analysis

At this point, the Mac is still slightly more expensive. But an initial price difference of $1200 was quickly dwindled down to just over $500 by the time we were finished. Clearly, the stark difference in the initial price is quite different from TCO. I could easily keep going, purchasing software that I want and need, and each time find that the Mac edition of the software comes in priced far below the Windows version. But I think I’ve illustrated the pattern here. The cost of software for Windows is more expensive than corresponding software for the Mac, and throughout the lifetime of ownership, TCO for Windows will approach, if not exceed, that of the Mac. For instance, Apple offers the option of purchasing a 5-license family pack of iWork for a mere $20 more than a single copy. A 5-license copy of Office is going to be quite expensive ($595).

Additionally, I have not broached the subject surrounding the time you’ll spend maintaining the Windows machine to keep it running smoothly. In four years on a Mac, I haven’t devoted any effort to tuning the machine for performance. It just works. After about six months on a Windows machine, I find that I’m always tinkering with it to maintain optimal performance. Since I can’t put an accurate price tag on this, I haven’t discussed it. But it is significant. Ok, I’m a little biased here, aren’t I?

Finally, I could have gone with the 13″ MacBook Pro with the exact same specs (just a smaller display) as the Inspiron 15 for an entry price of $1448.00. And the 13″ Mac is a great machine. After purchasing the software I want, the price for the 13″ Mac comes to 1656.90. That’s only $155.00 more for the Mac!

The Clear Winner Is…

The winner…you decide! All too often we compare Mac and PCs based on initial cost, failing to factor in the longer term TCO. When comparing Apples-to-Apples (errr…I mean PCs) though, the difference isn’t as stark. If you want a low end machine with a short lifespan, Dell (and many other PC manufacturers using Windows) offer this, including their line of netbooks.

It is possible to purchase a Dell for under $500. But remember, you get what you pay for. And side-by-side, machines of equal horsepower and capability are going to result in similarly equal TCO. By the time you spec out the machines evenly, and factor in TCO, the price of a Mac is arguably equal too, if not less than, that of a Windows PC. Either way, I know what my choice is!

In the past, I’ve talked about the importance of designing the joints of a system. OSGi services represent these joints, since services are essentially a module’s published interface. In general, if the implementation details of a module are well encapsulated, then implementation details can change without impacting the rest of the system. In other words, we minimize that nasty ripple affect of change if we confine change to a single module. But changes that span modules is nasty. Let’s take a few visuals to illustrate the point here.

Where’s the Architecture

Commonly, we visualize the module structure of a software system using a simple diagram similar to what’s shown at right (click to enlarge). We see the modules and the relationships between them. Of course, the relationships between modules represent the joints. Pretty simple stuff! Unfortunately, our attention is immediately drawn to the modules themselves, but not so much the joints.

And we really must be much more concerned about the joints than we are the implementation details because changes in the joints of the system are more complex and costly. This is where we need flexibility and it’s also where we need stability. Emphasizing module implementation is important, but flexibility in the joints is more important.

Illustrating the Joints

In the session, Peter annotated the diagram using a new convention, which I’ve depicted in the diagram at right (click to enlarge). The red triangles more clearly illustrate the joints. Here, our attention is drawn more toward the design constructs that span module boundaries. No longer is emphasis placed on the implementation details, but instead has moved to focus on the relationships between the modules. And the direction of the red triangle also helps illustrate the direction of the dependency relationship between the modules. In other words, these red triangles are the services.

The Real Architecture

Now, because the module itself represents behavior that is encapsulated, we can remove the modules from the diagram altogether and show only the joints, as shown at right (click to enlarge). This is where the impact and cost of change is most significant. And this is where we really need to ensure we have the greatest flexibility. These are the decisions that are most architecturally significant.

A Short Digression

Now I don’t think it’s necessarily important to start drawing architecture diagrams this way. In fact, I’ve never felt that it’s critically important to draw these pretty diagrams in the first place. While diagrams (and documentation) can be useful, they are little more than a pimple on software development’s ass. There are much more important things to worry about than diagrams. But I digress, and in general, this simple exercise helps recognize the importance of designing the joints of the software system.

The Real Point

Of course, while this example illustrates the point using modules, it’s not just the joints between modules that are important, but the joints at many different levels of the system, including applications, services, packages, and classes. In other words, the point where two entities connect is where the real architecture lives.

The Cost of Complexity

The title of the session is OSGi in the Enterprise - Agility, Modularity, and Architecture’s Paradox. We all know that OSGi isn’t a new technology. It’s been around for over a decade. But the recent surge in interest in modularity on the Java platform is overdue. The complexity of software is increasing exponentially. Did you know:

• In 1990, there were 120 billion lines of code
• in 2000, there were 250 billion lines of code
• The number of lines of code doubles every 7 years
• 50% of development time is spent understanding code
• 90% of software cost is maintenance and evolution

Let’s put this in perspective using the visual at right (click to enlarge), where we can see that the amount of code doubles every seven years. Pretty staggering! But let’s put this in perspective for what it means over the course of the next seven years, as well. It means that between 2010 and 2017, we’ll produce more code than the total amount of code ever written combined! That’s staggering!

I decided to do a bit of research to see if I could find examples that supported these claims. I did:

• Since the Spring framework was released in 2002, the number of lines of code has grown more than 500% through the release of Spring framework 2.5 in 2008.
• FreeBSD contained roughly 8 million lines of code in 2002. In 2009, it was close to 16 million.
• In 2004, the Linux Kernel contained approximately 6 million lines of code. In 2009, it was around 12 million.

I’m sure many of us have experienced this phenomenon. Systems tend not to get smaller over time. And we also recognize that in almost every way, larger software systems are inherently more difficult to design, build, manage, and maintain than are smaller software systems.

Yet, such phenomenal growth is desirable. We can only hope that the systems we create are in such high demand that we have the opportunity to grow them over time. A software system that doesn’t change, dies. Evolution is a big deal! As Lehman’s law suggests:

As a system evolves, its complexity increases unless work is done to maintain or reduce it.

Add it all up, and there are some key takeaways here. We need something that will help us understand complex systems. We need something that help manage the complexity. We need something that will help ease maintenance. We need something that will help us deal with the natural evolution of software systems. We need something that will allow us deal with the natural architectural shifts that occur as a system grows to accommodate demand. For a long time, a central ingredient has been missing. But not for much longer, because the enterprise will get its OSGi!

For the rest of the story, see you in London.

Source of statistical information above: http://users.jyu.fi/~koskinen/smcosts.htm

It’s housecleaning time again, and like last time, I stumbled across an article I wrote back in 2006 that I don’t believe ever reached publication (at least, I don’t think it did…how am I expected to remember what I did in 2006?). For the most part, I’ve left it in its original state, except that I removed the Agile Manifesto and 12 supporting principles. There are easily enough found on the Agile Manifesto website, and the article is long enough without this duplication. The wordle at right shows the most common words used in this document (click to enlarge). Here, in it’s otherwise unadulterated glory, is Agile: A New Era of Software Development.

Agile: A New Era of Software Development

Embrace Change

Writing code is easy, but developing software is hard. While syntax errors are common, their severity pales in comparison to the logic flaws inherent in many software systems. The difficulty in software development is not writing code or applying a certain technology stack. Instead, the challenge lies in the specification and design of the software itself.  Therein lies the essential complexity of software development, an idea introduced by Frederick Brooks in his 1987 article titled, “No Silver Bullet” [Brooks]. The most difficult aspect of software development is deciding what, exactly, needs to be built.

There is certainly evidence backing this claim. The original Chaos Report shows the top three impediments to a successful development effort are lack of user input, incomplete requirements and specifications, and changing requirements and specifications [CHAOS]. No other activity, if done incorrectly, stands to compromise the system more than incorrect requirement specifications.

It might not be so difficult were software a concrete entity, existing in a world where we could easily visualize it’s structure and behavior, allowing us to more reliably assess and share the impact of change. But software is a highly conceptual, invisible construct. It is considered infinitely malleable by those not intimately familiar with the conceptual complexity of it’s structure and behavior. The contractor building your home would look at you with incredulous disbelief if you suggested that the house he has 90% complete no longer met your needs, and you asked that he move walls. Or imagine how ridiculous it would sound to suggest that a new third floor be inserted to a 100 story skyscraper. Physicists labor on with firm belief that there exist an underlying set of unifying principles to serve as guidance. Or at least, there are laws of physics that we hold to be true. There are no such rules or principles that guide software development. We are left with the imagination and dreams of our clients, and they demand and deserve rapid response to change.

We have made valiant attempts at conformity. Ceremonial processes attempting to define standardized activities that guide the development process have failed, however. We cannot define detailed up-front requirements specifications and expect them to survive the development lifecycle intact. We cannot establish an initial design of the conceptual construct and expect the structure to go unscathed throughout the process of construction. Software development is an error prone human activity involving experts with varying backgrounds and skills who must come together and attempt to communicate uniformly, working as a team toward a common goal. While tools and process do help, we must also accept that change is expected. We cannot treat change as evil. Instead, the tools and process used must allow us to accommodate change, treating it as an inherent part of software development. Changing requirements is a rule of our game. The software we develop must be malleable and adaptive to change, and the process we use must embrace change.

We often draw comparisons between software development and various manufacturing processes. As Larman points out, however, manufacturing is a predictive process [Larman]. Herein lies one of the greatest differences between software development and the manufacturing processes to which we often draw comparisons. Manufacturing is a repeatable activity, with high rates of near-identical creation where a consistent product is produced in assembly line fashion. Little change is expected, making it possible to reliably estimate and predict the outcome. Software development is much more like new product development, where evolutionary specifications and adaptive planning is necessary to deal with the many unknowns that lie ahead.

Agile Principles

In early 2001, a small group of industry experts converged in Utah to discuss alternatives to heavy, document driven software development methods. Emerging from this meeting was the Agile Manifesto, a symbolic proclamation endorsing the virtues of a lighter, more flexible, people-oriented approach to software development, giving birth to the agile software development movement. (Since this is already a long article, I’ve snipped the manifesto and principles, which were included in the original version. If you’re interested, you can view the manifesto and its 12 principles on the Agile Manifesto website.)

<Snipped the Manifesto for Agile Software Development and 12 Principles>

The ideas behind these 12 principles are simple, and contain no hidden messages. Of course, there are different techniques embodied within various agile processes that support these principles. The one certainty is that agile teams definitely work differently from their less agile peers. They recognize there is one end goal - to create a working, functional software product. With that in mind, they work very closely with project stakeholders throughout the development lifecycle, knowing it is the stakeholders who possess the knowledge the system must embody. Agile teams work very hard to deliver working software iteratively and incrementally, and they adopt techniques representative of that ideal.

Agile project managers tend to favor intense communication and collaboration over heavy documentation. Empowering team members to make decisions enables responsiveness to change. Facilitating and negotiating requirements scope provides important feedback, helping plan future iterations, where each iteration produces a deliverable that can be shared with clients and stakeholders. Instead of forcing the team to follow a predictive project plan, agile project managers are more opportunistic. They prioritize features based on stakeholder feedback, and make adjustments as the iterations progress. Concurrent and parallel activities are favored over a phased approach. Agile project managers tend to guide the team instead of manage the team, and strongly discourage unnecessary overhead.

Agile developers work with a similar set of goals, knowing functional software must be delivered early and often. They work judiciously to grow a code base built upon a solid foundation, where each day represents a step forward. They integrate frequently, and do not tolerate failed builds. A rich suite of tests provide the courage necessary to respond to change when the need arises. They avoid the notion of code ownership, empowering other developers to make improvements to any component of the software product.

A common misconception is that agile processes discourage all documentation. This is untrue. Agile processes discourage unnecessary documentation, favoring collaboration as the preferred technique. Instead of using documentation to drive communication, agile processes favor face-to-face communication. Documents are encouraged by agile processes, so long as the need is immediate and significant.

Transitioning to Agile

Agile software development is based upon the fundamental premise that we must drive and respond to change quickly. The Agile Manifesto and 12 supporting principles serve this premise well. Advocates of agility claim speedier delivery of software, software with more business value, increased team productivity, higher quality systems, and a more enjoyable development experience. I believe each of these to hold true. Agile teams not only welcome change, they are able to respond to change at all levels of development. A project manager might discuss a changing requirement with a business client, empower a business analyst to schedule a meeting with the client to discuss further details, while a developer assesses the impact of change knowing she has the courage to accommodate the request because of the rich suite of unit tests in place.

Saying you’ll be more responsive to change and creating an environment that embraces change are separate beasts, however. Practicing agility is hard work, especially if your team is accustomed to more traditional approaches. As with many things new and unfamiliar, some resistance will no doubt arise by those who aren’t fully convinced. Agile projects differ greatly from their less agile counterparts, and skeptics will have many opportunities to express their discontent. As someone experimenting with agility, you may even have doubts. But don’t be discouraged, and give your agile transition the time it deserves.

One of the most significant changes you may experience is a feeling that you’ve been thrust into a chaotic nightmare. I doubt it’s unusual to feel this way. You’ve lost the security of detailed requirements specification and user sign-off. You are writing code without the comfort of knowing exactly what your stakeholders want. The detailed plans that have served as your security blanket on past projects no longer exist. And the celebrations accompanying completion of your various phase milestones are gone. Of course, these were all false comforts anyway. Stakeholders always changed their minds. Your detailed requirements and plans were outdated as quickly as they were completed.

Instead, you’re now working in shorter iterations with vague requirements. Initial releases early in the lifecycle may be completely thrown away. Your first few weeks may seem wasted, with little or no valuable artifacts produced. Naysayers will immediately come forward and cite the lack of progress. Previously, those first few weeks or months were spent producing very detailed requirement specifications and beautiful design models. But don’t give up yet. In that previous world, you were only delaying risk and postponing integration, avoiding the most difficult aspect of software development until the end of the lifecycle. Now you’re attacking risk early, prioritizing features, and working hard to develop a functional piece of software as early as possible. Progress may not be at breakneck speeds, but you are learning a tremendous amount about the requirements of the system, and your velocity is sustainable. Additionally, you are also performing a number of other important lifecycle activities.

Depending on the level of ceremony and bureaucracy within your organization, you will experience varying degrees of success when adopting agile techniques. As with any new technology adoption, it’s best to phase the transition. Some agile techniques are easier to adopt than others, and some serve as valuable catalysts to adopting additional techniques in the future. Don’t attempt to completely redefine how you work. It’s relatively easy to phase the agile transition, and you’ll want to adopt those principles that offer you the greatest initial reward.

For instance, if you’re struggling to produce quality software at a consistent rate, implementing a continuous integration strategy will help you frequently verify the quality of your work. In addition to the comfort of knowing you have a product always in a functional state, the ability to share the product with clients using functional demos and prototypes will tighten the feedback loop and offer valuable insight to the client’s perception of the software. In a number of cases, I’ve found this to be valuable in identifying subtle requirements that can be difficult to identify in other requirements elicitation venues.

Empirical Evidence

In recent years, there has been a significant amount of research comparing agile development methods to their waterfall counterpart. In Agile and Iterative Development: A Manager’s Guide, Craig Larman illustrates the advantage of agile development through detailed analysis of multiple studies[Larman]. The compilation of his results are illustrated below.

A study by Alan MacCormack at Harvard Business School explored whether evolutionary development techniques yielded better results than the waterfall model. The study included applications ranging from application software to embedded systems, with median values of nine developers spanning a 14 month development cycle.

A key conclusion of the study, in which 75% of participants used agile techniques compared to 25% using waterfall, explained releasing software earlier, rather than later, contributed to a lower defect rate and higher productivity. There was little evidence showing that a detailed design specification resulted in a lower defect rate, however, reviews with peers did help in reducing the rate of defects. The study found that iterative and agile practices have a significant impact on defect and productivity factors, as indicated by the following points.

• Releasing a system with 20% of the functionality complete is associated with a decrease in the defect rate of 10 defects per month per million lines of code as compared to waiting to release a product until 40% of the functionality is complete, and an increase in productivity of eight more lines of source code per person-day.
• Continuous Integration, the idea of integrating and testing code as it is released to your source code repository, resulted in a decrease in the defect rate of 13 defects per month per million lines of code, and an increase in productivity of 17 lines of source code per person-day.

The study also found four practices that were consistently used by the most successful development teams. The first two are deeply embedded in the ideals of agile software development.

• Releasing early and often to project stakeholders, using an iterative lifecycle.
• Continuous integration, with daily builds including regression testing.
• Teams with broad experience delivering multiple projects.
• Careful attention to modular and loosely coupled, componentized architectures.

In a separate study [Shine], 88% of organizations cited improved productivity when using agile methods, and 84% cited improved productivity. 49% stated that the cost of development was less when using agile methods. Additionally, 83% claimed increased business satisfaction and 26% claimed significantly better satisfaction. Another study by Boehm and Papaccio [Boehm] discovered that a typical project experiences a 25% change in requirements, while yet another [Johnson] showed  that 45% of features were never used.

There have also been many research efforts devoted exclusively to the analysis of waterfall methods. Below is a summary of these findings, taken from a variety of studies.

• Scope management related to detailed up-front requirements was a significant contributing factor of failure [Thomas].
• The U.S. Department of Defense (DoD), when following a waterfall lifecycle, experienced a 75% failure rate [Jarzombek]. This resulted in the DoD adopting a more iterative and agile approach.
• On a study including 400 waterfall projects, only 10% of the code was deployed. Only 20% of code deployed was used. The main factors included changing and misunderstood requirements [Cohen].

As these studies clearly illustrate, there is significant evidence showing that agile and iterative techniques offer significant advantages over the waterfall model of development. In fact, for larger projects, the statistics supporting agility were even more pronounced.

Conclusion

There are a variety of agile processes available to choose from, and each abide by the spirit of the manifesto and it’s 12 supporting principles. The agile movement and it’s supporters recognize that software development is a human (though not always humane) activity. Instead of forcing process on people, agile methods allow process conformance to people. Good people, working toward a common goal, can achieve great things will little ceremonial process, assuming you give them an environment that empowers them. Solid empirical evidence backs this claim. And if the quality of people is in question, it’s doubtful that any process will produce success.

References

• [Alliance]. The Agile Alliance. Manifesto for Agile Software Development. 2001. http://www.agilemanifesto.org
• [Boehm]. Boehm, B, and Papaccio, P. Understanding and Controlling Software Costs. IEEE Transaction on Software Engineering. October 1988.
• [Brooks]. Brooks, Frederick. No Silver Bullet: Essence and Accidents of Software Engineering. 1987.
• [CHAOS]. The Standish Group International, Inc. The CHAOS Report. 1995.
• [Cohen]. Cohen, D., Larson, G., and Ware, B. Improving Software Investments through Requirements Validation. IEEE 26th Software Engineering Workshop. 2001.
• [Jarzombek]. Jarzombek, J. The 5th Annual JAWS S3 Proceedings. 1999.
• [Johnson]. Johnson, J. Keynote speech, XP 2002, Sardinia, Italy. 2002.
• [Larman]. Larman, Craig. Agile and Iterative Development: A Manager’s Guide. Addison-Wesley, 2004.
• [MacCormack]. MacCormack, A. Product-Development Practices That Work. MIT Sloan Management Review. 2001.
• [Shine]. Corporate Report. Agile Methodologies Survey Results. Shine Technologies Pty Ltd. Victoria, Australia. 2003.
• [Thomas]. Thomas, M. IT Projects Sink or Swim. British Computer Society Review. 2001.

On July 25th, 2000, flight 4590 crashed. It was the first, and only, crash of the famed Concorde. Eventually, it would lead to retirement for the amazing aircraft. Investigators spent countless hours poring over the wreckage, and placed blame on a piece of runway debris that slashed the tire. A piece of that tire struck one of the fuel tanks, causing it to rupture and the plane caught fire. Case closed, right? Not so fast.

Surely a small piece of runway debris shouldn’t bring down a commercial airliner! As it turns out, there is quite a bit of contention among experts surrounding other factors that may have contributed to the crash. Some argue that it was a complex chain of events, all coming together, that brought down the aircraft. The plane was missing a spacer between the two wheels, had too much fuel in the tank, attempted to takeoff in unstable conditions, and was overweight. The flight was also delayed, causing angst among the flight crew. Finally, a required daily runway inspection was not performed.

Perhaps, if the runway inspection had been performed, the piece of debris would have been spotted. Or had the aircraft not been overfueled, the piece of tire may not have caused an increase in fuel tank pressure that some say caused the tank to burst. Had the spacer been installed, it’s possible the tire would have never burst. A series of small events, each contributing in their own way, to the fatal crash.

No Matter How Small

The story reminds me of the importance of attention to detail in software development. Of how the aggregate of all of the small, seemingly insignificant decisions we make on a continuous basis can have long-term consequences on the future of the software system. Possibly even your organization. Every time you design a class, define a variable, write a test, create a package, build a module, modify a method, or make a design decision, you are affecting the future of your system in some unsuspecting way.

What may seem insignificant today, can have a detrimental affect tomorrow. Really, there are no small, insignificant decisions in software development. I’m reminded of how important it is to make conscious decisions that are given careful thought, no matter how small. It also reminded me of a poem on build automation and continuous integration that I read a while back on the Test Early blog.

FOR WANT OF A BUILD

For want of a build, a test case was not executed
For want of test case, a defect was not detected
For want of a defect report, a bad release was promoted
For want of a good release, a strategic customer was lost
For want of a customer, a development team was reduced
For want of developers, a product stagnated
For want of a product, a company was lost
And all for the want of a build…

Architects architect architecture!

Don’t let the triviality of this statement undermine its depth. While each of the three words are variations of the same thing, each have different contextual meaning. Let’s tease the statement apart to see what I mean.

• Architects - Humans create software architecture, and for architecture to be effective, we must also be able to understand the architecture. In Eliminate Architecture, I cited a definition of architecture that introduces the social dimension. Architects signify the social pillar.
• Architect - The way we arrive at architecture is through some process or series of steps. We might create diagrams or software architecture documents. We might write a little code (proofs, spikes, prototypes) to determine the viability of architecture. There are many different activities we perform as we create the architecture. Architect signifies the process pillar.
• Architecture - In Modularity and Architecture, I offered up a few industry definitions of architecture. Common keywords that span definitions include components, composition, interfaces, subsystems, and structure. Architecture signifies the technology pillar.

To ensure balance, we must give attention to each of the three pillars. Additionally, each pillar is related to the other. For instance, ignoring the social pillar impacts the other two in unexpected ways.

The Social Pillar

Turtles and Architecture generally focused on the social pillar of software architecture, but also talked a bit about how the technology pillar can increase understanding, visibility, and transparency. The general sentiment is that architects must focus on more than just concepts and developers must focus on more than just code. There is a middle ground that demands attention, as well.

I used a visual similar to what’s at right (click to enlarge) to illustrate the middle ground. It also illustrates how aspects of the technology pillar can help increase understanding and transparency of architecture. Increased understanding of the architecture hopefully leads to improved structural quality (technology pillar) and transparency of the process (process pillar).

The Technology Pillar

Architecture All the Way Down primarily focused on the technology pillar. The visual at right (click to enlarge) illustrates this view. Again, we see the huge gap that exists if we fail to emphasize the importance of modularity. The rightmost portion illustrates how modularity fills the gap - architecture all the way down. Of course, other gaps are also created if we ignore any of the other aspects, such as class design. Note that as we move from services to modules to packages and classes, we increase the granularity along the way. Our classes should not be as fine-grained as our modules, nor our modules as fine-grained as our services.

Additionally, each entity solves a different type of problem (or provides a different type of advantage), as illustrated by the bars at the bottom. All are units of composition, but only services and modules are units of deployment. Services are reused through distributed computing, whereas module reuse is constrained by process boundaries. The technology pillar affects the other pillars. An inflexible rigid architecture makes it difficult for people to understand and communicate (social pillar) and inhibits how quickly we’re able to respond to change (process pillar).

The Process Pillar

The process pillar is one that I’ve not spent much time discussing. Certainly, it’s important though, and includes various aspects like deferring commitment to significant architectural decisions, evolutionary and emergent architecture, and reversibility. The visual at right illustrates the process pillar (click to enlarge). It’s not as descriptive as the other diagrams, I admit.  Anyone have something better that illustrates the process pillar?

I did talk a little about these ideas in Agile Architecture, Lean Principles. But certainly more needs to be fleshed out surrounding the process pillar. This tends to be where most spend their time when discussing agile architecture, but the other pillars are certainly important. The process pillar affects the other pillars. A bad process accompanied by bad practices results in an inflexible architecture (technology pillar) that noone is able to understand (social pillar).

Oracle has offered us very little direction on their future plans. There was little fanfare surrounding WebLogic dm Server, with only sparse references and no official word from Oracle. I’m hopeful we can expect to hear some news of the direction they intend to take in their upcoming webcast, though I’m not expecting them to make any significant statements at such a fine level of technical detail. Coincidentally, Apple is holding their own event that very same day, and is expected to announce their new tablet. Let your imagination run wild. Those two events should make for a very exciting day.

Regardless, I’ve had quite a few conversations over the past several months surrounding modularity and OSGi. I have my suspicions on Oracle’s intent, though am reticent to share at this time. Read the APS post titled, Java Modularity - Time to Set Sail.