February 23 - Keynote titled OSGi in the Enterprise: Agility, Modularity, and Architecture’s Paradox
March 22 - 25 - Tutorial on Modular Architecture
June 14 - 17 - Sessions titled Turtles and Architecture and Patterns of Modular Architecture
July 26 - 30 - Two sessions on rich mobile applications and one on agile development. Half day tutorial on software process improvement.
Right on. I just won a Best Buy drawing worth $1000. Either that or I won a shiny new virus by clicking the link. Hmm...what to do. 2012-08-29
The opinions expressed on this site are my own, and not necessarily those of my employer.
I recently stumbled across the following e-mail. It was sent on Friday, March 01, 2002 at 2:12 pm to the project manager of a team I’d just joined. It has not been modified, aside from stripping out e-mail addresses and a few names in order to protect the innocent (and guilty) :-).
It was sent after I was brought onto a development team as a software developer at the beginning of February, 2002. I quickly realized the path the team was taking, and it was not good. When I arrived, the team was meeting with clients, developing design models, and spending a lot of time trying to figure out what to do and how to do it. But they weren’t actually doing anything.
The response was astounding. We began to change what we were doing. We started writing code, sharing the system with our customers, taking their feedback, and making improvements. It worked well, and several months later, we delivered a great system - on time. It was not easy. Some of the changes took months to implement, not because they were technically difficult, but because there were significant cultural and legacy process issues to overcome.
I still believe in the ideas expressed in this e-mail. I would send it again today, almost verbatim, if I were in a similar position.
Here are some of my initial thoughts. Some are a bit random, and if you’d like to discuss any of them, feel free to holler. I’m really dumping a lot of thoughts here, and I apologize for that. In addition, some represent personal recommendations based on what I’ve found useful on past projects. It’s quite possible that a lot of this stuff has been considered, but I just haven’t crossed paths with it yet. If you’d like to talk about this, either one-on-one, or in a group setting with Mike, Jim, Bill, Susan, etc., we might find it to be a valuable discussion.
- Architecture proof - A proof has many different levels of detail. I don’t feel that a proof needs to prove the feasibility of the entire application architecture. Instead, we might consider emphasizing the critical aspects only. For instance, CMS connectivity is a big issue. While we may want to prove that we can effectively pull documents from the CMS using Java and COM, I’m not sure it’s necessary to consider any business process related issues in the proof. The underlying technical implementation of any piece of the system can be separate from the business process supported. The proof need only emphasize this underlying technical implementation. As such, I’d favor many small proofs that test the ease with which we can connect to the CMS, Natural Adabas, CTG, etc. In addition, proofing will be something that we must continually do. New issues will arise months from now that will need to be proofed. We can, however, start a lot of this stuff right now.
- Horizontal frameworks - These are system components that are not reliant on the functional requirements. They represent complete and reusable components. For instance, items that fall into this category are database access (ensure a consistent db access mechanism regardless of whether we’re hitting db2, Natural Adabas, etc.), error handling, auditing, connection management, document generation, security. This represents a lot of the “plumbing” necessary to begin implementing the functional requirements using vertical frameworks. We can start a lot of this stuff right now, though we’ll try to keep it fairly lightweight to ensure we don’t over engineer.
- Vertical frameworks - These are dependent on the functional requirements. These vertical frameworks help enforce consistency across use case implementations. Vertical frameworks use a bunch of the horizontal frameworks, the results of the proofs, and add all the business functions necessary. Examples of this include workflow, journaling, and claims. We need the functional requirements to start this stuff.
- Development Environment - All developers need a consistent development environment set up. We need to make sure everyone is using consistent versions of Java classes, jars, etc. We can start this right now.
- Geography - It would be ideal if all team members could sit in a central location. Ideally, this location would have access to as many whiteboards as possible.
- Team Structure - We might consider a few different options on structuring the team. Presentation developers, back end developers, etc. would be one option. Another is to divvy up based on use case.
- Iterative - Shorter and full lifecycle iterations will hopefully help us minimize a lot of the typical late project woes (ie. integration, test, deployment).
- Continous Integration - We should always keep integration in mind. Ensuring a system is always completely integrated avoids the big bang integration nightmares at the end. While things will move more slowly at first, we should see the advantages of this as time progresses.
- Continous Testing - Developers must test the code they construct, and test it fairly often. It’s effective to write code that automates these test cases. This way, test cases can be run frequently. Top priority should be to fix any errors. If we can keep on top of this, we’ll *always* have an application that is as fully functional as the requirements it presently supports.
- Continous builds - Compile errors should be fixed immediately. Applications that don’t compile don’t work. When code is changed, it should be built, then tested, then integrated. An automated application build set up to run nightly would be valuable.
- End-to-End - After we’ve got some of the proofs in place, it’s important to implement an end-to-end solution as quickly as possible. This would be fairly lightweight, and may not do much to support the overall business process. By definition, I think this is what some of us are presently calling the architecture proof. The only problem with treating this as the architecture proof is that there are a lot of other things (horizontal frameworks, technical proofs, development environment, etc.) that should probably be done first. This end-to-end solution includes not just code, but a build process, testing, versioning, deployment, etc. As we build this end-to-end solution, we would continue to fill in the gaps of our horizontal frameworks. We would continue to add to this end-to-end solution over the course of many iterations.
To only a fraction of the human race does God give the privilege of
earning one’s bread doing what one would have gladly pursued free, for
I like that quote. But let me ask…
How many software developers have a strong desire to develop enterprise business applications? How many dream of working for banks, insurance companies, automobile manufacturers, retail chains, etc? These are places we start, not places we dream of. In general, developers do not aspire to develop enterprise software, but instead develop enterprise software while they aspire to do other things. Frankly, the jobs are boring, painful, arduous, and do not challenge. The enterprise is broken. Michael Nygard touches on this in explaining why enterprise applications suck.
So how do we fix it? To do great things, you have to learn to love the process. If you want to grow a beautiful garden, you have to learn to enjoy pulling weeds. If you want to build a beautiful courtyard, you have to learn to love landscaping. If you want to paint a masterpiece, you have to learn to love the stroke of the brush. It isn’t enough to just plant the seeds. It isn’t enough to just move dirt and pile block. It isn’t enough to just apply paint to the canvas. Likewise, if you want to develop great enterprise software, you have to learn to love the process. Fixing the enterprise demands that we first fix the process.
There are a lot of suggested ways to do this. Scrum. XP. Lean. CMMI. The list goes on. But the right process - the ideal process - is a product of people, culture, technology, project, and more. There is no one-size-fits-all process. Every project is different. Every team is different. Every organization is different. What should we do? How do we fix the process? The only way to fix it is to learn to love it.
The Agile 2009 program is live, and it’s going to be a great conference. Thank you to the Developer Jam review committee for all the effort they put into helping select the sessions. The selection process for Developer Jam was not easy due to the quantity of amazing submissions, but I’m confident we put together an exceptional program. Whittling down more than 100 submissions into less space than we had at Agile 2008 was challenging. In the end, we had enough space for just 26 sessions. I’m certain that Developer Jam would have been a great conference of it’s own. Any takers on organizing this conference?
In addition to the sessions you see listed, we’re also putting the finishing touches on Programming with the Stars, where select conference attendees will be paired with legendary agile programmers to perform live on stage before a panel of famous (and outspoken) judges. The audience will participate in the judging contest. Be sure to stay tuned to the Agile 2009 conference site for additional details on Programming with the Stars.
Software architecture is difficult to define. Ask five different developers their definition of software architecture, and you’ll likely get five different answers. Arguably though, we can agree that software architecture represents the significant technical decisions spanning the breadth of the system. For instance, managing the dependencies between modules is a significant aspect of software architecture. Developing application frameworks that promote consistency across use cases is architecturally significant. Identifying application layers and the behavioral granularity of those layers is architecturally significant, as well. As Booch states:
Architecture is design, but not all design is architecture. Rather, architecture focuses upon significant decisions, where significant is measured by the impact of changing that decision. For this reason, architectural decisions tend to concentrate upon identifying and controlling the seams in a system, which are described in terms of interfaces and mechanisms and so typically mark the units of change in the system.
Essentially, we differentiate between architecture and design based on the impact of change. How easy is it to modify system modules if dependencies are not carefully managed? How difficult is it to change the dependency structure? How easy is it to reuse software components or services if the behavioral granularity is incorrect? How easy is it to refactor behavioral granularity? Traditionally, we attempt to make the right architectural decisions early due to the significant anticipated cost affiliated with making incorrect decisions.
This contradicts agile practices which have taught us to avoid up front design and architecture, and instead embrace change throughout the software development lifecycle. So how do agile and architecture come together? Conceptually, the goal of agile architecture is to eliminate the architectural significance of change by crafting software that can easily adapt to change. I provide an example of how to do this by focusing on the joints within a system. In practice, developing agile architecture is much more difficult. There are two simple truths we must be willing to accept.
So how do agile practices promote agile architecture? To briefly paint a picture of agile architecture, we’ll explore a development scenario examining system growth and evolution.
Early in the lifecycle, change is rampant as new features are under development. An architectural vision that guides developers should be established that aligns technology adoption and implementation with business objectives. The time required to establish this initial architectural vision should be measured in days, not weeks or months.
Developers should stress closure by bundling code into larger modules, avoiding the unpredictability associated with complex implementation issues such as module granularity and transactionality spanning module boundaries. As the system grows, change and discovery occur less frequently. Modules should be broken apart, emphasizing independent deployment, clear separation of concerns, and even reuse. Smaller, self-contained modules are wired together via interfaces, further decoupling the structural relationships between modules.
Throughout the lifecycle, as business objectives change and new requirements emerge, the architectural vision evolves, yet still serves as the guide to implementation. Agile architecture is manifest as modules with minimal dependencies, concise behaviors, and clear points of extensibility. This drives architecture down to the class and code level, making the quality of your source code, and the coupling and cohesion of your classes, a critical factor in the resiliency of your architecture.
Agile practices play a critical role in agile architecture. Test-driven development lends you the courage to undergo architectural evolution. Continuous integration accommodates architectural shifts by ensuring issues encountered while refactoring are identified and corrected quickly. Supplementing your up-front design approach with an emergent strategy emphasizing spikes or proofs helps verify the architectural vision. Developing a robust infrastructure supporting incremental delivery allows your team to experience and identify architectural deficiencies related to performance, failover, usability, and more. Early and frequent discovery of inadequate architectural decisions provides the team with the time necessary to make architectural adjustments.
Shifts in architecture exists in a world beneath the trendy technologies du jour. It exists in a world separate from your decision on which web framework to use. Or which ORM framework you want to play with. SOA isn’t relied upon as a revolutionary architectural model. With agile architecture, technology becomes less significant because the architecture diminishes the impact of change. Crafting a robust architecture is not easy, but is at least possible if you allow architecture to evolve, and refactor as architecture must shift. Agile practices enable agile architecture.
That’s my definition of agile architecture. What’s yours?