I saw the news item on InfoQ regarding MicroSoft Oslo. There was quite a bit marketing mumbo jumbo surrounding Oslo, SOA, BPM, composite applications and MDD, and not a lot of it resonated with me all that much. However, one quote did stand out.

We want to send the model to the server not the printer.

This got me thinking about Big Software in general, even if it isn’t about SOA, BPM, and a lot of other standard industry buzzwords. Developing large enterprise systems comes with a lot of baggage that smaller development efforts don’t have, and a larger codebase is part of it. There are a lot of techniques that teams apply in attempts to architect, maintain and downright understand large software systems. In fact, the problem understanding Big Software is one of the compelling forces that led me to create JarAnalyzer. Since no one person can understand all detailed aspects of a large system, give people high level visual insight into the component relationships and let them drill down from that point. Unfortunately, there is no “drill down” capability in JarAnalyzer - you only get the .jar relationships and a few metrics. But combine it with tools like JDepend, and UMLGraph and you can get a few different views. Either way, there are quite a few different techniques that can be used when it comes to dealing with large codebases. But it seems like most approaches have serious holes.

• UML Modeling. UML Modeling is typically associated with Big Design Up-Front. While I don’t agree with detailed design up-front, establishing some idea of how you want to approach design is a good idea. I don’t like designing individual use cases, but do find value in modeling application frameworks, for example. There are a few advantages in doing this. The entire development team understands the general direction. Nothing is ever set in stone, so if something changes, it also serves as a vehicle for communication. Creating a design model also offers you the opportunity to structure the model in a way that best fits your team and how developers will want to interact with and navigate the model. Unfortunately, the greatest problem with modeling is that keeping your model in sync with the source code is impossible, and most reverse engineering solutions suck. So, any modeling done needs to be at a sufficiently high enough level of abstraction such that detailed change doesn’t impact it, or the models must be short-lived, serve their immediate purpose, and then get filed in the trash bin. Generally, I like to do a bit of modeling early, followed by TDD to drive out the details of my designs.
• Post-Construction Software Visualization. Quite a few static structural analysis tools generate some form of diagram. JarAnalyzer, JDepend, and UMLGraph are examples. Some exist for the .Net platform, too. The obvious advantage is that you get an up-to-date accurate snapshot of your software system. That’s valuable. Unfortunately, the diagrams aren’t always laid out to give you the view into the system that you’d like. Tools like GraphViz do a great job laying out the diagram, but a diagram is just a single view into a system, and possibly not the view I’m looking for. They aren’t complete models, and since the output of these tools are usually images, they typically aren’t navigable. Static structure analysis is only half the battle…maybe less. Understanding class, package, and component relationships is important, but with frameworks like Spring, it’s virtually impossible to get a feel for run-time structure and behavior. Two packages might have no structural dependency between the two, but the placement of abstractions could result in a run-time relationship. I know of no tool that does a good job analyzing the run-time structure.
• Metrics. Many static analysis tools generate metrics and other type of feedback that provide insight into code quality. Most of these don’t necessarily help you understand a system better, but they can help you create a system that is more easily understandable. I like metrics to help guide refactoring, because the design as it’s manifest in code is not always the design as it’s envisioned in the mind of the developer. Metrics offer objective advice on the quality of your application that should be used subjectively. A great resource for analysis tools for Java/JEE development is java-source.net.
• DSM matrices. I haven’t done much work with Dependency (aka. Design) Structure Matrices, but I intend to explore. I’ve talked to a few co-workers who have found them useful.
• MDD. Model-Driven-Development has been thrown around the industry for a while. If anyone has actually had any great deal of success developing Big Software using MDD, I’d love to hear about it. I’m very skeptical.

If I had the perfect solution, it would be real-time visualization of the source, be it static or dynamic, with the ability to create different views of the system. I’d be able to start high-level, with either .jar or package relationships. Then drill down into the .jar and packages to view details. But I’d easily be able to filter out information that I didn’t want to see. For instance, I might want to see the class diagram for a particular package, but in a different diagram I might want to see the class diagram for only those classes that span packages while suppressing classes that have relationships only to other classes in a package. I’d like to capture the behavior of a running system as a sequence diagram, and I’d like to double-click on any class or object and view the source. It’d be perfect if all this would integrate with my Spring configuration files, so I can build that into a view of the system, too. Of course, I want my metrics. I’d like an API available so that I could write unit tests that validate my design. And most important, if I have all of this coolness, I want the ability to filter out the noise. Essentially, I’d like to combine the navigability and view aspects of modelling with the accuracy of post-construction visualization. I need to make sure my models are accurate, but I also want different views into the system depending on what I’m looking for. And I’d like this wonderful tool available as an Eclipse plug-in, standalone IDE, and Ant task. I want all of this, and I want it to perform well, too. Vendors might claim their tool does this for me, but I know it doesn’t.

What’s your wishlist?

For years, OSGi technology has flourished in the embedded systems and networked devices market. Till now, it’s remained a relatively obscure technology for the enterprise developer. Now, in part thanks to Eclipse, OSGi is emerging as a viable and valuable technology in the enterprise.

Read more

Design Rot

We’ve all experienced that sinking feeling when maintaining a piece of crappy software. Has my change broken the system in some unintended way? What is the ramification of my change on other parts of the system? If you’re lucky, and the system has a robust suite of unit tests, they can offer some support in proving your work. In practice, however, few systems have thorough automated test coverage. Mostly we’re alone, left to verify our changes as best as possible. We might privately criticize the original developers for creating such garbage. It certainly lends a plausble excuse in explaining why the maintenance effort is so costly or time-consuming. Or it might serve as the basis upon which we recommend a re-write. But mostly, we should wonder how it happened.

Read more

Because of it’s emerging significance and my increased interest, I’ll be posting a number of entries related to OSGi. I’ll share some ideas on how OSGi stands to transform enterprise Java software development and application delivery, present some samples that illustrate these ideas, and offer a few opinions regarding the JCP and the various JSR’s that intend to accomplish similar ideals. I’m hopeful that my unbiased opinion will help developers more fully understand the value that OSGi brings to the enterprise. To help you quickly find all OSGi related entries on this site, I’ve created a separate blog category under which I’ll file these posts. I’ve also added a separate page that links to a number of other useful OSGi specific articles, websites, and blogs. If you are aware of an on-line resource related to OSGi, or know of one not on the current list, let me know and I’ll add it.

For quite some time, I’ve felt that the lack of a component technology has left a gaping hole on the Java platform. In fact, this is what led me to begin development on JarAnalyzer back in 2001 in the hope that I could help developers understand the component modularity of their existing applications. It’s also what led me to found Extensible Java in the hope that I could offer some design guidelines surrounding component modularity when developing large enterprise software systems in Java. The idea of a component technology that brings greater modularity to the Java platform is very exciting. This is my first OSGi-related post, so let’s start at the beginning. Why?

The distiction between software architecture and design is subtle, and countless trees have been sacrificed attempting to differentiate between the two. Booch sums it well, pointing out that it’s the cost of change separating the two.

All architecture is design but not all design is architecture. Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change.

Cost of change can be measured in many different ways, and is contextual. Financial cost, technological risk, and derived business value are examples of what might factor into that cost. However you slice it and dice it, the business value of design is to craft a software design that minimizes the cost of change by maximizing adaptability to change.

The ability of software to adapt, however, is not constant. As unexpected shifts in technology and business occur, the original design may not accommodate current needs. Should this occur in an architecturally significant area of the system, the cost of applying the correct change may exceed the immediate business value. As this happens throughout the life of a software system, software design degrades to the point where most change is no longer cost effective, and the temptation to hack a compromised solution is great. As this begins, and continues to occur, the software is beginning to rot.

Fowler discusses Design Stamina Health in making the compelling case that software design is a worthwhile activity. I doubt many of us disagree with his conclusion, assuming that the resulting design realizes the intended goal - that it can evolve. While good software designs are able to evolve based on the known factors today, the unforeseen factors of tomorrow reap havoc on design. I doubt we can ever entirely defeat these forces, but many techniques have been discovered that allow us to craft more adaptable software designs. Object-oriented development, design patterns, software code quality metrics, design quality principles, emergent design techniques such as Test-Driven Development, and Service Oriented Architecture all represent a positive step. Yet for large enterprise software systems, there is still a key ingredient missing in delaying design rot. On the horizon looms a disruptive technology, codename OSGi, that stands to redefine how we think about designing enterprise software on the Java platform.