February 23 - Keynote titled OSGi in the Enterprise: Agility, Modularity, and Architecture’s Paradox
March 22 - 25 - Tutorial on Modular Architecture
June 14 - 17 - Sessions titled Turtles and Architecture and Patterns of Modular Architecture
July 26 - 30 - Two sessions on rich mobile applications and one on agile development. Half day tutorial on software process improvement.
Right on. I just won a Best Buy drawing worth $1000. Either that or I won a shiny new virus by clicking the link. Hmm...what to do. 2012-08-29
The opinions expressed on this site are my own, and not necessarily those of my employer.
My book, Java Application Architecture: Modularity Patterns With Examples Using OSGi is now available. Uncle Bob and Peter Kriens each contributed Forewords to the book. The book itself is part of the Robert C. Martin series. The book is intended for all software developers interested in designing better software using modularity. Though the examples use Java, the techniques can be applied to other languages and platforms, such as .NET, with relative ease.
Even if you’re not using OSGi (or perhaps not even familiar with OSGi), I’m confident you’ll find the book valuable. The book (and patterns) has been designed to allow you to realize the benefits of modularity whether you’re using a module framework, or not. As Uncle Bob says in the Foreword, “This is how you build a Java application, people.” Peter sums it up nicely too, in saying “This book…will give you a view into the magic of modularity.”
You can order it online at Amazon (print edition & Kindle edition), iBooks, InformIT, or a number of other publishers. For more details on the book, please see the book’s website. Over the next couple of weeks, I plan to post a sample chapter or two that will give you a feel for the book’s contents.
Here’s what a few people have to say:
It seems my recent post on OSGi has ruffled a few feathers. I’ve also received a few personal e-mails suggesting that I have missed the mark because OSGi is a significant component of the next generation application platform. Major vendors such as IBM are building OSGi into their products, and other leading edge vendors leverage OSGi as an enabling technology upon which their technical architecture is built.
My intent wasn’t to indict OSGi, so I was pretty surprised with the defensive responses I received…it’s a great technology and the benefits of modularity are real. As I’ve mentioned, and as many already know, I’m an ardent supporter of OSGi and modularity.
Based on the type of the feedback I’ve gotten on my previous post though, I’ve come to the realization that various constituencies have different perspectives that is contributing to the perception of OSGi adoption. Let me explain.
In my post, The Two Faces of Modularity & OSGi, I talk about the runtime model and the development model. There are significant advantages to each. But interestingly, it’s possible for one to succeed while the other never sees widespread adoption. That is, OSGi might lie at the core of the next generation platform, but it doesn’t necessarily imply that enterprise developers will leverage it to build applications with a modular architecture.
Today, most vendors emphasize the runtime model, touting the cost savings, improved responsiveness, and flexibility of a dynamic platform. So I can understand why a lot of folks have reached out to explain to me the error of my ways. From their perspective, OSGi is flourishing.
An excellent example is the Eclipse platform, which released Eclipse 3.0 in June of 2004. However, I’d wager that only a fraction of Java developers using the Eclipse IDE know that OSGi is the major enabling technology upon which Eclipse is built. Instead, they use Eclipse, install plug-ins, but know very little about the underpinnings of the platform.
From a development perspective I’m incredibly interested in the benefits of modular architecture, and I want tools that reduce the accidental complexity of the development model. However, these tools don’t exist today, and developing enterprise server-side Java software applications leveraging OSGi is painful. So while vendors are leveraging OSGi for various platform benefits, as long as they encourage the traditional deployment model and the dearth of tools persist, enterprise developers will not use OSGi to build applications with a modular architecture. We don’t have it today, and we won’t see it in the future until this changes.
I do believe that OSGI has the potential to be a major component of the next generation application platform. Though there are no guarantees, and I still have concerns. Let’s take SpringSource as an example. After seeing almost zero adoption of dm Server, they donated it to the Eclipse Foundation. And the recent VMForce announcement makes nary a mention of OSGi. Instead, tc Server provides the execution environment. So while some might argue that the runtime platform benefits of OSGi are real, there is a far more significant trend called the cloud that provides many of the same advantages. And the cloud certainly doesn’t need OSGi. Though OSGi could thrive in the cloud.
I’m also not convinced that the benefits of modularity will be realized by enterprise development teams. Will enterprise developers leverage OSGi and modularity as a core component of next generation architecture? I hope so, but I don’t believe it’s the slam dunk that many believe it is. Modularity is important. The benefits are real. It helps teams overcome a serious challenge when developing large enterprise software systems. But we need more developer advocacy surrounding OSGi and the benefits of modular architecture. Software is a fashion industry, and we must attach modularity to something trendy and more aggressively advocate adoption.
It is what it is. And unfortunately, that’s just the way it is.
I’d like to take a short moment to offer an additional perspective to my discussion on OSGi: Complexity is NOT the Problem. I believe this perspective adds clarity to that previous discussion, as well. All initiated thanks to a tweet, which summed up the situation in much less than 140 characters.
So over the weekend, I turned to the essay that said tweet refers, and reviewed the essence and the accidents. From the article, I quote Mr. Brooks:
“…to see what rate of progress we can expect in software technology, let us examine its difficulties. Following Aristotle, I divide them into essence - the difficulties inherent in the nature of the software - and accidents - those difficulties that today attend its production but that are not inherent.”
This leads me, pretty clearly, to the following simple conclusion.
Today, development teams leveraging OSGi to build server-side applications are burdened by accidental complexity. As platforms and tools mature, the accidental complexity of OSGi will be reduced to near zero. Modularity though, enabled and enforced by OSGi, attacks the essence.
If the difference between essential and accidental complexity isn’t quite clear, I highly recommend you take a few moments and read Mr. Brook’s essay (linked above). And then, if you can find a copy of the book’s 2nd edition, take a look at Chapter 17, “No Silver Bullet” Refired (Note: If anyone can find the Refired essay online, please post the link in comments).
Throughout these two discussions, you’ll find subtle hints extolling the virtues of modularity. But only do this if you’re willing to exercise your brain with thought, because the connection you’ll discover might just be transformational!
Last week at OSGi DevCon, I attended Peter Krien’s Advanced OSGi tutorial. The tutorial focused on OSGi services, and I had just come off a rather lengthy e-mail discussion with Peter on a similar topic. So this was pretty good timing. I want to share with you one aspect of the discussion, and its relationship to architecture. It’s quite simple, though makes an important point.
In the past, I’ve talked about the importance of designing the joints of a system. OSGi services represent these joints, since services are essentially a module’s published interface. In general, if the implementation details of a module are well encapsulated, then implementation details can change without impacting the rest of the system. In other words, we minimize that nasty ripple affect of change if we confine change to a single module. But changes that span modules is nasty. Let’s take a few visuals to illustrate the point here.
Commonly, we visualize the module structure of a software system using a simple diagram similar to what’s shown at right (click to enlarge). We see the modules and the relationships between them. Of course, the relationships between modules represent the joints. Pretty simple stuff! Unfortunately, our attention is immediately drawn to the modules themselves, but not so much the joints.
And we really must be much more concerned about the joints than we are the implementation details because changes in the joints of the system are more complex and costly. This is where we need flexibility and it’s also where we need stability. Emphasizing module implementation is important, but flexibility in the joints is more important.
In the session, Peter annotated the diagram using a new convention, which I’ve depicted in the diagram at right (click to enlarge). The red triangles more clearly illustrate the joints. Here, our attention is drawn more toward the design constructs that span module boundaries. No longer is emphasis placed on the implementation details, but instead has moved to focus on the relationships between the modules. And the direction of the red triangle also helps illustrate the direction of the dependency relationship between the modules. In other words, these red triangles are the services.
Now, because the module itself represents behavior that is encapsulated, we can remove the modules from the diagram altogether and show only the joints, as shown at right (click to enlarge). This is where the impact and cost of change is most significant. And this is where we really need to ensure we have the greatest flexibility. These are the decisions that are most architecturally significant.
Now I don’t think it’s necessarily important to start drawing architecture diagrams this way. In fact, I’ve never felt that it’s critically important to draw these pretty diagrams in the first place. While diagrams (and documentation) can be useful, they are little more than a pimple on software development’s ass. There are much more important things to worry about than diagrams. But I digress, and in general, this simple exercise helps recognize the importance of designing the joints of the software system.
Of course, while this example illustrates the point using modules, it’s not just the joints between modules that are important, but the joints at many different levels of the system, including applications, services, packages, and classes. In other words, the point where two entities connect is where the real architecture lives.
After the OSGi DevCon keynote, I had a few folks stop by and ask me for the slides. They are now available on slideshare. Usually, when browsing the slides without discussion, the lack of context can cause some confusion. So if you have any questions regarding the content, please feel free to reach out. Enjoy!
I’ll be at OSGi DevCon next week, having been given the opportunity to present the keynote session on Tuesday. The conference is co-hosted with JAX London. I thought I’d take a moment before jumping on the flight to offer a preview of what I’ll be speaking about.
The title of the session is OSGi in the Enterprise - Agility, Modularity, and Architecture’s Paradox. We all know that OSGi isn’t a new technology. It’s been around for over a decade. But the recent surge in interest in modularity on the Java platform is overdue. The complexity of software is increasing exponentially. Did you know:
Let’s put this in perspective using the visual at right (click to enlarge), where we can see that the amount of code doubles every seven years. Pretty staggering! But let’s put this in perspective for what it means over the course of the next seven years, as well. It means that between 2010 and 2017, we’ll produce more code than the total amount of code ever written combined! That’s staggering!
I decided to do a bit of research to see if I could find examples that supported these claims. I did:
I’m sure many of us have experienced this phenomenon. Systems tend not to get smaller over time. And we also recognize that in almost every way, larger software systems are inherently more difficult to design, build, manage, and maintain than are smaller software systems.
Yet, such phenomenal growth is desirable. We can only hope that the systems we create are in such high demand that we have the opportunity to grow them over time. A software system that doesn’t change, dies. Evolution is a big deal! As Lehman’s law suggests:
As a system evolves, its complexity increases unless work is done to maintain or reduce it.
Add it all up, and there are some key takeaways here. We need something that will help us understand complex systems. We need something that help manage the complexity. We need something that will help ease maintenance. We need something that will help us deal with the natural evolution of software systems. We need something that will allow us deal with the natural architectural shifts that occur as a system grows to accommodate demand. For a long time, a central ingredient has been missing. But not for much longer, because the enterprise will get its OSGi!
For the rest of the story, see you in London.
Source of statistical information above: http://users.jyu.fi/~koskinen/smcosts.htm
For those that follow this blog, but not the APS blog, I’ve just published a synopsis of modularity on the Java platform. With a lot of momentum continuing to build around OSGi, we’re still in a state of limbo surrounding a standard module system on the Java platform. Some might argue that OSGi has already won - it is the defacto standard module system. Possibly.
Oracle has offered us very little direction on their future plans. There was little fanfare surrounding WebLogic dm Server, with only sparse references and no official word from Oracle. I’m hopeful we can expect to hear some news of the direction they intend to take in their upcoming webcast, though I’m not expecting them to make any significant statements at such a fine level of technical detail. Coincidentally, Apple is holding their own event that very same day, and is expected to announce their new tablet. Let your imagination run wild. Those two events should make for a very exciting day.
Regardless, I’ve had quite a few conversations over the past several months surrounding modularity and OSGi. I have my suspicions on Oracle’s intent, though am reticent to share at this time. Read the APS post titled, Java Modularity - Time to Set Sail.
2010 is going to be a big year for modularity on the Java platform. So based on recent industry trends, and as momentum continues to build around modularity on the Java platform:
I hereby officially declare 2010 as the year of modularity on the Java platform.
So that’s a relatively silly statement, and I really don’t have any authority to make such sweeping declarations. But the statement is backed by some pretty serious trends. Here are a few nuggets to chew on that has led me to believe modularity is going to be bigger in 2010 than ever before!
Between 2007 and 2009, we saw amazing momentum build around OSGi. GlassFish, Jetty, Spring DM, SpringSource dm Server, Paremus Service Fabric, Nimble, PAX, Sigil, Aries, WebSphere, JBoss, Maven, and of course Eclipse are just a few products that leverage OSGi. The list goes on. The number of 3rd party frameworks that have been “osgi-ified” has also grown substantially. And whether you love it or hate it, let’s not forget about Jigsaw and JSR-294.
But there has been a nagging problem…an elephant in the room you might say…that’s hindered widespread adoption. Without gluing together a platform, assembling it yourself, or adopting an entirely new platform, aside from the vendors who were already exposing the virtues of OSGi in their products, the majority of enterprise developers were unable to leverage any of it to build modular enterprise applications! But in 2010, that’s all going to change.
Modularity is coming to the enterprise and you’re going to be able to use it to build modular applications. IBM has already made this announcement. I suspect others will follow suit. For those using SpringSource dm Server or Infiniflow, you’re probably already doing it. With the major platform vendors showing their support for a technology that’s already embedded in products such as dm Server and Infiniflow, I suspect these platforms will see an increase in market penetration through 2010 and beyond. And I suspect you can expect some big announcements surrounding these products soon. Finally, let’s not forget, if it can avoid another delay, JDK 7 (with support for modularity) will be released in September.
Whereas prior to 2010, enterprise developers could only dream about building modular applications (or work very hard to do so), modularity is going to take center stage. And since modularity lies at the heart of the platform, it doesn’t matter what language you use. Neil showed us how to do it with Scala way back in 2007. And Groovy does OSGi, as does Clojure. Yep, so does Scala. So no matter which language you use, you’ll be able to take advantage of modularity on the Java platform.
In 2010 modularity on the Java platform is going to be a game changer…a disruptor…that reaches from the developer to the data center. As 2010 progresses, more and more developers will be touched by modularity. It’s not something we should ignore. Really, it’s not something we can ignore anymore.
This is my last post of the year, and I want to take a moment to thank everyone that spends their precious time reading my long-winded entries. In 2009, I saw a four fold increase in blog traffic over 2008, and December 2009 saw a five fold increase in traffic over January 2009. Out of a total of 74 posts this year, more than half were related to modularity. I’d like to think the increase in traffic is related to the topics I’ve been writing about. Modularity is something that folks are interested in. But we still have a ways to go.
Interestingly, I was speaking to a conference organizer yesterday, and I sensed his mild surprise when I began to talk about modularity. He noted that modularity is certainly not a new concept. I agreed. It’s an idea that has been around since at least the early 1970’s when David Parnas published his essay. But things are different now. Until recently, modularity was not a first class concept on the Java platform. If we wanted to develop software with a modular architecture, we were required to do so without much framework or platform support. In fact, we hadn’t even identified any standard definition of module on the platform.
That’s changing. As the application platform vendors continue to bake OSGi into their products, it’s clear that a module is a JAR file and there’s a framework to back it up. In 2010 modularity on the Java platform will gain considerable visibility in our industry. The stackless stack is coming to fruition, and it’s a game changer…a disruptor…that reaches from the developer to the data center. It’s not something we should ignore. Really, it’s not something we can ignore anymore. So, as we close out one decade and usher in another, modularity will celebrate at least its 40th birthday. Maybe we’re finally starting to get it.
A final note before closing out 2009. For those interested, I’ve created a custom Wordpress page that shows a summary of all posts published on this blog. Enjoy!
Note: This is a re-post, with slight modifications, from an entry in October 2007. And now, two years later, we’re just about there! For the abridged version of this post, focus on the text in bold. You can see the original version here.
We’ve all experienced that sinking feeling when maintaining a piece of crappy software. Has my change broken the system in some unintended way? What is the ramification of my change on other parts of the system? If you’re lucky, and the system has a robust suite of unit tests, they can offer some support in proving your work. In practice, however, few systems have thorough automated test coverage. Mostly we’re alone, left to verify our changes as best as possible. We might privately criticize the original developers for creating such garbage. It certainly lends a plausble excuse in explaining why the maintenance effort is so costly or time-consuming. Or it might serve as the basis upon which we recommend a re-write. But mostly, we should wonder how it happened.
For sure, most software doesn’t start out this way. Most software starts out clean, with a clear design strategy. But as the system grows over time, strange things begin to happen. Business rules change. Deadline pressures mount. Test coverage slips. Refactoring is a forgotten luxury. And the inherent flaws present in every initial design begin to surface. Reality has proven that few enterprise development teams have the time or resources to fix a broken design. More often, we are left to work within the constraints of the original design. As change continues, our compromises exacerbate the problem. The consequence of rotting design is seen throughout the enterprise on a daily basis. Most apparent is the affect on software maintenance. But rotting design leads to buggy software and performance degradation, as well. Over time, at least a portion of every enterprise software system experiences the problem of rotting design. A quote from Brook’s sums it well:
All repairs tend to destroy the structure, to increase the entropy and disorder of the system. Less and less effort is spent on fixing the original design flaws; more and more is spent on fixing flaws introduced by earlier fixes. As time passes, the system becomes less and less well-ordered. Sooner or later the fixing ceases to gain any ground. Each forward step is matched by a backward one. Although in principle usable forever, the system has worn out as a base for progress.
The most obvious question is, “How do we prevent rotting design?” Unfortunately, rotting design is not preventable, only reducable. Of the past ten years, the design patterns movement has provided insight to the qualities of good design. Dissecting design patterns reveals many important design principles that contribute to more resilient software design. Favor object composition over class inheritance, and program to an interface, not an implementation, are two examples. Of the 23 patterns in the GOF book, all adhere to these fundamental statements. Alone however, design patterns that emphasize class structure are not enough to help reduce rotting design.
Most patterns emphasize class design, and present techniques that can be used in specific contexts to minimize dependencies between classes. Teasing apart the underlying goal of most patterns shows us that each aim to manage the dependencies between classes through abstract coupling. Conceptually, classes with the fewest dependencies are highly reusable, extensible, and testable. The greatest influence in reducing design rot is minimizing unnecessary dependencies. Yet enterprise development involves creating many more entities beyond only classes. Teams must define the package structure in which those classes live, and the module structure in which they are deployed. Increasing the survivability of your design involves managing dependencies between all software entities - classes, packages, and modules.
But if minimal dependencies were the only traits of great design, developers would lean towards creating very heavy, self-contained software entities with a rich API. While these entities might have minimal dependencies, extreme attempts to minimize dependencies results in excessive redundancy across entities with each providing its own built-in implementation of common behavior. Ironically, avoiding redundant implementations, thereby maximizing reuse, requires that we delegate to external entities, increasing dependencies. Attempts to maximize reuse results in excessive dependencies and attempts to minimize dependencies results in excessive redundancy. Neither is ideal, and a gentle balance must be sought when defining the behavior, or granularity, of all software entities - classes, packages, and modules. For more on the use/reuse paradox, see Reuse: Is the Dream Dead?
Software design is in a constant quandary. Any single element key to crafting great designs, if taken to its individual extreme, results in directly the opposite - a brittle design. The essential complexity surrounding design is different for every software development effort. The ideal design for a software system is always the product of it’s current set of behavioral specifications. As behavior changes, so too must the granularity of the software entities and the dependencies between them. The most successful designs are not characterized by their initial brilliance, but instead through their ability to withstand the test, and evolve over the course, of time. As the complexity of software design is an essential complexity surrounding software development, our hopes lie with technologies and principles that help increase the ability of your design to survive. Such is the reason why agile architecture is so important.
I’m hopeful that all software developers have experienced the pleasure of a design that, through the course of time, has withstood the test of time. Unfortunately, many enterprise development teams have too few of these experiences. Likewise, few enterprise development teams devote adequate effort to package and module design. It’s unreasonable to believe that even the most flexible class structure can survive should the higher level software entities containing those classes not exhibit similarily flexible qualities. The problems are rampant. Increased dependencies between packages and modules inhibit reusability, hinder maintenance, prevent extensibility, restrict testability, and limit a developer’s ability to understand the ramification of change.
Services offer some promise to remedy our failures with object-oriented development. Yet, while services may offer tangible business value, within each awaits a rotting design. There exists a world between class design and web services that deserves more exploration, and as an industry, we are beginning to notice. OSGi is a proven module system for the Java platform, while Jigsaw aims to modularize the JDK. JSR-294 aims to improve modularity on the Java platform. While some friction might exist between the constituencies involved, it’s only because they too recognize that something has been missing, and are passionate about fixing the problem. Of course, it doesn’t stop there. A plethora of application servers and tools are also including support for modularity using OSGi, which has grown into the defacto standard module system on the Java platform.
All aim to help manage the complexity, from design through deployment, of enterprise software development. With each, new practices, heuristics, and patterns will emerge that increase the ability of a design to grow and adapt.