February 23 - Keynote titled OSGi in the Enterprise: Agility, Modularity, and Architecture’s Paradox
March 22 - 25 - Tutorial on Modular Architecture
June 14 - 17 - Sessions titled Turtles and Architecture and Patterns of Modular Architecture
July 26 - 30 - Two sessions on rich mobile applications and one on agile development. Half day tutorial on software process improvement.
Right on. I just won a Best Buy drawing worth $1000. Either that or I won a shiny new virus by clicking the link. Hmm...what to do. 2012-08-29
The opinions expressed on this site are my own, and not necessarily those of my employer.
Architects architect architecture!
Don’t let the triviality of this statement undermine its depth. While each of the three words are variations of the same thing, each have different contextual meaning. Let’s tease the statement apart to see what I mean.
To ensure balance, we must give attention to each of the three pillars. Additionally, each pillar is related to the other. For instance, ignoring the social pillar impacts the other two in unexpected ways.
Turtles and Architecture generally focused on the social pillar of software architecture, but also talked a bit about how the technology pillar can increase understanding, visibility, and transparency. The general sentiment is that architects must focus on more than just concepts and developers must focus on more than just code. There is a middle ground that demands attention, as well.
I used a visual similar to what’s at right (click to enlarge) to illustrate the middle ground. It also illustrates how aspects of the technology pillar can help increase understanding and transparency of architecture. Increased understanding of the architecture hopefully leads to improved structural quality (technology pillar) and transparency of the process (process pillar).
Architecture All the Way Down primarily focused on the technology pillar. The visual at right (click to enlarge) illustrates this view. Again, we see the huge gap that exists if we fail to emphasize the importance of modularity. The rightmost portion illustrates how modularity fills the gap - architecture all the way down. Of course, other gaps are also created if we ignore any of the other aspects, such as class design. Note that as we move from services to modules to packages and classes, we increase the granularity along the way. Our classes should not be as fine-grained as our modules, nor our modules as fine-grained as our services.
Additionally, each entity solves a different type of problem (or provides a different type of advantage), as illustrated by the bars at the bottom. All are units of composition, but only services and modules are units of deployment. Services are reused through distributed computing, whereas module reuse is constrained by process boundaries. The technology pillar affects the other pillars. An inflexible rigid architecture makes it difficult for people to understand and communicate (social pillar) and inhibits how quickly we’re able to respond to change (process pillar).
The process pillar is one that I’ve not spent much time discussing. Certainly, it’s important though, and includes various aspects like deferring commitment to significant architectural decisions, evolutionary and emergent architecture, and reversibility. The visual at right illustrates the process pillar (click to enlarge). It’s not as descriptive as the other diagrams, I admit. Anyone have something better that illustrates the process pillar?
I did talk a little about these ideas in Agile Architecture, Lean Principles. But certainly more needs to be fleshed out surrounding the process pillar. This tends to be where most spend their time when discussing agile architecture, but the other pillars are certainly important. The process pillar affects the other pillars. A bad process accompanied by bad practices results in an inflexible architecture (technology pillar) that noone is able to understand (social pillar).
For those that follow this blog, but not the APS blog, I’ve just published a synopsis of modularity on the Java platform. With a lot of momentum continuing to build around OSGi, we’re still in a state of limbo surrounding a standard module system on the Java platform. Some might argue that OSGi has already won - it is the defacto standard module system. Possibly.
Oracle has offered us very little direction on their future plans. There was little fanfare surrounding WebLogic dm Server, with only sparse references and no official word from Oracle. I’m hopeful we can expect to hear some news of the direction they intend to take in their upcoming webcast, though I’m not expecting them to make any significant statements at such a fine level of technical detail. Coincidentally, Apple is holding their own event that very same day, and is expected to announce their new tablet. Let your imagination run wild. Those two events should make for a very exciting day.
Regardless, I’ve had quite a few conversations over the past several months surrounding modularity and OSGi. I have my suspicions on Oracle’s intent, though am reticent to share at this time. Read the APS post titled, Java Modularity - Time to Set Sail.
I’ve been thinking a bit more about the list of disruptive technologies on Richard’s list, and then I watched Dan Pink’s TED talk on the surprising science of motivation. There must be some relationship between the list, which is comprised of almost all open source software products, and Dan’s assertion that the 20th century reward system won’t work for the cognitive tasks performed by workers in the 21st century. What is it, though?
We develop open source software to scratch a personal itch, ease the pain in performing a certain type of task, or create a more compelling alternative to a commercial product. While the professional open source model has emerged the last few years as a way for the open source community to create a sustainable business model, none of the open source technologies on the list were initially developed that way. They were developed in response to need. Ant because Java lacked a good build system. Spring because Java EE was cumbersome and bloated. JUnit to help increase quality. In many cases, these tools have grown to become defacto standard technologies widely used by enterprise development teams.
In Dan’s mind, the new incentive program within organizations must revolve around three things - autonomy, opportunity, and purpose. We must be given autonomy, or the empowerment to make our own decisions. We must be given the opportunity to master something that matters to us. And we must be given purpose, which is the desire to do what we do in the service of something larger than ourselves. Dan notes that financial incentive is also important, but is not the decisive factor in what motivates us.
There are a lot of ways to connect the dots between the open source products so prevalent on Richard’s list and Dan’s point about incentives. I’ll allow you the opportunity to connect these dots any way you wish. A few that immediately come to mind for me include the way commercial software is sold, corporate incentive programs, empowering developers, and flaws in corporate culture. But here’s something else to chew on.
In most of the cases, it was the developer who spurred adoption of the most disruptive application development technologies of the last decade. We weren’t motivated to develop or adopt these great products because of financial incentive or a reward system. Sure, in some cases that’s a positive side affect, but it is not the force that motivates. Instead, we were motivated because they make our jobs a little bit easier and our software a little bit better.
Developers are fighting like hell to create better software. While a lot of commercial vendors are selling shelfware (they aren’t selling it to the developer, mind you), developers are driving adoption of the technologies that are making a difference. Developers seek autonomy, opportunity, and purpose. Given corporate culture, it’s not always easy to find. But if the last decade is any indication, we are finding it, developers do have a voice, and that voice is being heard.
Tight coupling between modules is a bad idea, and the worst form of coupling is cyclic dependencies between modules. Fortunately, there are a few techniques we can use to break the cycles. They are Callback, Escalation, and Demotion, and I’m going to walk through some examples that show each of them in action.
Then, once the dependencies are broken, we’ll look at two more techniques that allow us to invert and eliminate the relationship altogether. The code for each of the samples can be found in the edcie project on my Google code repository. Each example includes a build script and a simple test case. To execute them though, you’ll need GraphViz if you want to use JarAnalyzer. To invoke the build scripts without invoking JarAnalyzer, you can simply type:
Keep in mind that each variation of the system has the exact same behavior!
The example we’re going to use to drive the remainder of our discussion is incredibly simple. We’ve got a Customer and a Bill class that we’re going to bundle into two separate modules - cust.jar and bill.jar. There’s also a test case called PaymentTest that serves as a sample client to drive the interactions between the two classes. The test case is bundled into the billtest.jar module. The initial class diagram is seen at right. Note the bi-directional relationship between the two classes.
As we progress, we’ll add more classes and abstractions to the system to help increase the modularity. Additionally, we’re going to use JarAnalyzer to illustrate the relationships between the modules and also help us assess the quality of our design. The module structure is below, as generated by JarAnalyzer. You can see how to use JarAnalyzer in the build by reviewing the build file. Again, our goal is to break the cyclic dependency between cust.jar and bill.jar, and we’re going to look at three different ways to do this before moving on to examine different ways to massage acyclic module relationships.
Initial Module Structure with Cyclic Dependencies
The first technique we’re going to apply is called Escalation. With Escalation, we break the cyclic dependencies by escalating the cause of the dependency to a higher level entity. Before we do that, we need to more fully understand why a cyclic dependency exists in this example. This reason follows:
A Customer has a list of Bill instances. When the pay method on Bill is invoked, the Bill needs to determine if a discount should be applied. The discount is a product of the Customer the bill belongs to, not necessarily the Bill. Therefore, the Bill class calls a method on Customer to determine the appropriate discount amount. Think of it this way…The Customer represents a payee and we negotiate a discount with each payee. The calculation of this discounted amount is encapsulated within the Customer.
To break this dependency, we want to escalate the cause of the dependency up to a higher level class - the CustomerMediator. The mediator now encapsulates calculation of the discount and passes that to the bill class. The best way to see this change is to look at the modified PaymentTest class. Now, I’ve modified the build script and have bundled the mediator into it’s own module, as shown below. If you dig a bit more deeply into the class structure, you’ll wonder why I didn’t just pass the discount amount from Customer into Bill. Don’t worry about that. This example is slightly contrived because escalation isn’t the best way to solve this type of problem. The key takeaway is that we’ve escalated the dependency up to the mediator.jar bundle, breaking the cyclic dependency.
Escalating the Cause of the Cyclic Dependency
A slightly better way to solve this particular type of cyclic dependency (where we have a true composite relationship between Customer and Bill) is to use demotion. With demotion, we push the cause of the dependency to a lower level module. Exactly the opposite of escalation. We do this by introducing a DiscountCalculator class that will be passed into the Bill class. Our modified PaymentTest class will create the calculator and pass it in for us. The Customer class will serve as the factory for the DiscountCalculator, since it’s the Customer that knows the discount that must be applied. The new class structure can be seen at right.
Now we’ll modify our build script to create an additional calc.jar bundle which will contain the DiscountCalculator class. Our resulting module structure is shown below.
Demoting the Cause of the Cyclic Dependency
Already you can see how this is a more natural solution than escalation for this particular type of cyclic dependency problem. What’s the key difference you might ask? With escalation, notice how I would be able to deploy the cust.jar and bill.jar modules independently. While demotion is a more natural solution in this situation, it also means that to deploy bill.jar or cust.jar, I must also deploy calc.jar. The right solution is always going to be contextual and the ideal solution is likely to shift throughout the development lifecycle.
Using a Callback is similar to the Observer pattern. With this approach, we’ll refactor our DiscountCalculator class to an interface, and then modify the Customer class to implement this interface. This new class structure can be seen at right.
As it happens in this specific situation, using a Callback represents a combination of demotion and our initial solution. We’ll go back to passing the Customer into the Bill, but will pass it in as a DiscountCalculator type. Whereas in the Demotion example we bundled the DiscountCalculator in a separate module, we’ll now just include it in our bill.jar module. Note that putting the DiscountCalculator in the cust.jar module would introduce the cyclic dependency we’re trying to get rid of. The new module structure, which resembles our original version minus the cyclic dependency, is shown below.
Using a Callback to Eliminate the Cyclic Dependency
Now we’re going to play around a bit with the module relationships. While Callback seems like the most logical solution, what if we wanted to use the cust.jar module without the bill.jar module? Callback, as it’s implemented, doesn’t allow us to do this. But with a bit of trickery, I can actually invert the relationship between the cust.jar and bill.jar modules.
I start by refactoring the Bill class to an interface. Then, to avoid split packages (where classes in the same package are bundled into separate modules), I move the Bill class into the same package as the Customer class. The new class diagram is shown at right, and the inverted module structure is shown below.
Inverted Module Structure
Inverting the relationships allows us to deploy the cust.jar module independent of the bill.jar module. Again, it’s all about need. But I’d like to explore another option based on another important need - the ability to test modules independently. Before inverting the relationships, I am able to test the bill.jar module independently. After inverting the relationships, I can test the cust.jar module independently. But what if I want to test (or deploy) both modules independently? To do this, I need to completely eliminate the relationship altogether.
As it turns out, because I’ve got a pretty flexible class structure after I inverted the relationships (lot’s of abstract coupling), I can do this by simply bundling the two interfaces - Bill and DiscountCalculator - into a separate module. No other coding changes required. I start by moving them to a new package called base. Then, I modify my build script to bundle these two interfaces into a separate base.jar module, and we have successfully eliminated the relationship between the bill.jar and cust.jar modules, as shown below.
Eliminating Relationships Between Modules
We’ve come a long ways from our original version of the system. Two modules with a cyclic relationship to a module structure where the original modules don’t have any relationship to each other. This means these modules can be tested and deployed independently. If you follow this blog, you know I’ve written plenty about the tradeoffs between flexibility and complexity, use and reuse, and many other architectural and design challenges. I hope this little exercise has helped drive some of those concepts home.
As a final note, to explore some of these design decisions a bit more deeply, and to examine the tradeoffs a bit more objectively, I encourage you to run the builds for each of the projects and examine the dependencies.html file in the stats directory. To do this, you’ll need to make sure JarAnalyzer is executed, which requires GraphViz. As you’ll see when comparing the initial version to the final version, we made considerable progress in improving the quality of the design.
2010 is going to be a big year for modularity on the Java platform. So based on recent industry trends, and as momentum continues to build around modularity on the Java platform:
I hereby officially declare 2010 as the year of modularity on the Java platform.
So that’s a relatively silly statement, and I really don’t have any authority to make such sweeping declarations. But the statement is backed by some pretty serious trends. Here are a few nuggets to chew on that has led me to believe modularity is going to be bigger in 2010 than ever before!
Between 2007 and 2009, we saw amazing momentum build around OSGi. GlassFish, Jetty, Spring DM, SpringSource dm Server, Paremus Service Fabric, Nimble, PAX, Sigil, Aries, WebSphere, JBoss, Maven, and of course Eclipse are just a few products that leverage OSGi. The list goes on. The number of 3rd party frameworks that have been “osgi-ified” has also grown substantially. And whether you love it or hate it, let’s not forget about Jigsaw and JSR-294.
But there has been a nagging problem…an elephant in the room you might say…that’s hindered widespread adoption. Without gluing together a platform, assembling it yourself, or adopting an entirely new platform, aside from the vendors who were already exposing the virtues of OSGi in their products, the majority of enterprise developers were unable to leverage any of it to build modular enterprise applications! But in 2010, that’s all going to change.
Modularity is coming to the enterprise and you’re going to be able to use it to build modular applications. IBM has already made this announcement. I suspect others will follow suit. For those using SpringSource dm Server or Infiniflow, you’re probably already doing it. With the major platform vendors showing their support for a technology that’s already embedded in products such as dm Server and Infiniflow, I suspect these platforms will see an increase in market penetration through 2010 and beyond. And I suspect you can expect some big announcements surrounding these products soon. Finally, let’s not forget, if it can avoid another delay, JDK 7 (with support for modularity) will be released in September.
Whereas prior to 2010, enterprise developers could only dream about building modular applications (or work very hard to do so), modularity is going to take center stage. And since modularity lies at the heart of the platform, it doesn’t matter what language you use. Neil showed us how to do it with Scala way back in 2007. And Groovy does OSGi, as does Clojure. Yep, so does Scala. So no matter which language you use, you’ll be able to take advantage of modularity on the Java platform.
In 2010 modularity on the Java platform is going to be a game changer…a disruptor…that reaches from the developer to the data center. As 2010 progresses, more and more developers will be touched by modularity. It’s not something we should ignore. Really, it’s not something we can ignore anymore.