My Stuff

2010 Conferences

OSGi DevCon @ JAX London

February 23 - Keynote titled OSGi in the Enterprise: Agility, Modularity, and Architecture’s Paradox

EclipseCon

March 22 - 25 - Tutorial on Modular Architecture

Über Conf

June 14 - 17 - Sessions titled Turtles and Architecture and Patterns of Modular Architecture

Catalyst

July 26 - 30 - Two sessions on rich mobile applications and one on agile development. Half day tutorial on software process improvement.

Tweets @ Twitter

re: #apple event "We sold more iPads than any PC manufacturer sold of their entire PC line." 2012-09-12

re: #Apple Event ""Our notebooks now rank #1 in the US in Market share in the last three months." 2012-09-12

Right on. I just won a Best Buy drawing worth $1000. Either that or I won a shiny new virus by clicking the link. Hmm...what to do. 2012-08-29

The #osgi alliance response (http://t.co/KrN8XNWg) to the @mreinhold #jigsaw announcement (http://t.co/9YvcDdqC). 2012-08-29

Good Q&A with @mreinhold on project #jigsaw. http://t.co/9YvcDdqC. Modularity will change the way we design and run apps! 2012-08-28

LinkedIn Profile

The opinions expressed on this site are my own, and not necessarily those of my employer.

OSGi Perspectives

Filed Under Architecture & Design, Java, OSGi | 2 Comments 

It seems my recent post on OSGi has ruffled a few feathers. I’ve also received a few personal e-mails suggesting that I have missed the mark because OSGi is a significant component of the next generation application platform. Major vendors such as IBM are building OSGi into their products, and other leading edge vendors leverage OSGi as an enabling technology upon which their technical architecture is built.

My intent wasn’t to indict OSGi, so I was pretty surprised with the defensive responses I received…it’s a great technology and the benefits of modularity are real. As I’ve mentioned, and as many already know, I’m an ardent supporter of OSGi and modularity.

Based on the type of the feedback I’ve gotten on my previous post though, I’ve come to the realization that various constituencies have different perspectives that is contributing to the perception of OSGi adoption. Let me explain.

Differing Perspectives

In my post, The Two Faces of Modularity & OSGi, I talk about the runtime model and the development model. There are significant advantages to each. But interestingly, it’s possible for one to succeed while the other never sees widespread adoption. That is, OSGi might lie at the core of the next generation platform, but it doesn’t necessarily imply that enterprise developers will leverage it to build applications with a modular architecture.

Today, most vendors emphasize the runtime model, touting the cost savings, improved responsiveness, and flexibility of a dynamic platform. So I can understand why a lot of folks have reached out to explain to me the error of my ways. From their perspective, OSGi is flourishing.

An excellent example is the Eclipse platform, which released Eclipse 3.0 in June of 2004. However, I’d wager that only a fraction of Java developers using the Eclipse IDE know that OSGi is the major enabling technology upon which Eclipse is built. Instead, they use Eclipse, install plug-ins, but know very little about the underpinnings of the platform.

From a development perspective I’m incredibly interested in the benefits of modular architecture, and I want tools that reduce the accidental complexity of the development model. However, these tools don’t exist today, and developing enterprise server-side Java software applications leveraging OSGi is painful. So while vendors are leveraging OSGi for various platform benefits, as long as they encourage the traditional deployment model and the dearth of tools persist, enterprise developers will not use OSGi to build applications with a modular architecture. We don’t have it today, and we won’t see it in the future until this changes.

Parting Thoughts

I do believe that OSGI has the potential to be a major component of the next generation application platform. Though there are no guarantees, and I still have concerns. Let’s take SpringSource as an example. After seeing almost zero adoption of dm Server, they donated it to the Eclipse Foundation. And the recent VMForce announcement makes nary a mention of OSGi. Instead, tc Server provides the execution environment. So while some might argue that the runtime platform benefits of OSGi are real, there is a far more significant trend called the cloud that provides many of the same advantages. And the cloud certainly doesn’t need OSGi. Though OSGi could thrive in the cloud.

I’m also not convinced that the benefits of modularity will be realized by enterprise development teams. Will enterprise developers leverage OSGi and modularity as a core component of next generation architecture? I hope so, but I don’t believe it’s the slam dunk that many believe it is. Modularity is important. The benefits are real. It helps teams overcome a serious challenge when developing large enterprise software systems. But we need more developer advocacy surrounding OSGi and the benefits of modular architecture. Software is a fashion industry, and we must attach modularity to something trendy and more aggressively advocate adoption.

It is what it is. And unfortunately, that’s just the way it is.

The Exciting Enterprise

Filed Under Agile, Development | 1 Comment 

Developing enterprise software is hard work. Bureaucracy. Large teams. Legacy systems. The list of challenges is immense. But enterprise development can also offer tremendous opportunity and reward. It can even be fun! I hope that at some point, you have the opportunity to experience this feeling.

The Exciting Enterprise

I worked here once. In the exciting enterprise. Working here was different. It was actually exciting. What made it different? Well, it was the process. Exciting process? What’s that? It’s the process that makes software development fun again, productive, meaningful, and satisfying.

So what kind of process is this, you ask? Iterative? Agile? RUP? XP? Scrum? Kanban? Hell no! We never talked process lingo. We didn’t care how agile people thought we were. We didn’t want to get caught up in the bureaucracy and political mayhem surrounding software process improvement. There was work to be done. That’s what we talked about. And then we did what we needed to do to get it done.

Not Always Easy

Working in the exciting enterprise wasn’t always easy though. It required stamina, determination, and discipline. There were lots of people who didn’t appreciate how we went about our business. We didn’t submit weekly status reports. We didn’t have the Gantt chart. Actually, we didn’t have a lot of things. But we had what we needed. To us, it all seemed like common sense. Of course, we had a very experienced team, and we knew what worked and what didn’t.

Plan driven? Predictive? Estimates? Models? Oh yeah…we had it all. We had a 18 month project road map. It was quite a plan. Took a small group of us about 1.5 days to develop. All on a single spreadsheet. Showed all the major systems that we’d retire right along with the new functionality that was going to come online. Probably could have gotten another three months of project time if we wanted to do more planning. But we had code to write.

Oh That Build

Code? But what about the requirements? Yep. Gathered them as we went. And when developers had questions, they asked them. Right to the customers face too. Blasphemy! The business analyst working with the developers. Those BAs were awesome too. They worked real hard to clear any confusion. Made sure developers always had what they needed. And we got ‘em as quickly as we could handle them. A steady stream of requirements flowing in and right back out as an executable piece of software.

Oh sure, it wasn’t always easy. We had lots of important checkpoints along the way to make sure we were on the right track. Weekly checkpoints. Daily checkpoints. Hourly checkpoints. Developers and customers sitting in a room together to see what we’d gotten done the past week. Continuous deployment to an environment where folks could actually use the software. Hourly builds that made sure we never strayed too far from a working system.

I still think about that build. Actually, it was more than a build. It was a piece of frickin’ art. That build was the coolest piece of process I’ve ever seen. It was staged. It was fast. It did pretty much anything we asked it to do. It never got in our way. It just went on it’s merry way, building our system. Hourly. Every hour. Automatically. It was the glue that held the team together as we grew in size from a fledgling crew of six developers to upwards of 100 at times.

We protected that build. When somebody cause it to fail, they’d feel the wrath. Eventually, they all grew to love what it could do. Honestly, what choice did we have if we wanted to ensure we could have weekly checkpoints throughout the process? The only way to pull it off was to emphasize software that works. It always had to work. ALWAYS! Guard the source. The prime directive - software that works!

The Source

Now don’t get me wrong. We had documentation. We had lots of documentation. Some of it was pretty nice looking stuff too. But we weren’t afraid to let it go. Let it serve it’s purpose and then move onto what was real important…the source code. What else really matters, anyway? All the pretty documents, models and plans don’t amount to a hill of beans if the source code isn’t provably correct. That was our focus. It was the teams focus. And we guarded that source code very closely.

Improving the Right Process

I look back now. An 18 month project. A team that clicked. Not just a development team. A team. Including the customers. Honestly, we didn’t just build a piece of software, though. We improved the process. We automated it. We optimized it. No, I’m not talking about the software process. I mean the business process! That’s what made it so cool…

Granularity: Architecture’s Nemesis

Filed Under Architecture & Design, Development, OSGi | 10 Comments 

Granularity is the extent to which a system is broken down into it’s behavioral entities. Coarse-grained entities tend to be richer in behavior than fine-grained entities. Because coarse-grained entities do more, they tend to be larger than fine-grained entities, and are easier to use. In general, a fine-grained entity is more reusable than a coarse-grained entity because it does a little bit less. This is pretty natural. If it does less, then it’s more likely to apply across a wider variety of usage scenarios. I spent some time discussing this tension in Reuse: Is the Dream Dead?

Unfortunately, while there are a lot of patterns, principles, and idioms that offer really good guidance surrounding many aspects of architecture and design, there isn’t a lot of guidance surrounding granularity. Let’s start with a really simple example that illustrates the challenge.

A Really Simple Example

Consider the following save method:

public class Person {
   private String firstName;
   private String lastName;
   private String ssNumber;
   public void save() {
      ValidationErrors errors = this.validate();
      if (errors != null) {
         //handle the errors somehow
      } else {
         //save to the database.
      }
   }
   private ValidationErrors validate() {
      //perform validation
   }
}

As can be seen, the save method invokes the validate method before saving the information to the database. Generally speaking, this makes saving the information easier (and possibly safer) since whatever invokes save doesn’t have to worry about validating. And this seems to be a good thing.

Yet, what happens when a new consumer of this method wants to save the information but apply a slightly different set of rules than the current validation method provides? Here’s where the existing save method is simply too coarse-grained. It does just a little bit too much. We’ve made it easier to use, but also less reusable.

There are quite a few variations I could go with at this point to improve the design. One would be to make the validate method public, and have clients invoke the validate method before invoking save. Then, if validate didn’t do what it was supposed to, clients could always opt out of invoking the validate method and apply their own validation before invoking save. Certainly less safe though because validation is optional.

Yet another alternative might be the following:

public class Person {
   private String firstName;
   private String lastName;
   private String ssNumber;
   public void save(Validator validator) {
      ValidationErrors errors = validator.validate();
      if (errors != null) {
         //handle the errors somehow
      } else {
         //save to the database.
      }
   }
}

This seems to offer the best of both worlds. I now require a Validator be passed in before I can save, and if the Validator is an interface, I can certainly allow clients to pass in the Validator they need. Of course, I could easily pass in a NOP validator.

Whatever…there are lots of different options, and each has their own set of tradeoffs. I’d like to avoid extensive debate surrounding which approach to save and validation is best and instead focus on the main point here, which is that achieving the right level of granularity for an entity can be very challenging. It requires more than just looking at the problem from a code level viewpoint and demands that we possess contextual information that will help us answer the question, “What’s the right level of granularity?”

Bring It Up a Level

The code above was a pretty simple example that illustrates the challenges of granularity at the method level. But the same challenges exist when developing classes, packages, modules and services. I discussed another, much higher level example, in the post on architecture all the way down. So what I really want to know is, “How do I determine the appropriate level of granularity?

This is the million dollar question. Because granularity is a significant inhibitor to creating reusable software that’s easy to use. (Oh, and managing dependencies too.) If something does too much, it’s less reusable. If something does too little, it’s more difficult to reuse.

In the past, I’ve used the diagram at right (click to enlarge) to help illustrate one view of granularity. As can be seen, services are more coarse-grained than modules which are more coarse-grained than packages which in turn are slightly more coarse-grained than classes. This begins to help answer the question, “What’s the right level of granularity?”

If I’m concerned that a service I create is simply too coarse-grained and fails to maximize its reuse potential, I can break the behaviors of the service out into modules that are a bit finer grained and more reusable (of course, one might consider doing this in general). Then I can compose the services from the modules and reuse the modules across services. The result is different entities at different levels of granularity that lends tremendous flexibility to how I compose, use, and reuse software entities.

This provides some guidance on the level of granularity at which different types of software entities should be defined. However, it still doesn’t offer enough guidance to determine the right level of granularity for the save method in our example above.

Another Dimension

There’s another dimension that is relevant, and that ties in nicely (though in a rather subtle way) to the initial coding example. It has to do with the way we traditionally layer our software. The diagram at right illustrates this (click to enlarge). As we can see, as we move down the layered hierarchy, entities become finer grained.

While this isn’t absolute, and depends on your architectural principles and style, entities in higher level layers tend to be more coarse-grained than entities in lower level layers because they are an amalgamation of their own behavior and the behavior of entities in the lower level layers.

Armed with this information, we can determine the right level of granularity for the save method as long as we understand in which layer the code containing the save method lives. If it’s in the data access layer, and we place architectural constraints on business rules living in the data access layer, then we shouldn’t have any validation code that lives in the data access layer. If the save method lives in a class in the domain layer, however, it may be suitable for the save method to contain validation code in the domain layer and invoke another method in the data access layer that actually performs the save operation.

The Complete Picture

At this point we can make the following general statements:

  • Entities in higher level layers are more coarse-grained than entities in lower level layers.
  • Services are coarser than modules which are coarser than packages which are coarser than classes.

And if we combine these two ideas into a single thought, and overlay the two diagrams, we can begin to visualize the level of granularity for different types of software entities.

Certainly this doesn’t offer hard guidance. Realistically, there are very few architecture and design principles and patterns that are universally applicable. In fact, I’m not certain you’d ever create a “presentation service.” But certainly you might create a data service that is composed of multiple data access modules. And you might have separate data services that reuse a data access module. And you might have a business service that uses a data service and is also composed of multiple domain modules. But you definitely do not want a data access module that invokes a business service, nor a business service that references a presentation module.

So this general guidance can serve as useful information to help answer our question, “How do I determine the appropriate level of granularity for a software entity?” And it can also serve as guidance when establishing architectural principles and constraints that help determine where software entities with specific behaviors should reside. Because without some scheme that will help determine the appropriate level of granularity, it’s hopeless to imagine that we’ll be able to design reusable software entities that are also easy to use.

Complexity: Essence and Accidents

Filed Under Architecture & Design, Development, Java, OSGi | 2 Comments 

I’d like to take a short moment to offer an additional perspective to my discussion on OSGi: Complexity is NOT the Problem. I believe this perspective adds clarity to that previous discussion, as well. All initiated thanks to a tweet, which summed up the situation in much less than 140 characters.

So over the weekend, I turned to the essay that said tweet refers, and reviewed the essence and the accidents. From the article, I quote Mr. Brooks:

“…to see what rate of progress we can expect in software technology, let us examine its difficulties. Following Aristotle, I divide them into essence - the difficulties inherent in the nature of the software - and accidents - those difficulties that today attend its production but that are not inherent.”

This leads me, pretty clearly, to the following simple conclusion.

Today, development teams leveraging OSGi to build server-side applications are burdened by accidental complexity. As platforms and tools mature, the accidental complexity of OSGi will be reduced to near zero. Modularity though, enabled and enforced by OSGi, attacks the essence.

If the difference between essential and accidental complexity isn’t quite clear, I highly recommend you take a few moments and read Mr. Brook’s essay (linked above). And then, if you can find a copy of the book’s 2nd edition, take a look at Chapter 17, “No Silver Bullet” Refired (Note: If anyone can find the Refired essay online, please post the link in comments).

Throughout these two discussions, you’ll find subtle hints extolling the virtues of modularity. But only do this if you’re willing to exercise your brain with thought, because the connection you’ll discover might just be transformational!

OSGi: Complexity is NOT the Problem

Filed Under Architecture & Design, Development, OSGi | 3 Comments 

The editors at SD Times have proclaimed that OSGi is too complex. Unfortunately, they miss the mark, and the following statement taken from the article is misleading.

We believe that OSGi, rather than simplifying server application development, has actually made it more complex, and there aren’t sufficient benefits to justify the added complexity.

This statement should have been tempered with context. It is not universally applicable, though is made to sound as if it were. OSGi may face challenges, but complexity of the OSGi framework is not one of the long-term inhibitors to adoption.

Technology Adoption Lifecycle

All technology goes through various phases of adoption. A lack of platform support and tooling may be inhibiting adoption today, but that is today’s problem, and one that is being dealt with. And there is a stark difference between complexity brought on by lack of tooling and platform support versus complexity associated with an ill-conceived idea. OSGi suffers today from lack of enterprise-class tooling and platform support. OSGi is not an ill-conceived idea.

Whenever an organization adopts a new technology, they’ll undoubtedly face some degree of initial complexity. New technology takes time to learn. As the innovators and early adopters continue to use OSGi, tooling is going to get better and developing modular applications using OSGi is going to get easier.

The question you must answer is whether the pain of adoption today is worth the gain of a modular architecture. There are many factors to consider when answering this question, but as the technology matures, the question will be easier to answer. As I’ve alluded to, other factors that have little to do with OSGi’s technical benefits will likely determine if it’s is able to cross the chasm.

Long Term Value

But the article’s biggest flaw is in stating:

And what’s the benefit again? Enterprise developers have written many, many server-side Java applications without using OSGi.

It’s important to understand its value over time. Let me pose some simple questions.

  • Is Spring too complex? It is far easier to create an application with concrete class references than it is to use dependency injection and abstractions.
  • Is Hibernate too complex? It is far easier to use JDBC than it is an ORM framework.
  • Is unit testing too costly? It is far easier to forego unit testing than it is to create a robust suite of tests.

Yet each contribute to more flexible software systems, and in that, there is significant benefit. So it is with OSGi and modularity, as well.

Leveraging OSGi to design modular applications will require us to learn new concepts, tools, and design paradigms, but there is no doubt that modularity has a tremendous upside. This makes the SD Times article paradoxical. OSGi does not increase complexity; the very benefit of modularity is that it helps us manage complexity.

More on Benefits

If you’re interested in exploring some of the benefits of modularity, you can start with the following:

OSGi - Feast or Famine?

Filed Under Architecture & Design, Development, Industry, OSGi | 11 Comments 

I want to be careful here, but also very clear - I advocate OSGi and modularity, though am concerned about its future. In some circles, OSGi is hailed as a disruptive technology; in others, it lives in relative obscurity. There are some indications that adoption is growing, and possibly at some point, it’ll catch like wildfire. But right now, OSGi is the most widely acclaimed technology that nobody is using.

Maybe I’m too harsh. Or too impatient. Maybe I need to give it more time. I know some will argue that people are using OSGi, and they’re right. Some are. But of the more than 6 million Java developers worldwide, those using OSGi and designing modular applications is only a very small fraction.

So what’s the problem? What’s needed to drive adoption? How will OSGi and modularity succeed?

What’s In a Name?

One of the greatest inhibitors to adoption has been platform support. Until recently, few platforms exposed the virtues of OSGi to the enterprise developer. That’s beginning to change, and should help drive adoption. Another obstacle is tooling, which I discussed a while back when lamenting that one of the biggest barriers to adoption is no migration path. With better tools, we’re beginning to overcome that challenge.

But will it be enough? Will platform support and great tools help drive adoption? Sadly, possibly not. And part of the problem, I believe, has very little to do with the technology, and much more to do with its image.

So, what’s in a name?

OSGi is the dynamic module system for Java!

Modularity - Old and Stodgy

Recently, I commented on a recent conversation I had with a conference organizer and his reluctance to accept my speaking proposal on modularity. The general sentiment was that it’s an old, boring concept that can be traced back 40 years! He was interested in my talk on agility, but I explained that given the choice, I’d prefer to speak on modularity.

Currently, I’m participating in a similar conversation with the organizers of the Agile 2010 conference on a few sessions I’ve proposed on modularity. And now, Peter writes about a similar experience. This sends an important message to the OSGi community, and highlights an important challenge surrounding modularity. The message: Modularity is boring! The problem: The importance of modularity, and its support as a first class citizen on the Java platform, is not well communicated nor understood.

Not a Technology Problem

This is a serious problem that requires attention. In his post, Peter suggested that micro-services may be a more apt term for OSGi services, so as not to confuse the term with web services. He even hints at proposing a new JSR that would introduce the idea of micro-services as a new primitive on the Java platform. This is a good start, and micro-services isn’t a bad idea, but more needs to be done. The problem is not that OSGi isn’t a great technology with real benefits, the problem is that nobody cares about modularity.

Possibly it isn’t that they don’t care, but instead that modularity won’t sell. Nobody wants to buy a 40 year old technology, they want to buy the next great thing. And without question, developing modular software is not a new idea. But in general, the ideas surrounding SOA were not new either. Yet, without question, SOA has garnered significantly more attention than OSGi over the past decade. SOA has had its day in the sun; OSGi and modularity have not.

This is not a technology problem. The real problem is that OSGi doesn’t have a clearly articulated identity that explains why an organization must be using it, and organizations will adopt OSGi only if its adoption can be tied to real business benefits. Just like object-orientation in the 1990’s and SOA in the 2000’s, each were sold with a wonderful accompaniment of benefits that would reduce cost, speed delivery, model the real world, blah, blah, and blah. Whether they did or did not help organizations realize these benefits is arguable. Whether each were widely adopted is not!

Crossing the Chasm

OSGi has the potential to be this decade’s most disruptive technology. Unfortunately, the virtues of OSGi run the very real risk of never coming to fruition if it’s unable to cross the chasm. And sadly, it can never cross the chasm unless we find an effective way to sell, or reinvent, modularity.

Foremost, OSGi, modularity, and micro-services do not represent the paradigm shift that’s necessary. Too often, we talk about OSGi in terms of its technical merits alone. Unfortunately, that’s not enough. If OSGi is to cross the chasm, it is going to require something different. Something that people can really latch onto. Something that gets them excited. Modularity isn’t enough. Nor is plug-in architecture. Nor any other technically superior thing that OSGi is.

Naturally, this is only my perspective. My opinion. But OSGi has been a widely acclaimed technology for several years, and it has achieved very little enterprise adoption. While the move by SpringSource to donate dm Server to the Eclipse Foundation was hailed as a great contribution to open source, it’s also a clear indication that adoption of dm Server was gaining little traction.

Now, it is possible that an 800 pound gorilla could throw their weight behind OSGi and modularity that will help drive enterprise adoption. Maybe IBM’s move to expose the virtues of OSGi to the enterprise will be enough. With other vendors following suit, and up and coming vendors such as Paremus on the rise, that just might be enough. But if it is enough, I still believe the enterprise isn’t going to buy OSGi and modularity for the reasons we believe they should, they’ll buy something else very trendy in which OSGi and modularity plays a central role. Something new. Something fashionable. Because as I recall Ivar Jacobson alluding to at a keynote a couple of years ago,

The software industry is more driven by fashion than the fashion industry.

Sad. But true! And hopefully when it happens it won’t dilute the value of OSGi and modularity.

Ignore my Rant if You Want

OSGi is a great technology solution. But we also recognize that technologically superior solutions often fail to win. One of the most popular stories is of the video tape format wars. So my little pontification here should in no way be interpreted as questioning whether OSGi is capable, but instead whether OSGi will.

You can choose to ignore me if you’d like, scoff at me, or call this a rant. Maybe I’m just impatient. Either way, I’ll still continue to proclaim the virtues of OSGi and modularity. I believe in it…firmly! But the nagging feeling I have surrounding its ability to cement its place in the world of enterprise software development will continue to tug at me. And I wonder…will it ever succeed? And if so, how? And…when?

What do you think?