My Stuff

2010 Conferences

OSGi DevCon @ JAX London

February 23 - Keynote titled OSGi in the Enterprise: Agility, Modularity, and Architecture’s Paradox

EclipseCon

March 22 - 25 - Tutorial on Modular Architecture

Über Conf

June 14 - 17 - Sessions titled Turtles and Architecture and Patterns of Modular Architecture

Catalyst

July 26 - 30 - Two sessions on rich mobile applications and one on agile development. Half day tutorial on software process improvement.

Tweets @ Twitter

re: #apple event "We sold more iPads than any PC manufacturer sold of their entire PC line." 2012-09-12

re: #Apple Event ""Our notebooks now rank #1 in the US in Market share in the last three months." 2012-09-12

Right on. I just won a Best Buy drawing worth $1000. Either that or I won a shiny new virus by clicking the link. Hmm...what to do. 2012-08-29

The #osgi alliance response (http://t.co/KrN8XNWg) to the @mreinhold #jigsaw announcement (http://t.co/9YvcDdqC). 2012-08-29

Good Q&A with @mreinhold on project #jigsaw. http://t.co/9YvcDdqC. Modularity will change the way we design and run apps! 2012-08-28

LinkedIn Profile

The opinions expressed on this site are my own, and not necessarily those of my employer.

Java With a Bit of OSGi - The Book

Filed Under Agile, Architecture & Design, Development, Java, OSGi | 2 Comments 

I’m dancing. By god I’m dancing on the walls. I’m dancing on the ceiling. I’m ecstatic. I’m overjoyed. I’m really, really pleased.
- An excerpt from the Foreword by Uncle Bob (aka. Robert C. Martin)


My book, Java Application Architecture: Modularity Patterns With Examples Using OSGi is now available. Uncle Bob and Peter Kriens each contributed Forewords to the book. The book itself is part of the Robert C. Martin series. The book is intended for all software developers interested in designing better software using modularity. Though the examples use Java, the techniques can be applied to other languages and platforms, such as .NET, with relative ease.

Even if you’re not using OSGi (or perhaps not even familiar with OSGi), I’m confident you’ll find the book valuable. The book (and patterns) has been designed to allow you to realize the benefits of modularity whether you’re using a module framework, or not. As Uncle Bob says in the Foreword, “This is how you build a Java application, people.” Peter sums it up nicely too, in saying “This book…will give you a view into the magic of modularity.

You can order it online at Amazon (print edition & Kindle edition), iBooks, InformIT, or a number of other publishers. For more details on the book, please see the book’s website. Over the next couple of weeks, I plan to post a sample chapter or two that will give you a feel for the book’s contents.

Here’s what a few people have to say:

  • “The fundamentals never go out of style, and in this book Kirk returns us to the fundamentals of architecting economically interesting software-intensive systems of quality. You’ll find this work to be well-written, timely, and full of pragmatic ideas.” Grady Booch, IBM Fellow
  • “Along with GOF ‘Design Patterns’ - ’Java Application Architecture’ is a must own for every enterprise developer and architect, and on the required reading list for all Paremus engineers.” – Richard Nicholson, Paremus CEO & President of the OSGi Alliance
  • “In writing this book, Kirk has done the software community a great service: he’s captured much of the received wisdom about modularity in a form which can be understood by newcomers, taught in Computer Science courses, and referred to by experienced programmers. I hope this book finds the wide audience it deserves.” Glyn Normington, Eclipse Virgo Project Lead
  • “Our industry needs to start thinking in terms of modules – it needs this book!” Chris Chedgey, Founder and CEO of Structure 101
  • “In this book Kirk Knoernschild provides us with the design patterns we need to make modular software development work in the real world. While it’s true that modularity can help us manage complexity and create more maintainable software, there’s no free lunch. If you want to achieve the benefits modularity has to offer, buy this book.” Patrick Paulin, Consultant and Trainer at Modular Mind
  • “Kirk has expertly documented the best practices for using OSGi and Eclipse runtime technology.  A book any senior Java developer needs to read to better understand how to create great software.’ Mike Milinkovich, Executive Director Eclipse Foundation
I’d like to thank all of you who helped me along this journey. I hope you enjoy the book.

Farewell Blog, Hello POMA

Filed Under Development, General, OSGi | 6 Comments 

As the saying goes…all good things must end. As of today, my blog is shutting down!

I do intend to leave all content on-line so you’ll always be able to see a list of all 134 posts, but there won’t be any new posts for the foreseeable future. I’d like to thank everyone that has taken the time to read my often times long-winded posts. I hope you’ve found them useful.

For the past few years, I’ve written almost exclusively about modularity and OSGi (with a few other topics sprinkled in on occasion). To that end, I’ve decided to pursue a long overdue book project, tentatively titled Patterns of Modular Architecture (POMA). I hope you’ll decide to join me in my journey. Also, be sure to follow the book updates on Twitter.

So as to not disappoint those who have come to expect something a bit more verbose, I have managed to cobble together a few words explaining my decision in more detail. So if you want to know more, read on!

Story Behind the Story

Over the past few years of blogging, I’ve focused considerable energy on espousing the virtues of modularity and OSGi. From conceptual posts that discuss the important architectural benefits of modularity to simple examples that illustrate the benefit. From the beginning to the end, I’ve written about the benefits of modular architecture at a conceptual level, practical level, and have even provided some concrete examples. To the chagrin of some, I’ve even discussed the challenges that lie ahead. Of course, modularity and OSGi aren’t the only topics I write about. I’ve also written about agility, IT labor, metrics, and more. At some point though, the topics always came back to modularity and OSGi.

In the time that I’ve been focused on modularity and OSGi, the number of folks that access the content on this site have gone from less than 1000 visitors per month to more than 10,000, culminating in more than 100,000 pages served up and over a quarter million hits to the site each month. The blog was listed as one of the Top 200 Blogs for Developers, a Top Analyst Blog, and many of my posts are syndicated on JavaLobby. While still small by many standards, I recognized progress. And in general, the feedback I’ve received from the community is positive, though there is no doubt that at times, I’ve struck some nerves.

For those that read my posts, you already know that most tend to span many paragraphs. They require extensive writing time, considerable editing time, and careful review. It’s taken me hours to author many of them. And all of the content on this site has always been licensed under a Creative Commons Attribution-Noncommercial-Share Alike license. I vowed to never include any noise on the site, such as Google AdSense. It’s always been an earnest attempt to help developers improve the systems they create. A labor of love, you might say.

A while back, I introduced the modularity patterns, but didn’t elaborate much on the patterns themselves. Alas, that time has come. I’ve quickly realized that I do not have the bandwidth to continue with my long-winded posts and write a book. I’ve decided to pursue the latter and am hopeful that a book on modular architecture will have a far greater, longer lasting impact that serves as a capable ambassador for modular architecture. So while my blog may be silent, I have every intent to continue my advocacy of modular architecture, albeit in a way that I hope is more impactful.

I’ve decided to adopt an open and transparent approach to writing the book. The book’s website illustrates the book in it’s current form - a rough and incomplete draft to be sure. But I hope you’ll take the time to check it out, and offer any feedback you might have. I’ve created a Reviewers page that provides some guidance on the type of feedback I hope to receive. Also, be sure to follow the updates on Twitter.

By the way, this does not mean that I’ll necessarily cease my blogging activities altogether. In the past, I’ve published some articles to the APS Blog. Soon, the APS blog will also be shut down, and we’ll begin to blog on the Gartner Blog Network (GBN). So be sure to checkout GBN on occasion!

Thank you!

Ecosystems, Modularity, & OSGi

Filed Under Industry, OSGi, Platforms | 4 Comments 

Recently, I questioned whether OSGi and modularity would succeed in penetrating the enterprise. But what I really meant to question is whether OSGi will have the disruptive impact of which it’s capable. I asserted that if OSGi does succeed, it won’t be based on the technical merits of OSGi. It’ll be because of something else. Something trendy. Something fashionable.

Now, I could be wrong. OSGi adoption is certainly increasing, albeit slowly. And as case studies begin to emerge that tout the cost reduction, improved responsiveness, and time-to-market advantages of OSGi, adoption will likely continue to rise.

But adoption is one thing, disruption another, and I still have this nagging sensation that serves as cause for pause. What if something trendier, more fashionable surfaces, and OSGi is pushed into the backwaters? Will it really have the impact it’s capable of? You know…an “iPhone-esque” impact that raises the bar and redefines an industry. Not just evolutionary, but truly disruptive.

OSGi As Enabler

For OSGi to cross the chasm, it must enable something big that a business wants to buy. Maybe cost reduction, improved responsiveness and time-to-market benefits will be enough. But that’s easily perceived by many as a rather evolutionary impact. Not really disruptive.

Quite possibly OSGi will flourish in the data center, as organizations seek more adaptable platforms that lend them these benefits. But this doesn’t necessarily guarantee that development teams will leverage OSGi to build systems with a modular architecture. Because leveraging a platform built atop OSGi is separate from building modular software systems, even though OSGi enables both.

Without something trendy that promises real benefits, it’ll get brushed under the carpet like what has happened to many technologically superior solutions. Perhaps it’s already happening, with all of the hype surrounding the cloud. Or maybe it’ll be the cloud that helps OSGi cross the chasm. OSGi doesn’t obviously enable the cloud, but OSGi can enable the dynamic and adaptive runtime benefits of the cloud.

But again, this doesn’t mean that teams will begin leveraging OSGi to design modular software. It only means that the platform itself is adaptable. Of course, there is benefit in that. But if that’s the route that is taken, teams will still continue to develop monolithic applications that lack architecturally resiliency, and the full benefit of OSGi (and modularity) will not be realized.

The Disruption

OSGi has the potential to have have a much broader impact, affecting everyone from the developer to those working in the data center.  So what might this trend be that will propel OSGi to stardom?

OSGi enables ecosystems!

Now, you’re thinking I’ve gone on the deep end, perhaps? But give me a chance…let me explain.

A Bit of [Recent] Platform History

To start, I want to take a brief walk down memory lane. Not too far back though, but far enough so that we can see how important the ecosystem is in today’s most successful platforms. And these platforms span a range of markets, from mobile to social media. But each are successful in large part due to a thriving ecosystem.

Apple

In 2007, Apple released their first generation iPhone. Without question, the device revolutionized the mobile phone industry. While the device offers a great user experience that has certainly played a role in its surging popularity, people flock to the iPhone today because of the wealth of applications available. Yeah, “there’s an app for that.”

Apple, recognizing the power of this ecosystem, now delivers the iPad. With fewer preinstalled applications than the iPhone, Apple is counting on the ecosystem to drive adoption. The more consumers who flock to the device, the more developers who flock to the platform to deliver applications. As more applications become available, consumers will buy more iPads. The ecosystem fuels itself. Apple has simply provided the environment for the ecosystem to thrive.

Eclipse

In 2003, the Eclipse team was thinking of ways to make Eclipse more dynamic. Their decision to use OSGi to create a rich client platform that supports plug-in architecture was the first step toward the resulting ecosystem we know today. One of the reasons developers use Eclipse is because there are an abundance of plug-ins available that allow them to do their jobs more effectively. Other developers create Eclipse plug-ins because Eclipse is a popular IDE used by many developers. Again, the ecosystem fuels itself. The Eclipse team provided the environment that allows today’s Eclipse ecosystem to thrive.

If you’re interested, you can read more about the history of Eclipse and OSGi.

Hudson

Arguably, Hudson is today’s most popular continuous integration server. But it hasn’t always been. Before Hudson was CruiseControl. And while CruiseControl did help development teams get started on their path toward continuous integration, it was also unwieldy to use in many ways. With Hudson’s plug-in architecture, developers have the ability to extend the tool in ways the original creator couldn’t imagine or couldn’t find the time to do himself. Kohsuke created Hudson and gave the development community a new platform for continuous integration. With its plug-in architecture though, he also provided an environment that allows the Hudson ecosystem to thrive.

Social Media

Facebook. MySpace. Twitter. Each are examples of social media tools with a strong developer community that creates extensions to the platform that users can leverage to enhance the experience. Facebook Developers. MySpace Developer Platform. Twitter API. Each allows the ecosystem to thrive.

And Others

These are just a few examples. It’s easy to find other platforms with similar ecosystems, as well. The ease with which Wordpress themes and plug-ins can be developed and used to enhance a Wordpress website is another example. In fact, many content management systems provide similar capabilities. A large reason why the Firefox web browser has emerged as the preferred web browser is the ease with which add-ons can be installed that extend the capabilities of the browser. The Atlassian Plugin Framework is another example that uses OSGi, and platforms such as Force.com and SharePoint have built (or are trying to build) similar ecosystems.

The Power of Ecosystems

Aside from Eclipse (and Atlassian), none of these other platforms leverage OSGi. Yet each are wildly successful because of two reasons:

  • An environment was created that allowed an ecosystem to form and flourish. This environment includes a platform and a marketplace.
  • A group of customers and developers converged on the marketplace and fueled growth of the platform. The result is a self-sustaining ecosystem.

If you look at many of the more popular platforms that have emerged over the past decade, they tend to possess a similar characteristic - a community of individuals dedicated to providing great solutions leveraging the foundation of the platform. OSGi and modularity enables ecosystems on the Java platform.

Ecosystems and the Two Faces

I’ve already talked about the two faces of OSGi - the runtime model and the development model. I’ve also explained how one could possibly see widespread adoption while the other has little impact. A strong ecosystem surrounding OSGi and modularity must leverage both. Developers would create reusable modules, implying they are designing modular software. For development teams to leverage these modules, they must be using a platform that supports the runtime model.

CBD Has Already Had Its Day, You Say?

Now some of you might be arguing that this sounds a lot like the component based development (CBD) fad of the 1990’s. True…to an extent. Certainly these ideas are not new. But there are also some striking differences between that which OSGi enables and the CBD fad that has come and largely gone, or whose promise was never fully realized.

Foremost, the CBD fad was focused almost exclusively on visual components, such as ActiveX. While some attempted to create components for the Java platform, the movement largely failed to go mainstream. Instead, Java EE grew in popularity and for a number of years, garnered everyone’s attention. Why did this  happen?

IMHO, the answer is fairly simple. Even though numerous marketplaces emerged that allowed the consumer and producer to come together to buy and sell components, there was never a suitable component execution environment. That is, an environment that would support dynamic deployment, support for multiple versions, dependency management, and, in general, complete control over all components currently executing within the environment. ActiveX components did have an execution environment (though did not support each of these capabilities), but Java did not. Today, in OSGi, Java has the requisite execution environment!

The Ecosystem

It’s easy find holes in this idea. To explain why it cannot work. Yet, it’s happening elsewhere, so why not on the server…in the enterprise? Certainly there are a variety of different ways such an ecosystem could manifest itself. Possibly multiple ecosystems emerge like what we see in the mobile market today.

But for a moment, imagine the world where you have the ability to easily assemble a platform from pre-built infrastructure modules that exactly meet the demands of your application. You might purchase these modules, you might choose to use open source modules, or you might build them yourself. For those you don’t build, you obtain from a module marketplace, possibly deploying them to your [cloud] environment.

And when you choose to use a module, it’s dynamically deployed to your environment. The modules it depends upon? You’re given the option to purchase and deploy them, as well. You develop your software modules using the sound principles and patterns of modular design to ensure loose coupling and high cohesion. As you roll out your business solution modules, you simultaneously deploy the additional infrastructure modules that are needed.

In this marketplace, modules are sourced by multiple vendors. Some large. Some small. Neither the stack, nor your applications, are monolithic beasts. Instead, they are a composition of collaborating software modules. Your infrastructure isn’t necessarily tied to a specific vendor solution. The option always exists for organizations to purchase modules from different providers, easily swapping one provider module out with another.

The ecosystem flourishes. Developers flock to sell their latest creation. Organizations seek to add amazing capabilities to their rightsized environment at a fraction of the cost compared to what they are accustomed to today. The business benefits are real. The technical advantages are real. And the resulting ecosystem is sustainable.

A successful ecosystem demands both the runtime model and development model. And today, OSGi is the only standard technology that will allow this type of successful ecosystem to form on the Java platform. But will it happen? We may have a ways to go, but it sure would be cool! And it would be a shame if we lost this opportunity.

Thoughts?

Note: If you’re interested in ecosystems more generally, you might want to see the great TED talk by Dan Barber, “How I fell in love with a fish.” It’s very entertaining, informative, and worth 20 minutes of your time. His discussion on sustainability is truly fascinating!

Image Source: http://en.wikipedia.org/wiki/File:Blue_Linckia_Starfish.JPG

OSGi Perspectives

Filed Under Architecture & Design, Java, OSGi | 2 Comments 

It seems my recent post on OSGi has ruffled a few feathers. I’ve also received a few personal e-mails suggesting that I have missed the mark because OSGi is a significant component of the next generation application platform. Major vendors such as IBM are building OSGi into their products, and other leading edge vendors leverage OSGi as an enabling technology upon which their technical architecture is built.

My intent wasn’t to indict OSGi, so I was pretty surprised with the defensive responses I received…it’s a great technology and the benefits of modularity are real. As I’ve mentioned, and as many already know, I’m an ardent supporter of OSGi and modularity.

Based on the type of the feedback I’ve gotten on my previous post though, I’ve come to the realization that various constituencies have different perspectives that is contributing to the perception of OSGi adoption. Let me explain.

Differing Perspectives

In my post, The Two Faces of Modularity & OSGi, I talk about the runtime model and the development model. There are significant advantages to each. But interestingly, it’s possible for one to succeed while the other never sees widespread adoption. That is, OSGi might lie at the core of the next generation platform, but it doesn’t necessarily imply that enterprise developers will leverage it to build applications with a modular architecture.

Today, most vendors emphasize the runtime model, touting the cost savings, improved responsiveness, and flexibility of a dynamic platform. So I can understand why a lot of folks have reached out to explain to me the error of my ways. From their perspective, OSGi is flourishing.

An excellent example is the Eclipse platform, which released Eclipse 3.0 in June of 2004. However, I’d wager that only a fraction of Java developers using the Eclipse IDE know that OSGi is the major enabling technology upon which Eclipse is built. Instead, they use Eclipse, install plug-ins, but know very little about the underpinnings of the platform.

From a development perspective I’m incredibly interested in the benefits of modular architecture, and I want tools that reduce the accidental complexity of the development model. However, these tools don’t exist today, and developing enterprise server-side Java software applications leveraging OSGi is painful. So while vendors are leveraging OSGi for various platform benefits, as long as they encourage the traditional deployment model and the dearth of tools persist, enterprise developers will not use OSGi to build applications with a modular architecture. We don’t have it today, and we won’t see it in the future until this changes.

Parting Thoughts

I do believe that OSGI has the potential to be a major component of the next generation application platform. Though there are no guarantees, and I still have concerns. Let’s take SpringSource as an example. After seeing almost zero adoption of dm Server, they donated it to the Eclipse Foundation. And the recent VMForce announcement makes nary a mention of OSGi. Instead, tc Server provides the execution environment. So while some might argue that the runtime platform benefits of OSGi are real, there is a far more significant trend called the cloud that provides many of the same advantages. And the cloud certainly doesn’t need OSGi. Though OSGi could thrive in the cloud.

I’m also not convinced that the benefits of modularity will be realized by enterprise development teams. Will enterprise developers leverage OSGi and modularity as a core component of next generation architecture? I hope so, but I don’t believe it’s the slam dunk that many believe it is. Modularity is important. The benefits are real. It helps teams overcome a serious challenge when developing large enterprise software systems. But we need more developer advocacy surrounding OSGi and the benefits of modular architecture. Software is a fashion industry, and we must attach modularity to something trendy and more aggressively advocate adoption.

It is what it is. And unfortunately, that’s just the way it is.

The Exciting Enterprise

Filed Under Agile, Development | 1 Comment 

Developing enterprise software is hard work. Bureaucracy. Large teams. Legacy systems. The list of challenges is immense. But enterprise development can also offer tremendous opportunity and reward. It can even be fun! I hope that at some point, you have the opportunity to experience this feeling.

The Exciting Enterprise

I worked here once. In the exciting enterprise. Working here was different. It was actually exciting. What made it different? Well, it was the process. Exciting process? What’s that? It’s the process that makes software development fun again, productive, meaningful, and satisfying.

So what kind of process is this, you ask? Iterative? Agile? RUP? XP? Scrum? Kanban? Hell no! We never talked process lingo. We didn’t care how agile people thought we were. We didn’t want to get caught up in the bureaucracy and political mayhem surrounding software process improvement. There was work to be done. That’s what we talked about. And then we did what we needed to do to get it done.

Not Always Easy

Working in the exciting enterprise wasn’t always easy though. It required stamina, determination, and discipline. There were lots of people who didn’t appreciate how we went about our business. We didn’t submit weekly status reports. We didn’t have the Gantt chart. Actually, we didn’t have a lot of things. But we had what we needed. To us, it all seemed like common sense. Of course, we had a very experienced team, and we knew what worked and what didn’t.

Plan driven? Predictive? Estimates? Models? Oh yeah…we had it all. We had a 18 month project road map. It was quite a plan. Took a small group of us about 1.5 days to develop. All on a single spreadsheet. Showed all the major systems that we’d retire right along with the new functionality that was going to come online. Probably could have gotten another three months of project time if we wanted to do more planning. But we had code to write.

Oh That Build

Code? But what about the requirements? Yep. Gathered them as we went. And when developers had questions, they asked them. Right to the customers face too. Blasphemy! The business analyst working with the developers. Those BAs were awesome too. They worked real hard to clear any confusion. Made sure developers always had what they needed. And we got ‘em as quickly as we could handle them. A steady stream of requirements flowing in and right back out as an executable piece of software.

Oh sure, it wasn’t always easy. We had lots of important checkpoints along the way to make sure we were on the right track. Weekly checkpoints. Daily checkpoints. Hourly checkpoints. Developers and customers sitting in a room together to see what we’d gotten done the past week. Continuous deployment to an environment where folks could actually use the software. Hourly builds that made sure we never strayed too far from a working system.

I still think about that build. Actually, it was more than a build. It was a piece of frickin’ art. That build was the coolest piece of process I’ve ever seen. It was staged. It was fast. It did pretty much anything we asked it to do. It never got in our way. It just went on it’s merry way, building our system. Hourly. Every hour. Automatically. It was the glue that held the team together as we grew in size from a fledgling crew of six developers to upwards of 100 at times.

We protected that build. When somebody cause it to fail, they’d feel the wrath. Eventually, they all grew to love what it could do. Honestly, what choice did we have if we wanted to ensure we could have weekly checkpoints throughout the process? The only way to pull it off was to emphasize software that works. It always had to work. ALWAYS! Guard the source. The prime directive - software that works!

The Source

Now don’t get me wrong. We had documentation. We had lots of documentation. Some of it was pretty nice looking stuff too. But we weren’t afraid to let it go. Let it serve it’s purpose and then move onto what was real important…the source code. What else really matters, anyway? All the pretty documents, models and plans don’t amount to a hill of beans if the source code isn’t provably correct. That was our focus. It was the teams focus. And we guarded that source code very closely.

Improving the Right Process

I look back now. An 18 month project. A team that clicked. Not just a development team. A team. Including the customers. Honestly, we didn’t just build a piece of software, though. We improved the process. We automated it. We optimized it. No, I’m not talking about the software process. I mean the business process! That’s what made it so cool…

Granularity: Architecture’s Nemesis

Filed Under Architecture & Design, Development, OSGi | 7 Comments 

Granularity is the extent to which a system is broken down into it’s behavioral entities. Coarse-grained entities tend to be richer in behavior than fine-grained entities. Because coarse-grained entities do more, they tend to be larger than fine-grained entities, and are easier to use. In general, a fine-grained entity is more reusable than a coarse-grained entity because it does a little bit less. This is pretty natural. If it does less, then it’s more likely to apply across a wider variety of usage scenarios. I spent some time discussing this tension in Reuse: Is the Dream Dead?

Unfortunately, while there are a lot of patterns, principles, and idioms that offer really good guidance surrounding many aspects of architecture and design, there isn’t a lot of guidance surrounding granularity. Let’s start with a really simple example that illustrates the challenge.

A Really Simple Example

Consider the following save method:

public class Person {
   private String firstName;
   private String lastName;
   private String ssNumber;
   public void save() {
      ValidationErrors errors = this.validate();
      if (errors != null) {
         //handle the errors somehow
      } else {
         //save to the database.
      }
   }
   private ValidationErrors validate() {
      //perform validation
   }
}

As can be seen, the save method invokes the validate method before saving the information to the database. Generally speaking, this makes saving the information easier (and possibly safer) since whatever invokes save doesn’t have to worry about validating. And this seems to be a good thing.

Yet, what happens when a new consumer of this method wants to save the information but apply a slightly different set of rules than the current validation method provides? Here’s where the existing save method is simply too coarse-grained. It does just a little bit too much. We’ve made it easier to use, but also less reusable.

There are quite a few variations I could go with at this point to improve the design. One would be to make the validate method public, and have clients invoke the validate method before invoking save. Then, if validate didn’t do what it was supposed to, clients could always opt out of invoking the validate method and apply their own validation before invoking save. Certainly less safe though because validation is optional.

Yet another alternative might be the following:

public class Person {
   private String firstName;
   private String lastName;
   private String ssNumber;
   public void save(Validator validator) {
      ValidationErrors errors = validator.validate();
      if (errors != null) {
         //handle the errors somehow
      } else {
         //save to the database.
      }
   }
}

This seems to offer the best of both worlds. I now require a Validator be passed in before I can save, and if the Validator is an interface, I can certainly allow clients to pass in the Validator they need. Of course, I could easily pass in a NOP validator.

Whatever…there are lots of different options, and each has their own set of tradeoffs. I’d like to avoid extensive debate surrounding which approach to save and validation is best and instead focus on the main point here, which is that achieving the right level of granularity for an entity can be very challenging. It requires more than just looking at the problem from a code level viewpoint and demands that we possess contextual information that will help us answer the question, “What’s the right level of granularity?”

Bring It Up a Level

The code above was a pretty simple example that illustrates the challenges of granularity at the method level. But the same challenges exist when developing classes, packages, modules and services. I discussed another, much higher level example, in the post on architecture all the way down. So what I really want to know is, “How do I determine the appropriate level of granularity?

This is the million dollar question. Because granularity is a significant inhibitor to creating reusable software that’s easy to use. (Oh, and managing dependencies too.) If something does too much, it’s less reusable. If something does too little, it’s more difficult to reuse.

In the past, I’ve used the diagram at right (click to enlarge) to help illustrate one view of granularity. As can be seen, services are more coarse-grained than modules which are more coarse-grained than packages which in turn are slightly more coarse-grained than classes. This begins to help answer the question, “What’s the right level of granularity?”

If I’m concerned that a service I create is simply too coarse-grained and fails to maximize its reuse potential, I can break the behaviors of the service out into modules that are a bit finer grained and more reusable (of course, one might consider doing this in general). Then I can compose the services from the modules and reuse the modules across services. The result is different entities at different levels of granularity that lends tremendous flexibility to how I compose, use, and reuse software entities.

This provides some guidance on the level of granularity at which different types of software entities should be defined. However, it still doesn’t offer enough guidance to determine the right level of granularity for the save method in our example above.

Another Dimension

There’s another dimension that is relevant, and that ties in nicely (though in a rather subtle way) to the initial coding example. It has to do with the way we traditionally layer our software. The diagram at right illustrates this (click to enlarge). As we can see, as we move down the layered hierarchy, entities become finer grained.

While this isn’t absolute, and depends on your architectural principles and style, entities in higher level layers tend to be more coarse-grained than entities in lower level layers because they are an amalgamation of their own behavior and the behavior of entities in the lower level layers.

Armed with this information, we can determine the right level of granularity for the save method as long as we understand in which layer the code containing the save method lives. If it’s in the data access layer, and we place architectural constraints on business rules living in the data access layer, then we shouldn’t have any validation code that lives in the data access layer. If the save method lives in a class in the domain layer, however, it may be suitable for the save method to contain validation code in the domain layer and invoke another method in the data access layer that actually performs the save operation.

The Complete Picture

At this point we can make the following general statements:

  • Entities in higher level layers are more coarse-grained than entities in lower level layers.
  • Services are coarser than modules which are coarser than packages which are coarser than classes.

And if we combine these two ideas into a single thought, and overlay the two diagrams, we can begin to visualize the level of granularity for different types of software entities.

Certainly this doesn’t offer hard guidance. Realistically, there are very few architecture and design principles and patterns that are universally applicable. In fact, I’m not certain you’d ever create a “presentation service.” But certainly you might create a data service that is composed of multiple data access modules. And you might have separate data services that reuse a data access module. And you might have a business service that uses a data service and is also composed of multiple domain modules. But you definitely do not want a data access module that invokes a business service, nor a business service that references a presentation module.

So this general guidance can serve as useful information to help answer our question, “How do I determine the appropriate level of granularity for a software entity?” And it can also serve as guidance when establishing architectural principles and constraints that help determine where software entities with specific behaviors should reside. Because without some scheme that will help determine the appropriate level of granularity, it’s hopeless to imagine that we’ll be able to design reusable software entities that are also easy to use.

Complexity: Essence and Accidents

Filed Under Architecture & Design, Development, Java, OSGi | 2 Comments 

I’d like to take a short moment to offer an additional perspective to my discussion on OSGi: Complexity is NOT the Problem. I believe this perspective adds clarity to that previous discussion, as well. All initiated thanks to a tweet, which summed up the situation in much less than 140 characters.

So over the weekend, I turned to the essay that said tweet refers, and reviewed the essence and the accidents. From the article, I quote Mr. Brooks:

“…to see what rate of progress we can expect in software technology, let us examine its difficulties. Following Aristotle, I divide them into essence - the difficulties inherent in the nature of the software - and accidents - those difficulties that today attend its production but that are not inherent.”

This leads me, pretty clearly, to the following simple conclusion.

Today, development teams leveraging OSGi to build server-side applications are burdened by accidental complexity. As platforms and tools mature, the accidental complexity of OSGi will be reduced to near zero. Modularity though, enabled and enforced by OSGi, attacks the essence.

If the difference between essential and accidental complexity isn’t quite clear, I highly recommend you take a few moments and read Mr. Brook’s essay (linked above). And then, if you can find a copy of the book’s 2nd edition, take a look at Chapter 17, “No Silver Bullet” Refired (Note: If anyone can find the Refired essay online, please post the link in comments).

Throughout these two discussions, you’ll find subtle hints extolling the virtues of modularity. But only do this if you’re willing to exercise your brain with thought, because the connection you’ll discover might just be transformational!

OSGi: Complexity is NOT the Problem

Filed Under Architecture & Design, Development, OSGi | 3 Comments 

The editors at SD Times have proclaimed that OSGi is too complex. Unfortunately, they miss the mark, and the following statement taken from the article is misleading.

We believe that OSGi, rather than simplifying server application development, has actually made it more complex, and there aren’t sufficient benefits to justify the added complexity.

This statement should have been tempered with context. It is not universally applicable, though is made to sound as if it were. OSGi may face challenges, but complexity of the OSGi framework is not one of the long-term inhibitors to adoption.

Technology Adoption Lifecycle

All technology goes through various phases of adoption. A lack of platform support and tooling may be inhibiting adoption today, but that is today’s problem, and one that is being dealt with. And there is a stark difference between complexity brought on by lack of tooling and platform support versus complexity associated with an ill-conceived idea. OSGi suffers today from lack of enterprise-class tooling and platform support. OSGi is not an ill-conceived idea.

Whenever an organization adopts a new technology, they’ll undoubtedly face some degree of initial complexity. New technology takes time to learn. As the innovators and early adopters continue to use OSGi, tooling is going to get better and developing modular applications using OSGi is going to get easier.

The question you must answer is whether the pain of adoption today is worth the gain of a modular architecture. There are many factors to consider when answering this question, but as the technology matures, the question will be easier to answer. As I’ve alluded to, other factors that have little to do with OSGi’s technical benefits will likely determine if it’s is able to cross the chasm.

Long Term Value

But the article’s biggest flaw is in stating:

And what’s the benefit again? Enterprise developers have written many, many server-side Java applications without using OSGi.

It’s important to understand its value over time. Let me pose some simple questions.

  • Is Spring too complex? It is far easier to create an application with concrete class references than it is to use dependency injection and abstractions.
  • Is Hibernate too complex? It is far easier to use JDBC than it is an ORM framework.
  • Is unit testing too costly? It is far easier to forego unit testing than it is to create a robust suite of tests.

Yet each contribute to more flexible software systems, and in that, there is significant benefit. So it is with OSGi and modularity, as well.

Leveraging OSGi to design modular applications will require us to learn new concepts, tools, and design paradigms, but there is no doubt that modularity has a tremendous upside. This makes the SD Times article paradoxical. OSGi does not increase complexity; the very benefit of modularity is that it helps us manage complexity.

More on Benefits

If you’re interested in exploring some of the benefits of modularity, you can start with the following:

OSGi - Feast or Famine?

Filed Under Architecture & Design, Development, Industry, OSGi | 11 Comments 

I want to be careful here, but also very clear - I advocate OSGi and modularity, though am concerned about its future. In some circles, OSGi is hailed as a disruptive technology; in others, it lives in relative obscurity. There are some indications that adoption is growing, and possibly at some point, it’ll catch like wildfire. But right now, OSGi is the most widely acclaimed technology that nobody is using.

Maybe I’m too harsh. Or too impatient. Maybe I need to give it more time. I know some will argue that people are using OSGi, and they’re right. Some are. But of the more than 6 million Java developers worldwide, those using OSGi and designing modular applications is only a very small fraction.

So what’s the problem? What’s needed to drive adoption? How will OSGi and modularity succeed?

What’s In a Name?

One of the greatest inhibitors to adoption has been platform support. Until recently, few platforms exposed the virtues of OSGi to the enterprise developer. That’s beginning to change, and should help drive adoption. Another obstacle is tooling, which I discussed a while back when lamenting that one of the biggest barriers to adoption is no migration path. With better tools, we’re beginning to overcome that challenge.

But will it be enough? Will platform support and great tools help drive adoption? Sadly, possibly not. And part of the problem, I believe, has very little to do with the technology, and much more to do with its image.

So, what’s in a name?

OSGi is the dynamic module system for Java!

Modularity - Old and Stodgy

Recently, I commented on a recent conversation I had with a conference organizer and his reluctance to accept my speaking proposal on modularity. The general sentiment was that it’s an old, boring concept that can be traced back 40 years! He was interested in my talk on agility, but I explained that given the choice, I’d prefer to speak on modularity.

Currently, I’m participating in a similar conversation with the organizers of the Agile 2010 conference on a few sessions I’ve proposed on modularity. And now, Peter writes about a similar experience. This sends an important message to the OSGi community, and highlights an important challenge surrounding modularity. The message: Modularity is boring! The problem: The importance of modularity, and its support as a first class citizen on the Java platform, is not well communicated nor understood.

Not a Technology Problem

This is a serious problem that requires attention. In his post, Peter suggested that micro-services may be a more apt term for OSGi services, so as not to confuse the term with web services. He even hints at proposing a new JSR that would introduce the idea of micro-services as a new primitive on the Java platform. This is a good start, and micro-services isn’t a bad idea, but more needs to be done. The problem is not that OSGi isn’t a great technology with real benefits, the problem is that nobody cares about modularity.

Possibly it isn’t that they don’t care, but instead that modularity won’t sell. Nobody wants to buy a 40 year old technology, they want to buy the next great thing. And without question, developing modular software is not a new idea. But in general, the ideas surrounding SOA were not new either. Yet, without question, SOA has garnered significantly more attention than OSGi over the past decade. SOA has had its day in the sun; OSGi and modularity have not.

This is not a technology problem. The real problem is that OSGi doesn’t have a clearly articulated identity that explains why an organization must be using it, and organizations will adopt OSGi only if its adoption can be tied to real business benefits. Just like object-orientation in the 1990’s and SOA in the 2000’s, each were sold with a wonderful accompaniment of benefits that would reduce cost, speed delivery, model the real world, blah, blah, and blah. Whether they did or did not help organizations realize these benefits is arguable. Whether each were widely adopted is not!

Crossing the Chasm

OSGi has the potential to be this decade’s most disruptive technology. Unfortunately, the virtues of OSGi run the very real risk of never coming to fruition if it’s unable to cross the chasm. And sadly, it can never cross the chasm unless we find an effective way to sell, or reinvent, modularity.

Foremost, OSGi, modularity, and micro-services do not represent the paradigm shift that’s necessary. Too often, we talk about OSGi in terms of its technical merits alone. Unfortunately, that’s not enough. If OSGi is to cross the chasm, it is going to require something different. Something that people can really latch onto. Something that gets them excited. Modularity isn’t enough. Nor is plug-in architecture. Nor any other technically superior thing that OSGi is.

Naturally, this is only my perspective. My opinion. But OSGi has been a widely acclaimed technology for several years, and it has achieved very little enterprise adoption. While the move by SpringSource to donate dm Server to the Eclipse Foundation was hailed as a great contribution to open source, it’s also a clear indication that adoption of dm Server was gaining little traction.

Now, it is possible that an 800 pound gorilla could throw their weight behind OSGi and modularity that will help drive enterprise adoption. Maybe IBM’s move to expose the virtues of OSGi to the enterprise will be enough. With other vendors following suit, and up and coming vendors such as Paremus on the rise, that just might be enough. But if it is enough, I still believe the enterprise isn’t going to buy OSGi and modularity for the reasons we believe they should, they’ll buy something else very trendy in which OSGi and modularity plays a central role. Something new. Something fashionable. Because as I recall Ivar Jacobson alluding to at a keynote a couple of years ago,

The software industry is more driven by fashion than the fashion industry.

Sad. But true! And hopefully when it happens it won’t dilute the value of OSGi and modularity.

Ignore my Rant if You Want

OSGi is a great technology solution. But we also recognize that technologically superior solutions often fail to win. One of the most popular stories is of the video tape format wars. So my little pontification here should in no way be interpreted as questioning whether OSGi is capable, but instead whether OSGi will.

You can choose to ignore me if you’d like, scoff at me, or call this a rant. Maybe I’m just impatient. Either way, I’ll still continue to proclaim the virtues of OSGi and modularity. I believe in it…firmly! But the nagging feeling I have surrounding its ability to cement its place in the world of enterprise software development will continue to tug at me. And I wonder…will it ever succeed? And if so, how? And…when?

What do you think?

A Badge of Honor

Filed Under Development | 4 Comments 

Often times, I hear folks exclaim that they’ve been on “40 development projects over the past 10 years.” Or, “70 projects spanning a 20 year career.” They say this as if it’s some badge of honor. But I’m not so sure that it is.

Of course, there is value in widespread project exposure. Gaining experience with new technologies. Experiencing the dynamics of different teams. Understanding the challenges surrounding different organizational cultures. Each project is unique, bringing its own set of experiences, and these experiences are all incredibly valuable.

But when I see that someone has jumped from one project to another, it also leaves me wondering! Have they ever stuck around long enough on any single project to see their decisions through to the end? Have they ever had to live with the decisions they’ve made? An amazing compilation of knowledge is obtained by not only participating in the early stages of development, but also in maintaining the software system after it’s been released. To name just a few:

  • Dealing with the challenges of Phase 2, while also maintaining Phase 1.
  • Managing multiple branches of the software system. Merging those branches back together. Creating new branches.
  • A critical production issue arising about the same time you need to start heading out for that important customer demo.
  • Attempting to change a piece of code that hasn’t been touched in 10 months.
  • Watching the software system grow from nothing to a system that’s more than 500,000 lines of code. Keeping the build performant. Keeping the build working!
  • Being forced to live with seemingly meaningless early design decisions that haunt you over time.
  • Trying to upgrade versions of third party libraries while development is ongoing. Simply recognizing the right time to upgrade.

And there is so much more. Priorities tugging you in five different directions at one time. If you’ve had the luxury of living with a system (and the mistakes) you created, you’ll realize that there are very few, if any, decisions that shouldn’t be given conscious thought.

When an individual sticks with a project for a long time, they realize the importance of maintaining a clean design, a robust suite of unit tests, and how they package their software system. There is significant knowledge gained by sticking with a project for a long period of time. Perhaps the next time you hire a developer, it might be wise to ask them, “What’s the longest you’ve spent on any given project?” That may be more important than the number of projects they’ve been on.

Somewhere along the way, I picked up a small piece of advice that has helped serve as a valuable guide. There is a big difference between ten years of experience and one year of experience ten times. When you jump from project to project, never living with the decisions you’ve made, you have lost an opportunity to learn what works well and what does not.

Next Page →