February 23 - Keynote titled OSGi in the Enterprise: Agility, Modularity, and Architecture’s Paradox
March 22 - 25 - Tutorial on Modular Architecture
June 14 - 17 - Sessions titled Turtles and Architecture and Patterns of Modular Architecture
July 26 - 30 - Two sessions on rich mobile applications and one on agile development. Half day tutorial on software process improvement.
Right on. I just won a Best Buy drawing worth $1000. Either that or I won a shiny new virus by clicking the link. Hmm...what to do. 2012-08-29
The opinions expressed on this site are my own, and not necessarily those of my employer.
Recently, I questioned whether OSGi and modularity would succeed in penetrating the enterprise. But what I really meant to question is whether OSGi will have the disruptive impact of which it’s capable. I asserted that if OSGi does succeed, it won’t be based on the technical merits of OSGi. It’ll be because of something else. Something trendy. Something fashionable.
Now, I could be wrong. OSGi adoption is certainly increasing, albeit slowly. And as case studies begin to emerge that tout the cost reduction, improved responsiveness, and time-to-market advantages of OSGi, adoption will likely continue to rise.
But adoption is one thing, disruption another, and I still have this nagging sensation that serves as cause for pause. What if something trendier, more fashionable surfaces, and OSGi is pushed into the backwaters? Will it really have the impact it’s capable of? You know…an “iPhone-esque” impact that raises the bar and redefines an industry. Not just evolutionary, but truly disruptive.
For OSGi to cross the chasm, it must enable something big that a business wants to buy. Maybe cost reduction, improved responsiveness and time-to-market benefits will be enough. But that’s easily perceived by many as a rather evolutionary impact. Not really disruptive.
Quite possibly OSGi will flourish in the data center, as organizations seek more adaptable platforms that lend them these benefits. But this doesn’t necessarily guarantee that development teams will leverage OSGi to build systems with a modular architecture. Because leveraging a platform built atop OSGi is separate from building modular software systems, even though OSGi enables both.
Without something trendy that promises real benefits, it’ll get brushed under the carpet like what has happened to many technologically superior solutions. Perhaps it’s already happening, with all of the hype surrounding the cloud. Or maybe it’ll be the cloud that helps OSGi cross the chasm. OSGi doesn’t obviously enable the cloud, but OSGi can enable the dynamic and adaptive runtime benefits of the cloud.
But again, this doesn’t mean that teams will begin leveraging OSGi to design modular software. It only means that the platform itself is adaptable. Of course, there is benefit in that. But if that’s the route that is taken, teams will still continue to develop monolithic applications that lack architecturally resiliency, and the full benefit of OSGi (and modularity) will not be realized.
OSGi has the potential to have have a much broader impact, affecting everyone from the developer to those working in the data center. So what might this trend be that will propel OSGi to stardom?
OSGi enables ecosystems!
Now, you’re thinking I’ve gone on the deep end, perhaps? But give me a chance…let me explain.
To start, I want to take a brief walk down memory lane. Not too far back though, but far enough so that we can see how important the ecosystem is in today’s most successful platforms. And these platforms span a range of markets, from mobile to social media. But each are successful in large part due to a thriving ecosystem.
In 2007, Apple released their first generation iPhone. Without question, the device revolutionized the mobile phone industry. While the device offers a great user experience that has certainly played a role in its surging popularity, people flock to the iPhone today because of the wealth of applications available. Yeah, “there’s an app for that.”
Apple, recognizing the power of this ecosystem, now delivers the iPad. With fewer preinstalled applications than the iPhone, Apple is counting on the ecosystem to drive adoption. The more consumers who flock to the device, the more developers who flock to the platform to deliver applications. As more applications become available, consumers will buy more iPads. The ecosystem fuels itself. Apple has simply provided the environment for the ecosystem to thrive.
In 2003, the Eclipse team was thinking of ways to make Eclipse more dynamic. Their decision to use OSGi to create a rich client platform that supports plug-in architecture was the first step toward the resulting ecosystem we know today. One of the reasons developers use Eclipse is because there are an abundance of plug-ins available that allow them to do their jobs more effectively. Other developers create Eclipse plug-ins because Eclipse is a popular IDE used by many developers. Again, the ecosystem fuels itself. The Eclipse team provided the environment that allows today’s Eclipse ecosystem to thrive.
If you’re interested, you can read more about the history of Eclipse and OSGi.
Arguably, Hudson is today’s most popular continuous integration server. But it hasn’t always been. Before Hudson was CruiseControl. And while CruiseControl did help development teams get started on their path toward continuous integration, it was also unwieldy to use in many ways. With Hudson’s plug-in architecture, developers have the ability to extend the tool in ways the original creator couldn’t imagine or couldn’t find the time to do himself. Kohsuke created Hudson and gave the development community a new platform for continuous integration. With its plug-in architecture though, he also provided an environment that allows the Hudson ecosystem to thrive.
Facebook. MySpace. Twitter. Each are examples of social media tools with a strong developer community that creates extensions to the platform that users can leverage to enhance the experience. Facebook Developers. MySpace Developer Platform. Twitter API. Each allows the ecosystem to thrive.
These are just a few examples. It’s easy to find other platforms with similar ecosystems, as well. The ease with which Wordpress themes and plug-ins can be developed and used to enhance a Wordpress website is another example. In fact, many content management systems provide similar capabilities. A large reason why the Firefox web browser has emerged as the preferred web browser is the ease with which add-ons can be installed that extend the capabilities of the browser. The Atlassian Plugin Framework is another example that uses OSGi, and platforms such as Force.com and SharePoint have built (or are trying to build) similar ecosystems.
Aside from Eclipse (and Atlassian), none of these other platforms leverage OSGi. Yet each are wildly successful because of two reasons:
If you look at many of the more popular platforms that have emerged over the past decade, they tend to possess a similar characteristic - a community of individuals dedicated to providing great solutions leveraging the foundation of the platform. OSGi and modularity enables ecosystems on the Java platform.
I’ve already talked about the two faces of OSGi - the runtime model and the development model. I’ve also explained how one could possibly see widespread adoption while the other has little impact. A strong ecosystem surrounding OSGi and modularity must leverage both. Developers would create reusable modules, implying they are designing modular software. For development teams to leverage these modules, they must be using a platform that supports the runtime model.
Now some of you might be arguing that this sounds a lot like the component based development (CBD) fad of the 1990’s. True…to an extent. Certainly these ideas are not new. But there are also some striking differences between that which OSGi enables and the CBD fad that has come and largely gone, or whose promise was never fully realized.
Foremost, the CBD fad was focused almost exclusively on visual components, such as ActiveX. While some attempted to create components for the Java platform, the movement largely failed to go mainstream. Instead, Java EE grew in popularity and for a number of years, garnered everyone’s attention. Why did this happen?
IMHO, the answer is fairly simple. Even though numerous marketplaces emerged that allowed the consumer and producer to come together to buy and sell components, there was never a suitable component execution environment. That is, an environment that would support dynamic deployment, support for multiple versions, dependency management, and, in general, complete control over all components currently executing within the environment. ActiveX components did have an execution environment (though did not support each of these capabilities), but Java did not. Today, in OSGi, Java has the requisite execution environment!
It’s easy find holes in this idea. To explain why it cannot work. Yet, it’s happening elsewhere, so why not on the server…in the enterprise? Certainly there are a variety of different ways such an ecosystem could manifest itself. Possibly multiple ecosystems emerge like what we see in the mobile market today.
But for a moment, imagine the world where you have the ability to easily assemble a platform from pre-built infrastructure modules that exactly meet the demands of your application. You might purchase these modules, you might choose to use open source modules, or you might build them yourself. For those you don’t build, you obtain from a module marketplace, possibly deploying them to your [cloud] environment.
And when you choose to use a module, it’s dynamically deployed to your environment. The modules it depends upon? You’re given the option to purchase and deploy them, as well. You develop your software modules using the sound principles and patterns of modular design to ensure loose coupling and high cohesion. As you roll out your business solution modules, you simultaneously deploy the additional infrastructure modules that are needed.
In this marketplace, modules are sourced by multiple vendors. Some large. Some small. Neither the stack, nor your applications, are monolithic beasts. Instead, they are a composition of collaborating software modules. Your infrastructure isn’t necessarily tied to a specific vendor solution. The option always exists for organizations to purchase modules from different providers, easily swapping one provider module out with another.
The ecosystem flourishes. Developers flock to sell their latest creation. Organizations seek to add amazing capabilities to their rightsized environment at a fraction of the cost compared to what they are accustomed to today. The business benefits are real. The technical advantages are real. And the resulting ecosystem is sustainable.
A successful ecosystem demands both the runtime model and development model. And today, OSGi is the only standard technology that will allow this type of successful ecosystem to form on the Java platform. But will it happen? We may have a ways to go, but it sure would be cool! And it would be a shame if we lost this opportunity.
Note: If you’re interested in ecosystems more generally, you might want to see the great TED talk by Dan Barber, “How I fell in love with a fish.” It’s very entertaining, informative, and worth 20 minutes of your time. His discussion on sustainability is truly fascinating!
Image Source: http://en.wikipedia.org/wiki/File:Blue_Linckia_Starfish.JPG
I want to be careful here, but also very clear - I advocate OSGi and modularity, though am concerned about its future. In some circles, OSGi is hailed as a disruptive technology; in others, it lives in relative obscurity. There are some indications that adoption is growing, and possibly at some point, it’ll catch like wildfire. But right now, OSGi is the most widely acclaimed technology that nobody is using.
Maybe I’m too harsh. Or too impatient. Maybe I need to give it more time. I know some will argue that people are using OSGi, and they’re right. Some are. But of the more than 6 million Java developers worldwide, those using OSGi and designing modular applications is only a very small fraction.
So what’s the problem? What’s needed to drive adoption? How will OSGi and modularity succeed?
One of the greatest inhibitors to adoption has been platform support. Until recently, few platforms exposed the virtues of OSGi to the enterprise developer. That’s beginning to change, and should help drive adoption. Another obstacle is tooling, which I discussed a while back when lamenting that one of the biggest barriers to adoption is no migration path. With better tools, we’re beginning to overcome that challenge.
But will it be enough? Will platform support and great tools help drive adoption? Sadly, possibly not. And part of the problem, I believe, has very little to do with the technology, and much more to do with its image.
So, what’s in a name?
OSGi is the dynamic module system for Java!
Recently, I commented on a recent conversation I had with a conference organizer and his reluctance to accept my speaking proposal on modularity. The general sentiment was that it’s an old, boring concept that can be traced back 40 years! He was interested in my talk on agility, but I explained that given the choice, I’d prefer to speak on modularity.
Currently, I’m participating in a similar conversation with the organizers of the Agile 2010 conference on a few sessions I’ve proposed on modularity. And now, Peter writes about a similar experience. This sends an important message to the OSGi community, and highlights an important challenge surrounding modularity. The message: Modularity is boring! The problem: The importance of modularity, and its support as a first class citizen on the Java platform, is not well communicated nor understood.
This is a serious problem that requires attention. In his post, Peter suggested that micro-services may be a more apt term for OSGi services, so as not to confuse the term with web services. He even hints at proposing a new JSR that would introduce the idea of micro-services as a new primitive on the Java platform. This is a good start, and micro-services isn’t a bad idea, but more needs to be done. The problem is not that OSGi isn’t a great technology with real benefits, the problem is that nobody cares about modularity.
Possibly it isn’t that they don’t care, but instead that modularity won’t sell. Nobody wants to buy a 40 year old technology, they want to buy the next great thing. And without question, developing modular software is not a new idea. But in general, the ideas surrounding SOA were not new either. Yet, without question, SOA has garnered significantly more attention than OSGi over the past decade. SOA has had its day in the sun; OSGi and modularity have not.
This is not a technology problem. The real problem is that OSGi doesn’t have a clearly articulated identity that explains why an organization must be using it, and organizations will adopt OSGi only if its adoption can be tied to real business benefits. Just like object-orientation in the 1990’s and SOA in the 2000’s, each were sold with a wonderful accompaniment of benefits that would reduce cost, speed delivery, model the real world, blah, blah, and blah. Whether they did or did not help organizations realize these benefits is arguable. Whether each were widely adopted is not!
OSGi has the potential to be this decade’s most disruptive technology. Unfortunately, the virtues of OSGi run the very real risk of never coming to fruition if it’s unable to cross the chasm. And sadly, it can never cross the chasm unless we find an effective way to sell, or reinvent, modularity.
Foremost, OSGi, modularity, and micro-services do not represent the paradigm shift that’s necessary. Too often, we talk about OSGi in terms of its technical merits alone. Unfortunately, that’s not enough. If OSGi is to cross the chasm, it is going to require something different. Something that people can really latch onto. Something that gets them excited. Modularity isn’t enough. Nor is plug-in architecture. Nor any other technically superior thing that OSGi is.
Naturally, this is only my perspective. My opinion. But OSGi has been a widely acclaimed technology for several years, and it has achieved very little enterprise adoption. While the move by SpringSource to donate dm Server to the Eclipse Foundation was hailed as a great contribution to open source, it’s also a clear indication that adoption of dm Server was gaining little traction.
Now, it is possible that an 800 pound gorilla could throw their weight behind OSGi and modularity that will help drive enterprise adoption. Maybe IBM’s move to expose the virtues of OSGi to the enterprise will be enough. With other vendors following suit, and up and coming vendors such as Paremus on the rise, that just might be enough. But if it is enough, I still believe the enterprise isn’t going to buy OSGi and modularity for the reasons we believe they should, they’ll buy something else very trendy in which OSGi and modularity plays a central role. Something new. Something fashionable. Because as I recall Ivar Jacobson alluding to at a keynote a couple of years ago,
The software industry is more driven by fashion than the fashion industry.
Sad. But true! And hopefully when it happens it won’t dilute the value of OSGi and modularity.
OSGi is a great technology solution. But we also recognize that technologically superior solutions often fail to win. One of the most popular stories is of the video tape format wars. So my little pontification here should in no way be interpreted as questioning whether OSGi is capable, but instead whether OSGi will.
You can choose to ignore me if you’d like, scoff at me, or call this a rant. Maybe I’m just impatient. Either way, I’ll still continue to proclaim the virtues of OSGi and modularity. I believe in it…firmly! But the nagging feeling I have surrounding its ability to cement its place in the world of enterprise software development will continue to tug at me. And I wonder…will it ever succeed? And if so, how? And…when?
What do you think?
I’ve been thinking a bit more about the list of disruptive technologies on Richard’s list, and then I watched Dan Pink’s TED talk on the surprising science of motivation. There must be some relationship between the list, which is comprised of almost all open source software products, and Dan’s assertion that the 20th century reward system won’t work for the cognitive tasks performed by workers in the 21st century. What is it, though?
We develop open source software to scratch a personal itch, ease the pain in performing a certain type of task, or create a more compelling alternative to a commercial product. While the professional open source model has emerged the last few years as a way for the open source community to create a sustainable business model, none of the open source technologies on the list were initially developed that way. They were developed in response to need. Ant because Java lacked a good build system. Spring because Java EE was cumbersome and bloated. JUnit to help increase quality. In many cases, these tools have grown to become defacto standard technologies widely used by enterprise development teams.
In Dan’s mind, the new incentive program within organizations must revolve around three things - autonomy, opportunity, and purpose. We must be given autonomy, or the empowerment to make our own decisions. We must be given the opportunity to master something that matters to us. And we must be given purpose, which is the desire to do what we do in the service of something larger than ourselves. Dan notes that financial incentive is also important, but is not the decisive factor in what motivates us.
There are a lot of ways to connect the dots between the open source products so prevalent on Richard’s list and Dan’s point about incentives. I’ll allow you the opportunity to connect these dots any way you wish. A few that immediately come to mind for me include the way commercial software is sold, corporate incentive programs, empowering developers, and flaws in corporate culture. But here’s something else to chew on.
In most of the cases, it was the developer who spurred adoption of the most disruptive application development technologies of the last decade. We weren’t motivated to develop or adopt these great products because of financial incentive or a reward system. Sure, in some cases that’s a positive side affect, but it is not the force that motivates. Instead, we were motivated because they make our jobs a little bit easier and our software a little bit better.
Developers are fighting like hell to create better software. While a lot of commercial vendors are selling shelfware (they aren’t selling it to the developer, mind you), developers are driving adoption of the technologies that are making a difference. Developers seek autonomy, opportunity, and purpose. Given corporate culture, it’s not always easy to find. But if the last decade is any indication, we are finding it, developers do have a voice, and that voice is being heard.
The Agile 2009 program is live, and it’s going to be a great conference. Thank you to the Developer Jam review committee for all the effort they put into helping select the sessions. The selection process for Developer Jam was not easy due to the quantity of amazing submissions, but I’m confident we put together an exceptional program. Whittling down more than 100 submissions into less space than we had at Agile 2008 was challenging. In the end, we had enough space for just 26 sessions. I’m certain that Developer Jam would have been a great conference of it’s own. Any takers on organizing this conference?
In addition to the sessions you see listed, we’re also putting the finishing touches on Programming with the Stars, where select conference attendees will be paired with legendary agile programmers to perform live on stage before a panel of famous (and outspoken) judges. The audience will participate in the judging contest. Be sure to stay tuned to the Agile 2009 conference site for additional details on Programming with the Stars.
We all like metrics, statistics, and measurements that let us know how we’re doing. Released about a week ago, the 2009 Standish CHAOS Report should have plenty of numbers. Jim Johnson, the guy behind it all, cites some numbers pulled from the report. It appears failure is on the rise. In fact, it’s the highest failure rate in over a decade.
I’ve always liked the Standish report. It’s packed full of interesting and high impact statistics that can really help drive a point home - especially when you’re on stage presenting. But you also have to be careful about reading too deeply into those number. It’s very easy to metriculate, and some have challenged the accuracy of the numbers within prior reports.
About 10 years ago I recall studying profusely so that I might pass my Java programmer’s certification exam. I purchased a copy of an exam cram book that had sample questions similar to what I might encounter on my certification test. I recall dealing with questions like the following:
I’m sure many of us are familar with these questions, as well as others of similar ilk. Of course, I’d taken similar exams prior to this for a variety of technologies, but the Java certification exam was the last one I’ve taken. I’d come to the realization that time spent studying for these exams is largely a waste of time. I was no better at developing software systems in Java after taking the exam than I was before. I simply possessed a bunch of useless knowledge that I could have easily gotten by looking at reference material.
The skill that I needed to develop enterprise applications could only be obtained by developing applications. It could be found no other way. There are no shortcuts. Making mistakes and fixing them. Discovering new ways of doing things that worked better than what I had done before. Knowing what assumptions I could make, and those I could not. The reality is that certifications exist largely to sell training courses, while the marketing that surrounds them leads organizations to believe they are relevant. A while back, the agile alliance issued their public policy denouncing agile certification. They state:
Knowledge is a wonderful thing, but businesses pay for performance. Performance requires skill.
A skill is not as simple to acquire as knowledge: the learner has to
perform the skill badly, recover from mistakes, do it a bit better, and
keep repeating the whole process. Especially for the interrelated and
interpersonal skills required of Agile software development, much of
the learning has to take place on real projects. It is that learning
that a certification should vouch for.
And unfortunately, it is that learning and skill that certifications do not provide nor account for. In fact, this is exactly why the software craftsmanship movement has begun to gain so much traction. Through trial and tribulation, a software craftsman has obtained the practiced skill required to develop great software. Of course, it’s easy to dilute the notion of craftsmanship if we are not careful.
Tom DeMarco also published an open letter to Cutter IT Journal expressing his dismay with certification. The debate on certification certainly is not new. But with the Scrum certifications, among the many others that abound today, the debate is still relevant. Where do you stand on certification? Would you rather work with a craftsman or someone with a wall plastered with certifications?
By the way, if you stand on the side favoring certification, and would like to get an agile certification, then maybe you should visit Agile Certification Now to get started.
The Agile 2009 CFP is open. This year, I’m the stage producer for Developer Jam. As you can see, Developer Jam has an impressive review committee. We’re hopeful that 2009 will be as successful as 2008.
Once again, we’re planning to run a series of clinics. They’ll be hands-on, instructor led exercises that will teach attendees something practical. If you have an idea, write up a submission, but don’t wait until the submission deadline of February 15th. Doing it sooner ensures increased visibility with the group of reviewers, and that means greater likelihood that the session is accepted. Of course, there are no guarantees. As a bit of additional incentive, be sure to check out the speaker compensation rates.
Software failure statistics are abundant and serve as clear evidence that we must reform software development. While industry claims an IT labor shortage is the motivating force behind outsourcing, the greatest factor is directly related to our inability to deliver value-add software. As organizations continue to lose faith in IT as a trusted partner, the services we offer are little more than an ample commodity, and the search for cheaper labor will persist. But, there is no IT labor shortage.
Some of you may have seen this video explaining the placement of phony job ads that are subsequently used to prove to the Department of Labor that there is an IT labor shortage. Lou Dobbs also got in on the mix as shown in this YouTube video, or take a look at the transcript. This ammunition is used to secure green cards for H-1B Visa workers. It’s repulsive. Bottom line - there is no IT labor shortage.
Here are some more numbers from the Dobbs video. Universities are pumping out over 300,000 bachelors, masters, or PhD degrees annually in computer or information science, math, and engineering. The Department of Labor predicts the average yearly job creation in those fields to be 120,000 jobs.
I’m a believer in competition, but it must be fair. Data suggests that on average, H-1B Visa Holders are paid between $12,500 and $20,000 less than their American counterparts. I’m not anti-H-1B. I’ve worked with a large share of very good developers who were H-1B visa holders. Unfortunately, the H-1B visa program is being used to replace the jobs of U.S. IT professionals with cheaper labor.
A two pronged approach is required to fix the problem and requires a professional code of conduct between employees and employers. The result is a win-win-win situation for all involved. First, we need not eliminate or minimize the H-1B visa program, but instead must bring the salaries of visa holders up to levels equal to that of their American peers. Second, we must reform IT through incremental delivery of quality software. Until these happen, U.S. citizens will continue to suffer job loss due to anti-competitive and fraudulent practices.
There is no IT labor shortage in the U.S. There is no dearth of software developers. Instead, this shortage is reinforced through repetitious pronouncements by industry of the impending labor crisis, and is used as outsourcing ammunition. In reality, organizations outsource because of two simple and related factors: