February 23 - Keynote titled OSGi in the Enterprise: Agility, Modularity, and Architecture’s Paradox
March 22 - 25 - Tutorial on Modular Architecture
June 14 - 17 - Sessions titled Turtles and Architecture and Patterns of Modular Architecture
July 26 - 30 - Two sessions on rich mobile applications and one on agile development. Half day tutorial on software process improvement.
Right on. I just won a Best Buy drawing worth $1000. Either that or I won a shiny new virus by clicking the link. Hmm...what to do. 2012-08-29
The opinions expressed on this site are my own, and not necessarily those of my employer.
I stumbled across Uncle Bob’s SOLID principles of OOD quite a few years ago. They serve as the foundation of knowledge for designing object-oriented software. In fact, if you analyze the GOF patterns, you’ll find that virtually all of them adhere to the class design principles. I felt strongly enough about these principles that I included them in Chapter 1 of JOUP.
When used judiciously, they serve as excellent guidance in designing object-oriented software. But it’s not the SOLID class design principles I want to talk about here, at least not exclusively. Instead, it’s their less famous brethren that are often lost in the discussion of the SOLID class design principles…the principles of package design that I want to discuss.
The ill fate of the package design principles is not due to their lack of value, but due to misinterpretation. As a result, the package design principles are not used nearly as widely as the SOLID class design principles. The obvious reason for this is that more teams spend time designing class relationships than anything else. Anecdotal evidence has shown this to be true. But this is not the only reason. Another surrounds their perceived value, and this is where misinterpretation of the package design principles plays a crucial role. Instead of the package design principles, they should be referred to as module design principles.
I’m unsure why the term package was initially chosen, but I postulate that at the time the principles were documented, C++ was the target language. Unfortunately, when translating these principles to Java, the mapping between the package principles and Java packages was carried over literally, and the foundation for misinterpretation was born. For example, JDepend is a tool that reports the measurements related to many of the package principles, but does so for Java packages. The literal interpretation of the package design principles when translated to Java packages diminishes their value.
Don’t get me wrong. There is value in taking care to design Java packages, but there is greater value when applying the principles to design JAR files. In fact, the principles map more clearly to JAR files than Java packages, and the Reuse Release Equivalency principle supports this point by stating:
The granule of reuse is the granule of release.
The granule of release in Java is the JAR file, not the package. The greatest value of the package design principles is not when applied to Java packages, but when applied to Java JAR files. We recognize this today, but it hasn’t always been that way, and initial interpretations along with a misleading naming convention still lead people astray. It did me, initially. But my discoveries in applying these principles are also what led me to develop JarAnalyzer a few years ago. I finally recognized the value in applying these principles to JAR files. And combining the package (can I say module now?) design principles in conjunction with the SOLID class design principles sheds an entirely new and interesting light on how we go about designing large enterprise software systems.
In his recent blog entry, Uncle Bob provides insight to how you might get started with the SOLID principles. It’s tough work. Similarly, I recall working hard to learn the GOF patterns in the mid-90’s. They twisted my brain like a pretzel, but over time, through study and application, I was able to grasp and master many of the concepts. A key element I had to learn is that the patterns don’t represent a universal truth - they shouldn’t always be used. You have to recognize where the extra flexibility is worth the increase in complexity. It’s the same with these principles. Uncle Bob makes this point clear in his discussion of the SOLID principles, as well:
These principles are heuristics. They are common-sense solutions to common problems. They are common-sense disciplines that can help you stay out of trouble. But like any heuristic, they are empirical in nature. They have been observed to work in many cases; but there is no proof that they always work, nor any proof that they should always be followed.
That being the case, what are the most important areas of the system that need a SOLID design?
Every system has joints, which is the point where two modules connect. It is these joints within the system that require the greatest flexibility and resiliency. The reason for this is driven by change. Change that is encapsulated within a single module poses less threat than change that ripples across many modules. This is trivial, and it’s why changing the private methods of a class are easier than changing the public methods. Likewise, change confined to a single module is easier than change that spans modules. That’s logical. So if we’re able to create well-designed modules, it makes our life a little bit easier. And using the module design principles allows us to bring greater modularity to our applications if we apply the principles when designing JAR files.
It is these joints, or module boundaries, within a system that represent the ideal opportunity to leverage the SOLID principles. Using the SOLID principles to minimize coupling between JAR files offers the biggest bang for the buck. Realistically, we cannot make an entire system open for extension but closed to modification (the Open Closed Principle). But we can use OCP at the module boundaries (aka. the joints) to create JAR files with enough design flexibility to confine change to a single module. Using the SOLID principles when designing the relationships between modules provides a low barrier point of entry for using these principles to create better designs. For instance, it’s much easier to refactor the classes if we know the refactoring is confined to a single JAR file. With a rich suite of unit tests, we can prove it has no impact to the outside world. But if we have to refactor the classes and the refactoring spans the joints of the system, well…that’s an architectural refactoring. It gets difficult and messy.
Today, technologies such as OSGi and JSR 294 aim to leverage the JAR file in bringing greater modularity to the Java platform by treating the JAR file as a first class component. It’s why these technologies are so important…they make it just a little bit easier to design really big software systems. And if we apply the SOLID principles in conjunction with the module principles, we have guidance on how to best design large software systems. As interest in designing more modular software continues to increase, and especially as OSGi continues to gain momentum, the module design principles will grow in importance.
Over the past several months, I’ve written quite a few blog entries about OSGi. Here’s a recap of those entries. If you’re not familar with OSGi, the posts under How OSGi might be of interest. If you’re familar with OSGi, but aren’t clear how it can help, the posts under Why OSGi will be of interest. And if you want to get a sense for where OSGi stands in the market today, the posts under OSGi Market may peak your fancy.
Here are some posts that describe the motivating factors behind OSGi.
Here are some posts describing how to use OSGi. They are simple tutorials.
Here are some posts on the OSGi market.
Late last year, I also published an in-depth research overview on OSGi. A Burton Group client subscription is required to access the report. The Wordle image embedded in this post shows the word cloud for that paper.
My last post embedded OSGi in an application server using Felix, Jetty, and PAX WEB. Here, I’m going to embed Equinox in Tomcat. I originally set out to embed Felix in Tomcat, but the dearth of tools and frameworks available for embedding Felix made using Equinox much easier.
I use the word “much” pretty loosely, though, because there were still some hoops I had to jump through to get JSP compilation working. Of course, PAX WEB provided that capability when embedding Jetty in Felix. Unfortunately, no framework stepped forward that offered these capabilities in this embedding scenario. At least, I wasn’t able to find them.
I want to keep this post strictly educational (like a tutorial), but I’m going to post a follow-up shortly that discusses these two scenarios. To give you something to chew on in the meantime, both scenarios presented a different set of challenges, and we have a long ways to go before many development teams will be able to leverage OSGi for building web applications. Like I said, I’ll explain why in a different post, but for now let’s move on with this example.
In the spirit of using the simplest tools possible, all you’ll need for this example is Tomcat, Ant, and Subversion. If you went through the previous example, you’ve already got Ant and Subversion installed. If you already have Tomcat, you should be set. But if not, download Tomcat 6, which is the version I used for this embedding exercise.
Next, you’ll need to checkout the project from my Google code repository. You can put the project anywhere when checking out, but you’ll have to make sure you know the path relative to each directory so that you can install the necessary bundles. I have the project sitting in a directory right alongside Tomcat. To checkout, open up a Terminal window or DOS prompt and do the following:
svn checkout http://kcode.googlecode.com/svn/trunk/osgi/HelloWorldEmbedWebJSP
Of course, if you have a Subversion client installed like TortoiseSVN, you can checkout from the directory browser. I don’t use these tools, though. Next step!
This step has nothing whatsoever to do with OSGi. But as a basis for comparision, I felt it would be interesting to package the sample as a web application. Since this is the exact same example I used when embedding Jetty in Felix, it does offer a nice basis for comparing how we can deploy this functionality in different ways. To build the web application, simply navigate to the web directory in the console and type the following:
The build script spits out a web.war file in the bin directory. Simply take this web.war and copy it to the Tomcat webapps directory. If you haven’t started Tomcat yet, you should do so now. Just startup another DOS prompt or console window, navigate to the Tomcat bin directory, and execute the startup.bat or startup.sh. Once this is done, give it a second to explode the web application, and navigate to http://localhost:8080/web/hi. Like I said, if you went through the exercise on my previous post of embedding Jetty in Felix, you’ll see the same thing here deployed as a web application instead of a bundle. Onto OSGi.
The Bridge.war web application is the main reason I chose to use Equinox instead of Felix. It provides two very important functions that are critical when embedding OSGi in an application container. In fact, this is one of the items I’ll be speaking about in my next post - how difficult it is to use OSGi with the current generation of application servers. It’s virtually prohibitive. Anyway, here are the two necessary functions provided by the Bridge.
Deploying the Bridge is easy because it’s a webapp. Just take the Bridge.war from the web/lib directory of the HelloWorldEmbedWebJSP project and drop it into the Tomcat webapps directory. Give Tomcat a second to explode the web application, and test it by accessing http://localhost:8080/bridge/sp_test. You should see a page that says the Servlet Delegate is registered. You’ve now got OSGi embedded within Tomcat with a bridge that will feed requests to Tomcat and pass them onto Equinox.
Linux and Mac OS/X users take note. As you’ll recall in the previous examples, we install OSGi bundles from the OSGi console after starting OSGi. Well, since we embedded OSGi in Tomcat, where’s the console? The console is the same console window you used to start Tomcat. But because Tomcat redirects all output to a file, you may not see it. To enable the console, you’ll need to make some changes to the catalina.sh file in the Tomcat bin directory. Windows users shouldn’t have this problem, though I’m sure Windows users are dealing with many other types of problems. Anyway, once you’ve opened catalina.sh, comment out the following line so it looks like this. If you’ve got the same catalina.sh as I do that’s bundled with Tomcat 6, it’s line number 298.
#>> "$CATALINA_BASE"/logs/catalina.out 2>&1 &
Now, restart Tomcat and voila…in the console you used to start Tomcat, you should see the OSGi console. Type ss to see the list of installed bundles. Onward!
Before we deploy the bundle, we need to configure the environment. If we weren’t using JSP, we’d be done. But because JSPs require compilation, and the JSPs will run within Equinox, they can’t use the JSP compiler included with Tomcat. We need to include our own JSP compiler, and we’ll use Jasper. I’ve included all the bundles needed in the same directory where you found Bridge.war. To install the bundles, jump over to the OSGi console, and do the following:
osgi> install file:path to bundle/ javax.servlet.jsp_2.0.0.v200806031607.jar osgi> install file:path to bundle/ org.apache.commons.el_1.0.0.v200806031608.jar osgi> install file:path to bundle/ org.apache.commons.logging_1.0.4.v20080605-1930.jar osgi> install file:path to bundle/ org.apache.jasper_5.5.17.v200806031609.jar osgi> install file:path to bundle/ org.eclipse.equinox.jsp.jasper_1.0.100.v20080427-0830.jar osgi> install file:path to bundle/ org.eclipse.equinox.jsp.jasper.registry_1.0.0.v20080427-0830.jar
Now, type ss to view the state of the bundles. If they aren’t active, you’ll need to start each one of them. Again, in the OSGi console, just type:
osgi> start bundle id
Dependening on the dependencies, multiple bundles may start. Just do an ss in between each start to see which ones haven’t been started yet. Wash, Rinse, Repeat!
Unlike my previous example where I gave you a pre-configured environment that didn’t require you to build anything, we have to do the build and deploy of the bundle ourselves. Why did I do this? Only because Tomcat has a larger footprint than Felix and Jetty, and I didn’t want to put Tomcat in my Google code repository. Building is pretty easy though. We’ve already built and deployed the web application. Building the bundle is done the same way, except we’ll use a different Ant build script. For this step, you can either shut down Tomcat, or open up another DOS or terminal window. Navigate to the HelloWorldEmbedWebJSP/web directory, and type the following:
ant -f buildjar.xml
The JAR file overwrites the web.war in the bin directory and places a valid OSGi bundle named web.jar there for you. To install the bundle, make sure Tomcat is started, and type the following in the OSGi console:
install file:relative path to web.jar/web.jar
The bundle gets installed. Start it up using the OSGi start command. We can now access the OSGi enabled web application by navigating to http://localhost:8080/bridge/hi. Again, it’s the exact same functionality as the other web applications we’ve deployed, except using a different deployment topology.
To shut down, just type close in the OSGi console. We’re done. If you had any problems, let me know via comments here or by contacting me. Next up…some general notes on OSGi and hopefully messing around with Distributed OSGi.
A few additional notes about this exercise. Obviously it appeared a bit more difficult than embedding Jetty in Felix. But were it not for the PAX WEB framework, that exercise would have proven equally difficult. Also, the Bridge.war performs two very important functions for us. Felix has supporting documentation that shows how to embed Felix, but I found no framework that did it for me. The Bridge.war was available, so I used it. But were a bridge available for Felix, it would be easy to deploy that bridge and install web.jar with Felix embedded within Tomcat.
A recent post on InfoQ discusses a two part article by Israel Gat that argues in favor of internal vs external releases by decoupling the engineering aspects of a release from the marketing and sales aspects. By doing so, the engineering team can release the product internally on a frequent basis and the business is free to temporally choose the release they would like to promote. I agree. That sounds grand. But in part two, the author diminishes the effort of all that goes into a software release by stating:
From an engineering standpoint, a release is merely a branch in the
code tree, representing a packaging and distribution decision.
A software release involves much more, especially if the business is allowed to freely choose which release they promote to the public. In general, a software release requires that certain lifecycle activities be performed, including build, testing, and more. The frequency with which a team can release is going to be directly tied to the ability of the team to perform these other lifecycle activities. And there is a cost to these lifecycle activities.
In a session at Agile 2008, David Anderson discussed Future Directions for Agile, where he elaborated on these costs. The beginning and end of any value-added activity involves transaction costs. He goes on to state:
The prevailing paradigm in Western management were that overheads
(transaction and coordination costs) were inevitable and efficiency was
achieved through economies of scale (large batch sizes)
Because of the transaction costs surrounding SDLC activities, including build, test, and other release management tasks, the economies of scale motivate teams to release software less frequently. Is it really possible to adequately perform all engineering activities necessary to ready a software system for release on a weekly basis? And isn’t there a lot of waste involved in doing this if the business only chooses to promote a small percentage of internal releases? Absolutely…NOT! Dr. Gat is spot-on in that it can be done, and it should be done, but he doesn’t really tell us how to do it.
There is significant advantage to ensuring a software system is in a continuously releasable state because it allows the organization to bring the right product to market at the right time for the right customer. So the million dollar question must be this. How are transaction costs surrounding a software release effectively eliminated to provide the increased efficiencies without requiring economy of scale? Beyond saying that agile practices play a role, I don’t believe Dr. Gat answered that question in his articles. But if I were a betting man, I’d say that Dr. Gat’s teams leverage a pretty effective continuous integration strategy. Because if they don’t, I don’t know how they can pull it off.
An article on Search Software Quality discusses IBM’s transition to agile. There are a few key points to take away from this article.
Each of these are important points to keep in mind as teams seek to improve their software development process.
There are two approaches to working with OSGi when developing web applications.
In this post, I’m going to embed Jetty, the application server, into Felix, the OSGi runtime. As with my previous posts (Simple OSGi Service, OSGi & Modularity, and OSGi & Spring), I’m trying to use the simplest tools for the job to maximize the experience. That means Ant, Felix, Subversion, a simple text editor, and a terminal window. The code can be found in my Google code repository. Look for the project called HelloWorldWebJSP.
As the project name suggests, not only am I going to embed Jetty in Felix, I’m also going to make sure I can serve JSP pages, which requires a bit of extra trickery. If you need additional details, leave a comment or contact me. Let’s go!
Next thing to do is to checkout the project from the google code repository. If you want to use the scripts included with the project to startup Felix, you should checkout the project in a directory right alongside Felix. To checkout the project, you’ll need subversion. Then, navigate to the directory containing Felix and do the following:
svn checkout http://kcode.googlecode.com/svn/trunk/osgi/HelloWorldWebJSP
For convenience, I checked in the Felix bundle cache, so you don’t need to install all the necessary bundles. The environment is already configured. However, if you’d like to setup the environment yourself, you should be able to delete the felix-cache directory, and after starting Felix, go through the steps of installing the necessary bundles. I’d wait to do this until after you’ve had a chance to step through the rest of the example. But it’s up to you. If something gets messed up, you can always start over with a clean checkout. The bundles you’ll need to install can be found in the /web/lib directory. They are
jetty-6.1.7.jar jetty-util-6.1.7.jar pax-web-service-0.5.1.jar osgi.cmpn.jar pax-web-jsp-0.5.1.jar
Next step is to start Felix. Open a terminal window and navigate to the HelloWorldWebJSP directory. You can use the pre-written scripts (assuming your directory structure is correct), or you can invoke it from command-line like the following, replacing the path to felix with the location of felix:
java -Dfelix.config.properties=file:./config.properties -jar path to felix/felix-1.4.1/bin/felix.jar
Note that you should use the config.properties included with the project, and not the default Felix config.properties. This is because we need to specify an additional package import to make things work (javax.security.cert). Also, if the HelloWorldWebJSP and Felix directories aren’t side-by-side, you’ll have to change the file paths to the Felix bundles in config.properties.
At this point, Jetty should be running inside Felix, and you can navigate to the homepage in your browser. To do this, simply point your browser to http://localhost:8380/. Click hello. Click goodbye. To see the JSP, click the link that serves a JSP.
There isn’t a lot of magic here, but it did take some time to find all the bundles necessary to make it happen, especially to get JSP pages working. All in all, there are a total of 10 bundles, and that includes the four Felix bundles, and the bundle we developed as part of the web application (web.jar in ./web/bin directory). To see a list of all the bundles installed, type the following in the terminal window.
Of particular interest, you’ll see two PAX Web bundles. The OPS4J Pax Web - Service (0.5.1) is the bundle that provides Jetty integration (and actually starts Jetty), while OPS4J Pax Web - Jsp Support (0.5.1) provides JSP compilation via Jasper. Note that most of the examples on the PAX Web website use Maven. Since I didn’t want to use Maven, I installed these bundles manually through the Felix console.
The key element of the web application is the Activator. This registers the servlet, html pages, and JSP page so that Jetty can serve them up. This is where PAX Web makes the job much easier, because it encapsulates integration with Jasper.
There are a few interesting things you can do to experiment a bit further.
First, try setting up the environment yourself. Doing this isn’t that difficult since all of the bundles are included with the project when you checked it out. To do this, close down the existing Felix instance by typing the following in the terminal window.
Next, delete the felix-cache directory. This gives you a clean Felix instance with none of the bundles installed. Startup Felix and then, one-by-one, install the appropriate bundles (listed above) from within felix using the install command, as follows:
install file:./path to bundle
Note that if HelloWorldWebJSP and Felix aren’t sitting side-by-side in the file system, you’ll need to change config.properties to point to the location of the Felix bundles. They are found in the bundle directory under felix. Also, don’t forget to install web.jar from the bin directory (you may have to run the build script), since that’s the JAR that contains our application classes. Then, start the bundles using the following command at the Felix command line in the terminal, where the bundle id is the id of the bundle obtained using the ps command.
start bundle id
Second, make a change to the JSP, html, or servlet. There is an Ant build script in the web directory that allows you to recompile. Depending on what you changed or added, you may have to modify the build script to make sure any new files are included in the JAR file created. Once compiled, redeploy the web.jar to Felix from within the console. You shouldn’t need to restart Felix when doing this. Just do the following, where bundle id is the id of the Hello World Web bundle (ie. the web application):
stop bundle id update bundle id refresh bundle id start bundle id
Third, try deploying another servlet after registering it in the Activator. Or create and deploy another JSP. Or Add some images and additional resources. You can modify the build script to include them in the web.jar that’s already been deployed to Felix, or you can create your own JAR file and deploy it alongside web.jar. You can deploy these changes without interrupting what’s already running within Felix, providing a great feel for the flexibility of OSGi and the adaptable environments possible. Whatever you do, this base install should give you a foundation for experimenting with web applications using OSGi.
Soon I hope, I’ll be experimenting with the alternative configuration - embedding OSGi into the application server.
Update: While the example above uses Felix, I have also tested this example with Equinox. You can follow the same steps and the result is the same. Except the default port for the Equinox solution is 8080, and if you want that changed, you’ll have to do it yourself. Also, when using Equinox, replace the ps command with ss to see the bundle listing.