Sit amet felis. Mauris semper,

Welcome to WordPress. This is your first post. Edit or delete it, then start blogging!Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Quisque sed felis. Aliquam sit amet felis. Mauris semper, velit semper laoreet dictum, quam diam dictum urna, nec placerat elit nisl in quam. Etiam augue pede, molestie eget, ...

Category name clash

Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Quisque sed felis. Aliquam sit amet felis. Mauris semper, velit semper laoreet dictum, quam diam dictum urna, nec placerat elit nisl in quam. Etiam augue pede, molestie eget, rhoncus at, convallis ut, eros. Aliquam pharetra. Nulla in tellus eget odio sagittis blandit. ...

Test with enclosures

Here's an mp3 file that was uploaded as an attachment: Juan Manuel Fangio by Yue And here's a link to an external mp3 file: Acclimate by General Fuzz Both are CC licensed. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Quisque sed felis. Aliquam sit amet felis. Mauris semper, velit semper laoreet dictum, ...

Block quotes

Some block quote tests: Here's a one line quote. This part isn't quoted. Here's a much longer quote: Lorem ipsum dolor sit amet, consectetuer adipiscing elit. In dapibus. In pretium pede. Donec molestie facilisis ante. Ut a turpis ut ipsum pellentesque tincidunt. Morbi blandit sapien in mauris. Nulla lectus lorem, varius aliquet, ...

Contributor post, approved

I'm just a lowly contributor. My posts must be approved by the editor.Mauris semper, velit semper laoreet dictum, quam diam dictum urna, nec placerat elit nisl in quam. Etiam augue pede, molestie eget, rhoncus at, convallis ut, eros. Aliquam pharetra. Nulla in tellus eget odio sagittis blandit. Maecenas at ...

Posted by The Java Monkey - - 23 comments

Ok, so I've had to work a lot with GWT over the last 12 months. First up, I was really impressed at what GWT provides, especially the fact that it eliminates having to write browser code and forms. This quickly was overridden by the extremely poor compile times, the bulkiness of the GWT shell, and the fact that it allows developers to write truly terrible applications (due to its asynchronous nature) if you don't constantly monitor what they are doing.

Designing a GWT application up front is important. It is essential to make strict use of the Model View Controller pattern and obviously the Observer pattern (due to the asynchronous nature of GWT). The projects I have been developing GWT client's for, all have SOAP backend services that provide their actual data access (using Apache Axis 2). The SOAP services are all code-first POJO based services. The easiest way to put this together is to let the backend service assemble the GWT beans and perform all the transport and marshalling work to get them into your browser via the GWT RPC server. This way, you don't need to do any conversion from SOAP in to GWT beans and you don't need data transfer objects, because the GWT beans go all the way through from your server to your browser. That is, this cuts out a truckload of marshalling code, as everything is talking in GWT, and the SOAP layer handles all the SOAP stuff.

To do this, you need to create a separate jar/project for your GWT beans, that only contains the beans. This project is nothing but a pure, basic Java project that builds a jar file. It should not have the GWT framework or libraries applied to it. There are two minor things you need to do so that your GWT client project can work with these beans.

  1. Include the Java source of your beans in the jar file, so the GWT compiler can create the appropriate browser/RPC code.
  2. Include a .gwt.xml in your jar so GWT knows it contains GWT resources that it needs to use.

1. If you are using NetBeans, you can override one of the NetBeans build targets to make sure your Java files get built into the jar. The -pre-jar target should be overridden in your NetBeans generated build.xml script as follows:

<target name="-pre-jar">
<copy todir="build/classes">
<fileset dir="src">
<include name="**/*"/>

2. The gwt.xml file mentioned above is purely a standard .gwt.xml file, however there is one very important gotcha. GWT expects by default that your beans will be in a sub package of the package named client. This client package is in the same parent package as the gwt.xml file for your project. This is all well and good for most people, however I prefer to have more control over my package structure. I don't want to have my beans in a package called client, when the beans are a re-usable library that servers and clients both use.

To override this requirement, you simply add an extra line to your gwt.xml file with the name of the package that they can be found beneath. That is, if they are in a package called beans, then use the following definition line:

<source path="beans"/>

Note that you can have further packages beneath this folder, GWT just needs to know that it isn't the client folder.

If you fail to point this out to GWT, you will get the following exceptions during the GWT compile phase about it not being able to find the source code of your GWT beans:

    [java]       [ERROR] Line 206: No source code is available for type com.mym2.m2w.common.movements.Movement; did you forget to inherit a required module?
[java] [ERROR] Line 208: No source code is available for type com.mym2.m2w.common.movements.CashDeposit; did you forget to inherit a required module?
[java] Computing all possible rebind results for 'com.mym2.apps.client.LoginForm'
[java] Rebinding com.mym2.apps.client.LoginForm
[java] Checking rule
[java] [ERROR] Unable to find type 'com.mym2.apps.client.LoginForm'
[java] [ERROR] Hint: Previous compiler errors may have made this type unavailable
[java] [ERROR] Hint: Check the inheritance chain from your module; it may not be inheriting a required module or a module may not be adding its source path entries properly
[java] [ERROR] Build failed

So thats my first GWT blog, nice and simple to begin with. I will most likely be putting up further blogs as theres a lot of little tricks and techniques that should be known to get the most out of GWT. I'm also very interested in the architectures of GWT applications, as large applications absolutely require a good design in GWT.
[ Read More ]

Posted by The Java Monkey - - 12 comments

As part of my new job, I have been migrating our various development servers from a company in New Zealand to hosted servers in the States at Slice Host. Slice Host is a great setup, as it allows experienced developers to host their own Linux machines, without having to pay extra for support that you could do yourself.

Anyway, every time that I need to set up an Ubuntu machine, I always end up having to try and remember how I setup Java and Tomcat (among other stuff) on them. Hence, recording the steps and any gotchas down in a blog was worthwhile for me.

Note that this blog is written from the point of view of a Java developer who uses Linux, not a system administrator!

Installing Sun Java 6 SDK

To install the Sun Java 6 SDK, the Advanced Packaging Tool apt-get is used. Note that you need super-user access before running these commands. Before attempting the install, it is useful to update your list of installable packages using the following command:

apt-get update

Once the packages have been updated, execute the following command to install the Java 6 SDK:

apt-get install sun-java6-sdk

This should start installing Java 6. You will be prompted within the installer to agree to a license - use the TAB key to select the the OK and Yes buttons.

Java Gotcha One - Sometimes you will get the following errors, due to the Java 6 packages not being available from your package sources:

Reading package lists... Done
Building dependency tree... Done
Package sun-java6-jdk is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or is only available from another source
E: Package sun-java6-jdk has no installation candidate

To solve this, I had to add two more repositories to my /etc/apt/sources.list file:

deb hardy universe multiverse
deb-src hardy universe multiverse

deb hardy-updates universe multiverse
deb-src hardy-updates universe multiverse

Following this, running the update and then install commands works fine.

Sun Java SDK 6 should now be installed to your machine in the /usr/lib/jvm/java-6-sun (or similar).

Installing Apache Tomcat 6.0

To install Tomcat 6.0, I sidestep using apt in favour of directly downloading the tarball from an Apache download mirror and installing it manually. This way always seems to work a lot easier, and allows you to install it to custom locations if necessary.

To find out the exact URL of the Tomcat tarball you want to get, you can navigate around an Apache download mirror in a web browser. For example, enter an Apache mirror URL into a web-browser, and navigate down to a bin folder (where the binary tarballs are kept), for example:

A list of Apache mirrors can be found at

Once you have chosen the version of Tomcat (e.g. 5.5.25, 6.0.20) you want to use, grab the exact URL of the tarball for Tomcat, such as

To download this to your Ubuntu box, use the wget command:


Now decide where you want the Tomcat binary to install to, and what user account you want to own it. It is a good idea to not run the Tomcat from your superuser account for security reasons. Once you have decided where your tomcat is going, switch to that folder and extract the tarball to it by using the following command:

tar xzvf /path-to-tarball/apache-tomcat-6.0.20.tar.gz

This will create your Tomcat home area (and base if you intend to have base=home). If you want to, you can change directory into the bin folder and simply run to get your Tomcat running.

Seeing as this is going to be a server, you should also setup the Tomcat as a service, so that it starts up and shuts down as appropriate when the machine bounces.

Tomcat as a Service

The following simple Tomcat startup script can be used to control Tomcat as a service. Note that this script doesn't have non-superuser support in it yet, and only has start/stop/restart control cases. However, it is a good starting point, and is a lot better than having nothing:

#!/bin/sh -e
# Startup script for Tomcat


start_tomcat="$CATALINA_HOME/bin/ start"
stop_tomcat="$CATALINA_HOME/bin/ stop"

start() {
echo -n "Starting tomcat: "
echo "done."
stop() {
echo -n "Shutting down tomcat: "
echo "done."

case "$1" in
sleep 10
echo "Usage: $0 {start|stop|restart}"

exit 0

Create a file in your /etc/init.d folder, called tomcat (or whatever you want it to be called). Run the following command on the file to make it executable:

chmod +x /etc/init.d/tomcat

Now you can control the Tomcat using this script and the start/stop/restart parameters.

To let the operating system know it is a service that needs automatic controlling, you run the following command from the /etc/init.d folder (note that the following command assumes the script is named tomcat):

update-rc.d tomcat defaults

Once you've done this you have successfully installed Java 6 and Tomcat 6 and set it up as a service! I suggest looking at some of the following advanced topics if you're doing enterprise Java development:

If anyone wants a blog on any of these topics then let me know, particularly the tunneling one as its pretty cool and very useful!
[ Read More ]

Posted by The Java Monkey - - 12 comments

Its been half a year since my last post. In that time, I've separated from my wife, settled out my affairs with her, got a new job and worked overseas. Its pretty understandable, why I haven't devoted much time to my blog then!

The new company I work for, happens to use Axis 2 as their SOAP service engine. The setup is more complicated than I've previously used. There are about eight satellite Axis 2 SOAP services running on Windows servers, all talking to a central Axis 2 SOAP service, fronted by a GWT client. The communications is two way between the satellite web services and the central service. That is each satellite service knows about the central service, and vice versa.

That is, the central service acts a client to the satellite services and has client stubs for each of them. The GWT client, by its very nature, ends up causing multiple threads in the central SOAP service to be talking to the satellite SOAP services at the same time. There could be eight clients in the central service simultaneously talking to the satellite services, each with multiple threads in it.

Now this is all fine, and has been working since well before I started my new job (i.e. its not my fault!!). However, the other day, several times in a row, the central service suddenly started hanging with the Tomcat servlet in it printing the following exception out over and over:

SEVERE: Socket accept failed Too many open files
at Method)
May 19, 2009 5:31:32 PM$Acceptor run

This error essentially started occurring out of the blue, and then to make matters worse after restarting the Tomcat it happened 2 more times over the course of the next 24 hours. For a 24 hour available system, this is not acceptable!


Firstly, the open files concept relates to both file system handles and sockets. When running a Tomcat application, any jars in your WAR file will remain open during the lifetime of the tomcat, as well as JRE libraries, servlet API libraries, and many socket connections. What is happening islikely to be that your application is opening files and not closing them or opening sockets and not cleaning them up afterwards.

Ok, so its obvious what the issue is, however, we need to know how to diagnose this. Thankfully, on certain flavours of Unix/Linux there is the lsof command, or list open files.

In my case, I'm using a Debian install, which has lsof installed under /usr/sbin.

lsof lists all open files and sockets by your process, all it needs to know is the process ID. Note that its probably best to have superuser privileges when executing this command. For example:

lsof -p 14121

This will list all open files and sockets owned by your process. For example:

java 1456 tomcat 23r REG 253,1 27699 104136845 /data/tomcat6/lib/el-api.jar
java 1456 tomcat 24r REG 253,1 742089 104136851 /data/tomcat6/lib/tomcat-coyote.jar
java 1456 tomcat 25r REG 253,1 511240 104136848 /data/tomcat6/lib/jasper.jar
java 1456 tomcat 26r REG 253,1 228175 104136843 /data/tomcat6/lib/catalina-tribes.jar
java 1456 tomcat 27r REG 253,1 1128229 104136844 /data/tomcat6/lib/catalina.jar
java 1456 tomcat 28r REG 253,1 1395270 104136847 /data/tomcat6/lib/jasper-jdt.jar
java 1456 tomcat 29r REG 253,1 42558 104136854 /data/tomcat6/lib/tomcat-i18n-fr.jar
java 1456 tomcat 30r REG 253,0 840171 7799295 /usr/java/jdk1.6.0_06/jre/lib/ext/localedata.jar
java 1456 tomcat 31u IPv6 219309528 TCP *:xprint-server (LISTEN)
java 1456 tomcat 32u sock 0,5 219309526 can't identify protocol
java 1456 tomcat 33w REG 253,1 775017 81362966 /data/tomcat-operator/logs/www_mym2_com_access_log.20090520.log
java 1456 tomcat 34r CHR 1,9 2796 /dev/urandom
java 1456 tomcat 35u IPv6 219317480 TCP>wsm2app01:http (CLOSE_WAIT)
java 1456 tomcat 36u IPv6 219310003 TCP *:8011 (LISTEN)
java 1456 tomcat 37u IPv6 219310007 TCP localhost.localdomain:8007 (LISTEN)
java 1456 tomcat 40u IPv6 219317497 TCP>wsm2app02:http (CLOSE_WAIT)
java 1456 tomcat 41u IPv6 219317481 TCP>wsm2app01:http (CLOSE_WAIT)
java 1456 tomcat 42u IPv6 219317512 TCP>wsm2app01:http (CLOSE_WAIT)
java 1456 tomcat 43u IPv6 219317496 TCP>wsm2app01:http (CLOSE_WAIT)
java 1456 tomcat 45u IPv6 219317513 TCP>wsm2app01:http (CLOSE_WAIT)
java 1456 tomcat 46u IPv6 219317505 TCP>wsm2app01:http (CLOSE_WAIT)
java 1456 tomcat 47u IPv6 219317504 TCP>wsm2app01:http (CLOSE_WAIT)
java 1456 tomcat 48u IPv6 219317484 TCP>wsm2app01:http (CLOSE_WAIT)

You can see in this output snippet, that there are jars and sockets currently open by the process. If you examine the ouput further, you would be able to tell if your program has not being closing files that it should or if it has opened files multiple times.

How does this relate to Axis 2?

In the case of my Axis 2 client, there were two issues. The first issue, was, that I was creating an Axis 2 ConfigurationContext for every remote satellite server. This ends up causing your process to open multiple instances of .jar and .mar (Axis 2 modules) files to create each context. You should generally only ever need to create one context for the lifetime of your application. When piping the lsof output to grep, and grepping for jar and mar it becomes obvious if you're loading mars and jars too many times in Axis 2, as you will see the same ones appearing multiple times in the output.

The second issue, and the more important one (in my case), is the fact that once socket connections were finished with, they were not being thrown away. I was getting hundreds of sockets in the CLOSE_WAIT state. The definition of CLOSE_WAIT is as follows:

CLOSE_WAIT: The socket connection has been closed by the remote peer, and the operating system is waiting for the local application to close its half of the connection.

When I listed the open files for my tomcat, there were around 800 socket connections with a CLOSE_WAIT state. Whats more none of them were actually closing, they were seemingly hanging there indefinitely.

Note that in most Unix operating systems, the maximum number of file/sockets allowed open by a given process at any given time is 1024. Using the ulimit command, you can view this limit (ulimit -a) will print out the limits for your system. The one to pay attention to is open files.

It is possible to increase this limit, however the best solution is to fix your (Axis 2) application up to be using resources correctly.

Ok, so once you've made sure that you're only creating one context, the next step is to set your Axis 2 client stubs to using a multi-threaded HTTP connection manager, and forcing them to clean up idle connections when possible.

The way I do this is by having a Service Stub Connection Pool for each target service. Every time that my service wants to talk to a remote Axis 2 service, it borrows a service stub for the particular service it wants to talk to and then returns it after its finished with.

Within my connection pool class, I then create a multi threaded HTTP connection manager and use that instead of the standard Axis 2 connection manager. Subsequently, every time that a service stub is returned to the pool, I clean up any idle connections on the HTTP client it is using. After running tests against this connection pool in my target environment I have found it has a much better file/socket usage profile than the previous codebase I had to inherit.

Note that if you are only talking to one remote client, you only need one instance of ServiceStubConnectionPool, not one for each target service/URL. Additionally, if you are not multi-threading your calls to remote services then you will probably not even have this connection resource issue.

The rough code for this Connection Pool is as follows below. Note you will need to change the constructor for the ServiceStub to match the class of your Service Stub. Alternatively you could make ServiceStubConnectionPool abstract so that it can be re-used, featuring an abstract method constructServiceStub to do the construction.

package com.thejavamonkey.axis2.client;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.apache.axis2.AxisFault;
import org.apache.axis2.Constants;
import org.apache.axis2.client.Options;
import org.apache.axis2.client.Stub;
import org.apache.axis2.context.ConfigurationContext;
import org.apache.axis2.context.ConfigurationContextFactory;
import org.apache.axis2.transport.http.HTTPConstants;
import org.apache.commons.httpclient.HostConfiguration;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.MultiThreadedHttpConnectionManager;
import org.apache.commons.httpclient.params.HttpConnectionManagerParams;
import org.apache.log4j.Logger;
* <p><code>Service Stub Connection Pool</code> is used to pool
* Axis 2 client service stubs. It provides functionality for
* setting the service stubs to run with a multi-threaded
* connection manager and for closing idle socket connections.</p>
* @author thejavamonkey

public class ServiceStubConnectionPool {
* Creates a new instance of ServiceStubConnectionPool

private ServiceStubConnectionPool() {
// The logger for this class.
private final static Logger logger =
Logger.getLogger( ServiceStubConnectionPool.class);
// The axis2 client configuration context - only one of these should be around
// for your whole application. This is staticly created on demand.
private static ConfigurationContext context = null;
// Http connection management - these two member variables are needed
// per remote service.
private MultiThreadedHttpConnectionManager httpConnectionManager = null;
private HttpClient httpClient = null;
// Setup the multi-threaded connection manager. This is only done once
// per target service.
private void setupHttpConnectionManager( String host, int port, String scheme) {
if ( httpConnectionManager == null) { "setupHttpConnectionManager:" + scheme + " " + host + " " + port);
httpConnectionManager = new MultiThreadedHttpConnectionManager();
HttpConnectionManagerParams params = new HttpConnectionManagerParams();
HostConfiguration hostConfiguration = new HostConfiguration();
hostConfiguration.setHost( host, port, scheme);
params.setMaxTotalConnections( 100);
params.setMaxConnectionsPerHost( hostConfiguration, 25);
httpConnectionManager.setParams( params);
httpClient = new HttpClient( httpConnectionManager);
// Configure the connection for the client stub to use.
private void configureConnection( Stub client, String serviceURL) { "configureConnection: " + serviceURL);
if ( httpConnectionManager == null) {
// Parse the hostname, protocol and port for the target service.
String host = "";
int port = 80;
int portIndex = -1;
String scheme = serviceURL.substring( 0, serviceURL.indexOf( ":"));
if ( scheme.equalsIgnoreCase( "https")) {
port = 443;
for ( int i=scheme.length()+3; i<serviceURL.length(); i++) {
if ( serviceURL.charAt( i) == ':') {
portIndex = i + 1;
else if ( serviceURL.charAt( i) == '/') {
if ( portIndex > -1) {
port = Integer.parseInt( serviceURL.substring( portIndex, i));
host = serviceURL.substring( scheme.length()+3, portIndex - 1);
else {
host = serviceURL.substring( scheme.length()+3, i);
}"Configuring connection manager for URL [" +
serviceURL + "]: scheme = [" + scheme + "] host = [" + host +
"] port = [" + port + "]");
setupHttpConnectionManager( host, port, scheme);
Options options = client._getServiceClient().getOptions();
options.setProperty( HTTPConstants.REUSE_HTTP_CLIENT, Constants.VALUE_TRUE);
options.setProperty( HTTPConstants.CACHED_HTTP_CLIENT, httpClient);
// Get the axis 2 context.
private static synchronized ConfigurationContext getContext( URL axis2confURL,
URL repositoryURL) throws AxisFault {
if ( context == null) {
context = ConfigurationContextFactory.createConfigurationContextFromURIs(
axis2confURL, repositoryURL);
return context;
// Map of ServiceStubConnectionPool instances. Keyed on url.
private final static Map<String, ServiceStubConnectionPool> factories =
new HashMap<String, ServiceStubConnectionPool>();
// Get a factory by url. Create one if it doesn't exist. These are required per
// remote URL.
public static synchronized final ServiceStubConnectionPool getInstance( String url ) {
ServiceStubConnectionPool connPool = factories.get( url);
if ( connPool == null ) {
connPool = new ServiceStubConnectionPool();
factories.put( url, ServiceStubConnectionPool);
return connPool;
// The client stub pool - one of these is required per connection pool.
private final List<Stub> servicePool = new ArrayList();
* Borrow a service stub from the pool. This creates stubs as
* they are required. The stub MUST be returned to the pool after it has
* been used (in a finally block if possible!)
* @param axis2confURL the url location to the Axis 2 client configuration file
* @param repositoryURL the url location to the Axis 2 repository (modules,conf etc)
* @param serviceURL the http URL of the user authentication service
* @return the stub

public synchronized Stub borrowService(
String serviceURL, URL axis2confURL, URL repositoryURL,
String username, String password) throws AxisFault {
logger.debug( "Method call: borrowService [repositoryURL = " + repositoryURL +
", axis2confURL = " + axis2confURL + "]");
MonitorService10Stub result = null;
if ( servicePool.size() > 0) {
result = servicePool.remove( 0);
else {
// ******** NOTE: change the following line to use your own ServiceStub class
result = new MonitorService10Stub( getContext( axis2confURL, repositoryURL),
// Configure connection management to support a multiple threaded, reusable
// http connection client.
configureConnection( result, serviceURL);
// Configure your outflow security (rampart/WS-Security) here!
//configureOutflowSecurity( username, password, result);

logger.debug( "Method exit: borrowService [" + result + "]");
return result;
* Return a stub to the pool. Call this after you have made your client call.
* @param service the service stub

public synchronized void returnService( Stub service) {
logger.debug( "Method call: returnService [service = " +
service + "]");
// Clean up idle connections.
httpConnectionManager.closeIdleConnections( 20000);
servicePool.add( service);
logger.debug( "Method exit: returnService");
[ Read More ]

Posted by The Java Monkey - - 9 comments

If you are doing enterprise Java development I can guarantee that you will at some point have to deal with XML. XML has become ubiquitous with enterprise software as a medium for storing, processing and communicating information.

Depending on the parameters of your system, you will need to make an up-front choice of how you are going to process the XML. The three major options are:

  1. Read the the document in as a Document Object Model (DOM) and process it in memory in its entirety.
  2. Implement SAX Event Handling (SAX) to stream the document in, grabbing the data you need out of it.
  3. Use XQuery to query, transform and extract the data you need from the document.


If you are processing relatively small documents at a low frequency, the DOM approach wins hands down - because its simply a matter of dropping in dom4j.jar and adding the following code:

SAXReader reader = new SAXReader();
Document document = MyClass.class.getClassLoader().
getResourceAsStream( "document.xml"));
However, if you have a high frequency of documents, and/or they are large in size, then using the DOM is problematic. When writing performant Java code, it is important to ensure that the code is not I/O bound and that you are not needlessly creating objects. In Java object creation is fairly expensive.

The problem with the DOM is that it creates a lot of objects to represent even a simple XML document.

Safe SAX?

If the DOM doesn't suit, then the next approach is to use SAX event handling to process the documents. This requires you to write event handlers that get called when start, close and text nodes events are found in the document. Your event handler can then build up a simple object model in memory to represent the document. This has the advantage that it keeps the object creation to a minimum, and features you don't care about in the document can be ignored. The downside to this is that complex XML documents require a lot of code to handle them. As soon as a SAX parser becomes complicated, it becomes incredibly difficult to understand what is going on, as it can span hundreds of lines of code.


While SAX handling of documents is fine, sometimes you will be faced with truly massive XML documents to process, and in some cases you will not even know the exact schema - e.g. 500 megabytes in size. Situations like this call for a different set of tools to handle. Often when required to process big documents you are only interested in certain elements buried deep in the document structure. Writing a SAX handler to find these in a non-trivial or even unknown schema format is incredibly complicated and long-winded. In these cases, XQuery is the hands down winner for both complexity (or lack thereof) and performance.

XQuery is essentially SQL for XML. You write a query, and run it against a document. Anything in the document that matches your query can be returned to work on in your code. This has the advantage of not requiring lots of objects to be created as well as being simple to configure/use. Many of the more advanced features of SQL are also available such as joins and string functions. The XQuery syntax uses XPath as base for its queries.

There are many articles on the net about XQuery syntax, so I will not re-write that here. The initial difficulty in using XQuery for those new to it, is actually selecting a (free) implementation to use, and getting it running in your code.

The rest of this article is a focus on getting set up quickly with an XQuery implementation in your Java project, so you don't get stuck trying to learn how to use the XQuery Java API.

Saxon XQuery

The implementation of XQuery that we will be using is known as Saxon. Saxon has a commercial flavour and an open-source version. The open-source version differs in that it doesn't provide schema aware processing. This shouldn't trouble you - and if it is necessary you can simply get your work to buy the commercial version.

Once you've downloaded the latest release of the open-source variant (Saxon-B), you need to add the jar file(s) to your project. In my Maven 2 POM file, I have something similar to the following:

There is a lot of boiler plate code required to run an XQuery in Saxon. For that reason, it is very useful to encapsulate this in a class. To achieve this, I've created two classes to do most of this work for you.

Saxon needs to know about all the possible namespaces that will appear in the XML document that is being queried in order to understand the document. If it doesn't know them it will throw an exception during the document processing. For these purposes, I have created a simple Namespace bean that contains the namespace prefix and URI:

* Describes an XML namespace. A namespace consists of a prefix and an URI.
* @author Ants

public class Namespace {
private final String uri;
private final String prefix;
public Namespace( String uri, String prefix) {
this.uri = uri;
this.prefix = prefix;
public String getPrefix() {
return prefix;
public String getURI() {
return uri;

The class does that all the work is called XQueryRunner. This has one method on it, queryForXmlDocument. This takes a list of namespaces, an XML document dressed up in an Input Source (see below for example) and the x-query itself. It returns a string that contains an XML document. Check it out:

import java.util.List;
import java.util.Properties;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.sax.SAXSource;
import net.sf.saxon.Configuration;
import net.sf.saxon.query.DynamicQueryContext;
import net.sf.saxon.query.StaticQueryContext;
import net.sf.saxon.query.XQueryExpression;
import net.sf.saxon.trans.XPathException;
import org.xml.sax.InputSource;
import org.apache.log4j.Logger;
* X Query Runner provides an API to compile and run x-queries against an
* XML document.
* @author Ants

public class XQueryRunner {
// The logger for this class.
private static final Logger logger = Logger.getLogger( XQueryRunner.class);
* Run a query against the given xml document.
* @param namespaces defines the namespaces that are expected in the document
* that is being queried
* @param xmlDocument the document to query
* @param queryString the x-query
* @return the result of the x-query
* @throws net.sf.saxon.trans.XPathException on any XPath errors in the query
* compilation or execution
* @throws on any errors processing the input document

public String queryForXmlDocument( List<Namespace> namespaces,
InputSource xmlDocument, String queryString)
throws XPathException, IOException {
logger.debug( "entering XQueryRunner.queryForXmlDocument() query = [" +
queryString + "]");
StringWriter output = new StringWriter();
XQueryExpression expression = null;
Configuration configuration = new Configuration();
StaticQueryContext SQC = new StaticQueryContext( configuration);
DynamicQueryContext DQC = new DynamicQueryContext( configuration);
try {
try {
// Compile the query.
for ( Namespace namespace : namespaces) {
SQC.declareNamespace( namespace.getURI(), namespace.getPrefix());
expression = SQC.compileQuery( queryString);
SQC = expression.getStaticContext().getUserQueryContext();
SAXSource documentSource = new SAXSource( xmlDocument);
DocumentInfo docInfo = SQC.buildDocument( documentSource);
DQC.setContextItem( docInfo);
// Choose to output XML from the XQuery results (instead of plain text).
Properties props = new Properties();
props.setProperty( OutputKeys.METHOD, "xml");
props.setProperty( OutputKeys.INDENT, "yes");
// Run the query. DQC, new StreamResult( output), props);
String outputString = output.getBuffer().toString();
//outputString = outputString.substring( outputString.indexOf( ">") + 1);
logger.debug( "XQueryRunner.queryForXmlDocument() result = " +
return outputString;
finally {
catch( XPathException e) {
logger.error( "XPath error in query: " + queryString + " - " +
e.getMessage(), e);
throw e;
catch ( IOException e) {
logger.error( "Error reading XML document from input source " +
e.getMessage(), e);
throw e;

Lets get to it then...
Ok, so a quick example of calling it is probably a good idea also. This grabs an XML file and wraps it in an Input Source, then runs a query over it. Here goes:

import org.xml.sax.InputSource;

InputSource inputSource = new InputSource( new FileReader( "./document.xml"));
String query = "for $element in /rdf:RDF/rdf:desc[@xlink:arcrole=" +
"\"\"] return data($element/@rdf:desc)";
// Define the namespaces.
List namespaces = new ArrayList();
Namespace namespace = new Namespace( "rdf",
namespaces.add( namespace);
namespace = new Namespace( "dcterms",
namespaces.add( namespace);
namespace = new Namespace( "xlink",
namespaces.add( namespace);
String result = new XQueryRunner().queryForXmlDocument( namespaces, inputSource,
[ Read More ]

Posted by The Java Monkey - - 36 comments

I have mentioned in many of my previous blogs about having to implement Kerberos authentication in Axis 2 for a project at work. The project in question was an interoperable web service that had to also work out of the box with .Net WSE3 Kerberos enabled web services.

This blog, is a work in progress build of my results, that works for most cases - a lot of people have asked me about this stuff, so I thought I'd release the binaries for it with some instructions so people can try it out.

In the beginning...

Axis 2 uses the Rampart security module to provide its WS-Security features. Rampart in turn uses WSS4J which actually implements these features.

The bulk of the code changes I have made are to the WSS4J 1.5.3 codebase, with a few minor ones to Rampart 1.3, so that the Axis 2 configuration files can access the Kerberos functionality I've added.

Please note that to get Kerberos working with Axis 2 you absolutely must have a working knowledge of setting up Kerberos service principals, configuring KDCs and how to troubleshoot. If you still are in the dark on some of these topics, try my previous blogs where I have covered these topics in depth:

Lets get started...

The requirements are:

  • Java 1.5
  • Axis 2 1.3
  • Rampart 1.3
  • A KDC - preferably Active Directory or Apache Directory 1.5.1
  • Client user configured in the KDC.
  • Service principal configured in the KDC.
  • Web service client and associated web service.

Client Side Connections

The client-side outflow security configurations are based around a new security action, KerberosTokenSigned. My initial implementation requires the SOAP messages to be digitally signed, using the Kerberos session key. That is, the shared session key that the KDC supplies both the client and server, is used to digitally sign various parts of the message. The client and server are the only entities besides the KDC who know what the session key is - therefore the act of signing requests and responses provide an indisputable guarantee that the messages are from either the client or server.

JAAS configuration

As JAAS is used to provide the Kerberos services in Java, we have to use a JAAS properties file to configure both our client and server. Note that I'm working on implementing a JAAS extension so that all this stuff can be put into the Rampart configuration in the future, as it IS annoying having so many properties files to configure something. In fact, even having one properties file other than Spring is annoying to say the least.

The following or a similar JAAS properties file needs to be accessible from both your client and server side code:

ClientConfiguration { required

ServiceConfiguration { required
Note that the service principal in your config file should be set to your service principal and domain so that it works with your KDC. If you are an experienced Kerberos user, you can toy around with the key-tab and ticket-cache settings. I prefer to always talk to the KDC for authentication, instead of relying on caching - especially in a real network where user's passwords are changing every couple of months.

You can obviously keep the client and service configs in separate files on separate servers. When testing stuff on my laptop I usually keep them in the same file for expedience, as the example file above illustrates.

Setting debug to true causes the JAAS libraries to print copious amounts of debug information when you are authenticating. This is EXTREMELY useful in getting things to work initially.

In your code, you will also need to set some system properties when your client starts up. The code should be similar to the following:

System.setProperty( "", "true");
System.setProperty( "", "EXAMPLE.COM");
System.setProperty( "", "localhost");
System.setProperty( "", "C:/jaas.conf");
System.setProperty( "", useCreds);
Obviously, you will need to put your realm, your KDC hostname and the path to your JAAS configuration file into these properties.

Rampart Client Configuration

If you are using the old-style axis2.xml config file to configure your rampart security, configure your OutFlowSecurity as follows:

<parameter name="OutflowSecurity">
<!-- Timestamp header, message signature, and kerberos security token -->
<items>Timestamp KerberosTokenSigned</items>

<!-- The kerberos client principal credentials -->
<!-- Apache Directory account -->
<!-- Your callback handler goes here -->

<!-- The parts of the message to sign -->
<!-- Defines how the key used for signing is identified in the message -->

<!-- Identifies the kerberos service principal the session is with -->
<!-- Apache Directory account -->

The only notes are that you MUST use DirectReference as your key identifier (for interoperability reasons) and your client and service principals must work with your chosen KDC/domain controller. Also, you will need to point it to your own password/credentials callback handler, as is standard with Rampart configurations. (i.e. don't use the my.client.CredentialsHandler handler above because it doesn't exist!).

You can choose to sign whatever parts of the message you want to - the key is that you sign at least one part, preferably the body of the message.

When the response comes back, it will also be signed by the web service. Therefore, an InflowSecurity handler must also be setup to verify the signatures. The following is the matching InflowSecurity handler that achieves this:

<parameter name="InflowSecurity">
<items>Timestamp KerberosTokenSigned</items>
<!-- Verify the signatures from the server -->
<!-- Your callback handler goes here -->
This is much simpler, due to the fact that inflows only need to verify signatures rather than create them. Additionally, the kerberos principals and session details are already known from the request that was sent out.

Code Please?

Ok, so most people don't like configuring Axis 2 clients from config files - I know because I especially hate it myself. When logging into an application, having the username in a config is just plain old annoying! Due to the need to have loads of different users with Kerberos accounts using my services, which themselves contain axis 2 clients for other services, I always have to configure the security settings via code. To configure the client-side using code in Axis 2/Rampart 1.3, use the following (which is analogus to the previous two config files):

Note: serviceClient is your standard Axis 2 service client or RPC service client.

// Use this method to configure the inflow and outflow security for this client.
private void configureSecurity( String username, ServiceClient serviceClient,
String servicePrincipal) {
Options options = serviceClient.getOptions();
options.setProperty( WSSHandlerConstants.OUTFLOW_SECURITY,
buildOutflowConfiguration( username, servicePrincipal));
options.setProperty( WSSHandlerConstants.INFLOW_SECURITY,
buildInflowConfiguration( username, servicePrincipal));

// Build the outflow security configuration parameter.
private Parameter buildOutflowConfiguration( String username, String servicePrincipal) {
OutflowConfiguration ofc = new OutflowConfiguration();
ofc.setUser( username);
ofc.setPasswordType( "PasswordText");
ofc.setPasswordCallbackClass( "my.client.CredentialsHandler");
ofc.setActionItems( "Timestamp KerberosTokenSigned");
ofc.setEnableSignatureConfirmation( true);
"{Element}{" +
"/oasis-200401-wss-wssecurity-utility-1.0.xsd}Timestamp;" +
"{Element}{}Body;" +
"{Element}{}To;" +
"{Element}{}Action;" +
"{Element}{}MessageID;" +
ofc.setKerberosServicePrincipal( servicePrincipal);
ofc.setSignatureKeyIdentifier( "DirectReference");
return ofc.getProperty();

// Build the inflow security configuration parameter.
private Parameter buildInflowConfiguration( String username, String servicePrincipal) {
InflowConfiguration config = new InflowConfiguration();
config.setPasswordCallbackClass( "my.client.CredentialsHandler");
config.setActionItems( "Timestamp KerberosTokenSigned");
config.setEnableSignatureConfirmation( true);
return config.getProperty();
Again, you will need to configure this to point to your own password/credentials callback handler, instead of the my.client.CredentialsHandler class in my example.

Turning to the Server Side of the Force

To configure the server, it is an extremely similar process. A jaas.conf file is needed as per the client. The server side Kerberos security is configured via the services.xml file for the web service. Here is an example of the inflow/outflow security settings required:

<parameter name="InflowSecurity">
<items>Timestamp KerberosTokenSigned</items>

<parameter name="OutflowSecurity">
<items>Timestamp KerberosTokenSigned</items>
The important parts above are that the signature key reference on the input is DirectReference and the output signature key reference is Thumbprint. More specifically, the WS-Security spec requires the thumbprint to be the SHA-1 hash of the shared session key.

The kerberosPropFile element is a file that contains the Java Kerberos system properties for the server side. The best place for these were either in services.xml or external. I kept them external as it made the services.xml overly complicated. An example of such a file is:
Note that this file in turn points to the JAAS config file mentioned above. This indirection is kind've nasty, but it works at the moment, and is forced upon us by how JAAS works.

Thats all folks!

Ok, so thats how to configure Axis 2 1.3/Rampart 1.3 to use Kerberos authentication, so it can play nicely with other Kerberos enabled web services. The final step is to download the rampart-core, wss4j and kerberos-fix jar files below and replace the matching jars in your client and service with them.

Once you run it, you will either get a whole heap of weird errors due to not setting your Kerberos configs up correctly, or not configuring your KDC correctly, or you will see a whole load of extra stuff put into your SOAP messages such as BinarySecurityTokens. Here is a snippet of a SOAP request that has Kerberos added to it (click to view full size):

Note that the kerberos-fix jar contains a tweaked version of the Java Kerberos libraries so that the web service can pull the Kerberos session key out of the GSS Context. In the next release of this stuff, I'll be using my own Kerberos ticket decoding algorithm to achieve this (see my previous blog for more info).

And finally, I am happy to help people through their errors/problems - as the intent of this release is to get it tested and get feedback on it, with the aim of publishing the code back to Apache for integration back into Axis 2/wss4j. Enjoy!

PS: I also have an Axis 2 client and web service that can be used for this Kerberos stuff if anyone wants. Let me know and I'll put up a blog about it.
[ Read More ]

Posted by The Java Monkey - - 3 comments

I have been using the excellent Apache Synapse lightweight Enterprise Service Bus over the last year or so, including on a live, production project. I recently started on a new project that requires Synapse, and used it as an opportunity to switch to the new production release, version 1.2.

To my dismay, I discovered that to run the basic server on a Windows PC, you are now forced to use the Java Service Wrapper program, which runs Java programs as Windows services. There are two major problems with this requirement however:

  1. Wrapper.exe has some nasty bugs, and wouldn't work on any PC I tried it on.
  2. You shouldn't be forced to run Synapse as a Windows service when all you want is a simple instance to test your configuration files and mediators against during development.

Dr Watson!

The actual error we got on every Windows XP machine we tried Synapse 1.2 on is:

wrapper-windows-x86-32.exe has encountered a problem and needs to close.

Previously, Synapse was run using a .bat file. I grabbed this from a development snapshot of version 1.0 I still had lying around and modified it slightly so that it would run with the command-line parameter format of the newer version of Synapse (I had to look at the source code to work this format out).

To run Synapse 1.2 on Windows using this .bat file (see below), you need to define the following environment variables:


For example:

set SYNAPSE_HOME=C:\Java\synapse-1.2
set SYNAPSE_XML=C:\Java\synapse-1.2\synapse-config.xml
And, then simply put the run-synapse.bat file below into the bin folder of your Synapse install, and you're free from the wrapper-hell. Note that this batch file has port 8080 set as the port to run Synapse on.

[ Read More ]

Posted by The Java Monkey - - 10 comments

This is the first in a series of articles around the Java Developer Certification course. The plan is to do a new blog each month around this topic, so stay tuned!

At work, they have been pushing me to get the Java Developer Certification for a few years now. I've actually booked my project and almost completed it, but it has taken me several years to do so. It goes something like:

  1. Get fired up and work on it for a few hours.
  2. Wait 6-9 months.
  3. Go back to step 1 and repeat.

There is something that is extremely boring and unfulfilling about being at work 40+ hours a week, working on my own side projects for 20 hours a week and then on top of that having to write a University style room booking client/server application in a technology I absolutely despise! (there are many limitations to RMI that I do not like).

Anyway, in a move of desperation I have decided to discuss architectures related to the developer certification, which will force me to get cracking (hopefully).

So, over the next few weeks I've planned a series of blogs about the various parts of the certification project. Note that I'm not going to post any of my code and the purpose of these blogs is for purely a reasoning/discussion view point. Hopefully this may help some other developers to come up with a good plan for their developer certification project.

Personally, I'm hoping that it forces me to finish off the remaining bits in my project.

Project X

The developer certification is all about showing you are a capable developer. That is, can you write clean, maintainable code, and more importantly, can you take some requirements for a system and come up with a clean, pragmatic design, then implement that. The key point about the design for this project is that it has to be simple - so that a junior programmer could jump in and understand it and maintain it if necessary. An important requirement for such a system is to keep the number of classes low - that is, make your design efficient - which isn't a bad thing. A problem I have with developers who get a bit of experience is that they start going down a path that I call "Design Pattern Overload" - with the end result often becoming a disaster, where everytime you have to do something with the code, you have to spend 10 minutes trawling through the classes trying to remember exactly how they work together. This is more commonly known as The Simpleton Pattern.

You have your project assignment - so where to start? The first choice that needs to be made is RMI or sockets? This should be a no-brainer - unless you have to write socket communication layers as part of your job, you should choose RMI. All the other more important stuff in the project should have your attention rather than worrying about also implementing a socket layer with its own messaging protocol.

The next step, as it should be in any software project, is to map out the architecture of your project. In todays blog, I'm going to focus on the networked/stand-alone client design.

Are you In or Out?

The biggest design point for the developer certification is the requirement to run the server networked, or internally in a fat-client mode. This part should be simple, but from reading sites like Java Ranch, it seems that people have a lot of trouble with it.

The Adapter Pattern

The adapter design pattern translates the interface of a class to another interface, so that it can be used by other objects which are incompatible with the original interface. That is, the adapter takes the adaptee and allows another object to use it.

Requirement: The program must be able to work in a non-networked mode.
In this mode, the database and GUI must run in the same VM and must perform no networking,
must not use loopback networking, and must not involve the serialization of any objects
when communicating between the GUI and database elements.
So what does the adapter pattern have to do with an application that has to be both RMI/non-networked?

The GUI needs to access the business layer. It has to do so in both non-networked form and over an RMI connection, with transparency. That is, the GUI layer should not need to worry about whether or not the business layer it is talking to is remote or local. To achieve this, the GUI layer needs to talk to a service interface.

There will be two implementations of this interface, each which will adapt the business model layer for use with the client. The client is expecting an interface to the business model - the local, non-networked adaptor is simply a wrapper for the business model, whereas the remote adaptor is a remote, RMI object. As far as the client knows, it is simply talking to an implementation of an interface, and does not need to worry about whether that instance is local or running on a remote server. This completely decouples the user interface from the networking code which is a good thing. It means in the future the system could be expanded to have a third adapter which implements everything as a SOAP web service.

The following class diagram illustrates the local and remote adapters.

A key point in keeping your design simple is that your service interface will need to throw Exception so that it covers the need for remote methods to declare that they throw Remote Exception

Lets Get Cracking!

Ok, so thats the first part of your project underway - a nice, basic architecture to work with. My next topic in this series is going to cover the persistence layer code (including the notorious row locking topic!). Have fun.
[ Read More ]

Posted by The Java Monkey - - 9 comments

Axis 2 is an enterprise solution for an enterprise problem - Service Oriented Architectures. When running Axis 2 over the standard HTTP/S transports it is excellent. However, when running it over JMS, it has a major shortcoming (as of Axis 2 1.3).

The key feature of a system based around messaging is that messages must be 100% guaranteed to reach their destination. In the SOA world, this is handled via the Durable Subscriber enterprise integration pattern. A durable subscription instructs the messaging server to save messages, that are published while a subscriber is inactive. When the subscriber reconnects to the messaging server, the subscriber receives all the messages that it missed while it was inactive. This is a standard JMS feature which is one of the key patterns in enterprise integration systems.

Axis 2 1.3, does not support this, so I had to do some custom modifications to the Axis 2 transport layer so that I could make use of durable subscriptions. This blog is about how to configure your Axis 2 to support durable subscriptions, and contains a downloadable jar that contains the modified JMS transport layer for Axis 2 1.3.

Don't Shoot the Messenger!


I have modified Axis 2 so that the configuration fits in with the standard JMS transport configuration. The following example JMS transport receiver highlights the two new transport receiver configuration parameters. axis2.xml is generally located in the WEB-INF/conf folder of your Axis 2 server.

  1. UseDurableSubscriptions - this speaks for itself.
  2. DurableSubscriptionClientIdentifier - this is a unique identifier that the messaging server uses to identify your specific application server instance (for example, your dev tomcat).

Note that the example transport receiver below is configured to work with Web Sphere MQ, so the JNDI parameters should be changed depending on the messaging platform you are using.

<transportReceiver name="jms" class="org.apache.axis2.transport.jms.JMSListener">
<parameter name="defaultCF">
<parameter name="java.naming.factory.initial">
<parameter name="java.naming.provider.url">
<parameter name="transport.jms.ConnectionFactoryJNDIName">

<parameter name="transport.jms.UseDurableSubscriptions">
<parameter name="transport.jms.DurableSubscriptionClientIdentifier">



The final part of your configuration, is to configure your individual web services to be durable. My implementation is such, that you can have multiple web services and can define per web service, whether it is durable, or runs over HTTP or JMS (or both) or any combination.

The only parameter required for your service definition is, DurableSubscriptionName - this parameter identifies your specific web service uniquely to the messaging server so that when your web service connects to it, it knows what messages you have missed. As soon as you deploy your web service, you will see it start to pull all the messages off MQ that it missed!

If your JMS-enabled web service does not require to be durable, then simply leave this parameter out and it will resort to being a plain-old Axis 2 non-durable JMS service. Any of your web services running over standard message queues should be configured this non-durable way, as durability is only for publish-subscribe.

Note that the following services.xml excerpt is for a web service that is available over both HTTP and JMS and is subscribed to the topic MQ.BOOKING.TOPIC.


<parameter name="transport.jms.ConnectionFactory" locked="true">
<parameter name="transport.jms.Destination" locked="true">

<parameter name="transport.jms.DurableSubscriptionName" locked="true">

Note that I have not modified Axis 2 to have the ability to de-register a web service as a durable subscriber. Also note that durable subscriptions only work for publish-subscribe/asynchronous messaging - it does not work for point-point message queues and never will!

To anyone who is interested, I am also working on a major update to the JMS transport for Axis 2 so that it can heal its connections to messaging servers without having to restart the Axis 2 server.

Distribute Me

The jar file attached to this blog (see below) contains the JMS transport classes I fixed up for Axis 2 1.3 only. These either need to be put into the Axis 2 kernel jar so they take the place of the existing ones, or expanded into the WEB-INF/classes folder of your Axis 2, so they override the versions of these classes in the Axis 2 kernel jar file.

[ Read More ]

Recent Posts