Serving Up SOAP

Taste Tests

We invited BEA, Cape Clear, IBM, Iona, Microsoft, MindElectric Co., Novell, Oracle, Systinet and Sun to our Green Bay, Wis., Real-World Labs® to see which of their secret sauces could withstand our rigorous testing. IBM passed, saying a forthcoming release would be a better fit, and Microsoft reluctantly declined as well, preferring to serve up Windows Server 2003 at a later date. MindElectric bowed out, citing resource issues, and though Oracle worked hard to meet our deadlines, resource issues also prevented its participation.

So we tossed, whipped, mixed and beat products from BEA, Cape Clear, Iona, Novell, Systinet and Sun with the frenetic energy of the Swedish Chef. All the products deserve a standing ovation for their interoperability--during our testing, a failure to interoperate with our clients, regardless of the operating system/ development language platform used to build each client was negligible, and total interoperability was achieved with minimal tweaks to product-generated WSDL (Web Services Definition Language).

Happy SOAPing

One of the beautiful things about Web services is the agnostic nature of the standards behind SOAP (Simple Object Access Protocol). Because every existing Web services standard--from SOAP to WSDL to UDDI (Universal Description, Discovery and Integration) to WS-Security--is based on XML, it really doesn't matter what underlying operating system the product resides on; to provide Web services functionality the Web service must process XML. That means that half of the Web services' secret sauce, regardless of whether it's served on a J2EE or Microsoft .Net plate, is the XML parser used by the product. The second half is the way in which a product handles marshaling and unmarshaling of arguments. Marshaling is the process of taking arguments from the submitted XML document, such as a purchase order or customer ID, and putting it in a format usable by the developer. Unmarshaling is the opposite process--taking a language-specific construct and turning it into XML. If either of these ingredients is substandard, it means poorer performance and a more difficult time for developers to create Web services of their own.

The success of any such initiative is also dependent on the development environment and its support for building Web services. There's a multitude of new standards in play, and all the products we tested try to minimize the need for you to learn these standards, thus making your foray into the Web services world painless. The degree to which a development environment is seamlessly integrated with the target deployment platform is crucial, as is the product's ability to develop Web services rapidly--with minimal training. If a product can generate the framework and require only that the developer write business logic code and push a button to deploy, it has the chops to move your developers into the world of Web services.

As for security, we were disappointed in the general lack of support for XML-SIG and XML-Encryption in almost every product we tested. We understand the reluctance of most vendors to include support for

WS-Security--it's still under development--but XML-SIG, albeit still a work in progress, has been available since 1999, and XML-Encryption appeared in 2001. Both BEA and Sun have indicated that support for both standards will be available in their next product incarnations, while others are waiting for finalization of WS-Security before offering support.

Taking all this into consideration, we gave our Editor's Choice award to Novell's Extend 4.0. for its appearance, easy-to-follow Web services development and management, and broad range of features. BEA's WebLogic 7.0 was hot on Extend's trail, followed closely by Cape Clear's CapeConnect.Novell got itself a winner with its acquisition of SilverStream's Extend. Rebranded as Novell Extend, this application server offers both the corporate-class functionality required of an enterprise application and the features we'd expect for successful deployment of Web services.

Extend uses the standard J2EE deployment model, but the ease with which this is accomplished astounded us. The ability to deploy from an IDE to an application server is standard in most development environments that complement an application server, but configuration can be challenging, and getting it to work right the first time is always a chore.Not so with Extend. A seamless integration between its IDE, Extend Workbench and the application server was evident from the moment we hit the deploy button. Not only does Workbench support Ant (ant.apache.org), it supports deployment to other popular application servers, such as BEA's WebLogic Server and IBM's WebSphere Application Server. We were able to quickly create our echoInt Web service from an existing WSDL file, as well as from an existing Java class. All the products we tested offered this capability, though finding the option to do this in BEA's Workshop was difficult.

Novell's Extend, like Iona's XMLBus and BEA's WebLogic, automatically generates Web-based test clients for deployed Web services. All products we tested can build both Web- and Java-based clients, but the others require that this process be performed manually. The Web-based test clients are an excellent resource for quickly testing a Web service for correctness of deployment and functionality--one we used in every case we could to test our echoInt Web service after deployment to ensure the service was functional.

In addition to real-time monitoring of the application server via the management GUI, Extend provides a modularized management system for multiple instances of Extend servers, regardless of where they are physically deployed. Unlike those of its competitors, Extend's console is fully interactive--you can issue commands to the application server via the console--great for CLI (command-line interface) jocks or quick management that does not require a GUI console.

When a Web service is deployed to a cluster it is automatically deployed to all servers within that cluster. This feature, which was not available in most of the other products we tested, is enabled by Extend's use of a database as its repository as opposed to a file-based system. Novell does not currently support a file-based repository, but will offer this capability in a future release. The other products we tested support some type of scripting solution, usually Ant-based, to replicate applications across servers, even in a cluster. Systinet's product can propagate security across servers automatically, but still requires manual replication of applications.

Although we found Extend's support of XML security standards subpar, with no direct support for XML-Sig or XML-Encryption, like all the products in our review Extend does support both client and server certificates using SSL and offers a unique feature: built-in support for several leading SSL-acceleration products, such as those from Rainbow Technologies and nCipher.Novell's Web services strategy is comprehensive and includes the ability to package Web services run time within a deployable J2EE standard archive for a complete package that can be deployed to BEA's WebLogic, IBM's WebSphere and Apache's Tomcat and, of course, Novell's own application server, Extend 4.0. Although the company would prefer that you deploy to Extend, Novell has taken care to ensure flexibility in its architecture, letting you build a best-of-breed environment for your Web services initiatives. This move puts Novell in line with the pure Web services platforms, which care little about what application server an engine is contained within and focus more on providing a comprehensive Web services suite.

Novell Extend Application Server 4.0 (including Novell Extend Workbench 4.1 and jBroker Web 2.1, $500 per seat. Novell, (888) 321-4272, (781) 464-8000. www.novell.com BEA Systems WebLogic 7.0 | Cape Clear CapeConnect 4 Server | Systinet Corp. Web Application and Services Platform (WASP) Server for Java 4.5; Web Application and Services Platform (WASP) Server for C++ 4.5 | Sun's Sun ONE Application Server 7 | IonaOrbix E2A Web Services Integration Platform with XMLBus Edition 5.4

BEA Systems WebLogic 7.0

BEA's Web services strategy is comprehensive and forward-looking. WebLogic was held back from the top spot only by pricing and lack of support for XML security standards--issues that BEA says will be addressed in the next release of the product, which became available after we concluded our testing.

Offering a unique blend of support for J2EE experts as well as novice Web services developers, BEA's development environments are nothing short of impressive. We were delighted by the ability to develop both synchronous and asynchronous Web services within WebLogic Workshop without writing a line of code. BEA employs a unique set of JavaDoc tags that enable this unique functionality to orchestrate conversations.

For J2EE developers, exposing existing EJBs (Enterprise JavaBeans) and Java applications as Web services, as well as one-click deployment, can be accomplished via BEA's Web Service Builder. Unlike the other products we tested, however, generating Web services from an existing XML schema was not supported. The ability to create a Web service from a WSDL file is provided, but it's tucked away and hard to find. We developed our echoInt service from scratch within Workshop, and found it to be a pleasant process.

Also notably absent was support for XML-SIG and XML-Encryption. Although extensive support for existing security mechanisms is provided--and in fact blew away the competition with the number of options offered by WebLogic Server--the lack of support for XML-specific security detracted from the product's overall appeal. Support for HTTP Basic/Digest was welcome, as was full SSL support, including client and server certificates, but we would have liked to see at least some support for upcoming XML security specifications, as is offered by Cape Clear and Systinet. BEA indicates that support for these specifications will be included in the next release.BEA lets you use XML maps via WebLogic Workshop. These maps allow true loose coupling of Web services by mapping specific XML elements in SOAP requests to Java methods and parameters. This allows the underlying Web services code to be changed without requiring a change to clients. Changing our echoInt Web service from a DOC/LIT model to an RPC/ENC model required only a change in the properties of the JWS (Java Web Service). The other products we tested offer a similar approach, but their implementations represented a marked difference from Sun's offering, which generates a different set of code to support each model and therefore required that a new project be built.

BEA's management interface is extensive but falls short of Systinet's granularity in terms of Web services monitoring. The only product to come close to the level of sophistication offered by WebLogic in the general server-management arena was Novell, and even our winner fell short of providing the level of detail offered by BEA. Administrators accustomed to having complete control over every detail of their application-server configuration from a single, unified console will not be disappointed in BEA's offering. We'd still like to see more detail in all the products regarding Web services, as they present unique challenges in monitoring and management that are not easily covered by traditional J2EE management and monitoring mechanisms.

For example, a single endpoint may provide multiple messages, as does WeatherService with its WeatherByZipCode and getWeatherByCode. It's important to provide monitoring of not only the endpoint, but message-specific access information. Increasing the logging level in products from the default to Debug may provide this level of detail, but increasing the data being logged to such a level can degrade performance, and it means wading through information that may not be of interest to the administrator. Vendors need to address this issue and provide a means of monitoring Web services-specific information without degrading performance and without requiring a lot of extraneous information in the logs.

WebLogic 7.0 with WebLogic Workshop. For development, both products can be obtained free via a renewable 12-month subscription license. For deployment, WebLogic Server 7.0 is $10,000 per CPU. BEA Systems (800) 817-4BEA, (408) 570-8000. www.bea.com

Cape Clear CapeConnect 4 Server

Cape Clear's Web services platform consists of the CapeConnect 4 server and CapeStudio 4, an elegant IDE offering with a smorgasbord of options for developing all aspects of Web services. Included in the distribution were a Web services test client, XML and XSLT editors, and an integrated WSDL editor (also offered separately as a free download). Options on the server side are a bit more limited, with no integrated monitoring of Web services activity, as is offered by Novell and Systinet, and without a single, integrated console for managing distributed instances of CapeConnect. On its own, it's not enterprise-class, but if deployed into an existing corporate-class J2EE application server, the product should do just fine.

Although Cape Clear's offerings are completely J2EE-based, its products do not support deployment of J2EE-standard WAR or EAR files, as is found in more traditional application-server-based implementations, such as Sun's Sun ONE Application Server 7 and Novell's Extend 4.0. While CapeConnect can be deployed into a J2EE application server, deployment of Web services to CapeConnect is still limited.

Cape Clear uses Web services to deploy, publish and manage Web services deployed to CapeConnect, whether it's deployed as a standalone or within a J2EE application server. This is a common thread for platforms specializing in Web services. Iona and Systinet, both offering a standalone deployment configuration, also take advantage of Web services as a deployment model. We liked this model during testing--configuration within the IDE for deployment was greatly simplified, and we could deploy our echoInt service with the click of a button and minimal configuration. However, it's difficult to move from one platform to another, since a service packaged by one platform cannot easily be deployed to another platform without repackaging, the management model becomes proprietary in nature.

CapeConnect 4 Server and CapeStudio 4, $1,500 per developer. Cape Clear Software, (888) 227-3439, (781) 622-2258. www.capeclear.com

Systinet Corp. Web Application and Services Platform (WASP) Server for Java 4.5; Web Application and Services Platform (WASP) Server for C++ 4.5

Systinet WASP took us by surprise by incorporating corporate-class features into a platform and making it all look easy. Systinet's was the only product we tested to provide message-level security from the administration console as well as detailed monitoring on a per-service basis. Although Novell did an excellent job offering monitoring of Extend via the management console, no one came close to providing the level of detail available in Systinet's console. For our deployed echoInt service it was possible to examine the number of requests (both total, successful and failed) as well as average, longest and last transaction times. Letting us apply security to our Web service was also impressive. We could allow only specified users to access our echoInt service from the console--without server-side code modification. Other vendors should take Systinet's lead and incorporate this type of functionality.Systinet also uses Web services to manage Web services; therefore, deployment is not J2EE-standard. Like BEA, Cape Clear, Novell and Sun, Systinet takes advantage of Ant scripts to assist in packaging and deploying services. Ant support is becoming a de facto standard for Java-based development because of its portability and usefulness. This open-source Java "make" replacement is supported by many of the most popular commercial and open source Java IDEs, such as Cape Clear CapeStudio, Eclipse, IBM WebSphere, IntelliJ IDEA and Sun ONE Studio. Of the products we tested, only Iona's failed to support Ant.

Unfortunately, Systinet's is the only offering in our review without a replaceable XML parser. While other products can use a commonly available parser, like Apache's Xerces, or a custom-developed parser, only Systinet disallows replacement of this integral piece. The company says that using JAXP (Java API for XML Processing) to allow replacement of the parser would cause a performance degradation. Our tests showed that this is not likely the case, and we'd prefer being able to switch to a faster, more flexible parser if one becomes available.

Systinet is also the only vendor providing direct support for a language that is not Java-based; WASP supports C++ as well as C (provided it's compiled with a C++ compiler). Sun does support C++ via its NSAPI (Netscape API), and CapeClear can support it via CORBA. While the model of deployment is different--services are built into standalone servers rather than deployed to an application server--the performance was outstanding, and the code for our echoInt Web service was almost exactly the same, taking into account the slight syntactical differences between Java and C++.

Web Application and Services Platform (WASP) Server for Java 4.5; Web Application and Services Platform (WASP) Server for C++ 4.5. WASP Developer: free; WASP Server for Java: free for development, test and single-CPU deployments, $2,000 per CPU thereafter. WASP Server for C++: free for development, test and single-CPU deployments, $2,000 per CPU thereafter. Systinet Corp., (617) 868-2224. www.systinet.com

Sun's Sun ONE Application Server 7

Sun's Sun ONE Application Server 7 (AS7) screamed its way through our performance tests, offering the most scalable Web services environment among the products we tested (see "How We Tested Web Services," page 48). Its ability to process requests as load increased made it stand out from the pack. A standard J2EE deployment model put it on even terms with Novell and makes it easy for administrators familiar with a J2EE environment to manage Web services.

Sun's IDE, Sun ONE Studio, is not only free, it's also a comprehensive J2EE development environment that simply adds Web services as an additional option. The capability to develop a Web service or Web service client from a remote or local WSDL is shared by all the products we tested. We found the process of building and deploying a Web service in Sun's offering clunky compared with tools from Cape Clear and Novell, but it does work as advertised. We also noted that more code was generated by the application-server-based products in general, making the deployed services larger than those created by Cape Clear, Iona and Systinet.

Sun's submission was hindered by its lack of high-availability options--clustering, load balancing and failover--and a complete lack of support for XML security standards. These features are reserved for the enterprise edition of the product, which is scheduled for release in June.

Application Server 7 does, however, match BEA WebLogic and Novell Extend in supporting a wide range of authentication and authorization mechanisms, including LDAP, ADS, HTTP Basic/Digest and IP-based ACLs (access-control lists). Overall, we'd like to see more integration at the message level for each of these mechanisms.

Sun's Sun ONE Application Server 7 (AS7), unlimited free development seats for both platform and standard editions, $0/CPU for platform edition and $2,000/CPU for standard edition. Sun Microsystems, (800) 786-7638, (650) 960-1300. www.sun.comIonaOrbix E2A Web Services Integration Platform with XMLBus Edition 5.4

Iona's XMLBus is a platform, like Cape Clear's and Systinet's offerings, but it requires the use of an application server regardless of deployment. When it is deployed in a standalone configuration, a Tomcat J2EE application server is deployed as well to provide XMLBus with the framework necessary to perform. XMLBus also can be integrated into BEA's WebLogic, IBM's WebSphere Application Server and Iona's own Orbix E2A Application Server. The integration is performed during installation, making it a breeze to deploy into an existing environment.

XMLBus offers some additional value-adds in its IDE Web Services Builder, including a workflow component that makes developing composite Web services a snap. Iona says it expects WS-Choreography support to be included as soon as the standard is complete, which makes the platform an excellent choice for building collaborative and workflow-based applications that take advantage of Web services. Even without support for WS-Choreography, XMLBus' current solution is unique among those we tested in its support of choreographing the flow of business logic among disparate Web services.

XMLBus ships with a private UDDI server, intended for internal use only. Although this is useful for internal publishing of

services, we preferred the customizable taxonomies offered by Systinet's private UDDI server. Iona's default UDDI registry searches are based on the premise that services are public, and therefore require a search by organization, which is unnecessary in an internal-only scenario. The ability to customize taxonomies would be a good addition.

Orbix E2A Web Services Integration Platform, XMLBus Edition 5.4, $495 per development license. Iona Technologies, (800) 672-4948, (781) 902-8000. www.iona.comLori Macvittie, a Network Computing technology editor, has been a software developer, network administrator and member of the technical architecture team for a global transportation and logistics organization. Write to her at [email protected].

Post a comment or question on this story.

To evaluate the performance of Web services platforms, we considered three things:

1. Number of messages per second successfully processed

2. Consistent processing of messages under increased load

3. Differences in performance between processing of the DOC/LIT and RPC/ENC encoding modelsThe number of messages per second a product can process is dependent on the XML parser employed. The SOAP request must be decoded by the parser and then handed off to the appropriate Java method. An inefficient XML parser will degrade performance significantly as the size of the request increases, because there is more XML to decode.

Also entering the equation is the way in which a platform marshals and unmarshals arguments--in English, the way a platform transforms an XML element into a Java argument and vice versa. The way in which this is handled varies according to implementation. For example, Cape Clear's implementation handles DOC/LIT differently from the way it does RPC/ENC. In the DOC/LIT model, every argument--even primitive data types, such as an integer--is encoded into a corresponding Java class. That class may be simple, consisting of a single piece of data, or complex, a complete object that may comprise individual objects within it. In the RPC/ENC model, the arguments are often passed to the Web service as a primitive, which can increase performance.

Some platforms, such as those from Novell or Systinet, use the DOC/LIT model of processing even for RPC/ENC, which can degrade performance because of the additional instantiation of objects which are, in some regard, spurious. BEA provides an extra level of abstraction that allows elements within a SOAP request to be mapped to Java methods or arguments that do not necessarily share the same name. This is a definite value-add, making BEA's implementation more flexible, but it does add additional time to the process of marshaling and unmarshaling arguments.

Because Web services are in their infancy, it is not expected that the implementation of a Web service will result in it being inundated with thousands of messages per second. If that happens, we suggest taking advantage of clustering and load balancing, either via the products where available or through an external solution. Because of the real-world use of Web services, we were more concerned with the products' ability to maintain consistent performance as the number of concurrent users increased. In other words, we were examining the ability of each product to scale and were less concerned with the raw message throughput capability of the products under test.

For example, we were impressed with both BEA's and Iona's ability to remain consistent under load, even though the number of messages each processed per second appeared to be low as tested under an out-of-the-box configuration. Sun's product showed a high processing capability while maintaining consistency, while the other products decreased in their processing capabilities as the load increased.Take advantage of the tuning suggestions made by your vendor of choice. Changes to the Java Virtual Machine parameters, such as memory or threads, and modifications of activation policies, log levels and connection-level settings can drastically increase the performance of an application server, so consider tuning part of the deployment process. We set up a Hewlett-Packard Compaq ProLiant DL750, rolling in beefiness with its eight Xeon processors and 2 GB of RAM. Then we imaged eight SCSI 18-GB drives with a copy of Microsoft Windows 2000 Advanced Server, hooked up the Gigabit Ethernet NIC and prepared to test.

We installed each product on one of the prepared drives, then evaluated it in terms of management capabilities, monitoring and configuration options. We then built two Web services with the same functionality, one using DOC/Literal and one RPC/Encoded, and deployed them to the platform using the vendor's development environment. On each platform, we built two Web services, each based on a simple case within the WS-I's interoperability tests. We called both services echoInt. Both services took a single integer argument and returned that argument, plus one. The only difference between the two services was the encoding model. We also implemented a service that connected to our NWC Inc. customer database. This "getName" service took the user name of a customer and returned the first and last name of that customer. Each product was tested on the same version of Sun Microsystems' Java Virtual Machine, version 1.3.1.

Performance testing was accomplished by harnessing five Dell Optiplex machines, all running Red Hat Linux 8.0, and sending SOAP requests via ApacheBench to the product under test. Each test was configured to run for one minute or 50,000 requests (reaching either threshold completed the test) with a concurrency level of 10 on each machine, and was run three times against each Web service. Additional tests were run from a single client machine with concurrency levels of 10, 20 and 30, under the same time- and request-limit constraints.

Each client machine was time-synchronized to a public NTP (Network Time Protocol) server, and a script scheduled via at kicked off the tests to coordinate the five load-generating clients.

We tested interoperability by building MindElectric GLUE (Java) and Microsoft .Net (C#) clients from the WSDL (Web Services Definition Language) served up by each product and then running the respective clients against the service. ApacheBench, though used for our performance tests, was also indicative of the interoperability of each product, as the service was required to interact with the tool in order to perform our tests. We encountered some problems, specifically when trying to access the Web services from a .Net client. Minor tweaks to the WSDL provided the solution, but proved that concerns over interoperability are not completely unfounded.

R E V I E W

Web Services

Sorry,
your browser
is not Java
enabled

------------------->

Welcome toNETWORK COMPUTING's Interactive Report Card, v2. To launch it, click on the Interactive Report Card ® icon

above. The program components take a few moments to load.

Once launched, enter your own product feature weights and click the Recalc button. The Interactive Report Card ® will re-sort (and re-grade!) the products based on the new category weights you entered.

Click here for more information about our Interactive Report Card ®.