(This is an abridged version of a report I did at my job; I might post of copy of it once I remove anything that might be considered proprietary.)
I was tasked at my job with looking at ways of doing web services in our main application (which for an upcoming delivery is to be separated out into client and server portions). Said application is written primarily in C++, so naturally our first look was into frameworks written for C or C++ so that we would not need to bother with language bindings, foreign function interfaces, porting, new runtimes, or anything of the sort.
Our search led us to Apache Axis2/C. We’d examined this last year at a basic level and found that it looked suitable. Its primary intended purpose was as the framework that the client and server communicated over in order to transfer our various DTOs; that it worked over SOAP and handled most HTTP details (so it appeared) was a bonus.
I discovered after investing considerable effort that we were quite wrong about Axis2/C. I’ll enumerate a partial list of issues here:
- Lack of support: There was a distinct lack of good information online. I could find no real record of anyone using this framework in production anywhere. Mailing lists and message forums seemed nonexistent. I found a number of articles that were often pretty well-written, but almost invariably by WSO2 employees.
- Development is largely stagnant: The last update was in 2009. In and of itself this is not a critical issue, but combined with its extensive list of unsolved bugs and a very dense, undocumented code base, this is unacceptable.
- Lack of documentation: Some documentation is online, but the vast majority of the extensive API lacks any documentation, whether a formal reference or a set of examples. The most troubling aspect of this is that not even the developers of Axis2/C seemed to comprehend its memory management (and indeed our own tests showed some extensive memory leaks).
- Large set of outstanding bugs: When I encountered the bug-tracking website for Axis2/C (which I seem to have lost the link for), I discovered a multitude of troubling bugs. Most of them pertain to unfixed memory leaks (for code that will be running inside of a web server, this is really not good). On top of this, a 2-year-old unfixed bug had broken the functionality for binary MTOM transfers if you had enabled libcurl support, and this feature was rather essential to the application.
- Necessity of repetitive code: It lacked any production-ready means to automatically generate code for turning native C/C++ objects to and from SOAP. While it had WSDL2C this still left considerable repetitive work for the programmer (in many cases causing more work rather than less) and its generated code was very ambiguous as to its memory-management habits.
- Limited webserver support: Axis2/C provided modules only for working with three web servers: Apache HTTPD, Microsoft IIS, and their built-in test server, axis2_http_server. Our intended target was Microsoft IIS, and the support for IIS was considerably less developed than the support for Apache HTTPD. To be honest, though, most of my woes came from Microsoft here – and the somewhat pathetic functionality for logging and configuration that IIS has. I’m sorry for anyone who loves IIS, but I should not be required to manually search through a dump of Windows system calls to determine that the reason for IIS silently failing is that I gave a 64-bit pool a 32-bit DLL, or that said DLL has unmet dependencies. Whether it’s Axis2/C’s fault or IIS’s fault that the ISAPI DLL managed to either take IIS down or leave it an an indeterminate state no less than a hundred times doesn’t much matter to me. (However, on the upside, I did learn that Process Monitor from Sysinternals can be very useful in cases where you have otherwise no real source of diagnostic information. This is not the first time I had to dump system calls to diagnose an Axis2/C problem.)
- Poor performance: Even the examples provided in the Axis2/C source code itself had a tendency to fail to work properly.
- Their MTOM transfer example failed to work at all with Microsoft IIS and had horrid performance with Apache HTTPD.
- On top of this, the default configuration of Apache Axis2/C opens up a new TCP connection for every single request that is initiated. Each TCP connection, of course, occupies a port on the client side. On Windows, something like 240 seconds (by default) must pass upon that connection closing before the port may be reused; on Linux, I think it’s 30 seconds. There are 16384 ports available for this purpose. Practical upshot of this: A client with the default configuration of Axis2/C cannot sustain more than 68 requests per second on Windows or 273 requests per second on Linux. If you exceed that rate, it will simply start failing. How did I eventually figure this out? By reading documentation carefully? By looking at an API reference? By looking at comments in the source code? No, by looking at a packet dump in Wireshark, which pointed out to me the steadily increasing port numbers and flagged that ports were being reused unexpectedly. I later found out that I needed to compile Axis2/C with libcurl support and then it would use a persistent HTTP connection (and also completely break MTOM support because of that unfixed bug I mentioned). None of this was documented anywhere, unless a cryptic mailing-list message from years ago counts.
So, I’m sorry, esteemed employees of WSO2, but to claim that Apache Axis2/C is enterprise ready is a horrid mockery of the term.
This concluded about 2 weeks of work on the matter. In approximately 6 hours (and I’ll add that my starting point was knowing nothing about the Java technologies), I had a nearly identical version using Java web services (JAX-WS particularly) that was performing on the order of twice as fast and with none of the issues with memory leaks or stability.
P.S. Is it unique to Windows-related forums that the pattern of support frequently goes like this?
- Me: This software is messed up. It’s not behaving as it should.
- Them: It’s not messed up; it works for me. You are just too dumb to use it. Try pressing this button, and it will work.
- Me: Okay, I pressed it. It’s not working.
- Them: Oh. Your software is messed up. You should fix it.