Web Service Testing with soapUI

For a few years now I have used the open source application soapUI to simplify Web Service testing.  Both to test services that I have written and to test external services that I have had to consume.  In this post I'll cover the basic types of testing you can do with soapUI and provide some practical tips on how to use soapUI when working with Web Services.  If you don't want to go through the detailed examples I suggest jumping straight to the Practical Tips section at the end.

Getting Started

After downloading and installing soapUI the easiest way to get started is to create a new project from an initial WSDL or WADL.  For demonstration purposes I'll use the free Weather Web Service at http://www.webservicex.net/globalweather.asmx.  We will create a test for each operation in this SOAP Web Service.  We will also create a Test Suite, that allows us to run all our tests at the click of a button.  Furthermore, we will create a Load Test for our Test Suite and a Mock Service to simulate the functionality of the Web Service.

To do this go to File and select New soapUI Project.  Populate the New soapUI Project screen in the following manner and click OK:

Basically, paste the URL to the web service WSDL file into the Initial WSDL/WADL field and additionally check the field to generate a MockService.  SoapUI will now load the Web Service definition and ask you questions about how to create the MockService:

For this demonstration change the path to "/WeatherMock" and check the Starts the MockService immediately checkbox, but otherwise accept the defaults.  After clicking OK, specify the name of the Mock Service as "Weather MockService" (or a name of your choice) and click OK again. 

This will complete generating artifacts for the SOAP 1.2 version of the GlobalWeather Web Service.
Next you will be asked the exact same questions, for the SOAP 1.1 version.  Since we don't plan to use SOAP 1.1 you can just hit Cancel.  (I haven't found a way to configure soapUI to only generate artifacts for SOAP 1.2.  If you know how to, please let me know! :)

Running Tests

To execute a given test, drill down to the requests that soapUI auto-generated and insert values that make sense for the given Web Service operation.  In our case, in the Navigator on the left, click on GlobalWeatherSoap12, then GetCitiesByCountry and then double-click on Request 1.
We'll set the Country name as Iceland and then hit the green arrow button to execute the test.

Voila, you should get results back from the Web Service, a list of Icelandic cities.  To check if the Web Service response is valid (conforms to the WSDL), right-click in the results window and click Validate.

Lets give this test the name "City Test".  (Right-click Request 1 and select rename) 
Then do a similar test for the GetWeather operation.  Click on GetWeather and then double-click on Request 1. Put in "Reykjavik" for CityName and "Iceland" for CountryName and run the test.  Also rename the test to "Weather Test".  After running the test, the results should look something like this:

Build a TestSuite

Now that we have created two tests, lets add them to a TestSuite so we can easily re-run them at any time.  In the Navigator on the left, right-click GlobalWeatherSoap12 and select Generate TestSuite.  Select Use existing Requests in interface, and check Generates a default LoadTest for each created TestCase and Single TestCase with one Request for each Operation:  

After clicking OK give the TestSuite the name "Weather TestSuite".   Double-click on GlobalWeatherSoap12 TestSuite under Weather TestSuite to show the TestCase editor.
Then click the green arrow button to run the TestSuite.  If all goes well you should be presented with a results-screen like this, indicating a successful test run:

If the TestSuite run fails you'll see FAILED at the top instead of FINISHED and you should get a message explaining why a given test failed.

Add Assertions

To make the tests more meaningful lets add some assertions to validate the responses from the Web Service we are calling.  You do this by opening up the test Request Editor (double-click on a given test step under TestSuite in the Navigator) and then click the plus-sign next to the green arrow button (second from the left).

For both tests lets add the assertion called SOAP Response, to ensure the Web Service is returning a valid SOAP response.  Then add the assertion called Response SLA and specify the response time as 2000 ms.  That basically means that we are going to consider the test a failure if we don't get a response within two seconds.  Lastly, lets add some content validation by selecting a Contains validation.  For the GetCitiesByCountry operation add the string "Iceland" as the content to expect in the response, and for the GetWeather operation add the string "Success". 

Then run the TestSuite again to make sure you get success-results.

Run a Load Test

When we created the TestSuite in last step, we told soapUI to generate a Load Test as well.  You'll find it under the TestSuite in the Navigator on the left, under the heading Load Test.  The default name is LoadTest 1.  Double click on it in the Navigator to open it up.  Before running the LoadTest you can tweak such parameters as number of concurrent threads to run, length of the test-run, and delay between tests.  Once you have made the desired configurations hit the green arrow button to run the Load Test:

In this sample, 5 concurrent threads are running the TestSuite for 60 seconds with a random wait of up to 1 second between the start of each test.
You can track the progress of the load test with the progress bar in the upper right corner.  If it reaches a 100% without reporting any test errors you are good to go.

Use the Mock Service

Back when we imported the Weather Web Service we told soapUI to generate and start a Mock Service.  That service can now be accessed at http://localhost:8088/WeatherMock.  This is  convenient for example if you are developing against a Web Service that has been designed (WSDL/WADL available) but not yet implemented.  Then you can have the Mock return an actual Web Service response to test your code even though the actual implementation hasn't been completed.  A default response has already been generated (under WeatherMock Service,  GetCitiesByCountry, and Response 1), which you can edit as you like.

You can also have the Mock service return different responses depending on which request it receives.  To demonstrate this, lets create two new MockResponses for the GetWeather operation:

• Click on GetWeather under the Weather MockSerive in the Navigator and select New MockResponse.  
• Give it the name "ReykjavikWeatherResponse".  Accept the automatically generated response, but put the value "Reykjavik" in the GetWeatherResult tag.  (Or even better copy the actual response from calling GetWeather for Reykjavik, which should give you a fully valid response).  
• Create another response called AkureyriWeatherResponse and put the text "Akureyri" in the GetWeatherResult tag.  

Now put logic in the Mock service for when to return each response:

• Double-click on GetWeather in the Navigator to show the MockOperation Editor.  
• Select ReykjavikWeatherResponse under MockResponses.
• Select QUERY_MATCH under Dispatch.
• Click the plus sign to add a new match. 
• Give it the name "Reykjavik". 
• Select Reykjavik and then populate the XPath value with:
  
  declare namespace web='http://www.webserviceX.NET';
  declare namespace soap='http://www.w3.org/2003/05/soap-envelope';
  /soap:Envelope/soap:Body/web:GetWeather/web:CityName
  
  This XPath query will grab the value from the CityName tag.
• Under Expected Value enter "Reykjavik".
• Under Dispatch to, select ReykjavikWeatherResponse.
• Repeat the same steps to create a Match that returns AkureyriWeatherResponse when the city name in the request is Akureyri.

Now you can test your mock Match-logic by opening up your GetWeather test and adding http://localhost:8088/WeatherMock as the endpoint to use (select the current endpoint in the dropdown and pick add new endpoint...).  Then run the test and play around with changing the city name in the request to get different responses from the Mock service.

Additionally, if you want to test your client-side error handling you can have soapUI generate a soap:Fault response and have your Mock return it.  You do that by creating a MockResponse and then clicking the apostrophe icon in the MockResponse Editor.  Then edit the auto generated response as appropriate.

Practical Tips: What to use soapUI for

Now that we have covered the basics of soapUI, here are some practical tips for putting it to use during your software development.

Test Web Services You Have to Consume

When consuming external services, before delving into code, use soapUI to "kick the tires" of the Web Service.  This especially applies when consuming newly written services.  Rather than potentially spending hours pulling your hair over why your client code isn't working, spend a few minutes with soapUI validating and getting familiar with the Web Service you are about to consume. In particular:

• Create and run simple tests for key Web Service operations
• Make sure there are no security/access problems
• Is the web service using some proprietary authentication protocol (NTLMv2 comes to mind) that might give you trouble during implementation?
  
• Have soapUI validate that responses conform to the Web Service contract (WSDL/WADL)
• Visually inspect responses 
• Do they make sense or are they some illegible auto generated garble that should really be cleaned up and restructured by the Web Service developer? 
• Do the responses meet your needs?
• Test a few boundary cases
• Does the Web Service implement proper error handling?  Or does it blow up with an HTTP 500 error or some non SOAP compliant text message?  Does the level of error reporting meet your needs?
• Add all your tests to a TestSuite.  That way you can quickly "ping" the Web Service to make sure everything is working on the other end.  When a problem arises, taking your code out of the loop is a good way to make sure the issue is on the remote end and not with your own code.

When developing against a newly written external Web Service, it is very rare that the Web Service works 100% out of the box as expected.  There is usually a fair amount of communication needed between the Web Service developer and the client-side developer to tweak things until the Web Service works as needed.  By using soapUI, you can inspect the Web Service right at the time it is delivered to you, and quickly spot things that may need to be fixed.  I usually take 5 or 10 minutes to do so and almost always have a list of things that need to be modified.  The response time for getting those changes implemented is usually very short, as the Web Service developer is still engaged in the project and things fresh in his mind.  If I don't get back to him/her until some weeks later when I finally get around to implementing my client-side code, the other developer has probably moved onto other things and/or forgotten why he implemented the service in a certain way.   

Lastly, if you have received a WSDL/WADL file, but the service hasn't actually been implemented, and you NEED to start implementation against the service (not ideal), then consider using soapUI to create a Mock service from the WSDL/WADL file.  That way you can have your code hit the Mock service and at least get some preliminary feedback on whether your client code is working.  Since creating a Mock service is really a breeze with soapUI it can sometimes be more practical than implementing Mock objects in your code.

Test Your Own Web Services

When writing a Web Service for others to consume it can be handy to have a soapUI TestSuite to sanity test your service.  Of course you should still write unit and integration tests for your code, but having a good soapUI TestSuite can be quick and easy way to find out if all your services are running as expected.  When you get that 4 AM phone call saying that something is broke, fire up soapUI and at the click of a button sanity test all of your Web Service operations.  If you are smart, you'll hand the TestSuite over to a support team so that you only get woken when the issue truly is on your end ;-)  Just make sure to add proper assertions for Web Service responses and include SLA assertions to test that things are not running dead slow.

If you are concerned about the performance of your Web Service or whether it can handle a given load, then a soapUI Load Test can be a convenient way to test that.  Set the number of threads to imitate the expected number of concurrent users for your service and add SLA assertions to make sure all requests are handled in a timely fashion.  Of course generating all the load from a single machine does not quite imitate real traffic, so for truer numbers consider having coworkers assist you in running simultaneous Load Tests from multiple machines.

Kanban with David Anderson

This week I enjoyed attending a two day Kanban training class by David Anderson, the Father of Kanban-style software development.  The class provided an overview of Kanban and offered many tips on how to get the most out of Kanban.  David rolled through a deck of some 150 slides and answered various questions.  On the practical side, the class was divided into four teams and each team had to design a Kanban board for an actual company and then play an entertaining Kanban game, which turned out to be quite competitive.  The team I was on, Grand Kanban :) managed to squeeze out a slim victory with around $62,000 in revenue, which according to David was a pretty good score ;-)

Few points from the class:

The Kanban Method is based on three core principles:

•    Start with what you do now
•    Agree to pursue incremental evolutionary change
•    Initially, respect current processes, roles, responsibilities & job titles

The five keys to a successful Kanban implementation are:

•    Visualize Workflow
•    Limit Work-in-Progress
•    Manage Flow
•    Make Process Policies Explicit
•    Improve Collaboratively (using models & scientific method)

Key class takeaways according to Dave were:

• Kanban is like water 
• It goes around obstacles and slowly changes them, rather than removing them like a bulldozer
• Change has to come from within
• How many experts does it take to change an organization?  Answer: One.  But people are going to have to want to change

Surprising fact from the class:

According to a research done by David, getting more done only ranks number 4 on most manager's lists!  The list of their preference being:

1. Predictability
2. Business Agility (ability to respond to changes in market)
3. Good Governments (managing budgets, money spent the way intended, etc.)
4. Getting more done

Playing the Kanban game

All in all a good class and I look forward to continuing the improvement of Kanban at Hugsmiðjan where we develop the Eplica CMS.

Jfokus Conference

This week I had the pleasure of attending the JFokus conference in Stockholm, Sweden.  This was a gathering of some 1250 Java geeks and some excellent speakers.  I thoroughly enjoyed sessions by Arun Gupta, Guillaume Laforge, Mario Fusco, Henrik Kniberg, Neal Ford, and many others.  Here are some of the technologies that caught my attention and I will be considering putting to use at my current work on the Eplica CMS system.

• Lambdaj - This library is just brilliant and really simplifies working with Collections.  Write more readable code in fewer lines.  Stop writing loops!  Mario Fusco's slides.
• CKJM - Use this library to intelligently pick areas of your code base to refactor.  To get bang for your buck you want to pick classes with high Cyclomatic Complexity and high Afferent Coupling.  That is, complex code in frequent use.  Comes as part of a Sonar plugin.  And BTW, if you are not already using Sonar, use it!
• Groovy - I have been intrigued by this language for some time now but never actually put it to use in my work.  I think it is time to change that.  Groovy to me is a natural progression of the Java language.  Making the boring/tedious parts of Java optional and adding syntax improvements and useful features like closures.  The kind of changes you might envision seeing in Java 10 or somewhere way down the road.  Why wait so long?
  As a starting point I am thinking about using these Groovy frameworks for our web based testing:
• Geb - Browser automation framework based on Selenium.
• HTTP Builder - A convenient API for complex HTTP requests.
• WADL & Jersey - When consuming/publishing RESTful web services from/to our customers I prefer to have a contract to work by.  WADL is just that and Jersey an API (JAX-RS 1.1 implementation) for creating/consuming REST services.
• JDK 7 - Obviously eagerly awaiting that one.  Planned GA date July 28, 2011, according to Tomas Nilsson at Oracle.

Other technologies I found interesting but don't envision using in the near future (just not a good fit for Eplica CMS): Scala, Clojure, deploying to the cloud,  NoSQL, Vaadin and DDD.

All in all a great conference which I highly recommend to Java developers.

Photos from the trip

Tweets

Easily Size and Arrange Windows on Mac OS X

One idiosyncrasy of Mac OS X is that application windows can only be sized from their lower right corner, and Mac OS does not have any kind of "snap windows into place" feature similar to Windows 7.  At work I have the pleasure of working on a large 27" iMac screen, so I frequently arrange two apps side by side while working, or might even have more than 2 apps visible on the screen at any given time.   Arranging that kind of setup is tedious when done manually with the mouse, but with a nifty little tool called Divvy the task becomes quite easy.   As can be seen in the Divvy in Action video on their site, when activated, Divvy overlays a grid on your screen and can be used to visually size an application to a specific portion of the screen.  That is great, but what I found faster and easier was to assign global OS X shortcuts to window arranging operations I frequently use and let Divvy do its magic without bringing up the grid panel at all.  Creating shortcuts is covered towards the end of the video, but to make them global you need to check the "Global Shortcut" check box.  Here is a screen shot of some of the configurations I set up:

For example if I need to have an application fill the left 50% of my screen I just hit ctrl-shift-1 (with the app in focus) and it will jump right into place.  While developing, I often arrange things so that my IDE uses the entire right-side of the screen, the website I am testing covers the upper left-side of the screen and the application sever console (logging output) covers the lower left-side.  To put the windows in place I simply put them in focus one at a time and hit ctrl-shift-2, ctrl-shift-3 and ctrl-shift-4, in that order, and the apps snap into place, filling the screen like this:

No space is wasted on the screen and no annoying mouse-dragging operations are needed. Of course you can assign whatever keyboard shortcuts make sense to you.
Divvy is not freeware, but low cost, and can be used in demo mode (periodically pops up a registration message) if you don't want to cough up the $14.  It supports multiple monitors and is available for Windows too.

Evernote - A "shoe box" organizer that works

I have been an iPhone owner since the iPhone 3G was released in the USA on July 11, 2008. Why do I remember the exact day? Because I was one of the nerds that stood in line outside Tuttle Mall, in Columbus, Ohio, way before the AT&T store opened! Anywho, I often get asked what my favorite iPhone app is. Having tried out a fair share of apps from the App Store, I have to say my all time favorite is Evernote.

If you are the type of person that tends to forget things, loose things, and hates to spend a long time organizing information, then this is app is for you! And you don't really need to have an iPhone to use it. It is available for all major smart phones. Plus you can use it directly through www.evernote.com, or via multiple Web Browser addons or use one of the many thick client versions they have for all major operating systems.

If I need to remember anything for later (a store receipt, notes from a lecture, a good deal I see in the paper, a shopping list, recipe from a friend, a note from my kids' school, the name of a good wine bottle,.. you name it) I simply snap a picture of what I want to remember and send it to my Evernote account via the Evernote iPhone App.  If I feel up for it I might tag the note (e.g. Recipe) and add a descriptive title (e.g. "Chili Recipe from Bob").  But often I am to lazy to do that, and it typically doesn't matter.  I just file it and forget it.  What Evernote does when it files my note is to automatically perform OCR on the picture and store any text it extracts as metadata along with the note.  So if Bob showed me a handwritten recipe that he had titled "Bob's kick ass Chili" then that piece of text (granted it is written in legible writing) would automatically be stored in the metadata index for the note.  So if years later I searched my Evernote account for "Chili", or "Bob" AND "Chili" I would find the recipe.

To me, Evernote is the shoe box method for organizing, that actually works!  Meaning, you can get away with throwing all your junk into one large shoe box and thanks to Evernote's powerful search capabilities have it at your fingertip anytime you need it.  Feeling left out at a chili cooking party? Just whip out your cell phone and say "One second, I have a better recipe!" and voila you are the star of the party! ;-)

For more info on Evernote, check out one of their many introductory videos on www.evernote.com.  Then sign up and get an account!  The basic account is free and should be sufficient for most users.

Notes on Test Driven Development and Testing

This week I attended the Agilis 2010 conference, including a 2 day course on Test Driven Development by Nat Pryce. Below are a few notes from his class. Not necessarily the key teachings, but rather nuggets of information that I found interesting.

• One of the main arguments for using TDD is that it encourages you to improve the design of your code.  Writing the test for a software component that you have designed, but have yet to implement gets you to think about and question the purpose and nature of your design.  E.g., if when writing a unit test for a given class you discover that the test code gets way to complex or convoluted, then that is probably an indication that the class being tested has too broad responsibility and should be refactored into smaller, simpler units.  I had always thought of TDD as more of a means to ensure you build a comprehensive test suite and as a result have fewer bugs in your software, but had not given much thought to the fact that it is in essence a way to get you to improve your software design; hence making your code easier to understand, use, and maintain. 
• We did some exercises using the JMock framework, that Nat Pryce co-wrote.  JMock is a neat tool to help you mock out interfaces that your code interacts with and to validate that your code is using the interfaces as expected.  JMock allows you to command the mock object of the interface to behave a certain way (e.g. return specific results) and set up expectations for how you are planning to call the interface (e.g. methodA will be called once and only once with arguments "ABC" and 99).  These expectations are integrated with JUnit and if not fulfilled by the end of your test run then JUnit will fail the test.  
• The following JMock Cheet Sheet page provides an overview of he JMock syntax.
• During the Q&A session with Nat he admitted that JMock was probably best suited for green-field projects (new systems) while frameworks like Mockito where better suited for brown-field projects (preexisting systems that you are trying to create tests for).
• We spent some time discussing Monitoring Events, which is the concept of having your system broadcast notifications (events) about your code execution.  E.g. when an order is placed or when a user logs into your system a notification of the event is sent to a JMS Topic that interested parties can subscribe to.
• This is great for logging.  A logger component can subscribe to the topic and log all events in the system.  It can then allow you to filter out certain events or do things like group events by requestid and provide a holistic overview of a single user transaction (as opposed to having to grep through numerous log files on multiple servers to try piece together what happened when a given user transaction ran through the system, which may have spawned multiple threads in multiple JVMs).
• Monitoring Events are great for testing too.  Imagine trying to assert that a call to a checkout service (to complete the purchase of a product) will result in a proper inventory reduction in asynchronous inventory system.  If your test runs straight through and checks the inventory status as soon as it has completed the purchase, then the test will likely fail, since the inventory system hasn't had time to process its update inventory request.  One might try to fix such a test by adding a sleep statement of say 10 seconds after the checkout call but before the inventory is checked (which still might fail if the system is running slow).  Or (which is a little better) one might implement a loop that every 1 second pulls the inventory system to see if the update has been received (succeed fast).  In both cases we are polluting our tests with sleep statements and lengthening the time to feedback, when we run a suite of tests.  A better way would be to have the test subscribe to the Monitoring Event (topic) and complete (succeed) as soon as it has received an inventory update notification.
• You can also use Monitoring Events to build a support tool for your system.  E.g. the tool could send an email or SMS text message to a support person when a certain event is received (OutOfMemoryError,  External service not responding, etc) or a when certain number of events have been received over a given time frame.
• Miscellaneous tips on testing and coding
• If you are ever testing code that depends on the system clock (e.g. at noon every day the system is supposed to execute some function) then a neat trick to make that code more testable is to refactor out the dependency on the system clock.  E.g. instead of your class making a direct call to say System.currentTimeInMillis(), have your constructor take in a generic Clock object (or define a class variable and use dependency injection to inject a Clock implementation).  Then you can have a SystemClock implementation that simply uses the current system clock, where as during your test run, the class under test is initialized with a FakeClock implementation that hardcodes the time to 12 PM)
• In your system tests, which load up and work with data in a database, have your JUnit setUp method clear out the database, rather than the tearDown method.  This way you have  actual data to look at if a test fails (rather than a clean swiped database).
• Use Simplicators to simplify communication with 3rd party APIs.  That is, a facade that maps the 3rd party interface and artifacts over to your domain model or something that makes sense in your system.  This way you can more easily test your own code (by using mock Simplicators that don't have a dependency on a 3rd party system) and likewise your code is more easily maintained (e.g. a type change or renaming of a field in some 3rd party XML response may require you to update your Simplicator implementation, but might have no impact on your business code if the response has already been mapped over to your domain model.
• Have separate tests that test your production setup.  Basically tests that you can run in production to verify a deployment.  Don't deploy your unit/system tests into production. Avoid the painful lesson that GitHub recently experienced where a test run in production cleared out their entire production database!
• When programming, don't have your methods return null!  Null checks pollute you code and make it harder to debug. Return empty objects instead. 

Agile netið Founded

This week I took part in founding the Agile Network (Agile netið), a consortium of agile minded Icelandic companies, one of which is Hugsmiðjan, my current employer. The purpose of the non-profit organization is to advocate Agile and Lean development practices and sponsor agile lectures and conferences in Iceland. The founding partners are 10 companies that have been doing agile for awhile and are interested in sharing their experiences and learning from each other. At the founding meeting we elected a 3 person board for the organization, of which I got elected secretary :-) Hopefully it wont be too much work :) I am very excited to be part of this process and hope to learn a lot from my fellow members. The website for the organization is www.agilenetid.is. No English version yet, but we are working on it. Then there is a Facebook group for those interested in keeping up with what were are doing. Fans wanted :)

Join MP3 files with a UNIX shell command

I have been listening to a lot of audio books lately, which I have typically imported from CDs onto my iPhone via iTunes. Most of the Icelandic audio CDs I have listened to are in MP3 format and described in the SMIL markup language, which iTunes has no concept of. The files are named 01.mp3, 02.mp3, ... and so on and are void of any ID3 information, with all the metadata in those .smil files. Anywho, to my frustration once imported into iTunes, neither iTunes nor my iPhone have the good sense of playing the files in the correct order. Why they play them in some arbitrary order other than by file name, I have no idea, and iTunes does not seem to have any way of changing the play order. So I figured I needed some way to join the files into a single MP3 file so iTunes/iPhone couldn't mess up the order. After googling around for products to join MP3 files and mainly finding shareware software that I wasn't willing to pay for, I stumbled up on the fact that since MP3 is a streaming format you can simply concatenate MP3 files using the good old Unix cat command! So I came up with the following one-liner to join the files together (in alphabetic order), which happily runs on my Mac:

find /Volumes/My\ Disc/*.mp3 -exec cat {} >> MyAudioBook.mp3 \;

Where "/Volumes/My\ Disc/" is the Mac CD drive, and MyAudioBook.mp3 is the final output file that can be imported into iTunes.
Anyways, I figured I'd share this one, as I have to admit I was surprised to learn that MP3 files could simply be "cat"ed together :-)