Agile Tips from Remote Seminar with Henrik Kniberg

Last week Agile netið organized a remote seminar with Henrik Kniberg, award-winning keynote speaker and author of multiple Agile and Lean books.  The seminar was held at the Grand Hotel in Reykjavík, where around 10 Icelandic Agile enthusiasts gathered to remote connect to Henrik via Skype and Google Hangout.  We listened to Henrik talk about lessons learned at one of his recent projects and then had a long Q&A session, where we got to ask him all sorts of Agile and Lean related questions.  Despite an initial technical hiccup with poor sound quality the event was very enjoyable and informative.  Here are some of my take-aways. Basically random tips from the discussions that I found useful.

• If you need to explain to someone the benefits of limiting work in progress when running Kanban, use a traffic jam analogy: When there are not that many cars on a given highway, everything runs smoothly and we have a lot of cars running through the highway. But as we add more cars on the highway, basically everything starts slowing down to a jam and we have fewer and fewer cars that are able to drive the highway to the end.  The same thing happens in Kanban if we don't put WIP limits on our queues. Starting more tasks actually causes us to end up finishing fewer.

• If you ever need to sell to management the need to pay off a technical debt and refactor some legacy code: print out the longest class, tape the pages together and bring it with you to a meeting!  When you show up with a 5 meter long paper strip, showing some monstrosity of a class, people have an easier time visualizing the problem.
• When running a Scrum team, think about establishing a rule that each team member that completes a user story needs to ask the team if he can help out with a story that is already in progress before he is allowed to start work on a new one.  That encourages cooperation and limits work in progress, increasing the likelihood that we have fully completed stories at the end of the sprint, as opposed to stories that are partially done.  If our sprint backlog is 10 stories we would much rather have 8 fully completed stories at the end of the sprint than 6 completed stories and 4 partially done.
• According to Henrik, research has shown that incorporating QA into your agile development teams (testing continuously) saves on overall time spent in testing and bug fixing.  Meaning that if you test and bug fix as part of each sprint, as opposed to doing a large round of testing at the end of the project followed by a lot of bug fixes, you save time.  One reason for this is that when it comes time to fix a bug the code is still fresh in the developer's mind.  Also the continuous QA feedback encourages developers to deliver higher quality code.  This is visualized in the following illustration from Henrik's slides:

• Size doesn't matter.  According to Henrik having the team categorize user stories into small, medium and large before implementation is not very useful.  At the end of his project he calculated actual times spent on user stories and found out that it actually took about the same time on average to implement a story that had been categorized as small as it took to implement a story that had been categorized as medium.
• When you are on a large project that comprises multiple teams, try doing finger voting to assess the overall confidence that the project will get delivered on time.   Basically, in weekly meetings ask team leads to hold up fingers on one hand representing their confidence that their team will get the required work done on time.  Where 5 fingers mean "definitely", 4 fingers mean "likely", 3 fingers "it's a gray line", two fingers "probably not" and one finger means "no way".  Track this on a weekly basis.  If you get few fingers in the air, you know you have a problem that needs to be addressed.
• Try to make your development teams cross functional.  Each team should comprise all the skills required to fully deliver a feature (e.g. graphics designer, front-end developer, back-end developer, DBA, tester).  If a resource is needed at least 50% of the time put him on the team.  Make sure everyone has a home team, even though they might be outsourced to other teams from time to time.
• To check the health of a Scrum team use the Scrum check-list.

Ský Development Conference (Links in Icelandic)

Last week I had the pleasure of presenting at the annual Ský Development Conference (Hugbúnaðarrástefna Ský) in Reykjavík, Iceland.  This year's conference theme was "The Work Environment of Software Developers".   The conference offered presentations on the following topics: Programming Languages, Quality Assurance, Issue Tracking, User Interfaces, Project Management, Databases and Data Warehousing.  My presentation was under the Project Management topic, titled "Practical Tips for Software Development Leads".  In it I tried to provide some practical tips to team leads and software development managers.  Suggestions on such things as how to get the most out of your team, how to manage your time, how to use programming to help with your managerial duties, how to achieve continuous improvement and how to motivate developers.  The conference was all in Icleandic, so unfortunately so are the slides.  They are available on the conference web site and on Slideshare (mine).

I especially liked Ólafur Gauti Guðmundsson's presentation on the Play framework, which I have been playing with a bit and Stefán Baxter's presentation on NoSQL databases.  Margrét Dóra also had a great talk on user interfaces, reminding us that "stupid users" are really our own fault, a result of poor user interface design.  And Birna Íris Jónsdóttir had a good QA talk titled "It works on my machine!" where she took some shots at us programmers for all the lame excuses we provide when our stuff breaks! :)  All in all a fun day and an enjoyable conference.

Quickly Launch Remote Desktop Connections on Mac OS X

At work and at home I have a few PC servers that I frequently need to remote connect to. My Mac launcher application of choice is Alfred. I wanted to be able to use it to instantly launch connections to those servers without needing to type in anything but the first few letters of the server name. Getting this setup on Mac OS X is a bit of a hassle, but here is basically what I do:

• I use the Remote Desktop Connection Client for Mac to connect to the PC servers.
• After installing the client and typing in the name of the server I want to connect to I go to preferences and customize the connection:

• Under the Login tab I enter account information
• Under the Display tab I specify my preferred remote desktop size
• Sometimes I tweak some of the other settings, but those two are of main importance

• Then I go to File and Save As... and save this connection as a .rdp file on my Mac
• Now I can double-click on the file to be instantly connected to the remote server
• Problem is that application launchers don't really know how to run .rdp files, so I want to take it a step further and create a runnable Mac Application for each .rdp file.  To do this I use an open source program called Platypus, that a coworker of mine told me about.  
• Unfortunately Platypus does not know how to run .rdp files, but it does know how create Applications that launch shell scripts.  Hence, I create a simple shell script to launch my remote desktop connection.  E.g. if I want to launch SteiniServer.rdp, then my script would look like this:
  
  open "/Applications/Remote Desktop Connection.app" ~/Documents/RDC/SteiniServer.rdp
  
  
• Then I save the script on my computer in a file called rdcSteiniServer.sh
• Now I open Platypus and use it to create a Mac Application with the following settings:

(If you want to add a custom icon for your application you can right-click the image and choose Select Image File...)
        

The only thing that really matters is selecting Script Type Shell and the path to your script.

• Then I hit Create and select:

• Now I am ready to launch my remote connection with Alfred, which I do like this:

• Basically, I activate Alfred (Ctrl-A for me) and as soon as I have typed in RDC it finds the RDC Steini Server Application and when I hit enter I am automatically logged into the remote PC server without having to type in anything else.

If you have a simpler way of achieving the same I am eager to hear :-)

Web Service Testing with soapUI

For a few years now I have used the open source application soapUI to simplify Web Service testing.  Both to test services that I have written and to test external services that I have had to consume.  In this post I'll cover the basic types of testing you can do with soapUI and provide some practical tips on how to use soapUI when working with Web Services.  If you don't want to go through the detailed examples I suggest jumping straight to the Practical Tips section at the end.

Getting Started

After downloading and installing soapUI the easiest way to get started is to create a new project from an initial WSDL or WADL.  For demonstration purposes I'll use the free Weather Web Service at http://www.webservicex.net/globalweather.asmx.  We will create a test for each operation in this SOAP Web Service.  We will also create a Test Suite, that allows us to run all our tests at the click of a button.  Furthermore, we will create a Load Test for our Test Suite and a Mock Service to simulate the functionality of the Web Service.

To do this go to File and select New soapUI Project.  Populate the New soapUI Project screen in the following manner and click OK:

Basically, paste the URL to the web service WSDL file into the Initial WSDL/WADL field and additionally check the field to generate a MockService.  SoapUI will now load the Web Service definition and ask you questions about how to create the MockService:

For this demonstration change the path to "/WeatherMock" and check the Starts the MockService immediately checkbox, but otherwise accept the defaults.  After clicking OK, specify the name of the Mock Service as "Weather MockService" (or a name of your choice) and click OK again. 

This will complete generating artifacts for the SOAP 1.2 version of the GlobalWeather Web Service.
Next you will be asked the exact same questions, for the SOAP 1.1 version.  Since we don't plan to use SOAP 1.1 you can just hit Cancel.  (I haven't found a way to configure soapUI to only generate artifacts for SOAP 1.2.  If you know how to, please let me know! :)

Running Tests

To execute a given test, drill down to the requests that soapUI auto-generated and insert values that make sense for the given Web Service operation.  In our case, in the Navigator on the left, click on GlobalWeatherSoap12, then GetCitiesByCountry and then double-click on Request 1.
We'll set the Country name as Iceland and then hit the green arrow button to execute the test.

Voila, you should get results back from the Web Service, a list of Icelandic cities.  To check if the Web Service response is valid (conforms to the WSDL), right-click in the results window and click Validate.

Lets give this test the name "City Test".  (Right-click Request 1 and select rename) 
Then do a similar test for the GetWeather operation.  Click on GetWeather and then double-click on Request 1. Put in "Reykjavik" for CityName and "Iceland" for CountryName and run the test.  Also rename the test to "Weather Test".  After running the test, the results should look something like this:

Build a TestSuite

Now that we have created two tests, lets add them to a TestSuite so we can easily re-run them at any time.  In the Navigator on the left, right-click GlobalWeatherSoap12 and select Generate TestSuite.  Select Use existing Requests in interface, and check Generates a default LoadTest for each created TestCase and Single TestCase with one Request for each Operation:  

After clicking OK give the TestSuite the name "Weather TestSuite".   Double-click on GlobalWeatherSoap12 TestSuite under Weather TestSuite to show the TestCase editor.
Then click the green arrow button to run the TestSuite.  If all goes well you should be presented with a results-screen like this, indicating a successful test run:

If the TestSuite run fails you'll see FAILED at the top instead of FINISHED and you should get a message explaining why a given test failed.

Add Assertions

To make the tests more meaningful lets add some assertions to validate the responses from the Web Service we are calling.  You do this by opening up the test Request Editor (double-click on a given test step under TestSuite in the Navigator) and then click the plus-sign next to the green arrow button (second from the left).

For both tests lets add the assertion called SOAP Response, to ensure the Web Service is returning a valid SOAP response.  Then add the assertion called Response SLA and specify the response time as 2000 ms.  That basically means that we are going to consider the test a failure if we don't get a response within two seconds.  Lastly, lets add some content validation by selecting a Contains validation.  For the GetCitiesByCountry operation add the string "Iceland" as the content to expect in the response, and for the GetWeather operation add the string "Success". 

Then run the TestSuite again to make sure you get success-results.

Run a Load Test

When we created the TestSuite in last step, we told soapUI to generate a Load Test as well.  You'll find it under the TestSuite in the Navigator on the left, under the heading Load Test.  The default name is LoadTest 1.  Double click on it in the Navigator to open it up.  Before running the LoadTest you can tweak such parameters as number of concurrent threads to run, length of the test-run, and delay between tests.  Once you have made the desired configurations hit the green arrow button to run the Load Test:

In this sample, 5 concurrent threads are running the TestSuite for 60 seconds with a random wait of up to 1 second between the start of each test.
You can track the progress of the load test with the progress bar in the upper right corner.  If it reaches a 100% without reporting any test errors you are good to go.

Use the Mock Service

Back when we imported the Weather Web Service we told soapUI to generate and start a Mock Service.  That service can now be accessed at http://localhost:8088/WeatherMock.  This is  convenient for example if you are developing against a Web Service that has been designed (WSDL/WADL available) but not yet implemented.  Then you can have the Mock return an actual Web Service response to test your code even though the actual implementation hasn't been completed.  A default response has already been generated (under WeatherMock Service,  GetCitiesByCountry, and Response 1), which you can edit as you like.

You can also have the Mock service return different responses depending on which request it receives.  To demonstrate this, lets create two new MockResponses for the GetWeather operation:

• Click on GetWeather under the Weather MockSerive in the Navigator and select New MockResponse.  
• Give it the name "ReykjavikWeatherResponse".  Accept the automatically generated response, but put the value "Reykjavik" in the GetWeatherResult tag.  (Or even better copy the actual response from calling GetWeather for Reykjavik, which should give you a fully valid response).  
• Create another response called AkureyriWeatherResponse and put the text "Akureyri" in the GetWeatherResult tag.  

Now put logic in the Mock service for when to return each response:

• Double-click on GetWeather in the Navigator to show the MockOperation Editor.  
• Select ReykjavikWeatherResponse under MockResponses.
• Select QUERY_MATCH under Dispatch.
• Click the plus sign to add a new match. 
• Give it the name "Reykjavik". 
• Select Reykjavik and then populate the XPath value with:
  
  declare namespace web='http://www.webserviceX.NET';
  declare namespace soap='http://www.w3.org/2003/05/soap-envelope';
  /soap:Envelope/soap:Body/web:GetWeather/web:CityName
  
  This XPath query will grab the value from the CityName tag.
• Under Expected Value enter "Reykjavik".
• Under Dispatch to, select ReykjavikWeatherResponse.
• Repeat the same steps to create a Match that returns AkureyriWeatherResponse when the city name in the request is Akureyri.

Now you can test your mock Match-logic by opening up your GetWeather test and adding http://localhost:8088/WeatherMock as the endpoint to use (select the current endpoint in the dropdown and pick add new endpoint...).  Then run the test and play around with changing the city name in the request to get different responses from the Mock service.

Additionally, if you want to test your client-side error handling you can have soapUI generate a soap:Fault response and have your Mock return it.  You do that by creating a MockResponse and then clicking the apostrophe icon in the MockResponse Editor.  Then edit the auto generated response as appropriate.

Practical Tips: What to use soapUI for

Now that we have covered the basics of soapUI, here are some practical tips for putting it to use during your software development.

Test Web Services You Have to Consume

When consuming external services, before delving into code, use soapUI to "kick the tires" of the Web Service.  This especially applies when consuming newly written services.  Rather than potentially spending hours pulling your hair over why your client code isn't working, spend a few minutes with soapUI validating and getting familiar with the Web Service you are about to consume. In particular:

• Create and run simple tests for key Web Service operations
• Make sure there are no security/access problems
• Is the web service using some proprietary authentication protocol (NTLMv2 comes to mind) that might give you trouble during implementation?
  
• Have soapUI validate that responses conform to the Web Service contract (WSDL/WADL)
• Visually inspect responses 
• Do they make sense or are they some illegible auto generated garble that should really be cleaned up and restructured by the Web Service developer? 
• Do the responses meet your needs?
• Test a few boundary cases
• Does the Web Service implement proper error handling?  Or does it blow up with an HTTP 500 error or some non SOAP compliant text message?  Does the level of error reporting meet your needs?
• Add all your tests to a TestSuite.  That way you can quickly "ping" the Web Service to make sure everything is working on the other end.  When a problem arises, taking your code out of the loop is a good way to make sure the issue is on the remote end and not with your own code.

When developing against a newly written external Web Service, it is very rare that the Web Service works 100% out of the box as expected.  There is usually a fair amount of communication needed between the Web Service developer and the client-side developer to tweak things until the Web Service works as needed.  By using soapUI, you can inspect the Web Service right at the time it is delivered to you, and quickly spot things that may need to be fixed.  I usually take 5 or 10 minutes to do so and almost always have a list of things that need to be modified.  The response time for getting those changes implemented is usually very short, as the Web Service developer is still engaged in the project and things fresh in his mind.  If I don't get back to him/her until some weeks later when I finally get around to implementing my client-side code, the other developer has probably moved onto other things and/or forgotten why he implemented the service in a certain way.   

Lastly, if you have received a WSDL/WADL file, but the service hasn't actually been implemented, and you NEED to start implementation against the service (not ideal), then consider using soapUI to create a Mock service from the WSDL/WADL file.  That way you can have your code hit the Mock service and at least get some preliminary feedback on whether your client code is working.  Since creating a Mock service is really a breeze with soapUI it can sometimes be more practical than implementing Mock objects in your code.

Test Your Own Web Services

When writing a Web Service for others to consume it can be handy to have a soapUI TestSuite to sanity test your service.  Of course you should still write unit and integration tests for your code, but having a good soapUI TestSuite can be quick and easy way to find out if all your services are running as expected.  When you get that 4 AM phone call saying that something is broke, fire up soapUI and at the click of a button sanity test all of your Web Service operations.  If you are smart, you'll hand the TestSuite over to a support team so that you only get woken when the issue truly is on your end ;-)  Just make sure to add proper assertions for Web Service responses and include SLA assertions to test that things are not running dead slow.

If you are concerned about the performance of your Web Service or whether it can handle a given load, then a soapUI Load Test can be a convenient way to test that.  Set the number of threads to imitate the expected number of concurrent users for your service and add SLA assertions to make sure all requests are handled in a timely fashion.  Of course generating all the load from a single machine does not quite imitate real traffic, so for truer numbers consider having coworkers assist you in running simultaneous Load Tests from multiple machines.

Kanban with David Anderson

This week I enjoyed attending a two day Kanban training class by David Anderson, the Father of Kanban-style software development.  The class provided an overview of Kanban and offered many tips on how to get the most out of Kanban.  David rolled through a deck of some 150 slides and answered various questions.  On the practical side, the class was divided into four teams and each team had to design a Kanban board for an actual company and then play an entertaining Kanban game, which turned out to be quite competitive.  The team I was on, Grand Kanban :) managed to squeeze out a slim victory with around $62,000 in revenue, which according to David was a pretty good score ;-)

Few points from the class:

The Kanban Method is based on three core principles:

•    Start with what you do now
•    Agree to pursue incremental evolutionary change
•    Initially, respect current processes, roles, responsibilities & job titles

The five keys to a successful Kanban implementation are:

•    Visualize Workflow
•    Limit Work-in-Progress
•    Manage Flow
•    Make Process Policies Explicit
•    Improve Collaboratively (using models & scientific method)

Key class takeaways according to Dave were:

• Kanban is like water 
• It goes around obstacles and slowly changes them, rather than removing them like a bulldozer
• Change has to come from within
• How many experts does it take to change an organization?  Answer: One.  But people are going to have to want to change

Surprising fact from the class:

According to a research done by David, getting more done only ranks number 4 on most manager's lists!  The list of their preference being:

1. Predictability
2. Business Agility (ability to respond to changes in market)
3. Good Governments (managing budgets, money spent the way intended, etc.)
4. Getting more done

Playing the Kanban game

All in all a good class and I look forward to continuing the improvement of Kanban at Hugsmiðjan where we develop the Eplica CMS.

Jfokus Conference

This week I had the pleasure of attending the JFokus conference in Stockholm, Sweden.  This was a gathering of some 1250 Java geeks and some excellent speakers.  I thoroughly enjoyed sessions by Arun Gupta, Guillaume Laforge, Mario Fusco, Henrik Kniberg, Neal Ford, and many others.  Here are some of the technologies that caught my attention and I will be considering putting to use at my current work on the Eplica CMS system.

• Lambdaj - This library is just brilliant and really simplifies working with Collections.  Write more readable code in fewer lines.  Stop writing loops!  Mario Fusco's slides.
• CKJM - Use this library to intelligently pick areas of your code base to refactor.  To get bang for your buck you want to pick classes with high Cyclomatic Complexity and high Afferent Coupling.  That is, complex code in frequent use.  Comes as part of a Sonar plugin.  And BTW, if you are not already using Sonar, use it!
• Groovy - I have been intrigued by this language for some time now but never actually put it to use in my work.  I think it is time to change that.  Groovy to me is a natural progression of the Java language.  Making the boring/tedious parts of Java optional and adding syntax improvements and useful features like closures.  The kind of changes you might envision seeing in Java 10 or somewhere way down the road.  Why wait so long?
  As a starting point I am thinking about using these Groovy frameworks for our web based testing:
• Geb - Browser automation framework based on Selenium.
• HTTP Builder - A convenient API for complex HTTP requests.
• WADL & Jersey - When consuming/publishing RESTful web services from/to our customers I prefer to have a contract to work by.  WADL is just that and Jersey an API (JAX-RS 1.1 implementation) for creating/consuming REST services.
• JDK 7 - Obviously eagerly awaiting that one.  Planned GA date July 28, 2011, according to Tomas Nilsson at Oracle.

Other technologies I found interesting but don't envision using in the near future (just not a good fit for Eplica CMS): Scala, Clojure, deploying to the cloud,  NoSQL, Vaadin and DDD.

All in all a great conference which I highly recommend to Java developers.

Photos from the trip

Tweets

Easily Size and Arrange Windows on Mac OS X

One idiosyncrasy of Mac OS X is that application windows can only be sized from their lower right corner, and Mac OS does not have any kind of "snap windows into place" feature similar to Windows 7.  At work I have the pleasure of working on a large 27" iMac screen, so I frequently arrange two apps side by side while working, or might even have more than 2 apps visible on the screen at any given time.   Arranging that kind of setup is tedious when done manually with the mouse, but with a nifty little tool called Divvy the task becomes quite easy.   As can be seen in the Divvy in Action video on their site, when activated, Divvy overlays a grid on your screen and can be used to visually size an application to a specific portion of the screen.  That is great, but what I found faster and easier was to assign global OS X shortcuts to window arranging operations I frequently use and let Divvy do its magic without bringing up the grid panel at all.  Creating shortcuts is covered towards the end of the video, but to make them global you need to check the "Global Shortcut" check box.  Here is a screen shot of some of the configurations I set up:

For example if I need to have an application fill the left 50% of my screen I just hit ctrl-shift-1 (with the app in focus) and it will jump right into place.  While developing, I often arrange things so that my IDE uses the entire right-side of the screen, the website I am testing covers the upper left-side of the screen and the application sever console (logging output) covers the lower left-side.  To put the windows in place I simply put them in focus one at a time and hit ctrl-shift-2, ctrl-shift-3 and ctrl-shift-4, in that order, and the apps snap into place, filling the screen like this:

No space is wasted on the screen and no annoying mouse-dragging operations are needed. Of course you can assign whatever keyboard shortcuts make sense to you.
Divvy is not freeware, but low cost, and can be used in demo mode (periodically pops up a registration message) if you don't want to cough up the $14.  It supports multiple monitors and is available for Windows too.

Evernote - A "shoe box" organizer that works

I have been an iPhone owner since the iPhone 3G was released in the USA on July 11, 2008. Why do I remember the exact day? Because I was one of the nerds that stood in line outside Tuttle Mall, in Columbus, Ohio, way before the AT&T store opened! Anywho, I often get asked what my favorite iPhone app is. Having tried out a fair share of apps from the App Store, I have to say my all time favorite is Evernote.

If you are the type of person that tends to forget things, loose things, and hates to spend a long time organizing information, then this is app is for you! And you don't really need to have an iPhone to use it. It is available for all major smart phones. Plus you can use it directly through www.evernote.com, or via multiple Web Browser addons or use one of the many thick client versions they have for all major operating systems.

If I need to remember anything for later (a store receipt, notes from a lecture, a good deal I see in the paper, a shopping list, recipe from a friend, a note from my kids' school, the name of a good wine bottle,.. you name it) I simply snap a picture of what I want to remember and send it to my Evernote account via the Evernote iPhone App.  If I feel up for it I might tag the note (e.g. Recipe) and add a descriptive title (e.g. "Chili Recipe from Bob").  But often I am to lazy to do that, and it typically doesn't matter.  I just file it and forget it.  What Evernote does when it files my note is to automatically perform OCR on the picture and store any text it extracts as metadata along with the note.  So if Bob showed me a handwritten recipe that he had titled "Bob's kick ass Chili" then that piece of text (granted it is written in legible writing) would automatically be stored in the metadata index for the note.  So if years later I searched my Evernote account for "Chili", or "Bob" AND "Chili" I would find the recipe.

To me, Evernote is the shoe box method for organizing, that actually works!  Meaning, you can get away with throwing all your junk into one large shoe box and thanks to Evernote's powerful search capabilities have it at your fingertip anytime you need it.  Feeling left out at a chili cooking party? Just whip out your cell phone and say "One second, I have a better recipe!" and voila you are the star of the party! ;-)

For more info on Evernote, check out one of their many introductory videos on www.evernote.com.  Then sign up and get an account!  The basic account is free and should be sufficient for most users.