More sweetness…

I already blogged about ‘OctopusDeploy &TestComplete sweetness!’ and now I ‘d like to add a little more…

First, since we bought a TestComplete Enterprise license we were able to run TestExecute on multiple machines and thus add more servers that could execute tests.

Second, I could see problems coming our way if we we were to keep our test scripts outside of source control or outside the branch.We would run the risk that we’d change the system in such a way that the tests would no longer be correct. I spend some time today on mitigating that problem. Here’s what I did:

  • I wrapped the TestComplete project inside a class library
  • added a deploy.ps1 that does the magic of starting TextExecute
  • set all files to ‘Content & Do not copy’
  • created a package with _PublishedApplications and Octopack
  • set up a server in the environment in the role ‘TestComplete’
  • added a step to the octopus project to deploy the TestComplete package to that server.

Done!

Seriously, this setup is starting making me happier every day!

BTW: still bothered by the SMS that is send with every test run…

OctopusDeploy &TestComplete sweetness!

Seriously, this is just sw33t! After a deploy our system is being tested with no less than 2 tests! Smile with tongue out 

image

 

Okay, it’s actually a lot sweeter than it sounds. The first test is to check whether our website will return the correct response in case we do not specify an id. This is just a basic test.

The sweetness is in the second test. This test performs a XML post to the API and checks afterwards whether the returned id from the API is collectable on our website. And apparently it is! This test tests the API, the windows service, and the website. That is all there is for our system.

Now on to check whether the SMS that we send arrives…

Running Testcomplete scripts with Octopus Deploy

It was becoming more and more clear that just running a deployment without testing that the deployment has gone as planned, and yesterday we had a deployment gone bad…Luckily it was on the test environment and not on production.

So my conclusion was that we have to do some automated system or UI testing apart from the unit & scenario tests we run in the build. We have just bought a license for Testcomplete (http://smartbear.com/products/qa-tools/automated-testing-tools) to automate our tests, so it would be sweet if we could run our tests right after a deployment.

I took a relatively simple test to try to implement in the deployment process: a simple text-check on a webpage. First I had to find out how to start testcomplete from the command line and specify a test. Refer to the documentation for this, because it is well specified how to do this. The step from command line to powershell is a breeze after that.

There is however a problem: Testcomplete doesn’t just return success or failure, it generates logfiles. So we are stuck with the process of extracting the results from the logfile. I run TestComplete in silent mode and export the log to an mht file, which is a multipart mime file. The script unpacks this file, finds the root.xml file amongst all the parts (decodes base64 format) and gets the status code from the xml. You can look in this script to see the details.

After this worked locally I installed a Tentacle on the testcomplete server and tried to start kick off the powershell script with Octopus. It started, but the test result failed because I could not navigate to the website in my script. This is because of the fact that the Tentacle runs as local system and is thus bound to the system. I used an AD user as the service account for Tentacle and it worked as charm.

Happy deploying!

(PS: There might be easier ways to get the results from TestComplete or to parse a multipart mime document. If you know, please tell me, because this is not the most elegant way…)