-
Want to receive the latest contracting news and advice straight to your inbox? Sign up to the ContractorUK newsletter here. Every sign up will also be entered into a draw to WIN £100 Amazon vouchers!
Automated behaviour-driven integration testing
- Thread starter d000hg
- Start date
- Thanks received
- 10,020
Selenium is this freaky (but rather cool) thing that does... well, automated behaviour-driven testing. We made extensive use of it a a former client, although I wrongly tried to avoid it for the most part because it was weird
Basically, in the context in which I encountered it (who knows how it may have spread?) it allows you to set up test cases for user interactions which are then automated. So you'll have a test case that specifies that the browser will load a certain page, then the user will give the focus to a certain field and type in a certain value, then they'll click on a certain button, and when they do so a certain thing X should happen resulting in the page (or part thereof) adopting a certain state or the browser navigating to another page which will, once loaded, satisfy certain criteria... and it magically turns a declarative representation of all this user activity (which you can generate by actually performing the interactions yourself the first time, or regenerate later by doing it again, or if you're ubergeeky write yourself from scratch) into a weird mixture of server- and client-side code that goes through the motions (using some very clever JS on the client and Ajax to report things back to the server) such that you finish up with an automated test that can perform its own user-like behaviour on the client, logging it all back to the server, and comparing the outcome with the desired outcome, thereby determining success or failure, which the server then turns into a test report after the manner of automated unit tests
It's seriously
but totally 
once you get your head round it. I wished I'd stayed on that gig a bit longer because I'd finally realised that this was actual magic, but in a good way because all the protocols and such were clearly specified so it wasn't magic really, but it still felt like it 
- Thanks received
- 10,020
Given that my previous post was a bit over-generalised, let's take an example: this isn't an actual Selenium script (as should become apparent), but it's the kind of way they're structured, as I recall. N.B. XPath uses index "1" for the first element of a set rather than index "0" as nature intended. Also, I'm not using the real XPath expressions that would be necessary, just ones that show what I mean.
This script will make the browser go to General, check that it's logged in as me, check that there's a lunch thread on the first page, go to the lunch thread, and finally check that the first post in the lunch thread is from AtW. Note that it will fail miserably (as a test) if "lunch" isn't capitalised as "Lunch"
"Assert" fails if the first argument is false, with the second argument converted to a string (if necessary) and logged as the reason for failure. # starts a comment. Lines are delimited by line-ends (leaving aside obvious line-wrapping in the browser). The management accepts no responsibility for the destruction of your career if you use this for blagging purposes.
Start
# go to CUK forum "General"
Navigate: url(http://forums.contractoruk.com/general/)
# are we logged in?
Assert: xpath(contains(//body//welcome, "NickFitz")), "Not logged in as user NickFitz"
# is there a lunch thread?
Assert: xpath(contains(//body//content//thread//h3//a, "Lunch")), "AtW hasn't been in"
# go to the first lunch thread:
# Note that here, we're telling the browser to click on the link the way a user would
Click: xpath(contains(//body//content//thread//h3//a, "Lunch")[1])
# make sure AtW started the lunch thread, otherwise the thing we're actually testing
# that prevents anybody but AtW starting the lunch threads isn't working
Assert: xpath(contains(//body//content//posts[1]//username, "AtW")), "Non-AtW lunch thread detected"
End
You could then extend this script to, say, click on the "Reply" button, automatically type in "FFS" (yes, automatically in the browser - it will generate a bunch of keypress events in JS to make it seem that someone is typing), click the preview button to preview the reply (with a check that the preview of the reply wasn't missing any text or otherwise corrupted), submit it, make sure that the reply was present on the thread page after it had been submitted, confirm that the reply is being displayed under the correct username... and so on, all automated once you know what ought to happen
This script will make the browser go to General, check that it's logged in as me, check that there's a lunch thread on the first page, go to the lunch thread, and finally check that the first post in the lunch thread is from AtW. Note that it will fail miserably (as a test) if "lunch" isn't capitalised as "Lunch"
"Assert" fails if the first argument is false, with the second argument converted to a string (if necessary) and logged as the reason for failure. # starts a comment. Lines are delimited by line-ends (leaving aside obvious line-wrapping in the browser). The management accepts no responsibility for the destruction of your career if you use this for blagging purposes.
Start
# go to CUK forum "General"
Navigate: url(http://forums.contractoruk.com/general/)
# are we logged in?
Assert: xpath(contains(//body//welcome, "NickFitz")), "Not logged in as user NickFitz"
# is there a lunch thread?
Assert: xpath(contains(//body//content//thread//h3//a, "Lunch")), "AtW hasn't been in"
# go to the first lunch thread:
# Note that here, we're telling the browser to click on the link the way a user would
Click: xpath(contains(//body//content//thread//h3//a, "Lunch")[1])
# make sure AtW started the lunch thread, otherwise the thing we're actually testing
# that prevents anybody but AtW starting the lunch threads isn't working
Assert: xpath(contains(//body//content//posts[1]//username, "AtW")), "Non-AtW lunch thread detected"
End
You could then extend this script to, say, click on the "Reply" button, automatically type in "FFS" (yes, automatically in the browser - it will generate a bunch of keypress events in JS to make it seem that someone is typing), click the preview button to preview the reply (with a check that the preview of the reply wasn't missing any text or otherwise corrupted), submit it, make sure that the reply was present on the thread page after it had been submitted, confirm that the reply is being displayed under the correct username... and so on, all automated once you know what ought to happen
Last edited:
05:00
- Thanks received
- 38
I was about to reply: "that is what the AtW bot is". Then noticed someone beat me too it.
- Thanks received
- 1,499
That looks very similar to the Junit tests that I had created to run using Rational Functional Tester a few years ago. I've seen other tools which do exactly this but instead of you writing the code, they automatically did it for you. Using a client-server set-up and a web based interface, you just entered your data via forms and a test case was automatically generated. You would then say which servers the software was running on and it would execute tha test case against it and display the results once again via a web browser, Quite neat.
- Thanks received
- 34
Selenium is very powerful and free to download so I would suggest getting it yourself and just writing some random tests on a random website to get yourself started.
My top tip for selenium is learn to use xPaths properly, it's surprising how many people know nothing about them and suggest you use CSS locators instead (there are arguments for and against which I won't go into here but most people don't realise just how powerful xPaths can be).
To expand a bit on what nick posted above you can cater for unknown case using xPath using the translate command (There are more commands to deal with it in xPath 2, but most browsers only implement xPath 1):
//a[translate(., 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz')='Lunch']
The above is not an ideal xPath, for a start it's doing a fuzzy match on all anchors on the page which is not ideal (you should really key it into an ID to reduce the amount of elements in the DOM is has to search through) but it should give you an idea. You also don't have to translate the whole alphabet if you know the word you are looking for, but it's easy to have a base one for you to build from.
Anyway back to a more on topic theme.
Behaviour driven testing is basically testing plugged into test driven development. TDD is really a development methodology which (people who don't really know what it means) try to shove testers into. Behaviour driven can be seen as a way of integrating testers into a TDD team. Once the code written by the devs has passed their unit tests they will provide the testers with a functional block of work for the testers to check. The testers will then validate this block using a series of automated tests that they have written.
This process will usually be running on a CI server (like Hudson) so that every time the devs commit new code the tests will automatically run without the devs having to wait for the testers to become available. This frees the testers up to write more test scripts and keep the current scripts up to date and should result in fast code iterations that are well tested every time.
Selenium can be run using jUnit so the syntax will look very familliar to anybody testing in it, I would suggest Selenium and Java as the way forward since Selenium is written primarily in Java and then ported across to other languages. Almost all the new functionality they create will hit the Java code base first.
Bit rambling but hopefully helpful (I'm being hassled by the kids to take them out
)
My top tip for selenium is learn to use xPaths properly, it's surprising how many people know nothing about them and suggest you use CSS locators instead (there are arguments for and against which I won't go into here but most people don't realise just how powerful xPaths can be).
To expand a bit on what nick posted above you can cater for unknown case using xPath using the translate command (There are more commands to deal with it in xPath 2, but most browsers only implement xPath 1):
//a[translate(., 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz')='Lunch']
The above is not an ideal xPath, for a start it's doing a fuzzy match on all anchors on the page which is not ideal (you should really key it into an ID to reduce the amount of elements in the DOM is has to search through) but it should give you an idea. You also don't have to translate the whole alphabet if you know the word you are looking for, but it's easy to have a base one for you to build from.
Anyway back to a more on topic theme.
Behaviour driven testing is basically testing plugged into test driven development. TDD is really a development methodology which (people who don't really know what it means) try to shove testers into. Behaviour driven can be seen as a way of integrating testers into a TDD team. Once the code written by the devs has passed their unit tests they will provide the testers with a functional block of work for the testers to check. The testers will then validate this block using a series of automated tests that they have written.
This process will usually be running on a CI server (like Hudson) so that every time the devs commit new code the tests will automatically run without the devs having to wait for the testers to become available. This frees the testers up to write more test scripts and keep the current scripts up to date and should result in fast code iterations that are well tested every time.
Selenium can be run using jUnit so the syntax will look very familliar to anybody testing in it, I would suggest Selenium and Java as the way forward since Selenium is written primarily in Java and then ported across to other languages. Almost all the new functionality they create will hit the Java code base first.
Bit rambling but hopefully helpful (I'm being hassled by the kids to take them out
)
- Thanks received
- 1,499
- Thanks received
- 678
- Thanks received
- 678
Oh, one salient fact, this is in a Flex-oriented position I saw it mentioned. Do these tools work with Flex, or have they just thrown a bunch of general web-dev technologies into the spec?
EDIT: http://www.adobe.com/devnet/flash/articles/flash_selenium.html
EDIT: http://www.adobe.com/devnet/flash/articles/flash_selenium.html
Last edited:
13:41
- Thanks received
- 34
Selenium 2 was planned to support AAT at some point if I remember correctly, but right now you would have to hack it a bit to get it working. You could always try using something like this:
http://corlan.org/2008/08/15/functional-testing-framework-for-air-ajax-apps-based-on-selenium/
Of course YMMV
http://corlan.org/2008/08/15/functional-testing-framework-for-air-ajax-apps-based-on-selenium/
Of course YMMV
