beyond selenium ide

Wartbare Alternativen zu "record and playback"

Stefan Hildebrandt / @hildebrandttk

web application test automation

  • required for fast delivery
  • is not hip
  • complex environment
  • browser automation is error-prone
  • browsers are slow
    • slow tests are expensive
    • have to create a high value
  • required for fast delivery

record and playback

Some Tools

  • Selenium IDE (Firefox)
  • Microsoft Test Manager/Visual Studio 2015
  • Sahi PRO
  • TestingWhiz
  • Rapise
  • Ranorex
  • HP Unified Functional Testing (Quick Test Professional)
  • ...

common pros

  • without programming
  • quick test creation
  • simple
  • can be done by everyone
    • utilisation of current testing stuff

Bullshit !!!

common pros

  • without programming
    • visual programming
  • quick test creation
    • permanent fixing of redundant test parts
  • simple
    • flaky tests
  • can be done by everyone
    • utilisation of current testing stuff
    • but don't get the job done

common flaws

  • automatic anchor selection
  • code duplications
    • navigation
    • complex elements
    • ...
  • test scripts not human editable
    • recapture instead of change
      • no regression tests
    • for some tools: very limited version control capabilities
  • capture tool on desktop PCs
  • test execution on desktop PCs

conclusions

  • high maintenance costs
  • no regression tests after structural changes
    • did not help when required
  • limited value
  • cite from hp website not promoting UFT/QTP
    • “The applications I am testing are rapidly changing. I need a way to quickly develop robust and stable tests.”

modified approaches

metamodel

  • model of the website elements with virtual names
  • create tests on virtual model

navigation on text and pattern matching on visible elements

selenium

selenium

features

selenium

example from docs

public static void main(String[] args) {
   // Create a new instance of the Firefox driver
   // Notice that the remainder of the code relies on the interface,
   // not the implementation.
   WebDriver driver = new FirefoxDriver();

   // And now use this to visit Google
   driver.get("http://www.google.com");
   // Alternatively the same thing can be done like this
   // driver.navigate().to("http://www.google.com");

   // Find the text input element by its name
   WebElement element = driver.findElement(By.name("q"));

   // Enter something to search for
   element.sendKeys("Cheese!");

   // Now submit the form. WebDriver will find the form for us from the element
   element.submit();

   // Check the title of the page
   System.out.println("Page title is: " + driver.getTitle());

   // Google's search is rendered dynamically with JavaScript.
   // Wait for the page to load, timeout after 10 seconds
   (new WebDriverWait(driver, 10)).until(new ExpectedCondition<Boolean>() {
      public Boolean apply(WebDriver d) {
         return d.getTitle().toLowerCase().startsWith("cheese!");
      }
   });

   // Should see: "cheese! - Google Search"
   System.out.println("Page title is: " + driver.getTitle());

   //Close the browser
   driver.quit();
}
selenium

Problems

  • spaghetti code
  • bad identifiers
  • no possibility of reuse of
    • identities
    • navigation/access logic
  • but: typical result
selenium

common problem:

how find existing code

selenium

solution

  • conventions
  • maps
  • how to get one for all?

page object pattern

  • use page layout of application for code structure
page object pattern

mapping

page object pattern

Pages

  • every logical page is an class
    • hierarchies are possible
  • fields for page elements
page object pattern

actions

  • methods for actions
  • unit testing: fine grained
    • fillA
    • clickB
    • getC
  • functional testing:
    • searchFor
    • getResultByIndex
page object pattern

Navigation

  • by method return types
    • goToLoginPage(): LoginPage
    • searchFor(query:String): ResultPage
page object pattern

fluent api

  • every method returns type for next action
page object pattern

assertions

  • as method of a page
  • self describing name
page object pattern

return values

  • java lamdas are limited
    • value objects
    • groovy closures
    • other jvm languages
page object pattern

closures

  • Method on page
    GoogleResultsPage doSomething(Closure closure) {      .
       closure.call(this)
       this
    }
  • call
    List<GoogeResultModule> results;
    googleResultsPage
      .doSomething (
        { GoogleResultsPage page ->
          results = page.listResults()
    })
      .goToNextPage()
page object pattern

ajax and async updates

  • a mass of complexity
  • hide all async calls in page objects
page object pattern

facade pattern

  • build an own interface-facade to remote system
  • domain specific language over html interface
page object pattern

smaller parts

  • repeating elements
  • js enhanced html (jQuery UI, Kendo UI, extjs, ...)
  • complex web frameworks (angular/gwt/vadin/...)
  • complex jsf components (richfaces/primefaces/butterfaces)
  • selenium way:
    • UtilClasses

Modules / Fragments

  • like pages
  • with anchor element like WebElements
page object pattern

jvm implementations

page object pattern

selenium page objects

  • basic implementation
    • many factory calls required
  • finders by annotations
  • no abstraction of webdriver api
  • no support for sub elements
page object pattern

arquillian graphene

  • page and fragment support
  • finders by annotations
  • by default no abstraction of webdriver
  • fragments for RichFaces (and others?)
  • Waiting API
  • simplified screenshots, ...
page object pattern

webTester

  • page and fragment support
  • finders by annotations
  • waiting- and state-api by annotations
  • wrapper for html-Elements
  • simplified screenshots, ...
page object pattern

geb

  • groovy
    • no fear, it's just like java
    • very powerfull language for DSLs
  • page and module support
  • modern api layer around webdriver
  • jQuery like $-function
  • waiting support
  • simplified screenshots, ...

example

alternative concepts

CatJS

  • test code embedded in html source
  • js runner in page
    • both removed on minification for production
  • runs in any html5 browser
  • node.js based trigger available
  • example

common problems

long run duration

  • results
    • delayed feedback
    • time consuming test development
    • time consuming error fixing
    • less feedback
    • red tests
    • valueless tests

strategies

  • test on lowest level
  • replace slow systems as much as possible
  • isolate test cases
  • optimise setup
    • performance
    • maintenance
    • reliability

select the level

  • system test on service- or ui-layer?
  • ui-test requires real backend?
  • spread required test cases over all layers
  • example:
    • calculations on unit level
    • parameter and exception handling on service level
    • ui input and visualisation on isolated ui level
    • one full integration test for communication

mocks, simulators, ...

  • faster
  • reliable
  • easy way to create edge cases
  • tools:
    • test modules with @Alternative implementations (spring/cdi)
    • modify deployments with arquillian
    • simulator containers with docker
    • service interaction frameworks like citrus
    • "consumer driven contract tests"
      • automated verification of consumer and provider by pact

isolated test pages

  • create test pages for your ui-components
    • faster backend
    • single point of failure
  • better test coverage
  • may use different toolset
  • fast feedback
    • arquillian
    • test module

pre defined test systems

  • single stage for every run of a test suite
    • parallel runs
  • automate system setup
  • use containers, vms, ...
    • automated data migration

duplications

  • developer driven tests on code base
    • unit tests
    • integration tests
  • QA-division driven automated tests
    • interface tests
    • ui tests
    • acceptance tests
    • performance tests
    • penetration tests
  • QA-division driven manuel tests
  • anti-pattern called: software testing cupcake anti pattern by Fabio Pereira

strategies

  • co-working / pairing
  • combine measurements
  • agree on metrics
  • align targets
  • build a room of trust
  • trust!!!
short note

visual comparison

Slides

http://h9t.eu/s/bsi

Stefan Hildebrandt - consulting.hildebrandt.tk

  • Beratung, Coaching und Projektunterstützung
  • Java EE
  • Buildsysteme gradle und maven/ant-Migration
  • Testautomatisierung
  • Coach in agilen Projekten
  • DevOps
Datenschutz Impressum