Abstract

Update 2020-06-16: Thanks to Jordan Okene, this post and associated github repo were updated to LTS-16 and webdriver-0.9.

As a software engineer I have heard a lot about test automation and even about Selenium. But never tried it. Since I have an outstanding activity with web scrapping I tried instead of emulating web browsers to launch them in the automatic way. And Selenium should be the best choice for it.

What is it? You have the web browser that supports Selenium via webdriver implementation as interface for Selenium. You have a Selenium server. And you have your client test automation app with necessary scenarios inside.

How it works?

Your client test automation app sends requests that are complaint with Selenium server to it.
Selenium server handles these requests and invoke web browser through webdriver and receives response from browser.
Selenium server transfers response to your client app as a proxy middleware.

And that’s actually it. With Haskell’s REPL ghci I can show you how to build test automation scenarios interactively and ship them into single test case. Issues/questions raised on webdriver package github page motivated me to write this blogpost.

Installation

Prerequisites

stack, selenium standalone server, chromedriver.

stack is a tool that will help you to avoid all possible complexity with Haskell initial setup if you are not familiar with Haskell. You can obtain it from its download page.

selenium standalone server could be downloaded from official site.

chromedriver could be obtained from here.

Once you followed all instructions, set up ${JAVA_HOME} (or %JAVA_HOME% in Windows) environmental variable, put chromedriver and stack executables somewhere visible, i.e. in $PATH, you can enjoy playing with it. We will use customized webdriver haskell package (see Cookies section on this page for reasons of using custom code instead of published).

Configuration

$ stack new selenium-example

Modify selenium-example/stack.yaml configuration file by modifying extra-deps section:

extra-deps:
- webdriver-0.9.0.1

If in your project directory you do have package.yaml file then add a dependency right there:

dependencies:
- base >= 4.7 && < 5
- webdriver <= 0.9.0.1
- text  <= 1.2.3.0

Otherwise, modify selenium-example.cabal file library/build-depends subsection:

  build-depends:       base >=4.7 && <5
                     , webdriver <= 0.9.0.1
                     , text <= 1.2.3.0

Launch following command and wait a couple minutes to finish.

$ stack build

Initial browser setup

By default you have a couple of files in your project dir. What you need is to replace Main module app/Main.hs with following content:

module Main where
    
import Lib
import Data.Text
import Test.WebDriver 
import Test.WebDriver.Session
import Test.WebDriver.Commands
import Test.WebDriver.JSON (ignoreReturn)

chromeConfig :: WDConfig
chromeConfig = useBrowser chr defaultConfig
  { wdHost = "0.0.0.0", wdPort = 4444, wdHTTPRetryCount = 50 }
  where chr = chrome

main :: IO ()
main = someFunc

Now we are ready to go!

Launching

Step 1. Start selenium standalone server in console by running: java -jar ~/Downloads/selenium-server-standalone-x.x.x.jar where x.x.x is a version of downloaded selenium server. You can verify it by opening http://localhost:4444/ in your browser.

Step 2. Launch your client from console:

$ stack repl
The following GHC options are incompatible with GHCi...
Configuring GHCi with the following packages: selenium-example
Using main module: 1. Package `selenium-example' component ...
GHCi, version 8.4.3: http://www.haskell.org/ghc/  :? for help
[1 of 2] Compiling Lib              ( .../src/Lib.hs, interpreted )
[2 of 2] Compiling Main             ( .../app/Main.hs, interpreted )
Ok, two modules loaded.
Loaded GHCi configuration from ...
*Main Lib>

Ok, we are ready to go further.

Webdriver session

Session in terms of Selenium is the test scenario with particular browser instance. Each session triggers new browser. By default all sessions are available in your browser by the http://localhost:4444.

Step 3. Launch your web browser interactively in your client opened in Step 1:

*Main Lib> session <- runSession chromeConfig $ getSession
*Main Lib>

We started new session with configured chrome browser. Now we are going to interact with web browser by session that we received by getSession.

*Main Lib> runWD session (openPage "http://localhost:4444")
*Main Lib>

We passed our session and webdriver action in brackets (opening selenium start page) into webdriver context using runWD.

Elements Lookup

*Main Lib> runWD session (openPage "https://google.com")
*Main Lib> :t findElem
findElem
  :: Test.WebDriver.Class.WebDriver wd => Selector -> wd Element
*Main Lib> :i Selector 
data Selector
  = ById text-1.2.3.0:Data.Text.Internal.Text
  | ByName text-1.2.3.0:Data.Text.Internal.Text
  | ByClass text-1.2.3.0:Data.Text.Internal.Text
  | ByTag text-1.2.3.0:Data.Text.Internal.Text
  | ByLinkText text-1.2.3.0:Data.Text.Internal.Text
  | ByPartialLinkText text-1.2.3.0:Data.Text.Internal.Text
  | ByCSS text-1.2.3.0:Data.Text.Internal.Text
  | ByXPath text-1.2.3.0:Data.Text.Internal.Text
  	-- Defined in ‘Test.WebDriver.Commands’
instance Eq Selector -- Defined in ‘Test.WebDriver.Commands’
instance Ord Selector -- Defined in ‘Test.WebDriver.Commands’
instance Show Selector -- Defined in ‘Test.WebDriver.Commands’
*Main Lib> searchInput <- runWD session (findElem (ById "lst-ib"))

<interactive>:6:46: error:
    • Couldn't match expected type ‘text-1.2.3.0:Data.Text.Internal.Text’
                  with actual type ‘[Char]’
    • In the first argument of ‘ById’, namely ‘"lst-ib"’
      In the first argument of ‘findElem’, namely ‘(ById "lst-ib")’
      In the second argument of ‘runWD’, namely
        ‘(findElem (ById "lst-ib"))’
*Main Lib> :set -XOverloadedStrings
*Main Lib> searchInput <- runWD session (findElem (ById "lst-ib"))
*Main Lib>

Interaction with page

Sending text to input is very easy with sendKeys. Check the type of sendKeys and send something to google.

*Main Lib> :t sendKeys 
sendKeys
  :: Test.WebDriver.Class.WebDriver wd =>
     text-1.2.3.0:Data.Text.Internal.Text -> Element -> wd ()
*Main Lib> runWD session (sendKeys "Glasgow\n" searchInput)

Let’s find Wikipedia on result page.

*Main Lib> :t findElems
findElems
  :: Test.WebDriver.Class.WebDriver wd => Selector -> wd [Element]
*Main Lib> links <- runWD session (findElems (ByClass "r"))
*Main Lib> :t links
links :: [Element]
*Main Lib> :t getText 
getText
  :: Test.WebDriver.Class.WebDriver wd =>
     Element -> wd text-1.2.3.0:Data.Text.Internal.Text
*Main Lib> linkTexts <- runWD session (mapM getText links)
*Main Lib> :t linkTexts 
linkTexts :: [text-1.2.3.0:Data.Text.Internal.Text]
*Main Lib> print linkTexts
["Glasgow - Wikipedia","People Make Glasgow | Official Guide to the
City of Glasgow","First Time Must Sees in Glasgow | People Make
Glasgow","University of Glasgow - homepage","Glasgow 2018: Best of
Glasgow, Scotland Tourism - TripAdvisor","Things to do in
Glasgow","Glasgow travel - Lonely Planet","Glasgow City
Council","Glasgow - Holidays, City & Weekend Breaks | VisitScotland"]
*Main Lib>

Custom JS injection

*Main Lib> runWD session (ignoreReturn $ executeJS [] "alert('custom alert');")
*Main Lib>

Cookies

*Main Lib> myCookies <- runWD session (cookies)
*Main Lib> :t myCookies 
myCookies :: [Cookie]
*Main Lib> length myCookies 
5
*Main Lib> print myCookies 
[Cookie {cookName = "1P_JAR", cookValue = "2018-8-3-10", cookPath =
Just "/", cookDomain = Just ".google.com", cookSecure = Just False,
cookExpiry = Just 1.535885639e9},Cookie {cookName = "UULE", cookValue
=
"a+cm9sZToxIHByb2R1Y2VyOjEyIHByb3ZlbmFuY2U6NiB0aW1lc3RhbXA6MTUzMzI==",
cookPath = Just "/", cookDomain = Just "www.google.com", cookSecure =
Just False, cookExpiry = Just 1.533380041e9},Cookie {cookName = "NID",
cookValue =
"135=e6Jaak3KFazKszUmf60YNU45V_rp_Zbzw3t6zSvHWNzvuDHmN5d_Vf9iNGfpT",
cookPath = Just "/", cookDomain = Just ".google.com", cookSecure =
Just False, cookExpiry = Just 1.548845639106112e9},Cookie {cookName =
"DV", cookValue = "4xCQLaMEo_YeoPugrFVU4MaNxMP2TxY", cookPath = Just
"/", cookDomain = Just "www.google.com", cookSecure = Just False,
cookExpiry = Just 1.533294239e9},Cookie {cookName = "CGIC", cookValue
=
"IlV0ZXh0L2h0bWwsYXBwbGljYXRpb24veGh0bWwreG1sLGFwcGxpY2F0aW9uL3htb",
cookPath = Just "/search", cookDomain = Just ".google.com", cookSecure
= Just False, cookExpiry = Just 1.549072114233595e9}]
*Main Lib>

Let’s review expiration date of cookie. It’s Just a unix timestamp.

Closing session

*Main Lib> runWD session closeSession

Combine it together

Interactive mode provided by stack repl is better choice to create your automation tasks on flight. But how to combine them altogether to execute later? In ghci you have a history so you need to move it somehow and combine in single task. Moreover, you can avoid multiple repeatition of runWD session ... by using do-notation since WD () is a monad.

Replace following code with main function in your test application:

main :: IO ()
main = do
  result <- runSession chromeConfig googleIt
  print result

googleIt :: WD [Text]
googleIt = do
  openPage "https://google.com"
  searchInput <- findElem (ById "lst-ib")
  sendKeys "Glasgow\n" searchInput
  links <- findElems (ByClass "r")
  linkTexts <- mapM getText links
  closeSession
  return linkTexts

We hide session inside googleIt function that should return list of results. Reload our repl and let’s try again.

*Main Lib> :r
Ok, two modules loaded.
*Main Lib> main
["Glasgow - Wikipedia","People Make Glasgow | Official Guide to the
City of Glasgow","First Time Must Sees in Glasgow | People Make
Glasgow","University of Glasgow - homepage","Glasgow 2018: Best of
Glasgow, Scotland Tourism - TripAdvisor","Things to do in
Glasgow","Glasgow travel - Lonely Planet","Glasgow City
Council","Glasgow - Holidays, City & Weekend Breaks | VisitScotland"]

Trying different browsers

You can actually run chrome in headless mode by simple adding appropriate flags or even switching to chromium:

chromiumConfig :: WDConfig
chromiumConfig =
  useBrowser chr defaultConfig
    { wdHost = "0.0.0.0", wdPort = 4444, wdHTTPRetryCount = 50 }
  where chr = chrome
              { chromeBinary = Just "/usr/bin/chromium"
              , chromeOptions = ["--headless"
                                , "--mute-audio"
                                , "--disable-gpu"
                                , "--no-sandbox"
                                ]
              }

Using selenium-standalone-3.7.1.jar you can try to launch PhantomJS browser:

phantomConfig = useBrowser phantom defaultConfig
  where phantom =
    Phantomjs
      { phantomjsBinary = Just "/opt/phantomjs/bin/phantomjs"
      , phantomjsOptions =
        [ "/opt/phantomjs/src/ghostdriver/main.js"
        , "--webdriver=8910"
        , "--webdriver-selenium-grid-hub=http://127.0.0.1:4444"
        , "--webdriver-logfile=/var/log/phantomjs/webdriver.log"
        ]
      }

And one more headless browser:

htmlUnitConfig = useBrowser HTMLUnit defaultConfig

Actually PhantomJS is currently abandoned. And HtmlUnit cannot interact with React.js. However, maybe you will be lucky.

Thank you

Example is fully available on my github.

In case of questions or comments please contact me on Twitter: @marunarh.

Acknowledgments

Jordan Okene for update to

Posted on 2018-08-03 by agr . Powered by Hakyll. Inspired by Yann Esposito.

Andrey Prokopenko's Blog

Test automation with Haskell