Thursday, March 7, 2013

Robot Framework testing hints

Updates:
2013-03-15 Remove external content loaded by javascripts
2013-03-14 Be careful with Element Should Not Be Visible
2013-03-13 Using Chrome driver in RF tests
2013-03-12 Notes about Plone-specificity

Some things that come in handy for Plone developers working on Robot Framework tests:

Tags

RF supports tags. Add a line [Tags] tag1 tag2:
*** Test cases ***

Scenario: Clicking the submit button hides it
  Given i am logged in
    and i am on an article
   When i simulate clicking the comment submit button
   Then the submit button has class disabled

Scenario: Submitting a comment displays it in the page
[Tags] working_on_it
  Given i am logged in
    and i am on an article
   When i type something in the comment box
    and i click the comment submit button
   Then the page shows the comment
You can now run only the latter test: ./bin/test -m der.freitag -t working_on_it (This is Plone-specific. See Asko's comment below.)

"Typing" in TinyMCE

TinyMCE is in an iframe, get to it with Select Frame:
i type something in the comment box
    Select frame  form-widgets-comment-text_ifr
    Input text  tinymce  ${COMMENT}
    Unselect Frame
More useful commands and selectors at http://rtomac.github.com/robotframework-selenium2library/doc/Selenium2Library.html

Using Chrome driver

In order to also test in Google Chrome, get the chromedriver binary from http://code.google.com/p/chromedriver/downloads/list, install it as per http://code.google.com/p/selenium/wiki/ChromeDrive (short version: put the driver on your $PATH and chmod +x), and run your (Plone) tests (on Linux) with:
$ ROBOT_BROWSER=Chrome ./bin/test -m der.freitag -t wip 

Be careful with Element Should Not Be Visible

"Element should not be visible" will also return True (ie. not fail) if the element is not in the page... so probably you'll first want to check if it's there at all.

Remove external content loaded by javascripts

If your site loads external javascripts like ads, these may inject invalid HTML, or open popups, or just make the page load slower. How they behave is unpredictable, which can make creating tests all but impossible. You'll probably want to disable everything that depends on something from the outside.
To do this in Plone, you may create a Generic Setup profile for RF tests that disables the viewlets that load these external scripts. In our testing.py, we now have:
class DerFreitagAcceptanceLayer(PloneSandboxLayer):

    def setUpPloneSite(self, portal):
        applyProfile(portal, 'der.freitag:default')
        applyProfile(portal, 'der.freitag.tests:testing')
        applyProfile(portal, 'der.freitag.tests:robot')
The der.freitag.tests:robot profile lives in tests/robot/profile, it just has a viewlets.xml.
The tests layer DerFreitagAcceptanceLayer is a separate layer created specifically for RF tests. Actually DerFreitagRobotLayer would have been a better name.

3 comments:

Asko Soukka said...

Hi Kees! Thanks for publishing that TinyMCE tip!

Robot Framework users beyond Plone developers may be confused by bin/test -t ..., which is how we filter tests with zope.testrunner. Pure pybot would, of course, select tests by tags with pybot -i tagname.

(I think, we will be using more pure pybot also with Plone, once I manage to write a document, how to use it with our act_server test server :)

Anonymous said...

Hey that looks great but I'm having some problem with chromedriver.
I'm using ubuntu 13.04 and I checked my $PATH variable It contains nothing I mean I did $echo $PATH and an empty line shownup.

I went through these two links you've provide but doesn't help me.

though I've downloaded chromedriver from this link
http://chromedriver.storage.googleapis.com/index.html
and put it in same directory where my chrome executables are placed (/usr/bin/) also in /usr/lib/
Do I've missed something ?
I mean I'm not sure where to put it and wireup things. Is it occurring because of PATH variable?
plz help!

Kees Hink said...

If your $PATH is empty then that's your problem. Fixing that is beyond the scope of this post. Ask ubuntuforums.org or similar.