Assertions

We're now at a point where we can think about actually testing something. We have a test suite (a spec file), with some test groups (using describe()) and the tests themselves (using it()).

In each test we can locate an element on the page using element() and by(). There are two parts to a test. The first part is using a method in the element object to find a value that we want to check. This could be the text content, or a CSS attribute, or a testing framework value. The second part of the test is the assertion. Assertions are a comparison of some sort.

If we're going to run a test then we're going to need a value to make an assertion about. There are two ways to access data. If we want some page metadata, such as the page title, then we can use the browser global object. If we're going to test an element on the page then we need to use the element() method and a locator (using by.css() for example) to return the element from the DOM and then use one of the element's data methods to access the information to test.

Access the page metadata is a simple call to a browser object method.

browser.getTitle();

Accessing a page element's data is slightly more complex but not much.

element(by.css('a#account')).getText();

This code would access the DOM to find an element that matches the CSS selector and then return its textnode data. If we wanted to test one of its CSS attributes we could use getCssValue() instead. We could use getAttribute() to access one of its attributes.

When you have a method of getting data to test you can then you an expect() to perform the actual test itself.

expect(element(by.css('a#account')).getText()).toBe('Account');

In this test we're getting the textnode content for the account link and testing that it's exactly a string that matches "Account". It's essentially asking the page if "value === expectation".

Note: Remember that all of the access functions in Jasmine return promises; you'll always need to use a then() to access the actual value if you're doing more than a simple test. expect() waits for the promise to resolve automatically.

toBe() is about the most straightforward test we can perform, and while it's extremely useful it is rather limited. There are many things to check on a webpage that don't boil down to a simple string test.

The first couple of assertions to look at are toBeTruthy() and toBeFalsy(). These take a piece of data to be tested and cast it to a boolean, then check whether it's true or false. Sometimes that's all we want. What the thing is doesn't always matter. The important thing is that the data is there. Or not there.

element(by.css('input[name="forename"]')).getAttribute('value').toBeTruthy();

Here we're checking that the value attribute of an input evaluates to true.

If we want to check if a string matches a regular expression then we can use the toMatch() assertion.

element(by.css('input[name="forename"]')).getAttribute('value').toMatch(/[CK]h?ris/);

We can use any JavaScript compatible regular expression pattern with toMatch().

The next thing to check is whether or not a variable has been defined. Although this is more commonly checked in a unit test, it can be helpful to check it in an end-to-end test too, especially if the variable is set by something that's imported in to the page from outside. To test for a variable definition we should use toBeDefined().

it("The user name has been set", function() {
    var user = {
        name: "Chris"
    };
    expect(user.name).toBeDefined();
});

Jasmine also includes a method toBeUndefined() to test if something hasn't been set yet.

If you want to go deeper than a simple check that something has been set or whether it evaluates to true, then you can do a deep comparison of an object in your test using toEqual(). toEqual() does an equality match between two values, even if they're objects.

it("should work for objects", function() {
    var user = {
        username: 'Admin',
        name: 'Chris'
    };
    var admin = {
        username: 'Admin',
        name: 'Chris'
    };
    expect(user).toEqual(admin);
});

Next up we have some mathematical assertion methods. toBeLessThan() and toBeGreaterThan() should be quite obvious.

it("The user is the right age", function() {
    var user = {
        name: "Chris",
        age: 39
    };
    expect(user.age).toBeLessThan(50);
    expect(user.age).toBeGreaterThan(20);
});

There's also the mathematical function toBeCloseTo(), which enables us to test if a number is close to another number but not an exact match. This can be useful for testing functions that rely on time where external factors can mean code takes approximately a specified amount of time but there's no way to know exactly how long.

The last assertions that we might want to use are toThrow() to check for an exception and toThrowError() to check for an error, but these are rarely useful in end-to-end tests. They're much more common in unit testing.

Sometimes we want to test the opposite of an assertion. In these cases we can use the not method in a chain to test for the inverse.

it("The user age has not been set", function() {
    var name = 'Chris';
    expect(name).not.toBe('Paul');
});

This is exactly the same as the test where we used toBe() before, but we've added in the not method before the assertion.

In some cases it's hard to tell whether to use a not or to use a simple assertion. For example, we could use not.toBeDefined() or we could use toBeUndefined(). Which you use should be determined by the language of your tests - if you're testing that something isn't defined then you should use not.toBeDefined(). If you're testing whether something is undefined then you should use toBeUndefined(). It might sound like those are the same, and technically they are, but you should still use the one that matches the way you talk about your tests.

Very occasionally we may want to test whether or not a block of code has actually been run. Cleverly, Jasmine enables us to do this with it's spies.

In testing a spy is a test function that watches a function in our code and reports whether it was called, and how many times if it was. This can be incredibly useful to test that things are being run properly and that the code isn't running too often. If you're making calls to an API that should be cached after they've run then you would want to test that the code block that makes the API call has been run once, but only once. Using a spy is how you'd do that.

TODO: Spy example.

Spies are relatively complex, and there's a lot of different options available to us. We'll look at these in more depth later in the book.

The one really important thing about a test is that we absolutely have to make sure the test results are reproducible. A test that runs without us knowing exactly what the state was is of very little use - we can't know that the code works in a given scenario unless we know the code is in that scenario when it's tested. With this in mind, we need to take control of the data we pass in to our app when we're testing it.

There are a couple of common ways to do this.

The first is with mocks. A mock is a 'pretend' version of a code block that checks our code has called it properly. A mock object should present the test with the same interface as the same code as the real version, but instead it actually returns a Mock class. This enables us to have complete control over the data that it returns, but it also means we can verify that the code has been used - a mock has the ability to return information about whether or not methods have been called. This means we can create a 'pretend' version of a class during testing and then test that the code we're checking has called methods on it.

TODO: Example of a mock

The second method of returning test data to our code is with stubs. A stub is essentially the same as a mock but without the complexity of the mock object. It's like using a static object in place of something that loads data dynamically. For example, in a typical test we wouldn't actually want to make a call to an external API. That would be slow, it'd introduce a potential external failure (if the API was down), and it wouldn't give us much control over the data that the API returns. It'd be better to use a stub where we simply load a JSON object in to memory and use that. This means that our test would be the same every time. If we wanted to be really clever then we could even stub out different API responses - that would mean we could test scenarios where the API is unavailable or where it's returning incorrect data.

Making a robust application that can cope means testing what happens when things aren't right to check that it doesn't break. A great application should detect when something is wrong and fail graciously with a friendly message to the user especially when the API it's relying on is completely broken.