Frisby.js v2.0 – The REST API Testing Framework

August 14, 2019

I have been working on Frisby.js for the better part of a decade now. Frisby.js 2.0 has been released for just over 2 years (2.0.0 was released on July 26, 2017), and I realized that I never published a write-up of version 2.0, its differences from v0.x (effectively v1.x, though a 1.x was never officially released), or all the reasons behind the whole new approach I took in the 2.x branch. This is the missing writeup, that I am finally publishing now 🙂

Why Not Iterate on v0.x (effectively v1)?

There were a lot of issues with Frisby.js 0.x that have existed for a while, and some well known problems that are still not fixed due to lingering issues with running nested expectations in Jasmine. The main root of the issue is that Jasmine does not support nested tests the way that Frisby 0.x generated them when you nest calls with after and afterJSON. Unfortunately, due to the design of the Frisby API where Frisby controls and generates the whole Jasmine test for you, this is not possible to fix. By the time Frisby 0.x knows that another test will be generated (calling after), the current test has already been created and run, leading to some weird issues with Jasmine in certain contexts for any other tests generated after that (like custom function matchers –#93#94, #307, etc.). This ultimately meant that the test is not reliable, and can pass when it should fail when features like custom matchers are used. This lead to a loss of confidence in Frisby 0.x that undermined the purpose of testing, and (rightfully) causes a lot of frustration in the v0.x release.

Big Breaking Changes

Fixing the problems in Frisby 0.x required large breaking changes, because the whole API had to change. Starting in v2.0, Frisby sits neatly inside your own Jasmine test function instead of generating it for you. This is best illustrated by example.

Old Frisby.js v0.85 API (generates the whole test “it” for you):

var frisby = require('frisby');

frisby.create('should get user Joe Schmoe')
  .get(testHost + '/users/1')
  .expectStatus(200)
  .expectJSON({
    id: 1,
    email: 'joe.schmoe@example.com'
  })
  .toss();

New Frisby.js v2.x API (sits inside your own Jasmine-style test):

var frisby = require('frisby');

it('should get user Joe Schmoe', function() {
  return frisby.get(testHost + '/users/1')
    .expect('status', 200)
    .expect('json', {
      id: 1,
      email: 'joe.schmoe@example.com'
    });
});

This means that Frisby.js 2.x is now focused solely on the HTTP request itself and the assertions made against the HTTP response. You are in control of everything else. This provides a lot of possibilities that simply did not exist in Frisby v0.x, like the ability to use Jasmine’s beforeAll and beforeEach methods, test grouping with describe, and more. Basically, Frisby.js v2 is all about focusing on its strengths, playing to Jasmine’s strengths, and getting out of your way for everything else.

Why Is The New API Better?

The difference may be a bit subtle, but this means that there will never be nested Jasmine tests generated with nested Frisby calls. This single-handedly eliminates all of the most difficult and frustrating issues in Frisby v0.x, and makes Frisby v2.x much more reliable.

Modernization

JavaScript has grown up a lot since Frisby.js was first released back in 2011. A full rewrite gives me the opportunity to modernize the codebase based on new standards, like the fetch API and some newer ES6 features supported by Node (but without transpiling).

A brief summary of the new features in Frisby.js v2.0:

  • Frisby now uses the fetch API and is Promise based
  • Nested tests use then instead of after or afterJSON functions, so the nesting level can stay consistent instead of requiring deeply nested structures (I Promise you this is much better).
  • Each Frisby test is a new FrisbySpec instance, which keeps state between tests much cleaner
  • Easily supports negative assertions with expectNot as a companion to expect
  • Supports user-defined custom named expect matchers with frisby.addExpectHandler('foo', function (args) { ... }), which you can then use in tests with .expect('foo', args)
  • Supports testing JSON directly via frisby.fromJSON() without making any HTTP requests.
  • Frisby now uses Jest as its test runner instead of Jasmine. This allows each test to be run in its own sandbox and async process. This provides a significant reduction in the overall time it takes to run large Frisby test suites.

Nested Tests

Nested tests in Frisby v2.0 only require that you return the FrisbySpec object from your Jasmine test. This acts just like a Promise:

var frisby = require('frisby');

it('should get user Joe Schmoe, then delete him', function() {
  // Get him
  return frisby.get(testHost + '/users/1')
    .expect('status', 200)
    .expect('json', {
      id: 1,
      email: 'joe.schmoe@example.com'
    })
    .then(function (json) {
      // Delete him
      return frisby.del(testHost + '/users/1')
        .expect('status', 204);
    })
});

Flow Control

Many people were frustrated with the lack of setup/teardown methods in Frisby v0.x, which can sometimes be very useful for REST APIs for performing tasks like creating a user and logging in before performing actions that require authentication. The old Frisby API would have made this a chore to implement, because it would mean manually exposing, wrapping, or re-creating these methods by hand. With the new API in v2.0, this problem is non-existent. Since Frisby no longer wraps the Jasmine test, the end user can now use Jasmine’s methods like beforeAll directly:

describe('jobs API', function () {

  let access_token;
  beforeAll(function () {
    // Login as admin user to get and store access_token, then send it in each subsequent request
    return frisby.post(baseUrl + '/users/login', {
        body: {
          email: 'user@example.com',
          password: 'password'
        }
      })
      .expect('status', 200)
      .then(function (json) {
        access_token = json.access_token;

        // Set 'Authorization' header for all further requests
        frisby.globalSetup({
          request: {
            headers: {
              'Authorization': 'Bearer ' + access_token
            }
          }
        });
      });
  });

  it('should get list of all active jobs', function () {
    return frisby.get(baseUrl + '/jobs')
      .expect('status', 200);
  });

});

The best code truly is no code at all. It’s amazing what you can achieve when you just get out of the user’s way.

Looking To The Future

The philosophy behind Frisby.js 2.0 is all about making things easier, faster, more modern, and giving you more control and extensibility. Frisby 2.0 will continue to build on that as new issues come in and as new features are released.

If you are looking to help ensure your API is rock solid across your different product releases and versions, check out the Frisby.js git repo or read the Frisby.js docs. I think you are going to like what you see.


Tags: , , , ,

Categories: , , ,