Test Coverage Report (May 1st)

Unit Tests - Jest

Components

We use Jest to test our React Component. We mostly do snapshot testing to check for changes over time. We also have interaction tests where we test to see if user input like a “click” will fire the appropriate handler.

Jest’s default react shallow renderer with enzyme works well for us because it limits the scope of the test and lets us focus on the component under testing and not necessarily the children. We also test wrapped components to avoid the connected function and passing in the full application state.

With that said, a typical component test will still exercise several the utilities and other imported dependencies. We just started writing component tests and our 6 tests cover 60 files! We’re not trying to write isolated unit tests!

Snapshot Test
it("should render the component", () => {
  const wrapper = shallow(ResultList(payload));
  expect(wrapper).toMatchSnapshot();
});
Interaction Test
it("should call onClick function", () => {
  const wrapper = shallow(ResultList(payload));

  wrapper.childAt(selectedIndex).simulate("click");
  expect(selectItem).toBeCalled();
});

Utilities

Our utilities are the best tested part of the debugger. We try to move as much logic to a util so that we can reasonably test them in isolation.

The best example of this is our parser utils, which are like the brain of the debugger. Here’s a really cool example of how we test getting local variables. The best part about these tests is that they’re really easy to TDD and get confidence. It lets us have 90-100% coverage and refactor really easily.

it("only gets local variables", () => {
  const scope = getClosestScope(getSourceText("math"), {
    line: 3,
    column: 5
  });

  var vars = getVariablesInLocalScope(scope);

  expect(vars.map(v => v.name)).to.eql(["n"]);
  expect(vars[0].references[0].node.loc.start).to.eql({
    column: 4,
    line: 3
  });
});

Actions / Reducers

The old Debugger UI only had integration tests. When we switched to React, we knew we wanted to write tests that covered our Redux system.

The first we wrote tested our breakpoint logic. I still think it reads really well and covers a lot of ground. Notice that we have a mock client that we pass into the store so that we don’t make any API calls.

it("should add a breakpoint", async () => {
  const { dispatch, getState } = createStore(simpleMockThreadClient);

  await dispatch(actions.addBreakpoint({ sourceId: "a", line: 5 }));
  await dispatch(actions.addBreakpoint({ sourceId: "b", line: 6 }));

  expect(selectors.getBreakpoints(getState()).size).to.be(2);
});

We’ve recently made a concerted effort to add more redux integration test. We’re now testing all of our action files and have good coverage in everything, but source actions.


Integration Tests

While we try to get most of our coverage with unit tests, it is still helpful to have integration tests to confirm that we did not break an important code path like pausing on a breakpoint.

Switching to redux has helped us write really clean tests, that are both fun to write, and easy to debug.

Here is a typical test where we open the debugger, toggle breakpoints by clicking in the gutter, and check the redux store if the beakpoint is registered.

async function toggle(ctx) {
  const { ok, is, info } = ctx;
  const dbg = await initDebugger("doc-scripts.html", "simple2");

  // Create two breakpoints
  await selectSource(dbg, "simple2");
  await addBreakpoint(dbg, "simple2", 3);
  await addBreakpoint(dbg, "simple2", 5);

  // Disable the first one
  await disableBreakpoint(dbg, 1);
  let bp1 = findBreakpoint(dbg, "simple2", 3);
  let bp2 = findBreakpoint(dbg, "simple2", 5);
  is(bp1.disabled, true, "first breakpoint is disabled");
  is(bp2.disabled, false, "second breakpoint is enabled");

  // Disable and Re-Enable the second one
  await disableBreakpoint(dbg, 2);
  await enableBreakpoint(dbg, 2);
  bp2 = findBreakpoint(dbg, "simple2", 5);
  is(bp2.disabled, false, "second breakpoint is enabled");
}

Our integration tests are our last line of defense and have caught the most would be bugs. We are currently making a push to test all of our UI components with at least one integration test.