Integration and End-to-end Tests Made Easy with Node.js and MongoDB

Tests are an essential part of building a robust Node.js application. Proper tests can easily overcome a lot of shortcomings that developers may point out about Node.js development solutions.

While many developers focus on 100% coverage with unit tests, it is important that the code you write is not just tested in isolation. Integration and end-to-end tests give you that extra confidence by testing parts of your application together. These parts may be working just fine on their own, but in a large system, units of code rarely work separately.

Node.js and MongoDB together form one of the most popular duos of recent times. If you happen to be one of the many people using them, you are in luck.

In this article, you will learn how to write integration and end-to-end tests easily for your Node.js and MongoDB application that run on real instances of the database all without needing to set up an elaborate environment or complicated setup/teardown code.

You will see how the mongo-unit package helps with integration and end-to-end testing in Node.js. For a more comprehensive overview of Node.js integration tests, see this article.

Dealing with a Real Database

Typically, for integration or end-to-end tests, your scripts will need to connect to a real dedicated database for testing purposes. This involves writing code that runs at the beginning and end of every test case/suite to ensure that the database is in a clean predictable state.

This may work well for some projects, but has some limitations:

• The testing environment can be quite complex. You will need to keep the database running somewhere. This often requires extra effort to set up with CI servers.
• The database and the operations can be relatively slow. Since the database will use network connections and the operations will require file system activity, it may not be easy to run thousands of tests quickly.
• The database keeps state, and it’s not very convenient for tests. Tests should be independent of each other, but using a common DB could make one test affect others.

On the other hand, using a real database makes the test environment as close to production as possible. This can be looked at as a particular advantage of this approach.

Using a Real, In-Memory Database

Using a real database for testing does seem to have some challenges. But, the advantage of using a real database is too good to pass on. How can we work around the challenges and keep the advantage?

Reusing a good solution from another platform and applying it to the Node.js world can be the way to go here.

Java projects widely use DBUnit with an in-memory database (e.g., H2) for this purpose.

DBUnit is integrated with JUnit (the Java test runner) and lets you define the database state for each test/testing suite, etc. It removes the constraints discussed above:

• DBUnit and H2 are Java libraries, so you do not need to set up an extra environment. It all runs in the JVM.
• The in-memory database makes this state management very fast.
• DBUnit makes the database configuration very simple and allows you to keep a clear database state for each case.
• H2 is a SQL database and it is partially compatible with MySQL so, in major cases, the application can work with it as with a production database.

Taking from these concepts, I decided to make something similar for Node.js and MongoDB: Mongo-unit.

Mongo-unit is a Node.js package that can be installed using NPM or Yarn. It runs MongoDB in-memory. It makes integration tests easy by integrating well with Mocha and providing a simple API to manage the database state.

The library uses the mongodb-prebuilt NPM package, which contains prebuilt MongoDB binaries for the popular operating systems. These MongoDB instances can run in in-memory mode.

Installing Mongo-unit

To add mongo-unit to your project, you can run:

npm install -D mongo-unit

or

yarn add mongo-unit

And, that is it. You do not even need MongoDB installed on your computer to use this package.

Using Mongo-unit for Integration Tests

Let’s imagine you have a simple Node.js application to manage tasks:

// service.js

const mongoose = require('mongoose')
const mongoUrl = process.env.MONGO_URL || 'mongodb://localhost:27017/example'
mongoose.connect(mongoUrl)
const TaskSchema = new mongoose.Schema({
 name: String,
 started: Date,
 completed: Boolean,
})
const Task = mongoose.model('tasks', TaskSchema)

module.exports = {
 getTasks: () => Task.find(),
 addTask: data => new Task(data).save(),
 deleteTask: taskId => Task.findByIdAndRemove(taskId)
}

The MongoDB connection URL is not hard-coded here. As with most web application back-ends, we are taking it from the environment variable. This will let us substitute it for any URL during tests.

const express = require('express')
const bodyParser = require('body-parser')
const service = require('./service')
const app = express()
app.use(bodyParser.json())

app.use(express.static(`${__dirname}/static`))
app.get('/example', (req, res) => {
 service.getTasks().then(tasks => res.json(tasks))
})
app.post('/example', (req, res) => {
 service.addTask(req.body).then(data => res.json(data))
})
app.delete('/example/:taskId', (req, res) => {
 service.deleteTask(req.params.taskId).then(data => res.json(data))
})
app.listen(3000, () => console.log('started on port 3000'))

This is a snippet of an example application that has a user interface. The code for the UI has been omitted for brevity. You can check out the complete example on GitHub.

Integrating with Mocha

To make Mocha run integration tests against mongo-unit, we need to run the mongo-unit database instance before the application code is loaded in the Node.js context. To do this, we can use the mocha --require parameter and Mocha-prepare library, which allows you to perform asynchronous operations in the require scripts.

// it-helper.js
const prepare = require('mocha-prepare')
const mongoUnit = require('mongo-unit')

prepare(done => mongoUnit.start()
 .then(testMongoUrl => {
   process.env.MONGO_URL = testMongoUrl
   done()
 }))

Writing Integration Tests

The first step is to add a test to the test database (testData.json):

{
   "tasks": [
   {
     "name": "test",
     "started": "2017-08-28T16:07:38.268Z",
     "completed": false
   }
 ]
}

The next step is to add the tests themselves:

const expect = require('chai').expect
const mongoose = require('mongoose')
const mongoUnit = require('../index')
const service = require('./app/service')
const testMongoUrl = process.env.MONGO_URL

describe('service', () => {
 const testData = require('./fixtures/testData.json')
 beforeEach(() => mongoUnit.initDb(testMongoUrl, testData))
 afterEach(() => mongoUnit.drop())

 it('should find all tasks', () => {
   return service.getTasks()
     .then(tasks => {
       expect(tasks.length).to.equal(1)
       expect(tasks[0].name).to.equal('test')
     })
 })

 it('should create new task', () => {
   return service.addTask({ name: 'next', completed: false })
     .then(task => {
       expect(task.name).to.equal('next')
       expect(task.completed).to.equal(false)
     })
     .then(() => service.getTasks())
     .then(tasks => {
       expect(tasks.length).to.equal(2)
       expect(tasks[1].name).to.equal('next')
     })
 })

 it('should remove task', () => {
   return service.getTasks()
     .then(tasks => tasks[0]._id)
     .then(taskId => service.deleteTask(taskId))
     .then(() => service.getTasks())
     .then(tasks => {
       expect(tasks.length).to.equal(0)
     })
 })
})

And, voila!

Notice how there are just a couple of lines of code dealing with setup and teardown.

As you can see, it’s very easy to write integration tests using the mongo-unit library. We do not mock MongoDB itself, and we can use the same Mongoose models. We have full control of the database data and do not lose much on test performances since the fake MongoDB is running in memory.

This also allows us to apply the best unit testing practices for integration tests:

• Make each test independent of other tests. We load fresh data before each test, giving us a totally independent state for each test.
• Use the minimum required state for each test. We do not need to populate the whole database. We only need to set the minimum required data for each particular test.
• We can reuse one connection for the database. It increases test performance.

As a bonus, we can even run the application itself against mongo-unit. It allows us to make end-to-end tests for our application against a mocked database.

End-to-end Tests with Selenium

For end-to-end testing, we will be using Selenium WebDriver and Hermione E2E test runner.

First, we will bootstrap the driver and the test runner:

const mongoUnit = require('mongo-unit')
const selenium = require('selenium-standalone')
const Hermione = require('hermione')
const hermione = new Hermione('./e2e/hermione.conf.js') //hermione config

seleniumInstall() //make sure selenium is installed
 .then(seleniumStart) //start selenium web driver
 .then(mongoUnit.start) // start mongo unit
 .then(testMongoUrl => {
   process.env.MONGO_URL = testMongoUrl //store mongo url
 })
 .then(() => {
   require('./index.js') //start application
 })
 .then(delay(1000)) // wait a second till application is started
 .then(() => hermione.run('', hermioneOpts)) // run hermiona e2e tests
 .then(() => process.exit(0))
 .catch(() => process.exit(1))

We will also need some helper functions (error handling removed for brevity):

function seleniumInstall() {
 return new Promise(resolve => selenium.install({}, resolve))
}

function seleniumStart() {
 return new Promise(resolve => selenium.start(resolve))
}

function delay(timeout) {
 return new Promise(resolve => setTimeout(resolve, timeout))
}

After filling the database with some data and cleaning it once the tests are done, we can run our first tests:

const expect = require('chai').expect
const co = require('co')
const mongoUnit = require('../index')
const testMongoUrl = process.env.MONGO_URL
const DATA = require('./fixtures/testData.json')

const ui = {
 task: '.task',
 remove: '.task .remove',
 name: '#name',
 date: '#date',
 addTask: '#addTask'
}

describe('Tasks', () => {

 beforeEach(function () {
   return mongoUnit.initDb(testMongoUrl, DATA)
     .then(() => this.browser.url('http://localhost:3000'))
 })

 afterEach(() => mongoUnit.dropDb(testMongoUrl))

 it('should display list of tasks', function () {
   const browser = this.browser
   return co(function* () {
     const tasks = yield browser.elements(ui.task)
     expect(tasks.length, 1)
   })
 })

 it('should create task', function () {
   const browser = this.browser
   return co(function* () {
     yield browser.element(ui.name).setValue('test')
     yield browser.element(ui.addTask).click()
     const tasks = yield browser.elements(ui.task)
     expect(tasks.length, 2)
   })
 })

 it('should remove task', function () {
   const browser = this.browser
   return co(function* () {
     yield browser.element(ui.remove).click()
     const tasks = yield browser.elements(ui.task)
     expect(tasks.length, 0)
   })
 })
})

As you can see, the end-to-end tests look very similar to the integration tests.

Wrap Up

Integration and end-to-end testing are important for any large-scale application. Node.js applications, in particular, can benefit tremendously from automated testing. With mongo-unit, you can write integration and end-to-end testing without worrying about all the challenges that come with such tests.

You can find complete examples of how to use mongo-unit on GitHub.