Handle MongoDB transactions in Node.js using Mongoose

A transaction is an execution of many operations (insert, update, or delete) as a single unit that takes the database from a consistent state and lets it to a consistent one. If one operation fails, all the operations are canceled. The properties of a transaction are Atomicity, Consistency, Isolation, and Durability (ACID).

Transactions are common for Relational databases, while with NoSQL databases, the philosophy of database management goes against the strict ACID rules. MongoDB is one of the most used NoSQL databases, and it has added support for transactions since version 4.0.

This tutorial will show how to use MongoDB transactions in a Node.js application.

The use case

We will implement product ordering on an e-commerce website as a real-world use case. Below is our simplified class diagram.

Let's explain the relationships between the classes:

• A user can make one or many orders
• An order can have one or many payments (when the payment fails and we try with a new method, pay in many times)
• An order has one or many items
• An order item is associated with a product

Additional requirements:

• When the user places his order, if the payment succeeds and exceeds 100 euros, he gets a 5-mark bonus.
• When the order is completed, we update the product quantity in stock.

Here is the sequence diagram of the workflow to implement:

The importance of transaction

In the sequence diagram above, five steps constitute the order workflow. Let's analyze our system if we have a failure at each step:

• Step 1: If the application fails to create the order (duplicate order reference, insufficient permissions, etc.), the application will break, and the other steps will not continue.
• Step 2: If the payment execution fails (uncaught errors, service unavailable, etc.), we have an order created with payment status unknown, meaning if the user checks out the payment, he will have a message like, "We are processing your payment..." for minutes, hours, days if we don't change the status manually.
• Step 3: If the payment execution succeeds but the system fails to update the status (wrong API response parsing, bad update query, etc.), we have an order already paid, but the user receives a message asking him to pay again.
• Step 4: If the application fails to update the quantity of products in stock, users could have a bad experience since they can order an unavailable product.
• Step 5: If the payment is greater than 100€ and the application fails to increment the user bonus mark, you will have angry shouting everywhere that your promotion is a scam.

As we can see, to have a consistent system, all five steps must succeed. If one step fails, we should cancel the previous changes on the system. That is why we need a transaction of these steps as a single unit.

Prerequisites

MongoDB supports transactions since version 4.0; you must use at least this version to follow this tutorial.

For transactions to work on MongoDB, a replica set must be created. I wrote this full post where I guide you on setting it up in your local environment.

You can also use MongoDB Atlas, which gives a free instance that comes with a replica set out of the box.

I will use MongoDB with a replica set on my local environment, so you will also need Docker and Docker compose

Finally, Node.js 18+ must be installed since we will use it to build the sample application.

Set up the project

To start, we will use the project created in the tutorial, where I show how to create a replica set. Let's clone it from GitHub.

git clone https://github.com/tericcabrel/blog-tutorials.git

cd blog-tutorials/mongodb-replica-set

Initialize a Node.js project and also Typescript

yarn init -y

yarn add -D typescript

yarn tsc --init

Start the MongoDB server

First, start the MongoDB server with a replica set by running the command below:

./dbstart.sh

To connect to the database, we need the URL to that server. Each MongoDB runs on port 27017 internally; this port is mapped to the host's specific port, respectively: 27021, 27022, and 27023.

The URL will be: mongodb://localhost:27021,localhost:27022,localhost:27023/test?replicaSet=dbrs

The database name is test is the name of the database and dbrs is the name of the replica set.

Connect to the replication set

We will use Mongoose, the ORM for MongoDB in Node.js; we also need ts-node to run a .ts file. Let's install them.

yarn add mongoose
yarn add -D ts-node

Create a folder named src then, create the file called app.ts and finally, add the code below:

import mongoose, { ConnectOptions } from "mongoose";

const DATABASE_URL = "mongodb://localhost:27021,localhost:27022,localhost:27023/test?replicaSet=dbrs";
const options: ConnectOptions = {
  readPreference: 'secondary',
};

(async () => {
  try {
    await mongoose.connect(DATABASE_URL, options);

    console.log("Connected to the database successfully ");
  } catch (e) {
    console.error("Failed to connect to the database!", e);
  }
})();

Execute the file to test the connection:

yarn ts-node ./src/app.ts

The connection should work as expected. The option readPreference indicates if the read queries should be executed on the primary or secondary replica.

Define the models

We need to create Mongoose models for the application based on our diagram. Be careful! Modeling for NoSQL is not the same as for relational databases.

If we use a relational database, we will create 5 tables, but with MongoDB, a NoSQL database, we will create 3 collections.

I will not cover the creation of the models since it is not the main part of the topic, but you can find their definitions in the project repository here.

Create database collections

When performing MongoDB queries without transactions, you don't need to create the collections because they are created automatically when we insert the first document.

You have to do it manually on a replica set; otherwise, transactions will not work. Let's update the file "./src/app.ts" to create the collections from the model.

import mongoose, { ConnectOptions, Model, Document } from "mongoose";
import { models } from "./models";

const DATABASE_URL = "mongodb://localhost:27021,localhost:27022,localhost:27023/test?replicaSet=dbrs";

const options: ConnectOptions = {
  readPreference: 'secondary',
};

const createCollections = async (models: Model<Document>[]) => {
  await Promise.all(
      models.map((model) => model.createCollection())
  );
};


(async () => {
  try {
    await mongoose.connect(DATABASE_URL, options);

    await createCollections(models);

    console.log("Connected to the database successfully ");
  } catch (e) {
    console.error("Failed to connect to the database!", e);
  }
})();

The collections are created. Let's add some users and products to use in the upcoming part.

Find the source code for the database seed in the GitHub repository in the file src/seed.ts.

Anatomy of a transaction's execution

The code below shows the template of what a transaction looks like:

const session: ClientSession = await mongoose.startSession();

session.startTransaction();

try {
	// TODO Add your statement here

	// Commit the changes
	await session.commitTransaction();
} catch (error) {
	// Rollback any changes made in the database
	await session.abortTransaction();

	// Rethrow the error
	throw error;
} finally {
	// Ending the session
	await session.endSession();
}

• We create a transaction session and then initialize the transaction
• Inside the "try" block, we will add our operations to the database that must be executed atomically.
• Once done, we commit the changes.
• If an error occurs, the transaction is aborted inside the catch block, and the error is rethrown.
• Finally, we end the session.

Simulate a MongoDB transaction failure

To see the transaction in action, we will first simulate a failure case where I explicitly make the execution fail. In the function "proceedToPayment()", I intentionally throw an error after 2 seconds to simulate the failure of payment execution.

We expect nothing to be saved in the database at the end of the execution.

Inside the folder src, create a file called failure-transaction.ts, then add the code below:

import mongoose, { ClientSession } from "mongoose";
import { Order, OrderInput, OrderStatusEnum, PaymentStatusEnum } from "./models/order.model";
import { Product } from "./models/product.model";
import { User } from "./models/user.model";

const userId = '619052e09f31f6bcc5cf0234'; // ID of "User First" in the users collection

const orderInput: OrderInput = {
  amount: 50,
  vat: 20,
  user: new mongoose.Types.ObjectId(userId),
  status: OrderStatusEnum.CREATED,
  reference: Math.random().toString(),
  payment: {
    amount: 50,
    reference: Math.random().toString(),
    status: PaymentStatusEnum.CREATED,
  },
  items: [
    {
      product: '61903d0c3e658edce54265b6', // ID of "Product One" in the products collection
      priceUnit: 5,
      quantity: 4,
    },
    {
      product: '61903cdabe8df95babed6698', // ID of the "Product Two" in the products collection
      priceUnit: 10,
      quantity: 3,
    },
  ]
};

const proceedToPayment = async (paymentInput: OrderInput["payment"]) => {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      reject('Payment failed for unknown reason');
    }, 2000);
  })
};

export const processUserOrderFail = async () => {
  const session: ClientSession = await mongoose.startSession();

  session.startTransaction();

  try {
    // Step 1: create the order
    const [createdOrder] = await Order.create([orderInput], { session });

    // Step 2: process the payment
    const paymentResult: any = await proceedToPayment(orderInput.payment);

    if (paymentResult.failed) {
      // Step 3: update the payment status
      await Order.updateOne({ reference: createdOrder.reference }, { 'payment.status': PaymentStatusEnum.FAILED }, { session });
    } else {
      // Step 3: update the payment status
      await Order.updateOne({ reference: createdOrder.reference }, { 'payment.status': PaymentStatusEnum.SUCCEEDED }, { session });

      // Step 4: Update the product quantity in stock
      await Promise.all(orderInput.items.map((item) => {
        Product.findOneAndUpdate({ _id: item.product }, { $inc: { inStock: item.quantity * -1 } }, { new: true, session });
      }));

      // Step 5: Update bonus mark
      if (orderInput.payment.amount > 100) {
        await User.findOneAndUpdate({ _id: orderInput.user }, { $inc: { bonusMark: 5 } }, { new: true, session });
      }
    }

    // Commit the changes
    await session.commitTransaction();
  } catch (error) {
    // Rollback any changes made in the database
    await session.abortTransaction();

    // logging the error
    console.error(error);

    // Rethrow the error
    throw error;
  } finally {
    // Ending the session
    await session.endSession();
  }
};

Call the function processUserOrderFail() in the file src/app.ts then, run the code with:

yarn ts-node ./src/app.ts

You get the following output

As we can see, there is no document inside the collection orders.

Execute a transaction successfully

For the preview code to work as we expect, we just need to update the function proceedToPayment() to return a valid response.

const proceedToPayment = async (paymentInput: OrderInput["payment"]) => {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      resolve({ failed: false });
    }, 2000);
  })
};

Run the code and see the result.

Bonus: A wrapper for transaction execution

We have seen before that many lines of code are used to define a transaction statement with Mongoose. If you have to run your code at many places, you will end with many lines of code that look the same.

The solution is to create a function that handles transaction management; all you have to do is add your database query instructions.

Here is the function called runInTransaction() :

import mongoose, { ClientSession } from "mongoose";

type TransactionCallback = (session: ClientSession) => Promise<void>;

export const runInTransaction = async (callback: TransactionCallback) => {
  const session: ClientSession = await mongoose.startSession();

  session.startTransaction();

  try {
    await callback(session);

    // Commit the changes
    await session.commitTransaction();
  } catch (error) {
    // Rollback any changes made in the database
    await session.abortTransaction();

    // logging the error
    console.error(error);

    // Rethrow the error
    throw error;
  } finally {
    // Ending the session
    await session.endSession();
  }
};

Here is how to use it:

await runInTransaction(async (session) => {
  await Product.findOneAndUpdate({ _id: '61912208082a0397ff955e53' }, { $inc: { inStock: -5 } }, { session });
  
  await User.findOneAndUpdate({ _id: '61912208082a0397ff955e51' }, { $inc: { bonusMark: 5 } }, { session });
});

No more boilerplate code to add everywhere you need to use transactions.

Wrap up

This tutorial ends, and you now know how to execute a transaction in MongoDB. A transaction can prevent an inconsistent state in your database and your application. It is helpful for a suit of operations that needs to be completed atomically.

You can find the code source on the GitHub repository.

Follow me on Twitter or subscribe to my newsletter to avoid missing the upcoming posts and the tips and tricks I occasionally share.