Understanding middleware in Express.js
Over the last few months I've been making an effort to spend regular time answering questions on StackOverflow. I'm doing this for two reasons, first I want to give back to the community which helps me daily, and second it's proving a fantastic way to deliberately practice my written communication.
During my adventures in Q&A land, I've noticed a breed of questions which continue to come up. Most of these seem to stem from a lack of understanding about how Express' middleware concept works. This often seems to come around as a result of people following various guides to get going with Nodejs and Express but without understanding the technology itself. In this article I'm going to take a deep dive into Express' middleware concept, how it works, and how you can leverage it to build modular applications.
What is a "request handler"?
A request handler is a function which takes three parameters; a request object, a response object, and a function that triggers the next request handler in the chain. Typically these are called req, res, and next respectively. You often see these attached to a path, they are executed when the user hits an endpoint in your application. They can operate on the request, set response headers, cause a response to be sent early (finishing a request early) or call the next request handler in the chain.
What is "middleware"?
The Express.js glossary defines middleware as follows:
A function that is invoked by the Express routing layer before the final request handler, and thus sits in the middle between a raw request and the final intended route.
So a middleware function is a request handler that comes before the "final request handler".
We will take a concrete example to help us expand on this. Lets assume you have a JSON API for a private web forum, to see a list of forum posts you must be logged in. Anyone can register for the forum, but only approved users are allowed to read posts and create new ones.
Taking the above example, there are several ways we could do this in Express. Here is how you might do it if you're new to Express.
Note: for accessibility I'm using callbacks as this is what most nodejs and expressjs material on the web uses.
// [...Setup your express application...]
// Endpoint to get a list of all forum posts
app.get('/posts', (req, res, next) => {
// Check to see if the client has a session cookie
const session = req.cookies.session;
if (!session) {
// The client has no session, tell them to login
return res.status(401).json({ msg: 'you must first login' });
}
// Get the clients user account from the database
db.getAccount(session, (account) => {
if (!account) {
// The session is invalid
return res.status(401).json({ msg: 'you must first login' });
}
if (!account.approved) {
// The user has not been approved to see the content
return res.status(403).json({ msg: 'you have not been approved by an admin yet, try again in a few hours.' });
}
// The user is valid and approved, grab the posts and send them back
db.getPosts((foundPosts) => {
return res.status(200).json({ posts: foundPosts});
});
});
});The above code seems reasonable, the function names seem quite clear, and we're not doing too much in this request handler. However, a good chunk of this code is likely to be shared in other places. For example, finding the user from a session cookie is pretty generic session authentication, you're likely to want this in a number of places. The same can be said for checking the user is an approved user.
The next step you might be thinking would be to wrap the logic into small functions, but we can do one better, we can use Middleware.
The request chain goes roughly like this:
- Check the user has a session, if they don't then exit early, if they do then get the user account from it.
- Check the user is "approved", if they're not then exit early with an error, otherwise continue.
- Get the posts from the database and return them to the user.
If we split this out into smaller middleware functions, it might look like this.
// [...Setup your express application...]
function assertUserAuthenticated(req, res, next) {
// Check to see if the client has a session cookie
const session = req.cookies.session;
if (!session) {
// The client has no session, tell them to login
return res.status(401).json({ msg: 'you must first login' });
}
db.getAccount(session, (account) => {
if (!account) {
// The session is invalid
return res.status(401).json({ msg: 'you must first login' });
}
// Set the user on the request for future middleware handlers
req.user = account;
next();
});
}
function assertUserApproved(req, res, next) {
if (!account.approved) {
// The user has not been approved to see the content
res.status(403).json({ msg: 'you have not been approved by an admin yet, try again in a few hours.' });
}
}
// Endpoint to get a list of all forum posts
app.get(
'/posts',
assertUserAuthenticated,
assertUserApproved,
(req, res, next) => {
db.getPosts((foundPosts) => {
res.status(200).json({ posts: foundPosts});
});
});
);In this refactored version, we now have two pieces of reusable middleware and one "final request handler". The final request handler is specific to the endpoint so we leave that as an anonymous function.
Now we can knockout endpoints for an individual post really quickly, this might look something like this.
app.get(
'/posts/:id',
assertUserAuthenticated,
assertUserApproved,
(req, res, next) => {
const postId = req.params.id;
db.getPostById(postId, (foundPost) => {
if (!foundPost) {
return res.status(404).json({ msg: 'post not found' });
}
return res.send(200).json({ post: foundPost });
});
})
);Middleware as a plugin
We've covered middleware in the traditional sense, but middleware functions don't need to be attached to a specific path. They can also be attached to the top level express application with a simple app.use(myMiddleware). You may have done this before without knowing it when using other Express libraries like Express-Sessions.
When applying middleware without a path, it'll be run for every request (unless middleware earlier in the chain causes an early exit). This is how a lot of loggers work. The following middleware could be used for some crude request logging to see every incoming request.
// [... Setup express application ...]
function requestLogger(req, res, next) {
// Construct the log message, i.e:
// 2018-10-02T12:00:13 - GET - /post
const time = Date.now();
const logMsg = `${time} - ${req.method} - ${req.path}`;
// Write the log
console.log(logMsg);
// Continue with the chain
next();
}
app.use(requestLogger);
// [... Setup application endpoints ...]Express itself is middleware
An express application itself can also be treated as middleware. This is something many people don't discover for quite a while when using Express, but it allows you to make numerous sub-application and "mount" them to a single overall application. This allows you to make complex application trees without coupling all your components together, making it easy to split them out into micro-services further down the road. Here's an example that we might use for our forum application.
const path = require('path');
const express = require('express');
const expressLogger = require('express-logging');
const hbs = require('express-hbs');
function parentApp() {
// This is the "parent" application, all other
// applications will be mounted onto it.
const app = express();
// Log every request that comes into any endpoint
// of the parent or sub-application endpoint.
app.use(expressLogger);
return app;
}
function api() {
const apiApp = express();
apiApp.get('/ping', (req, res, next) => {
return res.status(200).json({ msg: 'pong' });
});
return apiApp;
}
function frontend() {
const frontendApp = express();
// Sever static frontend assets like css, imgs, etc
const staticAssetsPath = path.join(__dirname, '/static');
frontendApp.use('/static', express.static(staticAssetsPath));
// Configure the view engine
app.engine('hbs', hbs.express4({
partialsDir: __dirname + '/views/partials'
}));
app.set('view engine', 'hbs');
app.set('views', __dirname + '/views');
frontendApp.get('/', (req, res, next) => {
return res.render('index');
});
}
// Get the parent app as our "root app"
const rootApp = parentApp();
// Mount the API module for endpoints starting with /api/v1
rootApp.use('/api/v1', api());
// Mount the frontend module on the top level endpoint
rootApp.use(frontend());From this example application, we can see we're creating a single root/parent application. This has middleware which will run on any request going to any endpoint or any child app's endpoint.
Then there is the api application, this is attached to the /api/v1 application and has a sample endpoint which lives at /api/v1/ping. The parent apps middleware will be run, then any additional middleware for the api application, and then ending with the ping request handler.
We finally have the frontend application which is attached directly to the parent application. It has no sub-path but it's middleware does not effect the api's middleware since it is attached to the parent app after the API. We are using the frontend application to serve static content and an index page.
Error handlers
It's worth noting that error handlers work in a similar way as normal middleware, but they must have a function signature of accepting 4 parameters. These are traditionally, err, req, res, next. This new parameter at the start represents the error. Error handling middleware is only called if another request handler calls next with a parameter. That parameter is given to us as the err parameter to our error handling middleware.
Error handlers are only called if there is an error to handle, but they can still be chained together in the same way (by calling next with a single parameter).
Closing remarks
Express is built upon request handlers (your req, res, next functions), and the fact that the express application itself is also just a request handler makes it very easy to compose large applications from small independent modules.
When you begin to think in terms of small request handlers, you end up creating small functions and modules which can be reused across multiple applications or open sourced. You can then apply this to your larger components of your application to make them easy to break out into their own free-standing applications.
Member discussion