In this article, you’ll learn how to build a URL Shortener application with Node.js and MongoDB. Here’s a live demo of what we’ll be building. You can find the complete source code for this project in this GitHub repo.
You also need to have Node.js and npm installed on your computer. You can visit the Node.js website to view installation instructions for your operating system. npm comes bundled with Node, so once you install Node, you’ll have access to the
npm command too.
The versions I used while building this project are as follows:
- Node.js v11.2.0
- npm v6.6.0
You can view the version of Node and
npm you have installed by running the following commands in your terminal:
Grab the starter files
Grab the starter files for this project at this GitHub repository. Clone the repo to your computer and
cd into the created directory. Then run
npm install from the project root to install all the dependencies specified in the
package.json file. I’ll get into what each of the dependencies do later on as we progress.
MongoDB is a free and open-source NoSQL document database commonly used in modern web applications. You’ll need to have it installed on your machine. At the time of writing, the latest stable version is 4.0.5. This is the version I used throughout this tutorial.
You can checkout what version of mongoDB you have installed using
The database server should be automatically started after the installation process, but you should verify this before moving on from this step. On Ubuntu, you can use the following command to check the status of the mongoDB server:
You should see this output:
If not, you can start it using the following command:
Set up a basic Node server
Looking through the
src folder within the project directory, you’ll see that we’ve got a
server.js file and a
public folder containing the markup and styles for the application frontend. The
server.js file is where the bulk of the application code will be written in.
Unlike the previous Node tutorial where I used the built-in http module to set up the Node server, we’ll be using Express, a popular Node.js web application framework in this instance.
There are other web frameworks out there, but Express is simple enough, well documented and well supported so you shouldn’t run into many issues when using it in your applications.
If you look through the
package.json file, you will see the
express package is part of the dependencies that we installed earlier. Let’s go ahead and use it to set up the Node server in
You can start the server by running
npm start in the terminal. I am making use of the Nodemon package to auto restart the Node server when changes are made to it so we don’t have to do so ourselves.
Set up the application frontend
As mentioned earlier, the app frontend lives in the
public folder. We need to set up a new route on the server so that when a user visits the application, the HTML file will be sent and rendered in the browser.
server.js file to look like this:
path is a built-in module in Node.js. It allows us to link to directories and file paths in Node.js. The
sendFile() method takes an absolute path to the file, so
__dirname is used to avoid hardcoding the path.
__dirname is the directory in which the executing file is located, so
path.join(__dirname, 'public', 'index.html') will resolve to
http://localhost:4100 in your browser. Notice that the HTML is rendered correctly. However, the styles are missing even though
style.css was linked correctly in
When the browser encounters the reference to
To fix this situation, we need to configure express to handle requests for static files correctly. We can do this using the a built-in middleware function in express as follows:
Now reload the page. It should work correctly:
Submit the form to the server
action attribute of the form to a route on the server and setting the
method attribute to
Create a new
main.js file in the
public directory and add the following code into it:
This code listens for the
submit event on the form, prevents the form submission and fires of a POST request to the server with the value of the form input in request body. The reason we wrap the body object in
JSON.stringify is so that we can consume it as JSON on the server.
Note that we’re posting the data to the
/new route which hasn’t been created on the server yet. We’ll create it in the next section. Before that, make sure you reference
main.js in your
index.html file before the closing body tag:
Access the form body on the server
Let’s go ahead and create the
/new route that will process the URLs to be shortened. Add this below the root route in
The first thing we need to do is access the JSON data that was sent from the client in the request body. To do this, we need to use of the body-parser package. This package parses all incoming request bodies and makes them accessible on
Since this package has been installed already, we can use it right away in
Now, we should be able to access the request body in the
req.body property in the
You can try this out by entering a URL into the form and submitting it. Then navigate to the terminal where your server is running to see the JSON data printed in the terminal.
This means we can access the url using
Validate the URL
Before we shorten the URL, we need to validate if the URL that was submitted is valid. Client side validation is handled for us in the browser because we’ve set the type of the input to
url so the form won’t be submitted if the value doesn’t have a valid URL structure.
To make the application more robust, we need to validate the URL on the server as well. There are several npm packages that can handle this, but I’ve opted to do so using a few built-in Node modules.
The first thing we need to do is check if the URL has a valid structure, then we perform a DNS lookup to see if the domain is operational. A domain like
https://google.com will pass both tests, but
http://jidfsdm.com will fail the second one since that site does not exist.
Require the built-in dns module at the top of
Then change up the
/new route as follows:
The URL class returns a new
URL object with several properties if the input URL has a valid structure. Otherwise, it throws an error which we can
catch and send back to the client.
If the URL input passes the first test, we then check to see if the domain is operational by passing the hostname portion of the URL (the domain) to
dns.lookup which checks if the domain is live. If so, we can connect to our MongoDB instance and create the shortened version of the url as you’ll see.
Set up environmental variables
Environmental variables are a great way to configure how your program should work. They are key-value pairs that are stored on the local system where your program is being run, and are accessible from within your code.
It’s considered best practice to set app configuration data such as API keys, tokens, passwords and other sensitive details as environmental variables instead of hardcoding it into the program itself. This prevents you from accidentally exposing it to others, and also makes the values really easy to change without having to touch your code.
In Node.js, you can access variables defined in your environment via the
process.env object. You can check the contents of this project via the Node.js REPL as shown below:
Aside from the operating system variables that are present by default, we can create project specific variables using a
.env file at the root of your project directory and paste the following code into it:
Here, we’ve added the URL of our local MongoDB instance as an environmental variable. Port 27017 is the port that MongoDB runs on. The next thing to do is load the values defined in
process.env. The easiest way to do this is with the dotenv package which is already part of our app dependencies.
Add the following at the very top of
This will read the contents of the
.env file in the root of your project, parse its contents and initialize the values on
process.env. Now, you’ll be able to access any set variable under
Connect to MongoDB
Let’s go ahead and connect to our local MongoDB instance in
server.js as shown below:
First, we’re importing
MongoClient from mongodb which is the the native driver for interacting with a MongoDB instance in Node.js. Then we connect to the MongoDB instance specified in the
DATABASE environmental variable.
If the connection is successful, we get a reference to the MongoDB instance client and can select a database using the
client.db() method. Note that this method creates the database if it does not already exist.
Here, we are selecting a reference to the
shortener database and storing that reference in
app.locals which is an object provided by
express. This object allows us to set local variables that persist throughout the life of the application, and can be accessed in other middleware functions (functions that have access to the
res objects) via
The reason we’re storing a reference to the database in
app.locals.db is so that we can reuse the db object without having to open another connection to the MongoDB instance.
The next step is to actually shorten the URL and store it in the database. To create a unique short id for each url, we will be making use of the nanoid package.
Require it at the top of
server.js below the other
Then create a new
shortenURL function in
server.js as follows:
We need to reference a collection before we can add any data into the database. We can do so using the using the
db.collection() method. If the collection does not exist yet, it’s created.
Before we shorten the URL and add it to the database, we need to check if the URL has not been shortened already to prevent duplicate database entries for one URL. We can do this using the
findOneAndUpdate() method on the collection which allows us to modify a document that already exists in the database collection or create it if it doesn’t exist.
This method takes a few arguments. The first one is an object that is used to filter the collection. Here we are passing an object whose
original_url property matches the url we are about to shorten. If a document in the database matches this filter, it will be returned and updated according to the update operators set in the second argument.
$setOnInsert operator allows us to set the value of the document only if its being inserted. This means that the document will not be modified if it already exists, but if it doesn’t, it will be created with its value set to whatever we set in
In this case, the document will have two properties:
original_url which is the url to be shortened, and
short_id which is a unique 7 character id for that url.
We also need to set the
upsert option to
true. This ensures that the document is created if it doesn’t exist. Otherwise,
$setOnInsert has no effect. Finally, setting
false ensures that
findOneAndUpdate returns the new document if one is upserted, which is what we want in this case.
We can make use of the
shortenURL() function in the
/new route like this:
At this point, the document that was inserted will be sent to the client as JSON. It will be displayed on the page like this:
Set up a catch-all route for all shortened URLs
Apart from the
/new routes, we need to handle the other requests for the shortened URLs. Specifically, we need to redirect them to the original URLs. Here’s how we can do so using Express:
Here, we’re using named route parameters to capture the value of the
short_id part of the URL. This value can then be accessed in the
req.params object under the
Once we have the short id, we need to check if a url with that short id exists in the database. Let’s create a new function for this purpose just below
findOne method returns a document that matches the filter object passed to it or
null if no documents match the filter.
We can then use the function in our catch-all route like this:
If the short id exists in our database collection, we will redirect the user to the
original_url associated with that short id. Otherwise, an error message is sent to the user.
Now you should be able to shorten long urls, visit the shortened url and be redirected to the original url.
Visualise your database with a GUI
A GUI will allow us to connect to our MongoDB instance and visualise all the data that is present. You also be able to create new data, update existing data and perform other similar operations.
Once you install and open the app, it should connect to your local MongoDB instance by default. Otherwise, you can click the Connect button in the top left and connect to it from there.
Once connected, you’ll able to view all the collections present in the database, and interact with the data present in those collections.
Deploy to Heroku
Before we deploy the app to Heroku, we need to create a cloud hosted MongoDB instance first. We can do so using MongoDB Atlas. Create an account at that link and create a new cluster when you are redirected to the Create New Cluster page.
Next, click the Security tab and hit the Add new user button. Give the user a name and password, then click the Add user button.
Following that, click IP Whitelist and then Add IP Address. This allows you to choose what IP addresses can access your cluster. Since this is just a demo application, you can allow access from anywhere by clicking the Allow Access From Anywhere button.
Next, go back to the Overview, and hit the Connect button. Choose Connect your Application and then Short SRV connection string. Take note of that string as we’ll be using it shortly on Heroku.
Next, follow the instructions here to install the Heroku CLI on your machine. Then run the
heroku login command in the terminal to login to your Heroku account.
Make sure you’ve initialised a git repository for your project. If not, run the
git init command at the root of your project directory, then run the command below to set heroku as a remote for your git repo. Replace
<app name> with the name of your application.
Next, create a
Procfile in the root of your project directory (
touch Procfile) and paste in the following contents:
Following that, specify the version of Node you are running in your
package.json file under the
engines key. I specified version
11.2.0 since that’s the version I’m running on my computer. You should change that value to match the version of Node you have on your machine.
Before you deploy the app, head over to the Settings tab in the Heroku dashboard and hit Reveal Config Vars. This is where you will set the environmental variables for your app.
As we did earlier on the local
.env file, enter
DATABASE as the KEY and the SRV string from MongoDB Atlas as the value, then click Add. Remember to replace
<PASSWORD> in that string with password for the user you created earlier.
Notice how easy it is to change the database location depending on the environment the program is running without touching the application code. This is one huge advantage of using environmental variables for project configuration.
Finally, commit your code and push it to the Heroku remote using the following commands:
Once the deployment process is done, you can open
https://<your-app-name>.herokuapp.com to view and test your project.
We’ve successfully created a fully featured URL shortener and learnt the basics of Express and MongoDB along the way. We also learnt how to set up a cloud based MongoDB instance on MongoDB Atlas and how to deploy the app to Heroku.
I hope this exercise was helpful to you. If you have any questions regarding this tutorial, please leave a comment below and I’ll get back to you.
Thanks for reading!