Mongo Clustering, integration with Node JS
Updated: Jul 15, 2021
Ever wondered, what will happen if the server that is hosting your database goes down due to an unknown reason. Probably if you are not aware of how scalability works in the case of databases, probably you are losing on the concept of making your application highly available. Clustering is one such concept that can be used which can help you run your database in the clustered mode. By clustering, it means running the database instances on a primary and secondary node. Primary is usually used to read and write the data. The secondary is by default used to replicate the database. The sole purpose of secondary is to keep a copy of your data. Now in case, the primary node goes down, we have something to rely on. Probably if you wish to access the secondary node for reading query, we can surely do so, but the data which we will be getting could be stale. In this article, we will spin up a mongo DB cluster, and connect it with the NodeJS application just to make sure we understand end to end implementation of a cluster. Since the implementation of connecting NodeJS with the mongo cluster is slightly different than connecting it with a single instance of the mongo server. Note: Understanding the concept will help you in excelling in your career as a software developer.
Setting up a mongo cluster
Starting a mongo service on port 2717. Step 1. First, we need to have a folder ready where we are intending to store the data of our database.
On a Linux system or Mac use the following terminal command to create a folder.
> mkdir data1This will help in generating the folder, which will be named as data1.
Step 2. Now since we have our location ready where we intend to keep the data. Now we can start the mongo service.
> mongod --port 2717 --dbpath Cluster/data1/ --replSet replset
- port: Instruct which port we are planning to run mongo daemon process on.
- dbpath: Instruct the location where we want to keep our data. I have created folder named as Cluster and inside that data1 is my folder where I am planning to keep the data.
- replSet: Replica set, has the name of the replica set which are planning to spin up.
Note: Set the dbpath properly, to find the correct path of a folder using a terminal, you can use pwd.
- pwd: Command lets to know about the current location where you are present. We need to start the process for two other nodes of MongoDB. Just to explain in case you are confused, we are planning to run 3 different nodes on a different port of the same system. Just to replicate the original system.
Starting a mongo service on port 2718. Step 3. First, we need to have a folder ready where we are intending to store the data of our database.
On a Linux system or Mac use the following terminal command to create a folder.
> mkdir data2This will help in generating the folder, which will be named as data1.
Step 4. Now since we have our location ready where we intend to keep the data. Now we can start the mongo service.
> mongod --port 2718 --dbpath Cluster/data2/ --replSet replset
- port: Instruct which port we are planning to run mongo daemon process on.
- dbpath: Instruct the location where we want to keep our data. I have created folder named as Cluster and inside that data1 is my folder where I am planning to keep the data.
- replSet: Replica set, has the name of the replica set which are planning to spin up.
Note: Set the dbpath properly, to find the correct path of a folder using a terminal, you can use pwd.
- pwd: Command lets to know about the current location where you are present. Starting a mongo service on port 2719.
Step 5. First, we need to have a folder ready where we are intending to store the data of our database.
On a Linux system or Mac use the following terminal command to create a folder.
> mkdir data3This will help in generating the folder, which will be named as data1.
Step 6. Now since we have our location ready where we intend to keep the data. Now we can start the mongo service.
> mongod --port 2719 --dbpath Cluster/data3/ --replSet replset
- port: Instruct which port we are planning to run mongo daemon process on.
- dbpath: Instruct the location where we want to keep our data. I have created folder named as Cluster and inside that data1 is my folder where I am planning to keep the data.
- replSet: Replica set, has the name of the replica set which are planning to spin up.
Note: Set the dbpath properly, to find the correct path of a folder using a terminal, you can use pwd.
- pwd: Command lets to know about the current location where you are present. Once all three nodes of MongoDB are working. we can now initiate our cluster and add nodes into it.
Step 7. Open any mongo instance with the following command
> mongo --port 2717
- You can use, any of the port 2717/2718/2719
Step 8. Now once you are inside the mongo, now you have to initiate the mongo with the following command.
> rs.initiate({
_id: "replset",
version: 1,
members: [
{ _id: 0, host : "localhost:2717" },
{ _id: 1, host : "localhost:2718" },
{ _id: 2, host : "localhost:2719" }]
})
In this command, we have provided the IP and port of all the members of the replicate set, now our cluster is ready with all the mongo nodes connected. Now you will probably see something like this on your console
replset:PRIMARY >
One of the nodes, either 2717, 2718, or 2719 will becomes primary and the other two will be the secondary nodes. Now we can play around with our mongo cluster, to understand it better. Now we need to check the status of the replica set. Step 9. To check the rs.status(), we can get all the members and other meta-information about the cluster.
replset:PRIMARY > rs.status()
- You can find the members which are part of the cluster.
"members" : [ "members" : [
{
"_id" : 0,
"name" : "localhost:2717",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 6171,
"optime" : {
"ts" : Timestamp(1624473118, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1624473118, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2021-06-23T18:31:58Z"),
"optimeDurableDate" : ISODate("2021-06-23T18:31:58Z"),
"lastHeartbeat" : ISODate("2021-06-23T18:32:04.154Z"),
"lastHeartbeatRecv" : ISODate("2021-06-23T18:32:05.248Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "localhost:2718",
"configVersion" : 1
},
{
"_id" : 1,
"name" : "localhost:2718",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 8439,
"optime" : {
"ts" : Timestamp(1624473118, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2021-06-23T18:31:58Z"),
"electionTime" : Timestamp(1624466964, 1),
"electionDate" : ISODate("2021-06-23T16:49:24Z"),
"configVersion" : 1,
"self" : true
},
{
"_id" : 2,
"name" : "localhost:2719",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 6171,
"optime" : {
"ts" : Timestamp(1624473118, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1624473118, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2021-06-23T18:31:58Z"),
"optimeDurableDate" : ISODate("2021-06-23T18:31:58Z"),
"lastHeartbeat" : ISODate("2021-06-23T18:32:04.154Z"),
"lastHeartbeatRecv" : ISODate("2021-06-23T18:32:05.248Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "localhost:2718",
"configVersion" : 1
}
],
Currently, you can see inside the member that the node having "name": "localhost:2718", is the primary node out of the other two nodes, which are the secondary node.
Now if you kill the Mongo daemon which is running on port 2718, any one of the available two secondaries becomes the primary node.
Now we are able to start the mongo cluster, now next step would be to connect our NodeJS Application with the cluster. Connecting with a single Node JS mongo cluster is slightly different as compared to a cluster since now we have more than one server IP. As we want high availability, so we need to provide all the IP, and in case any server goes down NodeJS is capable enough to connect to another working mongo node. We are using MongoDB client here. The entire source code can be founded here. Click to reach.
const { MongoClient } = require('mongodb');
async function main () {
const uri = "mongodb://localhost:2717,localhost:2718,localhost:2719/?replicaSet=replset";
const client = new MongoClient(uri);
try {
// Connect to the MongoDB cluster
await client.connect();
const result = client.db('employee').collection('office').find()
if (result) {
console.log(await result.toArray())
}
} catch (e) {
console.error(e);
} finally {
await client.close();
}
}
main().catch(console.error);
const uri = "mongodb://localhost:2717,localhost:2718,localhost:2719/?replicaSet=replset"; This is the URL, in which we have given all the Mongo Node and arbiter ports and IP so that our application is aware of all the Mongo nodes in a specific mongo cluster.
On the contrary, when we wanted to connect to a single Mongo Node through our application we simply gave a single IP and port-like, "mongodb://localhost:2717/databaseName"
I hope this will help you in creating a mongo cluster and connecting it with your application.
Comments