82. Section Intro: REST APIs and Mongoose
83. Setting up Mongoose and Creating a Mongoose User Collection
Navigate to the Mongoose website.
Mongoose allows us to model entities, create instances of the entities, and perform CRUD operations on the documents representing the instances in a mongoDB database.
Let’s install the mongoose module in our task-manager app. Change the working directory to the task-manager directory and from the command line, execute the following command.
$ npm i mongoose
Using your IDE, add a task-manager/src directory, a task-manager/src/db directory, and a file task-manager/src/db/mongoose.js.
Inside mongoose.js, lets load the mongoose module and connect to the database. Instead of using the MongoClient object to connect to the database, we’re going to use mongoose’s connect() function.
const mongoose = require('mongoose')
const Schema = mongoose.Schema
mongoose.connect('mongodb://127.0.0.1:27017/task-manager-api').catch(() => {
console.log('error connecting mongoose to the mongodb')
})
Next we’ll create a User model with two fields (name and age) and set the type of data that will be stored in those fields. In mongoose, each schema maps to a MongoDB collection. In this case, the User model will map to a collection named users. Interestingly, I tested this and created a model named Fox and saved an instance of it in the database. Mongoose created a collection named foxes.
const User = mongoose.model('User', new Schema({
name: {
type: String
},
age: {
type: Number
}
}))
We can create instances of the User model by calling the User() function, passing to it an object containing the values for the fields. Each instance of a mongoose model represents a document in the collection that the model is mapped to.
const me = new User({name: 'Eric', age: 28})
We can save the data in the instance as a document in the collection users by using the save() function. The save() function is described here.
me.save().then(() => {
console.log(me)
}).catch((error) => {
console.log('Error', error)
})
When we run mongoose.js, we see that the object printed has a field named __v with a value of 0. This field is the version of the document.
84. Creating a Mongoose Task Collection Exercise
To model the Task entity, create an instance, and save it, we will use the following code.
const Task = mongoose.model('Task', new Schema({
desc: {
type: String
},
completed: {
type: Boolean
}
}))
const task1 = new Task({desc: "FEC Meeting", completed: false})
task1.save().then(() => {
console.log(task1)
}).catch((error) => {
console.log('Error', error)
})
85. Data Validation and Sanitization: Part I
We can use the required validator and create custom validator as shown below.
const User = mongoose.model('User', new Schema({
name: {
type: String,
required: true
},
age: {
type: Number,
validate(value) {
if (value < 0) {
throw new Error('Age must be a positive number')
}
}
}
}))
validator.js is a popular npm module that contains an extensive library of validation functions. Let’s install the module and use it in our code.
$ npm i validator@13.7.0
We can use the validator module’s isEmail() function to verify that the value is correct.
...
const validator = require('validator')
...
const User = mongoose.model('User', new Schema({
name: {
...
},
email: {
type: String,
validate(value) {
if (!validator.isEmail(value)) {
throw new Error('Email is invalid')
}
}
},
...
We can set a default value of a field if no value is provided. We can also convert strings to lower case and trim the whitespace. All these (and other) Schema options are viewable in the Mongoose Schema Types documentation page.
With these schema options include our User entity looks like this.
const User = mongoose.model('User', new Schema({
name: {
type: String,
required: true,
trim: true
},
email: {
type: String,
required: true,
trim: true,
lowercase: true,
validate(value) {
if (!validator.isEmail(value)) {
throw new Error('Email is invalid')
}
}
},
age: {
type: Number,
default: 0,
validate(value) {
if (value < 0) {
throw new Error('Age must be a positive number')
}
}
}
}))
86. Data Validation and Sanitization: Part II
Let’s create a password field in the User collection and configure it so that the password field is required, is at least 7 characters long, and does not contain the string ‘password’.
const User = mongoose.model('User', new Schema({
...
password: {
type: String,
required: true,
trim: true,
minLength: 7,
validate(value) {
if (value.toLowerCase().includes('password')) {
throw new Error('password cannot be password')
}
}
},
...
87. Structuring a REST API
REST = “representational state transfer”.
An API is a set of tools that allows you build an application.
A RESTful API allows clients to access and manipulate resources uses a set of predefined operations.
In practice, the requests are made via HTTP requests to a specific URL on the server. The request will contain the operation and all the data needed by the server in order to fulfill the request.
GET methods from a client are requests for data from the server and may return a 200 status to indicate success. POST methods request that resources be created on the server and may return a 201 status to indicate success.
The Task Resource
-
- Create: POST /task
- Read (all): GET /task
- Read (single): GET /task/:id
- Update: PATCH /task/:id
- Delete: DELETE /task/:id
The structure of an HTTP request is text based with 3 main pieces.
POST /task HTTP/1.1
Accept: application/json
Connection: Keep-Alive
Authorization: Bearer eyJjbGci0iJIUzINiIsInRcCI6IkXVC9.ey...
{"desc":"Order new drill bits"}
-
- The request line which contains the HTTP method, the path, and the HTTP protocol
- 0 or more request headers (e.g. Accept, Connection, Authorization)
- The request body which contains data needed by the server to fulfill the request.
A response looks similar to a request.
HTTP/1.1 201 Created
Date: Sun, 28 Jul 2019 15:37:37 GMT
Server: Express
Content-Type: application/json
{"_id": "5c13ecx6400d1465ed7e5b5", "desc": "Order new drill bits", "completed": false, "__v": 0}
-
- Status line which contains the protocol, status code, and string description of the status code.
- Response headers (e.g. Date, Server, and Content-Type)
- Response body which contains the created resource
88. Installing Postman
What is postman? From their website, “Postman is an API platform for building and using APIs. Postman simplifies each step of the API lifecycle and streamlines collaboration so you can create better APIs—faster”. Postman will allows us to test our server without having to create a client app.
Let’s install postman by navigating to postman.com/downloads, download the appropriate app, and installing it. Note, you do not need to sign up for an account in order to use postman for our purposes (usually a link to skip).
Let’s fire off an HTTP request to ensure it is working properly. First, start up postman. Press + in the left pane, and in the middle pane enter a name your collection (e.g. “weather app”). Then, in the left pane, choose Add a request.
In the URL field enter the following:
https://mead-weather-application.herokuapp.com/weather?address=dayton
Notice that the params fields are filled in automatically. When we press Send, postman will send the HTTP request to the heroku server and will display the response below.
89. Resource Creation Endpoints: Part I
Lets first install nodemon as a dev dependency, and express as a regular dependency.
$ npm i nodemon --save-dev $ npm i express
Inside task-manager/src, create a file named index.js. This will server as the starting point for our application.
In index.js, let’s set up express.
const express = require('express')
const app = express()
const port = process.env.PORT || 3000
app.listen(port, () => {
console.log('Server is up on port ' + port)
})
In package.json, let’s create scripts to run our server both locally and on heroku.
...
"scripts": {
"start": "node src/index.js",
"dev": "nodemon src/index.js"
},
...
Now test that the dev script works.
$ npm run dev
In index.js, let’s add our first api endpoint. After the port declaration, we add a call to app.post(). Post is the HTTP method and ‘/users’ is the path that is included in the URL by the client.
... // port
app.post('/users', (req, res) => {
res.send('testing!')
})
... //listen
In postman, create a new collection for the task-app, and create a new request.
Change the HTTP method to POST, enter the following in the URL field, and press Send.
localhost:3000/users
You should see ‘testing!’ in the response pane.
We will want to test our server by passing data to the server. To do this in postman, we select the Body tab and choose the raw option. When we click on the raw option, a new drop-down appears to the right. Select JSON in the drop down. Then in the editor pane add the following JSON object.
{
"name": "Eric",
"email": "eric@example.com",
"password", "red123!$"
}
Inside index.js, we need to customize express to automatically parse the JSON data passed from the client into an object that we can immediately use.
app.use(express.json())
In the app.post() function call, let’s print out the request’s body.
app.post('/users', (req, res) => {
console.log(req.body)
res.send('testing!')
})
When we send the request via postman, we see in the server’s terminal that req.body is a JSON object representing a new user.
Create a new directory task-manager/src/models.
Inside the models directory, create a file named user.js and a file named task.js. Then move the User object definition from mongoose.js to user.js. Include the mongoose, Schema, and validator constants.
At the bottom of user.js, add the following line to allow other files (like our index.js) to import the User object.
module.exports = User
Similarly, move the Task object definition from mongoose.js to task.js and include the mongoose and Schema constants. Then add a module.exports statement at the bottom of the file.
We can now remove the code in mongoose.js that we don’t need, like the code that creates instances of User and Task, and the code that saves the instances to the database.
Now, in index.js, let’s add code to allow us to use the code in mongoose.js and the User and Task objects. At the top of the file after we’ve required express, add the following code.
require('./db/mongoose')
const User = require('./models/user')
const Task = require('./models/task')
Still inside index.js, let’s add code to create a new user and add it as a document to the users collection when our server receives a new user request. We also change the status of the response to 201 (Created) when successful and 400 (Bad Request) when a new user was not able to be added to the database. Note that we can chain status() and send() together.
app.post('/users', (req, res) => {
const user = new User(req.body)
user.save().then(() => {
res.status(201).send(user)
}).catch((error) => {
res.status(400).send(error)
})
})
When we use postman to send a valid new user request, we see that the new user’s JSON object that is stored in the database is passed back to postman. If we change the data being sent by postman and shorten the password to a length of 2, making it an invalid request, we will see the error object is sent back to postman.
90. Resource Creation Endpoints: Part II
This video asked us to add task.js and the code for the /tasks endpoint. We did this when we working on the previous video.
91. Resource Reading Endpoints: Part I
In this section we add a route for getting a specific user’s info and a route for getting all of the users’ information.
When we navigate to the Queries guide in the mongoose documentation we find a list of methods that we can use on mongoose models (User and Task) to query and manipulate the database.
To find and return multiple documents in the collection we use Model.find(). The first argument to find() is an object that allows us to filter the results. Using an empty object will return all of the documents in the collection associated with the model.
app.get('/users', (req, res) => {
User.find({}).then((users) => {
res.send(users)
}).catch((error) => {
res.status(500).send()
})
})
To retrieve a single document in a collection based on the _id field, we can use Model.getById(). Mongoose can parse the path of the URL and place parts of it (the parts we specify) in the req.params object. We specify the parts we want using the colon character followed by an identifier.
For example, if the client sends a user id in the URL after /users, we can get the id using the following code.
app.get('/users/:id', (req, res) => {
const id = req.params.id
...
})
We can then user User.findByID(id) to find the document associated with the id (if any). If no user is found we send a 404 status code back. If the id cannot be converted to a string an error occurs and we send a 500 status code along with the error.
app.get('/users/:id', (req, res) => {
const id = req.params.id
User.findById(id).then((user) => {
if (!user) {
return res.status(404).send()
}
res.send(user)
}).catch((error) => {
res.status(500).send(error)
})
})
92. Resource Reading Endpoints: Part II
This is a challenge endpoints.
-
- Create an endpoint to fetch all tasks
- Create an endpoint for fetching a task by id
- Setup new requests in Postman and test our work.
This exercise is a simple copy and past of the code in the previous section, replacing user references with task references.
93. Promise Chaining
Suppose we want to mark a task as complete and then get the number of tasks that still need to be completed. There we have two asynchronous tasks that have to happen one after another. To do this, we can us Promise chaining.
Navigate to the playground directory, then open and remove all contents of the 8-promises.js file.
A typical use of Promises looks like this.
// suppose adding two numbers took a while
const add = (arg1, arg2) => {
return new Promise((resolve, reject) => {
setTimeout(() => {
if (typeof arg1 === 'number' && typeof arg2 === 'number') {
resolve(arg1 + arg2);
}
reject('invalid argument')
}, 2000)
})
}
const a = 1, b = 2
add(a,b).then((sum) =>{
console.log('sum: ' + sum)
}).catch((error) => {
console.log(error)
})
Notice that the inside the Promise constructor, resolve() is invoked when it is able to successfully return the sum and reject() is invoked when it is not.
When resolve() is invoked the Promise is created with a then() function. When we call then() and pass it a lambda expression, the Promise passes the sum to the lambda expression. When reject() is invoked, the Promise is created with a catch() function. When we call catch() and pass it a lambda expression, the Promise passes the string ‘invalid argument’ to the lambda expression.
Now, suppose I want to add a third number to the sum of the first two. We can accomplish this with promise chaining. Recall that add() returns a Promise with either a then() function or a catch() function. If add() was successful then the lambda expression that we pass to then() contains the numeric sum of the arguments passed to add(). Inside that lambda express, we can return the Promise returned from a second call to add.
const a = 1, b = 2, c = 3
//const a = 1, b = '2', c = 3
//const a = 1, b = 2, c = '3'
add(a,b).then((sum) => {
console.log('sum: ' + sum)
return add(sum, c)
}).then((sum2) => {
console.log("sum2: " + sum2)
}).catch((e) => {
console.log(e)
})
If the second call to add() is successful, it will return a Promise with a then() function, that we can chain to the initial then() call. If the first call to add() or the second call to add() is unsuccessful, then one of the Promises returned from add() will have a catch() function that will inevitably be executed.
Using mongoose and our models, we can use Promise chaining to change the age of a user and get the number of the users of a given age.
require('./db/mongoose')
const User = require('./models/user')
const userId = '61bf52dbb3362a79380b2c98'
User.findByIdAndUpdate(userId, {
age: 1
}).then((user) => {
console.log(user)
return User.countDocuments({age: 1})
}).then((result) => {
console.log("num docs: " + result)
}).catch((error) => {
console.log(error)
})
94. Promise Chaining Challenge
This is a challenge video.
-
- Create a file promise-chaining-2.js
- Load in mongoose and the task model
- Remove a task by id using Model.findByIdAndDelete()
- Get and print the total number of incomplete tasks
- Test by running
const taskId = '61be95265e5cc5dcbea24d7e'
Task.findByIdAndDelete(taskId).then((task) => {
console.log(task)
return Task.countDocuments({completed: false})
}).then((numDocs) => {
console.log('num docs not completed: ' + numDocs)
}).catch((error) => {
console.log(error)
})
95. Async/Await: Part I
Async/Await is a set of tools that helps us work with Promises.
Let’s create a new playground file named async-await.js and in it add a simple function named doWork() that returns 1 and then print to the console the value returned by doWork().
const doWork = () => {
return 1
}
console.log(doWork())
When we run this code we see that 1 is returned.
Now let’s add the keyword async before the lambda expression.
const doWork = async () => {
return 1
}
console.log(doWork())
Now when we run it we see that a Promise containing the value 1 is returned. Async functions always return a Promises. That Promise is fulfilled with the value returned by the async function.
Since the async function returns a Promise we can use then() and catch() on the Promise.
const doWork = async () => {
return 1
}
doWork().then((result) => {
console.log('result: ' + result)
}).catch((e) => {
console.log('error: ' + e)
})
Here we see that ‘result’ is printed followed by the value 1. If we want to force the catch() function to be invoked, we can throw an error in doWork().
const doWork = async () => {
throw new Error('error in doWork')
}
The await operator only works in async functions. Recall that an async funtion returns a Promise. The await operator is applied on functions that also return Promises, like our add() function created in an earlier video.
In our doWork() function we can call add multiple times using await. For example, if we want to add 3 numbers we can call await add() to add the first two variables. If add() returns a Promise that was unfulfilled, i.e. it called reject(), then the async function stops executing the code within the async function and returns the Promise returned by add(). If add() returns a Promise that was fulfilled then the value returned by resolve() is stored in sum and node continues to execute the code within the async function. The same pattern occurs when the second add() function is called.
const doWork = async () => {
const sum = await add(a, b)
const sum2 = await add(sum, c)
return sum2
}
The complete code example using the add() function and the doWork() function is below. Notice that nothing changes in the call to doWork. We do not need to chain Promises because the await operator handles them internally.
const add = (arg1, arg2) => {
return new Promise((resolve, reject) => {
setTimeout(() => {
if (typeof arg1 === 'number' && typeof arg2 === 'number') {
resolve(arg1 + arg2);
}
reject('invalid argument')
}, 2000)
})
}
const a = 2, b = 3, c = 4
//const a = 2, b = '3', c = 4
//const a = 2, b = 3, c = '4'
const doWork = async () => {
const sum = await add(a, b)
const sum2 = await add(sum, c)
return sum2
}
doWork().then((result) => {
console.log('result: ' + result)
}).catch((e) => {
console.log('error: ' + e)
})
96. Async/Await: Part II
Open promise-chaining.js file. We’re going to rewrite, using async/await, the code that changes a user’s age and returns the number of users with an age of 1.
The new code looks like this.
const updateAgeAndCount = async (userId, newAge) => { const user = await User.findByIdAndUpdate(userId, {age: newAge}) const count = await User.countDocuments({age: newAge}) return { 'user': user, 'count' : count} } const userId = '61bf52dbb3362a79380b2c98' const newAge = 1 updateAgeAndCount(userId, newAge).then((result) => { console.log('user: ' + result.user) console.log('num users: ' + result.count) }).catch((e)=> { console.log('error: ' + e) })
The next challenge is as follows.
-
- Create an async function named deleteTaskAndCount (accept id of task to remove)
- use await to delete the task and count the number of incomplete tasks.
- Return the count
- Call the function and attach then/catch to log results
- Test
The resulting code for the challenge is below.
const deleteTaskAndCount = async (taskId) => {
const task = await Task.findByIdAndDelete(taskId)
const count = await Task.countDocuments({completed: false})
return {'task': task, 'count': count }
}
deleteTaskAndCount(taskId).then((result) => {
console.log('task removed: ' + result.task)
console.log('num incomplete: ' + result.count)
}).catch((e) => {
console.log('error: ' + e)
})
97. Integrating Async/Await
We’re now going to use async/await to rewrite the routes in index.js.
Let’s first rewrite the code to add a new user. The new code is below with the old code commented out.
We first make the lambda expression async since we can only call await inside an async function. When we use the await operator on user.save(), we need to determine the Promise in user.save() was fulfilled or not. We determine this by making the call in a try/catch block. If the Promise was fulfilled, then we set the status to 201 and return the user data to the client. If it was not fulfilled, then the value returned by reject() is passed to the catch clause. In which case we set the status to 300 and return the error message.
app.post('/users', async (req, res) => {
const user = new User(req.body)
try {
await user.save()
res.status(201).send(user)
} catch(error) {
res.status(400).send(error)
}
// user.save().then(() => {
// res.status(201).send(user)
// }).catch((error) => {
// res.status(400).send(error)
// })
})
The new code (and old) for the get all users route is below. Again we make the lambda expression async, use a try/catch block, call User.find() with the await operator, and catch the error if the User.find() Promise is unfulfilled.
app.get('/users', async (req, res) => {
try {
const users = await User.find({})
res.send(users)
} catch (error) {
res.status(500).send()
}
// User.find({}).then((users) => {
// res.send(users)
// }).catch((error) => {
// res.status(500).send()
// })
})
The new and old code for the route that gets a user by id is below.
app.get('/users/:id', async (req, res) => {
const id = req.params.id
try {
const user = await User.findById(id)
if (!user) {
return res.status(404).send()
}
res.send(user)
} catch (error) {
res.status(500).send(error)
}
// User.findById(id).then((user) => {
// if (!user) {
// return res.status(404).send()
// }
// res.send(user)
// }).catch((error) => {
// res.status(500).send(error)
// })
})
The next challenge is to refactor the task routes. The new code for the task routes is similar to the user routes, so I omit the new code here.
98. Resource Updating Endpoints: Part I
Let’s add a route that allows the client to change properties in a user document.
Here we’ll use the PATCH HTTP method and will require the client to send us the user id in the path of the URL and an object holding the properties and new values to be stored in the document.
We use mongoose’s Model.findByIdAndUpdate() function which takes a user id, object containing the changes, and an options object as arguments. We’re going to set the new option to true so that(if successful) the function returns the a User object containing the requested changes. We’re also going to set runValidators to true so that the model validation code is run on the new values.
app.patch('/users/:id', async (req, res) => { try { const user = await User.findByIdAndUpdate( req.params.id, req.body, {new: true, runValidators: true}) if (!user) { return res.status(404).send() } res.send(user) } catch (error) { res.status(400).send(error) } })
Note that if we try to add a property that is not in the User model, the update will fail, but the status code will be 200. If we want to inform the client that the update is not allowed we can add the following code before the try/catch block.
Here we first specify the properties that are allowed to be changed in an array named allowedUpdates. Next, we extract the property names in the object passed in the body and store them in updates. We then declare a variable named isValidOperation which is set to true if every item in updates is in allowedUpdates and is set to false otherwise. Last, we send the client an error if isValidOperation is false.
const allowedUpdates = ['name', 'email', 'password', 'age']
const updates = Object.keys(req.body)
const isValidOperation = updates.every((item) => {
return allowedUpdates.includes(item)
})
if (!isValidOperation) {
return res.status(400).send({error: 'invalid update property'})
}
99. Resource Updating Endpoints: Part II
In this section the challenge is to set up an endpoint that will allow tasks to be updated. The code is very similar to the patch users code in the previous section so we omit it here.
100. Resource Deleting Endpoints
In this section we’ll write routes to delete users and tasks. The code for deleting a user is below.
The only thing of note here is that use the delete function.
app.delete('/users/:id', async (req, res) => {
try {
const user = await User.findByIdAndDelete(req.params.id)
if (!user) {
res.status(404).send()
}
res.send(user)
} catch (error) {
res.status(400).send()
}
})
Code for deleting as task is very similar to the code above, so we omit it here.
101. Separate Route Files
Currently, our server uses theapp object to call app.post(), app.get(), app.patch(), and app.delete() in src/index.js. Express includes a Router() object that we can also call our HTTP methods (post(), get(), etc) on. This will allow us to separate our route handlers into separate files, making our code easier to maintain. For instance, we can create a new directory and file named routes/users.js with the following code.
const express = require('express')
const router = new express.Router()
router.get('/test', (req, res) => {
res.send('This is my new router')
})
module.exports = router
Then in index.js, we simply require routes/users.js and tell the express engine to use the route.
...
const userRouter = require('./routers/user')
const app = express()
...
app.use(userRouter)
...
In our browser, if we navigate to localhost:3000/test, we will see the message, ‘This is my new router’.
Now, we can take all of the user routes out of index.js and move them into routers/users.js. There are a couple of things we need to do in routers/users.js in order to get it all to work as before. First, we need to require models/users and second we need to change the app references to router. The code in routers/users.js should looks similar to below. I’ve included only the post() endpoint to save time. You’d of course, include all 5 route endpoints.
const express = require('express')
const User = require('../models/user')
const router = new express.Router()
// Add a new user
router.post('/users', async (req, res) => {
const user = new User(req.body)
try {
await user.save()
res.status(201).send(user)
} catch(error) {
res.status(400).send(error)
}
})
// Get all users route
// Get a single user by id route
// Update a user by id route
// delete a user route
module.exports = router
The code for routers/task.js is similar and so omitted here.
© 2021 – 2024, Eric McGregor. All rights reserved. Unauthorized use, distribution and/or duplication of this material without express and written permission from the author is strictly prohibited.