OK. So that title is a bold claim. And this is a read-and-follow-along sort of tutorial. So, it might be 6 minutes or 4 minutes depending on how fast you type. Regardless, this should get you building something useful quickly and could make a nice foundation for something bigger.
Oh, and you might be wondering what Redis OM is. Well, there's an extensive README on GitHub. Go check it out!
Also, this document, and the code that we're about to implement, and the data needed to test it are all out on GitHub. Refer to them as you need.
So, what are we going to build? We're going to build a RESTful service that lets you manage songs. It'll let you do all the CRUD things (that's create, read, update, and delete for the uninitiated) with songs. Plus, we'll add some cool search endpoints to the service as well. That way, we can find songs by an artist or genre, from a particular year, or with certain lyrics.
Test data for this problem was a little tricky. Most song lyrics are copyrighted and getting permission to use them for a little old tutorial wasn't really an option. And we definitely want to be able to search on song lyrics. How else are we going to find that song that goes "oooo ah ah ah ah"?
Fortunately, my buddy Dylan Beattie is literally the original Rockstar developer. In addition to coding cool things, he writes parody songs with tech themes. And, he has given me permission to use them as test data.
We're using Redis as our database—that's the whole idea behind Redis OM. So, you'll need some Redis, specifically with Search and JSON installed. The easiest way to do this is to set up a free Redis Cloud instance. But, you can also use Docker:
$ docker run -p 6379:6379 redis/redis-stack:latestI'm assuming you are relatively Node.js savvy so you should be able to get that installed on your own. We'll be using the top-level await feature for modules that was introduced in Node v14.8.0 so do make sure you have that version, or a newer one. If you don't, go and get it.
Once you have that, it's time to create a project:
$ npm initGive it a name, version, and description. Use whatever you like. I called mine "Metalpedia".
Install Express and Redis OM for Node.js:
$ npm install express redis-om --saveAnd, just to make our lives easy, we'll use nodemon:
$ npm install nodemon --save-devNow that stuff is installed, let's set up some other details in our package.json. First, set the "type" to "module", so we can use ES6 Modules:
"type": "module",The "test" script that npm init generates isn't super useful for us. Replace that with a "start" script that calls nodemon. This will allow the service we build to restart automatically whenever we change a file. Very convenient:
"scripts": {
"start": "nodemon server.js"
},I like to make my packages private, so they don't accidentally get pushed to NPM:
"private": true,Oh, and you don't need the "main" entry. We're not building a package to share. So go ahead and remove that.
Now, you should have a package.json that looks something like this:
{
"name": "metalpedia",
"version": "1.0.0",
"description": "Sample application for building a music repository backed by Redis and Redis OM.",
"type": "module",
"scripts": {
"start": "nodemon server.js"
},
"author": "Guy Royse <guy@guyroyse.com> (http://guyroyse.com/)",
"license": "MIT",
"private": true,
"dependencies": {
"express": "^4.17.1",
"redis-om": "^0.2.0"
},
"devDependencies": {
"nodemon": "^2.0.14"
}
}Excellent. Set up done. Let's write some code!
I like to write my services with a little version and name endpoint at the root. That way if some random developer hits the site of the service, they'll get a clue as to what it is. So let's do that:
Create a file named server.js in the root of your project folder and populate it thus:
import express from 'express';
// create an express app and use JSON
let app = new express();
app.use(express.json());
// setup the root level GET to return name and version from package.json
app.get('/', (req, res) => {
res.send({
name: process.env.npm_package_name,
version: process.env.npm_package_version,
});
});
// start listening
app.listen(8080);We now have enough to actually run something. So let's run it:
$ npm startThen, hit http://localhost:8080/ in your favorite browser. You should see something like this:
{
"name": "metalpedia",
"version": "1.0.0"
}Or, hit your service using curl (and json_pp if you want to be fancy):
$ curl -X GET http://localhost:8080 -s | json_pp
{
"name": "metalpedia",
"version": "1.0.0"
}Cool. Let's add some Redis.
We're going to use Redis OM to map data for a song from JSON data in Redis to JavaScript objects.
Create a file named song-repository.js in the root of your project folder. In it, import all the parts from Redis OM that you'll need:
import { Entity, Schema, Client, Repository } from 'redis-om';Entities are the classes that you work with—the thing being mapped to. They are what you create, read, update, and delete. Any class that extends Entity is an entity. We'll define our Song entity with a single line for now, but we'll add some more to it later:
class Song extends Entity {}Schemas define the fields on your entity, their types, and how they are mapped internally to Redis. By default, entities map to Hashes in Redis but we want ours to use JSON instead. When a Schema is created, it will add properties to the provided entity class based on the schema information provided. Here's a Schema that maps to our Song:
let schema = new Schema(Song, {
title: { type: 'string' }, // the title of the song
artist: { type: 'string' }, // who performed the song
genres: { type: 'string[]' }, // array of strings for the genres of the song
lyrics: { type: 'text' }, // the full lyrics of the song
music: { type: 'text' }, // who wrote the music for the song
year: { type: 'number' }, // the year the song was releases
duration: { type: 'number' }, // the duration of the song in seconds
link: { type: 'string' }, // link to a YouTube video of the song
});Clients are used to connect to Redis. Create a Client and pass your Redis URL in the constructor. If you don't specify a URL, it will default to redis://localhost:6379. Clients have methods to .open, .close, and .execute raw Redis commands, but we're just going to open it:
let client = await new Client().open();Remember that top-level await stuff I mentioned at the top of the document? There it is!
Now we have all the pieces that we need to create a Repository. Repositories are the main interface into Redis OM. They give us the methods to read, write, and remove entities. Create a repository—and make sure it's exported as you'll need it when we get into the Express stuff:
export let songRepository = client.fetchRepository(schema);We're almost done with setting up our repository. But we still need to create an index or we won't be able to search on anything. We do that by calling .createIndex. If an index already exists and it's the same, this function won't do anything. If it is different, it'll drop it and create a new one. In a real environment, you'd probably want to create your index as part of CI/CD. But we'll just cram them into our main code for this example:
await songRepository.createIndex();We have what we need to talk to Redis. Now, let's use it to make some routes in Express.
Let's create a truly RESTful API with the CRUD operations mapping to PUT, GET, POST, and DELETE respectively. We're going to do this using Express Routers as this makes our code nice and tidy. So, create a file called song-router.js in the root of your project folder. Then add the imports and create a Router:
import { Router } from 'express';
import { songRepository as repository } from './song-repository.js';
export let router = Router();This router needs to be added in server.js under the /song path so let's do that next. Add the following line of code to at the top of server.js—with all the other imports—to import the song router:
import { router as songRouter } from './song-router.js';Also add a line of code to call .use so that the router we are about to implement is, well, used:
app.use('/song', songRouter);Our server.js should now look like this:
import express from 'express';
import { router as songRouter } from './song-router.js';
// create an express app and use JSON
let app = new express();
app.use(express.json());
// bring in some routers
app.use('/song', songRouter);
// setup the root level GET to return name and version from package.json
app.get('/', (req, res) => {
res.send({
name: process.env.npm_package_name,
version: process.env.npm_package_version,
});
});
// start listening
app.listen(8080);Now, let's start putting some routes in our song-router.js. We'll create a song first as you need to have songs in Redis before you can do any of the reading, updating, or deleting of them. Add the PUT route below. This route will call .createEntity to create an entity, set all the properties on the newly created entity, and then call .save to persist it:
router.put('/', async (req, res) => {
// create the Song so we can save it
let song = repository.createEntity();
// set all the properties, converting missing properties to null
song.title = req.body.title ?? null;
song.artist = req.body.artist ?? null;
song.genres = req.body.genres ?? null;
song.lyrics = req.body.lyrics ?? null;
song.music = req.body.music ?? null;
song.year = req.body.year ?? null;
song.duration = req.body.duration ?? null;
song.link = req.body.link ?? null;
// save the Song to Redis
let id = await repository.save(song);
// return the id of the newly created Song
res.send({ id });
});Now that we have a way to shove songs into Redis, let's start shoving. Out on GitHub, there are a bunch of JSON files with song data in them. (Thanks Dylan!) Go ahead and pull those down and place them in a folder under your project root called songs.
Let's use curl to load in a song. I'm partial to HTML, sung to the tune of AC/DC's , so let's use that one:
$ curl -X PUT -H "Content-Type: application/json" -d "@songs/html.json" http://localhost:8080/song -s | json_ppYou should get back the ID of that newly inserted song:
{
"id": "01FKRW9WMVXTGF71NBEM3EBRPY"
}We're shipping HTML indeed. If you have the redis-cli handy—or want to use RedisInsight—you can take a look and see how Redis has stored this:
> json.get Song:01FKRW9WMVXTGF71NBEM3EBRPY
"{\"title\":\"HTML\",\"artist\":\"Dylan Beattie and the Linebreakers\",\"genres\":[\"blues rock\",\"hard rock\",\"parody\",\"rock\"],\"lyrics\":\"W3C, RFC, a JIRA ticket and a style guide.\\\\nI deploy with FTP, run it all on the client side\\\\nDon\xe2\x80\x99t need Ruby, don\xe2\x80\x99t need Rails,\\\\nAin\xe2\x80\x99t nothing running on my stack\\\\nI\xe2\x80\x99m hard wired, for web scale,\\\\nYeah, I\xe2\x80\x99m gonna bring the 90s back\\\\n\\\\nI\xe2\x80\x99m shipping HTML,\\\\nHTML,\\\\nI\xe2\x80\x99m shipping HTML,\\\\nHTML\xe2\x80\xa6\\\\n\\\\nNo logins, no trackers,\\\\nNo cookie banners to ignore\\\\nI ain\xe2\x80\x99t afraid of, no hackers\\\\nJust the occasional 404\\\\nThey hatin\xe2\x80\x99, what I do,\\\\nBut that\xe2\x80\x99s \xe2\x80\x98cos they don\xe2\x80\x99t understand\\\\nMark it up, break it down,\\\\nRemember to escape your ampersands\xe2\x80\xa6\\\\n\\\\nI\xe2\x80\x99m shipping HTML,\\\\nHTML,\\\\nI\xe2\x80\x99m shipping HTML,\\\\nHTML\xe2\x80\xa6\\\\n\\\\n(But it\xe2\x80\x99s really just markdown.)\",\"music\":\"\\\"Highway to Hell\\\" by AC/DC\",\"year\":2020,\"duration\":220,\"link\":\"https://www.youtube.com/watch?v=woKUEIJkwxI\"}"Yep. Looks like JSON.
Create down, let's add a GET route to read this song from HTTP instead of using the redis-cli:
router.get('/:id', async (req, res) => {
// fetch the Song
let song = await repository.fetch(req.params.id);
// return the Song we just fetched
res.send(song);
});Now you can use curl or your browser to load http://localhost:8080/song/01FKRW9WMVXTGF71NBEM3EBRPY to fetch the song:
$ curl -X GET http://localhost:8080/song/01FKRW9WMVXTGF71NBEM3EBRPY -s | json_ppAnd you should get back the JSON for the song:
{
"link": "https://www.youtube.com/watch?v=woKUEIJkwxI",
"genres": ["blues rock", "hard rock", "parody", "rock"],
"entityId": "01FKRW9WMVXTGF71NBEM3EBRPY",
"title": "HTML",
"lyrics": "W3C, RFC, a JIRA ticket and a style guide.\\nI deploy with FTP, run it all on the client side\\nDon’t need Ruby, don’t need Rails,\\nAin’t nothing running on my stack\\nI’m hard wired, for web scale,\\nYeah, I’m gonna bring the 90s back\\n\\nI’m shipping HTML,\\nHTML,\\nI’m shipping HTML,\\nHTML…\\n\\nNo logins, no trackers,\\nNo cookie banners to ignore\\nI ain’t afraid of, no hackers\\nJust the occasional 404\\nThey hatin’, what I do,\\nBut that’s ‘cos they don’t understand\\nMark it up, break it down,\\nRemember to escape your ampersands…\\n\\nI’m shipping HTML,\\nHTML,\\nI’m shipping HTML,\\nHTML…\\n\\n(But it’s really just markdown.)",
"duration": 220,
"artist": "Dylan Beattie and the Linebreakers",
"music": "\"Highway to Hell\" by AC/DC",
"year": 2020
}Now that we can read and write, let's implement the REST of the HTTP verbs. REST... get it?
Here's the code to update using a POST route. You'll note this code is nearly identical to the GET route. Feel free to refactor to a helper function but since this is just a tutorial, I'll skip that for now.:
router.post('/:id', async (req, res) => {
// fetch the Song we are replacing
let song = await repository.fetch(req.params.id);
// set all the properties, converting missing properties to null
song.title = req.body.title ?? null;
song.artist = req.body.artist ?? null;
song.genres = req.body.genres ?? null;
song.lyrics = req.body.lyrics ?? null;
song.music = req.body.music ?? null;
song.year = req.body.year ?? null;
song.duration = req.body.duration ?? null;
song.link = req.body.link ?? null;
// save the Song to Redis
let id = await repository.save(song);
// return the id of the Song we just saved
res.send({ id });
});And the curl command to try it out, replacing Dylan's with —sung to the tune of by the Village People:
$ curl -X POST -H "Content-Type: application/json" -d "@songs/d-m-c-a.json" http://localhost:8080/song/01FKRW9WMVXTGF71NBEM3EBRPY -s | json_ppYou should get back the ID of that updated song:
{
"id" : "01FKRW9WMVXTGF71NBEM3EBRPY"
}And, finally, let's implement a DELETE route:
router.delete('/:id', async (req, res) => {
// delete the Song with its id
await repository.remove(req.params.id);
// respond with OK
res.type('application/json');
res.send('OK');
});And test it out:
$ curl -X DELETE http://localhost:8080/song/01FKRW9WMVXTGF71NBEM3EBRPY -s
OKThis just returns "OK", which is technically JSON but aside from the response header, is indistinguishable from plain text.
All the CRUD is done. Let's add some search. Search is where Redis OM really starts to shine. We're going to create routes to:
Before we get started, let's load up Redis with a bunch of songs—so we have stuff to search for. I've written a short shell script that loads all the song data on GitHub into Redis using the server we just made. It just calls curl in a loop. It's on GitHub, so go grab it and put it in your project root. Then run it:
$ ./load-data.shYou should get something like:
{"id":"01FM310A8AVVM643X13WGFQ2AR"} <- songs/big-rewrite.json
{"id":"01FM310A8Q07D6S7R3TNJB146W"} <- songs/bug-in-the-javascript.json
{"id":"01FM310A918W0JCQZ8E57JQJ07"} <- songs/d-m-c-a.json
{"id":"01FM310A9CMJGQHMHY01AP0SG4"} <- songs/enterprise-waterfall.json
{"id":"01FM310A9PA6DK4P4YR275M58X"} <- songs/flatscreens.json
{"id":"01FM310AA2XTEQV2NZE3V7K3M7"} <- songs/html.json
{"id":"01FM310AADVHEZXF7769W6PQZW"} <- songs/lost-it-on-the-blockchain.json
{"id":"01FM310AASNA81Y9ACFMCGP05P"} <- songs/meetup-2020.json
{"id":"01FM310AB4M2FKTDPGEEMM3VTV"} <- songs/re-bass.json
{"id":"01FM310ABFGFYYJXVABX2YXGM3"} <- songs/teams.json
{"id":"01FM310ABW0ANYSKN9Q1XEP8BJ"} <- songs/tech-sales.json
{"id":"01FM310AC6H4NRCGDVFMKNGKK3"} <- songs/these-are-my-own-devices.json
{"id":"01FM310ACH44414RMRHPCVR1G8"} <- songs/were-gonna-build-a-framework.json
{"id":"01FM310ACV8C72Y69VDQHA12C1"} <- songs/you-give-rest-a-bad-name.jsonNote that this script will not erase any data. So any songs that you have in there already will still be there, alongside these. And if you run this script more than once, it will gleefully add the songs a second time.
Like with the CRUD operations for songs, we need to first create a router. This time we'll name the file songs-router.js. Note the plural. Add all the imports and exports to that file like before:
import { Router } from 'express';
import { songRepository as repository } from './song-repository.js';
export let router = Router();Add this router to Express in server.js under /songs, also like we did before. And, again, note the plural. Your server.js should now look like this:
import express from 'express';
import { router as songRouter } from './song-router.js';
import { router as songsRouter } from './songs-router.js';
// create an express app and use JSON
let app = new express();
app.use(express.json());
// bring in some routers
app.use('/song', songRouter);
app.use('/songs', songsRouter);
// setup the root level GET to return name and version from package.json
app.get('/', (req, res) => {
res.send({
name: process.env.npm_package_name,
version: process.env.npm_package_version,
});
});
// start listening
app.listen(8080);Now we can add some search routes. We initiate a search by calling .search on our repository. Then we call .where to add any filters we want—if we want any at all. Once we've specified the filters, we call .returnAll to get all the matching entities.
Here's the simplest search—it just returns everything. Go ahead and add it to songs-router.js:
router.get('/', async (req, res) => {
let songs = await repository.search().returnAll();
res.send(songs);
});Then try it out with curl or your browser:
$ curl -X GET http://localhost:8080/songs -s | json_ppWe can search for a specific field by calling .where and .eq. This route finds all songs by a particular artist. Note that you must specify the complete name of the artist for this to work:
router.get('/by-artist/:artist', async (req, res) => {
let artist = req.params.artist;
let songs = await repository.search().where('artist').eq(artist).returnAll();
res.send(songs);
});Then try it out with curl or your browser too:
$ curl -X GET http://localhost:8080/songs/by-artist/Dylan%20Beattie -s | json_ppGenres are stored as an array of strings. You can use .contains to see if the array contains that genre or not:
router.get('/by-genre/:genre', async (req, res) => {
let genre = req.params.genre;
let songs = await repository
.search()
.where('genres')
.contains(genre)
.returnAll();
res.send(songs);
});And try it out:
$ curl -X GET http://localhost:8080/songs/by-genre/rock -s | json_pp
$ curl -X GET http://localhost:8080/songs/by-genre/parody -s | json_ppThis route lets you get all the songs between two years. Great for finding all those 80s hits. Of course, all of Dylan's songs are more recent than that, so we'll go a little more narrow when we try it out:
router.get('/between-years/:start-:stop', async (req, res) => {
let start = Number.parseInt(req.params.start);
let stop = Number.parseInt(req.params.stop);
let songs = await repository
.search()
.where('year')
.between(start, stop)
.returnAll();
res.send(songs);
});And, try it out, of course:
$ curl -X GET http://localhost:8080/songs/between-years/2020-2021 -s | json_ppLet's add the final route to find songs that have certain words in the lyrics using .match:
router.get('/with-lyrics/:lyrics', async (req, res) => {
let lyrics = req.params.lyrics;
let songs = await repository
.search()
.where('lyrics')
.match(lyrics)
.returnAll();
res.send(songs);
});We can try this out too, getting all the songs that contain both the words "html" and "markdown":
$ curl -X GET http://localhost:8080/songs/with-lyrics/html%20markdown -s | json_ppAnd that's a wrap. I've walked you through some of the basics with this tutorial. But you should totally go deeper. If you want to learn more, go ahead and check out Redis OM for Node.js on GitHub. It explains the capabilities of Redis OM for Node.js in greater detail.
If you have any questions or are stuck, feel free to jump on the Redis Discord server and ask there. I'm always hanging out and happy to help.
And, if you find a flaw, bug, or just think this tutorial could be improved, send a pull request or open an issue.
Thanks!
