Client-side databases remain a sore spot in cross browser offline application development. In one corner, are Safari and Opera ≤ 12. Both of these browsers support Web SQL exclusively. In the other corner, we have Firefox and Internet Explorer (10+), which support IndexedDB exclusively. Chrome (and Opera 15+), for what it’s worth, supports both.
Now this split wouldn’t be so terrible if Web SQL and IndexedDB weren’t radically dissimilar databases with different models for storing data. Supporting both is a nontrivial task. Luckily for us, PouchDB exists.
PouchDB is a client-side database API. It’s written in JavaScript and modeled after the CouchDB API. It’s even capable of synching with a CouchDB instance. We’re interested in PouchDB, however, because it abstracts the differences between Web SQL and IndexedDB, and wraps them in a single interface.
In this article, we’ll get to know PouchDB by building a simple note taking application that works offline. Only portions of the code will be covered here. Many of the functions have been simplified for readability. You can download the entire thing from GitHub.
What You’ll Need
For this project, you will need the following.
- A copy of the PouchDB script
- A web browser that supports IndexedDB or Web SQL. Current versions of Opera, Safari, Internet Explorer, Chrome, and Firefox fit the bill.
- An HTTP server such as Nginx, Lighttpd, or Apache HTTP.
It’s not really necessary, but if you’d like to view the data stored in your local database, use a browser with database inspector tools. Chrome, Opera, and Safari all support database inspection with their native developer tools. The following figure shows a PouchDB database in Chrome.
Because of the origin restrictions baked into IndexedDB and Web SQL, you will also need to use an HTTP server for development. Use whichever server you’d like — Apache, Nginx and Lighttpd are three solid options. Or you can use packages such as MAMP for Mac OS X, WAMP for Windows, or XAMPP for Mac, Windows, and Linux.
Add PouchDB to your HTML document as you would any other JavaScript file:
<script src="pouchdb-nightly.min.js"></script>
Creating a PouchDB Database
All PouchDB databases or database connections are created with the PouchDB
constructor:
var pdb = new PouchDB('pouchnotes');
This creates a database named _pouch_pouchnotes
. PouchDB prefixes each database name with _pouch_
. If you’re also using “raw” IndexedDB or Web SQL for other areas of your web site, avoid using the _pouch_
prefix for those databases.
Planning our application
So what might a note-taking application look like? Well, we probably want each note to have a title. Each note will also have text that makes up the note’s body. We might want to tag our notes too, so we’ll have a field for that. And wouldn’t it be nice if we were able to attach a file? We’ll use an HTML form such as the one below.
We’ll base our database structure on this form.
Designing a Schema (of sorts)
What’s fun about PouchDB is that it has a flexible schema. Each object in the database is really a self-contained document. PouchDB doesn’t use a relational model of data organization, so we can just add fields or properties to a document as we need them.
Rather than the SELECT * FROM tablename
syntax of SQL/relational databases, PouchDB queries use MapReduce. You write functions to filter and sort your data. It requires a bit of a mental shift when compared to SQL, but it’s easy once you get the hang of it. We’ll see an example of this a little later.
Adding and Updating Notes
We’ll add our note to the database when our form is submitted. PouchDB offers two methods for saving a document: post
and put
. Each method accepts two arguments.
document
(required): An object containing properties and their values. In this case, it will be form fields and their values.callback
(optional): A function to call when the operation completes. It accepts two parameters:error
andresponse
.
The primary difference is this: post
adds a new document and generates an identifier (_id
); with put
, we need to supply one. This means you can use put
to add or update documents. But post
is strictly for adding new documents to the database. Now, let’s take a look at an example using put
.
var form, savenote;
form = document.getElementById('addnote');
savenote = function(event) {
var o = {};
o.notetitle = form.notetitle.value;
o.note = form.note.value;
o.tags = form.tags.value;
/* Generate an _id if we don't have one.
It should be a string, which is why we're
adding '' to it. */
if (event.target._id.value == '') {
o._id = new Date().getTime() + '';
} else {
o._id = event.target._id.value;
}
pdb.put(o, function(error, response) {
if (error) {
console.log(error);
return;
} else if(response && response.ok) {
/* Do something with the response. */
}
});
}
/* Add the event handler */
form.addEventListener('submit', savenote);
If there isn’t an _id
value in our form, we’ll generate a timestamp to use for it. Otherwise, we’ll use the value of form._id
. Our other form fields will become properties and values for our document object. By using put
rather than post
, we can use our savenote
function both to add and update notes.
If all goes well, our callback will receive a JSON-formatted response. An example of a successful response is shown below.
{ok: true, id: "1391406871281", rev: "1-1d95025598a94304a87ef14c108db7be"}
We haven’t done anything with our response. Depending on your application, you may not want to. But, for our note-taking application, we want the ability to associate a file with a note. PouchDB calls these files attachments.
Saving Attachments
Saving an attachment is little more complicated than saving text. We can’t just query the value
attribute of the input type="file"
field. Instead, we have to read the file data using the File API, then save it using PouchDB’s putAttachment
method. Let’s add to our savenote
method from the previous section.
savenote = function(event) {
var o = {};
o.notetitle = form.notetitle.value;
o.note = form.note.value;
o.tags = form.tags.value;
/* Generate an _id if we don't have one.
It should be a string, which is why we're
adding '' to it. */
if (event.target._id.value == '') {
o._id = new Date().getTime() + '';
} else {
o._id = event.target._id.value;
}
pdb.put(o, function(error, response) {
if (error) {
console.log(error);
return;
}
/* New code for saving attachments */
if (response && response.ok) {
if (form.attachment.files.length) {
var reader = new FileReader();
/* Using a closure so that we can extract the
File's attributes in the function. */
reader.onload = (function(file) {
return function(e) {
pdb.putAttachment(response.id, file.name, response.rev, e.target.result, file.type);
};
})(form.attachment.files.item(0));
reader.readAsDataURL(form.attachment.files.item(0));
}
}
});
}
Every file input type also has a files
attribute that returns a FileList
object. In this case, that’s form.attachment.files
. As its name suggests, a FileList
object is an array containing the file or files submitted using that field. We can determine the number of files in the list with the length
property. Each file in the list can be referenced using its index and the item
method, as we’ve done here (form.attachment.files.item(0)
). Alternatively, you can use square bracket syntax (form.attachment.files[0]
).
If the note is added successfully, we’ll get a response.id
. Then, we can check to see whether or not there’s also a file to save as an attachment. If there is, we will read it using a FileReader
object (var reader = new FileReader()
). PouchDB attachments must be base64-encoded. The easiest way to encode files is to use the readAsDataURL()
. Once the file loads, we can save it to the database using putAttachment
.
PouchDB’s putAttachment
method accepts up to six arguments. Five are required, one is optional.
docID
(required): The identifier for the document that this attachment will be associated with. In this case, it’sresponse.id
.Attachment ID
(required): The name of the attachment. Here we’re using the file’s name.rev
(required): The parent document’s revision number.attachment_doc
(required): The base64-encoded file data. In this case theresult
property of ourFileReader
object.type
(required): The MIME type for this data. For example,image/png
orapplication/pdf
.callback
(optional): the function to invoke when the operation completes. As with all PouchDB callback functions, it accepts two arguments,error
andresponse
. We’ve left it out in our example.
In this example, we also wrapped our onload
event handler within a closure. The closure makes it possible to access our file properties from within our event handler (for example, with file.name
and file.type
).
Now that we’ve looked at saving notes and attachments, let’s look at retrieving records, both individually and in sets.
Retrieving All Notes
What if we want to view a list of the notes in our database? This is where PouchDB’s allDocs
is useful. PouchDB.allDocs
lets us retrieve a batch of documents at a time.
The name allDocs
is a bit misleading. We certainly can use it to retrieve all of our documents. However, we can also use it to retrieve documents that fall within a certain range, or retrieve documents that match particular keys. This method accepts two arguments, neither of which are required.
options
(optional): An object containing one or more of the following properties.include_docs
(Boolean): Include the entire document for each row. Whenfalse
, will only return the document’sid
andrev
number.
*conflicts
(Boolean): Include conflicts.startkey
andendkey
: Include documents with keys within this range.descending
(boolean): Sort the results in descending order instead.
*options.keys
(array): Return only documents matching the specified keys.
*options.attachments
(Boolean): Return attachments with documents.
*callback
(optional): A function to call when when the retrieval completes. As with other PouchDB callbacks, it receives anerror
argument, and aresponse
argument.
In the simplified example below, we’ve retrieved all of the documents in our database. In order to retrieve the document title, created date, and modified date, we need to set the value of include_docs
to true
. Here’s our viewnoteset
function.
var viewnoteset = function() {
var df = document.createDocumentFragment(),
options = {},
nl = document.querySelector('#notelist tbody');
options.include_docs = true;
this.pdb.allDocs(options, function(error, response) {
var row = response.rows.map(addrow); // Calls an addrow() function
row.map(function(f) {
if (f) {
df.appendChild(f);
}
});
nl.appendChild(df);
});
};
The value of response
is an object containing three properties: total_rows
, offset
, and rows
. We’re most interested in response.rows
, since it is an array of document objects. Here we’ve used map
, one of JavaScript’s built-in array methods, on response.rows
. Using map
invokes our addrow
function for every note, and adds it to the table that lists our notes.
Retrieving Individual Notes
Retrieving an individual note is a bit easier, as we can use PouchDB’s get
method. The only required argument is the document ID. We can, however, include an options
argument and a callback function to handle the results.
Our options argument, {attachments: true}
ensures that if a particular note has any attachments, it will be shown with the note when viewed. Here, our callback function takes our note data and uses it to fill in our form fields and show any attachment.
var viewnote = function(noteid) {
var noteform = document.querySelector('#noteform');
pdb.get(noteid, {attachments: true}, function(error, response) {
var fields = Object.keys(response), o, link, attachments, li;
if (error) {
return;
} else {
/* Since our note field names and form field names match,
We can just iterate over them. */
fields.map(function(f) {
if (noteform[f] !== undefined && noteform[f].type != 'file') {
noteform[f].value = response[f];
}
if (f == '_attachments') {
attachments = response[f];
for (o in attachments) {
li = document.createElement('li');
link = document.createElement('a');
link.href = 'data:' + attachments[o].content_type + ';base64,' + attachments[o].data;
link.target = "_blank";
link.appendChild(document.createTextNode(o));
li.appendChild(link);
}
document.getElementById('attachmentlist').appendChild(li);
}
});
}
});
}
In our demo application, we’re passing the id
for each note using a link. Each href
points to /#/view/xxxxx
where xxxxx
is the note id
. Clicking a link triggers a hashchange
event, and the hashchange
event handler (shown below) is where we pass the id
to viewnote
.
window.addEventListener('hashchange', function(e) {
var noteid;
/* Replacing # for compatibility with IE */
if (window.location.hash.replace(/#/,'')) {
noteid = window.location.hash.match(/\d/g).join('');
viewnote(noteid);
}
});
Making Notes Searchable
Notes are especially useful when they are searchable. So, let’s add a search feature to our application. We will take input from our search form, and use it as the basis for our search query. The following figure shows what our application will look like when we use the search feature.
PouchDB queries look very different from SQL. With SQL, you specify what to select, from which table, and according to what criteria. For example, a simple note search query might look like this: SELECT * FROM notes WHERE title, text, tags LIKE %interview%
. But with PouchDB we run queries using functions.
To run a query, we’ll use PouchDB’s query
method. It accepts three arguments.
fun
(required): The name of a function.options
(optional): An object containing options for the search results. You can specify a reduce function or restrict results to a specific key or range of keys.callback
(optional): A function to call when the query completes.
Let’s look at our search function below.
var search = function(searchkey) {
var map = function(doc) {
var searchkey, regex;
/* Escape characters with special RegExp meaning */
searchkey = document.getElementById('q').value.replace(/[$-\/?[-^{|}]/g, '\\$&');
regex = new RegExp(searchkey,'i');
/* If the notetitle, note, or tags fields match,
return only the fields we need to create the result list. */
if (doc.notetitle.match(regex) ||
doc.note.match(regex) ||
doc.tags.match(regex)) {
emit(doc._id, {notetitle: doc.notetitle, note: doc.note, tags: doc.tags});
}
}
db.query(map, function(err, response) {
if (err) {
console.log(err);
}
if (response) {
var df, rows, nl, results;
/* Rewrite the response so that our object has the
correct structure for our addrow function. */
results = response.rows.map(function(r) {
r.doc = r.value;
delete r.value;
return r;
});
nl = document.querySelector('#notelist tbody');
df = document.createDocumentFragment(),
rows = results.map(addrow, that);
rows.map(function(f) {
if (f) {
df.appendChild(f);
}
});
nl.innerHTML = '';
nl.appendChild(df);
}
});
}
Within our search function, we’ve defined a map
function which is how we find and filter our records. The map
function always receives a PouchDB document as its sole argument. We don’t have to name this function map
, but it must be the first argument.
Within map
, we’ve created a regular expression object from our search form input. We’ll test our notetitle
, note
, and tags
fields, to see if any of these fields match our regular expression. If they do, we’ll return the notetitle
, id
(which is a time stamp), and modified properties using the emit
method. The emit
method is built into PouchDB. As its name suggests, it selects and returns the properties specified, in the format specified. The first argument of emit
becomes the key for our results.
Our map
function becomes the first argument for query
. And the second argument for query
is — as you’ve probably guessed — a callback function. Assuming everything went okay, our response
argument will be an object containing three properties: total_rows
, offset
, and rows
. We want rows
. It’s an array containing the notes that match our search term. The following code sample shows what a response might look like.
[{
value: {
id: "1388703769529",
modified: 1391742787884,
notetitle: "Fluffernutter sandwich recipe"
},
id:"1388703769529",
key:"1388703769529"
},
{
value: {
id: "1391656570611",
modified: 1391656570611,
notetitle: "Browned-butter Rice Krispie Treats recipe"
},
id:"1391656570611",
key:"1391656570611"
}]
Because our response is an array, we can use native Array.prototype
methods to manipulate the results. In this case, we’ve used Array.prototype.map
to rewrite each note object so that our value
property becomes doc
instead, and again to invoke addrow
for every result.
Working Offline with Application Cache
To make this application work completely off-line, we also need to save the HTML, CSS, and JavaScript offline using Application Cache. Application Cache is a plain text file, served with a Content-type: text/cache-manifest
header that tells the browser which assets to store locally. We won’t do a “deep-dive” on Application Cache here, but let’s look at the manifest file, pouchnotes.cache
, for our demo application.
CACHE MANIFEST
# Version 2014.02.10.01
CACHE:
index.html
css/style.css
js/pouchdb-nightly.min.js
js/application.js
We’ve started it with the line CACHE MANIFEST
, which is how all cache manifests must begin. The second line tells us what version of the file this is. Browsers will only update the cache when the cache manifest changes. Changing the version number is the easiest way to trigger an update should we modify our CSS, JavaScript, or HTML files.
We still need to do one more thing, though. We need to add our manifest to our HTML document. That requires adding a manifest
attribute to our <html>
tag, like this:
<html lang="en-us" manifest="pouchnotes.manifest">
Now our database and our files will be available even when we’re offline.
Be forewarned: Application Cache adds a layer of development complexity. Because the cache manifest must change in order for the browser to download new files, you should wait until you’re ready to release a version of your application before adding it.
Conclusion
There’s more to PouchDB that we haven’t covered here. For example, you can synchronize PouchDB with a CouchDB server. Synchronizing with a database server lets us build applications that can easily share data and files across multiple browsers and computers.
I hope this article has left you with an understanding of what PouchDB is and how you can use it to build software that works even when our Internet connection does not.