Wednesday, March 24, 2010

Getting Started with node.js and CouchDB

‹prev | My Chain | next›

Tonight, I would like to get started with node.js. First downloading and installing:
cstrom@whitefall:~/tmp$ wget http://nodejs.org/dist/node-v0.1.33.tar.gz
--2010-03-24 20:51:40-- http://nodejs.org/dist/node-v0.1.33.tar.gz
Resolving nodejs.org... 97.107.132.72
Connecting to nodejs.org|97.107.132.72|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4016600 (3.8M) [application/octet-stream]
Saving to: `node-v0.1.33.tar.gz'

100%[======================================================================================================>] 4,016,600 142K/s in 26s

2010-03-24 20:52:07 (151 KB/s) - `node-v0.1.33.tar.gz' saved [4016600/4016600]
I store source code repositories in the repos directory, but code without source code management (e.g. tarballs) in my src directory:
cstrom@whitefall:~/tmp$ cd /home/cstrom/src/
cstrom@whitefall:~/src$ tar zxf ../tmp/node-v0.1.33.tar.gz
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
Not sure what's up with the "unknown extended header keyword", but stuff was created:
cstrom@whitefall:~/src/node-v0.1.33$ ls
AUTHORS benchmark bin ChangeLog configure deps doc lib LICENSE Makefile README src test tools wscript
Since I am just messing around, I do not want to use sudo to install. For now, I will install into my "local" directory:
cstrom@whitefall:~/src/node-v0.1.33$ ./configure --prefix=/home/cstrom/local
Check for program g++ or c++ : /usr/bin/g++
Check for program cpp : /usr/bin/cpp
Check for program ar : /usr/bin/ar
Check for program ranlib : /usr/bin/ranlib
Checking for g++ : ok
...
There were a few missing development libraries. Configure completed OK without them, but I note them for later:
...
Checking for library execinfo : not found
Checking for gnutls >= 2.5.0 : fail
...
--- libev ---
...
Checking for header port.h : not found
Checking for header poll.h : ok
Checking for function poll : ok
Checking for header sys/event.h : not found
Checking for header sys/queue.h : ok
Checking for function kqueue : not found
...
I would not expect them to matter because configure completes successfully:
...
creating config.h... ok
creating Makefile... ok
creating config.status... ok
all done.
'configure' finished successfully (9.562s)
With that, I can build the executables:
cstrom@whitefall:~/src/node-v0.1.33$ make
Waf: Entering directory `/home/cstrom/src/node-v0.1.33/build'
...
[22/23] cxx: src/node_idle_watcher.cc -> build/default/src/node_idle_watcher_7.o
[23/23] cxx_link: build/default/src/node_7.o build/default/src/node_child_process_7.o build/default/src/node_constants_7.o build/default/src/node_dns_7.o build/default/src/node_events_7.o build/default/src/node_file_7.o build/default/src/node_http_7.o build/default/src/node_net_7.o build/default/src/node_signal_watcher_7.o build/default/src/node_stat_watcher_7.o build/default/src/node_stdio_7.o build/default/src/node_timer_7.o build/default/src/node_idle_watcher_7.o build/default/deps/libev/ev_1.o build/default/deps/libeio/eio_1.o build/default/deps/evcom/evcom_3.o build/default/deps/http_parser/http_parser_4.o build/default/deps/coupling/coupling_5.o -> build/default/node
Waf: Leaving directory `/home/cstrom/src/node-v0.1.33/build'
'build' finished successfully (9m21.816s)
Cool. So now a make install:
cstrom@whitefall:~/src/node-v0.1.33$ make install
Waf: Entering directory `/home/cstrom/src/node-v0.1.33/build'
* installing deps/libeio/eio.h as /home/cstrom/local/include/node/eio.h
* installing deps/libev/ev.h as /home/cstrom/local/include/node/ev.h
* installing deps/udns/udns.h as /home/cstrom/local/include/node/udns.h
...
* installing build/default/node as /home/cstrom/local/bin/node
* installing build/default/src/node_version.h as /home/cstrom/local/include/node/node_version.h
Waf: Leaving directory `/home/cstrom/src/node-v0.1.33/build'
'install' finished successfully (0.476s)
It looks as though the node.js build scripts have honored my --prefix=/home/cstrom/local and, running the node binary would seem to confirm it:
cstrom@whitefall:~$ ./local/bin/node
No script was specified.
Usage: node [options] script.js [arguments]
Options:
-v, --version print node's version
--debug[=port] enable remote debugging via given TCP port
without stopping the execution
--debug-brk[=port] as above, but break in script.js and
wait for remote debugger to connect
--v8-options print v8 command line options
--vars print various compiled-in variables

Enviromental variables:
NODE_PATH ':'-separated list of directories
prefixed to the module search path,
require.paths.
NODE_DEBUG Print additional debugging output.

Documentation can be found at http://nodejs.org/api.html or with 'man node'
Cool!

Before trying something more complicated, I will give the delayed execution example from the nodejs.org site a try. I save this in ~/tmp/delay.js and execute it with node:
cstrom@whitefall:~$ ./local/bin/node ./tmp/delay.js 
Server running at http://127.0.0.1:8000/
And, accessing this resource does incur a delay of 2 seconds:
cstrom@whitefall:~$ time curl http://127.0.0.1:8000/
Hello World
real 0m2.036s
user 0m0.012s
sys 0m0.016s
That's all well and good, but what about something a little more complex? This current chain is supposed to be about updating CouchDB. Since CouchDB is HTTP from the ground up, I ought to be able to pull data back from CouchDB inside node.js. After some fiddling, I am able to produce a list of all CouchDB databases (_all_dbs) on my local CouchDB server with this node.js script:
var sys = require('sys'),
http = require('http');

http.createServer(function (req, res) {
var client = http.createClient(5984, "127.0.0.1");
var request = client.request("GET", "/_all_dbs");

request.addListener('response', function(response) {
var responseBody = "";

response.addListener("data", function(chunk) {
responseBody += chunk;
});

response.addListener("end", function() {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.write(responseBody);
res.close();
});
});

request.close();

}).listen(8000);
sys.puts('Server running at http://127.0.0.1:8000/');
After running the script with node couch.js, I can indeed retrieve the list of databases with curl:
cstrom@whitefall:~$ curl http://127.0.0.1:8000/
["eee","test","seed"]
Piece by piece, this script creates a client and builds a request of the _all_dbs resource:
  var client = http.createClient(5984, "127.0.0.1");
var request = client.request("GET", "/_all_dbs");
The request needs an event listener:
  request.addListener('response', function(response) {
// ...
});
I turn, the response object also needs an event listener to build up the response:
    response.addListener("data", function(chunk) {
responseBody += chunk;
});
And to handle the end of the response from the CouchDB server (which is when the node.js action responds):
    response.addListener("end", function() {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.write(responseBody);
res.close();
});
Finally, and it took me more than a little while to realize this, I need to close the request to get it processed (otherwise things just hang):
  request.close();
The rest of the script is identical to the delayed job script above.

I am quite sure that is a silly example, but it helps get me on my way with node.js. Tomorrow, I will likely play with node-couch, from which I borrowed liberally while putting this example together.

Day #52

7 comments:

  1. I would recommend you play with http://github.com/felixge/node-couchdb first. It has better documentation for one. Also I like the code of the lib better. Good luck with node.js

    ReplyDelete
  2. Awesome!

    I certainly did not expect you to jump from ruby/sinatra to javascript/nodejs.

    I've been following nodejs/commonjs for some time and its getting very exciting!

    ReplyDelete
  3. @Scott: thanks for the pointer -- I'll definitely check it out. Thanks!

    @Neville: nodejs does seem pretty exciting nowadays. I don't know if I'll end up using it, but I don't mind having an excuse to experiment with it :)

    ReplyDelete
  4. Felix's node-couchdb library is great.

    There is a lot of node related CouchDB stuff:

    relaximation : concurrent performance testing for CouchDB, concurrent build script, graph server
    git://github.com/jhs/relaximation.git

    node.couchapp.js : alternate implementation of couchapps, defined in pure javascript without the big directory heirarchy. Also an isolated test environment.
    http://github.com/mikeal/node.couchapp.js

    js-registry : couchapp for a javascript package index, uses node.couchapp.js
    http://github.com/mikeal/js-registry

    balance : programmable http reverse proxy load balancer, we'll be using this to route requests to different db's and update the routing table based on a _changes listener
    http://github.com/mikeal/balance

    node.couch.js : something of a dumping ground for node related couchdb stuff i'm doing, a changes consumer that is updated from a ddoc property is currently working
    http://github.com/mikeal/node.couch.js

    couchcache : reverse proxy cache for CouchDB, invalidates cache using _changes listeners
    http://github.com/mikeal/couchcache

    lode : randall is re-implementing lounge in node :)
    http://github.com/tilgovi/lode

    ReplyDelete
  5. Thanks a lot for posting this. It saved me a lot of time getting started.

    ReplyDelete
  6. I'm not sure if the syntax changed, but for me there request.close() and res.close() caused errors.
    I used request.end() and res.end() and that works fine

    ReplyDelete
  7. Feel free to check out nano if you like: http://github.com/dscape/nano

    ReplyDelete