Loggly middleware for Express.js (server visibility)

Hi!

Oh how long without posting, I’ve been missing it dearly, but in transition between jobs I was super busy, and now I can take the time to add some cool stuff =)

Use Case

I’ve been building an API, it is very cool, but the visibility for it is naturally low compared to websites. Loggly offer a great service for monitoring servers via a simple API available in node.js. While this API is cool, it is super broad and require some experimentation to implement an adequate visibility level.

Wouldn’t be amazing to be able to see every request that come in your server and how it respond? Utopic? Absolutely possible!

Loggly API

Implementation

This implementation has been heavily inspired by Connect logger middleware – which is great.

I deeply recommend you to tune it for your visibility needs, though. Make good use: https://gist.github.com/flockonus/5380753

// suggested use:
app.configure(function() {
  app.set("port", process.env.PORT || 3005);
  app.use(logglyMiddleware);
  // ..
  app.use(express.bodyParser());
  app.use(app.router);
  // ..
});

Notes worth taking

  • This snippet will show you information in your console when o development
  • The info sent might be too much in terms of size or frequency, make good use of customization for your needs
  • If you are dealing with a website, I advice you to not send every path hit, it is just too much

Cheers!

ImageMagick on Joyent

It is really simple:


pkgin search ImageMagick # you will get some result like this:
ImageMagick-6.6.6.5 Package for display and interactive manipulation of images
p5-PerlMagick-6.6.6.5 Object-oriented Perl interface to ImageMagick
php53-imagick-5.3.5.2.2.2 Provides a wrapper to the ImageMagick library
ruby18-RMagick-2.13.1nb3 Ruby binding to ImageMagick
ruby18-mini-magick-2.1 Ruby wrapper for ImageMagick command line
ruby19-RMagick-2.13.1nb3 Ruby binding to ImageMagick
ruby19-mini-magick-2.1 Ruby wrapper for ImageMagick command line

Now, install the latest version, today it is 6.6.6.5, so run:

pkgin install ImageMagick-6.6.6.5

Done!

For the Node.js package “imagemagick” at “0.1.2″ works for my use cases, which are just resizes :)
If you are using other languages you may need the other packages as well.

Mongoose validate unique field (insensitive)

Making a validation function for Mongoose (Node.js – MongoDB ODM) that checks upon validation field.

function uniqueFieldInsensitive ( modelName, field ){
	return function(val, cb){
		if( val && val.length ){ // if string not empty/null
			
			var query = mongoose.models[modelName]
				.where( field, new RegExp('^'+val+'$', 'i') ) // lookup the collection for somthing that looks like this field 
			
			if( !this.isNew ){ // if update, make sure we are not colliding with itself
				query = query.where('_id').ne(this._id)
			}
			
			query.count(function(err,n){
				// false when validation fails
				cb( n < 1 )
			})
		} else { // raise error of unique if empty // may be confusing, but is rightful
			cb( false )
		}
	}
}

Then call it to a field (aka path):

UserSchema.path('nick').validate( uniqueFieldInsensitive('User', 'nick' ), 'unique' )

Recommend you organize all validations in a file for itself, or even a folder if grows too big.

If you have a unique index set for the path you may just save it and check for the error raised; the thing is the format of the error.

Basic Authentication on Node.js – Express and Mongoose

via Axiom Zen

Hello!

This post is about the most important parts I can think of implementing signup and login to Node.js powered by Mongoose and Express.js.

This post have a huge audience, but guys, please keep in mind that this post was done a while ago. It works pretty well as a cook-book, but there probably are better options for achieving this nowadays, new npm packages and such.

Implementing last night, I was impressed on how fast had the functional part done (around 3 hours), pretty much the same time I’d had it done in Rails (if not faster).

Goal:

  • 3 urls: GET login & signup, POST login, POST signup.
  • User fields: email, nick, password(encrypted).
  • Validate (and show) messages for all fields.
I’ve setup a small Github project that may be used as example (assuming you have mongoDB installed and running in localhost) – DOWNLOAD IT FROM GITHUB  

Model: user.js

var Schema = mongoose.Schema
  , ObjectId = Schema.ObjectId
  , Validations = require('./validations.js')
  , salt = 'mySaltyString'
  , SHA2 = new (require('jshashes').SHA512)()

function encodePassword( pass ){
	if( typeof pass === 'string' && pass.length < 6 ) return ''

	return SHA2.b64_hmac(pass, salt )
}

var UserSchema = new Schema({
    nick        : {type: String, required: true, unique: true, trim: true }
  , email       : {type: String, required: true, unique: true, trim: true, lowercase: true }
  , password    : {type: String, set: encodePassword, required: true }
});

UserSchema.statics.classicLogin = function(login, pass, cb) {
	if( login && pass ){
		mongoose.models.User
			.where( 'email', login )
			.where( 'password', encodePassword(pass) )
	  	.findOne( cb )
	} else {
		// just to launch the standard error
		var o = new this({nick: 'VeryUniquejerewelA', password: '', email: login+'aaa'})
		o.save(cb)
	}
}
UserSchema.path('nick').validate( Validations.uniqueFieldInsensitive('User', 'nick' ), 'unique' )
UserSchema.path('email').validate( Validations.uniqueFieldInsensitive('User', 'email' ), 'unique' )
UserSchema.path('email').validate( Validations.emailFormat, 'format' )
UserSchema.path('password').validate( Validations.cannotBeEmpty, 'password' )
UserSchema.plugin( mongoose.availablePlugins.timestamper )

mongoose.model('User', UserSchema)

.Highlights of this code: We are using the package jshashes, which supplies many convenient encryption methods, among those SHA512 –strong enough
It is a good practice to use a salt along, represented by the var salt. In practice it makes way difficult for a cracker that acquired access to the database do decipher the passwords stored.
The method encodePassword is used at two occasions, when setting the User password, and when retrieving it from database.
UserSchema.statics is a object that stores additional static methods our User model will offer.
The function classicLogin requires both login and pass to search the db for existence, otherwise, it will launch an error (kinda of a smelly workaround to make it work dry )
UserSchema.path(…).validate offers us validations, in our case, we do not allow repeated email or nick, and password should be bigger at least 6 characters long. Also email should at least look like a email.
All those validations work along with the own Schema definition: required, unique, trim, lowercase

Route: auth.js

// app.get( '/auth/popover', auth.popover);
exports.popover = function(req, res){
	//req.session.popover = new Date()
	console.log('My session:', req.session)
  res.render('auth/index_pop', req.viewVars);
};

// CLASSIC LOGIN / SIGNUP       --because everyauth seems too messy for login+pass
// app.post('/auth/classic-signup', auth.classicSignup)
exports.classicSignup = function(req,res,next) {
	if( !req.body ){
		console.log('why u signup nobody?')
		return res.redirect('/?nobodySignup')
	}

	var user = new app.models.User()

	user.set('nick', req.body.nick)
	user.set('email', req.body.email)
	user.set('password', req.body.pass)
	user.set('providers', ['signup:'+user.get('email')])
	user.set('profiles', [{ _name: 'signup'}])

	user.save( function(err) {
		if( err ){ // validation failed

			req.viewVars.u = user
			return classicYieldErr( req, res, 'signUp', err)

		} else { // signup successful

			req.session.user = {
				provider: 'signup',
				id: user.get('id'),
				nick: user.get('nick'),
			}

			req.flash('notice', 'Welcome!')
			req.viewVars.welcome_login = "Welcome, "+user.nick

  		res.render('auth/win_pop', req.viewVars )
		}
	})
};

// app.post('/auth/classic-login',  auth.classicLogin)
exports.classicLogin = function(req,res,next) {
	if( !req.body ){
		console.log('why u login nobody?')
		return res.redirect('/?nobodyLogin')
	}

	app.models.User.classicLogin( req.body.email, req.body.pass, function(err, user) {
		if( err ){ // validation failed

			return classicYieldErr( req, res, 'signIn', err)

		} else {

			if( user ){ // login

				req.session.user = {
					provider: 'signup',
					id: user.get('id'),
					nick: user.get('nick'),
				}

				req.flash('notice', 'Welcome!')
				req.viewVars.welcome_login = "Welcome, "+user.nick

	  		res.render('auth/win_pop', req.viewVars )

			} else { // not found
				return classicYieldErr( req, res, 'signIn', {errors:
					{'loginpass': {
						name: 'V',
						path: 'login+password',
						type: 'loginpass'
					}
				}})
			}

		}
	})
};

// display form error
function classicYieldErr( req, res, mode, err ){
	req.viewVars.erroredForm = mode
	if( mode === 'signIn' ){
		req.viewVars.signin_errors = app.helpers.displayErrors( err )
	} else {
		req.viewVars.signup_errors = app.helpers.displayErrors( err )
	}
	req.viewVars.email = req.body.email

	res.render('auth/index_pop', req.viewVars);
}

Here we define the 3 routes;
- one route(GET) that defines one page for both login or signup
- one route(POST) to submit login
- one route(POST) to submit signup
About the last 2, their only role is to allow access for valid data. That is, valid signup data, or login+password existing in our collection from MongoDB.
The function classicYieldErr is there just to serve the errors in sort of an uniform way. We will see about this function on the next file

Model Helper: validations.js

exports.uniqueFieldInsensitive =  function ( modelName, field ){
	return function(val, cb){
		if( val && val.length ){ // if string not empty/null
			// only for new docs
			if( this.isNew ){
				mongoose.models[modelName].where(
					field, new RegExp('^'+val+'$', 'i')
				).count(function(err,n){
					// false when validation fails
					cb( n < 1 )
				})
			} else {
				cb( true )
			}
		} else { // raise error of unique if empty // may be confusing, but is rightful
			cb( false )
		}
	}
}

exports.emailFormat = function( val ){
	// false when validation fails
	return (/^([^@\s]+)@((?:[-a-z0-9]+\.)+[a-z]{2,})$/i).test( val )
}

exports.cannotBeEmpty = function( val ){
	console.log('pass val is:', val)

	// not all passwords should be set, BUT when string, should be encoded
	if( typeof val === 'string' ){
		if( val.length ){ // when it comes empty, something went wrong!
			return true
		} else {
			return false
		}
	} else {
		return false
	}
}

This file contains some very important function, being intuitive, I will only comment on the first.
uniqueFieldInsensitive is actually a validation creator. It uses JS’s closure ability along with the powerful this object to wrap the context where this will represent whichever Model is calling it.
The intent of the function is to search the db for any repeated occurrence of the field, case-insensitive. This way the system will block a user “John” from signinup, if “john” is present.

Helper: form_helper.js


//app.helpers.displayErrors = require('./helpers/form_helper.js').displayErrors

stripErrors = function(mongooseErr){
	var prop, list = [];
	for( prop in mongooseErr.errors ){
		list.push( [mongooseErr.errors[prop].path, mongooseErr.errors[prop].type] )
	}
	return list
}

/*
 * Translate mongoose errors into a <li> of errors
 */
exports.displayErrors = function( mongooseErr ){

	console.log( 'mongoose errs', mongooseErr )

	var list = stripErrors( mongooseErr )

	var output = []
	list.forEach(function(e,i){
		switch( e[1] ){
			case( 'unique' ):
				output.push( e[0]+" is taken" )
				break;
			case( 'required' ):
				output.push( e[0]+" is "+e[1] )
				break;
			case( 'format' ):
				output.push( e[0]+" has a bad format" )
				break;
			case( 'password' ):
				output.push( "password should be at least 6 char long" )
				break;
			case( 'loginpass' ):
				output.push( "login+password not found" )
				break;

			default:
				output.push( e[0]+": "+e[1] )
				break;
		}
	})
	if( output.length ){
		// condense all items in an error list
		output = [output.join( '</li><li>\n' )]
		output.unshift( '<ul><li>' )
		output.push( '</li><ul>' )

		// wrap in a div
		output.unshift( "<div class='error block'>" )
		output.push('</div>')
	}
	return output.join('\n')
}

This file provides a translation from Mongoose errors to user-readable errors. In order to offer a good UX, it is important to give accurate feedback. Notice this file could benefit from some refinement :)

Don’t forget to download the code from github, it plays well along everyauth module :)

Cheers!

Ruby’s fear-cancer

This video: High Performance Ruby: Threading Versus Evented by Dr Nic Williams, Engine Yard

It meant so much to me!

For around 4 months I’ve been using Node.js, and before that, for 3 years, programming Rails.

As soon as I started on Node.js, I could feel that something was different. My little Computer Science bachelor conscience was starting to tell me: now you are starting to do it right :)

But then, lets go back, Ruby’s syntax is lovely.. Rails API is sugar! But still, it’s neverland. Why??

It does not fit real world! It is a thin layer of happiness, so delicate, that we feel fear of touching it and breaking it. So what we puny humans do? We delegate it to those genius heroes who will be able to take that code and maybe squeeze a few more couples of requests/sec!

But not even that is the cancer I mean.

The cancer here is represented by clown hosting the video. “Oh those scientific articles mess with our head and stuff O.o Let me chew that evil reality and I will give you the RIGHT (sugar coated) solution!” and all other non fun nor assertive statements. By the way, he may be referring to the C10K problem, and alike, articles.

The problem here is, VP  EngineYard and want ppl addicted on his junk. He does not host Node.js.

Ruby always had so many language implementations to solve some important language problems, as well as servers. This is not the first time a good solution for ruby is born (JRuby+Trinidad), but why would they share it before if he could have people paying for so much RAM?

The reason I believe he felt compelled to share it now, is that Node.js unoptimized code can outperform Rails by a LOT,  under 150Mb.

But my main problem is fear of complexity. The host talks about it all over the initial part of the video. He refers his audience under constant fear of it’s own ignorance all the time. This is a awful, and that’s how I felt using Rails. Such a large stack, complex language design, only understanding the framework code itself was hard task.

To keep people in bliss ignorance he just goes like: “Oh I promissed evented? We are past that, right?! HAAAA” so just be happy and keep the status quo. You would’t want to mess your pretty little head with asynchronous code, would you?

I like Node.js, besides performance, language is simple; it is JavaScript, open Objects, closure and more. Of course, don’t assume well-written JS is something easy to do. No great code in any modern language is easy to achieve.

If you’ve developed Rails, you have concepts of MRI, thin, mongrel, jruby, 1.8.7, 1.9.2, rubinius, unicorn, and LOT MORE, whilst in node.js world all that stands for: node.js. It is a unification point for language and server.

What about gems?  npm, a easier system of distributing packages.

Wrapping it up, node.js is no silver bullet, it just made me realize the problems I had while programming ruby, as much as ruby did it for me on php. If nothing else, node.js helped improve ruby’s community by adding options to the web development mainstream.

Would I work with Rails again? If a employer would point me that out as the only chosen solution, then yes, and I’d probably try that stack, but I prefer confidence instead of fear.

When to Ruby on Rails, when to Node.js

(update) Take this post as a naive overview, it may not reflect the most accurate reality

Hello!

I am trying to do a sort of indirect comparison between Rails and Node.js. The very main reason of being indirect, is that Rails is a Framework, while Node.js is an implementation of a programming language with custom libraries.

If it were be to put in a simple phrase, Rails is resourceful and Node.js is light and fast.

Lets elaborate some..

Rails

Is the most complete open-source framework available (that I know of)! Big companies use it. It can do lots of stuff well, in an organized manner; this meaning, Rails is more than just MVC, it has a full stack of features well-integrated, at the same time being very modular. Some of the features included out of the box:

  • Database adapter for the majority of them, supporting plug your own.
  • Database migrations, so multiple dev can sync and experiment with their DB.
  • Powerful engines for Views, Controllers and Models.
  • Support to code generator.
  • Has structure to all sorts of tests and friendly to TDD.
  • Really awesome documentation.
  • Model has all kinds of hooks, validations and associations.
  • Controller has support to handle XML/JSON in the same action that serves HTML.
  • Gems that integrate, for instance, Memcached, MongoDB, Auth and lots more.
So Rails is war-proven, capable of integrating lots of features together without harass. There is also a very cool post of Fabio Akita in the refs. about how it made possible to develop systems in periods before impossible.

Node.js

Two things make this platform suitable for web:

Its engine, V8 is very fast! In a very loose average, 8 times faster than Python (or even up to 200 at peak). Python already outperforms Ruby (ref. bottom)

Second point; and this argument is separated from the above, is that it async-driven (is built around reactor pattern). As in the case, requests can be performed in parallel, without any blocking I/O. A single server can handle a lot. (update) And with >0.6.0 Cluster API, it can scale to use all of available CPU cores.

So, it is a very new sort of backend language, but huge players, besides Joyent, who invented it, are adopting it, including LearnBoost and LinkedIn, which has an awesome article about using. The language, and it’s main web framework, Express, deserve a list of features (you can check more info in the references below).

  • It´s web server is able to handle a HUGE number of connections out of the box
  • Various libraries can be run on browser, the same as in the server
  • Very friendly to Websockets (real-time web apps)
  • Lots of libraries are being ported to it from other langs.
  • Express, inspired in ruby´s Sinatra; is very light on memory but also very powerful
Running a simple benchmark against a single server instance, I were able to get 587 req/s accessing MySQL without any external cache support. This number could scale, if I used Cluster to spawn at least a process per processor.

Summarizing, When to use each?

Rails really shines when..

  • The database is complex in terms of associations.
  • The app structure is well defined.
  • Business rules are complex, and validation is needed.
  • When the number of requests isn´t the a decisive factor.
  • Administrative interfaces.
  • Many developers in parallel keep the DB up-to-date with migrations
  • The database to be used is undefined, or may vary.
What about Node.js?
  • APIs
  • Real-time web/mobile apps.
  • Application that should scale to lots of concurrent requests.
  • Little memory footprint

This being said, there is no reason at all, a web-site or service can´t easily integrate both.

– I’d appreciate if you could leave a comment, either to talk about your case, or add up.

References

 

UPDATE: http://www.mikealrogers.com/posts/a-new-direction-for-web-applications-.html

http://guides.rubyonrails.org/

http://railscasts.com/

http://blog.heroku.com/archives/2011/6/22/the_new_heroku_2_node_js_new_http_routing_capabilities/

http://nodejs.org/

http://akitaonrails.com/2011/04/16/twitter-muda-de-ruby-para-java-ruby-e-3x-mais-lento-que-java

http://venturebeat.com/2011/08/16/linkedin-node/

http://blog.bossylobster.com/2011/08/lesson-v8-can-teach-python-and-other.html

https://github.com/LearnBoost

http://www.readwriteweb.com/hack/2011/01/how-3-companies-are-using-node.php

http://twitter.com/#!/FlockonUS/status/104655096956190720

https://github.com/LearnBoost/cluster

Testing a Node.js Express API server with Vows (functional)

How to test an API that must be authenticated via session?

vows: plural of vow

Noun: A solemn promise.
Verb: Solemnly promise to do a specified thing: “one fan vowed, “I’ll picket every home game.””.

Oh wait..

Vows.js
Asynchronous behaviour driven development for Node.

I should… test every method I expose to the client..
Since the Vows documentation is very awesome, and I already have another post that explains the strategy used on the API, there is no need to much talk; just not that this code is still very early in maturity.
Straight to current code [ test/api-test-authed.js ]

/*
 * INSTRUCTIONS
 *
 * run the site at localhost, port 8010
 *
 * run vows --spec test/api-test-authed.js
 *
 */


var request = require('request'),
    vows = require('vows'),
    assert = require('assert'),
    apiUrl = "http://localhost:8010/",
    cookie = null


var apiTest = {
  general: function( method, url, data, cb ){
    //console.log( 'cb?', cb )
    request(
      {
        method: method,
        url: apiUrl+(url||''),
        json: data || {},
        headers: {Cookie: cookie}
      },
      function(req, res){
        cb( res )
      }
    )
  },
  get: function( url, data, cb  ){ apiTest.general( 'GET', url, data, cb    )  },
  post: function( url, data, cb ){ apiTest.general( 'POST', url, data, cb   )  },
  put: function( url, data, cb  ){ apiTest.general( 'PUT', url, data, cb    )  },
  del: function( url, data, cb  ){ apiTest.general( 'DELETE', url, data, cb )  }
}

function assertStatus(code) {
  return function (res, b, c) {
    assert.equal(res.statusCode, code);
  };
}


function assertJSONHead(){
  return function(res, b, c ){
    assert.equal( res.headers['content-type'], 'application/json; charset=utf-8' )
  }
}

function assertValidJSON(){
  return function(res, b ){
    // this can either be a Object or Array
    assert.ok( typeof( res.body ) == 'object' )
    //assert.isObject( res.body)
  }
}





// TODO include unauthed tests
var suite = vows.describe('API Localhost HTTP Authenticated Tests')

// Very first test!
.addBatch({
  "Server should be UP as in: var apiUrl": {
    topic: function(){
      apiTest.get('', {} ,this.callback )
    },

    '/ should repond something' : function(res, b){
      assert.ok(res.body)
    }
  }
})

.addBatch({
  'Authenticate to /login': {
    topic: function(){
      request.post(
        {
          url: "http://localhost:8010/login",
          json: { user:{ username: 'flockin_lab', password: '123456' }}
        },
        this.callback
      );
    },



    'get a valid Cookie': function(req, res, body, err){
      try{
        cookie = res.headers['set-cookie'].pop().split(';')[0]
        console.log("GOT COOKIE!", cookie)
      } catch(e){ }

      assert.ok( typeof(cookie) == 'string' && cookie.length > 10 )
    }
  }
})
.addBatch({
  'Users#index': {
    topic: function(){
      apiTest.get('admin/employees', {}, this.callback)
    },
    'should be 200': assertStatus(200),
    'should have JSON header' : assertJSONHead(),
    'body is valid JSON' : assertValidJSON(),

  },
})
.addBatch({
  'Qrcodes#index': {
    topic: function(){
      apiTest.get('admin/qrcodes', {}, this.callback)
    },
    'should be 200': assertStatus(200),
    'should have JSON header' : assertJSONHead(),
    'body is valid JSON' : assertValidJSON(),

  },
})

//suite.run( )
suite.export( module )

Brief Discussion

This code is still state-of-art,
We depend on the lib ‘request’, which is pretty good,
The server should already be started, since I see no point in having the test being responsible to bring it up and handle it.

Can improve it? Please leave a comment! :)

Starting a Node.js Express API to serve json, (with auth, MySQL – based)

via Axiom Zen

Hello and welcome!

Foreword, and purpose

I will be putting this JSON-API up, to serve USERS as a restful resource, trying to be light as possible, still serving content only to authenticated peers (auth based on cookie).

The regular use will be something as, the client makes login, sending params via POST, ant then is able to access all resources we may serve.

I will be using some packages, all being possible to install via npm install, lets talk about them:

  • Express, itself is like the Ruby’s Sinatra Framework, but async, for Node.js
  • Express-resource, a simple and extendable way to define Restful resources
  • Sequelize, a MySQL ORM! Offer n-m relations, and a .sync() way to define your own Schema! (better than migrations?)
  • Faker, a generator of fake-data that we need
  • Mustache, String template engine. Will be using for minimal tasks, not required.
  • Jade, view template engine. Not required at all, but since I started coding there, will keep it for now.

The app skeleton

Install express with -g and generate an app from it.

We will be keeping 3 core files:

app.js  : The main file; require all other from there, and configure Express!

models.js : The file that has all models definitions exported.

routes.js : Acts as a general router and controller for the app. Also holds the functionality of authenticating all users that get past /admin/*. The authorization method here is static, in this case, one e-mail bounded to one password. See Auth Methods app.post(‘/login’.. for details.

Files

./app.js

// Detect Params  http://nodejs.org/docs/latest/api/process.html#process.argv
if (process.argv.indexOf('--seed') > -1) {
  /**
   * Indicates when the DB should be re-schemed and seeded again
   */
  GLOBAL.db_seed = true;
}

if (process.argv.indexOf('--no-auth') > -1 ){
  /**
   * Indicates when to run without requiring auth for API
   */
  GLOBAL.no_auth = true;
}

var express = require('express'),
    /**
     * Instantiate the unique Express instance
     */
    app = module.exports = express.createServer(),
    models = require('./models.js'),
    S = require('mustache').to_html;

/**
 * @type {Object}
 *
 * A list of all Sequelize Models available, representing the tables.
 */
GLOBAL.models = models;

/**
 * @type {Express}
 *
 * The Singleton of Express app instance
 */
GLOBAL.app = app;

// Configuration

app.configure(function() {
  app.use(express.logger({format: 'dev'}));
  app.set('views', __dirname + '/views');
  app.set('view engine', 'jade');
  app.use(express.bodyParser());
  app.use(express.cookieParser());
  app.use(express.session({ secret: 'evilWorldDom1nat10nPlanzisstillsmallshouldhaveNoWords' }));
  app.use(express.methodOverride());
  app.use(app.router);
  app.use(express.static(__dirname + '/public'));

});

app.configure('development', function() {
  app.use(express.errorHandler({ dumpExceptions: true, showStack: true }));
});

app.configure('production', function() {
  app.use(express.errorHandler());
});

// Routes

require('./routes.js');

app.listen(8010);
console.log('Express server listening on port %d in %s mode',
   app.address().port, app.settings.env);

./models.js

var Sequelize = require('sequelize'),
    db = new Sequelize('dev_', 'root', '123456');

  /**
   * @type {Object}
   * Map all attributes of the registry
   * (Instance method useful to every sequelize Table)
   * @this {SequelizeRegistry}
   * @return {Object} All attributes in a Object.
   */
    var map_attributes = function() {
      var obj = new Object(),
          ctx = this;
      ctx.attributes.forEach(
        function(attr) {
          obj[attr] = ctx[attr];
        }
      );
      return obj;
    };

/**
 * @type {Object}
 * All models we have defined over Sequelize, plus the db instance itself
 */
var self = module.exports = {
  'db' : db,

  User: db.define('user',
    {
      name: {
              type: Sequelize.STRING,
              defaultValue: 'Not Big Deal',
              allowNull: false
            }
    },
    {
      timestamps: true,
      freezeTableName: true,

      classMethods: {
        staticExample: function() { this.name }
      },
      instanceMethods: {
        mapAttributes: map_attributes
      }
    }
  ),

  Activity: db.define('activity',
    {
      type: { type: Sequelize.STRING },
      value: { type: Sequelize.STRING }
    },
    {
      timestamps: true,
      freezeTableName: true,

      instanceMethods: {
        mapAttributes: map_attributes
      }
    }
  ),

  Company: db.define('company',
  {
    name: { type: Sequelize.STRING, defaultValue: 'MegaCorp', allowNull: false }
  },
  {
    timestamps: true,
    freezeTableName: true,

    instanceMethods: {
        mapAttributes: map_attributes
      }
  })

};

self.Activity.belongsTo(self.User, {foreignKey: 'user_id'});
self.User.hasMany(self.Activity, {foreignKey: 'user_id'});

self.User.belongsTo(self.Company, {foreignKey: 'company_id'});
self.Company.hasMany(self.User, {foreignKey: 'company_id'});

if (GLOBAL.db_seed) {
  console.info('SEED TIME!');

  var chainer = new Sequelize.Utils.QueryChainer;

  chainer
    .add(self.Company.sync({force: true}))
    .add(self.User.sync({force: true}))
    .add(self.Activity.sync({force: true}))
    .run()
    .on('success', function() {

      var seeds = require('./seeds');
      for (var model_name in seeds) {
        console.log('MODEL', model_name);
        for (var i = 0; i < seeds[model_name].length; i++) {
          self[model_name].create(seeds[model_name][i])
            .on('success', function(novo) { })
            .on('failure', function(erro) {
              console.error('DB SEED ERROR!!', erro);
              process.kill();
            });
          console.log('  ', seeds[model_name][i]);
        }
      }
    })
    .on('failure', function(errors) {

      console.error('DB schema change FAIL', errors);
      process.kill();
    });

}

// User.staticExample()
// User.build({}).instanceExample()

./routes.js

S = require('mustache').to_html;
Resource = require('express-resource'); // paths Express, add .resource

/**
 * ROOT
 */
app.get('/', function(req, res) {

  // This big mess just points out all routes we have, along with the verbs
  var path_list = S(
    '{{#routes_obj}}' +
    "<a href='{{path}}'>{{method}}: {{path}} </a> <br/>" +
    '{{/routes_obj}}',
    {
      routes_obj: (app.routes.routes.get ? app.routes.routes.get : [])
        .concat((app.routes.routes.post ? app.routes.routes.post : [])
          .concat((app.routes.routes.put ? app.routes.routes.put : [])
            .concat(app.routes.routes.delete ? app.routes.routes.delete : [])
              .concat([{ path: '/auth.html', method: 'get'}])))
    }
  );

  res.render('index', {
    title: 'DBServer',
    'path_list': path_list,
    logged: req.session.authed ? 'YES :) ' : 'NO :( '
  });

});

/**
 * AUTH methods
 */
app.post('/login', function(req, res) {

  console.info('login PARAM: ', req.body);

  var credentials = req.body.user;

  if (GLOBAL.no_auth ||
       (credentials &&
        credentials.username == 'super_safe' &&
        credentials.password == '123456')
      ) {

    req.session.authed = true;
    res.json(['OK']);
  } else {
    res.json(['FAIL']);
  }

});

app.get('/logout', function(req, res) {
  req.session.authed = null;
  res.json(['OK']);
});

/*
 * API Authentication filter
 */
app.all('/admin*', function(req, res, next) {

  if (req.session.authed) {
    next();
  } else {
    res.json(['must auth']);
  }
});

/*
 * Resources
 */
app.resource('admin/users', require('./resources/users'));

/**
 * Just say it is all fine.
 */
module.exports = true;

./seeds.js

// https://github.com/marak/Faker.js/
// npm install Faker
var Faker = require('Faker'),
    times = 0,
    id = 0

var self = {
  Company: [ ],
  User: [ ],
  Activity: [ ],
}

// Company: create 3 - 5
id = 1
times = Faker.Helpers.randomNumber(3)+3
for (; times > 0 ; times--) {
  var novo = {
    id: id++,
    name: Faker.Company.companyName(),
  }
  self.Company.push( novo )
  //console.log("Company", novo)
};

// Users: create 0 - 8 per company
id = 1
for (var i=0; i < self.Company.length; i++) {
  times = Faker.Helpers.randomNumber(9)
  for (; times > 0 ; times--) {
    var novo = {
      id: id++,
      name: Faker.Name.firstName(),
      company_id: Faker.Helpers.randomize( self.Company ).id,
    }
    self.User.push( novo )
    //console.log("User", novo)
  };
};

// Activities: create 0 - 10 per user
var TYPES = ['level_up', 'timeout' ],
    VALUES = ['win', 'fail']
id = 1
for (var i=0; i < self.User.length; i++) {
  times = Faker.Helpers.randomNumber(11)
  for (; times > 0 ; times--) {
    var novo = {
      id: id++,
      type: Faker.Helpers.randomize( TYPES ),
      value: Faker.Helpers.randomize( VALUES ),
      user_id: Faker.Helpers.randomize( self.User ).id,
    }
    self.Activity.push( novo )
    //console.log("Activity", novo)
  };
}

module.exports = self

./resources/users.js

exports.index = function(req, res) {
  models.User.findAll().on('success', function(users) {

    res.json(
      users.map(function(user) {
        return user.mapAttributes();
      })
    );
  });
};

./views/index.jade

h1= title
p Welcome to #{title}
p Paths we have:
div!= path_list
div .
footer Loggedin? #{ logged }

./public/auth.html

<!DOCTYPE html>
<html lang="en">
	<head>
		<meta charset="utf-8" />
		<title>auth</title>
		<link rel='stylesheet' href='/stylesheets/style.css' />
	</head>
	<body>
		<div>
			<header>
				<h1>Auth</h1>
			</header>
			<nav>
				<p>
					<a href="/">Home</a>
				</p>
			</nav>
			<div>
				<form action="/login" method="post">
					<label>Username: <input name="user[username]" type="text" /></label></br>
					<label>Password: <input name="user[password]" type="password" /></label></br>
					<button type="submit">OK</button>
				</form>
			</div>
			<footer>
				<p>
					Express API
				</p>
			</footer>
		</div>
	</body>
</html>

Brief Discussion

Overall, this post presents a short and extensible way to start a API that requires authentication, via cookie, and returns data in JSON format.

A fresh work enviroment for Node.js and Express on Ubuntu 10 in about 30 minutes

Hello and welcome.

Node.js came to me as likely the smartest choice for web development, as you are constrained at JavaScript for dynamic programming server side (ActionScript and Java, please don’t cry). Having all String, and Date operations the same as server or client is just a big thing! Also validations come to my mind as not-anymore duplicated code. Besides, the concurrency performance seems really nice. (most importantly, people smarter then me thinks the same)

This post intention is to document my efforts under Ubuntu 10.04 to install everything needed from Node.js to the actual work environment.

Lets start by some git and a good working java for Aptana, and some dependencies for Node.js install itself.

sudo aptitude install git-core sun-java6-bin sun-java6-jre sun-java6-plugin sun-java6-jdk python libssl-dev socat curl libev

Now, you should be good to both install version 0.4x as referenced here: https://github.com/joyent/node/wiki/Installation and I also recommend you start dl Aptana Studio 3 this is my IDE of choice for web dev. (as Rails developer)

If you are new to Aptana, I also recommend going to Window > Preferences and choosing a theme(Aptana>Themes) you like, as well as configuring JS to not alert apparent errors(General>Editor>Texte Editor>Errors : uncheck boxes), since it does not know Node.js functions.

Install NPM, which is the best package handler at the moment and all packages to head into web dev, I will explain later on

curl http://npmjs.org/install.sh | sh
npm install jade
npm install less
npm install -g express
npm install -g node-dev
npm install -g node-inspector

About Express, you can have noticed, it is close to ruby’s Sinatra web framework MVC, which is pretty cool, and has a great performance. I recommend some learning from their web-site videos.

In my machine, AMD Phenom II X4 965, 3.2GB Ram (bla bla) running some intensive apache ab, I got these fantastic results:

ab -n 100000 -c 100 http://127.0.0.1:3000/

Document Path: /
Document Length: 11 bytes

Concurrency Level: 100
Time taken for tests: 10.789 seconds
Complete requests: 100000
Failed requests: 0
Write errors: 0
Total transferred: 13200000 bytes
HTML transferred: 1100000 bytes
Requests per second: 9268.48 [#/sec] (mean)
Time per request: 10.789 [ms] (mean)
Time per request: 0.108 [ms] (mean, across all concurrent requests)

Jade and Less are about the View layer

Now… node-dev and node-inspector.. CLASSY STUFF!

If you are used to developing Rails or Sinatra, you are used to have your code auto-reloaded (development-mode or shotgun), node-dev accomplishes just that! everytime you make any changes to any file.js from the current folder it will kill and restart your Node app for you; nice!

Now, node-inspector this it the coolest debug tool I’ve ever seen! And you can use it along with node-dev with no problem :)

You can learn more from then at their Github page

If you want to develop along with CouchDB, you should install it too.


sudo aptitude install automake spidermonkey-bin autoconf subversion-tools libtool help2man build-essential erlang erlang-manpages libicu38 libmozjs-dev wget libicu-dev libreadline5-dev checkinstall
sudo aptitude install couchdb
npm install cradle

If couch installed fine, you can see it via http://127.0.0.1:5984/_utils/index.html. And a daemon is started along with your machine every boot.

About cradle, they have a real good documentation at their website, you should start there.

And that’s it! Due aptitude and npm you can have it all done in about 30 min, heavily depending on your network connection. Good coding! :D