Express.js public/static directory in one line

If you want to serve a static directory from express you can do it with the following line of code.

The first parameter is the route, the second parameter is the path to the directory.


app.use('/public', express.static(__dirname + '/public'));

Posting here for my own convenience.

Advertisements

HTTP requests with Node.js

I couldn’t find this code anywhere on the internets yesterday, posting here for my own convenience.



 var options = {
   host: 'api.twitter.com',
   port: 80,
   path: '/path',
   method: 'GET',
   encoding: 'utf8'
 };


 http.get({ host: options.host, path: options.path }, function(res2){
	var data = [];
	res2.on('data', function(chunk) { 
		data.push(chunk); 
	});


	res2.on('end', function() { // wait for the request to finish
		res.json(data.join('')); 
	})
});

Learning from Single Page Web Applications

Learning from Single Page Web Applications

I’ve spent the last three years working on single page applications of various shapes and sizes. I don’t like em,  this post isn’t about why but I will just say I like data to be exposed at the lowest level (HTTP) and not require Javascript to turn it into something useful.   All that being said I’ve been lucky enough to work with some clever folks and the end results have all been very interesting and pushed boundaries in their own way.

At Full Frontal I enjoyed listening to Nicholas Zakas talking about Scalable Javascript Application Architecture. In many ways he described the architecture we used for the Volkswagen Configurator. Earlier in the day Phil Hawksworth had been talking about Excessive Enhancement.

It struck me that while most front end MVC frameworks make it easy to build Javascript applications, making them work as web applications (without Javascript) becomes much more difficult. This is largely due to the fact that you define and build your application, templates/page structure using the browser’s Javascript interpreter.

I’m of the view that if you architect something properly you can get the same full experience provided by Javascript applications but still have a reliable fallback for those who do not have Javascript enabled. That is what I call a proper web application.

In the back of my mind I’ve been thinking about Charlie Robbins excellent post: Scaling Isomorphic Javascript Code which talks elegantly about why MVC might not be the best pattern in an environment where you can execute Javascript on the server. He suggests the Resource-View-Presenter pattern.

It seemed that the way we currently split our frameworks between front and back end reduces reuse of configuration/code and actually encourages duplication.

I wanted to try defining all of these things on the server, so they could be consumed on the server and client.

As soon as the talk finished I found myself writing code. This blog post talks about what I have been building and why.

Sharing URLs between client and server

Allow me to deviate for a moment.

On large applications it’s common practice to split the code and teams between front and back end developers.  This often results in duplication and unnecessary bugs.  A common example being maintaining URLs in two places.  In the browser we might a have a URLs object:

namespace.urls = {
     login: '/login',
     user: '/user/:username',
     ....
}

And then on the server the same URLs would be defined possibly using a different syntax.  Changing one does not change the other, and with split teams this can result in unnecessary bugs.

With Node.js it’s particularly easy to share exactly the same URL object on the client and server. If you ensure your HTML links are populated from the same URL object your application will continue to function when URLs change.

This is what I’ve been doing in some of my Node apps:

(function(exports){
    exports.HOME = '/';
    exports.LOGIN = '/login';
    exports.USERS = '/users';
    exports.USER = '/users/:user';
    // also add URL functions here that can be shared between client and server.
    exports.build = function(str, tokens) {
        for(token in tokens){
            str = str.replace(':'+token, tokens[token]);
        }
        return str;
    };
})(typeof exports === 'undefined' ? namespace.urls={} : exports);

That allows it to be used as a Node module using require(‘./urls.js’) and when served to the browser the URLs are available at namespace.urls.

I’ve been using Express.js, the parameter sytax seems to work nicely with Levi routes on the front end.

This seems like common sense. It’s a fine example of DRY.  Define as much as possible in server-side JS and then allow it to be consumed by the client-side JS reusing as much logic as possible.

Reusable modules

TiddlyWiki has the concept of plugins which are essentially just a chunk of HTML/CSS/JS relevant to a particular piece of functionality which could be added to HTML using a special syntax:

<<pluginName>>

On the Volkswagen Configurator we also have the concept of UI’s which were reusable chunks of HTML/CSS/JS which could be appended into any DOM node.

Both are essentially variants of the Module Pattern with added support for HTML and CSS as part of the module.  Both methods work really nicely just so long as you’ve got Javascript turned on.

I started to build a simple Node.js app which could parse a modules folder and serve the resources at appropriate URLs. See the following folder structure:

Generated the following URLs:

/contact.css
/contact.html
/contact.js
/content.css
/content.html
/content.js
/flickr.css
/flickr.html
/flickr.js

You will notice there is an app.js file in the flickr folder which contains the server-side JS required by the module. Later this will provide a getData method to asynchronously generate a templating object which can populate the HTML.

Define your views on the server

In browserland it’s really easy to mess around with the DOM. You can completely transform a page, especially when you start appending modules of HTML/CSS/JS to DOM nodes. The problem comes when you want to show the same view to something that doesn’t understand Javascript, an RSS feed or search engine for example.

I decided to create a view specification which could be read by the server and also served to the browser. Doing so should make it really easy to render the exact same HTML with or without Javascript enabled.  Using Sizlate I started with this (it has since changed):

var views = [{
    url: '/user/:user'
    view: 'userpage',
    modules: [
        'photos',
        'login',
        'timeline'
    ]
 }];

In the following example I have left out the JS and CSS files.  The view is just a folder containing a HTML file of the same name and an optional app.js (as with the modules). It assumes the view HTML file contains a  tag with a id corresponding to each of the modules specified for the view.

I like this simple approach but in order for it to scale I may soon need to start using HTML data attributes instead of ids. This is what the folder structure looked like:

project/views/userpage/userpage.html (the view HTML file) contains:

<html>
<body>
    <nav>...</nav>
    <div id="photos" /> 
    <div id="login" />
    <div id="timeline" /> 
</body>
</html>

The end result would look something like this:

<html>
<body>
    <nav>...</nav>
    <div id="photos">
        .. contents of /modules/photos/photos.html appended in here
    </div> 
    <div id="login">
        .. contents of /modules/login/login.html appended in here
    </div> 
    <div id="timeline"> 
        .. contents of /modules/timeline/timeline.html appended in here
    </div>  
</body>
</html>

Concatenation vs Caching

At this stage I was able to build a sample app with my re-usable modules and a view. I thought it would be useful to concatenate all the JS and CSS files together for each view. It turns out I was wrong.

In a situation where you’re reusing modules across multiple views concatinating per view makes no sense because you end up serving the same CSS across multiple URLs. The CSS/JS will be cached per view, not per module.

I decided it was actually better to use the URLs generated for the CSS/JS by the modules so they can be cached at the most granular level.  Both methods will be possible.  Currently CSS modules are served inline with the module HTML (not the document HEAD).  There will be some work to ensure that all CSS LINK tags are moved to the document HEAD before the view is rendered. JSDOM should make that quite easy.

History API

With views and modules defined on the server I’ve started to put together a front end framework which can consume the same application specification and make the whole experience a bit more pleasant.  I’m currently experimenting with generating popstate listeners from the specification which can then fire off the default/custom transitions between views.

Whats going on here?

Essentially I have started building a framework for mixing up reusable modules of HTML/CSS/JS in ways usually associated with Javascript applications but with progressive enhancement as one of its core values.

The functionality described above will almost certainly change.  What I have built so far is a simple proof of concept to test the best way to define views in this way using Sizlate.

At the moment I’m working on a sample app pulling in data from Flickr to demonstrate how it might all fit together in the real world.  Its pretty messy and requires lots of work but you can see the code here.

I thought it was worth blogging about my approach just to get some feedback. Please do let me know what you think. All idea’s, contributions, criticisms welcomed.

I’m going to be talking about Sizlate at the London Node user group on the 25th January at the Forward Offices in Camden, London. Please register here if you would like to attend.

Latest Database design for ccTiddly

erm2.jpg

and the sql :

-- phpMyAdmin SQL Dump
-- version 2.10.1
-- http://www.phpmyadmin.net
--
-- Host: localhost
-- Generation Time: Nov 09, 2007 at 01:32 PM
-- Server version: 5.0.41
-- PHP Version: 5.2.2

SET SQL_MODE="NO_AUTO_VALUE_ON_ZERO";

--
-- Database: `permissions`
--

-- --------------------------------------------------------

--
-- Table structure for table `admin_of_instance`
--

CREATE TABLE `admin_of_instance` (
`user_id` varchar(255) NOT NULL,
`instance_name` varchar(100) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

--
-- Dumping data for table `admin_of_instance`
--

-- --------------------------------------------------------

--
-- Table structure for table `group`
--

CREATE TABLE `group` (
`name` varchar(50) NOT NULL,
`desc` mediumtext NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

--
-- Dumping data for table `group`
--

-- --------------------------------------------------------

--
-- Table structure for table `group_membership`
--

CREATE TABLE `group_membership` (
`user_id` varchar(255) NOT NULL,
`group_name` varchar(50) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

--
-- Dumping data for table `group_membership`
--

-- --------------------------------------------------------

--
-- Table structure for table `instance`
--

CREATE TABLE `instance` (
`name` varchar(100) NOT NULL,
`lang` varchar(10) NOT NULL,
`keep_revision` int(1) NOT NULL,
`require_login` int(1) NOT NULL,
`session_expire` int(10) NOT NULL,
`tag_tiddler_with_modifier` int(1) NOT NULL,
`char_set` varchar(10) NOT NULL,
`hashseed` varchar(50) NOT NULL,
`debug` int(1) NOT NULL,
`status` varchar(10) NOT NULL,
`tiddlywiki_type` varchar(30) NOT NULL,
`default_anonymous_perm` varchar(4) NOT NULL,
`default_user_perm` varchar(4) NOT NULL,
`rss_group` varchar(50) NOT NULL,
`markup_group` varchar(50) NOT NULL,
PRIMARY KEY  (`name`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

--
-- Dumping data for table `instance`
--

INSERT INTO `instance` (`name`, `lang`, `keep_revision`, `require_login`, `session_expire`, `tag_tiddler_with_modifier`, `char_set`, `hashseed`, `debug`, `status`, `tiddlywiki_type`, `default_anonymous_perm`, `default_user_perm`, `rss_group`, `markup_group`) VALUES
('permissions', 'en', 1, 0, 0, 0, 'utf8', '', 0, '', 'TiddlyWiki', '', '', '', '');

-- --------------------------------------------------------

--
-- Table structure for table `login_session`
--

CREATE TABLE `login_session` (
`user_id` varchar(255) NOT NULL,
`session_token` varchar(150) NOT NULL COMMENT 'username+password+time',
`expire` int(20) NOT NULL,
`ip` varchar(15) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

--
-- Dumping data for table `login_session`
--

-- --------------------------------------------------------

--
-- Table structure for table `permissions`
--

CREATE TABLE `permissions` (
`read` int(1) NOT NULL,
`insert` int(1) NOT NULL,
`edit` int(1) NOT NULL,
`delete` int(1) NOT NULL,
`group_name` varchar(50) NOT NULL,
`instance_name` varchar(100) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

--
-- Dumping data for table `permissions`
--

-- --------------------------------------------------------

--
-- Table structure for table `tiddler`
--

CREATE TABLE `tiddler` (
`id` int(11) NOT NULL auto_increment,
`instance_name` varchar(100) NOT NULL,
`title` text NOT NULL,
`body` mediumtext NOT NULL,
`fields` text NOT NULL,
`tags` text NOT NULL,
`modifier` varchar(255) NOT NULL,
`creator` varchar(255) NOT NULL,
`modified` varchar(12) NOT NULL,
`created` varchar(12) NOT NULL,
`version` int(11) NOT NULL,
PRIMARY KEY  (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;

--
-- Dumping data for table `tiddler`
--

-- --------------------------------------------------------

--
-- Table structure for table `tiddler_revision`
--

CREATE TABLE `tiddler_revision` (
`id` int(11) NOT NULL auto_increment,
`title` text NOT NULL,
`body` mediumtext NOT NULL,
`fields` text NOT NULL,
`modified` varchar(12) NOT NULL,
`modifier` varchar(255) NOT NULL,
`revision` int(11) NOT NULL,
`tags` text NOT NULL,
`tiddler_id` int(11) NOT NULL,
PRIMARY KEY  (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;

--
-- Dumping data for table `tiddler_revision`
--

-- --------------------------------------------------------

--
-- Table structure for table `user`
--

CREATE TABLE `user` (
`id` varchar(255) NOT NULL,
`password` varchar(50) NOT NULL,
`short_name` varchar(50) NOT NULL,
`long_name` varchar(100) NOT NULL,
PRIMARY KEY  (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

--
-- Dumping data for table `user`
--

links for 2007-11-04

  • Discover places around you. The places you touch stand out in the map. As your network grows, you see all the places everyone you know has touched as well. It’s easy and it’s fun!
    (tags: location)
  • whrrl demo
  • The parent company of whrrl, Pelago, has raised $7.4 million in funding from Kleiner Perkins Caufield & Byers, Jeff Bezos and Trilogy Equity Partners. Currently, whrrl supports AT&T, Sprint and T-Mobile customers, on about 10 phone models. There will also

Proposed ccTiddly Database Structure

Over the past few week I have been working with cool cold, Martin and Jeremy to decide where we want to take ccTiddly going forward. The idea is to make ccTiddly completely self service without removing any functionality. In order to do this we need to store what is currently stored in the /config/default.php file and store it in a single database.

This is the currently the proposed design. Any comments and suggestions are welcome. I will be publishing more about road map for this project very soon. erm_summary.jpg

erm1.jpg


Note : I have included the original ccTiddly tables in the below SQL.  They will be removed but are in place so that we can test as we build


-- --------------------------------------------------------

--
-- Table structure for table `admin_of_instance`
--

CREATE TABLE `admin_of_instance` (
`user_username` varchar(50) NOT NULL,
`instance_name` varchar(100) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

-- --------------------------------------------------------

--
-- Table structure for table `group_membership`
--

CREATE TABLE `group_membership` (
`user_username` varchar(50) NOT NULL,
`groupname` varchar(50) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

-- --------------------------------------------------------

--
-- Table structure for table `instance`
--

CREATE TABLE `instance` (
`name` varchar(100) NOT NULL,
`lang` varchar(10) NOT NULL,
`keep_revision` int(1) NOT NULL,
`require_login` int(1) NOT NULL,
`cookie_expire` int(1) NOT NULL,
`tag_tiddler_with_modifier` int(1) NOT NULL,
`char_set` varchar(10) NOT NULL,
`hashseed` varchar(50) NOT NULL,
`debug` int(1) NOT NULL,
`status` varchar(10) NOT NULL,
PRIMARY KEY  (`name`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

-- --------------------------------------------------------

--
-- Table structure for table `privileges`
--

CREATE TABLE `privileges` (
`id` int(11) NOT NULL auto_increment,
`read` int(1) NOT NULL,
`insert` int(1) NOT NULL,
`edit` int(1) NOT NULL,
`delete` int(1) NOT NULL,
`group_membership_groupname` varchar(50) NOT NULL,
PRIMARY KEY  (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;

-- --------------------------------------------------------

--
-- Table structure for table `tiddler`
--

CREATE TABLE `tiddler` (
`id` int(11) NOT NULL auto_increment,
`instance_name` varchar(100) NOT NULL,
`title` text NOT NULL,
`body` mediumtext NOT NULL,
`fields` text NOT NULL,
`tags` text NOT NULL,
`modifier` varchar(50) NOT NULL,
`creator` varchar(50) NOT NULL,
`modified` varchar(12) NOT NULL,
`created` varchar(12) NOT NULL,
`version` int(11) NOT NULL,
`epoch_modified` int(15) NOT NULL,
`epoch_created` int(15) NOT NULL,
PRIMARY KEY  (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;

-- --------------------------------------------------------

--
-- Table structure for table `tiddler_revision`
--

CREATE TABLE `tiddler_revision` (
`id` int(11) NOT NULL auto_increment,
`title` text NOT NULL,
`body` text NOT NULL,
`fields` text NOT NULL,
`modified` varchar(12) NOT NULL,
`modifier` varchar(50) NOT NULL,
`revision` int(11) NOT NULL,
`tags` text NOT NULL,
`tiddler_id` int(11) NOT NULL,
`epoch_modified` int(15) NOT NULL,
PRIMARY KEY  (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;

-- --------------------------------------------------------

--
-- Table structure for table `tiddly_wiki_entry`
--

CREATE TABLE `tiddly_wiki_entry` (
`id` int(11) NOT NULL auto_increment,
`title` varchar(255) NOT NULL default '',
`body` text NOT NULL,
`fields` text NOT NULL,
`modified` varchar(128) NOT NULL default '',
`created` varchar(128) NOT NULL default '',
`modifier` varchar(255) NOT NULL default '',
`creator` varchar(255) NOT NULL default '',
`version` int(11) NOT NULL default '0',
`tags` varchar(255) NOT NULL default '',
PRIMARY KEY  (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=2 ;

-- --------------------------------------------------------

--
-- Table structure for table `tiddly_wiki_entry_version`
--

CREATE TABLE `tiddly_wiki_entry_version` (
`id` int(11) NOT NULL auto_increment,
`title` varchar(255) NOT NULL default '',
`body` text NOT NULL,
`fields` text NOT NULL,
`modified` varchar(128) NOT NULL default '',
`modifier` varchar(255) NOT NULL default '',
`version` int(11) NOT NULL default '0',
`tags` varchar(255) NOT NULL default '',
`oid` int(11) NOT NULL,
PRIMARY KEY  (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=3 ;

-- --------------------------------------------------------

--
-- Table structure for table `user`
--

CREATE TABLE `user` (
`username` varchar(50) NOT NULL,
`password` varchar(50) NOT NULL,
`empoyee_id` varchar(200) NOT NULL,
PRIMARY KEY  (`username`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
<code><strong>
</strong></code>