pondelok 5. januára 2015

Message oriented development

Introduction

Happy New Year 2015! As usual start of new year is place to think about what went good and what went wrong in last year. For me last year was time when I felt limitations of OOP and I started rethink my approach to development and looked into alternative methods to achieve goals faster and better. I started to use lambdas and asynchronous chaining more, looked into functional programming approach in F# - highly recommended to at least try, and, as answer to my dissatisfaction in bigger front-end application project, tried to find out better way how to create software. I received initial impulse by reading article Object Oriented Programming is an expensive disaster which must end and I would build on some of the observations mentioned there.

Now, after 6 months of being part of the team working on this particular piece of software using alternative method I would like to share my story and outline architectural decisions and balance out benefits with constraints. I would like to point out first that there is no silver bullet in SW development and as all approaches this one has it drawbacks and cannot be applied across all projects.

Motivation

Last months I’m dealing with desktop application development mostly on frontend using C#, WPF and DirectX (but I believe that experience can easily translate to other languages/frameworks) and approximately after 3 months we started hitting first walls using OOP approach. Our target was to have unit-testable code that has good separation of responsibilities but somehow we always ended up in quite messy entanglement of code where any change was causing unpredictable side effects.

In following article I will try to list out conceptual issues that rises from using “traditional OOP” approach in frontend applications; of course those ideas are not new (I was able to find that this concept was mentioned by Alan Key as Actor Model) but I think they provide nice complement to OOP deficiencies.

One target to rule them all

Applications (especially frontend applications) very rarely have just one point of entry for actions. For example action Close application has usually multiple entry points - actions passed from operating system (user presses Alt+F4) and inside application code (user selects File->Exit from main menu). As developers we would like to have unified way how to handle this situation (e.g. check if there are unsaved data available and prompt user with Do you want to save your data? dialog before exiting).

Naturally we would like to have separate component that deals with this workflow in one place, let’s say DataManager:

public class DataManager
{
    public void CheckSaveBeforeExit()
    { /* Do your stuff */  }
}

However, how we are going to invoke this method? Usually code dealing with window management is in different class then one dealing with menu so we need to have instance of DataManager in both classes in order to invoke the method. We don’t want to create multiple DataManagers, singleton is not a good idea since it kills the unit testing so we have to either pass DataManager through property or constructor:

public class MyWindow
{
    private DataManager _dataManager;

    public MyWindow(DataManager dataManager)
    {
        _dataManager = dataManager;
    }

    protected void OnExit()
    {
        _dataManager.CheckSaveBeforeExit();
    }
}

This of course effectively prevents us to perform unit testing - in unit tests we don’t want to save data to HDD every time we pefrom the test but we would like to examine data for validity. Latest response to this issue is to come up with interface to be able to mock up DataManager:

public interface IDataManager
{
    void CheckSaveBeforeExit();
}

Now retrieve real implementation via IoC and configure IoCManager in some startup configuration file (for simplicity directly in code but this can be done by injecting into constructor or property):

protected void OnExit()
{
    IoCManager.GetInstance<IDataManager>().CheckSaveBeforeExit();
}

Please note amount of boilerplate code that we need to do in order to support this very common scenario while giving us fragile and hard to reuse structure - IDataManager will most probably also contain other methods (Load, Save, Reset etc.) which are completely irrelevant and there is no clear indication which parts of IDataManager are required. Worse yet DataManager will probably have dependencies on it’s own (database provider).

State is your enemy

Remembering and relying on state setup by other action one of the worst things that can be done in SW development since very few operations are strictly tied to one state. Continuing from example above while writing Load() method we will most probably would like to perform it in separate thread so we can keep UI responsive but we would also like to update UI components to be disabled.

OOP guides us to provide IsLoading accessor which will indicate if data are loaded:

public class DataManager
{
    private bool isLoading;
    public book IsLoading { get { return isLoading; } }

    public void Load()
    {
        isLoading = true;

        // ... Load your data asynchronously

        isLoading = false;
    }
}

Any other object that will try to observe this accessor and mutate it’s state based on this value (e.g. disable the button, reset the undo stack etc.) is creating another dimension of complexity in your code since the value of IsLoading can be changed in the instant program control leaves the IsLoading statement - DataManager is no longer loading data but button is still disabled.

To counteract this behavior you actually need to build some kind of Observer pattern - please note that promise of accessor (access the object data safely) is not kept and developers needs to create plenty of boilerplate code.

You wanted a banana but what you got was a gorilla holding the banana and the entire jungle

by Joe Armstrong

Let’s imagine scenario with class encapsulating raster graphics device (e.g. Window) and Rectangle - class that draws the rectangle to the Window using Draw() method. Per OOP philosophy Window should contain accessors (e.g. ones that holds size of the window using properties Width and Height) and actions of them have Draw() method which does the drawing. We derive MyWindow and implement the requirements. In our scenario this can be nicely demonstrated by requirement to draw rectangle into center of window.

First approach could be that instance of Rectangle is created inside MyWindow, positioned properly before each draw. Later on we find out that rectangle should not be displayed while loading the data and if data is corrupted it should be actually half the size and with red border. All this logic goes into MyWindow and it’s harder and harder to maintain so we want to separate all logic relative to this rectangle into own class MySpecialRectangle. The question is how to pass information about Window size in order to keep Rectangle centered:

  • Option #1 - passing whole Window object to Draw method; now we need to ensure that we actually have valid instance passed (at least check for null). This however goes against reusability - what about case that we would like to draw same Rectangle to different container component (e.g. Grid)? Procedure of drawing of rectangle is the same but we are now coupled to Window-related environment.
  • Option #2 - OK, let’s separate what we need into interface IUIComponent, implement this on Window and Grid. How to pass this interface to the MySpecialRectangle - we can e.g. pass it in constructor or pass it as property (and again face the possiblity that developer using this class in other context will forget to initialize properly).
  • Option #3 - pass the relevant data in constructor (pass two numbers indicating ParentWidth and ParentHeight) - how to then take care of possible Window size update when user resizes the window? We need to create new method OnSizeUpdate, OnLoadingUpdate, OnLoadingErrorUpdate and since loading of data is done in separate class MyWindow needs to listen to those changes and forward it to MySpecialRectangle.

In all cases we either end up solving problem with contextual class and not focusing on what our class should do. This is caused by a fact that we are trying to pass too much of the data - our class consumes only very specific subset of information however OOP doesn’t give us good tool how to pass just information that we are interested in.

Message oriented development

At core of all issues mentioned above are two facts:

  • OOP promotes “data with actions” approach
  • OOP doesn’t allow easy messaging

To counter this we are using following design principles when creating new classes:

  • No public accessors on components and limit public methods to minimum (e.g. to use 3rd party libraries)
  • Pass information only by using immutable messages and without knowing the sender
  • Prefer immutable data containers with no methods
  • Attach to relevant messages in constructor

To support this we are using Messenger class of MVVM Light framework (but this can be achieved also by custom messenger implementation). The idea is that each change in application should be distributed by specific message holding data that cannot be changed during processing.

Loose coupling invocation

When Window detects change of size it will Send new WindowChangedMessage which will contain readonly properties exposing updated Width and Height and Rectangle will Register to consume this message in constructor:

public class WindowChangedMessage
{
    public int Width { get; private set; }
    public int Height { get; private set; }

    public WindowChangedMessage(int width, int height)
    {
        Width = width;
        Height = height;
    }
}

public class Window
{
    // Called when size is updated
    protected void OnSizeUpdate(int width, int height)
    {
        Messenger.Default.Send(new WindowChangedMessage(width, height));
    }
}

public class Rectangle
{
    protected Point position;

    public void Draw()
    {
        // Draw rectangle at position
    }
}

public class MySpecialRectangle : Rectangle
{
    public MySpecialRectangle()
    {
        Messenger.Register<WindowChangedMessage>(this, OnWindowChangedMessage);
    }

    private void OnWindowChangedMessage(WindowChangedMessage message)
    {
        // Calculate new position to appear in center
        position = new Point(...);
    }
}

Note that classes are completely isolated and they have no direct knowledge about each other (compared to e.g. C# events) - each class holds all information that it needs for it’s proper work and can subscribe to additional messages.

Same approach can be utilized for loading sequence:

public class LoadDataMessage
{ /* No data needed */ }

public class DataLoadedMessage
{ /* No data needed */ }

public class DataManager
{
    public DataManager()
    {
        Messenger.Register<LoadDataMessage>(this, OnLoadDataMessage);
    }

    private void OnLoadDataMessage(LoadDataMessage message)
    {
        // ... Load data

        Messenger.Default.Send(new DataLoadedMessage());
    }
}

public class Rectangle
{
    protected bool isVisible;

    public void Draw()
    {
        if (isVisible)
        {
            // Draw rectangle at position
        }
    }
}

public class MySpecialRectangle : Rectangle
{
    public MySpecialRectangle()
    {
        Messenger.Register<LoadDataMessage>(this, OnLoadDataMessage);
        Messenger.Register<DataLoadedMessage>(this, OnDataLoadedMessage);
    }

    private void OnLoadDataMessage(LoadDataMessage message)
    {
        isVisible = false;
    }

    private void OnDataLoadedMessage(DataLoadedMessage message)
    {
        isVisible = true;
    }
}

This arrangement clearly points out to business rules attached to MySpecialRectangle - we don’t care who invoked the LoadDataMessage (it can sent at start of application or by user action to reload the data). If we want to change behavior of MySpecialRectangle we don’t need to browse through multiple classes and look from which context other classes sets isVisible - it’s set only in one class.

Chaining

It’s very easy to chain messages to achieve sequence of actions - for example if user clicks on Close button we would like to check if data needs to be saved. However user can save data also by selecting File->Save:

   [click Exit]           [click Save]
        |                      |
        v                      v
RequestExitMessage +->  SaveDataMessage --> DataSavedMessage -\
                    \------------------------------------------+> ExitApplicationMessage

We need to know context or continuation of action which can be reinforced in constructors:

public class RequestExitMessage
{ }

public class SaveDataMessage
{
    public bool ExitAfterSave { get; private set; }

    public SaveDataMessage() {}
    public SaveDataMessage(RequestExitMessage message) { ExitAfterSave = true; }
}

public class DataSavedMessage
{
    public bool ExitAfterSave { get; private set; }

    public DataSavedMessage() {}
    public DataSavedMessage(SaveDataMessage message) { ExitAfterSave = message.ExitAfterSave; }
}

public class DataManager
{
    private bool isDirty;

    public DataManager()
    {
        Messenger.Register<RequestExitMessage>(this, OnRequestExitMessage);
        Messenger.Register<SaveDataMessage>(this, OnSaveDataMessage);
        Messenger.Register<DataSavedMessage>(this, OnDataSavedMessage);
    }

    private void OnDataSavedMessage(DataSavedMessage message)
    {
        if (message.ExitAfterSave)
        {
            Messenger.Default.Send(new ExitApplicationMessage());
        }
    }

    private void OnRequestExitMessage(RequestExitMessage message)
    {
        if (isDirty)
        {
            Messenger.Default.Send(new SaveDataMessage(message));
        }
        else
        {
            Messenger.Default.Send(new ExitApplicationMessage());
        }
    }

    private void OnSaveDataMessage(SaveDataMessage message)
    {
        // Save data
        Messenger.Default.Send(new DataSavedMessage(message));
    }
}

By creating messages in this manner consumer of message will be able to quickly determine context of message - SaveDataMessage should be either constructed without parameters or developer should use RequestExitMessage to get proper chaining.

This example also shows that we are dealing with only one, very specific aspect of saving the data - actual saving to HDD. We are not mixing how UI is changed (indication that data are saved), we don’t care about how actual application exit looks like (close all windows etc.). Those aspects of this message can be implemented in separate classes by observing same messages. Better yet, we can very easily remove whole Save feature as long as we keep correct sequence of messages (e.g. in demo application).

Response messaging

So far we covered only fire-and-forget type of message when we actually doesn’t care about result value from the other component; in case that we need to get some values from receivers we need to incorporate two or more messages.
For example let’s imagine that we would like to build handling of click events from active components on the screen. Normally this goes in three steps:

  1. When mouse goes down determine if there is some element under mouse position
  2. When mouse goes up test if mouse position is still within the same element (if user moves the mouse outside element where it started the click and release it should not be considered as click)

Let’s create ClickHandler class which will receive mouse signals from OS and Button classes that will contain information about active element. Since we don’t want to expose internal values of elements we will share only necessary amount of information in following sequence of calls:

  1. ClickHandler will send HitTestMessage on mouse down with position to all elements to determine if any of them contains mouse position; if yes element will send HitTestPassedMessage with it’s identifier and valid area for element
  2. ClickHandler will store HitTestPassedMessage (if any)
  3. On mouse up ClickHandler will test if mouse position is still within element’s valid area and send ClickMessage with ID of element
  4. Element will process ClickMessage and check if it’s ID is matching message ID
public class HitTestMessage
{
    public Point Position { get; private set; }
    public HitTestMessage(Point position) { Position = position; }
}

public class HitTestPassedMessage
{
    public Rect ValidArea { get; private set; }
    public string Id { get; private set; }

    public HitTestPassedMessage(string id, Rect validArea) 
    {
        Id = id;
        ValidArea = validArea;
    }
}

public class ClickMessage
{
    public string Id { get; private set; }
    public ClickMessage(string id) { Id = id; }
}

public class Button
{
    protected readonly string _id;
    private Rect _rectangle;

    public Button()
    {
        _id = Guid.NewGuid().ToString();
        Messenger.Default.Register<HitTestMessage>(this, OnHitTestMessage);
        Messenger.Default.Register<ClickMessage>(this, OnClickMessage);
    }

    private void OnHitTestMessage(HitTestMessage message)
    {
        if (_rectangle.Contains(message.Position))
        {
            Messenger.Default.Send(new HitTestPassedMessage(_id, _rectangle));
        }
    }

    private void OnClickMessage(ClickMessage message)
    {
        if (message.Id == _id)
        {
            // Perform click action
        }
    }
}

public class ClickHandler
{
    private HitTestPassedMessage _lastMessage;

    public ClickHandler()
    {
        Messenger.Default.Register<HitTestPassedMessage>(this, OnHitTestPassedMessage);
    }

    public void OnHitTestPassedMessage(HitTestPassedMessage message)
    {
        _lastMessage = message;
    }

    private OnMouseDown(Point position)
    {
        Messenger.Default.Send(new HitTestMessage(position));
    }

    private OnMouseUp(Point position)
    {
        if (null != _lastMessage)
        {
            var message = _lastMessage;
            _lastMessage = null;

            if (message.ValidArea.Contains(position))
            {
                Messenger.Default.Send(new ClickMessage(message.Id));
            }
        }
    }
}

I would like to again point out complete isolation between classes - developers can easily create unit tests and send/observe messages and test proper behavior of the ClickHandler without creating mocks, configuring any IoC containers or other constructs. It’s also worth to mention here that with this architecture it’s easy to convert mouse handling in Button to separate component and use it as mixin for any rectangular-shaped active area.

Scalability/Extendability

Couple of additional points favoring this style of development:

  • It’s very easy to convert actions to asynchronous execution if needed without changing calling sequence (compared to e.g. await/async methodology which will at least require all callers to include await before invoking the method). In DataManager save example developer can easily convert OnSaveDataMessage to perform action asynchronously without having to worry about other components:
private void OnSaveDataMessage(SaveDataMessage message)
{
    Task.Factory.StartNew(message => {
        // Save data
    }).ContinueWith(task => {
        Messenger.Default.Send(new DataSavedMessage(message));
    });
} 
  • It’s also easy to reuse developed components to other project - we just need to extract related components and related messages
  • Due to nature most of the application is stateless so it can be easily transformed to multi-core or even multi-computer application by sending/receiving messages by inter-process messaging zeromq or via sockets.

streda 17. júla 2013

Mobile webapps with nodejs & typescript - Part III

In Part I we setup basic project and prepared output package for mobile device. This got us some benefits coming from statically typed system provided by Typescript, dependency loading and given us some automation in building and possible integration with Continuous Integration systems.
In Part II we layed out infrastructure for easy maintenance of large projects with Jade, Stylus and Q/A tools.
This part will cover bootstraping of Require.js application of data binding for applications, namely MVC and MVVC pattern with Knockout.js.

Bootstraping

In order to effectively use AMD system from Require.js library have to expose it's components via export keyword instead of providing global variables. Although this is becoming standard way how to distribute JS libraries there are still libraries that does it in old way and that might confuse Require.js. 
To be able to use such libraries we can bootstrap our application by using shims. In this part we will be using Zepto.js (lightweight counterpart to jQuery) and Knockout.js which both exposes global objects - in case of Zepto it's $ like in jQuery and in case of Knockout it's knockout.
We will continue in project from last time where add minified zepto.min.js and knockout-2.3.0.js into lib folder. Now we need to tell Require.js where to look for those files; this can be easily achieved by using shim functionality and defining configuration file:

requirejs.config({
    paths: {
        "zepto": "libs/zepto.min",
        "knockout": "libs/knockout-2.3.0"
    },
    shim: {
        zepto: {
            exports: "$"
        }
    }
});

Please note that we are not providing .js extension in paths section. 
Since we will be using those libraries across the project this configuration needs to occur before first module tries to load it up. This brings interesting chicken-and-egg problem since Typescript generates declare statements as first lines in the file so if we try to put this configuration into Typescript file and load e.g. application.ts which in turn uses Zepto it will fail since at the moment of loading configuration was not yet processed. 

There are two ways out of this problem - one is to write config.js in pure Javascript and do imports in correct places or use mixed approach. In this example we will use latter one where we benefit from fact that Javascript code is fully valid in Typescript files. We will update src/config.ts to following:

/// <reference path='../extern/require.d.ts'/>

"use strict";

requirejs.config({
    paths: {
        "zepto": "zepto.min",
        "knockout": "knockout-2.3.0"
    },
    shim: {
        zepto: {
            exports: "$"
        }
    }
});

// Stage one
require(['zepto', 'knockout'], (zepto, knockout) => {

         // Stage two
    require(['main'], main => {
    new main.Main().run();
    });
});


As you can see we are loading dependencies in two stages - reason for that is that Require.js is practicing lazy loading but in this special case we want to have all those libraries pre-loaded before continuing.  
Other thing that is worth noticing is that we are referencing .d.ts files in header - without this information Typescript will not know about Require.js objects and functions. We can obtain this file from e.g. DefinitelyTyped site and put:
  • require.d.ts files into /extern/require.d.ts
  • zepto.d.ts into /extern/zepto.d.ts (we will use this one later)
  • knockout.d.ts into /extern/knockout.d.ts (we will use this one later)
  • knockout.amd.d.ts into /extern/knockout.amd.d.ts (we will use this one later - this one is AMD wrapper around standard Knockout)
Those files stands for Definition TypeScript - they contain only definitions of objects not real implementation. You can find more information about this in Typescript Language Reference.

Note: Typescript recently got support for generics and some libraries are already using this concept; previous package.json was pointed to older version of Typescript so it should be updated (line 14) to:

"grunt-typescript": "0.2.1",

Last piece of change we need to do is to tell Grunt to copy all libraries from lib folder; change line 19 in Gruntfile.js to:

{ flatten: true, expand: true, src: ['lib/*.js'], dest: 'dist/www/js/'}

With all infrastructure in place we can start using all libraries as usual with all benefits of Typescript and lazy-loading. Quick example could be updating src/main.ts to following:

/// <reference path='../extern/zepto.d.ts'/>

var $ = require("zepto");

export class Main  {
    constructor() {
        console.log("main.init()");

$(".corpus").click(() => {
    alert("hello world!");
}));
    }

    run() {        
        console.log("main.run()");
    }
}

Which should display message box with message "hello world" after clicking on text in browser.

Knockout.js

So far it was more about talking on laying out infrastructure rather than doing real work. In this chapter we take a look on how to utilize all of the components we setup so far and we will build simple menu for us. 
First, let's introduce Knockout.js library; it's Model-View-ViewModel pattern based binding library which will be familiar to Windows Phone developers. 
For those unaware of concept I recommend try try live examples on Knockout web pages but in our case we will split functionality as follows:
  • Model - simple JSON files with data to be displayed
  • View - HTML page generated from Jade and CSS styles from Stylus
  • ViewModel - Typescript compiled into Javascript binding view to model and acting on events
Let's start with Model - we create new file src/menuModel.ts which will contain items that we wish to display:

export interface MenuItem {
    name: string;
    icon: string;
    id: number;
};

export var items: MenuItem[] = [
    { id: 0, name: "Home", icon: "home.png" },
    { id: 1, name: "Items", icon: "items.png" },
    { id: 2, name: "Settings", icon: "settings.png" }
];

File defines interface on data displayed in menu and menu items itself - this information can also come e.g. from AJAX call. 

Now to prepare ViewModel which will establish data relation between View and Model. We will update src/main.ts to following:

/// <reference path='../extern/knockout.amd.d.ts'/>

var $ = require("zepto");

import menuModel = module("./menuModel");
import ko = module("knockout");

export class Main  {
    menuItems: KnockoutObservableArray<menuModel.MenuItem>;

    constructor() {
        console.log("main.init()");
        this.menuItems = ko.observableArray(menuModel.items);
    }

    run() {        
        console.log("main.run()");
        ko.applyBindings(this);
    }
}

Here we prepare observable array of our menu items for Knockout.js and we will bind this as main object. This is everything we need to do for display purposes - Knockout will handle everything else for us.

Last part is preparing View - since we most probably will be using same menu on multiple pages we will extract all relevant definition into new file views\menu.jade:  

ul.menu(data-bind="foreach: menuItems")
    li(data-bind="text: name")

First line created UL element which will be bound to property named menuItems; since we are using foreach keyword it is expected that this property will be Array. Everything that is declared within this element will be copied as many times as is count of items in collection. Second line says it should be LI element and text should be bound to property name of every item of collection menuItems.

Since we want to create separate HTML files not one big one we need to update Gruntfile.js lines 45-47:

files: [
     { expand: true, ext: ".html", cwd: "views/", src: ['*.jade'], dest: 'dist/www/' }
]

Last bit of functionality is to include menu into our index view - we will update views/index.jade:

html
    head
         link(rel='stylesheet', href='css/index.css')
 script(data-main='js/config', src='js/require.js')
    body
        include menu.jade
                   .corpus Hello, world!

That was easy, wasn't it? Unfortunately this doesn't look too much like menu so we extend this example bit more; let's style it into more conventional way (styles/index.styl):

body
    font: 100% "Trebuchet MS", sans-serif

.canvas
    margin: 8px

.menu
    list-style-type none
    padding 0px
    margin 0px

.menu li
    width 64px
    height 64px
    display inline-block
    margin 0 2px

.menu .selected
    font-weight bold

So we would like to make menu item bold when it's selected; in order to achieve this we first need to know which menu item is selected. This should be extracted to menu component (e.g. src/menu.ts) but for sake of simplicity we put it into src/main.ts:

/// <reference path='../extern/knockout.amd.d.ts'/>

var $ = require("zepto");

import menuModel = module("./menuModel");
import ko = module("knockout");

export class Main  {
    menuItems: KnockoutObservableArray<menuModel.MenuItem>;
    selectedMenu: KnockoutObservable<number>;

    constructor() {
        console.log("main.init()");
        this.menuItems = ko.observableArray(menuModel.items);
        this.selectedMenu = ko.observable(menuModel.items[0].id);
    }

    run() {        
        console.log("main.run()");
        ko.applyBindings(this);
    }

    selectMenu(id: number) {    
    this.selectedMenu(id);
    }
}

Please note that in order to update value we need to perform function call instead. This is often source of issues.
Now we know which menu item is selected and we need to bind this information to UI and propagate this changes back to ViewModel; we will update views/menu.jade to achieve this:

ul.menu(data-bind="foreach: menuItems")
li(data-bind="text: name, css: { selected: $root.selectedMenu() == id }, event: { click: $root.selectMenu.bind($root, id) }")

Again, please note that in order to obtain value we need to perform function call. Object $root in this case contains reference to top-level ViewModel object (in our case instance of class Main). Since Knockout.js processes events on our behalf context of methods called this will be always within Knockout unless we will bind it to correct context (in this case $root).

Compile with grunt and enjoy your menu that was built without doing any event handling or CSS operations! As usual you can find zipped archive with complete example here.

Back to Part II.

utorok 9. apríla 2013

Mobile webapps with nodejs & typescript - Part II

Overview

In Part I we setup basic project and prepared output package for mobile device. This got us some benefits coming from statically typed system provided by Typescript, dependency loading and given us some automation in building and possible integration with Continuous Integration systems.
This part will cover additional tools that can be used to provide better cooperation with other developers and template engines. We will cover following topics:
  • Templates of view 
  • Templates of CSS styles
  • Maintenance

Jade - view templates

Jade is powerful template engine for HTML pages or snippets that can be controlled via models passed. Traditionally it's more used for web servers to provide base templates however with our setup we can easily reuse this component to prepare mobile web applications and take away some of redundancy coming to play with .
To start with we install Jade task for Grunt:

> npm install grunt-contrib-jade --save-dev

Now we can update our existing views/index.html file to views/index.jade file:

html
head
script(data-main='js/config', src='js/require.js')
body
#corpus Hello, world!

And finally update our Gruntfile.js to execute new task:

module.exports = function(grunt) {
  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    typescript: {
      base: {
        src: ['src/**/*.ts'],
        dest: './dist/www/js',
        options: {
          module: 'amd', //or commonjs
          target: 'es5', //or es3
          base_path: 'src'
        }
      }
    },
    copy: {
      libs: {
        files: [
          { flatten: true, expand: true, src: ['lib/require.js'], dest: 'dist/www/js/'}
        ]
      },     
      tizen: {
        files: [
          { flatten: true, expand: true, src: ['platform/tizen/**'], dest: 'dist/tizen'},          
          { flatten: true, expand: true, src: ['dist/www/index.html'], dest: 'dist/tizen'},
          { flatten: true, expand: true, src: ['dist/www/js/*'], dest: 'dist/tizen/js'}          
        ]
      }
    },
    zip: {
      tizen: {
        src: 'dist/tizen/*',
        cwd: 'dist/tizen',
        dest: 'dist/helloWorld.wgt'
      }
    },
    jade: {
      compile: {
        options: {
          data: {
            debug: true
          }
        },
        files: {
          "dist/www/index.html": ["views/*.jade"]
        }
      }
    }
  });

  grunt.loadNpmTasks('grunt-typescript');
  grunt.loadNpmTasks('grunt-contrib-copy');
  grunt.loadNpmTasks('grunt-zip');
  grunt.loadNpmTasks('grunt-contrib-jade');

  // Default task(s).
  grunt.registerTask('default', ['copy:libs', 'typescript', 'jade']);
  grunt.registerTask('tizen', ['default', 'copy:tizen', 'zip:tizen']);
};

This approach has added benefit of simpler structure (no need to use closing tags), better readability and ability to use inheritance, blocks and includes.

Stylus - CSS templates

Similar approach as for views can be used for CSS by using Stylus; in this case benefit is less obvious but with help of in-line conditioning and functions which can take away some of the maintenance problems. Approach to integrate with our solution is straightforward:

> npm install grunt-contrib-stylus --save-dev

Put initial style definition into styles/index.styl:

body
font: 62.5% "Trebuchet MS", sans-serif

#canvas
margin: 8px

Update views/index.jade:

html
head
link(rel='stylesheet', href='css/index.css')
script(data-main='js/config', src='js/require.js')
body
#corpus Hello, world!

And finally update Gruntfile.js:

module.exports = function(grunt) {
  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    typescript: {
      base: {
        src: ['src/**/*.ts'],
        dest: './dist/www/js',
        options: {
          module: 'amd', //or commonjs
          target: 'es5', //or es3
          base_path: 'src'
        }
      }
    },
    copy: {
      libs: {
        files: [
          { flatten: true, expand: true, src: ['lib/require.js'], dest: 'dist/www/js/'}
        ]
      },     
      tizen: {
        files: [
          { flatten: true, expand: true, src: ['platform/tizen/**'], dest: 'dist/tizen'},          
          { flatten: true, expand: true, src: ['dist/www/index.html'], dest: 'dist/tizen'},
          { flatten: true, expand: true, src: ['dist/www/js/*'], dest: 'dist/tizen/js'},
          { flatten: true, expand: true, src: ['dist/www/css/*'], dest: 'dist/tizen/css'}
        ]
      }
    },
    zip: {
      tizen: {
        src: 'dist/tizen/*',
        cwd: 'dist/tizen',
        dest: 'dist/helloWorld.wgt'
      }
    },
    jade: {
      compile: {
        options: {
          data: {
            debug: true
          }
        },
        files: {
          "dist/www/index.html": ["views/*.jade"]
        }
      }
    },
    stylus: {
      compile: {
        files: {
          'dist/www/css/index.css': 'styles/index.styl'
        }
      }
    }
  });

  grunt.loadNpmTasks('grunt-typescript');
  grunt.loadNpmTasks('grunt-contrib-copy');
  grunt.loadNpmTasks('grunt-zip');
  grunt.loadNpmTasks('grunt-contrib-jade');
  grunt.loadNpmTasks('grunt-contrib-stylus');

  // Default task(s).
  grunt.registerTask('default', ['copy:libs', 'typescript', 'jade', 'stylus']);
  grunt.registerTask('tizen', ['default', 'copy:tizen', 'zip:tizen']);
};

Maintenance

Important part of software development process is making sure that everyone in the team starts from same spot and follows same rules and avoid common mistakes. First part of this process can be automated by linting (CSSLint, JSLint, JSHint etc.), unit testing (QUnit), adding clean task and second part can be done by code peer reviews. Let's start by adding some more tasks:

> npm install grunt-contrib-csslint grunt-contrib-jshint grunt-contrib-qunit grunt-contrib-clean --save-dev

Now let's make clean & lint part of our daily workflow by updating Gruntfile.js:

module.exports = function(grunt) {
  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    typescript: {
      base: {
        src: ['src/**/*.ts'],
        dest: './dist/www/js',
        options: {
          module: 'amd', //or commonjs
          target: 'es5', //or es3
          base_path: 'src'
        }
      }
    },
    copy: {
      libs: {
        files: [
          { flatten: true, expand: true, src: ['lib/require.js'], dest: 'dist/www/js/'}
        ]
      },     
      tizen: {
        files: [
          { flatten: true, expand: true, src: ['platform/tizen/**'], dest: 'dist/tizen'},          
          { flatten: true, expand: true, src: ['dist/www/index.html'], dest: 'dist/tizen'},
          { flatten: true, expand: true, src: ['dist/www/js/*'], dest: 'dist/tizen/js'},
          { flatten: true, expand: true, src: ['dist/www/css/*'], dest: 'dist/tizen/css'}
        ]
      }
    },
    zip: {
      tizen: {
        src: 'dist/tizen/*',
        cwd: 'dist/tizen',
        dest: 'dist/helloWorld.wgt'
      }
    },
    jade: {
      compile: {
        options: {
          data: {
            debug: true
          }
        },
        files: {
          "dist/www/index.html": ["views/*.jade"]
        }
      }
    },
    stylus: {
      compile: {
        files: {
          'dist/www/css/index.css': 'styles/index.styl'
        }
      }
    },
    csslint: {
      strict: {
        options: {
          import: 2
        },
        src: ['dist/www/css/*.css']
      }
    },
    jshint: { 
      files: [
        'dist/www/js/main.js',
        'dist/www/js/config.js'
      ],            
      options: { 
        force: true,  // Don't fail hard ..
        browser: true,        
        devel: true,
        globals: {define: true},
      }
    },
    clean: ['dist']
  });

  grunt.loadNpmTasks('grunt-typescript');
  grunt.loadNpmTasks('grunt-contrib-copy');
  grunt.loadNpmTasks('grunt-zip');
  grunt.loadNpmTasks('grunt-contrib-jade');
  grunt.loadNpmTasks('grunt-contrib-stylus');
  grunt.loadNpmTasks('grunt-contrib-csslint');
  grunt.loadNpmTasks('grunt-contrib-jshint');
  grunt.loadNpmTasks('grunt-contrib-clean');

  // Default task(s).
  grunt.registerTask('default', ['copy:libs', 'typescript', 'jade', 'stylus', 'csslint', 'jshint']);  
  grunt.registerTask('tizen', ['clean', 'default', 'copy:tizen', 'zip:tizen']);
};

By running grunt command we will run into couple of issues in both CSS and JS files (bad, bad developer!). Most of those can be easily resolved except for W033 (missing semicolon) which is not generated by Typescript compiler in current version so we turn off hard fail for JS validation.
With clean task integrated into tizen target we can ensure that everybod will always receive same build from same source files; clean task can be run also directly to force cleaning of dist directory.

Complete project can be downloaded here without node_modules folder so it is necessary to run following command from project directory to load dependencies:

> npm install .

Continue to Part III or back to Part I

piatok 5. apríla 2013

Mobile webapps with nodejs & typescript - Part I

Overview

With more and more powerful mobile devices spreading into population we can see that mobile web applications are coming to boom also on mobile devices. This brings in new set of people working in this segment looking into this way of development in new way.

Traditionally web applications require desktop computer which is usually by magnitude better than average mobile device; in combination with known issues of web development (scattered platform, Javascript etc.) and the mobile-world limitations like limited available memory and external events like low battery, incoming calls etc. this can lead into quick frustration and dismissing the platform.

Webapps definitely have their place in current ecosystems as tool for quick prototyping and showcasing features but looking forward into heavy investments in Tizen, Firefox OS and Ubuntu Mobile promises that in short time frame this can become viable option also for production-quality application. Still, from the nature of tools used in this area it quickly becomes pain to maintain and add new features.

One of the major contributor to this fact is that Javascript as main language is designed with other purposes (simplicity, easy learning curve) while more traditional languages like Java, C# or C++ focuses more on larger scale project maintainability. In this article I’ll try to target following issues:

  • Static typing in Javascript
  • Module loading in larger projects
  • Packaging for mobile platform
  • Infrastructure

As main tool in this article I’ll be using Node.js and created setup that can be easily deployed to Continuous Integration system of your choice. Choice of Node.js is driven by fact that is server-side Javascript engine which easily integrates with our web app (no need of special XML files etc.) and provides good toolset for web app development.

Assuming installed Node.js (current version is v0.10.3) we start by laying down some infrastructure and creating new package in new directory (e.g. helloworld):

npm init 

Now we configure our new project by answering couple of questions (name, version, entry point etc. - defaults are fine) and we will be granted with newly created package.json file which serves as descriptor of our package.

Now to install Grunt (current version is v0.4.1) that we will be using to chain our toolset and save us some headaches (and of course create whole set of new ones). We will need to install it globally to get new grunt command by invoking:

npm install -g grunt-cli 

Main driver of Grunt system is Gruntfile.js file that can be created from template or by hand. Since templates are quite complex we will start by very simple Gruntfile that will be extended:

module.exports = function(grunt) {
    // Project configuration.
    grunt.initConfig({
      pkg: grunt.file.readJSON('package.json'),
      typescript: {
        base: {
          src: ['src/**/*.ts'],
          dest: './dist/www/js',
          options: {
            module: 'amd', //or commonjs
            target: 'es5', //or es3
            base_path: 'src'
          }
        }
      }
    });
    grunt.loadNpmTasks('grunt-typescript');
    grunt.registerTask('default', ['typescript']);
}; 

We still need to install local copy of Grunt + tasks into our project:

npm install grunt --save-dev 

Option --save-dev tells NPM that we will need to have this package only for development - our final codes will don’t need that. And now we should be able to invoke Grunt command from command line to run our builds system:

grunt 

Success! By invoking Grunt without parameters we run default task. Now to prepare some folder structure:

  • dist - all output JS, CSS & HTML pages, platform packages etc.
  • lib - external libraries
  • images - images for our project
  • platform - platform specific files
  • styles - Stylus or CSS styles
  • src - Typescript files
  • views - Jade or HTML pages

Typescript

Typescript is new kid on the block from Microsoft which aims to help with most critical issues of Javascript for larger projects - module loading, classes & interfaces and static typing. Great thing about Typescript is that it’s superset of JS - this means that every JS file is automatically Typescript and Typescript follows proposed features for ECMAScript 6 which in future should allow Typescript files without need of compilation.
For now we still need to compile Typescript into Javascript so we will install new Grunt task

npm install grunt-typescript --save-dev 

Now we need to tell Grunt to load new tasks and tell it how to translate Typescript files into Javascript by updating Gruntfile.js:

module.exports = function(grunt) {
  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    typescript: {
      base: {
        src: ['src/**/*.ts'],
        dest: './dist/www/js',
        options: {
          module: 'amd', //or commonjs
          target: 'es5', //or es3
          base_path: 'src'
        }
      }
    }
  });

  grunt.loadNpmTasks('grunt-typescript');

  grunt.registerTask('default', ['typescript']);
}; 

And create our main Typescript file (into src/main.ts):

export class Main  {
    constructor() {
        console.log("main.init()");
    }

    run() { 
     var self = this;
        console.log("main.run()");
    }
} 

Now we can run grunt command again and we should get compiled main.js in dist/www. Reason for adding specific www folder is that it’s usually faster to do first check in desktop browser like Chrome rather than building it for specific platform - this desktop version will be our default target that will be later incorporated into target for building for specific platform.

File that we got is build for dynamic module loading via AMD mechanism - unfortunately no current browser supports this so we need to add support for this.

Dynamic module loading

Usual problem with larger web projects is separation of code modules and code dependencies - in order to keep project maintainable we need to split functionality into multiple JS files, however there is no way to tell that this JS module requires other JS module directly in JS file since this needs to be specified in HTML file by script tag (and usually also in correct order).

To resolve this issue script loaders like RequireJS uses AMD format to specify which modules are required for script. Fortunately Typescript can export module dependencies in AMD format as well which makes it suitable for our purposes.

We will start by creating src/config.ts (which will serve as main entry point into application and in future will hold also RequireJS shim configuration for external libraries) but for now it will be pretty simple:

import main = module("main");

var Main = new main.Main();
Main.run(); 

This code will import main.ts file, instantiate Main class and runs Main.run() method. Now we need to create sample web page that will demonstrate module loading (views/index.html):

<html>
 <head>    
  <script data-main="js/config" src="js/require.js">
 </script>
</head>
<body>
 <div id="corpus">
  Hello world!
 </div>
 </body>
</html> 

Additionaly, we download RequireJS file into libs/require.js and tell Grunt to copy both HTML and JS files into their respective locations. For this we will need contrib-copy task:

npm install grunt-contrib-copy --save-dev 

And update Gruntfile.js:

module.exports = function(grunt) {
  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    typescript: {
      base: {
        src: ['src/**/*.ts'],
        dest: './dist/www/js',
        options: {
          module: 'amd', //or commonjs
          target: 'es5', //or es3
          base_path: 'src'
        }
      }
    },
    copy: {
      libs: {
        files: [
          { flatten: true, expand: true, src: ['lib/require.js'], dest: 'dist/www/js/'}
        ]
      },
      html: {
        files: [
          { flatten: true, expand: true, src: ['views/index.html'], dest: 'dist/www'}          
        ]
      }
    },
  });

  grunt.loadNpmTasks('grunt-typescript');
  grunt.loadNpmTasks('grunt-contrib-copy');

  // Default task(s).
  grunt.registerTask('default', ['copy:libs', 'typescript', 'copy:html']);
}; 

We now have 2 subtasks of copy task (libs and html) - this way we can create build steps of same task that can be used for different targets or purposes. By running grunt from command line we should get new structure in dist/www folder and opening dist/www/index.html should write two messages into development console:

Packaging for mobile platform

In this example we will focus on Tizen platform since it has native support for web applications but with few modifications it’s possible to integrate PhoneGap (Apache Cordova) framework as well and target Android, iOS, Windows Phone and other platforms.

Tizen works with HTML5 packaged web app files which is basically zip file with config.xml defining package metadata. We will start by creating this file into platform/tizen:

<?xml version="1.0" encoding="UTF-8"?>
<widget xmlns="http://www.w3.org/ns/widgets" xmlns:tizen="http://tizen.org/ns/widgets" id="http://org.example/helloWorld" version="1.0" viewmodes="fullscreen">
    <icon src="icon.png"/>
    <content src="index.html"/>
 <name>helloWorld</name>
 <tizen:application id="c8ETUJghqu" required_version="1.0"/>
</widget>

Additionally we will need application icon which will be displayed in main menu (platform/tizen/icon.png):

Last bit is zip task for Grunt:

npm install grunt-zip --save-dev 

And we can update Gruntfile.js to pull it together:

module.exports = function(grunt) {
  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    typescript: {
      base: {
        src: ['src/**/*.ts'],
        dest: './dist/www/js',
        options: {
          module: 'amd', //or commonjs
          target: 'es5', //or es3
          base_path: 'src'
        }
      }
    },
    copy: {
      libs: {
        files: [
          { flatten: true, expand: true, src: ['lib/require.js'], dest: 'dist/www/js/'}
        ]
      },
      html: {
        files: [
          { flatten: true, expand: true, src: ['views/index.html'], dest: 'dist/www'}
        ]
      },
      tizen: {
        files: [
          { flatten: true, expand: true, src: ['platform/tizen/**'], dest: 'dist/tizen'},          
          { flatten: true, expand: true, src: ['dist/www/index.html'], dest: 'dist/tizen'},
          { flatten: true, expand: true, src: ['dist/www/js/*'], dest: 'dist/tizen/js'}          
        ]
      }
    },
    zip: {
      tizen: {
        src: 'dist/tizen/*',
        cwd: 'dist/tizen',
        dest: 'dist/helloWorld.wgt'
      }
    }
  });

  grunt.loadNpmTasks('grunt-typescript');
  grunt.loadNpmTasks('grunt-contrib-copy');
  grunt.loadNpmTasks('grunt-zip');

  // Default task(s).
  grunt.registerTask('default', ['copy:libs', 'typescript', 'copy:html']);
  grunt.registerTask('tizen', ['default', 'copy:tizen', 'zip:tizen']);
};

Now we can invoke either grunt command without parameters or, if we wish to build Tizen target, grunt with parameter tizen:

grunt tizen

In next part we will take a look how to tie together additional template engines Jade and Stylus and how to build MVVC application using Knockout.js.

Complete project can be downloaded here without node_modules folder so it is necessary to run following command from project directory to load dependencies:

npm install .

Continue to Part II.