Complete Grunt deployment workflow for a client side app with RequireJS, Jasmine, LESS, and Amazon S3

This blog post will cover a complete Grunt workflow for a client side app with RequireJS. There are no server side components, and the data is retrieved through a web service API. The goal for this post is to cover all the different steps for a full Grunt build script with the goal of a complete release package that can be deployed to any environment - in this case, we are deploying to an Amazon S3 bucket.

For this post, we assume some experience of working with Grunt. We will not cover all the different options in the plugins we use, but there will be links to all the plugins for further documentation and reference.

What will we cover?

• Code check with JsHint.
• Unit test execution with Jasmine.
• Compilation and minification of JavaScript files
• Compilation and minification of LESS files to CSS.
• Asset management to ensure proper cache functionality
• Environment configuration by setting environment specific variables.
• Versioned package ready to be deployed.
• Deployment of our files to an Amazon AWS S3 bucket.

Let's start!

Verification Tasks

As part of the verification tasks we would like to:

• Check our code using JShint.
• Run our tests using Jasmine.

JShint

We are using the grunt jshint plugin which is pretty much standard in most grunt scripts. The configuration is simple and self explanatory:

jshint: {
    options:{
        globals: {
            angular: true
        }
    },
    files: ['app/**/*.js', 'test/unit/*.js', 'test/e2e/*.js']
}

Jasmine

Next up we will setup a grunt task to run our Jasmine unit tests. To allow for browser based testing, we will use phantom JS and the grunt connect plugin alongside the grunt jasmine plugin.

connect: {
    tests: {
        options: {
            port: 9090
        }
    }
},
jasmine: {
    src: [
        'app/controllers/*.js',
        'app/services/*.js',
        'app/app.js'
    ],
    options: {
        host: 'http://localhost:9090/',
        specs: 'test/unit/*Specs.js',
        keepRunner: true,
        template: require('grunt-template-jasmine-requirejs'),
        templateOptions: {
            requireConfigFile: 'app/main.js',
            requireConfig: {
                baseUrl: 'app/',
                paths: {
                    'angular-mocks': '../test/vendor/angular-mocks'
                },
                shim: {
                    'angular-mocks': ['angular']
                }
            }
        }
    }
}
...
grunt.registerTask('unit-tests', ['connect:tests', 'jasmine']);

Notice how we are using the same port for both the connect and jasmine task. Also, we are leveraging the jasmine requirejs template for grunt to generate the proper SpecRunner.html file.

We allow ourselves to keep the _SpecRunner.html file with the option keepRunner: true since it can be very useful for debugging. You could always run the

grunt connect:tests:keepalive

task to generate an ad-hoc webserver where you could load the _SpecRunner.html file and execute the tests in any browser you would like.

Let's add a final grunt task to put it all together:

grunt.registerTask('check', ['jshint', 'unit-tests']);

Build and Compilation

As part of the build and compilation process we would like to:

• Set all configuration variables.
• Compile the JavaScript files.
• Compile the LESS files into CSS.

Environment definition

In order to be able to deploy to different environemts (development, staging, production, etc.), it will be very useful to have different configurations for each environment that can be easily swaped when running our tasks.

In the root of your project, we will define a folder called environments which will contain different modules containing the different environment configurations. For demo purposes, we will create a dev.js module:

module.exports = {}

We will be filling up this object as we go along. We load this environment variable with the following code:

module.exports = function(grunt) {
        var envToUse = grunt.option('env') || 'dev';
        var env = require('./environments/' + envToUse + '.js');
        grunt.log.writeln('Using environment:', envToUse, env);
        ...

We default to dev if no environment is defined. We can set environments by calling grunt as follows:

grunt {task} --env={environment-to-use}

Configuration

For this task, we will leverage the string replace grunt plugin for basic string manipulation using regular expressions.

In our source code, we have a config.js object with some basic settings - for instance, the URL for the API we will access:

define(function(){
 return {
  serviceUrl: '/api'
 };
});

So our goal will be to modify that URL with an environment specific setting. For instance, if we were thinking of deploying to our staging environment, we would like to set this serviceUrl property to the staging service url.

We will first add the serviceUrl to our environment file:

module.exports = {
    apiUrl: https://my.staging.api/
}

We will then copy our source to a temporary folder where we can modify our files. We will use the grunt copy plugin to achieve this goal.

copy: {
    'pre-build': {
        files: [
            { src:['app/**', 'vendor/**'], dest: 'build-tmp/' }
        ]
    }
}

Then putting it all together in our string-replace task:

'string-replace': {
    release: {
        files: {
            'build-tmp/app/config.js': 'app/config.js' 
        },
        options:{
            replacements: [
                { 
                    pattern: /serviceUrl:\s*'\/\w+'/,
                    replacement: 'serviceUrl: "' + env.apiUrl + '"'
                }
            ]
        }
    }
}

JavaScript compilation

We will use the grunt requirejs plugin with very basic settings:

requirejs: {
    compile: {
        options: {
            mainConfigFile: 'build-tmp/app/main.js',
            out: 'build/app/main.js',
            name: 'main'
        }
    }
}

LESS to CSS

For transforming our LESS files, we will use the grunt less plugin. Also, very simple configuration:

less: {
    release: {
        options: {
            paths: ["styles"],
            cleancss: true
        },
        files: {
            "build/styles/styles.css": "styles/styles.less"
        }
    }
}

Notice how we are populating a build folder. This will be our source from were we will create a release folder.

Cleanup

Let's add a task to cleanup our temp folder using the grunt clean plugin. This will be useful to make sure we are starting with a clean slate.

clean: {
    build: ['build-tmp/']
},

Putting it all together

grunt.registerTask('build', ['clean:build', 'copy:pre-build', 'string-replace:release', 'requirejs', 'less:production']);

Asset Versioning

Browser caches are great and we love them because they help speed up the loading time of our pages. However, they can be a problem when deploying new files since we need to be sure that our users are always consuming the most up to date files.

Let's take a look to see how we can accomplish this by revving our files.

Revving

A good explanation and intro to revving and why it's important can be found here. We will use the grunt rev plugin to rev our assets. But first, lets copy over our remaining assets to our build folder. Remember, the JavaScript and CSS have already been copied by the requirejs and less tasks.

copy: {
    'pre-build': {...},
    'pre-rev':{
        files: [
            { src:['Index.html', 'app/views/**'], dest: 'build/' }
        ]
    }
}

For revving, we will follow the hashing strategy were we will append a hash of the content of the file to the file name. Another common strategy is to append a timestamp to each file, but then even those assets that have not changed will be timestamp and it will force users to download a new file unnecessarily. This plugin implements only the hashing strategy, but there are others that can let you use either depending on your preferences. Let's take a look at our task configuration:

rev: {
    files: {
        src: [
            'build/app/main.js',
            'build/styles/styles.css',
            'build/app/views/**/*.{html,htm}'
        ]
    }
},

Here we are revving our only .js file (all dependencies have been compiled into one in the require js task detailed above). We are also revving our only css file that we have also compiled and minimized, as well as our .html templates. The result of this task run will be that all of our assets in the build folder will be revved and ready to be released.

Our next step is to modify the references to all of these assets so that they point to the new filename. To achieve this, we will leverage the grunt usemin plugin.

usemin: {
    html: 'build/index.html',
    html_header: ['build/index.html', 'build/app/views/*.html'],
    js_routing: 'build/app/*.main.js',
    options: {
        assetsDirs: ['build/**', 'build/app/views/**'],
        patterns: {
            js_routing: [
                [/(\/app\/views\/.*?\.(?:html))/gm]
            ],
            html_header: [
                [/(\/app\/views\/.*?\.(?:html))/gm]
            ]
        }
    }
}

We are using the built in matchers for html files, while providing some custom matchers for replacing the references for our html templates in both .html files, and our main.js file. Notice how we match the main.js file with build/app/*.main.js to account for the new filename that includes the hash.

More cleanup

Let's extend our build cleanup task to include our newly populated build folder:

clean: {
  build: ['build-tmp/', 'build/'] 
}

We are now ready to release our code and deploy it to S3.

Release and Deploy

Release Folder

Now that our build folder is complete, we will copy our assets into a versioned release folder. A common practice in Node projects is to include the version of your app in the package.json file; therefore, we will read that version number to brand our release:

var envToUse = grunt.option('env') || 'dev';
    var env = require('./environments/' + envToUse + '.js');
    grunt.log.writeln('Using environment:', envToUse, env);

    grunt.initConfig({
        pkg: grunt.file.readJSON('package.json'),
        releaseDirectory: 'releases/<%= pkg.version %>-' + envToUse,
    ...
});

And then we will proceed to copying the source over to our release folder:

copy: {
    'pre-build': {...},
    'pre-rev':{...},
    release:{
        files: [
            {expand: true, cwd: 'build/', src:['**'], dest: '<%= releaseDirectory %>/'}
        ]
    }
}

The result will be a new folder under releases located in the root of our project with the name {version}-{environment} - for instance, 1.2.3-staging.

Amazon AWS S3 Deployment

For our S3 deployment, we are leveraging the grunt aws-s3 plugin. First, we will add the s3 bucket name to our env:

module.exports = {
    apiUrl: https://my.staging.api/,
    s3Bucket: 'my-s3-bucket
}

aws_s3: {
    dev: {
        options: {
            accessKeyId: 'your-access-key',
            secretAccessKey: 'your-secret',
            bucket: env.s3Bucket,
            httpOptions: {
                proxy: 'set this here if you happen to be behind a proxy'
            },
            sslEnabled: false //this was necessary in my case since the proxy was giving me issues with SSL
        },
        files: [
            { expand: true, dest: '.', cwd: '<%= releaseDirectory %>/', src: ['**'], action: 'upload', differential: true },
            { dest: '/', cwd: '<%= releaseDirectory %>/', action: 'delete', differential: true }</%>
        ]
    }
}

Some explanation is due here. The proxy and sslEnabled settings are not needed in most cases. However, when behind your typical draconian corporate firewall, they can be very useful. In fact, this was the reason for me to choose this plugin as opposed to grunt-s3 plugin. The latter depends on knox which is not compatible with proxies.

With the files settings we are saying that we want to upload only the files that are different in our release folder compared to our s3 bucket. Furthermore, we want to delete all files that are in the s3 bucket, but are not in our release folder.

Putting It All Together

grunt.registerTask('release', ['clean:release', 'copy:pre-rev', 'rev', 'usemin', 'copy:release', 'clean:build']);
grunt.registerTask('default', ['check', 'build', 'release', 'aws_s3']);

With these two tasks, we have completed our deployment process. For extra credit, we can also add some watch tasks using the grunt watch plugin to speed up our development:

less: {
    development: {
        options: {
            paths: ["styles"]
        },
        files: {
            "styles/styles.css": "styles/styles.less"
        }
    },
    production: {...}
},
watch:{
    unit: {
        files: ['app/**/*.js', 'test/unit/*.js'],
        tasks: ['check']
    },
    less: {
        files: ['styles/*.less'],
        tasks: ['less:development']
    }
}

We also included in there a task to automatically compile our css during development to save us the need to manually recompile.

Conclusion

On this blog post I showed a nice way to automate the deployment of your web app using Grunt. The posts first shows how to check your code with JSHint and how to run your tests with Jasmine. It then compiles and mimifies your javascript and css code. And, at the end, it shows how to do revving and deploying to an S3 environment.

If you made it all the way to the end, I surely hope it's been worth it. Please feel free to post any questions or comments and I'll do my best to reply.