Incremental SASS Builds with Gulp

I recently picked up a project I had worked on in 2017. It had a simple setup of HTML, a local server, SASS compiled to CSS using gulp, along with super-slow build times.

If I changed even a single line of CSS in that project, my gulp setup would take 8-10 seconds and compile all the SASS files again.

The root cause: this bit of code I had copied from the gulp-sass README page.

gulp.task('sass', function () {
  return gulp.src('./sass/**/*.scss')
    .pipe(sass().on('error', sass.logError))
    .pipe(gulp.dest('./css'));
});
 
gulp.task('sass:watch', function () {
  gulp.watch('./sass/**/*.scss', ['sass']);
});

Back then, I used to be in a constant hurry as I juggled between 5 projects. I wasn’t able to find a way to fix the SASS compilation issue after 15 minutes of Google search so I lived with the pain.

This time, with two years of professional experience - which brings a calmer approach to debugging, and a lighter schedule, I decided to fix the build time issue once and for all.

tldr; version

Here’s the final solution, in case you’re in a hurry like the 2017 version of me:

'use strict'

const gulp = require('gulp')
const sass = require('gulp-sass')
const dependents = require('gulp-dependents')

const cssRoot = 'css'

const glob = {
    sass: [`${cssRoot}/*.scss`, `${cssRoot}/**/*.scss`]
}

function compileCSS (cb) {
    gulp.src(glob.sass, { since: gulp.lastRun(compileCSS) }) // filter only changed files
        .pipe(dependents()) // find sass files to re-compile
        .pipe(sass().on('error', sass.logError))
        // other plugins like autoprefixer and clean-css
        .pipe(gulp.dest(`${cssRoot}`))
        .on('end', cb)
}

function watchCSS () {
    gulp.watch(glob.sass, compileCSS)
}
exports.css = compileCSS
exports.watch = watchCSS

P.S. You’ll notice some syntax changes in the snippet above compared to the gulp-sass version. That’s because I migrated from gulp v3 syntax to gulp v4. You can read more about the migration here

In case you want to deep-dive into the problem and its solution, read on below 👇

The Problem

Every time I updated a line of CSS, my gulp setup would try to compile all the SASS files again (not just the ones that were changed). The local server would detect a file change and refresh the HTML.

Since I had 125 SASS files in my project, the local server detected 125 file changes and would refresh constantly. It appeared that my browser was having a 10 second seizure.

Needless to say, it was a pretty annoying setup to work it.

Let’s figure out what the issue was by looking the 2017 version of gulpfile (converted to gulp v4 snytax for easy comparison):

const cssRoot = 'css'

const glob = {
    sass: [`${cssRoot}/*.scss`, `${cssRoot}/**/*.scss`]
}
/*
 * This bit works as expected. Whenever it detects a file change,
 * it calls the compileCSS function
 */
function watchCSS () {
    gulp.watch(glob.sass, compileCSS)
}

function compileCSS (cb) {
   /*
    * This issue here is that whenever compileCSS() runs, 
    * it compiles all the SASS files. Now only if there was a way to fix this
    */
    gulp.src(glob.sass)
        .pipe(sass().on('error', sass.logError))
        // other plugins like autoprefixer and clean-css
        .pipe(gulp.dest(`${cssRoot}`))
        .on('end', cb)
}

Solution 1: gulp-cached and gulp-dependents

After 4 hours of searching the interwebz, and looking for the right keywords to describe my problem, I found this stackoverflow question which saved my sanity.

The way to fix the issue was just adding two lines of code in the compileCSS function!

const cached = require('gulp-cached');
const dependents = require('gulp-dependents');

function compileCSS (cb) {
    gulp.src(glob.sass)
        .pipe(cached('sasscache')) // 1
        .pipe(dependents())// 2
        .pipe(sass().on('error', sass.logError))
        // other plugins like autoprefixer and clean-css
        .pipe(gulp.dest(`${cssRoot}`))
        .on('end', cb)
}

How the solution works

The first line .pipe(cached('sasscache')) uses the gulp-cached plugin which creates an in-memory cache of the files that have passed through it.

Initially, when there is no cache, it allows all files to pass to the next step and makes a note of their contents - aka building the cache.

The next time a file tries to pass through it, it acts like a gatekeeper. It checks if the file contents have changed from the version that is saved in the cache. If they haven’t, it filters out that file and passes only the changed ones to the next step.

The second line .pipe(dependents()) uses the gulp-dependents plugin which maintains a dependency graph of your files.

Basically, this is useful when you change the CSS in a SASS partial file. The plugin builds and maintains a dependency graph of your files to figure out the list of files that use that particular partial - which it then passes to the next step.

Why do we need gulp-dependents?

To answer that question, lets use the gulp-debug plugin to see the files as they are passed down. Modify your code like this:

const gulp = require('gulp')
const sass = require('gulp-sass')
const cached = require('gulp-cached');
const dependents = require('gulp-dependents');
const debug = require('gulp-debug'); // Add this

const cssRoot = 'css'

const glob = {
    sass: [`${cssRoot}/*.scss`, `${cssRoot}/**/*.scss`]
}

function watchCSS () {
    gulp.watch(glob.sass, compileCSS)
}

function compileCSS (cb) {
    gulp.src(glob.sass)
        .pipe(cached('sasscache'))
        .pipe(debug({title: 'cache pass:'})) // Add this
        .pipe(dependents())
        .pipe(debug({title: 'dependents:'})) // Add this
        .pipe(sass().on('error', sass.logError))
        // other plugins like autoprefixer and clean-css
        .pipe(gulp.dest(`${cssRoot}`))
        .on('end', cb)
}

exports.css = compileCSS
exports.watch = watchCSS

Now run gulp watch and change any SASS file. This will cause the files to pass through gulp-cache plugin once and it will build a cache for us.

Next, change a partial used by a lot of files like header.scss. This time around, you’ll see only one file being passed down from the cache as shown in the screenshot below:

You can also see why we need the gulp-dependents plugin. Even though just the header is modified, we actually need to re-compile all the files using the header, which can be a lot in any project.

In this example, there are 46 files which are a dependent of the header partial!

If we had skipped using this plugin, our build pipeline wouldn’t work at all and you’ll be stuck scratching your head about why your changes aren’t reflected in the browser.

Note: Modified a heavily dependent file would still take a long time to build. For example, in this case, changing the header can take up to 5 seconds in my project. That’s still half compared to the original build times of 8-10 seconds.

Overall, this solution works well, bringing down the average build time from 8 seconds to ~1 second for 125 files in my folder. But we can still do better.

Solution 2: Use the since option in gulp.src()

Having a cache is great but all 125 files are still being checked against the cache. You can verify this by adding a gulp-debug step before the cache step.

What if we could save these comparsions by somehow knowing the exact files that changed? What if we could throw away the cache altogether? There is a way. ✌🏽

Starting with gulp v4, you can pass an option since to gulp.src() which takes a timestamp (read docs). When this option is passed, gulp only operates on files modified after the timestamp that you have specified.

For our use case, we want the timestamp to be dynamic - the last time the compileCSS() function was called.

To achieve that, there’s another gulp method called lastRun(). When passed a task function, it retrieves the last time that task was successfully completed (read docs).

With both of these in our set of tools, we can drop the gulp-cached plugin and use gulp v4 native functionality

// const cached = require('gulp-cached');
const dependents = require('gulp-dependents');

function compileCSS (cb) {
    gulp.src(glob.sass, { since: gulp.lastRun(compileCSS) })
        // .pipe(cached('sasscache'))
        .pipe(dependents())
        .pipe(sass().on('error', sass.logError))
        // other plugins like autoprefixer and clean-css
        .pipe(gulp.dest(`${cssRoot}`))
        .on('end', cb)
}

Since gulp is internally taking care of figuring out which files have changed now, our build times have reduced to ~200-300ms.

And that’s going to be enough for now.

Here’s the complete solution once again

'use strict'

const gulp = require('gulp')
const sass = require('gulp-sass')
const dependents = require('gulp-dependents')

const cssRoot = 'css'

const glob = {
    sass: [`${cssRoot}/*.scss`, `${cssRoot}/**/*.scss`]
}

function compileCSS (cb) {
    gulp.src(glob.sass, { since: gulp.lastRun(compileCSS) }) // filter only changed files
        .pipe(dependents()) // find sass files to re-compile
        .pipe(sass().on('error', sass.logError))
        // other plugins like autoprefixer and clean-css
        .pipe(gulp.dest(`${cssRoot}`))
        .on('end', cb)
}

function watchCSS () {
    gulp.watch(glob.sass, compileCSS)
}
exports.css = compileCSS
exports.watch = watchCSS

If you found this post useful, please let me know on twitter!

thoughts about frontend dev, digital experiences and education
Follow me on Twitter