Rewriting Mojolicious::Plugin::AssetPack #mojolicious plugin
Sunday, February 21, 2016
This article is about proposed AssetPack changes and why. I would appreciate feedback from existing users, so if you are an existing AssetPack user and think the changes are bad, then please don’t hesitate to contact me.
AssetPack started out as something very simple: Just a single module with some code blocks that would process the assets: The code blocks took a reference to a scalar holding the content of the asset and allowed any modification to that text. After all the registered code blocks were done, AssetPack would write the final asset to the public folder, which Mojolicious could serve.
The code blocks worked for a while, but did not scale very well. The number of pre-processors grew and made the module very clunky to work with.
I then decided to move the callbacks to a new “Preprocessors” module, which made the code a bit easier to read. Unfortunately, this module lived on, even after I decided to split each pre-processor into separate classes. This design decision made each of the pre-processor objects impossible to access from the main AssetPack plugin. This means that you cannot customize a pre-processor after it has been added.
Another thing that makes AssetPack difficult to work with is how it figures out what to name the output asset files: The simple way to describe the logic is that it takes all the content of the input files and calculates an MD5 of the content, and uses that string as part of the name. (Example: “myapp-d3b07384d113edec49eaa6238ad5ff00.css”). If this file exists, it will skip the processing step, and simply use the existing file. For some unexplained reason, AssetPack sometimes calculates the checksum based on the wrong input files. I’ve still not completely understood why this happens, but it happens in some rare cases when you depend on assets automatically downloaded from the web.
Why the rewrite? I wanted to make AssetPack simple and understandable again. The module has grown into something that is very difficult to maintain and extend. The new design enables most of the core functionality to be overridden in a very simple way. This makes AssetPack more flexible for end users.
The plugin code is very thin in the new version: Most of the code that used to be core is now moved to the “store”. The “store” is an object that can persist assets and serve them to the browser.
Preprocessor:: classes have been converted to
Pipe:: classes with
a new and more simple API: There’s just one method (
process()) that receives a
list of input assets for a
This means that a “pipe” can change the list in any way it likes: It can
collapse it into one element, add more elements or change just parts of the
assets. One special example of this is the
pipe: This pipe merges all of the input sources into one asset.
Asset base URL
base_url attribute in the existing version has been replaced with a
attribute. This route defaults to dispatching to a callback which will render
the requested assets. This introduces much more flexibility: You can change
the path part completely and/or the
A custom route also makes it possible to protect your assets with
authentication if you like.
You can still customize response headers: headers is moved to the store as “default_headers”, and it has smart defaults: “Cache-Control” is set to “max-age=31536000”. This is in addition to the standard “ETag” and “Last-Modified”, which is already set by Mojolicious::Static.
Asset output directory
Figuring out where to store assets was quite painful in the previous version,
and I still don’t think it’s done right. That’s why there’s no
the new version. Instead there’s automatic decision making to save assets to
TMP_DIR, or as an actual file while developing. The output files are
not stored in the “public” directory though, so they can’t be accessed from
the static renderer directly.
Purging processed assets
purge() was added to clean up unused processed files. This was useful since
a development process could create hundreds of files in the “public/packed”
directory. In the new version, there’s simply no need for
purge() since the
checksum in the generated filenames does not change as much as it used to: The
checksum is calculated from the input filenames/locations, instead of the
content of the input assets.
(See Guessing output filenames if you worry about
Guessing output filenames
This is the main reason for the rewrite: The current version is not very good at naming the output files on disk. The new version on the other hand calculates the filename based on the locations of the input files, and not the content of those files. This means that a new filename will only be generated when the list of input files change. This makes finding processed assets a lot more robust. But what about browser caching? That is still accounted for, since the path generated by the route contains the checksum of the content of the input files, so it changes each time the input files are changed.
Where to find source assets
source_paths has been replaced with two more powerful attributes in the
The “store” is an object that inherits from
paths attributes allowing you
to also declare your assets in the DATA section of your classes.
Major changes for existing users
There is some existing functionality that will (probably) be dropped.
Assets cannot be defined with the default helper
asset() anymore. You need to call
There’s an option today to rebuild the assets on each request, instead of just
on startup. I’m not quite sure if I want to keep this or not. What I might
want to do instead is to add support for
MOJO_ASSETPACK_WATCH which will
rebuild the changed assets and
reload the page
in the browser.
Fetching online assets
fetch() method will be removed. I would much rather want to improve the
process step to do all the fetching.
Will die instead of showing an error asset in the browser
The current version creates an error asset, which gives visual feedback in the browser. This might be re-implemented, but right now I consider the complexity to simply not be worth it. This behavior is especially a bad idea in production, where you probably want a hot deployment of hypnotoad to fail instead of showing an ugly page.
Wildcards in source path/filenames
A source path cannot have “*” in it. The star used to be a way of including
all files from a given directory, matching an expression. This logic simply
does not work if you use the standard CPAN toolchain and remove files in the
distribution: Tools such as
cpan won’t delete files on install,
which again results in the AssetPack finding files that should not be there.
So instead there’s now support for creating a very simple
which should be easy to generate from the command line.
“jsx” support is incredibly difficult to get right, and the current version is, at best, buggy. I will not continue to support this.
Global variables, such as SASS_PATH
This just won’t happen. Unless I mess up. Variables such as
used internally, but it can’t be set from the command line. Environment and global
variables are just not predictable enough to be trusted in this setting.
The next version has some new cool features:
AssetPack can automatically install third party node and ruby tools for processing assets, such as CoffeeScript. “node” and “ruby” are still required to be pre-installed though. I’m also looking into support for automatically installing Perl modules, but, ironically, I find that a bit harder to do right.
Much better at downloading online assets
Access to the “pipe” objects.
The pipe objects can be accessed from the main plugin. This makes it much easier to control the settings of a pipe.
All dependencies are optional. This means that you need to install more modules manually while developing, but running an application in production does not require any. The idea is that all of the “pipes” will skip processing the asset when they are already available.
There is (at least) one missing feature that needs to be implemented - at least before I completely deprecate the “pre-processors”:
The idea here is to make a subclass of Asset which can hold all of the images. This object will again be able to create CSS from the images, just like today.
I hope people will try out the new version in the wild before the next release happens. The merge will happen at some point, but that point could be delayed if you report back breaking changes. The next version should also be fully backwards compatible (the test suite says so), but feedback on both new and old functionality is greatly appreciated.
How to get the next version:
$ cpanm https://github.com/jhthorsen/mojolicious-plugin-assetpack/archive/v2.tar.gz