Aug 13 2007
Introducing the YUI Compressor
Is there a need for a new JavaScript minifier?
According to Yahoo!’s Exceptional Performance Team, 40% to 60% of Yahoo!’s users have an empty cache experience and about 20% of all page views are done with an empty cache (see this article for more information on browser cache usage) This fact outlines the importance of keeping web pages as lightweight as possible. Improving the engineering design of a page or a web application usually yields the biggest savings. After that come many different techniques such as minification of the code, HTTP compression, etc. In terms of code minification, the most widely used tools to minify JavaScript code are Douglas Crockford’s JSMin and the Dojo compressor. Both tools however have pitfalls. JSMin, for example, does not yield optimal savings (due to its simple algorithm, it must leave many line feed characters in the code in order not to introduce any new bugs) The Dojo compressor, on the other hand, does a better job at compacting code (it obfuscates local variables) but is unsafe and potentially introduces bugs if you are unaware of its limitations.
What is the YUI Compressor?
The YUI Compressor is a new JavaScript minifier. Its level of compaction is higher than the Dojo compressor, and it is as safe as JSMin. Tests on the YUI library have shown savings of about 18% compared to JSMin and 10% compared to the Dojo compressor (these respectively become 10% and 5% after HTTP compression)
How does it work?
The YUI Compressor is written in Java (requires Java >= 1.4) and relies on Rhino to tokenize the source JavaScript file. It starts by analyzing the source JavaScript file to understand how it is structured. It then prints out the token stream, replacing all local symbols by a 1 (or 2, or 3) letter symbol wherever such a substitution is appropriate (in the face of evil features such as eval or with, the YUI Compressor takes a defensive approach by not obfuscating any of the scopes containing the evil statement). The YUI Compressor is open-source, so don’t hesitate to look at the code to understand exactly how it works.
Where can I get it?
An archive containing both source and binary is available for download on this site.
How do I run it?
java -jar yuicompressor-1.0.jar
[-h, --help] [--warn] [--nomunge]
[--charset character-set] [-o outfile] infile
The following command line for example:
java -jar yuicompressor-1.0.jar myfile.js
will minify the file myfile.js and output the file myfile-min.js. For more information on how to use the YUI Compressor, please refer to the documentation included in the archive.
Limitations
Unlike JSMin, the YUI Compressor is slow and cannot be used for on-the-fly code minification (see minify for a PHP implementation of JSMin by Ryan Grove, another Yahoo! engineer, that does on-the-fly JavaScript minification among other things)
Has Yahoo!’s official position on obfuscation changed?
Douglas Crockford wrote an interesting article last year on minification vs. obfuscation. While that article still holds true, the YUI Compressor actually represents a 3rd way, in which code compaction does not carry the risk of introducing new bugs. The YUI Compressor is currently considered as a replacement for JSMin to compact the entire YUI library.
Feedback appreciated
The YUI Compressor is still a fairly new tool, so your feedback is well appreciated. Don’t hesitate to drop me a line if you find a bug, or think an important feature is missing. I will make updates available on this site, so watch for posts related to the YUI Compressor on this blog. This software belongs to the community, so it’s up to the community to make it better!
Additional notes
- Do not hesitate to use the
--warnoption. The YUI Compressor will warn you if it finds anything that seems abnormal with your code, or that might reduce the level of compression. - Keep as little of your code as possible in the global scope. This has 2 benefits:
- You’ll get a better level of compression since the YUI Compressor does not obfuscate global symbols (which would make it unsafe)
- You won’t pollute the global object (see this article)
69 Responses to “Introducing the YUI Compressor”
Thanks for making this available, it seems to work great.
Out of curiousity, I tested it on Prototype (http://www.prototypejs.org/assets/2007/6/20/prototype.js), and I got the following exception:
Exception in thread “main” java.lang.NullPointerException at YUICompressor.getHighestFnScope(YUICompressor.java:470)
at UICompressor.parseScope(YUICompressor.java:704)
at YUICompressor.buildSymbolTree(YUICompressor.java:498)
at YUICompressor.main(YUICompressor.java:211)
May be it can help debug the compressor.
Very nice! I have seen around a 40% reduction in the size of my codebase. I’ve posted a link to your article on my own site, and in the jQuery mailing list.
@Martin
I just posted an update that fixes the specific problem you mentioned (thanks for reporting it!) The use of ‘with’ in the global scope in prototype.js reduces the level of compression for that library (both ‘with’ and ‘eval’ are not recommended)
[...] YUI Compressor is a new JavaScript minifier. Its level of compaction is higher than the Dojo compressor, and it is [...]
In my experience, Packer created by Dean Edwards has been the most efficient JavaScript compressor. I compared YUI Compressor and Packer and it looks like the results are very much alike. Sometimes YUI Compressor is better and sometimes Packer. If you want to get every extra byte out of your script, I recommend testing Packer, too.
BTW, why is this application called YUI Compressor? As far as I can see, it has no relation to the Yahoo! script library with the same name.
@Visa
Unfortunately, packer is not a safe compressor. Try the following with packer:
function foo(aaa) {
var yop = aaa;
eval(”yop”);
}
You’ll get (if you check the “shrink variables” option):
function foo(a){var b=a;eval(”yop”)}
which is clearly incorrect…
Also, the YUI Compressor will be shipped together with the YUI library in a not-to-distant future.
@Visa
I’ve had major problems with packer causing errors in my code, *so far* I haven’t had a problem with this packer yet on the same code.
@Julien
You say you have updated the release - is there any chance you have this up on a publicly available SVN/CVS? I’d like to be able to keep up to date with the latest code
@Tane
There is no plan to setup a publicly available SVN/CVS repository for the YUI Compressor at this time.
Thanks for this new tool
I’m trying to compress mootools (http://mootools.net) v1.11 full, but I get this error :
16:28:08:bpopoff@grosminet:/var/www/test_boris/mootools$ java -jar yuicompressor-1.0.jar mootools.v1.11.uncompressed.js
[INFO] Using charset Default
java.lang.AssertionError
at YUICompressor.parseCatch (YUICompressor.java:546)
at YUICompressor.parseScope (YUICompressor.java:722)
at YUICompressor.buildSymbolTree (YUICompressor.java:501)
at YUICompressor.main (YUICompressor.java:211)
at java.lang.reflect.Method.invoke0 (Method.java)
at java.lang.reflect.Method.invoke (Method.java:255)
at kaffe.jar.ExecJarName.main (ExecJarName.java:67)
at kaffe.jar.ExecJar.main (ExecJar.java:75)
Is it a bug, or perhaps I missed something ? It works whith some other scripts though. I can send you the file if you want.
Martin Sévigny, re: your Prototype error. I had trouble with one line in prototype when I was using JSMin. I haven’t started using YUI yet so I don’t know if it is the same problem. Here’s the problem:
JSMin doesn’t like this line (hope it comes out in the post!). It says there is an unterminated quote.
attr: /\[((?:[\w-]*:)?[\w-]+)\s*(?:([!^$*~|]?=)\s*((['"])([^\]]*?)\4|([^'"][^\]]*?)))?\]/
Since in a regular expression ['"] is the same thing as ["'], just change the first ['"] to ["'] and JSMin is fooled into thinking there is a quote.
Impressive: I was setting up a “minification” step in the build process of Subtext (an OS .NET blogging engine), but now I think I’ll move from JSMin to this new YUI compressor.
[...] YUI Compressor is a new JavaScript minifier. Its level of compaction is higher than the Dojo compressor, and it is [...]
@boris
I am not able to reproduce your error on the same file. Which platform are you on? Which version of Java are you using? It seems like it’s not able pick up the platform’s default charset. Could you try it on another computer maybe? You can also try to specify the charset explicitly using the –charset option. For example:
java -jar yuicompressor-1.0.jar –charset US-ASCII myfile.js
Well, on another one I had
[INFO] Using charset UTF-8
Exception in thread “main” java.lang.AssertionError
at YUICompressor.parseCatch(YUICompressor.java:546)
at YUICompressor.parseScope(YUICompressor.java:722)
at YUICompressor.buildSymbolTree(YUICompressor.java:501)
at YUICompressor.main(YUICompressor.java:211)
But finally, it worked on another one :
[INFO] Using charset ANSI_X3.4-1968
Quite strange …
Here are the results for mootoools :
Uncompressed : 180K
JSmin : 73K
YUICompressor : 65K
Packer : 43K
So for Mootools packer seems to be the best, but as you said, packer is not safe for all scripts.
[...] Ajaxian I learned the Yahoo has released a new javascript compressor that reduces file sizes 18%. Compression of web site code is something that too many web site [...]
For on the fly compression with JSP apps, I would recommend pack:tag (http://packtag.sf.net/)
It currently uses JSMin, but since it only compresses once and caches that result, I wonder if YUI Compressor could be used instead.
[...] YUI Compressor. Se trata de una aplicación capaz de superar los niveles de compresión actuales. Pruebas sobre la biblioteca de Yahoo! (YUI) han demostrado que puede superarlos hasta entre un 10 y un 18%.. [...]
@boris
The difference between Packer and the YUI Compressor most likely comes from the presence of either ‘eval’ or ‘with’ statements somewhere in the code. In that case, it is impossible to know for sure (without human intervention) whether the obfuscation has introduced a bug that was not there in the original code. The YUI Compressor will always adopt a defensive approach to that regard.
Great work and great utility! Based on testing with our own libraries, YUI Compressor was able to beat about Dojo ShrinkSafe by an average of 3% across all of our JS files. 3% may not be a double-digit number, but when you’re serving up thousands upon thousands of requests for those files per day, it really adds up.
However, that said, I’m curious to know why you think ShrinkSafe is unsafe. Since it also uses Rhino to tokenize the JS, shouldn’t it be just as safe as YUI Compressor? The only thing I can think of that wouldn’t be safe is the correction of variables when there are eval’s or with’s in the scope. Since “good” JS programmers don’t use those though, is there anything else about YUI Compressor that is more safe than ShrinkSafe?
Also, speaking with regards to YUI Compressor not changing variables when there’s eval’s/with’s in the scope, couldn’t one make the argument that it’s unsafe overall then because of the possibility of global objects being referenced in eval’s? For example, if someone is actually using eval’s in their code, it’s possibly, if not likely, that they are also using global variables and referencing them within the eval’s. At that point, YUI Compressor would break their code. All I’m saying is that it’s almost impossible for anyone really to build a truly safe JS compressor that changes variable names.
Finally, one more thing. I notice that YUI Compressor shares one of the same pitfalls with ShrinkSafe in that it doesn’t work with IE’s conditional comments. See http://dean.edwards.name/weblog/2006/06/again/ for an example; all the code within /* */ in those examples would be wiped clean with either compressor. ShrinkSafe hadn’t / hasn’t acknowledged this, and I had to learn about it the hard way.
Anyway, once again, fantastic work!
@Andrew
Thanks for the kind words. You are perfectly correct with regard to ‘eval’ and ‘with’ (and I invite you to take a look at the code) Also, the YUI Compressor seems to do a better job (than Packer and ShrinkSafe) at parsing complex variable declarations. For example:
var foo = function () {…}, bar;
// do something with foo and bar…
Packer will not obfuscate bar while the YUI Compressor will.
Finally, note that symbols in the global scope are not obfuscated. Otherwise, it would create bugs for sure when importing several JavaScript files in the same page. So the way the YUI Compressor deals with ‘eval’ and ‘with’ IS 100% safe.
[...] You guys can go and have a look at this blog: Julien Lecomte’s Blog [...]
great work,
but the following gives me an error!!
-
['foo', 'bar'].forEach(function(item, index) {
alert (index+’: ‘+item);
});
-
First, great tool!
Secondly, I’m getting error’s when there’s a try/catch block.
The error:
Exception in thread “main” java.lang.AssertionError
at YUICompressor.parseCatch(YUICompressor.java:546)
at YUICompressor.parseScope(YUICompressor.java:722)
at YUICompressor.buildSymbolTree(YUICompressor.java:501)
at YUICompressor.main(YUICompressor.java:211)
Sample code which triggers error:
var do_it;
try {
if (window.external.IsSearchProviderInstalled(’http://www.stamboomgids.nl’) > 0) {
do_it=1;
}
} catch(err) {
do_it=2;
}
@Julien
I’ve found a bug.
Take the module pattern as described here:
http://yuiblog.com/blog/2007/06/12/module-pattern/
An eval(…) statement in a public method causes all private variables to remain unobfuscated by the compressor.
A simplified example:
—-
var Foo = (function () {
var privateVar1 = 1;
var privateVar2 = 2;
return {
publicFunc1 : function() {},
publicFunc2 : function(param) {
eval(param);
}
};
})();
—-
Comment out the eval(…) statement, and the private members will be obfuscated by the compressor.
However, the eval(…) statement can’t possibly change the scope of privateVar1 etc., so the interpretation by Rhino/YUI Compressor seems to be wrong in this case.
@bob
Indeed, there was a bad assertion in my code. I uploaded a new version (same file, but make sure you empty your browser cache before downloading it) that fixes this.
@Nikola
This is not a bug, this is what makes the YUI Compressor 100% safe! Even if the specific case you are exposing seems trivial, it is not possible to reliably obfuscates what is inside an ‘eval’ statement! Therefore, it is not safe to obfuscate anything that is in a scope higher than the scope that contains the ‘eval’ statement. The solution: don’t use ‘eval’ (and don’t use ‘with’ either) For more info on why you shouldn’t use these, please refer to:
http://www.jslint.com/lint.html
@gintro
I don’t get any error with the snippet you mention.
@Julien
Wouldn’t it be more adequate to introduce a directive that tells YUI Compressor that it shouldn’t draw the above noted conclusion? Example related to above use case which assumes that the parameter passed to eval() is well-known:
—-
publicFunc2 : function(param) {
/* [yuic:ignore] */
eval(param);
/* [/yuic:ignore] */
}
—-
As of right now, YUI compressor will leave the whole module unobfuscated, including the public method parameter symbols (param). Even with all the evils of eval() I don’t think this behavior is desirable.
In case you don’t see my comment on Ajaxian…
Did you modify Rhino or Jargs at all, or does this work just with the addition of the five java classes that you wrote? Is there any reason why the classes are packaged in the root package? Why is not packaged in com.yahoo…? Do you have any API for calling it from other Java classes. I would like to integrate this into Resource Accelerate. I really like the fact that it doesn’t appear to have any Rhino mods, like shrinksafe did.
@kris
- Indeed, there is no modification to both Rhino and Jargs (which makes updating them a trivial task)
- There is no reason why the classes are packaged in the root package. I’m just too lazy to write com.yahoo…;-) Seriously, I don’t know of any Java code publicly distributed by Yahoo, so I am not aware of any naming convention. However, this might change in a future version.
- If you want to integrate it in your own tool, just look at the code and start hacking it
[...] What: The YUI Compressor is a new JavaScript minifier. Its level of compaction is higher than the Dojo compressor, and it is [...]
[...] Julien Lecomte’s Blog » Introducing the YUI Compressor (tags: javascript yui compression compressor tool opensource performance yahoo optimization) [...]
[...] Another good new for the web developers is the YUI Compressor, a javascript minifier [...]
[...] Introducing the YUI Compressor The YUI Compressor is a new JavaScript minifier. Its level of compaction is higher than the Dojo compressor, and it is as safe as JSMin. (tags: Obfuscate_Minimize) [...]
[...] Julien Lecomte writes Is there a need for a new JavaScript minifier? YUI Compressor is said to be better than the DoJo copmpressor. [...]
Packer might not be safe for all javascript, it sure does minify the hardest! :)… Plus, JQuery gives it as an official package which will undoubtly be tested… No?
My results:
JQuery uncompressed: 61,4Kb
JQuery Compressed with Packer: 21,0Kb
JQuery Compressed with YUI Compressor: 31,4Kb
Cheers
[...] Julien Lecomte’s Blog » Introducing the YUI Compressor [...]
@Mike
Indeed, Packer yields higher savings on jQuery than the YUI Compressor. However, the generated code is very different from the original code (it is reconfigured for optimal compression) This may be fine for a library, but for your own code, I could not recommend it as it makes debugging a lot harder.
Again, the goal of the YUI Compressor was to be able to handle ANY JavaScript file without adding any cost (no risk of introducing any new bug, does not make debugging that much harder, etc.) In my own experience, working for both small companies and large multinationals, the approach adopted by the YUI Compressor is the most cost effective.
Apparently, YUI Compressor breaks Prototype’s new Inheritance Model (version 1.6 RC). Superclass methods can be called using “$super” as the first method parameter:
var Cat = Class.create(Animal, {
eat: function($super, food) {
if (food instanceof Mouse) return $super();
else return this.say(”Yuk! I only eat mice.”);
}
});
http://www.prototypejs.org/2007/8/15/prototype-1-6-0-release-candidate
“$super” gets renamed to “C” or whatever. Would it be possible to leave this keyword untouched without using –nomunged?
Cheers
@Severin
First of all, I hope you’re using the latest version of the compressor (at the time of this post, it’s 2.1.2)
Second, the answer is no. Renaming $super (which is a variable local to your function) should not have any impact at all. If you don’t get the result you were expecting when executing your code, it’s probably because you have a bug in your code, or in the Prototype library itself.
Regards
Thanks for your quick answer.
I updated to the latest version of the compressor, but the problem persists as expected.
Actually, it isn’t neither a bug in my code nor in Prototype’s. Prototype’s new inheritance model checks whether the first argument of the method (that overwrites the parent class method of the same name) is named “$super”; and wraps the new method if the keyword is found. If $super is renamed to something else, it gets treated like a normal argument.
Quote (Prototype Blog): “If you’re overriding a method from a parent class, you can now access the superclass method by naming the overriding function’s first argument $super. It works just like Function#wrap (in fact, it uses Function#wrap internally).”
Regards
Got it. Will introduce a new feature to the compressor to work around this in the next version. In the meantime, specify the –nomunge option.
[...] Ya hemos hablado por aquà de cómo comprimir ficheros javascript. Pero aún no está todo dicho. Hoy y aquÃ, YUI Compressor. [...]
Great tool, I need to look at it more fully as I’ve only really started thinking about compression this week. The last js file I compressed by hand so I thought I’d try this too and found it increased my file size!
The only caveat I found is that the tool converts:
if(t==0)c[i]=r;
into:
if(t==0){c[i]=r}
Okay so this only adds an extra character for any one-statement ‘if’s and this only meant my file ran through YUI added 3 additional but unnecessary bytes over my 769 byte file (the original file started off at over 2000 bytes).
One thing I’m currently looking into as I’m thinking of writing my own JS compressor is optimising it for gzip. What worked for me was figuring out what letters were used the most in the keywords used and then renaming the most common variable to the most common letter used. As compression tends to compress more commonly used items more this method, although painful to do by hand, saved me another 9 bytes once compressed. In order of use my variables became e, t, i, n, o.
[...] Julien Lecomte’s YUI Compressor [...]
[...] JavaScript and CSS are concatenated into one file each and compressed with YUI Compressor [...]
YUICompressor quotes every string in double quotes, but sometimes it’s cheaper to output a single one - ‘”‘ 3 characters - instead of “”" four characters. There are more cases like this
@Cezary
I just fixed this in the latest version (2.2.1) of the compressor now available for download.
Regards
I tested version (2.2.1) and found a bug
Input:
var str = ‘a = “50″‘ + ‘b’;
Output:
var str=’a = “50″b”
Will this run with gij instead of java?
@Alex
Good question. I guess it should, although I haven’t tried it. Please let us know!
I have tried it by accident. I didn’t realize I didn’t have sun’s jre on an Ubuntu box. So it’s been running (without a hitch) with gij 4.1.2 from Ubuntu Feisty.
I haven’t tried compiling it, though. I was just using the compiled jars. The other classes work fine too.
BTW: Is the version # needed in the jar names?
Ciao!
Yes, the version number is required in the jar name.
BUG:
The following CSS:
background-image: url(’grad_white.png’);
_background-image: none; /* ie sucks */
Is incorrectly turned into:
background-image:url(’grad_#fff.png’);_background-image:none;
Ciao!
BTW: I assume that this means that the CSS compression isn’t using the same technique as the JavaScript compression: building a parse tree and then unparsing it?
How hard would it be to do that? The CSS standard seems pretty straight forward as far as parsing.
Ciao!
The main problem is the use of CSS hacks, which are (unfortunately) widely used (and abused) These hacks rely on parsing errors, so building a CSS parser that can parse hacks correctly is no easy task. The CSS compressor uses a set of regular expressions. By the way, unlike the JavaScript compressor, the CSS compressor is NOT 100% safe (although it tries to be as safe as possible and should work in most cases)
Input:
.a {border: 1px solid #cccccc;}
Output:
.a{border:1px solid#ccc;}
I’m not quite sure what css standard says, but at least IE6 doesn’t seem to understand this compressed version where there is no space between solid and #ccc.
Could you post information on licensing? Do you have a publicly readable Subversion repository? Want one?
@Kris
The YUI Compressor, like the rest of the YUI library, is released under the BSD license (see README file at the very bottom) There is no public svn repository at this time, and I don’t wish to have one. Ultimately, the YUI Compressor will be shipped as part of the YUI library (along with other tools) and will follow the same development guidelines.
Regards
Sorry for the silly question… but
Can i run your YUI compressor in Windows?
@Sumeet
Sure. You can run the YUI Compressor on all the platforms that Java supports, and that includes Windows.
Regards
One idea about CSS compression. Currently (2.2.4) YUI Compressor packs
.a {
width: 10px;
height: 20px;
}
to
.a{width:10px;height:20px;}
I think that the last semicolon ‘;’ can be also quite easily removed, so the output should be
.a{width:10px;height:20px}
Maybe it requires only replacing all ‘;}’ sequences with ‘}’ at the end of the process. Or could it cause errors in some situation(s)?
@Aki
Unfortunately, removing the last semi-colon will break some older browsers (safari)
Regards
Thanks a lot for fixing the Prototype $super “bug”. Removing the –nomunge option made it possible to squeeze another 35 KB out of my code.
Keep the good work up!
Regards
BUG:
Version 2.2.4
pre {white-space: pre;}
becomes:
pre {#fff-space: pre;}
Ciao!
My mistake, this was fixed in 2.2.4…sorry.
Hello,
Can you distribute the Ant buildfile for those of use who want to experiment with modifications ourselves?
thanks
[...] YUI Compressor is a tool that will strip Javascript and CSS to a bare minimum. Comments are removed, variable names are shortened and excess whitespace is stripped. This is a great method to speed up your site. [...]
Hello,
Is it possible to pass several file names as arguments of YUI compressor? It would also be great to pass a wildcard expression to compress all JS/CSS files.