Introducing the YUI Compressor

Is there a need for a new JavaScript minifier?

Yahoo's YUI Compressor

According to Yahoo!’s Exceptional Performance Team, 40% to 60% of Yahoo!’s users have an empty cache experience and about 20% of all page views are done with an empty cache (see this article for more information on browser cache usage) This fact outlines the importance of keeping web pages as lightweight as possible. Improving the engineering design of a page or a web application usually yields the biggest savings. After that come many different techniques such as minification of the code, HTTP compression, etc. In terms of code minification, the most widely used tools to minify JavaScript code are Douglas Crockford’s JSMin and the Dojo compressor. Both tools however have pitfalls. JSMin, for example, does not yield optimal savings (due to its simple algorithm, it must leave many line feed characters in the code in order not to introduce any new bugs) The Dojo compressor, on the other hand, does a better job at compacting code (it obfuscates local variables) but is unsafe and potentially introduces bugs if you are unaware of its limitations.


What is the YUI Compressor?

The YUI Compressor is a new JavaScript minifier. Its level of compaction is higher than the Dojo compressor, and it is as safe as JSMin. Tests on the YUI library have shown savings of about 18% compared to JSMin and 10% compared to the Dojo compressor (these respectively become 10% and 5% after HTTP compression)

How does it work?

The YUI Compressor is written in Java (requires Java >= 1.4) and relies on Rhino to tokenize the source JavaScript file. It starts by analyzing the source JavaScript file to understand how it is structured. It then prints out the token stream, replacing all local symbols by a 1 (or 2, or 3) letter symbol wherever such a substitution is appropriate (in the face of evil features such as eval or with, the YUI Compressor takes a defensive approach by not obfuscating any of the scopes containing the evil statement). The YUI Compressor is open-source, so don’t hesitate to look at the code to understand exactly how it works.

Where can I get it?

An archive containing both source and binary is available for download on this site.

How do I run it?

java -jar yuicompressor-1.0.jar
    [-h, --help] [--warn] [--nomunge]
    [--charset character-set] [-o outfile] infile

The following command line for example:

java -jar yuicompressor-1.0.jar myfile.js

will minify the file myfile.js and output the file myfile-min.js. For more information on how to use the YUI Compressor, please refer to the documentation included in the archive.

Limitations

Unlike JSMin, the YUI Compressor is slow and cannot be used for on-the-fly code minification (see minify for a PHP implementation of JSMin by Ryan Grove, another Yahoo! engineer, that does on-the-fly JavaScript minification among other things)

Has Yahoo!’s official position on obfuscation changed?

Douglas Crockford wrote an interesting article last year on minification vs. obfuscation. While that article still holds true, the YUI Compressor actually represents a 3rd way, in which code compaction does not carry the risk of introducing new bugs. The YUI Compressor is currently considered as a replacement for JSMin to compact the entire YUI library.

Feedback appreciated

The YUI Compressor is still a fairly new tool, so your feedback is well appreciated. Don’t hesitate to drop me a line if you find a bug, or think an important feature is missing. I will make updates available on this site, so watch for posts related to the YUI Compressor on this blog. This software belongs to the community, so it’s up to the community to make it better!

Additional notes

  • Do not hesitate to use the --warn option. The YUI Compressor will warn you if it finds anything that seems abnormal with your code, or that might reduce the level of compression.
  • Keep as little of your code as possible in the global scope. This has 2 benefits:
    1. You’ll get a better level of compression since the YUI Compressor does not obfuscate global symbols (which would make it unsafe)
    2. You won’t pollute the global object (see this article)

69 thoughts on “Introducing the YUI Compressor

  1. Martin Sévigny

    Thanks for making this available, it seems to work great.

    Out of curiousity, I tested it on Prototype (http://www.prototypejs.org/assets/2007/6/20/prototype.js), and I got the following exception:

    Exception in thread “main” java.lang.NullPointerException at YUICompressor.getHighestFnScope(YUICompressor.java:470)
    at UICompressor.parseScope(YUICompressor.java:704)
    at YUICompressor.buildSymbolTree(YUICompressor.java:498)
    at YUICompressor.main(YUICompressor.java:211)

    May be it can help debug the compressor.

  2. Tane Piper

    Very nice! I have seen around a 40% reduction in the size of my codebase. I’ve posted a link to your article on my own site, and in the jQuery mailing list.

  3. Julien Lecomte Post author

    @Martin

    I just posted an update that fixes the specific problem you mentioned (thanks for reporting it!) The use of ‘with’ in the global scope in prototype.js reduces the level of compression for that library (both ‘with’ and ‘eval’ are not recommended)

  4. Pingback: YUI Compressor: The latest minification tool « outaTiME

  5. Visa Kopu

    In my experience, Packer created by Dean Edwards has been the most efficient JavaScript compressor. I compared YUI Compressor and Packer and it looks like the results are very much alike. Sometimes YUI Compressor is better and sometimes Packer. If you want to get every extra byte out of your script, I recommend testing Packer, too.

    BTW, why is this application called YUI Compressor? As far as I can see, it has no relation to the Yahoo! script library with the same name.

  6. Julien Lecomte Post author

    @Visa

    Unfortunately, packer is not a safe compressor. Try the following with packer:

    function foo(aaa) {
    var yop = aaa;
    eval(“yop”);
    }

    You’ll get (if you check the “shrink variables” option):

    function foo(a){var b=a;eval(“yop”)}

    which is clearly incorrect…

    Also, the YUI Compressor will be shipped together with the YUI library in a not-to-distant future.

  7. Tane Piper

    @Visa
    I’ve had major problems with packer causing errors in my code, *so far* I haven’t had a problem with this packer yet on the same code.

    @Julien
    You say you have updated the release – is there any chance you have this up on a publicly available SVN/CVS? I’d like to be able to keep up to date with the latest code

  8. boris

    Thanks for this new tool :-)

    I’m trying to compress mootools (http://mootools.net) v1.11 full, but I get this error :

    16:28:08:bpopoff@grosminet:/var/www/test_boris/mootools$ java -jar yuicompressor-1.0.jar mootools.v1.11.uncompressed.js

    [INFO] Using charset Default
    java.lang.AssertionError
    at YUICompressor.parseCatch (YUICompressor.java:546)
    at YUICompressor.parseScope (YUICompressor.java:722)
    at YUICompressor.buildSymbolTree (YUICompressor.java:501)
    at YUICompressor.main (YUICompressor.java:211)
    at java.lang.reflect.Method.invoke0 (Method.java)
    at java.lang.reflect.Method.invoke (Method.java:255)
    at kaffe.jar.ExecJarName.main (ExecJarName.java:67)
    at kaffe.jar.ExecJar.main (ExecJar.java:75)

    Is it a bug, or perhaps I missed something ? It works whith some other scripts though. I can send you the file if you want.

  9. Greg Sandell

    Martin Sévigny, re: your Prototype error. I had trouble with one line in prototype when I was using JSMin. I haven’t started using YUI yet so I don’t know if it is the same problem. Here’s the problem:

    JSMin doesn’t like this line (hope it comes out in the post!). It says there is an unterminated quote.

    attr: /\[((?:[\w-]*:)?[\w-]+)\s*(?:([!^$*~|]?=)\s*((['"])([^\]]*?)\4|([^'"][^\]]*?)))?\]/

    Since in a regular expression ['"] is the same thing as ["'], just change the first ['"] to ["'] and JSMin is fooled into thinking there is a quote.

  10. Simone

    Impressive: I was setting up a “minification” step in the build process of Subtext (an OS .NET blogging engine), but now I think I’ll move from JSMin to this new YUI compressor.

  11. Pingback: Ajax Girl » Blog Archive » YUI Compressor: The latest minification tool

  12. Julien Lecomte Post author

    @boris

    I am not able to reproduce your error on the same file. Which platform are you on? Which version of Java are you using? It seems like it’s not able pick up the platform’s default charset. Could you try it on another computer maybe? You can also try to specify the charset explicitly using the –charset option. For example:

    java -jar yuicompressor-1.0.jar –charset US-ASCII myfile.js

  13. Boris

    Well, on another one I had

    [INFO] Using charset UTF-8
    Exception in thread “main” java.lang.AssertionError
    at YUICompressor.parseCatch(YUICompressor.java:546)
    at YUICompressor.parseScope(YUICompressor.java:722)
    at YUICompressor.buildSymbolTree(YUICompressor.java:501)
    at YUICompressor.main(YUICompressor.java:211)

    But finally, it worked on another one :

    [INFO] Using charset ANSI_X3.4-1968

    Quite strange …

    Here are the results for mootoools :

    Uncompressed : 180K
    JSmin : 73K
    YUICompressor : 65K
    Packer : 43K

    So for Mootools packer seems to be the best, but as you said, packer is not safe for all scripts.

  14. Pingback: User First Web » YUI Compressor, Web Site Speed

  15. Tarun

    For on the fly compression with JSP apps, I would recommend pack:tag (http://packtag.sf.net/)

    It currently uses JSMin, but since it only compresses once and caches that result, I wonder if YUI Compressor could be used instead.

  16. Pingback: YUI Compressor: compresor de javascript » ingeniuz :: desarrollo web útil

  17. Julien Lecomte Post author

    @boris

    The difference between Packer and the YUI Compressor most likely comes from the presence of either ‘eval’ or ‘with’ statements somewhere in the code. In that case, it is impossible to know for sure (without human intervention) whether the obfuscation has introduced a bug that was not there in the original code. The YUI Compressor will always adopt a defensive approach to that regard.

  18. Andrew Mattie

    Great work and great utility! Based on testing with our own libraries, YUI Compressor was able to beat about Dojo ShrinkSafe by an average of 3% across all of our JS files. 3% may not be a double-digit number, but when you’re serving up thousands upon thousands of requests for those files per day, it really adds up.

    However, that said, I’m curious to know why you think ShrinkSafe is unsafe. Since it also uses Rhino to tokenize the JS, shouldn’t it be just as safe as YUI Compressor? The only thing I can think of that wouldn’t be safe is the correction of variables when there are eval’s or with’s in the scope. Since “good” JS programmers don’t use those though, is there anything else about YUI Compressor that is more safe than ShrinkSafe?

    Also, speaking with regards to YUI Compressor not changing variables when there’s eval’s/with’s in the scope, couldn’t one make the argument that it’s unsafe overall then because of the possibility of global objects being referenced in eval’s? For example, if someone is actually using eval’s in their code, it’s possibly, if not likely, that they are also using global variables and referencing them within the eval’s. At that point, YUI Compressor would break their code. All I’m saying is that it’s almost impossible for anyone really to build a truly safe JS compressor that changes variable names.

    Finally, one more thing. I notice that YUI Compressor shares one of the same pitfalls with ShrinkSafe in that it doesn’t work with IE’s conditional comments. See http://dean.edwards.name/weblog/2006/06/again/ for an example; all the code within /* */ in those examples would be wiped clean with either compressor. ShrinkSafe hadn’t / hasn’t acknowledged this, and I had to learn about it the hard way.

    Anyway, once again, fantastic work!

  19. Julien Lecomte Post author

    @Andrew

    Thanks for the kind words. You are perfectly correct with regard to ‘eval’ and ‘with’ (and I invite you to take a look at the code) Also, the YUI Compressor seems to do a better job (than Packer and ShrinkSafe) at parsing complex variable declarations. For example:

    var foo = function () {…}, bar;
    // do something with foo and bar…

    Packer will not obfuscate bar while the YUI Compressor will.

    Finally, note that symbols in the global scope are not obfuscated. Otherwise, it would create bugs for sure when importing several JavaScript files in the same page. So the way the YUI Compressor deals with ‘eval’ and ‘with’ IS 100% safe.

  20. Pingback: New JavaScript minifier from YUI (Yahoo)at Developer Snippets

  21. gintro

    great work,
    but the following gives me an error!!
    -
    ['foo', 'bar'].forEach(function(item, index) {
    alert (index+’: ‘+item);
    });
    -

  22. Bob Coret

    First, great tool!
    Secondly, I’m getting error’s when there’s a try/catch block.

    The error:

    Exception in thread “main” java.lang.AssertionError
    at YUICompressor.parseCatch(YUICompressor.java:546)
    at YUICompressor.parseScope(YUICompressor.java:722)
    at YUICompressor.buildSymbolTree(YUICompressor.java:501)
    at YUICompressor.main(YUICompressor.java:211)

    Sample code which triggers error:

    var do_it;
    try {
    if (window.external.IsSearchProviderInstalled(‘http://www.stamboomgids.nl’) > 0) {
    do_it=1;
    }
    } catch(err) {
    do_it=2;
    }

  23. Nikola Klaric

    @Julien

    I’ve found a bug.

    Take the module pattern as described here:

    http://yuiblog.com/blog/2007/06/12/module-pattern/

    An eval(…) statement in a public method causes all private variables to remain unobfuscated by the compressor.

    A simplified example:

    —-

    var Foo = (function () {

    var privateVar1 = 1;
    var privateVar2 = 2;

    return {
    publicFunc1 : function() {},
    publicFunc2 : function(param) {
    eval(param);
    }
    };
    })();

    —-

    Comment out the eval(…) statement, and the private members will be obfuscated by the compressor.

    However, the eval(…) statement can’t possibly change the scope of privateVar1 etc., so the interpretation by Rhino/YUI Compressor seems to be wrong in this case.

  24. Julien Lecomte Post author

    @bob

    Indeed, there was a bad assertion in my code. I uploaded a new version (same file, but make sure you empty your browser cache before downloading it) that fixes this.

  25. Julien Lecomte Post author

    @Nikola

    This is not a bug, this is what makes the YUI Compressor 100% safe! Even if the specific case you are exposing seems trivial, it is not possible to reliably obfuscates what is inside an ‘eval’ statement! Therefore, it is not safe to obfuscate anything that is in a scope higher than the scope that contains the ‘eval’ statement. The solution: don’t use ‘eval’ (and don’t use ‘with’ either) For more info on why you shouldn’t use these, please refer to:

    http://www.jslint.com/lint.html

  26. Nikola Klaric

    @Julien

    Wouldn’t it be more adequate to introduce a directive that tells YUI Compressor that it shouldn’t draw the above noted conclusion? Example related to above use case which assumes that the parameter passed to eval() is well-known:

    —-

    publicFunc2 : function(param) {
    /* [yuic:ignore] */
    eval(param);
    /* [/yuic:ignore] */
    }

    —-

    As of right now, YUI compressor will leave the whole module unobfuscated, including the public method parameter symbols (param). Even with all the evils of eval() I don’t think this behavior is desirable.

  27. Kris Zyp

    In case you don’t see my comment on Ajaxian…
    Did you modify Rhino or Jargs at all, or does this work just with the addition of the five java classes that you wrote? Is there any reason why the classes are packaged in the root package? Why is not packaged in com.yahoo…? Do you have any API for calling it from other Java classes. I would like to integrate this into Resource Accelerate. I really like the fact that it doesn’t appear to have any Rhino mods, like shrinksafe did.

  28. Julien Lecomte Post author

    @kris

    - Indeed, there is no modification to both Rhino and Jargs (which makes updating them a trivial task)
    - There is no reason why the classes are packaged in the root package. I’m just too lazy to write com.yahoo…;-) Seriously, I don’t know of any Java code publicly distributed by Yahoo, so I am not aware of any naming convention. However, this might change in a future version.
    - If you want to integrate it in your own tool, just look at the code and start hacking it ;-)

  29. Pingback: YUI Compressor » D’ Technology Weblog: Technology News & Reviews

  30. Pingback: links for 2007-08-14 « napyfab:blog

  31. Pingback: Felipe Gaucho's Blog

  32. Pingback: All in a days work…

  33. Pingback: Cloudy Thinking » Blog Archive » Julien Lecomte: YUI Compressor, a new JavaScript minifier

  34. Mike

    Packer might not be safe for all javascript, it sure does minify the hardest! :)… Plus, JQuery gives it as an official package which will undoubtly be tested… No?

    My results:
    JQuery uncompressed: 61,4Kb
    JQuery Compressed with Packer: 21,0Kb
    JQuery Compressed with YUI Compressor: 31,4Kb

    Cheers :)

  35. Pingback: A Modern Fable (AJM) » Blog Archive » YUI Compressor - JavaScript compressor / minifier

  36. Julien Lecomte Post author

    @Mike

    Indeed, Packer yields higher savings on jQuery than the YUI Compressor. However, the generated code is very different from the original code (it is reconfigured for optimal compression) This may be fine for a library, but for your own code, I could not recommend it as it makes debugging a lot harder.

    Again, the goal of the YUI Compressor was to be able to handle ANY JavaScript file without adding any cost (no risk of introducing any new bug, does not make debugging that much harder, etc.) In my own experience, working for both small companies and large multinationals, the approach adopted by the YUI Compressor is the most cost effective.

  37. Severin Heiniger

    Apparently, YUI Compressor breaks Prototype’s new Inheritance Model (version 1.6 RC). Superclass methods can be called using “$super” as the first method parameter:

    var Cat = Class.create(Animal, {
    eat: function($super, food) {
    if (food instanceof Mouse) return $super();
    else return this.say(“Yuk! I only eat mice.”);
    }
    });

    http://www.prototypejs.org/2007/8/15/prototype-1-6-0-release-candidate

    “$super” gets renamed to “C” or whatever. Would it be possible to leave this keyword untouched without using –nomunged?

    Cheers :-D

  38. Julien Lecomte Post author

    @Severin

    First of all, I hope you’re using the latest version of the compressor (at the time of this post, it’s 2.1.2)

    Second, the answer is no. Renaming $super (which is a variable local to your function) should not have any impact at all. If you don’t get the result you were expecting when executing your code, it’s probably because you have a bug in your code, or in the Prototype library itself.

    Regards

  39. Severin Heiniger

    Thanks for your quick answer.

    I updated to the latest version of the compressor, but the problem persists as expected.

    Actually, it isn’t neither a bug in my code nor in Prototype’s. Prototype’s new inheritance model checks whether the first argument of the method (that overwrites the parent class method of the same name) is named “$super”; and wraps the new method if the keyword is found. If $super is renamed to something else, it gets treated like a normal argument.

    Quote (Prototype Blog): “If you’re overriding a method from a parent class, you can now access the superclass method by naming the overriding function’s first argument $super. It works just like Function#wrap (in fact, it uses Function#wrap internally).”

    Regards

  40. Pingback: YUI Compressor comprime tus scripts y tus hojas de estilos - Scriptia

  41. Matt Dolan

    Great tool, I need to look at it more fully as I’ve only really started thinking about compression this week. The last js file I compressed by hand so I thought I’d try this too and found it increased my file size!

    The only caveat I found is that the tool converts:
    if(t==0)c[i]=r;
    into:
    if(t==0){c[i]=r}

    Okay so this only adds an extra character for any one-statement ‘if’s and this only meant my file ran through YUI added 3 additional but unnecessary bytes over my 769 byte file (the original file started off at over 2000 bytes).

    One thing I’m currently looking into as I’m thinking of writing my own JS compressor is optimising it for gzip. What worked for me was figuring out what letters were used the most in the keywords used and then renaming the most common variable to the most common letter used. As compression tends to compress more commonly used items more this method, although painful to do by hand, saved me another 9 bytes once compressed. In order of use my variables became e, t, i, n, o.

  42. Pingback: Compression & the Web by jpsykes

  43. Pingback: New Site Improvements; switching to the Sandbox theme and a real build system.

  44. Cezary

    YUICompressor quotes every string in double quotes, but sometimes it’s cheaper to output a single one – ‘”‘ 3 characters – instead of “”" four characters. There are more cases like this

  45. Aki Hiisilä

    I tested version (2.2.1) and found a bug

    Input:
    var str = ‘a = “50″‘ + ‘b’;

    Output:
    var str=’a = “50″b”

Comments are closed.