Oct 21 2008
YUI Compressor and Java Class Loader
The YUI Compressor uses a slightly modified version of the parser used in the Rhino JavaScript engine. The reason for modifying the parser came from the need to support JScript conditional comments, unescaped forward slashes in regular expressions — which all browsers support and many people use — and a few micro optimizations. The problem is that many users had the original Rhino Jar file somewhere on their system, either in their classpath, or in their JRE extension directory (<JRE_HOME>/lib/ext) This caused many headaches because the wrong classes were being loaded, leading to many weird bugs.
Today, I finally decided to do something about it. This meant writing a custom class loader to load the modified classes directly from the YUI Compressor Jar file. You can download the source and binary package here:
Download version 2.4 of the YUI Compressor
The skeleton of the custom class loader is pretty straightforward:
package com.yahoo.platform.yui.compressor;
public class JarClassLoader extends ClassLoader
{
public Class loadClass(String name) throws ClassNotFoundException
{
// First check if the class is already loaded
Class c = findLoadedClass(name);
if (c == null) {
// Try to load the class ourselves
c = findClass(name);
}
if (c == null) {
// Fall back to the system class loader
c = ClassLoader.getSystemClassLoader().loadClass(name);
}
return c;
}
protected Class findClass(String name)
{
// Most of the heavy lifting takes place here
}
}
The role of the findClass method is to first locate the YUI Compressor Jar file. To do that, we look in the classpath for a Jar file that contains the com.yahoo.platform.yui.compressor.JarClassLoader class:
private static String jarPath;
private static String getJarPath()
{
if (jarPath != null) {
return jarPath;
}
String classname = JarClassLoader.class.getName().replace('.', '/') + ".class";
String classpath = System.getProperty("java.class.path");
String classpaths[] = classpath.split(System.getProperty("path.separator"));
for (int i = 0; i < classpaths.length; i++) {
String path = classpaths[i];
JarFile jarFile = new JarFile(path);
JarEntry jarEntry = findJarEntry(jarFile, classname);
if (jarEntry != null) {
jarPath = path;
break;
}
}
return jarPath;
}
private static JarEntry findJarEntry(JarFile jarFile, String entryName)
{
Enumeration entries = jarFile.entries();
while (entries.hasMoreElements()) {
JarEntry entry = (JarEntry) entries.nextElement();
if (entry.getName().equals(entryName)) {
return entry;
}
}
return null;
}
Once we know where the YUI Compressor Jar file is, we can load the appropriate class from that file. Note the need to define the package the class belongs to before calling defineClass!
protected Class findClass(String name)
{
Class c = null;
String jarPath = getJarPath();
if (jarPath != null) {
JarFile jarFile = new JarFile(jarPath);
c = loadClassData(jarFile, name);
}
return c;
}
private Class loadClassData(JarFile jarFile, String className)
{
String entryName = className.replace('.', '/') + ".class";
JarEntry jarEntry = findJarEntry(jarFile, entryName);
if (jarEntry == null) {
return null;
}
// Create the necessary package if needed...
int index = className.lastIndexOf('.');
if (index >= 0) {
String packageName = className.substring(0, index);
if (getPackage(packageName) == null) {
definePackage(packageName, "", "", "", "", "", "", null);
}
}
// Read the Jar File entry and define the class...
InputStream is = jarFile.getInputStream(jarEntry);
ByteArrayOutputStream os = new ByteArrayOutputStream();
copy(is, os);
byte[] bytes = os.toByteArray();
return defineClass(className, bytes, 0, bytes.length);
}
private void copy(InputStream in, OutputStream out)
{
byte[] buf = new byte[1024];
while (true) {
int len = in.read(buf);
if (len < 0) break;
out.write(buf, 0, len);
}
}
The last thing we need to do is bootstrap the application. In order to do that, we simply load the main class (YUICompressor) using our new custom class loader. All the classes that will be needed at runtime will use the same class loader:
package com.yahoo.platform.yui.compressor;
public class Bootstrap
{
public static void main(String args[]) throws Exception
{
ClassLoader loader = new JarClassLoader();
Thread.currentThread().setContextClassLoader(loader);
Class c = loader.loadClass(YUICompressor.class.getName());
Method main = c.getMethod("main", new Class[]{String[].class});
main.invoke(null, new Object[]{args});
}
}
As you can see, it’s not terribly complicated to write a custom class loader. Note: I left out all the exception handling code and the import statements for clarity. The final code can be found in the downloadable archive. Cheers!
Why aren’t these optimizations directly injected into the Rhino project, it would be very helpful?
Wouldn’t it have been much easier to just change the modified-Rhino package name? Just like for instance the Xalan version included in recent JREs live in com.sun.org.apache.xalan.internal. GWT uses a modified Rhino parser too, which lives in the com.google.gwt.dev.js.rhino package.
@JY
As a reference implementation, Rhino must stick as close as possible to standard ECMAScript. The changes I made to the parser are to accommodate the more relaxed syntax supported by modern web browsers, and to support features that aren’t part of the standard.
@Thomas
My goal was to modify the smallest possible number of files from the Rhino distribution. That way, fixes in Rhino ca be easily backported into the YUI Compressor. Moreover, using a class loader is a slightly more elegant (not to say funner) solution!
Hello,
Do you plan to support Javascript 1.7 features in a future version?
We are using the compressor for a Firefox extension, and we use a lot of JS 1.7/1.8.
Thanks.
Hello, and thanks for all your work on YUI Compressor! Seems to be very useful and solid, and I was excited to try it on some very large scripts I’ve written.
One problem I have found, though, is that Rhino (apparently) chokes on the “debugger” keyword in JavaScript. I have some scripts that, under certain circumstances, execute the debugger keyword to force a breakpoint and activation of a JavaScript debugger (if present) — i.e. as a developer, if I’m testing my JavaScript while it’s running in an instance of IE embedded in another application, and I need to get in and inspect the DOM or the state of my code, I set things up such that I can give the code some secret input (in my case I click a certain button with Ctrl+Shift held down), which triggers execution of the debugger keyword. This causes the embedded instance of IE to invoke my JS debugger — it’s very useful in some cases.
When YUIC encounters the debugger keyword in JavaScript, though, it reports the following syntax error:
[ERROR] xxx:yy:identifier is a reserved word
…
org.mozilla.javascript.EvaluatorException: Compilation produced syntax error.
at com.yahoo.platform.yui.compressor.YUICompressor$1.runtimeError(YUICompressor.java:135)
at org.mozilla.javascript.Parser.parse(Parser.java:410)
at org.mozilla.javascript.Parser.parse(Parser.java:355)
at com.yahoo.platform.yui.compressor.JavaScriptCompressor.parse(JavaScriptCompressor.java:312)
at com.yahoo.platform.yui.compressor.JavaScriptCompressor.(JavaScriptCompressor.java:533)
at com.yahoo.platform.yui.compressor.YUICompressor.main(YUICompressor.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.yahoo.platform.yui.compressor.Bootstrap.main(Bootstrap.java:20)
Seems like “debugger” is on a list of reserved words that Rhino is aware or, but at the same time it doesn’t expect to ever find it in a script. As I said, it executes perfectly well under Internet Explorer; it’s also documented as working in Mozilla & derivatives. Any chance you’d consider (further) modifying Rhino to accept the debugger keyword?