Seeking Sophrosyne

Node.js Experiments

So I played with Node.js for a couple of months, and we even deployed a widget for MIT's OpenCourseWare at OpenStudy with it. It was a remarkably fun experience, not the least thanks to CoffeeScript, which is a fabulously awesome syntax layer over JavaScript that gets rid of many of the lamest aspects of JS sytnax (for my money, the biggest change is the ability to abbreviate function() to ->).

But, ultimately, we ran into a few issues. The biggest one was that our real-time push strategy of choice, socket.io, ended up behaving rather poorly under the kinds of load characteristics we subjected it to. MIT's pages see thousands of hits a day (that's not ginormous, by the way -- about 3 hits per minute) and users tend to stay for a few minutes.

The first issue is that we park our site behind nginx, and nginx does not speak HTTP 1.0 when proxying. The result is that WebSocket connections can't be proxied through nginx (indeed, WebSocket proxy support is currently less than spectacular, and WebSocket has since proven to have some security issues). Nothing to worry about, socket.io falls back on alternate mechanisms on the (many) browsers that are missing WebSocket support, so we should still have been okay. Moreover, we were okay for this experiment with removing ourselves from being behind nginx.

The next issue is that the static serving aspect of Node.js is not quite fleshed out yet. We were using express and connect, which come with a middleware to gzip outgoing requests. Unfortunately, it has serious performance issues, so using it at any scale is not really a good idea. Still no problems, for the purposes of our experiment, we were willing to disable gzip.

Finally, we found ourselves in a situation where Node.js would randomly quit on us with timeout errors (this appears to be a timeout firing without an associated handler or some such). After some investigation, it appeared that this was a known issue that was difficult to fix. We tried to apply some fixes to socket.io to handle this, but we were still getting the issues. Given chronic crashing, then, we couldn't keep up the experiment. We even tried switching back from node 0.3 (which may not have been supported by socket.io) to node 0.2, but this led to strange CPU spikes we didn't really have time to experiment with.

Ultimately, we turned our attention back to Lift and the improvements made to it since the 2.0 release. Between the designer-friendly views, CSS selector binding, and the inclusion of lift-mongodb, Lift is still treating us super-nicely. The compilation wait is still annoying, but in proper style I dealt with it by getting a quad-core Core i7 iMac, which tears through it a bit faster.

This shouldn't be taken to mean that I'm saying Node.js isn't ready for primetime. Obviously there are plenty of people employing it in production and loving every minute of it. It was fantastic to be able to code in JS both server- and client-side, and going back to a dynamic language was fun and exciting after a while in the statically typed world. Then again, going back to Lift after Node was also kind of cool, so I guess it's just change that I'm a fan of.

For the most part, Node treated us well. With CoffeeScript, the syntax was awesome, and the developers who are working on Node and its libraries are the kinds of passionate, fast-paced people you expect to find around surging technology. It just didn't happen to work out for us. But! All was not wasted. I did throw together a quick library for doing something very similar to Lift's view-first, CSS-selector based transformations in Node, so look for that over the next couple of days.

ADDENDUM: The timeout issues we faced with node.js are being tracked at https://github.com/joyent/node/issues/378 , where it's been mentioned that they may be gone with Node 0.4.0.

Errors and callbacks in Node.js's node-mongodb-native

node-mongodb-native has the usual error handling mechanism of Node.js: we pass in a callback to handle results, and the first parameter is an error that will be null if no error occurred. For example:

Most such structures in Node seem to execute the callback outside of an error context. However, in node-mongodb-native, the toArray and nextObject callbacks (and by extension a couple of others) are wrapped in a try-catch that triggers the error callback. What does this mean? If your callback throws an exception, the same callback will be invoked with the error parameter set. Thus your callback may be invoked twice: once with successful data, and once with an error corresponding to the exception thrown within your own callback.

So if you're seeing some strange double-invocations of your callback, that's probably what's cracking.

UPDATE: See issue 81 on the node-mongodb-driver github repository to track the resolution of this issue.

OpenStudy's Latest Features

We pushed several new features to OpenStudy today, and I wanted to give a brief rundown of a couple of them really quick.

Public Access

First off is what we call `public access'. Almost all content on the site is now accessible without logging on to OpenStudy. For a few months now, since we first started our beta, OpenStudy has been a login-only site, where login is required to view any of the study groups, studypads, or users on the site. When we were in closed beta, this made sense, and in the run up to our open beta we had a ton of other features we wanted to get in for our users to play with. Now, it's time to throw the doors open even further.

When you aren't logged in, you will see a banner on every page:

You can interact with the page as usual, or click any of these buttons to log in, sign up, or connect with facebook. Moreover, if you want to join a study group or follow a studypad or user, you can go ahead and click the appropriate buttons and you will be prompted to log in before the action is taken.

Facebook Like and Tweet Buttons

With public access also come facebook like and tweet buttons. Now, if you are on a study group or studypad, you can like it on facebook or tweet it to your followers. Just look for the two buttons in the sidebar!

Everyone who sees the links will be able to get to the page and have a look around before deciding whether they want to sign up or not!

Help

We've been working for a while on some great help videos to explain some of the features of OpenStudy and how they can be used. Starting today, you'll find a link to the help videos in the OpenStudy header on any page:

Click on the link to view our 6 videos about OpenStudy (with full HTML5 video support, thanks to Vimeo).

We're always working on cool new stuff, so keep an eye out for the next batch of new features!

A Passing Note About Facebook Like Buttons

In Facebook's documentation of the like button, they mention that the like button tag needs an href attribute that specifies the URL you are liking. A little further down, in the section on Open Graph tags, when discussing the og:url Open Graph tag, it says:

og:url - The canonical, permanent URL of the page representing the entity. When you use Open Graph tags, the Like button posts a link to the og:url instead of the URL in the Like button code.

Now, the problem here is that the iframe version of the like button does not, in fact, post a link to the og:url—it still requires the href attribute¹. If you have the href attribute set correctly, after someone likes the page, Facebook will crawl it and properly import your Open Graph tags; however, the href attribute on the iframe's URL must be correct for this to work.

I spent a while banging my head against the wall wondering why the like button wasn't working before I figured this one out. The symptoms the like button shows are flashing briefly to a `1 person' like message and then immediately reverting to the 0 people `Like' button. Also note that the way to debug this is to pull up Firebug or the WebKit Inspector and watch the AJAX that goes back in forth. Look for the response to the facebook AJAX call and check out the body to see what error message you received.

¹ – I'm not sure whether or not the XFBML version of the button correctly uses the og:url tag or not, as I didn't try using it.