Scala on Google App Engine

Google App Engine

As has been reported already, Scala works on Google App Engine. Gabriele Renzi has even used Scala with the Google Eclipse Plugin, meaning he gets to use tools such as the "deploy to app engine" button. David Pollak has verified that most of the Lift framework works on App Engine, and he has posted a demo site to prove it. Overall, it looks like the scalac compiler produces jars that are so close to Java's that App Engine just doesn't notice or care. We can second that: it works for us, too! We would like to describe two sample projects we were involved in and how they went.

Sudoku solver

One of the sample projects we built is a Sudoku solver (code available here, live on App Engine here) by Alex Rudnick. Its front end is made in GWT (so there's some Java code involved) and GWT RPC connects the client to the server. It grew out of an earlier Python version that Alex had written; part of the motivation was to try out translating from a more dynamic language to Scala. The transition turned out to be pretty straightforward -- moving the test cases to Scala early on was a great help for a new Scala programmer, and scalatest was very pleasant and familiar to use.

This version of the solver uses Dancing Links, which is a clever way to do recursive search on a constraint-satisfaction problem without copying state on the stack like a naive recursive search might do -- it makes a guess about a constraint and modifies the data structure in place, then undoes the guess when appropriate. The approach comes from a paper by Knuth.

Most of the programming on the solver itself was done in vim, and the test cases launched from a Makefile at first. Pretty primitive, but it got the job done. Mysteriously, though, once the Scala implementation of the solver was working, it was slower than the Python version (!!). This turned out to be a bottleneck where the solver was doing random access on nested linked lists. This was okay in Python, which has array-backed lists as the default. Switching from linked lists to nested arrays took the solver from slow to snappy.

Once there was a working solver, we packaged up the compiled Scala code in a jar and linked it into a GWT project, made in the Google Plugin for Eclipse. At first, the server-side was a servlet implemented in Java, which made calls to the packaged-up Scala code. In the next iteration, instead of having the solver in a separate project, we put the Scala code into the same source tree as the Java, and just extended GWT's RemoteServiceServlet with a Scala class -- no need to make a jar at all, just one bilingual project. In the end, it was a lot of fun to implement, and as soon as it worked at all, it uploaded to App Engine and ran cheerfully.

LotlRepls and the Scala interpreter

LotlRepls, by Toby Reyelts and James Robinson, includes read-eval-print loops -- repls -- for a number of languages. A language being in LotlRepls gives bragging rights to both the language and App Engine. It means that the language doesn't just barely work, but that even sophisticated code such as an interpreter works on App Engine. Scala is included in LotlRepls. Getting it to work meant facing a few challenges along the way, however.

1. Early versions of the Scala interpreter wrote generated class files out to disk, in a temporary directory. This approach has problems on App Engine, because App Engine apps cannot write to anywhere on the local disk. Instead, they must persist data through database APIs. Happily, more recent versions of the interpreter write to an in-memory virtual disk, so this problem is avoided.
2. The Scala interpreter takes multiple seconds to start. This is an eternity for a web application, where response times should be on the order of 100 ms. To avoid this, LotlRepls creates a Scala interpreter the first time one is requested and then caches it. At each request, LotlRepls calls reset() on the interpreter instance so as to start from a clean slate. Calling reset() takes milliseconds rather than seconds, so this is a big savings.
3. The Scala interpreter has no way to serialize the interpreter state for use in a future instance. The programming model on the App Engine is that each request comes to a possibly different servlet instance, so any persistent state must be saved to a database. Since interpreter sessions don't persist, there is no way to maintain a session. Each new request to the servlet goes to a freshly reset interpreter session.

Conclusion

Overall, Scala works so smoothly on App Engine that there is little to add. The main issues you face, such as needing to persist everything to a database, are issues that are exactly the same as in Java or Python. If you like Scala, and you need scalable server-side support, give Google App Engine a try.