Time Machine to the cloud.

Cool.

Then they announce Portal is FREE! (until May 24th, 2010 at least).

Awesome!

So I download the Steam .dmg to my Mac, copy Steam over to /Applications, try to run it, and promptly get:

Steam requires that Steam.app/Contents/MacOS be created on a case-insensitive file system, with read-write access.

Wut!?

Why!?

(I mean, why couldn’t they have ported the thing properly and made sure the paths/file names in the code were all lower case or something?)

You know—there are people who like case-sensitive file systems because it forces them to be more careful with how they code. You know—not every file system is like, FAT32 where “readme.txt”, “README.TXT” and “ReadMe.tXt” all refer to the same file!

Anyway—while I love to rant, I also like to fix things. So here goes:

1. Create a new partition, formatted simply as Mac OS Extended (Journaled)

Note—NOT case-sensitive. I made mine 10GB, and labeled it steam. Fortunately, Disk Utility lets us play around with partitions. Who needs Partition Magic?

2. Copy Steam.app to the new partition

But wait, there’s more! If you try to run Steam now, it’ll complain about ~/Library/Application Support/Steam (and later, ~/Documents/Steam Content). Hang in there with me for a second.

3. Create the necessary folders on your new partition

I ended up creating /Application Support/Steam and /Documents/Steam Content on the steam volume.

4. Create symbolic links to these new folders from where Steam looks for them

First make sure that the folders ~/Library/Application Support/Steam and ~/Documents/Steam Content don’t exist yet (the Steam app will try to create these for you). If they do, just delete them.

Time for some Terminal magic. Run Terminal, and go

cd ~/Library/Application\ Support/
ln -ns /Volumes/steam/Application\ Support/Steam/ Steam
cd ~/Documents
ln -ns /Volumes/steam/Documents/Steam\ Content/ "Steam Content"
sudo ln -s /Volumes/ /volumes


The last line will require you to enter your administrator password. I just added that in courtesy of Josh Anderson’s own post. Without that other symbolic link, it seems Steam will have trouble finding the content/games it’s already downloaded.

5. Run Steam.

6. Download Portal

7. …

8. Profit!

You’re welcome.

Added: Also, here’s another workaround by Josh Anderson that doesn’t require you to create a new partition, but uses a case-insensitive disk image instead.

SEPA Explained

Johnson discovered that a set of logical steps could be taken from one permutation to the next. Repeatedly calling the algorithm on the last output obtained generates all permutations in sorted order.

It works by first scanning for the rightmost ascent or pair of numbers where the first number is less than the one immediately after it.

If no ascent is found, then all numbers are in descending order and this is the last, lexicographic permutation.

Otherwise, the number at the start of the ascent is swapped with the smallest higher number to its right.

Finally, all the numbers to the right (which will be in descending order) are ‘flipped’ or reversed (to ascending order). The resulting array is the next permutation.

P(n) to P(n,k)

To begin generating P(n,k), let’s take a look at the first few permutations of 5 elements taken 3 at a time, P(5,3) and compare this with P(5):

If we look carefully at the first 3 values of P(5), a₀, a₁ and a₂, we actually find P(5, 3). Except, to get P(5, 3) from P(5) we need to ‘skip’ a few permutations.

Basically, we can safely ‘skip’ all permutations in P(5) that don’t permute the first 3 positions. For purposes of discussion, we designate the k^th or 3^rd position, a₂ as the ‘edge’.

Another way to look at this is that in generating P(n) using Johnson’s SEPA, we look for the rightmost ascent. In P(n, k) generation, we’re only interested in ascents at the ‘edge’ or to the left of the ‘edge’.

We first assume that the edge contains our ‘ascent’ and look for the next higher number to the right of the edge.

If all numbers to the right of the edge are smaller, then we know that the ascent must be to the left of the edge.

However, unlike with regular P(n) generation, at this point all numbers to the right of the edge will be in ascending order. We know this since we began with all numbers in ascending order, and all our operations at this point simply swap the value at the edge with the next higher number.

Fortunately, by simply reversing everything to the right of the edge we return to a state similar to P(n) where all values to the right of the actual ascent are in descending order. We can now proceed as with the regular SEPA.

A Simple, Efficient Algorithm for Generating P(n, k)

This leads us to the adapted algorithm for generating permutations of n items taken k at a time, in lexicographic order.

def a = { 0, 1, 2 … (n - 1) }
def edge = k - 1

// find j in (k…n-1) where aj > aedge
j = k
while j < n and aedge >= aj,
    ++j

if j < n {
    swap aedge, aj
} else {
    reverse ak to an-1

    // find rightmost ascent to left of edge
    i = edge - 1
    while i > 0 and ai >= ai+1,
          --i

    if i < 0,
          // no more permutations
          return 0

    // find j in (n-1…i+1) where aj > ai
    j = n - 1
    while j > i and ai >= aj
          --j

    swap ai, aj
    reverse ai+1 to an-1
}

output a0, a1 … ak-1

SEP(n,k) Algorithm Illustrated

To show how the algorithm works, let’s step through the major operations showing the state of the array a at each step.

Java Implementation

A Java implementation of this algorithm is provided as SepaPnkIterator. It is part of jcombinatorics, the Java Combinatorics Library up on GitHub and released under the MIT license.

Ad-hoc performance testing was conducted on a 2.4 GHz Intel Core 2 Duo with 2GB RAM running OS X 10.5.8 and Apple’s Java 1.6.0_15. Results revealed that Johnson’s SEPA can generate all 479,001,600 permutations of P(12) at over to 38,000 permutations/ms. The above algorithm computes P(12,12) at around 32,000 permutations/ms which makes it nearly 85% as fast as SEPA P(n).

This article is also available for download as a PDF.

I’d love to hear from others if you have any thoughts to share on a formal proof of this algorithm, or of analysis of its running time. Meanwhile, Enjoy!

Recap

In my previous post, I demonstrated how to use Sun’s HttpServer API to write a functional test of an HTTP ‘conversation’.

Recall that I thought my initial solution seemed inelegant. It was verbose, with some start up and shutdown code that would have to be repeated for each test, and which I felt cluttered the actual test code.

It was also tedious, in the sense that the raw HttpHandler and HttpExchange API required us to do quite a few things manually, and unintuitively (such as having to compute and write out the length of our response before the response itself).

In this post, we’ll explore how to use the new Interceptors feature ‘quietly’ released with JUnit 4.7 to write reusable, portable pre and post-test behaviour. I’ll also exhibit a convenient HttpHandler implementation that simplifies some of the effort required in responding to HTTP requests.

JUnit Interceptors at a glance

Developers familiar to JUnit will know that prior to JUnit 4.x, to perform pre/post test behaviour such as setting up and tearing down test scaffolding or external resources, we had no choice but to override TestCase#setup() and TestCase#tearDown(). To make this reusable across test classes, we had to extend TestCase and our actual tests had to extend that custom class.

JUnit 4.x introduced annotation-driven testing using the @Test annotation, which allowed us to write tests that were simple POJOs. However, this didn’t help when we wanted reusable pre/post test behavior. We still had to write our custom base class, this time with methods annotated with @Before and @After.

JUnit 4.7 introduces a new feature called “Interceptors” that aims to bring back to JUnit the ability to to ‘meta-testing’ with a much cleaner and simpler API. See Kent Beck’s¹ and David Saff’s² blog posts for more of the inside scoop.

Writing an Interceptor

A good place to start is to look at the source for ExternalResource.

If we need complete control over our interceptor’s behavior, we’d need to implement MethodRule from scratch.

In our case, we can get by simply by extending ExternalResource. We setup our HTTP server in the before() method, and tear it down in our after() method. We also provide a way to specify the port to listen on in our constructor:

Next, we provide a delegate method to register handlers:

Using Interceptors

To use our new JUnit interceptor, all we need to do is annotate a public field with the org.junit.Rule annotation. Note that the field has to be public—this might raise a warning if you use Checkstyle and personally, I wish JUnit would allow private @Rules.

The JUnit runner does the rest. It’ll transparently call our interceptor’s before() method, run the @Test then call our interceptor’s after() method.

Armed with this, we can rewrite the rest of our HTTP functional test as:

Simplifying our Handlers

We’ve managed to cut out some of our test scaffolding code. Now let’s simplify our actual handler. For this, I wrote SimpleHttpHandler, which is an HttpHandler which provides delegate methods to the actual HttpExchange. That is, by extending SimpleHttpHandler we can simply go getRequestURI() instead of having to go httpExchange.getRequestURI().

Additionally, SimpleHttpHandler uses an internal ByteArrayOutputStream, exposed via a PrintWriter in the getResponse() method. This lets us simply write our response as if, say, to System.out, and it takes care of computing the total response length later on when we call the sendResponse() method.

Here’s the same handler above rewritten using SimpleHttpHandler:

The complete source to HttpServerInterceptor and SimpleHttpHandler are provided as part of junit-rules, a ‘pet’ project I’ve made to explore and showcase JUnit Rules.

Hopefully, I’ll be able to add other, useful rules to junit-rules. Feel free to download, study and use it. Better yet, if you want to share any useful rules of your own, by all means, please fork or contribute to it!

(Edit 2009/10/16: Changed links/references to junit-interceptors to junit-rules, all because I misread Kent Beck’s post—they were renaming it to Rules from Interceptors, and not the other way around!)

References

¹ Interceptors in JUnit
² JUnit 4.7: Interceptors: expected exceptions

Forces

At work, we recently had the need to perform functional testing of a custom client that used HTTP as a transport. This isn’t strictly unit testing since we’re conducting actual HTTP over a socket & port instead of stubbing out or mocking the server, but in this case that was the only real way to test the client.

I could’ve fired up a standalone Web server and used that, but decided against it for a couple of reasons.

First, I wanted to have the server respond in a specific way to a particular client request. For example, if the request was for GET /1234.xml I might want to respond with an HTTP 200 and an XML response body. Another request for GET /0.xml might return an HTTP 404 instead.

To do that using, say, a Servlet container would mean writing multiple Servlets (mapped to various request URI) or a ‘rich’ Servlet with additional complexity. I didn’t want to have to write tests to test my test scaffolding!

Secondly, a standalone server would have to be started and stopped outside of our standard compile/test/package process (using Maven). Other people wouldn’t be able to run the tests successfully without having the test server up as well.

Clearly, the best way to go was to use an embedded HTTP server, which would allow us to provide specific responses tailored for each unit test.

As luck would have it, it turns out that Sun’s Java 6 implementation comes with a lightweight HTTP server API built in. Read on as I demonstrate the basic use of Sun’s HTTP server classes to write a functional test.

HTTP server in a box

The heart of our test solution involves taking advantage of the lightweight HTTP server API included in Sun’s Java 6 implementation. Note that since this isn’t part of the Java core API this package may not be available on all Java platforms. If this is a problem, you might be better off using another embedded HTTP server such as Jetty.

The class itself is com.sun.net.httpserver.HttpServer, and here’s how to use it in a nutshell:

1. Create the server
2. Create a server context and register a request handler
3. Start the server
4. Perform your test
5. Stop the server, and verify/assert your expected behavior

Now let’s look at each step in detail with corresponding code.

Create the server

We create an HttpServer using HttpServer.create(). To specify a port, we need to pass in an InetSocketAddress:

The second parameter to HttpServer.create() is the ‘backlog’, “the maximum number of queued incoming connections to allow on the listening socket”. Since that doesn’t really affect us, we can just pass in a 0 and a system default value is used.

Creating and registering a request handler

Now we get to the meat of actually stubbing our server’s behavior by creating and registering a request handler. HttpServer provides the createContext(String path, HttpHandler handler) method to do just that.

Now, HttpHandler is an interface which declares one method: handle(HttpExchange)

Obviously, HttpExchange is the class we need to work with when responding to HTTP requests. Let’s look at some code and I’ll explain what it does afterwards:

Basically: we convert our response string into a byte array. We send an HTTP 200 (OK) along with the number of bytes we’re about to send as the response body. We then write out the bytes of our response body, then close the HttpExchange. The complete HttpExchange life cycle is detailed in its API documentation.

We now create a context for the URI we’re interested in, passing in our newly created HttpHandler, then start the server:

At this point, an actual HTTP server will start running in a background thread ready to respond to requests. Let’s exercise our client code:

Our client code connects to our server, retrieves the associated URI, then, using a BufferedReader reads in lines from the input stream. We make a few JUnit assertions on what we received.

The only thing left to do is to stop our HttpServer:

The parameter is the number of seconds (NOTE: not milliseconds) to block to wait for our HttpServer to shutdown properly. Since we know we’re not serving any other requests, we can safely tell our HttpServer to shut down immediately.

At a glance

Here’s our functional test at a glance:

While functional, I find the above code rather verbose. If I want to write more HTTP tests, I certainly don’t want to keep repeating myself going httpServer.start(); … httpServer.stop(0);. I especially find it tedious to have to compute the length of the HTTP response body beforehand, then send it along with our HTTP response code before writing out the actual body.

In the next part of this article, we look at using some shiny new features in JUnit 4.7 that lets us write cleaner, more reusable tests that involve pre and post test behavior. Also, I’ll exhibit a helper class that simplifies some of the effort involved in implementing an HttpHandler.

If you read through the JFace wiki page above, though, it says that to use JFace + SWT outside of Eclipse we basically have to locate several dependencies, namely:

• org.eclipse.core.commands_<version info>.jar
• org.eclipse.equinox.common_<version info>.jar
• org.eclipse.jface_<version info>.jar
• org.eclipse.osgi_<version info>.jar
• org.eclipse.ui.workbench_<version info>.jar

All of these are available in the standard installation of Eclipse IDE in the ${ECLIPSE_HOME}/plugins directory.

Now, it would be a 10 minute job to manually search for, copy and paste the actual JAR filenames and execute the Maven commands to install those into the local repository with proper group and artifact ids.

Since I’m ‘lazy’, though, instead of 10 minutes I decided to spend a couple of hours writing a Ruby script to do this all for me (I originally started to write it in bash but I realized it’d be a lot easier in Ruby).

Here it is. Save this as mvn_install_jface.rb somewhere, and make it executable. To use the script, simply go:

mvn_install_jface.rb ${ECLIPSE_HOME}

Obviously, replace ${ECLIPSE_HOME} with the directory where you installed Eclipse into.

If you don’t supply the directory, the script defaults to /Applications/eclipse which is where Eclipse would normally be installed in OS X.

First (after the environmental/sanity checks), the script tries to search for all the JFace JARs and will exit with an error if it can’t find any of the JARs we need.

The script will run all the necessary mvn install:install-file ... commands and figure out/supply the appropriate groupId, artifactId and version for all the JARs.

Finally, it’ll helpfully print out a bunch of <dependency>...</dependency> snippets you can copy & paste directly into your Maven pom.xml to use the found JFace JARs immediately. You’re welcome.

I’ve only tried this on OS X, and would love to hear from people who’ve tried it and gotten it work on other operating systems, especially Windows. I figure under Windows, if you already have Ruby and Maven on the path the command should be something like:

ruby mvn_install_jface.rb C:\eclipse

But I don’t have a Windows machine with Ruby, Java, Eclipse and Maven handy right now to test it myself.

Added 2009.07.24: I just realized the `which mvn` part won’t work at all under Windows, so Windows users will have to edit that part out—just make sure mvn is on the path.

(Oh and legalese blah blah MIT License.)

Since I use Maven, I had to struggle a bit finding out how to configure my Maven project properly so that it’ll build an SWT app that’ll run both under Eclipse and from the terminal.

A little Googling brought me to Brice Lambi’s post on Maven SWT builds, but that was a little outdated.

In particular (and only after careful reading), the guide to Deploying SWT apps on Mac OS X says that the -Djava.library.path=.. option is needed for Eclipse 3.2.2 and earlier. Apparently, starting with 3.4 all you need is your OS-specific JAR (see here).

Anyway, after much fiddling about, I was able to get it all working like so:

Download and Install SWT to your local repository

First, you’ll want to download and extract the SWT release specific to your OS from eclipse.org.

(I’m using swt-3.4.2-carbon-macosx.zip for the rest of this article.)

Next, you’ll want to install the extracted swt.jar into your local Maven repository with a suitable groupId and artifactId. Following the conventions on the Ibiblio Maven repository, I used org.eclipse.swt.carbon and macosx, respectively. Here’s the Maven command to do this:

mvn install:install-file -DgroupId=org.eclipse.swt.carbon -DartifactId=macosx -Dversion=3.4.2 -Dpackaging=jar -Dfile=swt.jar

You might also want to point Maven to the SWT sources, so that if you use m2eclipse it can ‘attach’ those sources to the library JAR appropriately:

mvn install:install-file -DgroupId=org.eclipse.swt.carbon -DartifactId=macosx -Dversion=3.4.2 -Dpackaging=jar -Dfile=src.zip -Dclassifier=sources

Configure your POM

A thousand XML elements is worth a thousand words:

If you’re only ever going to target one OS, then you can do without the <profiles... > section and ‘hard-code’ your OS-specific dependencies directly. I’ve left it there as a template to make your Maven project more portable.

Run it!

To run this from the command line, I made a simple bash script that basically goes:

java -XstartOnFirstThread -cp ~/.m2/repository/org/eclipse/swt/carbon/macosx/3.4.2/macosx-3.4.2.jar:
target/swt-test-0.1.jar
com.example.swt.test.Main

The real trick there is the -XstartOnFirstThread option to the Java runtime.

This is the same trick you’ll need to run the app under Eclipse. All you need to do is in Run -> Run Configurations..., in the “Arguments” tab specify -XstartOnFirstThread in the “VM arguments” field and you’re all set!

(Edited 7/16/09 to align the SWT group and artifact id’s with those on the Ibiblio Maven repository, and added the Windows profile.)

Which brings me to the real reason for coming up with MagicTest—ActiveTest.

A code sample is worth a thousand words. Suppose we have a Spring JPA data access object:

WidgetDao needs a JPA EntityManager provided to it at construction time. Normally, to write a unit test for WidgetDao we’d have to create our mock objects and setup our test scaffolding manually.

Using ActiveTest, however, all we need to write is:

I’m lazy like that.

Under the hood

ActiveTest uses the ParameterizedType hack to discover the actual type parameter (the class under test) at run-time.

It then inspects the test class instance for any fields annotated with Mockito’s @Mock, and has Mockito construct those mock objects. At the same time, each mock object is registered into an internal Spring auto-wiring capable bean factory.

Finally, that Spring bean factory is itself used to construct an instance of the class under test. Because we’re using Spring itself to perform construction, we fully support field, setter or construction-based @Autowired injection of dependencies.

See the full source code of ActiveTest on the dirty-mockito project site on Google Code

Now, how often have you wished it were possible to simply go straight to @Test, do not pass setUp(), do not have to new Foo()?

Introducing MagicTest

So I came up with MagicTest, a parameterized base class for JUnit 4 tests that’ll let us do just that.
It lets us rewrite the above simply as:

It was bound to happen. I was bound to put to use my ‘hack’ on how to discover the type parameter to a generic base class.

Without further ado, here’s a simplified version of MagicTest:

That really wasn’t too hard, now, wasn’t it?

Why call it MagicTest? Well, because I also happen to like using the Mockito mocking framework, and I happen to already have a unit test base class called MagicMocker. Get it?

EDIT: steve k wrote in and opened my eyes to:

sudo port uninstall --follow-dependents portname

So use that instead, and read on only if you like bash.

(I have another script that does the inverse—traverses up the dependency chain removing ‘leaf’ nodes—that is, ports with no further dependencies. Maybe I’ll make a new post on that one—after making sure it’s also not already available as some hidden option. :p )

If you use MacPorts, then at some point you’ve probably tried to uninstall a port that had other ports as dependents. (If you’re a developer on OS X and you don’t use MacPorts, then you probably should.)

Even when using a friendly GUI like, say, Porticus, you’re prevented from uninstalling a port with dependents. Of course, you can force uninstall a port but MacPorts themselves warn us that doing so might cause problems later on (when rebuilding the dependents).

So, I used to manually traverse down the dependency tree and uninstall the leaf nodes (ports with no dependents) one by one until I could uninstall the ‘root’ port that I wanted to uninstall in the first place.

If you like doing that, then by all means keep doing so.

If, like me, you’re tired of having to do it by hand, then here’s a short shell script I wrote that can do all that for you.

Note that the script will fail if you have more than one installed version of any port. MacPorts itself will helpfully tell you which ports in the dependency tree have multiple versions installed. Maybe one of these days I’ll get motivated and challenged enough to update this script to handle (and uninstall) multiple versions properly.

I initially thought of writing this in Ruby, but since I used MacPorts to install Ruby itself I realized I very well couldn’t use the same script to uninstall anything up the Ruby dependency chain. Fortunately, I found that simple shell scripting (plus common tools like awk) was more than adequate for the task.

With that out of the way, here’s my deport.sh script:

If you find this useful, then you’re welcome! If it doesn’t work for you, please drop me a line. g