Generate less bytecode with default methods

Java’s default methods (introduced back in Java 8) are one of those features that solve the intended problem reasonably well, while at the same time allowing all kinds of nasty code and weird inheritance stuff when used in a general-purpose kind of way. Indeed, they’ve been the subject of a ton of posts and the way they (fail to) work still surprises people. They’ve been around for a while now, and there’s a lot of good advice out there on how they should and shouldn’t be used, so that’s not what I’m going to talk about in this post.

What I am going to talk about, however, is the way I’m using them in Moxy, as a way to do a lot less work in runtime-generated bytecode.

A bit of background

Moxy is a mock framework, and it does it’s job by generating mock classes at runtime. These mock classes naturally use a fair bit of bytecode generation, replacing real method implementations with new methods that implement the mock behaviour.

These mock methods do the standard mock thing of recording their invocation, and then they implement behaviour that has been set-up on the mock beforehand.

The problem

In order to implement this behaviour, mock objects need access to all kinds of state that’s specific to the instance – stubbed return values, exceptions they should throw, and more. They also need to have easy access to the mock engine that created them (so they can record invocations, for example).

This is often implemented via static methods on some class somewhere whose job it is to store state. The generated code will typically be peppered with a ton of invokestatic instructions that calls out to methods on those classes in order to get things done.

This works, but it has a few problems:

  • Some kind of mapping between instances and their state must be maintained.
  • It can make it difficult to swap out mock strategies in a clean fashion.
  • It can make testing (of the framework) difficult.

The Moxy approach

Instead of storing state statically, Moxy stores it right there in the mock, and all interaction with the state is done with instance methods. These methods can’t be inherited from a superclass (since mocks usually subclass the mocked class), and I really don’t want to generate them (I like to keep generated code to a minimum, as it’s harder to maintain and test that regular Java), so instead Moxy abuses default methods.

It works like this:

  • All mocks implement an interface, ASMMockSupport.
  • This interface defines one abstract method, __moxy_asm_ivars(). This method is implemented in generated code (since interfaces don’t have instance variables, the mocks hold the variables and expose them via this method)*.
  • The support interfaces defines a ton of default methods that do various kinds of work based on those instance variables.
  • The actual generated code for mocked methods uses invokeinterface calls on this to get stuff done as needed.
  • Other parts of the framework that interact with the mocks (e.g. stubbers) simple cast the mock instance to ASMMockSupport and call the methods they need.

* Yes, that’s a strange name for a method. Moxy does this to lessen the chances of its built-in methods clashing with methods being mocked.

 

The default methods on the support interface handle things like finding the engine that created the mock, adding stubbing, finding appropriate stubbing for a given invocation, running delegate methods and actions, and determining whether the current thread is currently in what Moxy calls a monitored invocation (used for recording purposes only, when mock behaviour is disabled).

This has the benefit that a whole lot of fairly-complex code that would otherwise be either generated, or stuffed into static methods, is now implemented, in Java, as instance methods. They’re easy to unit-test, and they’re easy to maintain. The added bonus is the generated bytecode is also easier to maintain, since it’s as short as possible.

Is this the intended use of default methods? Not at all. Is it correct, or appropriate? I’ll leave that to you to decide. I firmly believe, however, that in this very limited use-case it’s a good solution to a tricky problem, all things considered.

If you’re interested in looking at the code, you’ll find it here.

Advertisements

Mocking v2.0

We all love to use mocks in our Java tests, right? Add the usual mock framework to your test dependencies, sprinkle a few mock, when and thenWhatever calls in to your code, and happy days. You’re able to test things in isolation, and as an added bonus ensure your code is actually calling the stuff it’s supposed to be calling by verifying the mock afterwards. The sun is shining, life is good, and your code is tested to perfection.

But then the clouds roll in, and you have to test a final class. Or you need to mock out a static method in some legacy code. Or, horror, some code that directly calls new on a class you want to mock.

You could hit up StackOverflow, where you’ll find that all these things are possible with existing frameworks, assuming you’re prepared to add in another dependency, use a ton of boilerplate, and refactor your code a bit. Depending on which framework you’re currently using there are a ton of different ways you can kind of achieve what you need.

By now you’ve sunk another hour or two, and it’s still not quite working.

What if there was another way?

The other way

What if I told you there was a framework out there that let you take a final class, and just do this?

mockClasses(FinalClass.class);

FinalClass sc = new FinalClass();

assertThat(sc.returnHello()).isNull();

assertMock(() -> sc.returnHello()).wasCalledOnce();

when(() -> sc.returnHello()).thenReturn("Goodbye");

assertThat(sc.returnHello()).isEqualTo("Goodbye");

assertMock(() -> sc.returnHello()).wasCalledTwice();

What if it also did statics, with the same API, and no extra dependencies?

mockClasses(ClassWithStatic.class);

assertThat(ClassWithStatic.returnHello()).isNull();

when(() -> ClassWithStatic.returnHello()).thenReturn("Goodbye");

assertThat(ClassWithStatic.returnHello()).isEqualTo("Goodbye");

What if it even let you mock out constructors?

final RuntimeException rte = new RuntimeException("MARKER");

mockClasses(FinalClass.class);

when(() -> new FinalClass()).thenThrow(rte);

assertThatThrownBy(() -> new FinalClass()).isSameAs(rte);

What if we put all our arguments about whether we should be doing this kind of mocking to one side for now, because sometimes in life, you’re faced with a nail.

And when you’re faced with a nail, you need a hammer.

There’s one here: https://github.com/roscopeco/moxy

Or grab it with Maven:

<dependency>
	<groupId>com.roscopeco.moxy</groupId>
	<artifactId>moxy-core</artifactId>
	<version>0.90.1</version>
</dependency>

The “Scoring Predictions” kata

Official Ruby logo
Official Ruby logo (Photo credit: Wikipedia)

While idly surfing around earlier, I caught this post on coderbits, announcing that some of the Practicing Ruby archives are available under a free documentation license. When I saw that one of the released articles was written by James Edward Gray II, I just had to take a look.

In case you don’t know, JEGII is a “name” in the Ruby world, and for several years he ran the weekly ruby quiz on ruby-talk. At the time I was quite active on the list, and used to really look forward to the quiz as it always provided an interesting diversion from whatever I was working on that week. I even helped out by setting the occasional quiz myself (and that one won me a signed copy of James’ book).

Anyway, his article for Practicing Ruby was a great read, and proposed a problem that I’ve not specifically come across before. Just reading the intro re-awakened the old ruby-quizzer in me, so I had to have a go. Before I read any further than the problem definition, I wrote up some code. Here’s my first version:

def score(guesses, winners, scores = [15, 10, 5, 3, 1])
  guesses.zip(winners).inject([0, scores]) do |cscores, pair|
    guess, winner, cur_score, scores_ary = *pair, *cscores
    
    cur_score += if guess.eql? winner
      scores_ary.first
    elsif winners.include? guess
      1
    end || 0

    [cur_score, scores_ary[1..-1]]
  end.first
end

This works, but it feels funky in places. Using an array to pass multiple variables through the inject loop, for example, isn’t good coding. I don’t write anywhere near as much Ruby as I once did so I tend to forget things. Sure enough, reading the rest of the article reminded me that zip can take more than one argument!

This allowed me to refine my code a bit:

def score(guesses, winners, scores = [15, 10, 5, 3, 1])
  guesses.zip(winners, scores).inject(0) do |cur_score, (guess, winner, score)|
    cur_score + if guess.eql? winner
      score
    elsif winners.include? guess
      1
    end || 0
  end
end

This is also using another little trick I’d forgotten about (Mathias Meyer had a reminder for me in this post) – tuple arguments. Inject passes in the current score, plus the current three-element array from the zipped array. By surrounding the three argument names with parenthesis, Ruby splats the array out into three variables automatically for me. Sweet!

This fun little exercise reminded why I love Ruby so much. It’s the little things: zip and inject, ‘if’ as an expression (notice the ‘end || 0’ line at the end of the if block), tuple arguments and so much more.

Right now my day job project is a rails-based proprietary webapp, where I’m obviously writing Ruby, but somehow it’s a lot less fun than little snippets like this. My new resolution is to spend a little time every week seeking out these little problems – if Ruby Quiz no longer comes to me, then I must go to it!

 

Scratch: Teaching kids to think in blocks

The Scratch Cat
The Scratch Cat

Over the weekend I spent a few very rewarding hours with my ten-year-old son playing with Scratch, MIT’s drag-and-drop educational programming system. I’ve often wanted to “get him into” programming (I was a little younger than he is now when I started, and at ten I was beginning to get really interested in writing code on the old 8-bit machines) but haven’t really known where to begin. It turns out the ideal place was Scratch!

Although I’ve been aware of Scratch for a while (it comes pre-installed on the Raspberry Pi I got a few months ago, which I’ve not really done anything with yet) I only really looked at it properly last week, when a friend of mine asked me for a little help with an introduction to CS course he was doing, in which the first problem set was to create something in Scratch (I helped him make a Space Invaders clone). It was while working on that I realised I really should introduce Jake (that’s my son) to Scratch. After I mentioned it to him, it turned out he’d looked at it a little at school, and so it was decided – we’d spend a bit of time on Sunday playing around with it.

Continue reading

Biting the bullet – time for a rewrite

In response to a feature request from some of the guys here, my current work on Deelang focusses on implementing proper equality and comparison operators as part of the language. The current release has no support for operators beyond basic arithmetic – equality and comparison have traditionally been implemented as methods hacked on top of the standard library. This results in code such as:

1.eql(2)  => true
5.lt(10)  => true
10.gt(20) => false

Leaving aside for a moment that the above isn’t actually that bad (in my opinion at least), implementing this as operators is actually quite simple for both the dex and deevm compilers. However, while putting it together I ran up against an old elephant that’s been sitting quietly in the corner for some time now – the current parser is a mess, and it might just be time to rewrite the grammar.

The problem is not new, but I’ve ignored it for a while. Basically, it boils down to the fact that the current parser cannot handle code such as:

(1+2).foo()

Not only does this not work, but it fails miserably with nothing more specific than a MismatchedSetException at the terminator, after a lot of backtracking and ultimately ignoring the method call completely. The above code parses to the following tree:

Parse Tree Notice all the abandoned trees (in red), before the (erroneous) final parse, and the MismatchSetException up there on the right. The method call gets parsed at one point, but that tree was then abandoned in favour of a tree in which the call is quietly ignored. This is clearly one confused parser, and all over something as simple as (1+2).foo()! Clearly, this needs fixing.

As I say, I’ve ignored this for a while. It should be relatively simple to fix (and indeed it is) with a bit of rejigging in the grammar. However, this problem is actually symptomatic of something deeper – namely, that the Deelang grammar is a mess. In the past, as problems such as this have cropped up, they’ve been fixed by adding to the grammar. New productions, imaginary tokens, and syntactic predicates have all been added to cope with a specific case, with no real wider plan.As long as the tests still passed at the end, the additions stayed.

The result of all this is that things that should be handled in a unified way are actually handled in a variety of ways. My personal favourite example of this is the way chained method calls are handled – I won’t illustrate it here, but if you’re interested just debug something like “foo().bar().baz().quux()” in ANTLRWorks. Trust me, it’s not pretty. It’s inefficient, it’s inconsistent with other parts of the grammar, and it requires the compiler to jump through some pretty awkward hoops to keep track of who is using what target register. It worked well when the only target was the (stack-based) Dee vm, but as requirements have grown it’s become cumbersome – the only reason it still works this way is inertia.

With all this in mind, I’ve decided that now is the ideal time to rewrite the grammar to get rid of these issues. I’ve never been a big believer in planning to throw one away, but in this case it looks like I will, anyhow. To be fair, I’m not planning a complete rewrite – large parts of the grammar are fine as they are (literals, for example). But the meat of it – from atoms through function calls to method calls – will be rewritten in a way that’s more consistent, cleaner, and hopefully requires a lot less backtracking. I’m also aiming to reduce the lookahead where I can, although some of the actual language design makes this quite difficult.

Unfortunately this grand plan must remain just that for now – I’m very short on time to actually work on this at the moment. Since it’s not an actual key requirement I can’t allocate any actual work time to it (even though it will make things easier and save time down the line). So provisionally, I’ve set aside Sunday for the rewrite.

Now I just need to hope nothing more pressing crops up between now and then. Wish me luck!