Generate less bytecode with default methods

Java’s default methods (introduced back in Java 8) are one of those features that solve the intended problem reasonably well, while at the same time allowing all kinds of nasty code and weird inheritance stuff when used in a general-purpose kind of way. Indeed, they’ve been the subject of a ton of posts and the way they (fail to) work still surprises people. They’ve been around for a while now, and there’s a lot of good advice out there on how they should and shouldn’t be used, so that’s not what I’m going to talk about in this post.

What I am going to talk about, however, is the way I’m using them in Moxy, as a way to do a lot less work in runtime-generated bytecode.

A bit of background

Moxy is a mock framework, and it does it’s job by generating mock classes at runtime. These mock classes naturally use a fair bit of bytecode generation, replacing real method implementations with new methods that implement the mock behaviour.

These mock methods do the standard mock thing of recording their invocation, and then they implement behaviour that has been set-up on the mock beforehand.

The problem

In order to implement this behaviour, mock objects need access to all kinds of state that’s specific to the instance – stubbed return values, exceptions they should throw, and more. They also need to have easy access to the mock engine that created them (so they can record invocations, for example).

This is often implemented via static methods on some class somewhere whose job it is to store state. The generated code will typically be peppered with a ton of invokestatic instructions that calls out to methods on those classes in order to get things done.

This works, but it has a few problems:

  • Some kind of mapping between instances and their state must be maintained.
  • It can make it difficult to swap out mock strategies in a clean fashion.
  • It can make testing (of the framework) difficult.

The Moxy approach

Instead of storing state statically, Moxy stores it right there in the mock, and all interaction with the state is done with instance methods. These methods can’t be inherited from a superclass (since mocks usually subclass the mocked class), and I really don’t want to generate them (I like to keep generated code to a minimum, as it’s harder to maintain and test that regular Java), so instead Moxy abuses default methods.

It works like this:

  • All mocks implement an interface, ASMMockSupport.
  • This interface defines one abstract method, __moxy_asm_ivars(). This method is implemented in generated code (since interfaces don’t have instance variables, the mocks hold the variables and expose them via this method)*.
  • The support interfaces defines a ton of default methods that do various kinds of work based on those instance variables.
  • The actual generated code for mocked methods uses invokeinterface calls on this to get stuff done as needed.
  • Other parts of the framework that interact with the mocks (e.g. stubbers) simple cast the mock instance to ASMMockSupport and call the methods they need.

* Yes, that’s a strange name for a method. Moxy does this to lessen the chances of its built-in methods clashing with methods being mocked.

 

The default methods on the support interface handle things like finding the engine that created the mock, adding stubbing, finding appropriate stubbing for a given invocation, running delegate methods and actions, and determining whether the current thread is currently in what Moxy calls a monitored invocation (used for recording purposes only, when mock behaviour is disabled).

This has the benefit that a whole lot of fairly-complex code that would otherwise be either generated, or stuffed into static methods, is now implemented, in Java, as instance methods. They’re easy to unit-test, and they’re easy to maintain. The added bonus is the generated bytecode is also easier to maintain, since it’s as short as possible.

Is this the intended use of default methods? Not at all. Is it correct, or appropriate? I’ll leave that to you to decide. I firmly believe, however, that in this very limited use-case it’s a good solution to a tricky problem, all things considered.

If you’re interested in looking at the code, you’ll find it here.

Advertisements

Mocking v2.0

We all love to use mocks in our Java tests, right? Add the usual mock framework to your test dependencies, sprinkle a few mock, when and thenWhatever calls in to your code, and happy days. You’re able to test things in isolation, and as an added bonus ensure your code is actually calling the stuff it’s supposed to be calling by verifying the mock afterwards. The sun is shining, life is good, and your code is tested to perfection.

But then the clouds roll in, and you have to test a final class. Or you need to mock out a static method in some legacy code. Or, horror, some code that directly calls new on a class you want to mock.

You could hit up StackOverflow, where you’ll find that all these things are possible with existing frameworks, assuming you’re prepared to add in another dependency, use a ton of boilerplate, and refactor your code a bit. Depending on which framework you’re currently using there are a ton of different ways you can kind of achieve what you need.

By now you’ve sunk another hour or two, and it’s still not quite working.

What if there was another way?

The other way

What if I told you there was a framework out there that let you take a final class, and just do this?

mockClasses(FinalClass.class);

FinalClass sc = new FinalClass();

assertThat(sc.returnHello()).isNull();

assertMock(() -> sc.returnHello()).wasCalledOnce();

when(() -> sc.returnHello()).thenReturn("Goodbye");

assertThat(sc.returnHello()).isEqualTo("Goodbye");

assertMock(() -> sc.returnHello()).wasCalledTwice();

What if it also did statics, with the same API, and no extra dependencies?

mockClasses(ClassWithStatic.class);

assertThat(ClassWithStatic.returnHello()).isNull();

when(() -> ClassWithStatic.returnHello()).thenReturn("Goodbye");

assertThat(ClassWithStatic.returnHello()).isEqualTo("Goodbye");

What if it even let you mock out constructors?

final RuntimeException rte = new RuntimeException("MARKER");

mockClasses(FinalClass.class);

when(() -> new FinalClass()).thenThrow(rte);

assertThatThrownBy(() -> new FinalClass()).isSameAs(rte);

What if we put all our arguments about whether we should be doing this kind of mocking to one side for now, because sometimes in life, you’re faced with a nail.

And when you’re faced with a nail, you need a hammer.

There’s one here: https://github.com/roscopeco/moxy

Or grab it with Maven:

<dependency>
	<groupId>com.roscopeco.moxy</groupId>
	<artifactId>moxy-core</artifactId>
	<version>0.90.1</version>
</dependency>

Dex vs Dee VM – A microbenchmark

During a recent discussion, I had the need to show some numbers demonstrating why Deelang needs the new DEX compiler. As I said in that thread, it’s all about removing reflection from the compiled classes.

We all know that “reflection is slow” (for some value of slow) and especially on Android, that it isn’t generally a good solution for general use in method invocation. Looking up methods by reflection is slow, as is performing the actual invoke. But how slow? Could I quantify the difference?

Since the Deelang DEX compiler is still being actively developed, I’ve never paid much attention to performance before, and have certainly never benchmarked it. We knew that the Dee VM wasn’t fast enough for some of the things we were using it for, and I knew from experience that it would be faster without reflection, and we left it at that. This discussion got me thinking though, and I decided it would be nice to have some idea of the actual difference between the Dee VM and scripts compiled to DEX.

Despite the fact that microbenchmarks are often a bad idea, and are a bit of a minefield at the best of times, I wrote one in this instance. After all, I only want some idea of the comparative performance for the sake of discussion – I won’t actually be basing any design or implementation decisions on these results.

I expected a decent performance boost in the DEX compiled scripts, but I honestly wasn’t quite prepared for how much of a boost:

09-27 10:45:27.525: I/BENCHMARK(15853): Rehearsal
09-27 10:45:37.665: I/BENCHMARK(15853): DeeVM completed : 10000 runs : 10107ms (10.107 seconds)
09-27 10:45:37.725: I/BENCHMARK(15853): Dex completed   : 10000 runs : 31ms (0.031 seconds)
09-27 10:45:37.725: I/BENCHMARK(15853): Real
09-27 10:45:46.695: I/BENCHMARK(15853): DeeVM completed : 10000 runs : 8938ms (8.938 seconds)
09-27 10:45:46.745: I/BENCHMARK(15853): Dex completed   : 10000 runs : 25ms (0.025 seconds)

(This is logcat from the benchmark running on-device on a HTC One X)

That’s right – almost 9 seconds for the reflection-based Dee VM, compared to 0.025 seconds for the DEX compiled version. The difference is so marked that I at first didn’t believe the results. I changed the script to actually produce some output to logcat, to make sure that it was actually running each implementation properly, and it was. It turns out that, in this case, reflection has a much bigger impact that I expected.

In case you’re interested, you can see the benchmark code at the this page on the project wiki on Google Code.

Deelang – First file release available to download

After a few months in Subversion, with a few folks playing with the code and seeing what use they can make of it, the first file release of Deelang is now available from the project’s downloads page. This release is still a very early public release (as denoted by the 0.18 version number) but is already mostly functional (with the notable exception of first-class regular expressions, which may never make it in anyway) and ready for testing.

Like most of our Android stuff, Deelang started life as internal code for a (closed source) third-party app, that ended up being deemed “possibly generally-useful enough” for open-source. So, at the moment, it mostly does the things we needed it to do when it was written. And, as with ORMDroid, the intention here isn’t to just dump the code into Google Code and forget about it – I intent to keep developing these things. What I really need is for folks to pick up the code, start using it, and either come back to me and tell me what cool new things they want to see it do, or (even better!) send over a patch with the aforementioned cool new things.

In case you’ve read this far in the hope of finding out what the heck Deelang is, the project page summary puts it thus:

Deelang is a lightweight, embeddable, dynamically-typed scripting language for Java that is intended for use in user-scriptable applications in limited resource environments, and especially for Android devices.

Basically it’s a very simple scripting environment that allows you to provide some manner of customization to your users, or to others deploying your apps. You define an API (it comes with almost none, beyond a basic object hierarchy for ‘primitives’) for code to run against, and Deelang provides a compiler, and a virtual machine in which to run scripts against this API. To quote (again) from the project site on google, it allows you/your users/someone else to write scripts like:

pic = Camera.take_picture()
pic.save_to_media()

Facebook.share(pic) {
caption = "I just took this picture!"
  album = "Random pics I took"
} or {
  post_status("took a pic but couldn't upload it :(")
}

(Obviously, assuming you provide the appropriate Camera and Facebook implementations). These scripts can then be compiled and run on the fly, or compiled and cached for later, perhaps to run in response to some broadcast intent or whatever.

So anyway, if you have a minute, grab the code and take a look, have a look at the (sparse) wiki, see if Deelang might be useful in your own projects, and if you think of a killer feature that you could really, really use, get in touch via the comments and we’ll talk…

ORMDroid on Google Code

Further to my previous post, I spent a bit of time today getting the ORMDroid project set up on Google Code (Edit: Moved to Github), including adding a bit of sample code to get you going with the library. I’m also working on extracting a simple sample project that will also go in the repository, which will serve as a proof of concept, a working example, and a functional testbed all in one.

Getting up and running with ORMDroid is really simple. Firstly, you need to add a single XML tag to your AndroidManifest.xml (it should be a child of the Application tag):

<meta-data
    android:name="ormdroid.database.name"
    android:value="your_database_name" />

Next, you need to initialize the framework somewhere. Application.onCreate() is the logical choice if you’re using a custom Application class (configured in your manifest – see Google’s JavaDocs if you want to go this route), but  you can happily call this from pretty much anywhere – even your activity’s onCreate() – since there’s actually no penalty for calling the initialize method multiple times.

ORMDroidApplication.initialize(someContext);

Side note: Currently, ORMDroidApplication is itself a subclass of android.app.Application, so you could go down the route of setting it directly as your application class, or subclassing it with your custom behaviour, and have the initialization handled transparently for you. However, this isn’t the recommended way, and it’s possible this facility will be removed in future versions.

With the framework initialized, you can start loading and saving your models! These are just plain Java POJOs that subclass the ORMDroid Entity class. If you’re happy with the default table and column names provided by the framework, these can be as simple as:

package com.example.myapp;

import com.roscopeco.ormdroid.Entity;

public class Person extends Entity {
  public int id;
  public String name;
  public String telephone;
}

Working with the Person class is now as simple as:

Person p = Entity.query(Person.class).where("id").eq("1").execute();
p.telephone = "555-1234";  // for example...
p.save();

And that’s it. No, really, it’s that simple. You don’t have to worry about setting up the database, ensuring tables are created, transaction management, or any of that other stuff. Of course, you can handle the database yourself if you really want to, but ORMDroid is more than happy to take care of it for you if you’ll let it.

The entity class in the example above will be stored in the following table:

CREATE TABLE comexamplemyappPerson (id INTEGER PRIMARY KEY AUTOINCREMENT, name VARCHAR, phone VARCHAR);

If you aren’t happy with this default mapping, you can use the Table and Column annotations to customize things, e.g:

package com.example.myapp;

import com.roscopeco.ormdroid.Entity;
import com.roscopeco.ormdroid.Table;
import com.roscopeco.ormdroid.Column;

@Table(name="people")
public class Person extends Entity {
  @Column(name="person_id", primaryKey=true)
  public int id;

  @Column(name="fullname")
  public String name;

  public String telephone;
}

A couple of things to note here are that the primaryKey parameter is actually redundant, since id would be used as a primary key in the absence of another field that has primaryKey=true, and that, and that you don’t have to annotate all the columns – only those you want to change the default mapping for.

And that’s it. You’re up and running with ORMDroid! Obviously this example code is very simple, and misses out probably the most important point with using an ORM framework – relationships. You can play with this yourself though – create another entity class, and give it a Person field. By default, whenever you persist your new class the related Person will be persisted as well. One-to-one relationships are already supported, and it’s easy to add helper methods to support many relationships using the query facilities provided by Entity. We’ll take a detailed look at relationships at a later date (this post is already long enough), but for now take a look at the code and I guarantee you’ll pick it up in no time anyway.