rosco_m68k: August Update

Time for another update on the rosco_m68k, my homebrew retro computer project. As always, regular updates can be found on hackaday, so please do check there for the details (and like/follow if you want to!) – this will be more of a digest of what’s been going on over the past couple months.

The Headline

So I guess the headline thing that’s been happening recently is that I’ve started working toward having a PCB manufactured for this thing. I realised I needed to press on with this after I took the breadboard in to work for a geek-out session with some of my awesome colleagues, and had a totally nerve-wracking commute with the breadboard (it survived, save for one current-limiting resistor which developed a loose connection).

So I started work on reducing the component count (more on that later) and actually designing a PCB. It’s now pretty-much ready to go off to OSHPark for manufacture (Yes, I could get it cheaper but I like purple and, y’know, yield) and looks like this:

(The real one will be purple…)

So that’s the hot-of-the-press headline (I finished it today), and I’ll update more once the boards are made. In the meantime, if you want more details, follow the project on Hackaday for (usually) weekly updates.

Lowering the component count?

In the original design of this computer, I went for a full 7400-series TTL design for all of the glue logic, and that was cool. I learnt a lot about designing with discreet logic gates, was forced to really think about the way I wanted my circuit to work, came to understand propagation delay in a way I didn’t before, and bought a lot more breadboard. That last point is important – fifteen 7400-series chips, while cool, take up a lot of physical space. And PCB fabs charge by the square inch.

It had been pointed out to me before (thanks, Ken!) that I could do with a couple of programmable logic devices what I was previously doing with ~fifteen discreet logic ICs, so I decided (in the interests of staying sane and solvent) to investigate. After a good bit of reading and learning, I’ve now replaced most of the 7400-series on the board with two ATF16V8QL (link to PDF) CPLDs.

This is a win in more than one respect – not only has it saved me board space but it’s made routing the PCB much easier, cut down the current draw of the board, fixed up some propagation delay issues and allowed me to (finally) fix the bug with the expansion select line. These chips were available at the time the CPU I’m using was released, are still in production today, and cost me around £1 each (delivered, based on 5-off, from Farnell in the UK). They’ve taken over all address decoding and IO glue logic, and I highly recommend them.

What About Software?

Things haven’t stood still on the software front – since my last blog I’ve fully integrated the MC68901, gotten timers and interrupts working, and made the UART work such that the computer can now talk over serial (via a CP2102 Serial-to-USB converter I had lying around). Work on an OS for this thing continues apace, and it already has some basic memory management, a fully async serial driver, and has been demonstrated with some (very) basic multitasking. My immediate plan is to implement a way to load software via serial (Zmodem) in the immediate term to make it so that, as I continue to develop the kernel, I don’t have to physically pull the ROMs to reprogram them.

It’s Going Well, Then?

I like to think it is, yeah. I’m having tons of fun, and this thing is starting to look like a product. Once the PCBs are made and I’ve done some more on the software, I think this thing will be at a point where other people might want one. Obviously it’s a 16/32-bit computer in a 64-bit era, but I kinda feel like it might make a great starting point for people wanting to really get to grips with bare-metal, without all the lies abstractions that modern architectures force on them.

The PCB design gets me to a point where the core computer is done, and the expansion connector I put in means I can continue to develop new bits without worrying about disturbing the ever-so-fragile connections on the existing breadboard. This will mean graphics, networking, and who knows what else.

And even if it never goes anywhere beyond the three prototypes I’m going to be building soon, I’ve learnt a hell of a lot from this project, and continue to do so every time I work on it – and that’s really all that matters, right?

As always, the rosco_m68k is open hardware/open source. The complete set of design, documentation and code can be found on Githib at https://github.com/roscopeco/rosco_m68k.

Advertisements

Running Code and Talking to the Outside World

I’m still having a ton of fun with the rosco_m68k single board computer, and regularly writing project logs on Hackaday. If you’re interested, be sure to follow the project there as that’s where the regular updates happen, but here’s an in-brief update on what’s been happening.

Firstly, the board now has a lot more wires:

The ROM and RAM are both now hooked up to the buses, along with a MC68901 Multi-Function Peripheral that handles IO.

The system is successfully running code from the ROMs, and the RAM is working fine. I have code that does a full write/read test of the on-board RAM, and other code that just uses it and both are fine.

There’s some new glue logic to support the MC68901 (the top board in the picture) which has upped the chip-count a bit, and I’ve completely redone power distribution which was starting to get a bit shaky due to the way the project has grown.

Everything is connected together via the buses in the center of the picture, and the control bus has grown to accomodate all the CPU control lines and the address decoder outputs.

I’ve had a bit of a nightmare this weekend trying to get the MFP working properly (See this and this on Hackaday if you fancy a laugh) but it’s finally working, and I’m almost at the point of sending “Hello, World” to a serial terminal. After that, I’ll be off to the races with this project – debugging is so much easier when you can actually output stuff rather than just run logic traces.

That’s all for now – as I say if you want to know more be sure to like and follow on Hackaday!

Retro Computer Update

I’m writing regular logs on hackaday.io about the m68k-based retro computer I mentioned I was building a while back, but I thought I’d post a quick update here too. If you’re interested in the project, be sure to follow it on hackaday as that’s where the regular updates are going.

The main progress so far has been:

  • Get the CPU free-running
  • Decide on a memory layout, and design an address decoder to support that layout
  • Build the address decoder
  • Decide how I want to lay the buses out on the board for easier interconnection
  • Actually wire up the buses accordingly

I actually designed the address decoder a while ago, but gathering the parts and then wiring it up has taken some time – I finished it yesterday. Since then, I’ve been breaking out the CPU signals onto control, data and address buses, which I’m doing on an otherwise-bare breadboard in the middle of the main boards.

Right now, the rosco_m68k looks like this:

It’s still free-running at the moment (I’ve not wired the memory to the bus yet as you can see) but all the control pins I care about (basically everything except the legacy 6800 peripheral pins and the function code) are now on the control bus. The jumpers you can see there are configuring those control pins for free-run mode and will be replaced by connections to the RAM and ROM.

I did come across a couple of design errors while building the address decoder, some of which I fixed and others I’ve decided I can live with for now (e.g. an issue with the expansion select line I only discovered once I’d completely built the decoder and hooked up the LEDs at bottom left).

Anyway, that’s where it’s at right now. I expect that after another day’s wiring, it’ll be ready to run real code, and I can’t wait.

I’m building a retro computer!

Somewhere around the time I got bored with Advent of Code 2018 (well, the challenges grew to become total time-sinks), I decided my project for the new year was going to be a homebrew retro computer.

I decided that, rather than something 8-bit based on the Z80 or 6502, I wanted to go up a generation and build something 16-bit. 

Apart from gaming and a bit of BASIC programming on 8-bit machines like the Spectrum and C64, my first “real” computing was done on the Amiga, a machine which still has a special place in my heart today. For that reason, I decided I wanted to build something based on the Motorola 68000 series processor. 

I chose the 68010 which (as far as I’m aware) was never used in the Amiga line, but which is slightly more capable than the 68000 (e.g. interrupt vectors can be moved around and it can support virtual memory when used with a 68451 MMU, though I’m not likely to do the latter in this build). Mine is rated for 10MHz, but I’m only running it at 4MHz at the moment).

The key differences between this and an 8-bit Z80 or 6502 build are:

  • Dynamic registers, so the clock cannot be stopped or single-stepped (this can be emulated by manipulating /DTACK though).
  • Asynchronous data bus and DMA support, with /BGR, /BGA, /BERR and /DTACK to take care of.
  • 16-bit data bus, so more wires needed.
  • 24-bit (with some caveats) address bus, so lots more wires needed, but 16MB of address space available!
  • Internally, the m68k family is 32-bit.
  • /UDS and /LDS lines for selecting upper and lower data bytes, so a more complicated address decoder and ROM/RAM in pairs.
  • No separate IO space, further complicating the address decoder.
  • More complicated requirements around /RESET and /HALT handling, especially at power-on.

I started gathering parts in January, and so far have the crystal clock circuit, the reset/halt circuit, and have gotten the CPU freerunning with /DTACK grounded and the various other bus management lines tied high or low as appropriate.

The address decoder is designed, and I already have the RAM and ROM chips ready to go. I’ve also built an EEPROM programmer because I’m too impatient/cheap to order one and wait for it to arrive.

All this means that, pretty soon, I should be able to get it running some actual code!

Fun times are ahead! If you’re interested, you can follow the project over on Hackaday.

It’s Alive!!!!!

AoC 2018 – Day 10 Retro

Today is day 10 of Advent of Code 2018, the fun daily coding challenge I mentioned I was taking part in this year, and I thought it a good time to take a quick look back at my story so far.

I’ve still not managed to rank globally for any of the challenges (the closest I came was on day 5 when I completed part two 696th) but things are going reasonably well on our private BJSS leaderboard, and the challenges have mostly been fun. So far I’ve stuck with Ruby as my language of choice, although I almost broke out Unity 2D for today’s graphical challenge (Early-morning me obviously had grand ideas which were quickly set aside when I realised I could zoom out my terminal and stick with Ruby).

Here’s a quick rundown (with possible mild spoilers) of the challenges to date. The links take you to my solutions on GitLab, which obviously do contain spoilers, so don’t click them if you haven’t already done the puzzle for that day.

  • Day 1 was a very simple problem involving summing a list of numbers, and finding the first duplicated number. Nothing much to say here, nice and quick.
  • Day 2 involved finding characters duplicated two- and three-times in a string, with part two focusing on finding strings that differed by one character. I went with hash-based solutions for this one.
  • Day 3 was about finding overlapping rectangles. I made a simulation for this as the maths escaped me. This was the first solution that was properly object-oriented Ruby, and I got stuck for a while because I accidentally transposed my X and Y coordinates in one of the methods!
  • Day 4 involved finding the guard who slept the most, and (in part two) the minute in which they were most often asleep. Another simulation (I quite enjoyed this one).
  • Day 5 was about reducing a string by removing pairs until none remained. Regex and recursion to the rescue here – a really quick solution.
  • Day 6 was, frankly, a nightmare. I couldn’t really visualise the problem at first, and it took me a lot longer than it should have to really get to grips with it. Solved in the end, but after many hours on-and-off. I did learn something new though 🙂
  • Day 7 involved a dependency graph and another simulation. I had a lot of fun with this, and even made an ANSI-based visual mode so you could watch the simulation unfold in the terminal. Great fun.
  • Day 8 was a simple tree problem involving both breadth-first and depth-first traversal. The depth-first part had a little twist that meant it took a little thinking about.
  • Day 9 caused me a few problems – partly due to some vague wording in the puzzle, and partly due to my not understanding the problem properly for a while. It was made worse by the fact that I initially decided to write it with the wrong data-structure because I felt it would be quicker. After a few false starts I gave up on that and finally made it work using circular linked lists.
  • Day 10 was another fun one involving points and velocities to work out when the points (stars) align to display a message. Initially I planned to switch to something with proper graphical support (in a fit of complete overkill, I fired up Unity) but settled in the end on a Ruby solution after I realised I could make the output fit in a terminal.

And that’s it so far! The most fun I’ve had so far was probably on day 7, while my least favourite has been day 6.

Taking part? Let me know in the comments what your favourite puzzle has been thus far.

Advent of Code 2018

This year I’m taking part in the awesome Advent of Code, and because I haven’t done a lot of Ruby over the past couple of years I thought this would be an excellent opportunity to refresh my skills.  In case you’re unfamiliar, AoC is a series of twenty-five coding challenges. Each day leading up to Christmas a new challenge is posted on the site, and there’s a global leaderboard. You can also have local leaderboards (We have one at BJSS for example).

I did originally consider using the opportunity to learn a new language, but the desire to get back to Ruby proved too strong. In truth, I’m glad it did – every time I come back to Ruby after a break, I’m pleasantly surprised by both the bits I’d forgotten and the bits that have been added.

Advent of Code 2018 is at https://adventofcode.com/2018 . In case you’re interested, I’m posting my solutions day-by-day on GitLab here: https://gitlab.com/rosco.peco/aoc-2018 (they’re only spoilers if you decide to look!)

There’s also a subreddit where you can follow the discussion and see some very cool solutions.

In short, if you haven’t already, you should check it out.

Using GCC as cross-compiler with x86_64 target

This post is a little rough, and is intended both as a memory aid for myself and as an example for anyone else who needs to know this stuff. I’ve recently decided to push toward x86_64 support in my toy/research OS project, and needed to build a GCC cross compiler for that target. In my case I already had an x86 cross compiler built, but I’ve tried to make the steps general for those that don’t.

These steps are tested with binutils 2.26.1 and GCC 6.1.0. I’m using those versions as they match the version of my x86 cross compiler – These steps might work with the more recent versions, but I can’t guarantee it. I will at some point update my cross compilers, at which point I’ll update these steps.

1. Go to your $HOME/src. Create first if needed.

# mkdir ~/src
# cd ~/src

2. Set up prefixes. PATH is only needed if not already set (e.g. this is your first cross compiler on this system).

# export PREFIX="$HOME/opt/cross"
# export TARGET=x86_64-elf
# export PATH="$PREFIX/bin:$PATH"

3. Configure, build and install binutils

# tar xf /path/to/downloaded/binutils-2.26.1.tar.gz
# mkdir build-binutils-x86_64
# cd build-binutils-x86_64
# ../binutils-2.26.1/configure --target=$TARGET --prefix="$PREFIX" --with-sysroot --disable-nls --disable-werror
# make && make install

4. Extract GCC

# cd ..
# tar xf /path/to/downloaded/gcc-6.1.0.tar.gz

Ensure libgcc is built with no redzone – Add this:

MULTILIB_OPTIONS += mno-red-zone
MULTILIB_DIRNAMES += no-red-zone

to the following (new) file:

gcc-6.1.0/gcc/config/i386/t-x86_64-elf

And then make sure GCC configure knows about the new config:

# vim gcc-6.1.0/gcc/config.gcc
/x86_64-\*-elf\*)

And insert:

tmake_file="${tmake_file} i386/t-x86_64-elf"

above the tm_file line.

5. Configure GCC

# mkdir build-gcc-x86_64
# cd build-gcc-x86_64
# ../gcc-x.y.z/configure --target=$TARGET --prefix="$PREFIX" --disable-nls --enable-languages=c,c++ --without-headers

6. Build GCC

# make all-gcc

Go make coffee/eat out at a restaurant/vacation in Barbados…

Note: If building with newer GCC (Found with 7.3.1), there’s an issue in the 6.1.0 source. If you see error:

error: ISO C++ forbids comparison between pointer and integer [-fpermissive]

relating to gcc/ubsan.c, then you’ll need to edit that file, and change line 1473 from:

|| xloc.file == '\0' || xloc.file[0] == '\xff'

to:

|| xloc.file[0] == '\0' || xloc.file[0] == '\xff'

7. Build libgcc

# make all-target-libgcc

8. Install GCC and libgcc

# make install-gcc
# make install-target-libgcc

9. Check the no-redzone libgcc is being used properly when -mno-red-zone is passed:

# x86_64-elf-gcc -mno-red-zone -print-libgcc-file-name

Should print (note the no-red-zone in the path):

$HOME/opt/cross/lib/gcc/x86_64-elf/6.1.0/no-red-zone/libgcc.a

10. Don’t forget to pass -mno-red-zone in LDFLAGS in the actual build!

Generate less bytecode with default methods

Java’s default methods (introduced back in Java 8) are one of those features that solve the intended problem reasonably well, while at the same time allowing all kinds of nasty code and weird inheritance stuff when used in a general-purpose kind of way. Indeed, they’ve been the subject of a ton of posts and the way they (fail to) work still surprises people. They’ve been around for a while now, and there’s a lot of good advice out there on how they should and shouldn’t be used, so that’s not what I’m going to talk about in this post.

What I am going to talk about, however, is the way I’m using them in Moxy, as a way to do a lot less work in runtime-generated bytecode.

A bit of background

Moxy is a mock framework, and it does it’s job by generating mock classes at runtime. These mock classes naturally use a fair bit of bytecode generation, replacing real method implementations with new methods that implement the mock behaviour.

These mock methods do the standard mock thing of recording their invocation, and then they implement behaviour that has been set-up on the mock beforehand.

The problem

In order to implement this behaviour, mock objects need access to all kinds of state that’s specific to the instance – stubbed return values, exceptions they should throw, and more. They also need to have easy access to the mock engine that created them (so they can record invocations, for example).

This is often implemented via static methods on some class somewhere whose job it is to store state. The generated code will typically be peppered with a ton of invokestatic instructions that calls out to methods on those classes in order to get things done.

This works, but it has a few problems:

  • Some kind of mapping between instances and their state must be maintained.
  • It can make it difficult to swap out mock strategies in a clean fashion.
  • It can make testing (of the framework) difficult.

The Moxy approach

Instead of storing state statically, Moxy stores it right there in the mock, and all interaction with the state is done with instance methods. These methods can’t be inherited from a superclass (since mocks usually subclass the mocked class), and I really don’t want to generate them (I like to keep generated code to a minimum, as it’s harder to maintain and test that regular Java), so instead Moxy abuses default methods.

It works like this:

  • All mocks implement an interface, ASMMockSupport.
  • This interface defines one abstract method, __moxy_asm_ivars(). This method is implemented in generated code (since interfaces don’t have instance variables, the mocks hold the variables and expose them via this method)*.
  • The support interfaces defines a ton of default methods that do various kinds of work based on those instance variables.
  • The actual generated code for mocked methods uses invokeinterface calls on this to get stuff done as needed.
  • Other parts of the framework that interact with the mocks (e.g. stubbers) simple cast the mock instance to ASMMockSupport and call the methods they need.

* Yes, that’s a strange name for a method. Moxy does this to lessen the chances of its built-in methods clashing with methods being mocked.

 

The default methods on the support interface handle things like finding the engine that created the mock, adding stubbing, finding appropriate stubbing for a given invocation, running delegate methods and actions, and determining whether the current thread is currently in what Moxy calls a monitored invocation (used for recording purposes only, when mock behaviour is disabled).

This has the benefit that a whole lot of fairly-complex code that would otherwise be either generated, or stuffed into static methods, is now implemented, in Java, as instance methods. They’re easy to unit-test, and they’re easy to maintain. The added bonus is the generated bytecode is also easier to maintain, since it’s as short as possible.

Is this the intended use of default methods? Not at all. Is it correct, or appropriate? I’ll leave that to you to decide. I firmly believe, however, that in this very limited use-case it’s a good solution to a tricky problem, all things considered.

If you’re interested in looking at the code, you’ll find it here.

Mocking v2.0

We all love to use mocks in our Java tests, right? Add the usual mock framework to your test dependencies, sprinkle a few mock, when and thenWhatever calls in to your code, and happy days. You’re able to test things in isolation, and as an added bonus ensure your code is actually calling the stuff it’s supposed to be calling by verifying the mock afterwards. The sun is shining, life is good, and your code is tested to perfection.

But then the clouds roll in, and you have to test a final class. Or you need to mock out a static method in some legacy code. Or, horror, some code that directly calls new on a class you want to mock.

You could hit up StackOverflow, where you’ll find that all these things are possible with existing frameworks, assuming you’re prepared to add in another dependency, use a ton of boilerplate, and refactor your (production) code a bit. Depending on which framework you’re currently using there are a ton of different ways you can kind of achieve what you need.

By now you’ve sunk another hour or two, and it’s still not quite working.

What if there was another way?

The other way

What if I told you there was a framework out there that let you take a final class, and just do this?

mockClasses(FinalClass.class);
FinalClass sc = new FinalClass();
assertThat(sc.returnHello()).isNull();
assertMock(() -> sc.returnHello()).wasCalledOnce();
when(() -> sc.returnHello()).thenReturn("Goodbye");
assertThat(sc.returnHello()).isEqualTo("Goodbye");
assertMock(() -> sc.returnHello()).wasCalledTwice();

What if it also did statics, with the same API, and no extra dependencies?

mockClasses(ClassWithStatic.class);
assertThat(ClassWithStatic.returnHello()).isNull();
when(() -> ClassWithStatic.returnHello()).thenReturn("Goodbye");
assertThat(ClassWithStatic.returnHello()).isEqualTo("Goodbye");

What if it even let you mock out constructors?

final RuntimeException rte = new RuntimeException("MARKER");
mockClasses(FinalClass.class);
when(() -> new FinalClass()).thenThrow(rte);
assertThatThrownBy(() -> new FinalClass()).isSameAs(rte);

What if we put all our arguments about whether we should be doing this kind of mocking to one side for now, because sometimes in life, you’re faced with a nail.

And when you’re faced with a nail, you need a hammer.

There’s one here: https://github.com/roscopeco/moxy

Or grab it with Maven (instructions on the Github page).