More hot emulator action

After seeing a random Facebook comment from a friend, I spent quite a few hours this weekend working on my NES emulator to get the game “Low G Man” working. Turns out the issue was related to the fact that an NMI (Non-maskable interrupt) can occur mid-cycle, and that particular game waits around for the VBLANK (which is what generates the NMI) to occur (other games do this as well). Unfortunately, because my emulator doesn’t really support intra-instruction events (the emulator is single-threaded), the loop waiting for the VBLANK would never see it.

Essentially what I think was happening was

; wait for VBLANK
LDA $2002
BNE ...
(let PPU run... VBLANK occurs... causing NMI)
(NMI interrupt handler runs... which reads $2002, clearing it)
LDA $2002; this now doesn't see the VBLANK because the NMI handler cleared it
BNE ...
LDA $2002;
BNE ...

I added in a modification that the PPU (or anyone) could tell the CPU “hey, this NMI actually occurred a litte later than you think”, allowing for one extra instruction to execute. Which, in this case, is enough to get it out of the infinite loop.

So, effectively it now does

LDA $2002
BNE ...
(notify PPU to do work, generating VBLANK + NMI)
LDA $2002; see the VBLANK
(NMI handler runs, but the LDA has already had a chance to run)
BNE ...
; yay, outside loop!

I had previously done a hack of setting the “sign” flag (which is what the BNE is actually looking for, because it is that highest bit) in the CPU NMI callback, based on a recommendation of something I found. But this felt like a horrible hack. I’m not certain that the new way isn’t a hack as well, but I think it is at least more “realistic”.

On another note, I’ve put the emulator into github:
https://github.com/jdewald/qjnes

I couldn’t think of a name (JNES is already taken), so I just did Quay’s Java NES emulator (qjnes). Note that this is still technically both a C64 as well as an NES emulator.

If you do happen to download it, you can fire it up with ant:
ant nes -Drom=

While you can play a lot of games with it (Super Mario Brothers 3 plays well, Kirby’s Dreamland played well last time I check)… there is no sound and many glitches (it is remarkably how ‘perfect’ some games require the system to be). This is not the emulator you want to get if you actually want it for “fun” (Nestopia and the likes are for that). It was purely so I could “see if I could”.

It doesn’t do anything cool like bytecode manipulation and JIT compiling… because it doesn’t need to on a modern computer. I finally added code to sleep, because on my Macbook PRO it runs well over full speed, so I now try to keep it around 60fps.

Holidays and a New Year

It’s been a while…

I’m ending a nice long 3-or-so week extended Christmas (just me, the wife, and pictures of gifts from family), New Year’s (very cool dinner on a ship permanently docked on the Thames. Those who know me won’t be surprised to know that I spilled red wine all over the table within about 2 minutes of sitting down. The 9 others at the table were quite nice about it) and 5-day Moroccan holiday in Marrakesh (مراكش). The last was quite cool (finally something different from Europe, you can only handle so many castles) but hectic and wearing at times (I can only handle so much haggling.. even though it’s satisfying to only pay 50% of the original price, I know I’m still paying way more than the price a local would pay). Again those who know me will not be surprised to know that I dropped (and shattered) the large tagine that we had purchased… was about 10 feet from our boarding gate at the airport.

And to really but an end to the holiday, my wife is now on a plane back to the States to get us going for our repatriation there. I will be following 3 weeks later, as my year-long stint here in the UK is ending. I have have had an awesome time here, both at work in out and about. Met some great people who I will definitely miss.

And now for something completely different..

To bring things back around to geeky stuff (I tend to skim over other people’s personal stuff, so I understand if you, Reader, have done the same) I have finally started working on my Super Nintendo (SNES) emulator. It is still using the same basic core as the C64 and NES emulators. Main difference is that the SNES using a 65816, a successor to the 6502 which can address 16MB of memory (24 bits) and implements all of the 256 op codes and adds some more addressing modes. When it initially starts up, it is actually in 6502 emulation mode (with bugs that the original 6502 had fixed, which I’m sure provided frustration to many developers who depended on undocumented instructions and bugs). I have gotten some basic graphics to work in the ‘test.smc’ demo included in Y0shi’s documentation, but it is nowhere near even able to get screenshots, but hopefully only a week or so (I’ve spent a feverish 3 or 4 days dedicated to SNES stuff, but probably spent another couple of weeks previously working toward getting an Apple IIGS emulator working, which uses the same processor) to get there.

I have started adding some JUnit tests of the actual instruction implementations, as even minor bugs can truly spiral out of control in the running of the system.

As usual, Zophar’s domain has proved invaluable for obtaining documentation, but I have also used Wikipedia for system overview (memory sizes and the like) and another site I stumbled on just called “Emu-Docs”

I will make the code available via CVS or Subversion once it is in a usable state. Apparently my wife never really played the SNES, so we shall see if I can find anything to drive me like getting SMB3 working on the NES did.  I would love to get Super Mario Kart working.

I have been using ZSNES as my “reference” for what a working system should look like (I don’t know if it’s open source or not, but I am only using the executable in any case).

Shoutout goes to the many people who dedicated hours and hours dissecting and analyzing the SNES and writing emulators in C and Assembly which ran on old Pentiums. My Java emulator may one day run at a full speed on a 2 Gig machine 🙂

Combination Java NES/C64 Emulator source release

I finally bit the bullet and “packaged” up my Java-based NES and Commodore 64 Emulator.

The delay has been for multiple reasons:

• Laziness – I knew that I couldn’t just tar it up and send it out, but needed to do some cleanup, take out some files, etc. I had some time on my hands, so figured why not.
• Fear – I don’t think it’s just me, but it’s kind of scary to let other people see the code you’ve written when you know that it’s not perfect and poetic and all that. However, it does in fact work so I figured that was a plus. I don’t believe the code is monumentally horendous either. There is a diagram of of the architecture available on my wiki.
• Potential Uselessness – There are a lot of other emulators out there at have been in development for a lot longer and are open source. I did this purely out of curiosity to see if I could, so don’t download this expecting to pop in any ROM and go with it. Some work, many don’t. There’s also no sound. Nestopia is the emulator I used when I wanted to see how a game was supposed to look/work. I don’t know if it’s open source or not however.

I’ve finally overcome those and just decided to get it out there.

I’ve decided on the BSD software license. What this means is that you can take this and basically do whatever you want with it, but if you release something based on it, you have to include my license with it. I initially planned on going with GPL, but it felt too restrictive in this case. In my dream world, people will download this, fix bugs and contribute them back. That would also be highly unexpected.

It is my hope that the architecture is simple enough that a person could at least use this to see how they might go about creating an emulator (especially 6502/6510-based ones such as the Apple II). It would suck if it serves as a way to _not_ make an emulator, but I can accept that too. It’s fairly object oriented in structure, but in my testing (I modified it so that the instructions were executed as methods rather than objects) this did not prove to be a major performance issue. That said, it just barely runs at full speed on my machine and it doesn’t have any timing code in it. So clearly there are some inefficiencies in it.

As I’ve mentioned previously, the greatest moment of development (which also represented the end for the most part) was when my wife actually sat and played Super Mario Brothers 3 for 4 hours straight on it. The idea that kept me going after I had it “working” was that for her, SMB3 was what it meant to have a Nintendo. Unfortunately, at that time I didn’t have the gamepad adapter so she had to use the keyboard. By the time I got game controller stuff working, she and I were basically over it. Such is life.

Anyhow, you can download it here.

It is also available on github now: https://github.com/jdewald/qjnes

Note that to use it easily, you should have Ant installed, but it’s not required. Please put any comments here if you do download it and have questions about usage, why it sucks, why it’s cool, or literally anything at all.

NES Documentation

I’ve started to wikify (not that it existed in the first place) some documentation for the NES Emulator. I promise I’ll post the code, but I’d rather not shame myself so I’d prefer to clean it up a bit 🙂 However, if you’re interested just pop me some mail to nes@40two.org (bring on the SPAM…)

In other news, my wife bought me a Garmin eTrex Venture GPS system that I can’t wait to play with when we vacation in Europe this July with her family.

A great past few days

Great Weather

First off the weather here in my neck of England has been amazing. Haven’t really needed any warm gear and there’s been absolutely no rain. Of course as I write this it’s a bit cloudy but, no matter, it’s been amazing. Rachel and I made good use of this weather on Saturday and had a great time walking around Notting Hill. I’ve never seen so many antique stands/shops in my life. Naturally we hit the first book store we found, Notting Hill Book and Comic Exchange on Pembridge Road. Ended up getting about 20 books between us for the equivalent of $2 a book (some only 10p!). In my haste I didn’t notice that one of the Len Deighton books I’d grabbed didn’t have the second half of the book contained within it. Such is life.

Super Mario Brothers 3 

On Saturday and Sunday I did some more work on my Java-based NES emulator and on Saturday managed to get Super Mario Brothers 3 and Zelda both working in fully playable states. I think this is something of a Holy Grail for me and in fact when I started this in January, my wife said all she wanted to play was her favorite game (Super Mario Brothers 3) so that has kept the drive. She had beat it already on the real system and still loved it. So it made me enormously proud when she asked excitedly if she could play it (she had seen that it looked decent). Quickly into playing as reality hit, we both wanted to have a tad more lives (how about infinite?) and to use an actual controller. Had to spend an hour or so figuring out how the Game Genie worked (more on that in a bit). Yesterday she was able to turn that on and played for 4 hours and got through most of World 3. Will have to use the GG again to get her back there (SMB3 doesn’t have save game feature and I haven’t implemented state dumping, doh!). Have also ordered an adapter for my PS2 controllers so we can hook it up, that turned out to be cheaper than getting a modified NES controller (although that would have been more authentic, but ah well). Rachel has apparently never played the NES version of Legend of Zelda, so she’ll be trying that next.

Thanks

I fixed a lot of bugs in getting SMB3 to look and play decently. I had about halfway understood the documentation around rendering 8×16 sprites but managed to combine the PPU documentation from Everything2 (btw that’s one of the greatest sites ever, easy to spend hours on) in the 2c02 Technical Reference. My predecessors have spent an amazing amount of time accurately documenting things on the real NES/Famicom and have produced awesome documentation for it. Most larger NES cartridges use circuitry-based “mappers” which handle the requests for Program ROM and Character/Pattern ROM to serve out the correct data based on internal state. So a note for people writing NES emulators is don’t make the assumption that the cartridge (or whatever your representation of it) is necessarily read-only (the Character ROM is) because writes can be performed to the registers of the mapper. They will look like writes to the program address space (0x8000 – 0xFFFF) because that’s all the NES’s CPU knows about. Anyhow, Super Mario Brothers 3 uses the MMC3 mapper and there a few documents out there but the one I found the most useful was Kevin Horton’s MMC3 guide. It looks like he has physical ones that he tests for their behavior. I actually learned that I completely wasn’t even thinking about or emulating the built-in timer of the 2A03 (The NES’s modified 6502). Thankfully SMB3 doesn’t seem to need it, as it uses raster counting from the mapper intercepting read calls from the 2c02 PPU (Picture Processing Unit). I’ve got loads of other documentation but I think those were the main ones for getting that game working (as well as tons of other things). Kevin Horton has documented some of the other mappers as well (I used his MMC1 guide for Zelda). I almost forgot, throughout I’ve used Nestopia as my “reference” emulator for how things should look and behave. It’s an open source project, but I never looked at the code.

Game Genie

 As I’ve mentioned before, I may be dense but when looking for documentation on the Game Genie I wanted something that actually said “this is how it works.” All the documentation was really around how to translate the Game Genie codes into memory address + value + check value. So what I ended up doing is loading the Game Genie as a normal game and handing it the game’s ROM. Once a write of 0x00 occurred to address 0 (0x8000 to the CPU) then I put the genie in “passthrough” mode and reset the CPU. Now any reads that occurred would be checked against what was stored in the codes and if they didn’t match it would just pass through. My holdup was that I was somehow thinking that the ROM would actually do this (but of course that’s impossible since it’s being emulated and is just a bunch of instructions to the CPU and so won’t be able to make “real” reads to the cartridge on it’s own). Once it clicked that I had to do the final step of actually checking the addresses, it all came together. I guess when I just plug in my own Game Genie it just works and so I expected this too as well 🙂

End

I still need to get sound working. Didn’t ever do that on the C64 but I want to do that on this one definitely. After that I will probably move on to another system. Writing hardware emulators is really fun, but I actually don’t care too much about playing with them. However I do have that option now and I’m sure I will take it up. When I started my Commodore 64 Emulator I had really never dealt with machine architecture except in some basic classes at University. Perhaps surprisingly, it will still take me quite a while to write anything in Assembly (at least I think so, I haven’t tried or needed to). When you are the machine you get to be “dumb” and only worry about things at the level of each instruction and store some state in registers. It’s actually up to the programmer to be “smart” and make use of those registers and instructions in a smart way, especially on the older 8-bit machines. So I really have to hand it to those who can crank out beautiful and complex programs written in 6502 Assembler. My job is actually fairly easy here, it’s all been documented by people before me and each part does one thing. I’ve learned a lot more about sprite rendering and double buffering and “interface-based” OO development than I ever really had to before. It’s been a great experience. You put it all together and you have…

 

Update on NES Emulator

I went 2 or 3 weeks without doing much on the emulator, but the time I have spent has been modestly productive. It’s that type of productivity though where I’ve spent most of the time within single methods fixing bugs and optimizing. I have the Name/Attribute Table mirroring working mostly. I literally swapped a value and suddenly Mario began looking significantly better. There were actually quite a bit of other code changes (which allowed screen scrolling to work properly) but I had the mirroring wrong which made the wrong sprites show up most of the time.

Oddly, “fixing” the mirroring made it so that Metal Gear doesn’t work properly, so I need to look into that. I believe that I’ve done Vertical mirroring incorrectly, but I actually suspect some other logic that determines what memory address to look for objects at.

I’ve fully redone the way that the internal rendering works. Using the documentation available, I’ve made it much closer to how the real PPU functions in loading the addresses from the latches at the start of screen rendering while also allowing some bits to be modified *during* rendering. This is actually what makes scrolling work. The downside is that it makes the logic a bit harder to understand because previously I did a more human-readable version of the logic but it was also more naive. I’d like to find a compromise between “here’s what’s happening” while allowing for mid-rendering writes.

Anyhow, here are some new screenshots:

Again, there are definite flaws in it (what’s up with the green shading in the middle?) but I think it’s a significant improvement from the previous version. I still haven’t fixed the “bottom pixel” issue ont he sprites which is why there are lines running through them. That’s also why the bottom is cut off from the top status bar (which I only got working yesterday by properly implementing the “Sprite 0” detection which I previously had a completely naive version of). I also did a quick fix to get the transparent pixels working.

I’m not sure when I should move onto sound… I’m getting a little anxious but feel that I should probably get Mario actually playing! For some reason it won’t actually start. Bot Pacman and Metal Gear will start once I hit the START key, but I cannot get Super Mario Brothers to leave the demo mode. Very strange.

I played a full game of Othello last night against the computer, I think I beat it but it was a close game!

As before, want to do a shoutout. There is a good compendium of available documentation that’s been made into a nice color PDF available at from parodius written by Patrick Diskin. I think they also have emulators that actually work 🙂 In addition to NES-specific docs, it has a great introduction to how the 6502 functions which probably would ahve been useful when I was writing the core.

Working (sort of) NES Emulator

Well, I spent all day hacking on Sunday and managed to get some graphics to display on the NES emulator. As I hoped, I was able to use the same architecture as the C64 Emulator and actually am using the exact same processor implementation. The ADC and SBC instructions have slightly different modifications (the version of the 6502 used in the NES does not make use of the Decimal flag for BCD mode), but that’s *it*.

I had a business trip and hacked on the plane and manged to get Pacman to become playable (but look horrible).

I was having some major palette issues. Using the tried and true bug-fixing technique of…stepping away…I was able to track down why the graphics were totally jacked (had some incorrect handling of the Attribute tables as they relate to the tiles displayed on the screen). So now Pacman looks *almost* correct. At least I thought so until I made the screenshot. There is a lot of work to do to get the sprites to draw correctly. Thankfully the background is at least correct.

My “end goal” is to get it so that my wife can play Super Mario Brothers 3. I’m hoping to implement the mapping for it soon. Currently Super Mario Brothers (1) will play the starting demo (but has all sorts of artifacts on the screen including the fact that I’m not handling transparent pixels).

So, as you can tell, a lot of work left… but I’m pleased with it as a proof of concept and a definite work in progress. I’d like to also implement the audio for this one (which can hopefully be reused for the C64).

Update (11/30/2007): Here are screenshots from the more recent version of the emulator (these are actually from April) of which I have released the source. 

Testing Emulator Architecture

Even though my Java-based Commodore 64 emulator isn’t yet complete, I’ve decided to start on an NES emulator. One of my reasons is that I’d like to test out my design for how the emulator is structured to see how reusable the components are. Technically the only shared component is the MOS 6502/6510 that acts as the main processor. In addition to that same physical component, the two will share the way that memory reads and interrupts are handled.

My hope is that coming at the system from a different angle will help me work out kinks in the design in addition to any inaccuracies in the emulation at the low level. Not all of the instructions in my 6502 having correct timing, and I believe that could relate to why some of the games have odd behavior. The actual implementations (aside from the BCD code in the SBC instruction) are all correct and pass the tests in the excellent C64 Emulator Test suite developed by Wolfgang Lorenz as part of his PC64 emulator.

Eventually I would like to be able to “build” virtual machines a property file and some form of IoC (Inversion of Control) with Spring or the likes to wire everything together. As it stands, most components are fairly independent from each other and only know about some interfaces. Without doing any modifications to the core really (only added support for NMI), I’ve already managed to read in an NES ROM and start executing instructions. The graphics aren’t implemented yet, so there it’s not doing anything obvious, but we shall see.

This page also serves as a marker for me to know when I started it and track some progress.

I want to put some thanks right now to all of those out there (much of which can be found through NesDev) who have created excellent documentation.