System Of Levers

Solving problems no one else seems to be having

GameBoy Twitter Client Proof-of-Concept

I didn't have an interesting demo of my WiFi GameBoy cartridge. I decided that it would be fun to use it to like @diconx's tweet about his WiFi cartridge and tweet a video of it at him!

Of course up to this point I've just been sending handfuls of bytes between the GameBoy and ESP so that I could figure out the hardware and basic communication protocol. Everything seemed to be working reliably though and I figured there was probably a Twitter API library for the ESP8266. How hard could it be? Actually not that hard but I made some silly mistakes and banged my head against the Twitter API a bit.

What Is It

A proof-of-concept Twitter Client for the GameBoy. It's able to load a tweet and render it on the GameBoy. A user can then like that tweet by pressing right and the A-button.

There's no way to change what tweet is viewed. The tweet ID is hardcoded but the tweet content is loaded using the Twitter API. There's no way to unlike the tweet, or to show if the tweet is currently liked. You also can't write a tweet. Also the auth tokens are hardcoded so it can only make me like the tweet!

How Does It Work

There are three parts:

  1. A web service that talks to the Twitter api and formats the tweet text for easier rendering on the GameBoy.
  2. An ESP8266 that talks to the web service.
  3. A GameBoy ROM that renders the tweet and handles user input.

You can find the code for it here.

Web Service

In principle this isn't needed. The ESP could be made to talk directly to the Twitter API. I was trying to get this done quickly though, and I was having trouble working out the right authentication steps so I figured doing it in python with a mature library would be easier. It means that the ESP code can be a lot simpler and I was able to do some pre-processing of the tweet text to make it much simpler for the GameBoy to render.

Talking to Twitter

Twitter has an API! To use it you'll need to setup a new app at https://apps.twitter.com/app/new. Then you'll be able to get credentials that you can use to talk to twitter. If you're not familiar with web API authentication I wrote a brief overview covering some of the basics. You can also check out the twitter docs.

To talk to the Twitter API from my web service I used the Tweepy library. I'm using the Twitter V2 API so I used the tweepy.Client class. You can follow their authentication tutorial to get the right tokens to have the service act on your behalf. You'll have to do the OAuth dance to get them. Using the consumer_key and consumer_secret that you get from the Twitter dev portal for your app.

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
print(auth.get_authorization_url())
# Visit that URL and take tone of the oauth token and verifier code. Fill them
# in in the following lines.

auth.request_token = { 'oauth_token' : "[TOKEN]",
                       'oauth_token_secret' : "[VERIFIER]" }
auth.get_access_token("[VERIFIER]")
print(auth.access_token)
print(auth.access_token_secret)

Save the access_token and access_token_secret and fill those in in the web service code. Now the web service will be able to access twitter on your behalf.

Formatting Tweets

Since I was already using python to fetch the tweet I decided to also do some pre-processing of the tweet to make it easier for the GameBoy to display it. The web service does three things:

  1. Make all lines exactly 20 characters, padding with spaces if needed.
  2. Make all lines start and end with a space to give a blank border on either side when displayed on the GameBoy.
  3. Avoid having a line break in the middle of a word when possible.

This makes the GameBoy code to display the tweet much simpler.

ESP Code

The ESP is a Wemos D1 mini programmed with Arduino. It communicates with the GameBoy using a pair of shift registers, which are normally connected to the ESP's SPI bus but I had them wired up differently while experimenting and using Arduino's shiftIn() and shiftOut() functions. I hadn't realized that at first and spent a bunch of time debugging why nothing was working! A few more details about this hardware setup can be found in my first post.

The ESP takes care of fetching the tweet for the hardcoded tweet-ID, sending it to the GameBoy, and waiting for input to like the tweet. This is managed with a sequence of states:

  1. kConnecting
    1. Send the byte value 255 to the GameBoy (the actual value is ignored).
  2. kConnected
    1. Wait for the GameBoy to read the sent byte (this is just to synchronize with the GameBoy).
  3. kGettingTweet
    1. Request the tweet from the web service.
  4. kSendingToGB
    1. Send the tweet to the GameBoy 1 character byte at a time (assumes ASCII).
    2. When done, send the value 0 to indicate the end.
  5. kWaitingForLike
    1. Wait until the GameBoy sends a byte.
    2. When it does, send like request to the web service.
    3. When that completes send a byte to the GameBoy to indicate it's done.
  6. kDone
    1. do nothing.

Apparently I never used kConnecting, oops! Basically this state is entirely handled by the setup function, and once it's out of there it's assumed to be connected.

GameBoy Code

The GameBoy code is written in assembly. Due to the limited nature of the proof of concept it fairly straight forward. It basically runs in lockstep with the esp code. It also displays some text so I know what step of the initialization it's on. There are two things that help keep the GameBoy and ESP in sync. The GameBoy can read a "status-bit" from the least-significant bit at address $B000. This is how the ESP tells the GameBoy it's ready. The other way is for the GameBoy to read from or write to $A000, which I call the ESP register, or rESP. The ESP receives an interrupt for a read from or write to that address (separate ones for read/write).

The sequence is as follows:

  1. Do some display related initialization.
  2. Copy over the string "Connecting..".
  3. Wait for the ESP to indicate it's ready, while blinking the last "." in "Connecting..".
  4. Display "Connected!!" for a few frames.
  5. Ready a byte from the ESP to tell it to continue.
  6. Display "Loading.." with animated ".".
  7. Wait for the ESP to indicate it's ready.
  8. Copy the tweet from the ESP and display it.
  9. Display a little bar at the bottom with an empty heart in heart.
  10. Wait for the the right-button to be pressed.
  11. Display an arrow icon next to the heart.
  12. Wait for the A-button to be pressed.
  13. Send the byte for the ASCII character "L" to the GameBoy.
  14. Wait for the ESP to indicate it's ready, which the GameBoy assumes means it finished liking the tweet.
  15. Read a byte from the ESP to acknowledge.
  16. Fill in the heart icon to indicate the tweet was liked.
  17. Loop forever.

The text (and any graphics) can only be updated when the GameBoy's display hardware isn't accessing video RAM (VRAM). You can find more details about this in the Pan Docs' rendering section. The section on the LCD status register has a good overview of what parts of VRAM are accessible when and the section on the pixel FIFO gives details about timing.

Since I didn't need precise screen updates I did all my updates during VBlank, which is a chunk of time after the GameBoy has finished drawing an entire frame. The GameBoy has an interrupt to tell you when VBlank is. It also has a halt command that will stop code execution until an interrupt occurs. So VBlank interrupt handler just returns immediately and I have halts whenever I need to wait for a VBlank. There are other types of interrupts but I only have the VBlank one turned on.

For displaying the Tweet, rather than worry about fitting it all in VBlank I just turn off the the display. Only do this during VBlank see here for details. Once the display is off I can copy everything over and turn it on.