Thursday, December 24, 2015

2015 Year in Review

Here's my take on tech trends in 2015 and predictions for 2016.

Personal trends in 2015.
  • I started using Twitter, following a mix of optimistic tech bloggers, economists and comic-book artists. Always something interesting to read. I don't tweet much. (Nothing to say :-P)
  • Podcasts. I'm following a bunch of tech, gamer, and comedy podcasts. I especially like The Voicemail, Accidental Tech Podcast, Melton, and Guys we F*cked (NSFW) .
  • I switched to a large-sized iPhone 6s+. The big screen is great.
  • I stopped maintaining my "Terminal Emulator for Android" program, because I lost interest in the idea of an on-device terminal emulator for Android. (And I lost interest in maintaining the project in the face of frequent Android UX and build system churn.)

Open Source

I've been doing less open-source software work than in previous years. My OSS work has been driven by emotion and "hack value". This year I haven't come up with any ideas that were exciting enough to work on. Partly this is because we're in the midst of a change from PCs to mobile, and it's not clear to me what needs to be done in the new mobile-first world. And partly it's because things are working pretty well. I feel that I have my basic computing needs taken care of by existing apps.

More than once this year, I came up with an idea for a project, only to find a perfectly serviceable implementation already available for free or only a few dollars. Each time I installed the existing app rather than writing my own version.

Video Games

I found myself playing fewer video games. I bought lots of mobile games, but mostly for my family rather than for myself. My wife briefly held the world record high score for the puzzle game Spl-t.

I'm still waiting for The Witness and The Last Guardian to ship. I may buy a PS4 to play TLG. Or I may just watch the inevitable "Let's Play" walkthroughs. It seems like the kind of game that would be almost as much fun to watch someone else play as to play myself.


I bought an Apple TV 4th Gen and an Apple iPad Pro. I'm using both primarily for media consumption, although once I obtain an Apple Pencil I hope to use the iPad Pro for some sketching.

I had hoped to write games for the Apple TV, but the bundled controller is too limited to support interesting games. And the development model is clunky, requiring either two Apple TV units, or a long cable. I think it makes more sense to concentrate on iPhone/iPad apps than Apple TV apps.

Computer Languages

I'm studying Swift, trying to decide if it's good or not. It's a positive sign that Apple open sourced it. I like the "Playground" feature.

I wish I could use Go more, but I don't currently have a project for which Go is suitable.

Similarly, I'm impressed by recent developments in Clojurescript. I wish I had a project idea for which Clojurescript was suitable.

2016 Trends

  • Mobile
  • VR
  • Machine Learning

Family IT Information, end-of-year edition

Just an update on my family IT use.

The T-Mobile family plan has worked great for us. T-Mobile's plans are nice for us because:
  • The third, fourth, and fifth lines are only $10 / month.
  • When the paid-for data is exhausted, the plans automatically switch over to unlimited free low-speed data for the rest of the month.
  • Streaming music doesn't count against the data caps.
  • Free phone calls, texts, and low-speed data in Canada and Taiwan. (It was great using Google Maps to get around Vancouver. I was using many short cuts that I didn't know about when I was navigating using paper maps.)
  • For the last 3 months of 2015 T-Mobile had a special where they gave everyone unlimited high-speed data for free.
I ended up getting used iPhone 5s's for all my kids. I had the kids pick their own otterbox commuter cases. 

My son was initially frustrated at having to give up his rooted and customized Android phone for the smaller, less customizable iPhone. He's grown used to it, and likes it now. Everyone loves the fingerprint unlocking feature of the 5s.

I have the phones set up with restrictions, so that the kids can't install their own apps. I also confiscate-and-recharge their phones and laptops each night. This is fairly foolproof, and gives the kids 8 hours a day to sleep without electronic distractions.

I bought an Anker 6-port USB charger for my bedside table. I use it to recharge everyone's phones while keeping an eye on them. I have the phones on "Do Not Disturb" mode, so they don't bother me over night.

The "Find my Friends" app has proved helpful for keeping track of where everyone is, especially for things like picking kids up at school and at bus stops.

We now have 5 laptops: four 13" Macbook Airs and one 13" Macbook Pro. They work great and last a long time. They are mostly used for web surfing, YouTube and Minecraft.

We have had problems with headphones -- the kids are rough on headphone cables. They've already gone through one set of headphones each. Currently we're using Beats headphones due to them being relatively cheap on sale and/or included in Apple Educational bundles. They look stylish and work OK. Apple has a fairly good warranty repair process for their Beats headphones.

Sunday, July 26, 2015

Happiest recent purchases

A few products that made my summer family travels happier:

Anker 4-port USB car cigaret lighter charger.

Car vent phone holder (don't remember the brand.)

Short stereo audio cable. It's ghetto compared to bluetooth, but more reliable.

Family Computers, 2015 edition

As summer draws to a close, I am planning my family's computer use for the 2015-2016 school year.

My plans for this year are:
  • Each family member gets their own mobile phone and laptop.
  • We also have a shared desktop and a few shared tablets.
  • Shared network scanner/laser printer.
  • We print color documents and pictures at the library or at the drugstore.
  • Android TV for shared movie watching.
  • Chromecast for audio sharing.
  • Comcast Business Internet
  • Apple Time Capsule for backup / WiFi / NAT
Note the lack of a dedicated game console. The kids play games on mobile, tablets, and laptops (for Minecraft specifically!).

My two youngest kids are getting their first phones this year. I'm worried about their phones getting lost, broken, or stolen. So for the first six months my kids will use hand-me-down phones. After they've learned to take care of phones, I'm going to give them better phones. (Still used, though!)

To achieve my plan I need to buy one laptop and three mobile phones. For the laptop I'm leaning towards a 13" Retina Macbook Pro. For the phones I'm leaning towards used iPhone 5Ss.

Why a Macbook and not a Chromebook? Build quality and applications. I have a low-end ARM Chromebook, and I noticed that nobody in the family uses it by choice, due to its low speed and poor quality screen. I _could_ get a Chromebook Pixel, but for my family, at that price level a Macbook is a better deal.

Why iOS and not Android? It comes down to ease-of-administration. I want to lock down my kids' phones, and unfortunately experience with my oldest child using Android is that it's all-too-easy for him to defeat the aftermarket Android parental control apps.

For phone service I'm probably going to go with the BYOD T-Mobile Family Plan, because:
  • It is cheap.
  • Unmetered music streaming.
  • When you hit your data cap it switches to low speed data for the rest of the month, rather than charging more.
  • It has free 2G international roaming.

Thoughts on laptops and other legacy hardware

If I were on a tighter budget, or starting from scratch, I'd consider dropping the laptops, the Comcast Internet, and the home WiFi, and going pure mobile. I would get bluetooth keyboards to make typing school assignments easier.

There's (Already) an App for That

Story of my hobby hacking life these days:

  1. Think of an idea for a small application to write to learn a new technology and incidentally make my life better.
  2. Prototype the app.
  3. Plan a MVP, estimate costs in time and money to develop.
  4. Search Play Market and/or IOS App Store, find that reasonable equivalent already exists, and is only $2.
  5. Buy the existing app, get on with life.
This happened to me last week with the concept of a "comic book reader". I wrote a prototype that let me browse my collection. I was starting to list out all the features I needed to add (zooming, panning, sorting,RAR archive support...). And then I did a web search for comic book reader, spent a couple of minutes reading reviews, and bought one of the popular ones for $2. Sure it's got UI issues, and bugs, and doesn't quite work like it should, but I saved myself weeks of development time.

I need to think through how best to spend my hacking time in today's world of super abundance. What's my comparative advantage in this new world? What's my compliment? What am I trying to learn, trying to achieve? What is worth working on? Existential questions on a Sunday morning. :-)

Pixar Non-Commercial Renderman for OS X

Pixar released their Non Commercial version of Renderman. Woo! Hurray for patent expiration dates!

If you don't have Maya and you do have Mac and you just want to play with the command-line version, you have to jump through several hoops:

You need to install XQuartz before trying to install Renderman. (Otherwise the Renderman installer will fail.)

You need to copy /Applications/Pixar/RenderManProServer-19.0/etc/rendermn.ini to ~/.rendermn.ini  (Note the added "." in the front.)

You need to edit .rendermn.ini to add the line

    /licenseserver    ${RMANTREE}/../pixar.license

You need to add these lines to your bashrc (or equivalent, depending on your shell.) 

export RMANTREE=/Applications/Pixar/RenderManProServer-19.0/
export DYLD_LIBRARY_PATH=/Applications/Pixar/RenderManProServer-19.0/lib/
export RMANFB=it
export export

Once you've done that, you can use the "prman" command line tool to render rib files.

Wednesday, April 15, 2015

An Update on "Terminal Emulator for Android" development

I am shutting down "Terminal Emulator for Android" development again.

Why am I doing this?

  • I have lost interest in the core idea of an on-device terminal emulator.
  • Maintaining project, even in its mostly stable state, is taking up too much of my time.
  • I do not want to give control of the app to other developers, for fear that they will ruin the app by adding bugs, ads, in-app purchases, or malware.

What this means:

  • I will make one or two more releases based on the current source tree. (Which has a few small bug fixes.)
  • I will be closing all open bugs as "won't fix".
  • I will be rejecting all future pull requests.

What you should do:

  • If you're a user, the app will continue to be available in its current state.
  • If you're a developer, you are welcome to fork the app to start your own version. Maybe get together with other developers and make something great!

Friday, March 27, 2015

Crash Bandicoot Dev on rendering techniques

As seen on Hacker News:

Maybe Andy forgot to mention it; it's been a while since I've read the whole series.
The code was C and lisp so it didn't really require any effort to port other than replacing the rendering pipeline. And we used the SGIs to pre-render every frame anyway, to precompute the polygon sort order. (The PS1 had no Z-buffer, so you were stuck sorting polygons at run-time if you didn't do something clever.)
So we already had the rendering pipeline ported. Obviously you couldn't save your game to the memory card, etc. -- some stuff didn't work. But the game was playable (albeit very frustrating with keyboard controls).

That blows my mind, I always assumed it did have a z buffer. So what did you guys do to remedy 'z fighting' triangles? My interest is; I wrote a 3d renderer in Java using fillPolygon() many years ago, and used polygon sorting in place of a z buffer. Z fighting was of course a problem.

Some day I will write this up for real, but without going into detail, here's a summary.
The camera in Crash was on a rail. It could rotate left, right, up, and down (in Crash 2 and beyond, at least), but could not translate except by moving forward/backward on the rail. This motivates a key insight: if you're only rotating the camera, the sort order of the polygons in the scene cannot change.
This allowed us to sample points on the rail and render the frame at each sample point ahead of time, as a batch job, on the SGI using a Z-buffer. (We may have done the Z-buffer with software; I don't remember.) Then we could recover the polygon order of each frame by looking at the Z-buffer. And, even better, at run-time we could simply not render at all those polygons that weren't ultimately visible in the pre-rendered scene. This solved both the sorting and clipping problem nicely, and made the look of the game closer to 3K polygons/frame vs. the 1K polygons we were actually rendering in real time. (Many polygons were occluded by other polygons.)
The trick, though, was what exactly to do with this sort/occlusion information. In a nutshell, what I did was write a custom delta-compression algorithm tailored to the purpose of maintaining the sorted polygon list from frame to frame, in R3000 assembly language. Miraculously, this ended up being quite feasible because the delta between frames was in practice very small -- a hundred bytes or so was typical. And if a transition was too heavyweight (i.e., the delta was too big) we'd either sample more finely in that area or tell the artists to take stuff out. :)
One thing nobody talks about but which is obvious in retrospect is that without a Z-buffer you're pretty screwed: sorting polygons is not O(N lg N) -- it's O(N^2). This is because polygons don't obey the transitivity property, because you can have cyclic overlap. (I.e., A > B and B > C does not imply A > C). This is why virtually every game from that era has flickery polygons -- they were using bucket sorting, which has the advantage of being linear time complexity, but the disadvantage of being wrong, and producing this flickery effect as polygons jump from bucket to bucket between frames.
I'll leave the matter of weaving the foreground characters -- Crash himself and the other creatures -- into the pre-sorted background for another day.

Thursday, February 12, 2015

Run a Minecraft Server on OSX Boot2Docker

Run a Minecraft Server on OSX Boot2Docker

Here's how I did it. I hope you find it useful!

One-time Setup

Do these steps once, to initialize Boot2Docker:

Step 1: Install Docker for OS X

Step 2: Create a directory to hold your Minecraft files. This needs to be under the /Users part of your file system because boot2docker automatically mounts /Users to the boot2docker-vm.

  mkdir /Users/yourname/minecraft/data

Step 3: Initialize boot2docker

  boot2docker init

Step 4: Forward the TCP port Minecraft uses from the Mac to the boot2docker-vm.

  VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port25565,tcp,,25565,,25565";

Start Minecraft

Do these steps every time you want to start your Minecraft server.

Step 1: Start boot2docker.

 boot2docker start

Step 2: Set up the shell variables so you can use the docker command.

 $(boot2docker shellinit)

Step 3: Run the minecraft container.

 CONTAINER=$(docker run -v /Users/yourname/minecraft/data:/data -d -e EULA=TRUE -e VERSION=LATEST -p 25565:25565 itzg/minecraft-server)

The first time your run this it will take a few minutes to download and install minecraft. After that it should be much faster

View the Minecraft Server Log

  docker logs $CONTAINER

This prints out the logs from the container (you set the CONTAINER variable as part of the docker run command above.)

If you've lost track of your container, you can list all currently running containers.

  docker ps

If you don't see any containers, you container may have already exited. The Minecraft server will exit if it encounters an error while running.

Shut Down

You can shut down all running containers and quit boot2Docker by using the stop command:

  boot2docker stop

Note that the Minecraft Server files will be stored in /Users/yourname/minecraft/data, and when you've stopped the server you can edit the files using your mac. (You might want to edit the files in order to modify the server settings.)

Sunday, February 8, 2015

"Masters of Doom"

I've been reading the book Masters of Doom about the careers of John Carmack and John Romero. I have ported their games Doom and Quake to many different platforms. It was interesting to read about their lives and game business.

The book brought back memories of development in the '80's, and '90s. Things were simpler (and worse) then.

Some other John & John links:

The Rise and Fall of Ion Storm
Stormy Weather

Carmack QuakeCon Keynotes

Sunday, January 11, 2015

Reverse engineering my own game

I've long-since misplaced the source code to my Atari 800 game Dandy Dungeon. But thanks to the Atari800MacX emulator and the emulation scene, I've been able to play an emulated version of my original game. That's been helpful for remembering all the little details of gameplay.

For example, I was able to determine that the original game animated the arrows at 15 Hz and the players and monsters at 7.5 Hz.

FWIW I think the emulator may be slightly incorrect about the HBLANK processing emulation. I'm pretty sure that the color background for the 4th line of text should be a different color from the color background of the 3rd line of text.

The iOS version of the game is progressing -- the dual thumbstick virtual controls work well.

The next step (and it's a big one) is going to be multiplayer support. GameKit here I come.

Sunday, January 4, 2015

Fun with shaders

There are a total of 3 draw calls and 2 textures in this scene:

The whole tile map is rendered as a single draw call: a single 2-triangle tile that's instanced Row x Column times, using a 3D texture as a texture atlas. I originally used point sprites, but switched to instanced triangles because I wanted to use non-square tiles.

The virtual joystick is rendered as two coarse triangle strip rings, using a 1D radial texture. Note the anti aliasing. (I could have used quads, but wanted to minimize overdraw.)

So far Metal has been fairly straightforward to use, at least for someone like me coming from a DirectX 9 / Xbox 360 / Android OpenGL ES 2.0 background.

Friday, January 2, 2015

3*(N+1) Devices for N People

At my house we are trending towards having N+1 laptops for N people, because (a) I need to keep my work laptop separate from my home laptop, and (b) frequently everyone in the family wants to use their laptops at the same time.

The same goes for tablets, and when the kids are old enough to have phones I expect it will be the same for phones.

I tried using multi-user accounts on shared family tablets and laptops, but ended up assigning each kid their own devices. It was simpler from an account management point of view, and the kids like personalizing their devices with stickers and cases.

Having assigned devices also makes it easier to give different Internet and gaming privileges to different kids, depending on age and maturity.

A downside of assigned devices is that not all the devices have the same features. People complain about hand-me-down devices, as well as the perverse incentive created when an accidentally broken device is replaced by a brand new, better device.

Letting go of the Web and Embracing Mobile

When I started working on Android in 2007, I had never owned a mobile phone. When Andy Rubin heard this, he looked at me, grinned, and said "man, you're on the wrong project!"

But actually, being late to mobile worked out well. In the early days of Android the daily build was rough. Our Sooner and G1 prototypes often wouldn't work reliably as phones, and that drove the other Android developers crazy. But since I was not yet relying on a mobile phone, it didn't bother me much.

Seven years later, mobile's eaten the world. But I still haven't internalized what that means. I think I'm still too personal-computer-centric in my thinking and my planning.

Here's some recent changes that I'm still trying to come to grips with:

  • Android and iOS are the important client operating systems. The web is now a legacy system.
  • Containerized Linux is the important server operating system. Everything else is legacy.
  • OS X is the important programmer's desktop OS (because it's required for iOS development, and adequate for Android and containerized Linux development.)
  • The phone is the most important form factor, with tablet in second place.
  • Media has moved from local storage to streaming.
  • Programming cultural discussion has moved from blogs & mailing lists to Hacker News, Reddit & Twitter. (To be fair, these new forums mostly link back to blog posts for the actual content.)

In reaction, I've stopped working on the following projects:

  • Terminal Emulator for Android. When I started this project, all Android devices had hardware keyboards. But those days are long gone. And unfortunately for most people there isn't a compelling use case for an on-the-device terminal emulator. The compelling command-line use cases for mobile are SSH-ing from the mobile device to another machine, and adb-ing into the Android device from a desktop.
  • BitTorrent clients. My clients were written just for fun, to learn how to use the Golang and node.js networking libraries. With the fun/learning task accomplished, and with BitTorrent usage in decline, there isn't much point in working on these clients. (Plus I didn't like dealing with bug reports related to sketchy torrent sites.)
  • New languages. For the platforms I'm interested in, the practical languages are C/C++, Java, Objective C, and Swift. (And Golang for server-side work.)
    • I spent much of the past seven years experimenting with dynamic languages, but a year of using Python and JavaScript in production was discouraging. The brevity was great, but the loss of control was not.
Personal Projects for 2015

First, I'm going to port my ancient game Dandy to mobile. It needs a lot of work to "work" on mobile, but it's a simple enough game that the port should be possible to do on a hobby time budget. I'm probably going to go closed-source on this project, but I may blog the progress, because the process of writing down my thoughts should be helpful.

After that, we'll see how it goes!

Game Programming Patterns Book

I've been reading Game Programming Patterns by Bob Nystrom.

It's available to read online for free, as well as for purchase in a variety of formats.

A good book for people who are writing a video game engine. I found myself agreeing with pretty much everything in this book.

Note - this book is about internal software design. It's not about game design, or graphics, physics, audio, input, monetization strategies, etc. So you won't be able to write a hit video game after reading this book. But if you happen to be writing an engine for a video game, this book will help you write a better one.

Edit -- and I've finished reading it. It was a quick read, but a good one. I consider myself an intermediate level game developer. I've written a few simple games and I've worked on several other games. (For example, I've ported Quake to many different computers over the years.)

For me the most educational chapters were Game Loop and Component, although Bytecode and Data Locality were also quite interesting.

I like that the chapters have links to relevant external documents for further research.

I felt smarter after reading this book.

What I was up to 2012-2014

It's been a while since my last post -- I've been posting inside the Google internal ecosystem, but haven't posted much publicly.

What have I been up to in the past 3 years?


Prototyped a Dart runtime for Android. Amusingly enough, this involved almost no Dart code. It was 90% Python coding (wrangling Gyp build system scripts) and 10% C++ coding (calling the Dart VM).

Extended the Audio players for Google Play Music's Web client. I learned ActionScript, the Closure dialect of JavaScript, and HTML5 Audio APIs (Web Audio and EME.)

Started working on the Google Play Music iOS client.  I learned Objective C, Swift, iOS and Sqlite.

Personal Projects

Prototyped a Go language runtime for Android. Unpublished, but luckily the Go team is picking up the slack.

Finished working on Terminal Emulator for Android. I'm keeping it on life support, but no new features.

Personal Life

Started exercising again after a 10 year hiatus. It's good to get back into shape.

Switched to a low cholesterol diet. Google's cafes make this pretty easy to do.

Watched my kids grow!