Saturday, December 29, 2007

Hot Chips Conference archives

Curious about the internal designs of GPUs, CPUs, and game consoles? Tired of lame articles full of uninformed speculation and fanboy rants? Then check out the archives of the "Hot Chips" conference, an annual conference where computer chip designers get together to brag about their latest chips. The conference presentation slides are all online, and they're full of good technical information on GPUs, CPUs. and even game consoles. Of course, presenters often gloss over any technical problems with their chips, so you won't get the full picture. But these presentations offer a detailed technical look inside otherwise secret system architectures.

(For what it's worth the web site is poorly organized, and many links are broken -- you sometimes have to edit the URLs slightly to find the correct links.)

Some highlights:

Reality Co-Processor, Ken Hayes (Silicon Graphics, Inc.)
- All about the Nintendo 64.

Gekko: A PowerPC compatible processor supporting high-performance 3D Graphics - Gamecube CPU

Multiple Cell Papers (PS3)


Xbox 360 System Architecture

Nintendo Wii security defeated by the Tweezer Attack

According to a presentation at the 24th Chaos Communication Congress, hackers have apparently been able to defeat the Nintendo Wii game console's security system using tweezers to bypass the hardware memory protection.

The way it works is that the Wii runs in two modes: a GameCube emulation mode, which has access to just 1/8th of the total memory, and Wii mode, that has access to all the memory.

Hackers had already figured out how to run their own code in GameCube mode. So the trick was to run their code in GameCube mode, then use the tweezers to short out the address lines to allow the hacker's code to access parts of rest of the memory. By shorting different address lines different portions of memory were made available. By collecting enough shards they eventually mapped all of memory.

Apparently the Wii operating system keeps its digital signature keys in this protected memory, and once the digital signatures were found it was possible to sign and run homebrew code on the Wii.

It is not clear to me whether the attack is a per-machine attack or a break-once-run-everywhere attack.

Searching for old web pages: The Wayback machine is cool!

I was searching for information on LINJ, a Lisp language that compiles into human-readable Java code. Unfortunately, the LINJ web site http://www.evaluator.pt/linj.html, is currently offline.

Luckily, it turned out that the Internet Archive Wayback machine had cached both that page, and the download files that that page had pointed to. Very cool!

Similarly, I was looking for the source to the Windows CE port of Quake 3, and found that the project's web site http://www.noctemware.com/q3ce.html had been abandoned and taken over by spammers. Luckily the Wayback machine had cached both the original web page and the downloads.

Let this be a lesson to you aspiring open source developers out there: It's better to store small open-source projects in a large "won't-ever-go-away" source repository like SourceForge or Google Code than to use your own vanity domain hosting. Of course, even using a large popular repository is not failure-proof. Some large code repositories from the early days of the Internet, like DEC's ftp site, have gone away after their owning company was bought by another company. Perhaps some sort of distributed system of discoverable git repositories is the answer.

In the case of Quake 3 for Windows CE, I've contacted the author, Christien Rioux, and with his kind permission I've set up a code.google.com project so that other people can more easily find the sources (and binaries): http://code.google.com/p/q3ce .

Thursday, December 27, 2007

Why the Farmers won

I've been thinking about Lisp lately. A powerful language, with many excellent features, but not hugely successful. And I thought of one reason why:

The excellent book Guns, Germs and Steel hypothesizes that agriculture displaced hunting and gathering because people who practiced agriculture stayed in one place, and were able to have one child per year, as opposed to the hunter-gatherers, who had to wait until their children were old enough to walk before having another child. This reproductive rate difference was amplified by the ability for a given unit of land to support more farmers than hunters. As a result, agriculture displaced hunting even though individual hunters were far healthier (as seen by their skeleton height) than the farmers that displaced them.

So it is possible for a poorer technology to displace a better one, if it has compensating advantages. And I think that's what's hit Lisp. C and Java, which are each less powerful and more wordy than Lisp, are more successful for reasons other than power and brevity. Perhaps because both languages allow lots of reusable code to be written by ordinary programmers. Maybe Lispers are like healthy hunters, being displaced by hordes of sickly farmers.

Well, no doubt the Lispers will take some comfort in the Java and C programmers being displaced in turn by whatever language next becomes even more successful, just as hunter-gatherers may take pleasure in the trend that farmers are gradually being displaced by urban dwellers. (Come to think of it, Lispers have already seen their original competitor Fortran displaced by C/C++, and in turn much of C++ has been displaced by Java and C#.)

I think something similar is happening in productivity applications, as a generation of not-very-good-but-web-based productivity applications is displacing the Microsoft Office suite. (For example, I am typing this blog entry into a simple and ugly HTML-based web form rather than a beautiful Word document.)

Sunday, December 23, 2007

Back to Mac

This week I converted my family's main computer from Vista to OS X. (It's a Mac Mini).

We pretty much use it for web surfing, web email, really old DOS children's games, and photo editing. Macs do that pretty well.

I still have a Vista machine that I use for the excellent Windows Media Center -- love the record-by-keyword feature and the free programming guide.

But for day-to-day use we're back to the Mac.

Alternative language blues

I've spent roughly 4 years of midnight-engineering time looking into the cool languages to see if they would make game programming easier or more fun. Haskell, Ocaml, F#, Erlang, Scheme, Lisp, D, Factor, Scala, Python, I've looked at them all.

F# held my attention for quite a while, but now my platform-of-choice has moved away from F#'s design center. (I'm into Linux-based mobile platforms now.) And to be honest, I'm still happier in a C-like language.

I'm depressed. Sure, I learned a lot about fancy language features, but I could have written quite a few games in plain-old-C++ (or C#, or Java or Flash or Basic) in the same time.

P.S. Someone else has done this more impressively than I have. Do a Google Groups search for Brandon van Every, who has had a five year odyssey to find the perfect non-C++ game programming language. I corresponded with him back when we were both interested in O'Caml. Since then he's managed to annoy pretty much everyone by harping on their favorite language's shortcomings. In the end (at least as of six months ago) he'd given up and gone back to C++. I look forward to seeing what he does next.

Lisp Hacking and Science Fiction

I've been poking around with Lisp and Scheme again, and am reminded of some of my favorite science fiction books (warning, plot spoilers follow):

  • Verner Vinge's "A Fire Upon the Deep" begins with a group of scientists mining an ancient civilization's web archives. They need to build interpreters for the ancient civilization's programs. All goes well until they reconstitute a malevolent AI that they spend the rest of the book fighting.
  • Piers Anthony's Macroscope involves a group of people trying to decode an Extra-Terestrial message, that other ETs are trying to jam. Over the course of the book your opinion as to which group of ETs has humanity's best interests at heart changes back and forth several times.
  • Any number of SF stories are set in the far future where people poke around in the ruins of a once-great civilization. (See Gene Wolfe, Cordwainer Smith.)

Working with Lisp reminds me of these books. Lisp's a seductive, ancient, powerful language that has been worked on for years by very smart, very motivated hackers. Pretty much everything one can think of to do with Lisp has been done, multiple times, by really smart people.

For example, I want to have a system where I can interactively write a game, changing code on the fly while the game is running. I want to be able to use macros and garbage collection and free serialization and inspection and all that cool stuff that Lisp provides.

And Naughty Dog had all that, in GOAL, for the PS2. Their implementation was apparently much better (more efficient, less buggy, more features) than one I could cobble together out of open-source-Lisp parts. And they had used it to successfully write two or three games. But they they walked away from it. Gave it up. Went back to C++.

They said it was because they had a lot of pressure from their parent company to make their engine more reusable. But I think it's also because the results they were getting just weren't that much better than the results that all the other developers, who use C++-and-some-cheap-scripting-language-like-Lua, were getting.

I think Lisp is like a powerful alien technology that may not be in your best interests to use.

Tuesday, December 4, 2007

More evidence that garbage collection is expensive

As seen on Lambda the Ultimate.org

Quantifying the Performance of Garbage Collection vs. Explicit Memory Management

We compare explicit memory management to both copying and non-copying garbage collectors across a range of benchmarks, and include real (non-simulated) runs that validate our results. These results quantify the time-space tradeoff of garbage collection: with five times as much memory, an Appel-style generational garbage collector with a non-copying mature space matches the performance of explicit memory management. With only three times as much memory, it runs on average 17% slower than explicit memory management. However, with only twice as much memory, garbage collection degrades performance by nearly 70%. When physical memory is scarce, paging causes garbage collection to run an order of magnitude slower than explicit memory management.


The Economics of Selling a MacBook on Craig's List vs. Ebay

I just sold my MacBook over the web. I sold it on Craig's List. I also considered using eBay, but Craig's List turned out to be a better deal, for both buyer and seller.

The reason is that listing and selling on Craig's List is free, while selling on eBay is expensive, especially for items priced higher than $500. First, there's the listing fee of $0.20, then the Final Value fee of $0.50 + 3.25% * price. But wait, there's more: EBay all-but-requires you to use their PayPal service to settle transactions, and PayPal in turn requires you to use a "Premier" account if you receive more than $500 in eBay payments in one month, which you automatically would if the item you're selling is more than $500. Using a "Premier" account requires that you pay PayPal 2.9%+$0.30 per transaction, even for cash transactions. (The processing fee is less if you are doing a high volume of business through them.)

So the total eBay selling cost is in the range of 6.15%. That's $50 on a $800 item.

You could avoid accepting PayPal, but since accepting PayPal is the norm on eBay, it's very likely that your auction will be shunned, and you will receive a lower price.

Besides the hefty fees, using PayPal is riskier for the seller than using cash. This is because it is possible for a disgruntled buyer to reverse the transaction. (Of course this same ability is a plus for the buyer, as is the ability to use credit cards.)

Even without using eBay to sell the product, it makes sense to use eBay as a price setter. eBay makes this easy to do by reporting the final price for closed auctions that you are "watching". Since auctions last between 3 and 10 days, and an auction must be active in order for you to watch it, you will need at least a week to track enough auctions of similar items to make a fairly accurate estimate of the market price.