Embracing Chaos

Rhapsody Profiles FTW!

Excuse my newbie exuberance, but OMG Rhapsody.com finally launched profile pages!!!  They've been up for a while now, which makes me think they're for real this time.  A couple of you might remember that this feature was live for something like a week in early 2007.  But it was very slow and didn't live long.  Sniff.

I worked hard to make this feature possible when I was working at Real.  The fact that I couldn't get it re-launched was a big motivator for me to move on to greener pastures.  I saw making Rhapsody social as an important evolution of the music catalog's organizational schema.  It's also an attempt to bring the product into what Tim O'Reilly would call Web 2.0.  Tim's canonical essay is long-winded, but I really liked how he summarized it in a recent interview on NPR -- basically the product gets better as people use it.  The millions of people who use Rhapsody are an asset that has been almost completely unused, except to take their money.  I saw it as a way to take on one of the product's biggest shortcomings.

Rhapsody has tons of music.  TONS.  Rhapsody almost certainly has something you want to listen to right now, regardless of who you are or what your current mood or situation is.  It's a strong statement, but there really is that much music.  The problem is figuring out what you want to listen to.  Rhapsody has a great categorical index of music, so if you know you want to listen to D&B or Emo or Vocal Jazz, no problem.  Or if you know specifically the name of something you want to listen, just search for it.  Other than that, you can take the homepage recommendations, browse the catalog manually, or sift through Playlist Central, a dumping ground for unvetted playlists that is a case study in how not to use user-generated-content (UGC) on a website.

Picking good music is difficult.  This is what DJ's get paid for.  I originally wanted this feature to be called "DJ Pages."  The idea was to give a voice to the small fraction of Rhapsody users who are fanatical about the product.  People who are serious music buffs love Rhapsody, and if given a voice would and still might add tremendous value to the music catalog.  Right now the editorial voice in Rhapsody is controlled by a politburo of paid editors.  They're really good, but they're just a handful of hands.  DJ Pages would democratize the music editorial process so anybody with an opinion can contribute.  The social graph becomes the voting process to select who's worth paying attention to, just like with pagerank.  What Tim calls Web 2.0, I like to refer to the democratization of information.  Partly because it's fun to call people Communists when they cling to control of information, but mostly because the analogy is apt and helpful.

The Rhapsody team has made an important step in this direction of openness.  I hope they keep running with it.  If you want to see what's been playing on my Sonos at home, check out my profile page.  But most importantly, I'd like to express my CONGRATULATIONS to everybody who made this possible again and the first time!!!!11!!1

Moving from Typepad to Blogger

For a while now I've wanted to move this blog from Typepad to something else like Blogger.  I keep finding more reasons to do this as Blogger improves and Typepad stagnates.  Some reasons include:

• Better WYSIWYG editing in Blogger.  (I can't change any font characteristics in Typepad without going into raw HTML. Ugh.)
• Typepad's lack of support for Chrome, Google's awesome new browser
• The awkward and limiting way Typepad assigns human-readable URLs to posts
• Blogger is free, while Typepad costs >$100 per year
• Cool new features being added to Blogger like Followers
• Adding new features in Typepad is a p.i.t.a. (i.e. I'm too old to be messing with perl-based templating languages.  Did more than my share of that in 1996.)

Typepad has long supported an export feature, which dumps out all the posts, drafts, comments, etc into a text file.  And now blogger has an import feature, which takes an XML file that looks kinda like an Atom feed.  I looked around for a tool to convert between the two and found only others asking the same question. 

So I thought here's a chance for me to contribute something to society and help myself at the same time.  Converting one text file to another shouldn't be that hard.  So I rolled up my sleeves and started playing with it.  I realized one problem fairly quickly -- the typepad export format doesn't include the URL for each post.  I really don't want to break all the previous inbound links because that's how people get to my content.  But I'm going to need to crawl the old blog to get those old links.

Since the blogger format looks like a regular Atom feed, I thought I'd try to just grab the Atom feed off the blog and import it into Blogger.  Well to that, blogger said:

Blogger does not currently support import files generated by TypePad.

Fine.  But what are the differences?  The blogger format includes a bunch of fake "entry" elements which are really configuration -- like a giant "layout" node and a bunch of other settings.  So I tried grafting the two together -- just taking the legitimate "entry" nodes from the typepad feed and putting them into the blogger export that includes all the layout and settings.  By converting everything to look like the blogger format, it imports, but that also loses information.  So I set about looking for the minimal set of changes that will work.

One that definitely breaks it is a side-efect of being hooked-up to feedburner.  There's extra stuff in there like the feedburner:origLink tags:

    <feedburner:origLink>http://www.embracingchaos.com/2008/12/repairing-a-deg.html</feedburner:origLink>

These break the import, which shouldn't be much of a surprise since the feedburner: namespace isn't defined in the Blogger export.

I gave up looking for today because blogger was having too many server errors on import.  They're intermittent, so I was happy to refresh through them for a while.  But at some point 90% of my imports gave me funny error codes like bX-gxxw6w so I'm giving up for now. 

I wrote a stub of a python program to do the merge.  I called it typepad2blogger.py and happily donate it to the public domain.  It's definitely not done and is useful only insofar as it shows one way to approach the problem and has enough stuff in place that somebody else should be able to get a running start.  I'll hopefully continue work on this at some point.  But even a couple hours of work on it has triggered my RSI so I'll have to take a break.  If you want to pick this up, let me know and we can coordinate efforts.

RAID repair successful

For everybody who has been waiting with baited breath to hear whether or not the repair of the RAID array worked or not, it did. It took several days, but since we were away on vacation seeing my dad receive the Fleming Medal from the American Geophysical Union, the waiting was pretty easy.

To convince myself that the repair was successful, I unplugged one of the previously functional drives, and saw that all my files were still there when the array was running just on the new drive and the other previous drive. I recommend this to anybody who thinks they're running a RAID system -- until you've seen the RAID array work with a drive removed, how can you be sure it's really working? If your system is set up better than mine is, you'll get some kind of warning message too.

Repairing a degraded EVMS RAID 5 array

A while back, lightning scrambled one of the disks in my home RAID 5 array.  I figured out how to recover it.  And I got the critical data off.   Here I describe the steps I took to add a new drive and get it working with the old RAID array.  I share this with the net in hopes it will make it easier for somebody else who has to go through this process themselves, and selfishly as notes for me to refer to.  It's a testament to the power of EVMS and a warning to anybody who thinks it might be fun to run their own open-source software RAID server at home. 

My advice for people seeking reliable storage: go with a hosted solution.  Understanding the arcane nuances of these software systems is an extremely specific skill that doesn't translate well to many real-life necessities.  If you're smart, you can figure it out, but it doesn't teach you much of anything except how to do exactly that.  Each person who understands this stuff should be keeping petabytes of data happy, rather than one couple's pictures and music collections.  I hear Microsoft's "home server" actually makes this pretty easy, but I can't recommend anybody willingly lock themselves into Microsoft's business model.

Background

So I bought a new drive, following my own advice about picking drives from different manufacturers when building a raid array, and plugged it in to the mobo and booted the machine.  After futzing with /etc/fstab to get it to find the boot disk and load up, I logged into evms and got these messages:

MDRaid5RegMgr: RAID5 array md/md1 is miissing the member  with RAID index 0.  The array is running in degrade mode.

and

MDRaid5RegMgr: Region md/md1 is currently in degraded mode.  To bring it back to normal state, add 1 new spare device to replace the faulty or missing device.

Conceptually easy.  I've got a new 500 GB drive in the system.  Linux sees it.  It didn't take me too long to figure out it's called /dev/sda, while the previous 2 disks in the array are sdb and sdc, with a small boot drive at sdd.  Now the fun part is figuring out EVMS terminology enough to tell it to use the new disk.

The hierarchy of the array in EVMS land seems to be as follows:

• Logical Volume teraraid (contains)
• Region md/md1 (which contains)
• Segments sdb1 and sdc1 (which are built on)
• Logical disks sdb, sdc.

What I tried, and what seems to have worked

I see that logical disk sda has no segments.  So I try Action -> Create -> Segment.  It only gives me one choice for "Segment Manager" which is "GPT Segment Manager."  But when I choose it, it doesn't let me make a segment on sda.  Only the tiny free space on sdb and sdc.  So sda needs something else done to it before we can use it.  What?

sda also shows up in the list of Logical Volumes, next to Teraraid and the formatted boot partition.  Hmmm.

Well I tried converting it to an EVMS Volume.  It complained that sda does not have a File System Interface Module (FSIM) associated with it, but it made the new logical volume anyway.  This wasn't getting me anywhere.  So I erased it.

Next I tried "Add" -> "Segment Manager to Storage Object".  I noticed that all of the Disk Segments associated with the array were listed as using "Plug-in" "GptSegM" and this gave me the choice of adding Gpt Segment Manager to sda.  W00t.  I said "No" to make this a system disk.  This seems to be working.  Now I see a bunch of Disk Segments starting with sda, including a big one (465 GB) labelled sda_freespace1. 

Now when I tried to Create -> Segment, it let me use GPT Segment Manager on sda_freespace1 and allocate a 450 GB disk segment to match the others.  (I left 15 GB off each disk with the idea I could put a boot segment in that space, but I've never gotten around to it.)

Now in "Available Objects" there is sda1 with 450.0 ready for me.  Alrighty we're getting there.

Now I look at "Storage Regions" and in the context menu for md/md1 I see an option that says "Add spare to fix degraded array..."  I didn't see it there before -- it might have not shown up when there weren't any spares, or maybe I was just being thick.  In any case, selecting it now gives me a menu with one choice -- sda1. 

Now in details of md/md1 it shows:

 
 Na┌──────────────────── Detailed Information - md/md1 ─────────────────────┐
 ──│                                                                        │──
 lv│     Name                    Value                                      │
 lv│ ────────────────────────────────────────────────────────────────────── │
 lv│     Major Number            9                                          │
 md│     Minor Number            1                                          │
   │     Name                    md/md1                                     │
   │     State                   Discovered, Degraded, Active               │
   │     Personality             RAID5                                      │
   │ +   Working SuperBlock                                                 │
   │     Number of disks         3                                          │
   │ +   Disk 1                  sdb1                                       │
   │ +   Disk 2                  sdc1                                       │
   │     Number of stale disks   1                                          │
   │ +   Stale disk 0            sda1                                       │
   │                                                                        │
   │                                                                        │
   │                                                                        │
   │                                                                        │
   │                                                                        │
   │    Use spacebar on fields marked with "+" to view more information     │
   │                                                                        │
   │ [Help]                                                          [OK]   │
   │                                                                        │
   └────────────────────────────────────────────────────────────────────────┘

That last line about the Stale disk is new.

Actions -> Save commits these changes to disk.  Now looking at Detailed information for md/md1 shows

 Na┌──────────────────── Detailed Information - md/md1 ─────────────────────┐
 ──│                                                                        │──
 lv│     Name                    Value                                      │
 lv│ ────────────────────────────────────────────────────────────────────── │
 lv│     Major Number            9                                          │
 md│     Minor Number            1                                          │
   │     Name                    md/md1                                     │
   │     State                   Discovered, Degraded, Active, Syncing =  0 │
   │     Personality             RAID5                                      │
   │ +   Working SuperBlock                                                 │
   │     Number of disks         3                                          │
   │ +   Disk 1                  sdb1                                       │
   │ +   Disk 2                  sdc1                                       │
   │     Number of stale disks   1                                          │
   │ +   Stale disk 0            sda1                                       │
   │                                                                        │
   │                                                                        │
   │                                                                        │
   │                                                                        │
   │                                                                        │
   │    Use spacebar on fields marked with "+" to view more information     │
   │                                                                        │
   │ [Help]                                                          [OK]   │
   │                                                                        │
   └────────────────────────────────────────────────────────────────────────┘

Emotionally I feel like I should be done now.  But I don't hear the thrashing noise of a half-terabyte of of checksums being unwound and copied onto a fresh disk.  And it says "Syncing = 0".  Hmmm.

I quit evmsn and reload it to see two new messages.  One familiar:

MDRaid5RegMgr: Region md/md1 is currently in degraded mode.  To bring it
back to normal state, add 1 new spare device to replace the faulty or missing device.

And one novel:

MDRaid5RegMgr: RAID5 array md/md1 is missing the member  with RAID index 0.  The array is running in degrade mode.  The MD recovery process is running, please wait...

But this novel message saying it's recovering is "Number 0" implying that it came before the other message (Number 1) which tells me I need to take action for it to fix itself.  And the drives are not thrashing.  Again I look at the details for md/md1 and now I see:

 Na┌──────────────────── Detailed Information - md/md1 ─────────────────────┐
 ──│                                                                        │──
 lv│     Name                 Value                                         │
 lv│ ────────────────────────────────────────────────────────────────────── │
 lv│     Major Number         9                                             │
 md│     Minor Number         1                                             │
   │     Name                 md/md1                                        │
   │     State                Discovered, Degraded, Active, Syncing =  0.3% │
   │     Personality          RAID5                                         │
   │ +   Working SuperBlock                                                 │
   │     Number of disks      3                                             │
   │ +   Disk 1               sdb1                                          │
   │ +   Disk 2               sdc1                                          │
   │ +   Disk 3               sda1                                          │
   │                                                                        │
   │                                                                        │
   │                                                                        │
   │                                                                        │
   │                                                                        │
   │                                                                        │
   │    Use spacebar on fields marked with "+" to view more information     │
   │                                                                        │
   │ [Help]                                                          [OK]   │
   │                                                                        │
   └────────────────────────────────────────────────────────────────────────┘

Which really seems to say its doing its thing.  Maybe I don't hear the disks because it's formating the disk first, which is a linear process.  Or maybe the whole copy process is very linear and I won't hear it thrashing.  Its progress implies it's going to take a couple/few days to finish, which is what I'd expect.  So maybe it's working. I'll let it run for a while and see what happens to the array if I try to unplug one of the previously working drives.

Pretty cool that I didn't even need to unmount the array to do this.

Now if I could just figure out why my laser printer periodically decides it needs to print it internal test page, I'd be even happier.

LHC blue-screens the world

I've been thinking about writing this post for quite a while, and I figured tonight might be my last chance.  Plenty of people have been worrying about how the Large Hadron Collider (LHC) could destroy the planet by creating small black-holes that might suck in the entire earth.  As the good folks at CERN re-assure us, everything is fine.  I pretty much believe this.  That is to say, I'm pretty sure LHC will not destroy all life as we know it.  Pretty sure.  Otherwise, we've all got a few more hours to live.

So long as my buddy Stephen Hawking's theories about black holes are true, we're fine.  They'll dissipate by themselves and will not suck in the planet.  But to be clear, we are testing this theory.  (I just heard a scientist on the radio trip all over himself as he tried to spurt out a believable "there really is no chance these black-holes will devour the entire earth.")

Last year I wrote about a then-briefly-popular idea that all the world we see is actually a computer simulation.  (Pointless personal anecdaote -- I had this idea in grade-school and tried to marry it with special relativity's universal speed-limit in terms of a primitively digitized simulation where exceeding the speed of light would cause objects to skip pixels during a single time step.  Anyway.)  It's all as if our whole universe is a game of The Sims on some hyper-intelligent alien teenager's computer.  In a fairly religious way, this idea is unrefutable.  It's like a virtual machine trying to hack its host operating system.  Can't do it.

Some theories of simulated worlds hold that what we experience is a simplification of real physical laws.  If this is true, high-energy experiments like LHC could probe the limits of these simplifications.  It could cause an exception to get thrown in the simulation code.  Us clever scientists set up some extremely complex scenario that caused one of the simulation's assumptions to fail.  What happens when the simulation crashes?  Maybe it's a dialog box saying "Abort, Retry, Ignore."  Maybe it's a universe-scale Blue screen of death.  Teenager's response?  Maybe Abort.  How different is that from our whole planet getting sucked into a black hole?

Don't panic.

Covers for Kindles

My girlfriend has a kindle that she very much enjoys.  One of the biggest benefits from it she gets is having a large amount of content in a very small device.  She is a scientist who is very much an information worker.  Having access to a great many research papers in searchable form is very useful for her.  (If only the PDF import worked on multi-column papers!)  She also tends to live out of a backpack, so being able to have several interesting things to read at any give time is very appealing.

So she's often reading her kindle on the bus.  She's noted one interesting difference between reading her Kindle and reading a regular book while on the bus.  When she's reading a normal book, people will ask her what booj she's reading or will look at the cover and just talk to her about the book itself.  With the kindle the question is always "how do you like the gizmo?"  Which gets old after a while.

Here's a suggestion to Amazon on how to address this social problzem: offer full-color PDFs of the covers of books that you purchase for the Kindle, so people can print out their own covers.  These could slide into a convenient holder on the Kindle's attractive leather case.  Long-term it'd be great to have a color e-paper cover for the book, but we're not holding our breath for that one.

Spinning Spaghetti Monster

Last week I went to a different kind of Dork Bot meeting here in Seattle called the Catastrophic Cacophony Workshop.  It was a maker-style event where we broke up into teams of people and in 90 minutes had to build a robotic musical instrument out of one supplied motor and whatever other parts we could scrounge together. 

Our team started with the basic idea of a repetitive percussive instrument with multiple hammers hitting multiple objects.  So we constructed a platform to hold the motor and used tennis rackets to make a rotating frame that held a series of arms that struck objects as they went around.  We then built frames to hold up things like pot lids or copper pipes to be struck, and attached everything from small chains to bundles of spaghetti to the rotating frame to strike them.  The real enabling insight came from the women on the team who realized that by spacing the arms un-evenly we could create an interesting rhythm.  The one-two-and-three rhythm across a variety of instruments ended up sounding much like a drum circle that actually got people dancing.

Thanks to Espressobuzz for capturing our creation on video.  Thanks to Josh Kopel for bringing the event to Seattle, and Shelly Farnham for organizing.  And to my wonderful teammates whose energy and creativity made the Spinning Spaghetti Monster possible.

Apple moves downmarket: iPhone as a services platform

Coincident with the launch of the iPhone 3G hardware, Apple has started a couple of new online services: MobileMe and the iPhone App Store.  In some ways these are natural extensions of existing product lines.  But I believe their launch actually represents a fairly substantial strategic shift as Apple attempts to diversify from a hardware-only company to one that runs on a mix of hardware and services. In order to make this strategy work, Apple will need to sacrifice its much coveted high hardware margins.

Diversifying from hardware 

Since Jobs pulled the company out of the doldrums, Apple has been a manufacturer of high-end hardware.  This is a very nice place to be. Their margins are very high.  If you compare the hardware bits that go into a Mac to those in any PC, Apple's prices are much higher.  PC manufacturers squabble over low single digit margins because their product is almost completely commoditized. HP, Dell and IBM struggle to differentiate themselves in the market. Meanwhile Apple can charge a hefty premium for good industrial design and software that is slicker than windows.  iPod followed in this tradition of high margins by setting the bar for usability in portable media devices and following up with fantastic marketing.

Being a niche retailer of high-end products is a comfortable and stable place to be.  However, having a single line of revenue isn't good for a large company, so diversifying makes a lot of sense.  Thus Apple's current push into services.

The iTunes music store was an important pre-cursor to the current push into services.  Even though iTunes moves a massive volume of music, if you work through the accounting confusion, they're not actually making much money there.  Because the established music industry giants are completely fucked and have no idea what to do, they drive the margins of all big online retailers down to a pittance.  But iTunes did something very important for Apple: it established a billing relationship with customers.  Cellular operators have this kind of ongoing billing relationship with their customers which enables them to push high margin, low utility products like ring-tones.  Similarly, people are used to spending small quantities of money in iTunes to get music. So iTunes is the perfect precursor for an App Store.  In this way, Apple doesn't even need to rely on the carrier's billing relationship to build a services business.  This will be very important for Apple in coming years as carriers increasingly become just another provider of wireless bandwidth.  In the coming decades, the value will not come from piping bits around.  It will come from the services built on those pipes. 

MobileMe is a very Apple service

MobileMe is a cloud-hosted email, contact and calendaring solution. There's nothing revolutionary about this.  Google offers all of these services for free.  By charging for these services, Apple is implicitly promising to provide a better solution.  Considering their vertical integration into popular hardware, it's not hard to imagine that they will succeed at this.  Google will probably remain committed to supporting open standards for working with hardware.  By using proprietary protocols, Apple can provide a higher-quality product and support it better on the few platforms they care about.  It's a classic story we've seen in this industry before.

MobileMe's launch was a disaster.  This isn't surprising as Apple is yet to demonstrated great skill in online services.  Personally I believe they'll figure it out, because it's important to their long-term objectives, and they're a smart company, and the skills to do this well are getting easier and easier to find.  But from a marketing perspective, MobileMe is nothing new for Apple.  Because MobileMe is a premium service for which there is a very good free alternative, MobileMe still targets Apple's classic market segment. They're still targeting people who are willing to pay extra to have something really polished.  They can stick to their classic bag of tricks, like the ads that make fun of people who aren't as cool as the mac devotees.

App Store is new: a platform play

App Store on the other hand is going to be much harder for them to pull off.  For App Store to succeed, their primary challenge is not to attract paying customers, but developers.  Because independent software developers (ISVs) are the ones who are actually creating value in the App Store.  App is just a distributor taking a cut on that.  So what attracts developers to the App Store?  Customers do. 

It's a real chicken and egg problem.  Nobody's going to build software unless there are customers to buy it, and it's hard to get customers without cool apps.  Well right now Apple has the whole farm, but it's a very small farm.  There are millions of iPhones out there, representing potential customers.  Moreover, the iPhone SDK is very rich and capable when compared to its competitors.  Qualcomm's BREW, Windows Mobile, Palm OS, Blackberry and Symbian have all faltered for one reason or several.  Google's Android holds much promise and hope, but at this point it's complete vapor-ware.  So for now, Apple has almost all the mindshare of mobile application developers.

But how long will this dominance last?  Software platforms are natural monopolies, meaning the economics tend to be winner-take-all.  The chicken and egg "problem" can easily turn into a virtuous cycle, pushing a winner to the top. Application developers are fickle and will code to whatever platform has the best distribution.  iPhones are very popular, but they are still only used by a small fraction of all mobile subscribers.  Until the distribution of iPhones reaches a critical mass, their dominance as a mobile application platform is very shaky.

The mobile app challenger is HTML

But the laundry list of alternatives shows that the competition is fragmented.  What could unseat Apple?  IMHO it's not another application platform, open or proprietary.  It's the web.  Every high-end phone can display web pages, and increasingly they're using high-quality javascript engines that can run real web applications. Webkit, the super-fast open-source HTML/JS engine behind Safari is showing up in Symbian devices, Blackberries and Windows Mobile phones.

Apple's challenge is to make the proprieray iPhone SDK compelling to developers.  The alternative is to build a web application that works well on webkit, and works on every high-end phone.  They can attract developers in two basic ways -- make the native features of their SDK more compelling, and provide a large market for distribution of the applications. 

Charging for distribution of these applications is a gimick that won't last long.  Soon all the interesting applications will be free, but tied to cloud services that have their own business models independent of the mobile client.  Premium applications will start to seem a lot like premium ringtones pretty fast.  Still, it will help bootstrap this market for Apple so long as there are no serious competitors.

To stay on top, iPhones need distribution quickly

One thing that ties all these points together is that Apple's continued success with App Store hinges on having wide distribution of iPhones. They are currently throttling the distribution of iPhones for some reason, possibly because of software glitches.  But the aggressive $199 pricing is clearly aimed at attracting a new larger customer base that will help maintain their dominance in the mobile application space.

Long term they might be happy getting by offering premium versions of applications that are freely available on the web.  But something tells me they're actually trying to break open the mass market on this one. This is Jobs' big play.  It'll be really interesting to see how it works out once Android hits the streets.

Is Apple using scarcity to hide iPhone quality problems?

Here I propose an alternative explanation for iPhone scarcity: the difficulty in obtaining a new iPhone keeps people from complaining about problems with it.  I will explore this sophisticated marketing technique that Apple may or may not be employing to cover up quality problems with the new iPhone 3G.  Even if Apple is not doing this deliberately, I assert that it is a valid and potentially very useful technique if your product is lucky enough to have the prerequisites.

New iPhones are hard to get

The blogosphere is full of speculation about whether or not Apple deliberately made the iPhone scarce on opening day and since then.  Most assume that this is deliberate on Apple's part for a variety of reasons, mostly to attract more attention, increase demand, etc.  I assume most of these rants are from bloggers who want their new iPhones but haven't overcome the barriers to obtain one yet.

But if Apple's goal was purely to meter out their distribution, why not sell them online?  To get a phone you need to place an order for one, wait a week or two, and then you can get it.  This seems reasonable in conditions of scarcity.  But to get an iPhone 3G, you need to walk into an at&t store to place your order, and then walk into the store again to pick it up.  Think about this.  If the limitation was purely lack of supply then there are several ways this could be easier for customers:

1. You could order a phone online to be delivered to your house.
2. You could order a phone to be delivered to your nearest at&t store.
3. You could call the nearest at&t store to place your order, but still have to walk in to pick it up.

Try asking them why you can't do any of these things and they will answer with one word: policy.  Clearly Apple & at&t have gone out of their way to make it difficult for people to get their hands on a phone.   This goes above and beyond just preserving a limited supply.  You have to work to get an iPhone 3G.

New iPhones have Issues

From all the reports I've read, the problems with the new iPhone are in the software not the hardware.  I conclude this because my friends with first generation iPhones are experiencing the same problems as those with the new 3G iPhones.  Moreover everybody seems to agree that these problems only showed up after they upgraded their iPhone software.  Problems include:

• Frequent crashes of applications, especially Safari
• Increased lag in common operations
• Significant problems with large contact lists (>200 contacts)
• Extended delays before placing a call

Apple is legendary for their high quality software.  People buy Macs because they "just work."  It's really not like Apple to release a buggy piece of software.  But it sure seems that they did in this case.  Why?  Obvious answers of fierce competition for high-end smartphones.  The more interesting question for me is "How did they get away with it?"  Which it sure seems they are.

Escalation of Commitment: The Hush-factor

There's a well-document psychological principal at play which prevents people from objectively critiquing things that they are personally invested in.  Sometimes called escalation of commitment, or irrational escalation, the idea is the same.  If somebody works really hard to obtain something, they will blind themselves to its faults.  Imagine this conversation:

"Dude, I can't believe you waited in line for hours to get that phone.  What do you think of it?"

"Actually, it's just okay.  The applications crash a lot.  And it's not nearly as fast as I'd hoped it would be -- sometimes it just hangs for like 10 seconds.  But at least it's pretty."

Very few people have the objectivity to imply that their personal sacrifice was not worth while.  This effect is commonly observed in people who buy high-end items. 

The flip side of this effect is buyer's remorse.  But since the phone itself is not actually at all expensive (when compared to the monthly fees), that's unlikely.  Also, it has become a positional good, whereby it has value simply because other people don't have one.  That fact remains regardless of how unreliable it is.

Speculative Conclusion

I posit that Apple knew about the software problems with the iPhone 3G before launch.  They did manage to iron out all the performance and stability problems they encountered before launch.  They felt they needed to launch it this summer to get ahead of other notable smartphones like the Blackberry Bold, HTC Touch, and Android which are hot on their heels.  So they rushed it out the door at sub-standard quality.

In order to partially cover for this mistake, they have made this device especially hard to get.  This covers their tracks in two ways: people make even more noise about scarcity.  And those who do jump through the whoops to obtain one are far less likely to complain about it.

Stupid Prius Tricks - Traction Control Distractions

I love my Toyota Prius.  I'm very grateful to Google for giving me a financial incentive to buy a brand new car which does everything I want.  It's totally big and practical, it's very high-tech, it both supports and projects my values, etc.  I could sing its virtues for pages.  But for now, I want to share some non-obvious things about it in a series I'm calling "Stupid Prius Tricks."  (With a tip of a hat to David Letterman.)  Several of these articles (like this one) are complaints, but this must be taken in the context that overall I love the car.

The Prius's High-Tech Safety Features

The Prius has at least a couple of smart traction control systems which use technology to help maintain control of the car in difficult situations.  Of course there's anti-lock brakes (ABS).  In addition to ABS the braking system does things called Electronic Brake-force Distribution (EBD) and Brake Assist.  Beyond that, it's got Traction Control, and in all but the stripped-down cars something called Vehicle Stability Control (VSC). What each of these features does, or how they differ from each other I don't know and don't frankly care.  They help the car do what I'd want it to do under good driving conditions when the road is slick or uneven or I'm driving too fast or what have you.  All this is great.  But there's one subtle and very bad flaw in the implementation of at least one of these systems.

The Stupid UX for these Features

When the Prius needs to engage one of its TLA control systems, an orange light appears on the otherwise dark dashboard.  Sometimes a sound will beep.  This is presumably done to let you know that the fancy feature that you paid for is working on your behalf.  Call it a marketing feature if you will, because there's nothing you can do with this information, or way to react to it within the vehicle's controls. The stupid thing about this feature is that it's distracting to have lights flashing and sounds beeping.  When you're having trouble controlling the car is the last time you want something distracting you.  So the engineers at Toyota have done something completely moronic -- a complex system senses when you're having trouble controlling your car, and chooses that time to throw more distractions at you.  I've observed the light and beeping  on when driving over bumpy roads, or down something very steep that has a rut in it, or when the back-right wheel accidentally jumps up the curb.  None of these situations I've gotten into have been serious or very dangerous.  But even still, they were a bit confusing, and the confusion was confounded by having the car meow at me demanding attention.  When the car is not doing what you expect it to, the last thing you want is additional distractions.  Bad choice, Toyota.  These features should be absolutely silent.

What you can do about it

If anybody knows a cheat code to disable this light or beeping, please post it in a comment.  Until then, you can help a little bit by putting a small piece of electrical tape over the orange light with the symbol for "I think you're losing control of the car."