mypod.me

This is all Lachlan Hardy’s fault. He told me that I couldn’t come to WebJam 10 unless I presented. And, because I was a bozo and didn’t get a ticket within four hours after they were announced, well, I reckoned I was out of luck.

Unless I presented.

At the start of last week I gave Vodafone and Apple a whole bunch of money to upgrade my wimpy 8GB iPhone 3G to a supercharged, ‘roid-raged 32GB iPhone 3GS. The reasons for this upgrade were many and varied, but mostly centered on one thing: AR.

From my earliest days doing research into virtual reality (way back in 1990), I knew the best way to use the virtual world was to marry it intimately to the real world. Hence WebEarth, which I coded up in a weekend in 1996. Since then, Google Earth has come along to steal my thunder (heh), and we’ve seen a lot of other apps visualize and virtualize the real. But, just in the past few months – since the release of high-powered Android phones and iPhone 3GS – we’ve started to see some real, handheld AR apps, like RobotVision and Wikitude.

Both of them suck.

I say this out of love: they’re both technical tour-de-forces, but neither were designed by anyone who knows anything about UI/UX. The fact that you look like a complete wanker when you’re using AR should have been the first tip off. The fact that half the time you can’t trigger the AR content – because it’s too small, too far, too jiggly, too something – should have been another.

The concept of AR is fantastic. The execution is flawed. It’s time to step back and do a re-think.

ARG is my version of a rethought Augmented Reality application. It is entirely text-based. (Yes, that may seem perverse coming from someone who spent a decade working in virtual reality, but there it is.) It’s not just mere perversion – I’m making a point: you can visualize your AR data however you please, and it need not be visualized through a camera. We’ve confused augmented reality with overlaid reality. We need to separate these two until they can get their interface issues sorted out. Right now, and for the rest of us, there’s the simplicity and versatility of text.

In a hundred lines of JavaScript and HTML, ARG uses the new whiz-bang HTML 5 Geolocation API to access to coordinates of the device’s location, then passes them along to a Google Local search, displays the results, and repeats the process every minute. That’s all it does — and why it only requires a hundred lines of code. (Download the code by right-clicking here.)

ARG on iPhone OS

I’m doing a Google Local search on everything – literally. My search term is ‘*’. The thing I learned immediately is that search results are not returned by proximity, but rather by relevance, which is a very different metric. Objects further away show up higher in the list because they’re (presumably) more important.

I understand why Google does this, but it’s broken. When you give Google precise coordinates, it should rank its responses proximally: closer items have a higher priority than items further away. Otherwise, well, what’s the point? You haven’t actually sorted anything – or, at best, kinda-sorta sorted them. Really, you’ve handed me a pile of results that you’ve ranked according to some thoroughly opaque standard of relevance. Not very helpful. Google will need to be fixing that, because (drum roll) the next big phase of the Web is now here, the one we can finally call Web 3.0: the geolocative Web.

The world is the database of itself. That database needs to be mirrored, in its dynamic entirety, online. Right now, we’ve barely started to populate that database. We’ve got a few buildings in there, and some streets, but very little of the material stuff of our lives. A lot of that stuff doesn’t move around very much, or only moves around in a very limited proximity: all of that belongs in this database. The things that move around, those need to go into this database too, and we need to develop ways for those moving things to report where they are to keep that database up-to-date. And we need to have information in depth on every one of these things. Not just what Amazon or IMDB or Wikipedia might tell us, but what we want to tell ourselves. Every bit of our stuff, located in space, has its own tag cloud associated with it – though we’ve yet to make that explicit. It’s time to begin. It’s time to build the web of the real.

***

Just a few practical notes:

1. ARG has been tested and runs fine under iPhone OS 3.0+, Firefox 3.5+, Google Chrome for Windows, and on Firefox 3.0+ and MSIE 6+ if they have Google Gears installed. There is no support for Safari 4.x at present (weirdly), though Safari 3.x with Gears should work. If it says “FAKE” in the coordinate readout, it isn’t working. Finally, ARG should work on Android phones, but in an initial test it failed. I may have fixed that, but I haven’t had time (or a phone) for testing. Let me know.
2. Make sure to include the file gears_init.js in whatever directory you’re serving ARG from. That’s a required Google Gears initialization file. You can get it here.
3. Sometimes the geolocation query fails on Firefox 3.5, particularly when ARG is first starting up. I have no idea why this happens.
4. ARG has a companion, Feen, which runs a search on “coffee”. It returns results which are more location-specific than a general query.
5. My very few slides from the WebJam presentation can be found here.

Thanks to this nifty new WordPress plugin, this blog (and my other blogs) now looks very pretty when accessed via iPhone or iPod Touch. Give it a try.

And if anyone wants to access this blog using another WebKit based mobile browser (Nokia N95 or equivalent), I’d be interested in hearing about your user experience.

A just released app from Simplify Media now allows me to stream my iTunes library to my iPhone and my iPod Touch. From anywhere. It’ll work within the house over the wifi, but – far more significantly – it allows me to stream my iTunes library over the 3G network.

So now, my music is entirely portable. And yes, that sound you heard was commercial radio, dying.

The iPhone app is free for the first 100,000 downloads. So if you have an iPhone, get it now!

Qik, the streaming video shot out for jailbroken iPhone 3G. Which is yet another reason to jailbreak. Probably the best one so far.

After two days of hackery, a very basic version of iServe is up and running and available for all of you to play with here. Please be nice, don’t upload anything too large (and be prepared to wait during the bigger uploads) and if you upload anything other than the stated file types, you won’t see it. But please do upload – and download. It’s all running on my iPod Touch.

There are lots of implementation details and scripts to share, and I’ll be doing that tomorrow.

Let me explain why you really do need a web server on your iPhone.

When I mentioned this on Twitter and thereabouts, a few days ago, I was greeted with incredulity. After all, the web is big and wide – why would you also need a web server that you can carry around in your pocket?

The answer is simple, and so very obvious that I’m now shocked Apple didn’t think of this themselves.

Let me present my latest invention: iServe.

iServe leverages the ubiquitous technology of the World Wide Web with the fast-becoming-ubiquitous technology of ZeroConf (known as Bonjour in the Apple universe). When you put these two technologies together into a jailbroken iPod Touch or iPhone, you get a portable and fully scalable document viewing platform. Let me explain…

I have been using my iPhone to read PDF files – specifically, I’ve been reading Apple’s Safari Web Content Guide for iPhone OS. It’s useful – and came in handy for iServe – but it’s a book-style PDF file, which means that the pages are too big to be comfortably read on the iPhone’s screen. Silly for Apple to make a mistake like this, but they’re still thinking of a “book” as something that’s printed on 8 1/2 x 11″ or A4 size pages. It isn’t, not really, but PDF format (or rather, the viewers for PDF format) don’t take this into account. So, I read the Apple PDF in landscape mode on my iPhone, but, even with the 480 pixel-width screen, I had to constantly swipe a finger across the screen to see the end of the line I was reading. Not at all pleasant. (Yes, I could shrink the document to fit across the display, but I can not read 4-point font sizes. Reading should not be an optometric exam.)

iServe provides an elegant solution to this problem. Using the lighttpd web server I installed this morning, which has a document root at /var/www, I created a new subdirectory, /var/www/iserve. Into this directory I place whatever PDF, DOC and XLS files I might want to view. I then wrote a Python script, index.py, placed into this directory, so it is executed whenever a request for the iserve directory contents is received. This script simply reads the contents of the directory and gives a nicely-formatted HTML output in response. This script either works directly from Mobile Safari on the iPhone (where it’s accessed at http://localhost/iserve/), or, because of Bonjour services which are always running on the iPhone, it can be accessed at http://lugh.local/iserve/ from any machine residing on the same physical network as my iPhone. And that’s the magic of it: if I want to read a PDF on my iPhone, I can do it, using Mobile Safari as a launchpad. If I want to read it from, say, my MacBook Pro, I can do that too, using Safari (or Firefox) as a launchpad.

Suddenly I have a scaleable, ubiquitous display capability for any of the documents on my iPhone.

A few pictures might help. Here’s the icon for iServe on my iPhone:

When I tap on that icon, I launch Mobile Safari, and this is what I see:

I can scroll through and tap on any of these documents, launching the appropriate viewer.

Now, if I should wander into my flat, I can walk up to my MacBookPro, and do precisely the same thing:

Voila. Portability and scalability. This is more than a convenience. I could perhaps be carrying documents to a client for review. I can review them on the train as I travel to the meeting, then, when I arrive in the client’s offices, I can simply present my iPhone, connect to the client’s network, and browse the documents. The client can do a “Save As…” command on their browser, should they want their own copy of these documents.

It’s all very simple, and very elegant. And it makes me very confused that Apple overlooked this. Now perhaps some enterprising entrepreneur is putting together something in the App Store which does this. In some sense, FileMagnet (an iPhone app which I own) does this. But it’s entirely proprietary and requires a client program to be installed on a Mac or PC. Using standard web technologies makes this much easier. Web browsers are everywhere. Wireless networks are everywhere. Shouldn’t an iPhone or iPod Touch put those ubiquitous resources to work?

Yes, and there are some very good reasons why you’d want a webserver running on your iPhone. Those reasons come later. First comes the webserver.

To install lighttpd, a tiny and insanely full-featured HTTP server, either use the Cydia GUI, or use the command apt-get install lighttpd. Now a few more steps of manual configuration will complete the installation:

• Create the directory /var/log/lighttpd to store the log files;
• Create the directory /var/www as the document root for your web site;
• Create a configuration file in /etc/lighttpd.conf for server configuration.

Mine looks like this:

server.document-root = "/var/www/"
server.port = 80 # Default
server.modules = ( "mod_cgi","mod_indexfile","mod_dirlisting", "mod_accesslog" )
server.tag = "iPhone lighttpd"
server.name = "Aiode.webearth.org"

mimetype.assign = (
".pdf" => "application/pdf",
".sig" => "application/pgp-signature",
".spl" => "application/futuresplash",
".class" => "application/octet-stream",
".ps" => "application/postscript",
".torrent" => "application/x-bittorrent",
".dvi" => "application/x-dvi",
".gz" => "application/x-gzip",
".pac" => "application/x-ns-proxy-autoconfig",
".swf" => "application/x-shockwave-flash",
".tar.gz" => "application/x-tgz",
".tgz" => "application/x-tgz",
".tar" => "application/x-tar",
".zip" => "application/zip",
".mp3" => "audio/mpeg",
".m3u" => "audio/x-mpegurl",
".wma" => "audio/x-ms-wma",
".wax" => "audio/x-ms-wax",
".ogg" => "application/ogg",
".wav" => "audio/x-wav",
".gif" => "image/gif",
".jpg" => "image/jpeg",
".jpeg" => "image/jpeg",
".png" => "image/png",
".xbm" => "image/x-xbitmap",
".xpm" => "image/x-xpixmap",
".xwd" => "image/x-xwindowdump",
".css" => "text/css",
".html" => "text/html",
".htm" => "text/html",
".js" => "text/javascript",
".asc" => "text/plain",
".c" => "text/plain",
".cpp" => "text/plain",
".log" => "text/plain",
".conf" => "text/plain",
".text" => "text/plain",
".txt" => "text/plain",
".dtd" => "text/xml",
".xml" => "text/xml",
".mpeg" => "video/mpeg",
".mpg" => "video/mpeg",
".mov" => "video/quicktime",
".qt" => "video/quicktime",
".avi" => "video/x-msvideo",
".asf" => "video/x-ms-asf",
".asx" => "video/x-ms-asf",
".wmv" => "video/x-ms-wmv",
".bz2" => "application/x-bzip",
".tbz" => "application/x-bzip-compressed-tar",
".tar.bz2" => "application/x-bzip-compressed-tar",
# default mime type
"" => "application/octet-stream",
)

In order to make sure that the web server launches every time the iPhone reboots, we need to add a launchd control file. This file, com.lighttpd.plist is placed into the /Library/LaunchDaemons directory, where the launchd process will start it and keep it running if it should crash or be killed.

And once I can figure out how to get tags into Wordpress, well, then I’ll show you the contents of that file. Sigh. The launchd control file is an XML property list, which means that Wordpress (and other web tools) attempt to parse it, rather than display it.

When iPhone is rebooted, lighttpd will be running as one the background processes.

Twitter has been quite helpful. This post, from one of the iPhone boards, seems to be echoing the outcome of today’s research: I believe that the iPhone OS kernel lacks the capability to perform IP forwarding.

If that truly is the case, I’m not sure what can be done – short of writing a kernel module – to make the iPhone route the way any normal *NIX box would. I don’t really know enough to write a kernel module, and I have better things to do with my time than learn. Since the demand is there for something along these lines, I strongly suspect that someone else out in the wilds of this totally anarchic community of iPhone hackers is working on it.

This is odd. IP forwarding is pretty much bog-standard stuff. Perhaps they left it out of the kernel compilation for space reasons. It’s hard to say. Or, perhaps, somewhat more conspiratorially,  did this to prevent the kind of tethering that iPhone is truly capable of.

I don’t have any answers. I do have a lot of data, and I’ve learned a huge amount about the iPhone. It’s basically a very stripped-down BSD boxen, running “SpringBoard” as its interface – the equivalent of the OSX Finder. There are lots of services running, but they’re mostly geared to the iPhone as media device.

Update: I did manage to buy NetShare while it was briefly available through the Australian iTunes App Store. And it certainly does seem to work nicely. It’s only a web proxy, and therefore of limited utility (Twitter would work, but Mail wouldn’t).Another update: I have been informed that I may be wrong about this, and will investigate. Will be best to see if routed can be configured to make the iPhone a true router. It clearly already has a full routing implementation – that’s just part of the kernel. And the route command is there. So it may be rather less complicated than people are making it out to be.

Another update: As has been pointed out by people who actually do these sorts of things for a living, routed isn’t important. IP Forwarding is. That’s done with the ipfw tool, which is part of iPhone OS, but may be lacking all sorts of internal goodness which actually makes it, well, useful to do packet forwarding. Research continues…

And it’s used to keep a link from a device that may be behind a firewall. Or perhaps it’s connected to a highly variable mobile network. When it’s opened, it allows users to “tunnel” through the connection, to wherever the device might be.

Here’s what I put into the shell script “tunnel”, on my iPhone’s ~/mobile directory:

#!/bin/sh
ssh -R 8022:localhost:22 -f -N mpesce@webearth.org

That fairly simple line opens a connection via port 8022 on webearth.org (which sits underneath my telly), to port 22 (which is the normal SSH port) on my iPhone. Thus, if I go to the command line on webearth.org and type the following:

ssh -p 8022 mobile@localhost

Voila, I have a shell into my iPhone.

Yes, those of you who are not serious network geeks, this seems like rocket science. It’s not. Reverse SSH tunnels are used all the time by folks behind, say, the Great Firewall of China, or simply the NSW Public Schools network.

Update: I’ve realized that SSH, as a security precaution, will shut down open connections if they’re not used. In ~/mobile/.ssh, create a new file named config, and put the following line in the file:

ServerAliveInterval 60

That should keep things connected indefinitely.