Architecture and Coding

I have the word “architect” in my title at work, which is cool cause it apparently means I know what I am doing but is bad because it means that people assume that I always know what I am doing.

I consider myself a pretty good software engineer, in that I can generally produce working software against the requirements given to me. I actually do not tend to be up on the latest trends in WS-* or frameworks, etc so most of my code is pretty straightforward (I hope). Thankfully others I work with (both at work and on hobby stuff) keep up with things like Hibernate, Spring, Quartz so it allows me to keep track of what people are actually using, versus what the hype is. I will shamelessly take the skeletons of how those systems are used by other co-workers and friends. However, I always try to figure out how to make using them easier if I find bits that are in any way non-intuitive.

I tend to write in an “agile” way. What this really means is that I think about the problem and do some drawings, pseudocode, sketches, pictures, whatever that I need to get something up and running that solves an immediate problem. Then I move on to other components. Realize that I missed a detail. Rinse. Repeat As I move onto other requirements I integrate them with my solution and always try to have a working system that does something.

Back to the point of my story… this is a horrible way to work if you are supposed to “architect” (in the sense of “here is the design, build it”) a system based on a set of requirements. It is very easy to draw some lines and boxes with cool names and leave the “details as an exercise for the programmer.” Bad idea. By the time it has made it to the programmer (unless they work intimately with you on your own team) many of the requirements have been lost, and they are left only with the design. So they implement, or try to implement, what has been given to them. And surprise, surprise, it meets the requirements but does not actually work.

This is when I realize that what I really gave was a design for a “first pass” that would really be a prototype of the system. If I were implementing, it would be easy to turn around and enhance to go to the next level. But in this case, it has already gone down through two other levels of tech lead and off-shore architect. So not very easy to say “ok, so that we aren’t quite there yet, we actually have a million other details to work out”.

So lesson learned there is that however long you would normally give yourself to “design” the system, multiply it by 10 so that you can basically look at every single API and external system being called, and run through every use case you can think of to see if the design does in fact meet the requirements and produce a working system. It really is not enough to see that API X has a method that looks like Y and takes parameters A and B and probably does Z. What happens if you have 1 million sets of {A,B}?

My real point, I suppose, is that being fairly good at designing and implementing your own systems does not make you an architect. And if you are in that position, do not let someone tell you that you only need “maybe a day or 2” to design something if it is of any real complexity (where a good way to measure complexity is how many interfaces exist between various systems). Give yourself at least a couple of days per non-trivial interface.

I hate the feeling of thinking that if only I had had more time my architecture would have been perfect after the first implementation. I hate even more the feeling that it honestly probably would not have been. It is never right the first time.

Or maybe I just really suck as an architect.

In any case, I am currently getting to do the implementation on a quick-turnaround (aren’t they always in the corporate world?) implementation and I am absolutely loving it. Sure, the deadline is quickly approaching. But you know what, it is my project to succeed or fail at. I am using parts of the system that I “architected” and seeing parts where I totally did not think about the problem in low-level enough detail, and trying to fix those oversights. I am getting to play with systems that I have not worked on previously. And most importantly, I am getting to code! Maybe it is the design aspects of the coding I enjoy, but I think not. What I really, truly, absolutely, love is seeing something grow before my eyes into a working system. I love the list of “//TODO” items (and love even more removing them as they get done) and bits of stub code. I love when I get surprised that it works even better than I expected. I love making a database query 10,20,30 times faster through simple application of a decent index. I love keeping 10 things in my head as I move from file to file and implement my ever-growing APIs.

I have no doubt that my methods would scare many of the purists out there. At heart, I am a “just-in-time” developer. For my emulator, I did not implement an instruction until I ran into my “not yet implemented!” exception. Same goes often for other things that I implement. It is a bit of an extreme example of YAGNE, I suppose.

Update (11/30/2007): Added some words I have left out.

iPhone Native Look and Feel Wikipedia

As I mentioned before, I used the “schools edition” of Wikipedia and stored it on my iPod Touch (will work on the iPhone as well) for some hot off-line Wikipedia action. What quickly became apparent is that the set of subject index pages that come with it are actually kind of hard to navigate (small text, etc). I figured that somebody had made some set of style sheets that made web apps look like native iPhone apps, and of course someone did.

So what I did was make a basic perl script would crawl through the subject index pages (looking for “categoryname” div tags) and generate an html file that had the same look and feel. Currently all of the index goes into a single page, which gets you the “wipe” effect, but it could just as easily be put into different files. I had to modify the javascript slightly to allow it to link into the actual content pages (the default javascript assumes that all the links are internal to the file… either that or I just couldn’t figure it out). Basically, if an “a” tag has a class of “external” then it’s… external.

So if you grab iphonenav.js from the link above, and modify the click eventListener like so:

addEventListener(“click”, function(event)
{

var link = event.target;

if (link.className.indexOf(“external”) == -1){
event.preventDefault();

while (link && link.localName.toLowerCase() != “a”)
link = link.parentNode;

if (link && link.hash)
{
var page = document.getElementById(link.hash.substr(1));
showPage(page);
}
}
}, true);

You can download the perl script from my site. Just run it against “wikipedia/wp/index/subject.html”. Save the script with a .pl extension. My host seems to be trying to interpret the script, even if I have the extension be “.pl.txt”.

You can see what the end result looks like here. By the way… you should be able to just take that html file and drop it next to the main index page for the Wikipedia schools edition.

As usual with this type of thing, I’m sure people will look at the script and think “what are you doing!?” But it seems to do the job.

My very own Hitchhiker’s Guide

So I recently bought one of the new iPod Touches (basically, an iPhone without the GSM radio, camera or mic).

It’s cool that it plays music and all, but that’s not what I bought it for. I really was impressed by the way the browser worked and that essentially full-fledged Cocoa applications could be installed onto it.

What I really got it for was so that I can have my dream of an internet-enabled hand-held encyclopedia. I want The Hitchhiker’s Guide to the Galaxy. And now I’m that much closer.

It turns out that there is a “school edition” of Wikipedia, which is a static dump of Wikipedia content that’s been slightly edited and chosen for entries that “appeal to children.” Don’t let that dissuade you, it has an entry on MKULTRA. In their words:

It is about the size of a fifteen volume encyclopaedia (24,000 images and 14 million words), and includes essentially all topics in Wikipedia rated “Good” or higher by Wikipedia itself at date of production. Volunteers at Wikipedia have also suggested more than a thousand other articles considered valuable and relevant to this project which SOS Children volunteers have checked and tidied up.

They have a 750MB download available at http://www.soschildrensvillages.org.uk/charity-news/wikipedia-for-schools.htm. Once you de-compress it, you’re left with the exact same thing as what is at the Schools’ version site. Until we truly have ubiquitous, and free, Wi-Fi access (including during flights), this will be an awesome set of information to have on hand.

Unfortunately, Safari on the iPod/iPhone does not support viewing of local files directly with a “file:///” protocol. It reports “Cannot view the page because it is a local file.” It’s a good thing I had already “jailbroken” my iPod, so I just had to install the ‘lighttpd’ web server.

After copying the files to my iPod at /var/root/Sites using WinSCP (you can use any scp program though) into a ‘wikipedia’ directory, I was able to browse to http://localhost/wikipedia on my phone. And voila!

I assume this all works on the iPhone as well as the iPod, as most of the apps are actually meant for the iPhone.

I know this isn’t what Apple intended, yet (they are coming out with the official SDK in February), but if I wasn’t able to do something like this I would not be as pleased with my purchase. I don’t care if there are actually tons of other ways to have a hand-held encyclopedia, how many of them are also a music-playing, mail-synchronizing, calendaring web browser?