More Ramblings from a Los Angeles Programmer

November 5, 2008

Blu Ray DRM “cracked”

Filed under: coding, technology — Tags: , , , , — Josh DeWald @ 8:32 pm

I absolutely love to hear about supposedly “unbreakable” DRM mechanisms being cracked (well, circumvented in this case).

http://it.slashdot.org/article.pl?sid=08/11/01/1728222&from=rss

To Content Producers: Most people pay for their content. Those who don’t pay, never will. You are not protecting anything or making any more money by implementing these insane schemes to stop being from copying. All this does is make engineers that much more persistent in figuring it out. For every great engineer at your company, there are 100 out in the wild who are going to crack you software. Again, I will happily pay for a BluRay disc, but I want to play it on the player of my choice (such as on a MythTV installation). But when you have your stupid restrictions, you have lost a sale.

So I am hoping that I can in fact get a MythTV box that can play Blu-Ray “out of the box” with a drive.

October 25, 2008

Interesting bits about the JVM

Filed under: coding, technology — Tags: , , , , , — Josh DeWald @ 11:53 pm

Here are some random things about the JVM that you might (or least I didn’t) not know:

  • boolean is represented as a 4-byte integer internally and treated as such in all bytecode-level operations (method parameters can be specified as being booleans)
  • Often when a NullPointerException is thrown, the JVM actually has access to the method that was being called, but the NPE is generally thrown in the calling method, rather than the called method.
  • Reflection is really inherant to how the JVM is specified and operates; all methods are located by {class,method name,method parameters} dynamically
  • The JVM truly has no concept for the Java language; String and Object are for the most part the only classes treated specially
  • The JVM is stack-based rather than register-based

Some cool things about C# (.NET as a whole?):

  • C# has the notion of Properties, which let clients of your class access members “directly” while at the same time allowing for change to underlying implementation (as they are actually getter/setter methods)
  • C# allows for the notion of first-class methods, which it calls delegates. This lets you avoid defining a whole interface to get access to a single generic method.
  • C# has first-class support for firing events, which makes use of the delegates (not sure if it has to or not, I’m still learning this stuff)
  • C# requires you to use override prefix for a method if it is overriding a parent method, which makes it clear to readers of that class that this is in fact overriding something.

I learned these little tidbits while learning C# by writing a JVM in that language. I am fascinating by emulators and VMs because they are software that represents something that is well-defined (http://java.sun.com/docs/books/jvms/) . So in a sense the (naive) implementation is fairly straightforward, if tedious. (I also learned that you can actually write a non-trivial C# application by effectively writing Java and renaming it with .cs and doing a few basic replacements… or just about.)

This project would not have been at all possible without the GNU Classpath project, which has worked tirelessly to implement all of the standard Java classes as well as reference implementations the classes needed to tie to an actual Virtual Machine implementation. I have not gotten JNI to work yet, so for the time being I am implementing the native stuff directly in C# (which makes sense actually as that is my JVM implementation language and is sitting “below” the JVM).

Both Classpath and the official Sun jre implmentation (which is now open source as OpenJDK) provide real world implementations of that stuff you have not looked at since university (Hashtables, Linked Lists, etc) in a fairly readable format. And because they are real world, they offer glimpses into the optimizations and workarounds that have to be done to make these data structures work in the real world.

There is also a project called IKVM which is a very complete .NET-based implmentation of the JVM as well as the class libraries which allows for .NET applications to actually execute Java classes. I think it includes a combination of GNU Classpath and OpenJDK classes inluding managed .NET implementations of the native methods. If I continue this project (not terribly likely, I don’t call it “ToyVM” for nothing) I will probably migrate to using that so I can focus on the internals of the JVM itself. When I started I just wanted to get going and I had issues with the version of Visual Studio .NET that I had and then could not get GNU Classpath or OpenJDK to build or work in cygwin. I actually used MonoDevelop to do the C# development (so I wrote a JVM in C# using a Linux-based .NET implementation running on x86 VM on top of Windows XP).

A couple of nights ago, I finally got the “Hello, World!” application to work after about 2 months of development on and off, not sure what the actual man-hours was.

I used C#’s event handling/delegate set up to do gather some runtime statistics for the basic “Hello World!” application and apparently it loaded 142 classes before it finally did the output. Most of these were not used obviously, but are part of the environment that is statically loaded by key classes (like Charsets).

Next steps:

  • Make it look more like C# (Cxx languages tend to use GetBlah() rather than getBlah(), and C# supports Properties which I would like to make use of)
  • Implement Garbage Collection (pluggable perhaps). I am curious about the various methods that are used and which are better in various situations
  • Do some additional refactoring into additional namespaces
  • Optimize the most heavily used bytecodes if possible
  • See what breaks when I run things other than HelloWorld.class 🙂 I am very much a just-in-time developer so I only got the stuff working that were absolutely required to get Hello World to work. 84 byte codes have been implemented (out of the 107 that were encountered, but some load xload,xstore,if,if_icmp get reused with different initialization parameters)
  • As mentioned before, look into integrating with the IKVM libraries so I can worry less about the native aspects of it

I have put the code into a local git repository, which is another tool that I have been wanting to play around with. I am happy to push that out somewhere, with the normal caveat that this is your typical homebrew code that is not as well commented as it should be (but is hopefully well structured enough for it to make sense).

Happy Coding.

April 9, 2008

Strange keyboard issue

Filed under: daily life, technology — Tags: , , , — Josh DeWald @ 3:19 pm

I have been having this issue for over a year now, and have not been able to find a solution via Google nor through company IT (not for lack of trying on their part, it’s a whacky issue).

Very often (but it can also go for weeks) some of my keys will stop working on my Dell Latitude D610. It is generally the same keys, and will even occur in similar orders after reboot. The “favored” keys are: r,1,e,=,and ESC. Sometimes ‘o‘ and ‘y‘ won’t work. Even better, there are times when the ‘m’ key will actually output a ‘,’.

I need help!

It is very unlikely that it is either the keyboard or the actual laptop as we have tried:

  1. Using external keyboard (the exact same keys will not work)
  2. Replacing the laptop keyboard from another one
  3. Replacing the entire laptop itself (moving the hard drive to another)
  4. Running full system checks from IT department to check for viruses, corrupt files, etc (everything checks out)

So, to me, this would imply that it’s a driver problem in Windows. I have no attempted to replicate the issue in another Operating System. I don’t work in Linux enough (at all) on this machine to be able to sit around until it happens.

Most answers on the internet blame the keyboard, but based on the evidence I do not see that as being it at all.

I have recently downloaded a keyboard test application to see if anything is being detected when I hit the keys in question during one of these incidents.

Anyhow, hoping my few readers perhaps have an idea or have seen similar issues.

Update (4/22/2008): So a couple of days I had this issue again but I really wanted to be able to get logged in so I went through each of my keys to see what would happen. It definitely was like some bit was getting forced to be set or some wiring issue (and still I’m convinced that’s not it). For instance, the ‘m’ key was outputting a ‘c’ I think. I finally tracked down the ‘m’ key by hitting the RIGHT SHIFT key (not the left). I had downloaded a “keyboard test” program and when I ran that there weren’t any actual discrepancies (which is what I was hoping for):when I hit ‘m’, it output the same results for both the BIOS read and the Windows read.

January 30, 2008

Amazing

Filed under: delirium, mathematics, technology, uk life — Tags: , , , , , — Josh DeWald @ 11:13 pm

Lately I have been more and more amazed by two things:

  1. Gravity
  2. Magnetism

Sure, we have a bunch of equations for their effects (i.e. the force they impart), but do we really truly understand how they could actually come about?

Think about it, our Earth is completely locked into orbit around an object 93 million miles away. 93 million. Heck, Pluto is 3.6 billion (3.6 thousand million for the Brits out there) miles away and it ain’t even going to break away. That is pretty damn amazing and we totally take it for granted.

One of the more entertaining things to do for us simple-minded folks is to take two magnets and try to push their like-poles together. Those puppies will really fight you! You can use the effect and move another magnet around a table. Japan uses this for friction-less high speed trains. Kids play with magnets from the youngest age, and it’s always fascinating, but it is completely taken in stride.

It is very interesting that another instance of moving things without touching them, telekinesis, is complete hogwash (presumably…) despite there being electric currents running all through us.

Maybe this is why every guy has, at some point in his life, been convinced the he could use the Force.

Don’t even get me started with quantum entanglement (aka “spooky action at a distance”). The world around us is truly stunning, even when we can “explain” it.

Update:Had 36 thousand million instead of 3.6

January 27, 2008

Schools and that jazz

Filed under: coding, technology — Tags: , , , , , , — Josh DeWald @ 7:31 pm

There has been a lot of furor (at least from all the links popping up on proggit) around the “worth” of CS degrees and how bad the programs are.

My personal take is that people are expecting the wrong thing out of it. There is certainly a mechanical/trade aspect to programming. That’s the part that they can teach quite well: syntax, basic algorithms, etc. Strangely, this is the part they only teach in Software “Engineering” courses versus pure “Computer Science.” Most people go into these programs expecting to be able to walk into a typical business programming job and get to work. They really do no want to learn about Big O, Finite State Machines, or Data Structure Implementation. Who wants to know about all that damn math!?

The bit that they do not teach well is the part that actually makes you good at “programming”: critical thinking problem solving skills. Much of your time is spent figuring out how to go from problem to solution and, after doing that, why the apparent solution does not actually solve the problem correctly. You will spend a lot of time debugging software and fixing bugs. That’s just the way it is. Yes, each language makes some aspect of expression easier, but at the end of the day the actual algorithm is exactly the same. There are really only two ways a bit of program can be wrong:

  1. The algorithm is incorrect
  2. The expression of the algorithm is incorrect

I would argue that the analysis of either of these problems requires slightly different skills. One of them is the heart of “computer science” and it is the creation of algorithms that solve problems in faster and more innovative ways that are researched. The majority of us will never come up with a truly new algorithm; rather we will solve a problem that is just being defined in terms of different nouns. So a key skill of any programmer (during the design phase of construction, however short that may be) is recognizing how the problem can be re-phrased in another light and use a known algorithm. The site TopCoder is an excellent way to practice this.

Assuming that the proper algorithm has been chosen, the next step is to actually implement it. Theoretically this is the “easy” part, but it is also where the majority of effort is placed in the real world. An absolutely necessary skill of a software engineer is to be able to follow the logic of code (usually people speak of reading code but I really think you follow the logic of code instead. While I have seen some poetic code before, it really isn’t literary in nature) and trace what it is doing with a particular input. It is this skill (or the lack of) which is why, i believe, people complain about bad Computer Science education. You can whine and moan about Java or C++ being used (instead of “pure” languages like Haskell) but frankly that is a bunch of hogwash. If a person is getting the right education, or has the right innate talent, then they will be able to solve problems in any language given to them.

I have always said to people that one of the most useful classes I ever had in college was my Physics class. The professor was smart and did not allow calculators on the exams. You see, it is not the answer that matters, but how you get there. Your algorithm. The most important lessons in Computer Science (and medicine, and law, and….) is those that teach critical thinking and being methodical about solving a problem.

Ruby will not make you magically a better programmer. Java does not turn you into some brainless idiot. Perl will not turn you into a person incapable of writing clear code. Using RAD tools will not prevent you from learning how your code actually works. It is the person behind the code that matters.

Update: I found this response to the debate by Brian Hurt at Enfranchised Mind to be very good (and much better written than mine) in the sense of mentioning that, effectively, you want a “Developer that knows Java” rather than a “Java Developer”.  The reality, though, even if we don’t want to admit it, is that companies want Java developers. What do they care if the person will be useless 10 or 20 years from now, they’ll just get a developers that are trained up on New Fangled Language X.

January 9, 2008

Holidays and a New Year

Filed under: coding, daily life, technology, uk life — Tags: , , , , — Josh DeWald @ 5:52 pm

It’s been a while…

I’m ending a nice long 3-or-so week extended Christmas (just me, the wife, and pictures of gifts from family), New Year’s (very cool dinner on a ship permanently docked on the Thames. Those who know me won’t be surprised to know that I spilled red wine all over the table within about 2 minutes of sitting down. The 9 others at the table were quite nice about it) and 5-day Moroccan holiday in Marrakesh (مراكش). The last was quite cool (finally something different from Europe, you can only handle so many castles) but hectic and wearing at times (I can only handle so much haggling.. even though it’s satisfying to only pay 50% of the original price, I know I’m still paying way more than the price a local would pay). Again those who know me will not be surprised to know that I dropped (and shattered) the large tagine that we had purchased… was about 10 feet from our boarding gate at the airport.

And to really but an end to the holiday, my wife is now on a plane back to the States to get us going for our repatriation there. I will be following 3 weeks later, as my year-long stint here in the UK is ending. I have have had an awesome time here, both at work in out and about. Met some great people who I will definitely miss.

And now for something completely different..

To bring things back around to geeky stuff (I tend to skim over other people’s personal stuff, so I understand if you, Reader, have done the same) I have finally started working on my Super Nintendo (SNES) emulator. It is still using the same basic core as the C64 and NES emulators. Main difference is that the SNES using a 65816, a successor to the 6502 which can address 16MB of memory (24 bits) and implements all of the 256 op codes and adds some more addressing modes. When it initially starts up, it is actually in 6502 emulation mode (with bugs that the original 6502 had fixed, which I’m sure provided frustration to many developers who depended on undocumented instructions and bugs). I have gotten some basic graphics to work in the ‘test.smc’ demo included in Y0shi’s documentation, but it is nowhere near even able to get screenshots, but hopefully only a week or so (I’ve spent a feverish 3 or 4 days dedicated to SNES stuff, but probably spent another couple of weeks previously working toward getting an Apple IIGS emulator working, which uses the same processor) to get there.

I have started adding some JUnit tests of the actual instruction implementations, as even minor bugs can truly spiral out of control in the running of the system.

As usual, Zophar’s domain has proved invaluable for obtaining documentation, but I have also used Wikipedia for system overview (memory sizes and the like) and another site I stumbled on just called “Emu-Docs

I will make the code available via CVS or Subversion once it is in a usable state. Apparently my wife never really played the SNES, so we shall see if I can find anything to drive me like getting SMB3 working on the NES did.  I would love to get Super Mario Kart working.

I have been using ZSNES as my “reference” for what a working system should look like (I don’t know if it’s open source or not, but I am only using the executable in any case).

Shoutout goes to the many people who dedicated hours and hours dissecting and analyzing the SNES and writing emulators in C and Assembly which ran on old Pentiums. My Java emulator may one day run at a full speed on a 2 Gig machine 🙂

November 27, 2007

Architecture and Coding

Filed under: coding, technology — Tags: , , , — Josh DeWald @ 8:49 pm

I have the word “architect” in my title at work, which is cool cause it apparently means I know what I am doing but is bad because it means that people assume that I always know what I am doing.

I consider myself a pretty good software engineer, in that I can generally produce working software against the requirements given to me. I actually do not tend to be up on the latest trends in WS-* or frameworks, etc so most of my code is pretty straightforward (I hope). Thankfully others I work with (both at work and on hobby stuff) keep up with things like Hibernate, Spring, Quartz so it allows me to keep track of what people are actually using, versus what the hype is. I will shamelessly take the skeletons of how those systems are used by other co-workers and friends. However, I always try to figure out how to make using them easier if I find bits that are in any way non-intuitive.

I tend to write in an “agile” way. What this really means is that I think about the problem and do some drawings, pseudocode, sketches, pictures, whatever that I need to get something up and running that solves an immediate problem. Then I move on to other components. Realize that I missed a detail. Rinse. Repeat As I move onto other requirements I integrate them with my solution and always try to have a working system that does something.

Back to the point of my story… this is a horrible way to work if you are supposed to “architect” (in the sense of “here is the design, build it”) a system based on a set of requirements. It is very easy to draw some lines and boxes with cool names and leave the “details as an exercise for the programmer.” Bad idea. By the time it has made it to the programmer (unless they work intimately with you on your own team) many of the requirements have been lost, and they are left only with the design. So they implement, or try to implement, what has been given to them. And surprise, surprise, it meets the requirements but does not actually work.

This is when I realize that what I really gave was a design for a “first pass” that would really be a prototype of the system. If I were implementing, it would be easy to turn around and enhance to go to the next level. But in this case, it has already gone down through two other levels of tech lead and off-shore architect. So not very easy to say “ok, so that we aren’t quite there yet, we actually have a million other details to work out”.

So lesson learned there is that however long you would normally give yourself to “design” the system, multiply it by 10 so that you can basically look at every single API and external system being called, and run through every use case you can think of to see if the design does in fact meet the requirements and produce a working system. It really is not enough to see that API X has a method that looks like Y and takes parameters A and B and probably does Z. What happens if you have 1 million sets of {A,B}?

My real point, I suppose, is that being fairly good at designing and implementing your own systems does not make you an architect. And if you are in that position, do not let someone tell you that you only need “maybe a day or 2” to design something if it is of any real complexity (where a good way to measure complexity is how many interfaces exist between various systems). Give yourself at least a couple of days per non-trivial interface.

I hate the feeling of thinking that if only I had had more time my architecture would have been perfect after the first implementation. I hate even more the feeling that it honestly probably would not have been. It is never right the first time.

Or maybe I just really suck as an architect.

In any case, I am currently getting to do the implementation on a quick-turnaround (aren’t they always in the corporate world?) implementation and I am absolutely loving it. Sure, the deadline is quickly approaching. But you know what, it is my project to succeed or fail at. I am using parts of the system that I “architected” and seeing parts where I totally did not think about the problem in low-level enough detail, and trying to fix those oversights. I am getting to play with systems that I have not worked on previously. And most importantly, I am getting to code! Maybe it is the design aspects of the coding I enjoy, but I think not. What I really, truly, absolutely, love is seeing something grow before my eyes into a working system. I love the list of “//TODO” items (and love even more removing them as they get done) and bits of stub code. I love when I get surprised that it works even better than I expected. I love making a database query 10,20,30 times faster through simple application of a decent index. I love keeping 10 things in my head as I move from file to file and implement my ever-growing APIs.

I have no doubt that my methods would scare many of the purists out there. At heart, I am a “just-in-time” developer. For my emulator, I did not implement an instruction until I ran into my “not yet implemented!” exception. Same goes often for other things that I implement. It is a bit of an extreme example of YAGNE, I suppose.

Update (11/30/2007): Added some words I have left out.

November 17, 2007

iPhone Native Look and Feel Wikipedia

Filed under: coding, daily life, iphone, technology — Tags: , , , , — Josh DeWald @ 4:28 pm

As I mentioned before, I used the “schools edition” of Wikipedia and stored it on my iPod Touch (will work on the iPhone as well) for some hot off-line Wikipedia action. What quickly became apparent is that the set of subject index pages that come with it are actually kind of hard to navigate (small text, etc). I figured that somebody had made some set of style sheets that made web apps look like native iPhone apps, and of course someone did.

So what I did was make a basic perl script would crawl through the subject index pages (looking for “categoryname” div tags) and generate an html file that had the same look and feel. Currently all of the index goes into a single page, which gets you the “wipe” effect, but it could just as easily be put into different files. I had to modify the javascript slightly to allow it to link into the actual content pages (the default javascript assumes that all the links are internal to the file… either that or I just couldn’t figure it out). Basically, if an “a” tag has a class of “external” then it’s… external.

So if you grab iphonenav.js from the link above, and modify the click eventListener like so:

addEventListener(“click”, function(event)
{

var link = event.target;

if (link.className.indexOf(“external”) == -1){
event.preventDefault();

while (link && link.localName.toLowerCase() != “a”)
link = link.parentNode;

if (link && link.hash)
{
var page = document.getElementById(link.hash.substr(1));
showPage(page);
}
}
}, true);

You can download the perl script from my site. Just run it against “wikipedia/wp/index/subject.html”. Save the script with a .pl extension. My host seems to be trying to interpret the script, even if I have the extension be “.pl.txt”.

You can see what the end result looks like here. By the way… you should be able to just take that html file and drop it next to the main index page for the Wikipedia schools edition.

As usual with this type of thing, I’m sure people will look at the script and think “what are you doing!?” But it seems to do the job.

November 12, 2007

My very own Hitchhiker’s Guide

Filed under: daily life, technology, uk life — Tags: , , , , — Josh DeWald @ 9:32 am

So I recently bought one of the new iPod Touches (basically, an iPhone without the GSM radio, camera or mic).

It’s cool that it plays music and all, but that’s not what I bought it for. I really was impressed by the way the browser worked and that essentially full-fledged Cocoa applications could be installed onto it.

What I really got it for was so that I can have my dream of an internet-enabled hand-held encyclopedia. I want The Hitchhiker’s Guide to the Galaxy. And now I’m that much closer.

It turns out that there is a “school edition” of Wikipedia, which is a static dump of Wikipedia content that’s been slightly edited and chosen for entries that “appeal to children.” Don’t let that dissuade you, it has an entry on MKULTRA. In their words:

It is about the size of a fifteen volume encyclopaedia (24,000 images and 14 million words), and includes essentially all topics in Wikipedia rated “Good” or higher by Wikipedia itself at date of production. Volunteers at Wikipedia have also suggested more than a thousand other articles considered valuable and relevant to this project which SOS Children volunteers have checked and tidied up.

They have a 750MB download available at http://www.soschildrensvillages.org.uk/charity-news/wikipedia-for-schools.htm. Once you de-compress it, you’re left with the exact same thing as what is at the Schools’ version site. Until we truly have ubiquitous, and free, Wi-Fi access (including during flights), this will be an awesome set of information to have on hand.

Unfortunately, Safari on the iPod/iPhone does not support viewing of local files directly with a “file:///” protocol. It reports “Cannot view the page because it is a local file.” It’s a good thing I had already “jailbroken” my iPod, so I just had to install the ‘lighttpd’ web server.

After copying the files to my iPod at /var/root/Sites using WinSCP (you can use any scp program though) into a ‘wikipedia’ directory, I was able to browse to http://localhost/wikipedia on my phone. And voila!

I assume this all works on the iPhone as well as the iPod, as most of the apps are actually meant for the iPhone.

I know this isn’t what Apple intended, yet (they are coming out with the official SDK in February), but if I wasn’t able to do something like this I would not be as pleased with my purchase. I don’t care if there are actually tons of other ways to have a hand-held encyclopedia, how many of them are also a music-playing, mail-synchronizing, calendaring web browser?

October 3, 2007

Design Smart, Code Stupid

Filed under: coding, technology — Tags: , — Josh DeWald @ 11:42 am

“Design Smart, Code Stupid” is just another way of saying “Don’t be clever” at the code level. Remember that as soon as you write some code, you are now its primary maintainer. If the code base is a hobby project, you may be the only maintainer, so you owe it to yourself to make your code easy to read. If its not a hobby project, it becomes even more important (if only to save face!); the last thing you want is other developer’s looking at your code and thinking “what does this line do?!” Instead, another developer should look at your code and think, “this makes total sense, anybody could write this!” A little self-test is to revisit your code a week or two after you wrote it. If it confuses you and produces a lot of “wtf?” then you need to simplify it.

Do you really need to use a ternary operator? Is it honestly easier to do a bit mask and a bit shift in the same statement? You may want to reconsider that nested a.getB().getC().getD().doSomething() you have going there.

So that’s the “code stupid” part… kinda starting backwards here.

So every line should be obvious, but each one should also go toward solving a problem (why else is it there anyhow). As you take the view higher and higher through the system, it should continue to make just as much sense. Each method should be well-named so that a person does not even need to look at it to know what the purpose is: the calling of a method should be as obvious as a built-in language construct. At the class or module level, it should be obvious what role it serves in the big picture. Multiple modules and classes should be “packaged” together as a cohesive unit that solves some problem.

Essentially, at whatever level a person looks, it should be obvious what what the system is doing, and why it is being done in the way chosen (even if there are alternative methods).

And you know what, another developer will look at the design and think “Man, that’s smart!” Because it is not easy to do for any non-trivial problem.

Because you know what, if you don’t have a good view of the big picture design of your system, then neither will anybody else. Instead, you will have a hodge podge of classes and methods that sort of work together. You need to know what it is you are building before you build it. Always think about the Use Cases!

I have no idea if Steve McCcconell uses the phrase in “Code Complete.” I would not be surprised, as entire book is really devoted to the idea of building a system that makes sense at every level. So I know that I am havingaa completely un-original thought, and I’ll freely admit it. I have no doubt read it so many times that its worked its way into me into some sort of meme. Nonetheless, I just thought I’d talk about it for a bit. I try to live it and know that I screw it up all the time.

Update: Fixed the sentence on Code Complete, realized that I had left out like 4 words 😉

Older Posts »

Blog at WordPress.com.