Interactive D3 Experiments

With my first week of sabbatical completed, I dug into interactive graphical web programming - which turned into a deep dive into D3, SVG, and JavaScript DOM and events. Go ahead and try my experiment:

Source

I don't know if I should be proud that I accomplished something, or horrified at how long it took and how complex modern interactive web programming is today (or perhaps how atrophied my programming brain has become). In any case, I wanted 100% control over the interaction, and I finally got what I wanted - but it was hard. I could have done this in Java or in iOS much, much, much faster (well, at least if I had a graph layout algorithm handy).

Anyway, here is a high-level summary of what I learned:

• Interactive Graphics Layer: HTML5 "canvas" v. "svg".  First exploration was to analyze canvas (pixel-based write-only renderer) v. SVG (DOM-based, animatable, eventable vector graphics). SVG is much closer to my beloved Pad++ & Piccolo days of yore - and scalable and interactive is what I need. So, SVG it is.
• JavaScript SVG Wrapper: SVG is ugly, and not meant for humans to write. So, I went looking for a library. I found 3:
• D3: Advanced. Written by the brilliant and experienced Mike Bostock. High level, focused on data bound visualizations. A slight impedance mismatch for what I am looking for - but in the end, the clear winner (more below)
• Raphael: Low-level and multi-platform compatible. Meaning it supports the least common denominator of features across all browsers (including IE's VML). This means no groups and no classes. No groups means it is very difficult to hang data off a collection of items in the DOM - and so you are obligated to create a shadow data structure in the app. No classes means no easy styling, and no generic event handlers.
• JQuery SVG Plugin: This is actually about the feature set I was initially looking for, but the tool just isn't sophisticated enough. The architecture isn't clean enough, the docs and examples aren't sufficient, and I didn't have confidence that this project has legs. Close though - and with some TLC, it could become great.
• D3: D3 is well known to have a high learning curve - and it does. There are a handful of issues, some of which could definitely be helped by more documentation. It took me about a solid week to really feel comfortable with it - although admittedly that included brushing up on my JavaScript and DOM knowledge. However, I bet I'm not that atypical here.  Anyway, here are a few issues I ran into:
• The basic data binding model (w/ "enter" and "exit" mechanisms for more and fewer bound data items) just takes a while to wrap your head around. The docs here are good - just new to me.
• The relationship between D3 selection and underlying objects took a while to get a handle on. When you get a selection, are those DOM objects, and if not, what are they? This should be spelled out really clearly in the docs.
• Guidance on use with JQuery. Turns out this wasn't hard, but I (like just about everyone I know) rely on JQuery - so what are the catches? When is it a good idea to use D3 v. JQuery binding? Are they interchangable? Can you even use JQuery to select SVG DOM elements? A section on this in the docs would be really helpful.
• Events bound to selections are *statically* bound. This is the same as JQuery, and I should have known better - but it really threw me for a loop. This means that when you bind an event to a class name, and then change the class name - the event handler doesn't get unbound. You have to manually unbind and then bind new event handlers.
• Knowing what is a DOM attribute and what is a CSS style is confusing. A summary table of supported SVG objects, attributes and styles would be really helpful.
• I kept on wanting to write event handlers that worked on only some elements of a selection, but couldn't figure out how.  For example, I would have a bunch of active nodes, and want to act on all of them except the one the mouse was over. With 20 years of iterative programming in my brain, I spent an inordinate amount of time before I realized that in this new world of selection-based programming, the right approach was to change "class" tags on the elements so just the things I wanted had the right class names - and then select on that class.
• Finally, the D3 API docs are pretty good, but badly need complete examples. Many have no examples, and hunting through the demos to see how something is actually used is very time consuming. And where there are examples, they are usually abstracted. That is fine for conceptual understanding, but I kept wishing they were linked to full, simple standalone examples.  For example, I spent 3 hours getting the syntax of a path right - and then figured out later I could have just created a line.  Sigh...

Well, there's more - but that's a start. Modern web graphics technology is amazing. I'm heading out of the country for a few weeks, but when I return, I expect to be able to start building cool stuff!

How to merge different Google Docs accounts

I regularly get requests from people I share Google Docs with to re-share with a different account. That is, I know their primary university email address, so I share with that, and then I get an email asking to share with their gmail account. It turns out that a largely hidden, but very useful feature exists that makes it possible to link multiple email addresses to a single Google Docs account so that which ever of those email addresses a doc is shared with, you will have access to it through your single primary account.

For example, my google acct id is @gmail.com, but I want docs shared with @cs.umd.edu to also be available there (rather than having 2 or more google docs silos). This does not affect your email or anything else - but rather is just a way to consolidate docs.

To set this up, log into google at http://docs.google.com (or google.com, etc.) and select Account Settings:



Then, add an email address by clicking on Edit under Personal Settings.



This will affect all new documents shared with you. If you have existing docs that you want to make available with your gmail account, you can:

• Login with your other account and share it with your gmail account
• Ask the person that originally shared the doc to re-share it with your gmail account

If you find your self with multiple google docs accounts that you want to merge (because people had shared docs with a university account), then you should:

• Save all docs from your extra accounts that you want (by exporting or sharing with your gmail acct)
• Delete that account
• Go through the process above associating that email address with your gmail acct.

Hopefully that is more helpful than confusing...

QuicKeys - fix bugs, speed things up

After, literally, years of griping about various annoyances of my beloved Mac, I finally found a way work around for many of them.

QuicKeys is one of the oldest end-user customization tools (for PC and Mac). I've known about it for years, but finally started to put it to use because I was just so annoyed by a few minor things that I kept running into over and over again. Here are a few of the (admittedly minor) Mac annoyances that are now fixed!

iCal: Pressing Shift-Command-T brings up a dialog to go to another date. But there is a bug that makes it impossible to use the keyboard to either activate the dialog (to actually go the date you've specified) or dismiss the dialog. I wrote two simple shortcuts so that pressing return activates the dialog and pressing escape dismisses it. Shame on Apple for making me do this after years of iCal's existence, but anyway now I can use iCal without cursing Apple 5 times a day.

iPhone Simulator: When debugging code with XCode, you press Shift-Command-Return to stop the process. But if the focus happens to be on iPhone Simulator (as it often is when debugging an iPhone app), you have to press Shift-Command-H to stop the process. This is typical Apple keyboard shortcuts. Each one makes some sense on its own, but they don't work together (on iPhone, you pressing the "Home" button, thus the "H"). So I added a shortcut for iPhone so that Shift-Command-Return generates a Shift-Command-H to stop the process. Problem solved.

WriteRoom: The control-A and control-E keys move the cursor to the beginning and end of the paragraph instead of the line, which they do in most other apps. I remapped them.

Mail: Having to use the cursor keys and delete key is adequate, but I do *SO MUCH* cursoring and deleting that moving hands off the home keys turns out to be a significant burden. So, I made Control-N & Control-P move to next and previous emails and Control-D delete the current message. Ahhhh.

Firefox/Instapaper: I like to use Instapaper to read things offline later. The normal way to do this is to put a bookmarklet in your bookmark bar on the top of your browser. But I hate extra bars - especially on my laptop. So I started by making a shortcut to hide/show the bookmark bar so at least I don't have to dig around in the menu for it. Of course, then I realized I should just make a key to run Instapaper (which I'll do as soon as I finish this post).

Finder/Terminal: I regularly want to use Terminal to look at the contents of a directory I am looking at in finder. You can copy a directory from Finder, open Terminal, type "cd ", paste the directory, ... to get there. Instead, I made a single key that does that.

You get the picture. If there is a slow and consistent activity that you do frequently, it takes about 1 minute to write a quickeys shortcut. They can be applied narrowly so they only work for the app in question - and only when, say, a certain dialog is open. So they don't hurt anything else. I haven't observed it slowing down the system. The only issue is that you have to remember the new shortcut. But if the task you are speeding up really is one that you do frequently, then it isn't a problem. If you don't do it frequently, then don't bother speeding it up.

New Windows laptop rant

From a very experience friend of mine who knows Windows and Mac very well, has worked at Microsoft as a software engineer for several years. He prefers to remain anonymous. Sadly, my experience is about the same, and I am very happy living on a Mac (sometimes using windows in a VM, and sometimes not):

-----------
[Company] bought me an HP Envy 17 windows laptop because I realized that I couldn't take my primary windows development machine (a mac pro) to meetings and demos. It was only $1200, so I figured it was worth it.

The specs were nice -- slightly slower core i7 than I have in my MBP, but 6gb ram built in and a 1920x1080 display. All the reviews I read basically said this was the closest you can get in build quality to a mac without buying something from Apple. A similar MBP 17 starts at $2k.

Holy crap, this thing blows. It _looks_ like a Mac, except it's made of plastic and flexes dangerously when held in one hand. They've shamelessly ripped off the apple packaging experience, except forgot the little things (like having the battery be pre-charged so you can use your computer right away, or designing a power brick that (a) doesn't look and weigh as much as a real brick and (b) plugs into the computer on the right side via a hard plastic connector that sticks out 2-3 inches from the machine and looks terrible).

Then you boot Windows.

The first time through, you get a bunch of HP popups that are a ripoff of the Apple FTUE, except the design is atrocious and in fact the screens don't really match from one to the next. Fine. You get through that, and 14 -- yes 14 -- different preinstalled crapware apps want to update themselves. I have the entire Corel suite on this machine.

You can't connect to an 802.1x wireless network via the HP Wireless network utility. This may be a limitation of the windows home premium that ships on this laptop, but the result is I have no in-office wireless. Fine. I plug in an ethernet cord, BUT THE ETHERNET DOES NOT WORK. It turns out that you need to get a Windows wizard to run that lets you choose if your new connection is a Home, Work, or Public network. I manage to get the wizard to run by Disabling the ethernet port and then Enabling it. I assume at this point that most people would have called HP by now.

I go into the control panel and change the name of the computer on the network, since 'pc-8994892x' wasn't something I could remember. Once I actually find the correct place in the Control Panel for this (don't get me started on the control panel having grown to nearly 100 applets in win 7), the machine wants to reboot. Fine, it's 2010, and Win7 has a brand new networking stack that was developed from the ground up for Vista, but we still haven't solved the problem of changing the hostname requiring a reboot.

So the machine reboots. The SECOND TIME USER EXPERIENCE is that the user DOES NOT BOOT WINDOWS. You boot some kind of crazy HP overlay OS that lets you browse the web or read email, I assume through some kind of Linux-based layer that "boots faster" (not really). I have to dig around to turn this thing off.

I am now convinced that the "Apple Tax" is incredibly worth paying for all computers, even if you intend only to run Windows. At least then you get a Windows without all the crapware.