(check out the video from about 30 seconds in case this post gets a bit dry and drifting)
I find it funny that programmers still generate code by tapping buttons on a keyboard. Code, unlike normal prose, is highly structured with a relatively tiny lexicon. We make up notation like curly braces and semicolons to show this structure to the computer. To help ourselves, we graphically highlight the structure using tricks like indentation and syntactic colouring. Code editors also do their bit with tab-completion to select entities and block folding to hide structural complexity away. Spelling mistakes, dodgy formatting and syntax errors inevitably creep in nevertheless. Is there a better way?
When you recognise code is effectively built rather than typed then graphical environments are the natural way to go. Far beyond representations such as the Block Editor in Google Apps Inventor and Visual Studio Architecture views, I’d love to see the development of a fully-fledged, expressive and scalable software visualization model.
More than that, this representation prompts the use of a new interaction model based on physical gestures. Crack a method open to check it’s innards. Tug a variable to see where it is declared. Pinch for a breakpoint. Much of coding is actually exploring to see how it all fits together. That’s why I’m excited by demos of Kinect hacking like this:
The range of gestures that you can express with your whole body (or even just the upper part!) is far richer that you can get out of a keyboard and mouse. This ought to make interaction more efficient and, I suspect, satisfying. The ergonomic improvements over slouching in front of a keyboard and screen should not be underestimated.
Peering even further into the future, I look forward to the blooming of a sub-field of HCI based around physical gestures. I like to imagine it’ll converge on something resembling Tai Chi, combining efficient, expressive and intuitive movements that are also highly ergonomic. It wouldn’t even be surprising if different camps emerged, akin to the different martial art styles. Hopefully this will result in healthy, chilled-out geeks that happen look ultra cool in movies.
Code ninjas might even start looking the part.