Rocketpanda

So I seem to have become possessed by the alternative UI demons of late.

For some time now I’ve been following Johhny Chung Lee’s UI investigations into the potential of the wiimote among other things. The most accessible of these investgations for me was the wiimote whiteboard concept. If you’ve not read about it the premise is fairly simple. A wiimote is one of the coolest pieces of integrated tech on the market today. With three accelerometers, Bluetooth and an infra red (IR) camera that can track up to four points simultaneously it’s a steal of a device for £30, even without the Wii to go with it. With this cool device then, you can Bluetooth connect it to a pc and have access to the IR and accelerometer output. In the case of the wiimote whiteboard (available on Johnny’s site link above) the idea is to inverse what the wiimote’s normal use is and instead of waggling it in front of a stationary IR bar you use it as a stationary camera to track the movement of an IR source. In our case that IR source is a custom made pen that has a push button IR LED which the camera tracks and can emulate through the software as a mouse.

So knowing the concept I’ve been keen to try for some time but only recently have I acquired an IR pen and brought together all the constituent parts. After eBaying some IR pens, I borrowed the time of our IT department and our resident Wii and projector. A very short time later we had the wiimote whiteboard software set up with one wiimote tracked on the projection of the test laptop on a nicely blank wall. IR pen at the ready we went through the calibration process and fired up my colleague Gavin’s blog to try out his Silverlight painting test app.

The whole experience is really, really nice to use

It’s mostly pixel accurate if you’ve calibrated it well and for chunkier ui apps and drawing based tasks it is infinitely more usable than a mouse in a large-scale wall-presentation style environment.

There are a couple of small issues that the software can also help you get around. The wiimotebeing a camera in this instance means that the camera is looking at the surface you’re drawing. At some point you’re going to get in front of it meaning that it will stop drawing. Fortunately you can hook up more than one wiimote for redundancy which means if you get in the way of one wiimote the other is still tracking. We hooked up three and it makes a massive difference. I have to say though that the enhancement you get from connecting two is much bigger than it is for adding a third.

Our other issue was that as we were using a projector we were also walking in front of the display, so we want to try a rear projection set up which will eliminate this completely. Along the same line we could also use a rear projection setup for the wiimote though the surface we’d project on would need to allow th IR to show through.

All in all it works great and is really fast to set up. I bought a second hand wiimote on the weekend and hooked it up to my tv at home which took all of 10 minutes. If you try this out yourself please test it with Kloonigames’ Crayon Physics Deluxe demo, it’s a perfect example of what this tech does well and you won’t be disappointed.

After all the fun and excitement the last item we’re now interested in is as the wiimote can track up to four points at once it is possible for it to do multi touch as well. There is source code available and although it needs a bit more work to be a non developer project, believe me when I say it’s at the top of my list to get a multitouch wall surface created!

Happy wiimote whiteboarding!

So you know that Silverlight 3 supports pixel shaders right? Great! You are filled with joy, nay even rapture. But how to actually start using them?

From a designer’s point of view you can web search what’s available and all the juicy examples but there are a few hurdles before you get your crayon-crusted mitts on them.

Things you should have baked earlier:

• Visual Studio with service pack 1
• Silverlight 3 tools for visual studio (including SL3 sdk, SL3 developer plugin)
• Blend 3 release candidate
• WPF pixel shader library
• A new Silverlight 3 application project for testing

My way to victory was starting with the WPF pixel shader effects currently published on codeplex that use a workaround to get them running in SL3.

Download and extract the WPF shader library, open up the solution and have a look at the project structure, taking note of the SLShaderEffectLibrary folder… we’re stealing that one very shortly.

Then open up Visual Studio (resist the urge to go visual in Blend just yet) and create a new Silverlight application.

Next right click on the solution and click ‘Add > Existing Project…’ and then hunt for the SLShaderEffectsLibrary folder we found earlier from inside the wpf effects toolkit.

With that in place we need to add a reference from our base project so that it knows about all those shiny effects. Right click on the references folder in the base project and in the popup window choose the projects tab.

Then choose the imported project that appears in the list.

Finally, in your mainpage.xaml file we need to add an import statement to the added project; using ShaderEffectLibrary will do it.

Now that our base project knows of the magic that will shortly be, probably gratuitously, gracing your screen. With no delay, open blend with shaky sweaty fingers and slap down something interesting like an image onto mainpage.xaml. Then click new effect and in the popup window notice that instead of the two effects available as standard the project you added now appears in the list …

…and opening the list reveals the full list of effects to play with!

As a caveat I have noticed that some of the effects simply don’t work, though happily these are in the minority.

Also take note that while some effects just like the standard blur and dropshadow have exposed properties for you to play with variations of each effect, others are as is and don’t expose any properties at all.

So!
Now you’ve got some playing to do here’s some points to muse on:

• any exposed effect property can be animated
• only one effect instance can be attached per xaml object (canvas, image, textblock for eg)…
• BUT you can stack effects by adding effects to a parent container
• keep in mind that these effects are doing some fairly serious calculations, if you can get away with one instance of an effect on a parent container rather than 10 instances of the effect on each object in that container, that’s probably the way you want to go

With this new effect playground it provides a framework for adding new effects you find (or make) into this library with only minor wrangling. I’ll cover this approach in a future post, so for now happy shading and I hope this provides some image effect joy!