Advertisement

The Leap Motion Controller and Mac: Part 3

by

This Cyber Monday Tuts+ courses will be reduced to just $3 (usually $15). Don't miss out.

In the two previous tutorials I introduced you to the Leap Motion Controller and showed you how to get started with it and I showed you how to use the Leap Motion Controller and BetterTouchTool to add custom gestures to your workflows.

This tutorial is more experimental. I’ll show you how to chain actions in BetterTouchTool and integrate it with other applications to explore what is possible with the Leap Motion Controller.

Prerequisites

This tutorial builds on things I have shown in three previous tutorials. Before starting this tutorial it is best if you’ve read:

I won’t repeat the information in those tutorials in detail so if, at any point, you are lost, please refer back to them.

As with previous tutorials, I'm providing my custom gestures as a preset. 

Chaining Actions in BetterTouchTool

Chaining actions with BetterTouchTool lets you trigger multiple events with a single gesture. For example, you can open an application and then trigger a keyboard shortcut with a single hand motion. The actions trigger in sequence and the next action triggers immediately after the previous one completes.

You can create an action sequence that lets you search Google with your voice from anywhere on your Mac without touching your keyboard. Creating this action sequence will give you the tips and tricks you need to create your own by modifying the preset I provide or starting from scratch. I’m also going to suggest a few ideas for other series of actions and gestures you can create on your own.

Keyboard-less Googling

It is possible to use a series of Leap Motion Controller gestures to open Google Chrome, open a new tab, activate the omnibar, trigger dictation and submit the resulting text as a Google search.

  • Open the BetterTouchTools preference pane and click on Configure New Leap Motion Gesture. Select a gesture from the dropdown menu; as I mentioned in the previous tutorial, I like five finger gestures so I selected Five Finger Right
  • The first action is assigned as normal. You need to use the predefined action Open Application so select it from the Controlling Other Applications in the Predefined Action dropdown menu. In the Finder window that appears, navigate to the Mac’s Applications folder and select Google Chrome
  • To add a second action, click on the Attach Additional Action tab. It is right next to the New Leap Motion Gesture option
Adding a second action.
  • This additional action is configured just like a typical BetterTouchTool gesture. You need to tell Google Chrome to open a new tab so enter Chrome’s new tab keyboard shortcut, Command-T in the Custom Keyboard Shortcut input
  • Add a third action and assign it the keyboard shortcut Command-L; this is Chrome’s shortcut for highlighting the omnibar
  • Before adding a fourth action, you need to configure Dictation. Dictation’s default keyboard shortcut, Function-Function can’t be triggered using BetterTouchTool
  • Navigate to the Dictation & Speech preference pane in System Preferences. In the Shortcut dropdown menu select Customize… and enter a new keyboard shortcut. I used Command-Option-Control-Shift-Fn-F12 as I knew I would never actually have to press it and I’d be able to trigger it with a Leap Motion Gesture in the future.
The Dictation & Speech preference pane.
  • Return to the BetterTouchTool preferences and add a fourth action. Assign it to the new Dictation keyboard shortcut

Now is a good time to test your work to make sure everything runs as planned. Perform the Leap Motion Gesture you configured. Chrome should activate, a new tab be created and Dictation triggered. Dictation also allows you to dictate long documents so it does not stop automatically. For that we will need an additional Leap Motion Controller gesture.

  • Add a new Leap Motion Controller gesture in BetterTouchTools. I assigned it to Five Finger Left so that it would pair nicely with the first gesture. Stopping Dictation uses the same keyboard shortcut as starting it so assign your Dictation shortcut once again.
  • Unless you have downloaded the 500MB add on, Dictation takes a second or two to process your input. As BetterTouchTool triggers actions in quick succession, adding an additional action interferes with this slight processing delay, so you need to add a third gesture to submit the resulting text as a Google search.
  • Add this third gesture and assign it the keyboard shortcut Return. I used the gesture Five Finger Down to keep things consistent.
The finished series of gestures and actions.

You’re all set! To perform keyboard-less Google search perform the first gesture, if you followed along exactly, Five Finger Right. When Dictation activates speak what you want to search for and then do the second gesture, in my case Five Finger Left. Once Dictation returns the results do the third gesture–Five Finger Down for me–and the Google results will appear.

As a side effect of this set up, you now also have a custom Dictation gesture. If you want to dictate an email or text document, the same gesture that stops Dictation will start it.

Other Ideas

What you can do with chained actions in BetterTouchTool is pretty much limitless. If you can imagine it, more than likely you can figure out a way to make it possible using an action sequence. While not all your ideas may not work as well as the keyboard-less search I showed you above, it is a fun area to experiment with. Below are a few of the ideas I’ve experimented with that you might like to try as well.

It’s possible to use BetterTouchTool to open more than one application. I have experimented with using a gesture to open two or three different apps and position them in specific places on my screen. 

A three finger swipe in one direction could configure the workspace for writing by opening a browser window and a writing app side by side while a three finger swipe in a different direction could get you set for photo editing by opening Adobe Photoshop Lightroom and hiding the Dock.

Another idea I’ve experimented with is creating an action sequence that tweets a link. A single gesture can highlight the omnibar, copy the link address, activate your twitter app, compose a new tweet, paste in the link, move the input cursor to the start and activate dictation. 

As with the keyboard-less search, you could then use two additional gestures to submit the dictated input and send the tweet.

Finally, BetterTouchTool can also trigger Automator workflows. Some of my colleagues here at Tuts+ have shown you how to use Automator so you can even integrate what they show into your own custom action sequences.

Conclusion

In this tutorial I’ve shown you how to create an action sequence in BetterTouchTool that gives you the ability to search Google from anywhere on your Mac without using the keyboard. This action sequence illustrated the tips and tricks to creating your own complex series of actions that you can trigger with a Leap Motion Controller. I also gave a few suggestions of other ideas that might be fun to experimented with. 

Experimenting with the Leap Motion Controller and BetterTouchTool is really playing around with the leading edge of computing–you’re using a novel input device and the latest features from Apple and third parties to interact with your Mac in a new way. I suspect that in the next ten years some of the things that you are able to create on your own now with the tools in this tutorial will become standard in computers.

If you have any great ideas for action sequences using the Leap Motion Controller and BetterTouchTool, or any other crazy futuristic ideas, please let me know in the comments.


Advertisement