A handheld device includes a gesture database maintaining a plurality of
gestures, each gesture defined by a motion of the device with respect to
a first position of the handheld device. The gestures comprise a
plurality of remote command gestures and at least one device selection
gesture. The device includes a gesture mapping database comprising a
plurality of command maps, each of the command maps corresponding to a
particular controllable device and mapping at least one of the remote
command gestures to a command for controlling operation of the particular
controllable device. The device also includes a motion detection module
operable to detect motion of the handheld device within three dimensions
and a device selection module operable to detect the device selection
gesture based on the motion of the handheld device and to select a
currently controlled one of the controllable devices in response to the
device selection gesture. The device includes a control module operable
to select one of the command maps corresponding to the currently
controlled controllable device, to track movement of the handheld device
using the motion detection module, to compare the tracked movement
against the remote command gestures to determine a matching gesture, and
to identify, using the selected command map, the command mapped to the
matching gesture. The device also includes a wireless interface operable
to transmit the identified command to a remote receiver for delivery to
the currently controlled controllable device.