The recent finding that Broca's area, the motor center for speech, is activated during action observation lends support to the idea that human language may have evolved from neural substrates already involved in gesture recognition. Although fascinating, this hypothesis can be questioned because while observing actions of others we may evoke some internal, verbal description of the observed scene. Here we present fMRI evidence that the involvement of Broca's area during action observation is genuine. Observation of meaningful hand shadows resembling moving animals induces a bilateral activation of frontal language areas. This activation survives the subtraction of activation by semantically equivalent stimuli, as well as by meaningless hand movements. Our results demonstrate that Broca's area plays a role in interpreting actions of others. It might act as a motor-assembly system, which links and interprets motor sequences for both speech and hand gestures.