How a semantic representation is represented in human brain is an important problem in cognitive psychology. Although some previous studies suggested that semantic processing is related to sensory or motor processes, they have not shown clear evidence that the sensory and motor processes contribute to semantic processing per se. In the present study, I examined the effects of visual and motor dual tasks on the semantic processing of animal names, tool names, and abstract words. In order to surely evoke interference effect onto each visual and motor modal system, the dynamic visual noise presentation is used as the visual interference task, and the repetitive tapping task is used as the motor interference task. In Experiment 1, twelve participants performed word semantic judgment task under visual, motor, and no interference (control) conditions. As a result, there was no significant interference effect in any planned comparison. In Experiment 2, six participants completed the same task in which a foot-pedal was used as response device to avoid interference with repetitive tapping task. In latter experiment, a significant visual interference effect on the processing of animal name was found. This result suggests that the low level visual processing may contribute to the semantic processing of animal name.