This purpose of present study was to develop an evaluation scale in order to study how native speakers evaluate the speech and behavior of Japanese learners. To confirm the reliability and the validity of this scale, I carried out a questionnaire survey by viewing how 83 native speakers responded these items after they saw a videotape recording of a learner's free conversation with a native speaker. The questionnaires contain of 55 items (34 items for learners' competence evaluated by native speakers and 21 items for overall evaluation) were selected from the former related studies and the result of follow-up interview to 50 native speakers. In this analysis, 5 factors for learners' competence evaluated by native speakers and 3 factors for overall evaluation were extracted from the factor analysis first. The reliability of the scale was confirmed by the Good-Poor analysis and Cronbach's coefficient alpha. Furthermore, I propose the hypothesis of causal relation model of native speakers' evaluation based on Peason's product-moment correlation coefficient and the multiple regression analysis. As a result, the reliability and the validity of the scale are proved. Arid from my hypothesis of the causal relation model of native speakers' evaluation, it was found that 1) 'Expressiveness', 'interlocution competence', 'Listener-friendliness and comprehensibility' and 'vocabulary and expression' influence the evaluation of 'Japanese oral proficiency'. 2) 'Communication strategies' influences the evaluation of 'approachability'. 3) 'Communication strategies' and 'expressiveness' influence the evaluation of 'positive attitude'.