In a comprehensive longitudinal study, the English communicative abilities of 102 Japanese university freshmen were analyzed in detail during one semester. Two teachers each taught two courses; in one course a textbook was used, and in the other course podcasts were used. There were several controlled variables, including starting language abilities and student majors. A series of communicative language tests were given at the beginning, middle, and end of the semester. Importantly, the experimental method used here partially mirrored an influential study published by Gilmore (2011), which found that utilizing “authentic materials”—which were mostly Web-based—seemed to be more effective than utilizing textbooks. Thus, the goal of the present study was to confirm Gilmore’s findings by determining which pedagogical materials were more effective in improving students’ oral abilities: textbooks or podcasts.
It was found that students progressed in similar ways, regardless of the pedagogical materials used. For example, students in all four classes improved their scores on listening tests by an average of about 10%, this improvement being statistically significant (p < .05). And, on average, students in none of the classes noticeably improved their vocabulary-grammar scores much. On audio recordings of students having conversations in English, there were remarkable improvements during the semester in all classes with respect to both fluency (number of words uttered during 3 minutes) and the correct pronunciation of /l/. But none of the classes, when taken as a whole, seemed to improve other spoken grammatical trouble spots, such as use of grammatical articles or prepositions. The results seriously question Gilmore’s contention that Web-based materials are better than textbooks.