大規模継続学生調査の可能性と課題

Universities and colleges in Japan have experienced sweeping and sudden changes during the past ten years. Many factors have contributed to the changes: universalization of HE, obligatory accreditation systems, and so on. For example, "quality assurance" has rushed us into reform of the curriculum, program, and pedagogy. Many universities and colleges have now put an emphasis on teaching and learning rather than research. So, what is happening in Japanese higher education and what are the impacts of the emergence of the accountability issue and the progression of massification? One of the new trends is exemplified as the rising movement for learning outcome assessment. However, in Japan, we have little historical background to assess learning outcomes and students' growth through standardized tests and student self-reported surveys.

Our research group has developed a student surveys systems for learning improvement, the so-called JCIRP (Japanese Cooperative Institutional Research Program) since 2004. Our program consists of three student surveys, the Japanese Freshman Survey (JFS), the Japanese College Student Survey (JCSS), and the Japanese Junior College Student Survey (JJCSS). Up to 2010, more than 54,000 students from 310 four-year and two-year colleges have participated in our program. It seems that our student survey system has institutionalized a culture for assessment of student learning outcomes based on the self-reported data that has been widely accepted in Japanese higher education institutions as an indirect assessment system.

The learning outcomes issue can be approached through diversified perspectives. Learning outcomes can be examined as knowledge acquired through university education. It can also be examined by measurement of outcomes assessment tools. Definition of learning outcomes is wide and complicated. It covers the field of general education, academic disciplines and the method of assessment. In the United States, various tests have been developed in order to assess the learning outcomes of general education, of the academic disciplines and of high-school learning. However, assessment toward students' learning is not well developed in Japan.

In the United States, the Spellings' Commission on the Future of Higher Education issued a report that considered introduction of national tests for assessment. However, as is mentioned by Banta, the context for considering a national test for college students was actually much fuller in 1990 than in 2006, and approaches for designing the test was more careful and deliberate (Banta, 2007). Various arguments about the assessment issue have been developed and research on assessment has accumulated. In other words, many researchers have engaged in measurement, testing and assessment study and thus have developed various tools for assessing cognitive skills, learning outcomes and the affective aspects of students.

In contrast, there is little research on assessment in Japan. Thus, it is not easy to define what assessment is and what the assessment of students' learning is in Japanese higher education. However, it seems that there are two kinds of outcome assessment. These are divided into direct assessment and indirect assessment. Direct assessment is assessment of the direct learning outcome of students through tests, paper, portfolio, graduation examination, graduation research paper or standardized tests in the fields of general and disciplinebased education. Indirect assessment can be utilized to assess the learning process through student surveys asking about learning behavior, student experience, self-perception and satisfaction. These surveys are practiced when students are entered the college, at the end of the first-year, at senior-year level and after graduation. Direct assessment and indirect assessment can be used to complement each other. Thus, our JCIRP is expected to function as the indirect assessment for student learning outcomes.

This paper first demonstrates how we have tackled the development of student survey systems in Japan, what problems are to be focused on in terms of assessment of learning outcomes and what we have to do in the next stage for development of our survey system. Second, by using the JFS2008 data, the method used to explore the complex structure of college student analysis is indicated, that is, by analysis of JFS2008 data by a multi-level model. Multi-level model analysis is identified as a mixed-effects model or Hierarchical Linear Model (HLM). This model can be applied to nested or hierarchical data in which individuals belong to various types of groups. In this case, college students belong to different gender groups, different school year groups, and various colleges at different levels, and so on. Using multi-level model analysis, the study shows that college student satisfaction is caused by personal factors rather than college factors.