This study investigated whether Chinese and Japanese speakers rely on phonological and orthographic codes differently in a verbal serial recall task. Twenty-four native Chinese speakers and 24 native Japanese speakers participated in the task, which involved single-kanji words. The phonological and orthographic similarity of the set of kanji to be remembered was classified as follows: (1) phonologically similar and orthographically dissimilar; (2) phonologically dissimilar and orthographically similar; and (3) phonologically and orthographically dissimilar. The cognitive load was manipulated by increasing the length of the list of words to be remembered from 4 to 8 words. A clear effect of phonological similarity was observed in native Chinese speakers’ recall performance. That is, Chinese speakers relied heavily on phonological information in their maintenance of kanji words. Although the impact of phonological similarity was significant among Japanese speakers, the decrease was small. In addition, there was no significant difference in recall performance between phonologically and orthographically similar conditions. That is, Japanese speakers seemed not to rely heavily on phonology in the maintenance of kanji words. They also used orthographic information. This study directly compared native Chinese and Japanese speakers and found clear differences in the processing of kanji in the two languages.