The cross product of vectors is a fundamental notion in the vector analysis and the applications of mathematics. It is a peculiar one to the space vectors, and a sort of product with the alternating law and without the associative law. One purpose of the article is to describe such peculiarity of the cross product from the fundamental mathematical view point and consider a generalization of the cross product to the general dimensional vectors. Then, in the case of the plane vectors, the generalization corresponds to the vertical vector, and to explore the notion of the vertical vector as a teaching material is another ingredient of the article, where we describe some useful aspect of the notion showing several concrete examples.