We report on the electrical resistivity, thermal expansion, and x-ray diffraction measurements of single-crystalline sample of the so-called Kondo semiconductor CeRhAs under pressures up to 3 GPa. This compound undergoes successive structural phase transitions at T1=360, T2=235, and T3=165 K at ambient pressure. On cooling below T1, the crystal structure changes from the hexagonal LiGaGe-type to the orthorhombic ε-TiNiSi-type with a 2b × 2c superlattice. By applying pressure up to 1.5 GPa, T1 increases with a ratio of 270 K/GPa, whereas both T2 and T3 decrease with -100 K/GPa. The concurrent decrease of both the a parameter and the energy gap along the a axis with increasing pressure contradict the c-f hybridization gap model in which the gap is enlarged by the enhancement of hybridization between the 4f electrons and conduction band. Instead, a sort of charge-density-wave transition at T I is proposed for the origin of gap formation of this compound. The semiconducting behavior in the resistivity vanishes when the phase with the 2b × 2c superlattice decomposes into two orthorhombic phases below 100 K and above 1.5 GPa.