Previous research shows that there are two architectures for speechto-speech translation (S2ST) system implementation. One is client-server based systems that should be built on the server computer but not available anytime or anywhere. The other is to build portable stand-alone devices but lacks the real-time performance. Therefore, this work presents an embedded system design for portable S2ST applications. This system is characterized by small size, low cost, real-time operation, and high portability. For realization of the proposed S2ST system, this work designs the ARM-based SoPC architecture, the speech translation intellectual property, and software procedures of the proposed SoPC. The entire design was implemented on ALTERA EPXA10. The English-to-Mandarin translation process can be completed within 0.5 second at a 40 MHz clock frequency with 1,200 translation patterns. The maximum frequency is 46.22 MHz, and the usage of logic elements is 19,318 (50% of the total logic elements of the EPXA10 device).