In this paper, we present the efficient VLSI architecture for the stand-alone application of the speech recognition system. With the analysis of the computation complexity, mel frequency cepstrum extraction and Bayesian neural network operations are the most time consuming computation tasks in the recognition algorithm. The specific recognition core to deal with them is proposed based on much algorithm improvement. The construction of the special logarithm look-up table saves the computation time and drastically reduces the memory size. Moreover, the cost efficient programmable architecture is designed for other non computation-intensive operations. The best aspects of both programmable and application specific architectures including the performance, design complexity, and flexibility are incorporated in the proposed VLSI speech recognizer.