In this paper, we proposed a low complexity architecture design for Psycho-Acoustic Model (PAM). PAM is key component of MPEG-2/4 Advanced Audio Coding (AAC) Encoder. It occupies heavy computation load in AAC encoder and makes the AAC encoder hard to be implemented on portable devices for real-time condition. In order to conquer questions describe above, we propose a MDCT-based PAM algorithm and its dedicated architecture to accelerate PAM calculation. The main advantage of MDCT-Based PAM is the filterbanks in AAC can be reduced form three to two. Furthermore, we use look-up table method to replace computation of spreading-function. Second, the Logarithmic Number System (LNS) is used to reduce computation load of many special functions and data word-length. In the hardware architecture design, the pipeline is used to increase throughput of the PAM. Besides, a logarithmic unit that converts data into log scale at one cycle uses in our design. The proposed PAM architecture is implemented in UMC 0.18 CMOS technology. The total gate count is 69476.