This paper presents a high speed search and memory efficient Huffman decoding for MPEG AAC audio decoder. First, the leading sign bit characteristic in AAC Huffman codewords is utilized. Based on the leading sign grouping, a two-level table lookup approach is introduced. It can branch to the correct group by first table and decode the symbols by second table. Two optimization approaches are developed, one is to reduce the branch cycles and the other is to combine direct table lookup approach. Simulation results show that these two optimizations can totally save about 18.25% cycles further comparing to the non-optimized version and about 9x speed up comparing to the linear search approach. Besides, the codeword packing scheme is used that the 12 Huffman codebooks only consume 3.6 Kbytes.