In this research, a privacy protection mechanism in H.264/AVC videos is proposed. The sensitive or private visual information in frames, which should not be viewable by the general public or regular users, will be scrambled by directly modifying or removing the related data in H.264/AVC compressed bitstreams. In order to allow the authorized users to recover the partially scrambled video frames, the methodology of information hiding is employed; that is, the correct information is embedded and transmitted along with the video bitstream. After retrieving the data, the authorized users can descramble the protected areas in frames. Experimental results show that the partial scrambling can be achieved effectively and the size of the resulting video is kept under good control.