Evaluating speech quality in large wireless networks: A case for hybrid simulation

In this paper we address the evaluation of speech quality through a wireless network as perceived by the user. User perceived evaluation (in addition to the usual network metrics including delay, throughput, packet loss statistics, etc.) is critical in the design of wireless multimedia networks where speech and video play a key role and are affected by several factors, such as network performance and encoding scheme. We use two approaches for our evaluation: small scale testbed implementation and hybrid simulation. The testbeds are implemented either by UDP (user datagram protocol) connection or UCLA talk connection. The hybrid simulation is event driven and is based on the Maisie parallel simulation language. It uses speech traces for inputs and plays back the speech outputs off-line. There are several advantages in using the simulation tool, mainly from the point of view of scalability and new protocol feature testing. At the same time, the real life testbed provides verification of the simulator assumptions, albeit on a small scale.