The DDVPC Test & Evaluation Committee

TEC Members Return to DDVPC HomePage

Tandeming: - Tandeming occurs whenever the output of one voice coder is used as input for a second coder. Tandems with low data rate coders lead to significant losses in intelligibility and voice quality. The DDVPC has prepared an article on this issue: Effects of Tandeming on Voice Quality and Intelligibility.

Introduction

Although market forces currently drive large efforts in digital voice communications research and implementation, the unique needs of the Department of Defense (DoD) cannot always be met using commercial standards. DoD requirements generally call for operation at lower bit rates and in harsher environments than commercial coders. DoD communications systems must be able to interoperate and perform well over a variety of platforms, media, and across varied intermediate networks. The Test and Evaluation Committee (TEC) is made up of representatives from the Army, Navy, Air Force and National Security Agency. It has produced reports concerning voice processing and recommended several digital voice processor algorithms over the past 20 years. The TEC has amassed an in-depth knowledge of the test methods available for performance evaluation of communication systems and has successfully managed multi-candidate performance competitions. The DDVPC continues to provide this capability to the DoD community. The following section on narrowband digital speech performance test methodologies describes the test methods currently used by the DDVPC. In addition, government agencies may request from the DDVPC prerecorded audio input materials for these various test methods. The DDVPC and the TEC welcome input from the DoD communications community concerning the present and future of secure, narrowband, digital voice systems.

Test Methods for Performance Evaluation of Narrowband Digital Voice Systems

In order to insure a voice coding algorithm will meet the requirements of its intended users, the test plan must be multi-dimensional in design. During its latest major evaluation that resulted in the new 2.4kbps Federal Standard MELP, the TEC identified four facets of the algorithms' voice performance to evaluate:

Intelligibility - The degree to which the transmitted speech is interpreted and, therefore, understood as the talker intended. The Diagnostic Rhyme Test (DRT) is the method used for evaluation of intelligibility.
Quality - A measure of the personal opinion of a group of listeners in regards to the processed speech of a system or algorithm. The Mean Opinion Score (MOS) and the Degraded Mean Opinion Score (DMOS) measure voice quality. The Diagnostic Acceptability Measure (DAM) test method has also been used by the DDVPC. A comparison of the MOS/DMOS and DAM test methods is available.
Recognizability - The degree to which a processed voice no longer sounds like the talker. A new test method was developed at the Naval Research Laboratory (NRL) specifically for evaluating the talker recognizability of narrowband voice systems.
Communicability - This is a rating of user acceptance of a system in a conversational setting under adverse conditions. A new test method was developed at the Air Force Rome Laboratory ERC-1 Office by ARCON Corporation specifically for evaluating the communicability of narrowband voice systems.

In addition to voice performance considerations, hardware parameters such as processing power, memory usage, and throughput delay must also be considered if the algorithm is going to meet the needs of the DoD.

Ongoing Initiatives

The TEC is currently involved in the following efforts to provide the DoD community with high performance, secure, digital voice communications:

The New U.S. Federal Standard for 2.4kbps Voice Coding
After analyzing the performance and complexity measures of seven candidate algorithms, the Mixed Excitation Linear Prediction (MELP) algorithm was selected by the TEC as the recommended new federal standard algorithm for 2400bps voice communications. The MELP algorithm was jointly developed by Texas Instruments, Atlanta Signal Processors, Inc. and the National Security Agency.
The Standard - The federal standard document is currently being written. The review process will begin immediately thereafter. The expected publication by the General Services Administration (GSA) will be early 1998.
The Fixed Point Implementation -A performance verified, fixed point prototype of the MELP standard is scheduled to be available before January 1998.

New Concepts of Interest

The TEC is always considering the DoD's present and future communications needs. The following fields of research are currently under consideration as methods to provide for those needs.

Multi-rate Systems - Source independent rates from 9.6kbps to 1.2kbps.
7.2 and 9.6kbps Algorithms - With the deployment of the Secure Terminal Equipment (STE) and the current rate capacity of the public switched telephone network (PSTN), 7.2 and 9.6kbps interoperable standards are needed.
Improved 4.8kbps Algorithm - Improved performance while maintaining standard compatibility.
Robust Very Low Rate Algorithm - Interoperability in adverse communications environments.
Algorithms for Packet Networks - Packetized voice algorithms for transmission over existing data networks. Specifically designed or adapted for DoD communications needs. In conjunction, a study concerning the effects of mixed channels on current voice compression algorithms.

This website is sponsored by ARCON Corporation for the DDVPC and as such must conform to DoD policies concerning websites. Please read this disclaimer for more information.

Comments concerning design and content of these pages should be sent to info@arcon.com. This page was last updated on 09-Jan-2002.