Return to DDVPC HomePage
The following information is from a poster session given at The IEEE
International Conference on Acoustics, Speech, and Signal Processing (ICASSP
'97), Munich Germany, 1997.
A Comparison of the New 2400 BPS MELP Federal Standard
with other Standard Coders
M.A. Kohler
U.S. Department of Defense
Ft. Meade, MD
makohle@alpha.ncsc.mil
Introduction
Diagnostic Acceptability Measure (DAM)
Mean Opinion Score (MOS)
Degradation Mean
Opinion Score (DMOS)
Diagnostic Rhyme Test (DRT) -
Benign Conditions
Diagnostic Rhyme Test (DRT) - Harsh
Conditions
ARCON Communicability Exercise (ACE-95)
NRL Talker Recognizability Test (NTRT)
Complexity
Measures
Scores By Talker Gender
MOS Ranking
DMOS Ranking
DRT Ranking
ACE-95 Ranking
Diagnostic Information
References
Four Standard Coders Were Tested
- Continuously Variable Slope Delta Modulation (CVSD)
- Code Excited Linear Prediction (CELP)
- Mixed Excitation Linear Prediction (MELP)
- 2400 bps
- FIPS Publication 137
- Linear Predictive Coding (LPC)
Five Measurements
- Quality
- Diagnostic Acceptability Measure (DAM)
- Mean Opinion Score
- Intelligibility
- Diagnostic Rhyme Test (DRT)
- Communicability
- 1995 ARCON Communicability Exercise (ACE-95)
- Recognizability
- Naval Research Laboratory (NRL) Talker Recognizability Test (NTRT)
- Complexity
- Million Instructions Per Second (MIPS)
- Random Access Memory (RAM)
- Read Only Memory (ROM)
Top
of Presentation
Diagnostic Acceptability Measure
Two Conditions
- Quiet
- Anechoic Sound Chamber
- Dynamic Microphone
- Office
- Modern Office Environment
- Dynamic Microphone
Top
of Presentation
Mean Opinion Score
Six Conditions
- Quiet
- Anechoic Sound Chamber
- Dynamic Microphone
- Quiet - H250
- Anechoic Sound Chamber
- H250 Microphone
- 1% Random Bit Errors
- Anechoic Sound Chamber
- Dynamic Microphone
- 0.5% Random Block Errors
- Anechoic Sound Chamber
- Dynamic Microphone
- 50% Errors within a 35ms block
- Office
- Modern Office Environment
- Dynamic Microphone
- Mobile Command Environment
- Field Shelter
- EV M87 Microphone
Top
of Presentation
Degradation Mean Opinion Score
Five Conditions
- High Mobility Multi-Wheeled Vehicle
- Modern Jeep
- H250 Microphone
- E3A
- Airborne Warning and Control System (AWACS) Aircraft
- R215 Microphone
- Automobile
- Plymouth Reliant, Highway Speeds
- STU-III Microphone
- Single Tandem
- CVSD -> Coder
- Anechoic Chamber
- Dynamic Microphone
- Double Tandem
- CVSD -> Coder -> CVSD
- Anechoic Chamber
- Dynamic Microphone
Top
of Presentation
Diagnostic Rhyme Test
Seven Benign Conditions
- Quiet
- Anechoic Chamber
- Dynamic Microphone
- Quiet-H250
- Anechoic Chamber
- H250 Microphone
- 1% Random Bit Errors
- Anechoic Chamber
- Dynamic Microphone
- 0.5% Random Block Errors
- Anechoic Chamber
- Dynamic Microphone
- 50% Errors within a 35ms block
- P3C
- Orion Aircraft
- EV 985 Microphone
- Mobile Command Environment
- Field Shelter
- EV M87 Microphone
- Office
- Modern Office Environment
- Dynamic Microphone
Top
of Presentation
Diagnostic Rhyme Test
Eight Harsh Conditions
- Single Tandem
- Quiet->CVSD->Coder
- Dynamic Microphone
- Double Tandem
- Quiet->CVSD->Coder->CVSD
- Dynamic Microphone
- Automobile
- Plymouth Reliant, Highway Speeds
- STU-III Microphone
- M2 Tank
- Bradley Fighting Vehicle
- EV M138 Microphone
- High Mobility Multi-Wheeled Vehicle
- Modern Jeep
- H250 Microphone
- F15 Fighter Jet
- "Eagle" Fighter
- M101 Microphone
- E3A
- Airborne Warning and Control System (AWACS)
- R215 Microphone
- CH47 Helicopter
- "Chinook" Helicopter
- EV M87 Microphone
Top
of Presentation
ARCON Communicability Exercise
Four Scenarios
Asymmetrical Scenarios Produced Eight Conditions
- Office <-> POTS <-> Quiet
- E3A <-> JTIDS <-> MCE
- Aircraft Carrier <-> HF <-> Office
- HMMWV <-> Satellite <-> Quiet
Top
of Presentation
NRL Talker Recognizability Test
Two Comparisons
- Unprocessed versus Processed
- Processed versus Processed
Top
of Presentation
Complexity
Three Measurements
- Random Access Memory
- Read Only Memory
- Million Instructions Per Second
Top
of Presentation
Male Talker Score is Indicated to the Left of Each Symbol on Each Plot
Top
of Presentation
Diagnostic Acceptability Measure Gender Scores
Two Conditions
- Quiet
- Anechoic Sound Chamber
- Dynamic Microphone
- Office
- Modern Office Environment
- Dynamic Microphone
Male Talker Score is Indicated to the Left of Each Symbol
Top
of Presentation | Top
of Gender Scores
Mean Opinion Score
Six Conditions
- Quiet
- Anechoic Sound Chamber
- Dynamic Microphone
- Quiet - H250
- Anechoic Sound Chamber
- H250 Microphone
- 1% Random Bit Errors
- Anechoic Sound Chamber
- Dynamic Microphone
- 0.5% Random Block Errors
- Anechoic Sound Chamber
- Dynamic Microphone
- 50% Errors within a 35ms block
- Office
- Modern Office Environment
- Dynamic Microphone
- Mobile Command Environment
- Field Shelter
- EV M87 Microphone
Male Talker Score is Indicated to the Left of Each Symbol
Top
of Presentation | Top
of Gender Scores
Degradation Mean Opinion Score
Five Conditions
- High Mobility Multi-Wheeled Vehicle
- Modern Jeep
- H250 Microphone
- E3A
- Airborne Warning and Control System (AWACS) Aircraft
- R215 Microphone
- Automobile
- Plymouth Reliant, Highway Speeds
- STU-III Microphone
- Single Tandem
- CVSD -> Coder
- Anechoic Chamber
- Dynamic Microphone
- Double Tandem
- CVSD -> Coder -> CVSD
- Anechoic Chamber
- Dynamic Microphone
Male Talker Score is Indicated to the Left of Each Symbol
Top
of Presentation | Top
of Gender Scores
Diagnostic Rhyme Test
Seven Benign Conditions
- Quiet
- Anechoic Chamber
- Dynamic Microphone
- Quiet-H250
- Anechoic Chamber
- H250 Microphone
- 1% Random Bit Errors
- Anechoic Chamber
- Dynamic Microphone
- 0.5% Random Block Errors
- Anechoic Chamber
- Dynamic Microphone
- 50% Errors within a 35ms block
- P3C
- Orion Aircraft
- EV 985 Microphone
- Mobile Command Environment
- Field Shelter
- EV M87 Microphone
- Office
- Modern Office Environment
- Dynamic Microphone
Male Talker Score is Indicated to the Left of Each Symbol
Top
of Presentation | Top
of Gender Scores
Diagnostic Rhyme Test
Eight Harsh Conditions
- Single Tandem
- Quiet->CVSD->Coder
- Dynamic Microphone
- Double Tandem
- Quiet->CVSD->Coder->CVSD
- Dynamic Microphone
- Automobile
- Plymouth Reliant, Highway Speeds
- STU-III Microphone
- M2 Tank
- Bradley Fighting Vehicle
- EV M138 Microphone
- High Mobility Multi-Wheeled Vehicle
- Modern Jeep
- H250 Microphone
- F15 Fighter Jet
- "Eagle" Fighter
- M101 Microphone
- E3A
- Airborne Warning and Control System (AWACS)
- R215 Microphone
- CH47 Helicopter
- "Chinook" Helicopter
- EV M87 Microphone
Male Talker Score is Indicated to the Left of Each Symbol
Top
of Presentation | Top
of Gender Scores
NRL Talker Recognizability Test
Two Comparisons
- Unprocessed versus Processed
- Processed versus Processed
Male Talker Score is Indicated to the Left of Each Symbol
Top
of Presentation | Top
of Gender Scores
Top
of Presentation
Top
of Presentation
Top
of Presentation
Top
of Presentation
Elementary Perceptual Qualities (EPQs)
Signal
- Fluttering
- Thin (HPF)
- Harsh
- Muffled
- Interrupted
- Nasal
- Thin (LPF)
- Babble
Background
- Hissing
- Rushing
- Rumbling
- Buzzing
- Bubbling
- Staticy
- Chirping
Phonetic Factors
- Nasality (Grave/Acute)
- Sibilation (Voiced/Unvoiced)
- Compactness (Voiced/Unvoiced (Sustained/Interrupted))
- Voicing (Frictional/NonFrictional)
- Graveness (Voiced/Unvoiced (Plosive/Nonplosive))
- Produced by constriction toward anterior of vocal tract
- Relatively steep upward transitions of seconf formant
- "Weed" versus "Reed", and "Peak" versus "Teak"
- Sustention (Voiced/Unvoiced)
- Produced by incomplete constriction of vocal tract
- Gradual Onset, presence of mid-frequency noise
- "Vee" versus "Bee" and "Shoes" versus "Choose"
Top
of Presentation | Top
of Diagnostics
Top
of Presentation | Top
of Diagnostics
[1] L.M. Supplee, R.P. Cohn, J.S. Collura, A.V. McCree, "MELP: The New
Federal Standard at 2400 bps," IEEE International Conference on
Acoustics, Speech, and Signal Processing, Munich, Germany, 1997.
[2]
M.A. Kohler, P. LaFollette, M.R. Bielefeld, "Criteria for the D.o.D. 2400
bps Vocoder Selection," IEEE International Conference on Acoustics,
Speech, and Signal Processing, Atlanta Georgia, USA, 1996.
[3]
J.P.Campbell, Jr., T.E. Tremain, and V.C. Welch, "The Federal Standard 1016
4800 bps CELP Voice Coder." Digital Signal Processing 1, no. 3 (1991):
145-155.
[4] "Analog to Digital Conversion of Voice by 2400 bit/second
Linear Predictive Coding," Federal Standard 1015, Nov. 1984.
[5] J.D.
Tardelli, E.W. Kreamer, "Vocoder Intelligibility and Quality Test Methods,"IEEE
International Conference on Acoustics, Speech, and Signal Processing,
Atlanta, Georgia, USA, 1996.
[6] P.D. Gatewood, P.A. LaFollette, "Host
Laboratory Functions for the D.o.D. 2400 bps Vocoder Selection Process,"IEEE
International Conference on Acoustics, Speech, and Signal Processing,
Atlanta, Georgia, USA, 1996.
[7] E.W. Kreamer, J.D. Tardelli, "Communicability
Testing for Vocoders,"IEEE International Conference on Acoustics,
Speech, and Signal Processing, Atlanta, Georgia, USA, 1996.
[8] A.
Scmidt-Nielsen, D.P. Brock, "Speaker Recognizability Testing for Voice
Coders," IEEE International Conference on Acoustics, Speech, and Signal
Processing, Atlanta, Georgia, USA, 1996.
[9] T.E. Tremain, M.A. Kohler, "Philosophy
and Goals of the D.o.D. 2400 bps Vocoder Selection Process," IEEE
International Conference on Acoustics, Speech, and Signal Processing,
Atlanta, Georgia, USA, 1996.
[10] M.R. Bielefeld, "1995 Test and
Evaluation Plan for the Selection of a New U.S. Government Standard Voice
Processor at 2,400 bps," MITRE Technical Report 96W0000029, August 1996.
[11]
V.C. Welch, T.E. Tremain, J.P. Campbell, Jr., "A Comparison of U.S.
Government Standard Voice Coders," IEEE Military Communications Conference,
Boston MA, 1989.
[12] W.D. Voiers, "Evaluating Processed Speech Using
the Diagnostic Rhyme Test," Speech Technology, Jan/Feb 1983.
[13] M.A.
Kohler, "Analysis of Performance and Complexity for Four Government
Standard Voice Coding Algorithms," NSA Technical Report, R22-002-96,
S-243,720.
Top
of Presentation
This website is sponsored by ARCON Corporation for the DDVPC and as such must
conform to DoD policies concerning websites. Please read this
disclaimer for more information.
Comments concerning design and content of these pages should
be sent to
info@arcon.com. This page was last updated on
27-Dec-2001.