AbbotDemo mini-FAQ

  1 THIS
  1 SPEECH READING
  1 SPEECH RECOGNITION THE 
  1 SPEECH RECOGNITION IS A B.
  1 SPEECH RECOGNITION IS A PIECE OF THE
  1 SPEECH RECOGNITION IS A PIECE OF CAKE
  

Alternatively, if you do not have X or have problems associated with abbotAudio, you can send prerecorded files through the recogniser by specifying the names of the audio files on the command line. These files should be of speech sampled at 16 kHz with 16 bits/sample in the natural byte order and with no header. For example:

  unix$ srec -t 3 -s16000 -b16 test.raw
  Speed 16000 Hz (mono)
  16 bits per sample
  unix$ ~/AbbotDemo-0.4/AbbotDemo test.raw
  1 SPEECH RECOGNITION IS A PIECE OF CAKE

To select where the audio is recorded from and played back to, click on the "Audio" menu-button on abbotAudio, and highlight the appropriate option. Both input and output channels are controlled from this one menu.

• Did AbbotDemo produce any warning messages?
• Did a waveform appear after recording?
• Check the operation of the rest of the system with: AbbotDemo etc/test.raw

• microphone connected to inappropriate jack
• recording from incorrect audio channel
• line levels are set incorrectly
• recording levels are set incorrectly

• Was the signal recorded in noise-free conditions?
• Are you putting on your best British accent?
• Are there very many out of vocabulary words?
• Is the text similar to that of a business newspaper?

The system can be upgraded to use a larger, and therefore more accurate, language model by fetching the file spring98-20k-8-8.bin.gz from the AbbotDemo FTP directory (see Q3). You will need the GNU gzip utility to uncompress this file. To run this system we recommend 64 Mbyte RAM. Please note that this file is 22 Mbyte when compressed, which is why it is not included in the core distribution.

To run AbbotDemo with the new language model simply uncompress and specify the file name on the command line with the -lm flag like this:

  unix$ AbbotDemo -uk -lm spring98-20k-8-8.bin

In adddition we produced a language model for the EuroSpeech conference based on the EuroSpeech93 proceedings and some speech papers that were available locally. The amount of training data is much less than the standard North American Business news domain and hence the language model quality is not as good but it does show the use of speech recognition in another domain. Pick up a copy of ICASSP, EuroSpeech or ICSLP and start reading to both systems to see the effect of having an appropriate language model.

The files are available from host svr-ftp.eng.cam.ac.uk in directory pub/comp.speech/recognition/AbbotDemo/ as euro16k-00.bin.gz and euro16k.dict.gz. Only British English pronunciations are supported. The system is run as:

  AbbotDemo -uk -dictionary euro16k.dict -lm euro16k-00.bin

abbotAudio:

This is the front end to the whole system. It is written for X (which has proved to be remarkably non-portable). Basic use is described earlier in the FAQ. The data format of the output is a stream of 16 bit samples at 16 kHz. The special sample value of -32768 (the most negative 16 bit number) is used to flag the end of sentence.

abbotAudioCat:

A very simple substitute for AbbotAudio which glues together the files on the command line in the same format as abbotAudio.

plp:

Performs perceptual linear prediction (PLP) on the incoming waveform and writes out a set of acoustic vector coefficients. The format of the output stream is 13 floating point values at four bytes each.

rnnInputNorm:

Normalises each channel to zero mean and unity variance. This helps to compensate for variations in the channel. The first sentence is buffered before normalisation and subsequent sentences are either buffered and processed independently or not buffered and the means and variances are computed using an IIR filter. The output format is one byte to flag end-of-sentence followed by 13 bytes containing the (squashed) PLP coefficients and two padding bytes.

rnnForward:

Estimates the probabilities of each of the phonemes from the normalised PLP stream. This is achieved with a recurrent neural network and is one of the two biggest users of CPU cycles. The output is one byte to flag end-of-sentence and N bytes to encode the log probabilities of each of the phonemes where N is the size of the phone set used.

rnnGrabUtterance:

Sits in the pipe rather like a tee(1) command but just writes the last sentence that was passed through to the output file.

showGuts:

takes the single sentence provided by rnnGrabUtterance and displays it on the screen in a X window. For a very clean segment of speech it is often possible to read off the phonemes that were said, and if you are good at segmentation perhaps the words as well.

noway:

Converts the phone probabilities into a word string. This is the other process that needs a lot of CPU power. There is a tradeoff between speed and accuracy that can be adjusted via the values of -beam, -state_beam and -prob_min. 4, 3, and 0.0005 will run more slowly and more accurately than 2, 1, and 0.005. The -inc_output N option displays the best word string every N frames.

AbbotDemo uses one recurrent network to estimate phone probabilities. We find that using four networks and combining the outputs can result in 20% fewer errors.

AbbotDemo uses context independent phone models. Using context dependent phone models approximately doubles the number of parameters (and therefore the CPU required to run the network in real-time) but does result in about 15% fewer errors.

The vocabulary size of AbbotDemo is 20,000 words. If you pick up a copy of the Wall Street Journal chances are this will only cover about 97% of the words in a given passage of text and therefore there will be at least 3% errors as the system can not recognise words that are not in the vocabulary.

The size of the language models was constrained in order to allow ease of FTP access and reasonable disk usage. This results in a significant increase in the perplexity (the average number of words that are considered as the next word). The resulting increase in word error rate has been measured at 16%.

The decoder, noway, has tighter pruning than used in our "evaluation quality" decodes in order to achieve faster operation. These are adjustable with the -beam, -state_beam, -n_hyps and -prob_min options at the end of the AbbotDemo script. Our tests show that the faster options supplied with this demo result in about 10% increase in word error rate.

For the US English version of AbbotDemo, the "evaluation quality" pronunciation dictionary has been replaced by the publicly available CMU lexicon. This causes a mismatch between training and testing pronunciations and results in an additional 20-30% increase in word error rate for this system (please note that the increased error rate is due to the mismatch between lexica and is not necessarily due to the quality of the CMU pronunciations).

From this you may total all these numbers for a 20k system and arrive at about twice or three times the error rate for AbbotDemo than might otherwise be achieved (we have not yet measured the combined degradation). Our typical error word rate on clean read speech is about 11-17%.

• abbotAudio and showGuts have display problems if they are partially overlayed with another window
• There is a mismatch between the pronunciations used for training the American English system and those provided in this package

Much of the funding for the recent development of this system was provided by the ESPRIT Wernicke Project with partners:

CUED Cambridge University Engineering Department, UK
ICSI International Computer Science Institute, USA
INESC Instituto de Engenharia de Sistemas e Computadores, Portugal
LHS Lernout Hauspie SpeechSystems, Belgium
and associates:
SU Sheffield University, UK
FPMs Faculte Polytechnique de Mons, Belgium
Dedicated hardware for training the recurrent networks and system software for that hardware were provided by ICSI.

The acoustic and language models for AbbotDemo were derived from materials distributed by the Linguistic Data Consortium. ftp://ftp.cis.upenn.edu/pub/ldc

The CMU statistical language modelling toolkit was used to generate the trigram language model.

The BEEP dictionary was used for British English pronunciations. ftp://svr-ftp.eng.cam.ac.uk/pub/comp.speech/data/beep-0.7.tar.gz

The CMU dictionary was used for American English pronunciations. ftp://ftp.cs.cmu.edu/project/fgdata/dict/cmudict.0.4.Z

The CMU phone set was expanded using code provided by ICSI.

The X-windows interface for speech capture was derived from speech processing software developed at the Laboratory for Engineering Man/Machine Systems (LEMS) at Brown University.