Speech Recognition
Contents
Speech Recognition on Robotino
Robotino can now comprehend basic human speech commands. Robotino uses the open source speech recognition engine called Julius.
Setting up Speech Recognition
Setting up speech recognition is done in two parts. The first part involves training of the speech data and creation of the acoustic model and the second involves the actual execution of the speech recognition engine with the created acoustic model.
Creating an Acoustic Model
The HTK toolkit (version 3.4) shall be used to create an acoustic model. Please follow the following steps.
Step 1
You will need to register with HTK before you can download it. Please do so.
Step 2
Download the sources for HTK toolkit 3.4 from here. Also download the HTK samples from here.
Step 3
- Move to your home directory
cd ~
- Create a directory called 'bin'
mkdir bin
- Unpack the downloaded HTK toolkit sources and HTK samples in a folder called 'htk-3.4' in the 'bin' directory. The 'bin' directory should contain the following
htk-3.4 samples
- Move the 'samples' folder to the 'htk-3.4' folder as follows
cd bin mv samples htk-3.4
- If you have a newer version of the gcc compiler (version 4 or above), you will need to install gcc version 3.4 so that HTK will compile properly. Use the following gcc's version command to see which version is installed on your system
gcc -v
- If your gcc version is 4 and above, follow the listed commands to install gcc 3.4
sudo apt-get install gcc-3.4 sudo rm /usr/bin/gcc sudo ln -s /usr/bin/gcc-3.4 /usr/bin/gcc
- Install the external dependencies as follows
sudo apt-get install libx11-dev libesd0-dev libasound2-dev libzip1 flex
- Now move to the 'htk-3.4' dir and configure htk as follows. Note change %yourusername% from the command to your user name.
cd htk-3.4 ./configure --prefix=/home/%yourusername%/bin/htk-3.4
- Now run make all and make install. This should install the created binaries to the folder '/home/yourusername/bin/htk-3.4/bin' .
make all make install
- Change directory back to home and create a folder called 'voxforge' and then a folder called 'HTK_scripts' in the voxforge folder.
cd ~ mkdir voxforge cd voxforge mkdir HTK_scripts cd HTK_scripts
- Now copy some scripts from the 'htk-3.4/samples' folder to the 'HTK_scripts' folder as follows
cp ../../bin/htk-3.4/samples/RMHTK/perl_scripts/mkclscript.prl . cp ../../bin/htk-3.4/samples/HTKTutorial/maketrihed . cp ../../bin/htk-3.4/samples/HTKTutorial/prompts2mlf . cp ../../bin/htk-3.4/samples/HTKTutorial/prompts2wlist .
- Your 'HTK_scripts' folder should contain the following
maketrihed mkclscript.prl prompts2mlf prompts2wlist
Step 4
- Now we will download Julius (version 4.5.1). We shall be using pre-compiled libraries which can downloaded from here
- Once downloaded extract them to your '/home/%yourusername%/bin' folder. After that is done your 'bin' folder should contain the following
htk-3.4 julius-4.1.5-linuxbin
Step 5
- Now you will need to update your user path which can be done as follows. First change to your home directory and edit the .bashrc file.
cd ~ gedit .bashrc
- Add the following to the end of the .bashrc file. Note change %yourusername% from the command to your username.
# HTK and JULIUS scripts and executables PATH=$PATH:$HOME/bin:/home/%yourusename%/bin/htk-3.4/bin:/home/%yourusename%/bin/julius-4.1.5-linuxbin/bin
- Source your .bashrc file to reflect the changes
source ~/.bashrc
- Test if your HTK toolkit has been installed correctly by running the following command.
HVite -V
- You should see an output similar to the following.
HTK Version Information Module Version Who Date : CVS Info HVite 3.4 CUED 25/04/06 : $Id: HVite.c,v 1.1.1.1 2006/10/11 09:55:02 jal58 Exp $ HShell 3.4 CUED 25/04/06 : $Id: HShell.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $ HMem 3.4 CUED 25/04/06 : $Id: HMem.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $ HLabel 3.4 CUED 25/04/06 : $Id: HLabel.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $ HMath 3.4 CUED 25/04/06 : $Id: HMath.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $ HSigP 3.4 CUED 25/04/06 : $Id: HSigP.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $ HWave 3.4 CUED 25/04/06 : $Id: HWave.c,v 1.1.1.1 2006/10/11 09:54:59 jal58 Exp $ HAudio 3.4 CUED 25/04/06 : $Id: HAudio.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $ HVQ 3.4 CUED 25/04/06 : $Id: HVQ.c,v 1.1.1.1 2006/10/11 09:54:59 jal58 Exp $ HModel 3.4 CUED 25/04/06 : $Id: HModel.c,v 1.2 2006/12/07 11:09:08 mjfg Exp $ HParm 3.4 CUED 25/04/06 : $Id: HParm.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $ HDict 3.4 CUED 25/04/06 : $Id: HDict.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $ HNet 3.4 CUED 25/04/06 : $Id: HNet.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $ HRec 3.4 CUED 25/04/06 : $Id: HRec.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $ HUtil 3.4 CUED 25/04/06 : $Id: HUtil.c,v 1.1.1.1 2006/10/11 09:54:59 jal58 Exp $ HAdapt 3.4 CUED 25/04/06 : $Id: HAdapt.c,v 1.2 2006/12/07 11:09:07 mjfg Exp $ HMap 3.4 CUED 25/04/06 : $Id: HMap.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $
- Test if Julius has been installed correctly by entering the following command in the terminal
julius-4.1.5
- You should see an output similar to the following
Julius rev.4.1.5 - based on JuliusLib rev.4.1.5 (fast) built for i686-pc-linux Copyright (c) 1991-2009 Kawahara Lab., Kyoto University Copyright (c) 1997-2000 Information-technology Promotion Agency, Japan Copyright (c) 2000-2005 Shikano Lab., Nara Institute of Science and Technology Copyright (c) 2005-2009 Julius project team, Nagoya Institute of Technology Try '-setting' for built-in engine configuration. Try '-help' for run time options.
- Now to switch back to your original gcc version, do the following (The original version in my case was 4.3, yours may differ)
sudo rm /usr/bin/gcc sudo ln -s /usr/bin/gcc-4.3 /usr/bin/gcc
Step 6
Install Audacity as follows
sudo apt-get install audacity
Step 7
We will now compile the grammar and voca files.
- Create a folder called 'auto' in your '/home/%yourusename%/voxforge' directory
cd ~ cd voxforge mkdir auto cd auto
- Download the grammar and voca files from here. Extract them in your 'auto' folder you just created. After extraction your 'auto' folder should contain the following
robotino.grammar robotino.voca
- Now compile the grammar and voca files to Julius files. Make sure you are in the 'auto' folder. Run the following command
mkdfa.pl robotino
- You should see an output as follows
robotino.grammar has 11 rules robotino.voca has 10 categories and 27 words --- Now parsing grammar file Now modifying grammar to minimize states[7] Now parsing vocabulary file Now making nondeterministic finite automaton[31/31] Now making deterministic finite automaton[24/24] Now making triplet list[24/24] 10 categories, 24 nodes, 32 arcs -> minimized: 11 nodes, 19 arcs --- generated: robotino.dfa robotino.term robotino.dict
Step 8
Now we shall proceed to the training and creation of the acoustic model.
- Download the prompts file and save it in your '/home/%yourusername%/voxforge/auto' folder. Your 'voxforge/auto' folder should look like this
prompts robotino.dict robotino.term robotino.dfa robotino.grammar robotino.voca
- Now create a folder called 'lexicon' in the 'voxforge' directory.
cd ~ cd voxforge mkdir lexicon