Thread: How to Install Moses Statistical Machine Translation System on Ubuntu 10.04

Hi!

Follow this guide in order to install Moses on Ubuntu 10.04

Install required packages

Code:

apt-get install build-essential subversion zlib1g-dev autoconf automake g++ make gawk gzip tcl8.4 tcl8.4-dev tcsh tcl-dev tclx8.4 tclx8.4-dev libtool libboost-dev libboost-graph-dev libboost-iostreams-dev libboost-program-options-dev gcc-4.1 g++-4.1 git-core

Set GCC-4.1 as default compiler

Code:

update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.4 40
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.1 30
update-alternatives --config gcc

Code:

update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.4 40
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.1 30
update-alternatives --config g++

Default folders

Code:

mkdir -p /opt/tools/bin
cd /opt/tools

Install GIZA

Code:

wget http://giza-pp.googlecode.com/files/giza-pp-v1.0.5.tar.gz
tar -xzvf giza-pp-v1.0.5.tar.gz
cd /opt/tools/giza-pp
make
cd /opt/tools
cp giza-pp/GIZA++-v2/GIZA++ bin/
cp giza-pp/mkcls-v2/mkcls bin/
cp giza-pp/GIZA++-v2/snt2cooc.out bin/

Install SRILM

Code:

mkdir /opt/tools/srilm
cd /opt/tools/srilm

(get srilm download, requires web registration, you'll end up with a .tgz file to copy to this directory)

Code:

tar -xzvf srilm.tgz
rm  srilm.tgz
chmod +w Makefile
vim Makefile

change

Code:

7c7
< # SRILM = /home/speech/stolcke/project/srilm/devel
---
> SRILM = /opt/tools/srilm

Ubuntu 32 Bits

Code:

make World
make all

Ubuntu 64 Bits

Code:

make MACHINE_TYPE=i686-m64 World
make MACHINE_TYPE=i686-m64 all

Moses need a "i686" folder to run srlim, so we need to add some links. Remember to replace VERSION according to your computer.

Code:

cd /opt/tools/srilm/bin
ln -s i686-VERSION i686
cd /opt/tools/srilm/lib
ln -s i686-VERSION i686

Install IRSTLM

Code:

cd /opt/tools
mkdir irstlm
svn co https://irstlm.svn.sourceforge.net/svnroot/irstlm/trunk irstlm
cd /opt/tools/irstlm
. regenerate-makefiles.sh
./configure
make
make install

Install MOSES

Code:

cd /opt/tools
git clone https://github.com/moses-smt/mosesdecoder.git
mv mosesdecoder moses
cd moses
./regenerate-makefiles.sh
./configure --with-srilm=/opt/tools/srilm --with-irstlm=/usr/local/irstlm

Code:

make -j 2

(The -j 2 is optional. make -j X where X is number of simultaneous tasks is a speedier option for machines with multiple processors)

TEST

Code:

cd /opt/
mkdir data
cd data
wget http://www.statmt.org/moses/download/sample-models.tgz
tar -xzvf sample-models.tgz
cd sample-models
/opt/tools/moses/moses-cmd/src/moses -f /opt/data/sample-models/phrase-model/moses.ini < /opt/data/sample-models/phrase-model/in > out

Install MOSES SCRIPTS

Code:

cd /opt/tools
mkdir moses-scripts
cd /opt/tools/moses/scripts

Code:

vim Makefile

Code:

13,14c13,14
< TARGETDIR?=/home/s0565741/terabyte/bin
< BINDIR?=/home/s0565741/terabyte/bin
---
> TARGETDIR?=/opt/tools/moses-scripts
> BINDIR?=/opt/tools/bin/

Code:

make release

Once make is finished, a folder will be created with the compiled scripts using the date of the machine in the name. Now we need to set a SCRIPTS_ROOTDIR environment variable with the path to the scripts. Replace YYYYMMDD-HHMM in the following lines according to your machine.

Code:

vim /etc/bash.bashrc
export SCRIPTS_ROOTDIR=/opt/tools/moses-scripts/scripts-YYYYMMDD-HHMM

(add at the end)

Install OTHER SCRIPTS

Code:

cd /opt/tools/
wget http://homepages.inf.ed.ac.uk/jschroe1/how-to/scripts.tgz
tar -xzvf scripts.tgz
cd scripts
wget ftp://jaguar.ncsl.nist.gov/mt/resources/mteval-v11b.pl
chmod +x mteval-v11b.pl

More Information: Moses Installation and Training Run-Through

Hope it helps!

Muchas gracias por compartir esta información, Ari.

Originally Posted by hermeniko

I'm using ubuntu 10.10. g++-4.1library is not available here. Can this be the problem?

Yes! I can't compile Moses with a new version of g++, that's why this tutorial is for 10.04, because in 10.10 g++ 4.1 is not available.

As far as i know, people from DoMY project ("Do Moses Yourself") compile Moses with g++ 4.3, maybe you should give it a try.

Saludos!

Hello again,
for those who have the same problem and who doesn't want (or cannot for some reason) install ubuntu 10.04, have a look here:
[SOLVED] How to install g++4.1 in Ubintu 10.10 - Ubuntu Forums
This solves the problem (it did for me!)
Cheers.

P.S. I tried g++4.3. it didn't work.

Originally Posted by hermeniko

Hello again,
for those who have the same problem and who doesn't want (or cannot for some reason) install ubuntu 10.04, have a look here:
[SOLVED] How to install g++4.1 in Ubintu 10.10 - Ubuntu Forums
This solves the problem (it did for me!)
Cheers.

P.S. I tried g++4.3. it didn't work.

Thanks hermeniko!!

It would be great if you could tell us something more about your Moses instalation, like language pairs, corpus involved, train parameters, etc.

Saludos!

Hey,

I'm working with speech transcripts.
The general idea is to integrate a translation module into a dialog system (to obtain a multilingual dialog system, kind of... So the aim is to build a model which would translate the outputs of the speech recognition system (which are not always perfect). So I'm trying different models which use different kinds of info (like semantic annotation, for example) and different settings. I need to pick one with the best performance
Concerning the language pair, at the moment it is french-italian, but of course the extension to other languages is not out of question.

Cheers,

hello...
i'm ibnu.
is this tutorial is for ubuntu 10.04 64bit or 32 bit?
thanks.

For the latest installation instructions, follow this link:

Quick Start | Precision Translation Tools

Then, follow the "Next" links at the bottom of each page.

Tom

Originally Posted by ibnuhadi

hello...
i'm ibnu.
is this tutorial is for ubuntu 10.04 64bit or 32 bit?
thanks.