supervised multiple many-to-many aligner (sMM2M)

This program aligns multiple sequences in a monotone many-to-many manner. Programs have been tested on Ubuntu 12.04. They require Python 2.7x.

There are two aligners:

(1) multipleAlign_aligner.py: Slow (allows to align 4 or fewer strings in reasonable time) but more accurate

(2) multipleAlign_segmenter.py: Faster but less accurate

REQUIREMENTS

A bi-aligner, that is, an aligner that aligns pairs of strings/sequences. (In actuality, you only need bi-aligned training data.)

Good bi-aligners can be found from e.g.:

https://code.google.com/p/m2m-aligner/, M2M aligner

http://de.osdn.jp/projects/mpaligner/, Mpaligner

https://code.google.com/p/phonetisaurus/, Phonetisaurus aligner

DATA FORMAT

• (1) multipleAlign_aligner.py

  If you want to align N+1 sequences, you must have N bi-aligned files, each of the form

  segmented-X-string segmented-Y-string

  segmented-X-strings should always be from the same alphabet (e.g. letter words) in all N files segmented-Y-strings may be different (e.g. phonetic) representations of the X strings.

  Sequences must be separated by a tab symbol.

  More precisely: if you want to align sequences of the form X Y_1 Y_2 ... Y_N then the N files must be as follows:

  File 1: Sequences of type X and Y_1 are bi-aligned

  File 2: Sequences of type X and Y_2 are bi-aligned

  ...

  File N: Sequences of type X and Y_N are bi.aligned

  Have a look at e.g. sampleData/wiki_ga.bialigned

  The N different files do NOT need to have identical (number of) X-strings.

  Segment separators are, by default, "-".

  Character separators are, by default, "|".

  INPUT DATA: See e.g. sampleData/matchedup_3.data.

  Note: Input sequences must be separated by a tab symbol.

• (2) multipleAlign_aligner_segmenter.py

  If you want to align N sequences, you must have N bialigned files, each of the form as above. The first one contains segmented input strings in its second column (it may be the reverse of one of the N-1 other files).

RUNNING

• (1) multipleAlign_aligner.py:

Sample run is e.g.

    python multipleAlign_aligner.py sampleData/wiki_ga.bialigned sampleData/wiki_rp.bialigned <  sampleData/matchedup_3.data

or

    python multipleAlign_aligner.py sampleData/wiki_ga.bialigned sampleData/wiki_rp.bialigned sampleData/pte_ga.bialigned < sampleData/matchedup.data 

If you want to modify settings, you can do so in settings.txt

• (2) multipleAlign_segmenter.py:

  python multipleAlign_segmenter.py sampleData/wiki_ga.bialigned.rev sampleData/wiki_ga.bialigned sampleData/wiki_rp.bialigned sampleData/pte_ga.bialigned < sampleData/matchedup.data
  

REFERENCES

If you use this, please cite the following papers:

Eger, Steffen Multiple Many-To-Many Sequence Alignment For Combining String-Valued Variables: A G2P Experiment In: ACL, Association for Computational Linguistics, 2015. Accepted

Eger, Steffen Improving G2P from Wiktionary and other (web) resources. Interspeech 2015. Accepted.

Have fun! Steffen Eger, 07/04/2015

[email protected]