MySQL fulltext parser plugin

Starting with the version 3.2.36 mnoGoSearch offers a stemming plugin which is compatible with the MySQL 5.1 pluggable fulltext parsers interface.

When mnoGoSearch stemming plugin is used as a fulltext parser plugin, all words are normalized during INSERT, UPDATE and SELECT SQL queries. This allows to find different grammatical forms of the same words. For example, the search front-end will try to find the word test whenever the words testing or tests or tested are given in the search query, and vice versa. When processing INSERT and UPDATE SQL statements, MySQL will put all words into the fulltext index in their normalized form. When processing a SELECT SQL statement, the query words given in the MATCH operator are normalized again and then searched in the fulltext index.

To enable support for MySQL 5.1 fulltext parser plugin, mnoGoSearch should be configured with --enable-mysql-fulltext-plugin option:

./configure --with-mysql --enable-mysql-fulltext-plugin
make
make install
  

To install mnoGoSearch stemming plugin, libmnogosearch.so must be copied or sym-linked into the MySQL plugin directory. Also, you must have INSERT privileges for the mysql.plugin table to be able to run the INSTALL PLUGIN statement.

cp /usr/local/mnogosearch/lib/libmnogosearch.so /usr/lib/mysql
or
ln -s /usr/local/mnogosearch/lib/libmnogosearch.so /usr/lib/mysql

Before mnoGoSearch stemming plugin can be used for a fulltext index, libmnogosearch.so must be loaded into MySQL using the INSTALL PLUGIN statement:

  INSTALL PLUGIN stemming SONAME 'libmnogosearch.so';

To unload mnoGoSearch stemming plugin, use the UNINSTALL PLUGIN statement:

UNINSTALL PLUGIN stemming;

To create an index using mnoGoSearch stemming plugin, specify the WITH PARSER clause in the fulltext index definition:

CREATE TABLE my_table (
  my_column TEXT,
  FULLTEXT(my_column) WITH PARSER stemming
);

When the INSTALL PLUGIN statement is executed, mnoGoSearch stemming plugin reads its configuration from the file stemming.conf in MySQL datadir. The file stemming.conf supports these standard mnoGoSearch commands: Affix, Spell and MinWordLength.

An example of stemming.conf may look like this:

MinWordLength 2
Spell en latin1 american.xlg
Affix en latin1 english.aff

Usage example:

mysql> INSTALL PLUGIN stemming SONAME 'libmnogosearch.so';
Query OK, 0 rows affected (0.06 sec)

mysql> CREATE TABLE t(a TEXT, FULLTEXT(a) WITH PARSER stemming);
Query OK, 0 rows affected (0.01 sec)

mysql> INSERT INTO t VALUES('testing'),('tested'),('test'),('tests'),('tester');
Query OK, 5 rows affected (0.00 sec)
Records: 5  Duplicates: 0  Warnings: 0

mysql> SELECT * FROM t WHERE MATCH a AGAINST('test' IN BOOLEAN MODE);
+---------+
| a       |
+---------+
| testing |
| tested  |
| test    |
| tests   |
| tester  |
+---------+
5 rows in set (0.01 sec)

mysql> SELECT * FROM t WHERE MATCH a AGAINST('testing' IN BOOLEAN MODE);
+---------+
| a       |
+---------+
| testing |
| tested  |
| test    |
| tests   |
| tester  |
+---------+
5 rows in set (0.00 sec)