sphinx.util.stemmer.porter.PorterStemmer

class documentation

class PorterStemmer:

Known subclasses: sphinx.util.stemmer.StandardStemmer

View In Hierarchy

Undocumented

Method __init__ No summary
Method cons cons(i) is TRUE <=> b[i] is a consonant.
Method cvc No summary
Method doublec doublec(j) is TRUE <=> j,(j-1) contain a double consonant.
Method ends ends(s) is TRUE <=> k0,...k ends with the string s.
Method m m() measures the number of consonant sequences between k0 and j. if c is a consonant sequence and v a vowel sequence, and <..> indicates arbitrary presence,
Method r r(s) is used further down.
Method setto setto(s) sets (j+1),...k to the characters in the string s, readjusting k.
Method stem No summary
Method step1ab step1ab() gets rid of plurals and -ed or -ing. e.g.
Method step1c step1c() turns terminal y to i when there is another vowel in the stem.
Method step2 step2() maps double suffices to single ones. so -ization ( = -ize plus -ation) maps to -ize etc. note that the string before the suffix must give m() > 0.
Method step3 step3() dels with -ic-, -full, -ness etc. similar strategy to step2.
Method step4 step4() takes off -ant, -ence etc., in context <c>vcvc<v>.
Method step5 step5() removes a final -e if m() > 1, and changes -ll to -l if m() > 1.
Method vowelinstem vowelinstem() is TRUE <=> k0,...j contains a vowel
Instance Variable b Undocumented
Instance Variable j Undocumented
Instance Variable k Undocumented
Instance Variable k0 Undocumented
def __init__(self):

The main part of the stemming algorithm starts here. b is a buffer holding a word to be stemmed. The letters are in b[k0], b[k0+1] ... ending at b[k]. In fact k0 = 0 in this demo program. k is readjusted downwards as the stemming progresses. Zero termination is not in fact used in the algorithm.

Note that only lower case sequences are stemmed. Forcing to lower case should be done before stem(...) is called.

def cons(self, i):
cons(i) is TRUE <=> b[i] is a consonant.
Parameters
i:intUndocumented
Returns
intUndocumented
def cvc(self, i):
cvc(i) is TRUE <=> i-2,i-1,i has the form
consonant - vowel - consonant

and also if the second c is not w,x or y. this is used when trying to restore an e at the end of a short e.g.

cav(e), lov(e), hop(e), crim(e), but snow, box, tray.
Parameters
i:intUndocumented
Returns
intUndocumented
def doublec(self, j):
doublec(j) is TRUE <=> j,(j-1) contain a double consonant.
Parameters
j:intUndocumented
Returns
intUndocumented
def ends(self, s):
ends(s) is TRUE <=> k0,...k ends with the string s.
Parameters
s:strUndocumented
Returns
intUndocumented
def m(self):

m() measures the number of consonant sequences between k0 and j. if c is a consonant sequence and v a vowel sequence, and <..> indicates arbitrary presence,

<c><v> gives 0 <c>vc<v> gives 1 <c>vcvc<v> gives 2 <c>vcvcvc<v> gives 3 ....
Returns
intUndocumented
def r(self, s):
r(s) is used further down.
Parameters
s:strUndocumented
def setto(self, s):
setto(s) sets (j+1),...k to the characters in the string s, readjusting k.
Parameters
s:strUndocumented
def stem(self, p, i, j):
overridden in sphinx.util.stemmer.StandardStemmer
In stem(p,i,j), p is a char pointer, and the string to be stemmed is from p[i] to p[j] inclusive. Typically i is zero and j is the offset to the last character of a string, (p[j+1] == ''). The stemmer adjusts the characters p[i] ... p[j] and returns the new end-point of the string, k. Stemming never increases word length, so i <= k <= j. To turn the stemmer into a module, declare 'stem' as extern, and delete the remainder of this file.
Parameters
p:strUndocumented
i:intUndocumented
j:intUndocumented
Returns
strUndocumented
def step1ab(self):

step1ab() gets rid of plurals and -ed or -ing. e.g.

caresses -> caress ponies -> poni ties -> ti caress -> caress cats -> cat

feed -> feed agreed -> agree disabled -> disable

matting -> mat mating -> mate meeting -> meet milling -> mill messing -> mess

meetings -> meet

def step1c(self):
step1c() turns terminal y to i when there is another vowel in the stem.
def step2(self):
step2() maps double suffices to single ones. so -ization ( = -ize plus -ation) maps to -ize etc. note that the string before the suffix must give m() > 0.
def step3(self):
step3() dels with -ic-, -full, -ness etc. similar strategy to step2.
def step4(self):
step4() takes off -ant, -ence etc., in context <c>vcvc<v>.
def step5(self):
step5() removes a final -e if m() > 1, and changes -ll to -l if m() > 1.
def vowelinstem(self):
vowelinstem() is TRUE <=> k0,...j contains a vowel
Returns
intUndocumented
b =

Undocumented

j =

Undocumented

k =

Undocumented

k0 =

Undocumented