Search

Natural Language Processing

to be continued...

# regex for removing punctuation!
import re
# nltk preprocessing magic
import nltk
from nltk.tokenize import word_tokenize
from nltk.stem import PorterStemmer
from nltk.stem import WordNetLemmatizer
# grabbing a part of speech function:
from part_of_speech import get_part_of_speech

text = "So many squids are jumping out of suitcases these days that you can barely go anywhere without seeing one burst forth from a tightly packed valise. I went to the dentist the other day, and sure enough I saw an angry one jump out of my dentist's bag within minutes of arriving. She hardly even noticed."

cleaned = re.sub('\W+', ' ', text)
tokenized = word_tokenize(cleaned)

stemmer = PorterStemmer()
stemmed = [stemmer.stem(token) for token in tokenized]

## -- CHANGE these -- ##
lemmatizer = None
lemmatized = []

print("Stemmed text:")
print(stemmed)
print("\nLemmatized text:")
print(lemmatized)

Recent Posts

See All

Drug Discovery

A drug target is a molecule in the body, usually a protein, that is intrinsically associated with a particular disease process and that could be addressed. A biological target is anything within a liv

DevOps

DevOps is a collaboration of the development (Dev) and operations (Ops) teams with its foundation depending on providing IT automation. DevOps is an agile methodology that includes a set of practices