The big idea

Naïve Bayes predicts the most likely class using Bayes’ theorem.

It assumes features are conditionally independent given the class.

That assumption is often false, but it still works surprisingly well.

Bayes theorem

P(class | features) ∝ P(features | class) * P(class)P(class | features) ∝ P(features | class) * P(class)

Why it’s popular for text

Text features (word counts) produce high-dimensional sparse vectors.

Naïve Bayes handles this efficiently.

Common variants

• GaussianNB (continuous features)
• MultinomialNB (counts, text)
• BernoulliNB (binary features)

Scikit-learn example

from sklearn.naive_bayes import GaussianNB
 
nb = GaussianNB()

from sklearn.naive_bayes import GaussianNB
 
nb = GaussianNB()

Pros and cons

Pros:

• very fast
• strong baseline
• works well for text

Cons:

• independence assumption can limit accuracy

Mini-checkpoint

If you have TF-IDF features for spam detection, which NB variant is common?

(Usually MultinomialNB.)