intro to ML bgu

Occam's razor

• the simplest explanation is usally the correct one

• the simplest explanation is usally the wrongone

K fold use to

• create train/test split

• ignore train/test split

which one is the true confusion matrix ?

• predict

  TP | FP

  actual ______

  FN | TN

• predict

  FN | FP

  actual ______

  TP | TN

Overfitting

• means that it learns the training set too well – it overfits to the training set such that it performs poorly on the test set.

• when model is too simple, both training and test errors are large

Underfitting

• means that it learns the training set too well – it overfits to the training set such that it performs poorly on the test set.

• when model is too simple, both training and test errors are large

Bias

• is a learner’s tendency to consistently learn the same wrong thing.

• s the tendency to learn random things irrespective of the real signal.

Variance

• is a learner’s tendency to consistently learn the same wrong thing.

• is the tendency to learn random things irrespective of the real signal.

top right

top left

down right

doen left

• low variance - high bias

  high variance - high bias

  low variance - low bias

  high variance - low bias

• low variance - high bias

  high variance - low bias

  low variance - low bias

  high variance - high bias

what is Regularization and how it use?

• Fighting Overfitting

  Early Stopping • Inject Randomness • Ensemble Methods

• Loving Overfitting

  Never Stopping • avoid Randomness • Ensemble Methods

• AdaBoost (Freund & Schapire, 1996)

• Gradient Boosted Trees (Friedman, 1999)

• Stochastic gradient boosted distributed

decision trees (Ye et al., 2009)

are ____ methods

• Iterative Methods

• Non-Iterative Methods