CST383 Week 6

This week, we are learning about how to optimize machine learning algorithms.

Hyperparameters are values that are not able to be learned by the algorithm, so must be set before the training of data begins. To find the best values for hyperparameters, tuning tests on the existing data needs to be ran. With hyperparameter tuning, we can use three strategies: grid search, random search, or weighted KNN. Grid search goes through every combination in a data set to find the best value for the hyperparameters. Random search quickly calculates a best value from the minimum amount of tests, but can be inaccurate. Weighted KNN makes inverse distance of points matter more than the mode of the nearest # neighbors. The steps for hyperparameter tuning best practices is: 

1. Test/train split
2. Cross validation with training data
3. Train best model on full training set
4. Compute test score with test set

For supervised learning, classification predicts a category while regression predicts a quantifiable value. To predict a value x, we compute the distance from x to all other training examples, find the k training examples that are closest to x, then output the average (mean) of these k closest examples.

Linear regression applies a set of data points to the line y = w0 + w1*x1 (if multiple predictor variables xn: [+ w2*x2 + ... + wn*xn].) We use MSE (mean squared error) as an indicator of how good a linear model fits the data, the lower MSE the better.

To figure out regressor performance, we can find the lowest MSE, RMSE (root mean squared error), or MAE (mean absolute error). To calculate MSE, we can use ((predicted–actual)**2).mean() . For RMSE, we simply square root MSE. For MAE, we use np.abs(predicted–actual).mean() . To cross-validate RMSE, we use scoring='neg_root_mean_squared_error' in the cross_val_score() parameters. For MAE, use scoring='neg_root_mean_absolute_error' instead.