CST383 - Week 7

The final week of this course! This week, I learned about how to create logistic regression models, as well as understanding overfitting and how to reduce the effects of it.

We also learned how to encode categorical variables. One-hot encoding is what I've used for a number of homework problems. It does not create new columns unlike using dummy variables, and encodes two-value nominal variables with 1 or 0. One example is "sex", where a one-hot encoded version would be recreating the column as "sex_female" with 1 == female and 0 == male. For ordinal variables, replace the variables with increasing order, like "spicy". This can be convert the possible values of "spicy" into dummy variables "spicy_none", "spicy_mild", "spicy_hot", and "spicy_extra". We can encode as many columns as needed, as long as the number of rows >= the number of columns squared.

Logistic regression models are created using categorical variables. We set them up similarly to linear regression models, but we also use a squasher (sigmoid function) to map the data from 0 to 1.

Overfitting is a situation where the training set errors and test set errors have a large difference between them (high variance). Ideally, you would have some difference between the training set and test set errors, but not too small of a difference (low variance). In addition, bias can affect how well the model can capture the data's shape. A model can have high bias if there isn't enough data for the model to create an accurate shape.