ML lec 07-1: 학습 rate, Overfitting, 그리고 일반화 (Regularization)

Cost Function 정의 -> Gradient descent

 

Large learning

• Large learning rate : overshooting
• Small learning rate : takes too long, stops at local minimum

 

Try several learning rates

• Observe the cost function
• Check it goes down in a reasonable rate

 

Data Preprocessing

• Standardization

 

Overfitting

• 학습을 통해 모델을 만들어가는데
• 학습 데이터에만 딱 맞는 모델이 되어버려
• 실제 데이터에는 적합하지 않게됨
• 어느정도 오차를 허용 해야한다
• 오차를 허용하지 않는 모델 : overfitting 된 모델이다

 

Solutions for overfitting

• More training data
• Reduce the number of features
• Regularization : Let’s not have too big numbers in the weight
• regularization strength

 

Overshotting by large learning rate

Standardization for preprocessing