Ch6: Linear Regression and Its Cousins

• Robust regression
  • Normal least square regression is not robust against outliers.
  • Use the L1 norm for residual computation.
  • Huber approach: use L2 norm when the difference is small, L1 when the difference is large.
  • R: rlm function in MASS library.
• Partial least squares (PLS)
  • SIMPLS: An alternative approach to partial least squares regression by de Jong
• Principal component analysis (PCA)
  • Eigenvalue decomposision of the data covariance matrix
  • Singular value decomposition of the data matrix