residual sum of squares(ESL 读书笔记)

The learning algorithm has the property that it can modify its input/output relationship f-hat in response to differences y_i-f(x_i)-hat between the original and generated outputs.

For the linear model we get a simple closed form solution to the minimization problem. This is also true for the basis function methods, if the basis functions themselves do not have any hidden parameters.

Otherwise the solution requires either iterative methods or numerical optimization.