SKlearn库学习曲线

思想：

# 1.现将所有样本用交叉验证方法或者（随机抽样方法) 得到 K对 训练集-验证集
# 2.依次对K个训练集，拿出数量不断增加的子集如m个，并在这些K*m个子集上训练模型。
# 3.依次在对应训练集子集、验证集上计算得分。
# 4.对每种大小下的子集，计算K次训练集得分均值和K次验证集得分均值，共得到m对值。
# 5.绘制学习率曲线。x轴训练集样本量，y轴模型得分或预测准确率。

用到的方法：

learning_curve #直接得到1个模型在不同训练集大小参数下：1.训练集大小 2.训练得分 3.测试得分

ShuffleSplit  #实现交叉验证、或 随机抽样划分不同的训练集合验证集

plt.fill_between

python代码：

import numpy as np
from sklearn.model_selection import learning_curve, ShuffleSplit
from sklearn.datasets import load_digits
from sklearn.naive_bayes import GaussianNB
from sklearn import svm
import matplotlib.pyplot as plt
def plot_learning_curve(estimator, title, X, y, ylim=None, cv=None,n_jobs=1, train_size=np.linspace(.1, 1.0, 5 )):
    if __name__ == '__main__':
        plt.figure()
        plt.title(title)
        if ylim is not None:
            plt.ylim(*ylim)
        plt.xlabel('Training example')
        plt.ylabel('score')
        train_sizes, train_scores, test_scores = learning_curve(estimator, X, y, cv=cv, n_jobs=n_jobs, train_sizes=train_size)
        train_scores_mean = np.mean(train_scores, axis=1)
        train_scores_std = np.std(train_scores, axis=1)
        test_scores_mean = np.mean(test_scores, axis=1)
        test_scores_std = np.std(test_scores, axis=1)
        plt.grid()#区域
        plt.fill_between(train_sizes, train_scores_mean - train_scores_std,
                         train_scores_mean + train_scores_std, alpha=0.1,
                         color="r")
        plt.fill_between(train_sizes, test_scores_mean - test_scores_std,
                         test_scores_mean + test_scores_std, alpha=0.1,
                         color="g")
        plt.plot(train_sizes, train_scores_mean, 'o-', color='r',
                 label="Training score")
        plt.plot(train_sizes, test_scores_mean,'o-',color="g",
                 label="Cross-validation score")
        plt.legend(loc="best")
        return plt
digits = load_digits()
X = digits.data
y = digits.target
cv = ShuffleSplit(n_splits=100, test_size=0.2, random_state=0)#切割100ci
estimator = GaussianNB()
title = "Learning Curves(naive_bayes)"
plot_learning_curve(estimator, title, X, y, ylim=(0.7, 1.01), cv=cv, n_jobs=4)  

title = "Learning Curves(SVM,RBF kernel, $\gamma=0.001$)"
cv = ShuffleSplit(n_splits=10, test_size=0.2, random_state=0)#交叉验证传入别的方法，而不是默认的k折交叉验证
estimator = svm.SVC(gamma=0.001)
plot_learning_curve(estimator, title, X, y, (0.7, 1.01), cv=cv, n_jobs=4)
plt.show() 

结果：

说明：

1. 贝叶斯模型上，训练集规模达到1100时已经比较合适，太小的话会导致过拟合。。总的来看，该模型的准确率趋向于0.85

2. SVM模型上，训练集规模到800时已经比较合适，太小的话会导致训练集无代表性，对验证集无预测能力。。。总的来看，该模型在这份数据上的表现比第一个好。

参考

http://www.360doc.com/content/18/0424/22/50223086_748481549.shtml