1 概述

基础的理论知识参考线性SVM与Softmax分类器

代码实现环境:python3

2 数据处理

2.1 加载数据集

将原始数据集放入“data/cifar10/”文件夹下。

### 加载cifar10数据集

import os
import pickle
import random
import numpy as np
import matplotlib.pyplot as plt def load_CIFAR_batch(filename):
"""
cifar-10数据集是分batch存储的,这是载入单个batch @参数 filename: cifar文件名
@r返回值: X, Y: cifar batch中的 data 和 labels
""" with open(filename,'rb') as f:
datadict=pickle.load(f,encoding='bytes') X=datadict[b'data']
Y=datadict[b'labels'] X=X.reshape(10000, 3, 32, 32).transpose(0,2,3,1).astype("float")
Y=np.array(Y) return X, Y def load_CIFAR10(ROOT):
"""
读取载入整个 CIFAR-10 数据集 @参数 ROOT: 根目录名
@return: X_train, Y_train: 训练集 data 和 labels
X_test, Y_test: 测试集 data 和 labels
""" xs=[]
ys=[] for b in range(1,6):
f=os.path.join(ROOT, "data_batch_%d" % (b, ))
X, Y=load_CIFAR_batch(f)
xs.append(X)
ys.append(Y) X_train=np.concatenate(xs)
Y_train=np.concatenate(ys) del X, Y X_test, Y_test=load_CIFAR_batch(os.path.join(ROOT, "test_batch")) return X_train, Y_train, X_test, Y_test X_train, y_train, X_test, y_test = load_CIFAR10('data/cifar10/') print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
print( y_test.shape)

运行结果如下:

(50000, 32, 32, 3)
(50000,)
(10000, 32, 32, 3)
(10000,)

2.2 划分数据集

将加载好的数据集划分为训练集,验证集,以及测试集。

## 划分训练集,验证集,测试集

num_train = 49000
num_val = 1000
num_test = 1000 # Validation set
mask = range(num_train, num_train + num_val)
X_val = X_train[mask]
y_val = y_train[mask] # Train set
mask = range(num_train)
X_train = X_train[mask]
y_train = y_train[mask] # Test set
mask = range(num_test)
X_test = X_test[mask]
y_test = y_test[mask] print('Train data shape: ', X_train.shape)
print('Train labels shape: ', y_train.shape)
print('Validation data shape: ', X_val.shape)
print('Validation labels shape ', y_val.shape)
print('Test data shape: ', X_test.shape)
print('Test labels shape: ', y_test.shape)

运行结果为:

Train data shape:  (49000, 3072)
Validation data shape: (1000, 3072)
Test data shape: (1000, 3072)

2.3 去均值归一化

将划分好的数据集归一化,即:所有划分好的数据集减去均值图像。

# Processing: subtract the mean images
mean_image = np.mean(X_train, axis=0) X_train -= mean_image
X_val -= mean_image
X_test -= mean_image # append the bias dimension of ones (i.e. bias trick)
X_train = np.hstack([X_train, np.ones((X_train.shape[0], 1))])#堆叠数组
X_val = np.hstack([X_val, np.ones((X_val.shape[0], 1))])
X_test = np.hstack([X_test, np.ones((X_test.shape[0], 1))])
print('Train data shape: ', X_train.shape)
print('Validation data shape: ', X_val.shape)
print('Test data shape: ', X_test.shape)

运行结果为:

Train data shape:  (49000, 3073)
Validation data shape: (1000, 3073)
Test data shape: (1000, 3073)

3 线性SVM分类器

3.1 定义线性SVM分类器

关键的是线性SVM的梯度推导过程。具体的可以看看这篇文章

#Define a linear SVM classifier

class LinearSVM(object):
""" A subclass that uses the Multiclass SVM loss function """
def __init__(self):
self.W = None def loss_vectorized(self, X, y, reg):
"""
Structured SVM loss function, naive implementation (with loops).
Inputs:
- X: A numpy array of shape (num_train, D) contain the training data
consisting of num_train samples each of dimension D
- y: A numpy array of shape (num_train,) contain the training labels,
where y[i] is the label of X[i]
- reg: (float) regularization strength
Outputs:
- loss: the loss value between predict value and ground truth
- dW: gradient of W
""" # Initialize loss and dW
loss = 0.0
dW = np.zeros(self.W.shape) # Compute the loss
num_train = X.shape[0]
scores = np.dot(X, self.W)
correct_score = scores[range(num_train), list(y)].reshape(-1, 1)
margin = np.maximum(0, scores - correct_score + 1) # delta = 1
margin[range(num_train), list(y)] = 0 #分对的损失为0
loss = np.sum(margin) / num_train + 0.5 * reg * np.sum(self.W * self.W) #reg就是权重lamda # Compute the dW
num_classes = self.W.shape[1]
mask = np.zeros((num_train, num_classes))
mask[margin > 0] = 1
mask[range(num_train), list(y)] = 0
mask[range(num_train), list(y)] = -np.sum(mask, axis=1)
dW = np.dot(X.T, mask)
dW = dW / num_train + reg * self.W return loss, dW def train(self, X, y, learning_rate = 1e-3, reg = 1e-5, num_iters = 100,
batch_size = 200, print_flag = False):
"""
Train linear SVM classifier using SGD
Inputs:
- X: A numpy array of shape (num_train, D) contain the training data
consisting of num_train samples each of dimension D
- y: A numpy array of shape (num_train,) contain the training labels,
where y[i] is the label of X[i], y[i] = c, 0 <= c <= C
- learning rate: (float) learning rate for optimization
- reg: (float) regularization strength
- num_iters: (integer) numbers of steps to take when optimization
- batch_size: (integer) number of training examples to use at each step
- print_flag: (boolean) If true, print the progress during optimization
Outputs:
- loss_history: A list containing the loss at each training iteration
""" loss_history = []
num_train = X.shape[0]
dim = X.shape[1]
num_classes = np.max(y) + 1 # Initialize W
if self.W == None:
self.W = 0.001 * np.random.randn(dim, num_classes) # iteration and optimization
for t in range(num_iters):
idx_batch = np.random.choice(num_train, batch_size, replace=True)
X_batch = X[idx_batch]
y_batch = y[idx_batch]
loss, dW = self.loss_vectorized(X_batch, y_batch, reg)
loss_history.append(loss)
self.W += -learning_rate * dW if print_flag and t%100 == 0:
print('iteration %d / %d: loss %f' % (t, num_iters, loss)) return loss_history def predict(self, X):
"""
Use the trained weights of linear SVM to predict data labels
Inputs:
- X: A numpy array of shape (num_train, D) contain the training data
Outputs:
- y_pred: A numpy array, predicted labels for the data in X
""" y_pred = np.zeros(X.shape[0])
scores = np.dot(X, self.W)
y_pred = np.argmax(scores, axis=1) return y_pred

3.2 无交叉验证

3.2.1 训练模型

##Stochastic Gradient Descent

svm = LinearSVM()
loss_history = svm.train(X_train, y_train, learning_rate = 1e-7, reg = 2.5e4, num_iters = 2000,
batch_size = 200, print_flag = True)

运行结果如下:

iteration 0 / 2000: loss 407.076351
iteration 100 / 2000: loss 241.030820
iteration 200 / 2000: loss 147.135737
iteration 300 / 2000: loss 90.274781
iteration 400 / 2000: loss 56.509895
iteration 500 / 2000: loss 36.654007
iteration 600 / 2000: loss 23.732160
iteration 700 / 2000: loss 16.340341
iteration 800 / 2000: loss 11.538806
iteration 900 / 2000: loss 9.482515
iteration 1000 / 2000: loss 7.414343
iteration 1100 / 2000: loss 6.240377
iteration 1200 / 2000: loss 5.774960
iteration 1300 / 2000: loss 5.569365
iteration 1400 / 2000: loss 5.326023
iteration 1500 / 2000: loss 5.708757
iteration 1600 / 2000: loss 4.731255
iteration 1700 / 2000: loss 5.516500
iteration 1800 / 2000: loss 4.959480
iteration 1900 / 2000: loss 5.447249

3.2.2 预测

# Use svm to predict
# Training set
y_pred = svm.predict(X_train)
num_correct = np.sum(y_pred == y_train)
accuracy = np.mean(y_pred == y_train)
print('Training correct %d/%d: The accuracy is %f' % (num_correct, X_train.shape[0], accuracy)) # Test set
y_pred = svm.predict(X_test)
num_correct = np.sum(y_pred == y_test)
accuracy = np.mean(y_pred == y_test)
print('Test correct %d/%d: The accuracy is %f' % (num_correct, X_test.shape[0], accuracy))

运行结果如下:

Training correct 18799/49000: The accuracy is 0.383653
Test correct 386/1000: The accuracy is 0.386000

3.3 有交叉验证

3.3.1 训练模型

#Cross-validation

learning_rates = [1.4e-7, 1.5e-7, 1.6e-7]
regularization_strengths = [8000.0, 9000.0, 10000.0, 11000.0, 18000.0, 19000.0, 20000.0, 21000.0] results = {}
best_lr = None
best_reg = None
best_val = -1 # The highest validation accuracy that we have seen so far.
best_svm = None # The LinearSVM object that achieved the highest validation rate. for lr in learning_rates:
for reg in regularization_strengths:
svm = LinearSVM()
loss_history = svm.train(X_train, y_train, learning_rate = lr, reg = reg, num_iters = 2000)
y_train_pred = svm.predict(X_train)
accuracy_train = np.mean(y_train_pred == y_train)
y_val_pred = svm.predict(X_val)
accuracy_val = np.mean(y_val_pred == y_val)
if accuracy_val > best_val:
best_lr = lr
best_reg = reg
best_val = accuracy_val
best_svm = svm
results[(lr, reg)] = accuracy_train, accuracy_val
print('lr: %e reg: %e train accuracy: %f val accuracy: %f' %
(lr, reg, results[(lr, reg)][0], results[(lr, reg)][1]))
print('Best validation accuracy during cross-validation:\nlr = %e, reg = %e, best_val = %f' %
(best_lr, best_reg, best_val))

3.3.2 预测

# Use the best svm to test
y_test_pred = best_svm.predict(X_test)
num_correct = np.sum(y_test_pred == y_test)
accuracy = np.mean(y_test_pred == y_test)
print('Test correct %d/%d: The accuracy is %f' % (num_correct, X_test.shape[0], accuracy))

运行结果为:

Test correct 372/1000: The accuracy is 0.372000

线性SVM分类器实战的更多相关文章

  1. 线性Softmax分类器实战

    1 概述 基础的理论知识参考线性SVM与Softmax分类器. 代码实现环境:python3 2 数据预处理 2.1 加载数据 将原始数据集放入"data/cifar10/"文件夹 ...

  2. 基于sklearn的分类器实战

    已迁移到我新博客,阅读体验更佳基于sklearn的分类器实战 完整代码实现见github:click me 一.实验说明 1.1 任务描述 1.2 数据说明 一共有十个数据集,数据集中的数据属性有全部 ...

  3. SVM-支持向量机(一)线性SVM分类

    SVM-支持向量机 SVM(Support Vector Machine)-支持向量机,是一个功能非常强大的机器学习模型,可以处理线性与非线性的分类.回归,甚至是异常检测.它也是机器学习中非常热门的算 ...

  4. 深度学习与计算机视觉系列(3)_线性SVM与SoftMax分类器

    作者: 寒小阳 &&龙心尘 时间:2015年11月. 出处: http://blog.csdn.net/han_xiaoyang/article/details/49949535 ht ...

  5. 线性SVM与Softmax分类器

    1 引入 上一篇介绍了图像分类问题.图像分类的任务,就是从已有的固定分类标签集合中选择一个并分配给一张图像.我们还介绍了k-Nearest Neighbor (k-NN)分类器,该分类器的基本思想是通 ...

  6. SVM1 线性SVM

    一.Linear Support Vector Machine 接下来的讨论假设数据都是线性可分的. 1.1 SVM的引入:增大对测量误差的容忍度 假设有训练数据和分类曲线如下图所示: 很明显,三个分 ...

  7. 机器学习经典算法详解及Python实现--基于SMO的SVM分类器

    原文:http://blog.csdn.net/suipingsp/article/details/41645779 支持向量机基本上是最好的有监督学习算法,因其英文名为support vector  ...

  8. 支持向量机(Support Vector Machine,SVM)—— 线性SVM

      支持向量机(Support Vector Machine,简称 SVM)于 1995 年正式发表,由于其在文本分类任务中的卓越性能,很快就成为机器学习的主流技术.尽管现在 Deep Learnin ...

  9. 自己训练SVM分类器进行HOG行人检测

    正样本来源是INRIA数据集中的96*160大小的人体图片,使用时上下左右都去掉16个像素,截取中间的64*128大小的人体. 负样本是从不包含人体的图片中随机裁取的,大小同样是64*128(从完全不 ...

随机推荐

  1. 搭建HDFS HA

    搭建HDFS HA 1.服务器角色规划 hd-01(192.168.1.99) hd-02 (192.168.1.100) hd-03 (192.168.1.101) NameNode NameNod ...

  2. Python全栈开发:django网络框架(一)

    Python的WEB框架有Django.Tornado.Flask 等多种,Django相较与其他WEB框架其优势为:大而全,框架本身集成了ORM.模型绑定.模板引擎.缓存.Session等诸多功能. ...

  3. 微信小程序为什么看不到所有的console.log()的日志信息

    记录一个巨傻无比的问题 1.在首页的onLoad()函数里面,加了地理位置的加载,并打印到控制台上,可是今天就是没出现 2.然后纳闷的很久,各种google,发现没有人遇到这个问题 3.再然后,我就看 ...

  4. Java Collection - 遍历map的几种方式

    作者:zhaoguhong(赵孤鸿) 出处:http://www.cnblogs.com/zhaoguhong/ 本文版权归作者和博客园共有,转载请注明出处 ---------------- 总结 如 ...

  5. Json数据交换一Jackson

    依赖 <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>ja ...

  6. 广义欧拉降幂(欧拉定理)——bzoj3884,fzu1759

    广义欧拉降幂对于狭义欧拉降幂任然适用 https://blog.csdn.net/qq_37632935/article/details/81264965?tdsourcetag=s_pctim_ai ...

  7. 【转载】Delphi异常处理try except语句和try finally语句用法以及区别

    Delphi异常处理try except语句和try finally语句用法以及区别 一直写程序都没管他们,也尽量很少用,今天终于想把他给弄个明白,在网上找来,记下!主要是前面小部分,后面的是详细说明 ...

  8. PAT甲级——A1115 Counting Nodes in a BST【30】

    A Binary Search Tree (BST) is recursively defined as a binary tree which has the following propertie ...

  9. <day004>小娜显示空白+CSV文件的基本操作+普通的代理使用

    小知识: 当小娜搜索显示空白的时候,怎么解决? 任务管理器结束小娜进程就好了= =*(多半是惯得,关掉就好了!) 任务1:CSV文件的基本操作 import csv import pandas as ...

  10. 编写main方法