Logistic Regression 用于预测马是否生病

1.利用Logistic regression 进行分类的主要思想

根据现有数据对分类边界线建立回归公式，即寻找最佳拟合参数集，然后进行分类。

2.利用梯度下降找出最佳拟合参数

3.代码实现

 # -*- coding: utf-8 -*-
 """
 Created on Tue Mar 28 21:35:25 2017

 @author: MyHome
 """
 import numpy as np
 from random import uniform
 '''定义sigmoid函数'''
 def sigmoid(inX):
     return 1.0 /(1.0 +np.exp(-inX))

 '''使用随机梯度下降更新权重，并返回最终值'''
 def StocGradientDescent(dataMatrix,classLabels,numIter = 600):
     m,n = dataMatrix.shape
     #print m,n
     weights = np.ones(n)
     for j in xrange(numIter):
         dataIndex = range(m)

         for i in xrange(m):

             alpha = 4 / (1.0+j+i) + 0.01
             randIndex = int(uniform(0,len(dataIndex)))
             h = sigmoid(sum(dataMatrix[randIndex]*weights))
             gradient = (h - classLabels[randIndex])*dataMatrix[randIndex]
             weights = weights - alpha*gradient
             del(dataIndex[randIndex])

     return weights

 '''创建分类器'''
 def classifyVector(inX,weights):
     prob = sigmoid(sum(inX*weights))
     if prob > 0.5:
         return 1.0
     else:
         return 0.0

 '''测试'''
 def Test():

     frTrain = open("horseColicTraining.txt")
     frTest = open("horseColicTest.txt")
     trainingSet = []
     trainingLabel = []
     for line in frTrain.readlines():
         currLine = line.strip().split("\t")
         lineArr = []
         for i in range(21):
             lineArr.append(float(currLine[i]))
         trainingSet.append(lineArr)
         trainingLabel.append(float(currLine[21]))
     trainWeights = StocGradientDescent(np.array(trainingSet),trainingLabel)
     errorCount = 0.0
     numTestVec = 0.0
     for line in frTest.readlines():
         numTestVec += 1.0
         currLine = line.strip().split("\t")
         lineArr = []
         for i in range(21):
             lineArr.append(float(currLine[i]))
         if int(classifyVector(np.array(lineArr),trainWeights)) != int(currLine[21]):
             errorCount += 1
     errorRate = (float(errorCount)/numTestVec)
     print "the error rate of this test is:%f"%errorRate
     return errorRate

 '''调用Test()10次求平均值'''
 def multiTest():
     numTest = 10
     errorSum = 0.0
     for k in range(numTest):
         errorSum += Test()
     print "after %d iterations the average errror rate is:\
         %f"%(numTest,errorSum/float(numTest))

 if __name__ == "__main__":
     multiTest()

结果：

the error rate of this test is:0.522388
the error rate of this test is:0.328358

the error rate of this test is:0.313433

the error rate of this test is:0.358209

the error rate of this test is:0.298507

the error rate of this test is:0.343284

the error rate of this test is:0.283582

the error rate of this test is:0.313433

the error rate of this test is:0.343284

the error rate of this test is:0.358209

after 10 iterations the average errror rate is:        0.346269

4.总结

Logistic regression is finding best-fit parameters to a nonlinear function called the sigmoid.

Methods of optimization can be used to find the best-fit parameters. Among the

optimization algorithms, one of the most common algorithms is gradient descent. Gradient

desent can be simplified with stochastic gradient descent.

Stochastic gradient descent can do as well as gradient descent using far fewer computing

resources. In addition, stochastic gradient descent is an online algorithm; it can

update what it has learned as new data comes in rather than reloading all of the data

as in batch processing.

One major problem in machine learning is how to deal with missing values in the

data. There’s no blanket answer to this question. It really depends on what you’re

doing with the data. There are a number of solutions, and each solution has its own

advantages and disadvantages.