Random-Forest-Python

1. 近期目标，实现随机森林进行点云分类

　　1）学习阶段：

【干货】Kaggle 数据挖掘比赛经验分享

Kaggle Machine Learning Competition: Predicting Titanic Survivors

Kaggle Titanic 生存预测 -- 详细流程吐血梳理

机器学习实战之Kaggle_Titanic预测

https://www.codeproject.com/Articles/1197167/Random-Forest-Python

https://blog.csdn.net/hexingwei/article/details/50740404

　　2）实践阶段：

　　（1）原始点云字段（X,Y,Z,density,curvature,Classification），利用点云的高程Z，密度和曲率进行train和分类。分类结果很差就是了。

　　　　需要考虑哪些特征对分类结果的影响比较大？用什么样的点云特征更好，特征工程问题?

 # -*- coding: utf-8 -*-
 """
 Created on Sat Nov 10 10:12:02 2018
 @author: yhexie
 """
 import numpy as np
 import pandas as pd
 from sklearn import model_selection
 from sklearn.ensemble import RandomForestClassifier

 df = pd.read_csv('C:/Users/yhexie/.spyder-py3/pointcloudcls/train_pcloud2.csv', header=0)
 x_train = df[['Z','Volume','Ncr']]
 y_train = df.Classification

 df2 = pd.read_csv('C:/Users/yhexie/.spyder-py3/pointcloudcls/test_pcloud2.csv', header=0)
 x_test = df2[['Z','Volume','Ncr']]

 clf = RandomForestClassifier(n_estimators=10)
 clf.fit(x_train, y_train)
 clf_y_predict = clf.predict(x_test)

 data_arry=[]
 data_arry.append(df2.X)
 data_arry.append(df2.Y)
 data_arry.append(df2.Z)
 data_arry.append(clf_y_predict)

 np_data = np.array(data_arry)
 np_data = np_data.T
 np.array(np_data)
 save = pd.DataFrame(np_data, columns = ['X','Y','Z','Classification'])
 save.to_csv('C:/Users/yhexie/.spyder-py3/pointcloudcls/predict_pcloud2.csv',index=False,header=True)  #index=False,header=False表示不保存行索引和列标题

　　（2）对训练集进行split，用75%的数据训练，25%的数据验证模型的拟合精度和泛化能力。

　　　　a. 增加定性特征，进行dummy处理。

　　目前采用Z值和8个特征相关的点云特征进行分类，点云近邻搜索半径2.5m

 # -*- coding: utf-8 -*-
 """
 Created on Wed Nov 28 10:54:48 2018

 @author: yhexie
 """

 import numpy as np
 import pandas as pd
 import matplotlib.pyplot as plt
 from sklearn import model_selection
 from sklearn.ensemble import RandomForestClassifier

 df = pd.read_csv('C:/Users/yhexie/.spyder-py3/pointcloudcls/train_pc.csv', header=0)
 x_train = df[['Z','Linearity',    'Planarity','Scattering','Omnivariance',    'Anisotropy',
               'EigenEntropy','eig_sum'    ,'changeOfcurvature']]
 y_train = df.Classification

 from sklearn.cross_validation import train_test_split
 train_data_X,test_data_X,train_data_Y,test_data_Y = train_test_split(x_train, y_train, test_size=0.25, random_state=33)

 df2 = pd.read_csv('C:/Users/yhexie/.spyder-py3/pointcloudcls/test_pc.csv', header=0)
 x_test = df2[['Z','Linearity',    'Planarity','Scattering','Omnivariance',    'Anisotropy',
               'EigenEntropy','eig_sum'    ,'changeOfcurvature']]

 clf = RandomForestClassifier(n_estimators=10)
 clf.fit(train_data_X, train_data_Y)

 print('Accuracy on training set:{:.3f}:'.format(clf.score(train_data_X,train_data_Y)))
 print('Accuracy on training set:{:.3f}:'.format(clf.score(test_data_X,test_data_Y)))
 print('Feature inportances:{}'.format(clf.feature_importances_))
 n_features=9
 plt.barh(range(n_features),clf.feature_importances_,align='center')
 plt.yticks(np.arange(n_features),['Z','Linearity',    'Planarity','Scattering','Omnivariance',    'Anisotropy',
               'EigenEntropy','eig_sum'    ,'changeOfcurvature'])
 plt.xlabel('Feature importance')
 plt.ylabel('Feature')

 clf_y_predict = clf.predict(x_test)

 data_arry=[]
 data_arry.append(df2.X)
 data_arry.append(df2.Y)
 data_arry.append(df2.Z)
 data_arry.append(clf_y_predict)

 np_data = np.array(data_arry)
 np_data = np_data.T
 np.array(np_data)
 save = pd.DataFrame(np_data, columns = ['X','Y','Z','Classification'])
 save.to_csv('C:/Users/yhexie/.spyder-py3/pointcloudcls/predict_pcloud2.csv',index=False,header=True)  #index=False,header=False表示不保存行索引和列标题

　　计算结果：可以看到在测试集上的结果还是很差

 Accuracy on training set:0.984:
 Accuracy on test set:0.776:

特征重要程度：

---

新的测试：

Accuracy on training set:0.994:
Accuracy on training set:0.891:
Feature inportances:[0.02188956 0.02742479 0.10124688 0.01996966 0.1253002 0.02563489
0.03265565 0.100919 0.15808224 0.01937961 0.02727676 0.05498342
0.0211147 0.02387439 0.01900164 0.023478 0.02833916 0.0302441
0.02249598 0.06629199 0.05039737]

感觉Z值的重要程度太高了。房屋分类结果应该是很差，绿色的很多被错误分类了。

问题：目前训练集中的每个类别的样本数目并不相同，这个对训练结果有没有影响？