python/numpy/pandas数据操作知识与技巧

pandas针对dataframe各种操作技巧集合:

filtering:

一般地，使用df.column > xx将会产生一个只有boolean值的series，以该series作为dataframe的选择器(index/slicing)将直接选中该series中所有value为true的记录。

df[df.salt>60]  # 返回所有salt大于60的行
df[(df.salt>50)&(df.eggs < 300)] # 返回salt大于50并且eggs小于300的行
print(df2.loc[:,df2.all()]) # 打印不含0值的所有列(所有行)
print(df2.loc[:,df2.any()]) #打印所有含非0值的所有列(所有行)
print(df2.loc[:,df2.isnull().any()]) #打印所有包含一个NaN值的列(所有行)
print(df2.loc[:,df2.notnull().all()]) #打印所有满值列(不含空值)(所有行)
df.dropna(how='any') # 将任何含有nan的行删除

filter过滤并赋值

# Create the boolean array: too_close
too_close = election['margin']<1
# Assign np.nan to the 'winner' column where the results were too close to call
election.loc[too_close,'winner'] = np.nan
# 等价于以下，需要注意的是[column][row]和loc[row,column]是反过来的哦！！！！
election['winner'][too_close] = np.nan

dict(list(zip()))创建DataFrame

就地修改某列数据类型为数值型，无法parse成功的则设为NaN

df['salt'] = pd.to_numeric(df['salt'],errors='coerce')

setting index with combined column:列组合作为index(比如股票名称+日期)

获取df.loc['rowname','colname']==df.iloc[x,y]中的x和y

x = election.index.get_loc('Bedford') # 行名称为Bedford
y = election.columns.get_loc('winner') #列名称为winner
# 这时:
election.loc['Bedford','winner'] == election.iloc[x,y]

election.winner[too_close] = np.nan