DbUtility-关于DataTable转成List的效率问题
DbUtility中的方法ExecuteDataTableAsync()得到的是一个DataTable，而我们常见的情况下，我们需要的不是DataTable，而是List或IList，所以现在需要考虑把DataTable转成List或IList，目前暂时有三种方案：
方案1：用动软生成代码：
public static List<Roles> GetRoleses(DataTable dt)
{
List<Roles> list = new List<Roles>();
)
list.AddRange(from DataRow dataRow in dt.Rows select DataRowToModel(dataRow));
return list;
}
/// <summary>
/// 得到一个对象实体
/// </summary>
private static Roles DataRowToModel(DataRow row)
{
Roles model = new Roles();
if (row != null)
{
if (row["RoleID"] != null && row["RoleID"].ToString() != "")
{
model.RoleID = new Guid(row["RoleID"].ToString());
}
if (row["RoleName"] != null)
{
model.RoleName = row["RoleName"].ToString();
}
if (row["Description"] != null)
{
model.Description = row["Description"].ToString();
}
if (row["TaskMask"] != null)
{
model.TaskMask = row["TaskMask"].ToString();
}
if (row["RoleFlags"] != null && row["RoleFlags"].ToString() != "")
{
model.RoleFlags = int.Parse(row["RoleFlags"].ToString());
}
}
return model;
}
循环遍历100W次，大约用时：14460毫秒。
方案2：用MVC.Net提供的反射方案：
public static IList<T> ConvertToModel<T>(this DataTable dt) where T : class, new()
{
// 定义集合
IList<T> ts = new List<T>();
// 获得此模型的类型
Type type = typeof(T);
string tempName = "";
foreach (DataRow dr in dt.Rows)
{
T t = new T();
// 获得此模型的公共属性
PropertyInfo[] propertys = t.GetType().GetProperties();
foreach (PropertyInfo pi in propertys)
{
tempName = pi.Name;
// 检查DataTable是否包含此列
if (dt.Columns.Contains(tempName))
{
// 判断此属性是否有Setter
if (!pi.CanWrite) continue;
object value = dr[tempName];
if (value != DBNull.Value)
pi.SetValue(t, value, null);
}
}
ts.Add(t);
}
return ts;
}
反射100W次，大约用时：20350毫秒。
方案3：用Melas提供的反射方案：
public static IList<T> ConvertTo<T>(DataTable table)
{
if (table == null)
{
return null;
}
List<DataRow> rows = new List<DataRow>();
foreach (DataRow row in table.Rows)
{
rows.Add(row);
}
return ConvertTo<T>(rows);
}
public static IList<T> ConvertTo<T>(IList<DataRow> rows)
{
IList<T> list = null;
if (rows != null)
{
list = new List<T>();
foreach (DataRow row in rows)
{
T item = CreateItem<T>(row);
list.Add(item);
}
}
return list;
}
public static T CreateItem<T>(DataRow row)
{
T obj = default(T);
if (row != null)
{
obj = Activator.CreateInstance<T>();
foreach (DataColumn column in row.Table.Columns)
{
object value = row[column.ColumnName];
PropertyInfo prop = obj.GetType().GetProperty(column.ColumnName);
try
{
if (value != DBNull.Value && prop != null)
prop.SetValue(obj, value, null);
}
catch
{
// You can log something here
throw;
}
}
}
return obj;
}
反射100W次，大约用时：20258毫秒。
数据库结构：
表名：Roles
|
原本想修改动软代码生成器的模板来生成改写过的DataRowToModel方法，想改成下面这样：
/// <summary>
/// 得到一个对象实体
/// </summary>
private static Roles DataRowToModel(DataRow row)
{
if (row != null)
{
Roles model = new Roles
{
RoleID = new Guid(row["RoleID"].ToString()),
RoleName = row["RoleName"].ToString(),
Description = row["Description"].ToString(),
TaskMask = row["TaskMask"].ToString(),
RoleFlags = int.Parse(row["RoleFlags"].ToString())
};
return model;
}
return null;
}
后来发现一个很悲剧的事，模板里不包含DataRowToModel方法，我看到了下面这个东西：
目前用到的代码生成，我用CodeSmith生成，下面贴出两个模板，可以直接用，已经测试通过。
实体层：
<%@ CodeTemplate Language="C#" TargetLanguage="C#" Description="Generates a very simple business object." %>
<%@ Property Name="SourceTable" Type="SchemaExplorer.TableSchema" Category="Context" Description="Table that the object is based on." %>
<%@ Property Name="SingleFileMode" Type="System.Boolean" Category="Options" Default="True" Description="Generate content for a complete cs file instead of just a class." %>
<%@ Property Name="ClassNamespace" Type="System.String" Category="Options" Default="BusinessObjects" Description="Namespace your class is in. Only used if SingleFileMode is true!" %>
<%@ Assembly Name="SchemaExplorer" %>
<%@ Assembly Name="System.Data" %>
<%@ Import Namespace="SchemaExplorer" %>
<%@ Import Namespace="System.Data" %>
<%@ Map Name="SqlCSharp" Src="Sql-CSharp" Description="System to C# Type Map" %>
<%@ Map Name="DbDataReader" Src="DbType-DataReaderMethod" Description="DbType to DataReader Method Map" %>
<%@ Map Name="SqlNativeSqlDb" Src="SqlNativeType-SqlDbType" Description="SqlNativeType to SqlDbType Map" %>
<%@ Map Name="DbTypeCSharp" Src="DbType-CSharp" Description="DbType to CSharp Map" %>
<% if(this.SingleFileMode) { %>
using System;
namespace <%= this.ClassNamespace %>
{
<% } %>
/// <summary>
/// <%= GetClassName(SourceTable) %>:实体类(属性说明自动提取数据库字段的描述信息)
/// </summary>
[Serializable]
public class <%= GetClassName(SourceTable) %>
{
#region Private Properties
<% foreach (ColumnSchema column in SourceTable.Columns) { %>
<%= GetMemberVariableDeclarationStatement(column) %>
<% } %>
#endregion
#region Public Properties
<% ; i < SourceTable.Columns.Count; i++) { %>
/// <summary>
/// <%=SourceTable.Columns[i].Description %>
/// </summary>
public <%= GetCSharpVariableType(SourceTable.Columns[i]) %> <%= GetPropertyName(SourceTable.Columns[i]) %>
{
get {return <%= GetMemberVariableName(SourceTable.Columns[i]) %>;}
set {<%= GetMemberVariableName(SourceTable.Columns[i]) %> = value;}
}
<% ) Response.Write("\r\n"); %>
<% } %>
#endregion
}
<% if(this.SingleFileMode) { %>
}
<% } %>
<script runat="template">
public string GetMemberVariableDeclarationStatement(ColumnSchema column)
{
return GetMemberVariableDeclarationStatement("private", column);
}
public string GetMemberVariableDeclarationStatement(string protectionLevel, ColumnSchema column)
{
string statement = protectionLevel + " ";
statement += GetCSharpVariableType(column) + " " + GetMemberVariableName(column);
string defaultValue = GetMemberVariableDefaultValue(column);
if (defaultValue != "")
{
statement += " = " + defaultValue;
}
statement += ";";
return statement;
}
public string GetCamelCaseName(string value)
{
, ).ToLower() + value.Substring();
}
public string GetMemberVariableName(ColumnSchema column)
{
string propertyName = GetPropertyName(column);
string memberVariableName = "_" + GetCamelCaseName(propertyName);
return memberVariableName;
}
public string GetPropertyName(ColumnSchema column)
{
string propertyName = column.Name;
if (propertyName == column.Table.Name + "Name") return "Name";
if (propertyName == column.Table.Name + "Description") return "Description";
, propertyName.Length - );
return propertyName;
}
public string GetMemberVariableDefaultValue(ColumnSchema column)
{
switch (column.DataType)
{
case DbType.Guid:
{
return "Guid.Empty";
}
case DbType.AnsiString:
case DbType.AnsiStringFixedLength:
case DbType.String:
case DbType.StringFixedLength:
{
return "String.Empty";
}
default:
{
return "";
}
}
}
public string GetCSharpVariableType(ColumnSchema column)
{
if (column.Name.EndsWith("TypeCode")) return column.Name;
return DbTypeCSharp[column.DataType.ToString()];
}
public string GetClassName(TableSchema table)
{
return table.Name;
}
public override string GetFileName()
{
return this.GetClassName(this.SourceTable) + ".cs";
}
</script>
数据访问层：
<%@ CodeTemplate Language="C#" TargetLanguage="C#" Description="Generates a very simple business object." %>
<%@ Property Name="SourceTable" Type="SchemaExplorer.TableSchema" Category="Context" Description="Table that the object is based on." %>
<%@ Property Name="SingleFileMode" Type="System.Boolean" Category="Options" Default="True" Description="Generate content for a complete cs file instead of just a class." %>
<%@ Property Name="ClassNamespace" Type="System.String" Category="Options" Default="BusinessObjects" Description="Namespace your class is in. Only used if SingleFileMode is true!" %>
<%@ Assembly Name="SchemaExplorer" %>
<%@ Assembly Name="System.Data" %>
<%@ Import Namespace="SchemaExplorer" %>
<%@ Import Namespace="System.Data" %>
<%@ Map Name="SqlCSharp" Src="Sql-CSharp" Description="System to C# Type Map" %>
<%@ Map Name="DbDataReader" Src="DbType-DataReaderMethod" Description="DbType to DataReader Method Map" %>
<%@ Map Name="SqlNativeSqlDb" Src="SqlNativeType-SqlDbType" Description="SqlNativeType to SqlDbType Map" %>
<%@ Map Name="DbTypeCSharp" Src="DbType-CSharp" Description="DbType to CSharp Map" %>
<% if(this.SingleFileMode) { %>
using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
namespace <%= this.ClassNamespace %>
{
<% } %>
public class <%= GetClassName(SourceTable) %>DAL
{
/// <summary>
/// 把DataTable转成一个List集合
/// </summary>
public static List<<%= GetClassName(SourceTable) %>> GetRoleses(DataTable dt)
{
List<<%= GetClassName(SourceTable) %>> list = new List<<%= GetClassName(SourceTable) %>>();
)
list.AddRange(from DataRow dataRow in dt.Rows select DataRowToModel(dataRow));
return list;
}
/// <summary>
/// 得到一个对象实体
/// </summary>
private static <%= GetClassName(SourceTable) %> DataRowToModel(DataRow row)
{
if (row != null)
{
<%= GetClassName(SourceTable) %> model = new <%= GetClassName(SourceTable) %>
{
<% ; i < SourceTable.Columns.Count; i++) { %>
<%= GetReaderAssignmentStatement(SourceTable.Columns[i], i,SourceTable.Columns.Count-) %>
<%}%>
};
return model;
}
return null;
}
}
<% if(this.SingleFileMode) { %>
}
<% } %>
<script runat="template">
public string GetReaderAssignmentStatement(ColumnSchema column, int index,int count)
{
string statement = GetMemberVariableName(column) + " = ";
if (column.Name.EndsWith("TypeCode"))
statement += "(" + column.Name + ")";
statement += GetMemberVariableDefaultValue(column);
if(index<count)
statement+=",";
return statement;
}
public string GetMemberVariableName(ColumnSchema column)
{
return GetPropertyName(column);
}
public string GetPropertyName(ColumnSchema column)
{
string propertyName = column.Name;
if (propertyName == column.Table.Name + "Name") return "Name";
if (propertyName == column.Table.Name + "Description") return "Description";
, propertyName.Length - );
return propertyName;
}
public string GetMemberVariableDefaultValue(ColumnSchema column)
{
switch (column.DataType)
{
case DbType.Guid:
{
return "new Guid(row[\""+column.Name+"\"].ToString())";
}
case DbType.AnsiString:
case DbType.AnsiStringFixedLength:
case DbType.String:
case DbType.StringFixedLength:
{
return "row[\""+column.Name+"\"].ToString()";
}
case DbType.Byte:
return "Byte.Parse(row[\""+column.Name+"\"].ToString())";
case DbType.Int16:
return "Int16.Parse(row[\""+column.Name+"\"].ToString())";
case DbType.Int32:
return "Int32.Parse(row[\""+column.Name+"\"].ToString())";
case DbType.Int64:
return "Int64.Parse(row[\""+column.Name+"\"].ToString())";
default:
{
return "aaaa";
}
}
}
public string GetCSharpVariableType(ColumnSchema column)
{
if (column.Name.EndsWith("TypeCode")) return column.Name;
return DbTypeCSharp[column.DataType.ToString()];
}
public string GetClassName(TableSchema table)
{
return table.Name;
}
public override string GetFileName()
{
return this.GetClassName(this.SourceTable) + "DAL.cs";
}
</script>
上面的模板生成的是1楼评论中的代码。
后面经过MVC.Net的提醒，改用构造方法来做，具体做法是：
数据访问层就不需要了，只需要一个实体层模板：
<%@ CodeTemplate Language="C#" TargetLanguage="C#" Description="Generates a very simple business object." %>
<%@ Property Name="SourceTable" Type="SchemaExplorer.TableSchema" Category="Context" Description="Table that the object is based on." %>
<%@ Property Name="SingleFileMode" Type="System.Boolean" Category="Options" Default="True" Description="Generate content for a complete cs file instead of just a class." %>
<%@ Property Name="ClassNamespace" Type="System.String" Category="Options" Default="BusinessObjects" Description="Namespace your class is in. Only used if SingleFileMode is true!" %>
<%@ Assembly Name="SchemaExplorer" %>
<%@ Assembly Name="System.Data" %>
<%@ Import Namespace="SchemaExplorer" %>
<%@ Import Namespace="System.Data" %>
<%@ Map Name="SqlCSharp" Src="Sql-CSharp" Description="System to C# Type Map" %>
<%@ Map Name="DbDataReader" Src="DbType-DataReaderMethod" Description="DbType to DataReader Method Map" %>
<%@ Map Name="SqlNativeSqlDb" Src="SqlNativeType-SqlDbType" Description="SqlNativeType to SqlDbType Map" %>
<%@ Map Name="DbTypeCSharp" Src="DbType-CSharp" Description="DbType to CSharp Map" %>
<% if(this.SingleFileMode) { %>
using System;
using System.Data;
namespace <%= this.ClassNamespace %>
{
<% } %>
/// <summary>
/// <%= GetClassName(SourceTable) %>:实体类(属性说明自动提取数据库字段的描述信息)
/// </summary>
[Serializable]
public class <%= GetClassName(SourceTable) %>
{
#region Private Properties
<% foreach (ColumnSchema column in SourceTable.Columns) { %>
<%= GetMemberVariableDeclarationStatement(column) %>
<% } %>
#endregion
#region Public Properties
public <%= GetClassName(SourceTable) %>(DataRow row)
{
<% ; i < SourceTable.Columns.Count; i++) { %>
<%=GetMemberVariableName(SourceTable.Columns[i])%> = <%= GetMemberVariableDefaultValue1(SourceTable.Columns[i])%>;
<%}%>
}
<% ; i < SourceTable.Columns.Count; i++) { %>
/// <summary>
/// <%=SourceTable.Columns[i].Description %>
/// </summary>
public <%= GetCSharpVariableType(SourceTable.Columns[i]) %> <%= GetPropertyName(SourceTable.Columns[i]) %>
{
get {return <%= GetMemberVariableName(SourceTable.Columns[i]) %>;}
set {<%= GetMemberVariableName(SourceTable.Columns[i]) %> = value;}
}
<% ) Response.Write("\r\n"); %>
<% } %>
#endregion
}
<% if(this.SingleFileMode) { %>
}
<% } %>
<script runat="template">
public string GetMemberVariableDeclarationStatement(ColumnSchema column)
{
return GetMemberVariableDeclarationStatement("private", column);
}
public string GetMemberVariableDeclarationStatement(string protectionLevel, ColumnSchema column)
{
string statement = protectionLevel + " ";
statement += GetCSharpVariableType(column) + " " + GetMemberVariableName(column);
string defaultValue = GetMemberVariableDefaultValue(column);
if (defaultValue != "")
{
statement += " = " + defaultValue;
}
statement += ";";
return statement;
}
public string GetCamelCaseName(string value)
{
, ).ToLower() + value.Substring();
}
public string GetMemberVariableName(ColumnSchema column)
{
string propertyName = GetPropertyName(column);
string memberVariableName = "_" + GetCamelCaseName(propertyName);
return memberVariableName;
}
public string GetPropertyName(ColumnSchema column)
{
string propertyName = column.Name;
if (propertyName == column.Table.Name + "Name") return "Name";
if (propertyName == column.Table.Name + "Description") return "Description";
, propertyName.Length - );
return propertyName;
}
public string GetMemberVariableDefaultValue(ColumnSchema column)
{
switch (column.DataType)
{
case DbType.Guid:
{
return "Guid.Empty";
}
case DbType.AnsiString:
case DbType.AnsiStringFixedLength:
case DbType.String:
case DbType.StringFixedLength:
{
return "String.Empty";
}
default:
{
return "";
}
}
}
public string GetMemberVariableDefaultValue1(ColumnSchema column)
{
switch (column.DataType)
{
case DbType.Guid:
{
return "new Guid(row[\""+column.Name+"\"].ToString())";
}
case DbType.AnsiString:
case DbType.AnsiStringFixedLength:
case DbType.String:
case DbType.StringFixedLength:
{
return "row[\""+column.Name+"\"].ToString()";
}
case DbType.Byte:
return "Byte.Parse(row[\""+column.Name+"\"].ToString())";
case DbType.Int16:
return "Int16.Parse(row[\""+column.Name+"\"].ToString())";
case DbType.Int32:
return "Int32.Parse(row[\""+column.Name+"\"].ToString())";
case DbType.Int64:
return "Int64.Parse(row[\""+column.Name+"\"].ToString())";
default:
{
return "aaaa";
}
}
}
public string GetCSharpVariableType(ColumnSchema column)
{
if (column.Name.EndsWith("TypeCode")) return column.Name;
return DbTypeCSharp[column.DataType.ToString()];
}
public string GetClassName(TableSchema table)
{
return table.Name;
}
public override string GetFileName()
{
return this.GetClassName(this.SourceTable) + ".cs";
}
</script>
生成的代码是：
using System;
using System.Data;
namespace TestDbUtilityConsole
{
/// <summary>
/// Roles:实体类(属性说明自动提取数据库字段的描述信息)
/// </summary>
[Serializable]
public class Roles
{
#region Private Properties
private Guid _roleID = Guid.Empty;
private string _roleName = String.Empty;
private string _description = String.Empty;
private string _taskMask = String.Empty;
private byte _roleFlags;
#endregion
#region Public Properties
public Roles(DataRow row)
{
_roleID = new Guid(row["RoleID"].ToString());
_roleName = row["RoleName"].ToString();
_description = row["Description"].ToString();
_taskMask = row["TaskMask"].ToString();
_roleFlags = Byte.Parse(row["RoleFlags"].ToString());
}
/// <summary>
/// 列1
/// </summary>
public Guid RoleID
{
get { return _roleID; }
set { _roleID = value; }
}
/// <summary>
/// 列2
/// </summary>
public string RoleName
{
get { return _roleName; }
set { _roleName = value; }
}
/// <summary>
/// 列3
/// </summary>
public string Description
{
get { return _description; }
set { _description = value; }
}
/// <summary>
/// 列4
/// </summary>
public string TaskMask
{
get { return _taskMask; }
set { _taskMask = value; }
}
/// <summary>
/// 列5
/// </summary>
public byte RoleFlags
{
get { return _roleFlags; }
set { _roleFlags = value; }
}
#endregion
}
}
调用方法：
using System;
using System.Collections.Generic;
using System.Data;
using System.Diagnostics;
using System.Threading.Tasks;
using DbUtility;
namespace TestDbUtilityConsole
{
class Program
{
static SqlDbUtility db = new SqlDbUtility("Data Source=(local);Initial Catalog=ReportServer;Integrated Security=True;Connect Timeout=15;Encrypt=False;TrustServerCertificate=False");
static void Main(string[] args)
{
DataTable dt = GetDataTable().Result;
Stopwatch timer = new Stopwatch();
timer.Start();
; i < ; i++)
{
List<Roles> list = new List<Roles>();
Roles roles = null;
foreach (DataRow dataRow in dt.Rows)
{
roles = new Roles(dataRow);
list.Add(roles);
}
}
timer.Stop();
Console.WriteLine("遍历100W次，共耗时{0}毫秒。", timer.ElapsedMilliseconds);
Console.ReadKey();
}
static async Task<DataTable> GetDataTable()
{
DataTable dt = await db.T("select * from Roles").ExecuteDataTableAsync();
return dt;
}
}
}
结果：遍历100W次，共耗时9568毫秒。
DbUtility-关于DataTable转成List的效率问题的更多相关文章
- 将list<对象>转换成DataTable,把DataTable转换成参数传入存储过程实现批量插入数据
领导让在存储过程中批量添加数据,找出效率最高的,我看到后台代码后,发现可以将list<对象>转换成DataTable,把DataTable转换成参数传入存储过程实现批量插入数据,知道还有其 ...
- DataTable 转换成 Json的3种方法
在web开发中,我们可能会有这样的需求,为了便于前台的JS的处理,我们需要将查询出的数据源格式比如:List<T>.DataTable转换为Json格式.特别在使用Extjs框架的时候,A ...
- DataTable转换成IList<T>的简单实现
DataTable的无奈 很多时候,我们需要去操作DataTable.但DataTable的操作,实在是太不方便了.Linq?lambda表达式?统统没有... 特别是对现有结果集做进一步筛选,这样的 ...
- asp.net dataTable转换成Json格式
/// <summary> /// dataTable转换成Json格式 /// </summary> /// <param name="dt"> ...
- 将DataTable转换成CSV文件
DataTable用于在.net项目中,用于缓存数据,DataTable表示内存中数据的一个表.CSV文件最早用在简单的数据库里,由于其格式简单,并具备很强的开放性,所以起初被扫图家用作自己图集的标记 ...
- C#:DataTable映射成Model
这是数据库开发中经常遇到的问题,当然,这可以用现成的ORM框架来解决,但有些时候,如果DataSet/DataTable是第三方接口返回的,ORM就不方便了,还得自己处理. 反射自然必不可少的,另外考 ...
- 将DataSet（DataTable)转换成JSON格式（生成JS文件存储）
public static string CreateJsonParameters(DataTable dt) { /**/ /**/ /**/ /* /*********************** ...
- C#将DataTable转换成list的方法
本文实例讲述了C#将DataTable转换成list及数据分页的方法.分享给大家供大家参考.具体如下: /// <summary> /// 酒店评论列表-分页 /// </su ...
- DataTable转换成List<T>
很多时候需要将DataTable转换成一组model,直接对model执行操作会更加方便直观. 代码如下: public static class DataTableToModel { public ...
随机推荐
- sicily 1119 Factstone Benchmark
题意:求满足n! < 2^k,n的最大值! 解题:指数比较转换成对数比较,达到降幂! 其中: log (n!) = log(n)+log(n-1)+...+log(1); log(2^k) = ...
- Shopilex - 开源免费网店系统
Shopilex - 开源免费网店系统 Shopilex是一套基于GPL开源协议的,MVC架构的开源电子商务网店系统,是目前中文领域里真正开源免费的网店系统.
- 修改上一篇文章的node.js代码，支持默认页及支持中文
服务端 app.js var app = require('http').createServer(handler) var io = require('socket.io')(app); var f ...
- (转)Maven实战（三）Eclipse构建Maven项目
1. 安装m2eclipse插件 要用Eclipse构建Maven项目,我们需要先安装meeclipse插件 点击eclipse菜单栏Help->Eclipse Marketplac ...
- 1001 Sum Problem [ACM刷题]
这一段时间一直都在刷OJ,这里建一个博客合集,用以记录和分享算法学习的进程. github传送门:https://github.com/haoyuanliu/Online_Judge/tree/mas ...
- Oracle V$SESSION
SADDR session address SID session identifier 常用于链接其他列 SERIAL# SID有可能会重复,当两个session的SID重复时,SERIAL#用来区 ...
- divmod(a,b)函数
python每日一函数 - divmod数字处理函数 divmod(a,b)函数 中文说明: divmod(a,b)方法返回的是a//b(除法取整)以及a对b的余数 返回结果类型为tuple 参数: ...
- MediaPlayer+SurfaceView 视频播放 示例
SurfaceView的原理 SurfaceView在视频播放中起到显示画面的作用,而视频的播放主要通过MediaPlayer来控制. SurfaceView 允许我们 ...
- 95秀-dialog 进度对话框 实用工具
工具Util public class DialogUtil { public static ProgressDialogView progressDialog; /** * ...
- HTML基础总结<段落>
HTML 段落 段落是通过 <p> 标签定义的. 实例 <p>This is a paragraph </p><p>This is another pa ...