DbUtility-关于DataTable转成List的效率问题
DbUtility中的方法ExecuteDataTableAsync()得到的是一个DataTable,而我们常见的情况下,我们需要的不是DataTable,而是List或IList,所以现在需要考虑把DataTable转成List或IList,目前暂时有三种方案:
方案1:用动软生成代码:
public static List<Roles> GetRoleses(DataTable dt) { List<Roles> list = new List<Roles>(); ) list.AddRange(from DataRow dataRow in dt.Rows select DataRowToModel(dataRow)); return list; } /// <summary> /// 得到一个对象实体 /// </summary> private static Roles DataRowToModel(DataRow row) { Roles model = new Roles(); if (row != null) { if (row["RoleID"] != null && row["RoleID"].ToString() != "") { model.RoleID = new Guid(row["RoleID"].ToString()); } if (row["RoleName"] != null) { model.RoleName = row["RoleName"].ToString(); } if (row["Description"] != null) { model.Description = row["Description"].ToString(); } if (row["TaskMask"] != null) { model.TaskMask = row["TaskMask"].ToString(); } if (row["RoleFlags"] != null && row["RoleFlags"].ToString() != "") { model.RoleFlags = int.Parse(row["RoleFlags"].ToString()); } } return model; }
循环遍历100W次,大约用时:14460毫秒。
方案2:用MVC.Net提供的反射方案:
public static IList<T> ConvertToModel<T>(this DataTable dt) where T : class, new() { // 定义集合 IList<T> ts = new List<T>(); // 获得此模型的类型 Type type = typeof(T); string tempName = ""; foreach (DataRow dr in dt.Rows) { T t = new T(); // 获得此模型的公共属性 PropertyInfo[] propertys = t.GetType().GetProperties(); foreach (PropertyInfo pi in propertys) { tempName = pi.Name; // 检查DataTable是否包含此列 if (dt.Columns.Contains(tempName)) { // 判断此属性是否有Setter if (!pi.CanWrite) continue; object value = dr[tempName]; if (value != DBNull.Value) pi.SetValue(t, value, null); } } ts.Add(t); } return ts; }
反射100W次,大约用时:20350毫秒。
方案3:用Melas提供的反射方案:
public static IList<T> ConvertTo<T>(DataTable table) { if (table == null) { return null; } List<DataRow> rows = new List<DataRow>(); foreach (DataRow row in table.Rows) { rows.Add(row); } return ConvertTo<T>(rows); } public static IList<T> ConvertTo<T>(IList<DataRow> rows) { IList<T> list = null; if (rows != null) { list = new List<T>(); foreach (DataRow row in rows) { T item = CreateItem<T>(row); list.Add(item); } } return list; } public static T CreateItem<T>(DataRow row) { T obj = default(T); if (row != null) { obj = Activator.CreateInstance<T>(); foreach (DataColumn column in row.Table.Columns) { object value = row[column.ColumnName]; PropertyInfo prop = obj.GetType().GetProperty(column.ColumnName); try { if (value != DBNull.Value && prop != null) prop.SetValue(obj, value, null); } catch { // You can log something here throw; } } } return obj; }
反射100W次,大约用时:20258毫秒。
数据库结构:
|
原本想修改动软代码生成器的模板来生成改写过的DataRowToModel方法,想改成下面这样:
/// <summary> /// 得到一个对象实体 /// </summary> private static Roles DataRowToModel(DataRow row) { if (row != null) { Roles model = new Roles { RoleID = new Guid(row["RoleID"].ToString()), RoleName = row["RoleName"].ToString(), Description = row["Description"].ToString(), TaskMask = row["TaskMask"].ToString(), RoleFlags = int.Parse(row["RoleFlags"].ToString()) }; return model; } return null; }
后来发现一个很悲剧的事,模板里不包含DataRowToModel方法,我看到了下面这个东西:
目前用到的代码生成,我用CodeSmith生成,下面贴出两个模板,可以直接用,已经测试通过。
实体层:
<%@ CodeTemplate Language="C#" TargetLanguage="C#" Description="Generates a very simple business object." %> <%@ Property Name="SourceTable" Type="SchemaExplorer.TableSchema" Category="Context" Description="Table that the object is based on." %> <%@ Property Name="SingleFileMode" Type="System.Boolean" Category="Options" Default="True" Description="Generate content for a complete cs file instead of just a class." %> <%@ Property Name="ClassNamespace" Type="System.String" Category="Options" Default="BusinessObjects" Description="Namespace your class is in. Only used if SingleFileMode is true!" %> <%@ Assembly Name="SchemaExplorer" %> <%@ Assembly Name="System.Data" %> <%@ Import Namespace="SchemaExplorer" %> <%@ Import Namespace="System.Data" %> <%@ Map Name="SqlCSharp" Src="Sql-CSharp" Description="System to C# Type Map" %> <%@ Map Name="DbDataReader" Src="DbType-DataReaderMethod" Description="DbType to DataReader Method Map" %> <%@ Map Name="SqlNativeSqlDb" Src="SqlNativeType-SqlDbType" Description="SqlNativeType to SqlDbType Map" %> <%@ Map Name="DbTypeCSharp" Src="DbType-CSharp" Description="DbType to CSharp Map" %> <% if(this.SingleFileMode) { %> using System; namespace <%= this.ClassNamespace %> { <% } %> /// <summary> /// <%= GetClassName(SourceTable) %>:实体类(属性说明自动提取数据库字段的描述信息) /// </summary> [Serializable] public class <%= GetClassName(SourceTable) %> { #region Private Properties <% foreach (ColumnSchema column in SourceTable.Columns) { %> <%= GetMemberVariableDeclarationStatement(column) %> <% } %> #endregion #region Public Properties <% ; i < SourceTable.Columns.Count; i++) { %> /// <summary> /// <%=SourceTable.Columns[i].Description %> /// </summary> public <%= GetCSharpVariableType(SourceTable.Columns[i]) %> <%= GetPropertyName(SourceTable.Columns[i]) %> { get {return <%= GetMemberVariableName(SourceTable.Columns[i]) %>;} set {<%= GetMemberVariableName(SourceTable.Columns[i]) %> = value;} } <% ) Response.Write("\r\n"); %> <% } %> #endregion } <% if(this.SingleFileMode) { %> } <% } %> <script runat="template"> public string GetMemberVariableDeclarationStatement(ColumnSchema column) { return GetMemberVariableDeclarationStatement("private", column); } public string GetMemberVariableDeclarationStatement(string protectionLevel, ColumnSchema column) { string statement = protectionLevel + " "; statement += GetCSharpVariableType(column) + " " + GetMemberVariableName(column); string defaultValue = GetMemberVariableDefaultValue(column); if (defaultValue != "") { statement += " = " + defaultValue; } statement += ";"; return statement; } public string GetCamelCaseName(string value) { , ).ToLower() + value.Substring(); } public string GetMemberVariableName(ColumnSchema column) { string propertyName = GetPropertyName(column); string memberVariableName = "_" + GetCamelCaseName(propertyName); return memberVariableName; } public string GetPropertyName(ColumnSchema column) { string propertyName = column.Name; if (propertyName == column.Table.Name + "Name") return "Name"; if (propertyName == column.Table.Name + "Description") return "Description"; , propertyName.Length - ); return propertyName; } public string GetMemberVariableDefaultValue(ColumnSchema column) { switch (column.DataType) { case DbType.Guid: { return "Guid.Empty"; } case DbType.AnsiString: case DbType.AnsiStringFixedLength: case DbType.String: case DbType.StringFixedLength: { return "String.Empty"; } default: { return ""; } } } public string GetCSharpVariableType(ColumnSchema column) { if (column.Name.EndsWith("TypeCode")) return column.Name; return DbTypeCSharp[column.DataType.ToString()]; } public string GetClassName(TableSchema table) { return table.Name; } public override string GetFileName() { return this.GetClassName(this.SourceTable) + ".cs"; } </script>
数据访问层:
<%@ CodeTemplate Language="C#" TargetLanguage="C#" Description="Generates a very simple business object." %> <%@ Property Name="SourceTable" Type="SchemaExplorer.TableSchema" Category="Context" Description="Table that the object is based on." %> <%@ Property Name="SingleFileMode" Type="System.Boolean" Category="Options" Default="True" Description="Generate content for a complete cs file instead of just a class." %> <%@ Property Name="ClassNamespace" Type="System.String" Category="Options" Default="BusinessObjects" Description="Namespace your class is in. Only used if SingleFileMode is true!" %> <%@ Assembly Name="SchemaExplorer" %> <%@ Assembly Name="System.Data" %> <%@ Import Namespace="SchemaExplorer" %> <%@ Import Namespace="System.Data" %> <%@ Map Name="SqlCSharp" Src="Sql-CSharp" Description="System to C# Type Map" %> <%@ Map Name="DbDataReader" Src="DbType-DataReaderMethod" Description="DbType to DataReader Method Map" %> <%@ Map Name="SqlNativeSqlDb" Src="SqlNativeType-SqlDbType" Description="SqlNativeType to SqlDbType Map" %> <%@ Map Name="DbTypeCSharp" Src="DbType-CSharp" Description="DbType to CSharp Map" %> <% if(this.SingleFileMode) { %> using System; using System.Collections.Generic; using System.Data; using System.Linq; namespace <%= this.ClassNamespace %> { <% } %> public class <%= GetClassName(SourceTable) %>DAL { /// <summary> /// 把DataTable转成一个List集合 /// </summary> public static List<<%= GetClassName(SourceTable) %>> GetRoleses(DataTable dt) { List<<%= GetClassName(SourceTable) %>> list = new List<<%= GetClassName(SourceTable) %>>(); ) list.AddRange(from DataRow dataRow in dt.Rows select DataRowToModel(dataRow)); return list; } /// <summary> /// 得到一个对象实体 /// </summary> private static <%= GetClassName(SourceTable) %> DataRowToModel(DataRow row) { if (row != null) { <%= GetClassName(SourceTable) %> model = new <%= GetClassName(SourceTable) %> { <% ; i < SourceTable.Columns.Count; i++) { %> <%= GetReaderAssignmentStatement(SourceTable.Columns[i], i,SourceTable.Columns.Count-) %> <%}%> }; return model; } return null; } } <% if(this.SingleFileMode) { %> } <% } %> <script runat="template"> public string GetReaderAssignmentStatement(ColumnSchema column, int index,int count) { string statement = GetMemberVariableName(column) + " = "; if (column.Name.EndsWith("TypeCode")) statement += "(" + column.Name + ")"; statement += GetMemberVariableDefaultValue(column); if(index<count) statement+=","; return statement; } public string GetMemberVariableName(ColumnSchema column) { return GetPropertyName(column); } public string GetPropertyName(ColumnSchema column) { string propertyName = column.Name; if (propertyName == column.Table.Name + "Name") return "Name"; if (propertyName == column.Table.Name + "Description") return "Description"; , propertyName.Length - ); return propertyName; } public string GetMemberVariableDefaultValue(ColumnSchema column) { switch (column.DataType) { case DbType.Guid: { return "new Guid(row[\""+column.Name+"\"].ToString())"; } case DbType.AnsiString: case DbType.AnsiStringFixedLength: case DbType.String: case DbType.StringFixedLength: { return "row[\""+column.Name+"\"].ToString()"; } case DbType.Byte: return "Byte.Parse(row[\""+column.Name+"\"].ToString())"; case DbType.Int16: return "Int16.Parse(row[\""+column.Name+"\"].ToString())"; case DbType.Int32: return "Int32.Parse(row[\""+column.Name+"\"].ToString())"; case DbType.Int64: return "Int64.Parse(row[\""+column.Name+"\"].ToString())"; default: { return "aaaa"; } } } public string GetCSharpVariableType(ColumnSchema column) { if (column.Name.EndsWith("TypeCode")) return column.Name; return DbTypeCSharp[column.DataType.ToString()]; } public string GetClassName(TableSchema table) { return table.Name; } public override string GetFileName() { return this.GetClassName(this.SourceTable) + "DAL.cs"; } </script>
上面的模板生成的是1楼评论中的代码。
后面经过MVC.Net的提醒,改用构造方法来做,具体做法是:
数据访问层就不需要了,只需要一个实体层模板:
<%@ CodeTemplate Language="C#" TargetLanguage="C#" Description="Generates a very simple business object." %> <%@ Property Name="SourceTable" Type="SchemaExplorer.TableSchema" Category="Context" Description="Table that the object is based on." %> <%@ Property Name="SingleFileMode" Type="System.Boolean" Category="Options" Default="True" Description="Generate content for a complete cs file instead of just a class." %> <%@ Property Name="ClassNamespace" Type="System.String" Category="Options" Default="BusinessObjects" Description="Namespace your class is in. Only used if SingleFileMode is true!" %> <%@ Assembly Name="SchemaExplorer" %> <%@ Assembly Name="System.Data" %> <%@ Import Namespace="SchemaExplorer" %> <%@ Import Namespace="System.Data" %> <%@ Map Name="SqlCSharp" Src="Sql-CSharp" Description="System to C# Type Map" %> <%@ Map Name="DbDataReader" Src="DbType-DataReaderMethod" Description="DbType to DataReader Method Map" %> <%@ Map Name="SqlNativeSqlDb" Src="SqlNativeType-SqlDbType" Description="SqlNativeType to SqlDbType Map" %> <%@ Map Name="DbTypeCSharp" Src="DbType-CSharp" Description="DbType to CSharp Map" %> <% if(this.SingleFileMode) { %> using System; using System.Data; namespace <%= this.ClassNamespace %> { <% } %> /// <summary> /// <%= GetClassName(SourceTable) %>:实体类(属性说明自动提取数据库字段的描述信息) /// </summary> [Serializable] public class <%= GetClassName(SourceTable) %> { #region Private Properties <% foreach (ColumnSchema column in SourceTable.Columns) { %> <%= GetMemberVariableDeclarationStatement(column) %> <% } %> #endregion #region Public Properties public <%= GetClassName(SourceTable) %>(DataRow row) { <% ; i < SourceTable.Columns.Count; i++) { %> <%=GetMemberVariableName(SourceTable.Columns[i])%> = <%= GetMemberVariableDefaultValue1(SourceTable.Columns[i])%>; <%}%> } <% ; i < SourceTable.Columns.Count; i++) { %> /// <summary> /// <%=SourceTable.Columns[i].Description %> /// </summary> public <%= GetCSharpVariableType(SourceTable.Columns[i]) %> <%= GetPropertyName(SourceTable.Columns[i]) %> { get {return <%= GetMemberVariableName(SourceTable.Columns[i]) %>;} set {<%= GetMemberVariableName(SourceTable.Columns[i]) %> = value;} } <% ) Response.Write("\r\n"); %> <% } %> #endregion } <% if(this.SingleFileMode) { %> } <% } %> <script runat="template"> public string GetMemberVariableDeclarationStatement(ColumnSchema column) { return GetMemberVariableDeclarationStatement("private", column); } public string GetMemberVariableDeclarationStatement(string protectionLevel, ColumnSchema column) { string statement = protectionLevel + " "; statement += GetCSharpVariableType(column) + " " + GetMemberVariableName(column); string defaultValue = GetMemberVariableDefaultValue(column); if (defaultValue != "") { statement += " = " + defaultValue; } statement += ";"; return statement; } public string GetCamelCaseName(string value) { , ).ToLower() + value.Substring(); } public string GetMemberVariableName(ColumnSchema column) { string propertyName = GetPropertyName(column); string memberVariableName = "_" + GetCamelCaseName(propertyName); return memberVariableName; } public string GetPropertyName(ColumnSchema column) { string propertyName = column.Name; if (propertyName == column.Table.Name + "Name") return "Name"; if (propertyName == column.Table.Name + "Description") return "Description"; , propertyName.Length - ); return propertyName; } public string GetMemberVariableDefaultValue(ColumnSchema column) { switch (column.DataType) { case DbType.Guid: { return "Guid.Empty"; } case DbType.AnsiString: case DbType.AnsiStringFixedLength: case DbType.String: case DbType.StringFixedLength: { return "String.Empty"; } default: { return ""; } } } public string GetMemberVariableDefaultValue1(ColumnSchema column) { switch (column.DataType) { case DbType.Guid: { return "new Guid(row[\""+column.Name+"\"].ToString())"; } case DbType.AnsiString: case DbType.AnsiStringFixedLength: case DbType.String: case DbType.StringFixedLength: { return "row[\""+column.Name+"\"].ToString()"; } case DbType.Byte: return "Byte.Parse(row[\""+column.Name+"\"].ToString())"; case DbType.Int16: return "Int16.Parse(row[\""+column.Name+"\"].ToString())"; case DbType.Int32: return "Int32.Parse(row[\""+column.Name+"\"].ToString())"; case DbType.Int64: return "Int64.Parse(row[\""+column.Name+"\"].ToString())"; default: { return "aaaa"; } } } public string GetCSharpVariableType(ColumnSchema column) { if (column.Name.EndsWith("TypeCode")) return column.Name; return DbTypeCSharp[column.DataType.ToString()]; } public string GetClassName(TableSchema table) { return table.Name; } public override string GetFileName() { return this.GetClassName(this.SourceTable) + ".cs"; } </script>
生成的代码是:
using System; using System.Data; namespace TestDbUtilityConsole { /// <summary> /// Roles:实体类(属性说明自动提取数据库字段的描述信息) /// </summary> [Serializable] public class Roles { #region Private Properties private Guid _roleID = Guid.Empty; private string _roleName = String.Empty; private string _description = String.Empty; private string _taskMask = String.Empty; private byte _roleFlags; #endregion #region Public Properties public Roles(DataRow row) { _roleID = new Guid(row["RoleID"].ToString()); _roleName = row["RoleName"].ToString(); _description = row["Description"].ToString(); _taskMask = row["TaskMask"].ToString(); _roleFlags = Byte.Parse(row["RoleFlags"].ToString()); } /// <summary> /// 列1 /// </summary> public Guid RoleID { get { return _roleID; } set { _roleID = value; } } /// <summary> /// 列2 /// </summary> public string RoleName { get { return _roleName; } set { _roleName = value; } } /// <summary> /// 列3 /// </summary> public string Description { get { return _description; } set { _description = value; } } /// <summary> /// 列4 /// </summary> public string TaskMask { get { return _taskMask; } set { _taskMask = value; } } /// <summary> /// 列5 /// </summary> public byte RoleFlags { get { return _roleFlags; } set { _roleFlags = value; } } #endregion } }
调用方法:
using System; using System.Collections.Generic; using System.Data; using System.Diagnostics; using System.Threading.Tasks; using DbUtility; namespace TestDbUtilityConsole { class Program { static SqlDbUtility db = new SqlDbUtility("Data Source=(local);Initial Catalog=ReportServer;Integrated Security=True;Connect Timeout=15;Encrypt=False;TrustServerCertificate=False"); static void Main(string[] args) { DataTable dt = GetDataTable().Result; Stopwatch timer = new Stopwatch(); timer.Start(); ; i < ; i++) { List<Roles> list = new List<Roles>(); Roles roles = null; foreach (DataRow dataRow in dt.Rows) { roles = new Roles(dataRow); list.Add(roles); } } timer.Stop(); Console.WriteLine("遍历100W次,共耗时{0}毫秒。", timer.ElapsedMilliseconds); Console.ReadKey(); } static async Task<DataTable> GetDataTable() { DataTable dt = await db.T("select * from Roles").ExecuteDataTableAsync(); return dt; } } }
结果:遍历100W次,共耗时9568毫秒。
DbUtility-关于DataTable转成List的效率问题的更多相关文章
- 将list<对象>转换成DataTable,把DataTable转换成参数传入存储过程实现批量插入数据
领导让在存储过程中批量添加数据,找出效率最高的,我看到后台代码后,发现可以将list<对象>转换成DataTable,把DataTable转换成参数传入存储过程实现批量插入数据,知道还有其 ...
- DataTable 转换成 Json的3种方法
在web开发中,我们可能会有这样的需求,为了便于前台的JS的处理,我们需要将查询出的数据源格式比如:List<T>.DataTable转换为Json格式.特别在使用Extjs框架的时候,A ...
- DataTable转换成IList<T>的简单实现
DataTable的无奈 很多时候,我们需要去操作DataTable.但DataTable的操作,实在是太不方便了.Linq?lambda表达式?统统没有... 特别是对现有结果集做进一步筛选,这样的 ...
- asp.net dataTable转换成Json格式
/// <summary> /// dataTable转换成Json格式 /// </summary> /// <param name="dt"> ...
- 将DataTable转换成CSV文件
DataTable用于在.net项目中,用于缓存数据,DataTable表示内存中数据的一个表.CSV文件最早用在简单的数据库里,由于其格式简单,并具备很强的开放性,所以起初被扫图家用作自己图集的标记 ...
- C#:DataTable映射成Model
这是数据库开发中经常遇到的问题,当然,这可以用现成的ORM框架来解决,但有些时候,如果DataSet/DataTable是第三方接口返回的,ORM就不方便了,还得自己处理. 反射自然必不可少的,另外考 ...
- 将DataSet(DataTable)转换成JSON格式(生成JS文件存储)
public static string CreateJsonParameters(DataTable dt) { /**/ /**/ /**/ /* /*********************** ...
- C#将DataTable转换成list的方法
本文实例讲述了C#将DataTable转换成list及数据分页的方法.分享给大家供大家参考.具体如下: /// <summary> /// 酒店评论列表-分页 /// </su ...
- DataTable转换成List<T>
很多时候需要将DataTable转换成一组model,直接对model执行操作会更加方便直观. 代码如下: public static class DataTableToModel { public ...
随机推荐
- Matlab:拟合(1)
拟合练习: function f = curvefun(x, tdata) f = (x()*x()*x()) / (x()-x()) * ( exp(-x()*tdata)/(x()-x()) + ...
- XCode7中不能使用http的临时配置解决办法
先看看iOS9新特性中关于ATS的官方文档: App Transport Security App Transport Security (ATS) enforces best practices i ...
- Python 列表(list)、字典(dict)、字符串(string)常用基本操作小结
创建列表 sample_list = ['a',1,('a','b')] Python 列表操作 sample_list = ['a','b',0,1,3] 得到列表中的某一个值 value_star ...
- Core Services 层
Core Services层为所有的应用程序提供基础系统服务.可能应用程序并不直接使用这些服务,但它们是系统很多部分赖以建构的基础. 高阶特性 下面的部分描述一些比较常见特性,也许您正打算让您的应用程 ...
- SKView类
继承自 UIView:UIResponder:NSObject 符合 NSCoding(UIView)UIAppearance(UIView)UIAppearanceContainer(UIView) ...
- Java经典23种设计模式之结构型模式(一)
结构型模式包含7种:适配器模式.桥接模式.组合模式.装饰模式.外观模式.享元模式.代理模式. 本文主要介绍适配器模式和桥接模式. 一.适配器模式(Adapter) 适配器模式事实上非常easy.就像手 ...
- mysql 修复表和优化表
REPAIR TABLE `table_name` 修复表 OPTIMIZE TABLE `table_name` 优化表
- Understanding AMQP, the protocol used by RabbitMQ--reference
RabbitMQ is a lightweight, reliable, scalable and portable message broker. But unlike many message b ...
- 开源消息中间件DotNetMQ
由于这个开源项目对我这种中间件菜鸟很有帮助,因此,我将官方的说明文档翻译如下: Introduction In this article, I will introduce a new and ind ...
- 怎么在Linux上下载并安装ESET NOD32 Antivirus 4桌面版
转自:怎么在Linux上下载并安装ESET NOD32 Antivirus 4桌面版 下载并安装ESET NOD32 Antivirus 4的Linux桌面版,根据下面的步骤一步一步的来: I. 下 ...