SqlBulkCopy Precision Error from C# Double to Sql Decimal?

https://forums.asp.net/t/1300113.aspx?SqlBulkCopy+Precision+Error+from+C+Double+to+Sql+Decimal+

Before I bulkcopy data into the database, I use DataTable to store the bulk of data.

Since the value to be stored is declared as double in the DataTable, I defined the datatype of that data column to be double too.

This causes a conversion problem as somewhere along the chain (I am suspecting the SqlBulkCopy is doing some kind of implicit conversion) the precision of my value was changed.

Ever since I changed the data type of that data column from double into decimal and explicitly convert the datatype before storing into the DataTable, the problem is fixed.

C# double to sql server lost precision

 private void FixDoubleToDecimal(DataSet dataSet)
        {
            if (dataSet == null)
            {
                return;
            }

            foreach (DataTable table in dataSet.Tables)
            {
                foreach (DataColumn column in table.Columns)
                {
                    if (column.DataType == typeof(double))
                    {
                        column.DataType = typeof(decimal);
                    }
                }
            }
        }

测试代码

 [Test]
        public void DecimalTest()
        {
            DataTable dataTable=new DataTable();
            DataColumn dataColumn=new DataColumn("column1"){DataType = typeof(double)};
            dataTable.Columns.Add(dataColumn);
            DataRow dataRow= dataTable.NewRow();
            dataRow["column1"] = 0.000002552;
            Console.WriteLine(dataRow["column1"]);

            dataTable.Columns["column1"].DataType = typeof(decimal);
            Console.WriteLine(dataRow["column1"]);
        }

有数据就无法转换

System.ArgumentException : Cannot change DataType of a column once it has data.
at System.Data.DataColumn.set_DataType(Type value)
at AssemblyTest.MainTest.DecimalTest() in C:\Users\clu\source\repos\Edenred\Test\AssemblyTest\MainTest.cs:line 49

2.552E-06

How To Change DataType of a DataColumn in a DataTable?

方案1

Old post, but I thought I'd weigh in, with a DataTable extension that can convert a single column at a time, to a given type:

public static class DataTableExt
{
    public static void ConvertColumnType(this DataTable dt, string columnName, Type newType)
    {
        using (DataColumn dc = new DataColumn(columnName + "_new", newType))
        {
            // Add the new column which has the new type, and move it to the ordinal of the old column
            int ordinal = dt.Columns[columnName].Ordinal;
            dt.Columns.Add(dc);
            dc.SetOrdinal(ordinal);

            // Get and convert the values of the old column, and insert them into the new
            foreach (DataRow dr in dt.Rows)
                dr[dc.ColumnName] = Convert.ChangeType(dr[columnName], newType);

            // Remove the old column
            dt.Columns.Remove(columnName);

            // Give the new column the old column's name
            dc.ColumnName = columnName;
        }
    }
}

It can then be called like this:

MyTable.ConvertColumnType("MyColumnName", typeof(int));

Of course using whatever type you desire, as long as each value in the column can actually be converted to the new type.

DataColumn.Ordinal   Gets the (zero-based) position of the column in the DataColumnCollection collection.

评论

问 Gives error "Object must implement IConvertible." while converting Byte[] to string type column.

答 Just add some generics to it public static void ConvertColumnType<T>(this DataTable dt, string columnName, TnewType) where T : Type, IConvertible – 

 

I think this is most elegant solution, just avoid converting DBNulls, e.g. dr[dc.ColumnName] = dr[columnName] == DBNull.Value ? DBNull.Value : Convert.ChangeType(dr[columnName], newType);

方案2

I created an extension function which allows changing the column type of a DataTable. Instead of cloning the entire table and importing all the data it just clones the column, parses the value and then deletes the original.

/// <summary>
    /// Changes the datatype of a column. More specifically it creates a new one and transfers the data to it
    /// </summary>
    /// <param name="column">The source column</param>
    /// <param name="type">The target type</param>
    /// <param name="parser">A lambda function for converting the value</param>
    public static void ChangeType(this DataColumn column, Type type, Func<object, object> parser)
    {
        //no table? just switch the type
        if (column.Table == null)
        {
            column.DataType = type;
            return;
        }

        //clone our table
        DataTable clonedtable = column.Table.Clone();

        //get our cloned column
        DataColumn clonedcolumn = clonedtable.Columns[column.ColumnName];

        //remove from our cloned table
        clonedtable.Columns.Remove(clonedcolumn);

        //change the data type
        clonedcolumn.DataType = type;

        //change our name
        clonedcolumn.ColumnName = Guid.NewGuid().ToString();

        //add our cloned column
        column.Table.Columns.Add(clonedcolumn);

        //interpret our rows
        foreach (DataRow drRow in column.Table.Rows)
        {
            drRow[clonedcolumn] = parser(drRow[column]);
        }

        //remove our original column
        column.Table.Columns.Remove(column);

        //change our name
        clonedcolumn.ColumnName = column.ColumnName;
    }
}

You can use it like so:

List<DataColumn> lsColumns = dtData.Columns
    .Cast<DataColumn>()
    .Where(i => i.DataType == typeof(decimal))
    .ToList()

//loop through each of our decimal columns
foreach(DataColumn column in lsColumns)
{
    //change to double
    column.ChangeType(typeof(double),(value) =>
    {
        double output = ;
        double.TryParse(value.ToString(), out output);
        return output;
    });
}

The above code changes all the decimal columns to doubles.

最终的解决方案

DataSet data;

data.ConvertColumnType(typeof(double), typeof(decimal));

 public static void ConvertColumnType(this DataTable dt, string columnName, Type newType)
        {
            using (DataColumn dc = new DataColumn($"{columnName}_new", newType))
            {
                // Add the new column which has the new type, and move it to the ordinal of the old column
                int ordinal = dt.Columns[columnName].Ordinal;
                dt.Columns.Add(dc);
                dc.SetOrdinal(ordinal);

                // Get and convert the values of the old column, and insert them into the new
                foreach (DataRow dr in dt.Rows)
                {
                    var obj = dr[columnName];
                    if (obj != null && obj != DBNull.Value)
                    {
                        dr[dc.ColumnName] = Convert.ChangeType(obj, newType);
                    }
                    else
                    {
                        dr[dc.ColumnName] = obj;
                    }
                }

                // Remove the old column
                dt.Columns.Remove(columnName);

                // Give the new column the old column's name
                dc.ColumnName = columnName;
            }
        }

        public static void ConvertColumnType(this DataSet dataSet, Type sourceType, Type targetType)
        {
            foreach (DataTable table in dataSet.Tables)
            {
                var columns = table.Columns.Cast<DataColumn>().Where(x => x.DataType == sourceType).ToList();
                //do not use foreach here, otherwise you will encounter "Collection was modified; enumeration operation may not execute."
                for (int i = ; i < columns.Count; i++)
                {
                    table.ConvertColumnType(columns[i].ColumnName, targetType);
                }
            }
        }

关于double的精度的问题

https://www.cnblogs.com/c-primer/p/5992696.html

https://blog.csdn.net/yansmile1/article/details/70145416

涉及到计算机原理中浮点数的存储问题，

float和double的范围是由指数的位数来决定的。

float的指数位有8位，而double的指数位有11位，分布如下：

float：

1bit（符号位）

8bits（指数位）

23bits（尾数位）

double： In c# double is always 8 bytes (64 bits)

1bit（符号位）

11bits（指数位）

52bits（尾数位）

float和double的精度是由尾数的位数来决定的。

浮点数在内存中是按科学计数法来存储的，其整数部分始终是一个隐含着的“1”，由于它是不变的，故不能对精度造成影响。

float：2^23 = 8388608，一共七位，这意味着最多能有7位有效数字，但绝对能保证的为6位，也即float的精度为6~7位有效数字；

double：2^52 = 4503599627370496，一共16位，同理，double的精度为15~16位。

https://stackoverflow.com/questions/1546113/double-to-string-conversion-without-scientific-notation

For a general-purpose¹ solution you need to preserve 339 places:

doubleValue.ToString("0." + new string('#', 339))

The maximum number of non-zero decimal digits is 16. 15 are on the right side of the decimal point. The exponent can move those 15 digits a maximum of 324 places to the right. (See the range and precision.)

It works for double.Epsilon, double.MinValue, double.MaxValue, and anything in between.

The performance will be much greater than the regex/string manipulation solutions since all formatting and string work is done in one pass by unmanaged CLR code. Also, the code is much simpler to prove correct.

For ease of use and even better performance, make it a constant:

public static class FormatStrings
{
    public const string DoubleFixedPoint = "0.###################################################################################################################################################################################################################################################################################################################################################";
}

¹ Update: I mistakenly said that this was also a lossless solution. In fact it is not, since ToString does its normal display rounding for all formats except r. Live example. Thanks, @Loathing! Please see Lothing’s answer if you need the ability to roundtrip in fixed point notation (i.e, if you’re using .ToString("r") today).

This solution is not "loseless".

Example:

String t1 = (0.0001/7).ToString("0." + new string('#', 339)); // 0.0000142857142857143

versus:

String t2 = (0.0001/7).ToString("r"); // 1.4285714285714287E-05

Precision is lost at the ending decimal places.

用r进行转换

 if (value is double d)
                    {
                        parameters[2].Value = d.ToString("r");
                    }

发现是我理解错了，下面2个方法，第一个虽然可以避免科学计数法，但是会丢失精度。

            String t1 = (0.0001 / ).ToString("0." + new string('#', )); // 0.0000142857142857143 versus:
            String t2 = (0.0001 / ).ToString("r"); // 1.4285714285714287E-05
            Console.WriteLine(t1);
            Console.WriteLine(t2);

 public static class FormatStrings
    {
        public const string DoubleFixedPoint = "0.###################################################################################################################################################################################################################################################################################################################################################";

        public static String ToStandardNotationString(this double d)
        {
            //Keeps precision of double up to is maximum
            return d.ToString("0.#####################################################################################################################################################################################################################################################################################################################################");

        }
    }

  public void DecimalTest()
        {
            double d = 0.0001 / ;
            Console.WriteLine(d);
            Console.WriteLine(d.ToStandardNotationString());
            Console.WriteLine(d.ToString("r"));
        }

输出

1.42857142857143E-05      科学计数法，小数点后14位。(加上科学计数法的位数，14+5=19位)
0.0000142857142857143   不使用科学计数法，小数点后19位。
1.4285714285714287E-05  //d.ToString("r")  这个小数点16位(16+5=21位置)