今天發(fā)現(xiàn)了一個超級快速插入大量數(shù)據(jù)到數(shù)據(jù)庫的方法SqlBulkCopy，由于我這里是讀Excel文件，所以我這里會提及到處理空行的方法和讀取Excel的方法。下面我們會通過數(shù)據(jù)庫跟蹤器跟蹤數(shù)據(jù)庫發(fā)生了什么。

我們開始創(chuàng)建一個模型

public class TBModel{public string name { get; set; }public string phone { get; set; }public string addr { get; set; }}

然后就是讀取Excel的方法

#region 讀取Excel文件到DataSet中/// <summary>/// 讀取Excel文件到DataSet中/// </summary>/// <param name="filePath">文件路徑</param>/// <returns></returns>public static DataSet ExcelToDataSet(string filePath, string fileName){var connStr = string.Empty;var fileType = Path.GetExtension(fileName);if (string.IsNullOrEmpty(fileType)) return null;if (fileType == ".xls")connStr = "Provider=Microsoft.Jet.OLEDB.4.0;" + "Data Source=" + filePath + ";" +";Extended Properties=\"Excel 8.0;HDR=YES;IMEX=1\"";elseconnStr = "Provider=Microsoft.ACE.OLEDB.12.0;" + "Data Source=" + filePath + ";" +";Extended Properties=\"Excel 12.0;HDR=YES;IMEX=1\"";var sql_F = "Select * FROM [{0}]";OleDbConnection conn = null;OleDbDataAdapter da = null;DataTable dtSheetName = null;var ds = new DataSet();try{// 初始化連接，并打開conn = new OleDbConnection(connStr);conn.Open();// 獲取數(shù)據(jù)源的表定義元數(shù)據(jù) var SheetName = "";dtSheetName = conn.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });// 初始化適配器da = new OleDbDataAdapter();for (var i = 0; i < dtSheetName.Rows.Count; i++){SheetName = (string)dtSheetName.Rows[i]["TABLE_NAME"];if (SheetName.Contains("$") && !SheetName.Replace("'", "").EndsWith("$")){continue;}da.SelectCommand = new OleDbCommand(string.Format(sql_F, SheetName), conn);var dsItem = new DataSet();da.Fill(dsItem, "OA_MeetingPositionArrangement");ds.Tables.Add(dsItem.Tables[0].Copy());}}catch (Exception ex){}finally{// 關閉連接if (conn.State == ConnectionState.Open){conn.Close();da.Dispose();conn.Dispose();}}return ds;}#endregion

去除datatable里面的空行

#region 去除datatable里面的空行public static void RemoveEmpty(DataTable dt){List<DataRow> removelist = new List<DataRow>();for (int i = 0; i < dt.Rows.Count; i++){bool IsNull = true;for (int j = 0; j < dt.Columns.Count; j++){if (!string.IsNullOrEmpty(dt.Rows[i][j].ToString().Trim())){IsNull = false;}}if (IsNull){removelist.Add(dt.Rows[i]);}}for (int i = 0; i < removelist.Count; i++){dt.Rows.Remove(removelist[i]);}}#endregion

然后就是使用SqlBulkCopy大批量導入數(shù)據(jù)

using System; using System.Collections.Generic; using System.Data; using System.Data.OleDb; using System.Data.SqlClient; using System.Diagnostics; using System.IO;namespace ConsoleAppTest {class Program{static void Main(string[] args){//讀取Excel文件var ds = ExcelToDataSet("D:/", "temp.xlsx");TBModel tbmodel = new TBModel();DataTable titledt = new DataTable();titledt.Columns.Add("Title");titledt.Rows.Add(tbmodel.name);titledt.Rows.Add(tbmodel.phone);titledt.Rows.Add(tbmodel.addr);//檢查文件和列是否相同if (ds.Tables[0].Columns.Count == titledt.Rows.Count){for (int i = 0; i < titledt.Rows.Count; i++){if (ds.Tables[0].Columns[i].ToString() != titledt.Rows[i]["Title"].ToString()){Console.WriteLine("表格有誤");}}}else{Console.WriteLine("表格有誤");}//去空行RemoveEmpty(ds.Tables[0]);//計時開始Stopwatch st = new Stopwatch();st.Start();using (SqlConnection conn = new SqlConnection("server=.;uid=sa;pwd=1234;database=MangoDB")){conn.Open();using (SqlBulkCopy bulkCopy = new SqlBulkCopy(conn)){try{//插入到數(shù)據(jù)庫的目標表：表名 bulkCopy.DestinationTableName = "tbs";//內(nèi)存表的字段 對應數(shù)據(jù)庫表的字段 bulkCopy.ColumnMappings.Add("name", "TempData");bulkCopy.WriteToServer(ds.Tables[0]);}catch (Exception ex){Console.WriteLine(ex);}}}st.Stop();Console.WriteLine("成功!測試時間為：" + st.ElapsedMilliseconds);Console.Read();}} }

這里導入一百萬條數(shù)據(jù)大概8秒9秒左右。。。測試了好幾次，快的不行

打開Sqlserver Profiler跟蹤，會發(fā)現(xiàn)執(zhí)行的是如下語句：

insert bulk tbs ([name] VarChar(50), phone VarChar(50) addr VarChar(50))

百度了一下msdn

BULK INSERT [ database_name . [ schema_name ] . | schema_name . ] [ table_name | view_name ] FROM 'data_file' [ WITH ( [ [ , ] BATCHSIZE = batch_size ] [ [ , ] CHECK_CONSTRAINTS ] [ [ , ] CODEPAGE = { 'ACP' | 'OEM' | 'RAW' | 'code_page' } ] [ [ , ] DATAFILETYPE = { 'char' | 'native'| 'widechar' | 'widenative' } ] [ [ , ] DATASOURCE = 'data_source_name' ][ [ , ] ERRORFILE = 'file_name' ][ [ , ] ERRORFILE_DATASOURCE = 'data_source_name' ] [ [ , ] FIRSTROW = first_row ] [ [ , ] FIRE_TRIGGERS ] [ [ , ] FORMATFILE_DATASOURCE = 'data_source_name' ][ [ , ] KEEPIDENTITY ] [ [ , ] KEEPNULLS ] [ [ , ] KILOBYTES_PER_BATCH = kilobytes_per_batch ] [ [ , ] LASTROW = last_row ] [ [ , ] MAXERRORS = max_errors ] [ [ , ] ORDER ( { column [ ASC | DESC ] } [ ,...n ] ) ] [ [ , ] ROWS_PER_BATCH = rows_per_batch ] [ [ , ] ROWTERMINATOR = 'row_terminator' ] [ [ , ] TABLOCK ] -- input file format options[ [ , ] FORMAT = 'CSV' ][ [ , ] FIELDQUOTE = 'quote_characters'][ [ , ] FORMATFILE = 'format_file_path' ] [ [ , ] FIELDTERMINATOR = 'field_terminator' ] [ [ , ] ROWTERMINATOR = 'row_terminator' ] )]

這里是MSDN的地址：https://docs.microsoft.com/zh-cn/sql/t-sql/statements/bulk-insert-transact-sql?view=sql-server-2017

看著很像insert into tbs select * from tb這種，然后試了幾次這種插入數(shù)據(jù)的方式，發(fā)現(xiàn)還是上面的那種比這種快了好幾秒，不過這兩種方式使用的地方不同，所以使用的時候各有所需吧

?

總結

以上是生活随笔為你收集整理的SqlBulkCopy批量插入数据库的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。