首页 \ 问答 \ 有效地向数据库添加大量文本(Efficiently adding large amounts of text to a database)

有效地向数据库添加大量文本(Efficiently adding large amounts of text to a database)

我有一个C#应用程序需要向SQL Server数据库添加大量文本(一行最多可以有1GB的文本)。 所以,我有一些看起来像的代码:

SqlParameter param = command.Parameters.Add("@text", SqlDbType.NVarChar);
param.Value = new string(buffer);

文本以块的形式添加,因此缓冲区为20 MB。 我试图通过重新使用缓冲区来减少内存压力(而不是每次我需要添加块时执行新的char [10000000])。 我注意到的是ADO.NET似乎每次调用我的sproc时都会分配一个新的缓冲区。 每次调用sproc时,我都会看到我的进程的工作集几乎上升了20 MB。

现在,您可能认为这是因为“新字符串(缓冲区)”(我做过),但是当我这样做时,我会得到相同的行为

param = command.Parameters.Add("@text", SqlDbType.Text, buffer.Length);
param.Value = buffer;

有没有办法阻止ADO.NET每次向数据库添加一大块文本时进行额外的内存分配?

谢谢!


I have a C# app that needs to add large amounts of text to a SQL Server database (one row could have up to 1GB of text). So, I have some code that looks like:

SqlParameter param = command.Parameters.Add("@text", SqlDbType.NVarChar);
param.Value = new string(buffer);

The text is added in chunks, so the buffer is say, 20 MB. I've tried to reduce memory pressure by re-using buffers (as opposed to doing a new char[10000000] every time I need to add a chunk). What I'm noticing is that ADO.NET seems to be allocating a new buffer for every time my sproc is called. Every time the sproc is called, I see my process's working set go up by almost exactly 20 MB.

Now, you might think that's because of the "new string(buffer)" (I did), but I get the same behavior when I do

param = command.Parameters.Add("@text", SqlDbType.Text, buffer.Length);
param.Value = buffer;

Is there any way to prevent ADO.NET from doing the extra memory allocation every time I add a chunk of text to the database?

Thanks!


原文:https://stackoverflow.com/questions/1928600
更新时间:2023-04-18 22:04

最满意答案

函数签名应该取当前序列的最小编号,当前序列的最大编号和剩余序列。 如果剩余列表的头部大于当前序列的最大号码,则只需递增新的最大号码。 否则,返回当前差异的最大值和剩余当前头部的递归调用作为最小值和最大值。


The function signature should take the lowest number of the current sequence, the highest number of the current sequence and the remainding sequence. If the head of the remaining list is bigger than the highest number of the current sequence, just recurse with the new highest number. Otherwise, return the maximum of the current difference and the recursive call for the remainder with the current head as minimum and maximum.

相关问答

更多

相关文章

更多

最新问答

更多
  • 如何在Laravel 5.2中使用paginate与关系?(How to use paginate with relationships in Laravel 5.2?)
  • linux的常用命令干什么用的
  • 由于有四个新控制器,Auth刀片是否有任何变化?(Are there any changes in Auth blades due to four new controllers?)
  • 如何交换返回集中的行?(How to swap rows in a return set?)
  • 在ios 7中的UITableView部分周围绘制边界线(draw borderline around UITableView section in ios 7)
  • 使用Boost.Spirit Qi和Lex时的空白队长(Whitespace skipper when using Boost.Spirit Qi and Lex)
  • Java中的不可变类(Immutable class in Java)
  • WordPress发布查询(WordPress post query)
  • 如何在关系数据库中存储与IPv6兼容的地址(How to store IPv6-compatible address in a relational database)
  • 是否可以检查对象值的条件并返回密钥?(Is it possible to check the condition of a value of an object and JUST return the key?)
  • GEP分段错误LLVM C ++ API(GEP segmentation fault LLVM C++ API)
  • 绑定属性设置器未被调用(Bound Property Setter not getting Called)
  • linux ubuntu14.04版没有那个文件或目录
  • 如何使用JSF EL表达式在param中迭代变量(How to iterate over variable in param using JSF EL expression)
  • 是否有可能在WPF中的一个单独的进程中隔离一些控件?(Is it possible to isolate some controls in a separate process in WPF?)
  • 使用Python 2.7的MSI安装的默认安装目录是什么?(What is the default installation directory with an MSI install of Python 2.7?)
  • 寻求多次出现的表达式(Seeking for more than one occurrence of an expression)
  • ckeditor config.protectedSource不适用于editor.insertHtml上的html元素属性(ckeditor config.protectedSource dont work for html element attributes on editor.insertHtml)
  • linux只知道文件名,不知道在哪个目录,怎么找到文件所在目录
  • Actionscript:检查字符串是否包含域或子域(Actionscript: check if string contains domain or subdomain)
  • 将CouchDB与AJAX一起使用是否安全?(Is it safe to use CouchDB with AJAX?)
  • 懒惰地初始化AutoMapper(Lazily initializing AutoMapper)
  • 使用hasclass为多个div与一个按钮问题(using hasclass for multiple divs with one button Problems)
  • Windows Phone 7:检查资源是否存在(Windows Phone 7: Check If Resource Exists)
  • 无法在新线程中从FREContext调用getActivity()?(Can't call getActivity() from FREContext in a new thread?)
  • 在Alpine上升级到postgres96(/ usr / bin / pg_dump:没有这样的文件或目录)(Upgrade to postgres96 on Alpine (/usr/bin/pg_dump: No such file or directory))
  • 如何按部门显示报告(How to display a report by Department wise)
  • Facebook墙贴在需要访问令牌密钥后无法正常工作(Facebook wall post not working after access token key required)
  • Javascript - 如何在不擦除输入的情况下更改标签的innerText(Javascript - how to change innerText of label while not wiping out the input)
  • WooCommerce / WordPress - 不显示具有特定标题的产品(WooCommerce/WordPress - Products with specific titles are not displayed)