首页 \ 问答 \ Hadoop将数据附加到hdfs文件并忽略重复的条目(Hadoop append data to hdfs file and ignore duplicate entries)

Hadoop将数据附加到hdfs文件并忽略重复的条目(Hadoop append data to hdfs file and ignore duplicate entries)

如何将数据附加到HDFS文件并忽略重复值?

我有一个巨大的HDFS文件(MainFile),我有两个来自不同来源的新文件,我想将这些文件中的数据附加到MainFile。

主文件和其他文件具有相同的结构。


How can I append data to HDFS files and ignore duplicate values?

I have a huge HDFS file (MainFile) and I have 2 other new files from different sources and I want to append data from this files to the MainFile.

Main File and the other files has same structure.


原文:https://stackoverflow.com/questions/30892705
更新时间:2023-05-22 15:05

最满意答案

只是重复前两个完美的答案。 这里它试图弄清楚至少必须根据用户提供的逻辑设置位置和文本

Private Sub Button1_Click(ByVal sender As System.Object, _
      ByVal e As System.EventArgs) Handles Button1.Click

      ' YOU decide where EACH new label goes and pass the X,Y for each 
      ' new label; YOU decide on the text and pass it.  you can make them
      ' variables, but YOU have to do some of the thinking....

      ' <...> == information YOU provide
      Private x As integer = <where you want the new label>
      Private y as integer = <where you want the new label>
      Private txt as String = <Text for this new label>

      ' EXAMPLEs
      ' a) set text from a textbox
           ' txt = txtLblText.Text
      ' b) Set X position from another code integer variable
           ' x = thisX
      ' c) Set Y position from textbox input
           ' y = Integer.Parse(txtLblYPos.Text)

      Dim lbl as Label = MakeNewLabel(x, y, txt As string)

      Me.Controls.Add(lbl)  

End Sub


Friend function MakeNewLabel(x as integer, y as Integer, txt As String) as label
    Dim lbl As New label

    ' add other label props here as needed

    lbl.Size = New System.Drawing.Size(159, 23)       'set your size
    lbl.Location = New System.Drawing.Point(x, y)  'set your location
    lbl.Text = txt

    Return lbl
End Function

Just a rehash of the previous 2 perfectly good answers. Here it tries to make clear that at least the location and text must be set based on logic provided by the user

Private Sub Button1_Click(ByVal sender As System.Object, _
      ByVal e As System.EventArgs) Handles Button1.Click

      ' YOU decide where EACH new label goes and pass the X,Y for each 
      ' new label; YOU decide on the text and pass it.  you can make them
      ' variables, but YOU have to do some of the thinking....

      ' <...> == information YOU provide
      Private x As integer = <where you want the new label>
      Private y as integer = <where you want the new label>
      Private txt as String = <Text for this new label>

      ' EXAMPLEs
      ' a) set text from a textbox
           ' txt = txtLblText.Text
      ' b) Set X position from another code integer variable
           ' x = thisX
      ' c) Set Y position from textbox input
           ' y = Integer.Parse(txtLblYPos.Text)

      Dim lbl as Label = MakeNewLabel(x, y, txt As string)

      Me.Controls.Add(lbl)  

End Sub


Friend function MakeNewLabel(x as integer, y as Integer, txt As String) as label
    Dim lbl As New label

    ' add other label props here as needed

    lbl.Size = New System.Drawing.Size(159, 23)       'set your size
    lbl.Location = New System.Drawing.Point(x, y)  'set your location
    lbl.Text = txt

    Return lbl
End Function

相关问答

更多

相关文章

更多

最新问答

更多
  • 您如何使用git diff文件,并将其应用于同一存储库的副本的本地分支?(How do you take a git diff file, and apply it to a local branch that is a copy of the same repository?)
  • 将长浮点值剪切为2个小数点并复制到字符数组(Cut Long Float Value to 2 decimal points and copy to Character Array)
  • OctoberCMS侧边栏不呈现(OctoberCMS Sidebar not rendering)
  • 页面加载后对象是否有资格进行垃圾回收?(Are objects eligible for garbage collection after the page loads?)
  • codeigniter中的语言不能按预期工作(language in codeigniter doesn' t work as expected)
  • 在计算机拍照在哪里进入
  • 使用cin.get()从c ++中的输入流中丢弃不需要的字符(Using cin.get() to discard unwanted characters from the input stream in c++)
  • No for循环将在for循环中运行。(No for loop will run inside for loop. Testing for primes)
  • 单页应用程序:页面重新加载(Single Page Application: page reload)
  • 在循环中选择具有相似模式的列名称(Selecting Column Name With Similar Pattern in a Loop)
  • System.StackOverflow错误(System.StackOverflow error)
  • KnockoutJS未在嵌套模板上应用beforeRemove和afterAdd(KnockoutJS not applying beforeRemove and afterAdd on nested templates)
  • 散列包括方法和/或嵌套属性(Hash include methods and/or nested attributes)
  • android - 如何避免使用Samsung RFS文件系统延迟/冻结?(android - how to avoid lag/freezes with Samsung RFS filesystem?)
  • TensorFlow:基于索引列表创建新张量(TensorFlow: Create a new tensor based on list of indices)
  • 企业安全培训的各项内容
  • 错误:RPC失败;(error: RPC failed; curl transfer closed with outstanding read data remaining)
  • C#类名中允许哪些字符?(What characters are allowed in C# class name?)
  • NumPy:将int64值存储在np.array中并使用dtype float64并将其转换回整数是否安全?(NumPy: Is it safe to store an int64 value in an np.array with dtype float64 and later convert it back to integer?)
  • 注销后如何隐藏导航portlet?(How to hide navigation portlet after logout?)
  • 将多个行和可变行移动到列(moving multiple and variable rows to columns)
  • 提交表单时忽略基础href,而不使用Javascript(ignore base href when submitting form, without using Javascript)
  • 对setOnInfoWindowClickListener的意图(Intent on setOnInfoWindowClickListener)
  • Angular $资源不会改变方法(Angular $resource doesn't change method)
  • 在Angular 5中不是一个函数(is not a function in Angular 5)
  • 如何配置Composite C1以将.m和桌面作为同一站点提供服务(How to configure Composite C1 to serve .m and desktop as the same site)
  • 不适用:悬停在悬停时:在元素之前[复制](Don't apply :hover when hovering on :before element [duplicate])
  • 常见的python rpc和cli接口(Common python rpc and cli interface)
  • Mysql DB单个字段匹配多个其他字段(Mysql DB single field matching to multiple other fields)
  • 产品页面上的Magento Up出售对齐问题(Magento Up sell alignment issue on the products page)