首页 \ 问答 \ 在AWS EMR上的MapReduce代码中导入自定义函数(import custom function in MapReduce code on AWS EMR)

在AWS EMR上的MapReduce代码中导入自定义函数(import custom function in MapReduce code on AWS EMR)

我一直在努力奋斗2个小时吧!

我在python中创建了一个mapper脚本,它在其他python脚本中导入我的一个自定义函数。

    #!/usr/bin/env python
    import sys

    import testImport

    for line in sys.stdin:
        if line and line!='':
            words = line.strip().lower().split('\t')
            print '%s\t%s' % (words[0].strip(),testImport.age_classify(int(words[1])))

这段代码在我的终端上运行良好....问题是我将这个映射器函数上传到AWS Elastic MapReduce。 我的工作失败,错误地说“导入模块testImport失败”。

testImport是一个文件'testImport.py',它包含我的一些辅助函数(比如age_classify函数),我需要在每行标准输入上操作。

我将脚本与我的映射器脚本(给定脚本)放在同一个桶中。

当我添加'Streaming program'步骤时,我试图在参数部分传递它。 即使看到所有相关问题,我也不知道该怎么办。

我怎么能这样做???

任何帮助都会非常棒!

谢谢!


I have been struggling with this for 2 hours now!

I created a mapper script in python which is importing one of my custom functions in other python script.

    #!/usr/bin/env python
    import sys

    import testImport

    for line in sys.stdin:
        if line and line!='':
            words = line.strip().lower().split('\t')
            print '%s\t%s' % (words[0].strip(),testImport.age_classify(int(words[1])))

This code works well on my terminal....the problem is when i upload this mapper function to AWS Elastic MapReduce. My job fails with error saying "Failed to import module testImport".

testImport is a file 'testImport.py' which contains some of my helper functions (like the age_classify function), which i need to operate on each line of standard input.

I uploaded the script in the same bucket as my mapper script(the given script).

I tried to pass it in the arguments section when i add 'Streaming program' step. I have no clue what to do even after seeing all the related questions.

How can i get this done???

Any help would be really great!

Thank you!


原文:https://stackoverflow.com/questions/26824319
更新时间:2022-11-11 08:11

最满意答案

在分配和呈现MFMailComposeViewController之前,检查设备是否可以发送邮件:

if ([MFMailComposeViewController canSendMail]) {
//your code here

}

Before to alloc and present the MFMailComposeViewController check if the device can send mails:

if ([MFMailComposeViewController canSendMail]) {
//your code here

}

相关问答

更多
  • 我终于找到了一个有效的解决方案 - 所以这里是完成/记录之前: UINavigationController* savedUinvc = self.navigationController; UIViewController *one = nil; one = [savedUinvc popViewControllerAnimated:NO]; UIViewController *two = nil; ...
  • 在iOS7中, kCLAuthorizationStatusAuthorizedAlways和kCLAuthorizationStatusAuthorizedWhenInUse枚举值映射到kCLAuthorizationStatusAuthorized (在iOS 8中不推荐使用)。 这就是为什么你不会有任何崩溃。 In iOS7, kCLAuthorizationStatusAuthorizedAlways and kCLAuthorizationStatusAuthorizedWhenInUse enu ...
  • 在分配和呈现MFMailComposeViewController之前,检查设备是否可以发送邮件: if ([MFMailComposeViewController canSendMail]) { //your code here } Before to alloc and present the MFMailComposeViewController check if the device can send mails: if ([MFMailComposeViewController canSend ...
  • 我终于找到了这个问题的根本原因:我的UIScrollView子类中的subviews属性覆盖( EARestrictedScrollView.m#L54-56 )。 Autolayout以某种方式使用子视图属性并覆盖它会破坏事物。 这是不一致的,取决于许多其他事情,但修复subviews在100%的情况下为我处理它。 I finally found a root cause of this issue: subviews property override in my UIScrollView subcla ...
  • 在iOS 8中,应用程序容器的Te文件系统布局已更改。 应用程序及其内容不再存储在一个根目录中。 从iOS 8发行说明: 应用程序容器的文件系统布局在磁盘上已更改。 使用NSSearchPathForDirectoriesInDomains函数或URLForDirectory:inDomain:appropriateForURL:create:error:方法,而不是依赖硬编码的目录结构。 请参阅文件系统编程指南中的 访问文件和目录 。 我已经简要了解了这个问题,并且我可以看到VFRReader不遵循App ...
  • 出于某种原因Sergiy Salyuk没有发表回答,但他的评论有效。 问题在于从NSUserDefaults替换“AppleLanguages”中的初始语言数组 它节省了很多次,因为问题只出现在测试仪设备上,在其他国家,我不知道如何找到崩溃与此跟踪的连接以及NSUserDefaults中的“AppleLanguages”。 再次感谢Sergiy。 For some reason Sergiy Salyuk didn't post answer, but his comment works. Problem ...
  • 当您销毁之前添加了观察者的对象时,通常会抛出此异常。 要解决此问题,您需要在调用销毁该对象的方法之前在同一对象上调用removeObserver:forKeyPath: This exception is usually thrown when you are destroying an object that has an observer previously added to it. To resolve the problem you need to call removeObserver:forK ...
  • 看起来第一个BETA版本不支持模拟器上的Keychain Access开发人员API。 希望这很快就会出现。 更新:看起来问题在Beta 2中得到修复 Looks like the first BETA version doesn't support Keychain Access developer APIs on the simulator. Hopefully this is brought in soon. Update: looks like issue is fixed in Beta 2
  • 我最后用JSONModel替换了YYModel然后它工作正常。 I replaced YYModel with JSONModel at last then it work fine.
  • 您的路径只包含一个点。 由于这仅在iOS 7中崩溃,我认为iOS 8将此视为特殊情况,并且可能通过将节点的位置用作另一个点来完成路径。 无论如何,如果你想让它与iOS 7兼容,你将始终必须创建两个点的路径。 Your path only contains a single point. Since this is crashing only in iOS 7 I would assume that iOS 8 considers this as a special case and presumably c ...

相关文章

更多

最新问答

更多
  • 您如何使用git diff文件,并将其应用于同一存储库的副本的本地分支?(How do you take a git diff file, and apply it to a local branch that is a copy of the same repository?)
  • 将长浮点值剪切为2个小数点并复制到字符数组(Cut Long Float Value to 2 decimal points and copy to Character Array)
  • OctoberCMS侧边栏不呈现(OctoberCMS Sidebar not rendering)
  • 页面加载后对象是否有资格进行垃圾回收?(Are objects eligible for garbage collection after the page loads?)
  • codeigniter中的语言不能按预期工作(language in codeigniter doesn' t work as expected)
  • 在计算机拍照在哪里进入
  • 使用cin.get()从c ++中的输入流中丢弃不需要的字符(Using cin.get() to discard unwanted characters from the input stream in c++)
  • No for循环将在for循环中运行。(No for loop will run inside for loop. Testing for primes)
  • 单页应用程序:页面重新加载(Single Page Application: page reload)
  • 在循环中选择具有相似模式的列名称(Selecting Column Name With Similar Pattern in a Loop)
  • System.StackOverflow错误(System.StackOverflow error)
  • KnockoutJS未在嵌套模板上应用beforeRemove和afterAdd(KnockoutJS not applying beforeRemove and afterAdd on nested templates)
  • 散列包括方法和/或嵌套属性(Hash include methods and/or nested attributes)
  • android - 如何避免使用Samsung RFS文件系统延迟/冻结?(android - how to avoid lag/freezes with Samsung RFS filesystem?)
  • TensorFlow:基于索引列表创建新张量(TensorFlow: Create a new tensor based on list of indices)
  • 企业安全培训的各项内容
  • 错误:RPC失败;(error: RPC failed; curl transfer closed with outstanding read data remaining)
  • C#类名中允许哪些字符?(What characters are allowed in C# class name?)
  • NumPy:将int64值存储在np.array中并使用dtype float64并将其转换回整数是否安全?(NumPy: Is it safe to store an int64 value in an np.array with dtype float64 and later convert it back to integer?)
  • 注销后如何隐藏导航portlet?(How to hide navigation portlet after logout?)
  • 将多个行和可变行移动到列(moving multiple and variable rows to columns)
  • 提交表单时忽略基础href,而不使用Javascript(ignore base href when submitting form, without using Javascript)
  • 对setOnInfoWindowClickListener的意图(Intent on setOnInfoWindowClickListener)
  • Angular $资源不会改变方法(Angular $resource doesn't change method)
  • 在Angular 5中不是一个函数(is not a function in Angular 5)
  • 如何配置Composite C1以将.m和桌面作为同一站点提供服务(How to configure Composite C1 to serve .m and desktop as the same site)
  • 不适用:悬停在悬停时:在元素之前[复制](Don't apply :hover when hovering on :before element [duplicate])
  • 常见的python rpc和cli接口(Common python rpc and cli interface)
  • Mysql DB单个字段匹配多个其他字段(Mysql DB single field matching to multiple other fields)
  • 产品页面上的Magento Up出售对齐问题(Magento Up sell alignment issue on the products page)