首页
\
问答
\
设置数据节点时Hadoop中的特权操作异常？(Privileged action exception in Hadoop while setting up data nodes?)
设置数据节点时Hadoop中的特权操作异常？(Privileged action exception in Hadoop while setting up data nodes?)
我在我的mac OS X中设置了hadoop框架。使用virtualbox，我安装了ubuntu服务器（一个nameNode）和另外两个ubuntu服务器作为datanode。 我已经根据我的知识正确设置了core-site.xml和其他配置。 当我使用start-all.sh启动框架时，一切似乎都很好。 在NameNode上执行
jps，每个数据节点都显示该进程处于活动状态。 但是在GUI网址上，我看到没有节点存在。 看附图。 以下是来自日志文件的错误消息：2014-06-23 18:00:01,167 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at: 0.0.0.0:50070 2014-06-23 18:00:01,174 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2014-06-23 18:00:01,175 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 10001: starting 2014-06-23 18:00:01,178 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 10001: starting 2014-06-23 18:00:01,186 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 10001: starting 2014-06-23 18:00:01,190 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 10001: starting 2014-06-23 18:00:01,195 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 10001: starting 2014-06-23 18:00:01,195 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 10001: starting 2014-06-23 18:00:01,196 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 10001: starting 2014-06-23 18:00:01,198 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 10001: starting 2014-06-23 18:00:01,201 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 10001: starting 2014-06-23 18:00:01,206 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 10001: starting 2014-06-23 18:00:01,213 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 10001: starting 2014-06-23 18:00:05,692 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:sridhar cause:java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 2014-06-23 18:00:05,692 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 10001, call addBlock(/usr/local/hadoop/tmp/mapred/system/jobtracker.info, DFSClient_NONMAPREDUCE_1288778011_1, null) from 127.0.0.1:56524: error: java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920) at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426) 2014-06-23 18:00:06,716 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:sridhar cause:java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 2014-06-23 18:00:06,716 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 10001, call addBlock(/usr/local/hadoop/tmp/mapred/system/jobtracker.info, DFSClient_NONMAPREDUCE_1288778011_1, null) from 127.0.0.1:56524: error: java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
PS：我确实在寻找其他显示类似错误的帖子。 我试过了。 但没有成功
编辑：
namenode服务器在发出
jps命令时不显示datanode和tasktracker。 datanode上的日志如下所示:(可能datanode没有建立连接）
2014-06-23 17:14:16,003 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: ubuntuhadoop/69.91.135.178:10001. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS) 2014-06-23 17:14:17,005 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: ubuntuhadoop/69.91.135.178:10001. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS) 2014-06-23 17:14:18,006 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: ubuntuhadoop/69.91.135.178:10001. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)I have set-up hadoop framework in my mac OS X. Using virtualbox, I installed ubuntu server (one nameNode) and two more ubuntu server to act as datanodes. I have set up core-site.xml and other configurations correctly to my knowledge. when I start the framework using start-all.sh, everything seems fine. doing
jpson NameNode and each datanodes shows that the process is active. but on the GUI url, I see that no nodes are live. see the attached image. Here is the error message from log file:2014-06-23 18:00:01,167 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at: 0.0.0.0:50070 2014-06-23 18:00:01,174 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2014-06-23 18:00:01,175 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 10001: starting 2014-06-23 18:00:01,178 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 10001: starting 2014-06-23 18:00:01,186 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 10001: starting 2014-06-23 18:00:01,190 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 10001: starting 2014-06-23 18:00:01,195 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 10001: starting 2014-06-23 18:00:01,195 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 10001: starting 2014-06-23 18:00:01,196 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 10001: starting 2014-06-23 18:00:01,198 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 10001: starting 2014-06-23 18:00:01,201 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 10001: starting 2014-06-23 18:00:01,206 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 10001: starting 2014-06-23 18:00:01,213 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 10001: starting 2014-06-23 18:00:05,692 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:sridhar cause:java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 2014-06-23 18:00:05,692 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 10001, call addBlock(/usr/local/hadoop/tmp/mapred/system/jobtracker.info, DFSClient_NONMAPREDUCE_1288778011_1, null) from 127.0.0.1:56524: error: java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920) at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426) 2014-06-23 18:00:06,716 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:sridhar cause:java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 2014-06-23 18:00:06,716 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 10001, call addBlock(/usr/local/hadoop/tmp/mapred/system/jobtracker.info, DFSClient_NONMAPREDUCE_1288778011_1, null) from 127.0.0.1:56524: error: java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
PS: I did look for other posts that showed similar errors. I tried them. but not successful
EDIT:
The namenode server does not show datanodes and tasktracker on issuing
jpscommand. The log on the datanode looks as follows: (probably datanode is not establishing connection)
2014-06-23 17:14:16,003 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: ubuntuhadoop/69.91.135.178:10001. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS) 2014-06-23 17:14:17,005 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: ubuntuhadoop/69.91.135.178:10001. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS) 2014-06-23 17:14:18,006 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: ubuntuhadoop/69.91.135.178:10001. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
原文：
更新时间：2022-05-03 07:05
最满意答案
你在说什么是一个GUI。 最简单的示例使用matplotlib内置的小部件 ：
import numpy as np import matplotlib.pyplot as plt from matplotlib.mlab import bivariate_normal from matplotlib.widgets import Slider, Button #Setup figure and data fig, ax = plt.subplots() plt.subplots_adjust(bottom=0.25) delta = 0.5 t = np.arange(0.0, 100.0, 0.1) x = np.arange(-3.0, 4.001, delta) y = np.arange(-4.0, 3.001, delta) X, Y = np.meshgrid(x, y) Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0) Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1) Z = (Z1 - Z2) * 5. cmap = plt.cm.rainbow im = ax.pcolormesh(X, Y, Z, cmap=cmap) fig.colorbar(im) axcolor = 'lightgoldenrodyellow' axtime = plt.axes([0.25, 0.1, 0.65, 0.03], axisbg=axcolor) stime = Slider(axtime, 'Time', 0.0, 100.0, valinit=50.0) #Routines to reset and update sliding bar def reset(event): stime.reset() def update(val): time = stime.val/10. Z = (Z1 - Z2) * time im.set_array(Z.ravel()) fig.canvas.draw() #Bind sliding bar and reset button stime.on_changed(update) resetax = plt.axes([0.8, 0.025, 0.1, 0.04]) button = Button(resetax, 'Reset', color=axcolor, hovercolor='0.975') button.on_clicked(reset) plt.show()这应该是一个开始。 如果您希望它看起来更好（并添加更多功能），那么您需要转到像
wxpython这样的GUI框架，请查看此示例。更符合您数据结构的示例如下：
import numpy as np import matplotlib.pyplot as plt from matplotlib.mlab import bivariate_normal from matplotlib.widgets import Slider, Button #Setup figure and data fig, ax = plt.subplots() plt.subplots_adjust(bottom=0.25) delta = 0.5 t = np.linspace(0.0, 100.0, 256) x = np.linspace(-4.0, 4.001, 512) y = np.linspace(-4.0, 4.001, 512) X, Y = np.meshgrid(x, y) Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0) Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1) XZslice = np.zeros((256,512,512)) for i in range(t.shape[0]): XZslice[i,:,:] = (Z1 - Z2) * t[i]/10. cmap = plt.cm.rainbow im = ax.pcolormesh(XZslice[128,:,:], cmap=cmap) fig.colorbar(im) axcolor = 'lightgoldenrodyellow' axtime = plt.axes([0.25, 0.1, 0.65, 0.03], axisbg=axcolor) stime = Slider(axtime, 'Time', 0.0, 100.0, valinit=50.0) #Routines to reset and update sliding bar def reset(event): stime.reset() def update(val): time = int(stime.val/100.* 256) im.set_array(XZslice[time,:,:].ravel()) fig.canvas.draw() #Bind sliding bar and reset button stime.on_changed(update) resetax = plt.axes([0.8, 0.025, 0.1, 0.04]) button = Button(resetax, 'Reset', color=axcolor, hovercolor='0.975') button.on_clicked(reset) plt.show()What you are talking about is a GUI. The simplest example uses the matplotlib inbuilt widgets:
import numpy as np import matplotlib.pyplot as plt from matplotlib.mlab import bivariate_normal from matplotlib.widgets import Slider, Button #Setup figure and data fig, ax = plt.subplots() plt.subplots_adjust(bottom=0.25) delta = 0.5 t = np.arange(0.0, 100.0, 0.1) x = np.arange(-3.0, 4.001, delta) y = np.arange(-4.0, 3.001, delta) X, Y = np.meshgrid(x, y) Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0) Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1) Z = (Z1 - Z2) * 5. cmap = plt.cm.rainbow im = ax.pcolormesh(X, Y, Z, cmap=cmap) fig.colorbar(im) axcolor = 'lightgoldenrodyellow' axtime = plt.axes([0.25, 0.1, 0.65, 0.03], axisbg=axcolor) stime = Slider(axtime, 'Time', 0.0, 100.0, valinit=50.0) #Routines to reset and update sliding bar def reset(event): stime.reset() def update(val): time = stime.val/10. Z = (Z1 - Z2) * time im.set_array(Z.ravel()) fig.canvas.draw() #Bind sliding bar and reset button stime.on_changed(update) resetax = plt.axes([0.8, 0.025, 0.1, 0.04]) button = Button(resetax, 'Reset', color=axcolor, hovercolor='0.975') button.on_clicked(reset) plt.show()This should be a start. If you want it to look better (and add more functionality) then you need to go to a GUI framework like
wxpython, check out this example.An example which is more inline with your data-structure would go as follows:
import numpy as np import matplotlib.pyplot as plt from matplotlib.mlab import bivariate_normal from matplotlib.widgets import Slider, Button #Setup figure and data fig, ax = plt.subplots() plt.subplots_adjust(bottom=0.25) delta = 0.5 t = np.linspace(0.0, 100.0, 256) x = np.linspace(-4.0, 4.001, 512) y = np.linspace(-4.0, 4.001, 512) X, Y = np.meshgrid(x, y) Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0) Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1) XZslice = np.zeros((256,512,512)) for i in range(t.shape[0]): XZslice[i,:,:] = (Z1 - Z2) * t[i]/10. cmap = plt.cm.rainbow im = ax.pcolormesh(XZslice[128,:,:], cmap=cmap) fig.colorbar(im) axcolor = 'lightgoldenrodyellow' axtime = plt.axes([0.25, 0.1, 0.65, 0.03], axisbg=axcolor) stime = Slider(axtime, 'Time', 0.0, 100.0, valinit=50.0) #Routines to reset and update sliding bar def reset(event): stime.reset() def update(val): time = int(stime.val/100.* 256) im.set_array(XZslice[time,:,:].ravel()) fig.canvas.draw() #Bind sliding bar and reset button stime.on_changed(update) resetax = plt.axes([0.8, 0.025, 0.1, 0.04]) button = Button(resetax, 'Reset', color=axcolor, hovercolor='0.975') button.on_clicked(reset) plt.show()
相关问答
更多-
TCP/IP模型是一个________。[2023-05-19]
a -
下列中不属于面向对象的编程语言的是?[2022-05-30]
a -
1）不确定你的意思。 在init和animate你需要返回PLOT和time_text ，所以你用逗号分隔它们，这样每个函数都会返回一个元组。 对于PLOT, = sub.plot([],[])的'悬挂'逗号PLOT, = sub.plot([],[]) ， sub.plot返回一个包含单个元素的列表，即matplotlib.lines.Line2D对象。 PLOT,正在解压缩这个单一元素列表。 所以，你也可以这样做： PLOT = sub.plot([],[])[0] 获得元素。 2）您的代码确实更新了 ...
-
您的示例的唯一问题是如何填充animate函数中的新坐标。 set_offsets需要一个Nx2 ndarray，你提供了两个1d数组的元组。 所以只需使用： def animate(i): data = np.hstack((x[:i,np.newaxis], y[:i, np.newaxis])) scat.set_offsets(data) return scat, 并保存您可能想要调用的动画： anim.save('animation.mp4') The only pro ...
-
python Matplotlib gtk - 使用FuncAnimation制作动画图(python Matplotlib gtk - animate plot with FuncAnimation)[2023-08-16]
我想这就是你想要实现的目标。 请注意，我改变了您阅读文件的方式。 with open() as f:处理您忘记的文件关闭操作。 也可以在构建器文件中写入信号处理程序的名称，这样就可以简单地说self.builder.connect(self)并省略self.signals dicitonary。 import gtk from matplotlib.backends.backend_gtkagg import FigureCanvasGTKAgg as FigureCanvas import matplot ... -
Python Matplotlib FuncAnimation.save（）仅保存100帧(Python Matplotlib FuncAnimation.save() only saves 100 frames)[2022-03-14]
ani = animation.FuncAnimation(fig, simPoints, simData, blit=False, interval=10, repeat=True, save_count=200) 将解决这个问题。 在内部， save仅保存固定数量的帧。 如果传入固定长度的序列或数字，mpl可以正确地猜测长度。 如果传入（可能是无限的）生成器并且未传入save_count则默认为100。 ani = animation.F ... -
每帧更新600个矩形非常慢，代码中的cbar_blit模式更快，因为您只更新颜色更改的矩形。 您可以使用PatchCollection来加速绘图，这里是代码： import numpy as np import matplotlib matplotlib.use("TkAgg") import matplotlib.pyplot as plt import matplotlib.animation as manim from matplotlib.collections import PatchCollec ...
-
你的init函数不对，它应该是： def init(): line.set_ydata(U[1,:]) return line 您也不需要将numpy切片转换为列表。 Your init function isn't right, it should be: def init(): line.set_ydata(U[1,:]) return line You also don't need to convert the numpy slice to a list.
-
您可以为frames参数提供帧数，而不是用于绘图的值。 ani = animation.FuncAnimation(fig1, animate, frames=len(lead) ) 这相当于使用0和len(lead)之间的范围，并将使用当前帧的整数调用动画。 您可以使用此数字从动画功能中的lead和host数组中选择适当的值。 def animate(i): x1 = lead[i,0] x2 = host[i,0] rect_l.set_x(x1) rect_h.set ...
-
如何从matplotlib向FuncAnimation添加时间控制面板(How to add a time control panel to a FuncAnimation from matplotlib)[2023-05-26]
你在说什么是一个GUI。 最简单的示例使用matplotlib内置的小部件 ： import numpy as np import matplotlib.pyplot as plt from matplotlib.mlab import bivariate_normal from matplotlib.widgets import Slider, Button #Setup figure and data fig, ax = plt.subplots() plt.subplots_adjust(botto ...