Nutch以外的爬虫与Elasticsearch一起使用(Crawlers other than Nutch that work with Elasticsearch)
我正在尝试在设置数据系统时获得一些建议。 我想设置一个网络爬行系统。 它会定期爬行几百/千个站点。
我知道Nutch并且使用过Nutch,但是我想知道其他人是否知道比Nutch更好的爬虫。
我也使用Elasticsearch作为索引器,很难让Nutch使用更新版本的ES。
I'm trying to get some suggestions as I setup my data system. I'd like to setup a system for web crawling. It'll crawl probably a few hundred/thousand sites on a regular basis.
I'm aware of Nutch and have used Nutch, however I'd like to know if others know of a better crawler than Nutch.
I'm also using Elasticsearch as the indexer and its quite hard to get Nutch to work with newer versions of ES.
原文:https://stackoverflow.com/questions/48564305
最满意答案
您可以从HSV (色调,饱和度,值)映射到RGB,根据标量值改变色调,并将饱和度和值设置为常量。
import colorsys def scalar_to_rgb(scalar): return colorsys.hsv_to_rgb(scalar, 1, 1)
在将值传递给函数之前,将值缩放到0到1之间。 颜色环绕,1将给出与0相同的结果。如果这是一个问题,您只能使用0到1之间的部分范围。
You could map from HSV (hue, saturation, value) to RGB, varying the hue according to your scalar value and setting saturation and value to constants.
import colorsys def scalar_to_rgb(scalar): return colorsys.hsv_to_rgb(scalar, 1, 1)
Scale the value to between 0 and 1 before passing it to the function. The colours wrap around, 1 will give the same result as 0. If this is an issue you could use only part of the range between 0 and 1.
相关问答
更多-
TCP/IP模型是一个________。[2023-10-02]
a -
下列中不属于面向对象的编程语言的是?[2022-05-30]
a -
您可以这样做以创建具有所选颜色的平方位图。 // Here you create the bound of your shape Rect rect = new Rect(0, 0, 1, 1); // You then create a Bitmap and get a canvas to draw into it Bitmap image = Bitmap.createBitmap(rect.width(), rect.height(), Config.ARGB_8888); Canvas canva ...
-
连续的颜色转换(Continuous Color Transition)[2021-12-27]
检查此JSFiddle是否有花式图形的转换。 /* ==================== Required Functions ==================== */ // This is required to get the initial background-color of an element. // The element might have it's bg-color already set before the transition. // Transition sho ... -
RGB颜色排列(RGB color permutation)[2022-06-27]
请注意,RGB值只是三个字节。 您可以使用简单的int值并从1(0x000001)到16777215(0xffffff)进行迭代。 可以通过以下公式检索RGB值: unsigned char R = color % 256; unsigned char G = (color / 256) % 256; unsigned char B = (color / 65536) % 256; 要么 unsigned char R = (color & 0xff); unsigned char G = (color ... -
关于html rgb颜色(Regarding html rgb color)[2022-10-14]
尝试“风格”。 This is my font Try with "style". This is my font -
您可以从HSV (色调,饱和度,值)映射到RGB,根据标量值改变色调,并将饱和度和值设置为常量。 import colorsys def scalar_to_rgb(scalar): return colorsys.hsv_to_rgb(scalar, 1, 1) 在将值传递给函数之前,将值缩放到0到1之间。 颜色环绕,1将给出与0相同的结果。如果这是一个问题,您只能使用0到1之间的部分范围。 You could map from HSV (hue, saturation, value) to ...
-
你必须做浮点中间计算。 分页会让你失望; 你有很多“热”(饱和)像素。 def rgb2ycbcr(im): xform = np.array([[.299, .587, .114], [-.1687, -.3313, .5], [.5, -.4187, -.0813]]) ycbcr = im.dot(xform.T) ycbcr[:,:,[1,2]] += 128 return np.uint8(ycbcr) def ycbcr2rgb(im): xform = ...
-
您可以使用X11中的表格 http://www.astrouw.edu.pl/~jskowron/colors-x11/rgb.html 为了找到颜色接近度,最好先将颜色转换为Lab颜色空间,以便欧几里得距离具有更多含义,然后最近的邻居会给出好的结果。 You can use a table such as the one in X11 http://www.astrouw.edu.pl/~jskowron/colors-x11/rgb.html In order to find color proximi ...
-
MATLAB提供的默认函数不会产生此错误。 此错误是由用户定义的函数rgb2lab覆盖默认函数引起的。 可能产生该错误的覆盖函数如下,请参见GeometricContext\src\tools\misc function [L,a,b] =rgb2lab(R,G,B) % function [L, a, b] = RGB2Lab(R, G, B) % RGB2Lab takes matrices corresponding to Red, Green, and Blue, and % transforms ...