PHP-FPM和Nginx:502坏网关(PHP-FPM and Nginx: 502 Bad Gateway)
组态
- Ubuntu Server 11.10 64位
- 亚马逊AWS,Ec2,托管在云端
- t1.micro实例
在我写任何其他内容之前,我想表示,我已经检查了nginx 502坏网关和Nginx + PHP-FPM 502坏网关线程,这不幸没有帮助我在这方面。
这个问题似乎很常见:nginx或php-fpm的配置错误可能会导致
502 Bad Gateway
错误,这是我无法摆脱的错误。 请注意, 即使我进入我的域根 ,也没有指定任何特定的目录。我正在运行一个Amazon EC2网络服务器,启用端口9000,端口80打开等。
特别是这个问题,我该如何摆脱这个讨厌的错误? 或者更好的是,如何让
php5-fpm
实际工作 。我已经尝试了甚么
大多数编辑配置文件,特别是
php-fpm.conf
和nginx.conf
。一世。 PHP-fpm.conf
我添加了以下内容,这并没有太大帮助:
;;;;;;;;;;;;; ; Fpm Start ; ;;;;;;;;;;;;; ;pm.start_servers = 20 ;pm.min_spare_servers = 5 ;pm.max_spare_servers = 35
现在,我尝试包括我的配置文件:
include=/etc/php5/fpm/*.conf
这只是使我更进一步。
完全配置
;;;;;;;;;;;;;;;;;;;;; ; FPM Configuration ; ;;;;;;;;;;;;;;;;;;;;; ; All relative paths in this configuration file are relative to PHP's install ; prefix (/usr). This prefix can be dynamicaly changed by using the ; '-p' argument from the command line. ; Include one or more files. If glob(3) exists, it is used to include a bunch of ; files from a glob(3) pattern. This directive can be used everywhere in the ; file. ; Relative path can also be used. They will be prefixed by: ; - the global prefix if it's been set (-p arguement) ; - /usr otherwise ;include=/etc/php5/fpm/*.conf ;;;;;;;;;;;;;;;;;; ; Global Options ; ;;;;;;;;;;;;;;;;;; [global] ; Pid file ; Note: the default prefix is /var ; Default Value: none pid = /var/run/php5-fpm.pid ; Error log file ; Note: the default prefix is /var ; Default Value: log/php-fpm.log error_log = /var/log/php5-fpm.log ; Log level ; Possible Values: alert, error, warning, notice, debug ; Default Value: notice log_level = notice ; If this number of child processes exit with SIGSEGV or SIGBUS within the time ; interval set by emergency_restart_interval then FPM will restart. A value ; of '0' means 'Off'. ; Default Value: 0 ;emergency_restart_threshold = 0 ; Interval of time used by emergency_restart_interval to determine when ; a graceful restart will be initiated. This can be useful to work around ; accidental corruptions in an accelerator's shared memory. ; Available Units: s(econds), m(inutes), h(ours), or d(ays) ; Default Unit: seconds ; Default Value: 0 emergency_restart_interval = 0 ; Time limit for child processes to wait for a reaction on signals from master. ; Available units: s(econds), m(inutes), h(ours), or d(ays) ; Default Unit: seconds ; Default Value: 0 ;process_control_timeout = 0 ; Send FPM to background. Set to 'no' to keep FPM in foreground for debugging. ; Default Value: yes daemonize = no ;;;;;;;;;;;;; ; Fpm Start ; ;;;;;;;;;;;;; ;pm.start_servers = 20 ;pm.min_spare_servers = 5 ;pm.max_spare_servers = 35 ;;;;;;;;;;;;;;;;;;;; ; Pool Definitions ; ;;;;;;;;;;;;;;;;;;;; ; Multiple pools of child processes may be started with different listening ; ports and different management options. The name of the pool will be ; used in logs and stats. There is no limitation on the number of pools which ; FPM can handle. Your system will tell you anyway :) ; To configure the pools it is recommended to have one .conf file per ; pool in the following directory: include=/etc/php5/fpm/pool.d/*.conf
ii 。 nginx.conf
诚然,这个配置是我访问过的几个网站的一小部分,但是我可以告诉你,在这个502 Bad Gateway业务之前,服务器运行正常(没有PHP工作)。
这个问题主要在于一件事情是非常可怕的,非常错误的。 而现在,当我尝试
service php5-fpm restart
,它挂起在我猜测是一个无限循环或者什么,我甚至不能CTRL - C出来。完全配置
user www-data; worker_processes 1; pid /var/run/nginx.pid; events { worker_connections 64; # multi_accept on; } http { ## # Basic Settings ## sendfile on; tcp_nopush off; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 2048; # server_tokens off; # server_names_hash_bucket_size 64; # server_name_in_redirect off; include /etc/nginx/mime.types; default_type application/octet-stream; ## # Logging Settings ## access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; ## # Gzip Settings ## gzip on; gzip_disable "msie6"; # gzip_vary on; # gzip_proxied any; # gzip_comp_level 6; # gzip_buffers 16 8k; # gzip_http_version 1.1; # gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript; ## # Virtual Host Configs ## include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; server { listen 80; server_name ec2-xx-xx-xx-xx.compute-x.amazonaws.com; location ~ ^(.+\.php)(.*)$ { root /home/wayvac/public; fastcgi_pass unix:/var/run/php5-fpm.pid; #fastcgi_pass 127.0.0.1:9000; #Un-comment this and comment "fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;" if you are not using php-fpm. fastcgi_index index.php; set $document_root2 $document_root; if ($document_root2 ~ "^(.*\\\\).*?[\\\\|\/]\.\.\/(.*)$") { set $document_root2 $1$2; } if ($document_root2 ~ "^(.*\\\\).*?[\\\\|\/]\.\.\/(.*)$") { set $document_root2 $1$2; } if ($document_root2 ~ "^(.*\\\\).*?[\\\\|\/]\.\.\/(.*)$") { set $document_root2 $1$2; } if ($document_root2 ~ "^(.*\\\\).*?[\\\\|\/]\.\.\/(.*)$") { set $document_root2 $1$2; } if ($document_root2 ~ "^(.*\\\\).*?[\\\\|\/]\.\.\/(.*)$") { set $document_root2 $1$2; } fastcgi_split_path_info ^(.+\.php)(.*)$; fastcgi_param SCRIPT_FILENAME $document_root2$fastcgi_script_name; fastcgi_param PATH_INFO $fastcgi_path_info; fastcgi_param PATH_TRANSLATED $document_root2$fastcgi_path_info; include fastcgi_params; fastcgi_param DOCUMENT_ROOT $document_root2; } access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; location / { root /home/wayvac/public; index index.html index.htm index.php; } location ~* \.(?:ico|css|js|gif|jpe?g|png)$ { # Some basic cache-control for static files to be sent to the browser expires max; add_header Pragma public; add_header Cache-Control "public, must-revalidate, proxy-revalidate"; } #include drop.conf; #include php.conf; } }
Configuration
- Ubuntu Server 11.10 64 bit
- Amazon AWS, Ec2, hosted on the cloud
- t1.micro instance
Before I write anything else, I'd like to state that I've checked both nginx 502 bad gateway and Nginx + PHP-FPM 502 Bad Gateway threads, which unfortunately haven't helped me in this regard.
The issue appears to be rather common: a misconfiguration of nginx or php-fpm can lead to a
502 Bad Gateway
error, which is something that I haven't been able to get rid of. Note that this appears even when I go to my domain root, without specifying any particular directory.I'm running an Amazon EC2 webserver, with port 9000 enabled, port 80 open, etc.
The question in particular is, how can I get rid of this nasty error? Or, better yet, how can I get
php5-fpm
to actually work.What I Have Attempted so Far
Mostly consistent editing of configuration files, notably
php-fpm.conf
andnginx.conf
.i. php-fpm.conf
I've added the following, which hasn't quite helped much:
;;;;;;;;;;;;; ; Fpm Start ; ;;;;;;;;;;;;; ;pm.start_servers = 20 ;pm.min_spare_servers = 5 ;pm.max_spare_servers = 35
Now, afterward I tried including my configuration files:
include=/etc/php5/fpm/*.conf
Which only screwed me even further.
Full Configuration
;;;;;;;;;;;;;;;;;;;;; ; FPM Configuration ; ;;;;;;;;;;;;;;;;;;;;; ; All relative paths in this configuration file are relative to PHP's install ; prefix (/usr). This prefix can be dynamicaly changed by using the ; '-p' argument from the command line. ; Include one or more files. If glob(3) exists, it is used to include a bunch of ; files from a glob(3) pattern. This directive can be used everywhere in the ; file. ; Relative path can also be used. They will be prefixed by: ; - the global prefix if it's been set (-p arguement) ; - /usr otherwise ;include=/etc/php5/fpm/*.conf ;;;;;;;;;;;;;;;;;; ; Global Options ; ;;;;;;;;;;;;;;;;;; [global] ; Pid file ; Note: the default prefix is /var ; Default Value: none pid = /var/run/php5-fpm.pid ; Error log file ; Note: the default prefix is /var ; Default Value: log/php-fpm.log error_log = /var/log/php5-fpm.log ; Log level ; Possible Values: alert, error, warning, notice, debug ; Default Value: notice log_level = notice ; If this number of child processes exit with SIGSEGV or SIGBUS within the time ; interval set by emergency_restart_interval then FPM will restart. A value ; of '0' means 'Off'. ; Default Value: 0 ;emergency_restart_threshold = 0 ; Interval of time used by emergency_restart_interval to determine when ; a graceful restart will be initiated. This can be useful to work around ; accidental corruptions in an accelerator's shared memory. ; Available Units: s(econds), m(inutes), h(ours), or d(ays) ; Default Unit: seconds ; Default Value: 0 emergency_restart_interval = 0 ; Time limit for child processes to wait for a reaction on signals from master. ; Available units: s(econds), m(inutes), h(ours), or d(ays) ; Default Unit: seconds ; Default Value: 0 ;process_control_timeout = 0 ; Send FPM to background. Set to 'no' to keep FPM in foreground for debugging. ; Default Value: yes daemonize = no ;;;;;;;;;;;;; ; Fpm Start ; ;;;;;;;;;;;;; ;pm.start_servers = 20 ;pm.min_spare_servers = 5 ;pm.max_spare_servers = 35 ;;;;;;;;;;;;;;;;;;;; ; Pool Definitions ; ;;;;;;;;;;;;;;;;;;;; ; Multiple pools of child processes may be started with different listening ; ports and different management options. The name of the pool will be ; used in logs and stats. There is no limitation on the number of pools which ; FPM can handle. Your system will tell you anyway :) ; To configure the pools it is recommended to have one .conf file per ; pool in the following directory: include=/etc/php5/fpm/pool.d/*.conf
ii. nginx.conf
In all honesty this configuration is a smattering of a few websites I've visited, but I can tell you that before this 502 Bad Gateway business, the server was running fine (without PHP working. Period.).
The issue primarily lies in the fact that something is terribly, terribly wrong. And now, when I try to do a
service php5-fpm restart
, it hangs in what I'm guessing is an infinite loop or something, which I can't even CTRL-C out of.Full Configuration
user www-data; worker_processes 1; pid /var/run/nginx.pid; events { worker_connections 64; # multi_accept on; } http { ## # Basic Settings ## sendfile on; tcp_nopush off; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 2048; # server_tokens off; # server_names_hash_bucket_size 64; # server_name_in_redirect off; include /etc/nginx/mime.types; default_type application/octet-stream; ## # Logging Settings ## access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; ## # Gzip Settings ## gzip on; gzip_disable "msie6"; # gzip_vary on; # gzip_proxied any; # gzip_comp_level 6; # gzip_buffers 16 8k; # gzip_http_version 1.1; # gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript; ## # Virtual Host Configs ## include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; server { listen 80; server_name ec2-xx-xx-xx-xx.compute-x.amazonaws.com; location ~ ^(.+\.php)(.*)$ { root /home/wayvac/public; fastcgi_pass unix:/var/run/php5-fpm.pid; #fastcgi_pass 127.0.0.1:9000; #Un-comment this and comment "fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;" if you are not using php-fpm. fastcgi_index index.php; set $document_root2 $document_root; if ($document_root2 ~ "^(.*\\\\).*?[\\\\|\/]\.\.\/(.*)$") { set $document_root2 $1$2; } if ($document_root2 ~ "^(.*\\\\).*?[\\\\|\/]\.\.\/(.*)$") { set $document_root2 $1$2; } if ($document_root2 ~ "^(.*\\\\).*?[\\\\|\/]\.\.\/(.*)$") { set $document_root2 $1$2; } if ($document_root2 ~ "^(.*\\\\).*?[\\\\|\/]\.\.\/(.*)$") { set $document_root2 $1$2; } if ($document_root2 ~ "^(.*\\\\).*?[\\\\|\/]\.\.\/(.*)$") { set $document_root2 $1$2; } fastcgi_split_path_info ^(.+\.php)(.*)$; fastcgi_param SCRIPT_FILENAME $document_root2$fastcgi_script_name; fastcgi_param PATH_INFO $fastcgi_path_info; fastcgi_param PATH_TRANSLATED $document_root2$fastcgi_path_info; include fastcgi_params; fastcgi_param DOCUMENT_ROOT $document_root2; } access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; location / { root /home/wayvac/public; index index.html index.htm index.php; } location ~* \.(?:ico|css|js|gif|jpe?g|png)$ { # Some basic cache-control for static files to be sent to the browser expires max; add_header Pragma public; add_header Cache-Control "public, must-revalidate, proxy-revalidate"; } #include drop.conf; #include php.conf; } }
原文:https://stackoverflow.com/questions/10003978
最满意答案
如果您已经具有数值特征并且不需要其他转换,则可以使用
VectorAssembler
组合包含独立变量的列:from pyspark.ml.feature import VectorAssembler assembler = VectorAssembler( inputCols=["your", "independent", "variables"], outputCol="features") transformed = assembler.transform(parsedData)
接下来你可以简单地映射:
from pyspark.mllib.regression import LabeledPoint from pyspark.sql.functions import col (transformed.select(col("outcome_column").alias("label"), col("features")) .rdd .map(lambda row: LabeledPoint(row.label, row.features)))
从Spark 2.0
ml
和mllib
API不再兼容,后者正在进行弃用和移除。 如果你仍然需要这个,你必须将ml.Vectors
转换为mllib.Vectors
。from pyspark.mllib import linalg as mllib_linalg from pyspark.ml import linalg as ml_linalg def as_old(v): if isinstance(v, ml_linalg.SparseVector): return mllib_linalg.SparseVector(v.size, v.indices, v.values) if isinstance(v, ml_linalg.DenseVector): return mllib_linalg.DenseVector(v.values) raise ValueError("Unsupported type {0}".format(type(v)))
和地图:
lambda row: LabeledPoint(row.label, as_old(row.features)))
If you already have numerical features and which require no additional transformations you can use
VectorAssembler
to combine columns containing independent variables:from pyspark.ml.feature import VectorAssembler assembler = VectorAssembler( inputCols=["your", "independent", "variables"], outputCol="features") transformed = assembler.transform(parsedData)
Next you can simply map:
from pyspark.mllib.regression import LabeledPoint from pyspark.sql.functions import col (transformed.select(col("outcome_column").alias("label"), col("features")) .rdd .map(lambda row: LabeledPoint(row.label, row.features)))
As of Spark 2.0
ml
andmllib
API are no longer compatible and the latter one is going towards deprecation and removal. If you still need this you'll have to convertml.Vectors
tomllib.Vectors
.from pyspark.mllib import linalg as mllib_linalg from pyspark.ml import linalg as ml_linalg def as_old(v): if isinstance(v, ml_linalg.SparseVector): return mllib_linalg.SparseVector(v.size, v.indices, v.values) if isinstance(v, ml_linalg.DenseVector): return mllib_linalg.DenseVector(v.values) raise ValueError("Unsupported type {0}".format(type(v)))
and map:
lambda row: LabeledPoint(row.label, as_old(row.features)))
相关问答
更多-
如果您已经具有数值特征并且不需要其他转换,则可以使用VectorAssembler组合包含独立变量的列: from pyspark.ml.feature import VectorAssembler assembler = VectorAssembler( inputCols=["your", "independent", "variables"], outputCol="features") transformed = assembler.transform(parsedData) ...
-
sqlContext.createDataFrame(PANDASDATA) sqlContext.createDataFrame(PANDASDATA)
-
这是不可能的。 Dataframe是Spark中的一个分布式集合。 Dataframes只能在驱动程序节点上创建(即在转换/操作之外)。 此外,在Spark中,您不能对其他操作中的RDD / Dataframes / Datasets执行操作:例如,以下代码会产生错误。 rdd.map(v => rdd1.filter(e => e == v)) DF和DS也有RDD在下面,所以在那里有相同的行为。 That is not possible. Dataframe is a distributed coll ...
-
既然没有答案,我会分享我在研究中观察到的。 在Spark 2.1.0中使用现在处于维护模式的Spark MLlib时,使用标记点是可以的。 然而,我的功能是分类的,因此使用Spark ML使用DataFrame API,我不得不使用StringIndexer,OneHotEncoder和Pipelines将它们转换为向量来选择我的功能和标签。 回答这个问题 是的,Labeled Points可以与这些功能一起使用,但是在使用Spark MLlib时。 我无法实现多层感知器,因为它需要libsvm格式的数据, ...
-
将Spark DataFrame从Python移动到Scala和Zeppelin(Moving Spark DataFrame from Python to Scala whithn Zeppelin)[2023-07-10]
您可以put内部Java对象put Python包装器: %pyspark df = sc.parallelize([(1, "foo"), (2, "bar")]).toDF(["k", "v"]) z.put("df", df._jdf) 然后确保使用正确的类型: val df = z.get("df").asInstanceOf[org.apache.spark.sql.DataFrame] // df: org.apache.spark.sql.DataFrame = [k: bigint, v ... -
将Spark Dataframe拆分到每一行并转换为JSON - Python(Split Spark Dataframe to each row and convert to JSON - Python)[2021-08-13]
要读取json文件,可以使用sqlContext.jsonFile()。 您可以使用常规SQL查询进行处理。 您可以在此处查看更多信息 火花适用于分区。 您的数据将分为多个分区并在执行程序上运行。 这将基于您使用的模式由spark进行。 不确定您是否使用YARN。 在python中,您可以使用boto3将数据保存到amazon s3。 它是一个非常容易使用的包。 看这里 To read json files, you can use sqlContext.jsonFile(). The you can us ... -
Python Spark DataFrame:用SparseVector替换null(Python Spark DataFrame: replace null with SparseVector)[2023-07-21]
你可以使用udf : from pyspark.sql.functions import udf, lit from pyspark.ml.linalg import * fill_with_vector = udf( lambda x, i: x if x is not None else SparseVector(i, {}), VectorUDT() ) df = sc.parallelize([ (SparseVector(5, {1: 1.0}), SparseVect ... -
如果您使用的是spark.sql.hive.metastore.jars 1.5或更高版本,则可能有多个或错误版本的spark.sql.hive.metastore.jars 您可以将以下设置添加到$ SPARK_HOME / conf / spark_defaults.conf spark.sql.hive.metastore.version XXX spark.sql.hive.metastore.jars maven 如果将spark.sql.hive.metastore.jars设置为maven,则 ...
-
使用row_number窗口函数: from pyspark.sql.functions import row_number from pyspark.sql import Window w = Window.partitionBy("some_column").orderBy("some_other_column") df.withColumn("rn", row_number().over(w)) With row_number window function: from pyspark.sql.f ...
-
以下是Python中使用Dataframe API的示例实现(Spark 1.6 +)。 import pyspark.sql.functions as F import numpy as np from pyspark.sql.types import FloatType 假设我们在“工资”火花数据框中为客户提供月薪,例如: 月| customer_id | 薪水 我们希望在所有的月份中找到每个客户的平均工资 第一步:编写用户定义的函数来计算中位数 def find_median(values_list ...