折腾:
【已解决】把Python3的pipenv的Flask部署到CentOS服务器上
期间,接着去把本地用supervisor+gunicorn去部署没问题的Flask代码,想办法部署到服务器上。
其中此处是Python3的Flask,本地折腾时用的是supervisor和gunicorn。
先去服务器上确定,当前所用的Python3的环境在哪里
<code>[root@naturling-general-01 ~]# which python /usr/bin/python [root@naturling-general-01 ~]# where python -bash: where: command not found [root@naturling-general-01 ~]# which python3 /usr/bin/python3 [root@naturling-general-01 ~]# python --version Python 2.7.5 [root@naturling-general-01 ~]# python3 --version Python 3.4.5 [root@naturling-general-01 ~]# </code>
再去Python3中安装pipenv
吸取之前教训,不要强制重新安装pip3了。
先看看pip:
<code>[root@naturling-general-01 ~]# pip pip pip2 pip2.7 pip3 pip3.4 </code>
centos 部署Python3 Flask
先去安装Python3的pipenv
centos python3 pipenv
centos install pipenv python3
<code>[root@naturling-general-01 ~]# pip3 install pipenv --user Collecting pipenv Downloading http://mirrors.aliyun.com/pypi/packages/e5/fd/740a41c68c38262265d73191fb686f38d26c4ea1ea6080a65b2b626bedc7/pipenv-11.10.0-py3-none-any.whl (5.6MB) 100% || 5.6MB 81.0MB/s Requirement already satisfied: pip>=9.0.1 in /usr/lib/python3.4/site-packages (from pipenv) Collecting virtualenv-clone>=0.2.5 (from pipenv) Downloading http://mirrors.aliyun.com/pypi/packages/6d/c2/dccb5ccf599e0c5d1eea6acbd058af7a71384f9740179db67a9182a24798/virtualenv_clone-0.3.0-py2.py3-none-any.whl Requirement already satisfied: certifi in /usr/lib/python3.4/site-packages (from pipenv) Requirement already satisfied: setuptools>=36.2.1 in /usr/lib/python3.4/site-packages (from pipenv) Requirement already satisfied: virtualenv in /usr/lib/python3.4/site-packages (from pipenv) Installing collected packages: virtualenv-clone, pipenv Successfully installed pipenv-11.10.0 virtualenv-clone-0.3.0 </code>
但是安装后,没找到pipenv在哪
<code>[root@naturling-general-01 ~]# ll /usr/lib/python3.4/site-packages/ total 280 drwxr-xr-x 3 root root 4096 Mar 23 14:20 certifi drwxr-xr-x 2 root root 4096 Mar 23 14:20 certifi-2018.1.18.dist-info drwxr-xr-x 4 root root 4096 Mar 23 14:20 chardet drwxr-xr-x 2 root root 4096 Mar 23 14:20 chardet-3.0.4.dist-info drwxr-xr-x 3 root root 4096 Mar 23 14:20 cssselect drwxr-xr-x 2 root root 4096 Mar 23 14:20 cssselect-1.0.3.dist-info -rw-r--r-- 1 root root 195 Mar 23 14:21 easy-install.pth -rw-r--r-- 1 root root 126 Mar 23 14:20 easy_install.py drwxr-xr-x 4 root root 4096 Feb 4 23:09 et_xmlfile drwxr-xr-x 2 root root 4096 Feb 4 23:09 et_xmlfile-1.0.1-py3.4.egg-info drwxr-xr-x 3 root root 4096 Mar 23 14:20 fire drwxr-xr-x 2 root root 4096 Mar 23 14:20 fire-0.1.3-py3.4.egg-info drwxr-xr-x 3 root root 4096 Mar 23 14:20 idna drwxr-xr-x 2 root root 4096 Mar 23 14:20 idna-2.6.dist-info drwxr-xr-x 2 root root 4096 Feb 4 23:09 jdcal-1.3-py3.4.egg-info -rw-r--r-- 1 root root 12556 Sep 19 2016 jdcal.py drwxr-xr-x 21 root root 4096 Feb 4 23:09 openpyxl drwxr-xr-x 2 root root 4096 Feb 4 23:09 openpyxl-2.5.0-py3.4.egg-info drwxr-xr-x 11 root root 4096 Mar 23 14:21 pip drwxr-xr-x 2 root root 4096 Mar 23 14:21 pip-9.0.3.dist-info drwxr-xr-x 5 root root 4096 Mar 23 14:21 pkg_resources drwxr-xr-x 2 root root 4096 Mar 23 14:21 __pycache__ drwxr-xr-x 3 root root 4096 Mar 23 14:20 pyquery drwxr-xr-x 2 root root 4096 Mar 23 14:20 pyquery-1.4.0.dist-info drwxr-xr-x 3 root root 4096 Mar 23 14:20 requests drwxr-xr-x 2 root root 4096 Mar 23 14:20 requests-2.18.4.dist-info drwxr-xr-x 6 root root 4096 Mar 23 14:21 setuptools drwxr-xr-x 2 root root 4096 Mar 23 14:21 setuptools-39.0.1.dist-info -rw-r--r-- 1 root root 28 Dec 7 21:12 setuptools.pth drwxr-xr-x 2 root root 4096 Mar 23 14:20 six-1.11.0.dist-info -rw-r--r-- 1 root root 30888 Mar 23 14:20 six.py drwxr-xr-x 6 root root 4096 Mar 23 14:20 urllib3 drwxr-xr-x 2 root root 4096 Mar 23 14:20 urllib3-1.22.dist-info drwxr-xr-x 2 root root 4096 Dec 7 21:14 virtualenv-15.1.0.dist-info -rw-r--r-- 1 root root 99021 Dec 7 21:14 virtualenv.py drwxr-xr-x 3 root root 4096 Dec 7 21:14 virtualenv_support </code>
Pipenv – The Officially Recommended Python Packaging Tool – OSTechNix
然后找到了:
<code>[root@naturling-general-01 ~]# python3 -m site --user-base /root/.local [root@naturling-general-01 ~]# ll /root/.local/ bin/ lib/ [root@naturling-general-01 ~]# ll /root/.local/bin/pipenv pipenv pipenv-resolver [root@naturling-general-01 ~]# ll /root/.local/bin/ total 16 -rwxr-xr-x 1 root root 223 Apr 23 11:16 pewtwo -rwxr-xr-x 1 root root 207 Apr 23 11:16 pipenv -rwxr-xr-x 1 root root 218 Apr 23 11:16 pipenv-resolver -rwxr-xr-x 1 root root 218 Apr 23 11:16 virtualenv-clone [root@naturling-general-01 ~]# /root/.local/bin/pipenv --version pipenv, version 11.10.0 </code>
然后看到是有bash_profile的:
<code>[root@naturling-general-01 ~]# ll ~/. ./ .bash_logout .cache/ .dbshell .mysql_history .pydistutils.cfg .tcshrc .tmux.conf.local ../ .bash_profile .config/ .local/ .pip/ .python_history .tmux/ .viminfo .bash_history .bashrc .cshrc .mongorc.js .pki/ .ssh/ .tmux.conf .vimrc [root@naturling-general-01 ~]# ll ~/.bash* -rw------- 1 root root 19727 Apr 23 10:37 /root/.bash_history -rw-r--r--. 1 root root 18 Dec 29 2013 /root/.bash_logout -rw-r--r--. 1 root root 176 Dec 29 2013 /root/.bash_profile -rw-r--r--. 1 root root 176 Dec 29 2013 /root/.bashrc [root@naturling-general-01 ~]# cat ~/.bashrc # .bashrc # User specific aliases and functions alias rm='rm -i' alias cp='cp -i' alias mv='mv -i' # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi [root@naturling-general-01 ~]# cat ~/.bash_profile # .bash_profile # Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # User specific environment and startup programs PATH=$PATH:$HOME/bin export PATH </code>
然后去加到.bash_profile中并source一下
<code>vi ~/.bash_profile [root@naturling-general-01 ~]# cat ~/.bash_profile # .bash_profile # Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # User specific environment and startup programs PATH=$PATH:$HOME/bin #export PATH export PATH="$HOME/.local/bin:$PATH" </code>
再去source一下,就可以找到了:
<code>[root@naturling-general-01 ~]# echo $PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin [root@naturling-general-01 ~]# source ~/.bash_profile [root@naturling-general-01 ~]# echo $PATH /root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/root/bin [root@naturling-general-01 ~]# pipenv --version whipipenv, version 11.10.0 [root@naturling-general-01 ~]# which pipenv /root/.local/bin/pipenv </code>
然后去:
【基本解决】如何利用已有Pipfile和Pipfile.lock创建pipenv的虚拟环境
但是接着又出现:
【基本解决】pipenv用已有pipfile去创建环境失败:Warning Python 3.6 was not found on your system
然后还要继续安装只支持Python2的supervisor
不过在此之前,先去用gunicorn去运行app,看看是否正常能访问服务器端的Rest API接口
记得要去阿里云ECS的安全组中增加端口运行访问
另外在运行app之前,先要上传代码。
此处考虑用之前用过的PyCharm同步代码的功能:
然后服务器端运行gunicorn试试,结果可以运行,但是失败:
然后再去用supervisor去管理:
【已解决】CentOS中用supervisor去管理gunicorn的Flask的app
然后再去优化Flask:
【已解决】Flask中增加不同环境的app的config配置
但是折腾配置期间,本地再去用supervisorctl却出错:
【已解决】supervisorctl status all出错:http://localhost:9001 refused connection
然后服务器端的Flask的app终于可以正常工作了:
先去访问qa接口,可以正常返回内容了:
然后去浏览器打开地址:
http://x.x.x.x:port/files/5ad438d2a4bc71fa65bac96c/Green%20Eggs%20and%20Ham.mp3
最后可以下载到对应文件:
相关的一些log是:
<code>[root@naturling-general-01 logs]# cat gunicorn_access.log [24/Apr/2018:17:16:05 +0800] <1395> 112.4.64.141 "GET /qa?q=play+a+green+song HTTP/1.1" 500 0.008734 - -" "-" [24/Apr/2018:17:20:17 +0800] <1462> 112.4.64.141 "GET /qa?q=play+a+green+song HTTP/1.1" 200 0.026611 372 -" "Apache-HttpClient/4.5.2 (Java/1.8.0_152-release)" [24/Apr/2018:17:20:43 +0800] <1462> 112.4.64.141 "GET /files/5ad438d2a4bc71fa65bac96c/Green%20Eggs%20and%20Ham.mp3 HTTP/1.1" 200 15.673270 2401743 -" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1 Safari/605.1.15" [root@naturling-general-01 logs]# pwd /root/naturling_20180101/web/server/robotDemo/logs [root@naturling-general-01 logs]# cat gunicorn_error.log [2018-04-24 17:12:32 +0800] [1365] [INFO] Starting gunicorn 19.7.1 [2018-04-24 17:12:32 +0800] [1365] [INFO] Listening at: http://0.0.0.0:32851 (1365) [2018-04-24 17:12:32 +0800] [1365] [INFO] Using worker: threads [2018-04-24 17:12:32 +0800] [1368] [INFO] Booting worker with pid: 1368 。。。 [2018-04-24 17:12:33 +0800] [1395] [INFO] Booting worker with pid: 1395 [2018-04-24 17:16:05 +0800] [1395] [ERROR] Error handling request /qa?q=play+a+green+song ... [2018-04-24 17:16:23 +0800] [1391] [INFO] Worker reloading: /root/naturling_20180101/web/server/robotDemo/app.py modified [2018-04-24 17:16:23 +0800] [1395] [INFO] Worker reloading: /root/naturling_20180101/web/server/robotDemo/app.py modified [2018-04-24 17:16:23 +0800] [1395] [INFO] Worker exiting (pid: 1395) [2018-04-24 17:16:23 +0800] [1385] [INFO] Worker exiting (pid: 1385) [2018-04-24 17:16:23 +0800] [1391] [INFO] Worker exiting (pid: 1391) [2018-04-24 17:16:23 +0800] [1383] [INFO] Worker exiting (pid: 1383) [2018-04-24 17:16:23 +0800] [1370] [INFO] Worker exiting (pid: 1370) [2018-04-24 17:16:23 +0800] [1373] [INFO] Worker exiting (pid: 1373) [2018-04-24 17:16:23 +0800] [1380] [INFO] Worker exiting (pid: 1380) [2018-04-24 17:16:23 +0800] [1375] [INFO] Worker exiting (pid: 1375) [2018-04-24 17:16:23 +0800] [1437] [INFO] Booting worker with pid: 1437 [2018-04-24 17:16:23 +0800] [1438] [INFO] Booting worker with pid: 1438 [2018-04-24 17:16:23 +0800] [1441] [INFO] Booting worker with pid: 1441 [2018-04-24 17:16:23 +0800] [1368] [INFO] Worker reloading: /root/naturling_20180101/web/server/robotDemo/app.py modified [2018-04-24 17:16:23 +0800] [1450] [INFO] Booting worker with pid: 1450 。。。 [root@naturling-general-01 logs]# cat RobotQA.log ... [2018-04-24 17:20:17,077 INFO app.py:101 get] parser=<flask_restful.reqparse.RequestParser object at 0x7f1d628cd320> [2018-04-24 17:20:17,078 INFO app.py:104 get] parsedArgs={'q': 'play a green song'} [2018-04-24 17:20:17,078 INFO app.py:110 get] q=play a green song [2018-04-24 17:20:17,079 INFO app.py:119 get] foundSongName=<_sre.SRE_Match object; span=(0, 17), match='play a green song'> [2018-04-24 17:20:17,079 INFO app.py:125 get] songName=green [2018-04-24 17:20:17,079 INFO app.py:147 get] filenameRegex=re.compile('green', re.IGNORECASE) [2018-04-24 17:20:17,080 INFO app.py:153 get] findFileCursor=<gridfs.grid_file.GridOutCursor object at 0x7f1d628cd588> [2018-04-24 17:20:17,096 INFO app.py:155 get] findFileCount=8 [2018-04-24 17:20:17,101 INFO app.py:163 get] [ 1] _id=5ad44050a4bc71fa65bb28a5, contentType=audio/mpeg, filename=119 Chester Greenwood's Big Idea.mp3 [2018-04-24 17:20:17,101 INFO app.py:163 get] [ 2] _id=5ad4404fa4bc71fa65bb289b, contentType=application/pdf, filename=119 Chester Greenwood's Big Idea.pdf [2018-04-24 17:20:17,101 INFO app.py:163 get] [ 3] _id=5ad4406da4bc71fa65bb29db, contentType=audio/mpeg, filename=27 Green.mp3 [2018-04-24 17:20:17,101 INFO app.py:163 get] [ 4] _id=5ad43a6da4bc71fa65badfa5, contentType=audio/mpeg, filename=A green green garden.mp3 [2018-04-24 17:20:17,102 INFO app.py:163 get] [ 5] _id=5ad43a64a4bc71fa65badf54, contentType=application/pdf, filename=A green green garden.pdf [2018-04-24 17:20:17,102 INFO app.py:163 get] [ 6] _id=5ad4382ca4bc71fa65babfdd, contentType=application/pdf, filename=Go Away Big Green Monster.pdf [2018-04-24 17:20:17,102 INFO app.py:163 get] [ 7] _id=5ad4382da4bc71fa65babfed, contentType=audio/mpeg, filename=Go away big green monster.mp3 [2018-04-24 17:20:17,102 INFO app.py:163 get] [ 8] _id=5ad438d2a4bc71fa65bac96c, contentType=audio/mpeg, filename=Green Eggs and Ham.mp3 [2018-04-24 17:20:27,464 INFO app.py:186 get] fileId=5ad438d2a4bc71fa65bac96c, file_name=Green Eggs and Ham.mp3 [2018-04-24 17:20:27,465 INFO app.py:189 get] fileIdObj=5ad438d2a4bc71fa65bac96c [2018-04-24 17:20:27,468 INFO app.py:200 get] fileObj=<gridfs.grid_file.GridOut object at 0x7f1d75db1ba8>, filename=Green Eggs and Ham.mp3, chunkSize=261120, length=2401743, contentType=audio/mpeg [2018-04-24 17:20:27,468 INFO app.py:201 get] lengthInMB=2.29 MB [2018-04-24 17:20:27,469 INFO app.py:209 get] outputFilename=Green Eggs and Ham.mp3 [root@naturling-general-01 logs]# cat supervisord-robotDemo-stdout.log currentRootPath=/root/naturling_20180101/web/server/robotDemo flaskHost=0.0.0.0, flaskPort=32851 。。。 currentRootPath=/root/naturling_20180101/web/server/robotDemo flaskHost=0.0.0.0, flaskPort=32851 [root@naturling-general-01 logs]# cat supervisord-robotDemo-stderr.log -------------------------------------------------------------------------------- DEBUG in app [/root/naturling_20180101/web/server/robotDemo/app.py:42]: app=<Flask 'app'> ... </code>
【总结】
此处部署supervisor+gunicorn+flask到CentOS服务器上的步骤是:
1.先把本地的pipenv虚拟环境复制到服务器上
2.再去利用PyCharm去把本地Flask代码同步拷贝到服务器上
3.安装并使用gunicorn去启动Flask的app,试试是否正常
4.安装并使用supervisor去管理gunicorn
具体步骤,详见前面具体过程。