折腾：
【已解决】Flask的gunicorn中多进程多worker如何共享数据或单实例
期间，暂时不想去试试多线程的其他共享内存，mmap等复杂的方法。
而先去试试这个方法：gunicorn的gevent
即：gunicorn中，把之前多worker，多process线程，
改为单worker，worker的type
看看能否解决问题。
先去试试：gunicorn的gevent
How to get a concurrency of 1000 requests with Flask and Gunicorn – Stack Overflow
flask gunicorn gevent coroutine
django flask应用uwsgi和tornado异步回调在项目中的体验 | 峰云就她了
python – How many concurrent requests does a single Flask process receive? – Stack Overflow
Design — Gunicorn 19.9.0 documentation
Introduction — gevent 1.3.6.dev0 documentation
Gunicorn Workers and Threads – Stack Overflow
http://docs.gunicorn.org/en/stable/settings.html#worker-class
先去安装gevent：
<code>➜ xxx git:(master) ✗ pipenv install gevent Installing gevent... Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Collecting gevent Downloading https://pypi.tuna.tsinghua.edu.cn/packages/2b/a9/7c38605b9672a6ede6ccf822a645fdeec0c80fb467c87c5ce4976e4056dd/gevent-1.3.6-cp36-cp36m-macosx_10_6_intel.whl (3.1MB) Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" (from gevent) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/5d/82/2e53a8def6f99db51992ca3a0a2448c3bbec1a9db3a7cbf7d5dad011e138/greenlet-0.4.14.tar.gz (59kB) Building wheels for collected packages: greenlet Running setup.py bdist_wheel for greenlet: started Running setup.py bdist_wheel for greenlet: finished with status 'done' Stored in directory: /Users/crifan/Library/Caches/pipenv/wheels/3f/20/d5/59a7f616d60eb238802e9860811aa4a069307c8fba12fce6d6 Successfully built greenlet Installing collected packages: greenlet, gevent Successfully installed gevent-1.3.6 greenlet-0.4.14 Adding gevent to Pipfile's [packages]... Pipfile.lock not found, creating... Locking [dev-packages] dependencies... </code>
django flask应用uwsgi和tornado异步回调在项目中的体验 | 峰云就她了
截止到 2018 年 1 月，Python 的 asyncio 的生态如何？ – 知乎
所以现在是：
http://docs.gunicorn.org/en/stable/settings.html#worker-class
设置为gevent了，然后：
http://docs.gunicorn.org/en/stable/settings.html#workers
workers设置为1
而对于
threads
http://docs.gunicorn.org/en/stable/settings.html#threads
worker_connections
http://docs.gunicorn.org/en/stable/settings.html#worker-connections
max_requests
http://docs.gunicorn.org/en/stable/settings.html#max-requests
到底设置为多少，去参考：
Gunicorn Workers and Threads – Stack Overflow
目前结论：
threads：无需设置
官网说了：此选项只适用于Gthread类型的worker
worker_connections：此处随便设置，默认本身是1000
注：只适用于Eventlet和Gevent类型的worker
max_requests：默认是0，表示不限制，除非想要降低内存泄漏风险，否则可以不设置
但是CentOS中去用pipenv安装gevent失败：
<code>[root@xxx-general-01 robotDemo]# pipenv install Pipfile.lock (ab35a7) out of date, updating to (feef41)… Locking [dev-packages] dependencies… Locking [packages] dependencies… Updated Pipfile.lock (feef41)! Installing dependencies from Pipfile.lock (feef41)… Ignoring greenlet: markers 'platform_python_implementation == "cpython"' don't match your environment Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 62/62 — 00:00:13 To activate this project's virtualenv, run the following: $ pipenv shell </code>
不过通过graph发现已安装：
<code>[root@xxx-general-01 robotDemo]# pipenv graph celery==4.2.1 - billiard [required: >=3.5.0.2,<3.6.0, installed: 3.5.0.4] - kombu [required: >=4.2.0,<5.0, installed: 4.2.1] - amqp [required: >=2.1.4,<3.0, installed: 2.3.2] - vine [required: >=1.1.3, installed: 1.1.4] - pytz [required: >dev, installed: 2018.5] Flask-Cors==3.0.6 - Flask [required: >=0.9, installed: 1.0.2] - click [required: >=5.1, installed: 6.7] - itsdangerous [required: >=0.24, installed: 0.24] - Jinja2 [required: >=2.10, installed: 2.10] - MarkupSafe [required: >=0.23, installed: 1.0] - Werkzeug [required: >=0.14, installed: 0.14.1] - Six [required: Any, installed: 1.11.0] Flask-PyMongo==2.1.0 - Flask [required: >=0.11, installed: 1.0.2] - click [required: >=5.1, installed: 6.7] - itsdangerous [required: >=0.24, installed: 0.24] - Jinja2 [required: >=2.10, installed: 2.10] - MarkupSafe [required: >=0.23, installed: 1.0] - Werkzeug [required: >=0.14, installed: 0.14.1] - PyMongo [required: >=3.0, installed: 3.7.1] Flask-RESTful==0.3.6 - aniso8601 [required: >=0.82, installed: 3.0.2] - Flask [required: >=0.8, installed: 1.0.2] - click [required: >=5.1, installed: 6.7] - itsdangerous [required: >=0.24, installed: 0.24] - Jinja2 [required: >=2.10, installed: 2.10] - MarkupSafe [required: >=0.23, installed: 1.0] - Werkzeug [required: >=0.14, installed: 0.14.1] - pytz [required: Any, installed: 2018.5] - six [required: >=1.3.0, installed: 1.11.0] gensim==3.5.0 - numpy [required: >=1.11.3, installed: 1.15.1] - scipy [required: >=0.18.1, installed: 1.1.0] - numpy [required: >=1.8.2, installed: 1.15.1] - six [required: >=1.5.0, installed: 1.11.0] - smart-open [required: >=1.2.1, installed: 1.6.0] - boto [required: >=2.32, installed: 2.49.0] - boto3 [required: Any, installed: 1.8.3] - botocore [required: <1.12.0,>=1.11.3, installed: 1.11.3] - docutils [required: >=0.10, installed: 0.14] - jmespath [required: <1.0.0,>=0.7.1, installed: 0.9.3] - python-dateutil [required: >=2.1,<3.0.0, installed: 2.7.3] - six [required: >=1.5, installed: 1.11.0] - urllib3 [required: <1.24,>=1.20, installed: 1.23] - jmespath [required: <1.0.0,>=0.7.1, installed: 0.9.3] - s3transfer [required: <0.2.0,>=0.1.10, installed: 0.1.13] - botocore [required: >=1.3.0,<2.0.0, installed: 1.11.3] - docutils [required: >=0.10, installed: 0.14] - jmespath [required: <1.0.0,>=0.7.1, installed: 0.9.3] - python-dateutil [required: >=2.1,<3.0.0, installed: 2.7.3] - six [required: >=1.5, installed: 1.11.0] - urllib3 [required: <1.24,>=1.20, installed: 1.23] - bz2file [required: Any, installed: 0.98] - requests [required: Any, installed: 2.19.1] - certifi [required: >=2017.4.17, installed: 2018.8.24] - chardet [required: <3.1.0,>=3.0.2, installed: 3.0.4] - idna [required: <2.8,>=2.5, installed: 2.7] - urllib3 [required: <1.24,>=1.21.1, installed: 1.23] gevent==1.3.6 - greenlet [required: >=0.4.14, installed: ?] gunicorn==19.9.0 jieba==0.39 openpyxl==2.5.5 - et-xmlfile [required: Any, installed: 1.0.1] - jdcal [required: Any, installed: 1.4] patchwork==1.0.1 - fabric [required: >=2.0,<3.0, installed: 2.3.1] - cryptography [required: >=1.1, installed: 2.3.1] - asn1crypto [required: >=0.21.0, installed: 0.24.0] - cffi [required: >=1.7,!=1.11.3, installed: 1.11.5] - pycparser [required: Any, installed: 2.18] - idna [required: >=2.1, installed: 2.7] - six [required: >=1.4.1, installed: 1.11.0] - invoke [required: <2.0,>=1.1, installed: 1.1.1] - paramiko [required: >=2.4, installed: 2.4.1] - bcrypt [required: >=3.1.3, installed: 3.1.4] - cffi [required: >=1.1, installed: 1.11.5] - pycparser [required: Any, installed: 2.18] - six [required: >=1.4.1, installed: 1.11.0] - cryptography [required: >=1.5, installed: 2.3.1] - asn1crypto [required: >=0.21.0, installed: 0.24.0] - cffi [required: >=1.7,!=1.11.3, installed: 1.11.5] - pycparser [required: Any, installed: 2.18] - idna [required: >=2.1, installed: 2.7] - six [required: >=1.4.1, installed: 1.11.0] - pyasn1 [required: >=0.1.7, installed: 0.4.4] - pynacl [required: >=1.0.1, installed: 1.2.1] - cffi [required: >=1.4.1, installed: 1.11.5] - pycparser [required: Any, installed: 2.18] - six [required: Any, installed: 1.11.0] pycnnum==1.0.1 pylru==1.1.0 PyMySQL==0.9.2 - cryptography [required: Any, installed: 2.3.1] - asn1crypto [required: >=0.21.0, installed: 0.24.0] - cffi [required: >=1.7,!=1.11.3, installed: 1.11.5] - pycparser [required: Any, installed: 2.18] - idna [required: >=2.1, installed: 2.7] - six [required: >=1.4.1, installed: 1.11.0] python-dotenv==0.9.1 redis==2.10.6 schedule==0.5.0 SolrClient==0.2.1 - kazoo [required: ==2.2.1, installed: 2.2.1] - six [required: Any, installed: 1.11.0] - requests [required: >=2.2.1, installed: 2.19.1] - certifi [required: >=2017.4.17, installed: 2018.8.24] - chardet [required: <3.1.0,>=3.0.2, installed: 3.0.4] - idna [required: <2.8,>=2.5, installed: 2.7] - urllib3 [required: <1.24,>=1.21.1, installed: 1.23] tqdm==4.25.0 </code>
然后去看看效果
发现log提示缺少greenlet，所以去本地和在线都安装：
<code>[root@xxx-general-01 logs]# pipenv install greenlet Installing greenlet… Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Collecting greenlet Using cached https://pypi.tuna.tsinghua.edu.cn/packages/de/7b/cb662640540725deb0627264f6b890ee2b7725848b8cbca49e27bf3273c6/greenlet-0.4.14-cp36-cp36m-manylinux1_x86_64.whl Installing collected packages: greenlet Successfully installed greenlet-0.4.14 Adding greenlet to Pipfile's [packages]… Pipfile.lock (feef41) out of date, updating to (36cd02)… Locking [dev-packages] dependencies… Locking [packages] dependencies… Updated Pipfile.lock (36cd02)! Installing dependencies from Pipfile.lock (36cd02)… ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 62/62 — 00:00:12 To activate this project's virtualenv, run the following: $ pipenv shell </code>
安装后，Pipfile中就有了：
<code>gevent = "*" greenlet = "*" </code>
然后再去 运行试试
为了更好的调试，发现时不时多个不同线程调用单例的初始化的，所以再去给logging中加上process信息：
参考：
https://docs.python.org/2/library/logging.html#logrecord-attributes
加上：
process | %(process)d | Process ID (if available). |
processName | %(processName)s | Process name (if available). |
变成：
<code>LOG_FORMAT = "[%(asctime)s %(levelname)s %(process)d %(processName)s %(filename)s:%(lineno)d %(funcName)s] %(message)s" </code>
然后再去看看log
然后看到对应的log日志输出是：
logs/gunicorn_error.log
<code>[2018-08-30 13:28:35 +0800] [26048] [INFO] Starting gunicorn 19.9.0 [2018-08-30 13:28:35 +0800] [26048] [DEBUG] Arbiter booted [2018-08-30 13:28:35 +0800] [26048] [INFO] Listening at: http://0.0.0.0:32851 (26048) [2018-08-30 13:28:35 +0800] [26048] [INFO] Using worker: gevent [2018-08-30 13:28:35 +0800] [26063] [INFO] Booting worker with pid: 26063 [2018-08-30 13:28:35 +0800] [26048] [DEBUG] 1 workers </code>
是所希望的
单个worker
type是gevent
然后对应的所希望的单例：
gMsTtsTokenSingleton=
对于Flask涉及的gunicorn中woker中process，的确也是单个实例了。
【总结】
此处，是可以通过gunicorn部署Flask时，把
conf/gunicorn/gunicorn_config.py
<code>worker_class = 'sync' #默认的是sync模式 workers = multiprocessing.cpu_count() * 2 + 1 #进程数 </code>
改为：
<code># http://docs.gunicorn.org/en/stable/settings.html#worker-class worker_class = 'gevent' workers = 1 #进程数 # http://docs.gunicorn.org/en/stable/settings.html#max-requests worker_connections = 2000 # 默认是1000 # http://docs.gunicorn.org/en/stable/settings.html#max-requests max_requests = 0 </code>
即：
从woker=9，type=sync改为worker=1，type=gevent
从而使得：确保Flask中的app，从9个Process，变成单个Process了。
从而单个Process中的Flask的APP的单例，就可以正常工作了。
只不过又出现其他方面的问题：
【未解决】Flask中gunicorn部署和supervisor管理celery的worker导致多线程导致单例失效
转载请注明：在路上 » 【已解决】用gunicorn的gevent解决之前多worker多Process线程的单例的数据共享