折腾:
【规避解决】Flask中开了多个worker时如何使用全局变量
期间,再去试试:
用Pickle+redis的方式保存对象
结果,用代码:
gLog.debug(“gWsDict=%s”, gWsDict) pickledGWsDict = pickle.dumps(gWsDict) gLog.debug(“pickledGWsDict=%s”, pickledGWsDict) savePickledGWsDictResult = redis_store.set(‘gWsDict’, pickledGWsDict) gLog.debug(“savePickledGWsDictResult=%s”, savePickledGWsDictResult) |
无法用redis+pickle去保存对象:
DEBUG in Websocket [/root/RunningFast/staging/runningfast/resources/Websocket.py:356]: added new ws for user user-cc680b0a-8d04-4f2b-8ad9-c6fefb527861 , gWsDict={u’user-cc680b0a-8d04-4f2b-8ad9-c6fefb527861′: <geventwebsocket.websocket.WebSocket object at 0x7f9934f53de0>} <div–<—————————————————————————— <div–<—————————————————————————— DEBUG in Websocket [/root/RunningFast/staging/runningfast/resources/Websocket.py:358]: gWsDict={u’user-cc680b0a-8d04-4f2b-8ad9-c6fefb527861′: <geventwebsocket.websocket.WebSocket object at 0x7f9934f53de0>} <div–<—————————————————————————— Traceback (most recent call last): File “/root/Envs/RunningFast/lib/python2.7/site-packages/gevent/pywsgi.py”, line 884, in handle_one_response self.run_application() File “/root/Envs/RunningFast/lib/python2.7/site-packages/geventwebsocket/handler.py”, line 76, in run_application self.run_websocket() File “/root/Envs/RunningFast/lib/python2.7/site-packages/geventwebsocket/handler.py”, line 52, in run_websocket self.application(self.environ, lambda s, h, e=None: []) File “/root/Envs/RunningFast/lib/python2.7/site-packages/flask/app.py”, line 2000, in __call__ return self.wsgi_app(environ, start_response) File “/root/Envs/RunningFast/lib/python2.7/site-packages/flask_sockets.py”, line 45, in __call__ handler(environment, **values) File “/root/RunningFast/staging/runningfast/resources/Websocket.py”, line 359, in userWebsocket pickledGWsDict = pickle.dumps(gWsDict) File “/usr/local/lib/python2.7/pickle.py”, line 1380, in dumps Pickler(file, protocol).dump(obj) File “/usr/local/lib/python2.7/pickle.py”, line 224, in dump self.save(obj) File “/usr/local/lib/python2.7/pickle.py”, line 286, in save f(self, obj) # Call unbound method with explicit self File “/usr/local/lib/python2.7/pickle.py”, line 655, in save_dict self._batch_setitems(obj.iteritems()) File “/usr/local/lib/python2.7/pickle.py”, line 669, in _batch_setitems save(v) File “/usr/local/lib/python2.7/pickle.py”, line 306, in save rv = reduce(self.proto) File “/root/Envs/RunningFast/lib/python2.7/copy_reg.py”, line 77, in _reduce_ex raise TypeError(“a class that defines __slots__ without “ TypeError: a class that defines __slots__ without defining __getstate__ cannot be pickled |
看起来是此处的字典变量gWsDict保存了
geventwebsocket.websocket.WebSocket
的示例
但是
geventwebsocket.websocket.WebSocket
本身无法被pickle
因为是:
websocket TypeError __getstate__ cannot be pickled
geventwebsocket TypeError __getstate__ cannot be pickled
geventwebsocket global
flask global websocket object
flask-sockets active socket
Recently Active ‘flask-sockets’ Questions – Stack Overflow
python – Using gevent and Flask to implement websocket, how to achieve concurrency? – Stack Overflow
和我的问题类似:
ws可以被打开
但是别的地方,无法访问到。。。
无法用有效的办法,把当前active的ws保存起来,供别处调用。
暂时还是没有找到有效的办法,解决:
当别处需要通过ws给某个用户发送信息时
此时却无法获得对应用户的ws
因为:
无法直接有效的,把已打开的ws,加入到全局变量中
-》全局变量的话,此处由于gunicorn开了多个worker,导致不同woker中的全局变量是独立的
想要把全局变量保存到redis中,结果此处的ws的对象:
geventwebsocket.websocket.WebSocket
无法被pickle
搜:
geventwebsocket pickle
pickle geventwebsocket
geventwebsocket cannot be pickled
geventwebsocket __getstate__ cannot be pickled
pickle – Python Pickling Slots Error – Stack Overflow
python – Pickle all attributes except one – Stack Overflow
gevent WebSocket __getstate__ cannot be pickled
WebSocket __getstate__ cannot be pickled
WebSocket cannot be pickled
python – Sharing websockets object between tornado processes – Stack Overflow
也是遇到:
实时的,在线的连接,是无法被pickle序列化,保存到数据库的。
-》每个ws是被一个线程打开的
-》找到对应的线程,线程里面,或许就可以获得对应的ws了?
那人尝试了:
multiprocessing.Manager()
这个库,但是太复杂了。
-》可以考虑进程间通信,这种库,相对还是很多的。
其中redis是相对来说比较好的选择
-》那就去试试:
用redis实现进程间通信
handler exception: can’t pickle <type ‘thread.lock> · Issue #203 · kanaka/websockify
【总结】
注:
如果是一般的数据(可以(通过pickel等)序列化和反序列化的),可以通过其他方式去存储。比如用redis,memchached等等。
而此处的ws,即geventwebsocket.websocket.WebSocket对象,是个实时的对象,不能用序列化的办法去保存(到redis中),因为即使可以被序列化为字符串,但是反序列化后得到的对象,也不是原先的对象了(内部的ws也就失效了,无法用ws去send消息了)
(注:且此WebSocket本身就不支持,无法被pickle序列化为字符串)
而解决的思路是:
只能去想办法拿到,实时的ws,然后去处理
-》可以通过线程间的通信,把ws传递出去,进行处理
-》而线程间通信,有多种凡事,redis是其中一种用的比较多的。
-》所以最后去通过:
【已解决】Flask中的线程或进程间通信
而通过线程间通信,解决了如何获取实时的ws对象,然后继续去处理。
转载请注明:在路上 » 【无法解决】Flask中用pickle去保存Websocket对象出错:TypeError a class that defines __slots__ without defining __getstate__ cannot be pickled