折腾：
期间，在Flask的app中，写了对应的代码：
/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/runningfast/app.py
from runningfast.resources.BackgroundWorker import * |
/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/runningfast/resources/BackgroundWorker.py
############################################################ # Current Project ############################################################ from runningfast.app import gLog from runningfast.app import db from runningfast.app import rq ############################################################ # Flask-RQ2 related ############################################################ @rq.job def updateUserStatus(): global gOnlineUserIdList curDatetime = datetime.now() 。。。 return # schedule periodic job to update user status updateUserStatusJob = updateUserStatus.schedule( timedelta(seconds=BACKGROUND_UPDATE_USER_STATUS_INTERVAL_SECONDS), repeat=0) gLog.debug("updateUserStatusJob=%s", updateUserStatusJob) |
结果出错了：
对应的log是：
[2016-11-06 14:39:09 +0000] [21725] [INFO] Listening at: http://0.0.0.0:21085 (21725) [2016-11-06 14:39:09 +0000] [21725] [INFO] Using worker: flask_sockets.worker [2016-11-06 14:39:09 +0000] [21736] [INFO] Booting worker with pid: 21736 <div–<—————————————————————————— DEBUG in app [/root/RunningFast/staging/runningfast/app.py:90]: app=<Flask ‘runningfast.app’>, server_port=21085, api=<flask_restful.Api object at 0x7fb8b33aedd0>, redis_store=<flask_redis.FlaskRedis object at 0x7fb8b45a4090>, db=<SQLAlchemy engine=’mysql://runningfast:Jiandao123@localhost/runningfast_dev’>, server_mode=staging, server_type=develop, rq=<flask_rq2.app.RQ object at 0x7fb8b33ca090>, sockets=<flask_sockets.Sockets object at 0x7fb8b33ca210> <div–<—————————————————————————— [2016-11-06 14:39:09 +0000] [21736] [ERROR] Exception in worker process Traceback (most recent call last): File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/arbiter.py", line 557, in spawn_worker worker.init_process() File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 190, in init_process super(GeventWorker, self).init_process() File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/workers/base.py", line 126, in init_process self.load_wsgi() File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/workers/base.py", line 136, in load_wsgi self.wsgi = self.app.wsgi() File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi self.callable = self.load() File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load return self.load_wsgiapp() File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp return util.import_app(self.app_uri) File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app __import__(module) File "/root/Envs/RunningFast/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__ result = _import(*args, **kwargs) File "/root/RunningFast/staging/run.py", line 4, in <module> from runningfast.app import app File "/root/Envs/RunningFast/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__ result = _import(*args, **kwargs) File "/root/RunningFast/staging/runningfast/app.py", line 102, in <module> from runningfast.resources.BackgroundWorker import * File "/root/Envs/RunningFast/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__ result = _import(*args, **kwargs) File "/root/RunningFast/staging/runningfast/resources/BackgroundWorker.py", line 81, in <module> timedelta(seconds=BACKGROUND_UPDATE_USER_STATUS_INTERVAL_SECONDS), repeat=0) File "/root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/helpers.py", line 81, in schedule return self.rq.get_scheduler().schedule( File "/root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/app.py", line 294, in get_scheduler connection=self.connection, File "/root/Envs/RunningFast/lib/python2.7/site-packages/rq_scheduler/scheduler.py", line 25, in __init__ self.connection = resolve_connection(connection) File "/root/Envs/RunningFast/lib/python2.7/site-packages/rq/connections.py", line 70, in resolve_connection raise NoRedisConnectionException(‘Could not resolve a Redis connection’) NoRedisConnectionException: Could not resolve a Redis connection |
很明显，是缺少了后台的redis
注：
此处，没有配置redis的配置
-》因为之前另外一个Flask的app，也没有添加redis的配置，都是用的默认的，是可以正常工作的。
所以这里是去模仿之前的操作，去再次利用Falsk-RQ2去实现后台任务。
然后去参考自己之前的：
［已解决］为Flask的后台任务而在CentOS中安装Redis
先去确认的确还存在对应的redis-server和配置文件：
(RunningFast) ➜ staging which redis-server /usr/local/bin/redis-server (RunningFast) ➜ staging ls /etc/redis/redis.conf /etc/redis/redis.conf |
去运行redis的sever：
(RunningFast) ➜ staging nohup redis-server /etc/redis/redis.conf >> logs/redis.log 2>&1 & [1] 21944 (RunningFast) ➜ staging [1] + 21944 exit 1 nohup redis-server /etc/redis/redis.conf >> logs/redis.log 2>&1 |
同时，参考：
再去运行对应的：
rqscheduler
和：
worker
(RunningFast) ➜ staging which rqscheduler /root/Envs/RunningFast/bin/rqscheduler (RunningFast) ➜ staging which rq /root/Envs/RunningFast/bin/rq (RunningFast) ➜ staging nohup rqscheduler -i 4 > logs/rqscheduler.log 2>&1 & [1] 22176 (RunningFast) ➜ staging [1] + 22176 exit 1 nohup rqscheduler -i 4 > logs/rqscheduler.log 2>&1 (RunningFast) ➜ staging nohup rq worker > logs/rq_worker.log 2>&1 & [1] 22190 |
然后可以看到对应的logs：
(RunningFast) ➜ staging ll logs total 728K -rw-r–r– 1 root root 479K Nov 6 14:39 gunicorn.err -rw-r–r– 1 root root 0 Nov 5 16:37 gunicorn.log -rw-r–r– 1 root root 134 Nov 6 14:54 redis.log -rw-r–r– 1 root root 689 Nov 6 15:06 rqscheduler.log -rw-r–r– 1 root root 200 Nov 6 15:06 rq_worker.log -rw-r–r– 1 root root 235K Nov 6 14:39 RunningFast.log |
再去运行，结果出错：
(RunningFast) ➜ staging supervisorctl -c supervisor.conf restart runningfast_staging runningfast_staging: ERROR (not running) runningfast_staging: started |
去看log
看到：
/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/logs/redis.log
nohup: ignoring input 21944:M 06 Nov 14:54:46.246 # Creating Server TCP listening socket 127.0.0.1:6379: bind: Address already in use |
/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/logs/rq_worker.log
nohup: ignoring input 15:06:49 RQ worker u’rq:worker:AY140128113754462e2eZ.22190′ started, version 0.6.0 15:06:49 Cleaning registries for queue: default 15:06:49 15:06:49 *** Listening on default… |
/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/logs/rqscheduler.log
nohup: ignoring input 15:06:43 Running RQ scheduler… Traceback (most recent call last): File "/root/Envs/RunningFast/bin/rqscheduler", line 11, in <module> sys.exit(main()) File "/root/Envs/RunningFast/lib/python2.7/site-packages/rq_scheduler/scripts/rqscheduler.py", line 53, in main scheduler.run(burst=args.burst) File "/root/Envs/RunningFast/lib/python2.7/site-packages/rq_scheduler/scheduler.py", line 336, in run self.register_birth() File "/root/Envs/RunningFast/lib/python2.7/site-packages/rq_scheduler/scheduler.py", line 33, in register_birth raise ValueError("There’s already an active RQ scheduler") ValueError: There’s already an active RQ scheduler |
但是错误依旧：
去看后台的redis：
(RunningFast) ➜ staging netstat -tupln Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:8998 0.0.0.0:* LISTEN 32463/java tcp 0 0 0.0.0.0:10022 0.0.0.0:* LISTEN 17286/docker-proxy tcp 0 0 0.0.0.0:8999 0.0.0.0:* LISTEN 32463/java tcp 0 0 0.0.0.0:9000 0.0.0.0:* LISTEN 19706/docker-proxy tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 17851/mysqld tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 19659/redis-server tcp 0 0 0.0.0.0:9003 0.0.0.0:* LISTEN 19669/docker-proxy tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 2222/nginx tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 24318/python tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1335/sshd tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN 17240/docker-proxy tcp 0 0 0.0.0.0:8088 0.0.0.0:* LISTEN 29962/docker-proxy tcp 0 0 0.0.0.0:5432 0.0.0.0:* LISTEN 18421/docker-proxy tcp 0 0 0.0.0.0:8090 0.0.0.0:* LISTEN 6675/java tcp 0 0 0.0.0.0:8091 0.0.0.0:* LISTEN 6675/java tcp 0 0 0.0.0.0:21084 0.0.0.0:* LISTEN 7846/python tcp 0 0 0.0.0.0:8093 0.0.0.0:* LISTEN 28556/java tcp 0 0 0.0.0.0:8094 0.0.0.0:* LISTEN 28556/java tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 19505/docker-proxy udp 0 0 192.168.42.1:123 0.0.0.0:* 1346/ntpd udp 0 0 115.29.173.126:123 0.0.0.0:* 1346/ntpd udp 0 0 10.161.170.247:123 0.0.0.0:* 1346/ntpd udp 0 0 127.0.0.1:123 0.0.0.0:* 1346/ntpd udp 0 0 0.0.0.0:123 0.0.0.0:* 1346/ntpd |
是有的：
tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 19659/redis-server
搜：
python rq NoRedisConnectionException Could not resolve a Redis connection
Could not resolve a Redis connection · Issue #164 · nvie/rq
get_current_job() throws NoRedisConnectionException in 0.5.0 · Issue #479 · nvie/rq
rq/connections.py at master · nvie/rq
redis – How to get Job by id in RQ python? – Stack Overflow
难道是：
当前的服务器的另外一个Flask的app已经连接了已有的redis的server
所以另外的flask的app，无法连接redis了？
NoRedisConnectionException Could not resolve a Redis connection
此处确认redis是正常工作的：
(RunningFast) ➜ staging redis-cli 127.0.0.1:6379> ping PONG 127.0.0.1:6379> quit |
-》
估计是：
rq worker
或
rqscheduler
没有正常工作？
当前好像是有rqworker工作的：
(RunningFast) ➜ staging ps aux | grep rq root 3 0.0 0.0 0 0 ? S Jun05 0:59 [ksoftirqd/0] root 43 0.0 0.0 0 0 ? S Jun05 0:33 [ksoftirqd/1] root 22190 0.0 0.3 201516 12540 pts/2 SN 15:06 0:00 /root/Envs/RunningFast/bin/python /root/Envs/RunningFast/bin/rq worker root 22472 0.0 0.0 103368 820 pts/2 S+ 15:28 0:00 grep –color=auto –exclude-dir=.bzr –exclude-dir=CVS –exclude-dir=.git –exclude-dir=.hg –exclude-dir=.svn rq root 26804 0.0 0.3 194612 12264 ? SN Sep04 2:35 /root/Envs/SIPEvents/bin/python /root/Envs/SIPEvents/bin/rqscheduler -i 30 root 26813 0.0 0.3 199684 12796 ? SN Sep04 0:08 /root/Envs/SIPEvents/bin/python /root/Envs/SIPEvents/bin/rq worker |
不过好像rqscheduler只有一个。
再去看rqscheduler的log：
There’s already an active RQ scheduler
-》
估计同时只能运行一个rqshecduler
Flask RQ2 NoRedisConnectionException
此处看了看安装的库，好像也是对的：
Flask-Redis==0.3.0 Flask-RQ2==16.1.1 redis==2.10.5 rq==0.6.0 rq-scheduler==0.7.0 |
NoRedisConnectionException on starting app · Issue #2 · jezdez/Flask-RQ2
貌似和我类似的问题？
jezdez/Flask-RQ2: A Flask extension for RQ.
“
- worker — Starts an RQ worker (required to run jobs).
- scheduler — Starts an RQ Scheduler (optional for scheduled jobs).
”
-》
才明白：
worker是针对于（无需调度，单次执行的）job的
scheduler：针对于需要调度，在不同时间，运行（多次）的任务的
搜：
Creating Server TCP listening socket 127.0.0.1:6379: bind: Address already in use
redis Creating Server TCP listening socket 127.0.0.1:6379: bind: Address already in use
redis-server in ubuntu14.04: Bind address already in use – Stack Overflow
去杀掉redis，重启
(RunningFast) ➜ staging netstat -tupln | grep redis tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 19659/redis-server (RunningFast) ➜ staging kill -9 19659 (RunningFast) ➜ staging [1] + 22190 exit 1 nohup rq worker > logs/rq_worker.log 2>&1 (RunningFast) ➜ staging netstat -tupln | grep redis (RunningFast) ➜ staging nohup redis-server /etc/redis/redis.conf >> logs/redis.log 2>&1 & [1] 22761 |
去看log：
/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/logs/redis.log
nohup: ignoring input 21944:M 06 Nov 14:54:46.246 # Creating Server TCP listening socket 127.0.0.1:6379: bind: Address already in use nohup: ignoring input _._ _.-“__ ”-._ _.-“ `. `_. ”-._ Redis 3.2.3 (00000000/0) 64 bit .-“ .-“`. “`\/ _.,_ ”-._ ( ‘ , .-` | `, ) Running in standalone mode |`-._`-…-` __…-.“-._|’` _.-‘| Port: 6379 | `-._ `._ / _.-‘ | PID: 22761 `-._ `-._ `-./ _.-‘ _.-‘ |`-._`-._ `-.__.-‘ _.-‘_.-‘| `-._ `-._`-.__.-‘_.-‘ _.-‘ |`-._`-._ `-.__.-‘ _.-‘_.-‘| | `-._`-._ _.-‘_.-‘ | `-._ `-._`-.__.-‘_.-‘ _.-‘ `-._ `-.__.-‘ _.-‘ `-._ _.-‘ `-.__.-‘ 22761:M 06 Nov 15:50:25.575 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 22761:M 06 Nov 15:50:25.575 # Server started, Redis version 3.2.3 22761:M 06 Nov 15:50:25.575 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add ‘vm.overcommit_memory = 1’ to /etc/sysctl.conf and then reboot or run the command ‘sysctl vm.overcommit_memory=1’ for this to take effect. 22761:M 06 Nov 15:50:25.575 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command ‘echo never > /sys/kernel/mm/transparent_hugepage/enabled’ as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled. 22761:M 06 Nov 15:50:25.575 * The server is now ready to accept connections on port 6379 |
然后再去运行程序看看。
错误依旧
NoRedisConnectionException: Could not resolve a Redis connection
再去重启：
(RunningFast) ➜ staging nohup rqscheduler -i 4 > logs/rqscheduler.log 2>&1 & [2] 22839 (RunningFast) ➜ staging nohup rq worker > logs/rq_worker.log 2>&1 & [3] 22850 |
看log，好像rq worker和rqscheduler没问题：
/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/logs/rq_worker.log
nohup: ignoring input 15:52:47 RQ worker u’rq:worker:AY140128113754462e2eZ.22850′ started, version 0.6.0 15:52:47 Cleaning registries for queue: default 15:52:47 15:52:47 *** Listening on default… |
/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/logs/rqscheduler.log
nohup: ignoring input 15:52:40 Running RQ scheduler… 15:52:40 Checking for scheduled jobs… 15:52:44 Checking for scheduled jobs… 15:52:48 Checking for scheduled jobs… 15:52:52 Checking for scheduled jobs… 15:52:56 Checking for scheduled jobs… |
参考了之前，可以工作的时候的库的版本，发现是：
Flask-Redis==0.3.0 Flask-RQ2==16.0.2 redis==2.10.5 rq==0.6.0 rq-scheduler==0.7.0 |
唯一不一样的只是：
之前工作的是：
Flask-RQ2==16.0.2
现在不工作的是：
Flask-RQ2==16.1.1
要不去把Flask-RQ2降版本到16.0.2
看看效果？
实在不行，再去试试这个办法。
搜：
self.rq.get_scheduler().schedule
参考：
Flask-RQ2/helpers.py at master · jezdez/Flask-RQ2
def schedule(self, time_or_delta, *args, **kwargs): """ A function to schedule running a RQ job at a given time or after a given timespan:: @rq.job def add(x, y): return x + y add.schedule(timedelta(hours=2), 1, 2) add.schedule(datetime(2016, 12, 31, 23, 59, 59), 1, 2) add.schedule(timedelta(days=14), 1, 2, repeat=1) """ repeat = kwargs.pop(‘repeat’, None) interval = kwargs.pop(‘interval’, None) description = kwargs.pop(‘description’, None) job_id = kwargs.pop(‘job_id’, None) if isinstance(time_or_delta, timedelta): time = datetime.utcnow() + time_or_delta else: time = time_or_delta return self.rq.get_scheduler().schedule( time, self.wrapped, args=args, kwargs=kwargs, interval=interval, repeat=repeat, result_ttl=self.result_ttl, ttl=self.ttl, timeout=self.timeout, id=job_id, description=description, queue_name=self.queue_name, ) |
->
https://github.com/jezdez/Flask-RQ2/blob/master/src/flask_rq2/app.py
def get_scheduler(self, interval=None): """ When installed returns a “rq_scheduler.Scheduler“ instance to schedule job execution, e.g.:: scheduler = rq.get_scheduler(interval=10) :param interval: Time in seconds of the periodic check for scheduled jobs. :type interval: int """ if self.scheduler_cls is None: raise RuntimeError(‘Cannot import rq-scheduler. Is it installed?’) if interval is None: interval = self.scheduler_interval return self.scheduler_cls( queue_name=self.scheduler_queue, interval=interval, connection=self.connection, ) |
->
ui/rq-scheduler: A light library that adds job scheduling capabilities to RQ (Redis Queue)
-》
rq-scheduler/scheduler.py at master · ui/rq-scheduler
class Scheduler(object): scheduler_key = ‘rq:scheduler’ scheduled_jobs_key = ‘rq:scheduler:scheduled_jobs’ def __init__(self, queue_name=’default’, interval=60, connection=None): from rq.connections import resolve_connection self.connection = resolve_connection(connection) self.queue_name = queue_name self._interval = interval self.log = logger |
-》
nvie/rq: Simple job queues for Python
-》
rq/connections.py at master · nvie/rq
def push_connection(redis): """Pushes the given connection on the stack.""" _connection_stack.push(patch_connection(redis)) def pop_connection(): """Pops the topmost connection from the stack.""" return _connection_stack.pop() def use_connection(redis=None): """Clears the stack and uses the given connection. Protects against mixed use of use_connection() and stacked connection contexts. """ assert len(_connection_stack) <= 1, \ ‘You should not mix Connection contexts with use_connection()’ release_local(_connection_stack) if redis is None: redis = StrictRedis() push_connection(redis) def get_current_connection(): """Returns the current Redis connection (i.e. the topmost on the connection stack). """ return _connection_stack.top def resolve_connection(connection=None): """Convenience function to resolve the given or the current connection. Raises an exception if it cannot resolve a connection now. """ if connection is not None: return patch_connection(connection) connection = get_current_connection() if connection is None: raise NoRedisConnectionException(‘Could not resolve a Redis connection’) return connection |
然后这里也有类似问题：
get_current_job() throws NoRedisConnectionException in 0.5.0 · Issue #479 · nvie/rq
对于rq，之前0.4.6也出现过类似bug，不过在0.5.1就解决了。
不过，难道是我此处的：
两个Flask的app，同时连接同一个本地的redis
redis://localhost:6379/0
所以连不上？
redis multiple connection NoRedisConnectionException
->
use_connection，用于实现单连接，（虽然很简单，但是会污染全局作用域namespace）所以已经废弃，不推荐使用了。
“Be aware of the fact that use_connection pollutes the global namespace. It also implies that you can only ever use a single connection.”
此处，有两个连接，所以出错了，应该就是这个原因。
不过好像又不是：
其rq已经自己实现了？
push_connection
和
pop_connection
了？
不是，上面是rq的自己的内部实现。
不是别人调用这两个函数。
Creating and managing multiple connections in Redis Python – Stack Overflow
提到了：
redis的另外的database，1（默认时0）
-》去搞清楚，如何创建redis的database 1
Redis Clients Handling – Redis
flask redis database 1
flask redis db
flask second redis db
Creating and managing multiple connections in Redis Python – Stack Overflow
算了，去参考：
jezdez/Flask-RQ2: A Flask extension for RQ.
“RQ_REDIS_URL
The URL to pass to redis-py’s Redis.from_url classmethod to create a Redis connetion pool. Defaults to connecting to the local Redis instance, database 0.
app.config[‘RQ_REDIS_URL’] = ‘redis://localhost:6379/0’”
去配置为1：
结果无论是：
/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/config.py
############################################################ # Flask-RQ2 ############################################################ RQ_REDIS_URL = ‘redis://localhost:6379/1’ |
还是：
/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/runningfast/app.py
app.config[‘RQ_REDIS_URL’] = ‘redis://localhost:6379/1’ app.config[‘RQ_QUEUES’] = [‘default’] app.config[‘RQ_ASYNC’] = True app.config[‘RQ_SCHEDULER_QUEUE’] = ‘default’ app.config[‘RQ_SCHEDULER_INTERVAL’] = 3 |
都是错误依旧。
去降级：
Flask-RQ2降版本到16.0.2
试试
(RunningFast) ➜ staging pip uninstall flask-rq2 Uninstalling Flask-RQ2-16.1.1: /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/DESCRIPTION.rst /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/INSTALLER /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/METADATA /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/RECORD /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/WHEEL /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/metadata.json /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/top_level.txt /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/__init__.py /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/__init__.pyc /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/app.py /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/app.pyc /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/cli.py /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/cli.pyc /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/helpers.py /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/helpers.pyc /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/script.py /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/script.pyc Proceed (y/n)? y Successfully uninstalled Flask-RQ2-16.1.1 You are using pip version 8.1.2, however version 9.0.0 is available. You should consider upgrading via the ‘pip install –upgrade pip’ command. (RunningFast) ➜ staging pip install flask-rq2==16.0.2 Collecting flask-rq2==16.0.2 Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by ‘NewConnectionError(‘<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f7fcf5047d0>: Failed to establish a new connection: [Errno 97] Address family not supported by protocol’,)’: /simple/flask-rq2/ Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by ‘ReadTimeoutError("HTTPSConnectionPool(host=’pypi.python.org’, port=443): Read timed out. (read timeout=15)",)’: /simple/flask-rq2/ Using cached Flask_RQ2-16.0.2-py2.py3-none-any.whl Requirement already satisfied (use –upgrade to upgrade): rq-scheduler>=0.6.1 in /root/Envs/RunningFast/lib/python2.7/site-packages (from flask-rq2==16.0.2) Requirement already satisfied (use –upgrade to upgrade): Flask>=0.10 in /root/Envs/RunningFast/lib/python2.7/site-packages (from flask-rq2==16.0.2) Requirement already satisfied (use –upgrade to upgrade): rq>=0.6.0 in /root/Envs/RunningFast/lib/python2.7/site-packages (from flask-rq2==16.0.2) Requirement already satisfied (use –upgrade to upgrade): croniter>=0.3.9 in /root/Envs/RunningFast/lib/python2.7/site-packages (from rq-scheduler>=0.6.1->flask-rq2==16.0.2) Requirement already satisfied (use –upgrade to upgrade): itsdangerous>=0.21 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2==16.0.2) Requirement already satisfied (use –upgrade to upgrade): Jinja2>=2.4 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2==16.0.2) Requirement already satisfied (use –upgrade to upgrade): Werkzeug>=0.7 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2==16.0.2) Requirement already satisfied (use –upgrade to upgrade): click>=2.0 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2==16.0.2) Requirement already satisfied (use –upgrade to upgrade): redis>=2.7.0 in /root/Envs/RunningFast/lib/python2.7/site-packages (from rq>=0.6.0->flask-rq2==16.0.2) Requirement already satisfied (use –upgrade to upgrade): python-dateutil in /root/Envs/RunningFast/lib/python2.7/site-packages (from croniter>=0.3.9->rq-scheduler>=0.6.1->flask-rq2==16.0.2) Requirement already satisfied (use –upgrade to upgrade): setuptools in /root/Envs/RunningFast/lib/python2.7/site-packages (from croniter>=0.3.9->rq-scheduler>=0.6.1->flask-rq2==16.0.2) Requirement already satisfied (use –upgrade to upgrade): MarkupSafe in /root/Envs/RunningFast/lib/python2.7/site-packages (from Jinja2>=2.4->Flask>=0.10->flask-rq2==16.0.2) Requirement already satisfied (use –upgrade to upgrade): six>=1.5 in /root/Envs/RunningFast/lib/python2.7/site-packages (from python-dateutil->croniter>=0.3.9->rq-scheduler>=0.6.1->flask-rq2==16.0.2) Installing collected packages: flask-rq2 Successfully installed flask-rq2-16.0.2 You are using pip version 8.1.2, however version 9.0.0 is available. You should consider upgrading via the ‘pip install –upgrade pip’ command. |
然后再去试试。
结果问题依旧。
再换回最新的Flask-RQ2：
(RunningFast) ➜ staging pip uninstall flask-rq2 Uninstalling Flask-RQ2-16.0.2: /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/DESCRIPTION.rst /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/INSTALLER /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/METADATA /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/RECORD /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/WHEEL /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/metadata.json /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/top_level.txt /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/__init__.py /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/__init__.pyc /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/app.py /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/app.pyc /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/cli.py /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/cli.pyc /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/helpers.py /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/helpers.pyc /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/script.py /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/script.pyc Proceed (y/n)? y Successfully uninstalled Flask-RQ2-16.0.2 You are using pip version 8.1.2, however version 9.0.0 is available. You should consider upgrading via the ‘pip install –upgrade pip’ command. (RunningFast) ➜ staging pip install flask-rq2 Collecting flask-rq2 Using cached Flask_RQ2-16.1.1-py2.py3-none-any.whl Requirement already satisfied (use –upgrade to upgrade): rq-scheduler>=0.6.1 in /root/Envs/RunningFast/lib/python2.7/site-packages (from flask-rq2) Requirement already satisfied (use –upgrade to upgrade): Flask>=0.10 in /root/Envs/RunningFast/lib/python2.7/site-packages (from flask-rq2) Requirement already satisfied (use –upgrade to upgrade): rq>=0.6.0 in /root/Envs/RunningFast/lib/python2.7/site-packages (from flask-rq2) Requirement already satisfied (use –upgrade to upgrade): croniter>=0.3.9 in /root/Envs/RunningFast/lib/python2.7/site-packages (from rq-scheduler>=0.6.1->flask-rq2) Requirement already satisfied (use –upgrade to upgrade): itsdangerous>=0.21 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2) Requirement already satisfied (use –upgrade to upgrade): Jinja2>=2.4 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2) Requirement already satisfied (use –upgrade to upgrade): Werkzeug>=0.7 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2) Requirement already satisfied (use –upgrade to upgrade): click>=2.0 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2) Requirement already satisfied (use –upgrade to upgrade): redis>=2.7.0 in /root/Envs/RunningFast/lib/python2.7/site-packages (from rq>=0.6.0->flask-rq2) Requirement already satisfied (use –upgrade to upgrade): python-dateutil in /root/Envs/RunningFast/lib/python2.7/site-packages (from croniter>=0.3.9->rq-scheduler>=0.6.1->flask-rq2) Requirement already satisfied (use –upgrade to upgrade): setuptools in /root/Envs/RunningFast/lib/python2.7/site-packages (from croniter>=0.3.9->rq-scheduler>=0.6.1->flask-rq2) Requirement already satisfied (use –upgrade to upgrade): MarkupSafe in /root/Envs/RunningFast/lib/python2.7/site-packages (from Jinja2>=2.4->Flask>=0.10->flask-rq2) Requirement already satisfied (use –upgrade to upgrade): six>=1.5 in /root/Envs/RunningFast/lib/python2.7/site-packages (from python-dateutil->croniter>=0.3.9->rq-scheduler>=0.6.1->flask-rq2) Installing collected packages: flask-rq2 Successfully installed flask-rq2-16.1.1 You are using pip version 8.1.2, however version 9.0.0 is available. You should consider upgrading via the ‘pip install –upgrade pip’ command. |
看了半天，感觉：
get_current_job() throws NoRedisConnectionException in 0.5.0 · Issue #479 · nvie/rq
就是个bug，到目前的最新的0.6.0，还是没解决。。。
那人给出的解决办法：
diff –git a/rq/cli/cli.py b/rq/cli/cli.py index 6b52bd7..97c5e4d 100755 <div–<- a/rq/cli/cli.py +++ b/rq/cli/cli.py @@ -13,6 +13,7 @@ from redis import StrictRedis from redis.exceptions import ConnectionError from rq import Connection, get_failed_queue, Queue +from rq.connections import push_connection from rq.contrib.legacy import cleanup_ghosts from rq.exceptions import InvalidJobOperationError from rq.utils import import_attribute @@ -171,6 +172,7 @@ def worker(url, config, burst, name, worker_class, job_class, queue_class, path, setup_loghandlers_from_args(verbose, quiet) conn = connect(url, config) + push_connection(conn) cleanup_ghosts(conn) worker_class = import_attribute(worker_class) queue_class = import_attribute(queue_class) |
目前0.6.0的代码，还是没有修复：
def worker(url, config, burst, name, worker_class, job_class, queue_class, connection_class, 。。。 setup_loghandlers_from_args(verbose, quiet) connection_class = import_attribute(connection_class) conn = connect(url, config, connection_class) cleanup_ghosts(conn) worker_class = import_attribute(worker_class) queue_class = import_attribute(queue_class) |
去看看自己此处的代码：
同样的：
(RunningFast) ➜ staging cat /root/Envs/RunningFast/lib/python2.7/site-packages/rq/cli/cli.py def worker(url, config, burst, name, worker_class, job_class, queue_class, path, results_ttl, 。。。 setup_loghandlers_from_args(verbose, quiet) conn = connect(url, config) cleanup_ghosts(conn) worker_class = import_attribute(worker_class) queue_class = import_attribute(queue_class) exception_handlers = [] |
去手动加上试试：
vim /root/Envs/RunningFast/lib/python2.7/site-packages/rq/cli/cli.py
加上：
from rq.connections import push_connection
和：
push_connection(conn)
然后重新启动那个rq worker：
(RunningFast) ➜ staging nohup rq worker > logs/rq_worker.log 2>&1 & [2] 24162 |
log中也是对应的24162:
nohup: ignoring input 17:18:32 RQ worker u’rq:worker:AY140128113754462e2eZ.24162′ started, version 0.6.0 17:18:32 Cleaning registries for queue: default 17:18:32 17:18:32 *** Listening on default… |
然后重启Flask的app试试。
问题依旧。
重启redis，rq worker，rqscheduler，结果还是不行：
(RunningFast) ➜ staging ps aux | grep rq root 3 0.0 0.0 0 0 ? S Jun05 0:59 [ksoftirqd/0] root 43 0.0 0.0 0 0 ? S Jun05 0:33 [ksoftirqd/1] root 22839 0.0 0.3 196428 12136 pts/2 SN 15:52 0:01 /root/Envs/RunningFast/bin/python /root/Envs/RunningFast/bin/rqscheduler -i 4 root 22850 0.0 0.3 201516 12536 pts/2 SN 15:52 0:00 /root/Envs/RunningFast/bin/python /root/Envs/RunningFast/bin/rq worker root 24147 0.0 0.3 201772 12556 pts/5 SN 17:17 0:00 /root/Envs/RunningFast/bin/python /root/Envs/RunningFast/bin/rq worker root 24162 0.0 0.3 201516 12540 pts/5 SN 17:18 0:00 /root/Envs/RunningFast/bin/python /root/Envs/RunningFast/bin/rq worker root 24303 0.0 0.0 103368 816 pts/5 S+ 17:24 0:00 grep –color=auto –exclude-dir=.bzr –exclude-dir=CVS –exclude-dir=.git –exclude-dir=.hg –exclude-dir=.svn rq (RunningFast) ➜ staging netstat -tupln Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:8998 0.0.0.0:* LISTEN 32463/java tcp 0 0 0.0.0.0:10022 0.0.0.0:* LISTEN 17286/docker-proxy tcp 0 0 0.0.0.0:8999 0.0.0.0:* LISTEN 32463/java tcp 0 0 0.0.0.0:9000 0.0.0.0:* LISTEN 19706/docker-proxy tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 17851/mysqld tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 22761/redis-server tcp 0 0 0.0.0.0:9003 0.0.0.0:* LISTEN 19669/docker-proxy tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 2222/nginx tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 24318/python tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1335/sshd tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN 17240/docker-proxy tcp 0 0 0.0.0.0:8088 0.0.0.0:* LISTEN 29962/docker-proxy tcp 0 0 0.0.0.0:5432 0.0.0.0:* LISTEN 18421/docker-proxy tcp 0 0 0.0.0.0:8090 0.0.0.0:* LISTEN 6675/java tcp 0 0 0.0.0.0:8091 0.0.0.0:* LISTEN 6675/java tcp 0 0 0.0.0.0:21084 0.0.0.0:* LISTEN 7846/python tcp 0 0 0.0.0.0:8093 0.0.0.0:* LISTEN 28556/java tcp 0 0 0.0.0.0:8094 0.0.0.0:* LISTEN 28556/java tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 19505/docker-proxy udp 0 0 192.168.42.1:123 0.0.0.0:* 1346/ntpd udp 0 0 115.29.173.126:123 0.0.0.0:* 1346/ntpd udp 0 0 10.161.170.247:123 0.0.0.0:* 1346/ntpd udp 0 0 127.0.0.1:123 0.0.0.0:* 1346/ntpd udp 0 0 0.0.0.0:123 0.0.0.0:* 1346/ntpd (RunningFast) ➜ staging kill -9 22761 (RunningFast) ➜ staging [2] + 24162 exit 1 nohup rq worker > logs/rq_worker.log 2>&1 (RunningFast) ➜ staging [1] + 24147 exit 1 nohup rq worker > logs/rq_worker.log 2>&1 (RunningFast) ➜ staging nohup redis-server /etc/redis/redis.conf >> logs/redis.log 2>&1 & [1] 24395 (RunningFast) ➜ staging nohup rqscheduler -i 4 > logs/rqscheduler.log 2>&1 & [2] 24403 (RunningFast) ➜ staging nohup rq worker > logs/rq_worker.log 2>&1 & [3] 24412 (RunningFast) ➜ staging supervisorctl -c supervisor.conf restart runningfast_staging runningfast_staging: ERROR (not running) runningfast_staging: started (RunningFast) ➜ staging supervisorctl -c supervisor.conf restart runningfast_staging runningfast_staging: ERROR (not running) runningfast_staging: started (RunningFast) ➜ staging supervisorctl -c supervisor.conf restart runningfast_staging runningfast_staging: ERROR (not running) runningfast_staging: started |
【总结】
此处，貌似是rq的bug，导致后续的别处调用rq的get_current_job却获取不到对应的redis的连接。
暂时是通过自己手动去添加：
push_connection
使得此处有了redis的连接，然后后续就可以获得到当前的连接了：
from redis import Redis from rq import Queue, Worker from rq.connections import push_connection conn = Redis.from_url(u’redis://localhost:6379/0′) queue = Queue(connection=conn) pushConnResult = push_connection(conn) gLog.debug("conn=%s, queue=%s, pushConnResult=%s", conn, queue, pushConnResult) |
至少暂时可以正常运行了：
输出了log：
conn=Redis<ConnectionPool<Connection<host=localhost,port=6379,db=0>>>, queue=<Queue u’default’>, pushConnResult=None |
此办法，很明显只是临时的办法，需要待官方有更新版本出来解决了此问题，再去更新版本。
转载请注明：在路上 » 【临时解决】Flask出错：NoRedisConnectionException Could not resolve a Redis connection