折腾:
【已解决】Mac中安装Solr的server并启动Solr服务
期间,现有python的solr相关代码:
SOLR = SolrClient(‘http://localhost:8999/solr’)
…
docs = self.solr.fq(q_mapper=q_mapper, fq_mapper=fq_mapper, retrieve_keys=retrieve_keys, rows=1)
去运行,出错:
File "/Users/crifan/dev/dev_root/xxx/search/qa/iqa.py", line 163, in main
reply = qa.get_responses(i, "qa")
File "/Users/crifan/dev/dev_root/xxx/search/qa/iqa.py", line 80, in get_responses
docs = self.solr.fq(q_mapper=q_mapper, fq_mapper=fq_mapper, retrieve_keys=retrieve_keys, rows=1)
File "/Users/crifan/dev/dev_root/company//xxx/sourcecode/xxx/xxx/search/utils/solr_util.py", line 240, in fq
resp = self.solr.query(self.core, params)
File "/Users/crifan/.virtualenvs/xxx-gXiJ4vtz/lib/python3.6/site-packages/SolrClient/solrclient.py", line 125, in query
**kwargs)
File "/Users/crifan/.virtualenvs/xxx-gXiJ4vtz/lib/python3.6/site-packages/SolrClient/transport/transportbase.py", line 39, in inner
return function(self, host, **kwargs)
File "/Users/crifan/.virtualenvs/xxx-gXiJ4vtz/lib/python3.6/site-packages/SolrClient/transport/transportbase.py", line 57, in send_request
res_dict, c_inf = self._send(host, **kwargs)
File "/Users/crifan/.virtualenvs/xxx-gXiJ4vtz/lib/python3.6/site-packages/SolrClient/transport/transportrequests.py", line 66, in _send
raise ConnectionError("404 – {}".format(res.url))
SolrClient.exceptions.ConnectionError: 404 – http://localhost:8983/solr/qa/select?q=question_str%3A%22sing+a++song%22&fq=%2A%3A%2A+AND+scene%3Aqa&rows=1&fl=question%2Canswer%2Cid&wt=json&indent=false
看起来错误和之前的:
很类似。
搜:
python SolrClient.exceptions.ConnectionError: 404
SolrClient/transportrequests.py at master · moonlitesolutions/SolrClient
if res.status_code == 404:
raise ConnectionError("404 – {}".format(res.url))
感觉是:
虽然connection有了,但是此处http://localhost:8983/solr
的
/qa/select
的api端口找不到,所以404
突然想起来了:
dev服务器上可以正常运行
-》所以去dev服务器上看看后台的solr是如何运行的
[root@xxx-general-01 xxx]# solr status
-bash: solr: command not found
[root@xxx-general-01 xxx]# ps aux | grep 8983
dev 5424 0.0 3.8 5819128 633084 ? Sl Jul18 38:45 java -server -Xms2g -Xmx2g -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:-OmitStackTraceInFastThrow -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/home/dev/nlp/qa/data/solr-7.4.0/server/logs/solr_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M -Dsolr.log.dir=/home/dev/nlp/qa/data/solr-7.4.0/server/logs -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -Djetty.home=/home/dev/nlp/qa/data/solr-7.4.0/server -Dsolr.solr.home=/home/dev/nlp/qa/data/solr-7.4.0/server/solr -Dsolr.data.home= -Dsolr.install.dir=/home/dev/nlp/qa/data/solr-7.4.0 -Dsolr.default.confdir=/home/dev/nlp/qa/data/solr-7.4.0/server/solr/configsets/_default/conf -Xss256k -Dsolr.jetty.https.port=8983 -Dsolr.log.muteconsole -XX:OnOutOfMemoryError=/home/dev/nlp/qa/data/solr-7.4.0/bin/oom_solr.sh 8983 /home/dev/nlp/qa/data/solr-7.4.0/server/logs -jar start.jar –module=http
root 24989 0.0 0.0 112664 976 pts/0 S+ 14:53 0:00 grep –color=auto 8983
[root@xxx-general-01 xxx]# ps aux | grep solr
dev 5424 0.0 3.8 5819128 633084 ? Sl Jul18 38:45 java -server -Xms2g -Xmx2g -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:-OmitStackTraceInFastThrow -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/home/dev/nlp/qa/data/solr-7.4.0/server/logs/sol_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M -Dsolr.log.dir=/home/dev/nlp/qa/data/solr-7.4.0/server/logs -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -Djetty.home=/home/dev/nlp/qa/data/solr-7.4.0/server -Dsolr.solr.home=/home/dev/nlp/qa/data/solr-7.4.0/server/solr -Dsolr.data.home= -Dsolr.install.dir=/home/dev/nlp/qa/data/solr-7.4.0 -Dsolr.default.confdir=/home/dev/nlp/qa/data/solr-7.4.0/server/solr/configsets/_default/conf -Xss256k -Dsolr.jetty.https.port=8983 -Dsolr.log.muteconsole -XX:OnOutOfMemoryError=/home/dev/nlp/qa/data/solr-7.4.0/bin/oom_solr.sh 8983 /home/dev/nlp/qa/data/solr-7.4.0/server/logs -jar start.jar –module=http
root 24991 0.0 0.0 112664 976 pts/0 S+ 14:53 0:00 grep –color=auto solr
也的确能看到对应solr的路径:
[root@xx-xx-01 xxx]# ls /home/dev/nlp/qa/data/solr-7.4.0/server/
contexts/ .DS_Store etc/ lib/ logs/ modules/ README.txt resources/ scripts/ solr/ solr-webapp/ start.jar
所以现在可以处理为:
除非本地想要调试solr,否则本地可以不去关心如何搭建solr的服务了。
因为在线服务器中已经正常运行了solr的服务器了。
发现此处是需要解决本地搭建solr的问题。
此处对比一下 线上CentOS 和 本地Mac中的不同
线上CentOS
java -server -Xms2g -Xmx2g -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:-OmitStackTraceInFastThrow -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/home/dev/nlp/qa/data/solr-7.4.0/server/logs/sol_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M -Dsolr.log.dir=/home/dev/nlp/qa/data/solr-7.4.0/server/logs -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -Djetty.home=/home/dev/nlp/qa/data/solr-7.4.0/server -Dsolr.solr.home=/home/dev/nlp/qa/data/solr-7.4.0/server/solr -Dsolr.data.home= -Dsolr.install.dir=/home/dev/nlp/qa/data/solr-7.4.0 -Dsolr.default.confdir=/home/dev/nlp/qa/data/solr-7.4.0/server/solr/configsets/_default/conf -Xss256k -Dsolr.jetty.https.port=8983 -Dsolr.log.muteconsole -XX:OnOutOfMemoryError=/home/dev/nlp/qa/data/solr-7.4.0/bin/oom_solr.sh 8983 /home/dev/nlp/qa/data/solr-7.4.0/server/logs -jar start.jar –module=http
[root@xxx-general-01 xxx_20180101]# cd /home/dev
[root@xxx-general-01 dev]# ll
total 12
drwxrwxr-x 2 dev dev 4096 Jul 18 22:30 downloads
drwxrwxr-x 13 dev dev 4096 Jul 18 22:31 miniconda2
drwxrwxrwx 3 dev dev 4096 Jul 18 22:08 nlp
[root@xxx-general-01 dev]# cd nlp/
[root@xxx-general-01 nlp]# ll
total 4
drwxrwxrwx 4 dev dev 4096 Jul 18 22:08 qa
本地Mac
➜ ~ solr -V
Using Solr root directory: /usr/local/Cellar/solr/7.2.1
Using Java: java
java version "1.8.0_112"
Java(TM) SE Runtime Environment (build 1.8.0_112-b16)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b16, mixed mode)
Port 8983 is already being used by another process (pid: 83817)
Please choose a different port using the -p option.
➜ ~ cd /usr/local/Cellar/solr/7.2.1
➜ 7.2.1 git:(master) ✗ ll
total 1600
-rw-r–r– 1 crifan admin 737K 1 10 2018 CHANGES.txt
-rw-r–r– 1 crifan admin 582B 8 16 14:22 INSTALL_RECEIPT.json
-rw-r–r– 1 crifan admin 12K 1 10 2018 LICENSE.txt
-rw-r–r– 1 crifan admin 24K 1 10 2018 NOTICE.txt
-rw-r–r– 1 crifan admin 7.3K 1 10 2018 README.txt
drwxr-xr-x 6 crifan admin 192B 8 16 14:24 bin
drwxr-xr-x 8 crifan admin 256B 1 10 2018 example
-rw-r–r– 1 crifan admin 583B 8 16 14:22 homebrew.mxcl.solr.plist
drwxr-xr-x 8 crifan admin 256B 8 16 14:22 libexec
drwxr-xr-x 13 crifan admin 416B 8 16 14:24 server
drwxr-xr-x 3 crifan admin 96B 8 16 14:22 share
➜ 7.2.1 git:(master) ✗ cd server
➜ server git:(master) ✗ ll
total 296
-rw-r–r– 1 crifan admin 3.9K 1 10 2018 README.txt
drwxr-xr-x 3 crifan admin 96B 1 10 2018 contexts
drwxr-xr-x 7 crifan admin 224B 1 10 2018 etc
drwxr-xr-x 23 crifan admin 736B 1 10 2018 lib
drwxr-xr-x 6 crifan admin 192B 8 16 14:24 logs
drwxr-xr-x 6 crifan admin 192B 1 10 2018 modules
drwxr-xr-x 4 crifan admin 128B 1 10 2018 resources
drwxr-xr-x 3 crifan admin 96B 1 10 2018 scripts
drwxr-xr-x 6 crifan admin 192B 1 10 2018 solr
drwxr-xr-x 3 crifan admin 96B 1 10 2018 solr-webapp
-rw-r–r– 1 crifan admin 142K 6 1 2017 start.jar
➜ server git:(master) ✗
本地是7.2.1
线上是7.4.0
都是在server中有个start.jar
然后才传递的参数
找找,如何确认当前正在运行的solr,所用的配置
-》想办法复制到本地Mac中
【无需解决】CentOS中正在运行的Solr如何得到其所有的配置
然后:
【记录】Mac本地Solr中新建qa的collection再运行脚本导入数据
然后再回来看看,是否还会出错
结果404的问题解决了,但是内部数据有问题,导致没有搜索到结果:
input: say hi
failed to find an answer
input:bye
failed to find an answer
【总结】
此处出现
SolrClient.exceptions.ConnectionError: 404
就是solr的server端,内部没有正常包含之前的qa的collection,没有数据,导致连接端口找不到而报错。
后来是确保solr中,正常创建collection(或core)后:
solr create -c qa -s 2 -rf 2
再导入数据,建立索引,然后再运行solr的sever:
solr stop -all
solr start
solr status
然后就可以正常访问:
http://localhost:8983/solr/qa/
这个url了,就不会出现404的问题了。
具体过程详见:
【记录】Mac本地Solr中新建qa的collection再运行脚本导入数据
转载请注明:在路上 » 【已解决】python的solr运行出错:SolrClient.exceptions.ConnectionError: 404