【已解决】python的solr运行出错：SolrClient.exceptions.ConnectionError: 404

折腾：

【已解决】Mac中安装Solr的server并启动Solr服务

期间，现有python的solr相关代码：

SOLR = SolrClient(‘http://localhost:8999/solr’)

…

docs = self.solr.fq(q_mapper=q_mapper, fq_mapper=fq_mapper, retrieve_keys=retrieve_keys, rows=1)

去运行，出错：

   File "/Users/crifan/dev/dev_root/xxx/search/qa/iqa.py", line 163, in main

    reply = qa.get_responses(i, "qa")

  File "/Users/crifan/dev/dev_root/xxx/search/qa/iqa.py", line 80, in get_responses

    docs = self.solr.fq(q_mapper=q_mapper, fq_mapper=fq_mapper, retrieve_keys=retrieve_keys, rows=1)

  File "/Users/crifan/dev/dev_root/company//xxx/sourcecode/xxx/xxx/search/utils/solr_util.py", line 240, in fq

    resp = self.solr.query(self.core, params)

  File "/Users/crifan/.virtualenvs/xxx-gXiJ4vtz/lib/python3.6/site-packages/SolrClient/solrclient.py", line 125, in query

    **kwargs)

  File "/Users/crifan/.virtualenvs/xxx-gXiJ4vtz/lib/python3.6/site-packages/SolrClient/transport/transportbase.py", line 39, in inner

    return function(self, host, **kwargs)

  File "/Users/crifan/.virtualenvs/xxx-gXiJ4vtz/lib/python3.6/site-packages/SolrClient/transport/transportbase.py", line 57, in send_request

    res_dict, c_inf = self._send(host, **kwargs)

  File "/Users/crifan/.virtualenvs/xxx-gXiJ4vtz/lib/python3.6/site-packages/SolrClient/transport/transportrequests.py", line 66, in _send

    raise ConnectionError("404 – {}".format(res.url))

SolrClient.exceptions.ConnectionError: 404 – http://localhost:8983/solr/qa/select?q=question_str%3A%22sing+a++song%22&fq=%2A%3A%2A+AND+scene%3Aqa&rows=1&fl=question%2Canswer%2Cid&wt=json&indent=false

看起来错误和之前的：

【已解决】Python中solr出错：SolrClient.exceptions.ConnectionError NewConnectionError Failed to establish a new connection Errno 61 Connection refused

很类似。

搜：

python SolrClient.exceptions.ConnectionError: 404

SolrClient/transportrequests.py at master · moonlitesolutions/SolrClient

        if res.status_code == 404:

            raise ConnectionError("404 – {}".format(res.url))

感觉是：

虽然connection有了，但是此处http://localhost:8983/solr

的

 /qa/select

的api端口找不到，所以404

突然想起来了：

dev服务器上可以正常运行

-》所以去dev服务器上看看后台的solr是如何运行的

[root@xxx-general-01 xxx]# solr status

-bash: solr: command not found

[root@xxx-general-01 xxx]# ps aux | grep 8983

dev       5424  0.0  3.8 5819128 633084 ?      Sl   Jul18  38:45 java -server -Xms2g -Xmx2g -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:-OmitStackTraceInFastThrow -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/home/dev/nlp/qa/data/solr-7.4.0/server/logs/solr_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M -Dsolr.log.dir=/home/dev/nlp/qa/data/solr-7.4.0/server/logs -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -Djetty.home=/home/dev/nlp/qa/data/solr-7.4.0/server -Dsolr.solr.home=/home/dev/nlp/qa/data/solr-7.4.0/server/solr -Dsolr.data.home= -Dsolr.install.dir=/home/dev/nlp/qa/data/solr-7.4.0 -Dsolr.default.confdir=/home/dev/nlp/qa/data/solr-7.4.0/server/solr/configsets/_default/conf -Xss256k -Dsolr.jetty.https.port=8983 -Dsolr.log.muteconsole -XX:OnOutOfMemoryError=/home/dev/nlp/qa/data/solr-7.4.0/bin/oom_solr.sh 8983 /home/dev/nlp/qa/data/solr-7.4.0/server/logs -jar start.jar –module=http

root     24989  0.0  0.0 112664   976 pts/0    S+   14:53   0:00 grep –color=auto 8983

[root@xxx-general-01 xxx]# ps aux | grep solr

dev       5424  0.0  3.8 5819128 633084 ?      Sl   Jul18  38:45 java -server -Xms2g -Xmx2g -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:-OmitStackTraceInFastThrow -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/home/dev/nlp/qa/data/solr-7.4.0/server/logs/sol_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M -Dsolr.log.dir=/home/dev/nlp/qa/data/solr-7.4.0/server/logs -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -Djetty.home=/home/dev/nlp/qa/data/solr-7.4.0/server -Dsolr.solr.home=/home/dev/nlp/qa/data/solr-7.4.0/server/solr -Dsolr.data.home= -Dsolr.install.dir=/home/dev/nlp/qa/data/solr-7.4.0 -Dsolr.default.confdir=/home/dev/nlp/qa/data/solr-7.4.0/server/solr/configsets/_default/conf -Xss256k -Dsolr.jetty.https.port=8983 -Dsolr.log.muteconsole -XX:OnOutOfMemoryError=/home/dev/nlp/qa/data/solr-7.4.0/bin/oom_solr.sh 8983 /home/dev/nlp/qa/data/solr-7.4.0/server/logs -jar start.jar –module=http

root     24991  0.0  0.0 112664   976 pts/0    S+   14:53   0:00 grep –color=auto solr

也的确能看到对应solr的路径：

[root@xx-xx-01 xxx]# ls /home/dev/nlp/qa/data/solr-7.4.0/server/

contexts/    .DS_Store    etc/         lib/         logs/        modules/     README.txt   resources/   scripts/     solr/        solr-webapp/ start.jar    

所以现在可以处理为：

除非本地想要调试solr，否则本地可以不去关心如何搭建solr的服务了。

因为在线服务器中已经正常运行了solr的服务器了。

发现此处是需要解决本地搭建solr的问题。

此处对比一下 线上CentOS 和 本地Mac中的不同

线上CentOS

java -server -Xms2g -Xmx2g -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:-OmitStackTraceInFastThrow -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/home/dev/nlp/qa/data/solr-7.4.0/server/logs/sol_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M -Dsolr.log.dir=/home/dev/nlp/qa/data/solr-7.4.0/server/logs -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -Djetty.home=/home/dev/nlp/qa/data/solr-7.4.0/server -Dsolr.solr.home=/home/dev/nlp/qa/data/solr-7.4.0/server/solr -Dsolr.data.home= -Dsolr.install.dir=/home/dev/nlp/qa/data/solr-7.4.0 -Dsolr.default.confdir=/home/dev/nlp/qa/data/solr-7.4.0/server/solr/configsets/_default/conf -Xss256k -Dsolr.jetty.https.port=8983 -Dsolr.log.muteconsole -XX:OnOutOfMemoryError=/home/dev/nlp/qa/data/solr-7.4.0/bin/oom_solr.sh 8983 /home/dev/nlp/qa/data/solr-7.4.0/server/logs -jar start.jar –module=http

[root@xxx-general-01 xxx_20180101]# cd /home/dev

[root@xxx-general-01 dev]# ll

total 12

drwxrwxr-x  2 dev dev 4096 Jul 18 22:30 downloads

drwxrwxr-x 13 dev dev 4096 Jul 18 22:31 miniconda2

drwxrwxrwx  3 dev dev 4096 Jul 18 22:08 nlp

[root@xxx-general-01 dev]# cd nlp/

[root@xxx-general-01 nlp]# ll

total 4

drwxrwxrwx 4 dev dev 4096 Jul 18 22:08 qa

本地Mac

➜  ~ solr -V

Using Solr root directory: /usr/local/Cellar/solr/7.2.1

Using Java: java

java version "1.8.0_112"

Java(TM) SE Runtime Environment (build 1.8.0_112-b16)

Java HotSpot(TM) 64-Bit Server VM (build 25.112-b16, mixed mode)

Port 8983 is already being used by another process (pid: 83817)

Please choose a different port using the -p option.

➜  ~ cd /usr/local/Cellar/solr/7.2.1

➜  7.2.1 git:(master) ✗ ll

total 1600

-rw-r–r–   1 crifan  admin   737K  1 10  2018 CHANGES.txt

-rw-r–r–   1 crifan  admin   582B  8 16 14:22 INSTALL_RECEIPT.json

-rw-r–r–   1 crifan  admin    12K  1 10  2018 LICENSE.txt

-rw-r–r–   1 crifan  admin    24K  1 10  2018 NOTICE.txt

-rw-r–r–   1 crifan  admin   7.3K  1 10  2018 README.txt

drwxr-xr-x   6 crifan  admin   192B  8 16 14:24 bin

drwxr-xr-x   8 crifan  admin   256B  1 10  2018 example

-rw-r–r–   1 crifan  admin   583B  8 16 14:22 homebrew.mxcl.solr.plist

drwxr-xr-x   8 crifan  admin   256B  8 16 14:22 libexec

drwxr-xr-x  13 crifan  admin   416B  8 16 14:24 server

drwxr-xr-x   3 crifan  admin    96B  8 16 14:22 share

➜  7.2.1 git:(master) ✗ cd server

➜  server git:(master) ✗ ll

total 296

-rw-r–r–   1 crifan  admin   3.9K  1 10  2018 README.txt

drwxr-xr-x   3 crifan  admin    96B  1 10  2018 contexts

drwxr-xr-x   7 crifan  admin   224B  1 10  2018 etc

drwxr-xr-x  23 crifan  admin   736B  1 10  2018 lib

drwxr-xr-x   6 crifan  admin   192B  8 16 14:24 logs

drwxr-xr-x   6 crifan  admin   192B  1 10  2018 modules

drwxr-xr-x   4 crifan  admin   128B  1 10  2018 resources

drwxr-xr-x   3 crifan  admin    96B  1 10  2018 scripts

drwxr-xr-x   6 crifan  admin   192B  1 10  2018 solr

drwxr-xr-x   3 crifan  admin    96B  1 10  2018 solr-webapp

-rw-r–r–   1 crifan  admin   142K  6  1  2017 start.jar

➜  server git:(master) ✗

本地是7.2.1

线上是7.4.0

都是在server中有个start.jar

然后才传递的参数

找找，如何确认当前正在运行的solr，所用的配置

-》想办法复制到本地Mac中

【无需解决】CentOS中正在运行的Solr如何得到其所有的配置

然后：

【记录】Mac本地Solr中新建qa的collection再运行脚本导入数据

然后再回来看看，是否还会出错

结果404的问题解决了，但是内部数据有问题，导致没有搜索到结果：

input: say hi

http://localhost:8983/solr/qa/select?q=question_str%3A%22+say+hi%22&fq=%2A%3A%2A+AND+scene%3Aqa&rows=1&fl=question%2Canswer%2Cid&wt=json&indent=false

http://localhost:8983/solr/qa/select?q=question%3A%22+say+hi%22&fq=%2A%3A%2A+AND+scene%3Aqa&rows=100&fl=question%2Canswer%2Cid&wt=json&indent=false

failed to find an answer

input:bye

http://localhost:8983/solr/qa/select?q=question_str%3A%22bye%22&fq=%2A%3A%2A+AND+scene%3Aqa&rows=1&fl=question%2Canswer%2Cid&wt=json&indent=false

http://localhost:8983/solr/qa/select?q=question%3A%22bye%22&fq=%2A%3A%2A+AND+scene%3Aqa&rows=100&fl=question%2Canswer%2Cid&wt=json&indent=false

failed to find an answer

【总结】

此处出现

SolrClient.exceptions.ConnectionError: 404

就是solr的server端，内部没有正常包含之前的qa的collection，没有数据，导致连接端口找不到而报错。

后来是确保solr中，正常创建collection（或core）后：

solr create -c qa -s 2 -rf 2

再导入数据，建立索引，然后再运行solr的sever：

solr stop -all

solr start

solr status

然后就可以正常访问：

http://localhost:8983/solr/qa/

这个url了，就不会出现404的问题了。

具体过程详见：

【记录】Mac本地Solr中新建qa的collection再运行脚本导入数据

转载请注明：在路上 » 【已解决】python的solr运行出错：SolrClient.exceptions.ConnectionError: 404