折腾:
【未解决】Fabric 2中如何实现Fabric 1中的rsync_project去实现同步代码以实现项目代码部署
期间,通过:
Recursive multi-file transfer (e.g. put(a_directory)uploads entire directory and all its contents) | Removed | This was another one of the buggiest parts of v1, and over time it became clear that its maintenance burden far outweighed the fact that it was poorly reinventing rsync and/or the use of archival file tools like ye olde tar“+“gzip. |
终于找到了解释了:
fabric1中的rsync,或者说put(某个文件夹),现在已经去除了,因为和rsync本身功能重复了,
暂时替换者是patchwork
所以现在感觉问题转换为:
如何在Fabric 2.x中调用patchwork去实现项目代码同步部署
先去Mac本地安装试试:
➜ Downloads brew install patchwork Updating Homebrew... Error: No available formula with the name "patchwork" ==> Searching for a previously deleted formula (in the last month)... Warning: homebrew/core is shallow clone. To get complete history run: git -C "$(brew --repo homebrew/core)" fetch --unshallow Error: No previously deleted formula found. ==> Searching for similarly named formulae... Error: No similarly named formulae found. ==> Searching taps... ==> Searching taps on GitHub... Error: No formulae found in taps. ➜ Downloads pip install patchwork Collecting patchwork Downloading https://files.pythonhosted.org/packages/52/52/c8f1fc421dcd85d61c7d5e346dd12491a8dfeb4ddc992cb9812b7f6c85b4/patchwork-1.0.1-py2.py3-none-any.whl Requirement already satisfied: fabric<3.0,>=2.0 in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from patchwork) (2.3.1) Requirement already satisfied: cryptography>=1.1 in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from fabric<3.0,>=2.0->patchwork) (2.3.1) Requirement already satisfied: paramiko>=2.4 in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from fabric<3.0,>=2.0->patchwork) (2.4.1) Requirement already satisfied: invoke<2.0,>=1.1 in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from fabric<3.0,>=2.0->patchwork) (1.1.1) Requirement already satisfied: six>=1.4.1 in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from cryptography>=1.1->fabric<3.0,>=2.0->patchwork) (1.4.1) Requirement already satisfied: cffi!=1.11.3,>=1.7 in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from cryptography>=1.1->fabric<3.0,>=2.0->patchwork) (1.11.5) Requirement already satisfied: enum34; python_version < "3" in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from cryptography>=1.1->fabric<3.0,>=2.0->patchwork) (1.1.6) Requirement already satisfied: idna>=2.1 in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from cryptography>=1.1->fabric<3.0,>=2.0->patchwork) (2.7) Requirement already satisfied: asn1crypto>=0.21.0 in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from cryptography>=1.1->fabric<3.0,>=2.0->patchwork) (0.24.0) Requirement already satisfied: ipaddress; python_version < "3" in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from cryptography>=1.1->fabric<3.0,>=2.0->patchwork) (1.0.22) Requirement already satisfied: pyasn1>=0.1.7 in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from paramiko>=2.4->fabric<3.0,>=2.0->patchwork) (0.4.4) Requirement already satisfied: bcrypt>=3.1.3 in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from paramiko>=2.4->fabric<3.0,>=2.0->patchwork) (3.1.4) Requirement already satisfied: pynacl>=1.0.1 in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from paramiko>=2.4->fabric<3.0,>=2.0->patchwork) (1.2.1) Requirement already satisfied: pycparser in /Users/crifan/Library/Python/2.7/lib/python/site-packages (from cffi!=1.11.3,>=1.7->cryptography>=1.1->fabric<3.0,>=2.0->patchwork) (2.18) pipenv 11.10.0 requires certifi, which is not installed. pipenv 11.10.0 requires requests[security], which is not installed. pipenv 11.10.0 requires virtualenv, which is not installed. matplotlib 1.3.1 requires nose, which is not installed. matplotlib 1.3.1 requires tornado, which is not installed. Installing collected packages: patchwork Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/patchwork-1.0.1.dist-info' Consider using the `--user` option or check the permissions. You are using pip version 10.0.1, however version 18.0 is available. You should consider upgrading via the 'pip install --upgrade pip' command.
即:mac中brew找不到patchwork,而只能用pip去安装
然后搜如何使用:
fabric patchwork rsync
还是之前的:
本来想说:
如果用基于Fabric的patchwork,那和自己直接使用rsync,何有区别、优势、好处呢
结果看到:
解释了:
“Convenient wrapper around your friendly local rsync.
Specifically, it calls your local rsync program via a subprocess, and fills in its arguments with Fabric’s current target host/user/port.”
是有好处的,因为自动patchwork自动把Fabric的当前的参数,比如host/user/port自动传递给了rsync了。和手动用rsync比,安全方便,不需要冗余的额外再传递一次。
然后就可以去试试效果了:
patchwork rsync example
结果用:
from invoke import task from fabric import Connection from patchwork.transfers import rsync @task def upload(context): syncResp = rsync(remoteConn, source="../deploy", target=RemotePathRoot) print("syncResp=%s" % (syncResp))
输出log:
Following command run in remote server: Linux xxx-general-01 3.10.0-693.21.1.el7.x86_64 #1 SMP Wed Mar 7 19:03:37 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux /root total 12K drwxr-xr-x 2 root root 4.0K Aug 17 13:56 . drwxr-xr-x 5 root root 4.0K Aug 17 13:52 .. -rw-r--r-- 1 root root 3.1K Aug 17 18:20 fabfile.py building file list ... done deploy/ deploy/fab_others.py deploy/fabfile.py sent 882 bytes received 70 bytes 1.90K bytes/sec total size is 1.48K speedup is 1.56 syncResp=Command exited with status 0. === stdout === building file list ... done deploy/ deploy/fab_others.py deploy/fabfile.py sent 882 bytes received 70 bytes 1.90K bytes/sec total size is 1.48K speedup is 1.56 (no stderr)
结果:
服务器上,只更新了有限的deploy文件夹下面的文件:
而不是本地的xxx文件夹下所有的文件:
继续参考:
试试:
get_proj_path
结果:
from invoke import task from fabric import Connection from patchwork.transfers import rsync ... def seperatorLine(seperatorChar='-', count=80): print(seperatorChar * count) def get_proj_path(): from fabric.main import program return program.collection.loaded_from @task def upload(context): print("upload: context=%s", context) seperatorLine() projectPath = get_proj_path() print("projectPath=%s" % projectPath)
结果是:
➜ deploy git:(master) ✗ fab upload upload: context=%s <Context: <Config: {'run': {'warn': False, 'hide': None, 'shell': '/bin/bash', 'pty': False, 'fallback': True, 'env': {}, 'replace_env': True, 'echo': False, 'encoding': None, 'out_stream': None, 'err_stream': None, 'in_stream': None, 'watchers': [], 'echo_stdin': None}, 'runners': {'local': <class 'invoke.runners.Local'>, 'remote': <class 'fabric.runners.Remote'>}, 'sudo': {'prompt': '[sudo] password: ', 'password': None, 'user': None}, 'tasks': {'dedupe': True, 'auto_dash_names': True, 'collection_name': 'fabfile', 'search_root': None}, 'connect_kwargs': {}, 'forward_agent': False, 'gateway': None, 'inline_ssh_env': False, 'load_ssh_configs': True, 'port': 22, 'ssh_config_path': None, 'timeouts': {'connect': None}, 'user': 'crifan'}>> -------------------------------------------------------------------------------- projectPath=/Users/crifan/dev/dev_root/company/xxx/projects/NLP/sourcecode/xxx/deploy ... ➜ xxx git:(master) ✗ fab -r deploy upload upload: context=%s <Context: <Config: {'run': {'warn': False, 'hide': None, 'shell': '/bin/bash', 'pty': False, 'fallback': True, 'env': {}, 'replace_env': True, 'echo': False, 'encoding': None, 'out_stream': None, 'err_stream': None, 'in_stream': None, 'watchers': [], 'echo_stdin': None}, 'runners': {'local': <class 'invoke.runners.Local'>, 'remote': <class 'fabric.runners.Remote'>}, 'sudo': {'prompt': '[sudo] password: ', 'password': None, 'user': None}, 'tasks': {'dedupe': True, 'auto_dash_names': True, 'collection_name': 'fabfile', 'search_root': None}, 'connect_kwargs': {}, 'forward_agent': False, 'gateway': None, 'inline_ssh_env': False, 'load_ssh_configs': True, 'port': 22, 'ssh_config_path': None, 'timeouts': {'connect': None}, 'user': 'crifan'}>> -------------------------------------------------------------------------------- projectPath=/Users/crifan/dev/dev_root/company/xxx/projects/NLP/sourcecode/xxx/deploy
即,不论是在deploy子目录,还是根目录,program.collection.loaded_from都是返回fabfile被加载的目录,即:
/Users/crifan/dev/dev_root/company/xxx/projects/NLP/sourcecode/xxx/deploy
【总结】
最后用如下代码:
from invoke import task from fabric import Connection from patchwork.transfers import rsync import os RemoteHost = '47.x.x.x' RemoteUser = 'UserName' RemotePathRoot = '/root/xxx_20180101/nlp/test_fabric' def seperatorLine(seperatorChar='-', count=80): print(seperatorChar * count) def fabLoadedPath(): from fabric.main import program return program.collection.loaded_from @task def upload(context): print("upload: context=%s", context) seperatorLine() fabFilePath = fabLoadedPath() print("fabFilePath=%s" % fabFilePath) localProjectRootPath = os.path.join(fabFilePath, "..") print("localProjectRootPath=%s" % localProjectRootPath) seperatorLine() print("Local environment:") context.run("uname -a") context.run("pwd") context.run("ls -lha") seperatorLine() remoteConn = Connection(host=RemoteHost, user=RemoteUser) print(remoteConn) seperatorLine() print("Remote Server:") remoteConn.run('uname -a') remoteConn.run('pwd') print("remote path: %s" % RemotePathRoot) remoteConn.run('ls -lha %s' % (RemotePathRoot)) # remoteConn.run('cd %s && pwd && ls -lha' % RemotePathRoot) # putFileResult = remoteConn.put('fabfile.py', remote=RemotePathRoot) # print("Uploaded {0.local} to {0.remote}".format(putFileResult)) # syncSource = "../deploy" syncSource = localProjectRootPath # /Users/crifan/dev/dev_root/company/xxx/projects/NLP/sourcecode/xxx/ syncTarget = RemotePathRoot # /root/xxx_20180101/nlp/test_fabric # syncExclude = [".DS_Store", "data/", "processData/", "*.log", "*.pyc", "__pycache__"] syncExclude = [".DS_Store", "data/", "processData/mysqlQa/output_summary/", "*.log", "*.pyc", "__pycache__"] syncResp = rsync(remoteConn, source=syncSource, target=syncTarget, delete=True, exclude=syncExclude) print("Sync lcoal %s to remote %s while exclude %s -> return %s" % (syncSource, syncTarget, syncExclude, syncResp))
输出的log是:
➜ deploy git:(master) ✗ fab upload upload: context=%s <Context: <Config: {'run': {'warn': False, 'hide': None, 'shell': '/bin/bash', 'pty': False, 'fallback': True, 'env': {}, 'replace_env': True, 'echo': False, 'encoding': None, 'out_stream': None, 'err_stream': None, 'in_stream': None, 'watchers': [], 'echo_stdin': None}, 'runners': {'local': <class 'invoke.runners.Local'>, 'remote': <class 'fabric.runners.Remote'>}, 'sudo': {'prompt': '[sudo] password: ', 'password': None, 'user': None}, 'tasks': {'dedupe': True, 'auto_dash_names': True, 'collection_name': 'fabfile', 'search_root': None}, 'connect_kwargs': {}, 'forward_agent': False, 'gateway': None, 'inline_ssh_env': False, 'load_ssh_configs': True, 'port': 22, 'ssh_config_path': None, 'timeouts': {'connect': None}, 'user': 'crifan'}>> -------------------------------------------------------------------------------- fabFilePath=/Users/crifan/dev/dev_root/company/xxx/projects/NLP/sourcecode/xxx/deploy localProjectRootPath=/Users/crifan/dev/dev_root/company/xxx/projects/NLP/sourcecode/xxx/deploy/.. -------------------------------------------------------------------------------- Local environment: Darwin licrifandeMacBook-Pro.local 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64 /Users/crifan/dev/dev_root/company/xxx/projects/NLP/sourcecode/xxx/deploy total 16 drwxr-xr-x 4 crifan staff 128B Aug 20 14:34 . drwxr-xr-x 16 crifan staff 512B Aug 20 13:45 .. -rw-r--r-- 1 crifan staff 424B Aug 15 09:53 fab_others.py -rw-r--r-- 1 crifan staff 2.0K Aug 20 14:34 fabfile.py -------------------------------------------------------------------------------- <Connection host=47.x.x.x user=username> -------------------------------------------------------------------------------- Remote Server: Linux xxx-general-01 3.10.0-693.21.1.el7.x86_64 #1 SMP Wed Mar 7 19:03:37 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux /root remote path: /root/xxx_20180101/nlp/test_fabric total 36K drwxr-xr-x 4 root root 4.0K Aug 20 14:39 . drwxr-xr-x 5 root root 4.0K Aug 17 13:52 .. -rw-r--r-- 1 root root 8.1K Aug 16 13:30 .DS_Store -rw-r--r-- 1 root root 4 Aug 20 14:39 file_to_delete.txt drwxr-xr-x 7 root root 4.0K Aug 20 14:37 .git -rw-r--r-- 1 root root 202 Aug 15 14:46 .gitignore drwxr-xr-x 5 root root 4.0K Aug 20 14:34 .idea building file list ... done deleting file_to_delete.txt ./ Pipfile Pipfile.lock README.md __init__.py .git/ conf/ conf/main.conf deploy/ deploy/fab_others.py deploy/fabfile.py nlp/ nlp/dialog/ nlp/dialog/AccessData.py nlp/dialog/Analyzer.py nlp/dialog/DialogueManager.py nlp/dialog/GenerateResponse.py nlp/dialog/SearchData.py nlp/dialog/__init__.py nlp/dialog/intent.py nlp/dialog/config/ nlp/dialog/config/config.ini nlp/dialog/files/ nlp/dialog/files/control.txt nlp/dialog/files/test.txt nlp/search/ nlp/search/app.conf nlp/search/feature_embedding.py nlp/search/web_qa.py nlp/search/amq/ nlp/search/amq/IMQ.py nlp/search/amq/hlt.py nlp/search/amq/jb.py nlp/search/amq/judge.py nlp/search/amq/nlp_property.py nlp/search/amq/random_helper.py nlp/search/amq/sim.py nlp/search/amq/vector.py nlp/search/amq/word_cut.py nlp/search/dmn/ nlp/search/dmn/attention_gru_cell.py nlp/search/dmn/char/ nlp/search/dmn/char/data_utils.py nlp/search/dmn/char/dmn_data_utils.py nlp/search/dmn/char/dmn_main.py nlp/search/dmn/char/dmn_plus.py nlp/search/dmn/char/dmn_session.py nlp/search/dmn/char/vector_helper.py nlp/search/dmn/dmn_fasttext/ nlp/search/dmn/dmn_fasttext/attention_gru_cell.py nlp/search/dmn/dmn_fasttext/config.py nlp/search/dmn/dmn_fasttext/data_helper.py nlp/search/dmn/dmn_fasttext/dmn_plus.py nlp/search/dmn/dmn_fasttext/dmn_session.py nlp/search/dmn/dmn_fasttext/main.py nlp/search/graph/ nlp/search/graph/automata.py nlp/search/graph/policy.py nlp/search/graph/state_card.py nlp/search/iob/ nlp/search/iob/config.py nlp/search/iob/data_helper.py nlp/search/iob/main.py nlp/search/iob/model.py nlp/search/iob/session.py nlp/search/iob/test.py nlp/search/memory/ nlp/search/memory/config.py nlp/search/memory/data_utils.py nlp/search/memory/dialog_simulator.py nlp/search/memory/feature_embedding.py nlp/search/memory/main.py nlp/search/memory/memn2n.py nlp/search/memory/memn2n2.py nlp/search/memory/memn2n_lstm.py nlp/search/memory/memn2n_session.py nlp/search/ml/ nlp/search/ml/__init__.py nlp/search/ml/belief_clf.py nlp/search/mypy/ nlp/search/mypy/singleton.py nlp/search/qa/ nlp/search/qa/false_instructor.py nlp/search/qa/iqa.py nlp/search/renderer/ nlp/search/renderer/conditional_render.py nlp/search/renderer/machine_profile.py nlp/search/renderer/render.py nlp/search/renderer/reply.py nlp/search/rnn/ nlp/search/rnn/config.py nlp/search/rnn/data_helper.py nlp/search/rnn/main.py nlp/search/rnn/model.py nlp/search/rnn/session.py nlp/search/slu/ nlp/search/slu/luis.py nlp/search/slu/nlu.py nlp/search/static/ nlp/search/static/context.py nlp/search/utils/ nlp/search/utils/cn2arab.py nlp/search/utils/json_util.py nlp/search/utils/mongodb_client.py nlp/search/utils/mysql2solr.py nlp/search/utils/query_util.py nlp/search/utils/regex_rank.py nlp/search/utils/solr_util.py nlp/search/utils/translator.py nlp/search/utils/embedding/ nlp/search/utils/embedding/__init__.py nlp/search/utils/embedding/embedding_helper.py nlp/search/utils/embedding/feature_embedding.py nlp/search/utils/embedding/glove_embedding_helper.py processData/ processData/__init__.py processData/mysqlMedia/ processData/mysqlMedia/README.md processData/mysqlMedia/mongoGridfsToMysqlMediaKeyword.py processData/mysqlQa/ processData/mysqlQa/README.md processData/mysqlQa/mongoDialogToMysqlQa.py processData/mysqlQa/vocabulary.txt processData/mysqlThesaurus/ processData/mysqlThesaurus/README.md processData/mysqlThesaurus/vocabularyToMysql.py processData/mysqlThesaurus/\#350\#257\#215\#346\#261\#207\#350\#241\#250 180627.xlsx processData/other/ processData/other/topic processData/other/update_topic.py processData/other/to_delete/ processData/other/to_delete/resource.2018.1.29.xlsx processData/other/to_delete/resource.xlsx processData/other/to_delete/xlstomysql.py util/ util/__init__.py util/configs.py util/database.py util/paths.py util/crifanLib/ util/crifanLib/__init__.py util/crifanLib/crifanBeautifulsoup.py util/crifanLib/crifanCookie.py util/crifanLib/crifanDatetime.py util/crifanLib/crifanDict.py util/crifanLib/crifanEmail.py util/crifanLib/crifanFile.py util/crifanLib/crifanGeography.py util/crifanLib/crifanHtml.py util/crifanLib/crifanHttp.py util/crifanLib/crifanList.py util/crifanLib/crifanLogging.py util/crifanLib/crifanMath.py util/crifanLib/crifanMysql.py util/crifanLib/crifanOpenpyxl.py util/crifanLib/crifanString.py util/crifanLib/crifanSystem.py util/crifanLib/crifanTemplate.py util/crifanLib/crifanUrl.py sent 542.83K bytes received 2.99K bytes 363.88K bytes/sec total size is 3.04M speedup is 5.56 Sync lcoal /Users/crifan/dev/dev_root/company/xxx/projects/NLP/sourcecode/xxx/deploy/.. to remote /root/xxx_20180101/nlp/test_fabric while exclude ['.DS_Store', 'data/', 'processData/mysqlQa/output_summary/', '*.log', '*.pyc', '__pycache__'] -> return Command exited with status 0. === stdout === building file list ... done deleting file_to_delete.txt ./ Pipfile Pipfile.lock README.md __init__.py .git/ conf/ conf/main.conf deploy/ deploy/fab_others.py deploy/fabfile.py nlp/ nlp/dialog/ nlp/dialog/AccessData.py nlp/dialog/Analyzer.py nlp/dialog/DialogueManager.py nlp/dialog/GenerateResponse.py nlp/dialog/SearchData.py nlp/dialog/__init__.py nlp/dialog/intent.py nlp/dialog/config/ nlp/dialog/config/config.ini nlp/dialog/files/ nlp/dialog/files/control.txt nlp/dialog/files/test.txt nlp/search/ nlp/search/app.conf nlp/search/feature_embedding.py nlp/search/web_qa.py nlp/search/amq/ nlp/search/amq/IMQ.py nlp/search/amq/hlt.py nlp/search/amq/jb.py nlp/search/amq/judge.py nlp/search/amq/nlp_property.py nlp/search/amq/random_helper.py nlp/search/amq/sim.py nlp/search/amq/vector.py nlp/search/amq/word_cut.py nlp/search/dmn/ nlp/search/dmn/attention_gru_cell.py nlp/search/dmn/char/ nlp/search/dmn/char/data_utils.py nlp/search/dmn/char/dmn_data_utils.py nlp/search/dmn/char/dmn_main.py nlp/search/dmn/char/dmn_plus.py nlp/search/dmn/char/dmn_session.py nlp/search/dmn/char/vector_helper.py nlp/search/dmn/dmn_fasttext/ nlp/search/dmn/dmn_fasttext/attention_gru_cell.py nlp/search/dmn/dmn_fasttext/config.py nlp/search/dmn/dmn_fasttext/data_helper.py nlp/search/dmn/dmn_fasttext/dmn_plus.py nlp/search/dmn/dmn_fasttext/dmn_session.py nlp/search/dmn/dmn_fasttext/main.py nlp/search/graph/ nlp/search/graph/automata.py nlp/search/graph/policy.py nlp/search/graph/state_card.py nlp/search/iob/ nlp/search/iob/config.py nlp/search/iob/data_helper.py nlp/search/iob/main.py nlp/search/iob/model.py nlp/search/iob/session.py nlp/search/iob/test.py nlp/search/memory/ nlp/search/memory/config.py nlp/search/memory/data_utils.py nlp/search/memory/dialog_simulator.py nlp/search/memory/feature_embedding.py nlp/search/memory/main.py nlp/search/memory/memn2n.py nlp/search/memory/memn2n2.py nlp/search/memory/memn2n_lstm.py nlp/search/memory/memn2n_session.py nlp/search/ml/ nlp/search/ml/__init__.py nlp/search/ml/belief_clf.py nlp/search/mypy/ nlp/search/mypy/singleton.py nlp/search/qa/ nlp/search/qa/false_instructor.py nlp/search/qa/iqa.py nlp/search/renderer/ nlp/search/renderer/conditional_render.py nlp/search/renderer/machine_profile.py nlp/search/renderer/render.py nlp/search/renderer/reply.py nlp/search/rnn/ nlp/search/rnn/config.py nlp/search/rnn/data_helper.py nlp/search/rnn/main.py nlp/search/rnn/model.py nlp/search/rnn/session.py nlp/search/slu/ nlp/search/slu/luis.py nlp/search/slu/nlu.py nlp/search/static/ nlp/search/static/context.py nlp/search/utils/ nlp/search/utils/cn2arab.py nlp/search/utils/json_util.py nlp/search/utils/mongodb_client.py nlp/search/utils/mysql2solr.py nlp/search/utils/query_util.py nlp/search/utils/regex_rank.py nlp/search/utils/solr_util.py nlp/search/utils/translator.py nlp/search/utils/embedding/ nlp/search/utils/embedding/__init__.py nlp/search/utils/embedding/embedding_helper.py nlp/search/utils/embedding/feature_embedding.py nlp/search/utils/embedding/glove_embedding_helper.py processData/ processData/__init__.py processData/mysqlMedia/ processData/mysqlMedia/README.md processData/mysqlMedia/mongoGridfsToMysqlMediaKeyword.py processData/mysqlQa/ processData/mysqlQa/README.md processData/mysqlQa/mongoDialogToMysqlQa.py processData/mysqlQa/vocabulary.txt processData/mysqlThesaurus/ processData/mysqlThesaurus/README.md processData/mysqlThesaurus/vocabularyToMysql.py processData/mysqlThesaurus/\#350\#257\#215\#346\#261\#207\#350\#241\#250 180627.xlsx processData/other/ processData/other/topic processData/other/update_topic.py processData/other/to_delete/ processData/other/to_delete/resource.2018.1.29.xlsx processData/other/to_delete/resource.xlsx processData/other/to_delete/xlstomysql.py util/ util/__init__.py util/configs.py util/database.py util/paths.py util/crifanLib/ util/crifanLib/__init__.py util/crifanLib/crifanBeautifulsoup.py util/crifanLib/crifanCookie.py util/crifanLib/crifanDatetime.py util/crifanLib/crifanDict.py util/crifanLib/crifanEmail.py util/crifanLib/crifanFile.py util/crifanLib/crifanGeography.py util/crifanLib/crifanHtml.py util/crifanLib/crifanHttp.py util/crifanLib/crifanList.py util/crifanLib/crifanLogging.py util/crifanLib/crifanMath.py util/crifanLib/crifanMysql.py util/crifanLib/crifanOpenpyxl.py util/crifanLib/crifanString.py util/crifanLib/crifanSystem.py util/crifanLib/crifanTemplate.py util/crifanLib/crifanUrl.py sent 542.83K bytes received 2.99K bytes 363.88K bytes/sec total size is 3.04M speedup is 5.56 (no stderr) ➜ deploy git:(master) ✗
实现了效果:
服务端,上传了对应的文件:
且:
- 排除了那些不想要上传的文件:
比如
各个log文件:
整个文件夹:
processData/mysqlQa/output_summary/
pyc:
- 删除了服务器中本身(估计添加用来测试的)额外的文件:
deleting file_to_delete.txt
另外更高级的rsync功能,可以参考:
比如:
Set the Max Size of Files to be Transferred
–max-size=’200k’
指定最大文件大小