之前通过代码批量把一堆的mp3和pdf等文件保存到在线的mongo中了。现在手上有3个mp3文件：
现在本地的fs.files中单个文件是：
<code>> db.fs.files.findOne()
{
"_id" : ObjectId("5b20d5ec7f4d384006e91f18"),
"contentType" : "audio/mpeg",
"chunkSize" : 261120,
"metadata" : {
"song" : {
"singers" : [ ]
},
"fitAgeStart" : 3,
"topics" : [
"Wisdom",
"Animals",
"Villain"
],
"storybook" : {
"publisher" : "Mathew Price",
"isFiction" : "是",
"lexileIndex" : "",
"awards" : "",
"authors" : [
"Atsuko Morozumi"
],
"foreignCountry" : "英国"
},
"keywords" : {
"fromName" : [
"go away Mr Wolf",
"Mr wolf",
"Go away",
"wolf"
],
"other" : [ ],
"fromContent" : [
"knock!",
"go away",
"Mr Wolf",
"quickly shut the door"
]
},
"name" : "Go away, Mr Wolf",
"resourceType" : "storybook",
"mainActors" : [
"three little pigs",
"Mr.Wolf"
],
"contentAbstract" : "一个教小朋友警惕坏人的故事，狼先生扮成各种样子前来敲小猪家的们，3只小猪识别了狼的真面目，把门关上不让狼进来，一直到爸爸回来把发生的事情告诉爸爸听。",
"isSeries" : false,
"series" : {
"number" : 0,
"name" : ""
},
"fitAgeEnd" : 4,
"fileInfo" : {
"isAudio" : true,
"contentType" : "audio/mpeg",
"name" : "Go away, Mr Wolf.mp3",
"suffix" : "mp3"
}
},
"filename" : "Go away, Mr Wolf.mp3",
"length" : 1613790,
"uploadDate" : ISODate("2018-06-13T08:29:32.493Z"),
"md5" : "3fa28e2ef5e7e56e15f797cd169845cc"
}
>
</code>需要保存mp3到mongo中，且把相关的信息都保存进去。
mongo save file with meta
mongo save file with metadata
mongofiles save with metadata
貌似没有metadata的选项啊
去看看Mongo Compass中是否有插入文件，且设置metadata的功能
貌似没有插入文件的选项？
去搜：
自己看了看：
貌似导入的是json
但是貌似没发指定mp3文件？
[JAVA-1013] Saved extra fields for Gridfs does not go in metadata sub-document – MongoDB
mongo 保存 metadata
mongo 保存 GridFS metadata
难道是只能和之前一样，去写python代码去保存file，同时传递metadata了？
mongo compass gridfs
mongo compass support gridfs
Creating Compass Plugins — MongoDB Compass stable
Visualizing your data with MongoDB Compass | MongoDB
【未解决】尝试用Mongo Management Studio去实现导入文件到Mongo的gridfs且带metadata信息
看到：
https://www.guru99.com/top-20-mongodb-tools.html
“15) NoSQL Manager:
* File Manager Tool to work with GridFS”
去看看
NoSQL Manager – GUI tool with intelligent Shell certified by MongoDB
貌似也没有gridfs上传文件的选项，更别说metadata了。
“File Manager tool to work with GridFS”
是支持gridfs上传文件的
不清楚是否支持修改metadata？
去试试
下载之前也先去看看功能：
Managing GridFS
File Manager tool to work with GridFS
free版本都支持。
Downloads – NoSQL Manager – GUI tool with intelligent Shell certified by MongoDB
下载：
NoSQL Manager for MongoDB Freeware
发现是exe版本的：
才注意到：
“System Requirements
* Microsoft Windows Vista or higher
* Microsoft .Net Framework 4.5”
而此处是Mac，用不了。
再去确认：
<code>mongofiles put --help </code>
和：
<code>mongofiles --help </code>
输出一样：
都不支持put时传递metadata
看来真的只能去写python代码去保存了。
【已解决】mongo的shell中find返回多个有限个数的结果
【总结】
最后是写代码实现：
把mp3文件存入进去后，且还带着metadata的：
<code># -*- coding: utf-8 -*-
import pymongo
from pymongo import MongoClient
import gridfs
# from pymongo.objectid import ObjectId
# from pymongo import objectid
from bson.objectid import ObjectId
from gridfs import GridFS
# import pprint
import os
import errno
import logging
import sys
sys.path.append("libs/crifan")
import crifanLib
import re
import time
import mime
from urllib import quote_plus
################################################################################
# Global Config/Setting
################################################################################
########## Mongo Settings ##########
# mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]
MongoHostRemote = "x.x.x.x"
# MongoHostLocal = "localhost"
# MongoHostLocal = "127.0.0.1"
# MongoHost = MongoHostLocal
MongoHost = MongoHostRemote
MongoPort = 27017
</code>MongoUseAuth = True
# MongoUseAuth = False
# with auth
MongoUsername = “gridfs”
MongoPassword = “P@wd”
MongoAuthenticationDatabase = “gridfs”
mongodbUri = “”
if MongoUseAuth :
mongodbUri = “mongodb://%s:%s@%s:%s/%s” % (
quote_plus(MongoUsername), \
quote_plus(MongoPassword), \
MongoHost, \
MongoPort, \
MongoAuthenticationDatabase \
)
#’mongodb://userName:[email protected]:27017/gridfs’
else:
mongodbUri = “mongodb://%s:%s” % (
MongoHost, \
MongoPort
)
# print “mongodbUri=”,mongodbUri
StorybookResourceType = “storybook”
SongResourceType = “song”
################################################################################
# Global Value
################################################################################
ToImportFileFolder = “files/to_import”
FileInfoList = [
{
“filename”: “Finger Family.mp3”,
“metadata”: {
“song”: {
“singers”: [
“ChuChu TV”
]
},
“fitAgeStart”: 3,
“topics”: [
],
“storybook”: {
“publisher”: “”,
“isFiction”: “未知”,
“lexileIndex”: “”,
“awards”: “”,
“authors”: [],
“foreignCountry”: “”
},
“keywords”: {
“fromName”: [
“Finger”,
“Family”
],
“other”: [],
“fromContent”: [
“Daddy Finger”
]
},
“name”: “Finger Family”,
“resourceType”: “song”,
“mainActors”: [],
“contentAbstract”: “”,
“isSeries”: False,
“series”: {
“number”: 0,
“name”: “”
},
“fitAgeEnd”: 6,
“fileInfo”: {
“isAudio”: True,
“contentType”: “audio/mpeg”,
“name”: “Finger Family.mp3”,
“suffix”: “mp3”
}
},
},
{
“filename”: “Finger Family Collection.mp3”,
“metadata”: {
“song”: {
“singers”: []
},
“fitAgeStart”: 3,
“topics”: [
],
“storybook”: {
“publisher”: “”,
“isFiction”: “未知”,
“lexileIndex”: “”,
“awards”: “”,
“authors”: [],
“foreignCountry”: “”
},
“keywords”: {
“fromName”: [
“Finger”,
“Family”,
“Collection”
],
“other”: [],
“fromContent”: [
“Daddy Finger Collection”
]
},
“name”: “Finger Family”,
“resourceType”: “song”,
“mainActors”: [],
“contentAbstract”: “”,
“isSeries”: False,
“series”: {
“number”: 0,
“name”: “”
},
“fitAgeEnd”: 6,
“fileInfo”: {
“isAudio”: True,
“contentType”: “audio/mpeg”,
“name”: “Finger Family Collection.mp3”,
“suffix”: “mp3”
}
},
},
{
“filename”: “Mr. Sun, Sun, Mr. Golden Sun.mp3”,
“metadata”: {
“song”: {
“singers”: [
“Super Simple Songs”
]
},
“fitAgeStart”: 3,
“topics”: [
],
“storybook”: {
“publisher”: “”,
“isFiction”: “未知”,
“lexileIndex”: “”,
“awards”: “”,
“authors”: [],
“foreignCountry”: “”
},
“keywords”: {
“fromName”: [
“Sun”,
“Golden Sun”
],
“other”: [],
“fromContent”: []
},
“name”: “Mr. Sun, Sun, Mr. Golden Sun”,
“resourceType”: “song”,
“mainActors”: [],
“contentAbstract”: “”,
“isSeries”: False,
“series”: {
“number”: 0,
“name”: “”
},
“fitAgeEnd”: 6,
“fileInfo”: {
“isAudio”: True,
“contentType”: “audio/mpeg”,
“name”: “Mr. Sun, Sun, Mr. Golden Sun.mp3”,
“suffix”: “mp3”
}
},
},
]
################################################################################
# Local Function
################################################################################
def initLogging():
“””
init logging
:return: log file name
“””
global gCfg
# init logging
filenameNoSufx = crifanLib.getInputFileBasenameNoSuffix()
logFilename = filenameNoSufx + “.log”
crifanLib.loggingInit(logFilename)
return logFilename
def saveFileToDb(curFullFilename, metadataDict):
“””
get more file meta info then save (text or audio) file into mongo db
:param curFullFilename:
:param metadataDict:
:return:
“””
resultDict = {
“hasSaved”: False,
“errMsg”: “Unknown error”
}
if not os.path.isfile(curFullFilename):
errMsg = “Not exist file for %s” % curFullFilename
logging.error(errMsg)
resultDict[“hasSaved”] = False
resultDict[“errMsg”] = errMsg
return resultDict
curFilename = crifanLib.getBasename(curFullFilename) #’Finger Family.mp3′
logging.info(“curFilename=%s”, curFilename)
metadataDict[“fileInfo”][“name”] = curFilename
fileSuffix = crifanLib.getFileSuffix(curFullFilename)
logging.info(“fileSuffix=%s”, fileSuffix)
metadataDict[“fileInfo”][“suffix”] = fileSuffix
# extract MIME
curFileMimeType = mime.Types.of(curFilename)[0].content_type
logging.info(“curFileMimeType=%s”, curFileMimeType) #’audio/mpeg’, ‘application/pdf’
metadataDict[“fileInfo”][“contentType”] = curFileMimeType
if re.match(“^audio/.+”, curFileMimeType):
metadataDict[“fileInfo”][“isAudio”] = True
with open(curFullFilename) as curFp:
curFileObjectId = fsCollection.put(
curFp,
filename=curFilename,
content_type=curFileMimeType,
metadata=metadataDict)
logging.info(“curFileObjectId=%s”, curFileObjectId)
# readOutFile = fsCollection.get(curFileObjectId)
# logging.info(“readOutFile=%s”, readOutFile)
# fileMedata = readOutFile.metadata
# logging.info(“fileMedata=%s”, fileMedata)
fileIdStr = str(curFileObjectId) #’5ac31bd2a4bc7156fd727d59′
resultDict[“hasSaved”] = True
resultDict[“errMsg”] = “”
logging.info(“+++ Saved file: id=%s, name=%s”, fileIdStr, curFilename)
return resultDict
def processFiles():
“””proces each file”””
curPath = os.getcwd()
for curIdx, curFileInfo in enumerate(FileInfoList):
curNum = curIdx + 1
curFilename = curFileInfo[“filename”]
logging.info(“-“*30 + ” [%d] %s ” + “-“*30, curNum, curFilename)
metadataDict = curFileInfo[“metadata”]
logging.info(“metadataDict=%s”, metadataDict)
curFullFilename = os.path.join(curPath, ToImportFileFolder, curFilename)
resultDict = saveFileToDb(curFullFilename, metadataDict)
logging.info(“resultDict=%s”, resultDict)
################################################################################
# Main Part
################################################################################
initLogging()
# init mongodb
mongoClient = MongoClient(mongodbUri)
logging.info(“mongoClient=%s”, mongoClient)
gridfsDb = mongoClient.gridfs
logging.info(“gridfsDb=%s”, gridfsDb)
fsCollection = GridFS(gridfsDb)
logging.info(“fsCollection=%s”, fsCollection)
processFiles()
效果：
<code>2018/07/23 02:19:25 LINE 375 INFO ------------------------------ [3] Mr. Sun, Sun, Mr. Golden Sun.mp3 ------------------------------
2018/07/23 02:19:25 LINE 378 INFO metadataDict={'song': {'singers': ['Super Simple Songs']}, 'series': {'number': 0, 'name': ''}, 'topics': [], 'storybook': {'publisher': '', 'isFiction': '\xe6\x9c\xaa\xe7\x9f\xa5', 'lexileIndex': '', 'awards': '', 'authors': [], 'foreignCountry': ''}, 'keywords': {'fromName': ['Sun', 'Golden Sun'], 'other': [], 'fromContent': []}, 'name': 'Mr. Sun, Sun, Mr. Golden Sun', 'resourceType': 'song', 'mainActors': [], 'contentAbstract': '', 'isSeries': False, 'fitAgeStart': 3, 'fitAgeEnd': 6, 'fileInfo': {'isAudio': True, 'contentType': 'audio/mpeg', 'name': 'Mr. Sun, Sun, Mr. Golden Sun.mp3', 'suffix': 'mp3'}}
2018/07/23 02:19:25 LINE 330 INFO curFilename=Mr. Sun, Sun, Mr. Golden Sun.mp3
2018/07/23 02:19:25 LINE 334 INFO fileSuffix=mp3
2018/07/23 02:19:25 LINE 339 INFO curFileMimeType=audio/mpeg
2018/07/23 02:19:25 LINE 351 INFO curFileObjectId=5b55736d7f4d38be4e02496c
2018/07/23 02:19:25 LINE 362 INFO +++ Saved file: id=5b55736d7f4d38be4e02496c, name=Mr. Sun, Sun, Mr. Golden Sun.mp3
2018/07/23 02:19:25 LINE 383 INFO resultDict={'hasSaved': True, 'errMsg': ''}
</code>系统中就可以选择了：
后续再去：
【未解决】Mongo中更新gridfs中的mp3文件的metadata信息且尽量保持id不变
转载请注明：在路上 » 【已解决】把本地mp3文件存入在线Mongo中且填写meta信息