成都网站建设设计

将想法与焦点和您一起共享

由正则表示式匹配($regex)引起的一次mongo数据库c-创新互联

某一天,监控到mongo数据库cpu使用率高了很多,查了一下,发现是下面这种语句引起的:

为临颍等地区用户提供了全套网页设计制作服务,及临颍网站建设行业解决方案。主营业务为网站建设、网站设计、临颍网站设计,以传统方式定制建设网站,并提供域名空间备案等一条龙服务,秉承以专业、用心的态度为用户提供真诚的服务。我们深信只要达到每一位用户的要求,就会得到认可,从而选择与我们长期合作。这样,我们也可以走得更远!
db.example_collection.find({ "idField" : { "$regex" : "123456789012345678"} , "dateField" : { "$regex" : "2019/10/10"}})

通常,遇到这种情况,我第一反应是缺少相关字段的索引,导致每执行一次这种语句都会全表扫描一次。

但是我用explain( )语句分析了下,发现上面所涉及的两个字段idField、dateField是有索引的,并且该语句也是有使用到索引的。如下为explain( )的结果:

mgset-11111111:PRIMARY> db.example_collection.find({ "idField" : { "$regex" : "123456789012345678"} , "dateField" : { "$regex" : "2019/10/10"}}).explain("queryPlanner")
{
"queryPlanner" : {
     "plannerVersion" : 1,
     "namespace" : "example_db.example_collection",
     "indexFilterSet" : false,
     "parsedQuery" : {
             "$and" : [
                     {
                             "idField" : {
                                     "$regex" : "123456789012345678"
                             }
                     },
                     {
                             "dateField" : {
                                     "$regex" : "2019/10/10"
                             }
                     }
             ]
     },
     "winningPlan" : {
             "stage" : "FETCH",
             "inputStage" : {
                     "stage" : "IXSCAN",
                     "filter" : {
                             "$and" : [
                                     {
                                             "idField" : {
                                                     "$regex" : "123456789012345678"
                                             }
                                     },
                                     {
                                             "dateField" : {
                                                     "$regex" : "2019/10/10"
                                             }
                                     }
                             ]
                     },
                     "keyPattern" : {
                             "idField" : 1,
                             "dateField" : 1
                     },
                     "indexName" : "idField_1_dateField_1",
                     "isMultiKey" : false,
                     "multiKeyPaths" : {
                             "idField" : [ ],
                             "dateField" : [ ]
                     },
                     "isUnique" : false,
                     "isSparse" : false,
                     "isPartial" : false,
                     "indexVersion" : 2,
                     "direction" : "forward",
                     "indexBounds" : {
                             "idField" : [
                                     "[\"\", {})",
                                     "[/123456789012345678/, /123456789012345678/]"
                             ],
                             "dateField" : [
                                     "[\"\", {})",
                                     "[/2019/10/10/, /2019/10/10/]"
                             ]
                     }
             }
     },
     "rejectedPlans" : [ ]
},
"ok" : 1
}

查看mongo的日志发现,这种语句执行一次就要800~900ms,的确是比较慢。除非数据库cpu核数很多,要不然只要这种语句每秒并发稍微高一点,cpu很快就被占满了。

之后搜索了下,发现有可能是正则表达式的问题。原来,虽然该语句的确是使用了索引,但是explain( )语句的输出中还有一个字段"indexBounds",表示执行该语句时所需扫描的索引范围。说实话,上面那个输出中,我始终没看明白它那个索引范围。上面的语句对idField、dateField这两个字段都进行了普通的正则表达式匹配,我猜测它应该是扫描了整个索引树,所以导致索引并未实际提升该语句的查询效率。

我看了下数据库里面的数据,发现idField、dateField这两个字段完全没有必要进行正则匹配,进行普通的文本匹配就行。将正则匹配操作$regex去掉之后,再分析一下,结果是这样的:

mgset-11111111:PRIMARY> db.example_collection.find({ "idField" : "123456789012345678", "dateField" : "2019/10/10"}).explain("queryPlanner")
{
"queryPlanner" : {
     "plannerVersion" : 1,
     "namespace" : "example_db.example_collection",
     "indexFilterSet" : false,
     "parsedQuery" : {
             "$and" : [
                     {
                             "idField" : {
                                     "$eq" : "123456789012345678"
                             }
                     },
                     {
                             "dateField" : {
                                     "$eq" : "2019/10/10"
                             }
                     }
             ]
     },
     "winningPlan" : {
             "stage" : "FETCH",
             "inputStage" : {
                     "stage" : "IXSCAN",
                     "keyPattern" : {
                             "idField" : 1,
                             "dateField" : 1
                     },
                     "indexName" : "idField_1_dateField_1",
                     "isMultiKey" : false,
                     "multiKeyPaths" : {
                             "idField" : [ ],
                             "dateField" : [ ]
                     },
                     "isUnique" : false,
                     "isSparse" : false,
                     "isPartial" : false,
                     "indexVersion" : 2,
                     "direction" : "forward",
                     "indexBounds" : {
                             "idField" : [
                                     "[\"123456789012345678\", \"123456789012345678\"]"
                             ],
                             "dateField" : [
                                     "[\"2019/10/10\", \"2019/10/10\"]"
                             ]
                     }
             }
     },
     "rejectedPlans" : [ ]
},
"ok" : 1
}

可以看到,仍然使用到了索引,并且索引扫描范围是仅限于一个值的。

后来跟开发人员确认了下,该语句确实没必要使用正则匹配,就让他把正则匹配去掉了。之后就没有再出现问题了,mongo慢日志中也未再出现该语句。

另外有需要云服务器可以了解下创新互联建站www.cdcxhl.com,海内外云服务器15元起步,三天无理由+7*72小时售后在线,公司持有idc许可证,提供“云服务器、裸金属服务器、高防服务器、香港服务器、美国服务器、虚拟主机、免备案服务器”等云主机租用服务以及企业上云的综合解决方案,具有“安全稳定、简单易用、服务可用性高、性价比高”等特点与优势,专为企业上云打造定制,能够满足用户丰富、多元化的应用场景需求。


名称栏目:由正则表示式匹配($regex)引起的一次mongo数据库c-创新互联
当前网址:http://chengdu.cdxwcx.cn/article/pgjdp.html