大数据分类利器---支持向量机(SVM)入门介绍

2015-10-25 MedSci MedSci原创

什么是分类分析? 让我们思考一个例子。假设某总体50%是男性,50% 是女性。你想要通过一个样本(集)获取一些规则,以确定总体其余部分成员的性别。使用支持向量机,相当于建立一个判断某人是男是女的机器。这是个典型的分类问题。我们试图通过一些规则,把总体分为两个可能的部分。出于简便考虑,这里假设两个用来区分的要素为:身高、头发长度。样本集的散点图如下: 图中的蓝色圆点代表女性,绿色方块代表男性

什么是分类分析? 让我们思考一个例子。假设某总体50%是男性,50% 是女性。你想要通过一个样本(集)获取一些规则,以确定总体其余部分成员的性别。使用支持向量机,相当于建立一个判断某人是男是女的机器。这是个典型的分类问题。我们试图通过一些规则,把总体分为两个可能的部分。出于简便考虑,这里假设两个用来区分的要素为:身高、头发长度。样本集的散点图如下: 图中的蓝色圆点代表女性,绿色方块代表男性。从图中可以推测: 1.总体中的男性平均身高更高 2.总体中的女性头发更长 如果某人身高180cm,头发4cm长,我们更可能将其判定为男性。这就是做分类分析的方式。 支持向量是什么?支持向量机(SVM)是什么? 支持向量就是单条记录的坐标。比如(45,150) 是对应一名女性的支持向量。支持向量机是能够区分男性和女性的最佳边界。在本例中,两个类别相隔清晰,所以更容易找到支持向量机。 如何确定当前案例的支持向量机? 当前的案例有很多可能的边界。下面是三种可能情况: 我们要如何确定哪条是最佳的边界? 要解释支持向量机目标函数(objective function)最简单的方法,是寻找与边界最近的支持向量(

版权声明:
本网站所有内容来源注明为“梅斯医学”或“MedSci原创”的文字、图片和音视频资料,版权均属于梅斯医学所有。非经授权,任何媒体、网站或个人不得转载,授权转载时须注明来源为“梅斯医学”。其它来源的文章系转载文章,或“梅斯号”自媒体发布的文章,仅系出于传递更多信息之目的,本站仅负责审核内容合规,其内容不代表本站立场,本站不负责内容的准确性和版权。如果存在侵权、或不希望被转载的媒体或个人可与我们联系,我们将立即进行删除处理。
在此留言
评论区 (6)
#插入话题
  1. [GetPortalCommentsPageByObjectIdResponse(id=1983967, encodeId=73c1198396e6f, content=<a href='/topic/show?id=cf795e0762b' target=_blank style='color:#2F92EE;'>#支持向量机#</a>, beContent=null, objectType=article, channel=null, level=null, likeNumber=28, replyNumber=0, topicName=null, topicId=null, topicList=[TopicDto(id=57076, encryptionId=cf795e0762b, topicName=支持向量机)], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=4dfa37, createdName=智者为医08, createdTime=Wed Mar 09 08:15:00 CST 2016, time=2016-03-09, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41311, encodeId=96fd413110b, content=不明觉厉, beContent=null, objectType=article, channel=null, level=null, likeNumber=97, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://thirdwx.qlogo.cn/mmopen/vi_32/Q0j4TwGTfTJXCR8W80NNgQuicJU3aIm3JtiaNAFLYSMlBuVfGkSlw5Z5VpmZK4PyiaDURLAorhvwhWVnaM1gQdJXA/132, createdBy=a01444383, createdName=lixh1719, createdTime=Mon Oct 26 11:14:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41296, encodeId=4dca41296ee, content=好复杂,看不明白, beContent=null, objectType=article, channel=null, level=null, likeNumber=113, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=90ea1614944, createdName=huangwukui, createdTime=Mon Oct 26 09:01:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41290, encodeId=96364129092, content=可以看看, beContent=null, objectType=article, channel=null, level=null, likeNumber=75, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://cacheapi.medsci.cn/resource/upload/20151021/IMG5627B03B7A49C7641.jpg, createdBy=cc6e1657094, createdName=风中云追忆, createdTime=Mon Oct 26 08:31:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41283, encodeId=bc4d4128353, content=好复杂, beContent=null, objectType=article, channel=null, level=null, likeNumber=159, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=2c0879410, createdName=m510429, createdTime=Mon Oct 26 06:29:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41260, encodeId=cc324126000, content=朕, beContent=null, objectType=article, channel=null, level=null, likeNumber=105, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://cacheapi.medsci.cn/resource/upload/20150904/IMG55E8DB68F05883098.jpg, createdBy=e00a1631985, createdName=fengzhigu, createdTime=Sun Oct 25 23:48:00 CST 2015, time=2015-10-25, status=1, ipAttribution=)]
  2. [GetPortalCommentsPageByObjectIdResponse(id=1983967, encodeId=73c1198396e6f, content=<a href='/topic/show?id=cf795e0762b' target=_blank style='color:#2F92EE;'>#支持向量机#</a>, beContent=null, objectType=article, channel=null, level=null, likeNumber=28, replyNumber=0, topicName=null, topicId=null, topicList=[TopicDto(id=57076, encryptionId=cf795e0762b, topicName=支持向量机)], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=4dfa37, createdName=智者为医08, createdTime=Wed Mar 09 08:15:00 CST 2016, time=2016-03-09, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41311, encodeId=96fd413110b, content=不明觉厉, beContent=null, objectType=article, channel=null, level=null, likeNumber=97, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://thirdwx.qlogo.cn/mmopen/vi_32/Q0j4TwGTfTJXCR8W80NNgQuicJU3aIm3JtiaNAFLYSMlBuVfGkSlw5Z5VpmZK4PyiaDURLAorhvwhWVnaM1gQdJXA/132, createdBy=a01444383, createdName=lixh1719, createdTime=Mon Oct 26 11:14:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41296, encodeId=4dca41296ee, content=好复杂,看不明白, beContent=null, objectType=article, channel=null, level=null, likeNumber=113, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=90ea1614944, createdName=huangwukui, createdTime=Mon Oct 26 09:01:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41290, encodeId=96364129092, content=可以看看, beContent=null, objectType=article, channel=null, level=null, likeNumber=75, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://cacheapi.medsci.cn/resource/upload/20151021/IMG5627B03B7A49C7641.jpg, createdBy=cc6e1657094, createdName=风中云追忆, createdTime=Mon Oct 26 08:31:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41283, encodeId=bc4d4128353, content=好复杂, beContent=null, objectType=article, channel=null, level=null, likeNumber=159, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=2c0879410, createdName=m510429, createdTime=Mon Oct 26 06:29:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41260, encodeId=cc324126000, content=朕, beContent=null, objectType=article, channel=null, level=null, likeNumber=105, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://cacheapi.medsci.cn/resource/upload/20150904/IMG55E8DB68F05883098.jpg, createdBy=e00a1631985, createdName=fengzhigu, createdTime=Sun Oct 25 23:48:00 CST 2015, time=2015-10-25, status=1, ipAttribution=)]
    2015-10-26 lixh1719

    不明觉厉

    0

  3. [GetPortalCommentsPageByObjectIdResponse(id=1983967, encodeId=73c1198396e6f, content=<a href='/topic/show?id=cf795e0762b' target=_blank style='color:#2F92EE;'>#支持向量机#</a>, beContent=null, objectType=article, channel=null, level=null, likeNumber=28, replyNumber=0, topicName=null, topicId=null, topicList=[TopicDto(id=57076, encryptionId=cf795e0762b, topicName=支持向量机)], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=4dfa37, createdName=智者为医08, createdTime=Wed Mar 09 08:15:00 CST 2016, time=2016-03-09, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41311, encodeId=96fd413110b, content=不明觉厉, beContent=null, objectType=article, channel=null, level=null, likeNumber=97, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://thirdwx.qlogo.cn/mmopen/vi_32/Q0j4TwGTfTJXCR8W80NNgQuicJU3aIm3JtiaNAFLYSMlBuVfGkSlw5Z5VpmZK4PyiaDURLAorhvwhWVnaM1gQdJXA/132, createdBy=a01444383, createdName=lixh1719, createdTime=Mon Oct 26 11:14:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41296, encodeId=4dca41296ee, content=好复杂,看不明白, beContent=null, objectType=article, channel=null, level=null, likeNumber=113, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=90ea1614944, createdName=huangwukui, createdTime=Mon Oct 26 09:01:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41290, encodeId=96364129092, content=可以看看, beContent=null, objectType=article, channel=null, level=null, likeNumber=75, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://cacheapi.medsci.cn/resource/upload/20151021/IMG5627B03B7A49C7641.jpg, createdBy=cc6e1657094, createdName=风中云追忆, createdTime=Mon Oct 26 08:31:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41283, encodeId=bc4d4128353, content=好复杂, beContent=null, objectType=article, channel=null, level=null, likeNumber=159, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=2c0879410, createdName=m510429, createdTime=Mon Oct 26 06:29:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41260, encodeId=cc324126000, content=朕, beContent=null, objectType=article, channel=null, level=null, likeNumber=105, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://cacheapi.medsci.cn/resource/upload/20150904/IMG55E8DB68F05883098.jpg, createdBy=e00a1631985, createdName=fengzhigu, createdTime=Sun Oct 25 23:48:00 CST 2015, time=2015-10-25, status=1, ipAttribution=)]
    2015-10-26 huangwukui

    好复杂,看不明白

    0

  4. [GetPortalCommentsPageByObjectIdResponse(id=1983967, encodeId=73c1198396e6f, content=<a href='/topic/show?id=cf795e0762b' target=_blank style='color:#2F92EE;'>#支持向量机#</a>, beContent=null, objectType=article, channel=null, level=null, likeNumber=28, replyNumber=0, topicName=null, topicId=null, topicList=[TopicDto(id=57076, encryptionId=cf795e0762b, topicName=支持向量机)], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=4dfa37, createdName=智者为医08, createdTime=Wed Mar 09 08:15:00 CST 2016, time=2016-03-09, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41311, encodeId=96fd413110b, content=不明觉厉, beContent=null, objectType=article, channel=null, level=null, likeNumber=97, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://thirdwx.qlogo.cn/mmopen/vi_32/Q0j4TwGTfTJXCR8W80NNgQuicJU3aIm3JtiaNAFLYSMlBuVfGkSlw5Z5VpmZK4PyiaDURLAorhvwhWVnaM1gQdJXA/132, createdBy=a01444383, createdName=lixh1719, createdTime=Mon Oct 26 11:14:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41296, encodeId=4dca41296ee, content=好复杂,看不明白, beContent=null, objectType=article, channel=null, level=null, likeNumber=113, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=90ea1614944, createdName=huangwukui, createdTime=Mon Oct 26 09:01:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41290, encodeId=96364129092, content=可以看看, beContent=null, objectType=article, channel=null, level=null, likeNumber=75, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://cacheapi.medsci.cn/resource/upload/20151021/IMG5627B03B7A49C7641.jpg, createdBy=cc6e1657094, createdName=风中云追忆, createdTime=Mon Oct 26 08:31:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41283, encodeId=bc4d4128353, content=好复杂, beContent=null, objectType=article, channel=null, level=null, likeNumber=159, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=2c0879410, createdName=m510429, createdTime=Mon Oct 26 06:29:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41260, encodeId=cc324126000, content=朕, beContent=null, objectType=article, channel=null, level=null, likeNumber=105, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://cacheapi.medsci.cn/resource/upload/20150904/IMG55E8DB68F05883098.jpg, createdBy=e00a1631985, createdName=fengzhigu, createdTime=Sun Oct 25 23:48:00 CST 2015, time=2015-10-25, status=1, ipAttribution=)]
    2015-10-26 风中云追忆

    可以看看

    0

  5. [GetPortalCommentsPageByObjectIdResponse(id=1983967, encodeId=73c1198396e6f, content=<a href='/topic/show?id=cf795e0762b' target=_blank style='color:#2F92EE;'>#支持向量机#</a>, beContent=null, objectType=article, channel=null, level=null, likeNumber=28, replyNumber=0, topicName=null, topicId=null, topicList=[TopicDto(id=57076, encryptionId=cf795e0762b, topicName=支持向量机)], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=4dfa37, createdName=智者为医08, createdTime=Wed Mar 09 08:15:00 CST 2016, time=2016-03-09, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41311, encodeId=96fd413110b, content=不明觉厉, beContent=null, objectType=article, channel=null, level=null, likeNumber=97, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://thirdwx.qlogo.cn/mmopen/vi_32/Q0j4TwGTfTJXCR8W80NNgQuicJU3aIm3JtiaNAFLYSMlBuVfGkSlw5Z5VpmZK4PyiaDURLAorhvwhWVnaM1gQdJXA/132, createdBy=a01444383, createdName=lixh1719, createdTime=Mon Oct 26 11:14:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41296, encodeId=4dca41296ee, content=好复杂,看不明白, beContent=null, objectType=article, channel=null, level=null, likeNumber=113, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=90ea1614944, createdName=huangwukui, createdTime=Mon Oct 26 09:01:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41290, encodeId=96364129092, content=可以看看, beContent=null, objectType=article, channel=null, level=null, likeNumber=75, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://cacheapi.medsci.cn/resource/upload/20151021/IMG5627B03B7A49C7641.jpg, createdBy=cc6e1657094, createdName=风中云追忆, createdTime=Mon Oct 26 08:31:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41283, encodeId=bc4d4128353, content=好复杂, beContent=null, objectType=article, channel=null, level=null, likeNumber=159, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=2c0879410, createdName=m510429, createdTime=Mon Oct 26 06:29:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41260, encodeId=cc324126000, content=朕, beContent=null, objectType=article, channel=null, level=null, likeNumber=105, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://cacheapi.medsci.cn/resource/upload/20150904/IMG55E8DB68F05883098.jpg, createdBy=e00a1631985, createdName=fengzhigu, createdTime=Sun Oct 25 23:48:00 CST 2015, time=2015-10-25, status=1, ipAttribution=)]
    2015-10-26 m510429

    好复杂

    0

  6. [GetPortalCommentsPageByObjectIdResponse(id=1983967, encodeId=73c1198396e6f, content=<a href='/topic/show?id=cf795e0762b' target=_blank style='color:#2F92EE;'>#支持向量机#</a>, beContent=null, objectType=article, channel=null, level=null, likeNumber=28, replyNumber=0, topicName=null, topicId=null, topicList=[TopicDto(id=57076, encryptionId=cf795e0762b, topicName=支持向量机)], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=4dfa37, createdName=智者为医08, createdTime=Wed Mar 09 08:15:00 CST 2016, time=2016-03-09, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41311, encodeId=96fd413110b, content=不明觉厉, beContent=null, objectType=article, channel=null, level=null, likeNumber=97, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://thirdwx.qlogo.cn/mmopen/vi_32/Q0j4TwGTfTJXCR8W80NNgQuicJU3aIm3JtiaNAFLYSMlBuVfGkSlw5Z5VpmZK4PyiaDURLAorhvwhWVnaM1gQdJXA/132, createdBy=a01444383, createdName=lixh1719, createdTime=Mon Oct 26 11:14:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41296, encodeId=4dca41296ee, content=好复杂,看不明白, beContent=null, objectType=article, channel=null, level=null, likeNumber=113, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=90ea1614944, createdName=huangwukui, createdTime=Mon Oct 26 09:01:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41290, encodeId=96364129092, content=可以看看, beContent=null, objectType=article, channel=null, level=null, likeNumber=75, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://cacheapi.medsci.cn/resource/upload/20151021/IMG5627B03B7A49C7641.jpg, createdBy=cc6e1657094, createdName=风中云追忆, createdTime=Mon Oct 26 08:31:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41283, encodeId=bc4d4128353, content=好复杂, beContent=null, objectType=article, channel=null, level=null, likeNumber=159, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=, createdBy=2c0879410, createdName=m510429, createdTime=Mon Oct 26 06:29:00 CST 2015, time=2015-10-26, status=1, ipAttribution=), GetPortalCommentsPageByObjectIdResponse(id=41260, encodeId=cc324126000, content=朕, beContent=null, objectType=article, channel=null, level=null, likeNumber=105, replyNumber=0, topicName=null, topicId=null, topicList=[], attachment=null, authenticateStatus=null, createdAvatar=http://cacheapi.medsci.cn/resource/upload/20150904/IMG55E8DB68F05883098.jpg, createdBy=e00a1631985, createdName=fengzhigu, createdTime=Sun Oct 25 23:48:00 CST 2015, time=2015-10-25, status=1, ipAttribution=)]
    2015-10-25 fengzhigu

    0

相关资讯

符合美国HIPAA法案的医疗健康大数据服务公司TrueVault

TrueVault是一家总部位于美国旧金山的医疗初创公司,由Jason Wang 和Trey Swann于2013年创立。因为在2013年9月23日,美国的立法机构强制通过HIPAA法案,要求医疗健康提供商在与商业伙伴合作的情况下,必须严格遵守相关隐私法规,该政策的出台一下助力打通了商业渠道,而两人也抓住了这一政策造就的机遇,意欲将公司发展为完全符合HIPAA法案的医疗健康数据服务提供商。

医疗健康大数据: 应用实例与系统分析

随着信息技术和物联网技术的发展、个人电脑和智能手机的普及以及社交网络的兴起,人类活动产生的数据正以惊人的速度增长。根据国际数据公司(International DataCorporation,IDC)的报告,仅2011年,全世界产生的数据就有1.8 ZB(1 ZB≈1 021 byte),并且平均

大数据研究常用软件工具与应用场景

如今,大数据日益成为研究行业的重要研究目标。面对其高数据量、多维度与异构化的特点,以及分析方法思路的扩展,传统统计工具已经难以应对。工欲善其事,必先利其器。众多新的软件分析工具作为深入大数据洞察研究的重要助力, 也成为数据科学家所必须掌握的知识技能。然而,现实情况的复杂性决定了并不存在解决一切问题的终极工具。实际研究过程中,需要根据实际情况灵活选择最合适的工具(甚至多种工具组合使用),才能更好

大数据技术人员的工具包汇总(下)

接上一部分:大数据技术人员的工具包汇总(上),第二部分主要收集整理的内容主要有日志收集系统、消息系统、分布式服务、集群管理、RPC、基础设施、搜索引擎、Iaas和监控管理等大数据开源工具。 第九类:日志收集系统 一、Facebook Scribe 贡献者:Facebook 简介:Scribe是Facebook开源的日志收集系统,在Facebook内部已经得到大量的应用

大数据技术人员的工具包汇总(上)

第一类:查询引擎 一、Phoenix 贡献者::Salesforce 简介:这是一个Java中间层,可以让开发者在Apache HBase上执行SQL查询。Phoenix完全使用Java编写,代码位于GitHub上,并且提供了一个客户端可嵌入的JDBC驱动。 Phoenix查询引擎会将SQL查询转换为一个或多个HBase scan,并编排执行以生成标准的JDBC结果集。直接使用HBas

大数据技术人员的工具包——开源大数据处理工具

下面是文档简版,收集整理者为Aquester详细信息,请关注MedSci官方微信,然后输入“大数据工具”,可以直接下载详细的工具包表格关注微信: 类别 名称 官网 备注 查询引擎 Phoenix http://phoenix.incubator.apac