网站建设资讯

NEWS

网站建设资讯

TimelineServicev.2(HDP3.1)参数配置以及相关环境

Timeline Service v2 默认集成嵌入HBase(HMaster、HRegionServer),进程启动 User: yarn-ats

吉安ssl适用于网站、小程序/APP、API接口等需要进行数据传输应用场景,ssl证书未来市场广阔!成为成都创新互联公司的ssl证书销售渠道,可以享受市场价格4-6折优惠!如果有意向欢迎电话联系或者加微信:18982081108(备注:SSL证书合作)期待与您的合作!

  1. 嵌入HBase配置文件路径:
    [yarn-ats@master1 embedded-yarn-ats-hbase]$ pwd
    /usr/hdp/3.1.0.0-78/hadoop-yarn/conf/embedded-yarn-ats-hbase
    [yarn-ats@master1 embedded-yarn-ats-hbase]$ ls -la
    total 40
    drwxr-xr-x 2 yarn-ats hadoop 4096 3月   9 10:32 .
    drwxr-xr-x 4 root     root   4096 4月  23 19:32 ..
    -rw-r--r-- 1 yarn-ats hadoop 1462 3月   7 09:22 hadoop-metrics2-hbase.properties
    -rw-r--r-- 1 yarn-ats hadoop 3374 4月  23 20:41 hbase-env.sh
    -rw-r--r-- 1 yarn-ats hadoop  818 3月   7 09:22 hbase_grant_permissions.sh
    -rw-r--r-- 1 yarn-ats hadoop  411 3月   7 09:22 hbase-policy.xml
    -rw-r--r-- 1 yarn-ats hadoop 5394 3月   7 09:22 hbase-site.xml
    -rw-r--r-- 1 yarn-ats hadoop 5315 3月   7 09:22 log4j.properties
  2. hbase-site.xml文件

    
    
    
      dfs.domain.socket.path
      /var/lib/hadoop-hdfs/dn_socket
    
    
    
      hbase.bucketcache.ioengine
      
    
    
    
      hbase.bucketcache.percentage.in.combinedcache
      
    
    
    
      hbase.bucketcache.size
      
    
    
    
      hbase.client.keyvalue.maxsize
      1048576
    
    
    
      hbase.client.retries.number
      7
    
    
    
      hbase.client.scanner.caching
      100
    
    
    
      hbase.cluster.distributed
      true
    
    
    
      hbase.coprocessor.master.classes
      
    
    
    
      hbase.coprocessor.region.classes
      
    
    
    
      hbase.coprocessor.regionserver.classes
      
    
    
    
      hbase.defaults.for.version.skip
      true
    
    
    
      hbase.hregion.majorcompaction
      604800000
    
    
    
      hbase.hregion.majorcompaction.jitter
      0.50
    
    
    
      hbase.hregion.max.filesize
      10737418240
    
    
    
      hbase.hregion.memstore.block.multiplier
      4
    
    
    
      hbase.hregion.memstore.flush.size
      134217728
    
    
    
      hbase.hregion.memstore.mslab.enabled
      true
    
    
    
      hbase.hstore.blockingStoreFiles
      10
    
    
    
      hbase.hstore.compaction.max
      10
    
    
    
      hbase.hstore.compactionThreshold
      3
    
    
    
      hbase.local.dir
      ${hbase.tmp.dir}/local
    
    
    
      hbase.master.info.bindAddress
      0.0.0.0
    
    
    
      hbase.master.info.port
      17010
    
    
    
      hbase.master.namespace.init.timeout
      2400000
    
    
    
      hbase.master.port
      17000
    
    
    
      hbase.master.ui.readonly
      false
    
    
    
      hbase.master.wait.on.regionservers.timeout
      30000
    
    
    
      hbase.regionserver.executor.openregion.threads
      20
    
    
    
      hbase.regionserver.global.memstore.size
      0.4
    
    
    
      hbase.regionserver.handler.count
      30
    
    
    
      hbase.regionserver.info.port
      17030
    
    
    
      hbase.regionserver.port
      17020
    
    
    
      hbase.rootdir
      /atsv2/hbase/data
    
    
    
      hbase.rpc.protection
      authentication
    
    
    
      hbase.rpc.timeout
      90000
    
    
    
      hbase.security.authentication
      simple
    
    
    
      hbase.security.authorization
      false
    
    
    
      hbase.superuser
      yarn
    
    
    
      hbase.tmp.dir
      /tmp/hbase-${user.name}
    
    
    
      hbase.zookeeper.property.clientPort
      2181
    
    
    
      hbase.zookeeper.quorum
      manager.bigdata.com,master2.bigdata.com,master1.bigdata.com
    
    
    
      hbase.zookeeper.useMulti
      true
    
    
    
      hfile.block.cache.size
      0.4
    
    
    
      zookeeper.recovery.retry
      6
    
    
    
      zookeeper.session.timeout
      90000
    
    
    
      zookeeper.znode.parent
      /atsv2-hbase-unsecure
    
    
    
  3. Zookeeper Cluster znode信息

    [zookeeper@master1 zookeeper-client]$ bin/zkCli.sh -server master1.bigdata.com:2181
    [zk: master1.bigdata.com:2181(CONNECTED) 1] ls /atsv2-hbase-unsecure
    [replication, meta-region-server, rs, splitWAL, backup-masters, table-lock, flush-table-proc, master-maintenance, online-snapshot, switch, master, running, draining, namespace, hbaseid, table]
    [zk: master1.bigdata.com:2181(CONNECTED) 2] ls /atsv2-hbase-unsecure/rs
  4. UI Interface
    http://master1.bigdata.com:17010/
  5. Table list
    prod.timelineservice.app_flow
    prod.timelineservice.application
    prod.timelineservice.entity 
    prod.timelineservice.flowactivity
    prod.timelineservice.flowrun
    prod.timelineservice.subapplication 

    Timeline Service v.2 (HDP3.1 )参数配置以及相关环境
    TTL: 默认30天

  6. Hbase shell执行
    hbase --config /usr/hdp/3.1.0.0-78/hadoop-yarn/conf/embedded-yarn-ats-hbase shell
    hbase(main):004:0* status
    1 active master, 0 backup masters, 1 servers, 0 dead, 124.0000 average load
    Took 1.2255 seconds                                                                                                                                                    
    hbase(main):005:0> list
    TABLE                                                                                                                                                                  
    prod.timelineservice.app_flow                                                                                                                                 
    prod.timelineservice.application                                                                                                                                
    prod.timelineservice.entity                                                                                                                                       
    prod.timelineservice.flowactivity                                                                                                                               
    prod.timelineservice.flowrun                                                                                                                                      
    prod.timelineservice.subapplication                                                                                                                          
    6 row(s)
    Took 0.0862 seconds                                                                                                                                                    
    => ["prod.timelineservice.app_flow", "prod.timelineservice.application", "prod.timelineservice.entity", "prod.timelineservice.flowactivity", "prod.timelineservice.flowrun", "prod.timelineservice.subapplication"]
    hbase(main):018:0> scan 'prod.timelineservice.app_flow', { COLUMNS => ['m'], LIMIT => 10 }
  7. HDFS存储路径
    /atsv2/hbase

网站栏目:TimelineServicev.2(HDP3.1)参数配置以及相关环境
文章路径:http://cdweb.net/article/gdeisd.html