FluentdからCloudWatchのカスタムメトリクスにデータを登録してみた(UPDATE2!)

以前紹介した記事(「 FluentdからCloudWatchのカスタムメトリクスにデータを登録してみた(UPDATE!) 」)を
更にアップデートしてみました。

○変更点

• DataCounterの一つの集計ログからCloudWatchへ複数のデータを登録可能に
  • DataCounterは一つでよい(patternNで区別)
  • Dimensionをタグ(Tag)からキー(Key)に変更
    • KeyにはpatternNのキーを登録

○設定ファイル(/etc/td-agent/td-agent.conf)

 type     tail
 format   syslog
 path     /opt/suz-lab/var/log/syslog/all.log
 pos_file /opt/suz-lab/var/lib/td-agent/pos/tail.syslog.pos
 tag      tail.syslog


 type copy

  type file
  path /tmp/tail.syslog


  type      datacounter
  unit      minute
  aggregate all
  count_key ident
  pattern1  history ^-bash$
  pattern2  audit   ^audispd$
  tag       datacounter.syslog



 type copy

  type file
  path /tmp/datacounter.syslog


  type                 cloudwatch
  buffer_type          file
  buffer_path          /opt/suz-lab/var/lib/td-agent/buf/cloudwatch.datacounter.syslog
  flush_interval       1m
  cloud_watch_endpoint monitoring.ap-northeast-1.amazonaws.com
  namespace            SUZ-LAB/TEST

   key  history_count
   name SyslogCount
   unit Count


   key  audit_count
   name SyslogCount
   unit Count

○CloudWatchプラグイン(/etc/td-agent/plugin/out_cloudwatch.rb)

module Fluent
 require 'aws-sdk'
 class CloudWatchOutput 
  MAX_METRIC_DATA_SIZE = 20
  Fluent::Plugin.register_output('cloudwatch', self)
  config_param :aws_key_id,           :string, :default => nil
  config_param :aws_sec_key,          :string, :default => nil
  config_param :cloud_watch_endpoint, :string, :default => 'monitoring.ap-northeast-1.amazonaws.com'
  config_param :namespace,            :string

  def initialize
   super
   @metrics = []
  end

  def configure(conf)
   super
   conf.elements.select {|e|
    e.name == 'metric'
   }.each do |e|
    @metrics      'name' => e['name'],
     'key'  => e['key'],
     'unit' => e['unit']
    }
   end
  end

  def start
   super
   AWS.config(
    :access_key_id        => @aws_key_id,
    :secret_access_key    => @aws_sec_key,
    :cloud_watch_endpoint => @cloud_watch_endpoint
   )
  end

  def shutdown
   super
  end

  def format(tag, time, record)
   record[&guot;tag&guot;]       =  tag
   record[&guot;timestamp&guot;] =  Time.at(time).iso8601
   record.to_msgpack
  end

  def write(chunk)
   metric_data = []
   chunk.msgpack_each do |record|
    $log.debug(record.inspect)
    @metrics.each do |metric|
     tmp_data_by_key = {
      :metric_name => metric['name'],
      :timestamp   => record['timestamp'],
      :value       => record[metric['key']],
      :unit        => metric['unit'],
      :dimensions  => [ { :name  => 'Key', :value => metric['key'] } ]
     }
     metric_data      tmp_data_by_key_and_hostname = Marshal.load(Marshal.dump(tmp_data_by_key))
     tmp_data_by_key_and_hostname[:dimensions]       :name  => 'HostName', :value => Socket.gethostname
     }
     metric_data     end
   end
   $log.debug(metric_data.inspect)
   until metric_data.length     tmp_data = metric_data.slice!(0, MAX_METRIC_DATA_SIZE)
    $log.debug(tmp_data.inspect)
    AWS::CloudWatch.new.put_metric_data(
     :namespace   => @namespace,
     :metric_data => tmp_data
    )
   end
  end

 end
end

こちらの記事はなかの人（suz-lab）監修のもと掲載しています。
元記事は、こちら