fluentd couchdb elasticsearch kibana ubuntu 14.04

- port -
elasticsearch : 9200
couchdb : 5984


1. elasticsearch install

$ sudo apt-get install openjdk-7-jre-headless -y
$ sudo wget -O - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | apt-key add -


$ sudo vi /etc/apt/sources.list
  -> deb http://packages.elasticsearch.org/elasticsearch/1.3/debian stable main

$ sudo apt-get update
$ sudo apt-get install elasticsearch
$ sudo update-rc.d elasticsearch defaults 95 10
$ sudo /etc/init.d/elasticsearch start


2. fluentd install
$ curl -L http://toolbelt.treasuredata.com/sh/install-ubuntu-lucid.sh | sh
$ sudo /usr/lib/fluent/ruby/bin/fluent-gem install fluent-plugin-elasticsearch




3. fluentd power up
- multiprocess
  http://docs.fluentd.org/articles/in_multiprocess
- ruby memory fix
  https://github.com/kzk/jemalloc-rb

4. elasticsearch + couchdb

$ curl -XPUT 'http://localhost:9200/news/'
$ cd /usr/share/elasticsearch/
$ ./bin/plugin -install elasticsearch/elasticsearch-river-couchdb/1.2.0

 - date set time format, it is options
$ curl -XPUT localhost:9200/news/news/_mapping -d '{
    "news" : {
         "numeric_detection" : false,
          "properties" : {
               "creation" : {"format":"date_time_no_millis","type" : "date"}
        }
    }
}'
 - add river (it's connect couchdb)
$ curl -XPUT 'localhost:9200/_river/news/_meta' -d '{
     "type" : "couchdb",
     "couchdb" : {
          "host" : "localhost",
          "port" : 8888,
          "db" : "news",
          "filter" : null
     },
     "index" : {
          "index" : "news",
          "type" : "news",
          "bulk_size" : "100",
          "bulk_timeout" : "10ms"
     }
}'

- river delete
curl -XDELETE 'http://localhost:9200/_river/news/'

- news delete
curl -XDELETE 'http://localhost:9200/news/'


 - option
5. fluentd + geoip
$ sudo apt-get install libgeoip-dev
$ sudo /usr/lib/fluent/ruby/bin/fluent-gem install fluent-plugin-geoip

<match low.lighttpd.log>
     type copy
     <store>
          type    geoip
          geoip_lookup_key  host
     <record>
          city  ${city['host']}
          lat   ${latitude['host']}
          lon   ${longitude['host']}
     </record>
     remove_tag_prefix low.
     tag     couch.${tag}
     </store>
</match>

6. kibana download
http://www.elasticsearch.org/overview/kibana/

vi config.js
elasticsearch: "http://127.0.0.1:9200"