etcd 服务发现

2018-05-23

etcd 是 CoreOS 团队于 2013 年 6 月发起的开源项目，它的目标是构建一个高可用的分布式键值(key-value)数据库。etcd 内部采用 raft 协议作为一致性算法，etcd 基于 Go 语言实现。

下载安装
简单使用
- k/v 存储

etcd 作为服务发现系统，有以下的特点：

简单：安装配置简单，而且提供了HTTP API进行交互，使用也很简单
安全：支持SSL证书验证
快速：根据官方提供的benchmark数据，单实例支持每秒2k+读操作
可靠：采用raft算法，实现分布式系统数据的可用性和一致性

下载安装

首先从 releases 列表中查找最新版本并下载

1 2	$ sudo wget https://github.com/coreos/etcd/releases/download/v3.3.5/etcd-v3.3.5-linux-amd64.tar.gz $ tar -zxvf etcd-v3.3.5-linux-amd64.tar.gz

因为etcd是go语言编写的，安装只需要下载对应的二进制文件 etcd 和 etcdctl 放入环境变量中即可

1 2	$ cd etcd-v3.3.5-linux-amd64 $ sudo cp -R etcd* /usr/local/bin

运行

$ etcd

2018-05-23 16:16:09.403005 I | etcdmain: etcd Version: 3.3.5
2018-05-23 16:16:09.403104 I | etcdmain: Git SHA: GitNotFound
2018-05-23 16:16:09.403109 I | etcdmain: Go Version: go1.10.2
2018-05-23 16:16:09.403114 I | etcdmain: Go OS/Arch: darwin/amd64
2018-05-23 16:16:09.403122 I | etcdmain: setting maximum number of CPUs to 8, total number of available CPUs is 8
2018-05-23 16:16:09.403130 N | etcdmain: failed to detect default host (default host not supported on darwin_amd64)
2018-05-23 16:16:09.403140 W | etcdmain: no data-dir provided, using default data-dir ./default.etcd
2018-05-23 16:16:09.403670 I | embed: listening for peers on http://localhost:2380
2018-05-23 16:16:09.403854 I | embed: listening for client requests on localhost:2379
2018-05-23 16:16:09.405312 I | etcdserver: name = default
2018-05-23 16:16:09.405332 I | etcdserver: data dir = default.etcd
2018-05-23 16:16:09.405341 I | etcdserver: member dir = default.etcd/member
2018-05-23 16:16:09.405347 I | etcdserver: heartbeat = 100ms
2018-05-23 16:16:09.405354 I | etcdserver: election = 1000ms
2018-05-23 16:16:09.405360 I | etcdserver: snapshot count = 100000
2018-05-23 16:16:09.405386 I | etcdserver: advertise client URLs = http://localhost:2379
2018-05-23 16:16:09.405394 I | etcdserver: initial advertise peer URLs = http://localhost:2380
2018-05-23 16:16:09.405406 I | etcdserver: initial cluster = default=http://localhost:2380
2018-05-23 16:16:09.574156 I | etcdserver: starting member 8e9e05c52164694d in cluster cdf818194e3a8c32
2018-05-23 16:16:09.574211 I | raft: 8e9e05c52164694d became follower at term 0
2018-05-23 16:16:09.574238 I | raft: newRaft 8e9e05c52164694d [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2018-05-23 16:16:09.574244 I | raft: 8e9e05c52164694d became follower at term 1
2018-05-23 16:16:09.581795 W | auth: simple token is not cryptographically signed
2018-05-23 16:16:09.582432 I | etcdserver: starting server... [version: 3.3.5, cluster version: to_be_decided]
2018-05-23 16:16:09.582519 E | etcdserver: cannot monitor file descriptor usage (cannot get FDUsage on darwin)
2018-05-23 16:16:09.582534 I | etcdserver: 8e9e05c52164694d as single-node; fast-forwarding 9 ticks (election ticks 10)
2018-05-23 16:16:09.582946 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32
2018-05-23 16:16:10.581028 I | raft: 8e9e05c52164694d is starting a new election at term 1
2018-05-23 16:16:10.581069 I | raft: 8e9e05c52164694d became candidate at term 2
2018-05-23 16:16:10.581097 I | raft: 8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 2
2018-05-23 16:16:10.581114 I | raft: 8e9e05c52164694d became leader at term 2
2018-05-23 16:16:10.581123 I | raft: raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 2
2018-05-23 16:16:10.581325 I | etcdserver: setting up the initial cluster version to 3.3
2018-05-23 16:16:10.581489 I | embed: ready to serve client requests
2018-05-23 16:16:10.581623 I | etcdserver: published {Name:default ClientURLs:[http://localhost:2379]} to cluster cdf818194e3a8c32
2018-05-23 16:16:10.582287 N | embed: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged!
2018-05-23 16:16:10.594227 N | etcdserver/membership: set the initial cluster version to 3.3
2018-05-23 16:16:10.594332 I | etcdserver/api: enabled capabilities for version 3.3

etcd 目前默认使用2379端口提供HTTP API服务，2380端口和peer通信(这两个端口已经被IANA官方预留给etcd)

简单使用

etcd 提供了 etcdctl 命令行和 RESTful 风格的 HTTP 接口

k/v 存储

k/v 存储也是 Nginx upsync 模块中主要用的功能，用于灵活的存储

添加/修改

1
2
3

$ etcdctl set upstreams/127.0.0.1:8001 '{"server": "127.0.0.1:8001"}'

{"server": "127.0.0.1:8001"}

HTTP 接口方式会返回完整的数据结构

1
2
3

$ curl -X PUT http://127.0.0.1:2379/v2/keys/upstreams/127.0.0.1:8004 -d value='{"server": "120.0.0.1:8004"}'

{"action":"set","node":{"key":"/upstreams/127.0.0.1:8004","value":"{\"server\": \"120.0.0.1:8004\"}","modifiedIndex":5,"createdIndex":5}}

获取

1
2
3

$ etcdctl get upstreams/127.0.0.1:8001

{"server": "127.0.0.1:8001"}

1
2
3

$ curl http://127.0.0.1:2379/v2/keys/upstreams/127.0.0.1:8004

{"action":"set","node":{"key":"/upstreams/127.0.0.1:8004","value":"{\"server\": \"120.0.0.1:8004\"}","modifiedIndex":5,"createdIndex":5}}

upstreams/127.0.0.1:8001 作为 key，中间的 / 相当于在文件夹标示。HTTP 接口方式可以获取文件夹下的所有内容，这也是 upsync 模块中使用的方式

1
2
3

$ curl http://127.0.0.1:2379/v2/keys/upstreams/

{"action":"get","node":{"key":"/upstreams","dir":true,"nodes":[{"key":"/upstreams/127.0.0.1:8001","value":"{\"server\": \"127.0.0.1:8001\"}","modifiedIndex":4,"createdIndex":4},{"key":"/upstreams/127.0.0.1:8004","value":"{\"server\": \"120.0.0.1:8004\"}","modifiedIndex":5,"createdIndex":5}],"modifiedIndex":4,"createdIndex":4}}

删除

1
2
3

$ etcdctl rm upstreams/127.0.0.1:8001

PrevNode.Value: {"server": "127.0.0.1:8001"}

1
2
3

$ curl -X DELETE http://127.0.0.1:2379/v2/keys/upstreams/127.0.0.1:8004

{"action":"delete","node":{"key":"/upstreams/127.0.0.1:8004","modifiedIndex":7,"createdIndex":5},"prevNode":{"key":"/upstreams/127.0.0.1:8004","value":"{\"server\": \"120.0.0.1:8004\"}","modifiedIndex":5,"createdIndex":5}}

TTL

我们也可以给 key 设置一个有效期，单位为秒

1
2
3

$ etcdctl set /foo "Expiring Soon" --ttl 20

Expiring Soon

1
2
3

$ curl -X PUT http://127.0.0.1:2379/v2/keys/foo?ttl=20 -d value=bar

{"action":"set","node":{"key":"/foo","value":"bar","expiration":"2014-02-10T19:54:49.357382223Z","ttl":20,"modifiedIndex":31,"createdIndex":31}}

更多使用见文档

etcd使用入门