How to Set Up and Use Hashicorp's Vault and Consul Projects

The following tutorial details how to set up and use Hashicorp's Vault and Consul projects to securely store and manage secrets.

Source: https://testdriven.io

#vault #consul 

How to Set Up and Use Hashicorp's Vault and Consul Projects
Thai  Son

Thai Son

1660640700

Cách Thiết Lập Và Sử Dụng Các Dự Án Vault Và Consul Của Hashicorp

Hướng dẫn sau đây trình bày chi tiết cách thiết lập và sử dụng các dự án Vault vàConsul của Hashicorp để lưu trữ và quản lý bí mật một cách an toàn.

Chúng tôi sẽ bắt đầu bằng cách xoay một phiên bản Vault duy nhất trong vùng chứa Docker và sau đó chuyển sang quản lý cả bí mật tĩnh và động cùng với tính năng "mã hóa dưới dạng dịch vụ" của Vault. Sau đó, chúng tôi sẽ thêm Consul vào hỗn hợp và xem xét cách mở rộng quy mô Vault.

Phụ thuộc chính:

  • Docker v20.10.8
  • Docker-Compose v1.29.2
  • Vault v1.8.2
  • Consul v1.10.2

Mục tiêu

Đến cuối hướng dẫn này, bạn sẽ có thể:

  1. Giải thích Vault là gì và tại sao bạn có thể muốn sử dụng nó
  2. Mô tả kiến ​​trúc Vault cơ bản cùng với các bí mật động và tĩnh, các phụ trợ khác nhau (lưu trữ, bí mật, xác thực, kiểm tra) và cách Vault có thể được sử dụng như một "mã hóa như một dịch vụ"
  3. Định cấu hình và chạy Vault and Consul với Docker
  4. Tăng tốc Vault với chương trình phụ trợ Hệ thống tập tin
  5. Init và unseal Vault
  6. Xác thực chống lại Vault
  7. Định cấu hình chương trình phụ trợ Kiểm tra để ghi lại tất cả các tương tác với Vault
  8. Làm việc với các bí mật tĩnh và động thông qua CLI, API HTTP và giao diện người dùng
  9. Tạo chính sách Vault để giới hạn quyền truy cập vào một đường dẫn cụ thể
  10. Sử dụng phần phụ trợ Chuyển tuyến làm "mã hóa dưới dạng dịch vụ"
  11. Thiết lập Consul để làm việc với Vault dưới dạng phụ trợ Storage để biết bí mật
  12. Xác định thời hạn thuê tùy chỉnh cho một bí mật và thu hồi một bí mật trước khi kết thúc thời hạn đó

Vault là gì?

Vault là một công cụ mã nguồn mở được sử dụng để lưu trữ và quản lý các bí mật một cách an toàn.

What is a secret? Secrets, in the context of this tutorial, are securely-sensitive or personally identifiable info like database credentials, SSH keys, usernames and passwords, AWS IAM credentials, API tokens, Social Security Numbers, credit card numbers, just to name a few.

Take a moment to think about how your team currently manages and distributes secrets:

  1. Who has access to them?
  2. Who manages them?
  3. How do you control who has access to them?
  4. How do your apps get them?
  5. How are they updated?
  6. How are they revoked?

Vault provides answers to those questions and helps to solve the following problems with regard to secret management:

ProblemsVault's Goals
Secrets are everywhere.Vault is the single source of truth for all secrets.
They are generally unencrypted.Vault manages encryption (during transit and at rest) out of the box.
It's difficult to dynamically generate them.Secrets can be dynamically generated.
It's even more difficult to lease and revoke them.Secrets can be leased and revoked.
There's no audit trail.There's an audit trail for generating and using secrets.

Vault has a number of moving pieces so it can take some time to get up to speed with the overall architecture. Take a moment to review the Architecture guide, taking note of the following backends:

BackendUseExamples
StorageWhere secrets are storedConsul*, Filesystem*, In-Memory, PostgreSQL, S3
SecretHandles static or dynamic secretsAWS*, Databases, Key/Value*, RabbitMQ, SSH
AuthHandles authentication and authorizationAWS, Azure, Google Cloud, GitHub, Tokens*, Username & Password
AuditLogs all requests and responsesFile*, Syslog, Socket

* used in this tutorial

With that, let's start using Vault.

Filesystem Backend

To get up and running quickly, we'll use the Filesystem backend to store secrets at rest.

The filesystem backend should only be used for local development or a single-server Vault deployment since it does not support high availability.

Create a new project directory:

$ mkdir vault-consul-docker && cd vault-consul-docker

Then add the following folders:

└── vault
    ├── config
    ├── data
    ├── logs
    └── policies

Add a Dockerfile to the "vault" directory:

# base image
FROM alpine:3.14

# set vault version
ENV VAULT_VERSION 1.8.2

# create a new directory
RUN mkdir /vault

# download dependencies
RUN apk --no-cache add \
      bash \
      ca-certificates \
      wget

# download and set up vault
RUN wget --quiet --output-document=/tmp/vault.zip https://releases.hashicorp.com/vault/${VAULT_VERSION}/vault_${VAULT_VERSION}_linux_amd64.zip && \
    unzip /tmp/vault.zip -d /vault && \
    rm -f /tmp/vault.zip && \
    chmod +x /vault

# update PATH
ENV PATH="PATH=$PATH:$PWD/vault"

# add the config file
COPY ./config/vault-config.json /vault/config/vault-config.json

# expose port 8200
EXPOSE 8200

# run vault
ENTRYPOINT ["vault"]

Next, add a docker-compose.yml file to the project root:

version: '3.8'

services:

  vault:
    build:
      context: ./vault
      dockerfile: Dockerfile
    ports:
      - 8200:8200
    volumes:
      - ./vault/config:/vault/config
      - ./vault/policies:/vault/policies
      - ./vault/data:/vault/data
      - ./vault/logs:/vault/logs
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_API_ADDR=http://127.0.0.1:8200
    command: server -config=/vault/config/vault-config.json
    cap_add:
      - IPC_LOCK

Add a config file called vault-config.json to "vault/config":

{
  "backend": {
    "file": {
      "path": "vault/data"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Here, we configured Vault to use the Filesystem backend, defined the listener for Vault, disabled TLS, and enabled the Vault UI. Review the docs for more info on configuring Vault.

Now we can build the image and spin up the container:

$ docker-compose up -d --build

Pull up the Docker logs to make sure there were no errors in the build:

$ docker-compose logs

You should see something similar to:

Attaching to vault-consul-docker_vault_1
vault_1  | ==> Vault server configuration:
vault_1  |
vault_1  |              Api Address: http://127.0.0.1:8200
vault_1  | 2021-09-08T14:48:35.014Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
vault_1  |                      Cgo: disabled
vault_1  |          Cluster Address: https://127.0.0.1:8201
vault_1  |               Go Version: go1.16.7
vault_1  |               Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
vault_1  |                Log Level: info
vault_1  |                    Mlock: supported: true, enabled: true
vault_1  |            Recovery Mode: false
vault_1  |                  Storage: file
vault_1  |                  Version: Vault v1.8.2
vault_1  |              Version Sha: aca76f63357041a43b49f3e8c11d67358496959f
vault_1  |
vault_1  | ==> Vault server started! Log data will stream in below:
vault_1  |

Initializing and Unsealing

Start a bash session within the running container:

$ docker-compose exec vault bash

Within the shell, initialize Vault:

bash-5.1# vault operator init

Take note of the unseal keys and the initial root token. You will need to provide three of the unseal keys every time the Vault server is resealed or restarted.

Why 3 keys? Review Shamir's Secret Sharing.

Now you can unseal Vault using three of the keys:

bash-5.1# vault operator unseal
Unseal Key (will be hidden):

Run this command two more times, using different keys each time. Once done, make sure Sealed is false:

Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    5
Threshold       3
Version         1.8.2
Storage Type    file
Cluster Name    vault-cluster-8fcf9d05
Cluster ID      d86e0274-ad9c-d2c1-d6ec-baeab410797b
HA Enabled      false

Using the root token, you can now authenticate:

bash-5.1# vault login
Token (will be hidden):

You should see something similar to:

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                s.c0kYHWiOTqQvtR8JuSeTz6sZ
token_accessor       3FQJVxOY5C1brzlHHQSFaCdZ
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Keep in mind that this uses the root policy. In production you'll want to set up policies with different levels of access. We'll look at how to do this shortly.

vault init

Vault is now unsealed and ready for use.

Auditing

Before we test out the functionality, let's enable an Audit Device:

bash-5.1# vault audit enable file file_path=/vault/logs/audit.log

Success! Enabled the file audit device at: file/

You should now be able to view the logs locally in "vault/logs". To test, run the following command to view all enabled Audit Devices:

bash-5.1# vault audit list

Path     Type    Description
----     ----    -----------
file/    file    n/a

The request and subsequent response should be logged in vault/logs/audit.log. Take a look.

Secrets

There are two types of secrets in Vault: static and dynamic.

Static secrets (think encrypted Redis or Memcached) have refresh intervals but they do not expire unless explicitly revoked. They are defined ahead of time with the Key/Value backend (formerly the "generic" backend) and then shared.

lưu trữ bí mật an toàn

Dynamic secrets are generated on demand. They have enforced leases and generally expire after a short period of time. Since they do not exist until they are accessed, there's less exposure -- so dynamic secrets are much more secure. Vault ships with a number of dynamic backends -- i.e., AWS, Databases, Google Cloud, Consul, and RabbitMQ.

Review the Why We Need Dynamic Secrets blog post for more info on the advantages of using dynamic secrets.

Static Secrets

Vault can be managed through the CLI, HTTP API, or UI.

CLI

Still within the bash session in the container, we can create, read, update, and delete secrets. We'll also look at how to version and roll back secrets.

Enable secrets with following command:

bash-5.1# vault secrets enable kv

Success! Enabled the kv secrets engine at: kv/

Create a new secret with a key of bar and value of precious within the kv/foo path:

bash-5.1# vault kv put kv/foo bar=precious

Success! Data written to: kv/foo

Read:

bash-5.1# vault kv get kv/foo

=== Data ===
Key    Value
---    -----
bar    precious

To work with different versions of a specific key, we'll need to upgrade to v2 of the Key/Value backend:

bash-5.1# vault kv enable-versioning kv/

Success! Tuned the secrets engine at: kv/

Add version 2 by updating the value to copper:

bash-5.1# vault kv put kv/foo bar=copper

Key              Value
---              -----
created_time     2021-09-08T18:23:14.4154928Z
deletion_time    n/a
destroyed        false
version          2

Read version 1:

bash-5.1# vault kv get -version=1 kv/foo

====== Metadata ======
Key              Value
---              -----
created_time     2021-09-08T18:22:37.2548824Z
deletion_time    n/a
destroyed        false
version          1

=== Data ===
Key    Value
---    -----
bar    precious

Read version 2:

bash-5.1# vault kv get -version=2 kv/foo

====== Metadata ======
Key              Value
---              -----
created_time     2021-09-08T18:23:14.4154928Z
deletion_time    n/a
destroyed        false
version          2

=== Data ===
Key    Value
---    -----
bar    copper

Delete the latest version (e.g., version 2):

bash-5.1# vault kv delete kv/foo

Success! Data deleted (if it existed) at: kv/foo

Delete version 1:

bash-5.1# vault kv delete -versions=1 kv/foo

Success! Data deleted (if it existed) at: kv/foo

You can undelete as well:

bash-5.1# vault kv undelete -versions=1 kv/foo

Success! Data written to: kv/undelete/foo

Delete is akin to a soft delete. If you want to remove the underlying metadata, you'll have to use the destroy command:

bash-5.1# vault kv destroy -versions=1 kv/foo

Success! Data written to: kv/destroy/foo

Review v1 and v2 to view all the available commands.

Take note of the audit log. Each of the above requests were logged!

API

You can also interact with Vault via the HTTP API. We'll make requests against v2 of the API. Open a new terminal tab, and then set the root token as an environment variable:

$ export VAULT_TOKEN=your_token_goes_here

Create a new secret called foo with a value of world:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -H "Content-Type: application/json" \
    -X POST \
    -d '{ "data": { "foo": "world" } }' \
    http://127.0.0.1:8200/v1/kv/data/hello

Read the secret:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/hello

The JSON response should contain a data key with a value similar to:

"data": {
  "data":{
    "foo": "world"
  },
  "metadata": {
    "created_time": "2021-09-08T18:30:32.5140484Z",
    "deletion_time": "",
    "destroyed": false,
    "version": 1
  }
}

vault api

Try adding new versions, deleting, and destroying on your own.

UI

The UI should be up at running at http://localhost:8200/ui/vault. Use the root token to login. Then, explore the Key/Value backend on your own:

vault ui

Policies

Thus far we've been using the root policy to interact with the API. Let's set up a policy that only has read access.

Add a new config file called app-policy.json to "vault/policies":

{
  "path": {
    "kv/data/app/*": {
      "policy": "read"
    }
  }
}

Create a new policy back in the bash session:

bash-5.1# vault policy write app /vault/policies/app-policy.json

Success! Uploaded policy: app

Then, create a new token:

bash-5.1# vault token create -policy=app

Key                  Value
---                  -----
token                s.ZOUMx3RIhVRhI4ijlZg8KXRQ
token_accessor       TT53xOxbIfGjI7l4392gjXcg
token_duration       768h
token_renewable      true
token_policies       ["app" "default"]
identity_policies    []
policies             ["app" "default"]

Within another new terminal tab (you should now have three), add the VAULT_TOKEN environment variable with the new token:

$ export VAULT_TOKEN=your_token_goes_here

Try to read the foo secret that we previously set:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/hello

You should not have the correct permissions to view that secret:

{
  "errors":[
    "1 error occurred:\n\t* permission denied\n\n"
  ]
}

Why can't we even read it? Jump back to the policy config in vault-config.json. kv/data/app/* indicates that the policy can only read from the app path.

As you've probably already noticed, nearly everything in Vault is path-based.

Back within the bash session in the container, add a new secret to the app/test path:

bash-5.1# vault kv put kv/app/test ping=pong

Key              Value
---              -----
created_time     2021-09-08T18:40:35.2694047Z
deletion_time    n/a
destroyed        false
version          1

You should be able to view the secret using the token associated with the app policy:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/app/test

Policies can be managed from the UI as well:

vault ui

Encryption as a Service

Before we look at dynamic secrets, let's quickly review the Transit backend, which can be used as an "encryption as a service" for:

  • Encrypting and decrypting data "in-transit" without storing it inside Vault
  • Easily integrating encryption into your application workflow

Back within the bash session in the container, enable Transit:

bash-5.1# vault secrets enable transit

Success! Enabled the transit secrets engine at: transit/

Configure a named encryption key:

bash-5.1# vault write -f transit/keys/foo

Success! Data written to: transit/keys/foo

Encrypt:

bash-5.1# vault write transit/encrypt/foo plaintext=$(base64 <<< "my precious")

Key           Value
---           -----
ciphertext    vault:v1:cFnk5AQLE9Mg+mZ7Ej17vRmYT5aqheikdZQ1FC4vre5jAod0L/uHDA==

Decrypt:

bash-5.1# vault write transit/decrypt/foo ciphertext=vault:v1:cFnk5AQLE9Mg+mZ7Ej17vRmYT5aqheikdZQ1FC4vre5jAod0L/uHDA==

Key          Value
---          -----
plaintext    bXkgcHJlY2lvdXMK

Decode:

bash-5.1# base64 -d <<< "bXkgcHJlY2lvdXMK"

my precious

Test it out in the UI as well:

vault ui

Dynamic Secrets

As mentioned, Vault supports a number of dynamic secret backends for generating secrets dynamically when needed. For example, with the AWS and Google Cloud backends, you can create access credentials based on IAM policies. The Databases backend, meanwhile, generates database credentials based on configured roles.

Dynamic Secrets:

  • are generated on demand
  • have limited access based on role
  • are leased for a period of time
  • can be revoked
  • come with an audit trail

Let's look at how to generate AWS credentials using the AWS backend.

AWS Credentials

Enable the AWS secrets backend:

bash-5.1# vault secrets enable -path=aws aws

Success! Enabled the aws secrets engine at: aws/

Authenticate:

bash-5.1# vault write aws/config/root access_key=foo secret_key=bar

Success! Data written to: aws/config/root

Make sure to replace foo and bar with your AWS access key id and secret key, respectively.

Create role:

bash-5.1# vault write aws/roles/ec2-read credential_type=iam_user policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Success! Data written to: aws/roles/ec2-read

Here, we created a new role based on AmazonEC2ReadOnlyAccess, which is an AWS-managed policy. As the name suggests, it give users read-only access to the EC2 console; they cannot perform any actions or create new resources. You can also use an inline policy to create a custom role based on your individual needs. We'll look at an example of this shortly. Refer to the AWS Secrets Engine docs for more info.

Remember: Dynamic Secrets are generated only when they are requested (i.e., a web app requests access to S3). They are not available in the store before this.

Create a new set of credentials:

bash-5.1# vault read aws/creds/ec2-read

Key                Value
---                -----
lease_id           aws/creds/ec2-read/9KdO6J7KVBiSwOPEvwrqqALG
lease_duration     768h
lease_renewable    true
access_key         AKIAZ4DZAKZKEULSDW5A
secret_key         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
security_token     <nil>

You should now be able to see the user within the "Users" section on the IAM console on AWS:

tôi xin lỗi

Leases and Revocation

In this section, we'll take a quick look at how to define a custom lease period and revoke a secret before the end of that period.

Create a new AWS role:

bash-5.1# vault write aws/roles/foo credential_type=iam_user policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Success! Data written to: aws/roles/foo

Take note of the lease_duration when you create a new AWS credential:

bash-5.1# vault read aws/creds/foo

Key                Value
---                -----
lease_id           aws/creds/foo/F0oBbnBIHEoz0ywVVtbuJB7r
lease_duration     768h
lease_renewable    true
access_key         AKIAZ4DZAKZKLJKB7CPX
secret_key         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
security_token     <nil>

What if you only wanted the lease period for all AWS IAM dynamic secrets to be 30 minutes?

bash-5.1# vault write aws/config/lease lease=1800s lease_max=1800s

In this example, since lease_max is the same as lease, you won't be able to renew the token. If you set the lease_max to 3600s, you'd be able to renew the lease once. For more, review the Tokens and Leases guide.

Create a new credential:

bash-5.1# vault read aws/creds/foo

Key                Value
---                -----
lease_id           aws/creds/foo/xQlJpKDS1ljE9Awz0aywXgbB
lease_duration     30m
lease_renewable    true
access_key         AKIAZ4DZAKZKJPL5OM5W
secret_key         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
security_token     <nil>

Want to quickly revoke this credential? Grab the lease_id and then run:

bash-5.1# vault lease revoke aws/creds/foo/xQlJpKDS1ljE9Awz0aywXgbB

Want to revoke all AWS creds?

bash-5.1# vault lease revoke -prefix aws/

Refer to the Lease, Renew, and Revoke guide for more info these concepts.

Consul Backend

Cho đến nay, chúng tôi đã sử dụng chương trình phụ trợ Hệ thống tệp. Điều này sẽ không mở rộng ra ngoài một máy chủ duy nhất, vì vậy nó không tận dụng được tính khả dụng cao của Vault. May mắn thay, có một số phần mềm phụ trợ Lưu trữ khác , như phần phụ trợ Consul , được thiết kế cho các hệ thống phân tán.

Để thiết lập Consul , hãy bắt đầu bằng cách cập nhật tệp docker-compost.yml :

version: '3.8'

services:

  vault:
    build:
      context: ./vault
      dockerfile: Dockerfile
    ports:
      - 8200:8200
    volumes:
      - ./vault/config:/vault/config
      - ./vault/policies:/vault/policies
      - ./vault/data:/vault/data
      - ./vault/logs:/vault/logs
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_API_ADDR=http://127.0.0.1:8200
    command: server -config=/vault/config/vault-config.json
    cap_add:
      - IPC_LOCK
    depends_on:
      - consul

  consul:
    build:
      context: ./consul
      dockerfile: Dockerfile
    ports:
      - 8500:8500
    command: agent -server -bind 0.0.0.0 -client 0.0.0.0 -bootstrap-expect 1 -config-file=/consul/config/config.json
    volumes:
      - ./consul/config/consul-config.json:/consul/config/config.json
      - ./consul/data:/consul/data

Thêm một thư mục mới trong thư mục gốc của dự án được gọi là "consul", sau đó thêm một Dockerfile mới vào thư mục mới tạo đó:

# base image
FROM alpine:3.14

# set consul version
ENV CONSUL_VERSION 1.10.2

# create a new directory
RUN mkdir /consul

# download dependencies
RUN apk --no-cache add \
      bash \
      ca-certificates \
      wget

# download and set up consul
RUN wget --quiet --output-document=/tmp/consul.zip https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_linux_amd64.zip && \
    unzip /tmp/consul.zip -d /consul && \
    rm -f /tmp/consul.zip && \
    chmod +x /consul/consul

# update PATH
ENV PATH="PATH=$PATH:$PWD/consul"

# add the config file
COPY ./config/consul-config.json /consul/config/config.json

# expose ports
EXPOSE 8300 8400 8500 8600

# run consul
ENTRYPOINT ["consul"]

Tiếp theo, trong thư mục "consul" thêm hai thư mục mới: "config" và "data". Sau đó, trong "config", thêm một tệp cấu hình có tên là consul-config.json :

{
  "datacenter": "localhost",
  "data_dir": "/consul/data",
  "log_level": "DEBUG",
  "server": true,
  "ui": true,
  "ports": {
    "dns": 53
  }
}

Hãy nhớ xem lại các tùy chọn Cấu hình từ tài liệu của Consul để biết thêm thông tin về các tùy chọn trên.

Thư mục "Consul" bây giờ sẽ giống như sau:

├── Dockerfile
├── config
│   └── consul-config.json
└── data

Thoát khỏi phiên bash. Đặt vùng chứa xuống, sau đó cập nhật tệp cấu hình Vault:

{
  "backend": {
    "consul": {
      "address": "consul:8500",
      "path": "vault/"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Vì vậy, bây giờ chúng tôi đang sử dụng chương trình phụ trợ Consul thay vì Hệ thống tệp. Chúng tôi đã sử dụng tên của dịch vụ consul, như một phần của địa chỉ. Khóa pathxác định đường dẫn trong kho lưu trữ khóa / giá trị của Consul nơi dữ liệu Vault sẽ được lưu trữ.

Xóa tất cả các tệp và thư mục trong thư mục "vault / data" để xóa phần phụ trợ Hệ thống tệp. Xây dựng các hình ảnh mới và quay các vùng chứa:

$ docker-compose down
$ docker-compose up -d --build

Đảm bảo tất cả đều tốt bằng cách điều hướng trong trình duyệt của bạn đến http: // localhost: 8500 / ui :

lãnh sự ui

Kiểm tra điều này từ CLI hoặc UI.

CLI

Tạo phiên bash mới trong vùng chứa Vault:

$ docker-compose exec vault bash

Sau đó chạy:

# Init
bash-5.1# vault operator init

# Unseal
bash-5.1# vault operator unseal

# Authenticate
bash-5.1# vault login

# Enable secrets
bash-5.1# vault secrets enable kv

# Add a new static secret
bash-5.1# vault kv put kv/foo bar=precious

# Read it back
bash-5.1# vault kv get kv/foo

Giao diện người dùng

lãnh sự kho tiền

Lưu ý rằng không có tệp hoặc thư mục nào trong "vault / data". Tại sao bạn nghĩ rằng đây là?

Bạn muốn thêm một máy chủ Consul khác vào hỗn hợp? Thêm một dịch vụ mới vào docker-compos.yml :

consul-worker:
  build:
    context: ./consul
    dockerfile: Dockerfile
  command: agent -server -join consul -config-file=/consul/config/config.json
  volumes:
    - ./consul/config/consul-config.json:/consul/config/config.json
  depends_on:
    - consul

Ở đây, chúng tôi đã sử dụng lệnh tham gia để kết nối tác nhân này với một cụm hiện có. Lưu ý rằng chúng tôi chỉ cần tham chiếu đến tên dịch vụ như thế nào consul:.

Sau đó:

  1. Thoát khỏi phiên bash (nếu cần)
  2. Mang các thùng chứa xuống
  3. Xóa thư mục dữ liệu trong "consul / data" (Tại sao?)
  4. Quay các thùng chứa sao lưu và kiểm tra

lãnh sự ui

Sự kết luận

Trong hướng dẫn này, chúng tôi đã giới thiệu cho các bạn cách thiết lập và chạy Vault and Consul bên trong vùng chứa Docker. Bây giờ bạn đã hiểu rõ về cách tương tác với Vault và thực hiện các thao tác cơ bản.

Lấy mã cuối cùng từ repo vault-consul-docker . Kiểm tra bản trình bày là tốt.

Nguồn:  https://testdriven.io

#vault #consul 

Cách Thiết Lập Và Sử Dụng Các Dự Án Vault Và Consul Của Hashicorp

Как настроить и использовать проекты Hashicorp Vault и Consul

В следующем руководстве подробно описано, как настроить и использовать проекты Hashicorp Vault и Consul для безопасного хранения секретов и управления ими.

Мы начнем с запуска одного экземпляра Vault в контейнере Docker, а затем перейдем к управлению как статическими, так и динамическими секретами вместе с функцией Vault «шифрование как услуга». Затем мы добавим Consul и посмотрим, как масштабировать Vault.

Основные зависимости:

  • Докер v20.10.8
  • Докер-композитор v1.29.2
  • Убежище v1.8.2
  • Консул v1.10.2

Цели

К концу этого урока вы должны уметь:

  1. Объясните, что такое Vault и почему вы можете захотеть его использовать.
  2. Описать базовую архитектуру Vault, а также динамические и статические секреты, различные серверные части (хранилище, секрет, аутентификация, аудит) и то, как Vault можно использовать в качестве «шифрования как услуги».
  3. Настройте и запустите Vault и Consul с помощью Docker
  4. Раскрутите Vault с помощью серверной части файловой системы
  5. Инициализировать и распечатать хранилище
  6. Аутентификация в хранилище
  7. Настройте серверную часть аудита для регистрации всех взаимодействий с Vault.
  8. Работайте со статическими и динамическими секретами через интерфейс командной строки, HTTP API и пользовательский интерфейс.
  9. Создайте политику Vault, чтобы ограничить доступ к определенному пути
  10. Используйте серверную часть Transit как «шифрование как услугу».
  11. Настройте Consul для работы с Vault в качестве хранилища для секретов.
  12. Определите пользовательский период аренды для секрета и отзовите секрет до окончания этого периода.

Что такое Сейф?

Vault — это инструмент с открытым исходным кодом, используемый для безопасного хранения секретов и управления ими.

Что такое секрет? Секреты в контексте этого руководства — это конфиденциальная или личная информация, такая как учетные данные базы данных, ключи SSH, имена пользователей и пароли, учетные данные AWS IAM, токены API, номера социального страхования, номера кредитных карт и т. д.

Найдите минутку, чтобы подумать о том, как ваша команда в настоящее время управляет секретами и распределяет их:

  1. Кто имеет к ним доступ?
  2. Кто ими управляет?
  3. Как вы контролируете, кто имеет к ним доступ?
  4. Как ваши приложения получают их?
  5. Как они обновляются?
  6. Как они отзываются?

Vault дает ответы на эти вопросы и помогает решить следующие проблемы, связанные с управлением секретами:

ПроблемыЦели Убежища
Секреты повсюду.Хранилище — единственный источник правды для всех секретов.
Обычно они не зашифрованы.Сейф автоматически управляет шифрованием (во время передачи и хранения).
Их сложно динамически генерировать.Секреты могут генерироваться динамически.
Еще труднее сдать в аренду и отозвать их.Секреты можно сдавать в аренду и отзывать.
Нет никакого контрольного следа.Существует контрольный журнал для создания и использования секретов.

Vault состоит из нескольких движущихся частей, поэтому может потребоваться некоторое время, чтобы привыкнуть к общей архитектуре. Найдите минутку, чтобы просмотреть руководство по архитектуре , приняв во внимание следующие серверные части:

БэкендИспользоватьПримеры
ХранилищеГде хранятся секретыКонсул *, файловая система *, в памяти, PostgreSQL, S3
СекретОбрабатывает статические или динамические секретыAWS *, базы данных, ключ/значение *, RabbitMQ, SSH
АвторизацияОбрабатывает аутентификацию и авторизациюAWS, Azure, Google Cloud, GitHub, токены *, имя пользователя и пароль
АудитРегистрирует все запросы и ответыФайл *, системный журнал, сокет

*используется в этом уроке

Итак, давайте начнем использовать Vault.

Бэкэнд файловой системы

Чтобы быстро приступить к работе, мы будем использовать серверную часть файловой системы для хранения секретов в состоянии покоя.

Серверную часть файловой системы следует использовать только для локальной разработки или развертывания Vault на одном сервере, поскольку она не поддерживает высокую доступность.

Создайте новый каталог проекта:

$ mkdir vault-consul-docker && cd vault-consul-docker

Затем добавьте следующие папки:

└── vault
    ├── config
    ├── data
    ├── logs
    └── policies

Добавьте Dockerfile в каталог «хранилище»:

# base image
FROM alpine:3.14

# set vault version
ENV VAULT_VERSION 1.8.2

# create a new directory
RUN mkdir /vault

# download dependencies
RUN apk --no-cache add \
      bash \
      ca-certificates \
      wget

# download and set up vault
RUN wget --quiet --output-document=/tmp/vault.zip https://releases.hashicorp.com/vault/${VAULT_VERSION}/vault_${VAULT_VERSION}_linux_amd64.zip && \
    unzip /tmp/vault.zip -d /vault && \
    rm -f /tmp/vault.zip && \
    chmod +x /vault

# update PATH
ENV PATH="PATH=$PATH:$PWD/vault"

# add the config file
COPY ./config/vault-config.json /vault/config/vault-config.json

# expose port 8200
EXPOSE 8200

# run vault
ENTRYPOINT ["vault"]

Затем добавьте файл docker-compose.yml в корень проекта:

version: '3.8'

services:

  vault:
    build:
      context: ./vault
      dockerfile: Dockerfile
    ports:
      - 8200:8200
    volumes:
      - ./vault/config:/vault/config
      - ./vault/policies:/vault/policies
      - ./vault/data:/vault/data
      - ./vault/logs:/vault/logs
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_API_ADDR=http://127.0.0.1:8200
    command: server -config=/vault/config/vault-config.json
    cap_add:
      - IPC_LOCK

Добавьте файл конфигурации с именем vault-config.json в «vault/config»:

{
  "backend": {
    "file": {
      "path": "vault/data"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Здесь мы настроили Vault для использования серверной части файловой системы, определили прослушиватель для Vault, отключили TLS и включили пользовательский интерфейс Vault . Ознакомьтесь с документацией для получения дополнительной информации о настройке Vault.

Теперь мы можем создать образ и запустить контейнер:

$ docker-compose up -d --build

Откройте журналы Docker, чтобы убедиться, что в сборке нет ошибок:

$ docker-compose logs

Вы должны увидеть что-то похожее на:

Attaching to vault-consul-docker_vault_1
vault_1  | ==> Vault server configuration:
vault_1  |
vault_1  |              Api Address: http://127.0.0.1:8200
vault_1  | 2021-09-08T14:48:35.014Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
vault_1  |                      Cgo: disabled
vault_1  |          Cluster Address: https://127.0.0.1:8201
vault_1  |               Go Version: go1.16.7
vault_1  |               Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
vault_1  |                Log Level: info
vault_1  |                    Mlock: supported: true, enabled: true
vault_1  |            Recovery Mode: false
vault_1  |                  Storage: file
vault_1  |                  Version: Vault v1.8.2
vault_1  |              Version Sha: aca76f63357041a43b49f3e8c11d67358496959f
vault_1  |
vault_1  | ==> Vault server started! Log data will stream in below:
vault_1  |

Инициализация и распечатка

Запустите сеанс bash в работающем контейнере:

$ docker-compose exec vault bash

В оболочке инициализируйте Vault:

bash-5.1# vault operator init

Обратите внимание на ключи распечатывания и первоначальный корневой токен. Вам нужно будет предоставлять три ключа распечатывания каждый раз, когда сервер Vault повторно запечатывается или перезапускается.

Почему 3 ключа? Просмотрите секретный обмен Шамира .

Теперь вы можете открыть Хранилище с помощью трех ключей:

bash-5.1# vault operator unseal
Unseal Key (will be hidden):

Запустите эту команду еще два раза, каждый раз используя разные клавиши. После этого убедитесь, Sealedчто false:

Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    5
Threshold       3
Version         1.8.2
Storage Type    file
Cluster Name    vault-cluster-8fcf9d05
Cluster ID      d86e0274-ad9c-d2c1-d6ec-baeab410797b
HA Enabled      false

Используя корневой токен, теперь вы можете аутентифицироваться:

bash-5.1# vault login
Token (will be hidden):

Вы должны увидеть что-то похожее на:

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                s.c0kYHWiOTqQvtR8JuSeTz6sZ
token_accessor       3FQJVxOY5C1brzlHHQSFaCdZ
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Имейте в виду, что при этом используется корневая политика. В рабочей среде вам потребуется настроить политики с разными уровнями доступа. Вскоре мы рассмотрим, как это сделать.

инициализация хранилища

Хранилище теперь распечатано и готово к использованию.

Аудиторская проверка

Прежде чем мы проверим функциональность, давайте включим устройство аудита :

bash-5.1# vault audit enable file file_path=/vault/logs/audit.log

Success! Enabled the file audit device at: file/

Теперь вы сможете просматривать журналы локально в «хранилище/журналах». Для проверки выполните следующую команду, чтобы просмотреть все включенные устройства аудита:

bash-5.1# vault audit list

Path     Type    Description
----     ----    -----------
file/    file    n/a

Запрос и последующий ответ должны регистрироваться в vault/logs/audit.log . Взглянем.

Секреты

В Vault есть два типа секретов: статические и динамические .

Статические секреты (например, зашифрованные Redis или Memcached) имеют интервалы обновления, но срок их действия не истекает, если они не отменены явным образом. Они определяются заранее с помощью бэкэнда Key/Value (ранее «общий» бэкэнд), а затем совместно используются.

надежное секретное хранилище

Динамические секреты генерируются по запросу. Они имеют принудительные договоры аренды и обычно истекают через короткий промежуток времени. Поскольку они не существуют до тех пор, пока к ним не будет осуществлен доступ, они менее уязвимы, поэтому динамические секреты гораздо более безопасны. Vault поставляется с несколькими динамическими бэкендами, например, AWS , Databases , Google Cloud , Consul и RabbitMQ .

Ознакомьтесь с записью блога « Почему нам нужны динамические секреты », чтобы получить дополнительную информацию о преимуществах использования динамических секретов.

Статические секреты

Хранилищем можно управлять через интерфейс командной строки , HTTP API или пользовательский интерфейс .

CLI

Все еще в сеансе bash в контейнере мы можем создавать, читать, обновлять и удалять секреты. Мы также рассмотрим, как создавать версии и откатывать секреты.

Включите секреты с помощью следующей команды:

bash-5.1# vault secrets enable kv

Success! Enabled the kv secrets engine at: kv/

Создайте новый секрет с ключом barи значением preciousвнутри kv/fooпути:

bash-5.1# vault kv put kv/foo bar=precious

Success! Data written to: kv/foo

Читать:

bash-5.1# vault kv get kv/foo

=== Data ===
Key    Value
---    -----
bar    precious

Чтобы работать с разными версиями определенного ключа, нам нужно обновить до версии 2 серверную часть Key/Value :

bash-5.1# vault kv enable-versioning kv/

Success! Tuned the secrets engine at: kv/

Добавьте версию 2, изменив значение на copper:

bash-5.1# vault kv put kv/foo bar=copper

Key              Value
---              -----
created_time     2021-09-08T18:23:14.4154928Z
deletion_time    n/a
destroyed        false
version          2

Читать версию 1:

bash-5.1# vault kv get -version=1 kv/foo

====== Metadata ======
Key              Value
---              -----
created_time     2021-09-08T18:22:37.2548824Z
deletion_time    n/a
destroyed        false
version          1

=== Data ===
Key    Value
---    -----
bar    precious

Читать версию 2:

bash-5.1# vault kv get -version=2 kv/foo

====== Metadata ======
Key              Value
---              -----
created_time     2021-09-08T18:23:14.4154928Z
deletion_time    n/a
destroyed        false
version          2

=== Data ===
Key    Value
---    -----
bar    copper

Удалить последнюю версию (например, версию 2):

bash-5.1# vault kv delete kv/foo

Success! Data deleted (if it existed) at: kv/foo

Удалить версию 1:

bash-5.1# vault kv delete -versions=1 kv/foo

Success! Data deleted (if it existed) at: kv/foo

Вы также можете восстановить:

bash-5.1# vault kv undelete -versions=1 kv/foo

Success! Data written to: kv/undelete/foo

Удаление сродни мягкому удалению. Если вы хотите удалить базовые метаданные, вам придется использовать команду destroy :

bash-5.1# vault kv destroy -versions=1 kv/foo

Success! Data written to: kv/destroy/foo

Просмотрите v1 и v2 , чтобы просмотреть все доступные команды.

Обратите внимание на журнал аудита. Каждый из вышеуказанных запросов был зарегистрирован!

API

Вы также можете взаимодействовать с Vault через HTTP API . Мы будем делать запросы к v2 API. Откройте новую вкладку терминала, а затем установите корневой токен в качестве переменной среды:

$ export VAULT_TOKEN=your_token_goes_here

Создайте новый секрет с именем fooсо значением world:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -H "Content-Type: application/json" \
    -X POST \
    -d '{ "data": { "foo": "world" } }' \
    http://127.0.0.1:8200/v1/kv/data/hello

Прочтите секрет:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/hello

Ответ JSON должен содержать dataключ со значением, подобным:

"data": {
  "data":{
    "foo": "world"
  },
  "metadata": {
    "created_time": "2021-09-08T18:30:32.5140484Z",
    "deletion_time": "",
    "destroyed": false,
    "version": 1
  }
}

API хранилища

Попробуйте добавлять новые версии, удалять и уничтожать самостоятельно.

интерфейс

Пользовательский интерфейс должен работать по адресу http://localhost:8200/ui/vault . Используйте корневой токен для входа. Затем самостоятельно изучите бэкэнд Key/Value:

интерфейс хранилища

Политики

До сих пор мы использовали корневую политику для взаимодействия с API. Давайте настроим политику, которая имеет доступ только для чтения.

Добавьте новый файл конфигурации с именем app-policy.json в «vault/policies»:

{
  "path": {
    "kv/data/app/*": {
      "policy": "read"
    }
  }
}

Создайте новую политику в сеансе bash:

bash-5.1# vault policy write app /vault/policies/app-policy.json

Success! Uploaded policy: app

Затем создайте новый токен:

bash-5.1# vault token create -policy=app

Key                  Value
---                  -----
token                s.ZOUMx3RIhVRhI4ijlZg8KXRQ
token_accessor       TT53xOxbIfGjI7l4392gjXcg
token_duration       768h
token_renewable      true
token_policies       ["app" "default"]
identity_policies    []
policies             ["app" "default"]

На другой новой вкладке терминала (теперь их должно быть три) добавьте VAULT_TOKENпеременную среды с новым токеном:

$ export VAULT_TOKEN=your_token_goes_here

Попробуйте прочитать fooсекрет, который мы установили ранее:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/hello

У вас не должно быть правильных разрешений для просмотра этого секрета:

{
  "errors":[
    "1 error occurred:\n\t* permission denied\n\n"
  ]
}

Почему мы даже не можем это прочитать? Вернитесь к конфигурации политики в vault-config.json . kv/data/app/*указывает, что политика может читать только из appпути.

Как вы, наверное, уже заметили, почти все в Vault основано на путях.

Вернувшись в сеанс bash в контейнере, добавьте новый секрет к app/testпути:

bash-5.1# vault kv put kv/app/test ping=pong

Key              Value
---              -----
created_time     2021-09-08T18:40:35.2694047Z
deletion_time    n/a
destroyed        false
version          1

Вы должны иметь возможность просматривать секрет, используя токен, связанный с appполитикой:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/app/test

Политиками также можно управлять из пользовательского интерфейса:

интерфейс хранилища

Шифрование как услуга

Прежде чем мы рассмотрим динамические секреты, давайте быстро рассмотрим серверную часть Transit , которую можно использовать как «шифрование как услугу» для:

  • Шифрование и дешифрование данных «в пути» без сохранения их в хранилище
  • Простая интеграция шифрования в рабочий процесс вашего приложения

Вернувшись в сеанс bash в контейнере, включите Transit:

bash-5.1# vault secrets enable transit

Success! Enabled the transit secrets engine at: transit/

Настройте именованный ключ шифрования:

bash-5.1# vault write -f transit/keys/foo

Success! Data written to: transit/keys/foo

Зашифровать:

bash-5.1# vault write transit/encrypt/foo plaintext=$(base64 <<< "my precious")

Key           Value
---           -----
ciphertext    vault:v1:cFnk5AQLE9Mg+mZ7Ej17vRmYT5aqheikdZQ1FC4vre5jAod0L/uHDA==

Расшифровать:

bash-5.1# vault write transit/decrypt/foo ciphertext=vault:v1:cFnk5AQLE9Mg+mZ7Ej17vRmYT5aqheikdZQ1FC4vre5jAod0L/uHDA==

Key          Value
---          -----
plaintext    bXkgcHJlY2lvdXMK

Расшифровать:

bash-5.1# base64 -d <<< "bXkgcHJlY2lvdXMK"

my precious

Проверьте это и в пользовательском интерфейсе:

интерфейс хранилища

Динамические секреты

Как уже упоминалось, Vault поддерживает ряд динамических секретных бэкэндов для динамического создания секретов при необходимости. Например, с помощью серверных частей AWS и Google Cloud вы можете создавать учетные данные для доступа на основе политик IAM. Тем временем серверная часть баз данных генерирует учетные данные базы данных на основе настроенных ролей .

Динамические секреты:

  • генерируются по запросу
  • иметь ограниченный доступ в зависимости от роли
  • сдаются в аренду на срок
  • может быть отозван
  • прийти с аудиторским следом

Давайте посмотрим, как сгенерировать учетные данные AWS с помощью серверной части AWS.

Учетные данные AWS

Включите серверную часть секретов AWS:

bash-5.1# vault secrets enable -path=aws aws

Success! Enabled the aws secrets engine at: aws/

Аутентификация:

bash-5.1# vault write aws/config/root access_key=foo secret_key=bar

Success! Data written to: aws/config/root

Обязательно замените fooи barна идентификатор ключа доступа AWS и секретный ключ соответственно.

Создать роль:

bash-5.1# vault write aws/roles/ec2-read credential_type=iam_user policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Success! Data written to: aws/roles/ec2-read

Здесь мы создали новую роль на основе политикиAmazonEC2ReadOnlyAccess , управляемой AWS . Как следует из названия, он предоставляет пользователям доступ только для чтения к консоли EC2; они не могут выполнять какие-либо действия или создавать новые ресурсы. Вы также можете использовать встроенную политику для создания пользовательской роли в соответствии с вашими индивидуальными потребностями. Вскоре мы рассмотрим это на примере. Дополнительную информацию см . в документации AWS Secrets Engine .

Помните : динамические секреты генерируются только тогда, когда они запрашиваются (например, веб-приложение запрашивает доступ к S3). До этого в магазине их нет.

Создайте новый набор учетных данных:

bash-5.1# vault read aws/creds/ec2-read

Key                Value
---                -----
lease_id           aws/creds/ec2-read/9KdO6J7KVBiSwOPEvwrqqALG
lease_duration     768h
lease_renewable    true
access_key         AKIAZ4DZAKZKEULSDW5A
secret_key         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
security_token     <nil>

Теперь вы должны увидеть пользователя в разделе «Пользователи» на консоли IAM на AWS:

Мне жаль

Аренда и отзыв

В этом разделе мы кратко рассмотрим, как определить пользовательский период аренды и отозвать секрет до окончания этого периода.

Создайте новую роль AWS:

bash-5.1# vault write aws/roles/foo credential_type=iam_user policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Success! Data written to: aws/roles/foo

Обратите внимание на то, что lease_durationпри создании новых учетных данных AWS:

bash-5.1# vault read aws/creds/foo

Key                Value
---                -----
lease_id           aws/creds/foo/F0oBbnBIHEoz0ywVVtbuJB7r
lease_duration     768h
lease_renewable    true
access_key         AKIAZ4DZAKZKLJKB7CPX
secret_key         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
security_token     <nil>

Что, если вы хотите, чтобы период аренды для всех динамических секретов AWS IAM составлял 30 минут?

bash-5.1# vault write aws/config/lease lease=1800s lease_max=1800s

В этом примере, поскольку lease_maxэто то же самое lease, что и , вы не сможете обновить токен. Если вы установите значение lease_max, 3600sвы сможете продлить аренду один раз. Для получения дополнительной информации ознакомьтесь с руководством по токенам и аренде .

Создайте новые учетные данные:

bash-5.1# vault read aws/creds/foo

Key                Value
---                -----
lease_id           aws/creds/foo/xQlJpKDS1ljE9Awz0aywXgbB
lease_duration     30m
lease_renewable    true
access_key         AKIAZ4DZAKZKJPL5OM5W
secret_key         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
security_token     <nil>

Хотите быстро отозвать эти учетные данные? Возьмите, lease_idа затем запустите:

bash-5.1# vault lease revoke aws/creds/foo/xQlJpKDS1ljE9Awz0aywXgbB

Хотите отозвать все кредиты AWS?

bash-5.1# vault lease revoke -prefix aws/

Дополнительные сведения об этих концепциях см. в руководстве по аренде, продлению и отзыву .

Консул Бэкэнд

До сих пор мы использовали серверную часть файловой системы . Он не будет масштабироваться за пределы одного сервера, поэтому он не использует преимущества высокой доступности Vault. К счастью, существует ряд других бэкендов Storage , таких как бэкенд Consul , предназначенных для распределенных систем.

Чтобы настроить Consul , начните с обновления файла docker-compose.yml :

version: '3.8'

services:

  vault:
    build:
      context: ./vault
      dockerfile: Dockerfile
    ports:
      - 8200:8200
    volumes:
      - ./vault/config:/vault/config
      - ./vault/policies:/vault/policies
      - ./vault/data:/vault/data
      - ./vault/logs:/vault/logs
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_API_ADDR=http://127.0.0.1:8200
    command: server -config=/vault/config/vault-config.json
    cap_add:
      - IPC_LOCK
    depends_on:
      - consul

  consul:
    build:
      context: ./consul
      dockerfile: Dockerfile
    ports:
      - 8500:8500
    command: agent -server -bind 0.0.0.0 -client 0.0.0.0 -bootstrap-expect 1 -config-file=/consul/config/config.json
    volumes:
      - ./consul/config/consul-config.json:/consul/config/config.json
      - ./consul/data:/consul/data

Добавьте новый каталог в корень проекта с именем «consul», а затем добавьте новый Dockerfile в этот вновь созданный каталог:

# base image
FROM alpine:3.14

# set consul version
ENV CONSUL_VERSION 1.10.2

# create a new directory
RUN mkdir /consul

# download dependencies
RUN apk --no-cache add \
      bash \
      ca-certificates \
      wget

# download and set up consul
RUN wget --quiet --output-document=/tmp/consul.zip https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_linux_amd64.zip && \
    unzip /tmp/consul.zip -d /consul && \
    rm -f /tmp/consul.zip && \
    chmod +x /consul/consul

# update PATH
ENV PATH="PATH=$PATH:$PWD/consul"

# add the config file
COPY ./config/consul-config.json /consul/config/config.json

# expose ports
EXPOSE 8300 8400 8500 8600

# run consul
ENTRYPOINT ["consul"]

Затем в каталоге «consul» добавьте два новых каталога: «config» и «data». Затем в «config» добавьте файл конфигурации с именем consul-config.json :

{
  "datacenter": "localhost",
  "data_dir": "/consul/data",
  "log_level": "DEBUG",
  "server": true,
  "ui": true,
  "ports": {
    "dns": 53
  }
}

Обязательно ознакомьтесь с параметрами конфигурации из документов Consul для получения дополнительной информации о вышеуказанных параметрах.

Теперь каталог «consul» должен выглядеть так:

├── Dockerfile
├── config
│   └── consul-config.json
└── data

Выйдите из сеанса bash. Перенесите контейнер, а затем обновите файл конфигурации Vault:

{
  "backend": {
    "consul": {
      "address": "consul:8500",
      "path": "vault/"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Итак, теперь мы используем бэкенд Consul вместо файловой системы. Мы использовали название службы consulкак часть адреса. Ключ определяет путь pathв хранилище ключей/значений Consul, где будут храниться данные Vault.

Очистите все файлы и папки в каталоге «vault/data», чтобы удалить серверную часть файловой системы. Создайте новые образы и разверните контейнеры:

$ docker-compose down
$ docker-compose up -d --build

Убедитесь, что все в порядке, перейдя в браузере по адресу http://localhost:8500/ui :

консул пользовательского интерфейса

Проверьте это из интерфейса командной строки или пользовательского интерфейса.

CLI

Создайте новую сессию bash в контейнере Vault:

$ docker-compose exec vault bash

Затем запустите:

# Init
bash-5.1# vault operator init

# Unseal
bash-5.1# vault operator unseal

# Authenticate
bash-5.1# vault login

# Enable secrets
bash-5.1# vault secrets enable kv

# Add a new static secret
bash-5.1# vault kv put kv/foo bar=precious

# Read it back
bash-5.1# vault kv get kv/foo

интерфейс

консул хранилища

Обратите внимание, что в «хранилище/данных» нет файлов или папок. Как вы думаете, почему это так?

Хотите добавить еще один сервер Consul? Добавьте новый сервис в docker-compose.yml :

consul-worker:
  build:
    context: ./consul
    dockerfile: Dockerfile
  command: agent -server -join consul -config-file=/consul/config/config.json
  volumes:
    - ./consul/config/consul-config.json:/consul/config/config.json
  depends_on:
    - consul

Здесь мы использовали команду join для подключения этого агента к существующему кластеру. Обратите внимание, что нам просто нужно было сослаться на имя службы: consul.

Затем:

  1. Выход из bash-сессии (при необходимости)
  2. Снести контейнеры
  3. Очистите каталог данных в «consul/data» (почему?)
  4. Вращайте контейнеры обратно и проверяйте

консул пользовательского интерфейса

Вывод

В этом руководстве мы рассмотрели, как настроить и запустить Vault и Consul внутри контейнера Docker. Теперь у вас должно быть четкое представление о том, как взаимодействовать с Vault и выполнять основные операции.

Возьмите окончательный код из репозитория vault-consul-docker . Ознакомьтесь также с презентацией .

Источник:  https://testdriven.io

#vault #consul 

Как настроить и использовать проекты Hashicorp Vault и Consul
工藤  晃

工藤 晃

1660625880

如何設置和使用 Hashicorp 的 Vault 和 Consul 項目

以下教程詳細介紹瞭如何設置和使用 Hashicorp 的 Vault 和 Consul 項目來安全地存儲和管理機密。

我們將從在 Docker 容器中啟動單個 Vault 實例開始,然後開始管理靜態和動態機密以及 Vault 的“加密即服務”功能。然後,我們將 Consul 添加到組合中,並研究如何擴展 Vault。

主要依賴:

  • 碼頭工人 v20.10.8
  • Docker-Compose v1.29.2
  • 保險櫃 v1.8.2
  • 領事 v1.10.2

目標

在本教程結束時,您應該能夠:

  1. 解釋 Vault 是什麼以及為什麼要使用它
  2. 描述基本的 Vault 架構以及動態和靜態機密、各種後端(存儲、機密、身份驗證、審計),以及如何將 Vault 用作“加密即服務”
  3. 使用 Docker 配置和運行 Vault 和 Consul
  4. 使用文件系統後端啟動 Vault
  5. 初始化和解封 Vault
  6. 針對 Vault 進行身份驗證
  7. 配置審核後端以記錄與 Vault 的所有交互
  8. 通過 CLI、HTTP API 和 UI 處理靜態和動態機密
  9. 創建 Vault 策略以限制對特定路徑的訪問
  10. 將 Transit 後端用作“加密即服務”
  11. 設置 Consul 以使用 Vault 作為機密的存儲後端
  12. 為密鑰定義自定義租期並在該期限結束前撤銷密鑰

什麼是保險櫃?

Vault是一種用於安全存儲和管理機密的開源工具。

What is a secret? Secrets, in the context of this tutorial, are securely-sensitive or personally identifiable info like database credentials, SSH keys, usernames and passwords, AWS IAM credentials, API tokens, Social Security Numbers, credit card numbers, just to name a few.

Take a moment to think about how your team currently manages and distributes secrets:

  1. Who has access to them?
  2. Who manages them?
  3. How do you control who has access to them?
  4. How do your apps get them?
  5. How are they updated?
  6. How are they revoked?

Vault provides answers to those questions and helps to solve the following problems with regard to secret management:

ProblemsVault's Goals
Secrets are everywhere.Vault is the single source of truth for all secrets.
They are generally unencrypted.Vault manages encryption (during transit and at rest) out of the box.
It's difficult to dynamically generate them.Secrets can be dynamically generated.
It's even more difficult to lease and revoke them.Secrets can be leased and revoked.
There's no audit trail.There's an audit trail for generating and using secrets.

Vault has a number of moving pieces so it can take some time to get up to speed with the overall architecture. Take a moment to review the Architecture guide, taking note of the following backends:

BackendUseExamples
StorageWhere secrets are storedConsul*, Filesystem*, In-Memory, PostgreSQL, S3
SecretHandles static or dynamic secretsAWS*, Databases, Key/Value*, RabbitMQ, SSH
AuthHandles authentication and authorizationAWS, Azure, Google Cloud, GitHub, Tokens*, Username & Password
AuditLogs all requests and responsesFile*, Syslog, Socket

* used in this tutorial

With that, let's start using Vault.

Filesystem Backend

To get up and running quickly, we'll use the Filesystem backend to store secrets at rest.

The filesystem backend should only be used for local development or a single-server Vault deployment since it does not support high availability.

Create a new project directory:

$ mkdir vault-consul-docker && cd vault-consul-docker

Then add the following folders:

└── vault
    ├── config
    ├── data
    ├── logs
    └── policies

Add a Dockerfile to the "vault" directory:

# base image
FROM alpine:3.14

# set vault version
ENV VAULT_VERSION 1.8.2

# create a new directory
RUN mkdir /vault

# download dependencies
RUN apk --no-cache add \
      bash \
      ca-certificates \
      wget

# download and set up vault
RUN wget --quiet --output-document=/tmp/vault.zip https://releases.hashicorp.com/vault/${VAULT_VERSION}/vault_${VAULT_VERSION}_linux_amd64.zip && \
    unzip /tmp/vault.zip -d /vault && \
    rm -f /tmp/vault.zip && \
    chmod +x /vault

# update PATH
ENV PATH="PATH=$PATH:$PWD/vault"

# add the config file
COPY ./config/vault-config.json /vault/config/vault-config.json

# expose port 8200
EXPOSE 8200

# run vault
ENTRYPOINT ["vault"]

Next, add a docker-compose.yml file to the project root:

version: '3.8'

services:

  vault:
    build:
      context: ./vault
      dockerfile: Dockerfile
    ports:
      - 8200:8200
    volumes:
      - ./vault/config:/vault/config
      - ./vault/policies:/vault/policies
      - ./vault/data:/vault/data
      - ./vault/logs:/vault/logs
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_API_ADDR=http://127.0.0.1:8200
    command: server -config=/vault/config/vault-config.json
    cap_add:
      - IPC_LOCK

Add a config file called vault-config.json to "vault/config":

{
  "backend": {
    "file": {
      "path": "vault/data"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Here, we configured Vault to use the Filesystem backend, defined the listener for Vault, disabled TLS, and enabled the Vault UI. Review the docs for more info on configuring Vault.

Now we can build the image and spin up the container:

$ docker-compose up -d --build

Pull up the Docker logs to make sure there were no errors in the build:

$ docker-compose logs

You should see something similar to:

Attaching to vault-consul-docker_vault_1
vault_1  | ==> Vault server configuration:
vault_1  |
vault_1  |              Api Address: http://127.0.0.1:8200
vault_1  | 2021-09-08T14:48:35.014Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
vault_1  |                      Cgo: disabled
vault_1  |          Cluster Address: https://127.0.0.1:8201
vault_1  |               Go Version: go1.16.7
vault_1  |               Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
vault_1  |                Log Level: info
vault_1  |                    Mlock: supported: true, enabled: true
vault_1  |            Recovery Mode: false
vault_1  |                  Storage: file
vault_1  |                  Version: Vault v1.8.2
vault_1  |              Version Sha: aca76f63357041a43b49f3e8c11d67358496959f
vault_1  |
vault_1  | ==> Vault server started! Log data will stream in below:
vault_1  |

Initializing and Unsealing

Start a bash session within the running container:

$ docker-compose exec vault bash

Within the shell, initialize Vault:

bash-5.1# vault operator init

Take note of the unseal keys and the initial root token. You will need to provide three of the unseal keys every time the Vault server is resealed or restarted.

Why 3 keys? Review Shamir's Secret Sharing.

Now you can unseal Vault using three of the keys:

bash-5.1# vault operator unseal
Unseal Key (will be hidden):

Run this command two more times, using different keys each time. Once done, make sure Sealed is false:

Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    5
Threshold       3
Version         1.8.2
Storage Type    file
Cluster Name    vault-cluster-8fcf9d05
Cluster ID      d86e0274-ad9c-d2c1-d6ec-baeab410797b
HA Enabled      false

Using the root token, you can now authenticate:

bash-5.1# vault login
Token (will be hidden):

You should see something similar to:

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                s.c0kYHWiOTqQvtR8JuSeTz6sZ
token_accessor       3FQJVxOY5C1brzlHHQSFaCdZ
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Keep in mind that this uses the root policy. In production you'll want to set up policies with different levels of access. We'll look at how to do this shortly.

保險庫初始化

Vault is now unsealed and ready for use.

Auditing

Before we test out the functionality, let's enable an Audit Device:

bash-5.1# vault audit enable file file_path=/vault/logs/audit.log

Success! Enabled the file audit device at: file/

You should now be able to view the logs locally in "vault/logs". To test, run the following command to view all enabled Audit Devices:

bash-5.1# vault audit list

Path     Type    Description
----     ----    -----------
file/    file    n/a

The request and subsequent response should be logged in vault/logs/audit.log. Take a look.

Secrets

There are two types of secrets in Vault: static and dynamic.

Static secrets (think encrypted Redis or Memcached) have refresh intervals but they do not expire unless explicitly revoked. They are defined ahead of time with the Key/Value backend (formerly the "generic" backend) and then shared.

安全的秘密存儲

Dynamic secrets are generated on demand. They have enforced leases and generally expire after a short period of time. Since they do not exist until they are accessed, there's less exposure -- so dynamic secrets are much more secure. Vault ships with a number of dynamic backends -- i.e., AWS, Databases, Google Cloud, Consul, and RabbitMQ.

Review the Why We Need Dynamic Secrets blog post for more info on the advantages of using dynamic secrets.

Static Secrets

Vault can be managed through the CLI, HTTP API, or UI.

CLI

Still within the bash session in the container, we can create, read, update, and delete secrets. We'll also look at how to version and roll back secrets.

Enable secrets with following command:

bash-5.1# vault secrets enable kv

Success! Enabled the kv secrets engine at: kv/

Create a new secret with a key of bar and value of precious within the kv/foo path:

bash-5.1# vault kv put kv/foo bar=precious

Success! Data written to: kv/foo

Read:

bash-5.1# vault kv get kv/foo

=== Data ===
Key    Value
---    -----
bar    precious

To work with different versions of a specific key, we'll need to upgrade to v2 of the Key/Value backend:

bash-5.1# vault kv enable-versioning kv/

Success! Tuned the secrets engine at: kv/

Add version 2 by updating the value to copper:

bash-5.1# vault kv put kv/foo bar=copper

Key              Value
---              -----
created_time     2021-09-08T18:23:14.4154928Z
deletion_time    n/a
destroyed        false
version          2

Read version 1:

bash-5.1# vault kv get -version=1 kv/foo

====== Metadata ======
Key              Value
---              -----
created_time     2021-09-08T18:22:37.2548824Z
deletion_time    n/a
destroyed        false
version          1

=== Data ===
Key    Value
---    -----
bar    precious

Read version 2:

bash-5.1# vault kv get -version=2 kv/foo

====== Metadata ======
Key              Value
---              -----
created_time     2021-09-08T18:23:14.4154928Z
deletion_time    n/a
destroyed        false
version          2

=== Data ===
Key    Value
---    -----
bar    copper

Delete the latest version (e.g., version 2):

bash-5.1# vault kv delete kv/foo

Success! Data deleted (if it existed) at: kv/foo

Delete version 1:

bash-5.1# vault kv delete -versions=1 kv/foo

Success! Data deleted (if it existed) at: kv/foo

You can undelete as well:

bash-5.1# vault kv undelete -versions=1 kv/foo

Success! Data written to: kv/undelete/foo

Delete is akin to a soft delete. If you want to remove the underlying metadata, you'll have to use the destroy command:

bash-5.1# vault kv destroy -versions=1 kv/foo

Success! Data written to: kv/destroy/foo

Review v1 and v2 to view all the available commands.

Take note of the audit log. Each of the above requests were logged!

API

You can also interact with Vault via the HTTP API. We'll make requests against v2 of the API. Open a new terminal tab, and then set the root token as an environment variable:

$ export VAULT_TOKEN=your_token_goes_here

Create a new secret called foo with a value of world:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -H "Content-Type: application/json" \
    -X POST \
    -d '{ "data": { "foo": "world" } }' \
    http://127.0.0.1:8200/v1/kv/data/hello

Read the secret:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/hello

The JSON response should contain a data key with a value similar to:

"data": {
  "data":{
    "foo": "world"
  },
  "metadata": {
    "created_time": "2021-09-08T18:30:32.5140484Z",
    "deletion_time": "",
    "destroyed": false,
    "version": 1
  }
}

保險庫 API

Try adding new versions, deleting, and destroying on your own.

UI

The UI should be up at running at http://localhost:8200/ui/vault. Use the root token to login. Then, explore the Key/Value backend on your own:

保險庫用戶界面

Policies

Thus far we've been using the root policy to interact with the API. Let's set up a policy that only has read access.

Add a new config file called app-policy.json to "vault/policies":

{
  "path": {
    "kv/data/app/*": {
      "policy": "read"
    }
  }
}

Create a new policy back in the bash session:

bash-5.1# vault policy write app /vault/policies/app-policy.json

Success! Uploaded policy: app

Then, create a new token:

bash-5.1# vault token create -policy=app

Key                  Value
---                  -----
token                s.ZOUMx3RIhVRhI4ijlZg8KXRQ
token_accessor       TT53xOxbIfGjI7l4392gjXcg
token_duration       768h
token_renewable      true
token_policies       ["app" "default"]
identity_policies    []
policies             ["app" "default"]

Within another new terminal tab (you should now have three), add the VAULT_TOKEN environment variable with the new token:

$ export VAULT_TOKEN=your_token_goes_here

Try to read the foo secret that we previously set:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/hello

You should not have the correct permissions to view that secret:

{
  "errors":[
    "1 error occurred:\n\t* permission denied\n\n"
  ]
}

Why can't we even read it? Jump back to the policy config in vault-config.json. kv/data/app/* indicates that the policy can only read from the app path.

As you've probably already noticed, nearly everything in Vault is path-based.

Back within the bash session in the container, add a new secret to the app/test path:

bash-5.1# vault kv put kv/app/test ping=pong

Key              Value
---              -----
created_time     2021-09-08T18:40:35.2694047Z
deletion_time    n/a
destroyed        false
version          1

You should be able to view the secret using the token associated with the app policy:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/app/test

Policies can be managed from the UI as well:

保險庫用戶界面

Encryption as a Service

Before we look at dynamic secrets, let's quickly review the Transit backend, which can be used as an "encryption as a service" for:

  • Encrypting and decrypting data "in-transit" without storing it inside Vault
  • Easily integrating encryption into your application workflow

Back within the bash session in the container, enable Transit:

bash-5.1# vault secrets enable transit

Success! Enabled the transit secrets engine at: transit/

Configure a named encryption key:

bash-5.1# vault write -f transit/keys/foo

Success! Data written to: transit/keys/foo

Encrypt:

bash-5.1# vault write transit/encrypt/foo plaintext=$(base64 <<< "my precious")

Key           Value
---           -----
ciphertext    vault:v1:cFnk5AQLE9Mg+mZ7Ej17vRmYT5aqheikdZQ1FC4vre5jAod0L/uHDA==

Decrypt:

bash-5.1# vault write transit/decrypt/foo ciphertext=vault:v1:cFnk5AQLE9Mg+mZ7Ej17vRmYT5aqheikdZQ1FC4vre5jAod0L/uHDA==

Key          Value
---          -----
plaintext    bXkgcHJlY2lvdXMK

Decode:

bash-5.1# base64 -d <<< "bXkgcHJlY2lvdXMK"

my precious

Test it out in the UI as well:

保險庫用戶界面

Dynamic Secrets

As mentioned, Vault supports a number of dynamic secret backends for generating secrets dynamically when needed. For example, with the AWS and Google Cloud backends, you can create access credentials based on IAM policies. The Databases backend, meanwhile, generates database credentials based on configured roles.

Dynamic Secrets:

  • are generated on demand
  • have limited access based on role
  • are leased for a period of time
  • can be revoked
  • come with an audit trail

Let's look at how to generate AWS credentials using the AWS backend.

AWS Credentials

Enable the AWS secrets backend:

bash-5.1# vault secrets enable -path=aws aws

Success! Enabled the aws secrets engine at: aws/

Authenticate:

bash-5.1# vault write aws/config/root access_key=foo secret_key=bar

Success! Data written to: aws/config/root

Make sure to replace foo and bar with your AWS access key id and secret key, respectively.

Create role:

bash-5.1# vault write aws/roles/ec2-read credential_type=iam_user policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Success! Data written to: aws/roles/ec2-read

Here, we created a new role based on AmazonEC2ReadOnlyAccess, which is an AWS-managed policy. As the name suggests, it give users read-only access to the EC2 console; they cannot perform any actions or create new resources. You can also use an inline policy to create a custom role based on your individual needs. We'll look at an example of this shortly. Refer to the AWS Secrets Engine docs for more info.

Remember: Dynamic Secrets are generated only when they are requested (i.e., a web app requests access to S3). They are not available in the store before this.

Create a new set of credentials:

bash-5.1# vault read aws/creds/ec2-read

Key                Value
---                -----
lease_id           aws/creds/ec2-read/9KdO6J7KVBiSwOPEvwrqqALG
lease_duration     768h
lease_renewable    true
access_key         AKIAZ4DZAKZKEULSDW5A
secret_key         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
security_token     <nil>

You should now be able to see the user within the "Users" section on the IAM console on AWS:

對不起

Leases and Revocation

In this section, we'll take a quick look at how to define a custom lease period and revoke a secret before the end of that period.

Create a new AWS role:

bash-5.1# vault write aws/roles/foo credential_type=iam_user policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Success! Data written to: aws/roles/foo

Take note of the lease_duration when you create a new AWS credential:

bash-5.1# vault read aws/creds/foo

Key                Value
---                -----
lease_id           aws/creds/foo/F0oBbnBIHEoz0ywVVtbuJB7r
lease_duration     768h
lease_renewable    true
access_key         AKIAZ4DZAKZKLJKB7CPX
secret_key         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
security_token     <nil>

What if you only wanted the lease period for all AWS IAM dynamic secrets to be 30 minutes?

bash-5.1# vault write aws/config/lease lease=1800s lease_max=1800s

In this example, since lease_max is the same as lease, you won't be able to renew the token. If you set the lease_max to 3600s, you'd be able to renew the lease once. For more, review the Tokens and Leases guide.

Create a new credential:

bash-5.1# vault read aws/creds/foo

Key                Value
---                -----
lease_id           aws/creds/foo/xQlJpKDS1ljE9Awz0aywXgbB
lease_duration     30m
lease_renewable    true
access_key         AKIAZ4DZAKZKJPL5OM5W
secret_key         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
security_token     <nil>

Want to quickly revoke this credential? Grab the lease_id and then run:

bash-5.1# vault lease revoke aws/creds/foo/xQlJpKDS1ljE9Awz0aywXgbB

Want to revoke all AWS creds?

bash-5.1# vault lease revoke -prefix aws/

Refer to the Lease, Renew, and Revoke guide for more info these concepts.

Consul Backend

Thus far, we've been using the Filesystem backend. This will not scale beyond a single server, so it does not take advantage of Vault's high availability. Fortunately, there are a number of other Storage backends, like the Consul backend, designed for distributed systems.

To set up Consul, start by updating the docker-compose.yml file:

version: '3.8'

services:

  vault:
    build:
      context: ./vault
      dockerfile: Dockerfile
    ports:
      - 8200:8200
    volumes:
      - ./vault/config:/vault/config
      - ./vault/policies:/vault/policies
      - ./vault/data:/vault/data
      - ./vault/logs:/vault/logs
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_API_ADDR=http://127.0.0.1:8200
    command: server -config=/vault/config/vault-config.json
    cap_add:
      - IPC_LOCK
    depends_on:
      - consul

  consul:
    build:
      context: ./consul
      dockerfile: Dockerfile
    ports:
      - 8500:8500
    command: agent -server -bind 0.0.0.0 -client 0.0.0.0 -bootstrap-expect 1 -config-file=/consul/config/config.json
    volumes:
      - ./consul/config/consul-config.json:/consul/config/config.json
      - ./consul/data:/consul/data

Add a new directory in the project root called "consul", and then add a new Dockerfile to that newly created directory:

# base image
FROM alpine:3.14

# set consul version
ENV CONSUL_VERSION 1.10.2

# create a new directory
RUN mkdir /consul

# download dependencies
RUN apk --no-cache add \
      bash \
      ca-certificates \
      wget

# download and set up consul
RUN wget --quiet --output-document=/tmp/consul.zip https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_linux_amd64.zip && \
    unzip /tmp/consul.zip -d /consul && \
    rm -f /tmp/consul.zip && \
    chmod +x /consul/consul

# update PATH
ENV PATH="PATH=$PATH:$PWD/consul"

# add the config file
COPY ./config/consul-config.json /consul/config/config.json

# expose ports
EXPOSE 8300 8400 8500 8600

# run consul
ENTRYPOINT ["consul"]

Next, within the "consul" directory add two new directories: "config" and "data". Then, within "config", add a config file called consul-config.json:

{
  "datacenter": "localhost",
  "data_dir": "/consul/data",
  "log_level": "DEBUG",
  "server": true,
  "ui": true,
  "ports": {
    "dns": 53
  }
}

請務必查看 Consul 文檔中的配置選項,以獲取有關上述選項的更多信息。

“consul”目錄現在應該如下所示:

├── Dockerfile
├── config
│   └── consul-config.json
└── data

退出 bash 會話。關閉容器,然後更新 Vault 配置文件:

{
  "backend": {
    "consul": {
      "address": "consul:8500",
      "path": "vault/"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

所以,現在我們使用Consul後端而不是文件系統。我們使用了服務的名稱consul,作為地址的一部分。path鍵定義了在 Consul 的鍵/值存儲中存儲 Vault 數據的路徑。

清除“vault/data”目錄中的所有文件和文件夾以刪除文件系統後端。構建新圖像並啟動容器:

$ docker-compose down
$ docker-compose up -d --build

通過在瀏覽器中導航到http://localhost:8500/ui來確保一切順利:

領事

從 CLI 或 UI 中對此進行測試。

命令行界面

在 Vault 容器中創建一個新的 bash 會話:

$ docker-compose exec vault bash

然後,運行:

# Init
bash-5.1# vault operator init

# Unseal
bash-5.1# vault operator unseal

# Authenticate
bash-5.1# vault login

# Enable secrets
bash-5.1# vault secrets enable kv

# Add a new static secret
bash-5.1# vault kv put kv/foo bar=precious

# Read it back
bash-5.1# vault kv get kv/foo

用戶界面

保險庫領事

請注意“保管庫/數據”中沒有文件或文件夾。你為什麼認為這是?

想要添加另一個 Consul 服務器嗎?向docker-compose.yml添加新服務:

consul-worker:
  build:
    context: ./consul
    dockerfile: Dockerfile
  command: agent -server -join consul -config-file=/consul/config/config.json
  volumes:
    - ./consul/config/consul-config.json:/consul/config/config.json
  depends_on:
    - consul

在這裡,我們使用join命令將此代理連接到現有集群。請注意我們如何只需要引用服務名稱:consul

然後:

  1. 退出 bash 會話(如有必要)
  2. 放下容器
  3. 清除“consul/data”中的數據目錄(為什麼?)
  4. 旋轉容器備份並測試

領事

結論

在本教程中,我們介紹瞭如何在 Docker 容器中設置和運行 Vault 和 Consul。您現在應該清楚地了解如何與 Vault 交互並執行基本操作。

vault-consul-docker存儲庫中獲取最終代碼。也請查看演示文稿

來源:  https ://testdriven.io

#vault #consul 

如何設置和使用 Hashicorp 的 Vault 和 Consul 項目
Noelia  Graham

Noelia Graham

1660618620

Gérer Les Secrets Avec Vault Et Consul

Le didacticiel suivant explique comment configurer et utiliser les projets Vault et Consul de Hashicorp pour stocker et gérer en toute sécurité des secrets.

Nous commencerons par lancer une seule instance de Vault dans un conteneur Docker, puis passerons à la gestion des secrets statiques et dynamiques avec la fonctionnalité de "chiffrement en tant que service" de Vault. Ensuite, nous ajouterons Consul au mélange et verrons comment mettre à l'échelle Vault.

Principales dépendances :

  • Docker v20.10.8
  • Docker Compose v1.29.2
  • Coffre-fort v1.8.2
  • Consul v1.10.2

Objectifs

À la fin de ce didacticiel, vous devriez être en mesure de :

  1. Expliquez ce qu'est Vault et pourquoi vous voudrez peut-être l'utiliser
  2. Décrire l'architecture de base de Vault ainsi que les secrets dynamiques et statiques, les différents backends (stockage, secret, authentification, audit) et comment Vault peut être utilisé comme "chiffrement en tant que service"
  3. Configurer et exécuter Vault et Consul avec Docker
  4. Faites tourner Vault avec le backend du système de fichiers
  5. Initialiser et desceller Vault
  6. S'authentifier auprès de Vault
  7. Configurer un backend d'audit pour consigner toutes les interactions avec Vault
  8. Travailler avec des secrets statiques et dynamiques via la CLI, l'API HTTP et l'interface utilisateur
  9. Créer une stratégie Vault pour limiter l'accès à un chemin spécifique
  10. Utilisez le backend Transit comme "chiffrement en tant que service"
  11. Configurer Consul pour qu'il fonctionne avec Vault en tant que backend de stockage pour les secrets
  12. Définir une période de bail personnalisée pour un secret et révoquer un secret avant la fin de cette période

Qu'est-ce que le coffre-fort ?

Vault est un outil open source utilisé pour stocker et gérer en toute sécurité des secrets.

Qu'est-ce qu'un secret ? Les secrets, dans le contexte de ce didacticiel, sont des informations sécurisées ou personnellement identifiables telles que les informations d'identification de la base de données, les clés SSH, les noms d'utilisateur et les mots de passe, les informations d'identification AWS IAM, les jetons API, les numéros de sécurité sociale, les numéros de carte de crédit, pour n'en nommer que quelques-uns.

Prenez un moment pour réfléchir à la manière dont votre équipe gère et distribue actuellement les secrets :

  1. Qui y a accès ?
  2. Qui les gère ?
  3. Comment contrôlez-vous qui y a accès ?
  4. Comment vos applications les obtiennent-elles ?
  5. Comment sont-ils mis à jour ?
  6. Comment sont-ils révoqués ?

Vault apporte des réponses à ces questions et aide à résoudre les problèmes suivants concernant la gestion des secrets :

ProblèmesObjectifs du coffre-fort
Les secrets sont partout.Vault est la seule source de vérité pour tous les secrets.
Ils sont généralement non cryptés.Vault gère le chiffrement (pendant le transit et au repos) prêt à l'emploi.
Il est difficile de les générer dynamiquement.Les secrets peuvent être générés dynamiquement.
Il est encore plus difficile de les louer et de les révoquer.Les secrets peuvent être loués et révoqués.
Il n'y a pas de piste d'audit.Il existe une piste d'audit pour générer et utiliser des secrets.

Vault a un certain nombre de pièces mobiles, il peut donc prendre un certain temps pour se familiariser avec l'architecture globale. Prenez un moment pour consulter le guide d' architecture , en prenant note des backends suivants :

BackendUtilisationExemples
StockageOù sont stockés les secretsConsul *, Système de fichiers *, In-Memory, PostgreSQL, S3
SecretGère les secrets statiques ou dynamiquesAWS *, Bases de données, Clé/Valeur *, RabbitMQ, SSH
AuthentificationGère l'authentification et l'autorisationAWS, Azure, Google Cloud, GitHub, jetons *, nom d'utilisateur et mot de passe
AuditEnregistre toutes les demandes et réponsesFichier *, Syslog, Socket

*utilisé dans ce tutoriel

Sur ce, commençons à utiliser Vault.

Backend du système de fichiers

Pour être opérationnel rapidement, nous utiliserons le backend du système de fichiers pour stocker les secrets au repos.

Le backend du système de fichiers ne doit être utilisé que pour le développement local ou un déploiement Vault sur un seul serveur, car il ne prend pas en charge la haute disponibilité.

Créez un nouveau répertoire de projet :

$ mkdir vault-consul-docker && cd vault-consul-docker

Ajoutez ensuite les dossiers suivants :

└── vault
    ├── config
    ├── data
    ├── logs
    └── policies

Ajoutez un Dockerfile au répertoire "vault":

# base image
FROM alpine:3.14

# set vault version
ENV VAULT_VERSION 1.8.2

# create a new directory
RUN mkdir /vault

# download dependencies
RUN apk --no-cache add \
      bash \
      ca-certificates \
      wget

# download and set up vault
RUN wget --quiet --output-document=/tmp/vault.zip https://releases.hashicorp.com/vault/${VAULT_VERSION}/vault_${VAULT_VERSION}_linux_amd64.zip && \
    unzip /tmp/vault.zip -d /vault && \
    rm -f /tmp/vault.zip && \
    chmod +x /vault

# update PATH
ENV PATH="PATH=$PATH:$PWD/vault"

# add the config file
COPY ./config/vault-config.json /vault/config/vault-config.json

# expose port 8200
EXPOSE 8200

# run vault
ENTRYPOINT ["vault"]

Ensuite, ajoutez un fichier docker-compose.yml à la racine du projet :

version: '3.8'

services:

  vault:
    build:
      context: ./vault
      dockerfile: Dockerfile
    ports:
      - 8200:8200
    volumes:
      - ./vault/config:/vault/config
      - ./vault/policies:/vault/policies
      - ./vault/data:/vault/data
      - ./vault/logs:/vault/logs
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_API_ADDR=http://127.0.0.1:8200
    command: server -config=/vault/config/vault-config.json
    cap_add:
      - IPC_LOCK

Ajoutez un fichier de configuration appelé vault-config.json à "vault/config":

{
  "backend": {
    "file": {
      "path": "vault/data"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Ici, nous avons configuré Vault pour utiliser le backend du système de fichiers, défini l' écouteur pour Vault, désactivé TLS et activé l' interface utilisateur Vault . Consultez la documentation pour plus d'informations sur la configuration de Vault.

Nous pouvons maintenant créer l'image et faire tourner le conteneur :

$ docker-compose up -d --build

Extrayez les journaux Docker pour vous assurer qu'il n'y a pas d'erreurs dans la compilation :

$ docker-compose logs

Vous devriez voir quelque chose de similaire à :

Attaching to vault-consul-docker_vault_1
vault_1  | ==> Vault server configuration:
vault_1  |
vault_1  |              Api Address: http://127.0.0.1:8200
vault_1  | 2021-09-08T14:48:35.014Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
vault_1  |                      Cgo: disabled
vault_1  |          Cluster Address: https://127.0.0.1:8201
vault_1  |               Go Version: go1.16.7
vault_1  |               Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
vault_1  |                Log Level: info
vault_1  |                    Mlock: supported: true, enabled: true
vault_1  |            Recovery Mode: false
vault_1  |                  Storage: file
vault_1  |                  Version: Vault v1.8.2
vault_1  |              Version Sha: aca76f63357041a43b49f3e8c11d67358496959f
vault_1  |
vault_1  | ==> Vault server started! Log data will stream in below:
vault_1  |

Initialisation et descellement

Démarrez une session bash dans le conteneur en cours d'exécution :

$ docker-compose exec vault bash

Dans le shell, initialisez Vault :

bash-5.1# vault operator init

Prenez note des clés de descellement et du jeton racine initial. Vous devrez fournir trois des clés de descellement chaque fois que le serveur Vault est rescellé ou redémarré.

Pourquoi 3 clés ? Passez en revue le partage secret de Shamir .

Vous pouvez désormais desceller Vault à l'aide de trois des clés :

bash-5.1# vault operator unseal
Unseal Key (will be hidden):

Exécutez cette commande deux fois de plus, en utilisant des clés différentes à chaque fois. Une fois fait, assurez Sealed- vous que false:

Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    5
Threshold       3
Version         1.8.2
Storage Type    file
Cluster Name    vault-cluster-8fcf9d05
Cluster ID      d86e0274-ad9c-d2c1-d6ec-baeab410797b
HA Enabled      false

À l'aide du jeton racine, vous pouvez désormais vous authentifier :

bash-5.1# vault login
Token (will be hidden):

Vous devriez voir quelque chose de similaire à :

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                s.c0kYHWiOTqQvtR8JuSeTz6sZ
token_accessor       3FQJVxOY5C1brzlHHQSFaCdZ
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Gardez à l'esprit que cela utilise la politique racine. En production, vous souhaiterez configurer des stratégies avec différents niveaux d'accès. Nous verrons comment procéder sous peu.

initialisation du coffre-fort

Vault est maintenant descellé et prêt à l'emploi.

Audit

Avant de tester la fonctionnalité, activons un périphérique d'audit :

bash-5.1# vault audit enable file file_path=/vault/logs/audit.log

Success! Enabled the file audit device at: file/

Vous devriez maintenant pouvoir afficher les journaux localement dans "vault/logs". Pour tester, exécutez la commande suivante pour afficher tous les appareils d'audit activés :

bash-5.1# vault audit list

Path     Type    Description
----     ----    -----------
file/    file    n/a

La demande et la réponse qui s'ensuit doivent être consignées dans le coffre-fort/logs/audit.log . Regarde.

secrets

There are two types of secrets in Vault: static and dynamic.

Static secrets (think encrypted Redis or Memcached) have refresh intervals but they do not expire unless explicitly revoked. They are defined ahead of time with the Key/Value backend (formerly the "generic" backend) and then shared.

stockage secret sécurisé

Les secrets dynamiques sont générés à la demande. Ils ont fait respecter les baux et expirent généralement après une courte période de temps. Puisqu'ils n'existent pas jusqu'à ce qu'ils soient accessibles, il y a moins d'exposition - donc les secrets dynamiques sont beaucoup plus sécurisés. Vault est livré avec un certain nombre de backends dynamiques, à savoir AWS , Databases , Google Cloud , Consul et RabbitMQ .

Consultez l' article de blog Why We Need Dynamic Secrets pour plus d'informations sur les avantages de l'utilisation des secrets dynamiques.

Secrets statiques

Vault peut être géré via l' interface de ligne de commande , l' API HTTP ou l' interface utilisateur .

CLI

Toujours dans la session bash du conteneur, nous pouvons créer, lire, mettre à jour et supprimer des secrets. Nous verrons également comment versionner et restaurer les secrets.

Activez les secrets avec la commande suivante :

bash-5.1# vault secrets enable kv

Success! Enabled the kv secrets engine at: kv/

Créez un nouveau secret avec une clé baret une valeur preciousdans le kv/foochemin :

bash-5.1# vault kv put kv/foo bar=precious

Success! Data written to: kv/foo

Lis:

bash-5.1# vault kv get kv/foo

=== Data ===
Key    Value
---    -----
bar    precious

Pour travailler avec différentes versions d'une clé spécifique, nous devrons effectuer une mise à niveau vers la v2 du backend clé/valeur :

bash-5.1# vault kv enable-versioning kv/

Success! Tuned the secrets engine at: kv/

Ajoutez la version 2 en mettant à jour la valeurcopper :

bash-5.1# vault kv put kv/foo bar=copper

Key              Value
---              -----
created_time     2021-09-08T18:23:14.4154928Z
deletion_time    n/a
destroyed        false
version          2

Lire la version 1 :

bash-5.1# vault kv get -version=1 kv/foo

====== Metadata ======
Key              Value
---              -----
created_time     2021-09-08T18:22:37.2548824Z
deletion_time    n/a
destroyed        false
version          1

=== Data ===
Key    Value
---    -----
bar    precious

Lire la version 2 :

bash-5.1# vault kv get -version=2 kv/foo

====== Metadata ======
Key              Value
---              -----
created_time     2021-09-08T18:23:14.4154928Z
deletion_time    n/a
destroyed        false
version          2

=== Data ===
Key    Value
---    -----
bar    copper

Supprimez la dernière version (par exemple, la version 2) :

bash-5.1# vault kv delete kv/foo

Success! Data deleted (if it existed) at: kv/foo

Supprimer la version 1 :

bash-5.1# vault kv delete -versions=1 kv/foo

Success! Data deleted (if it existed) at: kv/foo

Vous pouvez également restaurer :

bash-5.1# vault kv undelete -versions=1 kv/foo

Success! Data written to: kv/undelete/foo

La suppression s'apparente à une suppression douce. Si vous souhaitez supprimer les métadonnées sous-jacentes, vous devrez utiliser la commande destroy :

bash-5.1# vault kv destroy -versions=1 kv/foo

Success! Data written to: kv/destroy/foo

Passez en revue v1 et v2 pour afficher toutes les commandes disponibles.

Prenez note du journal d'audit. Chacune des demandes ci-dessus a été enregistrée !

API

Vous pouvez également interagir avec Vault via l' API HTTP . Nous ferons des requêtes contre la v2 de l'API. Ouvrez un nouvel onglet de terminal, puis définissez le jeton racine en tant que variable d'environnement :

$ export VAULT_TOKEN=your_token_goes_here

Créez un nouveau secret appelé fooavec une valeur deworld :

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -H "Content-Type: application/json" \
    -X POST \
    -d '{ "data": { "foo": "world" } }' \
    http://127.0.0.1:8200/v1/kv/data/hello

Lisez le secret :

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/hello

La réponse JSON doit contenir une dataclé avec une valeur similaire à :

"data": {
  "data":{
    "foo": "world"
  },
  "metadata": {
    "created_time": "2021-09-08T18:30:32.5140484Z",
    "deletion_time": "",
    "destroyed": false,
    "version": 1
  }
}

API de coffre-fort

Essayez d'ajouter de nouvelles versions, de supprimer et de détruire vous-même.

interface utilisateur

L' interface utilisateur doit être opérationnelle sur http://localhost:8200/ui/vault . Utilisez le jeton racine pour vous connecter. Ensuite, explorez le backend clé/valeur par vous-même :

interface utilisateur du coffre-fort

Stratégies

Jusqu'à présent, nous avons utilisé la politique racine pour interagir avec l'API. Configurons une stratégie qui n'a qu'un accès en lecture.

Ajoutez un nouveau fichier de configuration appelé app-policy.json à "vault/policies":

{
  "path": {
    "kv/data/app/*": {
      "policy": "read"
    }
  }
}

Créez une nouvelle stratégie dans la session bash :

bash-5.1# vault policy write app /vault/policies/app-policy.json

Success! Uploaded policy: app

Ensuite, créez un nouveau jeton :

bash-5.1# vault token create -policy=app

Key                  Value
---                  -----
token                s.ZOUMx3RIhVRhI4ijlZg8KXRQ
token_accessor       TT53xOxbIfGjI7l4392gjXcg
token_duration       768h
token_renewable      true
token_policies       ["app" "default"]
identity_policies    []
policies             ["app" "default"]

Dans un autre nouvel onglet de terminal (vous devriez maintenant en avoir trois), ajoutez la VAULT_TOKENvariable d'environnement avec le nouveau jeton :

$ export VAULT_TOKEN=your_token_goes_here

Essayez de lire le foosecret que nous avons défini précédemment :

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/hello

Vous ne devez pas disposer des autorisations appropriées pour afficher ce secret :

{
  "errors":[
    "1 error occurred:\n\t* permission denied\n\n"
  ]
}

Pourquoi ne pouvons-nous même pas le lire ? Revenez à la configuration de la stratégie dans le coffre-fort-config.json . kv/data/app/*indique que la stratégie ne peut lire qu'à partir du appchemin.

Comme vous l'avez probablement déjà remarqué, presque tout dans Vault est basé sur un chemin.

De retour dans la session bash du conteneur, ajoutez un nouveau secret au app/testchemin :

bash-5.1# vault kv put kv/app/test ping=pong

Key              Value
---              -----
created_time     2021-09-08T18:40:35.2694047Z
deletion_time    n/a
destroyed        false
version          1

Vous devriez pouvoir afficher le secret à l'aide du jeton associé à la appstratégie :

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/app/test

Les stratégies peuvent également être gérées à partir de l'interface utilisateur :

interface utilisateur du coffre-fort

Chiffrement en tant que service

Avant d'aborder les secrets dynamiques, passons rapidement en revue le backend Transit , qui peut être utilisé comme un "chiffrement en tant que service" pour :

  • Chiffrement et déchiffrement des données "en transit" sans les stocker dans Vault
  • Intégrez facilement le chiffrement dans le flux de travail de votre application

De retour dans la session bash du conteneur, activez Transit :

bash-5.1# vault secrets enable transit

Success! Enabled the transit secrets engine at: transit/

Configurez une clé de chiffrement nommée :

bash-5.1# vault write -f transit/keys/foo

Success! Data written to: transit/keys/foo

Crypter:

bash-5.1# vault write transit/encrypt/foo plaintext=$(base64 <<< "my precious")

Key           Value
---           -----
ciphertext    vault:v1:cFnk5AQLE9Mg+mZ7Ej17vRmYT5aqheikdZQ1FC4vre5jAod0L/uHDA==

Décrypter :

bash-5.1# vault write transit/decrypt/foo ciphertext=vault:v1:cFnk5AQLE9Mg+mZ7Ej17vRmYT5aqheikdZQ1FC4vre5jAod0L/uHDA==

Key          Value
---          -----
plaintext    bXkgcHJlY2lvdXMK

Décoder:

bash-5.1# base64 -d <<< "bXkgcHJlY2lvdXMK"

my precious

Testez-le également dans l'interface utilisateur :

interface utilisateur du coffre-fort

Secrets dynamiques

Comme mentionné, Vault prend en charge un certain nombre de backends secrets dynamiques pour générer des secrets de manière dynamique en cas de besoin. Par exemple, avec les backends AWS et Google Cloud , vous pouvez créer des identifiants d'accès basés sur des stratégies IAM. Le backend des bases de données , quant à lui, génère des informations d'identification de base de données en fonction des rôles configurés.

Secrets dynamiques :

  • sont générés à la demande
  • avoir un accès limité en fonction du rôle
  • sont loués pour une durée
  • peut être révoqué
  • venir avec une piste d'audit

Voyons comment générer des informations d'identification AWS à l'aide du backend AWS.

Identifiants AWS

Activez le backend des secrets AWS :

bash-5.1# vault secrets enable -path=aws aws

Success! Enabled the aws secrets engine at: aws/

Authentifier:

bash-5.1# vault write aws/config/root access_key=foo secret_key=bar

Success! Data written to: aws/config/root

Assurez-vous de remplacer fooet barpar votre ID de clé d'accès AWS et votre clé secrète, respectivement.

Créer un rôle :

bash-5.1# vault write aws/roles/ec2-read credential_type=iam_user policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Success! Data written to: aws/roles/ec2-read

Ici, nous avons créé un nouveau rôle basé sur , qui est une stratégieAmazonEC2ReadOnlyAccess gérée par AWS . Comme son nom l'indique, il donne aux utilisateurs un accès en lecture seule à la console EC2 ; ils ne peuvent effectuer aucune action ou créer de nouvelles ressources. Vous pouvez également utiliser une stratégie en ligne pour créer un rôle personnalisé en fonction de vos besoins individuels. Nous en verrons un exemple sous peu. Reportez-vous à la documentation AWS Secrets Engine pour plus d'informations.

N'oubliez pas : les secrets dynamiques sont générés uniquement lorsqu'ils sont demandés (par exemple, une application Web demande l'accès à S3). Ils ne sont pas disponibles dans le magasin avant cela.

Créez un nouvel ensemble d'identifiants :

bash-5.1# vault read aws/creds/ec2-read

Key                Value
---                -----
lease_id           aws/creds/ec2-read/9KdO6J7KVBiSwOPEvwrqqALG
lease_duration     768h
lease_renewable    true
access_key         AKIAZ4DZAKZKEULSDW5A
secret_key         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
security_token     <nil>

Vous devriez maintenant pouvoir voir l'utilisateur dans la section « Utilisateurs » de la console IAM sur AWS :

Je suis désolé

Baux et révocation

Dans cette section, nous verrons rapidement comment définir une période de bail personnalisée et révoquer un secret avant la fin de cette période.

Créez un nouveau rôle AWS :

bash-5.1# vault write aws/roles/foo credential_type=iam_user policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Success! Data written to: aws/roles/foo

Prenez note du lease_durationlorsque vous créez un nouvel identifiant AWS :

bash-5.1# vault read aws/creds/foo

Key                Value
---                -----
lease_id           aws/creds/foo/F0oBbnBIHEoz0ywVVtbuJB7r
lease_duration     768h
lease_renewable    true
access_key         AKIAZ4DZAKZKLJKB7CPX
secret_key         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
security_token     <nil>

Et si vous vouliez que la période de bail pour tous les secrets dynamiques AWS IAM soit de 30 minutes ?

bash-5.1# vault write aws/config/lease lease=1800s lease_max=1800s

Dans cet exemple, étant donné que lease_maxest identique à lease, vous ne pourrez pas renouveler le jeton. Si vous définissez le lease_maxsur 3600s, vous pourrez renouveler le bail une fois. Pour en savoir plus, consultez le guide Tokens and Leases .

Créez un nouvel identifiant :

bash-5.1# vault read aws/creds/foo

Key                Value
---                -----
lease_id           aws/creds/foo/xQlJpKDS1ljE9Awz0aywXgbB
lease_duration     30m
lease_renewable    true
access_key         AKIAZ4DZAKZKJPL5OM5W
secret_key         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
security_token     <nil>

Vous souhaitez révoquer rapidement cet identifiant ? Saisissez le lease_idpuis exécutez :

bash-5.1# vault lease revoke aws/creds/foo/xQlJpKDS1ljE9Awz0aywXgbB

Vous souhaitez révoquer tous les identifiants AWS ?

bash-5.1# vault lease revoke -prefix aws/

Reportez-vous au guide Bail, renouvellement et révocation pour plus d'informations sur ces concepts.

Back-end du consul

Jusqu'à présent, nous avons utilisé le backend du système de fichiers . Cela ne s'étendra pas au-delà d'un seul serveur, il ne tirera donc pas parti de la haute disponibilité de Vault. Heureusement, il existe un certain nombre d'autres backends de stockage , comme le backend Consul , conçu pour les systèmes distribués.

Pour configurer Consul , commencez par mettre à jour le fichier docker-compose.yml :

version: '3.8'

services:

  vault:
    build:
      context: ./vault
      dockerfile: Dockerfile
    ports:
      - 8200:8200
    volumes:
      - ./vault/config:/vault/config
      - ./vault/policies:/vault/policies
      - ./vault/data:/vault/data
      - ./vault/logs:/vault/logs
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_API_ADDR=http://127.0.0.1:8200
    command: server -config=/vault/config/vault-config.json
    cap_add:
      - IPC_LOCK
    depends_on:
      - consul

  consul:
    build:
      context: ./consul
      dockerfile: Dockerfile
    ports:
      - 8500:8500
    command: agent -server -bind 0.0.0.0 -client 0.0.0.0 -bootstrap-expect 1 -config-file=/consul/config/config.json
    volumes:
      - ./consul/config/consul-config.json:/consul/config/config.json
      - ./consul/data:/consul/data

Ajoutez un nouveau répertoire à la racine du projet appelé "consul", puis ajoutez un nouveau Dockerfile à ce répertoire nouvellement créé :

# base image
FROM alpine:3.14

# set consul version
ENV CONSUL_VERSION 1.10.2

# create a new directory
RUN mkdir /consul

# download dependencies
RUN apk --no-cache add \
      bash \
      ca-certificates \
      wget

# download and set up consul
RUN wget --quiet --output-document=/tmp/consul.zip https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_linux_amd64.zip && \
    unzip /tmp/consul.zip -d /consul && \
    rm -f /tmp/consul.zip && \
    chmod +x /consul/consul

# update PATH
ENV PATH="PATH=$PATH:$PWD/consul"

# add the config file
COPY ./config/consul-config.json /consul/config/config.json

# expose ports
EXPOSE 8300 8400 8500 8600

# run consul
ENTRYPOINT ["consul"]

Ensuite, dans le répertoire "consul", ajoutez deux nouveaux répertoires : "config" et "data". Ensuite, dans "config", ajoutez un fichier de configuration appelé consul-config.json :

{
  "datacenter": "localhost",
  "data_dir": "/consul/data",
  "log_level": "DEBUG",
  "server": true,
  "ui": true,
  "ports": {
    "dns": 53
  }
}

Assurez-vous de consulter les options de configuration de la documentation Consul pour plus d'informations sur les options ci-dessus.

Le répertoire "consul" devrait maintenant ressembler à :

├── Dockerfile
├── config
│   └── consul-config.json
└── data

Quittez la session bash. Arrêtez le conteneur, puis mettez à jour le fichier de configuration Vault :

{
  "backend": {
    "consul": {
      "address": "consul:8500",
      "path": "vault/"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Donc, nous utilisons maintenant le backend Consul au lieu du système de fichiers. Nous avons utilisé le nom du service, consul, dans le cadre de l'adresse. La pathclé définit le chemin dans le magasin clé/valeur de Consul où les données du coffre-fort seront stockées.

Effacez tous les fichiers et dossiers du répertoire "vault/data" pour supprimer le backend du système de fichiers. Créez les nouvelles images et faites tourner les conteneurs :

$ docker-compose down
$ docker-compose up -d --build

Assurez-vous que tout va bien en naviguant dans votre navigateur vers http://localhost:8500/ui :

interface utilisateur du consul

Testez ceci à partir de la CLI ou de l'interface utilisateur.

CLI

Créez une session bash dans le conteneur Vault :

$ docker-compose exec vault bash

Ensuite, lancez :

# Init
bash-5.1# vault operator init

# Unseal
bash-5.1# vault operator unseal

# Authenticate
bash-5.1# vault login

# Enable secrets
bash-5.1# vault secrets enable kv

# Add a new static secret
bash-5.1# vault kv put kv/foo bar=precious

# Read it back
bash-5.1# vault kv get kv/foo

interface utilisateur

consul du coffre-fort

Remarquez qu'il n'y a pas de fichiers ou de dossiers dans "vault/data". Pourquoi pensez-vous cela est?

Vous voulez ajouter un autre serveur Consul dans le mix ? Ajoutez un nouveau service à docker-compose.yml :

consul-worker:
  build:
    context: ./consul
    dockerfile: Dockerfile
  command: agent -server -join consul -config-file=/consul/config/config.json
  volumes:
    - ./consul/config/consul-config.json:/consul/config/config.json
  depends_on:
    - consul

Ici, nous avons utilisé la commande join pour connecter cet agent à un cluster existant. Remarquez comment nous devions simplement référencer le nom du service : consul.

Alors:

  1. Quitter la session bash (si nécessaire)
  2. Descendre les conteneurs
  3. Vider le répertoire data dans "consul/data" (Pourquoi ?)
  4. Faites tourner les conteneurs et testez

interface utilisateur du consul

Conclusion

Dans ce didacticiel, nous avons expliqué comment configurer et exécuter Vault et Consul dans un conteneur Docker. Vous devriez maintenant avoir une compréhension claire de la façon d'interagir avec Vault et d'effectuer des opérations de base.

Récupérez le code final dans le référentiel de coffre-fort-consul-docker . Découvrez également la présentation .

Source :  https://testdrive.io

#vault #consul 

Gérer Les Secrets Avec Vault Et Consul

Cómo Configurar Y Utilizar Los Proyectos Vault Y Consul De Hashicorp

El siguiente tutorial detalla cómo configurar y usar los proyectos Vault y Consul de Hashicorp para almacenar y administrar secretos de forma segura.

Comenzaremos activando una sola instancia de Vault dentro de un contenedor Docker y luego pasaremos a administrar secretos estáticos y dinámicos junto con la función de "cifrado como servicio" de Vault. Luego, agregaremos Consul a la mezcla y veremos cómo escalar Vault.

Dependencias principales:

  • Ventana acoplable v20.10.8
  • Docker-Componer v1.29.2
  • Bóveda v1.8.2
  • Cónsul v1.10.2

Objetivos

Al final de este tutorial, debería ser capaz de:

  1. Explicar qué es Vault y por qué es posible que desee usarlo
  2. Describir la arquitectura básica de Vault junto con los secretos dinámicos y estáticos, los diversos backends (almacenamiento, secreto, autenticación, auditoría) y cómo se puede usar Vault como un "cifrado como servicio".
  3. Configurar y ejecutar Vault y Consul con Docker
  4. Haga girar Vault con el backend del sistema de archivos
  5. Iniciar y desbloquear Vault
  6. Autenticar contra Vault
  7. Configure un backend de auditoría para registrar todas las interacciones con Vault
  8. Trabaje con secretos estáticos y dinámicos a través de la CLI, la API HTTP y la interfaz de usuario
  9. Cree una política de Vault para limitar el acceso a una ruta específica
  10. Use el backend de Transit como un "cifrado como servicio"
  11. Configure Consul para trabajar con Vault como backend de almacenamiento para secretos
  12. Definir un período de concesión personalizado para un secreto y revocar un secreto antes del final de ese período

¿Qué es la bóveda?

Vault es una herramienta de código abierto que se utiliza para almacenar y administrar secretos de forma segura.

What is a secret? Secrets, in the context of this tutorial, are securely-sensitive or personally identifiable info like database credentials, SSH keys, usernames and passwords, AWS IAM credentials, API tokens, Social Security Numbers, credit card numbers, just to name a few.

Take a moment to think about how your team currently manages and distributes secrets:

  1. Who has access to them?
  2. Who manages them?
  3. How do you control who has access to them?
  4. How do your apps get them?
  5. How are they updated?
  6. How are they revoked?

Vault provides answers to those questions and helps to solve the following problems with regard to secret management:

ProblemsVault's Goals
Secrets are everywhere.Vault is the single source of truth for all secrets.
They are generally unencrypted.Vault manages encryption (during transit and at rest) out of the box.
It's difficult to dynamically generate them.Secrets can be dynamically generated.
It's even more difficult to lease and revoke them.Secrets can be leased and revoked.
There's no audit trail.There's an audit trail for generating and using secrets.

Vault has a number of moving pieces so it can take some time to get up to speed with the overall architecture. Take a moment to review the Architecture guide, taking note of the following backends:

BackendUseExamples
StorageWhere secrets are storedConsul*, Filesystem*, In-Memory, PostgreSQL, S3
SecretHandles static or dynamic secretsAWS*, Databases, Key/Value*, RabbitMQ, SSH
AuthHandles authentication and authorizationAWS, Azure, Google Cloud, GitHub, Tokens*, Username & Password
AuditLogs all requests and responsesFile*, Syslog, Socket

* used in this tutorial

With that, let's start using Vault.

Filesystem Backend

To get up and running quickly, we'll use the Filesystem backend to store secrets at rest.

The filesystem backend should only be used for local development or a single-server Vault deployment since it does not support high availability.

Create a new project directory:

$ mkdir vault-consul-docker && cd vault-consul-docker

Then add the following folders:

└── vault
    ├── config
    ├── data
    ├── logs
    └── policies

Add a Dockerfile to the "vault" directory:

# base image
FROM alpine:3.14

# set vault version
ENV VAULT_VERSION 1.8.2

# create a new directory
RUN mkdir /vault

# download dependencies
RUN apk --no-cache add \
      bash \
      ca-certificates \
      wget

# download and set up vault
RUN wget --quiet --output-document=/tmp/vault.zip https://releases.hashicorp.com/vault/${VAULT_VERSION}/vault_${VAULT_VERSION}_linux_amd64.zip && \
    unzip /tmp/vault.zip -d /vault && \
    rm -f /tmp/vault.zip && \
    chmod +x /vault

# update PATH
ENV PATH="PATH=$PATH:$PWD/vault"

# add the config file
COPY ./config/vault-config.json /vault/config/vault-config.json

# expose port 8200
EXPOSE 8200

# run vault
ENTRYPOINT ["vault"]

Next, add a docker-compose.yml file to the project root:

version: '3.8'

services:

  vault:
    build:
      context: ./vault
      dockerfile: Dockerfile
    ports:
      - 8200:8200
    volumes:
      - ./vault/config:/vault/config
      - ./vault/policies:/vault/policies
      - ./vault/data:/vault/data
      - ./vault/logs:/vault/logs
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_API_ADDR=http://127.0.0.1:8200
    command: server -config=/vault/config/vault-config.json
    cap_add:
      - IPC_LOCK

Add a config file called vault-config.json to "vault/config":

{
  "backend": {
    "file": {
      "path": "vault/data"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Here, we configured Vault to use the Filesystem backend, defined the listener for Vault, disabled TLS, and enabled the Vault UI. Review the docs for more info on configuring Vault.

Now we can build the image and spin up the container:

$ docker-compose up -d --build

Pull up the Docker logs to make sure there were no errors in the build:

$ docker-compose logs

You should see something similar to:

Attaching to vault-consul-docker_vault_1
vault_1  | ==> Vault server configuration:
vault_1  |
vault_1  |              Api Address: http://127.0.0.1:8200
vault_1  | 2021-09-08T14:48:35.014Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
vault_1  |                      Cgo: disabled
vault_1  |          Cluster Address: https://127.0.0.1:8201
vault_1  |               Go Version: go1.16.7
vault_1  |               Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
vault_1  |                Log Level: info
vault_1  |                    Mlock: supported: true, enabled: true
vault_1  |            Recovery Mode: false
vault_1  |                  Storage: file
vault_1  |                  Version: Vault v1.8.2
vault_1  |              Version Sha: aca76f63357041a43b49f3e8c11d67358496959f
vault_1  |
vault_1  | ==> Vault server started! Log data will stream in below:
vault_1  |

Initializing and Unsealing

Start a bash session within the running container:

$ docker-compose exec vault bash

Within the shell, initialize Vault:

bash-5.1# vault operator init

Take note of the unseal keys and the initial root token. You will need to provide three of the unseal keys every time the Vault server is resealed or restarted.

Why 3 keys? Review Shamir's Secret Sharing.

Now you can unseal Vault using three of the keys:

bash-5.1# vault operator unseal
Unseal Key (will be hidden):

Run this command two more times, using different keys each time. Once done, make sure Sealed is false:

Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    5
Threshold       3
Version         1.8.2
Storage Type    file
Cluster Name    vault-cluster-8fcf9d05
Cluster ID      d86e0274-ad9c-d2c1-d6ec-baeab410797b
HA Enabled      false

Using the root token, you can now authenticate:

bash-5.1# vault login
Token (will be hidden):

You should see something similar to:

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                s.c0kYHWiOTqQvtR8JuSeTz6sZ
token_accessor       3FQJVxOY5C1brzlHHQSFaCdZ
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Keep in mind that this uses the root policy. In production you'll want to set up policies with different levels of access. We'll look at how to do this shortly.

inicio de bóveda

Vault is now unsealed and ready for use.

Auditing

Before we test out the functionality, let's enable an Audit Device:

bash-5.1# vault audit enable file file_path=/vault/logs/audit.log

Success! Enabled the file audit device at: file/

You should now be able to view the logs locally in "vault/logs". To test, run the following command to view all enabled Audit Devices:

bash-5.1# vault audit list

Path     Type    Description
----     ----    -----------
file/    file    n/a

The request and subsequent response should be logged in vault/logs/audit.log. Take a look.

Secrets

There are two types of secrets in Vault: static and dynamic.

Static secrets (think encrypted Redis or Memcached) have refresh intervals but they do not expire unless explicitly revoked. They are defined ahead of time with the Key/Value backend (formerly the "generic" backend) and then shared.

almacenamiento secreto seguro

Dynamic secrets are generated on demand. They have enforced leases and generally expire after a short period of time. Since they do not exist until they are accessed, there's less exposure -- so dynamic secrets are much more secure. Vault ships with a number of dynamic backends -- i.e., AWS, Databases, Google Cloud, Consul, and RabbitMQ.

Review the Why We Need Dynamic Secrets blog post for more info on the advantages of using dynamic secrets.

Static Secrets

Vault can be managed through the CLI, HTTP API, or UI.

CLI

Still within the bash session in the container, we can create, read, update, and delete secrets. We'll also look at how to version and roll back secrets.

Enable secrets with following command:

bash-5.1# vault secrets enable kv

Success! Enabled the kv secrets engine at: kv/

Create a new secret with a key of bar and value of precious within the kv/foo path:

bash-5.1# vault kv put kv/foo bar=precious

Success! Data written to: kv/foo

Read:

bash-5.1# vault kv get kv/foo

=== Data ===
Key    Value
---    -----
bar    precious

To work with different versions of a specific key, we'll need to upgrade to v2 of the Key/Value backend:

bash-5.1# vault kv enable-versioning kv/

Success! Tuned the secrets engine at: kv/

Add version 2 by updating the value to copper:

bash-5.1# vault kv put kv/foo bar=copper

Key              Value
---              -----
created_time     2021-09-08T18:23:14.4154928Z
deletion_time    n/a
destroyed        false
version          2

Read version 1:

bash-5.1# vault kv get -version=1 kv/foo

====== Metadata ======
Key              Value
---              -----
created_time     2021-09-08T18:22:37.2548824Z
deletion_time    n/a
destroyed        false
version          1

=== Data ===
Key    Value
---    -----
bar    precious

Read version 2:

bash-5.1# vault kv get -version=2 kv/foo

====== Metadata ======
Key              Value
---              -----
created_time     2021-09-08T18:23:14.4154928Z
deletion_time    n/a
destroyed        false
version          2

=== Data ===
Key    Value
---    -----
bar    copper

Delete the latest version (e.g., version 2):

bash-5.1# vault kv delete kv/foo

Success! Data deleted (if it existed) at: kv/foo

Delete version 1:

bash-5.1# vault kv delete -versions=1 kv/foo

Success! Data deleted (if it existed) at: kv/foo

You can undelete as well:

bash-5.1# vault kv undelete -versions=1 kv/foo

Success! Data written to: kv/undelete/foo

Delete is akin to a soft delete. If you want to remove the underlying metadata, you'll have to use the destroy command:

bash-5.1# vault kv destroy -versions=1 kv/foo

Success! Data written to: kv/destroy/foo

Review v1 and v2 to view all the available commands.

Take note of the audit log. Each of the above requests were logged!

API

You can also interact with Vault via the HTTP API. We'll make requests against v2 of the API. Open a new terminal tab, and then set the root token as an environment variable:

$ export VAULT_TOKEN=your_token_goes_here

Create a new secret called foo with a value of world:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -H "Content-Type: application/json" \
    -X POST \
    -d '{ "data": { "foo": "world" } }' \
    http://127.0.0.1:8200/v1/kv/data/hello

Read the secret:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/hello

The JSON response should contain a data key with a value similar to:

"data": {
  "data":{
    "foo": "world"
  },
  "metadata": {
    "created_time": "2021-09-08T18:30:32.5140484Z",
    "deletion_time": "",
    "destroyed": false,
    "version": 1
  }
}

API de bóveda

Try adding new versions, deleting, and destroying on your own.

UI

The UI should be up at running at http://localhost:8200/ui/vault. Use the root token to login. Then, explore the Key/Value backend on your own:

interfaz de usuario de la bóveda

Policies

Thus far we've been using the root policy to interact with the API. Let's set up a policy that only has read access.

Add a new config file called app-policy.json to "vault/policies":

{
  "path": {
    "kv/data/app/*": {
      "policy": "read"
    }
  }
}

Create a new policy back in the bash session:

bash-5.1# vault policy write app /vault/policies/app-policy.json

Success! Uploaded policy: app

Then, create a new token:

bash-5.1# vault token create -policy=app

Key                  Value
---                  -----
token                s.ZOUMx3RIhVRhI4ijlZg8KXRQ
token_accessor       TT53xOxbIfGjI7l4392gjXcg
token_duration       768h
token_renewable      true
token_policies       ["app" "default"]
identity_policies    []
policies             ["app" "default"]

Within another new terminal tab (you should now have three), add the VAULT_TOKEN environment variable with the new token:

$ export VAULT_TOKEN=your_token_goes_here

Try to read the foo secret that we previously set:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/hello

You should not have the correct permissions to view that secret:

{
  "errors":[
    "1 error occurred:\n\t* permission denied\n\n"
  ]
}

Why can't we even read it? Jump back to the policy config in vault-config.json. kv/data/app/* indicates that the policy can only read from the app path.

As you've probably already noticed, nearly everything in Vault is path-based.

Back within the bash session in the container, add a new secret to the app/test path:

bash-5.1# vault kv put kv/app/test ping=pong

Key              Value
---              -----
created_time     2021-09-08T18:40:35.2694047Z
deletion_time    n/a
destroyed        false
version          1

You should be able to view the secret using the token associated with the app policy:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/app/test

Policies can be managed from the UI as well:

interfaz de usuario de la bóveda

Encryption as a Service

Before we look at dynamic secrets, let's quickly review the Transit backend, which can be used as an "encryption as a service" for:

  • Encrypting and decrypting data "in-transit" without storing it inside Vault
  • Easily integrating encryption into your application workflow

Back within the bash session in the container, enable Transit:

bash-5.1# vault secrets enable transit

Success! Enabled the transit secrets engine at: transit/

Configure a named encryption key:

bash-5.1# vault write -f transit/keys/foo

Success! Data written to: transit/keys/foo

Encrypt:

bash-5.1# vault write transit/encrypt/foo plaintext=$(base64 <<< "my precious")

Key           Value
---           -----
ciphertext    vault:v1:cFnk5AQLE9Mg+mZ7Ej17vRmYT5aqheikdZQ1FC4vre5jAod0L/uHDA==

Decrypt:

bash-5.1# vault write transit/decrypt/foo ciphertext=vault:v1:cFnk5AQLE9Mg+mZ7Ej17vRmYT5aqheikdZQ1FC4vre5jAod0L/uHDA==

Key          Value
---          -----
plaintext    bXkgcHJlY2lvdXMK

Decode:

bash-5.1# base64 -d <<< "bXkgcHJlY2lvdXMK"

my precious

Test it out in the UI as well:

interfaz de usuario de la bóveda

Dynamic Secrets

As mentioned, Vault supports a number of dynamic secret backends for generating secrets dynamically when needed. For example, with the AWS and Google Cloud backends, you can create access credentials based on IAM policies. The Databases backend, meanwhile, generates database credentials based on configured roles.

Dynamic Secrets:

  • are generated on demand
  • have limited access based on role
  • are leased for a period of time
  • can be revoked
  • come with an audit trail

Let's look at how to generate AWS credentials using the AWS backend.

AWS Credentials

Enable the AWS secrets backend:

bash-5.1# vault secrets enable -path=aws aws

Success! Enabled the aws secrets engine at: aws/

Authenticate:

bash-5.1# vault write aws/config/root access_key=foo secret_key=bar

Success! Data written to: aws/config/root

Make sure to replace foo and bar with your AWS access key id and secret key, respectively.

Create role:

bash-5.1# vault write aws/roles/ec2-read credential_type=iam_user policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Success! Data written to: aws/roles/ec2-read

Here, we created a new role based on AmazonEC2ReadOnlyAccess, which is an AWS-managed policy. As the name suggests, it give users read-only access to the EC2 console; they cannot perform any actions or create new resources. You can also use an inline policy to create a custom role based on your individual needs. We'll look at an example of this shortly. Refer to the AWS Secrets Engine docs for more info.

Remember: Dynamic Secrets are generated only when they are requested (i.e., a web app requests access to S3). They are not available in the store before this.

Create a new set of credentials:

bash-5.1# vault read aws/creds/ec2-read

Key                Value
---                -----
lease_id           aws/creds/ec2-read/9KdO6J7KVBiSwOPEvwrqqALG
lease_duration     768h
lease_renewable    true
access_key         AKIAZ4DZAKZKEULSDW5A
secret_key         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
security_token     <nil>

You should now be able to see the user within the "Users" section on the IAM console on AWS:

Lo siento

Leases and Revocation

In this section, we'll take a quick look at how to define a custom lease period and revoke a secret before the end of that period.

Create a new AWS role:

bash-5.1# vault write aws/roles/foo credential_type=iam_user policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Success! Data written to: aws/roles/foo

Take note of the lease_duration when you create a new AWS credential:

bash-5.1# vault read aws/creds/foo

Key                Value
---                -----
lease_id           aws/creds/foo/F0oBbnBIHEoz0ywVVtbuJB7r
lease_duration     768h
lease_renewable    true
access_key         AKIAZ4DZAKZKLJKB7CPX
secret_key         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
security_token     <nil>

What if you only wanted the lease period for all AWS IAM dynamic secrets to be 30 minutes?

bash-5.1# vault write aws/config/lease lease=1800s lease_max=1800s

In this example, since lease_max is the same as lease, you won't be able to renew the token. If you set the lease_max to 3600s, you'd be able to renew the lease once. For more, review the Tokens and Leases guide.

Create a new credential:

bash-5.1# vault read aws/creds/foo

Key                Value
---                -----
lease_id           aws/creds/foo/xQlJpKDS1ljE9Awz0aywXgbB
lease_duration     30m
lease_renewable    true
access_key         AKIAZ4DZAKZKJPL5OM5W
secret_key         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
security_token     <nil>

Want to quickly revoke this credential? Grab the lease_id and then run:

bash-5.1# vault lease revoke aws/creds/foo/xQlJpKDS1ljE9Awz0aywXgbB

Want to revoke all AWS creds?

bash-5.1# vault lease revoke -prefix aws/

Refer to the Lease, Renew, and Revoke guide for more info these concepts.

Consul Backend

Thus far, we've been using the Filesystem backend. This will not scale beyond a single server, so it does not take advantage of Vault's high availability. Fortunately, there are a number of other Storage backends, like the Consul backend, designed for distributed systems.

To set up Consul, start by updating the docker-compose.yml file:

version: '3.8'

services:

  vault:
    build:
      context: ./vault
      dockerfile: Dockerfile
    ports:
      - 8200:8200
    volumes:
      - ./vault/config:/vault/config
      - ./vault/policies:/vault/policies
      - ./vault/data:/vault/data
      - ./vault/logs:/vault/logs
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_API_ADDR=http://127.0.0.1:8200
    command: server -config=/vault/config/vault-config.json
    cap_add:
      - IPC_LOCK
    depends_on:
      - consul

  consul:
    build:
      context: ./consul
      dockerfile: Dockerfile
    ports:
      - 8500:8500
    command: agent -server -bind 0.0.0.0 -client 0.0.0.0 -bootstrap-expect 1 -config-file=/consul/config/config.json
    volumes:
      - ./consul/config/consul-config.json:/consul/config/config.json
      - ./consul/data:/consul/data

Add a new directory in the project root called "consul", and then add a new Dockerfile to that newly created directory:

# base image
FROM alpine:3.14

# set consul version
ENV CONSUL_VERSION 1.10.2

# create a new directory
RUN mkdir /consul

# download dependencies
RUN apk --no-cache add \
      bash \
      ca-certificates \
      wget

# download and set up consul
RUN wget --quiet --output-document=/tmp/consul.zip https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_linux_amd64.zip && \
    unzip /tmp/consul.zip -d /consul && \
    rm -f /tmp/consul.zip && \
    chmod +x /consul/consul

# update PATH
ENV PATH="PATH=$PATH:$PWD/consul"

# add the config file
COPY ./config/consul-config.json /consul/config/config.json

# expose ports
EXPOSE 8300 8400 8500 8600

# run consul
ENTRYPOINT ["consul"]

Next, within the "consul" directory add two new directories: "config" and "data". Then, within "config", add a config file called consul-config.json:

{
  "datacenter": "localhost",
  "data_dir": "/consul/data",
  "log_level": "DEBUG",
  "server": true,
  "ui": true,
  "ports": {
    "dns": 53
  }
}

Asegúrese de revisar las opciones de configuración de los documentos de Consul para obtener más información sobre las opciones anteriores.

El directorio "cónsul" ahora debería verse así:

├── Dockerfile
├── config
│   └── consul-config.json
└── data

Salga de la sesión bash. Baje el contenedor y luego actualice el archivo de configuración de Vault:

{
  "backend": {
    "consul": {
      "address": "consul:8500",
      "path": "vault/"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Entonces, ahora estamos usando el backend de Consul en lugar del sistema de archivos. Usamos el nombre del servicio, consul, como parte de la dirección. La pathclave define la ruta en el almacén de clave/valor de Consul donde se almacenarán los datos de Vault.

Borre todos los archivos y carpetas dentro del directorio "bóveda/datos" para eliminar el backend del sistema de archivos. Cree las nuevas imágenes y haga girar los contenedores:

$ docker-compose down
$ docker-compose up -d --build

Asegúrese de que todo esté bien navegando en su navegador a http://localhost:8500/ui :

cónsul ui

Pruébelo desde la CLI o la interfaz de usuario.

CLI

Cree una nueva sesión de bash en el contenedor de Vault:

$ docker-compose exec vault bash

Entonces corre:

# Init
bash-5.1# vault operator init

# Unseal
bash-5.1# vault operator unseal

# Authenticate
bash-5.1# vault login

# Enable secrets
bash-5.1# vault secrets enable kv

# Add a new static secret
bash-5.1# vault kv put kv/foo bar=precious

# Read it back
bash-5.1# vault kv get kv/foo

interfaz de usuario

cónsul de bóveda

Observe cómo no hay archivos o carpetas dentro de "bóveda/datos". ¿Por qué crees que es esto?

¿Quiere agregar otro servidor Consul a la mezcla? Agregue un nuevo servicio a docker-compose.yml :

consul-worker:
  build:
    context: ./consul
    dockerfile: Dockerfile
  command: agent -server -join consul -config-file=/consul/config/config.json
  volumes:
    - ./consul/config/consul-config.json:/consul/config/config.json
  depends_on:
    - consul

Aquí, usamos el comando de unión para conectar este agente a un clúster existente. Observe cómo simplemente tuvimos que hacer referencia al nombre del servicio: consul.

Después:

  1. Salir de la sesión bash (si es necesario)
  2. Bajar los contenedores
  3. Limpiar el directorio de datos en "consul/data" (¿Por qué?)
  4. Haga girar los contenedores de nuevo y pruebe

cónsul ui

Conclusión

En este tutorial, repasamos cómo configurar y ejecutar Vault y Consul dentro de un contenedor Docker. Ahora debería tener una comprensión clara de cómo interactuar con Vault y realizar operaciones básicas.

Tome el código final del repositorio vault-consul-docker . Echa un vistazo a la presentación también.

Fuente:  https://testdriven.io

#vault #consul 

Cómo Configurar Y Utilizar Los Proyectos Vault Y Consul De Hashicorp
Neil  Morgan

Neil Morgan

1660603740

Como Configurar E Usar Os Projetos Vault E Consul Da Hashicorp

O tutorial a seguir detalha como configurar e usar os projetos Vault e Consul da Hashicorp para armazenar e gerenciar segredos com segurança.

Começaremos girando uma única instância do Vault em um contêiner do Docker e, em seguida, passaremos ao gerenciamento de segredos estáticos e dinâmicos junto com o recurso "criptografia como serviço" do Vault. Em seguida, adicionaremos o Consul à mistura e veremos como dimensionar o Vault.

Principais dependências:

  • Docker v20.10.8
  • Docker-Compose v1.29.2
  • Cofre v1.8.2
  • Cônsul v1.10.2

Objetivos

Ao final deste tutorial, você deverá ser capaz de:

  1. Explique o que é o Vault e por que você pode querer usá-lo
  2. Descrever a arquitetura básica do Vault juntamente com segredos dinâmicos e estáticos, os vários back-ends (armazenamento, segredo, autenticação, auditoria) e como o Vault pode ser usado como uma "criptografia como serviço"
  3. Configurar e executar o Vault e o Consul com o Docker
  4. Acelere o Vault com o back-end do sistema de arquivos
  5. Iniciar e deslacrar o Vault
  6. Autenticar no Vault
  7. Configure um back-end de auditoria para registrar todas as interações com o Vault
  8. Trabalhe com segredos estáticos e dinâmicos por meio da CLI, API HTTP e IU
  9. Crie uma política do Vault para limitar o acesso a um caminho específico
  10. Use o back-end do Transit como uma "criptografia como serviço"
  11. Configurar o Consul para trabalhar com o Vault como back-end de armazenamento para segredos
  12. Defina um período de concessão personalizado para um segredo e revogue um segredo antes do final desse período

O que é Cofre?

O Vault é uma ferramenta de código aberto usada para armazenar e gerenciar segredos com segurança.

O que é um segredo? Segredos, no contexto deste tutorial, são informações seguras ou de identificação pessoal, como credenciais de banco de dados, chaves SSH, nomes de usuário e senhas, credenciais do AWS IAM, tokens de API, números de CPF, números de cartão de crédito, apenas para citar alguns.

Reserve um momento para pensar em como sua equipe atualmente gerencia e distribui segredos:

  1. Quem tem acesso a eles?
  2. Quem os administra?
  3. Como você controla quem tem acesso a eles?
  4. Como seus aplicativos os obtêm?
  5. Como eles são atualizados?
  6. Como são revogados?

O Vault fornece respostas a essas perguntas e ajuda a resolver os seguintes problemas em relação ao gerenciamento de segredos:

ProblemasObjetivos do Vault
Os segredos estão por toda parte.O Vault é a única fonte de verdade para todos os segredos.
Eles geralmente não são criptografados.O Vault gerencia a criptografia (durante o trânsito e em repouso) imediatamente.
É difícil gerá-los dinamicamente.Os segredos podem ser gerados dinamicamente.
É ainda mais difícil arrendar e revogá-los.Os segredos podem ser alugados e revogados.
Não há trilha de auditoria.Há uma trilha de auditoria para gerar e usar segredos.

O Vault tem várias peças móveis, então pode levar algum tempo para se atualizar com a arquitetura geral. Reserve um momento para revisar o guia de arquitetura , observando os seguintes back-ends:

Processo internoUsarExemplos
ArmazenarOnde os segredos são armazenadosConsul *, Filesystem *, In-Memory, PostgreSQL, S3
SegredoLida com segredos estáticos ou dinâmicosAWS *, bancos de dados, chave/valor *, RabbitMQ, SSH
AutenticaçãoLida com autenticação e autorizaçãoAWS, Azure, Google Cloud, GitHub, Tokens *, nome de usuário e senha
AuditoriaRegistra todas as solicitações e respostasArquivo *, Syslog, Soquete

*usado neste tutorial

Com isso, vamos começar a usar o Vault.

Back-end do sistema de arquivos

Para começar a funcionar rapidamente, usaremos o back-end do Filesystem para armazenar segredos em repouso.

O back-end do sistema de arquivos deve ser usado apenas para desenvolvimento local ou implantação do Vault de servidor único, pois não suporta alta disponibilidade.

Crie um novo diretório de projeto:

$ mkdir vault-consul-docker && cd vault-consul-docker

Em seguida, adicione as seguintes pastas:

└── vault
    ├── config
    ├── data
    ├── logs
    └── policies

Adicione um Dockerfile ao diretório "vault":

# base image
FROM alpine:3.14

# set vault version
ENV VAULT_VERSION 1.8.2

# create a new directory
RUN mkdir /vault

# download dependencies
RUN apk --no-cache add \
      bash \
      ca-certificates \
      wget

# download and set up vault
RUN wget --quiet --output-document=/tmp/vault.zip https://releases.hashicorp.com/vault/${VAULT_VERSION}/vault_${VAULT_VERSION}_linux_amd64.zip && \
    unzip /tmp/vault.zip -d /vault && \
    rm -f /tmp/vault.zip && \
    chmod +x /vault

# update PATH
ENV PATH="PATH=$PATH:$PWD/vault"

# add the config file
COPY ./config/vault-config.json /vault/config/vault-config.json

# expose port 8200
EXPOSE 8200

# run vault
ENTRYPOINT ["vault"]

Em seguida, adicione um arquivo docker-compose.yml à raiz do projeto:

version: '3.8'

services:

  vault:
    build:
      context: ./vault
      dockerfile: Dockerfile
    ports:
      - 8200:8200
    volumes:
      - ./vault/config:/vault/config
      - ./vault/policies:/vault/policies
      - ./vault/data:/vault/data
      - ./vault/logs:/vault/logs
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_API_ADDR=http://127.0.0.1:8200
    command: server -config=/vault/config/vault-config.json
    cap_add:
      - IPC_LOCK

Adicione um arquivo de configuração chamado vault-config.json a "vault/config":

{
  "backend": {
    "file": {
      "path": "vault/data"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Aqui, configuramos o Vault para usar o back-end do sistema de arquivos, definimos o ouvinte para o Vault, desabilitamos o TLS e habilitamos a interface do usuário do Vault . Revise os documentos para obter mais informações sobre como configurar o Vault.

Agora podemos construir a imagem e girar o contêiner:

$ docker-compose up -d --build

Puxe os logs do Docker para verificar se não houve erros na compilação:

$ docker-compose logs

Você deve ver algo semelhante a:

Attaching to vault-consul-docker_vault_1
vault_1  | ==> Vault server configuration:
vault_1  |
vault_1  |              Api Address: http://127.0.0.1:8200
vault_1  | 2021-09-08T14:48:35.014Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
vault_1  |                      Cgo: disabled
vault_1  |          Cluster Address: https://127.0.0.1:8201
vault_1  |               Go Version: go1.16.7
vault_1  |               Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
vault_1  |                Log Level: info
vault_1  |                    Mlock: supported: true, enabled: true
vault_1  |            Recovery Mode: false
vault_1  |                  Storage: file
vault_1  |                  Version: Vault v1.8.2
vault_1  |              Version Sha: aca76f63357041a43b49f3e8c11d67358496959f
vault_1  |
vault_1  | ==> Vault server started! Log data will stream in below:
vault_1  |

Inicializando e deslacrando

Inicie uma sessão bash no contêiner em execução:

$ docker-compose exec vault bash

Dentro do shell, inicialize o Vault:

bash-5.1# vault operator init

Anote as chaves de unseal e o token raiz inicial. Você precisará fornecer três das chaves de remoção de lacre sempre que o servidor do Vault for relacrado ou reiniciado.

Por que 3 chaves? Revise o Compartilhamento Secreto de Shamir .

Agora você pode desbloquear o Vault usando três das chaves:

bash-5.1# vault operator unseal
Unseal Key (will be hidden):

Execute este comando mais duas vezes, usando chaves diferentes a cada vez. Uma vez feito, certifique- Sealedse de false:

Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    5
Threshold       3
Version         1.8.2
Storage Type    file
Cluster Name    vault-cluster-8fcf9d05
Cluster ID      d86e0274-ad9c-d2c1-d6ec-baeab410797b
HA Enabled      false

Usando o token raiz, agora você pode autenticar:

bash-5.1# vault login
Token (will be hidden):

Você deve ver algo semelhante a:

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                s.c0kYHWiOTqQvtR8JuSeTz6sZ
token_accessor       3FQJVxOY5C1brzlHHQSFaCdZ
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Tenha em mente que isso usa a política de raiz. Na produção, você desejará configurar políticas com diferentes níveis de acesso. Veremos como fazer isso em breve.

inicialização do cofre

O Vault agora está sem lacre e pronto para uso.

Auditoria

Antes de testarmos a funcionalidade, vamos habilitar um Dispositivo de Auditoria :

bash-5.1# vault audit enable file file_path=/vault/logs/audit.log

Success! Enabled the file audit device at: file/

Agora você deve poder visualizar os logs localmente em "cofre/logs". Para testar, execute o seguinte comando para visualizar todos os dispositivos de auditoria habilitados:

bash-5.1# vault audit list

Path     Type    Description
----     ----    -----------
file/    file    n/a

A solicitação e a resposta subsequente devem ser registradas em vault/logs/audit.log . Dê uma olhada.

Segredos

Existem dois tipos de segredos no Vault: estático e dinâmico .

Segredos estáticos (pense em Redis ou Memcached criptografados) têm intervalos de atualização, mas não expiram, a menos que sejam revogados explicitamente. Eles são definidos antecipadamente com o back-end de chave/valor (anteriormente o back-end "genérico") e, em seguida, compartilhados.

armazenamento secreto seguro

Os segredos dinâmicos são gerados sob demanda. Eles obrigaram os arrendamentos e geralmente expiram após um curto período de tempo. Como eles não existem até que sejam acessados, há menos exposição - portanto, os segredos dinâmicos são muito mais seguros. O Vault é fornecido com vários back-ends dinâmicos -- ou seja, AWS , Databases , Google Cloud , Consul e RabbitMQ .

Revise a postagem do blog Por que precisamos de segredos dinâmicos para obter mais informações sobre as vantagens de usar segredos dinâmicos.

Segredos estáticos

O Vault pode ser gerenciado por meio da CLI , HTTP API ou UI .

CLI

Ainda dentro da sessão bash no container, podemos criar, ler, atualizar e excluir segredos. Também veremos como versionar e reverter segredos.

Habilite segredos com o seguinte comando:

bash-5.1# vault secrets enable kv

Success! Enabled the kv secrets engine at: kv/

Crie um novo segredo com uma chave de bare um valor de preciousdentro do kv/foocaminho:

bash-5.1# vault kv put kv/foo bar=precious

Success! Data written to: kv/foo

Ler:

bash-5.1# vault kv get kv/foo

=== Data ===
Key    Value
---    -----
bar    precious

Para trabalhar com diferentes versões de uma chave específica, precisaremos atualizar para a v2 do back-end de chave/valor :

bash-5.1# vault kv enable-versioning kv/

Success! Tuned the secrets engine at: kv/

Adicione a versão 2 atualizando o valor para copper:

bash-5.1# vault kv put kv/foo bar=copper

Key              Value
---              -----
created_time     2021-09-08T18:23:14.4154928Z
deletion_time    n/a
destroyed        false
version          2

Leia a versão 1:

bash-5.1# vault kv get -version=1 kv/foo

====== Metadata ======
Key              Value
---              -----
created_time     2021-09-08T18:22:37.2548824Z
deletion_time    n/a
destroyed        false
version          1

=== Data ===
Key    Value
---    -----
bar    precious

Leia a versão 2:

bash-5.1# vault kv get -version=2 kv/foo

====== Metadata ======
Key              Value
---              -----
created_time     2021-09-08T18:23:14.4154928Z
deletion_time    n/a
destroyed        false
version          2

=== Data ===
Key    Value
---    -----
bar    copper

Exclua a versão mais recente (por exemplo, versão 2):

bash-5.1# vault kv delete kv/foo

Success! Data deleted (if it existed) at: kv/foo

Excluir versão 1:

bash-5.1# vault kv delete -versions=1 kv/foo

Success! Data deleted (if it existed) at: kv/foo

Você também pode recuperar:

bash-5.1# vault kv undelete -versions=1 kv/foo

Success! Data written to: kv/undelete/foo

Excluir é semelhante a uma exclusão reversível. Se você quiser remover os metadados subjacentes, terá que usar o comando destroy :

bash-5.1# vault kv destroy -versions=1 kv/foo

Success! Data written to: kv/destroy/foo

Revise v1 e v2 para visualizar todos os comandos disponíveis.

Anote o log de auditoria. Cada uma das solicitações acima foi registrada!

API

Você também pode interagir com o Vault por meio da API HTTP . Faremos solicitações na v2 da API. Abra uma nova guia de terminal e defina o token raiz como uma variável de ambiente:

$ export VAULT_TOKEN=your_token_goes_here

Crie um novo segredo chamado foocom um valor de world:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -H "Content-Type: application/json" \
    -X POST \
    -d '{ "data": { "foo": "world" } }' \
    http://127.0.0.1:8200/v1/kv/data/hello

Leia o segredo:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/hello

A resposta JSON deve conter uma datachave com um valor semelhante a:

"data": {
  "data":{
    "foo": "world"
  },
  "metadata": {
    "created_time": "2021-09-08T18:30:32.5140484Z",
    "deletion_time": "",
    "destroyed": false,
    "version": 1
  }
}

API de cofre

Tente adicionar novas versões, excluir e destruir por conta própria.

IU

A interface do usuário deve estar em execução em http://localhost:8200/ui/vault . Use o token raiz para fazer login. Em seguida, explore o back-end de chave/valor por conta própria:

interface do cofre

Políticas

Até agora, usamos a política de raiz para interagir com a API. Vamos configurar uma política que tenha apenas acesso de leitura.

Adicione um novo arquivo de configuração chamado app-policy.json a "vault/policies":

{
  "path": {
    "kv/data/app/*": {
      "policy": "read"
    }
  }
}

Crie uma nova política de volta na sessão bash:

bash-5.1# vault policy write app /vault/policies/app-policy.json

Success! Uploaded policy: app

Em seguida, crie um novo token:

bash-5.1# vault token create -policy=app

Key                  Value
---                  -----
token                s.ZOUMx3RIhVRhI4ijlZg8KXRQ
token_accessor       TT53xOxbIfGjI7l4392gjXcg
token_duration       768h
token_renewable      true
token_policies       ["app" "default"]
identity_policies    []
policies             ["app" "default"]

Dentro de outra nova guia do terminal (agora você deve ter três), adicione a VAULT_TOKENvariável de ambiente com o novo token:

$ export VAULT_TOKEN=your_token_goes_here

Tente ler o foosegredo que definimos anteriormente:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/hello

Você não deve ter as permissões corretas para visualizar esse segredo:

{
  "errors":[
    "1 error occurred:\n\t* permission denied\n\n"
  ]
}

Por que não podemos nem ler? Volte para a configuração de política em vault-config.json . kv/data/app/*indica que a política só pode ler do appcaminho.

Como você provavelmente já notou, quase tudo no Vault é baseado em caminhos.

De volta à sessão bash no contêiner, adicione um novo segredo ao app/testcaminho:

bash-5.1# vault kv put kv/app/test ping=pong

Key              Value
---              -----
created_time     2021-09-08T18:40:35.2694047Z
deletion_time    n/a
destroyed        false
version          1

Você deve conseguir visualizar o segredo usando o token associado à apppolítica:

$ curl \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -X GET \
    http://127.0.0.1:8200/v1/kv/data/app/test

As políticas também podem ser gerenciadas na interface do usuário:

interface do cofre

Criptografia como um serviço

Antes de analisarmos os segredos dinâmicos, vamos analisar rapidamente o back-end do Transit , que pode ser usado como uma "criptografia como serviço" para:

  • Criptografar e descriptografar dados "em trânsito" sem armazená-los no Vault
  • Integrando facilmente a criptografia ao fluxo de trabalho do seu aplicativo

De volta à sessão bash no contêiner, ative o Transit:

bash-5.1# vault secrets enable transit

Success! Enabled the transit secrets engine at: transit/

Configure uma chave de criptografia nomeada:

bash-5.1# vault write -f transit/keys/foo

Success! Data written to: transit/keys/foo

Criptografar:

bash-5.1# vault write transit/encrypt/foo plaintext=$(base64 <<< "my precious")

Key           Value
---           -----
ciphertext    vault:v1:cFnk5AQLE9Mg+mZ7Ej17vRmYT5aqheikdZQ1FC4vre5jAod0L/uHDA==

Descriptografar:

bash-5.1# vault write transit/decrypt/foo ciphertext=vault:v1:cFnk5AQLE9Mg+mZ7Ej17vRmYT5aqheikdZQ1FC4vre5jAod0L/uHDA==

Key          Value
---          -----
plaintext    bXkgcHJlY2lvdXMK

Decodificar:

bash-5.1# base64 -d <<< "bXkgcHJlY2lvdXMK"

my precious

Teste-o também na interface do usuário:

interface do cofre

Segredos dinâmicos

Conforme mencionado, o Vault oferece suporte a vários back-ends de segredo dinâmicos para gerar segredos dinamicamente quando necessário. Por exemplo, com os back-ends da AWS e do Google Cloud , você pode criar credenciais de acesso com base nas políticas do IAM. Enquanto isso, o back-end de bancos de dados gera credenciais de banco de dados com base nas funções configuradas.

Segredos dinâmicos:

  • são gerados sob demanda
  • têm acesso limitado com base na função
  • são alugados por um período de tempo
  • pode ser revogado
  • vem com uma trilha de auditoria

Vejamos como gerar credenciais da AWS usando o back-end da AWS.

Credenciais da AWS

Habilite o back-end de segredos da AWS:

bash-5.1# vault secrets enable -path=aws aws

Success! Enabled the aws secrets engine at: aws/

Autenticar:

bash-5.1# vault write aws/config/root access_key=foo secret_key=bar

Success! Data written to: aws/config/root

Certifique-se de substituir fooe barpelo ID da chave de acesso e chave secreta da AWS, respectivamente.

Criar função:

bash-5.1# vault write aws/roles/ec2-read credential_type=iam_user policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Success! Data written to: aws/roles/ec2-read

Aqui, criamos uma nova função baseada em , que é uma políticaAmazonEC2ReadOnlyAccess gerenciada pela AWS . Como o nome sugere, ele dá aos usuários acesso somente leitura ao console do EC2; eles não podem executar nenhuma ação ou criar novos recursos. Você também pode usar uma política em linha para criar uma função personalizada com base em suas necessidades individuais. Veremos um exemplo disso em breve. Consulte os documentos do AWS Secrets Engine para obter mais informações.

Lembre -se : Segredos dinâmicos são gerados apenas quando solicitados (ou seja, um aplicativo da Web solicita acesso ao S3). Eles não estão disponíveis na loja antes disso.

Crie um novo conjunto de credenciais:

bash-5.1# vault read aws/creds/ec2-read

Key                Value
---                -----
lease_id           aws/creds/ec2-read/9KdO6J7KVBiSwOPEvwrqqALG
lease_duration     768h
lease_renewable    true
access_key         AKIAZ4DZAKZKEULSDW5A
secret_key         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
security_token     <nil>

Agora você deve conseguir ver o usuário na seção "Usuários" no console do IAM na AWS:

Sinto muito

Arrendamentos e Revogação

Nesta seção, veremos rapidamente como definir um período de concessão personalizado e revogar um segredo antes do final desse período.

Crie uma nova função da AWS:

bash-5.1# vault write aws/roles/foo credential_type=iam_user policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Success! Data written to: aws/roles/foo

Observe lease_durationquando você cria uma nova credencial da AWS:

bash-5.1# vault read aws/creds/foo

Key                Value
---                -----
lease_id           aws/creds/foo/F0oBbnBIHEoz0ywVVtbuJB7r
lease_duration     768h
lease_renewable    true
access_key         AKIAZ4DZAKZKLJKB7CPX
secret_key         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
security_token     <nil>

E se você quisesse que o período de concessão de todos os segredos dinâmicos do AWS IAM fosse de 30 minutos?

bash-5.1# vault write aws/config/lease lease=1800s lease_max=1800s

Neste exemplo, como lease_maxé igual a lease, você não poderá renovar o token. Se você definir lease_maxcomo 3600s, poderá renovar a concessão uma vez. Para saber mais, consulte o guia Tokens e Leases .

Crie uma nova credencial:

bash-5.1# vault read aws/creds/foo

Key                Value
---                -----
lease_id           aws/creds/foo/xQlJpKDS1ljE9Awz0aywXgbB
lease_duration     30m
lease_renewable    true
access_key         AKIAZ4DZAKZKJPL5OM5W
secret_key         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
security_token     <nil>

Quer revogar rapidamente esta credencial? Pegue o lease_ide depois execute:

bash-5.1# vault lease revoke aws/creds/foo/xQlJpKDS1ljE9Awz0aywXgbB

Quer revogar todas as credenciais da AWS?

bash-5.1# vault lease revoke -prefix aws/

Consulte o guia Lease, Renew e Revoke para obter mais informações sobre esses conceitos.

Back-end do cônsul

Até agora, estamos usando o backend Filesystem . Isso não será dimensionado além de um único servidor, portanto, não aproveita a alta disponibilidade do Vault. Felizmente, existem vários outros back-ends de armazenamento , como o back-end Consul , projetados para sistemas distribuídos.

Para configurar o Consul , comece atualizando o arquivo docker-compose.yml :

version: '3.8'

services:

  vault:
    build:
      context: ./vault
      dockerfile: Dockerfile
    ports:
      - 8200:8200
    volumes:
      - ./vault/config:/vault/config
      - ./vault/policies:/vault/policies
      - ./vault/data:/vault/data
      - ./vault/logs:/vault/logs
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_API_ADDR=http://127.0.0.1:8200
    command: server -config=/vault/config/vault-config.json
    cap_add:
      - IPC_LOCK
    depends_on:
      - consul

  consul:
    build:
      context: ./consul
      dockerfile: Dockerfile
    ports:
      - 8500:8500
    command: agent -server -bind 0.0.0.0 -client 0.0.0.0 -bootstrap-expect 1 -config-file=/consul/config/config.json
    volumes:
      - ./consul/config/consul-config.json:/consul/config/config.json
      - ./consul/data:/consul/data

Adicione um novo diretório na raiz do projeto chamado "consul" e adicione um novo Dockerfile nesse diretório recém-criado:

# base image
FROM alpine:3.14

# set consul version
ENV CONSUL_VERSION 1.10.2

# create a new directory
RUN mkdir /consul

# download dependencies
RUN apk --no-cache add \
      bash \
      ca-certificates \
      wget

# download and set up consul
RUN wget --quiet --output-document=/tmp/consul.zip https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_linux_amd64.zip && \
    unzip /tmp/consul.zip -d /consul && \
    rm -f /tmp/consul.zip && \
    chmod +x /consul/consul

# update PATH
ENV PATH="PATH=$PATH:$PWD/consul"

# add the config file
COPY ./config/consul-config.json /consul/config/config.json

# expose ports
EXPOSE 8300 8400 8500 8600

# run consul
ENTRYPOINT ["consul"]

Em seguida, dentro do diretório "consul" adicione dois novos diretórios: "config" e "data". Em seguida, dentro de "config", adicione um arquivo de configuração chamado consul-config.json :

{
  "datacenter": "localhost",
  "data_dir": "/consul/data",
  "log_level": "DEBUG",
  "server": true,
  "ui": true,
  "ports": {
    "dns": 53
  }
}

Certifique-se de revisar as opções de configuração dos documentos do Consul para obter mais informações sobre as opções acima.

O diretório "consul" agora deve se parecer com:

├── Dockerfile
├── config
│   └── consul-config.json
└── data

Saia da sessão do bash. Desative o contêiner e atualize o arquivo de configuração do Vault:

{
  "backend": {
    "consul": {
      "address": "consul:8500",
      "path": "vault/"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Então, agora estamos usando o backend Consul em vez do Filesystem. Usamos o nome do serviço, consul, como parte do endereço. A pathchave define o caminho no armazenamento de chave/valor do Consul onde os dados do Vault serão armazenados.

Limpe todos os arquivos e pastas dentro do diretório "vault/data" para remover o backend do sistema de arquivos. Construa as novas imagens e gire os contêineres:

$ docker-compose down
$ docker-compose up -d --build

Certifique-se de que tudo está bem navegando em seu navegador para http://localhost:8500/ui :

cônsul ui

Teste isso na CLI ou na interface do usuário.

CLI

Crie uma nova sessão bash no contêiner do Vault:

$ docker-compose exec vault bash

Então corra:

# Init
bash-5.1# vault operator init

# Unseal
bash-5.1# vault operator unseal

# Authenticate
bash-5.1# vault login

# Enable secrets
bash-5.1# vault secrets enable kv

# Add a new static secret
bash-5.1# vault kv put kv/foo bar=precious

# Read it back
bash-5.1# vault kv get kv/foo

IU

cônsul do cofre

Observe como não há arquivos ou pastas dentro de "cofre/dados". Por que você acha que é isso?

Quer adicionar outro servidor Consul à mistura? Adicione um novo serviço ao docker-compose.yml :

consul-worker:
  build:
    context: ./consul
    dockerfile: Dockerfile
  command: agent -server -join consul -config-file=/consul/config/config.json
  volumes:
    - ./consul/config/consul-config.json:/consul/config/config.json
  depends_on:
    - consul

Aqui, usamos o comando join para conectar esse agente a um cluster existente. Observe como simplesmente tivemos que referenciar o nome do serviço: consul.

Então:

  1. Sair da sessão bash (se necessário)
  2. Derrube os recipientes
  3. Limpe o diretório de dados em "consul/data" (Por quê?)
  4. Gire os contêineres de volta e teste

cônsul ui

Conclusão

Neste tutorial, vimos como configurar e executar o Vault e o Consul dentro de um contêiner do Docker. Agora você deve ter uma compreensão clara de como interagir com o Vault e realizar operações básicas.

Pegue o código final do repositório vault-consul-docker . Confira também a apresentação .

Fonte:  https://testdrive.io

#vault #consul 

Como Configurar E Usar Os Projetos Vault E Consul Da Hashicorp
Audra  Haag

Audra Haag

1660582680

How to Deploy Hashicorp's Vault and Consul with Docker Swarm

Let's look at how to deploy Hashicorp's Vault and Consul to DigitalOcean with Docker Swarm.

Source: https://testdriven.io

#vault #docker 

How to Deploy Hashicorp's Vault and Consul with Docker Swarm
Thai  Son

Thai Son

1660575420

Cách Triển Khai Hashicorp's Vault và Consul Với Docker Swarm

Hãy xem cách triển khai Hashicorp's Vault and Consul đến DigitalOcean với Docker Swarm.

Sau khi hoàn thành, bạn sẽ có thể:

  1. Máy chủ cung cấp trên DigitalOcean với Máy Docker
  2. Định cấu hình một cụm Docker Swarm để chạy trên DigitalOcean
  3. Chạy Vault and Consul trên Docker Swarm

Phụ thuộc chính:

  • Docker v20.10.8
  • Docker-Compose v1.29.2
  • Docker-Machine v0.16.2
  • Vault v1.8.3
  • Consul v1.10.3

Consul

Tạo một thư mục dự án mới:

$ mkdir vault-consul-swarm && cd vault-consul-swarm

Sau đó, thêm tệp docker-compac.yml vào thư mục gốc của dự án:

version: "3.8"

services:

  server-bootstrap:
    image: consul:1.10.3
    ports:
      - 8500:8500
    command: "agent -server -bootstrap-expect 3 -ui -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"

  server:
    image: consul:1.10.3
    command: "agent -server -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"
    deploy:
      replicas: 2
    depends_on:
      - server-bootstrap

  client:
    image: consul:1.10.3
    command: "agent -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"
    deploy:
      replicas: 2
    depends_on:
      - server-bootstrap

networks:
  default:
    external: true
    name: core

Cấu hình này trông quen thuộc.

  1. Tham khảo phần Soạn tệp của bài đăng blog Running Flask trên Docker Swarm để biết thêm thông tin về cách sử dụng tệp soạn cho chế độ Docker Swarm.
  2. Xem lại hướng dẫn Cosul và Docker để biết thông tin về cấu hình Consul ở trên.

Docker Swarm

Đăng ký tài khoản DigitalOcean (nếu bạn chưa có), sau đó tạo mã thông báo truy cập để bạn có thể truy cập API DigitalOcean.

Thêm mã thông báo vào môi trường của bạn:

$ export DIGITAL_OCEAN_ACCESS_TOKEN=[your_digital_ocean_token]

Quay lên ba giọt:

$ for i in 1 2 3; do
    docker-machine create \
      --driver digitalocean \
      --digitalocean-region "nyc1" \
      --digitalocean-image=debian-10-x64 \
      --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
      --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
      node-$i;
done

Khởi tạo chế độ Swarm trên nút đầu tiên node-1,:

$ docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)

Sử dụng mã thông báo tham gia từ đầu ra của lệnh trước đó để thêm hai nút còn lại vào Swarm với tư cách là công nhân:

$ for i in 2 3; do
    docker-machine ssh node-$i -- docker swarm join --token YOUR_JOIN_TOKEN HOST:PORT;
done

Ví dụ:

for i in 2 3; do
    docker-machine ssh node-$i -- docker swarm join --token SWMTKN-1-18xrfgcgq7k6krqr7tvav3ydx5c5104y662lzh4pyct2t0ror3-e3ed1ggivhf8z15i40z6x55g5 67.205.165.166:2377;
done

Bạn nên thấy:

This node joined a swarm as a worker.
This node joined a swarm as a worker.

Trỏ trình nền Docker vào node-1, tạo một mạng lớp phủ có thể đính kèm (được gọi core) và triển khai ngăn xếp:

$ eval $(docker-machine env node-1)
$ docker network create -d overlay --attachable core
$ docker stack deploy --compose-file=docker-compose.yml secrets

Liệt kê các dịch vụ trong ngăn xếp:

$ docker stack ps -f "desired-state=running" secrets

Bạn sẽ thấy một cái gì đó tương tự như:

ID             NAME                         IMAGE          NODE     DESIRED STATE   CURRENT STATE
b5f5eycrhf3o   secrets_client.1             consul:1.10.3   node-1   Running         Running 7 seconds ago
zs7a5t8khcew   secrets_server.1             consul:1.10.3   node-2   Running         Running 9 seconds ago
qnhtlan6m0sp   secrets_server-bootstrap.1   consul:1.10.3   node-1   Running         Running 7 seconds ago
u61eycesmsl7   secrets_client.2             consul:1.10.3   node-2   Running         Running 9 seconds ago
vgpql8lfy5fi   secrets_server.2             consul:1.10.3   node-3   Running         Running 9 seconds ago

Lấy IP được liên kết với node-1:

$ docker-machine ip node-1

Sau đó, kiểm tra giao diện người dùng Consul trong trình duyệt của bạn tại http: // YOUR_MACHINE_IP: 8500 / ui . Cần có ba dịch vụ đang chạy và năm nút.

dịch vụ lãnh sự ui

nút lãnh sự ui

Vault

Thêm vaultdịch vụ vào docker-compo.yml :

vault:
  image: vault:1.8.3
  deploy:
    replicas: 1
  ports:
    - 8200:8200
  environment:
    - VAULT_ADDR=http://127.0.0.1:8200
    - VAULT_LOCAL_CONFIG={"backend":{"consul":{"address":"http://server-bootstrap:8500","path":"vault/"}},"listener":{"tcp":{"address":"0.0.0.0:8200","tls_disable":1}},"ui":true, "disable_mlock":true}
  command: server
  depends_on:
    - consul

Hãy lưu ý đến VAULT_LOCAL_CONFIGbiến môi trường:

{
  "backend": {
    "consul": {
      "address": "http://server-bootstrap:8500",
      "path": "vault/"
    }
  },
  "listener": {
    "tcp": {
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true,
  "disable_mlock": true
}

Xem lại phần Phụ trợ Consul từ bài đăng trên blog Quản lý Bí mật với Vault và Consul để biết thêm thông tin. Ngoài ra, việc đặt disable_mlock thành truekhông được khuyến khích cho các môi trường sản xuất; tuy nhiên, nó phải được bật vì --cap-addkhông khả dụng trong chế độ Docker Swarm. Xem các sự cố GitHub sau để biết chi tiết:

  1. --cap-add = IPC_LOCK không khả dụng trong nhóm docker
  2. Thiếu từ Swarmmode --cap-add

Bài kiểm tra

Triển khai lại ngăn xếp:

$ docker stack deploy --compose-file=docker-compose.yml secrets

Chờ một vài giây để các dịch vụ chạy lên, sau đó kiểm tra trạng thái:

$ docker stack ps -f "desired-state=running" secrets

Một lần nữa, bạn sẽ thấy một cái gì đó tương tự như:

ID             NAME                         IMAGE           NODE      DESIRED STATE   CURRENT STATE
xtfsetfrbrs7   secrets_client.1             consul:1.10.3   node-3    Running         Running 19 minutes ago
ydqxexgiyzb2   secrets_client.2             consul:1.10.3   node-1    Running         Running 19 minutes ago
izlku3y6j8rp   secrets_server-bootstrap.1   consul:1.10.3   node-2    Running         Running 19 minutes ago
zqpkcrhrix2x   secrets_server.1             consul:1.10.3   node-1    Running         Running 19 minutes ago
kmlxuhxw1akv   secrets_server.2             consul:1.10.3   node-2    Running         Running 19 minutes ago
wfmscoj53m39   secrets_vault.1              vault:1.8.3     node-3    Running         Running about a minute ago

Tiếp theo, đảm bảo Vault được liệt kê trong phần "Dịch vụ" của Giao diện người dùng Consul:

dịch vụ lãnh sự ui

Bây giờ bạn có thể tương tác với Vault thông qua CLI, API HTTP và giao diện người dùng. Bắt đầu bằng cách khởi tạo và mở khóa Vault. Sau đó, đăng nhập và tạo một bí mật mới.

Loại bỏ các nút sau khi hoàn thành:

$ docker-machine rm node-1 node-2 node-3 -y

Tập lệnh tự động hóa

Cuối cùng, hãy tạo một tập lệnh nhanh để tự động hóa quá trình triển khai:

  1. Cung cấp ba giọt DigitalOcean với Máy Docker
  2. Định cấu hình chế độ Docker Swarm
  3. Thêm các nút vào Swarm
  4. Triển khai ngăn xếp

Thêm một tệp mới có tên là deploy.sh vào thư mục gốc của dự án:

#!/bin/bash


echo "Spinning up three droplets..."

for i in 1 2 3; do
  docker-machine create \
    --driver digitalocean \
    --digitalocean-region "nyc1" \
    --digitalocean-image=debian-10-x64 \
    --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
    --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
    node-$i;
done


echo "Initializing Swarm mode..."

docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)


echo "Adding the nodes to the Swarm..."

TOKEN=`docker-machine ssh node-1 docker swarm join-token worker | grep token | awk '{ print $5 }'`

for i in 2 3; do
  docker-machine ssh node-$i \
    -- docker swarm join --token ${TOKEN} $(docker-machine ip node-1):2377;
done


echo "Creating networking..."

eval $(docker-machine env node-1)
docker network create -d overlay --attachable core


echo "Deploying the stack..."

docker stack deploy --compose-file=docker-compose.yml secrets

Hãy thử nó ra!

$ sh deploy.sh

Giảm các giọt xuống sau khi hoàn thành:

$ docker-machine rm node-1 node-2 node-3 -y

Bạn có thể tìm thấy mã này trong repo vault-consul-swarm . Chúc mừng!

Nguồn:  https://testdriven.io

#vault #docker 

Cách Triển Khai Hashicorp's Vault và Consul Với Docker Swarm

Как развернуть хранилище Hashicorp и Consul с помощью Docker Swarm

Давайте посмотрим, как развернуть хранилище Hashicorp и Consul в DigitalOcean с помощью Docker Swarm.

По окончании вы сможете:

  1. Предоставление хостов в DigitalOcean с помощью Docker Machine
  2. Настройте кластер Docker Swarm для работы в DigitalOcean.
  3. Запустите Vault и Consul на Docker Swarm

Основные зависимости:

  • Докер v20.10.8
  • Докер-композитор v1.29.2
  • Докер-машина v0.16.2
  • Убежище v1.8.3
  • Консул v1.10.3

Консул

Создайте новый каталог проекта:

$ mkdir vault-consul-swarm && cd vault-consul-swarm

Затем добавьте файл docker-compose.yml в корень проекта:

version: "3.8"

services:

  server-bootstrap:
    image: consul:1.10.3
    ports:
      - 8500:8500
    command: "agent -server -bootstrap-expect 3 -ui -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"

  server:
    image: consul:1.10.3
    command: "agent -server -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"
    deploy:
      replicas: 2
    depends_on:
      - server-bootstrap

  client:
    image: consul:1.10.3
    command: "agent -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"
    deploy:
      replicas: 2
    depends_on:
      - server-bootstrap

networks:
  default:
    external: true
    name: core

Эта конфигурация должна выглядеть знакомо.

  1. Обратитесь к разделу Compose File записи блога Running Flask on Docker Swarm для получения дополнительной информации об использовании файла compose для режима Docker Swarm.
  2. Просмотрите руководство Consul и Docker для получения информации о приведенной выше конфигурации Consul.

Докер Рой

Зарегистрируйте учетную запись DigitalOcean (если у вас ее еще нет), а затем создайте токен доступа, чтобы получить доступ к API DigitalOcean.

Добавьте токен в свою среду:

$ export DIGITAL_OCEAN_ACCESS_TOKEN=[your_digital_ocean_token]

Раскрутите три капли:

$ for i in 1 2 3; do
    docker-machine create \
      --driver digitalocean \
      --digitalocean-region "nyc1" \
      --digitalocean-image=debian-10-x64 \
      --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
      --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
      node-$i;
done

Инициализируйте режим Swarm на первом узле node-1:

$ docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)

Используйте токен соединения из вывода предыдущей команды, чтобы добавить оставшиеся два узла в Swarm в качестве рабочих:

$ for i in 2 3; do
    docker-machine ssh node-$i -- docker swarm join --token YOUR_JOIN_TOKEN HOST:PORT;
done

Например:

for i in 2 3; do
    docker-machine ssh node-$i -- docker swarm join --token SWMTKN-1-18xrfgcgq7k6krqr7tvav3ydx5c5104y662lzh4pyct2t0ror3-e3ed1ggivhf8z15i40z6x55g5 67.205.165.166:2377;
done

Тебе следует увидеть:

This node joined a swarm as a worker.
This node joined a swarm as a worker.

Укажите демону Docker на node-1, создайте присоединяемую оверлейную сеть (называемую core) и разверните стек:

$ eval $(docker-machine env node-1)
$ docker network create -d overlay --attachable core
$ docker stack deploy --compose-file=docker-compose.yml secrets

Перечислите службы в стеке:

$ docker stack ps -f "desired-state=running" secrets

Вы должны увидеть что-то похожее на:

ID             NAME                         IMAGE          NODE     DESIRED STATE   CURRENT STATE
b5f5eycrhf3o   secrets_client.1             consul:1.10.3   node-1   Running         Running 7 seconds ago
zs7a5t8khcew   secrets_server.1             consul:1.10.3   node-2   Running         Running 9 seconds ago
qnhtlan6m0sp   secrets_server-bootstrap.1   consul:1.10.3   node-1   Running         Running 7 seconds ago
u61eycesmsl7   secrets_client.2             consul:1.10.3   node-2   Running         Running 9 seconds ago
vgpql8lfy5fi   secrets_server.2             consul:1.10.3   node-3   Running         Running 9 seconds ago

Получите IP-адрес, связанный с node-1:

$ docker-machine ip node-1

Затем протестируйте пользовательский интерфейс Consul в своем браузере по адресу http://YOUR_MACHINE_IP:8500/ui . Должно быть три запущенных сервиса и пять узлов.

услуги консула

узлы пользовательского интерфейса консула

Свод

Добавьте vaultсервис в docker-compose.yml :

vault:
  image: vault:1.8.3
  deploy:
    replicas: 1
  ports:
    - 8200:8200
  environment:
    - VAULT_ADDR=http://127.0.0.1:8200
    - VAULT_LOCAL_CONFIG={"backend":{"consul":{"address":"http://server-bootstrap:8500","path":"vault/"}},"listener":{"tcp":{"address":"0.0.0.0:8200","tls_disable":1}},"ui":true, "disable_mlock":true}
  command: server
  depends_on:
    - consul

Обратите внимание на VAULT_LOCAL_CONFIGпеременную окружения:

{
  "backend": {
    "consul": {
      "address": "http://server-bootstrap:8500",
      "path": "vault/"
    }
  },
  "listener": {
    "tcp": {
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true,
  "disable_mlock": true
}

Просмотрите раздел Consul Backend в записи блога Управление секретами с помощью Vault и Consul для получения дополнительной информации. Кроме того, установка для параметра disabled_mlock значенияtrue не рекомендуется для производственных сред; однако его необходимо включить, поскольку --cap-addон недоступен в режиме Docker Swarm. Дополнительные сведения см. в следующих проблемах GitHub:

  1. --cap-add=IPC_LOCK недоступен в рое докеров
  2. Отсутствует в Swarmmode --cap-add

Тест

Повторно разверните стек:

$ docker stack deploy --compose-file=docker-compose.yml secrets

Подождите несколько секунд, пока сервисы запустятся, затем проверьте статус:

$ docker stack ps -f "desired-state=running" secrets

Опять же, вы должны увидеть что-то похожее на:

ID             NAME                         IMAGE           NODE      DESIRED STATE   CURRENT STATE
xtfsetfrbrs7   secrets_client.1             consul:1.10.3   node-3    Running         Running 19 minutes ago
ydqxexgiyzb2   secrets_client.2             consul:1.10.3   node-1    Running         Running 19 minutes ago
izlku3y6j8rp   secrets_server-bootstrap.1   consul:1.10.3   node-2    Running         Running 19 minutes ago
zqpkcrhrix2x   secrets_server.1             consul:1.10.3   node-1    Running         Running 19 minutes ago
kmlxuhxw1akv   secrets_server.2             consul:1.10.3   node-2    Running         Running 19 minutes ago
wfmscoj53m39   secrets_vault.1              vault:1.8.3     node-3    Running         Running about a minute ago

Затем убедитесь, что Vault указан в разделе «Сервисы» пользовательского интерфейса Consul:

услуги консула

Теперь вы сможете взаимодействовать с Vault через интерфейс командной строки, HTTP API и пользовательский интерфейс. Начните с инициализации и распечатывания Vault. Затем войдите в систему и создайте новый секрет.

Удалите узлы после завершения:

$ docker-machine rm node-1 node-2 node-3 -y

Скрипт автоматизации

Наконец, давайте создадим быстрый скрипт для автоматизации процесса развертывания:

  1. Предоставление трех капель DigitalOcean с помощью Docker Machine
  2. Настроить режим Docker Swarm
  3. Добавьте узлы в Swarm
  4. Развернуть стек

Добавьте новый файл с именем deploy.sh в корень проекта:

#!/bin/bash


echo "Spinning up three droplets..."

for i in 1 2 3; do
  docker-machine create \
    --driver digitalocean \
    --digitalocean-region "nyc1" \
    --digitalocean-image=debian-10-x64 \
    --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
    --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
    node-$i;
done


echo "Initializing Swarm mode..."

docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)


echo "Adding the nodes to the Swarm..."

TOKEN=`docker-machine ssh node-1 docker swarm join-token worker | grep token | awk '{ print $5 }'`

for i in 2 3; do
  docker-machine ssh node-$i \
    -- docker swarm join --token ${TOKEN} $(docker-machine ip node-1):2377;
done


echo "Creating networking..."

eval $(docker-machine env node-1)
docker network create -d overlay --attachable core


echo "Deploying the stack..."

docker stack deploy --compose-file=docker-compose.yml secrets

Попробуйте!

$ sh deploy.sh

Сбейте капли, как только закончите:

$ docker-machine rm node-1 node-2 node-3 -y

Код можно найти в репозитории vault-consul-swarm . Ваше здоровье!

Источник:  https://testdriven.io

#vault #docker 

Как развернуть хранилище Hashicorp и Consul с помощью Docker Swarm

如何使用 Docker Swarm 部署 Hashicorp 的 Vault 和 Consul

讓我們看看如何使用 Docker Swarm 將 Hashicorp 的 Vault 和 Consul 部署到 DigitalOcean。

完成後,您將能夠:

  1. 使用 Docker Machine 在 DigitalOcean 上配置主機
  2. 配置 Docker Swarm 集群以在 DigitalOcean 上運行
  3. 在 Docker Swarm 上運行 Vault 和 Consul

主要依賴:

  • 碼頭工人 v20.10.8
  • Docker-Compose v1.29.2
  • Docker-Machine v0.16.2
  • 保險櫃 v1.8.3
  • 領事 v1.10.3

領事

創建一個新的項目目錄:

$ mkdir vault-consul-swarm && cd vault-consul-swarm

然後,將docker-compose.yml文件添加到項目根目錄:

version: "3.8"

services:

  server-bootstrap:
    image: consul:1.10.3
    ports:
      - 8500:8500
    command: "agent -server -bootstrap-expect 3 -ui -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"

  server:
    image: consul:1.10.3
    command: "agent -server -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"
    deploy:
      replicas: 2
    depends_on:
      - server-bootstrap

  client:
    image: consul:1.10.3
    command: "agent -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"
    deploy:
      replicas: 2
    depends_on:
      - server-bootstrap

networks:
  default:
    external: true
    name: core

此配置應該看起來很熟悉。

  1. 有關在 Docker Swarm 模式下使用 compose 文件的更多信息,請參閱在 Docker Swarm 上運行 Flask博客文章的Compose File部分。
  2. 查看Consul 和 Docker指南以獲取有關上述 Consul 配置的信息。

碼頭工人群

註冊一個DigitalOcean帳戶(如果您還沒有),然後生成一個訪問令牌,以便您可以訪問 DigitalOcean API。

將令牌添加到您的環境中:

$ export DIGITAL_OCEAN_ACCESS_TOKEN=[your_digital_ocean_token]

旋轉三個液滴:

$ for i in 1 2 3; do
    docker-machine create \
      --driver digitalocean \
      --digitalocean-region "nyc1" \
      --digitalocean-image=debian-10-x64 \
      --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
      --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
      node-$i;
done

在第一個節點上初始化Swarm 模式node-1

$ docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)

使用上一個命令輸出中的連接令牌將剩餘的兩個節點作為工作人員添加到 Swarm:

$ for i in 2 3; do
    docker-machine ssh node-$i -- docker swarm join --token YOUR_JOIN_TOKEN HOST:PORT;
done

例如:

for i in 2 3; do
    docker-machine ssh node-$i -- docker swarm join --token SWMTKN-1-18xrfgcgq7k6krqr7tvav3ydx5c5104y662lzh4pyct2t0ror3-e3ed1ggivhf8z15i40z6x55g5 67.205.165.166:2377;
done

你應該看到:

This node joined a swarm as a worker.
This node joined a swarm as a worker.

將 Docker 守護進程指向node-1,創建一個可附加的覆蓋網絡(稱為core),然後部署堆棧:

$ eval $(docker-machine env node-1)
$ docker network create -d overlay --attachable core
$ docker stack deploy --compose-file=docker-compose.yml secrets

列出堆棧中的服務:

$ docker stack ps -f "desired-state=running" secrets

您應該會看到類似於以下內容的內容:

ID             NAME                         IMAGE          NODE     DESIRED STATE   CURRENT STATE
b5f5eycrhf3o   secrets_client.1             consul:1.10.3   node-1   Running         Running 7 seconds ago
zs7a5t8khcew   secrets_server.1             consul:1.10.3   node-2   Running         Running 9 seconds ago
qnhtlan6m0sp   secrets_server-bootstrap.1   consul:1.10.3   node-1   Running         Running 7 seconds ago
u61eycesmsl7   secrets_client.2             consul:1.10.3   node-2   Running         Running 9 seconds ago
vgpql8lfy5fi   secrets_server.2             consul:1.10.3   node-3   Running         Running 9 seconds ago

獲取與關聯的 IP node-1

$ docker-machine ip node-1

然後,在瀏覽器中的http://YOUR_MACHINE_IP:8500/ui中測試 Consul UI 。應該有三個正在運行的服務和五個節點。

領事用戶界面服務

領事用戶界面節點

保險庫

vault服務添加到docker-compose.yml

vault:
  image: vault:1.8.3
  deploy:
    replicas: 1
  ports:
    - 8200:8200
  environment:
    - VAULT_ADDR=http://127.0.0.1:8200
    - VAULT_LOCAL_CONFIG={"backend":{"consul":{"address":"http://server-bootstrap:8500","path":"vault/"}},"listener":{"tcp":{"address":"0.0.0.0:8200","tls_disable":1}},"ui":true, "disable_mlock":true}
  command: server
  depends_on:
    - consul

注意VAULT_LOCAL_CONFIG環境變量:

{
  "backend": {
    "consul": {
      "address": "http://server-bootstrap:8500",
      "path": "vault/"
    }
  },
  "listener": {
    "tcp": {
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true,
  "disable_mlock": true
}

查看使用 Vault 和 Consul 管理秘密博客文章中的Consul 後端部分以獲取更多信息。此外,不建議將disable_mlock設置為生產環境;但是,它必須啟用,因為它在 Docker Swarm 模式下不可用。有關詳細信息,請參閱以下 GitHub 問題:true--cap-add

  1. --cap-add=IPC_LOCK 在 docker swarm 中不可用
  2. 缺少 Swarmmode --cap-add

測試

重新部署堆棧:

$ docker stack deploy --compose-file=docker-compose.yml secrets

等待幾秒鐘讓服務啟動,然後檢查狀態:

$ docker stack ps -f "desired-state=running" secrets

同樣,您應該會看到類似於以下內容的內容:

ID             NAME                         IMAGE           NODE      DESIRED STATE   CURRENT STATE
xtfsetfrbrs7   secrets_client.1             consul:1.10.3   node-3    Running         Running 19 minutes ago
ydqxexgiyzb2   secrets_client.2             consul:1.10.3   node-1    Running         Running 19 minutes ago
izlku3y6j8rp   secrets_server-bootstrap.1   consul:1.10.3   node-2    Running         Running 19 minutes ago
zqpkcrhrix2x   secrets_server.1             consul:1.10.3   node-1    Running         Running 19 minutes ago
kmlxuhxw1akv   secrets_server.2             consul:1.10.3   node-2    Running         Running 19 minutes ago
wfmscoj53m39   secrets_vault.1              vault:1.8.3     node-3    Running         Running about a minute ago

接下來,確保 Vault 列在 Consul UI 的“服務”部分:

領事用戶界面服務

您現在應該能夠通過 CLI、HTTP API 和 UI 與 Vault 進行交互。首先初始化和解封 Vault。然後,登錄並創建一個新密碼。

完成後刪除節點:

$ docker-machine rm node-1 node-2 node-3 -y

自動化腳本

最後,讓我們創建一個快速腳本來自動化部署過程:

  1. 使用 Docker Machine 提供三個 DigitalOcean 液滴
  2. 配置 Docker Swarm 模式
  3. 將節點添加到 Swarm
  4. 部署堆棧

將名為deploy.sh的新文件添加到項目根目錄:

#!/bin/bash


echo "Spinning up three droplets..."

for i in 1 2 3; do
  docker-machine create \
    --driver digitalocean \
    --digitalocean-region "nyc1" \
    --digitalocean-image=debian-10-x64 \
    --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
    --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
    node-$i;
done


echo "Initializing Swarm mode..."

docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)


echo "Adding the nodes to the Swarm..."

TOKEN=`docker-machine ssh node-1 docker swarm join-token worker | grep token | awk '{ print $5 }'`

for i in 2 3; do
  docker-machine ssh node-$i \
    -- docker swarm join --token ${TOKEN} $(docker-machine ip node-1):2377;
done


echo "Creating networking..."

eval $(docker-machine env node-1)
docker network create -d overlay --attachable core


echo "Deploying the stack..."

docker stack deploy --compose-file=docker-compose.yml secrets

試試看!

$ sh deploy.sh

完成後放下水滴:

$ docker-machine rm node-1 node-2 node-3 -y

代碼可以在vault-consul-swarm 倉庫中找到。乾杯!

來源:  https ://testdriven.io

#docker #vault 

如何使用 Docker Swarm 部署 Hashicorp 的 Vault 和 Consul
Shayna  Lowe

Shayna Lowe

1660553040

Comment Déployer Vault et Consul de Hashicorp Avec Docker Swarm

Voyons comment déployer Vault et Consul de Hashicorp sur DigitalOcean avec Docker Swarm.

À la fin, vous serez en mesure de :

  1. Provisionner des hôtes sur DigitalOcean avec Docker Machine
  2. Configurer un cluster Docker Swarm pour qu'il s'exécute sur DigitalOcean
  3. Exécutez Vault et Consul sur Docker Swarm

Principales dépendances :

  • Docker v20.10.8
  • Docker Compose v1.29.2
  • Docker-Machine v0.16.2
  • Coffre-fort v1.8.3
  • Consul v1.10.3

Consul

Créez un nouveau répertoire de projet :

$ mkdir vault-consul-swarm && cd vault-consul-swarm

Ajoutez ensuite un fichier docker-compose.yml à la racine du projet :

version: "3.8"

services:

  server-bootstrap:
    image: consul:1.10.3
    ports:
      - 8500:8500
    command: "agent -server -bootstrap-expect 3 -ui -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"

  server:
    image: consul:1.10.3
    command: "agent -server -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"
    deploy:
      replicas: 2
    depends_on:
      - server-bootstrap

  client:
    image: consul:1.10.3
    command: "agent -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"
    deploy:
      replicas: 2
    depends_on:
      - server-bootstrap

networks:
  default:
    external: true
    name: core

Cette configuration devrait vous sembler familière.

  1. Reportez-vous à la section Compose File du billet de blog Running Flask on Docker Swarm pour plus d'informations sur l'utilisation d'un fichier de composition pour le mode Docker Swarm.
  2. Consultez le guide Consul et Docker pour plus d'informations sur la configuration Consul ci-dessus.

Essaim Docker

Créez un compte DigitalOcean (si vous n'en avez pas déjà un), puis générez un jeton d'accès pour pouvoir accéder à l'API DigitalOcean.

Ajoutez le jeton à votre environnement :

$ export DIGITAL_OCEAN_ACCESS_TOKEN=[your_digital_ocean_token]

Faites tourner trois gouttelettes :

$ for i in 1 2 3; do
    docker-machine create \
      --driver digitalocean \
      --digitalocean-region "nyc1" \
      --digitalocean-image=debian-10-x64 \
      --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
      --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
      node-$i;
done

Initialisez le mode Swarm sur le premier nœud,node-1 :

$ docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)

Utilisez le jeton de jointure de la sortie de la commande précédente pour ajouter les deux nœuds restants au Swarm en tant que nœuds de calcul :

$ for i in 2 3; do
    docker-machine ssh node-$i -- docker swarm join --token YOUR_JOIN_TOKEN HOST:PORT;
done

Par exemple:

for i in 2 3; do
    docker-machine ssh node-$i -- docker swarm join --token SWMTKN-1-18xrfgcgq7k6krqr7tvav3ydx5c5104y662lzh4pyct2t0ror3-e3ed1ggivhf8z15i40z6x55g5 67.205.165.166:2377;
done

Tu devrais voir:

This node joined a swarm as a worker.
This node joined a swarm as a worker.

Pointez le démon Docker sur , créez un réseau de superpositionnode-1 attachable (appelé ) et déployez la pile :core

$ eval $(docker-machine env node-1)
$ docker network create -d overlay --attachable core
$ docker stack deploy --compose-file=docker-compose.yml secrets

Répertoriez les services dans la pile :

$ docker stack ps -f "desired-state=running" secrets

Vous devriez voir quelque chose de similaire à :

ID             NAME                         IMAGE          NODE     DESIRED STATE   CURRENT STATE
b5f5eycrhf3o   secrets_client.1             consul:1.10.3   node-1   Running         Running 7 seconds ago
zs7a5t8khcew   secrets_server.1             consul:1.10.3   node-2   Running         Running 9 seconds ago
qnhtlan6m0sp   secrets_server-bootstrap.1   consul:1.10.3   node-1   Running         Running 7 seconds ago
u61eycesmsl7   secrets_client.2             consul:1.10.3   node-2   Running         Running 9 seconds ago
vgpql8lfy5fi   secrets_server.2             consul:1.10.3   node-3   Running         Running 9 seconds ago

Récupérez l'IP associée à node-1:

$ docker-machine ip node-1

Ensuite, testez l'interface utilisateur Consul dans votre navigateur à l' adresse http://YOUR_MACHINE_IP:8500/ui . Il devrait y avoir trois services en cours d'exécution et cinq nœuds.

services d'interface utilisateur consul

nœuds d'interface utilisateur consul

Sauter

Ajoutez le vaultservice à docker-compose.yml :

vault:
  image: vault:1.8.3
  deploy:
    replicas: 1
  ports:
    - 8200:8200
  environment:
    - VAULT_ADDR=http://127.0.0.1:8200
    - VAULT_LOCAL_CONFIG={"backend":{"consul":{"address":"http://server-bootstrap:8500","path":"vault/"}},"listener":{"tcp":{"address":"0.0.0.0:8200","tls_disable":1}},"ui":true, "disable_mlock":true}
  command: server
  depends_on:
    - consul

Prenez note de la VAULT_LOCAL_CONFIGvariable d'environnement :

{
  "backend": {
    "consul": {
      "address": "http://server-bootstrap:8500",
      "path": "vault/"
    }
  },
  "listener": {
    "tcp": {
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true,
  "disable_mlock": true
}

Consultez la section Consul Backend du billet de blog Managing Secrets with Vault and Consul pour plus d'informations. De plus, définir disable_mlock sur truen'est pas recommandé pour les environnements de production ; cependant, il doit être activé car il --cap-addn'est pas disponible en mode Docker Swarm. Consultez les problèmes GitHub suivants pour plus de détails :

  1. --cap-add=IPC_LOCK indisponible dans l'essaim docker
  2. Absent de Swarmmode --cap-add

Test

Redéployez la pile :

$ docker stack deploy --compose-file=docker-compose.yml secrets

Attendez quelques secondes que les services démarrent, puis vérifiez l'état :

$ docker stack ps -f "desired-state=running" secrets

Encore une fois, vous devriez voir quelque chose de similaire à :

ID             NAME                         IMAGE           NODE      DESIRED STATE   CURRENT STATE
xtfsetfrbrs7   secrets_client.1             consul:1.10.3   node-3    Running         Running 19 minutes ago
ydqxexgiyzb2   secrets_client.2             consul:1.10.3   node-1    Running         Running 19 minutes ago
izlku3y6j8rp   secrets_server-bootstrap.1   consul:1.10.3   node-2    Running         Running 19 minutes ago
zqpkcrhrix2x   secrets_server.1             consul:1.10.3   node-1    Running         Running 19 minutes ago
kmlxuhxw1akv   secrets_server.2             consul:1.10.3   node-2    Running         Running 19 minutes ago
wfmscoj53m39   secrets_vault.1              vault:1.8.3     node-3    Running         Running about a minute ago

Ensuite, assurez-vous que Vault est répertorié dans la section "Services" de l'interface utilisateur Consul :

services d'interface utilisateur consul

Vous devriez maintenant pouvoir interagir avec Vault via la CLI, l'API HTTP et l'interface utilisateur. Commencez par initialiser et desceller Vault. Ensuite, connectez-vous et créez un nouveau secret.

Supprimez les nœuds une fois terminé :

$ docker-machine rm node-1 node-2 node-3 -y

Script d'automatisation

Enfin, créons un script rapide pour automatiser le processus de déploiement :

  1. Provisionner trois droplets DigitalOcean avec Docker Machine
  2. Configurer le mode Docker Swarm
  3. Ajouter des nœuds au Swarm
  4. Déployer la pile

Ajoutez un nouveau fichier appelé deploy.sh à la racine du projet :

#!/bin/bash


echo "Spinning up three droplets..."

for i in 1 2 3; do
  docker-machine create \
    --driver digitalocean \
    --digitalocean-region "nyc1" \
    --digitalocean-image=debian-10-x64 \
    --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
    --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
    node-$i;
done


echo "Initializing Swarm mode..."

docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)


echo "Adding the nodes to the Swarm..."

TOKEN=`docker-machine ssh node-1 docker swarm join-token worker | grep token | awk '{ print $5 }'`

for i in 2 3; do
  docker-machine ssh node-$i \
    -- docker swarm join --token ${TOKEN} $(docker-machine ip node-1):2377;
done


echo "Creating networking..."

eval $(docker-machine env node-1)
docker network create -d overlay --attachable core


echo "Deploying the stack..."

docker stack deploy --compose-file=docker-compose.yml secrets

Essaye le!

$ sh deploy.sh

Faites tomber les gouttelettes une fois fait :

$ docker-machine rm node-1 node-2 node-3 -y

Le code peut être trouvé dans le référentiel Vault-consul- swarm. Acclamations!

Source :  https://testdrive.io

#vault #docker 

Comment Déployer Vault et Consul de Hashicorp Avec Docker Swarm

Cómo Implementar Vault Y Consul De Hashicorp Con Docker Swarm

Veamos cómo implementar Vault y Consul de Hashicorp en DigitalOcean con Docker Swarm.

Al finalizar, podrá:

  1. Aprovisione hosts en DigitalOcean con Docker Machine
  2. Configure un clúster de Docker Swarm para que se ejecute en DigitalOcean
  3. Ejecute Vault y Consul en Docker Swarm

Dependencias principales:

  • Ventana acoplable v20.10.8
  • Docker-Componer v1.29.2
  • Docker-Máquina v0.16.2
  • Bóveda v1.8.3
  • Cónsul v1.10.3

Cónsul

Cree un nuevo directorio de proyecto:

$ mkdir vault-consul-swarm && cd vault-consul-swarm

Luego, agregue un archivo docker-compose.yml a la raíz del proyecto:

version: "3.8"services:  server-bootstrap:    image: consul:1.10.3    ports:      - 8500:8500    command: "agent -server -bootstrap-expect 3 -ui -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"  server:    image: consul:1.10.3    command: "agent -server -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"    deploy:      replicas: 2    depends_on:      - server-bootstrap  client:    image: consul:1.10.3    command: "agent -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"    deploy:      replicas: 2    depends_on:      - server-bootstrapnetworks:  default:    external: true    name: core

Esta configuración debería resultarle familiar.

  1. Consulte la sección Componer archivo de la publicación de blog Running Flask en Docker Swarm para obtener más información sobre el uso de un archivo de composición para el modo Docker Swarm.
  2. Consulte la guía de Consul y Docker para obtener información sobre la configuración de Consul anterior.

Enjambre Docker

Regístrese para obtener una cuenta de DigitalOcean (si aún no tiene una) y luego genere un token de acceso para poder acceder a la API de DigitalOcean.

Agregue el token a su entorno:

$ export DIGITAL_OCEAN_ACCESS_TOKEN=[your_digital_ocean_token]

Haz girar tres gotas:

$ for i in 1 2 3; do    docker-machine create \      --driver digitalocean \      --digitalocean-region "nyc1" \      --digitalocean-image=debian-10-x64 \      --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \      --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \      node-$i;done

Inicialice el modo Swarm en el primer nodo node-1:

$ docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)

Use el token de unión de la salida del comando anterior para agregar los dos nodos restantes al Swarm como trabajadores:

$ for i in 2 3; do    docker-machine ssh node-$i -- docker swarm join --token YOUR_JOIN_TOKEN HOST:PORT;done

Por ejemplo:

for i in 2 3; do    docker-machine ssh node-$i -- docker swarm join --token SWMTKN-1-18xrfgcgq7k6krqr7tvav3ydx5c5104y662lzh4pyct2t0ror3-e3ed1ggivhf8z15i40z6x55g5 67.205.165.166:2377;done

Debería ver:

This node joined a swarm as a worker.This node joined a swarm as a worker.

Apunte el demonio de Docker a node-1, cree una red de superposición conectable (llamada core) e implemente la pila:

$ eval $(docker-machine env node-1)$ docker network create -d overlay --attachable core$ docker stack deploy --compose-file=docker-compose.yml secrets

Enumere los servicios en la pila:

$ docker stack ps -f "desired-state=running" secrets

Deberías ver algo similar a:

ID             NAME                         IMAGE          NODE     DESIRED STATE   CURRENT STATEb5f5eycrhf3o   secrets_client.1             consul:1.10.3   node-1   Running         Running 7 seconds agozs7a5t8khcew   secrets_server.1             consul:1.10.3   node-2   Running         Running 9 seconds agoqnhtlan6m0sp   secrets_server-bootstrap.1   consul:1.10.3   node-1   Running         Running 7 seconds agou61eycesmsl7   secrets_client.2             consul:1.10.3   node-2   Running         Running 9 seconds agovgpql8lfy5fi   secrets_server.2             consul:1.10.3   node-3   Running         Running 9 seconds ago

Tome la IP asociada con node-1:

$ docker-machine ip node-1

Luego, pruebe la interfaz de usuario de Consul en su navegador en http://YOUR_MACHINE_IP:8500/ui . Debe haber tres servicios en ejecución y cinco nodos.

servicios de interfaz de usuario del cónsul

nodos de la interfaz de usuario del cónsul

Bóveda

Agregue el vaultservicio a docker-compose.yml :

vault:  image: vault:1.8.3  deploy:    replicas: 1  ports:    - 8200:8200  environment:    - VAULT_ADDR=http://127.0.0.1:8200    - VAULT_LOCAL_CONFIG={"backend":{"consul":{"address":"http://server-bootstrap:8500","path":"vault/"}},"listener":{"tcp":{"address":"0.0.0.0:8200","tls_disable":1}},"ui":true, "disable_mlock":true}  command: server  depends_on:    - consul

Toma nota de la VAULT_LOCAL_CONFIGvariable de entorno:

{  "backend": {    "consul": {      "address": "http://server-bootstrap:8500",      "path": "vault/"    }  },  "listener": {    "tcp": {      "address": "0.0.0.0:8200",      "tls_disable": 1    }  },  "ui": true,  "disable_mlock": true}

Consulte la sección Backend de Consul de la publicación de blog Administración de secretos con Vault y Consul para obtener más información. Además, no se recomienda establecer disabled_mlock en trueentornos de producción; sin embargo, debe estar habilitado ya --cap-addque no está disponible en el modo Docker Swarm. Consulte los siguientes problemas de GitHub para obtener más detalles:

  1. --cap-add=IPC_LOCK no disponible en Docker Swarm
  2. Falta en Swarmmode --cap-add

Prueba

Vuelva a implementar la pila:

$ docker stack deploy --compose-file=docker-compose.yml secrets

Espere unos segundos para que los servicios se activen y luego verifique el estado:

$ docker stack ps -f "desired-state=running" secrets

De nuevo, deberías ver algo similar a:

ID             NAME                         IMAGE           NODE      DESIRED STATE   CURRENT STATExtfsetfrbrs7   secrets_client.1             consul:1.10.3   node-3    Running         Running 19 minutes agoydqxexgiyzb2   secrets_client.2             consul:1.10.3   node-1    Running         Running 19 minutes agoizlku3y6j8rp   secrets_server-bootstrap.1   consul:1.10.3   node-2    Running         Running 19 minutes agozqpkcrhrix2x   secrets_server.1             consul:1.10.3   node-1    Running         Running 19 minutes agokmlxuhxw1akv   secrets_server.2             consul:1.10.3   node-2    Running         Running 19 minutes agowfmscoj53m39   secrets_vault.1              vault:1.8.3     node-3    Running         Running about a minute ago

A continuación, asegúrese de que Vault aparezca en la sección "Servicios" de la interfaz de usuario de Consul:

servicios de interfaz de usuario del cónsul

Ahora debería poder interactuar con Vault a través de la CLI, la API HTTP y la interfaz de usuario. Comience por inicializar y abrir Vault. Luego, inicie sesión y cree un nuevo secreto.

Retire los nodos una vez hecho:

$ docker-machine rm node-1 node-2 node-3 -y

Guión de automatización

Finalmente, creemos un script rápido para automatizar el proceso de implementación:

  1. Aprovisione tres gotitas de DigitalOcean con Docker Machine
  2. Configurar el modo Docker Swarm
  3. Agregar nodos al enjambre
  4. Implementar la pila

Agrega un nuevo archivo llamado deployment.sh a la raíz del proyecto:

#!/bin/bashecho "Spinning up three droplets..."for i in 1 2 3; do  docker-machine create \    --driver digitalocean \    --digitalocean-region "nyc1" \    --digitalocean-image=debian-10-x64 \    --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \    --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \    node-$i;doneecho "Initializing Swarm mode..."docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)echo "Adding the nodes to the Swarm..."TOKEN=`docker-machine ssh node-1 docker swarm join-token worker | grep token | awk '{ print $5 }'`for i in 2 3; do  docker-machine ssh node-$i \    -- docker swarm join --token ${TOKEN} $(docker-machine ip node-1):2377;doneecho "Creating networking..."eval $(docker-machine env node-1)docker network create -d overlay --attachable coreecho "Deploying the stack..."docker stack deploy --compose-file=docker-compose.yml secrets

¡Pruébalo!

$ sh deploy.sh

Baje las gotas una vez hecho:

$ docker-machine rm node-1 node-2 node-3 -y

El código se puede encontrar en el repositorio vault-consul-swarm . ¡Salud!

Fuente:  https://testdriven.io

#vault #docker 

Cómo Implementar Vault Y Consul De Hashicorp Con Docker Swarm

Como Implantar O Vault E O Consul Da Hashicorp Com O Docker Swarm

Vejamos como implantar o Vault e o Consul da Hashicorp na DigitalOcean com o Docker Swarm.

Após a conclusão, você será capaz de:

  1. Provisionar hosts na DigitalOcean com Docker Machine
  2. Configurar um cluster do Docker Swarm para ser executado na DigitalOcean
  3. Execute o Vault e o Consul no Docker Swarm

Principais dependências:

  • Docker v20.10.8
  • Docker-Compose v1.29.2
  • Máquina Docker v0.16.2
  • Cofre v1.8.3
  • Cônsul v1.10.3

Cônsul

Crie um novo diretório de projeto:

$ mkdir vault-consul-swarm && cd vault-consul-swarm

Em seguida, adicione um arquivo docker-compose.yml à raiz do projeto:

version: "3.8"

services:

  server-bootstrap:
    image: consul:1.10.3
    ports:
      - 8500:8500
    command: "agent -server -bootstrap-expect 3 -ui -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"

  server:
    image: consul:1.10.3
    command: "agent -server -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"
    deploy:
      replicas: 2
    depends_on:
      - server-bootstrap

  client:
    image: consul:1.10.3
    command: "agent -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"
    deploy:
      replicas: 2
    depends_on:
      - server-bootstrap

networks:
  default:
    external: true
    name: core

Essa configuração deve parecer familiar.

  1. Consulte a seção Compose File da postagem do blog Running Flask on Docker Swarm para obter mais informações sobre como usar um arquivo de composição para o modo Docker Swarm.
  2. Revise o guia do Consul e do Docker para obter informações sobre a configuração do Consul acima.

Enxame Docker

Inscreva-se em uma conta da DigitalOcean (se você ainda não tiver uma) e gere um token de acesso para poder acessar a API da DigitalOcean.

Adicione o token ao seu ambiente:

$ export DIGITAL_OCEAN_ACCESS_TOKEN=[your_digital_ocean_token]

Gire três gotas:

$ for i in 1 2 3; do
    docker-machine create \
      --driver digitalocean \
      --digitalocean-region "nyc1" \
      --digitalocean-image=debian-10-x64 \
      --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
      --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
      node-$i;
done

Inicialize o modo Swarm no primeiro nó, node-1:

$ docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)

Use o token de junção da saída do comando anterior para adicionar os dois nós restantes ao Swarm como trabalhadores:

$ for i in 2 3; do
    docker-machine ssh node-$i -- docker swarm join --token YOUR_JOIN_TOKEN HOST:PORT;
done

Por exemplo:

for i in 2 3; do
    docker-machine ssh node-$i -- docker swarm join --token SWMTKN-1-18xrfgcgq7k6krqr7tvav3ydx5c5104y662lzh4pyct2t0ror3-e3ed1ggivhf8z15i40z6x55g5 67.205.165.166:2377;
done

Você deveria ver:

This node joined a swarm as a worker.
This node joined a swarm as a worker.

Aponte o daemon do Docker para node-1, crie uma rede de sobreposição anexável (chamada core) e implante a pilha:

$ eval $(docker-machine env node-1)
$ docker network create -d overlay --attachable core
$ docker stack deploy --compose-file=docker-compose.yml secrets

Liste os serviços na pilha:

$ docker stack ps -f "desired-state=running" secrets

Você deve ver algo semelhante a:

ID             NAME                         IMAGE          NODE     DESIRED STATE   CURRENT STATE
b5f5eycrhf3o   secrets_client.1             consul:1.10.3   node-1   Running         Running 7 seconds ago
zs7a5t8khcew   secrets_server.1             consul:1.10.3   node-2   Running         Running 9 seconds ago
qnhtlan6m0sp   secrets_server-bootstrap.1   consul:1.10.3   node-1   Running         Running 7 seconds ago
u61eycesmsl7   secrets_client.2             consul:1.10.3   node-2   Running         Running 9 seconds ago
vgpql8lfy5fi   secrets_server.2             consul:1.10.3   node-3   Running         Running 9 seconds ago

Grab the IP associated with node-1:

$ docker-machine ip node-1

Then, test out the Consul UI in your browser at http://YOUR_MACHINE_IP:8500/ui. There should be three running services and five nodes.

serviços de interface do cônsul

nós de interface do usuário do cônsul

Vault

Add the vault service to docker-compose.yml:

vault:
  image: vault:1.8.3
  deploy:
    replicas: 1
  ports:
    - 8200:8200
  environment:
    - VAULT_ADDR=http://127.0.0.1:8200
    - VAULT_LOCAL_CONFIG={"backend":{"consul":{"address":"http://server-bootstrap:8500","path":"vault/"}},"listener":{"tcp":{"address":"0.0.0.0:8200","tls_disable":1}},"ui":true, "disable_mlock":true}
  command: server
  depends_on:
    - consul

Take note of the VAULT_LOCAL_CONFIG environment variable:

{
  "backend": {
    "consul": {
      "address": "http://server-bootstrap:8500",
      "path": "vault/"
    }
  },
  "listener": {
    "tcp": {
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true,
  "disable_mlock": true
}

Review the Consul Backend section from the Managing Secrets with Vault and Consul blog post for more info. Also, setting disable_mlock to true is not recommended for production environments; however, it must be enabled since --cap-add is not available in Docker Swarm mode. See the following GitHub issues for details:

  1. --cap-add=IPC_LOCK unavailable in docker swarm
  2. Missing from Swarmmode --cap-add

Test

Re-deploy the stack:

$ docker stack deploy --compose-file=docker-compose.yml secrets

Wait a few seconds for the services to spin up, then check the status:

$ docker stack ps -f "desired-state=running" secrets

Again, you should see something similar to:

ID             NAME                         IMAGE           NODE      DESIRED STATE   CURRENT STATE
xtfsetfrbrs7   secrets_client.1             consul:1.10.3   node-3    Running         Running 19 minutes ago
ydqxexgiyzb2   secrets_client.2             consul:1.10.3   node-1    Running         Running 19 minutes ago
izlku3y6j8rp   secrets_server-bootstrap.1   consul:1.10.3   node-2    Running         Running 19 minutes ago
zqpkcrhrix2x   secrets_server.1             consul:1.10.3   node-1    Running         Running 19 minutes ago
kmlxuhxw1akv   secrets_server.2             consul:1.10.3   node-2    Running         Running 19 minutes ago
wfmscoj53m39   secrets_vault.1              vault:1.8.3     node-3    Running         Running about a minute ago

Next, ensure Vault is listed on the "Services" section of the Consul UI:

serviços de interface do cônsul

You should now be able to interact with Vault via the CLI, HTTP API, and the UI. Start by initializing and unsealing Vault. Then, log in and create a new secret.

Remove the nodes once done:

$ docker-machine rm node-1 node-2 node-3 -y

Automation Script

Finally, let's create a quick script to automate the deployment process:

  1. Provision three DigitalOcean droplets with Docker Machine
  2. Configure Docker Swarm mode
  3. Add nodes to the Swarm
  4. Deploy the stack

Add a new file called deploy.sh to the project root:

#!/bin/bash


echo "Spinning up three droplets..."

for i in 1 2 3; do
  docker-machine create \
    --driver digitalocean \
    --digitalocean-region "nyc1" \
    --digitalocean-image=debian-10-x64 \
    --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
    --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
    node-$i;
done


echo "Initializing Swarm mode..."

docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)


echo "Adding the nodes to the Swarm..."

TOKEN=`docker-machine ssh node-1 docker swarm join-token worker | grep token | awk '{ print $5 }'`

for i in 2 3; do
  docker-machine ssh node-$i \
    -- docker swarm join --token ${TOKEN} $(docker-machine ip node-1):2377;
done


echo "Creating networking..."

eval $(docker-machine env node-1)
docker network create -d overlay --attachable core


echo "Deploying the stack..."

docker stack deploy --compose-file=docker-compose.yml secrets

Try it out!

$ sh deploy.sh

Bring down the droplets once done:

$ docker-machine rm node-1 node-2 node-3 -y

The code can be found in the vault-consul-swarm repo. Cheers!

Fonte:  https://testdrive.io

#vault #docker 

Como Implantar O Vault E O Consul Da Hashicorp Com O Docker Swarm
Vicenta  Hauck

Vicenta Hauck

1660390620

How to Use Vault To Create Postgres Credentials for A Flask App

In this tutorial, we'll look at a quick, real-world example of using Hashicorp's Vault and Consul to create dynamic Postgres credentials for a Flask web app.

Source: https://testdriven.io

#vault #flask #postgres 

How to Use Vault To Create Postgres Credentials for A Flask App