Scroll to top
© 2019, DigitalOnUs

How to Set up a HashiCorp Vault Cluster in 15 Minutes


DigitalOnUs - January 4, 2019 - 0 comments

When you search online for a quick way to set up a HashiCorp vault cluster, you will get some results that cover the concepts, some that discuss how to use Vault’s API and some others that tell you how to enable Vault Enterprise features in general.

HashiCorp has an excellent Vault Deployment Guide and a solid Vault Reference Architecture but they come with some pre-requisites. For instance, in the Deployment Guide, you would first need to know how to install a Vault Server and how to network containers or other infrastructure to run it on, before starting off with the guide. The Reference Architecture is more of an overview of sample architecture, which does not help with the details, of the commands to run, which are not directly related to Vault or Consul. These are also not prescriptive about what kind of networking stack or other infrastructure can help you set up your Vault cluster.

This article helps those who are familiar with scripting, Git, configuring new SSH connections, installing software and Virtual Machines. As you install the cross-platform, open-source tools (Vagrant, VirtualBox, and Git), run universal commands and wait for 10 minutes, your new Vault Cluster is ready to play.

You can get this Vault environment up and running in 15 minutes’ time with just 4 command lines!* You get a fully operational set of 3 Vault servers with a backend protected by ACLs.

Let’s get started.

The diagram below shows 3 Virtual Machines on a “host-only” network. This means, rather than being a part of the Internet, they are on the equivalent of your local wireless network confined to your laptop. This is a Personal Network and you never have to worry about any of that because VirtualBox and Vagrant will do the work for you.

The names of the Virtual Machines, indicated in the diagram by the white rectangles, are instance5, instance6 and instance7. Each of them has both Consul and Vault installed on it. The Consuls are configured to form a cluster, which acts as a storage and High Availability “backend” for the Vault servers.

First, let’s get some prerequisites out of the way.

If you get stuck with the prerequisites or tools to install or download the code, please take a look at the resources on the Internet.

Once you have Vagrant and VirtualBox installed, the Getting Started guide takes about 30 minutes. If Vault is not working properly, you could post your concern on the Vault Discussion Group to seek help in finding out if it is a bug or not.

OS-Specific Prerequisites

  • MacOS: OSX 10.13 or later
  • Windows: Windows must be equipped with PowerShell 3.0 or its later version. If you’re on Windows 7, you could use Windows Management Framework 4.0 since it is easier to install.

Install VirtualBox and Git

The first tools that you could download are VirtualBox and Git, as they will let you run a “Virtual Machine” and allow you to get code for this walk-through.

Install Tools

  • Make sure you have Git installed.
  • Install the latest version of Vagrant.
  • Install the latest version of VMware or VirtualBox.

Vagrant is a tool that helps with all the minor details in development environments. It is like an Infrastructure as Code (IaC) for Virtual Machines and Containers and for automating the boring parts of setting up programs for development work.

Download the Code for the Vault Cluster Setup

Related Vendor Documentation: https://help.github.com/articles/cloning-a-repository

Git clone: https://github.com/v6/super-duper-vault-train.git

Code to Make a Vault Cluster

Related Vagrant Vendor Documentation: https://www.vagrantup.com/intro/index.html#why-vagrant-

  1. cd super-duper-vault-train
  2. vagrant up ## NOTE: You may have to wait a while for vagrant up to complete, and there will be some “connection retry” errors for a long time before a successful connection occurs, because the VM is booting. Make sure you have the latest version, and try the Vagrant getting started guide, too
  3. vagrant status
  4. vagrant ssh instance5 After you ssh to that VM named instance5, you’ll see your command prompt change to show vagrant@instance5. You can also vagrant ssh to other VMs listed in the output of vagrant status.
  5. You can now use Vault or Consul from within the VM for which you ran vagrant ssh. For example, try the commands consul members or vault status from within any of your VMs.

Vault

Explore the Vault Cluster

ps -ef | grep vault ## Check the Vault process (run while inside a Vagrant-managed Instance)

ps -ef | grep consul ## Check the Consul process (run while inside a Vagrant-managed Instance)

vault version ## Output should be Vault v0.10.2 ('3ee0802ed08cb7f4046c2151ec4671a076b76166')

consul version ## Output should show Consul Agent version and Raft Protocol version

The Vagrant boxen have the following IP addresses:

192.168.13.35

192.168.13.36

192.168.13.37
 Both Vault and Consul are running on each of them. Vault is on port 8200. Consul is on port 8500.

Open these links in tabs:

http://192.168.13.35:8200 (Vault)

http://192.168.13.35:8500 (Consul)

http://192.168.13.36:8200 (Vault)

http://192.168.13.36:8500 (Consul)

http://192.168.13.37:8200 (Vault)

http://192.168.13.37:8500 (Consul)

Start Vault Data

Related Vendor Documentation Link: https://www.vaultproject.io/api/system/init.html

Start Vault.

Run this curl command on one of the Vagrant-managed VMs, or somewhere on your computer that has curl installed.

curl -s --request PUT -d '{"secret_shares": 3,"secret_threshold": 2}' http://192.168.13.35:8200/v1/sys/init

Unseal Vault

Related Vendor Documentation: https://www.vaultproject.io/api/system/unseal.html

If successful, this process will unseal the Vault at 192.168.13.35:8200. You can use the same process for 192.168.13.36:8200 and 192.168.13.37:8200.

Use your unseal key to replace the value for key abcd1430890…, and run this on the Vagrant-managed VM.

    curl --request PUT --data '{"key":"abcd12345678..."}' http://192.168.13.35:8200/v1/sys/unseal

Run that curl command again. But use a different value for “key”:. Replace efgh2541901… with a different key than you used in the previous step, from the keys you received when running the v1/sys/init endpoint.

    curl --request PUT --data '{"key":"efgh910111213..."}' http://192.168.13.35:8200/v1/sys/unseal

Non-Vagrant

Please refer to the file PRODUCTION_INSTALLATION.md in the repository we cloned earlier.

Codified Vault Policies and Configuration

To Provision Vault via its API, please refer to the provision_vault folder in the repository that we cloned earlier with data and scripts.

The data folder’s tree corresponds to the HashiCorp Vault API endpoints that are similar to the: https://www.hashicorp.com/blog/codifying-vault-policies-and-configuration#layout-and-design

You can use the Codified Vault Policies and Configuration with your initial Root token, after initializing and unsealing Vault to configure Vault quickly via its API.

The .json files inside each folder correspond to the payloads to send to Vault via its API, but there may also be .hcl, .sample, and .sh files for added convenience.

Further reading for manual setup: https://medium.com/rigged-ops/building-a-local-hashicorp-vault-cluster-5575fe322a17

*Disclaimer: The 4 command lines work only after installing the latest versions of Git, VirtualBox and Vagrant.

Related posts

Post a Comment

Your email address will not be published. Required fields are marked *