I have recently started looking into Hashicorp Vault as a way to expand my kowledge of things for fun and profit. There is much and more about Vault on the interwebs, but here is my log of discovery. A blog is just a fancy way to remember things, amirite?
Getting Started
I'm going to just put vault binary on my laptop instead of in something like a container or vagrant box. So first step is to download and unzip it. I'm going to put it into /usr/local/bin:
ls -l vau*
-rw-r--r--@ 1 alan staff 17470290 Nov 24 20:09 vault_0.9.0_darwin_amd64.zip
unzip vault_0.9.0_darwin_amd64.zip
Archive: vault_0.9.0_darwin_amd64.zip
inflating: vault
ls -l vault
-rwxr-xr-x@ 1 alan staff 74993168 Nov 14 12:02 vault
sudo mv vault /usr/local/bin/.
Password:
which vault
/usr/local/bin/vault
Now that I have vault in my path, time to see what there is to do with it:
vault help
usage: vault [-version] [-help] [args]
Common commands:
delete Delete operation on secrets in Vault
path-help Look up the help for a path
read Read data or secrets from Vault
renew Renew the lease of a secret
revoke Revoke a secret.
server Start a Vault server
status Outputs status of whether Vault is sealed and if HA mode is enabled
unwrap Unwrap a wrapped secret
write Write secrets or configuration into Vault
All other commands:
audit-disable Disable an audit backend
audit-enable Enable an audit backend
audit-list Lists enabled audit backends in Vault
auth Prints information about how to authenticate with Vault
auth-disable Disable an auth provider
auth-enable Enable a new auth provider
capabilities Fetch the capabilities of a token on a given path
generate-root Generates a new root token
init Initialize a new Vault server
key-status Provides information about the active encryption key
list List data or secrets in Vault
mount Mount a logical backend
mount-tune Tune mount configuration parameters
mounts Lists mounted backends in Vault
policies List the policies on the server
policy-delete Delete a policy from the server
policy-write Write a policy to the server
rekey Rekeys Vault to generate new unseal keys
remount Remount a secret backend to a new path
rotate Rotates the backend encryption key used to persist data
seal Seals the Vault server
ssh Initiate an SSH session
step-down Force the Vault node to give up active duty
token-create Create a new auth token
token-lookup Display information about the specified token
token-renew Renew an auth token if there is an associated lease
token-revoke Revoke one or more auth tokens
unmount Unmount a secret backend
unseal Unseals the Vault server
version Prints the Vault version
Well, thats a lot of stuff. Luckily I've read that the first thing I have to do is create a configuration file.
mkdir ~/vault-stuff
mkdir ~/vault-stuff/data
vi ~/vault-stuff/vault.hcl
### MAKE CONFIG FILE ###
cat vault-stuff/vault.hcl
storage "file" {
path = "/Users/alan/vault-stuff/data"
}
listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = 1
}
So I:
- Created the filesystem storage for holding the config file and the on-disk data
- Created the configuration file, which will instruct vault to
- Use filesystem storage to keep secrets on disk (as opposed to using consul or one of the many other options)
- use localhost listener on port 8200, without TLS for now
Now I need to start the vault service, and send it to the background so I can close the terminal if needed
sudo vault server -config ~/vault-stuff/vault.hcl &
[1] 39278
Alans-Air:vault-stuff alan$ ==> WARNING: mlock not supported on this system!
An `mlockall(2)`-like syscall to prevent memory from being
swapped to disk is not supported on this system. Running
Vault on an mlockall(2) enabled system is much more secure.
==> Vault server configuration:
Cgo: disabled
Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", tls: "disabled")
Log Level: info
Mlock: supported: false, enabled: false
Storage: file
Version: Vault v0.9.0
Version Sha: bdac1854478538052ba5b7ec9a9ec688d35a3335
==> Vault server started! Log data will stream in below:
Looks like it started, but has an issue with mlock. I'll ignore that for now. Just playing around here. Next step is to export the address for the API client to talk to the server. Since I disabled TLS for now, I have to override the defualt, which will use https://. Then I can query vault status, and see if it reports back that it's sitting in an unsealed state.
export VAULT_ADDR=http://127.0.0.1:8200
vault status
2017/11/25 17:14:50.283528 [INFO ] core: seal configuration missing, not initialized
Error checking seal status: Error making API request.
URL: GET http://127.0.0.1:8200/v1/sys/seal-status
Code: 400. Errors:
* server is not yet initialized
SUCCESS! The above output shows that the server process is running, and correctly reports back that it is in the uninitialized state I expected it to be in.
Initializing Vault
Lets see if I can initialize it:
vault init
Unseal Key 1: tt/9Pz0a2zrCxcuaJwOfNv3oRbrM26XzWW7XTvFiYFJ/
Unseal Key 2: igN3kQIqel61o3UXFvNUksVXvscYXpr9pf9kq+rrL7w4
Unseal Key 3: W6bqfaitfHjgm0HavZCNbIxNuz6FqZATECVNkCpOlEQk
Unseal Key 4: 5rXPhU4/XNVGIYVKcnPBdtooKT94K1ASo9tx4u8AI/wf
Unseal Key 5: wjkZAJUOivD+Z+CeaR2lPoMGjir3nfK+687rlgFhpUcN
Initial Root Token: b1a61244-65ff-75ce-1126-9b5f5cfda220
Vault initialized with 5 keys and a key threshold of 3. Please
securely distribute the above keys. When the vault is re-sealed,
restarted, or stopped, you must provide at least 3 of these keys
to unseal it again.
Vault does not store the master key. Without at least 3 keys,
your vault will remain permanently sealed.
Cool, I got the root token and the Shamir's shards for unsealing the vault. Also, looks like it put stuff in the designated storage area
ls vault-stuff/data
core sys
Time to unseal the vault. So exciting!
vault unseal
Key (will be hidden):
Sealed: true
Key Shares: 5
Key Threshold: 3
Unseal Progress: 1
Unseal Nonce: 41b2f026-6ee4-dab6-8269-37e0de68de80
vault unseal
Key (will be hidden):
Sealed: true
Key Shares: 5
Key Threshold: 3
Unseal Progress: 2
Unseal Nonce: 41b2f026-6ee4-dab6-8269-37e0de68de80
vault unseal
Key (will be hidden):
Error: Error making API request.
URL: PUT http://127.0.0.1:8200/v1/sys/unseal
Code: 400. Errors:
* 'key' must be a valid hex or base64 string
vault unseal
Key (will be hidden):
Sealed: false
Key Shares: 5
Key Threshold: 3
Unseal Progress: 0
Unseal Nonce:
vault status
Type: shamir
Sealed: false
Key Shares: 5
Key Threshold: 3
Unseal Progress: 0
Unseal Nonce:
Version: 0.9.0
Cluster Name: vault-cluster-dc228e99
Cluster ID: f09d1904-67c6-4e4a-4118-b629649017eb
High-Availability Enabled: false
Awesome. So the vault is unsealed (with one bad copy/paste there in the middle)! Now it's time for me to authenticate to it...
vault auth
Token (will be hidden):
Successfully authenticated! You are now logged in.
token: b1a61244-65ff-75ce-1126-9b5f5cfda220
token_duration: 0
token_policies: [root]
Obviously I would replace the root token if this was something that was going to be shared. But for now, I'll just leave it to make my life easier :)
Backends
Now I can start to see about the real deal: secrets! Vault uses mounted paths for reading and writing secrets (along with everything else it does). So, lets see what mounts I have by out-of-the-box.
vault mounts
Path Type Accessor Plugin Default TTL Max TTL Force No Cache Replication Behavior Description
cubbyhole/ cubbyhole cubbyhole_6222f487 n/a n/a n/a false local per-token private secret storage
identity/ identity identity_902aecf8 n/a n/a n/a false replicated identity store
secret/ kv kv_aee6d2a6 n/a system system false replicated key/value secret storage
sys/ system system_042cc31d n/a n/a n/a false replicated system endpoints used for control, policy and debugging
The output is wide, but there are all the mounts I have to work with. Based on the description, there are two mounts that seem to be for specifically containing secrets: cubbyhole and secret. The others are more for the functionality of Vault itself. Identity is for identity management and sys is used for Vault internals. As indicated by the output, cubbyhole is a per user place to store secrets. Each token created will have it's own place to store secrets that are available only to it. If that token is removed, it's cubbyhole will also go away. Secret is a generic key/value (kv) store mounted by default. It can be accessed by any token given the ability via it's ACL. Let's write a secret:
vault list secret # see no secrets exists
No value found at secret/
vault write secret/foo value=bar
Success! Data written to: secret/foo
vault list secret # see secret exists now
Keys
- - - -
foo
vault read secret/foo
Key Value
- - - - - - - -
refresh_interval 768h0m0s
value bar
As the documentation says, never write secrets directly from the command line like that. But I trust you not to divulge my foo secret.
So now I have a minimally functional Vault installation running. Next time, I'll work on using it for something cool.
Thanks for reading.
Part 2 - Running vault as a daemon on OSX
Part 3 - Unseal Vault with LastPass