Fundamentals of HashiCorp-vault

HashiCorp Vault fundamentals

Hashicorp Vault is a prominent secret management solution today. It is currently used by the world's top financial solutions and enterprises.

What is the HashiCorp vault?

It’s a cloud-agnostic secrets management system. It doesn’t depend on a cloud provider and can work with any provider or environment.
If you have a diversified cloud infrastructure, with half of it on the public cloud and half on the private cloud, you can easily integrate Vault with it.
API driven

Used to generate dynamic short-lived credentials, or encrypt application data on the fly.

What is a secret below are some of the most common types of secrets

username and password
Certificates
SSH Keys
API Keys
Encryption Keys

What problems does Vault solve?

Secret Sprawl

Below are some common places where secrets get stored:

On a developer’s computer in Excel or Notepad files.
Hard-coded into the source code
On a sticky node under an engineer’s keyboard
In a version control system such as GitHub Sometimes exposed publicly

Vault use cases:

Secrets Management:

Centrally store, access, and distribute secrets.

Encrypting Application Data:

Keep application data secure with centralized key management.

Identity-Based Access:

Authenticate and access different clouds, systems, and endpoints using trusted identities.

When the client accesses the Foo secret in the vault, the Vault checks the policy associated with the user to confirm the permissions given to the user. If the permissions are enough it will provide the user with the secrets it is seeking by returning the secrets stored in the mentioned directory in the vault.

Foo is key

Secret is vault

So we call it Authentication

KV secret engine

Encrypting Application Data

EAAS —> Encrpypting as a service

These types of requests are usually done by apps to the vault to encrypt the data provided to the app. It as usual checks for permissions in the policy file of the application and if it has returned encrypted data which can be stored in the database of the applications.

Identity-Based Access

Dynamic AWS CREDS

Here is the data flow data of the user creating identity-based access creds.

The user requests the vault to Access the Secrets of AWS account, and the vault reads the policy associated with the user, if it has enough permission to create the AWS IAM user then the vault will request aws to create the IAM account and will receive access the secrets of the IAM user which it will pass to the user.

Basic vault CLI Commands

Vault by itself will give you a list of many vault cli commands.
```
  vault
```
The vault version tells you the version of the vault you are running.
```
  vault version
```
Vault read is used to read secrets from the vault.
```
  vault read
```
Vault write is used to write secrets to the vault.
```
  vault write
```
The -h, - - help, and -help flags can be added to get help for any vault cli command.

Vault Server modes

Vault server can be run in two different modes:

Dev mode that is only intended for development.
Prod mode that can be used in Q&A and production.

Vault’s Dev mode:

It is not secure
It stores everything in memory
vault is automatically unselected (You can only interact with vault when it is unsealed)
The root token can be specified before launching.

“Please never store actual secrets on a server run in ‘dev‘ mode“

vault server -dev

It will generate automatically

unseal key
Root token

The root token in this arrangement is very complex.

vault server server -dev -dev-root-token-id=root

This will specify the root token as given.

To specify the vault URL address you have to run this command

export VAULT_ADDR='http://127.0.0.1:8200'

To see the status of the vault.

vault status

To login into the vault

vault login

Enter your root token in this window.

Let’s start writing our secrets.

vault kv put secret/foo bar=baz

This will put a secret/foo file with bar=baz

So we wrote our first secret CONGURALATIONS.

Now how can we update it?

vault kv sceret/foo bar=baz hello=world

To retrieve a secrets we do:

 vault kv get sceret/foo

To delete a secret we can do

vault delete sceret/foo

If you want to undelete

vault kv undelete -version=<version of your secret> secret/foo

If you want to delete the file from permanent memory

vault kv destory secret/foo

Vault Architecture Internals Overview

Deeper knowledge of architecture

Vault seal and unseal

vault starts up in a sealed state.
Vault can access the physical storage, but can’t decrypt it.
No operations are possible with the vault except to unseal it and check its status.

Vault data is encrypted using the encryption key in the keyring. The keyring is encrypted by the master key and the master key is encrypted by the unseal key.

In this method, the key is split up into different stakeholders. And these custodians of the keys then come together to create what we call combined keys.

Shamir seal is the default mechanism.
Algorithm to split the key into shards.
A certain threshold of the shards is required to reconstruct the unsealed key, which is then used to decrypt the master key.
Shards are added one at a time (in any order) until enough shards are present to reconstruct the combined key.
To unseal: use

vault operator unseal

Or by the API or by UI.

Make sure that all five keys are not present in the same place.

By default, Shamir keys provide 5 keys, and the threshold of three by default will unseal the vault.

Seal the vault with a vault operator seal or via the API.
Sealing removes the master key in memory and requires another unseal process to restore it.
Sealing requires a single operator with root privileges.
Reason for sealing:

If there is a detected intrusion, the vault data can be locked quickly to try to minimize damages.
It is revealed via the API.
The server is restarted.
Vault’s storage layer encounters an unrecoverable error.

Unsealing with Auto Unseal

Auto unseal reduces the operational complexity of keeping the unseal key secure.

Delegates the responsibility of securing the unseal key from users to a trusted device or service.

At startup vault will connect to the device or service implementing the seal and ask it to decrypt the master key vault read from storage.
Initialization with Shamir=>unseal key
initialization with Auto unseal=>Recovery keys

Some operations such as generating a root token require unseal keys (in case of auto unseal we use the recovery keys.

The configure file:-

Starting a production vault server.

Running a vault server in ‘prod‘ mode involves multiple steps:

Specify the configuration in a configuration file.
Start the server.
Initialize the server to get unseal keys and an initial root token.
Unseal the vault server with the unseal keys.

Configuration File

A production vault server gets its configuration from a configuration file.

Vault configuration files can be specified in HCL or JSON.

To run a production server use this command:

vault server -config=./vault-config.hcl

Notice we don’t use the -dev flag.

Listener “tcp“ {
address = "0.0.0.0:8200"
tls_disable = true
}

tls is disabled because of the demo purposes in production you have to enable it for security purposes.

storage "raft"{
path = "./vault/data"
node-id=”node1”
}

Specifying storage type as a raft.

specifying path.

node-id=”node1”

Disable_mlock=true.

In production you don’t want to do it. It is system (block) which prevents memory being swaped to disk.

api_addr=”localhost:8200”

cluster_addr=”http://127.0.0.1:8200”

ui=true

For the above code

cd <prod dir>

mkdir -p vault/data

vault server -config=./vault-config.hcl

Now next step is to initialize the vault.

export VAULT_ADDR=127.0.0.1

Vault status

vault operator init -key-shares=1 -key-threshold=1

It will give you.

Unseal key 1:
Initial Root Token:

export VAULT_TOKEN=<root token>

Next is enabling audit logging

mkdir logs

vault audit enable file file_path=./logs/vault-audit.log

To take a look at the audit-file.log

tail -f ./logs/vault-audit.log | jq

Types of Auth methods

Methods for users

Userpass
GitHub
LDAP
JWT
OIDC
OKTA

Methods for Application

App role
AWS
Azure
Google Cloud
Ali cloud
Kuberenetes

Enabling Auth Methods

Most vault auth methods need to be explicitly enable.
This is done with the vault auth enable.
Each auth method has a default path.
Alternative path can bespecified to enablemultiple instances.

vault auth enable -path=aws-east aws

Custom path must be specified in CLI commands and API calls:

vault write aws-east/config/root

Policies overview

vault uses policies to :

Govern the behaviors of clients.
Instrument Role-Based Access Control(RBAC) by specifying access privileged principle.

Authorization to access secrets.

Denyed by default

empty policy= not authorized

Default Policy	Root policy
Default policy can’t be removed, but cab be modified.	Root policy can’t be removed nor modified
Attached to all tokens.	Any user associated with this policy becomes a root user (super user do anything)
Can be explicitly excluded at token creation time.e.g vault token create -no-default-policy	Best practice to revoke root tokens in production. vault token revoke “<token>“
Allow basic functionality such as letting the token look up data about itself and to use its cubbyhole data.	When vault server is first initialized, one root user is created to do the initial configuration and setup of the vault. After configure the initial root token should be revoked and more strictly controlled users and authentication should be used.

Tokens:

Tokens are the care methods for authentication within vault.
Tokens can be used directly or dynamically generated by auth methods.
Clients need valid token to interact with vault.

The Token Store:

Same as the token authentication backend.
Special auth back-end responsible for creating and storing token (can’t be disable)
Only auth method with no login capability.
Built-in and automatically available at /auth/token.
You can create tokens directly and bypass other auth methods.

Secrets Engines:

Secrets engines are the compoenet of the vault architecture that manages secrets.
They store, generate, or encrypt data.
They are enable at a path.
Secrets engines are flexible and pluggable.
Think of them in termsof their function.
Some just store and read data.
Other connect to external services and generate dynamic credential on demands, other provide encryption as a service.
TOTP generation.
Certificates.

Enable:

This enable a secrets engine at a given path. Each secrets engine is isolated to its path. By default, they are enable at their “type“ (e.g “aws“,enable at “aws“)

Disable:

This disables an existing secrets engine.When a secrets engine is disable, all of its secrets are revoked (if they support it) and all of the data stored for that engine in the physical storage layer is deleted.

Move:

This moves the path for an exiting secrets engine. This process revokes all secrets, since secret leases are tied to the path they were created at. The configuration data stored for the engine persists through the move.

Tune:

This tunes global configuration for the secrets engine such as the TTLs.

HashiCorp Vault

Table of contents