scaleway/gonsul

scaleway/gonsul

Description: A stand-alone alternative to git2consul

Language: Go

License: NOASSERTION

Stars: 1

Forks: 1

Open issues: 4

Created: 2025-04-15T14:23:21Z

Pushed: 2026-02-02T07:44:34Z

Default branch: main

Fork: yes

Parent repository: miniclip/gonsul

Archived: no

README:

Gonsul - A Git to Consul tool, in Go

This is an active fork of [miniclip/gonsul](https://github.com/miniclip/gonsul) than includes [jplanckeel/gonsul](https://github.com/jplanckeel/gonsul) patch for mTLS.

This tool serves as an entry point for the Hashicorp's Consul KV store. Not only because Consul lacks of a built in audit mechanism, but also because having configurations managed in GIT, using a gitflow or a normal development-to-master flow is much friendly and familiar to any development team to manage configurations.

Downloads in releases page.

Important Notes

The main branch is being refactored. No (big) breaking changes will happen but codebase is under significant changes. Mainly adding tests, makefile, and code refactor/cleanup.

Please use the releases/downloads page. Instructions for development will be added soon!

How It Processes a Repository

Gonsul will (optionally) clone your repository into the filesystem. After, it will recursively parse all the files in the directory. Whenever Gonsul moves one level deep into a folder, the folder name is added as a Consul KV path part and as soon as it finds a file (either .json, .txt or .ini - or any other given in parameters) it will take the file name (without the extension) and append to the Consul KV path, making it a key and the file content is added as the value.

Example: Take this repository folder structure:

+-- prod
| |
| +-- website1
| | |-- config.json
| | |-- db-pass.txt
| |
| +-- app2
| |-- config.json
|
+-- dev
|
+-- website1
| |-- config.json
|
+-- app2
|-- config.json

All the files content would be inserted, respectively, into the following Consul KV paths:

• prod/website1/config
• prod/website1/db-pass
• prod/app2/config
• dev/website1/config
• dev/app2/config

Note 1: Gonsul makes the assumption that only .txt, .json and .ini files are to be treated as keys and their content as values for Consul. Any other files are disregarded. Also, .json files are treated and validated as valid JSON, and inserted into Consul pretty formatted.

Note 2: You can instruct Gonsul to expand the JSON files, so it will parse the structure, appending the JSON path to the previous folder structure, creating single Consul KV entries for each value. Caveats: Any arrays found are inserted into Consul a bracketed comma separated value string. More details on this on the flags description.

Some Features

Gonsul was built out of necessity and so it provides some features that makes our team more comfortable regarding security and consistency, so below are the most important features (TL;DR's in bold):

• The most obvious feature is it's built in GO, so we have **one small binary

tool that just runs** in any bamboo, jenkins, *nix or OSX computer (does not support windows yet).

• Gonsul uses Consul KV transactions, so any operations (inserts, updates

and deletes) will be atomically done. Due to Consul limitations, a maximum of 64 operations can be done on one transaction. If there are more operations, multiple batches are created and multiple calls made. Each transaction is atomic on Consul side, and if a transaction fails, Gonsul will terminate. On most configurations changes on a normal workday, all changes are made using one atomic transaction. Either all configs go, or none, reducing inconsistent states.

• Gonsul can **operate on a *less destructive* manner**. As we (@Miniclip) have

multiple teams using the KV store without having the KV paths namespaced, we strongly rely on Consul ACL's. So to avoid a mass delete in case of an ACL mistake we would like to have Gonsul to operate on a mode where it does not perform any deletes. In such mode, when Gonsul detectes *delete* operation, it will terminate with a specific error number, printing a list of KV paths that it would delete, allowing a team member to decide whether to manually run the operation or investigate any ACL problem before disaster.

• Gonsul handles secrets substitution using

mustache.go avoiding the need of extra tooling for this small task. In our case, having all our configurations in GIT is a really important way to keep a familiar work flow, but having secrets out of GIT is crucial in any corporate environment. So just use some placeholders in your GIT files, and tell Gonsul where it can find a JSON key-value file of the placeholder-secrets map. (More about this below)

• Finally Gonsul does not rely on any OS external tool (such as GIT) as it

does not shell out any operations. All tasks are natively handled by Gonsul, as long as filesystem and network permission/access exists.

Available Flags

Below are all available command line flags for starting Gonsul. Flags may be specified on the command line, using environment variables (flag prefixed with "GONSUL_", converted to upper case, and "-" changed to "_", e.g. GONSUL_REPO_URL), or using a configuration file (--config=). The order of precedence for specifying flags is:

1. Command line options 2. Environment variables 3. Configuration file 4. Default values

--config=
--strategy=
--repo-url=
--repo-ssh-key=
--repo-ssh-user=
--repo-branch=
--repo-remote-name=
--repo-base-path=
--repo-root=
--consul-url=
--consul-acl=
--consul-base-path=
--ca-file=
--key-file=
--cert-file=
--log-level=
--expand-json=
--secrets-file=
--allow-deletes=
--poll-interval=
--input-ext=
--keep-ext=

Below is the full description for each individual command line flag.

--config

> require: no > example: `--config=gonsul.conf`

This specifies a file with configuration settings for the options described below. File syntax:

• Empty lines and lines beginning with a "#" character are ignored.
• Flags and values can be separated with whitespace or the "=" character

(strategy ONCE or strategy=DRYRUN).

• Booleans can be empty (true) or set with "1/0", "t/f", "T/F", "true/false",

"TRUE/FALSE", or "True/False".

--strategy

> require: yes > example: `--strategy=ONCE`

This defines the mode that Gonsul should run at,...