Playing with conftest and yum repository policies

A project to modernise an old Terraform code base came across my desk recently and while investigating the more recent developments in testing tools and workflows I stumbled onto conftest, a utility to help you write tests against structured configuration data. I was interested in trying the technology out but I don’t want to put something i have this little experience in on the main flow of work so I decided to do a few tests with it against a smaller, more self contained, use case.

My home lab runs a lot of CentOS and Fedora so when I found an old yum repository client configuration with gpgcheck disabled the streams aligned and I had my small problem to solve. I added the card to my list and in an impossible amount of hubris marked it as small. I’d read a few blog posts by this point and thought, “I can do that.” But I couldn’t.

I threw hours into getting the policies to fail in the correct way. Now I look back at the six lines, one of which is white space, I can’t help but ask “WTF was I doing to take that long?”

violation[msg] {
    configuration_block := input[configuration_block_name]

    not configuration_block.gpgcheck = 1
    msg = sprintf("gpgcheck should be enabled in %s", [configuration_block_name])

Every character of that policy took minutes off my life. I don’t want to consider how long I invested putting the sample configuration file through the same few commands. It started out promisingly, using parse I could see how my sample data was being structured.

$ conftest parse -i ini basic-single-repo.repo 
        "code": {
                "baseurl": "",
                "enabled": 1,
                "gpgcheck": 1,
                "gpgkey": "",
                "name": "Visual Studio Code"

Then I tried to write the policy. It honestly felt like trying to fix an engine through the exhaust pipe. Action at a distance didn’t quite match the level of frustration. I started scattering sprintf throughout the code and forced it to raise failure just to check a value. Then I started running under --trace to see more of what the internals were doing. This was oddly calming to watch, even though nothing was actually working, and eventually I stumbled on to the trace() function and trace(sprintf("%v", [input])) . Finally a way to output things. But only in the verbose --trace mode. Things did get easier once I’d found trace and searching through GitHub for other peoples examples and reading them helped me get a better perspective on what I was trying to do.

I’ve now managed to reach the first real milestone since I started playing with conftest. I have working Open policy agent policies that cover the original insecure yum client configuration and I can run them against arbitrary files using the code directly from GitHub.

conftest test -i ini --update basic-single-repo-broken.repo

FAIL - basic-single-repo-broken.repo - gpgcheck should be enabled in code
FAIL - basic-single-repo-broken.repo - baseurl in code should use https URLs
FAIL - basic-single-repo-broken.repo - gpgkey in code should use https URLs
FAIL - basic-single-repo-broken.repo - metalink in code should use https URLs
FAIL - basic-single-repo-broken.repo - mirrorlist in code should use https URLs

5 tests, 0 passed, 0 warnings, 5 failures

I’m still at the beginner stage of learning OPA / rego / conftest and I don’t feel comfortable suggesting the tooling for the Terraform project just yet but I can see the potential it offers. I intend to try a few more small policy cases to build up some more experience with the language, and hopefully get unit testing working (that’s still too raw to talk about in this post), but I’m glad I spent some time actually trying to get the code to work. I need to think some more about when I’d use Augeas and RSpec vs conftest and Rego but it’s always nice to have flexible options.

As an aside trying out rego with a project that has repo in the file names was a massive mistake. Typos? Oh yes. Auto complete not getting very far due to the shared root? Very much so.