Kubernetes Combine Example does not Work as Intended

[bijenkins]

The example in the kubernetes folder does not work on the combine example, there is NO matching goodbye service yet it passes the test that every deployment has a matching service.

https://github.com/open-policy-agent/conftest/tree/master/examples/kubernetes/combine

Specifically, there's a rule that shows if a deployment has no matching service with the same label, it should fail:

conftest/examples/kubernetes/combine/combine.rego

Line 8 in 56d742d

msg := sprintf("Deployment '%v' has no matching service", [deployment.metadata.name])

package main

violation[msg] {
    some i
    input[i].contents.kind == "Deployment"
    deployment := input[i].contents
    not service_selects_app(deployment.spec.selector.matchLabels.app)
    msg := sprintf("Deployment '%v' has no matching service", [deployment.metadata.name])
}

service_selects_app(app) {
    some i
    input[i].contents.kind == "Service"
    service := input[i].contents
    service.spec.selector.app == app
}

Yaml running against:
https://github.com/open-policy-agent/conftest/blob/56d742d241174bbc790d7b6e6cb3a8632ffff576/examples/kubernetes/combine/combine.yaml

Specifically to reproduce:

conftest test conftest/examples/kubernetes/combine/combine.yaml --policy conftest/examples/kubernetes/combine/


> conftest test combine/combine.yaml --policy combine -o json                                                                                                                                                       1 ↵ ──(Tue,Aug13)─┘
[
        {
                "filename": "combine/combine.yaml",
                "namespace": "main",
                "successes": 3
        }
]

This should not have passed the tests, really trying to grasp this exact scenario and the example given doesn't work. There is no goodbye service. It should fail.

[bijenkins]

Trace for exact run:

file: combine/combine.yaml | query: data.main.exception[_][_] == ""
TRAC   Enter __localq0__ = data.main.exception[_][_]; equal(__localq0__, "", _); _
TRAC   | Eval __localq0__ = data.main.exception[_][_]
TRAC   | Unify __localq0__ = data.main.exception[_][_]
TRAC   | Fail __localq0__ = data.main.exception[_][_]
file: combine/combine.yaml | query: data.main.violation
TRAC   Enter data.main.violation = _
TRAC   | Eval data.main.violation = _
TRAC   | Unify data.main.violation = _
TRAC   | Index data.main.violation (matched 1 rule)
TRAC   | Enter data.main.violation
TRAC   | | Eval input[i].contents.kind = "Deployment"
TRAC   | | Unify input[i].contents.kind = "Deployment"
TRAC   | | Unify "apiVersion" = i
TRAC   | | Unify "kind" = i
TRAC   | | Unify "metadata" = i
TRAC   | | Unify "spec" = i
TRAC   | | Fail input[i].contents.kind = "Deployment"
TRAC   | Unify set() = _
TRAC   | Exit data.main.violation = _
TRAC   Redo data.main.violation = _
TRAC   | Redo data.main.violation = _
file: combine/combine.yaml | query: data.main.exception[_][_] == ""
TRAC   Enter __localq0__ = data.main.exception[_][_]; equal(__localq0__, "", _); _
TRAC   | Eval __localq0__ = data.main.exception[_][_]
TRAC   | Unify __localq0__ = data.main.exception[_][_]
TRAC   | Fail __localq0__ = data.main.exception[_][_]
file: combine/combine.yaml | query: data.main.violation
TRAC   Enter data.main.violation = _
TRAC   | Eval data.main.violation = _
TRAC   | Unify data.main.violation = _
TRAC   | Index data.main.violation (matched 1 rule)
TRAC   | Enter data.main.violation
TRAC   | | Eval input[i].contents.kind = "Deployment"
TRAC   | | Unify input[i].contents.kind = "Deployment"
TRAC   | | Unify "apiVersion" = i
TRAC   | | Unify "kind" = i
TRAC   | | Unify "metadata" = i
TRAC   | | Unify "spec" = i
TRAC   | | Fail input[i].contents.kind = "Deployment"
TRAC   | Unify set() = _
TRAC   | Exit data.main.violation = _
TRAC   Redo data.main.violation = _
TRAC   | Redo data.main.violation = _
file: combine/combine.yaml | query: data.main.exception[_][_] == ""
TRAC   Enter __localq0__ = data.main.exception[_][_]; equal(__localq0__, "", _); _
TRAC   | Eval __localq0__ = data.main.exception[_][_]
TRAC   | Unify __localq0__ = data.main.exception[_][_]
TRAC   | Fail __localq0__ = data.main.exception[_][_]
file: combine/combine.yaml | query: data.main.violation
TRAC   Enter data.main.violation = _
TRAC   | Eval data.main.violation = _
TRAC   | Unify data.main.violation = _
TRAC   | Index data.main.violation (matched 1 rule)
TRAC   | Enter data.main.violation
TRAC   | | Eval input[i].contents.kind = "Deployment"
TRAC   | | Unify input[i].contents.kind = "Deployment"
TRAC   | | Unify "apiVersion" = i
TRAC   | | Unify "kind" = i
TRAC   | | Unify "metadata" = i
TRAC   | | Unify "spec" = i
TRAC   | | Fail input[i].contents.kind = "Deployment"
TRAC   | Unify set() = _
TRAC   | Exit data.main.violation = _
TRAC   Redo data.main.violation = _
TRAC   | Redo data.main.violation = _

[thevilledev]

Pass the --combine parameter to evaluate all objects at the same time.

$ conftest test examples/kubernetes/combine/combine.yaml \
--policy examples/kubernetes/combine/ \
--combine
FAIL - Combined - main - Deployment 'goodbye-kubernetes' has no matching service

1 test, 0 passed, 0 warnings, 1 failure, 0 exceptions