Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Verifier attributes persist across suites #241

Open
mattray opened this issue Aug 28, 2019 · 3 comments
Open

Verifier attributes persist across suites #241

mattray opened this issue Aug 28, 2019 · 3 comments
Labels
Type: Bug Doesn't work as expected.

Comments

@mattray
Copy link

mattray commented Aug 28, 2019

Version:

If you set a verifier attribute in a suite, it persists to the next one in the kitchen.yml. Conversely, if you put it last it doesn't persist. The linked repo has a bad.kitchen.yml that will reproduce the bug (the normal kitchen.yml has been rewritten to avoid this bug).

ChefDK 4.3.13
InSpec 4.10.4
Chef 15.2.20

BAD SUITES

suites:
  - name: noop
    provisioner:
      policyfile: policyfiles/noop.rb
    verifier:
      inspec_tests:
        - .
      attributes:
        file: /tmp/foo
  - name: audit
    provisioner:
      policyfile: policyfiles/audit.rb
    verifier:
      inspec_tests:
        - .
  - name: fc_cookbook
    provisioner:
      policyfile: policyfiles/fc_cookbook.rb
    verifier:
      inspec_tests:
        - .

BAD OUTPUT

$ kitchen verify
-----> Starting Kitchen (v2.2.5)
-----> Verifying <noop-centos-7>...
[2019-08-28T17:57:14+10:00] WARN: DEPRECATION: InSpec Attributes are being renamed to InSpec Inputs to avoid confusion with Chef Attributes. Use --input-file on the command line instead of --attrs.
       Loaded filecheck 
[2019-08-28T17:57:14+10:00] WARN: DEPRECATION: The 'default' option for attributes is being replaced by 'value' - please use it instead. attribute name: 'Inspec::Input' (used at /Users/mattray/ws/profiles/filecheck/controls/example.rb:5)

Profile: InSpec Profile (filecheck)
Version: 0.1.0
Target:  ssh://[email protected]:2222

  File /tmp/foo
     ✔  should not exist

Test Summary: 1 successful, 0 failures, 0 skipped
       Finished verifying <noop-centos-7> (0m0.38s).
-----> Verifying <audit-centos-7>...
[2019-08-28T17:57:14+10:00] WARN: DEPRECATION: InSpec Attributes are being renamed to InSpec Inputs to avoid confusion with Chef Attributes. Use --input-file on the command line instead of --attrs.
       Loaded filecheck 
[2019-08-28T17:57:14+10:00] WARN: DEPRECATION: The 'default' option for attributes is being replaced by 'value' - please use it instead. attribute name: 'Inspec::Input' (used at /Users/mattray/ws/profiles/filecheck/controls/example.rb:5)

Profile: InSpec Profile (filecheck)
Version: 0.1.0
Target:  ssh://[email protected]:2200

  File /tmp/foo
     ✔  should not exist

Test Summary: 1 successful, 0 failures, 0 skipped
       Finished verifying <audit-centos-7> (0m0.36s).
-----> Verifying <fc-cookbook-centos-7>...
[2019-08-28T17:57:15+10:00] WARN: DEPRECATION: InSpec Attributes are being renamed to InSpec Inputs to avoid confusion with Chef Attributes. Use --input-file on the command line instead of --attrs.
       Loaded filecheck 
[2019-08-28T17:57:15+10:00] WARN: DEPRECATION: The 'default' option for attributes is being replaced by 'value' - please use it instead. attribute name: 'Inspec::Input' (used at /Users/mattray/ws/profiles/filecheck/controls/example.rb:5)

Profile: InSpec Profile (filecheck)
Version: 0.1.0
Target:  ssh://[email protected]:2201

  File /tmp/foo
     ✔  should not exist

Test Summary: 1 successful, 0 failures, 0 skipped
       Finished verifying <fc-cookbook-centos-7> (0m0.41s).
-----> Kitchen is finished. (0m3.84s)

CORRECT OUTPUT

Using the kitchen.yml with the verifier attributes moved to the last suite works.

$ kitchen verify
-----> Starting Kitchen (v2.2.5)
-----> Verifying <audit-centos-7>...
[2019-08-28T17:58:21+10:00] WARN: DEPRECATION: InSpec Attributes are being renamed to InSpec Inputs to avoid confusion with Chef Attributes. Use --input-file on the command line instead of --attrs.
       Loaded filecheck 
[2019-08-28T17:58:21+10:00] WARN: DEPRECATION: The 'default' option for attributes is being replaced by 'value' - please use it instead. attribute name: 'Inspec::Input' (used at /Users/mattray/ws/profiles/filecheck/controls/example.rb:5)

Profile: InSpec Profile (filecheck)
Version: 0.1.0
Target:  ssh://[email protected]:2200

  File /etc/hosts
     ×  should not exist
     expected File /etc/hosts not to exist

Test Summary: 0 successful, 1 failure, 0 skipped
-----> Verifying <fc-cookbook-centos-7>...
[2019-08-28T17:58:21+10:00] WARN: DEPRECATION: InSpec Attributes are being renamed to InSpec Inputs to avoid confusion with Chef Attributes. Use --input-file on the command line instead of --attrs.
       Loaded filecheck 
[2019-08-28T17:58:21+10:00] WARN: DEPRECATION: The 'default' option for attributes is being replaced by 'value' - please use it instead. attribute name: 'Inspec::Input' (used at /Users/mattray/ws/profiles/filecheck/controls/example.rb:5)

Profile: InSpec Profile (filecheck)
Version: 0.1.0
Target:  ssh://[email protected]:2201

  File /etc/hosts
     ×  should not exist
     expected File /etc/hosts not to exist

Test Summary: 0 successful, 1 failure, 0 skipped
-----> Verifying <noop-centos-7>...
[2019-08-28T17:58:22+10:00] WARN: DEPRECATION: InSpec Attributes are being renamed to InSpec Inputs to avoid confusion with Chef Attributes. Use --input-file on the command line instead of --attrs.
       Loaded filecheck 
[2019-08-28T17:58:22+10:00] WARN: DEPRECATION: The 'default' option for attributes is being replaced by 'value' - please use it instead. attribute name: 'Inspec::Input' (used at /Users/mattray/ws/profiles/filecheck/controls/example.rb:5)

Profile: InSpec Profile (filecheck)
Version: 0.1.0
Target:  ssh://[email protected]:2222

  File /tmp/foo
     ✔  should not exist

Test Summary: 1 successful, 0 failures, 0 skipped
       Finished verifying <noop-centos-7> (0m0.36s).
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 2 actions failed.
>>>>>>     Verify failed on instance <audit-centos-7>.  Please see .kitchen/logs/audit-centos-7.log for more details
>>>>>>     Verify failed on instance <fc-cookbook-centos-7>.  Please see .kitchen/logs/fc-cookbook-centos-7.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration
@zenspider zenspider added the Type: Bug Doesn't work as expected. label Oct 14, 2019
@sean-nixon
Copy link

I've also noticed this behavior is still present when using the renamed "inputs". Setting an input on one suite will persist it to all subsequent suites.

# chef -v                   
Chef Workstation version: 20.8.125
Chef Infra Client version: 16.4.38
Chef InSpec version: 4.22.8
Chef CLI version: 3.0.24
Chef Habitat version: 1.6.56
Test Kitchen version: 2.6.0
Cookstyle version: 6.15.5

@clintoncwolfe
Copy link
Contributor

#258 addresses this

@Ambassadormathismathews
Copy link

Version:

If you set a verifier attribute in a suite, it persists to the next one in the kitchen.yml. Conversely, if you put it last it doesn't persist. The linked repo has a bad.kitchen.yml that will reproduce the bug (the normal kitchen.yml has been rewritten to avoid this bug).

ChefDK 4.3.13 InSpec 4.10.4 Chef 15.2.20

BAD SUITES

suites:
  - name: noop
    provisioner:
      policyfile: policyfiles/noop.rb
    verifier:
      inspec_tests:
        - .
      attributes:
        file: /tmp/foo
  - name: audit
    provisioner:
      policyfile: policyfiles/audit.rb
    verifier:
      inspec_tests:
        - .
  - name: fc_cookbook
    provisioner:
      policyfile: policyfiles/fc_cookbook.rb
    verifier:
      inspec_tests:
        - .

BAD OUTPUT

$ kitchen verify
-----> Starting Kitchen (v2.2.5)
-----> Verifying <noop-centos-7>...
[2019-08-28T17:57:14+10:00] WARN: DEPRECATION: InSpec Attributes are being renamed to InSpec Inputs to avoid confusion with Chef Attributes. Use --input-file on the command line instead of --attrs.
       Loaded filecheck 
[2019-08-28T17:57:14+10:00] WARN: DEPRECATION: The 'default' option for attributes is being replaced by 'value' - please use it instead. attribute name: 'Inspec::Input' (used at /Users/mattray/ws/profiles/filecheck/controls/example.rb:5)

Profile: InSpec Profile (filecheck)
Version: 0.1.0
Target:  ssh://[email protected]:2222

  File /tmp/foo
     ✔  should not exist

Test Summary: 1 successful, 0 failures, 0 skipped
       Finished verifying <noop-centos-7> (0m0.38s).
-----> Verifying <audit-centos-7>...
[2019-08-28T17:57:14+10:00] WARN: DEPRECATION: InSpec Attributes are being renamed to InSpec Inputs to avoid confusion with Chef Attributes. Use --input-file on the command line instead of --attrs.
       Loaded filecheck 
[2019-08-28T17:57:14+10:00] WARN: DEPRECATION: The 'default' option for attributes is being replaced by 'value' - please use it instead. attribute name: 'Inspec::Input' (used at /Users/mattray/ws/profiles/filecheck/controls/example.rb:5)

Profile: InSpec Profile (filecheck)
Version: 0.1.0
Target:  ssh://[email protected]:2200

  File /tmp/foo
     ✔  should not exist

Test Summary: 1 successful, 0 failures, 0 skipped
       Finished verifying <audit-centos-7> (0m0.36s).
-----> Verifying <fc-cookbook-centos-7>...
[2019-08-28T17:57:15+10:00] WARN: DEPRECATION: InSpec Attributes are being renamed to InSpec Inputs to avoid confusion with Chef Attributes. Use --input-file on the command line instead of --attrs.
       Loaded filecheck 
[2019-08-28T17:57:15+10:00] WARN: DEPRECATION: The 'default' option for attributes is being replaced by 'value' - please use it instead. attribute name: 'Inspec::Input' (used at /Users/mattray/ws/profiles/filecheck/controls/example.rb:5)

Profile: InSpec Profile (filecheck)
Version: 0.1.0
Target:  ssh://[email protected]:2201

  File /tmp/foo
     ✔  should not exist

Test Summary: 1 successful, 0 failures, 0 skipped
       Finished verifying <fc-cookbook-centos-7> (0m0.41s).
-----> Kitchen is finished. (0m3.84s)

CORRECT OUTPUT

Using the kitchen.yml with the verifier attributes moved to the last suite works.

$ kitchen verify
-----> Starting Kitchen (v2.2.5)
-----> Verifying <audit-centos-7>...
[2019-08-28T17:58:21+10:00] WARN: DEPRECATION: InSpec Attributes are being renamed to InSpec Inputs to avoid confusion with Chef Attributes. Use --input-file on the command line instead of --attrs.
       Loaded filecheck 
[2019-08-28T17:58:21+10:00] WARN: DEPRECATION: The 'default' option for attributes is being replaced by 'value' - please use it instead. attribute name: 'Inspec::Input' (used at /Users/mattray/ws/profiles/filecheck/controls/example.rb:5)

Profile: InSpec Profile (filecheck)
Version: 0.1.0
Target:  ssh://[email protected]:2200

  File /etc/hosts
     ×  should not exist
     expected File /etc/hosts not to exist

Test Summary: 0 successful, 1 failure, 0 skipped
-----> Verifying <fc-cookbook-centos-7>...
[2019-08-28T17:58:21+10:00] WARN: DEPRECATION: InSpec Attributes are being renamed to InSpec Inputs to avoid confusion with Chef Attributes. Use --input-file on the command line instead of --attrs.
       Loaded filecheck 
[2019-08-28T17:58:21+10:00] WARN: DEPRECATION: The 'default' option for attributes is being replaced by 'value' - please use it instead. attribute name: 'Inspec::Input' (used at /Users/mattray/ws/profiles/filecheck/controls/example.rb:5)

Profile: InSpec Profile (filecheck)
Version: 0.1.0
Target:  ssh://[email protected]:2201

  File /etc/hosts
     ×  should not exist
     expected File /etc/hosts not to exist

Test Summary: 0 successful, 1 failure, 0 skipped
-----> Verifying <noop-centos-7>...
[2019-08-28T17:58:22+10:00] WARN: DEPRECATION: InSpec Attributes are being renamed to InSpec Inputs to avoid confusion with Chef Attributes. Use --input-file on the command line instead of --attrs.
       Loaded filecheck 
[2019-08-28T17:58:22+10:00] WARN: DEPRECATION: The 'default' option for attributes is being replaced by 'value' - please use it instead. attribute name: 'Inspec::Input' (used at /Users/mattray/ws/profiles/filecheck/controls/example.rb:5)

Profile: InSpec Profile (filecheck)
Version: 0.1.0
Target:  ssh://[email protected]:2222

  File /tmp/foo
     ✔  should not exist

Test Summary: 1 successful, 0 failures, 0 skipped
       Finished verifying <noop-centos-7> (0m0.36s).
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 2 actions failed.
>>>>>>     Verify failed on instance <audit-centos-7>.  Please see .kitchen/logs/audit-centos-7.log for more details
>>>>>>     Verify failed on instance <fc-cookbook-centos-7>.  Please see .kitchen/logs/fc-cookbook-centos-7.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

gitlab-ctl kill

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type: Bug Doesn't work as expected.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@zenspider @clintoncwolfe @mattray @sean-nixon @Ambassadormathismathews