disable parallelism from our tests

[uturunku1]

Description

go-tfe tests are failing with errors such as:

FAIL TestPlansRead (13.29s)
helper_test.go:1205: Polling run "run-A1fXxv58khUzMKkN" for status included in ["cost_estimated" "errored"] with deadline of 2024-04-09 18:00:06.320719772 +0000 UTC m=+189.359325389
helper_test.go:1211: ...
helper_test.go:1269: Reading run "run-A1fXxv58khUzMKkN"
helper_test.go:1217: Run "run-A1fXxv58khUzMKkN" had status "planning"
helper_test.go:1211: ...
helper_test.go:1269: Reading run "run-A1fXxv58khUzMKkN"
helper_test.go:1217: Run "run-A1fXxv58khUzMKkN" had status "planning"
helper_test.go:1211: ...
helper_test.go:1269: Reading run "run-A1fXxv58khUzMKkN"
helper_test.go:1217: Run "run-A1fXxv58khUzMKkN" had status "errored"
helper_test.go:1290: Reading plan "plan-zap9jsnSo4ickPuV"
helper_test.go:1302: Reading logs of plan "plan-zap9jsnSo4ickPuV"
helper_test.go:1175: Run errored - here's some logs to help figure out what happened
helper_test.go:1175: ---Start of logs---
Terraform v1.7.5
on linux_amd64
Initializing plugins and modules...
{"@level":"info","@message":"Terraform 1.7.5","@module":"terraform.ui","@timestamp":"2024-04-09T17:58:10.165313Z","terraform":"1.7.5","type":"version","ui":"1.2"}
{"@level":"info","@message":"null_resource.foo: Plan to create","@module":"terraform.ui","@timestamp":"2024-04-09T17:58:10.875404Z","change":{"resource":{"addr":"null_resource.foo","module":"","resource":"null_resource.foo","implied_provider":"null","resource_type":"null_resource","resource_name":"foo","resource_key":null},"action":"create"},"type":"planned_change"}

        {"@level":"info","@message":"Plan: 1 to add, 0 to change, 0 to destroy.","@module":"terraform.ui","@timestamp":"2024-04-09T17:58:10.875592Z","changes":{"add":1,"change":0,"import":0,"remove":0,"operation":"plan"},"type":"change_summary"}
Failed generating provider schemas JSON
Exit code: 1
╷
│ Error: Failed to load plugin schemas

Because those same tests don't fail locally, only for the tflocal box, I approached the TDP team with the suspicion that the box configuration may need to be adjusted. Reason being that, to me, those errors seem to indicate that the agent cannot properly run terraform init (every time that it runs plan and apply) and it is timing out due to not having enough network capacity.

Well, turns out that the TDP team had recently approved a PR that actually, for the first time, forces integrations tests to use agents! In the past, the CI pipeline was running integration tests using an instance in terraform build worker, rather than using agent pools.

https://github.com/hashicorp/atlas/pull/19179
https://github.com/hashicorp/atlas/pull/19204

As a consequence of these changes, our go-tfe tests got thrown onto the agents by surprise. The tests failures that we are seeing are highlighting an underlying issue with the agent environment where you cannot run multiple concurrent runs because they will compete with each other and fail. All runs are executed from within a nomad container and they all touch the same directory path. Disabling parallelism for our test solves the issue. The test suite may be slower, however, now that we are using the same agent dispatch system that production uses, at least we can that the tests are more reliable.

Testing plan

External links

Output from tests

Including output from tests may require access to a TFE instance. Ignore this section if you have no environment to test against.

$ TFE_ADDRESS="https://example" TFE_TOKEN="example" go test ./... -v -run TestFunctionsAffectedByChange

...

[github-actions]

Reminder to the contributor that merged this PR: if your changes have added important functionality or fixed a relevant bug, open a follow-up PR to update CHANGELOG.md with a note on your changes.