Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update configs to use new TME tenant #37667

Merged
merged 1 commit into from
Oct 7, 2024
Merged

Update configs to use new TME tenant #37667

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 16 additions & 42 deletions eng/pipelines/templates/jobs/live.tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ parameters:
default: ''
- name: TestResourceDirectories
type: object
default:
default: null
- name: PreSteps
type: object
default: []
Expand Down Expand Up @@ -124,35 +124,20 @@ jobs:
Pool: $(Pool)
${{ insert }}: ${{ parameters.EnvVars }}

- ${{ if parameters.TestResourceDirectories }}:
- ${{ each directory in parameters.TestResourceDirectories }}:
- template: /eng/common/TestResources/deploy-test-resources.yml
parameters:
Location: ${{ coalesce(parameters.Location, parameters.CloudConfig.Location) }}
ServiceDirectory: '${{ directory }}'
SubscriptionConfiguration: $(SubscriptionConfiguration)
ArmTemplateParameters: $(ArmTemplateParameters)
UseFederatedAuth: ${{ parameters.UseFederatedAuth }}
ServiceConnection: ${{ parameters.CloudConfig.ServiceConnection }}
SubscriptionConfigurationFilePaths: ${{ parameters.CloudConfig.SubscriptionConfigurationFilePaths }}
EnvVars:
Pool: $(Pool)
${{ insert }}: ${{ parameters.EnvVars }}
- ${{ if not(parameters.TestResourceDirectories) }}:
- template: /eng/common/TestResources/deploy-test-resources.yml
parameters:
Location: ${{ coalesce(parameters.Location, parameters.CloudConfig.Location) }}
ServiceDirectory: '${{ parameters.ServiceDirectory }}'
SubscriptionConfiguration: $(SubscriptionConfiguration)
ArmTemplateParameters: $(ArmTemplateParameters)
UseFederatedAuth: ${{ parameters.UseFederatedAuth }}
ServiceConnection: ${{ parameters.CloudConfig.ServiceConnection }}
SubscriptionConfigurationFilePaths: ${{ parameters.CloudConfig.SubscriptionConfigurationFilePaths }}
EnvVars:
Pool: $(Pool)
${{ insert }}: ${{ parameters.EnvVars }}
- ${{ each directory in coalesce(parameters.TestResourceDirectories, split(parameters.ServiceDirectory, '|')) }}:
weshaggard marked this conversation as resolved.
Show resolved Hide resolved
- template: /eng/common/TestResources/deploy-test-resources.yml
parameters:
Location: ${{ coalesce(parameters.Location, parameters.CloudConfig.Location) }}
ServiceDirectory: '${{ directory }}'
SubscriptionConfiguration: $(SubscriptionConfiguration)
ArmTemplateParameters: $(ArmTemplateParameters)
UseFederatedAuth: ${{ parameters.UseFederatedAuth }}
ServiceConnection: ${{ parameters.CloudConfig.ServiceConnection }}
EnvVars:
Pool: $(Pool)
${{ insert }}: ${{ parameters.EnvVars }}

- template: ../steps/build-test.yml
- template: /eng/pipelines/templates/steps/build-test.yml
parameters:
ServiceDirectory: ${{ parameters.ServiceDirectory }}
CloudName: ${{ parameters.CloudConfig.Cloud }}
Expand All @@ -170,24 +155,13 @@ jobs:
TestProxy: ${{ parameters.TestProxy }}
UseFederatedAuth: ${{ parameters.UseFederatedAuth }}
ServiceConnection: ${{ parameters.CloudConfig.ServiceConnection }}
SubscriptionConfigurationFilePaths: ${{ parameters.CloudConfig.SubscriptionConfigurationFilePaths }}

- ${{ if parameters.TestResourceDirectories }}:
- ${{ each directory in parameters.TestResourceDirectories }}:
- template: /eng/common/TestResources/remove-test-resources.yml
parameters:
ServiceDirectory: '${{ directory }}'
SubscriptionConfiguration: $(SubscriptionConfiguration)
UseFederatedAuth: ${{ parameters.UseFederatedAuth }}
ServiceConnection: ${{ parameters.CloudConfig.ServiceConnection }}
SubscriptionConfigurationFilePaths: ${{ parameters.CloudConfig.SubscriptionConfigurationFilePaths }}
- ${{ if not(parameters.TestResourceDirectories) }}:
- ${{ each directory in coalesce(parameters.TestResourceDirectories, split(parameters.ServiceDirectory, '|')) }}:
- template: /eng/common/TestResources/remove-test-resources.yml
parameters:
ServiceDirectory: '${{ parameters.ServiceDirectory }}'
ServiceDirectory: '${{ directory }}'
SubscriptionConfiguration: $(SubscriptionConfiguration)
UseFederatedAuth: ${{ parameters.UseFederatedAuth }}
ServiceConnection: ${{ parameters.CloudConfig.ServiceConnection }}
SubscriptionConfigurationFilePaths: ${{ parameters.CloudConfig.SubscriptionConfigurationFilePaths }}

- ${{ parameters.PostSteps }}
193 changes: 61 additions & 132 deletions eng/pipelines/templates/stages/archetype-sdk-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,26 +57,12 @@ parameters:
type: object
default:
Public:
ServiceConnection: azure-sdk-tests
SubscriptionConfigurationFilePaths:
- eng/common/TestResources/sub-config/AzurePublicMsft.json
Preview:
ServiceConnection: azure-sdk-tests-preview
SubscriptionConfigurationFilePaths:
- eng/common/TestResources/sub-config/AzurePreviewMsft.json
Canary:
ServiceConnection: azure-sdk-tests
SubscriptionConfigurationFilePaths:
- eng/common/TestResources/sub-config/AzurePublicMsft.json
Location: 'centraluseuap'
UsGov:
ServiceConnection: usgov_azure-sdk-tests
SubscriptionConfigurationFilePaths:
- eng/common/TestResources/sub-config/AzureUsGovMsft.json
China:
ServiceConnection: china_azure-sdk-tests
SubscriptionConfigurationFilePaths:
- eng/common/TestResources/sub-config/AzureChinaMsft.json
Location: 'chinanorth3'
scbedd marked this conversation as resolved.
Show resolved Hide resolved
- name: MatrixConfigs
type: object
default:
Expand Down Expand Up @@ -107,131 +93,74 @@ parameters:
default: 'whl'
- name: Packages
type: object
default: []
default: null
- name: UseFederatedAuth
type: boolean
default: true

extends:
template: /eng/pipelines/templates/stages/1es-redirect.yml
parameters:
Use1ESOfficial: false
stages:
- ${{ if ne(length(parameters.Packages), 0) }}:
- ${{ each package in parameters.Packages }}:
- ${{ each cloud in parameters.CloudConfig }}:
- ${{ if or(contains(parameters.Clouds, cloud.key), and(contains(variables['Build.DefinitionName'], 'tests-weekly'), contains(parameters.SupportedClouds, cloud.key))) }}:
- ${{ if not(contains(parameters.UnsupportedClouds, cloud.key)) }}:
- stage: ${{ cloud.key }}_${{ parameters.JobName }}_${{ replace(package, '-', '_') }}
dependsOn: []
jobs:
- template: /eng/common/pipelines/templates/jobs/generate-job-matrix.yml
parameters:
SparseCheckoutPaths:
- sdk/${{ parameters.ServiceDirectory }}/**/*.json
JobTemplatePath: /eng/pipelines/templates/jobs/live.tests.yml
OsVmImage: azsdk-pool-mms-ubuntu-2004-1espt
Pool: azsdk-pool-mms-ubuntu-2004-general
AdditionalParameters:
ServiceDirectory: ${{ parameters.ServiceDirectory }}
TestResourceDirectories: ${{ parameters.TestResourceDirectories }}
PreSteps:
- ${{ parameters.PlatformPreSteps }}
- ${{ parameters.PreSteps }}
PostSteps:
- ${{ parameters.PlatformPostSteps }}
- ${{ parameters.PostSteps }}
EnvVars: ${{ parameters.EnvVars }}
MaxParallel: ${{ parameters.MaxParallel }}
BeforeTestSteps: ${{ parameters.BeforeTestSteps }}
AfterTestSteps: ${{ parameters.AfterTestSteps }}
AdditionalTestArgs: ${{ parameters.AdditionalTestArgs }}
BuildTargetingString: ${{ package }}
TestMarkArgument: ${{ parameters.TestMarkArgument }}
InjectedPackages: ${{ parameters.InjectedPackages }}
BuildDocs: ${{ parameters.BuildDocs }}
TestTimeoutInMinutes: ${{ parameters.TestTimeoutInMinutes }}
TestProxy: ${{ parameters.TestProxy }}
ToxTestEnv: ${{ parameters.ToxTestEnv }}
UseFederatedAuth: ${{ parameters.UseFederatedAuth }}
MatrixConfigs:
# Enumerate platforms and additional platforms based on supported clouds (sparse platform<-->cloud matrix).
- ${{ each config in parameters.MatrixConfigs }}:
- ${{ config }}
- ${{ each config in parameters.AdditionalMatrixConfigs }}:
- ${{ config }}
MatrixFilters:
- ${{ each cloudFilter in cloud.value.MatrixFilters }}:
- ${{ cloudFilter }}
- ${{ parameters.MatrixFilters }}
MatrixReplace:
- ${{ each cloudReplace in cloud.value.MatrixReplace }}:
- ${{ cloudReplace }}
- ${{ parameters.MatrixReplace }}
CloudConfig:
SubscriptionConfiguration: ${{ cloud.value.SubscriptionConfiguration }}
SubscriptionConfigurations: ${{ cloud.value.SubscriptionConfigurations }}
Location: ${{ coalesce(parameters.Location, cloud.value.Location) }}
Cloud: ${{ cloud.key }}
ServiceConnection: ${{ cloud.value.ServiceConnection }}
SubscriptionConfigurationFilePaths: ${{ cloud.value.SubscriptionConfigurationFilePaths }}
- ${{ else }}:
- ${{ each cloud in parameters.CloudConfig }}:
- ${{ if or(contains(parameters.Clouds, cloud.key), and(contains(variables['Build.DefinitionName'], 'tests-weekly'), contains(parameters.SupportedClouds, cloud.key))) }}:
- ${{ if not(contains(parameters.UnsupportedClouds, cloud.key)) }}:
- stage: ${{ cloud.key }}_${{ parameters.JobName }}
dependsOn: []
jobs:
- template: /eng/common/pipelines/templates/jobs/generate-job-matrix.yml
parameters:
SparseCheckoutPaths:
- sdk/${{ parameters.ServiceDirectory }}/**/*.json
JobTemplatePath: /eng/pipelines/templates/jobs/live.tests.yml
OsVmImage: azsdk-pool-mms-ubuntu-2004-1espt
Pool: azsdk-pool-mms-ubuntu-2004-general
AdditionalParameters:
ServiceDirectory: ${{ parameters.ServiceDirectory }}
TestResourceDirectories: ${{ parameters.TestResourceDirectories }}
PreSteps:
- ${{ parameters.PlatformPreSteps }}
- ${{ parameters.PreSteps }}
PostSteps:
- ${{ parameters.PlatformPostSteps }}
- ${{ parameters.PostSteps }}
EnvVars: ${{ parameters.EnvVars }}
MaxParallel: ${{ parameters.MaxParallel }}
BeforeTestSteps: ${{ parameters.BeforeTestSteps }}
AfterTestSteps: ${{ parameters.AfterTestSteps }}
AdditionalTestArgs: ${{ parameters.AdditionalTestArgs }}
BuildTargetingString: ${{ parameters.BuildTargetingString }}
TestMarkArgument: ${{ parameters.TestMarkArgument }}
InjectedPackages: ${{ parameters.InjectedPackages }}
BuildDocs: ${{ parameters.BuildDocs }}
TestTimeoutInMinutes: ${{ parameters.TestTimeoutInMinutes }}
TestProxy: ${{ parameters.TestProxy }}
ToxTestEnv: ${{ parameters.ToxTestEnv }}
UseFederatedAuth: ${{ parameters.UseFederatedAuth }}
MatrixConfigs:
# Enumerate platforms and additional platforms based on supported clouds (sparse platform<-->cloud matrix).
- ${{ each config in parameters.MatrixConfigs }}:
- ${{ config }}
- ${{ each config in parameters.AdditionalMatrixConfigs }}:
- ${{ config }}
MatrixFilters:
- ${{ each cloudFilter in cloud.value.MatrixFilters }}:
- ${{ cloudFilter }}
- ${{ parameters.MatrixFilters }}
MatrixReplace:
- ${{ each cloudReplace in cloud.value.MatrixReplace }}:
- ${{ cloudReplace }}
- ${{ parameters.MatrixReplace }}
CloudConfig:
SubscriptionConfiguration: ${{ cloud.value.SubscriptionConfiguration }}
SubscriptionConfigurations: ${{ cloud.value.SubscriptionConfigurations }}
Location: ${{ coalesce(parameters.Location, cloud.value.Location) }}
Cloud: ${{ cloud.key }}
ServiceConnection: ${{ cloud.value.ServiceConnection }}
SubscriptionConfigurationFilePaths: ${{ cloud.value.SubscriptionConfigurationFilePaths }}
- ${{ each package in coalesce(parameters.Packages, split(parameters.BuildTargetingString, '|')) }}:
- ${{ each cloud in parameters.CloudConfig }}:
- ${{ if or(contains(parameters.Clouds, cloud.key), and(contains(variables['Build.DefinitionName'], 'tests-weekly'), contains(parameters.SupportedClouds, cloud.key))) }}:
- ${{ if not(contains(parameters.UnsupportedClouds, cloud.key)) }}:
- stage:
displayName: ${{ format('{0} {1} {2}', cloud.key, parameters.JobName, package) }}
dependsOn: []
jobs:
- template: /eng/common/pipelines/templates/jobs/generate-job-matrix.yml
parameters:
SparseCheckoutPaths:
- sdk/${{ parameters.ServiceDirectory }}/**/*.json
JobTemplatePath: /eng/pipelines/templates/jobs/live.tests.yml
OsVmImage: azsdk-pool-mms-ubuntu-2004-1espt
Pool: azsdk-pool-mms-ubuntu-2004-general
AdditionalParameters:
ServiceDirectory: ${{ parameters.ServiceDirectory }}
TestResourceDirectories: ${{ parameters.TestResourceDirectories }}
PreSteps:
- ${{ parameters.PlatformPreSteps }}
- ${{ parameters.PreSteps }}
PostSteps:
- ${{ parameters.PlatformPostSteps }}
- ${{ parameters.PostSteps }}
EnvVars: ${{ parameters.EnvVars }}
MaxParallel: ${{ parameters.MaxParallel }}
BeforeTestSteps: ${{ parameters.BeforeTestSteps }}
AfterTestSteps: ${{ parameters.AfterTestSteps }}
AdditionalTestArgs: ${{ parameters.AdditionalTestArgs }}
BuildTargetingString: ${{ package }}
TestMarkArgument: ${{ parameters.TestMarkArgument }}
InjectedPackages: ${{ parameters.InjectedPackages }}
BuildDocs: ${{ parameters.BuildDocs }}
TestTimeoutInMinutes: ${{ parameters.TestTimeoutInMinutes }}
TestProxy: ${{ parameters.TestProxy }}
ToxTestEnv: ${{ parameters.ToxTestEnv }}
UseFederatedAuth: ${{ parameters.UseFederatedAuth }}
MatrixConfigs:
# Enumerate platforms and additional platforms based on supported clouds (sparse platform<-->cloud matrix).
- ${{ each config in parameters.MatrixConfigs }}:
- ${{ config }}
- ${{ each config in parameters.AdditionalMatrixConfigs }}:
- ${{ config }}
MatrixFilters:
- ${{ each cloudFilter in cloud.value.MatrixFilters }}:
- ${{ cloudFilter }}
- ${{ parameters.MatrixFilters }}
MatrixReplace:
- ${{ each cloudReplace in cloud.value.MatrixReplace }}:
- ${{ cloudReplace }}
- ${{ parameters.MatrixReplace }}
CloudConfig:
SubscriptionConfiguration: ${{ cloud.value.SubscriptionConfiguration }}
SubscriptionConfigurations: ${{ cloud.value.SubscriptionConfigurations }}
Location: ${{ coalesce(cloud.value.Location, parameters.Location) }}
weshaggard marked this conversation as resolved.
Show resolved Hide resolved
Cloud: ${{ cloud.key }}
ServiceConnection: ${{ coalesce(cloud.value.ServiceConnection, lower(format('azure-sdk-tests-{0}', cloud.key))) }}
weshaggard marked this conversation as resolved.
Show resolved Hide resolved
SubscriptionConfigurationFilePaths: ${{ cloud.value.SubscriptionConfigurationFilePaths }}

- template: /eng/pipelines/templates/stages/python-analyze-weekly.yml
parameters:
BuildTargetingString: ${{ parameters.BuildTargetingString }}
Expand Down
1 change: 1 addition & 0 deletions sdk/agrifood/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ extends:
Clouds: Public_FarmBeats
CloudConfig:
Public_Farmbeats:
ServiceConnection: azure-sdk-tests-public #Might need a specific service connection if this is a unique sub.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weshaggard I checked and they are using a custom sub (5bae917c-5fc9-40c5-923d-adbd52759048). I don't have access though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think they care much about this pipeline currently as the existing client secret is also not working. So, either way they will need to spend time to fix this when they are interested.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weshaggard what is the guidance for services that use a custom sub? Our openai tests use static resources from a non-SDK team owned sub.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no general guidance. We need to look into them case by case.

SubscriptionConfiguration: $(sub-config-farmbeats-azure-cloud-test-resources)
EnvVars:
AZURE_CLIENT_ID: $(AGRIFOOD_CLIENT_ID)
Expand Down
12 changes: 1 addition & 11 deletions sdk/appconfiguration/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,26 +8,16 @@ parameters:
# - azure-appconfiguration-provider

extends:
template: ../../eng/pipelines/templates/stages/archetype-sdk-tests.yml
template: /eng/pipelines/templates/stages/archetype-sdk-tests.yml
parameters:
ServiceDirectory: appconfiguration
Packages: ${{ parameters.Services }}
SupportedClouds: 'Public,UsGov,China'
TestTimeoutInMinutes: 240
UseFederatedAuth: true
CloudConfig:
Public:
ServiceConnection: azure-sdk-tests
SubscriptionConfigurationFilePaths:
- eng/common/TestResources/sub-config/AzurePublicMsft.json
UsGov:
ServiceConnection: usgov_azure-sdk-tests
SubscriptionConfigurationFilePaths:
- eng/common/TestResources/sub-config/AzureUsGovMsft.json
China:
ServiceConnection: china_azure-sdk-tests
SubscriptionConfigurationFilePaths:
- eng/common/TestResources/sub-config/AzureChinaMsft.json
Location: chinanorth3
MatrixReplace:
- TestSamples=.*/true
Expand Down
Loading
Loading