Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Colab runtime template #12706

Open
wants to merge 33 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
a966f6c
update documentation for instance_owners field
bcreddy-gcp Oct 31, 2024
71e2553
update documentation for instance_owners field
bcreddy-gcp Oct 31, 2024
6a7ddbb
lint fix
bcreddy-gcp Oct 31, 2024
85b523e
Merge branch 'GoogleCloudPlatform:main' into main
bcreddy-gcp Nov 6, 2024
b4f50e3
Merge branch 'GoogleCloudPlatform:main' into main
bcreddy-gcp Dec 5, 2024
268e2ae
gcs-data-bucket is now modifiable by the customer
bcreddy-gcp Dec 5, 2024
c9e07e6
Add suport for Third Party Identity
bcreddy-gcp Dec 18, 2024
b5377ef
Merge branch 'GoogleCloudPlatform:main' into main
bcreddy-gcp Dec 18, 2024
609f8ad
Merge branch 'GoogleCloudPlatform:main' into main
bcreddy-gcp Dec 23, 2024
cbd8a74
Merge remote-tracking branch 'upstream/main'
bcreddy-gcp Jan 6, 2025
92170ce
Colab runtime template draft
bcreddy-gcp Jan 6, 2025
0b90b9b
Ignore changes to other resources
bcreddy-gcp Jan 6, 2025
6893114
Add empty space
bcreddy-gcp Jan 6, 2025
e9845f3
Add examples for templates
bcreddy-gcp Jan 8, 2025
883aebf
empty lines
bcreddy-gcp Jan 8, 2025
c0d5a77
Merge branch 'GoogleCloudPlatform:main' into colab_template_changes
bcreddy-gcp Jan 8, 2025
c195e5c
empty lines
bcreddy-gcp Jan 8, 2025
808c8a3
lint fixes
bcreddy-gcp Jan 8, 2025
aa2f3c6
formatting
bcreddy-gcp Jan 8, 2025
cb60cca
undo wrong changes
bcreddy-gcp Jan 8, 2025
206748b
fix colab test
bcreddy-gcp Jan 8, 2025
01d38f3
fix teamcity tests
bcreddy-gcp Jan 8, 2025
28d57e1
keep TF consistent with API
bcreddy-gcp Jan 8, 2025
78bfb4b
Move name to parameters
bcreddy-gcp Jan 9, 2025
1eb0c28
remove unnecessary variables
bcreddy-gcp Jan 9, 2025
beb9849
Another unused var
bcreddy-gcp Jan 9, 2025
90a9a24
Add no name test
bcreddy-gcp Jan 9, 2025
2f4a67c
Added default_from_api to necessary fields
bcreddy-gcp Jan 9, 2025
7b919db
Add IAM
bcreddy-gcp Jan 10, 2025
d582945
Add post_create
bcreddy-gcp Jan 10, 2025
464e47b
use access policy
bcreddy-gcp Jan 11, 2025
8053cbd
Add a post create
bcreddy-gcp Jan 11, 2025
871e4d6
empty lines
bcreddy-gcp Jan 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 179 additions & 0 deletions mmv1/products/colab/RuntimeTemplate.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
# Copyright 2025 Google Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

---

name: 'RuntimeTemplate'
description: |
'A runtime template is a VM configuration that specifies a machine type and other characteristics of the VM,
as well as common settings such as the network and whether public internet access is enabled. When you create
a runtime, its VM is created according to the specifications of a runtime template.'

references:
guides:
'Create a runtime template': 'https://cloud.google.com/colab/docs/create-runtime-template'
api: 'https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.notebookRuntimeTemplates'
base_url: 'projects/{{project}}/locations/{{location}}/notebookRuntimeTemplates'
self_link: 'projects/{{project}}/locations/{{location}}/notebookRuntimeTemplates/{{name}}'
immutable: true
create_url: 'projects/{{project}}/locations/{{location}}/notebookRuntimeTemplates?notebook_runtime_template_id={{name}}'
autogen_async: true
async:
operation:
base_url: '{{op_id}}'
iam_policy:
parent_resource_attribute: 'runtime_template'
method_name_separator: ':'
fetch_iam_policy_verb: 'POST'
import_format:
- 'projects/{{project}}/locations/{{location}}/notebookRuntimeTemplates/{{runtime_template}}'
- '{{runtime_template}}'
custom_code:
post_create: 'templates/terraform/post_create/colab_runtime_template.tmpl'
examples:
- name: 'colab_runtime_template_basic'
primary_resource_id: 'runtime-template'
primary_resource_name: 'fmt.Sprintf("tf-test-colab-runtime-template%s", context["random_suffix"])'
vars:
runtime_template_name: 'colab-runtime-template'
- name: 'colab_runtime_template_no_name'
primary_resource_id: 'runtime-template'
exclude_import_test: true
- name: 'colab_runtime_template_full'
primary_resource_id: 'runtime-template'
primary_resource_name: 'fmt.Sprintf("tf-test-colab-runtime-template%s", context["random_suffix"])'
vars:
runtime_template_name: 'colab-runtime-template'
network_name: 'colab-test-default'
key_name: 'my-crypto-key'
test_vars_overrides:
key_name: 'acctest.BootstrapKMSKeyInLocation(t, "us-central1").CryptoKey.Name'
parameters:
- name: 'location'
type: String
required: true
url_param_only: true
description: 'The location for the resource: https://cloud.google.com/colab/docs/locations'
properties:
- name: 'name'
type: String
default_from_api: true
custom_flatten: 'templates/terraform/custom_flatten/name_from_self_link.tmpl'
description: 'The resource name of the Runtime Template'
- name: 'displayName'
type: String
description:
Required. The display name of the Runtime Template.
required: true
- name: description
type: String
description: 'The description of the Runtime Template.'
- name: machineSpec
type: NestedObject
default_from_api: true
description: |
'The machine configuration of the runtime.'
properties:
- name: 'machineType'
bcreddy-gcp marked this conversation as resolved.
Show resolved Hide resolved
type: string
default_from_api: true
description: |
The Compute Engine machine type selected for the runtime.
- name: 'acceleratorType'
type: enum
description: |
The type of hardware accelerator used by the runtime. If specified, acceleratorCount must also be specified.
enum_values:
- 'NVIDIA_TESLA_V100'
- 'NVIDIA_TESLA_T4'
- 'NVIDIA_TESLA_A100'
- 'NVIDIA_A100_80GB'
- 'NVIDIA_L4'
- name: 'acceleratorCount'
type: Integer
default_from_api: true
description: 'The number of accelerators used by the runtime.'
- name: dataPersistentDiskSpec
default_from_api: true
type: NestedObject
description: 'The configuration for the data disk of the runtime.'
properties:
- name: 'diskType'
type: enum
description: 'The type of the persistent disk.'
default_from_api: true
enum_values:
- 'pd-standard'
- 'pd-ssd'
- 'pd-balanced'
- 'pd-extreme'
- name: 'diskSizeGb'
type: int
default_from_api: true
description: |
The disk size of the runtime in GB. If specified, the diskType must also be specified. The minimum size is 10GB and the maximum is 65536GB.
- name: networkSpec
type: NestedObject
default_from_api: true
description: 'The network configuration for the runtime.'
properties:
- name: 'enableInternetAccess'
bcreddy-gcp marked this conversation as resolved.
Show resolved Hide resolved
type: Boolean
description: Enable public internet access for the runtime.
- name: 'network'
default_from_api: true
type: String
description: 'The name of the VPC that this runtime is in.'
diff_suppress_func: 'tpgresource.CompareSelfLinkRelativePaths'
- name: 'subnetwork'
type: String
description: 'The name of the subnetwork that this runtime is in.'
diff_suppress_func: 'tpgresource.CompareSelfLinkRelativePaths'
- name: 'labels'
type: KeyValueLabels
description: 'Labels to identify and group the runtime template.'
- name: idleShutdownConfig
type: NestedObject
default_from_api: true
description: 'Notebook Idle Shutdown configuration for the runtime.'
properties:
- name: 'idleTimeout'
default_from_api: true
type: String
description: 'The duration after which the runtime is automatically shut down. An input of 0s disables the idle shutdown feature, and a valid range is [10m, 24h].'
- name: eucConfig
type: NestedObject
description: 'EUC configuration of the NotebookRuntimeTemplate.'
properties:
- name: 'eucDisabled'
type: Boolean
description: 'Disable end user credential access for the runtime.'
- name: shieldedVmConfig
type: NestedObject
description: 'Runtime Shielded VM spec.'
properties:
- name: 'enableSecureBoot'
type: Boolean
description: 'Enables secure boot for the runtime.'
- name: 'networkTags'
type: Array
item_type:
type: String
description: 'Applies the given Compute Engine tags to the runtime.'
- name: encryptionSpec
type: NestedObject
description: 'Customer-managed encryption key spec for the notebook runtime.'
properties:
- name: 'kmsKeyName'
type: String
description: 'The Cloud KMS encryption key (customer-managed encryption key) used to protect the runtime.'
25 changes: 25 additions & 0 deletions mmv1/products/colab/product.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Copyright 2025 Google Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

---
name: 'Colab'
display_name: 'Colab Enterprise'
versions:
- name: 'ga'
base_url: 'https://{{region}}-aiplatform.googleapis.com/v1/'
cai_base_url: 'https://aiplatform.googleapis.com/v1/'
- name: 'beta'
base_url: 'https://{{region}}-aiplatform.googleapis.com/v1beta1/'
cai_base_url: 'https://aiplatform.googleapis.com/v1beta1/'
scopes:
- 'https://www.googleapis.com/auth/cloud-platform'
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
resource "google_colab_runtime_template" "{{$.PrimaryResourceId}}" {
name = "{{index $.Vars "runtime_template_name"}}"
display_name = "Runtime template basic"
location = "us-central1"

machine_spec {
machine_type = "e2-standard-4"
}

network_spec {
enable_internet_access = true
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
resource "google_compute_network" "my_network" {
name = "{{index $.Vars "network_name"}}"
auto_create_subnetworks = false
}

resource "google_compute_subnetwork" "my_subnetwork" {
name = "{{index $.Vars "network_name"}}"
network = google_compute_network.my_network.id
region = "us-central1"
ip_cidr_range = "10.0.1.0/24"
}

resource "google_colab_runtime_template" "{{$.PrimaryResourceId}}" {
name = "{{index $.Vars "runtime_template_name"}}"
display_name = "Runtime template full"
location = "us-central1"
description = "Full runtime template"
machine_spec {
machine_type = "n1-standard-2"
accelerator_type = "NVIDIA_TESLA_T4"
accelerator_count = "1"
}

data_persistent_disk_spec {
disk_type = "pd-standard"
disk_size_gb = 200
}

network_spec {
enable_internet_access = true
network = google_compute_network.my_network.id
subnetwork = google_compute_subnetwork.my_subnetwork.id
}

labels = {
k = "val"
}

idle_shutdown_config {
idle_timeout = "3600s"
}

euc_config {
euc_disabled = true
}

shielded_vm_config {
enable_secure_boot = true
}

network_tags = ["abc", "def"]

encryption_spec {
kms_key_name = "{{index $.Vars "key_name"}}"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
resource "google_colab_runtime_template" "{{$.PrimaryResourceId}}" {
display_name = "Runtime template no name"
location = "us-central1"

machine_spec {
machine_type = "e2-standard-4"
}

network_spec {
enable_internet_access = true
}
}
10 changes: 10 additions & 0 deletions mmv1/templates/terraform/post_create/colab_runtime_template.tmpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
// The operation for this resource contains the generated name that we need
// in order to perform a READ. We need to access the object inside of it as
// a map[string]interface, so let's do that.

resp := res["response"].(map[string]interface{})
name := tpgresource.GetResourceNameFromSelfLink(resp["name"].(string))
log.Printf("[DEBUG] Setting resource name, id to %s", name)
if err := d.Set("name", name); err != nil {
return fmt.Errorf("Error setting name: %s", err)
}
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,11 @@ var ServicesListBeta = mapOf(
"displayName" to "Cloudtasks",
"path" to "./google-beta/services/cloudtasks"
),
"colab" to mapOf(
"name" to "colab",
"displayName" to "Colab",
"path" to "./google-beta/services/colab"
),
"composer" to mapOf(
"name" to "composer",
"displayName" to "Composer",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,11 @@ var ServicesListGa = mapOf(
"displayName" to "Cloudtasks",
"path" to "./google/services/cloudtasks"
),
"colab" to mapOf(
"name" to "colab",
"displayName" to "Colab",
"path" to "./google/services/colab"
),
"composer" to mapOf(
"name" to "composer",
"displayName" to "Composer",
Expand Down
Loading