sdk/jobs/automl-standalone-jobs/automl-image-instance-segmentation-task-fridge-items/automl-image-instance-segmentation-task-fridge-items.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AutoML: Train \"the best\" Image Instance Segmentation model for a 'Fridge items' dataset.\n",
    "\n",
    "**Requirements** - In order to benefit from this tutorial, you will need:\n",
    "- A basic understanding of Machine Learning\n",
    "- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F)\n",
    "- An Azure ML workspace. [Check this notebook for creating a workspace](../../../resources/workspace/workspace.ipynb) \n",
    "- A Compute Cluster. [Check this notebook to create a compute cluster](../../../resources/compute/compute.ipynb)\n",
    "- A python environment\n",
    "- Installed Azure Machine Learning Python SDK v2 - [install instructions](../../../README.md) - check the getting started section\n",
    "\n",
    "**Learning Objectives** - By the end of this tutorial, you should be able to:\n",
    "- Connect to your AML workspace from the Python SDK\n",
    "- Create an `AutoML Image Instance Segmentation Training Job` with the 'image_instance_segmentation()' factory-function.\n",
    "- Train the model using AmlCompute by submitting/running the AutoML training job\n",
    "- Obtaing the model and score predictions with it\n",
    "\n",
    "**Motivations** - This notebook explains how to setup and run an AutoML image instance segmentation job. This is one of the nine ML-tasks supported by AutoML. Other ML-tasks are 'forecasting', 'classification', 'image object detection', 'nlp text classification', etc.\n",
    "\n",
    "In this notebook, we go over how you can use AutoML for training an Image Instance Segmentation model. We will use a small dataset to train the model, demonstrate how you can tune hyperparameters of the model to optimize model performance and deploy the model to use in inference scenarios. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Connect to Azure Machine Learning Workspace\n",
    "\n",
    "The [workspace](https://docs.microsoft.com/en-us/azure/machine-learning/concept-workspace) is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section we will connect to the workspace in which the job will be run.\n",
    "\n",
    "## 1.1. Import the required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1634852261599
    }
   },
   "outputs": [],
   "source": [
    "# Import required libraries\n",
    "from azure.identity import DefaultAzureCredential\n",
    "from azure.ai.ml import MLClient\n",
    "\n",
    "from azure.ai.ml.constants import AssetTypes, InputOutputModes\n",
    "from azure.ai.ml import Input\n",
    "from azure.ai.ml.automl import ImageObjectDetectionSearchSpace\n",
    "from azure.ai.ml.sweep import (\n",
    "    Choice,\n",
    "    Uniform,\n",
    "    BanditPolicy,\n",
    ")\n",
    "\n",
    "from azure.ai.ml import automl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2. Configure workspace details and get a handle to the workspace\n",
    "\n",
    "To connect to a workspace, we need identifier parameters - a subscription, resource group and workspace name. We will use these details in the `MLClient` from `azure.ai.ml` to get a handle to the required Azure Machine Learning workspace. We use the default [default azure authentication](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) for this tutorial. Check the [configuration notebook](../../configuration.ipynb) for more details on how to configure credentials and connect to a workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1634852261744
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "credential = DefaultAzureCredential()\n",
    "ml_client = None\n",
    "try:\n",
    "    ml_client = MLClient.from_config(credential)\n",
    "except Exception as ex:\n",
    "    print(ex)\n",
    "    # Enter details of your AML workspace\n",
    "    subscription_id = \"<SUBSCRIPTION_ID>\"\n",
    "    resource_group = \"<RESOURCE_GROUP>\"\n",
    "    workspace = \"<AML_WORKSPACE_NAME>\"\n",
    "    ml_client = MLClient(credential, subscription_id, resource_group, workspace)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. MLTable with input Training Data\n",
    "\n",
    "In order to generate models for computer vision tasks with automated machine learning, you need to bring labeled image data as input for model training in the form of an MLTable. You can create an MLTable from labeled training data in JSONL format. If your labeled training data is in a different format (like, pascal VOC or COCO), you can use a conversion script to first convert it to JSONL, and then create an MLTable. Alternatively, you can use Azure Machine Learning's [data labeling tool](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-create-image-labeling-projects) to manually label images, and export the labeled data to use for training your AutoML model.\n",
    "\n",
    "In this notebook, we use a toy dataset called Fridge Objects, which consists of 128 images of 4 labels of beverage container {can, carton, milk bottle, water bottle} photos taken on different backgrounds.\n",
    "\n",
    "All images in this notebook are hosted in [this repository](https://github.com/microsoft/computervision-recipes) and are made available under the [MIT license](https://github.com/microsoft/computervision-recipes/blob/master/LICENSE).\n",
    "\n",
    "**NOTE:** In this PRIVATE PREVIEW we're defining the MLTable in a separate folder and .YAML file.\n",
    "In later versions, you'll be able to do it all in Python APIs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1. Download the Data\n",
    "We first download and unzip the data locally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import urllib\n",
    "from zipfile import ZipFile\n",
    "\n",
    "# download data\n",
    "download_url = \"https://cvbp-secondary.z19.web.core.windows.net/datasets/object_detection/odFridgeObjectsMask.zip\"\n",
    "data_file = \"./data/odFridgeObjectsMask.zip\"\n",
    "urllib.request.urlretrieve(download_url, filename=data_file)\n",
    "\n",
    "# extract files\n",
    "with ZipFile(data_file, \"r\") as zip:\n",
    "    print(\"extracting files...\")\n",
    "    zip.extractall(path=\"./data\")\n",
    "    print(\"done\")\n",
    "# delete zip file\n",
    "os.remove(data_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a sample image from this dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Image\n",
    "\n",
    "sample_image = \"./data/odFridgeObjectsMask/images/31.jpg\"\n",
    "Image(filename=sample_image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2. Upload the images to Datastore through an AML Data asset (URI Folder)\n",
    "\n",
    "In order to use the data for training in Azure ML, we upload it to our default Azure Blob Storage of our  Azure ML Workspace.\n",
    "\n",
    "Reference to URI FOLDER data asset example for further details: https://github.com/Azure/azureml-examples/blob/samuel100/data-samples/sdk/assets/data/data.ipynb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uploading image files by creating a 'data asset URI FOLDER':\n",
    "\n",
    "from azure.ai.ml.entities import Data\n",
    "from azure.ai.ml.constants import AssetTypes\n",
    "\n",
    "my_data = Data(\n",
    "    path=\"./data/odFridgeObjectsMask\",\n",
    "    type=AssetTypes.URI_FOLDER,\n",
    "    description=\"Fridge-items images instance segmentation\",\n",
    "    name=\"fridge-items-images-instance-segmentation\",\n",
    ")\n",
    "\n",
    "uri_folder_data_asset = ml_client.data.create_or_update(my_data)\n",
    "\n",
    "print(uri_folder_data_asset)\n",
    "print(\"\")\n",
    "print(\"Path to folder in Blob Storage:\")\n",
    "print(uri_folder_data_asset.path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3. Convert the downloaded data to JSONL\n",
    "In this example, the fridge object dataset is annotated in Pascal VOC format, where each image corresponds to an xml file. Each xml file contains information on where its corresponding image file is located and also contains information about the bounding boxes and the object labels. In order to use this data to create an AzureML MLTable, we first need to convert it to the required JSONL format. Please refer to the [documentation on how to prepare datasets](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-prepare-datasets-for-automl-images).\n",
    "\n",
    "\n",
    "The following script is creating two .jsonl files (one for training and one for validation) in the corresponding MLTable folder. The train / validation ratio corresponds to 20% of the data going into the validation file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The jsonl_converter below relies on scikit-image and simplification.\n",
    "# If you don't have them installed, install them before converting data by runing this cell.\n",
    "%pip install \"scikit-image==0.17.2\" \"simplification\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from jsonl_converter import convert_mask_in_VOC_to_jsonl\n",
    "\n",
    "data_path = \"./data/odFridgeObjectsMask/\"\n",
    "convert_mask_in_VOC_to_jsonl(data_path, uri_folder_data_asset.path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.4. Create MLTable data input\n",
    "\n",
    "Create MLTable data input using the jsonl files created above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_mltable_path = \"./data/training-mltable-folder/\"\n",
    "validation_mltable_path = \"./data/validation-mltable-folder/\"\n",
    "\n",
    "# Training MLTable defined locally, with local data to be uploaded\n",
    "my_training_data_input = Input(type=AssetTypes.MLTABLE, path=training_mltable_path)\n",
    "\n",
    "# Validation MLTable defined locally, with local data to be uploaded\n",
    "my_validation_data_input = Input(type=AssetTypes.MLTABLE, path=validation_mltable_path)\n",
    "\n",
    "# WITH REMOTE PATH: If available already in the cloud/workspace-blob-store\n",
    "# my_training_data_input = Input(type=AssetTypes.MLTABLE, path=\"azureml://datastores/workspaceblobstore/paths/vision-classification/train\")\n",
    "# my_validation_data_input = Input(type=AssetTypes.MLTABLE, path=\"azureml://datastores/workspaceblobstore/paths/vision-classification/valid\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create data input from TabularDataset created using V1 sdk, specify the `type` as `AssetTypes.MLTABLE`, `mode` as `InputOutputModes.DIRECT` and `path` in the following format `azureml:<tabulardataset_name>:<version>`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "# Training MLTable with v1 TabularDataset\n",
    "my_training_data_input = Input(\n",
    "    type=AssetTypes.MLTABLE, path=\"azureml:odFridgeObjectsMaskTrainingDataset:1\",\n",
    "    mode=InputOutputModes.DIRECT\n",
    ")\n",
    "\n",
    "# Validation MLTable with v1 TabularDataset\n",
    "my_validation_data_input = Input(\n",
    "    type=AssetTypes.MLTABLE, path=\"azureml:odFridgeObjectsMaskValidationDataset:1\",\n",
    "    mode=InputOutputModes.DIRECT\n",
    ")\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Compute target setup\n",
    "\n",
    "You will need to provide a [Compute Target](https://docs.microsoft.com/en-us/azure/machine-learning/concept-azure-machine-learning-architecture#computes) that will be used for your AutoML model training. AutoML models for image tasks require [GPU SKUs](https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-gpu) such as the ones from the NC, NCv2, NCv3, ND, NDv2 and NCasT4 series. We recommend using the NCsv3-series (with v100 GPUs) for faster training. Using a compute target with a multi-GPU VM SKU will leverage the multiple GPUs to speed up training. Additionally, setting up a compute target with multiple nodes will allow for faster model training by leveraging parallelism, when tuning hyperparameters for your model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.ai.ml.entities import AmlCompute\n",
    "from azure.core.exceptions import ResourceNotFoundError\n",
    "\n",
    "compute_name = \"gpu-cluster\"\n",
    "\n",
    "try:\n",
    "    _ = ml_client.compute.get(compute_name)\n",
    "    print(\"Found existing compute target.\")\n",
    "except ResourceNotFoundError:\n",
    "    print(\"Creating a new compute target...\")\n",
    "    compute_config = AmlCompute(\n",
    "        name=compute_name,\n",
    "        type=\"amlcompute\",\n",
    "        size=\"Standard_NC6\",\n",
    "        idle_time_before_scale_down=120,\n",
    "        min_instances=0,\n",
    "        max_instances=4,\n",
    "    )\n",
    "    ml_client.begin_create_or_update(compute_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Configure and run the AutoML for Images Instance Segmentation training job\n",
    "\n",
    "AutoML allows you to easily train models for Image Classification, Object Detection & Instance Segmentation on your image data. You can control the model algorithm to be used, specify hyperparameter values for your model as well as perform a sweep across the hyperparameter space to generate an optimal model.\n",
    "\n",
    "When using AutoML for image tasks, you need to specify the model algorithms using the model_name parameter. You can either specify a single model or choose to sweep over multiple models. Please refer to the [documentation](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=CLI-v2#configure-model-algorithms-and-hyperparameters) for the list of supported model algorithms.\n",
    "\n",
    "## 4.1. Using default hyperparameter values for the specified algorithm\n",
    "Before doing a large sweep to search for the optimal models and hyperparameters, we recommend trying the default values for a given model to get a first baseline. Next, you can explore multiple hyperparameters for the same model before sweeping over multiple models and their parameters. This allows an iterative approach, as with multiple models and multiple hyperparameters for each (as we showcase in the next section), the search space grows exponentially, and  you need more iterations to find optimal configurations.\n",
    "\n",
    "Following functions are used to configure the AutoML image job:\n",
    "### image_instance_segmentation() function parameters:\n",
    "The `image_instance_segmentation()` factory function allows user to configure the training job.\n",
    "\n",
    "- `compute` - The compute on which the AutoML job will run. In this example we are using a compute called 'gpu-cluster' present in the workspace. You can replace it any other compute in the workspace.\n",
    "- `experiment_name` - The name of the experiment. An experiment is like a folder with multiple runs in Azure ML Workspace that should be related to the same logical machine learning experiment.\n",
    "- `name` - The name of the Job/Run. This is an optional property. If not specified, a random name will be generated.\n",
    "- `primary_metric` - The metric that AutoML will optimize for model selection.\n",
    "- `target_column_name` - The name of the column to target for predictions. It must always be specified. This parameter is applicable to 'training_data' and 'validation_data'.\n",
    "- `training_data` - The data to be used for training. It should contain both training feature columns and a target column. Optionally, this data can be split for segregating a validation or test dataset. \n",
    "You can use a registered MLTable in the workspace using the format '<mltable_name>:<version>' OR you can use a local file or folder as a MLTable. For e.g Input(mltable='my_mltable:1') OR Input(mltable=MLTable(local_path=\"./data\"))\n",
    "The parameter 'training_data' must always be provided.\n",
    "\n",
    "### set_limits() parameters:\n",
    "This is an optional configuration method to configure limits parameters such as timeouts.     \n",
    "    \n",
    "- `timeout_minutes` - Maximum amount of time in minutes that the whole AutoML job can take before the job terminates. If not specified, the default job's total timeout is 6 days (8,640 minutes).\n",
    "\n",
    "### set_image_model() function parameters:\n",
    "This is an optional configuration method to configure fixed settings or parameters that don't change during the parameter space sweep. Some of the key parameters of this function are:\n",
    "\n",
    "- `model_name` - The name of the ML algorithm that we want to use in training job. Please refer to this [documentation](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=CLI-v2#supported-model-algorithms) for supported model algorithm.\n",
    "- `number_of_epochs` - The number of training epochs. It must be positive integer (default value is 15).\n",
    "- `layers_to_freeze` - The number of layers to freeze in model for transfer learning. It must be a positive integer (default value is 0).\n",
    "- `early_stopping` - It enable early stopping logic during training, It must be boolean value (default is True).   \n",
    "- `optimizer` - Type of optimizer to use in training. It must be either sgd, adam, adamw (default is sgd).\n",
    "- `distributed` - It enable distributed training if compute target contain multiple GPUs. It must be boolean value (default is True).\n",
    "\n",
    "If you wish to use the default hyperparameter values for a given algorithm (say `maskrcnn`), you can specify the job for your AutoML Image runs as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "# general job parameters\n",
    "exp_name = \"dpv2-image-instance-segmentation-experiment\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the AutoML job with the related factory-function.\n",
    "\n",
    "image_instance_segmentation_job = automl.image_instance_segmentation(\n",
    "    compute=compute_name,\n",
    "    # name=\"dpv2-image-instance-segmentation-job-02\",\n",
    "    experiment_name=exp_name,\n",
    "    training_data=my_training_data_input,\n",
    "    validation_data=my_validation_data_input,\n",
    "    target_column_name=\"label\",\n",
    ")\n",
    "\n",
    "image_instance_segmentation_job.set_limits(timeout_minutes=60)\n",
    "\n",
    "image_instance_segmentation_job.set_image_model(model_name=\"maskrcnn_resnet50_fpn\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submitting an AutoML job for Computer Vision tasks\n",
    "Once you've configured your job, you can submit it as a job in the workspace in order to train a vision model using your training dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Submit the AutoML job\n",
    "returned_job = ml_client.jobs.create_or_update(image_instance_segmentation_job)\n",
    "\n",
    "print(f\"Created job: {returned_job}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.jobs.stream(returned_job.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2. Hyperparameter sweeping for your AutoML models for computer vision tasks\n",
    "\n",
    "When using AutoML for Images, you can perform a hyperparameter sweep over a defined parameter space to find the optimal model. In this example, we sweep over the hyperparameters for `maskrcnn_resnet50_fpn` model which is pretrained on COCO, a large-scale object detection, segmentation, and captioning dataset that contains over 200K labeled images with over 80 label categories, choosing from a range of values for learning_rate, optimizer, etc., to generate a model with the optimal 'MeanAveragePrecision'. If hyperparameter values are not specified, then default values are used for the specified algorithm.\n",
    "\n",
    "set_sweep function is used to configure the sweep settings:\n",
    "### set_sweep() parameters:\n",
    "\n",
    "- `max_trials` - Required parameter for maximum number of configurations to sweep. Must be an integer between 1 and 1000. When exploring just the default hyperparameters for a given model algorithm, set this parameter to 1.\n",
    "- `max_concurrent_trials` - Maximum number of runs that can run concurrently. If not specified, all runs launch in parallel. If specified, must be an integer between 1 and 100.\n",
    "    NOTE: The number of concurrent runs is gated on the resources available in the specified compute target. Ensure that the compute target has the available resources for the desired concurrency.\n",
    "- `sampling_algorithm` - Sampling method to use for sweeping over the defined parameter space. Please refer to this [documentation](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=SDK-v2#sampling-methods-for-the-sweep) for list of supported sampling methods.\n",
    "- `early_termination` - Early termination policy to end poorly performing runs. If no termination policy is specified, all configurations are run to completion. Please refer to this [documentation](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=SDK-v2#early-termination-policies) for supported early termination policies.\n",
    "\n",
    "We use Random Sampling to pick samples from this parameter space and try a total of 10 iterations with these different samples, running 2 iterations at a time on our compute target. Please note that the more parameters the space has, the more iterations you need to find optimal models.\n",
    "\n",
    "We leverage the Bandit early termination policy which will terminate poor performing configs (those that are not within 20% slack of the best performing config), thus significantly saving compute resources.\n",
    "\n",
    "For more details on model and hyperparameter sweeping, please refer to the [documentation](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-tune-hyperparameters)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1634852262026
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "name": "image-instance-segmentation-configuration",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "# Create the AutoML job with the related factory-function.\n",
    "\n",
    "image_instance_segmentation_job = automl.image_instance_segmentation(\n",
    "    compute=compute_name,\n",
    "    # name=\"dpv2-image-instance-segmentation-job-02\",\n",
    "    experiment_name=exp_name,\n",
    "    training_data=my_training_data_input,\n",
    "    validation_data=my_validation_data_input,\n",
    "    target_column_name=\"label\",\n",
    "    primary_metric=\"MeanAveragePrecision\",\n",
    "    tags={\"my_custom_tag\": \"My custom value\"},\n",
    ")\n",
    "\n",
    "image_instance_segmentation_job.set_limits(timeout_minutes=60)\n",
    "\n",
    "image_instance_segmentation_job.extend_search_space(\n",
    "    [\n",
    "        ImageObjectDetectionSearchSpace(\n",
    "            model_name=Choice([\"maskrcnn_resnet50_fpn\"]),\n",
    "            learning_rate=Uniform(0.0001, 0.001),\n",
    "            # warmup_cosine_lr_warmup_epochs=Choice([0, 3]),\n",
    "            optimizer=Choice([\"sgd\", \"adam\", \"adamw\"]),\n",
    "            min_size=Choice([600, 800]),\n",
    "        ),\n",
    "    ]\n",
    ")\n",
    "\n",
    "image_instance_segmentation_job.set_sweep(\n",
    "    max_trials=10,\n",
    "    max_concurrent_trials=2,\n",
    "    sampling_algorithm=\"Random\",\n",
    "    early_termination=BanditPolicy(\n",
    "        evaluation_interval=2, slack_factor=0.2, delay_evaluation=6\n",
    "    ),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1634852267930
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "# Submit the AutoML job\n",
    "returned_job = ml_client.jobs.create_or_update(\n",
    "    image_instance_segmentation_job\n",
    ")  # submit the job to the backend\n",
    "\n",
    "print(f\"Created job: {returned_job}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.jobs.stream(returned_job.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When doing a hyperparameter sweep, it can be useful to visualize the different configurations that were tried using the HyperDrive UI. You can navigate to this UI by going to the 'Child jobs' tab in the UI of the main automl image job from above, which is the HyperDrive parent run. Then you can go into the 'Trials' tab of this HyperDrive parent run. Alternatively, here below you can see directly the HyperDrive parent run and navigate to its 'Trials' tab:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hd_job = ml_client.jobs.get(returned_job.name + \"_HD\")\n",
    "hd_job"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Next Steps\n",
    "You can see further examples of other AutoML tasks such as Regression, Image-Object-Detection, NLP-Text-Classification, Time-Series-Forcasting, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernel_info": {
   "name": "python3-azureml"
  },
  "kernelspec": {
   "display_name": "Python 3.8 - AzureML",
   "language": "python",
   "name": "python38-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "microsoft": {
   "host": {
    "AzureML": {
     "notebookHasBeenCompleted": true
    }
   }
  },
  "nteract": {
   "version": "nteract-front-end@1.0.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}