stable-diffusion-finetuner.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d9ce06f8-083d-4238-ade3-8dbe9295409c",
   "metadata": {},
   "source": [
    "# Fine Tune [Stable Diffusion Instruct-Pix2Pix](https://huggingface.co/docs/diffusers/training/instructpix2pix)\n",
    "\n",
    "For this fine tuning, In have used [AWS Sagemaker](https://aws.amazon.com/sagemaker/) with the following instance:\n",
    "\n",
    "* Sagemaker Instance: `ml.g5.2xlarge`\n",
    "* Sagemaker Image: Sagemaker Distribution `1.4`, `1.5`\n",
    "* Region: [us-east-1](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html) (_North Virginia_)\n",
    "* GPU: 1 x [NVIDIA A10G](https://aws.amazon.com/blogs/aws/new-ec2-instances-g5-with-nvidia-a10g-tensor-core-gpus/)\n",
    "* VRAM: 24 GB\n",
    "* Storage: 90 GB"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "beb5392d-4d23-43b8-a91f-424cf1a46b00",
   "metadata": {},
   "source": [
    "## `nvidia-smi` output on IDLE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ee6f85fb-724e-4016-bdec-d0695872aff5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wed Mar 27 10:50:31 2024       \n",
      "+---------------------------------------------------------------------------------------+\n",
      "| NVIDIA-SMI 535.129.03             Driver Version: 535.129.03   CUDA Version: 12.2     |\n",
      "|-----------------------------------------+----------------------+----------------------+\n",
      "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
      "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
      "|                                         |                      |               MIG M. |\n",
      "|=========================================+======================+======================|\n",
      "|   0  NVIDIA A10G                    Off | 00000000:00:1E.0 Off |                    0 |\n",
      "|  0%   21C    P0              52W / 300W |      4MiB / 23028MiB |      4%      Default |\n",
      "|                                         |                      |                  N/A |\n",
      "+-----------------------------------------+----------------------+----------------------+\n",
      "                                                                                         \n",
      "+---------------------------------------------------------------------------------------+\n",
      "| Processes:                                                                            |\n",
      "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
      "|        ID   ID                                                             Usage      |\n",
      "|=======================================================================================|\n",
      "|  No running processes found                                                           |\n",
      "+---------------------------------------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "! nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0ae8984-54ee-4053-8e46-469aa9ca8007",
   "metadata": {},
   "source": [
    "__STEP 1:__ Setup runtime\n",
    "\n",
    "__STEP 1.1:__ Clone the [huggingface/diffusers](https://github.com/huggingface/diffusers) Github repository. It is recommended to use a specific version of the repository to avoid downloading a pre-release build. To see all the available versions, check out all the available [tags](https://github.com/huggingface/diffusers/tags). Then use the following command to clone the repository at that particular release version. \n",
    "\n",
    "In the following command, the `v0.27.0` tag is used which implies that diffusers v0.27.0 is being used. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3a74fffc-a098-40d9-8aef-81cceaeaf553",
   "metadata": {},
   "outputs": [],
   "source": [
    "! git clone --branch v0.27.0 https://github.com/huggingface/diffusers.git"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d41b4bdc-badc-418a-b4ce-2a0163659db3",
   "metadata": {},
   "source": [
    "__STEP 1.2:__ Install `diffusers` from source code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c9df27d3-cc69-43d4-899b-a5468c981fae",
   "metadata": {},
   "outputs": [],
   "source": [
    "! python3 -m pip install -e ./diffusers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be5db088-5609-472c-93fe-fc5a12c93f5f",
   "metadata": {},
   "source": [
    "__STEP 1.3:__ Install other dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7750f42a-b0f1-46b0-b369-1f5a5d21252a",
   "metadata": {},
   "outputs": [],
   "source": [
    "! python3 -m pip install --no-cache-dir xformers transformers[torch] ftfy datasets tensorboard"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b9237c8-854d-4e03-ba1a-93323c53f08e",
   "metadata": {},
   "source": [
    "__STEP 1.4:__ Configure [accelerate](https://github.com/huggingface/accelerate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "763fe1d1-6b53-4a6f-930a-8fbd719346a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "! accelerate config default"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7df5de9b",
   "metadata": {},
   "source": [
    "__STEP 2:__ Login to [huggingface hub](https://hf.co)\n",
    "\n",
    "Obtain a _Read/Write_ or _Full Access_ Access Token by following the steps:\n",
    "\n",
    "1. Login to your huggingface account.\n",
    "2. Go to _Settings -> Access Tokens -> New Token_\n",
    "3. Click on _New Token_. Provde a _Name_ for your key and select the Type as _Write_ for Full Access.\n",
    "4. Copy the Key "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1f1a590e",
   "metadata": {},
   "outputs": [],
   "source": [
    "! huggingface-cli login --token=<paste-your-token>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bd8c41c-f7c7-4907-8a1d-72e2a86cc89d",
   "metadata": {},
   "source": [
    "__STEP 5:__ Start Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ecee329b-93fb-4995-9a68-e2e1704c7f75",
   "metadata": {},
   "outputs": [],
   "source": [
    "! export BASE_MODEL=stabilityai/stable-diffusion-2-1\n",
    "! export DATASET=<your-dataset>\n",
    "! export TUNED_MODEL=<tuned-model-name>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e56c63f-b2e7-4f61-af4f-8788b70d7b59",
   "metadata": {},
   "outputs": [],
   "source": [
    "! accelerate launch --mixed_precision=\"fp16\" \\\n",
    "    ./diffusers/examples/instruct_pix2pix/train_instruct_pix2pix.py \\\n",
    "    --pretrained_model_name_or_path=${BASE_MODEL} \\\n",
    "    --dataset_name=${DATASET} \\\n",
    "    --enable_xformers_memory_efficient_attention \\\n",
    "    --resolution=512 --random_flip \\\n",
    "    --train_batch_size=2 --gradient_accumulation_steps=4 --gradient_checkpointing \\\n",
    "    --max_train_steps=100 \\\n",
    "    --checkpointing_steps=25 --checkpoints_total_limit=1 \\\n",
    "    --learning_rate=5e-05 --max_grad_norm=1 --lr_warmup_steps=20 \\\n",
    "    --conditioning_dropout_prob=0.1 \\\n",
    "    --mixed_precision=fp16 \\\n",
    "    --seed=42 \\\n",
    "    --push_to_hub \\\n",
    "    --report_to=tensorboard \\\n",
    "    --output_dir=${TUNED_MODEL} \\\n",
    "    --original_image_column=before \\\n",
    "    --edit_prompt=prompt \\\n",
    "    --edited_image=after"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}