descriptors.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3afedad0-8582-4382-9014-e03e4e2ad143",
   "metadata": {},
   "source": [
    "# Descriptors: secret weapon of frameworks\n",
    "\n",
    "A talk presented at [Python Cerrado / Plone Conference 2024](https://2024.ploneconf.org/en/schedule/talks/descriptors-secret-weapon-of-frameworks)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d10677c-ce2a-427d-926a-fe46e4ad920e",
   "metadata": {},
   "source": [
    "## Motivating example\n",
    "\n",
    "<img src=\"bulk_food_by_Dan_Bruell.jpg\" alt=\"Bulk food bins, by Dan Bruell\" width=\"600\"> \n",
    "\n",
    "Bulk food bins, by Dan Bruell (CC BY-NC-SA 2.0)\n",
    "\n",
    "### Class for an item in an order of bulk food\n",
    "\n",
    "Imagine an app for a store that sells organic food in bulk,\n",
    "where customers can order nuts, dried fruit, or cereals by weight.\n",
    "In that system, each order would hold a sequence of line items,\n",
    "and each line item could be represented by an instance of a\n",
    "class like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0b02dad5-89c9-4fa4-81a9-d938225c4d9e",
   "metadata": {},
   "outputs": [],
   "source": [
    "class LineItem:\n",
    "\n",
    "    def __init__(self, description, weight, price):\n",
    "        self.description = description\n",
    "        self.weight = weight\n",
    "        self.price = price\n",
    "\n",
    "    def subtotal(self):\n",
    "        return self.weight * self.price"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a9b3d6f-16a8-4263-93e9-eed8f8d8d339",
   "metadata": {},
   "source": [
    "That's nice and simple:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "4e628e11-8917-4a20-9d2c-d5e81e7d8acb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "69.5"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raisins = LineItem('Golden raisins', 10, 6.95)\n",
    "raisins.subtotal()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdb907b6-81fa-4463-a38f-3d38ca84e1ff",
   "metadata": {},
   "source": [
    "But there's a problem:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "4f3f2688-273f-4985-80dc-bfcb47de5e51",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-139.0"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raisins.weight = -20\n",
    "raisins.subtotal()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96f8e64a-8199-410f-832e-d58be98e5811",
   "metadata": {},
   "source": [
    "This is a toy example, but not as fanciful as you may think.\n",
    "Here is a story from the early days of Amazon.com:\n",
    "\n",
    ">  We found that customers could order a negative quantity of books! And we would\n",
    "  credit their credit card with the price and, I assume, wait around for them to ship the\n",
    "  books.<br>\n",
    "  — Jeff Bezos, founder and CEO of Amazon.com, interviewed by Wall Street Journal in “Birth of a Salesman” (October 15, 2011)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8eb9970e-30df-42cb-8afc-20829dd9b8cd",
   "metadata": {},
   "source": [
    "### Type checkers won't save us"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35746d01-e859-4837-8b83-4a7a6e0144b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "from dataclasses import dataclass\n",
    "\n",
    "@dataclass\n",
    "class LineItem:\n",
    "    description: str\n",
    "    weight: float\n",
    "    price: float\n",
    "\n",
    "    def subtotal(self):\n",
    "        return self.weight * self.price"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d01ce17e-3f19-49ec-8b08-98571fbccb83",
   "metadata": {},
   "source": [
    "There is no way to specify that `weight` must be a `float` greater than zero.\n",
    "\n",
    "Only [dependend types](https://en.wikipedia.org/wiki/Dependent_type) could fix this,\n",
    "but that's a feature that exists only in obscure, academic functional languages like Agda and Idris."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6915c39-fb3e-46d0-8bd0-0025adc4bf94",
   "metadata": {},
   "source": [
    "### Why not getters and setters?\n",
    "\n",
    "The traditional way of controlling attribute access in C++ or Java is to use _getters_ and _setters_: methods that get and set values for attributes that are declared private or protected in some way.\n",
    "\n",
    "But that is verbose, and produces what the famous Pythonista Alex Martelli calls \"goofy idioms\" like this:\n",
    "\n",
    "```python\n",
    "item.set_quantity(item.get_quantity() + 1)\n",
    "```\n",
    "\n",
    "Using public attributes, properties, or descriptors, the uglyness of getters and setters is hidden from the users of your class, and they can accomplish the same result writing this:\n",
    "\n",
    "```python\n",
    "item.quantity += 1\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bb176d5-a659-41db-8685-7e50ff892f09",
   "metadata": {},
   "source": [
    "### Properties to the rescue\n",
    "\n",
    "> The crucial importance of properties is that their existence makes it perfectly safe and\n",
    " indeed advisable for you to expose public data attributes as part of your class’s public\n",
    " interface. <br>\n",
    "  — Martelli, Ravenscroft, and Holden, “Why properties are important”, _Python in a Nutshell, 2nd edition_\n",
    "\n",
    "\n",
    "Properties enable the implementation of getters and setters without\n",
    "changing the public interface of a class that previously\n",
    "allowed reading and writing public attributes via `item.quantity` notation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7f6c8fe9-7882-4cf6-8ef0-c92c6e9dafa0",
   "metadata": {},
   "outputs": [],
   "source": [
    "class LineItem:\n",
    "\n",
    "    def __init__(self, description, weight, price):\n",
    "        self.description = description\n",
    "        self.weight = weight  # (1)\n",
    "        self.price = price\n",
    "\n",
    "    def subtotal(self):\n",
    "        return self.weight * self.price\n",
    "\n",
    "    @property  # (2)\n",
    "    def weight(self):  # (3)\n",
    "        return self.__weight  # (4)\n",
    "\n",
    "    @weight.setter  # (5)\n",
    "    def weight(self, value):\n",
    "        if value > 0:\n",
    "            self.__weight = value  # (6)\n",
    "        else:\n",
    "            raise ValueError('weight must be > 0')  # (7)\n",
    "            \n",
    "    @property\n",
    "    def price(self):\n",
    "        return self.__price\n",
    "\n",
    "    @price.setter  # (5)\n",
    "    def price(self, value):\n",
    "        if value > 0:\n",
    "            self.__price = value\n",
    "        else:\n",
    "            raise ValueError('price must be > 0')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "934c60b7-74df-44f6-8758-20137583c4b0",
   "metadata": {},
   "source": [
    "Uncomment to see a demonstration:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "c2077ad0-f603-4f82-a14b-8eb7675f0897",
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "weight must be > 0",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[6], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m raisins \u001b[38;5;241m=\u001b[39m \u001b[43mLineItem\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mGolden raisins\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m-\u001b[39;49m\u001b[38;5;241;43m10\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m6.95\u001b[39;49m\u001b[43m)\u001b[49m\n",
      "Cell \u001b[0;32mIn[4], line 5\u001b[0m, in \u001b[0;36mLineItem.__init__\u001b[0;34m(self, description, weight, price)\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__init__\u001b[39m(\u001b[38;5;28mself\u001b[39m, description, weight, price):\n\u001b[1;32m      4\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mdescription \u001b[38;5;241m=\u001b[39m description\n\u001b[0;32m----> 5\u001b[0m     \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mweight\u001b[49m \u001b[38;5;241m=\u001b[39m weight  \u001b[38;5;66;03m# (1)\u001b[39;00m\n\u001b[1;32m      6\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mprice \u001b[38;5;241m=\u001b[39m price\n",
      "Cell \u001b[0;32mIn[4], line 20\u001b[0m, in \u001b[0;36mLineItem.weight\u001b[0;34m(self, value)\u001b[0m\n\u001b[1;32m     18\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m__weight \u001b[38;5;241m=\u001b[39m value  \u001b[38;5;66;03m# (6)\u001b[39;00m\n\u001b[1;32m     19\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m---> 20\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mweight must be > 0\u001b[39m\u001b[38;5;124m'\u001b[39m)\n",
      "\u001b[0;31mValueError\u001b[0m: weight must be > 0"
     ]
    }
   ],
   "source": [
    "raisins = LineItem('Golden raisins', -10, 6.95)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0cebfbc-8a63-40f2-a5c2-5ba6b82f42b4",
   "metadata": {},
   "source": [
    "## Descriptors to the rescue\n",
    "\n",
    "The previous example is fine, but there's a lot of repetition.\n",
    "A descriptor class allows reuse of attribute validation logic.\n",
    "\n",
    "By the way, `@property` itself is implemented as a descriptor class.\n",
    "\n",
    "### `Quantity`: a validation descriptor\n",
    "\n",
    "We'll implement a descriptor class named `Quantity` that enforces the rule that a numeric attribute must be greater than zero.\n",
    "\n",
    "Here's how we want to write `LineItem` with quantity:\n",
    "\n",
    "```python\n",
    "class LineItem:\n",
    "    weight = Quantity()\n",
    "    price = Quantity()\n",
    "\n",
    "    def __init__(self, description, weight, price):\n",
    "        self.description = description\n",
    "        self.weight = weight\n",
    "        self.price = price\n",
    "\n",
    "    def subtotal(self):\n",
    "        return self.weight * self.price\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ed9d488-d512-4675-9020-3acd8da580f8",
   "metadata": {},
   "source": [
    "### A descriptor class and a managed class\n",
    "\n",
    "Before looking a the implementation of `Quantity`, consider just these lines of `LineItem`:\n",
    "\n",
    "```python\n",
    "\n",
    "class LineItem:\n",
    "    weight = Quantity()\n",
    "    price = Quantity()\n",
    "```\n",
    "\n",
    "What is happening here?\n",
    "\n",
    "Both `weight` and `price` are attributes of the `LineItem` **class**.\n",
    "\n",
    "Also, they are two separate instances of `Quantity`.\n",
    "\n",
    "Here is a UML class diagram for that:\n",
    "\n",
    "<img src=\"flpy_2301.png\" alt=\"UML class diagram showing relationship between `LineItem` and `Quantity`\" width=\"800\"> \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c427a3e-2938-4476-a797-1b3c5fb2b95e",
   "metadata": {},
   "source": [
    "Metaprogramming may involve unusual relationships between classes and instances that may be hard to read in UML.\n",
    "\n",
    "So, I invented the Mills & Gizmos Notation to enhance UML diagrams. Here is a sample of MGN:\n",
    "\n",
    "<img src=\"flpy_2303.png\" alt=\"Classes and instances drawn as mills and gizmos.\" width=\"800\"> \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "318d22a8-5133-4b41-91fd-4545c938fe22",
   "metadata": {},
   "source": [
    "Here is that same UML diagram annotated with Mills & Gizmos:\n",
    "\n",
    "<img src=\"flpy_2302.png\" alt=\"UML class diagram showing relationship between `LineItem` and `Quantity` with mills and gizmos\" width=\"800\"> \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26f0f33d-50db-4ae1-a511-c5688b3eeadb",
   "metadata": {},
   "source": [
    "### Implementing `Quantity`, the simplest way\n",
    "\n",
    "Descriptor classes are implemented in frameworks like Django or SQLAlchemy, not application code.\n",
    "\n",
    "For example, this could be part of a `validators.py` module in a framework:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "c0e9851d-d8c7-47bb-b991-f98c703f4dc3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# part of a validators.py module\n",
    "\n",
    "class Quantity:  # (1)\n",
    "\n",
    "    def __init__(self, storage_name):  # (2)\n",
    "        self.storage_name = storage_name\n",
    "\n",
    "    def __set__(self, instance, value):   # (3)\n",
    "        if value > 0:\n",
    "            instance.__dict__[self.storage_name] = value\n",
    "        else:\n",
    "            msg = f'{self.storage_name} must be > 0'\n",
    "            raise ValueError(msg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b64e2ab-0a67-4f70-8484-882726a262b3",
   "metadata": {},
   "source": [
    "**(1)** descriptor is a protocol-based feature: there's no need to subclass anything, just implement one of the relevant special methods.\n",
    "\n",
    "**(2)** The `storage_name` is the name of the attribute in each managed class instance that will hold the value of the attribute controlled by the descriptor.\n",
    "\n",
    "**(2)** Implementing `__set__` makes a class behave as a descriptor. `self` is the descriptor instance, `instance` is the managed instance, and `value` is the value being set.\n",
    "\n",
    "**(4)** If we have a valid `value`, then we save it in the `instance.__dict__`. We can't use `settattr(instance, self.storage_name, value`) because that would trigger the descriptor `__set__` again, leading to uncontrolled recursion.\n",
    "\n",
    "**(5)** We build a user friendly message explaining how to fix the problem, and raise `ValueError`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dacf5fd6-6cf0-49ea-8fca-a11a1a3253c8",
   "metadata": {},
   "source": [
    "Here is the application code that would use that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "3e37bea4-932d-45de-9425-267867e65378",
   "metadata": {},
   "outputs": [],
   "source": [
    "# from validators import Quantity\n",
    "\n",
    "class LineItem:\n",
    "    weight = Quantity('weight')\n",
    "    price = Quantity('price')\n",
    "\n",
    "    def __init__(self, description, weight, price):\n",
    "        self.description = description\n",
    "        self.weight = weight\n",
    "        self.price = price\n",
    "\n",
    "    def subtotal(self):\n",
    "        return self.weight * self.price"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "ab43a3f4-b7f8-4cc2-aa4c-92e5031a3446",
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "price must be > 0",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[10], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m granola \u001b[38;5;241m=\u001b[39m \u001b[43mLineItem\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mBacon granola\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m10\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m0\u001b[39;49m\u001b[43m)\u001b[49m\n",
      "Cell \u001b[0;32mIn[8], line 10\u001b[0m, in \u001b[0;36mLineItem.__init__\u001b[0;34m(self, description, weight, price)\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mdescription \u001b[38;5;241m=\u001b[39m description\n\u001b[1;32m      9\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mweight \u001b[38;5;241m=\u001b[39m weight\n\u001b[0;32m---> 10\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mprice\u001b[49m \u001b[38;5;241m=\u001b[39m price\n",
      "Cell \u001b[0;32mIn[7], line 13\u001b[0m, in \u001b[0;36mQuantity.__set__\u001b[0;34m(self, instance, value)\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m     12\u001b[0m     msg \u001b[38;5;241m=\u001b[39m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstorage_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m must be > 0\u001b[39m\u001b[38;5;124m'\u001b[39m\n\u001b[0;32m---> 13\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(msg)\n",
      "\u001b[0;31mValueError\u001b[0m: price must be > 0"
     ]
    }
   ],
   "source": [
    "granola = LineItem('Bacon granola', 10, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2c9dbfc-f5f6-4af4-87f6-d521ce0f08fc",
   "metadata": {},
   "source": [
    "Now consider this example. Can you spot the bug?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "aea942f3-82b1-44c0-ae4c-2773df9983f7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<__main__.Quantity at 0x73cdb06eeb70>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class Dog:\n",
    "    weight = Quantity('weigth')\n",
    "\n",
    "    def __init__(self, name, weight):\n",
    "        self.name = name\n",
    "        self.weight = weight\n",
    "\n",
    "\n",
    "rex = Dog('Rex', 8.5)\n",
    "rex.weight"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7980a513-e8cc-4f74-956b-9bae5180e9ed",
   "metadata": {},
   "source": [
    "### `Quantity` take #2: automatic storage_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "38f83c47-e044-49fc-bf93-08e18dad2e18",
   "metadata": {},
   "outputs": [],
   "source": [
    "# part of a validators.py module\n",
    "\n",
    "class Quantity:\n",
    "\n",
    "    # (1)\n",
    "    \n",
    "    def __set_name__(self, owner, name):  # (2)\n",
    "        self.storage_name = name          # (3)\n",
    "\n",
    "    def __set__(self, instance, value):   # (4)\n",
    "        if value > 0:\n",
    "            instance.__dict__[self.storage_name] = value\n",
    "        else:\n",
    "            msg = f'{self.storage_name} must be > 0'\n",
    "            raise ValueError(msg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bcd0eea-af2f-46ab-b8f4-b829fa823c31",
   "metadata": {},
   "source": [
    "**(1)** we don't need the `__init__` anymore in this example.\n",
    "\n",
    "**(2)** `__set_name__` is the newest special method of the descriptor protocol: `self` is the descriptor instance, owner is the managed class (where the descriptor is instantiated as a class attribute) and `name` is the name of the managed class attribute to which the descriptor is assigned, for example `\"weight\"` or `\"price\"` in the `LineItem` example.\n",
    "\n",
    "**(3)** here we set `self.storage_name`, which we previously did in `__init__`.\n",
    "\n",
    "**(4)** the rest of the code is the same as before.\n",
    "\n",
    "The `__set_name__` special method is called by the `type` metaclass, the factory of all Python classes, when a descriptor is assigned to a class attribute. It was added to the descriptor protocol in Python 3.6. Solving the problem of the `storage_name` was one of the main uses cases for custom metaclasses. Now we don't need them for this reason."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9c6636a-9125-4bb0-9724-18dc0f0266cd",
   "metadata": {},
   "source": [
    "Uncomment to see a test that will generate a validation error due to a negative `weight`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "586942b2-4adc-4719-91ae-a878972bcca0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# raisins = LineItem('Golden raisins', -10, 6.95)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1fe86b7-1f51-4902-83b5-36068e58698e",
   "metadata": {},
   "source": [
    "### `Quantity` is an overriding descriptor\n",
    "\n",
    "Any descriptor that implements the `__set__` or `__delete__` methods is an overriding descriptor,\n",
    "because although it is a class attribute, that descriptor will override\n",
    "attempts to assign or delete the corresponding instance attribute.\n",
    "\n",
    "The terminology about descriptors varied accross the Python documentation and books over the years.\n",
    "I adopted the terminology of _Python in a Nutshell_.\n",
    "\n",
    "| current term | alternative terms | definition | examples |\n",
    "| :- | :- | :- | :- |\n",
    "| **overriding descriptor** | data descriptor, enforced descriptor | A descriptor with `__set__` or `__delete__` | validators |\n",
    "| **non-overriding descriptor** | non-data descriptor, shadowable descriptor | A descriptor with `__get__` only | methods, caches |\n",
    "\n",
    "\n",
    "**TIP**: Properties are also **overriding descriptors**:\n",
    "if you don’t decorate a setter function,\n",
    "the default __set__ from the property class will\n",
    "raise AttributeError to signal that the\n",
    "attribute is read-only."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b147b71-3d33-4612-a27d-e432c0334d15",
   "metadata": {},
   "source": [
    "## A non-overriding descriptor for caching"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "e3cc5f54-acdc-4723-a6cc-79d98f908e72",
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import sleep\n",
    "from random import randint\n",
    "\n",
    "class Cached:\n",
    "\n",
    "    def __set_name__(self, owner, name):\n",
    "        self.storage_name = name\n",
    "\n",
    "    def __get__(self, instance, owner):\n",
    "        value = instance.__dict__.get(self.storage_name, ...)\n",
    "        if value is ...:\n",
    "            sleep(5)  # pretend to do a lot of work\n",
    "            value = 42\n",
    "            setattr(instance, self.storage_name, value)\n",
    "            return value\n",
    "\n",
    "class Question:\n",
    "    answer = Cached()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "7220229b-952d-4c27-ab02-6c11f0a47832",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.06 ms, sys: 2.24 ms, total: 3.31 ms\n",
      "Wall time: 5 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "42"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "q = Question()\n",
    "q.answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "d355b5e6-66e8-4d05-ba4f-58f6f4761b1e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4 µs, sys: 1 µs, total: 5 µs\n",
      "Wall time: 8.11 µs\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "42"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "q.answer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8041bfc3-dfb8-4f79-acf7-1b5f83791ed8",
   "metadata": {},
   "source": [
    "`Cached` is a **non-overriding** descriptor.\n",
    "\n",
    "The `__get__` method sets the attribute named `self.storage_name`, which is `\"answer\"` in this example.\n",
    "\n",
    "When the attribute `answer` exists in the managed instance `__dict__`, its value is returned directly,\n",
    "_without using the descriptor logic_.\n",
    "\n",
    "That's why it's a **non-overriding** descriptor. It does not override the instance attribute with the same name.\n",
    "\n",
    "The `@functools.cached_property` decorator from the standard library\n",
    "produces a nonoverriding descriptor."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd54dee3-0484-4b02-8157-57aec35251a8",
   "metadata": {},
   "source": [
    "## There's also a `__delete__` special method\n",
    "\n",
    "If present, it makes an **overriding descriptor**. The `__delete__` method handles attempts to delete an attribute:\n",
    "\n",
    "```python\n",
    "del item.weight\n",
    "```\n",
    "\n",
    "`del` and `__delete__` are rarely used in practice, but if you need them, now you know they exist. 😉"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3603b705-ea55-4d5e-8a9b-46c3dbdf9573",
   "metadata": {},
   "source": [
    "## Functions are descriptors\n",
    "\n",
    "Every Python function has a `__get__` method.\n",
    "\n",
    "Recall the signature of `__get__`:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51e5ce43-8a4d-4d03-8048-0877eaae2b66",
   "metadata": {},
   "outputs": [],
   "source": [
    "import collections\n",
    "\n",
    "class Text(collections.UserString):\n",
    "    def __repr__(self):\n",
    "        return 'Text({!r})'.format(self.data)\n",
    "    def reverse(self):\n",
    "        return self[::-1]\n",
    "\n",
    "word = Text('forward')\n",
    "word"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1abcea01-e846-4426-88cc-10f95cdced5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "word.reverse()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a576a8c-5de1-4613-8cab-a5039a21f613",
   "metadata": {},
   "outputs": [],
   "source": [
    "Text.reverse(Text('backward'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "466116e1-15fa-460a-aecc-b188cfed130c",
   "metadata": {},
   "outputs": [],
   "source": [
    "Text.reverse, word.reverse"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1fab228b-2b48-4bc8-b1b4-64f87560524a",
   "metadata": {},
   "outputs": [],
   "source": [
    "type(Text.reverse), type(word.reverse)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c1f04d9-dacb-4c0d-91a8-ce6cec6ad943",
   "metadata": {},
   "outputs": [],
   "source": [
    "list(map(Text.reverse, ['repaid', (10, 20, 30), Text('stressed')]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2dfbf1f3-1751-4907-a56e-104fff98f1d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "word.reverse.__self__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3f18614a-bd78-4c24-a819-0193fe3127c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "word.reverse.__func__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "652ab079-4593-42ee-8acd-777a13ee9ece",
   "metadata": {},
   "source": [
    "## Descriptor Usage Tips\n",
    "\n",
    "### Use property to keep it simple\n",
    "\n",
    "The property built-in creates overriding descriptors implementing `__set__` and\n",
    "`__get__` even if you do not define a setter method. The default `__set__` of a\n",
    "property raises AttributeError: can't set attribute, so a property is the\n",
    "easiest way to create a read-only attribute, avoiding the issue described next.\n",
    "\n",
    "### Read-only descriptors require `__set__`\n",
    "\n",
    "If you use a descriptor class to implement a read-only attribute, you must\n",
    "remember to code both `__get__` and `__set__`, otherwise setting a namesake\n",
    "attribute on an instance will shadow the descriptor.\n",
    "\n",
    "The `__set__` method of a\n",
    "read-only attribute should just raise AttributeError with a suitable message.8\n",
    "\n",
    "### Validation descriptors can work with `__set__` only\n",
    "\n",
    "In a descriptor designed only for validation, the `__set__` method should check\n",
    "the value argument it gets, and if valid, set it directly in the instance `__dict__`\n",
    "using the descriptor instance name as key. That way, reading the attribute with\n",
    "the same name from the instance will be as fast as possible, because it will not\n",
    "require a `__get__`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8616570-faca-434e-82b0-60df07a025f1",
   "metadata": {},
   "source": [
    "### Caching can be done efficiently with `__get__` only\n",
    "\n",
    "If you code just the `__get__` method, you have a nonoverriding descriptor.\n",
    "These are useful to make some expensive computation and then\n",
    "cache the result by setting an attribute by the same name on the instance.\n",
    "The namesake instance attribute will shadow the descriptor,\n",
    "so subsequent access to that attribute will fetch it\n",
    "directly from the instance `__dict__` and not trigger the descriptor\n",
    "`__get__` anymore."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed3e0d90-1725-48f7-aa0f-dff60e2ccb79",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}