lesson1-intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\" style=\"margin-top: 1em;\"><ul class=\"toc-item\"><li><span><a href=\"#Intro-to-Pytorch\" data-toc-modified-id=\"Intro-to-Pytorch-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>Intro to Pytorch</a></span><ul class=\"toc-item\"><li><span><a href=\"#Pytorch-tensors\" data-toc-modified-id=\"Pytorch-tensors-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span>Pytorch tensors</a></span></li><li><span><a href=\"#Pytorch-Autograd\" data-toc-modified-id=\"Pytorch-Autograd-1.2\"><span class=\"toc-item-num\">1.2&nbsp;&nbsp;</span>Pytorch Autograd</a></span></li><li><span><a href=\"#torch.nn-module\" data-toc-modified-id=\"torch.nn-module-1.3\"><span class=\"toc-item-num\">1.3&nbsp;&nbsp;</span>torch.nn module</a></span></li></ul></li><li><span><a href=\"#Linear-Regression-with-Pytorch\" data-toc-modified-id=\"Linear-Regression-with-Pytorch-2\"><span class=\"toc-item-num\">2&nbsp;&nbsp;</span>Linear Regression with Pytorch</a></span><ul class=\"toc-item\"><li><span><a href=\"#Gradient-Descent-with-Pytorch\" data-toc-modified-id=\"Gradient-Descent-with-Pytorch-2.1\"><span class=\"toc-item-num\">2.1&nbsp;&nbsp;</span>Gradient Descent with Pytorch</a></span></li><li><span><a href=\"#Simplified-GD-Loop\" data-toc-modified-id=\"Simplified-GD-Loop-2.2\"><span class=\"toc-item-num\">2.2&nbsp;&nbsp;</span>Simplified GD Loop</a></span><ul class=\"toc-item\"><li><span><a href=\"#Models-in-Pytorch\" data-toc-modified-id=\"Models-in-Pytorch-2.2.1\"><span class=\"toc-item-num\">2.2.1&nbsp;&nbsp;</span>Models in Pytorch</a></span></li></ul></li></ul></li><li><span><a href=\"#Logistic-Regression\" data-toc-modified-id=\"Logistic-Regression-3\"><span class=\"toc-item-num\">3&nbsp;&nbsp;</span>Logistic Regression</a></span></li><li><span><a href=\"#Dataset-and-Data-loaders\" data-toc-modified-id=\"Dataset-and-Data-loaders-4\"><span class=\"toc-item-num\">4&nbsp;&nbsp;</span>Dataset and Data loaders</a></span></li><li><span><a href=\"#Two-layer-neural-network\" data-toc-modified-id=\"Two-layer-neural-network-5\"><span class=\"toc-item-num\">5&nbsp;&nbsp;</span>Two layer neural network</a></span><ul class=\"toc-item\"><li><span><a href=\"#Lab\" data-toc-modified-id=\"Lab-5.1\"><span class=\"toc-item-num\">5.1&nbsp;&nbsp;</span>Lab</a></span></li></ul></li><li><span><a href=\"#References\" data-toc-modified-id=\"References-6\"><span class=\"toc-item-num\">6&nbsp;&nbsp;</span>References</a></span></li></ul></div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import pytorch libraries\n",
    "%matplotlib inline\n",
    "import torch \n",
    "import torch.autograd as autograd \n",
    "import torch.nn as nn \n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intro to Pytorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch consists of 4 main packages:\n",
    "* torch: a general purpose array library similar to Numpy that can do computations on GPU\n",
    "* torch.autograd: a package for automatically obtaining gradients\n",
    "* torch.nn: a neural net library with common layers and cost functions\n",
    "* torch.optim: an optimization package with common optimization algorithms like SGD, Adam, etc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pytorch tensors\n",
    "Like Numpy tensors but can utilize GPUs to accelerate its numerical computations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create random tensor\n",
    "N = 5\n",
    "x = torch.randn(N, 10).type(torch.FloatTensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 1.4179,  0.7172, -2.0251,  0.8929, -0.3741, -0.5346, -1.5780,  0.5497,\n",
      "         -0.6771,  1.1097],\n",
      "        [-0.4169,  1.2200, -0.4406, -1.0797, -0.5540,  0.2036, -1.5355, -0.0012,\n",
      "          0.4200, -0.5164],\n",
      "        [-1.2314, -1.1224,  0.0531, -1.3469, -0.0044, -1.1588, -0.1957, -0.8979,\n",
      "          1.1346, -0.6168],\n",
      "        [-1.5271, -0.7603,  0.5204, -1.6306, -0.2804,  2.1417,  1.4689,  1.0748,\n",
      "          0.6299, -1.0543],\n",
      "        [-0.1975,  0.4705,  0.5607, -0.6967,  0.9124,  0.1098,  0.3245,  1.7451,\n",
      "          0.9873, -1.9232]])\n",
      "torch.Size([5, 10])\n"
     ]
    }
   ],
   "source": [
    "print(x)\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that tensor is simply but a multi dimensional matrix where first value in shape tells you number of rows and second tells you number of columns as shown above. The contents of a tensor can be accessed and modified using Python’s indexing and slicing notation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 1.4179,  0.7172, -2.0251,  0.8929, -0.3741, -0.5346, -1.5780,  0.5497,\n",
      "         -0.6771,  1.1097, -0.4169,  1.2200, -0.4406, -1.0797, -0.5540,  0.2036,\n",
      "         -1.5355, -0.0012,  0.4200, -0.5164, -1.2314, -1.1224,  0.0531, -1.3469,\n",
      "         -0.0044, -1.1588, -0.1957, -0.8979,  1.1346, -0.6168, -1.5271, -0.7603,\n",
      "          0.5204, -1.6306, -0.2804,  2.1417,  1.4689,  1.0748,  0.6299, -1.0543,\n",
      "         -0.1975,  0.4705,  0.5607, -0.6967,  0.9124,  0.1098,  0.3245,  1.7451,\n",
      "          0.9873, -1.9232]])\n",
      "torch.Size([1, 50])\n"
     ]
    }
   ],
   "source": [
    "# reshaping of tensors using .view()\n",
    "print(x.view(1,-1)) #-1 makes torch infer the second dim\n",
    "print(x.view(1,-1).shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note in above code block we are asking view to return only one row and hence we get above shape."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pytorch Autograd\n",
    "The autograd package in PyTorch provides classes and functions implementing automatic differentiation of arbitrary scalar valued function. For example, the gradient of the error with respect to all parameters.\n",
    "\n",
    "In order for this to happen we need to declare our paramerers as Tensors with the requires_grad=True keyword. Here is an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor([1., 2., 3., 4., 5., 6.], requires_grad=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(188., grad_fn=<SumBackward0>)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L = (2*x**2+1).sum() #Defining a function in terms of x\n",
    "L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "L.backward() # computes the grad of L with respect to x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 4.,  8., 12., 16., 20., 24.])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.grad #Stores the gradient values of x with respect to function L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 8., 16., 24., 32., 40., 48.])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L = (2*x**2+1).sum()\n",
    "L.backward()\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that when we run the above code again we get twice the gradient values of x. This is because x.grads keeps adding the gradient values. We should initialize it to zero to avoid such issues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 4.,  8., 12., 16., 20., 24.])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L = (2*x**2+1).sum()\n",
    "x.grad.data.zero_()\n",
    "L.backward()\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## torch.nn module\n",
    "A neural net library with common layers and cost functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`nn.Linear(5, 3)` creates a linear transformation ($A\\cdot X+b$) of a $N \\times 5$ matrix into a $N \\times 3$ matrix, where N can be anything (number of observations)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "D = 5 # number of input featutes\n",
    "M = 3 # neurons in the first hidden layer\n",
    "linear_map = nn.Linear(D, M)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Simply but we are creating a neural network layers which has 5 neurons at input and 3 neurons at output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Parameter containing:\n",
       " tensor([[-0.4119, -0.1948, -0.3511,  0.3839,  0.2299],\n",
       "         [ 0.2582,  0.0336, -0.3060,  0.1271,  0.0833],\n",
       "         [ 0.0656,  0.1023, -0.2738,  0.2783,  0.2815]]), Parameter containing:\n",
       " tensor([-0.2190, -0.3817,  0.3718])]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# parameters are initialized randomly\n",
    "[p for p in linear_map.parameters()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we have 5X3 nn layer we have 15+3 trainable parameters as shown above. Biases are accounted seperately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[torch.Size([3, 5]), torch.Size([3])]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[p.shape for p in linear_map.parameters()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note the shape is [3,5] i.e mapping form output layer to input layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Linear Regression with Pytorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The goal of linear regression is to fit a line to a set of points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Here we generate some fake data\n",
    "def lin(a,b,x): return a*x+b\n",
    "\n",
    "def gen_fake_data(n, a, b):\n",
    "    x = np.random.uniform(0,1,n) \n",
    "    y = lin(a,b,x) + 0.1 * np.random.normal(0,3,n)\n",
    "    return x, y\n",
    "\n",
    "x, y = gen_fake_data(50, 3., 8.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAFn5JREFUeJzt3XuQpFd53/HvTytZFwS2pB2MEFoWxbILlUjZuCUDIUREFCWrXFLAEKQUhaAkNhAuNs6lSOUChXMB4pSrHLsC60iFSGJZBDtmA4oF4WIc26J2xHWFimhZhLRIWGOWiGxx00pP/uheNBr17NszO2+/b3d/P1VT05cz08/RrPrpc85zzpuqQpKkYzmh6wAkSf1nspAkNTJZSJIamSwkSY1MFpKkRiYLSVIjk4UkqZHJQpLUyGQhSWp0YtcBbJXt27fXzp07uw5DkmbK7bff/ldVtdTUbm6Sxc6dO1leXu46DEmaKUm+Pkk7p6EkSY1MFpKkRiYLSVIjk4UkqZHJQpLUyGQhSWpkspAkNTJZSFJPHVg5zM177+HAyuGuQ5mfTXmSNE8OrBzml/7D/6YKEvjwm57PeUundxaPIwtJ6qG9dx+iCr730MNUDe93yWQhST100c4zSeDUk7aRDO93yWkoSeqh85ZO58Nvej577z7ERTvP7HQKCkwWktRb5y2d3nmSOMppKElSI5OFJKmRyUKS1MhkIUlqZLKQJDUyWUiSGpksJEmNTBaSpEYmC0lSo9aSRZIbkjyQZN+qx16e5I4kjyQZHONnL0vylST7k7y1rRglSZNpc2TxPuCyNY/tA14KfHq9H0qyDfgd4BeBC4Crk1zQUoySpAm0liyq6tPAoTWP3VlVX2n40YuB/VV1oKp+CPw+cGVLYUqSJtDHNYtzgHtX3T84ekyS1JE+JouMeazGNkx2JVlOsryystJyWJK0uPqYLA4C5666/zTgvnENq2p3VQ2qarC0tDSV4CS1q0/Xndaj+ng9i73A+UmeAXwDuAr4e92GJGka+nbdaT2qzdLZm4C/AH4mycEk1yZ5SZKDwHOBjyS5ddT2qUluAaiqI8AbgVuBO4EPVNUdbcUpqT/6dt1pPaq1kUVVXb3OU/99TNv7gMtX3b8FuKWl0CT1VN+uO61H9XEaStKC6tt1p/Uok4WkXunTdaf1qD5WQ0mSesZkIUlqZLKQJDUyWUiSGpksJGkVd5CPZzWUJI24g3x9jiwkacQd5OszWUjSiDvI1+c0lCSNuIN8fSYLSVrFHeTjOQ0lqVWzUF00CzF2zZGFpNbMQnXRLMS4ngMrh6c2ZWaykNSa1dVFp560jb13H+rdG/EsxDjOtJOc01CSWjML1UWzEOM40y7zdWQhqTWzUF00CzGOM+0kl6pq9QWmZTAY1PLyctdhSNLUbMWaRZLbq2rQ1M6RhSTNqGmW+bpmIUlq1FqySHJDkgeS7Fv12JlJPpbkrtH3M9b52YeTfH70taetGCVJk2lzZPE+4LI1j70V+HhVnQ98fHR/nO9V1c+Ovq5oMUZJ0gRaSxZV9WlgbS3XlcCNo9s3An+nrdeXJG2daa9Z/GRV3Q8w+v7kddqdkmQ5yW1JTCiS1LG+VkPtqKr7kpwHfCLJl6rqq2sbJdkF7ALYsWPHtGOUpIUx7ZHFXyY5G2D0/YFxjarqvtH3A8CngJ9bp93uqhpU1WBpaamdiCVJU08We4BrRrevAT60tkGSM5KcPLq9HfgbwJenFqEk4Um0a7U2DZXkJuASYHuSg8DbgHcCH0hyLXAP8PJR2wHwuqq6Dngm8N4kjzBMZu+sKpOFpKmZ5ZNo29Jasqiqq9d56tIxbZeB60a3/xx4VltxSeqvaR65fSyzehJtm/q6wC1pwfTp0/ysnkTbJpOFpF7o06f5WT2Jtk0mC0m90LdP816L+7FMFpJ6wU/z/WaykNQbfprvL48ol3Rc3I+wGBxZSNq0PlUwqV2OLCRt2uoKpqrhfc0nk4WkTetbBZPa4zSUpE2zgmlxmCwkHRcrmBaD01CSpEYmC0m9ZVlufzgNJWli0zwV1rLcfjFZSJrItN+8+3SwoJyGkjShae+psCy3XxxZSJrItN68V091WZbbHyYLSROZxp6KcVNdr7hox5a/jjbOZCEJmGzxuu09FePWKY4+7uiiWyYLSb2pPFo71fWUJ53Si7jU4gJ3khuSPJBk36rHzkzysSR3jb6fsc7PXjNqc1eSa9qKUdJQXw4EPDrV9fYrLuDDb3o+3/zO93sRl9qthnofcNmax94KfLyqzgc+Prr/GEnOBN4G/AJwMfC29ZKKpK3Rp8qj85ZO5xUX7eC8pdN7FdeiS1W198uTncCHq+rC0f2vAJdU1f1JzgY+VVU/s+Znrh61+fuj++8dtbvpWK81GAxqeXm5hV5I82W9tYlpbrjbiL7GNS+S3F5Vg6Z2016z+Mmquh9glDCePKbNOcC9q+4fHD0m6Tgda22irwcC9jWuRdPHTXkZ89jY4U+SXUmWkyyvrKy0HJY0+/qyNqHZM+1k8Zej6SdG3x8Y0+YgcO6q+08D7hv3y6pqd1UNqmqwtLS05cFKXWrjED3XALRZ056G2gNcA7xz9P1DY9rcCvybVYvaLwb+6XTCk/qhrVJWL1akzWotWSS5CbgE2J7kIMMKp3cCH0hyLXAP8PJR2wHwuqq6rqoOJfl1YO/oV72jqhwra6G0eYieawDajNaSRVVdvc5Tl45puwxct+r+DcANLYUm9Z7TReobd3BLPeR0kfrGZCH1lNNF6pM+ls5KknrGZCHNIa9dra3mNJQ0Z/pygqzmiyMLac5MskvbkYc2ypGFNGeaym4deWgzTBbSnGkqu21zw5/ml8lCmkPHKrt1w582w2QhLRg3/GkzTBbSHJn0QkFu+NNGmSykKWn7im+TLlx75TlthslCmoJpVCBNsnBtJZQ2y30W0hRM4wp1kyxce6U8bZYjC03VIk6BHFg5zMr/+wFFtVqBNMnCtZVQ2iyThaZmEadAVvcZ4A0v/Gtc/qyzH9PvrUygTQvXVkJpsxqTRZI3Av+1qr49hXg0xxZxM9jaPi898eTHJYppJ9C1CWURR3vauElGFk8B9ib5LMOr191adfRzkjS5RZwCaerz3rsP8fAjxQ+OPMLJJ54w9QS6iKM9bU5jsqiqf57kXwAvBl4D/HaSDwDXV9VX2w5Q86PvUyBtfMJu6vNTnnQKPzjyCAA/OPIIT3nSKVvyuuOM698ijva0OROtWVRVJfkm8E3gCHAG8MEkH6uqf9JmgJovfd0M1uYn7GP1+Zvf+T4nn3jCj0YW3/zO97fkNddar3+LONrT5jSWziZ5c5LbgXcDfwY8q6peD/w88MubedEkv5JkX5I7kvzqmOcvSfJgks+Pvv7lZl5HmlRXJaUX7TyTbSeEU0/axrYT0tqb9Xr9OzryefsVFzgFpWOaZGSxHXhpVX199YNV9UiSX9roCya5EHgtcDHwQ+CPk3ykqu5a0/RPq2rDv1/ajK4+YU9rau5Y/evraE/9Msmaxbqf6qvqzk285jOB26rquwBJ/gR4CcORi9SJLtdTpvFm3ff1IvVfF/ss9gH/OslZwPeAy4HlMe2em+QLwH3AP6qqO6YYoxbQJG/as1xm6ghCx2PqyaKq7kzyLuBjwGHgCwwXzVf7LPD0qjqc5HLgj4Dz1/6uJLuAXQA7duxoNW7JMlMtsk7Ohqqq66vq2VX1AuAQcNea579TVYdHt28BTkqyfczv2V1Vg6oaLC0tTSV2LS7PVdIi6yRZJHny6PsO4KXATWuef0qSjG5fzDDOb007Tmk1y0y1yLo6G+oPRmsWDwFvqKpvJ3kdQFW9B3gZ8PokRxiua1zlrnF1zUViLbLMy3vwYDCo5eVx6+SSpPUkub2qBk3tvJ6FJKmRyUKS1MhkIUlqZLKQJDUyWUiSGpksJEmNTBZSSw6sHObmvfdwYOVw16FIx62rTXnSXPMcKc0bRxZSCzxHSvPGZCG1wHOkNG+chpJa4DlSmjcmC6klXmxI88RpKElSI5OFWmX5qDQfnIZSaywfleaHIwu1xvJRaX6YLNQay0el+eE0lFqzleWjB1YOW4YqdchkoVZtRfmoax9S95yGUu+59iF1r5NkkeRXkuxLckeSXx3zfJL8VpL9Sb6Y5NldxKl+cO1D6t7Up6GSXAi8FrgY+CHwx0k+UlV3rWr2i8D5o69fAP7j6LsWkEdnSN3rYs3imcBtVfVdgCR/ArwEePeqNlcC76+qAm5L8hNJzq6q+6cfrvrAozOkbnUxDbUPeEGSs5KcBlwOnLumzTnAvavuHxw9JknqwNRHFlV1Z5J3AR8DDgNfAI6saZZxP7r2gSS7gF0AO3bs2OJIJUlHdbLAXVXXV9Wzq+oFwCHgrjVNDvLY0cbTgPvG/J7dVTWoqsHS0lJ7AUvSguuqGurJo+87gJcCN61psgd41agq6jnAg65XaBF5EKP6oqtNeX+Q5CzgIeANVfXtJK8DqKr3ALcwXMvYD3wXeE1HcS4Md0j3j5sR1SedJIuq+ptjHnvPqtsFvGGqQS0w35T6afVmxFNP2sbeuw/5d1Fn3MEtd0j3lJsR1SeeDSXflHrKzYjqE5OFfFPqMTcjqi9MFgKa35RcAJcWm8lCjVwAl+QCtxq5AC7JZLHAJt3w5QK4JKehFtRGppZcAJdkslhQG93wZVWOtNichlpQTi1J2ghHFgvKqSVJG2GyWGBOLUmalNNQ2jCPzZYWjyMLbYgb9KTF5MhCG+IGPWkxmSy0IVZRSYvJaShtiFVU0mIyWWjDrKKSFo/TUJKkRiaLOWRpq6St5jTUnLG0VVIbOhlZJHlLkjuS7EtyU5JT1jz/6iQrST4/+rquizhnkaWtktow9WSR5BzgzcCgqi4EtgFXjWl6c1X97OjrP001yBlmaaukNnQ1DXUicGqSh4DTgPs6imPuWNoqqQ1TTxZV9Y0kvwHcA3wP+GhVfXRM019O8gLg/wBvqap71zZIsgvYBbBjx44Wo54tlrZK2mpdTEOdAVwJPAN4KvCEJK9c0+x/ADur6q8D/wu4cdzvqqrdVTWoqsHS0lKbYWtGWAkmtaOLaagXAV+rqhWAJH8IPA/4L0cbVNW3VrX/XeBdU41QM8lKMKk9XVRD3QM8J8lpSQJcCty5ukGSs1fdvWLt89I4VoJJ7elizeIzST4IfBY4AnwO2J3kHcByVe0B3pzkitHzh4BXTztOzR4rwaT2pKq6jmFLDAaDWl5e7joMdezAymErwaQNSHJ7VQ2a2rmDW3PFSjCpHZ4NNWf6Xg3U9/gkjefIYo70vRqo7/FJWp8jiznS92qgvscnaX0mi57Z6DTN6vZ9rwbqe3yS1mc1VI9sdJpmXHug19VAVitJ/WI11AxaPU1z6knb2Hv3oR+9oY57kx3X/hUX7ej1m7DVStJsMln0yHrTNOuNODYzreMne0mbYbLokfWOF19vxLGR48gPrBzmli/dz29/cj8hViNJ2hCTRc+Mm6Y51ghikmmdoyOThx5+hIceHq5RrZ3mkqRjMVnMgOO9oNHRkcnRRHHStliNJGlDTBYz4ngWhlePTIrijS/8KS5/1tmOKiRNzGSxALzUqqTjZbJYEJasSjoe7uDWxDwEUFpcjizm1Fbvp/AQQGmxmSxG5mmzWhtv7MfaXS5p/pksmL9PzW28sXsIoLTYTBbM36fmNt7YraiSFpvJgvn71NzWG7sVVdLi6iRZJHkLcB1QwJeA11TV91c9fzLwfuDngW8Br6iqu9uKZx4/NfvGLmkrTb10Nsk5wJuBQVVdCGwDrlrT7Frg21X1U8BvAu9qO67zlk7v/fHektSVrvZZnAicmuRE4DTgvjXPXwncOLr9QeDSJJlifJKkVaaeLKrqG8BvAPcA9wMPVtVH1zQ7B7h31P4I8CBw1trflWRXkuUkyysrK+0GLkkLrItpqDMYjhyeATwVeEKSV65tNuZHH3f916raXVWDqhosLS1tfbCSJKCbaagXAV+rqpWqegj4Q+B5a9ocBM4FGE1V/ThwaKpRSpJ+pItkcQ/wnCSnjdYhLgXuXNNmD3DN6PbLgE9U1eNGFpKk6ehizeIzDBetP8uwbPYEYHeSdyS5YtTseuCsJPuBXwPe2nZcHpInSevLvHxgHwwGtby8vKmfnbfjPiRpUklur6pBUzuPKOexx31UDe9Lkh5lsmD+jvuQpK3m2VDM53EfkrSVTBYjnqUkSetzGkqS1MhkIUlqZLKQJDUyWUiSGpksJEmNTBaSpEZzc9xHkhXg68fxK7YDf7VF4cwK+7wY7PNi2Gyfn15Vjdd4mJtkcbySLE9yPso8sc+LwT4vhrb77DSUJKmRyUKS1Mhk8ajdXQfQAfu8GOzzYmi1z65ZSJIaObKQJDVaqGSR5LIkX0myP8njLtWa5OQkN4+e/0ySndOPcmtN0OdfS/LlJF9M8vEkT+8izq3W1O9V7V6WpJLMfOXMJH1O8ndHf+87kvzetGPcahP8+96R5JNJPjf6N355F3FulSQ3JHkgyb51nk+S3xr99/hikmdv2YtX1UJ8AduArwLnAT8GfAG4YE2bfwC8Z3T7KuDmruOeQp9fCJw2uv36We/zpP0etXsi8GngNmDQddxT+FufD3wOOGN0/8ldxz2FPu8GXj+6fQFwd9dxH2efXwA8G9i3zvOXA/8TCPAc4DNb9dqLNLK4GNhfVQeq6ofA7wNXrmlzJXDj6PYHgUuTZIoxbrXGPlfVJ6vqu6O7twFPm3KMbZjkbw3w68C7ge9PM7iWTNLn1wK/U1XfBqiqB6Yc41abpM8FPGl0+8eB+6YY35arqk8Dx7ru85XA+2voNuAnkpy9Fa+9SMniHODeVfcPjh4b26aqjgAPAmdNJbp2TNLn1a5l+Klk1jX2O8nPAedW1YenGViLJvlb/zTw00n+LMltSS6bWnTtmKTPbwdemeQgcAvwpumE1pmN/j8/sUW6Ut64EcLaUrBJ2sySifuT5JXAAPhbrUY0Hcfsd5ITgN8EXj2tgKZgkr/1iQynoi5hOIL80yQXVtX/bTm2tkzS56uB91XVv0/yXOA/j/r8SPvhdaK197BFGlkcBM5ddf9pPH5I+qM2SU5kOGw91pCv7ybpM0leBPwz4Iqq+sGUYmtTU7+fCFwIfCrJ3QzndvfM+CL3pP++P1RVD1XV14CvMEwes2qSPl8LfACgqv4COIXhGUrzaqL/5zdjkZLFXuD8JM9I8mMMF7D3rGmzB7hmdPtlwCdqtGo0oxr7PJqOeS/DRDHrc9hHHbPfVfVgVW2vqp1VtZPhWs0VVbXcTbhbYpJ/33/EsKCBJNsZTksdmGqUW2uSPt8DXAqQ5JkMk8XKVKOcrj3Aq0ZVUc8BHqyq+7fiFy/MNFRVHUnyRuBWhlUUN1TVHUneASxX1R7geobD1P0MRxRXdRfx8Zuwz/8OOB34b6O1/Huq6orOgt4CE/Z7rkzY51uBFyf5MvAw8I+r6lvdRX18JuzzPwR+N8lbGE7HvHqWPwAmuYnhNOL20TrM24CTAKrqPQzXZS4H9gPfBV6zZa89w//dJElTskjTUJKkTTJZSJIamSwkSY1MFpKkRiYLSVIjk4UkqZHJQpLUyGQhtSTJRaNrCpyS5Amja0hc2HVc0ma4KU9qUZJ/xfCIiVOBg1X1bzsOSdoUk4XUotGZRXsZXjPjeVX1cMchSZviNJTUrjMZnr31RIYjDGkmObKQWpRkD8MruD0DOLuq3thxSNKmLMyps9K0JXkVcKSqfi/JNuDPk/ztqvpE17FJG+XIQpLUyDULSVIjk4UkqZHJQpLUyGQhSWpkspAkNTJZSJIamSwkSY1MFpKkRv8f9ogj7D9fercAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.scatter(x,y, s=8); plt.xlabel(\"x\"); plt.ylabel(\"y\"); "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You want to find **parameters** (weights) $a$ and $b$ such that you minimize the *error* between the points and the line $a\\cdot x + b$. Note that here $a$ and $b$ are unknown. For a regression problem the most common *error function* or *loss function* is the **mean squared error** ($\\sum_i (\\hat{y}_i - y_i)^2$). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def mse(y_hat, y): return ((y_hat - y) ** 2).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppose we believe $a = 10$ and $b = 5$ then we can compute `y_hat` which is our *prediction* and then compute our error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.2031367785906553"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_hat = lin(10,5,x)\n",
    "mse(y_hat, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "def mse_loss(a, b, x, y): return mse(lin(a,b,x), y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.2031367785906553"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mse_loss(10, 5, x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So far we have specified the *model* (linear regression) and the *evaluation criteria* (or *loss function*). Now we need to handle *optimization*; that is, how do we find the best values for $a$ and $b$? How do we find the best *fitting* linear regression."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient Descent with Pytorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a fixed dataset $x$ and $y$ `mse_loss(a,b)` is a function of $a$ and $b$. We would like to find the values of $a$ and $b$ that minimize that function.\n",
    "\n",
    "**Gradient descent** is an algorithm that minimizes functions. Given a function defined by a set of parameters, gradient descent starts with an initial set of parameter values and iteratively moves toward a set of parameter values that minimize the function. This iterative minimization is achieved by taking steps in the negative direction of the function gradient.\n",
    "\n",
    "Here is gradient descent implemented in [PyTorch](http://pytorch.org/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((10000,), (10000,))"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# generate some more data\n",
    "x, y = gen_fake_data(10000, 3., 8.)\n",
    "x.shape, y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([10000]) torch.Size([10000])\n"
     ]
    }
   ],
   "source": [
    "# Wrap x and y as tensor \n",
    "x = torch.tensor(x)\n",
    "y = torch.tensor(y)\n",
    "print(x.shape,y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([-0.9907], dtype=torch.float64, requires_grad=True),\n",
       " tensor([0.1186], dtype=torch.float64, requires_grad=True))"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create random Tensors for weights, and wrap them in tensors.\n",
    "# Setting requires_grad=True indicates that we want to compute gradients with\n",
    "# respect to these tensors during the backward pass.\n",
    "a, b = np.random.randn(1), np.random.randn(1)\n",
    "a = torch.tensor(a, requires_grad=True)\n",
    "b = torch.tensor(b, requires_grad=True)\n",
    "a,b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "99.38503008138994\n",
      "0.7008584326902876\n",
      "0.09615343021622287\n",
      "0.09192764321948345\n",
      "0.0914983814607665\n",
      "0.0911855151290231\n",
      "0.0909450057588368\n",
      "0.09076003868475682\n",
      "0.09061778669203424\n",
      "0.09050838543728189\n"
     ]
    }
   ],
   "source": [
    "learning_rate = 1e-3\n",
    "for t in range(10000):\n",
    "    # Forward pass: compute predicted y using operations on Variables\n",
    "    loss = mse_loss(a,b,x,y) #since a,b,x,y are tensor so is loss now\n",
    "    if t % 1000 == 0: print(loss.item())\n",
    "    \n",
    "    # Computes the gradient of loss with respect to all Variables with requires_grad=True.\n",
    "    # After this call a.grad and b.grad will be Variables holding the gradient\n",
    "    # of the loss with respect to a and b respectively\n",
    "    loss.backward()\n",
    "    \n",
    "    # Update a and b using gradient descent; a.data and b.data are Tensors,\n",
    "    # a.grad and b.grad are Variables and a.grad.data and b.grad.data are Tensors\n",
    "    a.data -= learning_rate * a.grad.data\n",
    "    b.data -= learning_rate * b.grad.data\n",
    "    \n",
    "    # Zero the gradients\n",
    "    a.grad.data.zero_()\n",
    "    b.grad.data.zero_()    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([3.0500], dtype=torch.float64, requires_grad=True) tensor([7.9760], dtype=torch.float64, requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "print(a,b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that gradient decent helped us predict a and b values with good precision."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simplified GD Loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Linear(in_features=1, out_features=1, bias=True)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# linear tranformation with input dimension=1 and output dimension=1\n",
    "nn.Linear(1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Models in Pytorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Linear(in_features=1, out_features=1, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# simple way of specifying a linear regression model\n",
    "model = torch.nn.Sequential(\n",
    "    nn.Linear(1, 1),\n",
    ")\n",
    "model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "# equivalent way of specifiying the same model\n",
    "class LinearRegression(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(LinearRegression, self).__init__()\n",
    "        self.lin = nn.Linear(1, 1)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = self.lin(x)\n",
    "        return x \n",
    "model =  LinearRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Parameter containing:\n",
      "tensor([[0.7967]], requires_grad=True), Parameter containing:\n",
      "tensor([0.8044], requires_grad=True)]\n"
     ]
    }
   ],
   "source": [
    "# note here we have just two parameters, why?\n",
    "print([p for p in model.parameters()])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([10000])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([0.6942, 0.1108, 0.0223,  ..., 0.2303, 0.7408, 0.3185])"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x, y = gen_fake_data(10000, 3., 8.)\n",
    "x = torch.tensor(x).float()\n",
    "y = torch.tensor(y).float()\n",
    "print(x.shape)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([10000, 1])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([[0.6942],\n",
       "        [0.1108],\n",
       "        [0.0223],\n",
       "        ...,\n",
       "        [0.2303],\n",
       "        [0.7408],\n",
       "        [0.3185]])"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# you have to be careful with the dimensions that your model is expecting\n",
    "x1 = torch.unsqueeze(x, 1)\n",
    "print(x1.shape)\n",
    "x1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1.3575],\n",
      "        [0.8927],\n",
      "        [0.8222],\n",
      "        ...,\n",
      "        [0.9879],\n",
      "        [1.3946],\n",
      "        [1.0582]], grad_fn=<ThAddmmBackward>)\n"
     ]
    }
   ],
   "source": [
    "y_hat = model(x1)\n",
    "print(y_hat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the optim package to define an Optimizer that will update the weights of\n",
    "# the model for us. Here we will use Adam\n",
    "learning_rate = 0.1\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "69.30413818359375\n",
      "0.08890427649021149\n",
      "0.0889040008187294\n",
      "0.0889039933681488\n",
      "0.0889040008187294\n",
      "0.0889040008187294\n",
      "0.0889040008187294\n",
      "0.0889039933681488\n",
      "0.0889039933681488\n",
      "0.0889039933681488\n"
     ]
    }
   ],
   "source": [
    "for t in range(10000):\n",
    "    # Forward pass: compute predicted y using operations on Variables\n",
    "    y_hat = model(x1)\n",
    "    loss = F.mse_loss(y_hat, y.unsqueeze(1))\n",
    "    if t % 1000 == 0: print(loss.item())\n",
    "       \n",
    "    # Before the backward pass, use the optimizer object to zero all of the\n",
    "    # gradients for the variables\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    \n",
    "    # Calling the step function on an Optimizer makes an update to its\n",
    "    # parameters\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Parameter containing:\n",
      "tensor([[3.0145]], requires_grad=True), Parameter containing:\n",
      "tensor([7.9916], requires_grad=True)]\n"
     ]
    }
   ],
   "source": [
    "print([p for p in model.parameters()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generating fake data\n",
    "# Here we generate some fake data\n",
    "def lin(a,b,x): return a*x+b\n",
    "\n",
    "def gen_logistic_fake_data(n, a, b):\n",
    "    x = np.random.uniform(-20,20, (n, 2))\n",
    "    x2_hat = lin(a,b, x[:,0])\n",
    "    y = x[:,1] > x2_hat\n",
    "    return x, y.astype(int)\n",
    "\n",
    "x, y = gen_logistic_fake_data(100, 1., 0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0,\n",
       "       1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,\n",
       "       0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0,\n",
       "       1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 1, 0, 1,\n",
       "       0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0])"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f263c154128>]"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEKCAYAAAAMzhLIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VNXWx/HvmslkUmgBQhEVELGgKAoqYO9gQ7GhXvUVFVHEdr323sXelXvtBWxgwd7BhoKCgHilK9JCTUidst4/zsCNGkJCZmafJOvzPHmYOTNzzs9xkjVn77P3FlXFGGOM2ZCA6wDGGGP8zQqFMcaYalmhMMYYUy0rFMYYY6plhcIYY0y1rFAYY4yplhUKY4wx1bJCYYwxplpWKIwxxlQrw3WAZGjdurV26tTJdQxjjKlXJk+evFxV8zf2vAZRKDp16sSkSZNcxzDGmHpFRBbU5HnW9GSMMaZaViiMMcZUywqFMcaYalmhMMYYUy0rFMYYY6plhcIYY0y1nBUKEdlCRD4TkZkiMkNELkxsbykiH4nIrMS/ea4yGmOMcXtGEQX+qarbA72BYSLSDbgC+ERVuwKfJO4bY4ypTBV+/jkth3JWKFR1sar+kLhdBMwEOgADgGcTT3sWONpNQmOM8anZs+GAA2CPPWDRopQfzhd9FCLSCdgFmAi0VdXF4BUToI27ZMYY4yOxGIwYAd27w48/wr33Qrt2KT+s80IhIk2A14GLVLWwFq8bIiKTRGRSQUFB6gImaHQBWvIyGvk15ccyxpgqBQLwySfQr5/X7HT22d62VB825UeohoiE8IrEi6o6JrF5qYi0TzzeHlhW1WtVdaSq9lLVXvn5G53Tqk40uhBdcTRaeCu64ng0MjOlx6strfgOLX4ajf7mOooxJtlKS+Hqq2H+fBCBsWNhzBjYbLO0RXB51ZMATwIzVfXeSg+9BZyeuH068Ga6s/1NZAooQBkQg4rvHQf6Hy2fgK48Cy262ytm8ZWuI5k60oofiK84mfjqf6LxGp9km4boiy9g553httvg7be9bTk5XsFII5dnFHsCpwIHiMiUxM9hwB3AwSIyCzg4cd+tzF1BAMkGgpC5h+tE62nFRLwCFvE2RGa5jGPqSLUCXTUYIpOg7H208BbXkYwLa9bAOefAfvt5/RIffwzDhzuL42yacVX9Eu/Pb1UOTGeWjZHgZtDqLaj4DjJ7IBldXEdaT8IHosXPgQS9Qhbq5jqSqQuNgFYk7kQgvtRpHOPILbfAf/4Dl14KN97onUU4JKrqNEAy9OrVSxvzehQanQvRWZC5OxKw8Yn1XbzoXij+N0gWkvcMkrmz60gmHZYsgVWrYPvtYfVq7xLYXr1SekgRmayqGz2IFQpjfEi1FAgh0iDWFjPVUYVnnoF//hO23homTkxbH0RNC4Xzy2ONMX8nkm1FojGYOxcOOQQGD4YddoDnn097R3VN2CfRGLNBqjFEgq5jNEwTJ3qjq4NBePRRr/M6DWMiNoU/UxljnFJV4qsvR5d2I15wEBqzTvWkKS31/t11V2/A3IwZcO65tS4SZSXlfPTcF3w7bjKp7kKwQmGM+bvoTCh/H1CI/YEWP+M6kVMa+wMteaVug23LyuCaa2C77bzO6lAI7r8ftthik3Z32UE38eCwf3PrSffx3I2vbHquGrBCYUwDpRU/Ei84kHjBIWhkRu1eLE28TlYAMiDQIun56guNFaDLj0ILb0FXnIhWTK39Tr78EnbZBW691RsbUcczgFg0xi8Tf6WsuJyy4nK+GvNdnfa3MVYojGmgdPUFEPsdYvPR1RfX6rWSsSU0uwGCXSH7CCT3/1KSsSZUI2h8ZcqbVzYoMg2Is35ga8U3NX9tRQUMGwZ77+01Ob3/Pjz7LOTV7TL2YEaQbn23JSs3TFZumL2P612n/W2MdWYb02DFNnDbGwHuXX674StsAjkDIWdgaqLVkEbnoytOBF0L4X2hxcOIpPn7bag7EPAGtKpCZp9avDYECxbARRfBzTdDkyZJizXio+v4csxEmrZsQq9DeyRtv1WxQmFMAyXN70XXXAIEkeZ3AV4ntRZeA6WvQ6A1tByFZGxaG3k6aPEzoKsBhYqvIPorhLZLawYJ5kPrt6H8awh1R0LbVv+CZcvg8svh+uuhUyd4803vyqYky8zK5ICT9076fqtiTU/GNFAS7k2gzdcE2kxAMnf1NsbmQ+nbQBziy9HikS4jblywAxD2bqtCoKWTGBLcDMk5rvoioeqNg+jWDV58Eb5JNFGloEikm51R1JCqoiWjIDoDyTkRCe3kOpIxtSe5JKZCBkIQaOUyzUZJ7v95MyJHZiBNzkSCyVvHTDUGZe+CFkP2AESyN31nCxbA0KFeH0SfPt48Td0azrxrVihqSEtGQ9GdQCla9g7kf4o4+nZjzKaSYBu0+d1Q/DiEtkWaDHUdqVoiIaTZ5SnZtxbeDKVjAYXSt5BWL236zu6+GyZMgAcfhPPOaxBnEZVZoaip6C9A6f/uxxY7Ow32K40tRtdcDVqMNLsesZlsfSmQfShkH+o6hnsVE1j/Ox35sfavnzEDolFvvYhbbvFmeu3YMakR/cL6KGpIck70TtslB4JbQ8Y2AKiWJSZwM7r6Uqj4GiI/oqvOch3HmOplHeVdySQ5EN6v5q8rL4cbbvDGRVycuOy4efMGWyTAzihqTELdIP8z70wioysiGcRLx8Ea77RYm93sXU7YmMVX4V1vDsSLnEYxZmOkyQXeImRaAuF9avaib76Bs87y1qs+5RS4777UhvQJ12tmPyUiy0RkeqVtN4jIH39Z9c4XJNACCW3/v1k9i27DW1kuAkXuF+JzTZpd5307IwOaXec6jjHVEhEk3BvJOqBmM/V+9BHsuScUFcE778ALL0B+fuqD+oDrM4pngIeB5/6y/T5VvTv9cWop0AbiKwGFQOP4wFRHwr2hzQ+A2oyjpuFYsQJatfKm3rjlFm9J0qZNXadKK6dnFKo6HljpMkNdSN6jEN4fwvsjeY+7juMLIgErEqZhWL4cTj0Vunf/3yR+V13V6IoE+Lcz+3wR+SnRNFXlpCgiMkREJonIpIKCgnTn8zIENyOQ9xiBvMd9PbrVGFMLqjBqlDcOYvRor08iuw5jLBoAPxaKx4AuQA9gMXBPVU9S1ZGq2ktVe+U3knbCVFBVtOxTtGQ0Gi90HWejVMuJF95MfMVpaPnXruOYhqa4GI48Ek4+GTp3hh9+gJtugnDYdTKnXPdR/I2qrl8hRUT+DYxzGKde03ghWvw4aBRpMrTKAYJa/CSsfQhQKH4aWr+X/knXakHXPgIlrwDl6Kop0GY80oinwDZJlpPj/dx7L1xwQYMbOLepfPcXQUTaV7p7DDB9Q8811dPVF0Dxs1DyArry7KqfVP4J3qCjMm9KavX5WUXsD6AicUehHpwFGZ/75Rfo1w/mzfPWq375ZW98hBWJ9VxfHjsK+AbYVkQWisiZwAgRmSYiPwH7A7WbSN/8T3QW3uW7UYjNrfo5WUcBiUFHGduANE9jwNqT3KEgzYAMyOoHQesbMpuoosK7imnnneG772DWLG97NVOv10Y0EmXx3KVEKiJJ2Z9LTpueVPWkKjY/mfYgDVX4cCh9EQhAzuAqnxLIPQkNdYV4AYT3q3Z9Aj+QUFdo8zVoCRLwd1EzPvb993DmmTBtGpxwgjdHU9u2Sdt98Zpizut1BSsWr6R562Y8NnkEzVrV36ulfNf0ZJIjXvQglL4MBCF8AIGmwzf4XMnshWT1r9vsmWkkErIiYerm6ae98RFvvuk1NSWxSAB8O+4HVi5ZRXlJBWuWF/LlmIlJ3X+6WaFoqEpfYX3fQ/nnjsMY4wMffwyTJ3u377zTm4bjqKNScqj2W7WptCy20L5LcgtRulmhaKhCPfEWfMmE0A6u0xjjzsqVMHgwHHww3Hqrt61pU28ivxTp1mdbLvnPUPoetRvDHz6TXQ7onrJjpYM4W7A8iXr16qWTJk1yHcNXVCug5GUgAtknIoFc15GMSS9VeO01b8qN5cvhssvg2msb/eC5ykRksqr22tjzfDeOwiSHSCbknuo6hjHuvP6611G9667eynM9erhOVG9Z05MxpuGIx73xEAADBnhLkk6caEWijqxQ1CManUu86CG07APXUYzxn1mz4IADoG9fWLPGm8TvzDMhw/8NJyuXrGLhr4vwa1eA/99BA4DGV6MrjgMtQSWMxssI5AxwHcsY9yIRuOceb9W5rCxv/epmzVynqrEvx07k9lMeQEQ48B/7cPET57iO9Dd2RlFfxH4HFIiDlkLEOu9Ncmh0HvHlxxJffjhaMdV1nNpZtQr22AOuvBIOPxxmzvRme03iwNFYLMadpz/EgBanccPAEUkfaf3SbWOoKItQXlrBe09+4suR3FYo6ouMbbyFkiQXyEayj3adyDQQuvqfEJ0O0Vno6qGu49TMuiaaFi28tatfe83rvG7fvvrXbYLv3v2RL8d+R0lhKZM+/InPRn2V1P137t6RUFYICQh5bVuQEfJfQ4//EpkqiYSh9RtQ8QMEOyIZm7uOZBoKLcE7WwW03GmUGvn8c7joIhg71psK/MnUzvoTCP7v+7TIn+8nw/CHz6TVZi1YvXQNJ1050JfT6NgZRT0iko2E97QiYZJKmt8KkudNDNnsdtdxNmz1ahgyBPbf31u3evnytBx2t349OOiUvWme34y+A3Zj/0F7JnX/WTlhBt9yMpf8+1zab+XPEdw24M4Y439vvAHnnQdLl8Ill8CNN3rrRpg6sQF3xpiG48MPIT8f3noLem3075pJMisUxjQQqnEg6o3Kr+9U4ZlnvHWr99jDu+Q1FPJ+TNq5XrjoKRFZJiLTK21rKSIficisxL95LjMaUx9oZBq6bDd06c7E1z6SxuP+gpZ9hMZLkrfTuXO9CfwGD/ZGVoPXzGRFwhnXndnPAP3+su0K4BNV7Qp8krhvjKmGFt4JWgTEYO0jqJam/phln6ArTkDXXIauGIhqHa//j0a9gXM77uitOPfYY/DEE8kJa+rEaaFQ1fHAyr9sHgA8m7j9LGADBjZAVdHSd9GSF9H4WtdxTBU0Mp14QX/iBf3RSAqXfw+2Zn1LsoRIR6uylr4BlIEWQ2wxxBbWbYfPPQeXXgoHHeStFTF0KARcf5c14M8+iraquhhAVReLSBvXgfxK194Pxc8AcSgZhbQe5zqS+QtdfWFiVL13W/I/SclxpNn1qMYgvgRpehkiaWimCe8NFeNBYxDIheAmDHYrK/PmaOreHU47Ddq08UZY+3AsQWPmx0JRIyIyBBgCsOWWWzpOk3oaXwvRmZDRBQm09DaWf4G3ih0QnY1quTcwz/iHVlR9Oxm7ji2H0jEQbAdZRxLIeyip+9+YQM4JaCAfYvMh63BEsmq3gwkT4OyzvQn85szx+iGOOCIlWU3d+PG8bqmItAdI/Lusqiep6khV7aWqvfLz89MaMN00vgZd3h9ddQ5acBAaXeA9kH00SLY3UCrYBV37GBqZ6Tas+RNpfhdIS5CW3u0kUY2hK45F196PrrkWLX40afuuDcnaH8k9AwnW4sS/sNAbE7HPPlBeDs8+a2MifM6PheIt4PTE7dOBNx1m8YeK70HXJn7KoPwjAAK5/4fkPQU5p3nNG8WPoitPQmNLHAc260i4N4G23xJo+y0S7p28HesaiC8HokAplH+TvH2n0uLFsMMO8Pjj3jQc06bBIYe4TmU2wvXlsaOAb4BtRWShiJwJ3AEcLCKzgIMT9xu3jK1B4+vuQMb26x+SzJ54/xvLElsCEJ2b5oAm7SQPQt0Tk0RmQfYJrhNVL5K4IqpdOzj+ePjmG7jvPmjSxG0uUyM2hUc9oRU/omUfIpm7I1n7//mxyC/oykFAAKQZ0nocErBfwHTS6HyIzoPM3dL23qtWQMV3EMhHQtum5Zi1pgovvABXX+1N5rfVVq4TmUpsCo8GRjJ3QTJ3qfqx0HbQ+gOIzoHQTlYk0kwrvkdXngkShEAetH639h27m0AkE8J7pfw4m2z+fO8S1w8+gD59IBZznchsIj/2UZhNIMG2SLivFQkHtHQc68cTxFdB5L+uI7n30EPewLmvvvJuf/kldO3qOpXZRFYojKkjr5M6GwgCAcjo5DaQH8yc6V3VNGMGnH++DZyr56zpyfxNvPRDqPgUCR/6t/4Q83eS1R/ysr0ziaz+SKC560jpV14Ot98Ohx7qNTPdf783N5MNnGsQrFCYP9GK72HNv4BStPRdaPUiEuruOpbvSXg/CO/nOoYb33zjrVP9889eP0SfPpDZAGawNevZ+aAD8dIPiK++iHiJD4eIROewfllMu9TWVGftWrjgAthzT2/FuXffhZtvdp3KpIAVijTTiqmw5jIoexcKr0P9NlAqfKA3b4/kQqAphPdxncj41dNPw8MPe30QM2ZA//6uE5kUsaandIv9Bqxrt1Vvnhz6uMvzFxLMR5s/BOUfJtrbbTkQU8ny5TB7NvTuDeee6zUz2YpzDZ6dUaRbeF8ItvTmaAo0h6xDXSf6E438AqsGQ8kLsPL/0Ohs15GMH6jCqFGw/fbeyOpIBDIyrEg0ElYo0kwCzZDW7yOtxiD5H/9vJli/iExJ3Ih6Jz4VP7lMY/zg99/hyCPh5JO9kdXvvWerzTUy1vTkgEgmZHSp8340tgSiv0Jo5+Rdkpm5pzfCmFxAIJkT2Zn6Z84c6NED4nFvbqbhwyEYdJ3KpJkVinpKo3PQFccBApLlTRsRaFHn/UrGFtD6PYhM9QpQsF3dw5r6p6gImjb1ziD+9S849VTo3Nl1KuOINT3VV2WfgpYnph4vhYrkTYoowXZI1qFWJBqjigq45Rbo2NE7mxCB665zUiRUlWgkmvbjmr+zQlFfZe6Md0IoQBwyfDp7qKk/vv/e65y+9lpvjYimTZ1FmTN1PsfmD+bwnFN45rrRznIYjxWKekoyd0fyHkeaDEdajvKajJJAY8vRso/Q2KKk7M/UA6pw6aXeJa8rV8Kbb8Lo0d761Y48edWLFK1cSzwW5+U732Dt6mJnWYz1UdRrEu4L4b5J25/GlqHLD8dbNU2h1Rgkw9YPaPBEvLmahgyBO+6A5u7nqmqR35yMUAbRSJRgKINQ2P5UueTbd19E5gNFQAyI1mRxDVNHFd8DUW+6bDKh/EuwQtEwrVzpnUWcdRb07QsPPuirCfzOu/8MKkorWLKggLNuP4Vwdjitx//h45945MKnyGvbgiueH07rDq3Seny/8W2hSNhfVZe7DtFohHYE4ngfiwBk9nAcyCSdKrz2mjftxooV0LOnVyh8VCTA68g+8/ZTaNe5DZLmbLFojOuOvpPykgoW/rqY+4eO5Ja3r0xrBr/xe6EwaSQZHaHlaO9MInM3JLST60gmmf74A4YN8/ogdt3VW3muh/++DEz9YgZXH347qkrfo3px9aiL03r8eDxONOKtxhePxSkpKk3r8f3Iz53ZCnwoIpNFZIjrMI2FhLZHmpyN2NlEwzN6NHz4Idx1F0yc6MsiATD6jrGUl5RTUVrBl2O/Y9XS1Wk9figzxLAHziCUFSKvXQuGPTA4rcf3I1HVjT/LARHZTFUXiUgb4CNguKqOr/T4EGAIwJZbbtlzwYIFjpIa42O//goLF8IBB0A06k3H4fOBc49f+ixvP/YhFaUV5DbP4ZXF/yYzy9a3SAURmVyT/l/fNj2p6qLEv8tEZCywOzC+0uMjgZEAvXr18me1SzGN/IwWXgeEkea3Ixlbuo5k/CISgXvugRtugE6dvEWFMjJ8XyQABt96Mlk5YRbNWcqgy4+2IuEDviwUIpILBFS1KHH7EOAmx7F8R1cNgfgyIICuuQRp9ZrrSMYPfvgBzjwTpkyBgQO9NSPq0ZrVmeEQ/3fTINcxTCW+LBRAW2Bs4mqHDOAlVX3fbSQf0pLEjTjEC51GMT7x00+w++6Qnw+vv+4VClMvLV1QQFZumOatm7mO4s/ObFWdq6o7J352UNVbXWfypWY3AJkgOUizG9dv1vgatOQ1tPxbZ9FMmi1KjKTv3h3uv99rarIiUW89cuFTDN7+Qk7ecihfv/m96zj+7cyujV69eumkScmbFK8+Wff/b9215qoRdPmhEFvhTQPV9DoCOcc6TGhSavVqb3bXF1+EadOgS92nrzduxaIxDss+mXgsDkCXHp14/Ie7UnKsmnZm+/KMwtSciPx5QFJ8GcSWA6XerLLlHzrLZlJs7Fjo1s1bu3r4cNhsM9eJTBIEggFatGmOiBAKZ9Cx2+auI/m2j8JsqkBbCLaBWIF3P2wL3jc48TgMGgSvvuqNhRg3zhtAZxoEEeHeL27khZtfo3l+M06/4QTXkazpqSHSeBGUfwzBzZHM3VzHMalw2WXQsiX885+2LKnZZNb0VI/Eix4gvmxP4quGo1pe5/1JoCmSfYwViYZkzhw4+GD4+mvv/ogRcMUVViRMWlihcEwrpkLJUxAvgPLPoeRl15GMn0SjcPfd3tVM333nzddkTJpZH4VzlZd6VCDiKoivaXQBuvZRCLT0FmsK5LiOlHpTp3oD5yZPhgED4JFHoEMH16lMI2RnFNVQLSe+cgjxpb2IF95ESvpzQrtC1jEgORDqAdk2IvWvVBVdeRKUvQElz6OFN278RQ3BRx95czO98op3hZMViaRbvmgl3733I4UrilxH8TXrzK6GlryIFt4BlINkI3n/RjJ3T/pxTPVUK9ClO+GtlQFk7Eig9RinmVJm/HhYuxYOO8xrdios9DqtTdItnLWYYb0uB4GMzAz+M/0+8tq4X90vnawzOykk8VP5vkk3kczEmVYYCCNNzncdKfnWrIFzz4V994Wbb/YWGMrIsCKRQt+98wOR8gglhaVEyiP89MXPriP5lhWK6mQf661JLc0h+wQI2WqsrgSa34C0fhdp8wWSdYDrOMn11luwww4wciRccgl8/LHvVpxriLbbY2sCGQEkIMRjSpedO7qO5FvW9GSMS19/DXvu6V3V9J//eBP6mbSZ+sUMpo2fyW79d2HbXo1v+pN6vx6FMQ2WKsyc6U2/0acPjBrlTeCXaesupNvO++7Azvvu4DqG71nTk2m0NDobLfvQG8meLvPmQb9+0LMnzJ/vNTENGmRFwvianVEYJzQ6By1+CoIdkNyzvA7rdB6//Ftv4ScJen1Q+e8jkpW6A8Zi8OCDcM013iJCd98NW9qKhKZ+qLZQiEgzIF9V5/xl+06q+lNKk5kGS7UCXTEIdA2QhcYLkWZXpDdD2btAmTfGESA6C0LdU3OwSMS7mumbb7zLXh97zIqEqVc22PQkIicAvwCvi8gMEak8cdAzqQ4mIv1E5L8iMltE0vtXpBqqimp04080G6ZrK63OVwbRX9IeQcJ7AtlACMiAYKfkHySeGPcRCsEhh3hrRowbZ0XC1DvV9VFcBfRU1R7AGcDzIrJuyayUXrsnIkHgEaA/0A04SUS6pfKYNaGR/6LL9kCXdide9IDrOPWWBFpC+AAgC8hCcs9Of4asQ5G8R5CmlyCt30ACTZN7gK+/hp12gq++8u7fcAOcfHKjvey1rKScV+95i1G3j6G4sGTjLzC+Ul3TU1BVFwOo6ncisj8wTkQ2538n7KmyOzBbVecCiMhoYADgdESMrrkSdLV3p3gkmvt/SKBxjeRMFmnxIMTmgOQhwVZuMoT3gvBeSd2nFhay7NRzaPP2y7D5FkikZnN3qSpFK9eS2yKHYDCY1Ex+cPspD/D9+1NAle/en8J9X9zkOpKpherOKIpEZP2FxYmisR/eH+xUX0/WAfi90v2FiW3ricgQEZkkIpMKCgpSHAfiJS9DdGalLQFIcwdsQyIiSMbWzopESnzwASVbbkX+W6MZl7EN13Y+Bfbbb6MvqyiPcNHe13LiZmdzetfhrC5Yk/qsafbLxFlEyiNEKqLM+XGe6zimlqorFOcCgcpNPqpaBPQDzkpxrqrOz/90FqOqI1W1l6r2ys/PT3EcoHw8EEvcEWh2EyLZqT+uqT+mT2dNqXIR+/NgtDvffTmb+Lp+impM+XQ686YtIBqJsWLRKj56bnwawqbXEeceQjgnTFZumH6DG9jI+kZgg01PqjoVQESmi8jzwAi8RuURQC/g+RTmWghsUen+5sCiFB5v47IGQPkE73LKQFsk+0incYwPqHqD5bKz4Zhj4MIL+WBpK+Y99jFZCl16dCIQ2PhQpZbtWhCPed+DMkJBWndoePM7nXrt8fQ5ohfRSJRtd9vadRxTSxudwkNEcoE7gZ5AU+BF4E5V3fhXpU0NJZIB/AocCPwBfA+crKozqnp+uqbw0MjPEFsImXsigdyUH8/42G+/eZP4vfsuHHmkN18TXl/D129+z9rVxex3Yl/C2eEa7e6j57/gvf98Qs+Dd+Lkq49FGmmnt0mvZE7hEQFK8a4lzALmpbJIAKhqVETOBz4AgsBTGyoS6SShbhByfvGVcSkeh0cfhSuv9G7ffz+c/7/ZbEWEPY+u/XxNB5+6Lwefum8ykxqTNDWZwuN7vEKxG7AX3qWqr6U0FaCq76rqNqraRVVvTfXxjKmRTz6B4cOhb1+YMQMuvBAayFVKqsqYB8ZxRb9b+OSlCa7jGB+pyRnFmaq6rl1nCTBARE5NYSZj/KWiAiZN8orDQQd504AfcECDGxPx9Zvf8/Q1oykrLmf6l7+wxbabsU3Pxjejqvm7jZ5RVCoSlbelsiPbGP+YONGbwO/AA2HxYq84HHhggysSAEsXFBCLeq3KgYBQ8PsKx4mMX9jsscZUpbgYLr7YmwZ81Spv3er27V2nSqkDTt6LFm2akZkVon2XtvQ8ZGfXkYxP2OyxxvxVSQnsvDPMmeNd2XTHHdCsmetUKdcivznPzX6Y1cvW0LJ9Xo0u7TWNg30SjFmnrMz7NycHzjkHxo/3rnBqBEVinYxQBq07tPJ9kXjm+tEcmz+YKw69mdK1pa7jpF08HmfWD3NZ9vvytBzP358GY9JB1Wta6twZvvzS2/avf8Hee7vNZao0Z+p8XrvnbQpXFPHT+J8Z++C7riOllapy/dEjuGTf6xi83YVMfGdyyo9phcI0bgsXwoABcOKJ0KEDNLdJHv3OGyScuJhA191v2GKx2PrbRSsixV01AAAfE0lEQVTXMumDKZQVl1NeWsGr976d8uNboTCN11NPeetWf/yxt+Lct99C9xQtXmSSZusenTnmgv40yctlhz2345gLDncdKWXi8Ti3nfIA/TNP4oztLmDVsjXkNs+hSV4uEhDC2Zlsl4YpUTY6hUd9kK4pPEwDc++93hQcTzzB8uw8/pi1mG16dSE7N4VLohpTCzMnzuKyg26krLicYEaQQVcczf/dNIhFc5Yw9sF3adsxn2MuOIxgxqYN+kzmFB7GNAyRiHfm0KULnHCCN6r64ouZPXU+F/e4kEBAaJ7fjCem3m3FwvhCbvMcNO59mQ+GgjRt2QSAzbq0Y9gDg9OWw5qeTOMwaRLsthtcdRV8+qm3LRgEET5/+WvKissoKSplTUEhv06aU/2+jEmTLbfrwHkPnEGnHbfgwFP25qjzDnWSw84oHFKNooXXQ/k3kHMMknt+g5g1VGMF6JprQFchTa9GMh0O3Copgeuv95qZ2raFMWO8KcEr2Xa3rQnnhCkvKSceVzp0Tf/AutLiMn78eBrtt2pD5+4d035841+HnXUQh511kNMMVihcKn0TSscBpbD2P5DZGzJ3c52qznTN1VAxAYihqwZDm0nuCuC6juqzz4YRI6BFi789Ze+BewDw6/ez2ffEvrTeLL3rQUQjUc7f/QoKFq4gHotz9aiL6XPkRpuNjUkbKxQuaSmQmLFdBLTMaZyk0RWsXw1w/X9jGmdYXbXKu4Kpf39vrYipU2Gnnap9yd4D91hfMNJtybxlLF2wnPKScgA+eu4LKxTGV6yPwqXsgRDaEQhB5r6Q2dd1oqSQpleB5AIZ0PRyRNJYJF5/3bvk9fjjvYIhstEi4VqbLVuT3SSLUGYGWblheh7i77ym8fHdGYWI3ACcDRQkNl2lqg1y6KUEcpBWo1zHSDrJ7AltJgMxRELpOejixTBsGIwdCz16wDvvQF5eeo5dR5lZmTw2+U4+G/01Hbq2o+9RdW9+VFUKfl9O8/xmNV5lz5gN8V2hSLhPVe92HcJsOpEAaTthXbUKdtzR67i+4w645BIIpalAJUnrDq04/p/JWYc9Fo1x2cE38cvEWYSzM3nwm9vYfJvNkrJv0zhZ05Opv1Yk1kvIy4Nbb4WffoLLL693RSLZZv0wl18nz6WiLMLa1SW8/fiHriOZes6vheJ8EflJRJ4SkfrRfuCQakWjmO9mvWgU7roLttwSJiSW7Bw6FLp2dZvLJ1pt1hKNexdJZGaH2NzB5b6mYXFSKETkYxGZXsXPAOAxoAvQA1gM3LOBfQwRkUkiMqmgoKCqp9SJxlej8ZKk7zeZVJX4mmvQpTuhBXuh0d9cR0q9KVOgd2+47DI4+GDYaivXiXwnf/NW3PTG5fQ+oicnXzWQw8852HUkU8/5eq4nEekEjFPVHat7XrLneooXPQzFjwFBJO8RJOzP6aY1Oh9dfhRQBgQg+1gCzW91HSt1br4ZbrwRWrWChx+G445rkEuSGpMuNZ3ryXdNTyJS+Tz5GGB6Oo+vGofiR4AIUIYW3ZXOw9eONAHWFfoMCLSq1ctVlXjxKOKrzkfLPkt6vKRr2hROOw1mzvQuf7UiYUxa+PGqpxEi0gPvL+B84Jz0Hl4g0ALiK4EgBLdI7+FrQYKtocW96NrHILQt0uTc2u2g/EMougMoRcvHQ+sxSEbqpyyusTVr4IorYJ994KSTvEn8rDgYk3a+KxSqeqrL44sItHweLbwLgnlI0ytdxtkoyToYydrENujob3hnToAEIfo7+KVQvPUWnHeeNz5is8SlnVYkjHHCd4XCDyRja6TlE65jpF72kVD8FGgxBDtAuI/rRLB0KVxwgbc0affu3gC63er//FfG1GdWKBoxCbaDNp9DbAkEOyDig4/DV1/BG2/ALbd461ZnZrpOZEyj57vObJNeImEko6PbIjFvHrz6qnd74ECYPRuuvtqKhPGdxfOWMnTXfzFo83OY8Pq3ruOkjRUK404sBvfd502/MWwYFBd727fw7wUEpnF7aNh/mDt1ASsWreT2fzxIpCLiOlJaWKEwbkybBn37evMy7b8/TJ4MubmuU5kqzP5xHhPGTKR0banrKOupKn/MXsya5YVpPW4sFv9ThsbCB43S/qLxlejakSAhJHcIEmjqOlLDs2SJ10HdtCm89BIMGmRXNPnUhDETufO0BwkEArTevBX//ukeghlpnDZ+A+449SG+GjsRBG4cexk9D07PKorDHz6La4+6g8LlhZx7/xmEMhvHvGJWKP5CVw6B6M+AoJHpSMunXUdqOObNg86doV07ePppbwqO1q1dp6rWfyfN4dOXJtCt9zbse0LDWC+kNj55cTzlJRUALPutgKULCtisSzunmYpWreWLV78mFvEWxxp1+9i0FYrNu7bn6ZkPbPDxb8dN5sVbX2ernTpy3v3/12CmeLdC8VexOUDUux391WmUBqOoCK66Ch59FL74AvbayxtA53MFC1dw6f7XU1Zczjs5HxMIBtj72N6uY6XV7v13YfKHU4lF4+Q2z6X15rUb/Z8K2U2yyGmSzdrVawmFM+ncfUvXkQBYtWwNN594LxWlFcydOp+8ts35v5sGuY6VFFYo/irndG9sAQq5Z7lOU/+9+643s+vChd74iB49XCeqsUWzlxAIeN145SXl/Dp5bqMrFIeddRCt2uexaM5S9j2hD5lh900tGaEM7ptwEy+PeJM2W7bm5KsGuo4EQPGaEtY1oEbKo6xYvMppnmSyQvEXgaYXodlHARlIhj++qdRb55wDI0d6S5N+9RX08cGAvlrYdvetyWvbHPA6Lg84aU/HidzY4/CeriP8TcduW3DZM+e7jvEnHbZux36D9uSj576gWaumDLr8aNeRksbXs8fWVLJnjzV1sO7zJAKPPAIFBXDllRCun2215aXlzP5xPptt3Y68Ns1dxzH1QFlJOZlZofVno35W09lj7YzCJM+CBXDuuXDKKd7PsGGuE9VZODvMDn23dR3D1CNZOfXzS1F1/F/yjP/FYvDQQ7DDDjB+PJSVuU5kjEkiO6Mwm6x0bSmhubPJGHoOfPMNHHooPPEEdOzoOpoxJonsjKKe0NgS4itOIV7QDy3/ynUcnrpmFMe0PIMRe5xPdMbP8Nxz8N57ViSMaYCsUNQTuuZ6iEyG2Fx01bmoRp1lKfvsC1bdcR+xaIzPyttwffchcOqpNrramAbKSaEQkeNFZIaIxEWk118eu1JEZovIf0XkUBf5fEmLgXXzzEQr3U6jtWvhoosIH7g/J+kvZGicjMwMmnbePP1ZjDFp4+qMYjowEBhfeaOIdAMGATsA/YBHRcT9xDI+IM2ugUAbIAxNr0XEm4JbVYmvuZ74kp2IrxiExotSE+DDD71ZXh94ADn3XIo++oId9u/Ofif05fyHzkzNMY0xvuCkM1tVZ0Ji2dE/GwCMVtVyYJ6IzAZ2B75Jb0L/kdB2SJsv//5AZCqUvgGUQWQaWvIS0iTJy4zPmwf9+0PXrjBhAuy1F9sCdx+wS3KPY0wDEIvFmDNlPi3aNKfNFv6ey6ym/NZH0QH4vdL9hYltfyMiQ0RkkohMKigoSEs4X5LKi/sE1p9p1JkqrBvE2LkzjBsHU6Z48zQZY6qkqlx75B38c7/rGbzdhXz33o+uIyVFygqFiHwsItOr+BlQ3cuq2Fbl0HFVHamqvVS1V35+fnJC10MS6gZNzoFAWwjvBzkn132nCxfCUUd5U4FPnOht698fsrLqvm9jGrDVy9bw46fTKSsup7y0gtfve9t1pKRIWdOTqh60CS9bCFRe3mxzYFFyEjVcgSbnQZPz6r6jeNwbB3H55RCNwj33QK+Nju43ptGJxWKMuf8d5k//naOH96frrlsB0CQvl9xm2RSuWEtmVohtd9/acdLk8NuAu7eAl0TkXmAzoCvwndtIjYQqHHGENxbioIO8grHVVq5TGeNLr937Ns/f+BrlJeV8/spXvLTgMZq3bk4oM8SD39zGGw+/R7tObRgwrJ/rqEnh6vLYY0RkIdAHeEdEPgBQ1RnAK8DPwPvAMFWNucjYaEQiXpEQgYED4amnvCucrEgYs0Fzpy6gvKQcgIrSCDcMvHv9Y5t1acd5953BwAsP98VqgMngpFCo6lhV3VxVw6raVlUPrfTYraraRVW3VdX3XOTzG42XoLElyV+jd9Ikr2npxRe9+2edBWecYQPnjNmIYy447E9Xbc746hdisYb7ndZvVz2Zv9DINLRgT7TgIHT1hckpFiUlcOmlsMcesHw5tGy5waf+MXsxp3YZxhG5p/BaA+mYMw3Hwl8XcdaOF3NihyF8/eb3aTvudrt35YihB5OZnUlmVoitdupEMLjhs4dIRYR43MEg2SSx9Sh8Lr7qAih/P3EvjOS/jwSrvGK4ZsaP984a5s6FIUNgxAhovuF1Fm4YOIKv35qExpWMzAxeXfIfmrTI3fTjG5NE/9z/eqaN/xlVyMwK8fbaF9K2DkQ8HufTl75k7apiDj5tH3KbV/178fKIN3j6mtGEszO5/f2r6dbHP9PW13Q9Cjuj8LuMzkBifnsJovEKtOJ7VDdxKu+VKyEYhM8/9zqsqykSAOGcLAIB7xRbRAgE7SNTHy2eu5T3nvyEBT//vvEn1yOVv+em+ytvIBDgoH/sw9HD+2+wSEQqIjx1zShi0RglRaU8celzaU6ZHH676sn8hTQZhmocYnMgfBCsPBol4I2baP0mIhtZJEUVXn8dli71FhI6+mg47DDIrNnAvKH3ns7KJatYtqCAs+74BzlNs5PwX5VeX73xHXef+Sjh7ExufONytu3VxXWktFr2WwFDd/lXog1deOCrW+iyc6eUHCsWi1XbBJNsFz0+hBsGjqBoVTEXPnq271aVC2YEycoOU1JUSjAjSF67Fq4jbRJreqpH4oU3Q8nz3h3JRfKeRjJ7bPgFixZ5xeGNN7z1qidM8M4mGpmjmp1K6VrvDGyrnTryxJS7N/KKhuWLV77mnrMfp7SolFBmBmfd+Q8GXnh4Uo8RqYhw1WG3MfWz6WzTa2vu+vR6snNtgCbAz9/+yhP/fJa8di246PEhtMj3z5K61vTUAEloF6DSN/rgFlU/MR6Hf/8bunWD99+HO+/0+iYaYZEACIa8/24RCIUb30n09r27InhLdAYygvTYf8ekH+Pbtyfzy3ezUYX5M37js5eqmJeskerWexse+OpWbnj9X74qErXR+H5r6rOswxECaGQGkn0UEmxV9fNmzoShQ2GffWDkSG8yv0bshjH/4t6zHycrN8xlzw53HSft2myZz+NT7mLqZzPYvndXOnbbwBeMOmiSl7u+w0BEvPumwbCmp4YiGoWPP4Z+iZGgEyd6czX5rM3WNEyqyqjbx/DZqK/Y44ienHnbyVXNDm18pqZNT1YoGoIff4Qzz/T+nToVdtrJdSJjTD1gfRSNQWkpXHGFd+awaBG89hp07+46lTGmgbFCUV/F497aEHfeCaef7vVLHHusTb9hjA/8NP5nBm1+DidtcQ7Tv/rFdZw6s0JR3xQVeZ2GgQBcconXL/Hkk5CX5zqZMSbh9lMeYMWilSz/YyV3/ONB13HqzApFffLmm7DddvDCC979U06BAw90m8kY8zeVZzCQQP0/y7dCUR8sWUL06GPg6KMpzsxBt9vOdSLn3nj4PS7a+xpeveet5M+qa0wdXT36Ytp1bkP7rdpy9aiLXMepMxtH4Xevvoqecw66upBnQzsztqAbJ36ykJN22811MmemfDadJ698kbLicuZMmU/Hbluwe/9dXMcyZr1uvbfh+TmPuI6RNK4WLjpeRGaISFxEelXa3klESkVkSuLncRf5fCUzk/h223Nu8BBeiHaluCTCt+Mmu07l1MrFq9bfVtU/3TfGJJ+rpqfpwEBgfBWPzVHVHomfoWnO5V4sBvfeC3cn5iMaMIDAlxMI77oT2U2yCOeEOfCUvd1mdKzPgN3o0LU9wVCQdp3asPdxvV1HMqZBc9L0pKozARu5+Vc//eStMvf993DcceuXKJVAgHs/v5GJ7/xAy/Z57Lhn4+6jyM7N4rHJI1i7upgmLXLtc5Rka1cX89nor2jVPo8+R/Wy99f4so+is4j8CBQC16jqBNeBUq6sDG65xRsTkZcHo0fDCSf8aUxEODvMPsf1cRjSX0SEpnlNXMdocOLxOMP7XMWy35YTCAinXHMsgy4/xnUs41jKmp5E5GMRmV7Fz4BqXrYY2FJVdwEuAV4SkWYb2P8QEZkkIpMKCgpS8Z+AxhajFVNQjaRk/+vNnAl33AEnn+zdPvFEGzhnnCheU8LiuUupKK2grLi80feHGU/KzihU9aBNeE05UJ64PVlE5gDbAH+byElVRwIjwZvrqW5pq8hS/jW6aihIADK2g5YvIZLEulpYCOPGecVhl128AlFPZ3md8Pq3vHzXm2zTqwvn3ns6ocyQ60hmEzVpkUuXHp34feYfqCoH/WMf15EatTXLCwnnhMnK2cgCZSnmq6YnEckHVqpqTES2AroCc11k0ZIXgTJvfcXIzxBbCBlbJmfn77zjTQO+aBHssQd06VJvi8Sy3wq447SHqCitYN6032izRWsGXX6061hmE4kI935+I9+9+yMt2+exQ1//rO/c2Dx60dO8/fiHZISC3Pbu1XTfe3tnWVxdHnuMiCwE+gDviMgHiYf2AX4SkanAa8BQVV3pIiOhXYAsQEDCEMyv+z6XLYOTToIjjvDWqv7qK69I1GNFq4rXr6kdLY/YpaoNQDg7zN7H9rYi4VBJUSlvPfoB0YooZcXlPHPdaKd5XF31NBYYW8X214HX05/o7yR3MCrNIDYPyTkBkTquFR2JeGcPf/wBN97ozfpaw3Wr/WyrnTrS+4iejH/tW1q0ac6xFx/hOpIx9V5mVois3DDFa0oIhUN02Lqd0zy2HkWqLV4M7dp5ndNjxnhzNXXr5jpV0pWVlBPOzqzzpZQfPP0Zj1z4FLktcrjtnavo3L1jkhIaU7/M+mEuz17/MvlbtGbIiH8w96ffuP+cJwjnhrni+QvYvGv7Oh/DFi5yLRaDRx6Bq66CRx+F005zncj3YtEYRzT5B9GKKAA77r0d931xs+NUxvjDsfmDKVxRhIiwzW5dePjb2+u8T1u4yKUZM7y1Ii68EPbeG/bd13Wi+kEgmOF9JCUghLPdXulhjJ9UlHuX6asq5cXlaT22FYpke+gh73LXWbPg+efh3XehozWf1EQwGOSGMZex2dbt2H6Prlwy8hzXkYzxjUufPJesJlk0b92Ui9P8u2FNT8n21lveyOoHHoD8JFwpZYwxKWJNT+mydi1cdBHceqt3/6ij4KWXrEgYYxoMKxR18cEHsOOO3tnDihWu0xhjUiQWi1FRVuE6hjNWKDZCNU589WXEl+xIfMVJaLwYli/3rmLq1w+ys+HLL72pwY0xDc7P3/yXY1r+H0c1O42XbvPFMK+0s0KxMRXfQfmHQAVEpkHpq7BgAbz6Klx7Lfz4I+y5p+uUZhMUF5bw+n3jGPfER0QjUddxjE898a/nKS0qIxaN8ewNr6y/+qgx8dVcT74kWYDCogh8WAwXZEHPnl6xaNMmZYctXFlEtCJKy3Z5KTtGY3fFobcwZ8o8AoEAM7/9lX89Pcx1JONDLdu1IBgKEovEyMoOr7+EuzGxQrERkrET+tK2yHVjIC5wah/IIaVF4otXv+HO0x+CuHLSVQM59brjU3asxmzWD3OJRWIA/DT+Z8dpjF9d9PgQRISVS1Yz5K5TCQaDriOlXeMrjbXxyy+w774ELnkV6b0/Mu2/SIfUj4l4+ppRRMoiRCqivHTbmJQfr7Ha9/g+ZCWWlz3srANdxzE+1bx1M6579Z/cP+FmuvXexnUcJ+yMYkNKSrxR1bEYPP00nH562hYT6tC1HUvnLyMWi9O6Q8u0HLMxuvy54UwbP5NwTibb7V4/p3k3Jh2sUPzVzJnexH05OfDCC9CjB7Rtm9YIlz83nKeueomSwlLOuOWktB47FeLxOAt+Xkhe2+a0yG/uOs56gUCAnffbwXUMY3zPRmavU1wM118P990Hzz4L//hHcsKlyMolq1i1dA2du29JIODfFkRV5cp+tzDj6/+Cwu3vX82Oe7lbgMUY8z82Mrs2PvkEuneHe+6Bs86CI490nahaP346jdO2Hs5Fe13D1Yffjp+L/ZL5y5j25S+UFZdTVlLOmPvfcR3JGFNLrla4u0tEfhGRn0RkrIi0qPTYlSIyW0T+KyKHpjzMVVfBQQdBRgZ8/jk88YS3+pyPvX7fOMpLyikrLmfq5zNY/oebRQBrokWb5oRCXgtnOCeTLrt0Ttq+Y7GYr4ukMQ2FqzOKj4AdVXUn4FfgSgAR6QYMAnYA+gGPikhqr0Xr29dbbW7q1HozHXjXnl0IZ2eCeH98m7du6jrSBmXnZnHfhJvod+YBnH7jiQy6Ijnrab902+scnn0yx7c7i3nTFiRln8aYqjnvoxCRY4DjVPUUEbkSQFVvTzz2AXCDqn5T3T58NXtsGsSiMcY++C5/zFrM0RccRsftN3cdKa2KC0s4Ln8w0cQYiF6H7szt713jOJUx9U9N+yj8cNXTYODlxO0OwLeVHluY2GYqCWYEOe4Sf/ejpFIoM4NgRpBoJEYwFKRZK/+eURnTEKSs6UlEPhaR6VX8DKj0nKuBKPDiuk1V7KrKUx4RGSIik0RkUkFBQfL/A4xvZWZlcuMbl9GlRyd2778Lwx4Y7DqSMQ1ays4oVPWg6h4XkdOBI4AD9X/tXwuBLSo9bXNg0Qb2PxIYCV7TU50Dm3ql58E70/PgnV3HMKZRcHXVUz/gcuAoVS2p9NBbwCARCYtIZ6Ar8J2LjMYYYzyu+igeBsLAR+JNi/Gtqg5V1Rki8grwM16T1DBVjTnKaIwxBkeFQlW3ruaxW4Fb0xjHGGNMNWxktjHGmGpZoTDGGFMtKxTGGGOqZYXCGGNMtZxP4ZEMIlIA1GXCn9bA8iTFSSbLVTuWq/b8ms1y1c6m5uqoqvkbe1KDKBR1JSKTajLfSbpZrtqxXLXn12yWq3ZSncuanowxxlTLCoUxxphqWaHwjHQdYAMsV+1YrtrzazbLVTspzWV9FMYYY6plZxTGGGOq1WgLha/W7f5zruNFZIaIxEWkV6XtnUSkVESmJH4eT2eu6rIlHnP2nv0lxw0i8kel9+kwV1kSefol3pPZInKFyyyVich8EZmWeI+cLg8pIk+JyDIRmV5pW0sR+UhEZiX+zfNJLuefLxHZQkQ+E5GZid/HCxPbU/eeqWqj/AEOATISt+8E7kzc7gZMxZvdtjMwBwimMdf2wLbA50CvSts7AdMdv2cbyub0PftLxhuAS11/vhJZgon3YisgM/EedXOdK5FtPtDadY5Eln2AXSt/voERwBWJ21es+/30QS7nny+gPbBr4nZT4NfE72DK3rNGe0ahqh+qajRx91u8RZIABgCjVbVcVecBs4Hd05hrpqr+N13Hq41qsjl9z3xsd2C2qs5V1QpgNN57ZSpR1fHAyr9sHgA8m7j9LHB0WkOxwVzOqepiVf0hcbsImIm3ZHTK3rNGWyj+YjDwXuJ2B+D3So/5ad3uziLyo4h8ISJ7uw5Tid/es/MTTYpPuWiyqMRv70tlCnwoIpNFZIjrMFVoq6qLwfvDCLRxnKcyv3y+EJFOwC7ARFL4nrlauCgtRORjoF0VD12tqm8mnrPJ63anMlcVFgNbquoKEekJvCEiO6hqoQ+ypfw9+9PBqskIPAbcnDj+zcA9eF8EXEjr+1JLe6rqIhFpg7eA2C+Jb9Cmer75fIlIE+B14CJVLUwsApcSDbpQaIrX7U5Vrg28phwoT9yeLCJzgG2ApHZEbko20vCeVVbTjCLyb2BcqnLUQFrfl9pQ1UWJf5eJyFi8ZjI/FYqlItJeVReLSHtgmetAAKq6dN1tl58vEQnhFYkXVXVMYnPK3rNG2/RU39btFpF8EQkmbm+Fl2uu21Tr+eY9S/yCrHMMMH1Dz02D74GuItJZRDKBQXjvlVMikisiTdfdxruww+X7VJW3gNMTt08HNnQ2m1Z++HyJd+rwJDBTVe+t9FDq3jOXvfeOrxyYjdd+PCXx83ilx67Gu1rlv0D/NOc6Bu+baDmwFPggsf1YYAbelTM/AEc6eM+qzOb6PftLxueBacBPiV+c9o4/Z4fhXZUyB6/5zlmWSpm2SnyOpiY+U05zAaPwmlYjic/XmUAr4BNgVuLflj7J5fzzBeyF1/T1U6W/X4el8j2zkdnGGGOq1WibnowxxtSMFQpjjDHVskJhjDGmWlYojDHGVMsKhTHGmGpZoTAmxUTkfRFZLSIuB/8Zs8msUBiTencBp7oOYcymskJhTJKIyG6JyeKyEqOfZ4jIjqr6CVDkOp8xm6pBz/VkTDqp6vci8hZwC5ANvKCqfpsaw5has0JhTHLdhDfHUxlwgeMsxiSFNT0Zk1wtgSZ4K49lOc5iTFJYoTAmuUYC1+Ktb3Kn4yzGJIU1PRmTJCJyGhBV1ZcSU8J/LSIHADcC2wFNRGQhcKaqfuAyqzG1YbPHGmOMqZY1PRljjKmWFQpjjDHVskJhjDGmWlYojDHGVMsKhTHGmGpZoTDGGFMtKxTGGGOqZYXCGGNMtf4fDBkaoHosYbYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "t = np.arange(-20, 20, 0.2)\n",
    "import matplotlib.pyplot as plt\n",
    "plt.scatter(x[:,0],x[:,1],c=y, s=8);\n",
    "plt.xlabel(\"x1\"); plt.ylabel(\"x2\");\n",
    "plt.plot(t, t + 0.5, 'r--')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor(x).float()\n",
    "y = torch.tensor(y).float()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Linear(in_features=2, out_features=1, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = torch.nn.Sequential(\n",
    "    torch.nn.Linear(2, 1),\n",
    ")\n",
    "model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([100, 1])"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model(x).shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "x, y = gen_logistic_fake_data(10000, 1., 0.5)\n",
    "x = torch.tensor(x).float()\n",
    "y = torch.tensor(y).float()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 0.1\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jyoti/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py:1006: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.\n",
      "  warnings.warn(\"nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.\")\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.1200413703918457\n",
      "0.014865457080304623\n",
      "0.010064764879643917\n",
      "0.007740796077996492\n",
      "0.006234072148799896\n",
      "0.005126570351421833\n",
      "0.004259850829839706\n",
      "0.0035587025340646505\n",
      "0.002980965655297041\n",
      "0.002499609487131238\n"
     ]
    }
   ],
   "source": [
    "for t in range(10000):\n",
    "    # Forward pass: compute predicted y using operations on Variables\n",
    "    y_hat = model(x)\n",
    "    loss = F.binary_cross_entropy(F.sigmoid(y_hat), y.unsqueeze(1))\n",
    "    if t % 1000 == 0: print(loss.item())\n",
    "       \n",
    "    # Before the backward pass, use the optimizer object to zero all of the\n",
    "    # gradients for the variables\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    \n",
    "    # Calling the step function on an Optimizer makes an update to its\n",
    "    # parameters\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Parameter containing:\n",
      "tensor([[-18.6118,  18.6125]], requires_grad=True), Parameter containing:\n",
      "tensor([-9.2036], requires_grad=True)]\n"
     ]
    }
   ],
   "source": [
    "print([p for p in model.parameters()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dataset and Data loaders "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nearly all of deep learning is powered by one very important algorithm: **stochastic gradient descent (SGD)**. SGD can be seeing as an approximation of **gradient descent** (GD). In GD you have to run through *all* the samples in your training set to do a single itaration. In SGD you use *only one* or *a subset*  of training samples to do the update for a parameter in a particular iteration. The subset use in every iteration is called a **batch** or **minibatch**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "model2 = torch.nn.Sequential(\n",
    "    torch.nn.Linear(1, 1),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import Dataset, DataLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "def lin(a,b,x): return a*x+b\n",
    "\n",
    "def gen_fake_data(n, a, b):\n",
    "    x = np.random.uniform(0,1,n) \n",
    "    y = lin(a,b,x) + 0.1 * np.random.normal(0,3,n)\n",
    "    return x.astype(np.float32), y.astype(np.float32)\n",
    "\n",
    "# create a dataset\n",
    "class RegressionDataset(Dataset):\n",
    "    def __init__(self, a=3, b=8, n=10000):\n",
    "        x, y = gen_fake_data(n, a, b)\n",
    "        x = torch.from_numpy(x).unsqueeze(1)\n",
    "        y = torch.from_numpy(y)\n",
    "        self.x, self.y = x, y\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.y)\n",
    "    \n",
    "    def __getitem__(self, idx):\n",
    "        return self.x[idx], self.y[idx]\n",
    "    \n",
    "fake_dataset = RegressionDataset()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we are going to create a data loader. The data loader provides the following features:\n",
    "* Batching the data\n",
    "* Shuffling the data\n",
    "* Load the data in parallel using multiprocessing workers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataloader = DataLoader(fake_dataset, batch_size=1000, shuffle=True)\n",
    "x, y = next(iter(dataloader))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([1000, 1]), torch.Size([1000]))"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.shape, y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 0.1\n",
    "optimizer = torch.optim.Adam(model2.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "77.5617446899414\n",
      "0.14563588798046112\n",
      "0.09156578034162521\n",
      "0.09121144562959671\n"
     ]
    }
   ],
   "source": [
    "for t in range(200):\n",
    "    for i, (x, y) in enumerate(dataloader): \n",
    "        \n",
    "        y_hat = model2(x)\n",
    "        loss = F.mse_loss(y_hat, y.unsqueeze(1))\n",
    "    \n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "    \n",
    "        optimizer.step()\n",
    "    if t % 50 == 0: print(loss.item())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Parameter containing:\n",
      "tensor([[ 2.9895]]), Parameter containing:\n",
      "tensor([ 8.0066])]\n"
     ]
    }
   ],
   "source": [
    "print([p for p in model2.parameters()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Two layer neural network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generating fake data\n",
    "# Here we generate some fake data\n",
    "def sigmoid(x):\n",
    "    return 1/(1 + np.exp(-x))\n",
    "\n",
    "def gen_nn_fake_data(n):\n",
    "    x = np.random.uniform(0,10, (n, 2))\n",
    "    x1 = x[:,0]\n",
    "    x2 = x[:,1]\n",
    "    score1 = sigmoid(-x1 - 8* x2 + 50)\n",
    "    score2 = sigmoid(-7*x1 - 2* x2 + 50)\n",
    "    score3 = 2* score1 + 3*score2 - 0.1\n",
    "    y = score3 < 0\n",
    "    return x, y.astype(int)\n",
    "\n",
    "x, y = gen_nn_fake_data(500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsnXd4U+UXxz/vzU4XLS17b5AhUwRxi6Ko4ETcoogDEUUFRfkpAoLgBhQVRREHuBiKoOy9d1myV6F0t9n3/f1xY1kdSZo0BfJ5Hh6a5r3ve5Im99x73nO+R0gpiRAhQoQIFy9KuA2IECFChAjhJeIIIkSIEOEiJ+IIIkSIEOEiJ+IIIkSIEOEiJ+IIIkSIEOEiJ+IIIkSIEOEiJ+IIIkSIEOEiJ+IIIkSIEOEiJ+IIIkSIEOEiRx9uA3whMTFR1qpVK9xmRIgQIcJ5xdq1a1OllEnFjTsvHEGtWrVYs2ZNuM2IECFChPMKIcR+X8ZFQkMRIkSIcJETcQQRIkSIcJETcQQRIkSIcJETcQQRIkSIcJETMkcghJgohDguhNhy2u8ShBBzhRC7vP/Hh2r9CBEiRIjgG6G8I/gauOms3w0E/pFS1gf+8T6OECFChAhhJGSOQEq5CEg769e3A5O8P08CuoVq/QgRIkQoCOk+iJr9AdL2G8Hq0CilRHpSkNIZlPlKm9KuI6gopTwKIKU8KoSoUMrrl0mcdid/fTUfoSh0fuRqjCZDuE3yG1VV+WLgZJbPWMvV93bgoSH3IIQIt1kRIpyBlDbkyTtBZiExgZqBiHrEj+PtYJ8HuiSEsa33dx5k+mPgXAvCCuWnIfQ1QvQKQkOZLSgTQvQGegPUqHF+van+8tbdY9gwT9tKWTt3I0OmDQizRf6z9LfVzBg/B3uug2ljZtC0YyNa39Ai3GZd1EjPCXDMB30DhPHScJtTNvCcAJkHqIANHAvBR0cgpUSe7AmePYBERr+EEvUAuLaAayPgBOlG5k1BxJ5fUe/SzhpKEUJUBvD+f7ywgVLKCVLKNlLKNklJxVZIn9dsWbIdh82Jw+Zk86Jt4TYnIHIz80Ce9ThC2JBqDjL1VmTWMGTaw0jHgnCbVDbQVQU8px67D/t+rMwG93bNkUgb2Gd650wCqXoHGUFXPVjWlhql7QimAw97f34Y+L2U1y+TXNOjI+YoE+YoE9fe3+mM57av2sWQ7qP4/JVvcTpcYbKweK6+twP1WtVGKIImHRpy+W1twm3SxY37X8AJ2AAb0r4gvPaUEYTQAabTfuFHUETEgIg98zEgdFUQ8Z+AsRNEP4Gw9giOsaWICNZmyTkTC/E9cDWQCKQAQ4DfgJ+AGsAB4G4p5dkbyufQpk0bWda1hv57HwOJi0sp2TB/C4qi0PyqJvlz2HJs3FulN7YcO0azge79buHxEfcH1e5gI6WM7A2UAbQ7gs6g5gISET8WYep07jjPEWR6X1BTIWYwiuWG0Njj2gkIhKF+SOb3B9X2F2QNAmFClBvvV9hMTWmt3RkAiPIoFZeHyMrgIIRYK6Us9qosZHsEUsr7CnnqulCtGS7mfb+Y0b3Go9fruOqeyylfJYE7nr+F2IQYn44XQtDy2mbn/D4nIw+3S7uNddpdHNpxJKh2h4KLyQlINRdc60BXC6EvW+EAoURD+RngXAj6+gjDuZ8vAJn5P3BvBVTI7I80r0MIY1BtUbM/hNwvtfWie6NEPxvU+X1BShXss0BmI8y3ISquC2wifV1tTwCh/XyBEKksDgIfPDkBl92FLcfO7Inz+XHkb7zaZViJ502smkDHbm0xmA1Yos30GBj8bNspw3/mFmtPHmnQl2P7Ct2yiXAWUtqRJ29HZjyHTO2KdK497Tknas5nqFnvID3Hwmaj0JVHWO4o1AlouDm1uSM5Y6MnWORNBuzav9xvgz+/D8jskciswcisEci0hwKeR8R/CtYHwPogIn5sEC0ML2U2a+h8whJtxpZjz3/sdnnYt+VgiecVQvDqlOdJPZxGdHwUlihziec8nbRj6Uwe+jMuh4uje1L4avAPDJr8XP7z21bs5NMXviYuMZYXPu9DfMVyQVk3Oz2Hae/NQG/QcdcLt2KJtgRl3kCRUrJo6nJ2r9/LdQ9cSa1LfLi6d+0A9STIXG0O20yEsbX2c9abYJsOuJD22ZA0v8zeKYnY15HpvUFNg5hXEcJU/EH+om+k3TkBGBoFf35fcCzSNngB3FuRUkUI/6+DhZKAiH01yMaFn8gdQRB4e+YgGrSuQ1L18uj0OgwmA137dMbj8fDJc1/yeNP+/PLRrIDmFkKQVK180J2A065lKeWvo1OwRJ86CUgpGXTT2ySv2MXq2et574lPg7b24FtH8NOo3/l++K+8fe/7QZs3UOZNWcK7j43jh5G/0a/Da2SmZhV7jNTVACkBAVgQxtPCsK4tgANQQT2CzA7/aywMoa+NkjQXpeJaFOudoVkjfhxE9YaoPohyobmKlmoaatpDqMevQc2bfu4AS3fAouX5G68IyAlcyETejdNYPmMNfdsP4r0nxuOwOXw+rn6rOoz6+w0cNic6vYIQ0PK6ZsydtJDZE+ezf9shJr46hR1r/g2h9b6zYf4W7kh8jMca9aPdzS2pXKciLa9txmPDe+aPUT0qjjzNUXjcKunHM4O2/r7NB3G7PLicbnat2xu0eQMleeUuHHna31tKydE9KUWOl1JC1mC0sIoAHMjs0Uj3AW2AtZf3917yvkCq2aEwvcwiPcdRMweiZg4CaUeJ6YcS01fbuwjFetmjwbkG1MOQ9RLqsTaoOacuXpTo3oiELxFx72vhnQhnEHEEXtJTMni7x/tsX7Wbf75bzJThv/h1/P5th3A73TjtLpx2F6tnbyA7PRfVo232KopCbkZuKEz3m89fmYwjz4Hb5WH59DVMTP6AEX++dsbmtk6v4/GRD6DT67DGWHj6/UeCtv5tz9yIyWrEZDFyZ/9bfDpm39aDzBj/F/u3lTzkdjbXP9AJc5QJS4yFxGrlqdO8ZtEHqEe0UANOtMKk/678RwOgWG8D0w2cirwaIMgbsGUdmf4k2H4H22/IjKd9O0ZK1NwpqJmDkM4Nfi5oR/tbgLbPkQU5Y5HufflDhLENwnwNQgRWuS8dS1DTHkPNfh8p3QHNUVaJ7BF4yc2y5f/scrhJO5ru1/G1mtbAZDGielSklHS4rQ31W9dh7jcL2LflIJde05QW11wSbLMDomLNJPZs3Ifb5SEq1opOrytw3F39u9Lt2ZtQdAqKErxrhl7D7+fa+65A0euo2bhaseP3Jx+ib/tBqKpEUQRjV4+kRqOqQbOnUbv6TEz+kCP/HqNRu3oYzcWctEUcCN1Z+6o6OO1qV8QNQ2a+DuoRRPSA0MTeA0S69522LzAoNCEhzyHyC7fcvjlvaZsG2SPR6h7+gMS5CJ1vKjQi+gWkayt4DpLvnAnevoz0pCDTnwbs4FyDFHGI6McCm8t9SLuYMFwa9AytQIk4Ai9V61Xi2vuuYM6kBcQmxtBjYHe/jrfGWJiwaQyr/9xA7WY1qNeyNgATNo7B4/Gg0xV8svUXW46NTYuSqdagMlXrVfbpmNysPHIzckmqnogQgv4TnsQUZSIjJZNeI3oWuZGpN4TmI1K7WTFX3aexdekOpJQ4bU7MUSa2LdsRVEcAkFStPEnVyvs0VijRkPANMudLQIJzHSgWMN9+2pg4RPxHQbUxWMisoeDZD0jIegNp6Rp8RxXdF7JHnfrZF9w70ArgAAR4joKvjkBfDcr/isz9Amy/gJoD0b0Q+lr+Wl4w6klOORaH1+H4j3Qs1+6WhA50NTVdIn+K2kJEyArKgklpFpQ5bA6MZmOZzPJw2p080exF0o9noHpUhv/xGs2vbFLkMZsXJzOoyzCkqtLpzvYM/Pa5IseHmqN7U0hevpMmHRpSqZZvX/JDu47yVKuXQWhfxU/Xv0uVupVCa6iPSOcqZNrjgFMrUCo/M+w1BdK5AWQO0tAebFPAuQZhvQdhugIANe1Jrb4AFTAiKga/dgBAelIBgdD55mClawcyrYe2Ca+vjSj/k19hHDW9DziWABIMzVDK/xCY4QXZJlXtBO5cDsKKKP8jQl/b73nUjP5aPQMAFkTiL4gQ1iOEvaDsfMVkKdmVkdvlZtIbP7Jr/V7uGXAbra5vHiTLYO+Wg6SnZOSnqs6bsqRYRzBl+M/5G6ELpy6nz3sPUy4pLmg2+cORf4/Rp+VL+Y99PaFXq1+Zzza8y9ZlO2jasRGV61QMpZn+4dyItmmsAgq4d0IYHYGa8znkfAJCgK6OV2rCpmkNJc5E6Gsg4oZ4q4lPQMxrIQtPCF2if+MNDSFpgXYnoK/n/5WyazPavg2aJlAQEUKB+AlaBbYSF/h7ZmitqZdiB2EApWxc0EQ2i4PM1DHT+fWjP1g7ZyNv3D6S9JSMoM1dpW5FFJ2CTqdgsppoflXRTgCgRqOqGM3aVZXJYsQaay2RDb+P/ZNHG/dj9OPjcDn90z7auGArqiqx5dhRVcnGBVt9PrZK3Urc8OBVBTqB4wdT+XvyIg7u8ENALFiYr9E2gkU0YAJjq9K34XRs0wCbJozm3oJWyIUWivAc8v5YBSXxZ5QKi1AsN4bN1IIQShzC0CiwcIn1AcAMWMByb7BNQwiB0CWVyHEK6/0Q3QcMHSBuDEKJCqKFgRO5Iwgyx/aewGH3XpUIQWZqdtAKsWLioxm3ZiQLflxG7WY1uPzW4oXdeo24H4PZyLE9KfQY1L1EvQ72bjnA5y9PxmFzcvxAKvVa1KZb3y4+H9+kQ0MEYDQbEEJ7XFJSj6TRu/mLeDwqSMmHS4cVn/VTCNtX7SJ5xS7a3nQp1RpU8ekYoa8HibO1K1DDpQglOH/rQFBtf2gyyyiAHnS1QD2uZdToaoLxwhYCVKKfQpquBdygL/4iKSy4d0LOp4AHMtch9b8FFGIKNhFHEGTu7N+VxdOWY89z0Or6ZtRoHNxNzSp1K9Hz1Tt8Hm80G4MmVJeXZUMo2t6J6vaQk5Hj1/E1G1fjoxXD2bhgKy2uviQ/Y8jpcDF97GxyMnLp1reLX6Gr5OU7karEnmNHb9Szdu6mgBzB1mU7eKXzW6geyVeDv+fLbR/4vnmsqwS68N7iSzUTMl9GC40oYGiCiJ/ovRNIAV11r/LmhY0wlPziIqS41qGlmzkBK7g2QMQRXHjUaFSVH458Tm5GLuUqxPm86Zx1MpstS7dTt0UtKtYMb/+FvybN55/Ji2l/a2u69705/zU0bl+fjt3aMe/7JVRrWJXbnj67JXXx1G5ag9pNz2w09OFTn7PghyV43CqLf17Jl1t9r8Rt2LYuUkrvXYbwKVxWEJsXbcPtcOPxqBhMenat2+OzIygTnJ3Xrqt6qngrWJkzEUqOsT0IBaQVEGBsF26LgEjWUJkg62Q2jzV5HpfdhaqqfLRsWIHplcumr2bamBk0bFePXsN7hiS1c9uKnbx8/Vs48hyYo0y8OuX5c0JQwZaa7nXJ8xxI1uL7QhH85frRr/kP7TrKurmbqHtpTXIzbVStX8nn1Nr/2L1hL/2veB2hE+gNer7c9gHxFcKzqR4oas44yBkHuoqI+K/Ou3aJFwvSfVDraGZshdD5FoIMlEjW0HnE1mU7cDlc5GXb0OkUVsxcd44jOH4wleH3fYDD5mTnmn9JqFSOu1+8Lei2pB46ifJf+EeVnDh48pwxwU6t7d7vFj7t/zVCEXS6s73f81erX5kK1ct7U2szvam1rxabUXU69S6tzfj177J73R6aX9UkKE5ASre2d6BUROhCf5enRD8N0b5V8RaEdB9EZg8DFETs4JCfpC5WhL56WDPLCiLiCMoAdVvURPVIdHodeqOOJh0anDMmKzU7Pz7vdLgKPEEXxe71e5n0vx9JqpbIEyPvz1f8XDNnI5Pe+IFqDarQd+zjtLu5FZXrVOTgjsOUr1SOq+/tUPIXWAxde99A8yubYMu20aBNYDnVe7ccJP14BrZsLUtm/vfFp9aeTbX6lalW3787icKQ0oNMe8CbxighfmK+OmlZQLr3gHs/GNvlZ67I9CfB8y8gkOlHEYm/htfIs1DtCyDzFa1eI/4ThCHw1Gyp5iBzPgKZi4h+5qJ3ehFHUAaoUCOJj1cMZ8XMtTS5vEGBJ7A6LWrS6obmrJy1jpj4aLr3u9nn+d0uNwOu/R+5mXkYTAbcThcvfP4UuVl5DOk+CqfNyb8b9xEVZ+XZj3sxft0oMo5nEpcUG7SK6OIoaaVwlboVURQFRadgMBlo5qcTCDqe/eBK5r9KWZk3ucw4AulYikx/SttIVpKQ0S8iPHtATSG/J4FaBntTZL6odQeTIDMGIpL+CHgqmfkKOBYAHqRzJSLp76CZeT4ScQRlhFqXVC9SB19RFN785WWy03OwxlgK1QcqCKfdhT1XKypzOVwc3q01S7Hn2vG4NT0Yt9NN2rH0/LUSKsUH+lLCQkx8NGNXv8OCH5dRp3lNn1JrQ4pS4TQ9IgsU2RymdJG26YBds83jhMwBSNxopwNvenHMwPAZWCinfeYDFI7Lx70b8NbBeMp+579Qc9EVlHk8HvZuOUDWyfNTFjgmPtovJwCaDtKtT3VGb9RjjjLx8Jtasc1Xg3/IF04zWkw89L/gF+GEGo9b6/nwZMsBbFywlftfu9NnJyCl5MD2w2ScCJ7E9n8IJRqR8ANY7oeYVxDWR4K+RsAYLwcsaCd+TUYbPFoj94RJiIprUCy3htXEghDxn4CuGujqIeLeLdlkUc+gNbE3gfXRYJh3XnNRZQ15PB5euu4tdq39F4Rg9D9DaNi2XhAsPD/IOJGJOcqM2arJaNxouBfVo0n3Vq1fia93fBy0tfZu3s/xA6lcem3TEst2FMWM8X/x2Uvf4MhzYrIa+XjFiHPSUwtCSslbd49h9ez1AAydPrDAvtFlDanmaGEnfR2ECLyzm7TPA/e/SF1VyByIdk3oVe00XYUo92GZEEMLJdJzAqRDE6y7QPE1a+iiuiM4tOMIO9f8iz3XgT3Hzu9jZ4fbpFKlXFJcvhMAqNG4GjqDDqPZQOPLzt2gDpQlv66k7+WvMqznBzzX4TU83p4MoSDzZDZu16meDznpvvV8yDiRxYqZa3HkOXHkOflx1O8hszFYSE8K8sT1yLQHkCc6I1X/5EtU21zUlJaoKe1AiUFEP4FiuRmROAOsPbyjnOBcCo5/gv8CyhhClwS6qloPhIz+SMeycJsUNi4qR5BQOR5Fp2XemKymgKUILhRGzxvCfQO788jb99F/wpNBm3fO1wtw5DmxZds5tPOo3xlO/tD1yRuoXLsCQgha3dCcSzr6VlkaFWfFHGVCCDBajNRtcR58Fhx/axpCMhfUbHAs9u/4rEHasTIDmXmq767Q10QYW2hCdQBIKEP9E0KKfRbkjAT7LGR6n1Nd5i4yLux7v7OIiY9mzPw3+X3sbOo0r+mXTs6FSFxibP5+QTBpdUNz1s/bjNvpJirWSkLl0G08l0uK46vtH+F2uf0qsDOaDHyweChTx8ygUu2K9Hjl9uIPCjf6+pzSxFfBX/liYQaZpc1xdljJdCOYV2iZNOYbwXhVye31E+k5hsz7CaGrjDTdCBm9wLUVzDch4kYHrc+wVDORab28SrENvN3N8MpxHIaLsBDvotoj8Benw0VOeg7xFcuVyf4EwWbL0u283/tTzFFmBn3XL+CceiklS35ZydE9KVx7fycSqyQE2dKLF2mfg3QsRJg7I0z+naylcyMy61XAhCg3ShPMKyNI6UaeuNrbAMagCeQ5V6H1ebAiyn2KMLUPylpq9keQ+xla1pAZ0IPwgK4movzUsHaTk9IBeVMBCdZ7SmxLpLK4hOxPPkT/KwZjz3PQ5sZL+d8vLwW1XWNZZEi3UWSdzEYIeOfBj/hkxYiA5hFCqxCOEHyEuTPC3DmwY40tEImzih8YDmQWqOlo7S092pX5f5FrKc+9gykJwsgZbSwTvkYoVtDVCssGuXQfBBwIfT1kRj9wLAUk2Bcjyk8oFRsijqAQfn5/Jtnejcd1f2/mQPLhIvP8LwScDi2vWkpweOsOIgQfKR2gZvvduCU4a9tBOhFKbKmvXSQiXmva4t4EUoWo58C1RLsrsNyl7WEEa6moh5GuzVojG+vDKMbgNY/yFzX3B8geBgik9T5wrEFL5wVcC5FqOkIJfU3PhX2JWwIq1a6AyeJtQCElcYkxpbb2oV1HmTdlMScOhW6TtSAGTHwaS4yFchVi6f95n1Jd+2JBupKRxzsgT1ylZaqUYmhWOlYgUy5DHm+Pmu27wmtpIIRAJExExE9AJP6GYr0FJW4EStI/KNFPBXktC0r8WK0xT3SvoM59OmredNTU7qiZ/0NKZ8GD8j5HO/HbIW8ymE5XIxVa/+VSIHJHUAh3v3greZl57N6wj3sG3Ba05jLFsXfLAZ67/FWEECg6hS+2vEdi1dKRQ77qrsu56q7LQzb/yaPpbFu2gwZt6p4htb3wp2X8+O7vNGhdh6c/fKxEzXPKOjJ3giaTAGD/G6IPnSNAptr+gOwRoFTQNHV0QdI/yhlDfnP43M+Q0c+ErE1lIAihLzOyzCVFuvdD1mDA7q3XqImILqBwTd9I6xeBqjUSinoaHIvQ+hWYQCmd/bWIIziLnWv/ZVCXYeRl2XjqvYd5/J0Hznh+06Jt2LJttLnxUr8rfH1h/d+b8bg9uBxuLDFmti7dwVX3hF74LdSkHj7JE81eRPWoSCkZu/odqjesSsr+E4x6dCxOm5P9Ww9SqVYFegzsHm5zQ4euOlpFqwNQQDlT5VRKx6kGM+oJZNZQRPy4IK1dFVzbAJe3tWbk6x8yZKaWjisBXCBTCxwm4kYhcz8HmYeI6o3QJaLGvg55P2oV4ObSyWaLfBLOYuxzE8lK1a7Yxj3/NTc/cX1+WuLkt6fx48jfEEJw6TVNeev3V4qdb8XMtfz2yZ9c0qEh9w++s9gN56adGqHoFK3PsISG7cpOZkdJ2LQoGY/bgy3HjsFkYO2cTVRvWJWcjNz8jCy3y0PaseD1eC6LiOhntZO9ezciuk8BsXqVfN0POLfhTEnWjn0LKczgOYGIeTFo6ZgXClqYTi22k5uUHmTeD+A5hLDeh9DXQDqWIDNfBoyI+I9B3xSMV4PjL02G3PpQgXMJJQoR8/wZv1Os94K1dOVeIo7gLKLLRaHoFFRvpypFd+rL8ve3i/LF21bMWltsg5aje1N4+973cNicbFmSTHzFcnR98oYi12/Qui4fLH6bLUu30/qG5lSqVSE4LyzMNGijdRLTG3QoisiX2q7TvCYdu7VjwQ9LSKgcz10vdA2zpaFFCCMidlARz1uQsW9C9jugJCJiBwdvbSUWEfdO0Oa7kJCuXci0B0FmIqOeQol5rvCxueMg53PAibT9AhUWIzP6a3cBgMx4BSXpD0T8h5rTx1jm08/D4giEEP2Bx9EufTYDj0r5X1VHeOn/eR/efeQTMo5n8fSHj55xBd+6c3NOHklDVSV1m9cs9o+bfiwD4XUkLruLY3tTfLKhXsva1GsZ/j6mwaRa/cp8tGwYa+ZsolmnRjRorRVDCSEYNPk5XvyiD39PXsSIBz/msptbcu/L3cr8lydUKNa7wHpXuM0IGdJzGDzHwdCszOgZyZwxINMBqe2fRD2EUArZF3RuALynK5kLagZnpKOedqcVzpoEfyj1v4IQoirwHNBESmkTQvwE9AC+Lm1bCiKxSgIj57xR4HNPf/AojS9rgC3bxvUPXlnsXA3b1qNR23psXbadqDgrXfsElv99PuJyuti1dg8VaiTmb3bXblazwBacALs37GPc81/jyHOwe90eqjWowhXdLytNkyOUEOk+pIVC9A0Qpk4Fj3EsQaY/rZ0s9U0g4btSc/jScxSZ9Q4IBRHzCkJX6dSTSjza6dCFJndd+Ca6iHoI6VytvQZDC1CSEOU+QmYO1JrmlFQZNQyEyx3rAYsQwgVYgfNCEFyn03H9A5oDyM3MZf+2g1RrWKXQ5i06vY5Rf79BekoGMQnRGIwXbjbM6XjcHvp1eI1DO48ipWTE7ME07dioyGPSjqafapHpUTl5JL00TL1gkdKFzP4I3NsRUY8jTKF1qlLNRp7srmkhoUfGvYNiOVfCReZNJr8XgmszqEehlLqDyfQ+4N6h/ezej0g8lZopYgYi1Swt7h8zQCswKwRhugqS/gRPKhiaao7M1B5RYUGoX0LIKPXdIinlYWA0cAA4CmRKKeeEar1j+44z95uFHNoZPF+zd/N+etZ8imcvG0T/Tq/nN3f5j5yMXH4fO5uFP2lqhgmV4oPiBKSUrPt7E8umrz5nzeLYn3yI5JW7SiVv/cD2wxzccQRbjh17roNv35pa7DHturSkeuOq6A06kqqV55r7Oga8/vGDqXw1+Htmfjb3DOXT3Kw83u7xPk+2HMCy6asDnv98QOZ+CXmTwLkQmf4EUk0L7YKefYAb7YraBs5ClDwNLdF6IXj1jpTSSY0GwHOUfKlt9egZTwklTqstSPy90LuZM8brqmqV2sVsLJ8vhCM0FA/cDtQGMoCpQogHpJSTzxrXG+gNUKNGYCJQKftP8GSLAahSBQkfrxgRlOrgWRPmkpel5WPv23KQ3Rv20dDba1dKyfNXDObonhQUncK+bQd5uJiGL/Y8BwaTvsi2kEf3pvDWXWPYu+UABqOeNp0vZcjPA3yyd+aEufnN4Tvc3pZBk/v5+EoDI6la+TNCppsWbGXdP5tpdV3Bev9Lf1vFrx//QesbmjNs5iBiy8cELOfhcXvoe9kgMk5kYTDpOXEolUeH3gfA14O/Z9lvq3A53Qzr8T4/HJ5ATHx0QOuUeTz7yY9j40F6UhGhzEnX1wfhTYWVHoS54FaqIuoJpCgHnn0Ia49Si6FLxxIwdgDHHECBaN++OxcL4cgfux7YK6U8IaV0Ab8A5yTKSyknSCnbSCnbJCUlnTOJL2xduh0pJfYcB6pHZcP8LSWz3EutpjUweXX9pZQkVTv1BXPYnBzccSS/PeSqP9YXOdf4F76mW7mHuKtCL/7duK/AMaqq0q/DYHav34vH5cGe62DFrLU+2/vrR3/gsDmx5zqY/8NSVFX1+dhAiC4XRe+RD6I3atcZqkc71O99AAAgAElEQVQlecXOAsce3ZPCiPs/ZOP8rfzywSwWT1tRIk2n3Mw8sk5mo3pUHHlOtizZnv9cRuqp3gVSgiMveDIah3YdZfVfG7AHcc4SYXmQU97YA455IV1OCDMicQYibiQi8ReEqeDCRCEUlKh7UWJfQehLR/pbzZuOTH9Gk/EWMZA0D8V6Z6msfb4QDkdwAGgvhLAKbZfoOiA5FAs1bt8AKTW9eaEoBTaFD4Sbn7ieXiN6cv0DVzJyzutn9Pc1W0006dAAS7QZk9VE54cLV4hMP57J9HF/4XGr5KTnMmnIjwWOc+Q5yDqZlf9YCFFszP10Lrm8AUaLEZ1BR/WGVUpFPO/qHh2JTYjGGmvBaDHS4baCBRDTU07LrHK4STlQcOGNr8QkRNP8qku0999ipNuzp+LUD795L/GVyiEUQbe+XYJWsb3un830afkSQ+8ZwzNtX8HldAVl3pIgdAmc2vBUwen7hUPAayoxmiheGVI1BcC5GK2i2gk4EH429LkYKPXQkJRypRBiGrAOLai4HgiJxF7lOhUZv3Yk6+dtoWnHhoVmrPiLoih071vwrS/AyDlvsGb2BuKSYrmkQ+GNUsxWIzq9gtsJBpOe8oXINVuiLXS6sz3LZ6zB4/Jw8xPX0/vdB322t+/Yx6nVrAYnDp4kulwUS39bRYfb24Y0WyMmPpovtr7PtuU7qd20OhVqFHxX17BdPS65vAEbF24jJj6a254qWWaVEILhf7xK8spdxFeMo2q9U/IM1epX5odDn6GqapFhOH+ZO2lB/t3F8QOpHNp51Kd2mSFFqQCGpuBOBqkiou4Prz1hRFhuQdr/0rJ8lPKgrxVuk8ockX4EYWb9vM1MGvIjVepW4tmPe2GNKVhuV0rJvi0HiI6P1mLwfuJ2uXmo3rNkHM9Cp1d49O0e3NGvbBRvSSnJOplNdLmokMh2BIsD2w+zZXEyza5sTPWGVfN/P/ureYx9biJOu4uoWAuT940v9O/4H1J6Qr7RKKULXBu0ytYLtNmKdCxH5n0PhhaIqEcLrZaWru3avomxI0K5QPeFCiDSj+A8oeW1zXxqmi6EKNEdTcbxTDJPZOFyuHA5YNWfG8qMIxBCEJfonyyyx+1h8tBp7Fy7hzv7dy10IzpYHNh+mGfavpJfTT5+3bv5jXtufOQaomKtHNh+mGvvu6JIJyClRGa9BrafkbqqiIQpZ+azBxEhDGBsW+QYqWaBiDovs1+k5ygy/UnArmVHiRhE1D0FjhWGRmDwPZx6sRFxBBcJCZXjqdagCkf3pKCqkhseKv1WhMHk14/+YOro6ThsTjYu2Mo3uz8+Y68m2GxZnOzdYHZijjKxZcn2fEfgVyMe9y6wzQQkeI4ic78IqoyEr0ipIjOe1VpTKuUg4SeEvlqp21EiPMe0cI9Eazfp2RNui85bIo7AS3Z6Dq/dMpy9Ww5y65M30PvdgkWizlcUReHDZcNYO2cjSdXL50s8nK8c3n0Uh13TeBcC0lMyQ+oIml3ZGCHAHGUCBM06BXh1qURxSlROX2oyw+fgTvZ2wnKDmobM+wYR+2qxh5UpDE1B3xBcySCMCGvBdwMRiifiCLz88sFMdq3dg9vlYfr4OVz3wJXUbVEr3GYFFbPVRMduF4bee/fnbmHBj8tw5DlpekVjajcLbQy8esOqjF87ii1Ld9CsU6MzNqH9QeiqImOHQd4E0DdFRIWuMUqRKPGcckhGUEITngoGUqrInE/AtRZhfQBh1oQbhTBAwvfgOajJPBRQDSzVTGTeDwgR5e0BbPTOWbRgZCiR0uXtEhcVlvULIuIIvOgNeoRX4gCvSmaEskuNRlX58fAEMlOzSayaUCpf6moNqlCtgSaHsPqvDRxMPkynu9r7vXmvWG8D622hMNFnhK4KlHsPmTsRDM0RUb5noZU6tqmQ+yVgQzrXQ+KM/BoEIRQooh5Bpj0M7l1IdODaAjH9kGkPgOcw0nIfIvaNEn12pJqGTHtCq6yOegIluujOftK1CZn2CEg70vooSuxLAa8dTCKOwMsdz99C8qrd7F63h+7P3UzNJhd2f+ILAaPZGFAGVSAs/W0Va//exJV3tiftaDrv9f4M1e1hyvCf+XbPWCzRQWyuXkoI8/UI8/XhNqNYpOcoWg0A2p6AegLwMXHCvQPwAC5wrUHmTADPYUAF288Q9SDo6wRuW/ZYLcyGG3LGIs1di9xrkdnvgczRHuR9hYzug1BKrw1uYVzQjmB/8iEObDvEpdc2LVZKwBJt4e3pA0vJsvOXvGwbK2asIal6Is06Nfb5uNysPHav30vNJtUolxRX/AGlgNvlRvWoGM0FK02mHklj+H0fcGT3UbLScnA53Mz5ej6trm+eXzfgtLs4uuc4dZqXTpXs+Yx070GmPwVqJsQOKVCUriCE9R6k7SdQs0DfHAyX+r6o+RatoliqYO2pzYEezbFIEIWLy/lm3FmRg+Ka/egqAwa0LnFGpJqFzBmrZW5F9SpS7C6UXLCOYNOibbx683AUnSAq1srE5A+CctXmtDuZ9/1SzFYjne5qH9TCpLKOqqo8d/mrpBxIRaqSpz94hJsfL/6KMjs9hyeavYAtR9O+GbdmZMAx9mCxYf4WBt/6Dm6Xm8dH9OSuF84N1Yx9biJbl+1A9ZyS5FB0CrWb1mDd35tQdArlEmOp1iC8r+V8QWYN8YrTSch8CWm+wad+BEJXBZIWaidxxb8woIgbBa51IKwIQxOktCE9B8C1HaL6lDh1V0Q/jXRt0zKWzLcjs95B6mogYp4rsB+0iHkNKT3gOYKI6Y9Mf9z7nuiQ7l2I+I9KZE+gXLCOYOmvK/Ov2gSCfzfu90uWAWDZ9NUsnraC9re24aq7Ne2U128bydZlOxACNi7cRr9xTwTddl9w2p38787RbFmynSvvbs8LE/qcIx2RlZatVbk2q4ElylzsnLvW7WHt3E20uPoSGl9W/5znM45ncuTfY7gcWvvEBT8u88kRbFq4jbxsO7ZsG3qDjmW/rebuAeGNkX/6wqT8z8cXA6dwx/Ndz3n/7HkO5Gm6TJZoM9Hlorh7wG1cfW8HDu06RqvrmxV6RxHhbM4+gftxQhcG0PkfBhRCAWOb0x5bEOXe83ueQudXyiHKT0ZKO/J4B2/Yx4TEjYg9N8IglGhEuVH5j6VnP1roygPubUGzy18uWEdw6bXNmPX5P6huD4pOoUZjrRJ0zZyNLJu+msu6tOSyW1oXevy092fy2YuTAFg4dTnxFeNofmUTNi3ahtupnQhX/1m0oFwomTNpIZsWbMVhc7Lwp+Vcc29HWt/QIv/5o3tSeLrNK3g8KjHxUUzYOJqouMKzFPYnH6L/lW/gdrrRG3W8v2go9VtpsdMJL3/DjPFzqNGkGolVE0g7loEQgo7dii5W+o+al1TPv6rWGXTUaxX+7mvlq8Szb+sBPG6VqDhrgVeZT733CIO6DCPzRBYPDbmbZlc2oVbT6liiNIcQLMmSiwUROxSZ8YzW0StmyHlZxFYoajZI7z4GDnBrNQ1SSnD8o131m7ueewdivQ/ypgISosJzUQkXsCO4/NY2DJ3+Cns27qdj93bEJsSwa90e/nfHKBx5TuZ8vYCRc14vVAvom//9lP+zy+ni4PbDNL+yCe26tGTdP5tBSq68u2CFxdJAp1dOXVBJeUZvZYDFP6/AnmvPV9vcuGAbHW4v/MS9c82/CKFV7OoMOrav3EX9VnXYu3k/08f9hSPPyZ4N++jWtwvVG1UjqXp52t7oW6y2Wv3KjJr7OgunLqPJ5Q19qqQONQO+eoaPnv6c7LQc+ox5uEBHUKNRVb7bOy5kNvz93SK+GfIjVetX4dUp/S5cSWwvQl8TkTjznN9LNRdc60FX65yNVqlmaI5DV3xr2LCiJIK5M9jnah3QvNlD0jYVsoahbSZ/ARUWIMSpu3MRMxgsPUCYEfrwJahcsI4AzpVv2Lf14KkPk5Ts3XygUEcQFWvBlq31HFB0Ch28+fdvTH2R5TPWYLKaaNO5RYHHlgbXP3gla//exMb5W7iuZycuvabpGc/XaVELnUGH2+VB9ahUb1R0F6iW1zZFb9BjjdUqNdt4T/I6g57/5KiEIrDGWrn58ev8tjc7PZeZn87l14/+ZMqwXxjy8wCq1A1f7np8hTiGTAufJn3GiUzef+JTnHYXKftTGfnQxwydPjBoJzspXd6TkhFM1xaqwRP4/E5Q00GpULL0S2lHnrwN1DRtQzfha4Sxpfaccw0yvZemGW7qBOU+KbPOQAgBcWMg5iiI2FN6Ro4laMqngLSB58gZWUpCCDCcG4YtbS4q0bmME5n0bj4Ah82B3qjnsw2jSSxE8XPXuj2MevQTVI/k5a+fzW88cz6x5NeVbJi3mSvv7nCGBPf2VbvYt+Ugl93SiviKpxp0px1LZ/vK3TRoU+cMieafRk/nlw9mUq9lbV77/vmANt17NXmeA9sP5z+uWCuJyXtCd7Vd1kk9fJKH6vXF5dAkqxW9wj0DbqPX8OCohKrpT4PD2yXM0h0lbkhQ5gWvxs/JO7XNW0NTRMK3Wgw/kLmcG5Dpj2pN4AEsD6LEvQ6Amv4kOOZ7RxoRSX+HTJcpVEjHAmT6c1p2kVIJkTg94PcqEHwVnbuoHAFo6Y/7tx2iRuOqRMWGJ1UrnKz6cz1v3T0agcAcbWbSro+LVcosKfY8Bw/UfprME6d6KuiNev60fx/SdYsjL9vGt29NJScth/tfv4tKtSqU6vqT357GpDdO9aCoUrcik3Z9EpS51WPNye9QplREqbDYr+OlcyMy72utEX3UE2dk96jZH0PuWEDVsnHKfRZwT2SppiNPXOftdWxCxI1AWDSJdzVrjNZuEzuIaESFZWeEVc4XpGuXVv1sbF/q6aER9dFCsMZYCsyIKSm71mmbQ/9tsJZVls9YgyNP29RS9AoHkg/RqF1g74eqqnz6wiSWz1jDVfd0oNfwngXeus/4dA65WXn5j3V6hfsHl6xDlC3XzsRB35FyIJUH37g7oPf9vcfHs2z6GtwuN+vnbWFyCPcDCuKBwXexYf4WdqzaDcDltxWjFOpJAftMLZZuLiY8Z+oAjuXen6/1yy6pZiHTH/aenP9BIvJj3uCVyRAmLdQhVdAF7kCFEg/lpyJtfyAMjc8ocBMxzyEVM7j3IaIeAwRq7hSwz9AcUOygstcEpwCEoX6ZCP8UxUXnCELBxNem8MuHfwDQrW8XHh9RdpuAdLitDXO/WYBAYDQbqdE4cMXJJb+s5M8v/8Ge6+D3T/6kxVVNaHtTy3PGqR41f1/bZDXyxtQBtOty7jh/+PTFScydtBCXw8XGBVuZdvxLDEb/brn3bTuUH5o5fjCVp9u+THyFcrz45VMhFbA7naG/v8KHT31OXlYetz1zU6HjpHQgT3bXirGEHqm+imItvBe2KPcx2P8CYQDTDf4ZpZ7UTvAA2MF9VptRSzdQU8C5CmHtCZ4U1KxhoG+EiOnnd+hD6OsiYvqe+3thQEQ/c5pZPbWaADTbZNoORIUlfq0VoWDC0arygmPWhLk48hw48hzMmjC30HF7txxg4dTlZKfnlKJ1Z9L2ppaMnvc/nvnoMSZsGlOisNB/BWIACHHm49O4tU9najerid6go9V1zWl2ZWMKCklmp+f43PP3yO5j+Sdxp82Zf5fjDw++cTdGswGj2YCiU9i1di9r52zk/d6f+T1XoHw/4leW/LKSFbPW0a/Dq7hd7oIHeo6Cmgu4tCtxR9GhHiEMCEtXhPlG/zeKdTXB2A4weyteHz1rbgUl+imUhK/A2BqZ3huciyDvG2Tul/6t5SNSesC1hv+cAKBtVkcIChFHEATqt6qDwaTHYNIXGqJY989m+rYfxJhe4+jdYgC23IJPmqAVi417/isGdXmbzYuD3865Ubv63PTYtcRXKJnUwzU9OtKoXX10eh3Nr2xcaHqqLcfOwe2Hcbs8rJy1ltvjHuKB2k+Tevhk/pjJb0/jnspPcGfio6yeXXx9xkND7sEcZcJg0nPTY9cSXc5/Jcer7r6cyfvGM2HjmHwhTo9HJTM12++5AiV55S4cNidSleRk5JGTkVvwQF010FUBEQWYEZY7QmaTEAoi/nOtGX3SYoShiHRfNe20Bw6tC1hIbNKBoRWn+jArEBO+rK8LjUhoKAgM+XkA08f9hapKuj1b8O394mnL869ahRAc2HaIhm0Ljm9O+t9PzJowF6fdxebF2/nxyAS/N7b//PIf5k1ZwuW3t+GO527x7wX5iNFs5N1/is9GObj9cP7egapqZ9zUw2lMH/cXjw3riaqqfPvm1Pyis89fmVxgiOl0mnVqzE/HvsCeYz8j88lf4ivEEV8hjt7vPsiEl77FEm3i6Q8e8fl4l9PF7InzcTlcdHn8Op8quE/nzue7krxiJ0IImnVqUminNiH0UP5ncC4HXTWEofBe2MFACFGkqmc+urpgugIci7RceOtjobMp4Wuwz0ZiBWMHFF146i6kmonMehM8KYiYlxHG4KeRS89xZObrILMQsa8hDE2LP6gERBxBELBEW7j35W5Fjmlz46XM/XYRqkdFZ9AVqU9zbE8KTrsW9pCqSm5Grl+OYMuSZMb2+wpHnoPtq3ZRrX6VEsfkS0LDtnWJTojC7fHg9DpDg1FPfCXtBC6EILZ8DJknMlH0OirV9m3z0RJl9vvEWxh39LuF25+9CUVR/MpVH/XIWJb/vhopJUt/W8WY+W/6tW77rq2ZuO0D0lMyqdeqdpFrC8UKxW0SBwmppiFzxgM6RPRTCKXgu0chBJQbqymCKnEIYQqZTUKYwdLND2GK0CCz3gT7bMCNTH8EKqw5p0paShVwBfx+yMyXwbkS8CDTeiEqriyp2UUScQSlRMdu7Rg6/RX2bj7AFXdcVqTcw32D7mDt3E04bE463dmepOqJfq114lBafm8FKSUnDqaWyPaSYom28PqPL/DiNf/DYNLnt8q8tU9nQDuZjJ43hC8GfkdMQjR9xjwcFjsDERDcvCgZh01zbv9l//hLhRpJVKiRFNCxoUBKiUy9G9TDgIJ0b0MkfFPoeCFEiTKHzjs8xwDvXo60Ay7g1GdHuvcgT/YEmYG03IsS59/FAaBt2OPxTpgd8kY6EUdQivjaqL5ey9r8dOwL8rLyApJsbn9ra6rUqcjBHYdJrJLAVfd0CMTcoLJ27ibcTk322Rpjof2tbdAbTn38ajapztDzUAb8xseu4Zf3ZwKCq+4tPckRNecTTaPG2A4RN6xApctAkXkTQT3430rgCszBXaiImJeQ6Y+BdEDUk+fUNsicsSDTAQm2X5DRvRG6qn6uMRiZ0RukC2IGhbyiOuIIyihGkwFjgLr9ligz49eNIjM1m9jy0WVCKrtRu3oYTHocNieqqlK7aXWmvTeD70f8StUGlRn6+yuFxsfLMo++1YPLurTE5XCTlZbNg3WfoUKNRF77/vmQpaBK5wbI/VzLHrL/BcZWmnhZsHCcFYaw9gje3BcAwtgSKqz2tpssIGSrJKFtajsA4d3g93MN02VQYT3gKZVK5IgjuEBRFKXEWUEAK2etZeG05Vx2c+t8Ke5AaH1DC4b8/BLJK3Zq2UVCMHHw97jsLnLX5PHNm1Pp+3F4+vduW7GTmZ/OoX7rOtz+zE3nyFEXR5PLG2LLtXNn0mO47C5OHEhlXP+vGfx9/9AYLO2cpjjofRw8hPVepHOFtoauBiL62aDOfyEghB4K6aUgop9Dykxw70ZEP4tQAktm0NJ+SyexM+IIwkhORi6vdB7Kno37ue6BTrz4xVNlSlRr59p/GXrvezjynCyauoJySbG0uPqSgOdre+Ol+Yqlx8O8b/Ef6cczeeWGt7DnOlg8bQU6vY7bnrrR73lUj4r0ZkSpUuZviocEYzswdQb7H2C4BCx3B3V6Yb4OEn/VYuHGtkEXrPsP6ViKzJsChlaIqEdDtk4okPb5mmyEpStCOVOvTChWRNyIMFkWGOfPO+8nUkrm/7CUb4dO5di+4+E2p0BmfjaHPZv243a5WfjTcrYt1yo4XU4X3741lZGPfMLezaHJy/aFQzuOILxXx1KVHEg+VOI5p70/g5st99Gvw2t079uF2PIxNGxbl4eGBPdk5iuph06C1/na8xzs3XwgoHmiYq08OfpBzFEm4ivGkZmaxacvfo3L6QqmuYC3oKvcKJRKW1DK/3BK6TKYa+jrIkwdg7r3cDrSc1hrW+mYCzkfge2XkKwTCtTcKcjM55HZ7yJTu2lKr+c5F+wdwYzxf/H5y5NxOlz89tGffLd/PGZr6FLbAsFsNaOclt1jsmpfuq9f/4HfPpmNy+5k+e+r+XbPJ8TEl36D67ZdWhITH4UQYLKY6Ni9XYnms+XY+HLgd7hdHk4eSWPvloP8fGJikKwNjDrNa1K7WQ32btqPoih0fdJPOYbT6PbszVzT4wrur/UU25bvZPeGfVhjrTw05J4gWnyB4Dmm9feVAHakZ1/Y00J9xrFA258BrbpZPaEV+53HXLCOYMOCrflyBQ6bk9TDaVSrX7Z6y97c+3q2r9rF1qU7uPXpG6l3qda5a+/mAzi9KYm5mXncU+kJXpz4NNfff2Wp2hcTH81X2z/k0M6jVKlXqcQ5+zq9DkWnA5fWNc4aG1rVU19ten/RWxzcfoTEqgkBVSifTnZaTn6VssvuJGXfiSLHr/pzPbvW7eGKOy6jZgl0n8oS0rECXBvBfF3honCG5qBvoPUOFiaEN7wlpQeZ/a6WQ2/tiWINz51ikVhu1+wTAnRVQakYbotKzAUrQ7169nrevGs0ik6hcp2KjFs9Ep0+/NkzvrBxwVZe6zoiv6cuQLkKsUw9FhodF1+QUpJxIouY+Kgz0j79ZeWstXz20jckVUtk4Ld9S1QVXBaRUjLsvg9Y+usqLNEm3l88lJpNCu48tWz6aob3/ACXw43JamTSzo/P+/dDOpYj058EXNoJPnF2oT0EpFTBcxh0SfkpmDJvKjJrKJqEthlRfmrIq6gDQbo2a01mjJ0QihXpOQlKdEgL6gLhopehbntTSz5dP5qUfcdp1qnxeeMEAFpcfQlTDoynX4fXOLTrKEKIsJ4gPG4Pg7oMY8uSZKLiovh4xfAitfuTV+5i1MMfo+gUBk3uR72Wp3oUX3ZL6yJ7RZ/vCCEY/EN/MlOzsMZailRE3bp0+xmyIwd3HDlvHIF0H9TaSxpandle0rUOcKKJwyng3gGFOAIhFDi7PaOaRn6xllDKrLCcMDQDQzOklKgZA8D+JwgTJHyLMASeUBEuLtjNYtB65ba+oQVGc2g2vEJJbEIMw/98jctubkW7Li1587eXw2bLjtW7SV65S8uVT81i5qdzihw/9O4xHNp5lAPJh3m7x/ulZGXg/LtxHz1r9uH2cg8x73v/GrgURlxibLGy2FfedTnmKBPWGAtRcVbqt6pd5PiygnTvR568FZn5hva/++CpJ03XaidEEaX9b/Ctr3U+1ru1cAsKGNqCsegeDWFHPeaVm3CBzEHmfBpuiwIiLHcEQohywBdAU7SI6mNSyuXhsKUsU7l2Rd6eMSisNhzccZjJQ6fl71kYzEYq1ixaDsHt9uT/7HF5ihhZNvjwqQmcOKgpoY5+bDxX39ux0FqCOd8sYFy/r4hJiObtmYNKFNdv2LYen29+jwPJh7mkY8OAWoCGBecKb78CO2AB56r8K3thaAzlZ2k9DIyt8jWKpFTB8be2yWruUmg2klASIHEO4CxzYZYCETFaG0oJYAJd+BrQl4RwhYY+BGZLKe8S2ifi4usZeZ7w0nVvknY0A6EI4pJiubVPZ27ufX2Rxwz8pi/D7/8QRRG8PKnsFyMZTAaEIpCqRKcrXHTO7XLz/hOf4Xa5ycvK45NnvyxWfdXj8bB16Q5i4qOo3excNc9KtSr43CLT6XDx9zcLkVJyw8NXYzSVXu/bMzC2Rito856ojWcKGgp9NdCf6SBl9nBNEkMAtt8RCYVni2nv/3ngBEBL3Y2fqN0J6OsgYvoVOE6qOWCbqt0lWe4KWVpuoJS6IxBCxAJXAo8ASCmdaEHFCEUgpcRpd2KylN4XRFVV0lMyNREyjyS+YhwPv1l4V6z/aHV9c6alhGdjOzczl2+9dzA9X7uTxCoJxR7z4hdP8XaP98lJz+G5cb0LdQRCCBSd0DTGhMBgKv7r8/Y977Nm7kakqtJnzMN0fbKzvy8pnzduH8nGBVtRFMGKWesY+vsrAc9VEoS+HpT/CZyrwXgZQu9Dm1D7P4BNu3J2hlZJs7QRxtaIhM+LHCPTnwDXZkAB51pEuTGlY5yPhGOPoA5wAvhKCLFeCPGFEOeKcQghegsh1ggh1pw4UXQK3oVOyv4T9KzRh9tiHuSdBz8qsLtXKFAUhfsGdcdg0mM0G3hsWM9SWbckDO/5Ib9/Mps/JvzNwBvf9umYKnUrMW71SL7ZPZY2nQvXltfpdbz+04tUrJlEg9Z16Te+9zlj7HkONi7cSurhk3jcHpb+thJ7jh1HnpPp4/4K+HX9u3Efa+dsxO1047S72LRga8BzBQNhaIiIekDrx+sL5s4grODtJXDR4dqCdr1rB6d/GZClQThCQ3qgFdBXSrlSCPEhMBB4/fRBUsoJwATQ0kdL3cqzkFLy+9jZrJu7ic6PXM0V3S8rtbWnvTeDtKPpqKpkya+ruHfLgQLDDKHgkbd6cPuzXTCaDX43xwk2xw+mMuqRT8hOy6Hvx71oekXjc8bs33YIt1PLOjm6JyXoNrTv2pr2XQvOenLYHPRp+RLpxzJQVZXR89+kaoMqHNuTgk6vo9mV59rrK6v+WK+FVbzfhCYdGgQ8VzgQMQPB2N67RxB40d55i/kWcMwGpLYhXsYIhyM4BBySUv53fzgNzRGEnAPbD7Pst1U0aFOXVtc39+vYJb+u4stB32HPdbDun018vGIEtZvWCJGlZxJfqRx6ox6n3YWUkqgSFj2dTsr+E/zy4SwSKpXjjjEMtwoAACAASURBVOdvKTDTJRjidcFg9GPj2LxwG6oqea3rCH5Ln3ROGOf+wXfySV8tLHXXC11L1b5/N+wj7Vg6tmxNBG7ed4v4YPFQZk2YS0xCDDc/HnhTmaZXNMJkMeJ2utEZ9PSf0CdYZheLdB8Az14wtEEopz57Ukpk3nfgXI2w3o0wXVHoHEIIMF9TGuYWiJQSmT1MS/M0dkTEDdeE40oJETcCXHcAppB0NCsppe4IpJTHhBAHhRANpZQ7gOuAbaFeN+1YOn0vG4Qjz4HepOeNqQP86tp1bE9K/pWmolM4vv9EqTmCu164lRMHUtm5dg/3vnw7FfxsVFMYqqrSr+NrpB/LQG8ycOLQSZ79KDwKoL6Qm5mX3+ryP6d4tiPo0us62tx4KW6nm8p1Srfis3LdSvlX7CaricbtGxCXGEvPV+8s8dzNOjXmnb9eJ3nFLi67pRUVqificXs4tPMISdUTscZYyErLZv3fm6l5SXVqXRKc7BXpXINM66Xl9CsJkDjrlP6+fRbkvAvShnTMh8QZCF/aW4YD51KwTQOZp0l3mzqApeiugsFECKGJBfqImvc7OBcjzLcgSsGBhitrqC/wnTdjaA/waKgX3L/tEAitObknz8nmJcl+OYJrel7B1DEzyMvKo3Ltilx6bcl7iCav3MX2lbto26VlkfIXRpOhwHj02djzHBzdk0LVepV8qp1w2Jykp2SiqhKnzcmO1f/6ZX9p0/eTXrx2y3BsOQ76ju1VaIpnUrXypWyZRnyFOD5c+jbzpiyh3v/ZO8vAKK4uDD+zGiMESEIgeHCHosWtUKy4U6BIcSmlOJTiUKxocS+llI/iFJfg7h4ChIQI8azPfD8WtqRENslGoHn+QLIjN8nunHvPPed9KxZKkWx3XJSuUZzSNYoD5gqiodXH4ffIH4VSwdxjPzKh+QxiIjSIosj0feMoVyfljU2SZg+WTV4Rc1mo0ryalow+ZnMWMJdQml5a53OcHkgGS5A2f21Mt6EkhqQ7DRGTAA2S9m/IsR1BWTxV75kugUCSpOtAom3PtqRoJS/snexAMs+Ea7ZKmoCaa+7sbH62jJBXb3DP55pis5fb3vcZ03gqokli3cTfWHNnYYoeYG8CQulf4Qe00VqcXbOw4urcRHVz7B3tqNW2Khf2XUUUJTp83zLZ908Lilcpwp9B61Ldti8lFCyTn94zbfcwPLLlFNvn/EWRioUYtryvJcDfO/eQV08C0EbrEGR6di7aiyZKiybKnJY6s/NCkgKByWgi6GUIOXJni50eVFUBzS7MG51mf4J3CPatkGI2mYOBPC+o0vQjnTTUtc1+z7ojoKwC9hn4vW70wWJTKcjA5AufYiBIDxydHVh9ewG3z9wnf8k8yUobqNRKchW0Tbrh1ql7GHRm60alWsHjaz4pCgRndl4kOjwGvdZciXvp4HXqdaqR6HnjfxvBkxvPcM7ulKF8cxMiowYBW/PaN4gFfVeg1xrwe+SPZxEPS5opZwE3RJN5iqu2V1Hq82Kc330ZpVqJTC7wWSPr89CaKA2Dqowl0DeILDmysPzKbItFqsy+GZJgb5aKsPsylsmKoMgLbifM3bXyfGmWc79z9gGrx2zGNU8Ohi7tQ5ZsictwC4I8w5VsxotdY4haBmhBlgNUiX+OU8p/JhAAOLk4xlvxkdZUblKerdP/RJALyBVyile1sgwvHvIUy20xrBdFkTxFrVNaFQTBonqaScYiJiLG8jc1GkyEB0daXvMo4M70vWPZv+YopWsUo2mfhlRuXB7vXZcoVC5/klYDFw9cJ/hlCDqNHik4gpPbz/HVoCaW1wW7+kD9OM8VZA4gs6KPwEYYDUbGNplmXvkI8PiaD4vOTMM5R9rLtKcWgjwnuB17m2orkCbNZ/+pQADmTs9Tf5xHF6OjfpeaSdIhCnwexL3zjyhRrUiKZ8+FKxRk+bW5PL76lLJ1Sqa4MqdigzKMXD2AC/uuUKfD5xSpmHYfzkxShwKl81G3Uw3+XncCt7w5aD+yRazXy9UtFcsxzj2fG62HNk3yfTwKuCGKIoBFrTejYtAbLUJ9SGbzpAktZ/GL93Sb3UOvMxAaEIZrnuzp5vdtDrBpVyL8ycpQx8eigas4sukkkgQlqxdhzuGEJQLe4e/zmv7lRyFhLudecW1uun5gfG4/Z3qnBehidHy/bpBNNgYzyZiYTKZUfyAd23aG41vPULX5ZzTvl7Hr/Dvn+5bgl28sXzu6OLDrzQabXDv41RsGVRpNVHgM+Up4sujMtI9StPId1spQf9Lqo3Fx8cBVtNE6dDE6bp2+b/V5N47fwWQS0URqMJlErh+/nYqjTJw5PRbje/clAc+CmNL253QdSyapS1rMSut3qsnU3WMyfBAAqN6iEnLFP4+udiNaJHC0mfN7rzC75xJObj+b4HF/LtjLm9dh6DV6Xtz34/rx9O3gTiv+c6mhuh1rsHvpQRAEKjYoY/V5JaqZc/hKleLt1+nb2Wk0iJb/iyYxgSNtR9DLEM7tvkyB0nm4dfo+d889pNXgJlRuYn0ZbiaZpJT+83uSzcOF5/f8aNqnPhXqJ9wc+vDKE6Z1mo8uRs/pHedxcc8aK6X2Pie3n7WUmeo0etzz2aZnJ6PznwsEfWZ2pUL9MuhidEnaOM5fMi+LvKdx4/gdytUrZbOGneTy/dqBTGk7F73GkCYKn1Fh0fSv8D3aaB0mk4hMJmDQGblx/DYzD02gSMVCGc4TOpNPE5VaSfeJ1ss0vHzoj/C250SSJJ7f94s3ELxrWARwyGKf7p/ztOI/FwgEQUhQWCwhvMoVwKtcAdsOKJkUq+TFVt+0M8F4ft8Po8GEXmsALFXO6HUGRjf8CTtHNb+cn5nhfKE/FkRR5ObJu+aO5BRWkKU34cERCDIB5+wZo5KncpPyOGd3MgtnO6ip0Sp+s5vhy/sxreN8REni+7WD0m6QmL0/RjeaSmhgOF//2IHOY1qn2b3/c5vFyUUUReZ+s4zTO85RolpRpu0Zk6aS0OlNTKSGXsWGEhOlRTSaUKiU6DU6swmNZA6w7b9vQd/Z3dN7qB8lM7st4uzuyyBJdB7b2iayFOnBriX7WTlqEwBDl/alyTdxl52mNTqNDr9HAeTyyom9o12Cx1oqqOLpXE8tJreew7ndl5EkCYVKwXb/VVb1SCRE5maxjbl29BZndl5AF6Pn3rmH/L3+RHoPKU1xyGLPypvzGLGiH7+cm8Hq2/OQK+WWfKpCpSBP0dzpO8iPmJN/nEMbpUUbrePAmmPpPZxks2HSdgw6IwadkXUTfkvv4VhQ26spVDZ/okEAzAEgrYMAgENWB8smuEwmoFCmXelqZiCwEoVKAe9WT8Lbr/9jZHV1pn6XWniVK0CATxByxT9v1OJVC9O4V/qpS2ZUJEni0Prj/Nx7GTdPxa+tWKRiIZRqJWp7FWVrl/zgdZPRxIJvV9C90CA2/Ph7ij0pHl/3oUPuvjRz6MK+VYdTdK33cc/nikwuQ66Q4VHQOue1TMz0//lrytcvQ97iuRm/bUSaWpcmmBp66ybmJknSk399v6wkSTdTe3DvyAipIUmSWD12C0c3n6Jc3VKMWjcIhfK/FwzeoYnS8E3JEcSEx2Ayicw5PJGS1YtZfX5GyyOnFqd2nGNur6Voo3WoHdSsujUvTpmS6IgYDq45hp2jmsa96lneWyaTiYjgSC7uv8biIWvQxeiwc1Qzfd+4OAOGtYyoNZHb3ubyablSzr7oLbECe3IJehnC6jFbkCtk9JnVlUdXnrJjwV5KVC1CjykdbXKPTKzH2tRQvE8yQRA6AAuBQEEQlEBPSZIuvX15PWZzmU+S4797s2jAShyy2DN19xi8yhVAEAT6zupG31nd0nt4GQJ7J3tW3ZzHzVN3yV8yD56Frd8kjpVHXtaXJr0yRh45NfC9+xKdxtwJK5ML+D8NjDMQODo70HZEbP+EqLBoBlf9RwPoXe4aQUAbrUvRuOyz2CGTCYiihFKpsEhZpBS3PDkYu3koYO7En9phPjqNOZ2azcOF1kM+7HwOfBHMvN7LiAqLYciS3hSv8nFvlsfFkS2nuHLoBvU610yS6nFakVBqaBzwmSRJ5THLRG8SBKHN29c+WdUvURSZ23Mp0WExBL0IYVH/lek9pA+QJInLf9/g8KaTaKK16TYOJxdHPm9ZOUlBADJuHjk1qNe5Jo7ODtg72eGe15VSnxfl6U1fZvdYzOapf2A0xC+HfPrP8wT7hWDQG4mJiCFnfjcUSjlVmpTnsy+SZqz0b0as7E+pGsXJW9yTKX+NTpWceGhghCXA6HUGXj+L23J2Ts8lXD92m4eXnzCu6QybjyO9uXTwGou+XcmRzaf4qf3P+Nx+nt5D+oCEchtySZL8ASRJuigIQj1gryAIeYit7P1JIQgCcoUMg878f6X6Q8eu9OaPebvZNOUPAHYu3Meyy7MzrCLnkxvP2PTTH7h5ZuebmV2xd7TDPZ8rMZEaBIF0zSOHvg7DzlGdqrnYPEVysdlnKQHPgshXwhPRJPJdnUlEh8egtleh1xn4ZlrcXtCueXLE+rt+t7I/ZWol3+7yfdzy5GD+yZ9scq13RIdHYzKJlnRf4QoFKFOrJNeO3cLJxSGWkN37RIZEWer3NVHaDC0znhyObfVGG2NewQmCgN8j/zQztbKWhAJBpCAIXu/2ByRJ8hcEoS6wC/hkhW0EQWDyn6P4ZeBKnFwc+W71h5aA9y48YlqnBYgmkTGbhqS5zs/pHectqYGnN33RRmvTdGPJWkxGE9/X+5GosGiUaiV6rYERK/szbe/YWHnk9GDpsLXs+/UwMoWc6fvGpurf0DGrI17lzN4QwX4h6HXmXgydRs/TG77xnle5cXl6z+yK9/8uUrfT5zYLAqnB8d+9mdtzKZIk0WdWV77s3QBBJjB931jCgyNwcnGMd09tyJLejG8+E73WwJAlvT+pIBAdHs3J7d6WrwVBSLJNbloQ72axIAjlgBhAKUnS3fe+rwQ6SZK0KW2GmDE2i9+nR5EhvHoSAED2XC787rcqTe+//efdbJqyHYA8RXNn2BVBTKSGNjl6YTKa28/K1C7B/BO2nYUmB02UhtbZ02dckiTxY9u5XDl0A0EmMGP/+Az9gLeWbgUH8trXnPpRqOSAgCAIjN44xCqnNkmSkCQpXco2k0rwqzfYO9nh6OyQ6LERIZF08uyHQW9EEARqtqnKpD9GpsEozaR4s1iSpBtvL3RbEIRNwBzA7u2/lYA0CwQZjffzumml8/M+7Ue2oFDZ/IS+DqNmm6oZMgiAufegad8GHFx7HJlcRo8fO6b3kABQ2amwc1ITHRaDUq1M8h5Hcnh45Qkzuy5CNEn8sGEQfWd1w9k1S4JVUw8uPyHAJ5DKTcrjkCXjrfjeJ2cBN4L9QhBFCZNRRHqb6lk5aqNVgUAQhAz7Pn6fRQNWcmi9+f08bc9YytdL2LLWOUcWek3vzIbJ23H1zE6/uRmz4TLRzmJBEByB2cBnQBZgCzBbkqQ0ewJmtBVBl/z9CXoRAkCeorlYd/+XVLuXJEnc8b6PySRStnbJj+LD8m9C/EOxd7Kz+cPMaDAytcN8Lh+6ToUGZZj85/exbRYT4MmNZ2yY/DvueV3pPbNLqqfWehYdgt9j8yrSNU92fnv+a4LHn9pxjjk9lyKTCeTInZ1Vt+Zl6HLl0MBwVo7aaPa+vvyEwOfBCIJAiapFWHhmWnoPzyZER8TQ1u0bTIaMtcJNiBSvCN7DAGgAe8wrAp+0DAIZkXe+sMA/JhmpxOqxW8xqqUCDrrUYvuLbVL1fapAjV7ZUua73rktcPXILvdbAjRN3OL3jPPW71LLqXK9yBfhp1+hY39PrDDy8/ISc+d1SZBsaFhTOid/PkqugO1WbmYUNTaakqcUe3Xoa3dsNxqCXwQQ+Dya3l0eyx5TaZHPPyugNQwB49SSAFd9tQKlWMGBhr3Qeme2wc1Bj72RHVKh5zytvMc/0HpLNsCYhdwlzIKgM1AQ6C4KwI1VHlcEZuLAXCpUCpVrJwEWp+0b/e/0JtNE6tNE6jm09k6r3+m3W/+hdegQrf9j4T816GvDszgsmt57Dgm9XEB0ebfV5SrWC9wvYZAoZj6/5EBVm/TXeYTQYGVp9HOO+nM43JYZx+8w9y2vv9hKsvc7ASqNZ9cMmpnVawO5l5iA+euMQcuTORracWRmzaWii16nW7DPsHNWo7JQ4uTjh+q/AdHjTSaZ1WsDZvy7FcwXb8eKBH+OazWBKu58JfvUm0eNze3nw01+jmbh9JK65s6f6+NIKuULOvONTqNW2Ki0GfEH/+T3Se0g2w5rUUCVJki7/63vd/8ubxWCePQoCsVIROo0On1vP8SySK8ViUe+Y0u5nLh24hiRJlK5Zgtl/T7TJdf/NjZN3mNB8Jtpoc+fqyNUDqNsx9U2zJUmirWsvIkPND+9SNYqz8PRUq84VRZHlI9bj/b+LVP6yArdO3yP4ZQgyuYzF52fEO2M79ttp/pi3h+KVCzNwUS+UKiU+t3wZWmMC2rervS961KXz2NYMrDQaTZSWXF45+fX6z7G0ai7su8KLB6+o2/FzXD3ND+rA50H0Kj7MotJapWkFpu8dl6zfzaWD13j15DW121ePZWV6+e8b/NhmLroYHWoHFQtPT6NwhdTzne5eaBCvfYMQZAIlqxVlgZV/n0zSH5ulhv4dBN5+76PdKL577gE7F+2nUNn8dBz9VbLdn1T/6i+IidTwbfnvCQ+OQCaTsezybKuX8u+CcVz5/7FbhnFwzTFEk0iT3qnXgRsdFmO5vyRKRIXFpNq93sdkNFmCAMCds/cxGU1WSRHIZDIGLfqGQYu+4erRW5zY5o0mSosgCBzbeoYeUz7cnPb3ec28PivQa/Q8v/sSj0LudBzVCre8rshlMhDMAmUlqhVhRpdFljSg/5PXHNl0ihb9vwDg0IbjLBm8BpPBxO9zdrHp6TLsHNTk8MxOLi8PAn2DEEWRBl2tS1XFRXyGPy8fvkJ6TyHT73EAhSsURJIklo9Yz4G1x/Aql5/pe8fimNUx2fd/R1hQuLmqxyRZtSLI5OMj49dq2ZCIkEhGfzGVk9vPsnXGn+xcsM9m17558i4RwZFoIrVoo7Wc+uOcdeedukvr7D1p5tCVw5tOfvC6Sq2k5cDGtBryZaoav1RpWoGSnxdDEATyl8pLg641bXLdc3su08a1F53zfsuDS48/eF2hVJAz/z8uUDK5DEmS0Gv1/Nx7Gf3Kj+To1tOJ3ie3V05LHl5lr4rXUCQqNBrZ225Xo8FEWGAEYO6SXnRuOh2+b8mw5X1p1q8RMnnsj4dTtn/KBa8cuoE2WodBb0QbreP1s0DAbCu5+PwMvl87kHnHp1C/c/IDQXzUblcNJxdH7BzVZMvpQqXGZn+Nh1eecmDNUbRRWh5cesKuJQdtcr9+c7qjUClQ2SkZML+nTa6ZScYi45YhpAJvAsIw6M2ln7oYPfcvPrLZtfMUy43JZM4lK1VKCpbNb9V5vwxcRXS4efa9aMBKGnWvY7MxJQWFUsGsgxMQRdGmtdyzv15MdHgMkcDPvZex6ub8D46ZeXAC45vPJCo0isFL+qBQKtgyZwfHfzuDXmtgfp/llPq8GB4F4u9C9ijgzswD4/l7w3FK1yxB7XhKFgtXKEj1FpU4sf0srp7ZaTOsmeW1/CXyxPJTGLN5CMNqTCDyTRRVvqxA3Q7/pMoadKvN2d2XkckFcuTKRu7C/6z+7B3tqN0u8ZLJ+DAajOhidPHO5rN7ZGPjkyW89g0mVyF3S3pSoZTHWl3aSiG3xYDGNOpRl5BXoTi5JF47n8nHx38qENg5qhCN/2yCGpOwCZgYeYrkYsb+8Zz43ZtydUtTtal1mnxOLo4IgoAkSdhZoZWe2iQ3CPg99md8s5mEBYbTf14PiyGJ7D2TcYVSwV9LD7Jm7Bayebgw88B4cnt5kLeYJxsfLYl1vYiQKIxvy/QEmYyYCE2iYyhTq0Ss5ixRFHl60xcXN2dLDl8QBMZtHc736wahVCniTMdFhUWzdvxvRIfHsOT8THIV+lAkrmrTiiw+N51XT15TsWEZq8tWE+PpTV9G1p2MJkrLFz3r8t3KDzvbwZy+ylc89h6IV7kCdB7Tmt3L/6ZYZa94JR2Sw6Ypf/C/X/YjCDBm01Bqta1ms2tnJEL8Q7l86Dpe5QtQuHzq7btkND5ph7KYSA2+d1+Sr4Qnjs4OhL4Oo0uBARh1RuRKOS0HNGZgEsrbTEYTq8du4d75R7QZ1jRFs753BDwLZG7PpWiiNAxb3o9ilQun+JrpwbhmM7h88LrFXWlX6HrU9mpunrrLz73N+fPvVvVnRO1JGN92WVZvWYkp//shzusFvQxhWM3xBD0PIaubM+O2DqdigzJWj0eSJCa3nsO1o7cQRYnJO763WvVxcus5XDxwDZPRhHteVzb7LLP6villaod5nNpxHjDP8Jv2aUizbxtRyMoVZmogiiJfqjtbyl7zl8rL6lsfruw+diLeRNKr2FD0WgOSJDFj//gUSX1nBP7zDmWhr8PoUWQIYxpPpWfRIbwJCCVbThdGrupPLq+cVGpcnh5TOiTpmn8tPcieZYe4432fOT2W4O/zOsXj9CjgzrwTU1h2ec5HGwTCgyMwaA28K+V8v0u0bO2SbHy0hJU35pG/VF7L92UyAaVd/LNotzw5qPxFeZQqBeFBEUz6alaSlFYjQiK5dPA62mgdeo2eP+bttvrcFw9eYdQbkUSJYL+QFJvAgDkwLRq4iuZOXRleawLREXFvxrvnc7P8XowGE7uXH2JYzQm8CQhN8RiSi0wmI0eubAgyc7rp3yuRT4Un159hNJjQRuvQxei5dOh6eg8pzfhkA8GF/dfQRGmJidCgidJyYd9VABp2q8PGR0uYtntMkisqQl69sewxCHIZ4UERNh93emAymvC9+yLeh1NC3D5zj24FB3L3/EPss9jj6pmdsZuHorJTfXCsvaMdozcOwT2/KyWqFU104/H9PR2TUUSvsb55zzGrA47O9giCgNpeRbFKXlaf2/OnjqjslCjVSjqObmWTbu575x9yZNNJdDF6Hl5+wp7lh+I8rsdPHWncoy5uef/pG5AJAn6PAlI8hpQw78QUGnStRfP+XzByzYB0HUtq4VWuADK5DLWDCrW9ikqNyqX3kNKMT3aPoECpPBjeqTzG6HG0wSbXV4OacGTzacKDIihfrxRFk/BwyagY9AaGfT6eFw9eIVfI+eXcjCTN+P5csPc9kxQVC05PTXBTt0776lZpzwD0mtaZO94PzK39w5uR1dXZ6nEplAoWnZ3O/xbtwz2/O22GfWiIEh+121WnfP3S6LUGmzVEKdXKf5xOBQGV+sNACebu1WHL+zG1wzyLjImdo5oinxWyyTiSS65COS2dw58qzjmysPLGPC4duIZXhYJJmjwkxK4lB9iz/BC5vTyo1LgcNdtUS7Vu++SSboFAEAQ5cBnwkySpeWLHJxV7JzvkCrklr3n18C1qt01ZTt89nxtbny9HG6W1SX12XAS+CObpDV9KVi+Kc47Ut3F8cPExLx/5mx/mAhxadyxW5UxieJUvwKVDN9DF6FCqFGR1s/5hnRiFyuZnR9AajAbTB30b1uBZOBeDF/dJ1r1tbaFZpGIhOo1pxd4VhylZvSjN+zeK91i9zsCZnRcsX2fPlc2q0mG/x/7oNXoKlM5nWcWIoogkScnul/mv4ZYnB037NrTZ9Z7ceMbqMZvRxeh5fs+PS4eus3nqDjY+XpKhpOPTc0UwDLgH2O7J8R5ZsjtZ3JFUdspYS+2UIJfLUy0I+N57yeCqY835c7WS1bfn4+KWNcFzgv1COLn9HHmK5ba6Uul93PO7IZrMU1W1vZoCpZJmmNF5bBtUdipePPCj9dBmsTpvbYFMJkOl/jQymN0mtKPbhHaJHqdUKXB2cyYsMBylSmnVRvGeFYdYMXIjggBNvqnP4F96c2HfFX7qMB9JFBm1fjD10qBTPJPYxERoEN6rxDMZTOg0Bl4+9KdIxUKc3X2JNWO3kMvLgx/WD0o3D+90qRp663K2AZgOfJfYiiC5VUMX9l9l26z/UbhCQfrN7W6zEr/UYvvcv1g74TdMBhMOWewZtX4QNVtXjfd4bYyO7oUGEh0Wg1wpZ/CS3jTuUS/J97156i77Vx2h5OfFaNH/i49S4fRT4+Ujf7ZM20H2nC50m9w+3gDr99ifC3uvsv3nvwh5Zd5QlivlHNRto2uBAQQ+DwbME6OdwevSbPz/JSRJwmgwxvl8EUWR6Z0XcmrHOQQE1A4qnFwcWffgF0xGEx08+qDXGlAo5TTsXoeRq227/2JL9dHUYCHwA2ZZ61SjatOKyZol24LXvkEEvQyheJXCVssHl6hW1NwUJEqIoohXuQKW18KCwnl05SmFKxay6M689g2ydLca9EYuH7yerEBQtnbJVCmTkySJK4dvoonSUr3FZ6kioxwREsnelYdxyupI074NMrRUc1LIUyRXvDl5URQ5uvk0z+6+YM/yQxj1RkxGEblSDhLkLZobMD/8zfsMEk7ZUmcVay3vPDyS8/fRaXQo1coMaVrz8pE/39WeRHhQOC0GNmbwL71jvS6TyZj4+3eIosiT6894fs+Pyl+Wx85BTVhQuMWi02QUkyWWaCvS/FMjCEJzIFCSpCtvrS/jO64f0A8gX76M5e+ZGFcO32By6znIZDIKls3PglM/WfUmLlOrBNP3jeOO9wOqNqtoaWQK9guhb9mRiCZz1++v1+fins+N3F45yZE7O2/8Q9/q2tRO7R8tSawdv5Vdiw+AIFC+bimm7h5j83uMrPcjLx/4IVfIeXLDhxG/xt2A9Snx++xdbJm+E4NWjyhJ5qpdAcrWKknxSoVp931LACb9MZIF/X7FaDAyfEW/dBuv2cbS3DA4dvMwq5vRJEliXp/lHN54EmfXLCw8PTVNTISSyMxKZwAAIABJREFUwuaf/iAs0KzFdGD1UdqPbEnO/G4fHCeTyShSsRBFKv6z6e/ilpWu49uyeeoOsnu40HtG3N7VaUF6TJ9qAC0FQWiK2d/AWRCEzZIkdXv/IEmSVgIrwZwaSvthxs2BtUe5uP8a9TvXjPWGPrHdm6VD15HVNQs5cmez+BQ8ueZD4PPgBCtp3qdcHXM1ks9NXyLeROKcPQtXDt/E+FbTRu2g4vKhGzTt2xClSsmyy7O5euQmub080rXpKC5O/H7WUlF06aDta7JFUcT37gskUcJoMHHr9H2b3+Md5/Zc5reZOylcoSD95/dM1ua1rbhx4o7FqwDMVUWSBP1md4/Vi5Lby4O5Rycn6dqSJBEZGoWjs0OCwn86jQ6VncqqNOLSIWsw6MwrgiVD11odCPwe+XNimzeiSSQ8MIJts3cxclXGKl11cc+KQiW3/Hz2TknbI+s2sR1dJ7RN93RsmgcCSZLGAmMB3q4Ivv93EMioXDxwjWVD16GN0XHp4DU8CrpTpGIhjAYjc3oswaAzEh4UgcFgRO2gRq/Vo7JXkS1nwhu+7xMVFk2/ciOJDo8xq5hemU2RioWQpLfqpBKxSgkdstgnuI+QnlRtVpFD644jSVC8iu2b5WQyGbXbVefC/qtIokjLgY1tfg8wyw5M67QAvUbPk+vPyJ4rm1WbvqlF8/5fcNv7PjKZjHwlPWn/XUu8yhcgz9uUkMlowv/pa9zy5kBtb71QoclkYkLzWVw/dhvnHE4sPj8D93yxZ7cGvYGxX07n1sm75CmWm4VnpiUque7smoWIkCgQIKur9dlgx6wOFrcJpVrxQcnljRN32L/6CKVrFqf5t+mzt9Xjp46EB0fw/J4f3Se1T1alX3oHAfiE+wgSI+JNJFPa/syLB6/oMq41rQYnXmf+6nGAxbBFJpPh//R1rKXeO3Lmd6Pz6Na8fORP0z4NkvRhvHHiDlFh0WgitShUCs7svEj7kS2Yc3giVw7fpEKDMnHeMyMycGEvytQsgSZKS73OiVes6HUGLh+6joubMyWrF7PqHuO2DuOO9wMcnO1j7amkBKPByIqRG7h3/iFtRzTHq1wB3n1WDXojIX7pK8Vcs3VVll+ZQ8irUErXLB4r766N0TG46lgCfAKxd1Kz4PQ0fO+8IGcBt0S1cx5cesJt7/sYDUbCgiLY++thvpkeO11x9fBNHl56gihKBPgEcmTTKVoPTfiz89NfY1g8eDWCIDBsWV+rf85sOV2YsG0EW6f/SaFyBeg8trXltYBngYxvPhNdjA7vXZdwdHaw2p3Oltg72qVKf0V0eDQzuizC995Luk1sR5NeqSdBD+kcCCRJOgGcSI97b5ryB3fOPsBkMLHqh818/lUV3PO6JnhOnQ7V+W3W/4gOjyF7rmxUalweMG+A/bBhMEuHrsPFzZlhy/omO5eZr4SnpZxTrpDjVc6c7ilZvZjVD8eMgkwmo06Hz60+fkzjqTy+6oMkSfSe2ZVWg7+06h7vC83FR1RYNC8evKJAqTyJ1m/vWnKAA2uOodfomddnBStv/EzN1lU5vs2brK7OdBj1ldU/U2qRt5hnnMY714/dJvB5ELoYHQa9ge9qT0QbrUMURX5YPzhBfaxs7lmR3vbdKNWKOO06nV2dLRucMrnMqr6RPEVyMftQ0g2VosKi2Tx1B0+uPyNPsdyxutVfPwtCJjdHZ71Gj++9l0m+fkZm45Q/uHr0Jka9icWDVlOpcflUdXv7z64IRJMI75XOSmLi2xDZcrqw6ckSAl+E4FHALdZMrG6HGrFkipNL3mKezDwwnlM7zlGhfhkqNiyb4mumNsd+O82RTaeo2qwiLQc2SdZSVxuj4473A0sD4OENJ6wKBNYQ+CKY/hVGYTKasHeyY+XNeQnWa4e+Dsf4VtpCJhOICo1mzKahDFvRD7W9KknVK3qdIV6V09TAo6C75XeoUCqIDI22/CyHN55MMBDkKpST0ZuGsuuX/ZT8vChN+33YWFWiahH6zu7KoXUnqNS4HHU7Wh/ok8reX//m6U1fTEYTZ/53kWZ971O6pjnol6helFyFcvLqcQAKpYJGX9dNtXGkB7oYnWVCCFj+hqnFJx8ILuy7wuLBa3B2zcLE7d+Rq6C5EqfbxHbcv/gIv0cBdPzhqzh3+uNCZaciT5HUrVz4t5xyRubBpcfM77sCXYyeW6fvkTO/O9Waf5bk66jtVeQtlhv/p6+RyWVUbmqdUqg1nN11CV20Dr3OgMlo4vKh6wkaxrQe8iUntnkT/DKEyl9WsOzJJKVZTpIkZnX/hRPbvHHNk4MFp6cmuuJMDJ1Gx7PbL/Askgsnl7jLQQuUysuEbd/x94YTlKtXit9m7CQiJBK5Qk7lJuUTvUetNlWp1SbhPaevBn3JV4NsE6QTws7RzmIOJIki6ve6q1VqJcsuzeblw1e45XXFIUvG6dK1Bd0mtuPW6XsE+ATS/vuWVhebJJdPWobaZDLRMkt39FoDgiBQoWGZZC1RMwqaaC13vB/gWdgjTo389ODUjnPM/WYZ2igtKjslfed0T/ZMPiosmiObT5HNPSu121e32Sz6xsk7jG0y3aI9VaxyYRafn5Hg9d+5pCVlf+d9Hl/3YUQtc1pGJpfRemhT+s9Lvtl5dEQM/SuMIiI4EpncbIVqzXvgTUAoJ7adxaOQO9VbVMoQG5PWYtAbmN93BbfP3Kd5/y/omAFSch8bGb2hLG2QsOQzQUrR8urcnsvsW3mY8vVL03Z48zT/QOl1BgZU/IHQgDBMJpFZB8dblslpTVRYNNM7LeDZnRe0HdkcjwJuvHocgIt71hSlCpxcHG2WDnqfcnVKUahsPh5cegKAzy1fgl4Ef1AR8z5m1dLkW4M6uTj+k6JRKZJUORYXZivUCGIitcgVMk7+cY5Oo1slel52j2y0Gd4s0eMyIkqVMt2E7kRR5NjWM4S+Dqdxz7ppovuVnnzSgUCukDNydX+WDF1LlmxODF2aPAGy5/f9mN5pATqNnhsn7pDdIxv1Oyfs6Xt400l2zN9DscqFGby4d4rrzp/dfs6bgFA0kWZN/iObT1sdCExGE1cO38QxqwOlPk/5hvOW6X9y7dgtTEaRtWO3subuQuQKOdk9XDJsZ2/1lpV5ducFeo0Btb3KpuJ4ceFRwJ3v1w1i+9y/KF65cIofxnmL5cb0NmessFJ/KJPks+HH7excsBejwcTeX/9m/YNfPqrVVFLJmJ9aG9KwWx0adkuZD3DQi2CL5aJBZ8D/acKGNK+eBLCo/0p0Gj0vH7zCs4gHHUclPntLiFyFciKTyd4auqiStIcwpe3PXD9+G0mS6DaxHR1/SNlYgp6HYHpr+WnQGQkNCEuTiibvXRfZvewgZWqVoMv4tknatO00phVO2Rzxe+RPywGNUzTbt5Z6HWvYTOgtT9HcTN83lhPbvClfr3S8bmuBL4KZ0Xkhoa/DGLjom3STWEktXj7yZ3zT6YQFRtDv5+406xu/imtKuHr4pqUZMsAnEG20NkOphdqaTz4Q2IIytUuSt5gnz+68wCGLHY2+TjiwRIfHWJRPjQYTEUGRKR5DlmxOLLkwk+PbvClYJp/VTWSSJHF+3xVLVdSh9SdSHAg8i/xj1K5UKwhLA4Oel4/8mdl1ETqNnjtnH+LqmcPii2wNcrmcrwbazsM3PShXpxTl6pRK8JhfBq7i3vmHiKLET+1+5q/wjRl2lZYclg1fh//TQCRJYsmQtTToWtsqie6k8mXv+vjceo5MJlC0ktcnHQQgMxBYhUqtZPH5GQS/DMElp0uiaZ7CFQpSo1UVjm09g3teV1rbKEebp2huuk9qn6RzBEGgSMVCPLv9HEEm47NGKS9Hrd6yEn8u2AuCOf2WFgY9b/xDLRUkBp2BAN/AVL/nx4hOo7dYa4om8b09sk8DlVqBIBOQTBKCICB7O+GKDo/m0PoTOGZ1oGH32in2X2japyFFKhYiLDCcCknwyv5Y+aSrhtIbo8GYIWZjMZEa/t5g/pDU71LTJiYlT24848HFx1RsVDbVS9vA/Lv8odFU7l94hJOLI0sufCh/kAn43n3BuGYziAyJYsCCnnzZu0F6D8mmBPuFMLXDfEJehdJ/fg/Lyrh/xVE8v+eHTC7jy971GbTomxTd58mNZ1w6cI3SNYunW1GGLbC2aigzEGTy0SBJEqGvw3DOkSVDBNhMMgaiKNJE2cmyEspXMg9rbi9I9vX8n76mX7mRGLQGFGoFs/+eZJMii/TA2kCQ8QS+M4kTvc6Av89rTEZTeg8l3RAEgewe2TKDQCaxkMlkVG1WEXsnO+wc1TTplbAnR0RIJHfOPkATrY3z9cfXfBBkAiaTiMkocu/8w9QYdoYi8xP1EfAmIJQBn40mOiwaj0I5WXJhZqpskGWSdvjc8mXZiPU4OtszZGnfDGdm/rHx485RXD16CycXR0pULRLvca+eBDCw0mgkUcLRxYFVN+d9YD1bulYJlColQhYBSZKo8olVXsVFZiD4CDix7SyRbyIx6IwE+gZx9fBNPv+qcnoPK5MUMKbJNN74hyGXy4iO0DD3SNJ8A+IiKiyaAJ9A8pfKk+FtWW2NXCGncuPEJTRO/3kBbYwOk8G8sr5x8i6ft4z9WcrmnpU1dxdw99xDClcomGJpkI+BzECQAXj58BVrx/+GY1YH+s7p9oEgmkchd+QKs/mFKIrkLJC5SfqxE/kmCgCTSeTNW6/hlPDigR9Dqo1DNIm453Nl6aVZadIrkVpEhUWzdtxWosKi6Tm1E7m9PBI/yQoKlcuPQqnAZDBhMonkK5EnzuNc3LJ+ECDSirCgcF4/C6Jg2fxpZoCUGQgyAKMaTiHELxS5QkZoYDjT/mXpWL1FJfrN7c6lA9dp9HUdm+nu2wqT0cTelYcJ8XtD8/5f/CdmUGDeVDy49hi5C3vwRY+6Seo87Te3O7+O3IhMIaf/gp4pHsvRLaeJidAgSRKBL0K4e+4hFepbX/b4JiCUmEgtnoU90qSD1mQ0cfPUXbK6OsfZJT2n5xIu7r+GyWTiztkHbHm23Cb3rdy4PGM3D+X6iTvUaVct1QUkk8qTG88YUcush+ZR0J0lF2elSTDIDATpjCRJhAaYPU+NBhP+TwI+OEYQBFr0b0yL/tY7cEmSxNObvihUCvLHM+uxFesmbmPX4v0YdEYOrT/B1ufLkUSJo1tOI0nQoGvNjz5V8do3iAXf/opOo2fokt7kLuzB4KpjiQyNQmWnIiIkkvYjW1p9vVaDm/Jl7wbIFXKbbH4XLJMflb3qrXyxmKQZ9OmdF5jVbREIAo2612b4im9TPJ53XNh/lT3LD1GmVgk6jPrKEmQmfjWb26fvIYoSgxb1+qDM9ebJu5bCiKCXIUiSZLMAVaNVFWq0qmKTa9mavzecQBNl3sQOeBbI46tP06RrP7NqKBEuHrjG6jGbuXvuQapcXxAEuk5oi1KlQGWnpPeMrja57oqRGxhWYwKDKo1mx/w9NrlmfNzxvo8uRm/2lg2OQBOpZdbXi1k8eA1LhqxmWsfkl/JlFKZ1WsDVIze5ffoe45vP5E1AGLoYHZIooYvRcds76X7Janu1zSqgarerxpClvWnSuz6zDo63WlYdYOuMP9FrDeg1evavPorRYBvt+0fXfJjYchYX9l1l3cRtHN54EgC9Vs/lQ9fRRGnRxejYs/zvD859v6LHPovdJ63z8z65C3ugfLsCkEQJj4Kp36MDmYEgQa4fv81P7efx+5y/+KHRVF4+fJUq9+k+qT1bX6xge8Bqm2wCv3jgx+5lh9DF6NBp9OxcuM8Go4yfNsOaobZXYeeo5rOGZXHM6sC1I7fM948xC/WlF9ERMawavZmlQ9eaRfuitYxqOIUWzt1Z2P9XEuqjiXgTyZR2PzOk+jiCXoRYZDqiwqLJmd+NgmXyYZ/FDrWDiubffpFWP1KcCIJA4x71GLlqQJIboAqVyY/STokgE8ju4ZKgaX1S+HP+HsvvzGQwce/CIwCUaiW5CuVEoZSjdlBTrt6HshnFKnmhUMpR2imp1izp/ha2QJIkwoLC06xk+9HVp6weswUA5xxOzPp7Itk90qaaLDM1lACPr/lY3gQyucCzOy8sBuG2xsUtZTLF73h09Snf1Z5kGbdSpYhldp8a1GpbDa/yBQgLDKdYlcIIgkCdDtUtM8Caba3TRUoNZnRZyNUjt5BEkRsn7/Bl7wbcPfsAvdbA0S2nqde5Zrz6PUuGrOXcnsuYDCaUaiUqexWiSWTQL98gk8mYf+on7p1/hFueHBnGHyI5DFnahxye2QgLjKDz2NY2m3275s0BArxzoK/51vBGEAQWeU/j4JpjZHVz5ouedT84d+bBCexfeQSVvYqmfdO+O9pkNDGm8TRue9/HOUcWllyYGad1py2ICInEwdmeQ+uOoX2bFlIo5cjlaTdPzwwECfD5V5XZ/NMOVHZK7BztKFunZHoPKVEuH7qBXmdAEiUUKjm12ldn+Ip+qX7f3F4esfLSQ5b0oXrLykiiRKXG5VL9/vHhc+uFxYfCbGsoj/WgS2j2G/o6zFJmKJpEdgSswd7RznKOUqWkbO2M/55IDDsHNd9M65L4gUmk8+hW3Dv3kKc3fGk15Es+e8921cUtK53GtI73XEdnB9p/b/2ei625c/YBDy49xqg3EhYYzr5Vh+k5pZNN7yFJElM7zufcX5exz2JHu+9aoHZQm1OOQM40kG55R2YgSIDcXh6sf/QLvndfUrhCQRydHdJ7SIlSpnYJlGqFORAoFfSd1TVJFou2QhAEq+q6U5su49uwfPg6ANqMaE7jb+pz7fhtbp++T+NedROUDug7uxujG/1ETJSWrhPa4pQ1bnvI5HBgzVEOrDlKxUZl+XpyhyRJan8sOGZ1ZN7xKal+n6iwaGZ0WcTz+y/pPqk9jXsm3FkcF6IocnH/NQQBKn9ZgeweLoiiWWpdqVbgntf2JdvP7/txcf9VjAYjkaFR+Pu8Zsji3jy6+oTGveqTzd02WQJryNQayiCYjCZunLiDs2sWCpcvmKJr3b/4iPsXHlOpSfkMVx6XHgQ+D8KgN+JZOOm/C1EUMRlNNq16enT1KSNqT0IXo8POUc2IX7+lfpf4PZQzSZhlw9exZ8XfGPVGlGolm32WJjm3PrfXUk7tOAcI1Otcg+9W9uf0n+f5a+lBytQuSfdJ7WwerEMDw+lecCA6jR6VvYou49rQdXxbm94j06ryI2NCi5nc8X6AKIoMXNiLpn0aJvtaxasUoXiV+Nvsk8vVo7cI8AmkZpsqHzS9ZWTiUik9/rs3a8duwaOgO+N+GxHv7EsmkyFT2fgB8DrcIp9sMpp4ExBm0+v/19C+LZkFEASSZUnr/ddFixGN9/8u8N3K/tRqW41abavZdKzvk809K1N2/cC22bvwKl+ADqPSLxX26a1HP0J0Gh1XD998W06nZ9+vh9N7SB9wYO1RJreazbLhaxlUeYzNSgzTg+jwaOb2XErAsyBunb7Hqh82pen9KzQoTeGKhZArZLjlyZGo0VEmCfP15PZ4FsmFyk5Jhx++SpY8ecUGZbFzVGPnqKZiw7Tb0/qsUTnmHplM/597pGuvTeaKIBloojT8PucvYiI0dBzdKsWCYSo7FbkK5eT182DkCjnl6pW20Uhth/euS5YZU2hAGCGvQpNUq56RMBnFf8xb3vYB2ILo8Ghmdl/M87sv6DqhXby5aqVKybzjPxITqcEhi/1/pkb+HSH+oczvs5zI0GgGLuyZotWrJEn8Nut/BD4PonDFQrQb0TzW66Io8uDSE5xcHMhbzDPe64zbOozj27wRBIF6nWxjL/oxkblHkAx+aj+P83svmzs4C+di7d2FKb5meHAEB9ceJ6ubM42+TrnDkq05svkkC/uvQhAgR65sFsP6j5U/5u1mw+Tt5MidjVkHJ9ik/HPFyPX8tfQQRr0RlZ2SjU+WZqqKxsGYJtO4dvQWoknEKZsjO4PXJTsY3r/4iFENpqCN1qFUKeg+uT2dx7axvD698wLO7zVbtQ5e0psmvay3N01tHl/34eDaYxSpUIgveiZNosRaMvcIUpEn159h0P1TkmgLsro60/GHr2xyLWsJfBHMq8cBlKhWJFGBsobd6uBRMCcBPoFUb/HZRx0EANqPbJkkSQhr0ET9k6sGs6VmRkKn0XHn7ENye+VME1e5+AgPjrD8njSR2hTJRyhUCsvqTpAJKN7T5TEajJzcfs7y+v9+2Z9hAkF4cATf1Z6EJkqLnYMaURTT1U0uc48gGXQa0wqVnRK1g4qvBn+chuh3zz+kd8nhTG41hwEVf0Cv1Sd6TukaxWnYrfYH+u2ZmOk+qR2eRXKhVCtoN7JFuj5s/41Bb2BgpTH82GYufUp/x+0z95J0flhQOPq3ge3R1ad0LTCAdjl7c2H/1SSPZdDCXthnMfdj9F/QI0XVOIXLF6TLuDa4emanStOKtBzwT4e3QqnAo5A7MrmA0k5JyWpFk32ff+P32J+rR25a9bmJi9e+QZb/a2N0PLz8xFZDSxaZKwIrCAsKZ/mI9cREaOgzuxtf9m5AhQZl0Gn0Hwi6mYwmNk/dwcMrT2k/sgXlk5Hv93/6mvDgCIpW8rJJydrd8w+Z/fViZHIZ47YMo0jFQhzZfMqS85deSfjcek6xyoVTfK//Mq6eOWySJkwNfO+8JOhFsEXQ7NCGE1ZJUexacoBlw9chiRL2WexY5D2dud8sJfB5MAAzuy5iV+iGJI2ldM0S7ArdgCRKyVpZGg1GTvx+FplcRp321ekyri2la5bg+G9nOPvXZUuO39/nNdFh0YCAe15Xvp3fI8n3iosrh28wufVcZHKBXIVysuzS7CT/HIXK5sezSC78HvkjSdAknb2lMwOBFcz+esnbnKaJR1d92Pby13hnezsX7eOPebstGjsbHy9OUk3zye1nmdNrKTKZQMVGZZmy84cUj39ax/kEvQgBYHrnhax/8Aulqhfj8PoTaDU6BISPWiIhk8TxKOiOTC5DEARU9ipK1yye6Dl6nYHl36236AVpIrX8PuevWPtXgix5KR2ZTJbsfMT0zgu5fOg6AJcOXKPrxHaMazodXYyeI5tPYe9kR7Xmn7F3xd9EhkYjiRIh/qE8veFrk1XBgTXHLAUGfo8CePUkIMGN6LhQKBX8cm46T64/I2d+N7LldEnxuFJCZmrICgJ9gzAZTUgShAeFJ3jsq8cB6DTm5aIgmGvGk8KfC/ei1+jRRuu4sPcKmihNssf9jvdFs0xvyz7rd6nJ9+sG0Xlsa345Nx3nHNb3Bbx44Md3dSczovZEfO+9TPH4Mkl9nFwcWXx+Bl0ntGXUukF88XXdRM+RyQQUitgPfY+CbvywYTD5SniSM78bk/4YmSrjfRMQyohaE+mcrz9Ht56O9drVIzfRRuvQRuu4dOg6fg9fIXsbnAw6Az63ngPgUcAdld07JU/RZhv3ZeuUxM5BjUwuQ22vxDWZGkRKlZKCZfIRFhSBTmObyrXkkuYrAkEQ8gIbAQ9ABFZKkrQorceRFPrN7c5P7echmkR6Tuuc4LGthjblxPaz6GL0lKlVgoJl8iXpXiWrFeXpDV8MeiPZcrqgtoE38ZhNQ5nRZREyucDojUMAswREnfbVqdO+epKv92Obn3lx/+Xb/89l3b3Yf77rx2+zbuJv5C7kweAlvT8KaY7/AnmLedJjSkerj1coFUzZNZoF/VYQE6mhdjtzGkalVrLmTuqmwFaO2sS9Cw8xGUXm9V5OtWYVLXtT1VtUwnvXRRAEarSqQpnaJXFxzQJIyGQy6nQwv6eb9mtIWFA4d84+pPXQpjYrd27R/wucszvx4uErGnarnWwJl7CgcL4tPwpNpAZHF0d+vTY3SRMyW5Lm5aOCIOQCckmSdFUQhCzAFaCVJEl34zsnI5SPmn1OjVZtlOq1esKDI3H1zJ7kagijwcieFX8T8iqUrwY1STXFw5TQ0bMfb/zN9oou7ln5I2C15TVtjI527r3RxehQqBQ07lWX4cttZ3SSyX+DaZ3mc2rH+bfiiQp+f7XS0s1uMpk4v+cKMrmMqs0qIpPJ0Gl0PLv9gtyFPciSzSmdR28d+1cfZdnwdRapkaFL+9q8uTDDlo9KkuQP+L/9f6QgCPcATyDeQJARsHNQA9bNzlV2qmQ/wBVKBa2HNE3WuWnFsGV9md55ARIwdFnfWK/pYnSWVJRRbyTEL5SwoHD2rvibLNmz0KxfQ5uZsaQmJpOJC3uvIsgEy8Mmk7Sjz6xu+Nx6TvDLN/Sc1imWpIlcLv/AYUxtr84QxQ43Ttxh769/U7J6MVoN+TLBiWDeYv9I2ksSeBZNP12wdG0oEwShAHAKKC1JUsS/XusH9APIly/fZ76+vmk+vkzi550yY1wPyJWjNvLnwn04ONvz87Efmd55Ia+eBKBQyGn0dR2GLU99WeyUMvvrxZzZdRGA+p1rMuLXzFXNfx2dRockvZsUfkjg8yC+KTkcXYwetYOaIUt6J6qEevKPc5zbfYkaratSq43tfTusXRGk2zRHEAQn4E9g+L+DAIAkSSslSaokSVIlN7ePU8rgU0Ymk8U7S+4392t2R2xkR+AaCpTOy8sHrzAZTOg0ek5sP8uDBGqmdRodAc8CLYEmvTi7+zLaKC3aKC3e/7uYrmPJxLY8vubD9rl/8fCK9bX7p/88T+vsvWidvSf7VsatBRb4PNjymdBr9Ly475fodeu0r86YTUNTJQgkhXQJBIIgKDEHgS2SJO1MjzFkkrqo7dXI5XLkcjm12lVDoTKng6JCo/m+3o+Evv5QcdPvsT+d8/and6kRjKg9KV2F7So2KGMRIfvsi7KJn5BJhuDR1aesGbuFs7svxfm6790XDK81kbUTfuO7OpN4etO6TMOKkRsw6AwY9UZWxiNSWKxKYfIW98TOQY1jVgeafJMxupitIT2qhgRgDXBPkqT5aX3/xPB77E/o63BKVC3yUchHJpB5AAAgAElEQVQo6LV6lg5bx9ObvnQe25rPW6bc89jWjP9tOFM7zOfMzguAuaz2tW/QB7XTB9YcIyo0GkmS8Lnpy/2LjyldI/F699Rg/LbhHN/mjVwuo06Hz9NlDP9Fntx4xvm9Vyhds3i8FqLxEfgimO/qTEIbrUPtoGbCthFUax7b7/j+xccIgtlDWaGQc//CIwqVzZ/otXPkykaw3xskUcIlHslypUrJL+em8+pxAK6e2bF3sk/S+NOT9Ni1qwF0B24JgnD97ffGSZK0Px3GEovTOy8wu/svyOQySlQryqxDEzK8MuRvM//HkU0n0WsNTO+0kE1Pl6SZ4bW1yGQyuk5oy5W/bwDgWSQXXuULfHCcZ2EPVPYqdDE6RFFM14ophVJBo+6ftjy0Xmfg0oFrZHXNkmTD+9TA/+lrhtecgF6jR2mnZOaBCZSplfi4TEYTM7os5NzeK4hvCxX0Gj0Przz5IBBUqF8auVyOg7M9SFChYRmrxjbxj5EsHboWg97IgAU94z1OLpcnubksI5AeVUNnMFtaZzj+WnLA0gx289RdIkIiyerqnM6jSpgQ/1CLAJ4gQHR4TJoEghsn7xAdFkOVphWsqgIqXL4gm56apQkKlskX5zmNe9UjIiSSu+ce0mJA449W5vp9JEni0sHrxETE8HmrKqjUKdecF0WRwxtP8sY/jCa9k29pOKbxVB5f9UGSJHpN60SbYc0TPykVeXLjGYJMhihKmAwm7l94ZFUg8N51kYsHrmHQGkAAlb0KuVwWp6mMez43Vt9dwP0LjyhWubDVkw23PDn4ceeoJP9MHwsZv44vDSlVszj3Lz7CoDPinN0JJxfbi6tFhESi1+px9bTNbLfT6Fac33OF8OAI6nT4nDxFcyd6zvFt3tw8eYcGXWslayb428ydbJ2xE0EQKFm9KLMOTbTqvKyuzgkGVplMRscfWiV5PNZgMpo4uPYYkaHRNO3TIM0ad9ZP3MbORfsAc934nMOTUnzNDZN+58+FezHqTexbdZiNj5ckubxVr9Vz+8x9i3zE4Y2n0j0QlKlVApWdEkEASZSo0qyiVee9P6lQ26toP7Ilzft/EW8nsWvu7NRsnb6bsxmNzEDwHl9Pbo+bZw4CnwfR/NtGNt8jOL3zArO6LUKSoO2IZvSe0TXF18zt5cE2v1/Raw3xlrW9z7k9l5nXZzm6GB2HN51i5Y2fye3lkaR7Ht1y2iJYd/XILURRtEmdvclkQhuljbdpL/B5EIc2nMDTy4N6nWsmKW23bMQ6Dq07jslo4simk6y+vSDF47WGUzvOWX5X14/fTpHk8juuH7+NLsa8cg1+GYImSpvk7m2lWkmBUnnxexyATCZQtbl1D93UJKurM2vvLuTe+YcUKlcg3tm6966LXNh/ldrtqlPpi3JUa/EZDbrW4uxfl6jSrCLdJrXLcH4eGZ1POhAYDUb0WgMOWazbtJHL5TT/tlGqjWfD5N/Ra81Svtvn7uab6V1S9FCQJIlD647z7M4Lvuxdn/wl8yZ6js+t5xadfLlcxsuH/kkOBFWaVuC1bxCSKJG/VF6bBAF/n9cMrT6eyDdRfP5VJSb8/l2s6+q1egZWHkPkmyhUaiVhQRG0GdbM6uvfPHnX8vB8fs8Pk8lkk4dF4PMg7px9SIlqReIUIqzesjJ7lh8CoGglL5vsOTXv/wVPbjxDJpNRvGoRq9/f7yMIAgtOT+XYltNkdXNOVW/epOCcIwtVm30W7+s3Tt5hZrdF6GL0HNt6mkXe0/EqV4DhK75l+IrMXo/k8skGgrvnHjCm8TT0WgPtv29hk9l3SslbLDd+j/wxGUzkyJ0txQ+F3csOsmr0FnQxOg6uOcYW32WJSmDU6VCd7XP/QhRFXNyzUqZW0qty+szqRtHPvIgKi6FB15rJHX4s/lywl/DgCCTRnFP3ufUcr3IFLK/fPfeQqNAoRJOINkbHjRN3YgUCo8GIyWiK12Cn+beNWDV6CzKZQIWGZWwSBAKeBfJtue+RkECC5Vfn4Fk4dndo39ndKF2jONHhMRYNnJTSqHsdClcoSOjrcMrVKRnv+8hkNHHr9D2y5cwa5yTB0dmBFgMa22RMacWz2y8s6SyZTMbze36x3ieZJI9PNhCsGbfVor3+x8976DKuTbqXc32/ZgDrJ24j4k00PX+yXvwrPu5ffGyRwzWZTAS9fJNoIPAsnItNT5fy6slrCpTKk6gzWVzIZDLqdrStr6tbnhwo1Ur0Gj2SKMXK4eu1en5qPw+T0dxkJlfKadq3oeX1a8duMbHlbIwGI71ndInTeeyrQV9SqkZxokKjKVPbNhUyN0/eRRRFc7mivYrrx25/EAgEQeDzr2xf0luwdD4KJmB1IUkSY5pM48HFx4gmkRGr+tOgS60Er3lgzVGWj1iPs2sWpu8bF8tr4+KBa/y/vfMMjKr4+vAzu9lNg4QSQu8gRaQpCIhSxQIooqgUAUWKilKlNxEBC1j4gwhIRxSwIUhRQdCX3iGUUEMCpABppG2b98OGlZKQ3WRLyM7zhezm3jtn2Ow9c8+c8zuHth6l2XONPJph9FinRiz7YBVanZbA4EAeeco1jeY3Ld7Kn8u282iHhrw4uEO+zx7MKwXWEZQoVxwfvRaTwYzeX4fOCdkaeSUwOJB3vurjtOu179uWf37cjUYjKF+z7G3aJfeicNFC1HgkfwlzdR7cnutX4gnfd44u7z93W3w4ITbxtgbzdR6rSaOn69tezx22xPb7b0d/x4tDOmQZrqpWv7JTba7VxNp0XeerAyGo3ayGU6+fF24kpHD0nxOYjdZ0yt++3nxPR2DIMPLV2/MxGc2kp6Qz+72Fto3tI9uPM7nLZ2SkGlg7ZzOz906/qyGTK4g4EUViXBIPPlbD9gQXUrY4S8/OJir8MhVqlbNrX8xRTuw+zayB35KRmsHJPacpU7VUvqzPcSYF1hEMnNUHoRHERV7jjand7guhM0ep07wWi8O/IvbiVR54uEq+LoCTUhK24xS+/nqqN6xy1+91eh1vff56lueGlCtOjUbVOHPwPEajieM7TtGhUA9GLhlIiy7NCClTjIiwSMwmC4FFAty2eitfoyxf7ZzKoS3HqNuiNpXrOCY57koCgvwpXroo1y7H46PXUq/VvYuzNBqBRqsBo9nWvOYmZw9dsPUY1mgFF45F5skRHPv3BFO7WaXLx6wcnGXR4J/Lt/FF/3lotBoeerwWH60f89/cCvvzwMNVcz1+TlyNuoYms+GOxWyxNXUqyHhUdM5e8oMMtSJvfPbmHLb9sAMpofu4F+k66gWHzjebzJzcc4bJXWbYJLCDihfix7hFxMcmMuudBSRfT6b/jF5OX/nfr1yPjmfjwi0UK12Mdr1a5Lipv/O3fXzWZw4pCamUq1GaqevHEFqhBNEXYhnQ4H2klPgF+PLhulFotVqq1K2YK6fbvdJbtlaXoRVCWHHh67uOGdhkNKf2nAFAo9XwS8KSXOv+O0p6agaDm4/j4vEoQsoW4397prsk3fjU3jOcPRxB42cbEFKmmNOvD/aLzilHoHALz/p3tRW+Zfflt4d+9YZZNwylpHTVkiw9/T+Hr5F4NYkj209QtV5FhzOmCjJpKel0Lv46JoMJjUbQ9PlGTPrRWkSVdC2ZC2GRRJyIYu7QJQghaPlqM4YveNvhcbpXfovYiExHULEEK87PueuYeSOWsnbOJkwGE8XLFmP5uTlujdNbLBYS45IICinsklTUA38dZcLz0wFr7cPi8Fku6aOQb/sRKLyTqvUrc/bgeYRG8GAO+kGxkVcZ0XYyMRFxvDz8OV6/pSvchDXD+bzvXMxmc67SBZOuJ/Pmg0PISLduSs/cNjnLUJVXcsuiUN7xOqh4Yeo+UZtZAxdgyKy+37z4b4Z809/hG+XY7wYztXtmaGjFoCyP6TOtO2WrleZ6dALt+7V1+2atRqNxaR/hfZsO2tKZtT5aTu45Q9i/J0hJTOOVUZ1c9oSQHcoROMi6bzazfc0unnipCR36t/O0OfkCKSVmk/me+zDTN45l3Td/4Ovvy7P92mZ7HMCyyau5ci4Gi9nC6pm/8Wzftja5iXLVSzPj7w9ybevJ3WcwZBhJS05HaAS71x9wiiMI23GKjDQD9Vs9eN82sfEv5M/guf2YP2IZxUoXZcDM3ncd88DDVbl8JhqT0UzJSiXQaDT8sXQbx3ed4smeLe1qDl+7aQ2Wn7v7KeBWtFot7fvdXdNjNpk5tfcMRUsVoXTlknbPLb/R+JmGrJ2zCSEEPnof1s/dzJ6NB7GYLOzddIjFp75yqz3KETjAwS1H+Wb4UtJTMjixM5yy1UvToLV9olUFldjIqwx5fDxxUddo2+MJ3l/0Tpart8DgQLvlI/wD/dBoNbYNSp2v8/5Mq9SriJTgo9Oi1Wmp0zzv6qZLP1jF6s/WghA079TY1hfaGZzad5bk6zesYmluSAZ4qnerezZTGfR1X8o9UJqka8m8OLQj21bt4Mu355ORmsGfy7az8MSXLhMLlFIy+pmPOLn7NBazhbHfD6FpxxyjHvmS+q3q8Pn2Dzl3JIJGT9dnWMuJttBp9PkYt9tzfy5dPMRt2QPi9tffTf2RPg8OZu6wJR5vquJO1sz4zSbPu33NLi4cu5jna/b64GUebleP0lVLMnRe/2xF9NJupDH7vW/54KXPOH/UPl35kDLFmL1nGm9M7c70jeOo3+oeyfh2smnRVtJTMki/kc621TvzfL2b/Dp7A8NaTmDyS58x/rnpDp+fdiONIU+M51n/rkzr8WWu/i53r9/PyHaTWTThe8xmM3o/PV1Hd6b/Z70IKVOMs4cv2FJ3hUZw+Wy0w2PYS0JcEsf+PUHajXQy0gz8MsvjgsV5onrDKjzVuxXFShXl1dEvoPfT4euv57m3n3a7LeqJwAEee6Ex33/8C9HnYyhRPoTHXrD2TT28LYyV034mPSWD2ItXqdm4mlMKrhLiErGYLflOVvpWioQG4aPTYjBbkFISGOyY5k1WBAYHMmXtqByPmz1oEVu++weTwcThrWGsjl1gV7y6fI2yTpUKrtuiNv/8uAuLRVLjEeelNf4+/y9bHHnfpsOYjCaH0qA3LtxK+L6zGDNM7Fi7j0NbjtGwrf1NdmIi4pj88kwMaQbCdpyiWKkiPH/HTaptjydYO3sTUkpCy4dQ69Hqdl/fUQoXDSQwOICkq8no/PTUbpp/6jbyylO9WtGg9UNkpGZ4RMZaOQIHCAwKYP7RGSTGJRFcIsh200lJTLVqQGNVTbyRkJrnsTYs/ItZ73wLwBsfvcpLQ++ulnUX8bGJfNhlBjEX4ugzvRutu/5XmPTS0I5En48lfP85Xh3ZidAKJbgQFsn4jtNJjr/BwFlv0LaH47r+kacu4aPzoXSV7OPAUeGXbY/TqcmpGNKN+Ae6JnxiNBg5vjOc0PIhd9k0bMFb1HmsJhlpBp7p47yuVA2frMulM9FYzGbKVCvlcC2Mr78ekZkPj5T4Olh8dT06wVpbABjTjcRciLvrmIq1y7Ps/Gyiz8dSqU4Fp8hsZ4ePzodZu6axbu5mSlYskeNe0/1GaPkQj42t0kedgMloYvxz09m/+QgPPFKVqb+PZtWna4kKv0LX0S9Qo1E1h6/ZtXx/rl66DlgLaH5NXOpss+1mbMdp7Fl/ALDmdP90deE9pSyGtZrIkW3HAfDR+7DuxnKH4tsLx63kp8/XIYE3p3fnhXefzfK4myl4FrPkmTdb8+6sN+2flAOYzWbeazqWyFOXsJgtTPppBI+0c420wZ3jblnxL0nXkmnXu6XD6YUmo4nP+33Dob+P8VTvVvSc+LLD449tP43Df4dRqEgAX+2cel9v0HojKn3UjfjofJi2YZxNjnn+yGX8MmsDhnQjB/46wg+X5ztcDFOmWiniYxKRUlKykmcbtJzYGW772WK2YDCYuJeikY/eB6ERSIu0rigdzPz7+cv1tgZBqz75NVtH0LDNQ/xweT5pN9Jdmm4XfT6WiOORtjDN+nl/uMURaLVanuyZ+y5pPjof3l/0Tp7Gn7ZhLAlxSRQuGpjvqvPDdpxiyYTvKV2lJANm9vK4ltj9jNosdiI30wYjT122yU2bDGZSElIcvtaE1cNo17slbbo/ztTfx+R8ggupUOu/mKXO14ciOXRtG/JNf6o3rELJSiWYsHqYw3nm5WuUQeujxUfvQ+WH7t1PNjAowOU518XLFEPvp0ejEfgG+NrVNaugIISgaGhwvnMCGWkZjHpqCge3HOOPZduY937WDeUV9pG/Pl0XYjKaOH3gPKEVQtiz4QBLJ66ibPXSTFg9zOnl411Hd+bQlmOYTGaad2pM8VzcqIJDghg6b4BT7cotY1YMYvprs0i6lsy7s9/MsbinVKVQZu9xPMvlJlM3jOX7j39Bp9fx6ijXdCxzBL8AX2bvmc6mxVspW700bXs84WmTvJ6MVAMmo3V/yJhhIibi7v0Lhf14xR6B2WTmvWZjiTx5CYvFgtlkwWQwodVpeeaN1gz6up8TrYV9mw8zb/hSipctyugVgwgq5p62iPmF2ItxjHpqCnFR1+gxoQslK4Sw7ps/aNC6Dl3HdL5vC67uxGKxYMww5krKW5F3Zg9ayLq5m9H76fnkzwm52osr6Kg9glu4ePISF09E2VoG3sykkBaJyWC663hDhpEJz03n0NYwGrSpw+RfR6LT25cNkZaSzqQXPiEjzUDkqUvMHbaEEYsGOm8y9wHfjlnJpdNXsFgki8etROOjxZBm4OSeM4RWLMGTr+U+7p1fOH/sIsNaTiQlMZXnBz7N29kopxYULp25Qnx0ArWaPJBvVG7f+fINen3wCr4Beru/n4qsKRhLsxwoUa64LQ3ON8CXph0fxr+wHxVrl6NXFg1i/v1xF2E7TmE2mTn270n++XG33WOZDCbMmRWxZpOZpKvJzplENoTvP8vHPWfxwye/YDabXTqWvfj66xGZ/98SbJK+JoOJq1HX3WZH+P6zzBuxlH9+sv/zs5elk1bZOqatm7uZ69Hxdp9rsViYP2o5/eoNY/WMtU63zdn88+Mu+tcbzphnpzL6mY/IT1GEQkUC3eYEzCYzZw9fIOm6a7/TnsArnggKFQnki3+nsOHbv6hcpwJPv9H6nnFuvzsyfPwL2Z/xU7hoIbqOfoEVU34kqHgh3pzeI9d250RKYgrDW00i7UY6vgF6LBbpsLyzK+gzrRtXzsdw+Uw0PSd1YePCrZzcc4YioUG0693SLTbERl5lWMuJ1u5hAZvRajVO7RRWrFQRfPQ+GDNMaDQah3L0t63aydrZG0lPyWDppFXUavIAhnQjGxduoW6L2rTv636RtXvx6+yNtiyuo/+cIOlaMsE5JAw4g7OHLzDrnQX4BugZMm9Alj2h3YXJaGJw83FEHI9CaAQz/55MtQYFR+7cKxyBxWLh3OEIylUvQ4uXm+X4JWv63CO07/ck//fLHpp3fpQmHbJvpp0VPSe+TPdxL6LRaFz6hY6PSbTJBmSkGjh/NO/yDs4gOCSIT/6YYJt7u16tSLqWTKGigS6R9M2KqPAriMy9iIy0DML3n83SESTH3+DsoQtUfqiCQze3PtO6k5qcRlT4FXpPfoXAIPsrqpOv37DpKAkhiAiL5Ouhi8lINbDz170ULhJIi5eb2X09V/Ng85qc3HMaY4aJoGKFKFTk3u1QncW4DtO4euk6QiP4qOsXzNo51S3jZsX5oxeJOHHJFl5eP/9PBs3p6zF7nI1XOIJ5w5eyfv6fWCwWNi7awpy9H9/zeCEEA2b0YsCMXrke05k3PLPZTEpiKoWLFrrNsZSpVooHm9UkbMcphEbQeVDW+fbu5GZaX9iOUzRs8xAf/jYKnV6X5U02PjaRxeNXYjFLen/4KsVLW6U0Dvx1lIiwSB5/8VFCyuZOwKxWk+oUCSmMENa9oKxurPExCfStOwxjhhGNRsM3hz4ltIJ9NRsBhf0ZuSR34nJtejzO7/P/5NyRC9RqUp3QCiG20KUhw8jFk5dydV1X0XNiF0LLFSf24lXa93/S6XsEhnQDqclpFCkRfNv7KYnWCn1pkSRfsz8ck3QtmYsnL1G1XkWn1RaElCueqc1tDS9XrXfvtOb7Da9wBHs3H7Z58jMHztsKv3KDlBKLxeK2le316HjebTKG61fiqd2sBh9vHm/L6dZoNEzbOJao8CsULRnsksYWjrL1+x2cOXAeaZGE7Qxnx6/7aNGlaZbHTur8ibULlbS2Q5yz72O2rd7Jp6/PRpotLJ+yhmVnZxNQ2PEvs3+gH98cmcHp/ecoX6NMltryB/48iiHNQNqNdPR+OnavP0DHt55yeCxHCQwKYO7BT23aQYZ0A6UqhXLlXAw6vU++S0/NThLaGZw5eJ5hrSZiSDfS8uVmjFgy0LbYeXf2m8zsOxcfHy3vzravavzy2WjefmQkUkoCggKYf2RGtk8wRoORHz751db34l4aP0VDg/lsy0TWfbOZ6g2ruOz/w1N4hSNo17MFyz9cgxCCOs1r5toJRByPZHjrSSRdu8Grozrx+oddcz4pj2xe/DfXrsRjNpo5vf8cR7afoGGb/6SvNRoNFWraJ1IVExHHzrX7qFq/ksuKogKD/P+rJM5BhO7ymRjMJmuI5Eqm9O6+jQdtapZavZbLZ6JzHYv1D/Sj7hO1s/19lboVbKE1IQTV3Nyg5qZD1/vpmbPvYy6dvkJohZDbVrE3ElIwGkwUDQ3O7jL3NSun/URqUhoA29fs5PWPuto0d558rQWtuzVHCGH3d/bfn3aTnpqB2WhGSjiy7Xi2e0MLRi5n3bw/MaQb2PHLXlZdmX/Pp50ajaoV2BRVr8gaemVEJ6ZtGMuY7wYz+deRdp2ze/1+Bj46ms/6zCE988a0aNxKEuOSsJgtrPp0LYlXk1xpNmCtar15w7BYLBQrZX/XJCmlLcMj6XoyAxq8z7wRyxj9zBT2bDjoEnsfe6ExHQa0o9wDpek8pAMPP5m92mXPSV3Q+erQ+eroMf4lAJ54uRm+Ab74F/KjcLFClK9ZxiV2AlR+qCLTNozj1ZGdmLx2VJ6UM01GEyf3nObaFfuzh27FR+dDxdrlb3MC29fs5JUyfelecQBLJv3AlXMx98wMc3U2z+oZa+lYuAdv1BrktAKuUlVKovezZv1otJq7Fg5ardahhVvV+pX++76YLbdVxd/J2SMR1m5rElKTUklNTsvFDP7j2P+dZO7wpexevz9P1/EEXlFQ5igJcYl0r/Q2hjQDOj8dnd97ljen92Bm37n8sWwbJoMJvb+eNTELcoxBnth9mk96zUKj1TB6+SCHV7cWi4Vlk9dweOsx2vd/kjbdHs/5JOCnL9Yxb8RyAosE8PGm8aSnpDO2wzTb6uvFIe0ZMKO3Q7a4gviYBKSUt0ltnzsSQeSpyzz8ZF23bUzmBbPJbMsosVgk0zeOJbRCCEEhQfg5qPh5K71rvMel01dsr3399VSoXY4v/p1ym8qnIcPIuA5TObQ1jLpP1Gbq72PQ++nzNKc7SYhLpFv5ARgz+xm3eLkZY74bnOfrGtINLBzzHRdPXqLbmM7UaZ73J9Uda/dy+O8wHn+xCXXu0RZ176ZDfND5UxDQpMPDjPt+aK7HvHjyEm8/MoKMVAO+AXo+XDuK8jXKEFwiyKM1DqqgLA+kJafb+rWaMkxcj04AoN+nr3EjIYXLZ6PpM7WbXRtRH3aZQVyUtYHNlFc/d7gFnUajodekl2GS/cqRhgwj80Yut9UxfD1kMZPXjsQvwBdpse5xNHu+sUN2uIqsYvdV6lakSt37ZzMuKvwy549F2kJaU175nOT4G+h8dXy+fXKOekk3SbyaxCe9ZxMbEUffT16jTNVSRF+IxWy0PgVkpBmICr/MsX9O3NZX4P9+3sOJXaeRFsmpvWfYvmaX0/cZtD5am9S60Ghsq/i8ovfTZ9kSMy80e64RzZ7LOVW40VP1WXZ+DsnXb1C+Rt6ePCOOR6HJ3Dc0Gc3M7DuXa5fjCQwOYPaeaXYnIXgKj4SGhBBPCyFOCSHOCCFy7kDiYkxGE1HhlzGkW3OlS1UOpV2vlmi0GoJLFKbbmM6AtR5hwuphzD3wKY2ebmDftU3/Pcrf/EKD9TF+zuBFdCjUg/eajeFGLoTpskOr1dgej7U+GgoXC8zspTCTofMH8L/d0+8ZO88PmIwmpvX4ki6l+jDr3QX5qojpTkLKFkPrY/0q6f10JFxNwpBuJDUplZ++WG/3deYOXcL+Pw5zISySD178jKHz+9OuZ0tqPlodvb91hW8xWwitcLtufUBhvzteO1+Fs3DRQgxdMIDQ8iHUaV6TNz92XX2MOykaGkyFmmXznOZdr2VtAgr7ERDkj07vQ0JsIsYMI0nXktm4cIuTrHUdbg8NCSG0QDjwJBAF7AW6SimPZ3eOK0NDqclpvP3wCK5djicgyJ+vD3xiC1MY0g3ofHV5+iM58OcRpnb/Eo1GMH7VMNsm7ekD5xjaYgLpKRn46LV0G/sir43v4pQ5ARz+O4yvhyymWOkivL/onSxX3vfiyvkYfvt6M6UqhdK+f1u3ZUnd5I+l22y9cP0K+TFh9TAaPVXf5eNej45n8+K/CSlXnDbdH7f7s78QFsn6+X9SqmIIi8Z9T0aaAV9/Pa9N7GJ3r+ZxHaex5/cDtp7Ka2K/JTA4ECklP3/1O4e3hfFsnzY82v72uhYpJYvGrWT7ml081qkxb07vnq8K0u4Hkq4lo/P1yVO6aWpyGmcPXUDn68PwVpPISDPgF+DLwP/1uWcfaFeSn0NDjYEzUspzAEKI74HngWwdgSvZu/EQ16MTSE/NwGKxsH31Ljq9+wyAU+KsDdvWZU3Mt3e976PT2la5QmjQ6Z37UdRr+SBzD36aq3ONBiPvNRlD4rVk9H464mMTreEpN3JTpuMmFrPr+0CbzWbeaTya+JgEfHQ+xFyIpfu4l+w6t9KD5XnnC6ve0AOPVGPNzLr1fhEAAAa1SURBVN+oWr8SLw3taPf4fT/uQfj+cyRdTaLnB6/Ymv8IIeg8qD2dB7XP8jwhBG981I03Pupm91iK/1j6wSpWTvsZjVbDxDXDafyMfU/7dxJQ2N+20Bv7/RB++3oTDz1RO089JdyFJxxBWSDyltdRwKMesAOAMlVL2lIINVoNZauXcsu4lR+qyGsTuvDzrA3UbFSVTtk0X/EEyddvkJKUirRIMlINnNh5yu02tO7WnB2/7uHw32E0f+FRGj3t+qeB5Os3SIhNxGw0YzaaObQ1zG5HcCsPPV4rV+m5FWuXZ9Xl+Ugp1YreTVgsFlZM+dG20Ph29IpcO4JbadrxEZp2zHEhnm/whCPI6i/8rviUEKIf0A+gQoUKLjOmesMqjFz6HltX/kvjZxvYHft3Bq+M6GR32MCdFC1ZhIcer83xXeFIi4UX3nO/k9L76pj8i32pvs4iOCSIGo2qcu5IBBazhWf7eqYnrnIC7kMIQXCJIBJiEtD6aCldxXN6Rp7EE3sETYFJUsqnMl+PBpBSTsvunPzes7ggYjabCd93jmKlilCyYv7OeHAmRoORw38fp3iZolSu47oFiCL/EHEiim9HryA4JIh+n76WLyr0nYW9ewSecAQ+WDeL2wCXsG4Wd5NShmV3jnIECoVC4Tj5drNYSmkSQgwENgFaYOG9nIBCoVAoXItHCsqklL8Dv3tibIVCoVDcjldoDSkUCoUie5QjUCgUCi9HOQKFQqHwcpQjUCgUCi9HOQKFQqHwcu6LfgRCiDggIhenhgBXnWzO/YA3zlvN2Xvwxnnnds4VpZQ5VoTeF44gtwgh9tlTTFHQ8MZ5qzl7D944b1fPWYWGFAqFwstRjkChUCi8nILuCOZ52gAP4Y3zVnP2Hrxx3i6dc4HeI1AoFApFzhT0JwKFQqFQ5ECBdQRCiKeFEKeEEGeEEKM8bY+rEUKUF0JsFUKcEEKECSEGedomdyGE0AohDgoh1nnaFnchhCgihFgjhDiZ+Zk39bRNrkYIMSTzb/uYEGKlEMLP0za5AiHEQiFErBDi2C3vFRNC/CGEOJ35b1FnjlkgHYEQQgvMBp4BagNdhRC1PWuVyzEBw6SUtYAmwDteMOebDAJOeNoIN/MlsFFKWROoRwGfvxCiLPAe8IiUsg5WCftXPWuVy1gMPH3He6OAv6SU1YG/Ml87jQLpCIDGwBkp5TkppQH4Hnjewza5FCnlFSnlgcyfk7HeGMp61irXI4QoB7QHFnjaFnchhAgCngC+BZBSGqSUCZ61yi34AP6Zza0CgMsetsclSCm3A9fvePt5YEnmz0sAp/a4LaiOoCwQecvrKLzgpngTIUQloAGw27OWuIUvgBGAxdOGuJEqQBywKDMktkAIEehpo1yJlPIS8BlwEbgCJEopN3vWKrdSUkp5BayLPsCpzZULqiPIqvu3V6RHCSEKAT8Cg6WUSZ62x5UIIToAsVLK/Z62xc34AA2Br6WUDYAUnBwqyG9kxsSfByoDZYBAIUQPz1pVcCiojiAKKH/L63IU0MfIWxFC6LA6gRVSyp88bY8beAx4TghxAWv4r7UQYrlnTXILUUCUlPLmE98arI6hINMWOC+ljJNSGoGfgGYetsmdxAghSgNk/hvrzIsXVEewF6guhKgshNBj3VRa62GbXIoQQmCNGZ+QUs70tD3uQEo5WkpZTkpZCetnvEVKWeBXiVLKaCBSCFEj8602wHEPmuQOLgJNhBABmX/rbSjgG+R3sBbolflzL+BXZ17cIz2LXY2U0iSEGAhswppdsFBKGeZhs1zNY8BrwFEhxKHM98Zk9odWFDzeBVZkLnTOAa972B6XIqXcLYRYAxzAmiF3kAJaYSyEWAm0BEKEEFHARGA6sEoI0QerU+zi1DFVZbFCoVB4NwU1NKRQKBQKO1GOQKFQKLwc5QgUCoXCy1GOQKFQKLwc5QgUCoXCy1GOQKHIA0KIjUKIBG9SPlUUPJQjUCjyxqdY6zcUivsW5QgUCjsQQjQSQhwRQvgJIQIzdfHrSCn/ApI9bZ9CkRcKZGWxQuFspJR7hRBrgSmAP7BcSnksh9MUivsC5QgUCvuZjFXHKh1rkxSFokCgQkMKhf0UAwoBhYEC2SZR4Z0oR6BQ2M88YDywAvjYw7YoFE5DhYYUCjsQQvQETFLK7zJ7Yu8QQrQGPgBqAoUylSL7SCk3edJWhcJRlPqoQqFQeDkqNKRQKBRejnIECoVC4eUoR6BQKBRejnIECoVC4eUoR6BQKBRejnIECoVC4eUoR6BQKBRejnIECoVC4eX8PxLdHgcP0uGbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.scatter(x[:,0],x[:,1],c=y, s=8);\n",
    "plt.xlabel(\"x1\"); plt.ylabel(\"x2\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Linear(in_features=2, out_features=2, bias=True)\n",
       "  (1): Sigmoid()\n",
       "  (2): Linear(in_features=2, out_features=1, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = torch.nn.Sequential(\n",
    "    torch.nn.Linear(2, 2),\n",
    "    torch.nn.Sigmoid(),\n",
    "    torch.nn.Linear(2, 1)\n",
    ")\n",
    "model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "x, y = gen_nn_fake_data(10000)\n",
    "x = torch.tensor(x).float()\n",
    "y = torch.tensor(y).float()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 0.01\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jyoti/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py:1006: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.\n",
      "  warnings.warn(\"nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.\")\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7252525687217712\n",
      "0.13772839307785034\n",
      "0.11242274194955826\n",
      "0.11006315797567368\n",
      "0.0606331005692482\n",
      "0.031591203063726425\n",
      "0.017416639253497124\n",
      "0.012376965023577213\n",
      "0.009449083358049393\n",
      "0.007461360190063715\n"
     ]
    }
   ],
   "source": [
    "for t in range(10000):\n",
    "    # Forward pass: compute predicted y using operations on Variables\n",
    "    y_hat = model(x)\n",
    "    loss = F.binary_cross_entropy(F.sigmoid(y_hat), y.unsqueeze(1))\n",
    "    if t % 1000 == 0: print(loss.item())\n",
    "       \n",
    "    # Before the backward pass, use the optimizer object to zero all of the\n",
    "    # gradients for the variables\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    \n",
    "    # Calling the step function on an Optimizer makes an update to its\n",
    "    # parameters\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Parameter containing:\n",
      "tensor([[-0.5389, -4.7348],\n",
      "        [ 6.1663,  1.6080]]), Parameter containing:\n",
      "tensor([ 30.6644, -45.6655]), Parameter containing:\n",
      "tensor([[-26.7002,  23.7534]]), Parameter containing:\n",
      "tensor([-11.9590])]\n"
     ]
    }
   ],
   "source": [
    "print([p for p in model.parameters()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lab\n",
    "* Get an AWS account and ask for GPU access.\n",
    "* Add a dataset and data loader to the neural network training.\n",
    "* Modify the neural network example to solve a regression problem.\n",
    "\n",
    "### Solutions:\n",
    "#### Data Loader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([1000, 2]), torch.Size([1000]))"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class NNDataset(Dataset):\n",
    "    def __init__(self,n=10000):\n",
    "        x, y = gen_nn_fake_data(n)\n",
    "        x = torch.tensor(x).float()\n",
    "        y = torch.tensor(y).float()\n",
    "        self.x, self.y = x, y\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.y)\n",
    "    \n",
    "    def __getitem__(self, idx):\n",
    "        return self.x[idx], self.y[idx]\n",
    "    \n",
    "nn_dataset = NNDataset()\n",
    "\n",
    "#Data Loader\n",
    "nn_dataloader = DataLoader(nn_dataset, batch_size=1000, shuffle=True)\n",
    "x, y = next(iter(nn_dataloader))\n",
    "x.shape, y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 0.01\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jyoti/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py:1006: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.\n",
      "  warnings.warn(\"nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.\")\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.010445968247950077\n",
      "0.003490675240755081\n",
      "0.00435412023216486\n",
      "0.004863296169787645\n"
     ]
    }
   ],
   "source": [
    "for t in range(200):\n",
    "    for i, (x, y) in enumerate(nn_dataloader):\n",
    "        # Forward pass: compute predicted y using operations on Variables\n",
    "        y_hat = model(x)\n",
    "        loss = F.binary_cross_entropy(F.sigmoid(y_hat), y.unsqueeze(1))\n",
    "       \n",
    "        # Before the backward pass, use the optimizer object to zero all of the\n",
    "        # gradients for the variables\n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "    \n",
    "        # Calling the step function on an Optimizer makes an update to its\n",
    "        # parameters\n",
    "        optimizer.step()\n",
    "    if t % 50 == 0: print(loss.item())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Parameter containing:\n",
      "tensor([[ 0.6900,  6.1085],\n",
      "        [-4.5064, -1.2060]], requires_grad=True), Parameter containing:\n",
      "tensor([-39.8663,  33.3423], requires_grad=True), Parameter containing:\n",
      "tensor([[ 22.6094, -28.0314]], requires_grad=True), Parameter containing:\n",
      "tensor([-10.6432], requires_grad=True)]\n"
     ]
    }
   ],
   "source": [
    "print([p for p in model.parameters()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solving regression problem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1000, 2]) torch.Size([1000])\n"
     ]
    }
   ],
   "source": [
    "# Here we generate some fake data for 2 layer input\n",
    "def lin(a,b,c,x1,x2): return a*x1+b*x2+c\n",
    "\n",
    "def gen_fake_data(n, a, b,c):\n",
    "    x1 = np.random.uniform(0,1,n) \n",
    "    x2 = np.random.uniform(0,1,n) \n",
    "    y = lin(a,b,c,x1,x2) + 0.1 * np.random.normal(0,3,n)\n",
    "    return x1,x2, y\n",
    "\n",
    "x1,x2, y = gen_fake_data(1000, 3., 8.,7)\n",
    "x=[[a1,a2] for a1,a2 in zip(x1,x2)]\n",
    "\n",
    "x = torch.tensor(x).float()\n",
    "y = torch.tensor(y).float()\n",
    "print(x.shape,y.shape)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Linear(in_features=2, out_features=2, bias=True)\n",
       "  (1): Sigmoid()\n",
       "  (2): Linear(in_features=2, out_features=1, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = torch.nn.Sequential(\n",
    "    torch.nn.Linear(2, 2),\n",
    "    torch.nn.Sigmoid(),\n",
    "    torch.nn.Linear(2, 1)\n",
    ")\n",
    "model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 0.01\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "160.61585998535156\n",
      "4.15768575668335\n",
      "0.27888235449790955\n",
      "0.1326553374528885\n",
      "0.10449597984552383\n",
      "0.09284494072198868\n",
      "0.0883951410651207\n",
      "0.08756507188081741\n",
      "0.08722952008247375\n",
      "0.08703199028968811\n"
     ]
    }
   ],
   "source": [
    "for t in range(10000):\n",
    "    # Forward pass: compute predicted y using operations on Variables\n",
    "    y_hat = model(x)\n",
    "    loss = F.mse_loss(y_hat, y.unsqueeze(1))\n",
    "    if t % 1000 == 0: print(loss.item())\n",
    "       \n",
    "    # Before the backward pass, use the optimizer object to zero all of the\n",
    "    # gradients for the variables\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    \n",
    "    # Calling the step function on an Optimizer makes an update to its\n",
    "    # parameters\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8.1\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([[8.0696]])"
      ]
     },
     "execution_count": 149,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Prediction of linear model\n",
    "print(lin(3,8,7,.1,.1))\n",
    "xnew=torch.tensor([[.1,0.1]])\n",
    "yhat=model(xnew)\n",
    "#NN model predictions\n",
    "yhat.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# References\n",
    "* https://pytorch.org/docs/stable/index.html\n",
    "* http://pytorch.org/tutorials/beginner/pytorch_with_examples.html\n",
    "* https://hsaghir.github.io/data_science/pytorch_starter/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "nav_menu": {},
  "toc": {
   "nav_menu": {
    "height": "116px",
    "width": "251px"
   },
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "toc_cell": true,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": false
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}