01 - Data Manipulation and Perceptron.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table align=\"left\">\n",
    "  <td>\n",
    "    <a href=\"https://colab.research.google.com/github/fmottes/unito-neural-networks/blob/master/01%20-%20Data%20Manipulation%20and%20Perceptron.ipynb\">\n",
    "      <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\" style=\"width:175px;\"/>\n",
    "    </a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <center> **Neural Networks** (MFN0824) </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lectures: _Dr. Matteo Osella_\n",
    "### Notebooks: _Francesco Mottes_\n",
    "<br>\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From this point on we dive deeper into the topics that are specific of the Neural Networks course. It is recommended to build a bit of confidence with the Python language and the Jupyter notebook environment before going through the following material. Reviewing the some fundamental concepts of machine learning is also advised, although we will try to give (brief) explanations of some basics.\n",
    "\n",
    "The reference book for the practical part of this course is:\n",
    "- *Hands-on Machine Learning with Scikit-Learn, Keras and Tensorflow - A. Géron*\n",
    "\n",
    "The whole first half of the book is dedicated to an introduction to classical Machine Learning, using the python scikit-learn library. It is an excellent place for reviewing basic concepts and to be introduced to the Jupyter notebook environment and the scikit-learn library. There is also a [free github page](https://github.com/ageron/handson-ml2) where you can find some notebooks with the code from the book, a good place to start if you want to look into more advanced topics.\n",
    "\n",
    "**NOTE:** You can easily (and legally) find online a raw and unedited version of the second edition of the book, released by the author before the book was finished. Be advised that, besides not being the revised version, **it is not even the whole book**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python modules references:\n",
    "- Numpy Docs: https://numpy.org/\n",
    "- Matplotlib Docs: https://matplotlib.org/\n",
    "- Scikit-learn Docs: https://scikit-learn.org/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <center> **01 - Data Manipulation and Perceptron** </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook we will show in which format data usually come in when facing machine learning problems. We will the implement from scratch the perceptron algorithm and a fully-connected feed-forward neural network with back-propagation and mini-batch Stochastic Gradient Descent (SGD). We will apply the algorithms to two simple mock datasets, in order to better understand the limits of the Perceptron."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <center> **Data Manipulation** </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In supervised learning tasks, we give to the chosen algorithm a set of *N* observations with an associated label (class to which the given observation belongs). In order to be manipulated in a mathematically convenient way, observations are represented as **feature vectors**: we take the set of *D* measured features associated with one observation and we put it into a vector, such that two different observations are represented as two points in a D-dimensional space.\n",
    "\n",
    "$$\n",
    "{\\rm feature~vector:~~~} {\\bf x}~=~ [x_1, x_2, x_3, \\cdots, x_D]\n",
    "$$\n",
    "\n",
    "Taking a step further, we can then stack all the N vectors of our observations horizontally, one on top of the other. What we get is a NxD matrix that summarizes all of our data: each row corresponds to a different observation, each column corresponds to a different feature of the observations. Such a matrix is generally called **data matrix** or **feature matrix** and is indicated with **X**.\n",
    "\n",
    "$$\n",
    "{\\rm feature~matrix:~~~} {\\bf X}~=~\\left[\n",
    "\\begin{matrix}\n",
    "x_{11} & x_{12} & \\cdots & x_{1D}\\\\\n",
    "x_{21} & x_{22} & \\cdots & x_{2D}\\\\\n",
    "x_{31} & x_{32} & \\cdots & x_{3D}\\\\\n",
    "\\vdots & \\vdots & \\ddots & \\vdots\\\\\n",
    "\\vdots & \\vdots & \\ddots & \\vdots\\\\\n",
    "x_{N1} & x_{N2} & \\cdots & x_{ND}\\\\\n",
    "\\end{matrix}\n",
    "\\right] ~=~\\left[\n",
    "\\begin{matrix}\n",
    "{\\bf x_{1}}\\\\\n",
    "{\\bf x_{2}}\\\\\n",
    "{\\bf x_{3}}\\\\\n",
    "\\vdots\\\\\n",
    "\\vdots\\\\\n",
    "{\\bf x_{N}}\\\\\n",
    "\\end{matrix}\n",
    "\\right]\n",
    "$$\n",
    "\n",
    "Lastly, we can organize also the labels of the observations in a convenient N-dimensional vector, called **label vector** and generally indicated with the **y** letter. In such a vector, the number at the n-th position represents the class to which the n-th row of the data matrix belongs.\n",
    "\n",
    "$$\n",
    "{\\rm label~vector:~~~} {\\bf y}~=~ [y_1, y_2, y_3, \\cdots, y_N]\n",
    "$$\n",
    "\n",
    "When dealing with neural networks and data with more than two classes, another notation is preferred for expressing the label of one observation, the so-called [one-hot encoding](https://www.kaggle.com/dansbecker/using-categorical-data-with-one-hot-encoding). If data come in *K* different categories, to each observation is associated a K-dimensional vector where all entries are zero, except for the one representing the class of the oservation, which is set to one.\n",
    "\n",
    "Suppose to have K=3 different classes in your data, and that the chosen sample belongs to the third class. The one-hot encoding of the class label \"3\" would be:\n",
    "\n",
    "$$\n",
    "\\left[\n",
    "\\begin{matrix}\n",
    "{0}\\\\\n",
    "{0}\\\\\n",
    "{1}\\\\\n",
    "\\end{matrix}\n",
    "\\right]\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#standard imports\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "#function for fast generation of training and test set from data\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <center> Synthetic Data </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now create two simple synthetic 2-dimensional datasets, that we will use in the following to visualize the performances of our hand-made Perceptron and feed-forward neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#generators of very simple 2d datasets we will use in the following\n",
    "from sklearn.datasets import make_blobs, make_moons"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#fix parameters\n",
    "N = 200 #number of observations\n",
    "D = 2 # features per sample\n",
    "test_size = .25 #test data are 25% of all data\n",
    "\n",
    "rnd_state = 42 #random numbers genereator initial state, fixed for reproducibility\n",
    "np.random.seed(rnd_state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GENERATE BLOBS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#generate two blobs, one for each class, of 2D data\n",
    "X_blob, y_blob = make_blobs(n_samples=N, n_features=D, centers=2)\n",
    "\n",
    "#generate training and test set\n",
    "X_blob_train, X_blob_test, y_blob_train, y_blob_test = train_test_split(X_blob, y_blob, test_size=test_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAGKCAYAAADngI2gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmcVOWZ9//PRdtog9rggIrdoj4zEY2KMrYLGhVZFBAJ5jE60YyYZCTGPMYlQwImUWLym+B0Ek0y8zyjccFEJ3GJookhuAUxCMbGVolrjIlLu8CIoEIj2/X741RDd62nqk/VOVX9fb9evOw+darqKkjuq+7tus3dERER6a5f3AGIiEjyKDmIiEgGJQcREcmg5CAiIhmUHEREJIOSg4iIZFByEAHMbI6Zebc/681shZnN6HbPmNRjBxf52ovM7M7ooxYpnx3iDkAkQdYCE1M/DwROBa41sw/d/b/jC0uk8pQcRLbb7O7Luv3+kJkdA0wDlBykT9Gwkkh+HwD1uR40swFm9mMze9vMNpjZE2Z2Uo57Z5jZ38ys08zuM7OmtMdnm9nLqdd5x8x+Z2Z7Rvx5REJRz0GkGzPr+v/EAGAqcALw+TxP+WnqvsuAl4HzgPvM7ER3/0O3+0YDI4BLgZ2Aq4D5wBGp9z0n9RpfB54F/g4YSzC8JVJxSg4i2/0dsCnt2o/d/WfZbjazA4HPAJ9z95tT1xYCzwDfAk7udvvuwDHu/mrqvleBP5jZRHf/HXAkcL+7/99uz7krgs8kUhINK4lst5bgm/wRwCeAi4DpZnZFjvuPAAy4o+uCu29N/f6JtHuf7EoMqfuWACsJkgLAU8BkM/u2mR1pZnURfB6RkqnnILLdZndv6/b7EjOrB/7NzH6S5f5hwIfuvj7t+jvAADPb0d0/Sl1bmeX5K1OvAXAjsAswA7gceNfM/h8wx923lPh5REqmnoNIfs8B/YG/z/LYW8DOZjYg7foewPpuiQGCYaV0u6deA3ff6u5Xu/uBwHDg+wRzEOf1Mn6Rkig5iOTXteHt9SyPPQE4cHrXBTOz1O9/SLv3H81seLf7jiVIDn9Mf1F3f93d5xJMcH+8V9GLlEjDSiLb7WBmR6d+7g8cDnwTuMfd3zazA7rf7O7Pm9kvgP8ws13ZvlrpAOBLaa+9EviNmc1h+2qlJ1OT0ZjZtcBqYBnB3MeJwMcIVi+JVJySg8h2jcDS1M+bgFeB/wK+m+c55xE09N8CBgErgClpy1hJve6DwDXAUGARwfxC98fPA75IkDxeBs5z9/mlfxyR0pmOCRURkXSacxARkQxKDiIikkHJQUREMig5iIhIBiUHERHJULVLWYcMGeL77rtv3GGIiFSV5cuX/4+7Dy10X9Umh3333Ze2trbCN4qIyDapisAFaVhJREQyKDmIiEiGiiYHM7vRzFaa2Z+6XWs1sxfM7Bkzu9vMBlUyJhERyVTpnsM8YGLatQeAg919JPASMLvCMYmISJqKJgd3X0xQebL7tfvdfXPq12VAcyVjEhGRTElbrfR54La4gxCR3N5//31WrlzJpk3px21LnOrr69l9993ZddddI3m9xCQHM/sGsBm4Nc89M0iVOR4+fHiu20SkTN5//33eeecdmpqaaGhoIDjbSOLm7nR2dtLR0QEQSYJIxGolM5sOTAHO9jw1xN39OndvcfeWoUML7uEQkYitXLmSpqYmBgwYoMSQIGbGgAEDaGpqYuXKbMeVFy/2noOZTSQ47eqELAe1i0iCbNq0iYaGhrjDkBwaGhoiG+6raHJIHak4BhhiZm8AVxCsTtoReCD1TWSZu59fybhq0fz2DloXvsibazrZa1ADM08ewbRRTXGHJTVAPYbkivLfpqLJwd0/k+XyDZWMoS+Y397B7LtW0LlpCwAdazqZfdcKACUIEQklEXMOEq3WhS9uSwxdOjdtoXXhizFFJCLVRsmhBr25prOo6yKy3bx58zAzPvzww7hDYe3atXzuc59j8ODBNDY2cvbZZ/Puu+9W5L2VHGrQXoOyTxjmui4iyXTmmWeyaNEirr/+eubNm8cTTzzBtGnTKvLesa9WkujNPHlEjzkHgIb6OmaePCLGqESkGEuXLmXhwoU88sgjHH/88QA0NTVx1FFH8eCDDzJ+/Piyvr96DjVo2qgmvvepQ2ga1IABTYMa+N6nDtFktCTK/PYOjp37MPvNuo9j5z7M/PaOir334sWLOfHEE9l5551pbGxkzJgxtLe357x/1qxZHHLIIey88840Nzdz9tln8/bbb/e459577+Xwww9n4MCBDB48mKOOOopHHnlk2+M33HADBx10EA0NDQwZMoQTTjiBZ599Nud7LliwgD322GNbYgA48sgj2W+//ViwYEEvPn046jnUqGmjmgomAy13lbjEuaJu0aJFTJgwgRNPPJGbb76ZgQMHsmTJEjo6Ohg1alTW56xcuZLLLruMvfbai1WrVvGDH/yAsWPHsmLFCurq6vjLX/7C6aefzkUXXURraysbNmxg+fLlrF4dlJJbvHgx559/PldeeSWjR4/m/fffZ+nSpaxduzZnnC+88AIHHHBAxvUDDzyQF154IZq/jDyUHPooLXeVOOVbUVfu//3Nnj2bQw89lIULF27bFzBxYnqx6J5uvPHGbT9v2bKF0aNH09zczJIlSzj++ONpb29nl112obW1ddt9kydP3vbzH//4R0aOHMns2duLTk+dOjXve7733nsMGpR5gsHgwYN55ZVX8n/ICGhYqY/ScleJU1wr6tatW8fjjz/O9OnTi9owtmDBAo455hgaGxvZYYcdaG4Oike/9NJLABxyyCGsXbuW6dOnc//997Nu3boezz/ssMNob2/nkksuYfHixWzcuDHU+2aL0d0rshFRyaGP0nJXiVNcK+ree+893J1hw4aFfs4TTzzB1KlTaW5u5uc//zlLly5l2bJlAGzYsAGAESNGcM899/DKK68wefJkhgwZwllnncWqVasAGD9+PDfddBOLFy9mzJgxDBkyhAsuuCAjiXQ3ePBg1qxZk3F9zZo1WXsUUVNy6KO03FXiNPPkETTU1/W4VokVdYMHD6Zfv3689dZboZ9z9913M3ToUG677TamTp3K0UcfzZ577plx3ymnnMKjjz7Ku+++yw033MCDDz7IhRdeuO3x6dOns3z5ct555x1aW1uZN28e3/nOd3K+7wEHHJB1biHXXETUlBz6qLj+zykC8a2oGzhwIEcddRQ/+9nPyFMAuofOzk7q6+t7DOXcemvOkwVobGzkrLPO4rTTTuO5557LeHzo0KF88Ytf5Ljjjsv6eJdJkybx9ttv84c//GHbtba2Nl555RUmTZoUKvbe0IR0H9X1f0KtVpK4hFlRVw5z585l/PjxTJo0iRkzZjBw4ECWLl1KS0sLU6ZMybh/woQJXHPNNVx88cWceuqpPPbYY9xyyy097rn22mtZunQpEydOZK+99uLPf/4zd9xxB+eccw4AV1xxBatXr942pNTe3s4jjzzC3Llzc8Y5evRoTj75ZM455xy+//3v069fP77+9a/ziU98oux7HIBgcqMa/xx++OEulXf3k2/4Md97yPf9+m/8mO895Hc/+UbcIUkFPffcc3GHEIlFixb5cccd5w0NDd7Y2Ohjxozx9vZ2d3e/6aabHPAPPvhg2/1XXXWVNzc3+4ABA3zcuHH+0ksvOeA/+clP3N39scce88mTJ/uwYcN8xx139H333de/9rWv+YYNG9zd/de//rWPHTvWhwwZ4jvuuKPvv//+/r3vfc+3bt2aN8733nvPzz33XG9sbPRddtnFP/OZz/iqVavyPqfQvxHQ5iHaWPOQXaukaWlp8ba2trjD6FPSl79CMBSlDXZ9x/PPP8+BBx4YdxiSR6F/IzNb7u4thV5Hcw4Smpa/ivQdSg4Smpa/ivQdSg4Smpa/ivQdSg4Smpa/ivQdWsoqoWn5q0jfoeQgRcm1Nl0VXkVqi5JDlUpSY6wKryK1R3MOVairMe5Y04mzvTGu5GEp3WmJq0jtUXKoQklrjLXEVaT2KDlUoaQ1xlriKrVk3rx5mBkffvhhrHFs3LiRmTNnctxxx9HQ0FCRMxy6U3KoQklrjLXEVSR669ev5/rrr2fAgAEcc8wxFX9/JYcqlLTGOK7yyyK1bNCgQaxevZqFCxdy2mmnVfz9lRyqUBIb42mjmlgyayx/nXsKS2aNVWKQwp65Ha4+GOYMCv77zO0Ve+vFixdz4oknsvPOO9PY2MiYMWNob2/Pef+sWbM45JBD2HnnnWlububss8/m7bff7nHPvffey+GHH87AgQMZPHgwRx11FI888si2x2+44QYOOuggGhoaGDJkCCeccALPPvts3jgrPZTUnZayVqm4auGLROKZ2+HXX4FNqXmyta8HvwOMPKOsb71o0SImTJjAiSeeyM0338zAgQNZsmQJHR0djBo1KutzVq5cyWWXXcZee+3FqlWr+MEPfsDYsWNZsWIFdXV1/OUvf+H000/noosuorW1lQ0bNrB8+XJWr14NBMno/PPP58orr2T06NG8//77LF26lLVr15b1s/aGkkNClWsfQ9jXTdI+CqlBD125PTF02dQZXC9zcpg9ezaHHnooCxcu3PbNfOLEiXmfc+ONN277ecuWLYwePZrm5maWLFnC8ccfT3t7O7vssgutra3b7ps8efK2n//4xz8ycuRIZs+eve3a1KlTo/pIZaFhpQQq1z6GsK+btH0UUoPWvlHc9YisW7eOxx9/nOnTpxc1ZLNgwQKOOeYYGhsb2WGHHWhubgbgpZdeAuCQQw5h7dq1TJ8+nfvvv59169b1eP5hhx1Ge3s7l1xyCYsXL2bjxo3RfagyUXJIoHLtYwj7uknbRyE1qLG5uOsRee+993B3hg0bFvo5TzzxBFOnTqW5uZmf//znLF26lGXLlgGwYcMGAEaMGME999zDK6+8wuTJkxkyZAhnnXUWq1atAmD8+PHcdNNNLF68eNtRoRdccEFGEkkSJYcEKtc+hlzP70i7nrR9FFKDxl0O9WlLr+sbgutlNHjwYPr168dbb70V+jl33303Q4cO5bbbbmPq1KkcffTR7Lnnnhn3nXLKKTz66KO8++673HDDDTz44INceOGF2x6fPn06y5cv55133qG1tZV58+bxne98J5LPVQ5KDglUrn0MuZ5v0GPIKGn7KKQGjTwDTv0xNO4NWPDfU39c9vmGgQMHctRRR/Gzn/2MsEckd3Z2Ul9f32MY6tZbb815f2NjI2eddRannXYazz33XMbjQ4cO5Ytf/CLHHXdc1seTQhPSCTTz5BFZz2ru7T6GmSeP4JLbniL9/xJOMJTUNeFcyvtrAluKNvKMsieDbObOncv48eOZNGkSM2bMYODAgSxdupSWlhamTJmScf+ECRO45ppruPjiizn11FN57LHHuOWWW3rcc+2117J06VImTpzIXnvtxZ///GfuuOMOzjnnHACuuOIKVq9evW1Iqb29nUceeYS5c+fmjXXBggWsW7eOp556CoA777wTgCOOOIJ99tknir+OnJQcEqhc5yZMG9XExbc9lfWxN9d09mjgBw2oZ8cd+rG2c1PB91dVVqkmxx9/PA888ADf+ta3+OxnP0v//v0ZNWoU06ZNy3r/5MmTueqqq/jJT37CT3/6U0aPHs1vfvMb9t9//233jBw5knvvvZdLL72U1atXM2zYMM477zyuvPJKIGjMr776an75y1/ywQcfsM8++zBnzhwuuuiivLF+6Utf4tVXX932+6c//WkAbrrpJs4999xe/k3kZ2G7VknT0tLibW1tcYdRdY6d+3DGHAPAoIZ6Ptq8NaO3EGZzXa7XbBrUwJJZY3sftCTG888/z4EHHhh3GJJHoX8jM1vu7i2FXkdzDn1MrtIbZpS8QkkT2CK1R8mhj8lVemPN+k1Z7w/TwGsCW6T2aM6hD8pWeqN14YtZh4bCNPDlmkAXkfio5yBA7yq9JrEQoIj0jnoOAvR+hZQKAYrUFiWHGhHFPoNyNPDa/1B73D3WUtKSW5SrTyuaHMzsRmAKsNLdD05d2w24DdgX+Btwhru/V8m4ql3YfQaVbqi1/6H21NfX09nZyYABA+IORbLo2s0dhUrPOcwD0mvjzgIecvePAQ+lfpcihCmUl6vS6jfnr+DYuQ+z36z7OHbuw5FWXlUBv9qz++6709HRwfr16yP9liq94+6sX7+ejo4Odt9990hes6I9B3dfbGb7pl3+JDAm9fPNwCLg6xULqgaE2WeQq6G+ddlr28ppRP3NXvsfas+uu+4KwJtvvsmmTdmXP0s86uvr2WOPPbb9G/VWEuYc9nD3twDc/S0zy5n2zGwGMANg+PDhFQov+fYa1FBwGWquBjn9u1/XN/sokkOYuKT67LrrrpE1QJJcVbWU1d2vc/cWd28ZOnRo3OEkxsyTR1Dfr+cEYX0/67EMtZgGuWNNZyTDTL1ZHisi8UpCz+EdMxuW6jUMA1aW642qZeVMSXGmLx5J+z3bRjUjs+fQpfu8BJQ2zFSuAoIiUn5J6DncC0xP/TwduKccb1ItR1+WEmfrwhfZtKVnM79pi/eY+M22Ue3so4dnfLNP19sJ5GmjmlgyayxXn3kYAJfc9lTBHsn89o6yTZKLSDiVXsr6C4LJ5yFm9gZwBTAXuN3MvgC8Bny6HO+db+VMkr7JlhJn2InfbPsYWvbZbds3+1y9iGzzBsUoZkmrlr+KJEOlVyt9JsdD48r93tWycqaUOHsz8ds9Yfz97N+yJcvyxLpebngqJuFVSxIXqXVJGFaqiGqpHFpKnFFN/GZLDPmuh1VMwquWJC5S6/pMcqiWlTOlxBlV4bumHAko1/Wwikl41ZLERWpdn0kO1VI5tNQ4uyZ+/zr3FJbMGlvS58qWmADWb9xcsSWt1ZLERWqdjgmtcvPbO5hz77Os6Qx2qw4eUM8Vpx5UctJLf70uYY8Mzfe6YZe0VsuSY5FqFPaYUCWHhPjm/BX84vHX2eJOnRmfOWpvvjvtkLzPmd/ewcw7nmbT1p7/hvV1Ruvph5bcoOpMaJHapTOkq8g356/glmWvbZv43eLOLcte45vzV+R9XuvCFzMSA2TucSiWJoVFJAk7pPu8Xzz+es7r+XoP+Rrr7o8VO0zT25pIGhYSqX5KDjGb395R8hLSXI1412MQ9ErCVF7t3qA3NtRTX2c9dl2HnRTWJjaR2qBhpRh1NaSF7sklW8E9COYcZp48gvntHT0SQ5dCZz2s6dzEpi3OwP51Ra/s0hkOIrVBPYcYZWtI0+X71t11LddqpWPnPpyzJEahsx4A1m/cwtVnHsa0UU3b6h0VGirSfIVIbVByiFGYBjO9dES28fynrjgp63Pz1UQKe9bDV29/mrZXV/Or5R2hhop0hoNIbdCwUoV1rzjaL2TNoq7Gu9iKrflqIoU962GLO7cuey30UJE2sYnUBvUcKih9sjZszSIn2Huw7qPNRRWly/f63e+fefIILrntqbxnO2STrcdRiTMctBpKpPyUHCoo19h+nRlb3WlsqGfdxs0ZZzNA/iGiXMNCTTmGeNJrJU0b1UTbq6uzTl7nk6vHka00eFS0GkqkMjSsVEG5GvGt7vx17ik8dcVJtJ5+aNGF7rp6FunDS8UM8Xx32iFcfeZhOYei0q/GNVSk1VAilaGeQ5lkG/oIM1nb9a17v1n3FfUtPts36GKHeLqupx8n2lBfx/8+vInfv7Aq9qEcrYYSqQwlhzLINfTxvw9v6rHqB3J/A8+VSAYPqGdA/x2yPpZt/qHYIZ6kn/us1VAilaFhpTLINfTx+xdWhS7HnWtI6IpTD2LJrLEZwzxdovgGHUX573LRaiiRylDPoQzyDX2E/SZf6Bt8Jb9BJ2l1UNJ7NiK1QsmhDKJquPMlkpknj8g6NxD1N+gkrg4q52ooEQloWKkMKjH0UamT7bQ6SKRvUnIog0o13F1zA1efeRgAl9z2VNYlrb2h1UEifZOGlcqkUkMf5R72qdTcRpLmNUREPYeqV+5hn0oMkRVbM0pEyk89h4Sb396RsyQ3hB/2yfbNHAqv+qnE6qB8CU69B5F4KDkk2Pz2Dmbe8XSPc6LfW7+JmXc+DQQNd5hhn2xDTzPveBqMbXWc8g1HlXuITPMaIsmjYaUEa134Yo/E0GXTFt82bBRm2CfbN/NNWz2jwF/Xt/XuZcWjnuDOJtf8hXY9i8RHySHB8n1z7noszMqoYr6Bd/UgKjn+r13PIsmjYaUEyzVk1PVYl0LDPvleJ12dWcXH/7XrWSR5lBwSbObJIzLmHADq66yob9XZdlPX97Mecw4QfFvPdaZ1ucf/tetZJFk0rJRg00Y10frpQxnUUL/t2uAB9bSefmjRlVbTh55aP33otrMjug9H5TpLQuP/In2Leg4RiWoTV7bXeeqKk3odX65v5tmuVaJmk4gkm5JDBKLapZyEInca/xcRUHKIRFSbuJKyGUzj/yKiOYcIRLWJS5vBRCQp1HOIQL5dysXMRegITBFJCvUcIpBrE9eJBwwtakOZNoOJSFIoOUQg1y7l37+wqqiKqemvM6ihnp3q+xV9TkOly19Ua0wikpu5Z9buqQYtLS3e1tYWdxh57TfrPrL97Rrw17mn5H1u+solCHoRhQ4NKvV5xSh22W4lYhKRcMxsubu3FLpPPYcy6k1BuVLPaSj3+Q7Zzl6YeefTHPbt+3P2CnTUqEj1UXIoo97MIZS6cqncK56yVnjd4qzp3JRzXkWrsESqj5JDGfXmLOlSex3lLn8dpkFP7xWoJLdI9UlMcjCzS8zsWTP7k5n9wsx2ijumKEwb1cSSWWP569xTWDJrbOgx9lJ7HeVe8RS2Qe+eRLQKS6T6JCI5mFkT8BWgxd0PBuqAf4o3qniV2usI87zerBzK1tBnk15SvNQelIjEIxGrlVLJYRlwKPA+MB/4sbvfn+s51bBaKYmiWDnUfbXSoAH1fLhhc4+y4lqJJJJcYVcrJWKHtLt3mNn3gdeATuD+bInBzGYAMwCGDx9e2SBrRK6VQ1+9ffu51IWk116KqiKtiCRHUnoOg4FfAWcCa4A7gDvd/ZZcz1HPoTT7zrov52P6xi9S+6ptn8N44K/uvsrdNwF3AcfEHFPNmd/egeV5vDd7D7QDWqS2JCU5vAYcbWYDzMyAccDzMcdUc1oXvph1x3Z3HWs6i27gs22My1dDSkSSLxHJwd0fB+4EngRWEMR1XaxB1aCwm86KbeC1A1qk9iQiOQC4+xXufoC7H+zu/+zuH8UdU60pdtNZ2AZeO6BFak9ikoOUX7Y9CvnmICBcA68d0CK1JxFLWSVTOZaHpp8PPWhAPe6wtnMT/czYkmXlWpgGfubJI7LunQizA1rLYEWSSckhgdI3qnWN/0P+fQhhGtquPQrp75EtMYRt4NOTTthGvtTPKSLlp+SQQPkmeHM1msU2tNneA6DOjK3uRX+LT98YF0Ypn1NEKkPJIYFyjfN3rOlkfntH6MY+X0Ob6z22uhc8iCgqmsgWSS5NSCdQvnH+XMtLi21okzCJnIQYRCQ7JYcEylf5NNfy0mIb2iSU0U5CDCKSnZJDAnWVuM4lW2+g2IY2CWW0kxCDiGSXiMJ7pegLhfeOnfswHVkSQdOgBpbMGptxXctCRaSQqirZLdkVu3+glBVDIiLZKDkkWKn7B6KgXohI36bkkHBx9Aa0OU1ENCEtGVRlVUSUHCSDNqeJSN7kYGajzOwGM/u1mc01s49luecwM3ulfCFKpXSd5pZr/Zo2p4n0HTmTg5m1AEuBsQSVnacDz5jZhWm37gjsU7YIpSK6n+aWjTanifQt+Sak/z/gQeA0d99kZv2BbwJXm9n+wFe8WjdJCNBzRVKukt0Q7KvQaiWRviVfcjgcONvdNwG4+0bgcjN7HPgl0GRmZ1UgRimDMCW7IegyZttwJyK1rdCEdMZBYe5+H8FQ07EEPYvdyhCXlFmukt3pNM8g0jflSw7PAeOyPeDuTwDHAXsBt5YhLimzMCuPNM8g0nflSw73AV8ws8ZsD7r7SwS9h9fKEZiUV64eQZ2ZiuCJSN45h1bgP4CcXzHd/S0zOxrYI+rApLxy1W1SQhARyJMc3H0rsK7QC7j7BuDVKIOS8ouzbpOIJJ9qK/VhquIqIrmofIaIiGRQchARkQxKDiIikqGo5GCBvc3sGDMbWK6gREQkXqGTg5ldAHQQrEx6FBiRun6XmV1cnvBERCQOoZKDmc0Efgj8lO1VWrssAs6MPDIREYlN2KWsXwYud/d/N7O6tMdeBPaPNiwREYlT2GGlPYHlOR7bCuwUTTgiIpIEYZPDy8AJOR47nqBIn4iI1Iiww0rXAP/XzDYCd6au7W5mXwAuBc4rR3AiIhKPUMnB3a83s8HA5cC3U5d/C6wH5rj7f5cpPhERiUHo2kru3mpm/wWMBoYAq4Gl7r62XMGJiEg8CiYHM9sJuBf4N3dfBNxf7qBERCReBSekUyW5jwDSl7CKiEiNCrta6V5gWjkDERGR5Ag757AQaDWzYQQT0e8A3v0Gd/9txLGJiEhMwiaHW1L//VTqTzpHw04iIjUjbHLYr6xRAGY2CLgeOJgg2Xze3ZeW+31FRCRT2H0OlTgj+kfA79z9dDPrDwyowHuKiEgWoZKDmX280D3uXnIJDTPblaAMx7mp19oIbCz19UREpHfCDiv9ibQJ6Cx6M+fwv4BVwE1mdihBkb+L3H1dL15TRERKFDY5nJjl2m7ASak/F0UQxz8CF7r742b2I2AW8K3uN5nZDGAGwPDhw3v5liIikou5F+oQFHgBs+8Cw939nF68xp7AMnffN/X7ccAsdz8l13NaWlq8ra2t1LcUEemTzGy5u7cUuq+oM6Rz+D3wyd68gLu/DbxuZiNSl8ahMuAiIrEJXXgvj1OANRG8zoXAramVSq8An4vgNUVEpARhVyvdnuVyf+AA4GPAZb0NxN2fAgp2dUREpPzC9hx2J3O10gbgUeBSlc4QEaktYTfBjSlzHCIikiChJqTN7EYzy1pCw8z2MbMbow1LRETiFHa10rnA0ByPDQGmRxKNiIgkQjFLWXNtiDiYYHeziIjUiJxzDmZ2Edt3Pjsw38ycMiMEAAAWcklEQVQ+SrttJ2APYF5ZohMRkVjkm5B+DvgVYMClBJvd3kq7ZyPwApBtqauIiFSpnMnB3R8AHgAwsw+A6929o1KBiYhIfMIuZf12uQMREZHkCF0+w8xGA18A9ieYa+jB3Y+MMC4REYlR2H0OE4DFQDPwCYLVSR8ChwJ/R3Deg4iI1IiwS1mvJDjGs6uE9rfcfSxBL2ITsCj60EREJC5hk8PHgQXAVoJlrQNh29nSc4BvlCM4ERGJR9jksAHo58HJQG8Bf9/tsfcJhptERKRGhJ2QfhoYQbC09SFgtpl1EOxzuBJYUZ7wREQkDmF7DtewvXzGZcA6YCHBxrjdgS9HH5qIiMQl7D6H33b7ucPMDgf+AWgAXnD3jWWKT0REYlD0MaFmZsAw4K/uvjn6kEREJG6hq7Ka2WQze5xgcvp1YGTq+k/N7LNlik9ERGIQdhPcOcC9BEX2ZhAU4+vyEsHOaRERqRFhew7fAFrdfTpwS9pjzxLsgxARkRoRNjnsQ6pCaxYbgF2jCUdERJIgbHJ4HRiV47EW4OVowhERkSQImxxuAK5ITTw3pK6ZmY0Dvgb8tBzBiYhIPMIuZb0K2Bu4GdiSuvYYUAdc6+4/LkNsIiISk7Cb4Bz4spn9EBgHDAFWAw+7+0tljE9ERGJQ1CY4d/8L8JcyxSIiIgmRc87BzO43sxFp18aa2cDyhyUiInHKNyE9Hmjs+sXM6giWs47I+QwREakJoctnpFjhW0REpNoVmxxERKQPKJQcPOQ1ERGpIYVWKy00s/Sy3A9luYa77x5dWCIiEqd8yeHbFYtCREQSJWdycHclBxGRPkoT0iIikkHJQUREMig5iIhIBiUHERHJoOQgIiIZlBxERCSDkoOIVNYzt8PVB8OcQcF/n7k97ogki6LOcyi3VOXXNqDD3afEHY+IROyZ2+HXX4FNncHva18PfgcYeUZ8cUmGpPUcLgKejzsIESmTh67cnhi6bOoMrkuiJCY5mFkzcApwfdyxiEiZrH2juOsSm8QkB+Aa4GvA1lw3mNkMM2szs7ZVq1ZVLjIRiUZjc3HXJTaJSA5mNgVY6e7L893n7te5e4u7twwdOrRC0YlIZMZdDvUNPa/VNwTXy0mT4EVLyoT0scBUM5sM7ATsama3uPtnY45LRPJ55vZgvmDtG8G3/3GX559Y7nqsmOdEEaMmwYtm7sk6u8fMxgD/Wmi1UktLi7e1tVUmKBHJlN7oQtALOPXHyWp0rz44SAjpGveGS/5U+XhiZmbL3b2l0H2JGFYSkSpULSuPNAleksQlB3dfpD0OIlWgWhpdTYKXJHHJQUSqRLU0unFNglc5JQcRKU21NLojzwjmQRr3Biz4b9LmRRIoKauVRKTaxLHyqFQjz0hmXAmm5CAipVOjW7M0rCQifY82xRWknoOI1Lb0jXofOwme/m9tiitAPQcRqV1dG/XWvg548N+2G6tjf0bMlBxEpHZl26hHjqoQSdufETMlBxHJVCtj8sU0+EnbnxEzJQcR6SnbUMyvv1KdCSJng289f03i/oyYKTmISE/VUjMpjFwb9Vo+r01xBWi1koj0VA01k8KWCq+mjXoJo+QgIj01NucocZ2QMfliz2fQRr2SaFhJRHpKes2kWhr2SjAlBxHpKemF6qIc9qqVVVlloGElEckU51BMvvmEZ24H6we+JfN5xQ576fjQvJQcRCQ58jXYEPycLTGUMuyVb3gqCcmh2PO5I6bkICLJUWg+IWO3M2B1pQ17JXlVVgJ6NZpzEJHkyNdg53rMt5bWYJbzJLvezmUkYNJdyUFEeifKSd18DXbUjXm5VmVFscM8Ab0aJQcRKV3UpTbyNdhRN+blWpUVxbf+BJzPrTkHESld1JO6YXY0RzFJmz7Z+6nrohvLj+Jb/7jLe845QMX3mig5iEjpyjH8kW8ZbRRLbMNM9vZmpVAUO8wTUPZDyUFESpf0Uhvpnrkd7j4/czls995Ob5NHVN/6Yy77oTkHESld0kttdNfV6GfbJwHbezuF5gwKzbMkfYd5SOo5iEjpEjD8EVrWU+G66ertFBoqCzPPUgPF/pQcRKR3ktQQ5hvuyTcP0r23k2uoDA+W6mZ9rMDrVyENK4lIbXjmdph/Qc/hnvkXbB/uyXcq3A4NcNeMoPH/2ElQ1z/7rWtfJ+MUuS5Rz7PEXBRQyUFEqkuuRnPB12Hrpp73bt0UXIfs8yP96qGuHjpXsy2htP8ctmzOE4BT9mNGE3BUq5KDiFSPfI1m5+rsz+m6nm2ieMddYMvGnvdv2QhsLRCIl3fCOQHlMzTnICKVEUWV0VyNZlfvoJD0+ZE5g4p7/y6Ne8MlfyrtuWEkoHyGkoOIlNcztweNd/dv9qVWGc3VOObqNQA07Jb7sZyTz3lUYqluAvaPaFhJRMqnaxgoW+NdyjBJsY1jXX+YdFXux7PNQ9T1D+YiekjNMUQ1hFRosjkB+0fUcxCR8im0t6DYYZJsu4/z+eR/5m/Ic+3TyHYtqjmFMDuwE7B/xNy9Ym8WpZaWFm9ra4s7DBHJZ84ggtU9OZQydp8+d7FxXfaeSbnnBUqVa69EheI1s+Xu3lLoPvUcRKR88o3plzpMkj6pnP5NvDevna4cR3UmYLI5DM05iEj5ZBs7B8C2zzn0du1+uWoZlWuvQQLOaghDw0oiUl7dv303DIaPPsjcrNawWzBxnJQyHFC+4Z9cPZ0KFecLO6yknoNIX1TJ0gwjzwga0zlroP/AzMQAwZxBhXcAF1SuGkpVUrVVcw4ifU2Y1TLlkq9h7c0JclF75naC5atZRlaiGP5JUrHCHNRzEOlr4izNUKhhXftG8b2acvSCHrqS7KusLJlnVZRBIpKDme1tZr83s+fN7FkzuyjumERqVpyrZcZdTs6qphDMSRQzCVyoEmupcv5deOK/8UclKcNKm4GvuvuTZrYLsNzMHnD35+IOTKTmxFmaYeQZcNd5+e8pdJBOd4UqsUZ+DvTe239On2gH6Hwv2QceFSERPQd3f8vdn0z9/AHwPNAUb1QiNSru0gzdG9juGnYLGtdsiq2p1DXBXeoy1EJ/R+nLXDtX9yz7nbTJ9RIkIjl0Z2b7AqOAx+ONRKRGxbFapvu8wMZ1mYfp1DcES1mj3APQm3mVQn9HhcqCVLi8djkkZVgJADPbGfgVcLG7v5/l8RnADIDhw4dXODqRGlLJ1TLpq6PSv+2n73EoZrdzw275K7KmK2ZeJd/fUZjXSdiO52IlpudgZvUEieFWd78r2z3ufp27t7h7y9ChQysboIiUptC37M3dHiu2VzPpqsxeSF3/3GW6o5pXCfM6CdvxXKxE9BzMzIAbgOfd/YdxxyMiESr0DTp9wrmYXk2+qqq9rbeUr65SoeqwFS6vXQ6JSA7AscA/AyvM7KnUtcvc/bcxxiQiUQhzoM7a14PGuJShrnzJJN9qpXyNf6GNgulJqQZXK6m2koiUV7ZaQtlUsL5QwfpGMZfVLifVVhKRZOgxj5BHqSt8StkhXWiXeJWU1S4nJQcRiUa+Rnpb8b21+c90LrbxLbWsdqHGv0rKapeTkoOI9F4xjXSujW5QfONbap2oQo1/3BsFE0DJQUR6r5hGOmcCKKGoXanDP9ka/7r+wQa9OYOCuA89K/FltcspKauVRKSaFdNIZ10GatDy+eIb31LrRGVbbfTRB9s31K19HZ7+7/wJoRxHiCaIeg4iEk6+OYVixuizbXT71HUwpYQtTvmGfwpNVBc6hCjf8FS5jhBNEPUcRPqSUr/tFlr3n603kG+MPqryHWE3wRU60KjY4al8w2g10ntQchDpK3pzAlyhxjBXI12JhjJborn64OzxLvh69hjzDU9lS6h9YKmrNsGJ9BWlbux65vY8ZzBYMCwTVqXG6ec0hruva+MbZO/5HHpWMPeQfn2HhuwF/6wOTvuvRPcewm6CU89BpK8o5dtuV28jl2KWnobtuZSSQLIdvBNGV++nKzmmv2+uHtMODUGSSH/Mt1TuPO4y04S0SF9RysaufBVVi133H2a5aykTvdkO3ilGV3LsPkF9yZ+C33MeMvRe0OOwuszHauAsB1ByEOk7StnYla9XUey6/zA9l1I2tRUqCV5IvuSYL6GOPCPoKWRTqNBgFVByEOkrSjkBLmfjuHdpexIKXS9l6Ks3k8CFkmOhhJqt55DvehVRchDpS7INneQTZRmJMK9VytBXb+odFUqOhRJqrp5DruvFKqWoYEQ0IS0iuUW5RDXMaxW7X6LrOXfNAIpceRm295NvT0bj3rlXgPVWb5YeR0BLWUUkWUpZrfSbS6HthvDvUdcfPvmfvW9kC50L0RtlOlNCS1lFpLzKtWehlN3TU34Iz94dfqVS/53Lu0M7iteOeaOdkoOIFC/mIY+sJl0V7sQ5yF82vFhRlQJJV2pRwYhoQlokTjFOOPZKqecolFO2yeNcBwtVw6E9MZ8poZ6DSFyS+O07rKTWFkr/Fp9rTqAaDu2Js14VSg4i8anmyp4xD3lklW8OpFrPXSjXkFUISg4icUnqt+8wSllyWk6FemHVkgwSRHMOInGp5kPsS9ltXU5JnAOpcuo5iMQlad++ixXmG3mlSnRXcy8sodRzEIlL0r59Ry3qozSjOqa00qp0RZp6DiJxquXx8Cgn3KM+prRSqnhFmnoOIlIeUQ71FJpTSGovrIrnQtRzEJHyiHK5a5hEk8ReWBXPhajnICLlEeUO3yTPKeRTrXGj5CAi5RLlUE/MpSSK1jUJvfZ1wHo+luS4u9GwkoiUT1RDPdW00zmjZIcTJAgPEmRS406j5CAi1SGJcwrZZD3T2nt9DkOlaVhJRCRKVTwJ3Z2Sg4gIRLdZrYonobtTchARiXI3d7VNnueg5CAiEuVmtaRuyCuSJqRFRKKeJ6iWyfM81HMQEamReYIoKTmIiNTIPEGUlBxERGpkniBKmnMQEYHqmCeo1OFJKDmIiFSHCp8NkZjkYGYTgR8BdcD17j435pBEROKT3kvYuC66w5NCSERyMLM64D+BCcAbwBNmdq+7PxdvZCIiMcjWS8ilTGU5kjIhfSTwsru/4u4bgV8Cn4w5JhGReGQt3pdDmZbbJiU5NAHdU+MbqWsiIn1P2N5AGZfbJiU5WJZrnnGT2QwzazOztlWrVlUgLBGRGOTqDTTsVrHltomYcyDoKezd7fdm4M30m9z9OuA6gJaWlozkISJSE8ZdnnZgEEEvYdJVFVtum5SewxPAx8xsPzPrD/wTcG/MMYmIxCMBm/IS0XNw981m9n+AhQRLWW9092djDktEJD4xb8pLRHIAcPffAr+NOw4REUnOsJKIiCSIkoOIiGRQchARkQxKDiIikkHJQUREMig5iIhIBiUHERHJoOQgIiIZlBxERCSDuVdn/TozWwW8GnccRRgC/E/cQURAnyN5auWz6HNUxj7uPrTQTVWbHKqNmbW5e0vccfSWPkfy1Mpn0edIFg0riYhIBiUHERHJoORQOdfFHUBE9DmSp1Y+iz5HgmjOQUREMqjnICIiGZQcKszM/tXM3MyGxB1Lqcys1cxeMLNnzOxuMxsUd0zFMLOJZvaimb1sZrPijqcUZra3mf3ezJ43s2fN7KK4Y+oNM6szs3Yz+03csfSGmQ0ysztT//943sxGxx1TqZQcKsjM9gYmAK/FHUsvPQAc7O4jgZeA2THHE5qZ1QH/CUwCPg58xsw+Hm9UJdkMfNXdDwSOBr5cpZ+jy0XA83EHEYEfAb9z9wOAQ6niz6TkUFlXA18Dqnqix93vd/fNqV+XAc1xxlOkI4GX3f0Vd98I/BL4ZMwxFc3d33L3J1M/f0DQCDXFG1VpzKwZOAW4Pu5YesPMdgWOB24AcPeN7r4m3qhKp+RQIWY2Fehw96fjjiVinwcWxB1EEZqA17v9/gZV2qh2MbN9gVHA4/FGUrJrCL40bY07kF76X8Aq4KbUENn1ZjYw7qBKtUPcAdQSM3sQ2DPLQ98ALgNOqmxEpcv3Wdz9ntQ93yAY3ri1krH1kmW5VrU9OTPbGfgVcLG7vx93PMUysynASndfbmZj4o6nl3YA/hG40N0fN7MfAbOAb8UbVmmUHCLk7uOzXTezQ4D9gKfNDIJhmCfN7Eh3f7uCIYaW67N0MbPpwBRgnFfXeug3gL27/d4MvBlTLL1iZvUEieFWd78r7nhKdCww1cwmAzsBu5rZLe7+2ZjjKsUbwBvu3tWDu5MgOVQl7XOIgZn9DWhx9yQX58rJzCYCPwROcPdVccdTDDPbgWASfRzQATwBnOXuz8YaWJEs+JZxM7Da3S+OO54opHoO/+ruU+KOpVRm9ijwL+7+opnNAQa6+8yYwyqJeg5Siv8AdgQeSPWElrn7+fGGFI67bzaz/wMsBOqAG6stMaQcC/wzsMLMnkpdu8zdfxtjTAIXAreaWX/gFeBzMcdTMvUcREQkg1YriYhIBiUHERHJoOQgIiIZlBxERCSDkoOIiGRQcpCaYGZzUtVu0/88WIb3OsnMErO3wMwmmNkvzOxvqc88J+6YpPppn4PUkrXAxCzXonYScDpBTaAkmAiMBB4C/inmWKRGKDlILdns7sviDqJYZtbg7p29eImZ7v7V1GtVXYVZSSYNK0mfYWb9zGxW6pCfj8zspVSNqO73nGJmD5jZSjN738yWmdlJ3R6fA3wV2Kfb0NW81GOLzOzOtNcbk7rn4NTv+6Z+P9vMfmZma4Bfd7v/X1KH93xkZq+a2dcKfS53r/ZqppJA6jlITUnVTupuS7fCgD8BpgNXAk8SHLx0o5m96+5dJ5DtR9BYf5+ghPQkYIGZHe/uSwjOHPgYMBY4LfWcUupLfR+4C/g0sCUV+0zg34B/BxYBhwPfMbP17v4fJbyHSMmUHKSW/B2wKe3aBOBBM/sH4EvA59z95tRjD5rZMOAK4DcA3RthM+sH/B44CPgCsMTd3zCzt4CPejmEtczdv9ztvXZNxfFdd/926vIDZjYA+KaZ/T9339KL9xMpioaVpJasBY5I+9NVPnkcQU/gbjPboesPwSTuYanjQzGzZjO72cw6CM6q2EQwAb1/xLHel/b7aGAgcEdafA8De1Bdp+1JDVDPQWrJZndvy/HYEIIqrLlWLw0zszeBe4FdgMuBl4F1BMNQu0cc6ztZ4gPIVSF2b+DViGMQyUnJQfqK1QQ9gWPJfhzlSuAfCI7bnOTuv+t6wMwaQr7HBqB/2rXdctybXg55deq/U8hMHAAvhoxBJBJKDtJXPEzQc2h09wey3dAtCXzU7do+BAnlmW63biQ4tSzdGwQHzHc3IWR8S4FOYC93Tx9yEqk4JQfpE1Inc/0X8Esz+3egjaCBPwjY393/BXiBoIH/gZl9i2B46dsEJ8Z19wKwh5mdC/wJ+B93/xtwN/AFM7uaYE7hRODkkPGtSS2T/VEqIS0mmBPcHzjR3U/L9dzU/Uekfu0PfNzMTgfWufuCMO8vksHd9Ud/qv4PMIegkc53jwEXE4zrf0SwBPUR4Jxu9xwB/JHgW/yfgXOBeUBbt3t2Am4iGIpyYF63x2YDrwMfALcAU1P3HJx6fN/U71NyxPhZYHnq/d8jmFC/tMDnOjf1mul//hb3v4v+VO8fnQQnIiIZtJRVREQyKDmIiEgGJQcREcmg5CAiIhmUHEREJIOSg4iIZFByEBGRDEoOIiKSQclBREQy/P/YTordRjg8LQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#plot the blob dataset\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(6,6))\n",
    "\n",
    "fontsize = 15\n",
    "\n",
    "#plot the points of the first class:\n",
    "mask = y_blob_train == 0\n",
    "\n",
    "ax.plot(X_blob_train[mask][:,0], X_blob_train[mask][:,1], 'o', label='class 0') #plot first feature on x-axis and second feature on y-axis\n",
    "\n",
    "\n",
    "#plot the points of the second class:\n",
    "mask = y_blob_train == 1\n",
    "\n",
    "ax.plot(X_blob_train[mask][:,0], X_blob_train[mask][:,1], 'o', label='class 1') #plot first feature on x-axis and second feature on y-axis\n",
    "\n",
    "\n",
    "\n",
    "ax.legend(fontsize=fontsize)\n",
    "\n",
    "ax.set_xlabel('Feature 1', fontsize=fontsize)\n",
    "ax.set_ylabel('Feature 2', fontsize=fontsize)\n",
    "\n",
    "ax.set_title('Blobs', fontsize=fontsize);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GENERATE SEMI-CIRCLES"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#generate two semi-circles, one for each class, of 2D data\n",
    "#(we don't get to choose the data dimension here, it's always 2D)\n",
    "X_moon, y_moon = make_moons(n_samples=N, noise=.1)\n",
    "\n",
    "#generate training and test set\n",
    "X_moon_train, X_moon_test, y_moon_train, y_moon_test = train_test_split(X_moon, y_moon, test_size=test_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZkAAAGKCAYAAADe5zz7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XucVPV9//HXR0SzoC4oqLAralpvUVCajYpGxTuQiGjVeMlPTE2Jzc2Y1gbSVAk2FUMSbdK00XiPVkWrBhMJXgiSIhqXYEC8S2pkMUJFMOKiCJ/fH2cWh9k5M2d2z5lzZub9fDz2sTvnnJn57rDMZ77f7+f7+Zq7IyIikoRt0m6AiIjULwUZERFJjIKMiIgkRkFGREQSoyAjIiKJUZAREZHEKMiI9IKZTTUzN7OXQs6/nDs/tcpNE8kEBRmR3tsA7G1mbfkHzewTwJ658yINSUFGpPfWA3OBswuOn507vr7qLRLJCAUZkXjcCZxlZgaQ+35W7vhWzOwsM1tqZu+Z2Wtm9h0z27bgmkPM7FEze9fM3jKz281st7zze+WG4c4ys2vNbJ2ZrTCzb5vZNnnXtZrZTDNbZWadZvaKmV2R2KsgUkBBRiQe9wK7AZ/M3T4KGAzcl3+RmZ0E3AX8DjgV+BHwD8C/510zGJgH9APOBb4CHAM8bGbbFTzvd4F3gDOA24DLcj93uRXYA5gEjAW+A2zfm19UpBLblr9ERMpx97Vm9iuCIbLf5L7/Knc8/9JpwDx3n5i7/avc+SvN7F/cfQXw97lzJ7v72wBm9iLwJPDXwB15jzff3buuf9jMxgCnAzNzxw4FznH3B3K358XyC4tEpJ6MSHzuBM4ws+0JehNbDZWZWR/gr4C7C+53F8H/xVG524cCD3UFGAB3/y3wv3zYU+ryUMHtZ4HWvNtPEwSwC8xsWKW/kEhvKciIxGcWsAPBkFR/4IGC84OAvsAbBce7bu+c+z6kyDVd1+1ccGxtwe33gY/k3f4M0A5cDbxqZk+b2fGlfw2R+CjIiMTE3dcDvwAuAR7I3c73f8BGYNeC410T+mty318vck3XdWuKHC/Vpg53vwDYhaCn9CdglpntUsnjiPSUgoxIvP6ToAfzk8IT7r4JWAScWXDqLGAzsDB3+0ngZDPbseuC3JqbvYD/6Umj3H2zuz8BfJsgoWDPnjyOSKU08S8SI3efR+nJ9cuBOWZ2E8GczXDgCuCnuUl/gB8Af5e77iqCIbjpwFLgv6O2xcyagTkEGWYvEmSV/T1Bb+a5yL+USC8oyIhUkbs/ZGZnA98CzgNWAd8nCD5d16w2s2Nzx+8gmGd5ELjE3d+v4Ok2EASmiwnSmN8FngBOcvfOGH4dkbJM2y+LiEhSNCcjIiKJUZAREZHEKMiIiEhiFGRERCQxCjIiIpKYhk9hHjRokO+1115pN0NEpKYsWrTo/9x9cLnrGj7I7LXXXrS3t6fdDBGRmmJmr0a5TsNlIiKSGAUZERFJjIKMiIgkRkFGREQSoyAjIiKJafjsMhGpvrfffptVq1axcePGtJsiBfr27cuuu+7KTjvtFMvjKciISFW9/fbbvPHGG7S0tNDU1ISZpd0kyXF3Ojs76ejoAIgl0Gi4TESqatWqVbS0tNCvXz8FmIwxM/r160dLSwurVq2K5TEVZESkqjZu3EhTU1PazZASmpqaYhvK1HCZpOr+xR3MmPMCK9d2MnRAE5eevB8TRrak3SxJmHow2Rbnv4+CjKTm/sUdTLl3KZ0bNwHQsbaTKfcuBVCgEakTGi6T1MyY88KWANOlc+MmZsx5IaUWiUjcFGQkNSvXdlZ0XKRW3HzzzZgZ77zzTtpNYd26dXzuc59j4MCBNDc3c9555/Hmm29W7fkVZCQ1QwcUn/wNOy4ilfvMZz7DvHnzuP7667n55pt56qmnmDBhQtWeX3MykppLT95vqzkZgKa+fbj05P1SbJVI/Vi4cCFz5szhscce4+ijjwagpaWFww47jEceeYQTTjgh8TaoJyOpmTCyhStPH07LgCYMaBnQxJWnD9ekv0R2/+IOjpw+l70n/5Ijp8/l/sUdVXvu+fPnc+yxx7LDDjvQ3NzM6NGjWbx4cej1kydPZvjw4eywww60trZy3nnn8ac//Wmra2bNmsXHP/5x+vfvz8CBAznssMN47LHHtpy/4YYbOPDAA2lqamLQoEEcc8wxLFu2LPQ5Z8+ezW677bYlwAAceuih7L333syePbsXv3106slIqiaMbFFQkR5JMztx3rx5nHjiiRx77LHccsst9O/fnwULFtDR0cHIkSOL3mfVqlV885vfZOjQoaxevZrvf//7HHfccSxdupQ+ffrwyiuvcMYZZ3DxxRczY8YMNmzYwKJFi1izZg0QBLWLLrqIadOmMWrUKN5++20WLlzIunXrQtv5/PPPs//++3c7fsABB/D888/H82KUoSAjIjWpVHZi0kFmypQpHHzwwcyZM2fLmpIxY8aUvM+NN9645edNmzYxatQoWltbWbBgAUcffTSLFy9mxx13ZMaMGVuuGzdu3Jaff/vb3zJixAimTJmy5dj48eNLPudbb73FgAEDuh0fOHAgy5cvL/1LxkTDZSJSk9LKTly/fj1PPvkkEydOrGjR4uzZszniiCNobm5m2223pbW1FYAXX3wRgOHDh7Nu3TomTpzIQw89xPr167e6/yGHHMLixYu55JJLmD9/Pu+//36k5y3WRnev2oLYzAUZM7vRzFaZ2TMh588zsyW5r8fN7OC8c/9rZkvN7Gkza69eq0Wk2tLKTnzrrbdwd4YMGRL5Pk899RTjx4+ntbWVn/3sZyxcuJAnnngCgA0bNgCw33778fOf/5zly5czbtw4Bg0axLnnnsvq1asBOOGEE7jpppuYP38+o0ePZtCgQXzxi1/sFozyDRw4kLVr13Y7vnbt2qI9nCRkLsgANwOl+p1/AI5x9xHAFcB1BeePdfdD3L0tofaJSAZcevJ+NPXts9WxamQnDhw4kG222YbXX3898n3uu+8+Bg8ezF133cX48eM5/PDD2X333btd96lPfYrf/OY3vPnmm9xwww088sgjfOUrX9lyfuLEiSxatIg33niDGTNmcPPNN3PFFVeEPu/+++9fdO4lbK4mCZkLMu4+H1hT4vzj7v5W7uYTQGtVGiYimZJWdmL//v057LDDuPXWW3H3SPfp7Oykb9++Ww1R3X777aHXNzc3c+6553Laaafx7LPPdjs/ePBgvvCFL3DUUUcVPd9l7Nix/OlPf+J//ud/thxrb29n+fLljB07NlLbe6vWJ/4vBPLz8Bx4yMwcuNbdC3s5IlJH0spOnD59OieccAJjx45l0qRJ9O/fn4ULF9LW1sanP/3pbtefeOKJXHPNNXzta1/jlFNO4fHHH+e2227b6pprr72WhQsXMmbMGIYOHcpLL73E3Xffzfnnnw/A5Zdfzpo1a7YMlS1evJjHHnuM6dOnh7Zz1KhRnHzyyZx//vl873vfY5tttuEb3/gGn/zkJ6uyRgYIJoCy9gXsBTxT5ppjgeeAXfKODc193xX4PXB0yH0nAe1A+7Bhw1xEqufZZ59NuwmxmDdvnh911FHe1NTkzc3NPnr0aF+8eLG7u990000O+J///Oct11911VXe2trq/fr18+OPP95ffPFFB/xHP/qRu7s//vjjPm7cOB8yZIhvv/32vtdee/k//uM/+oYNG9zd/YEHHvDjjjvOBw0a5Ntvv73vu+++fuWVV/rmzZtLtvOtt97yCy64wJubm33HHXf0c845x1evXl329yv37wS0e4T3c/OI3b1qMrO9gF+4+0Eh50cA9wFj3f3FkGumAu+4+/dKPVdbW5u3tytHQKRannvuOQ444IC0myFllPt3MrNFHmHuO3NzMuWY2TDgXuD/5QcYM+tvZjt2/QycBBTNUBMRkerI3JyMmd0BjAYGmdkK4HKgL4C7/wS4DNgF+I/cJNoHuWi6G3Bf7ti2wH+5+6+q/guIiMgWmQsy7n5OmfOfBz5f5Phy4ODu9xARkbTU3HCZiIjUDgUZERFJjIKMiIgkRkFGREQSoyAjIiKJUZAREZHEKMiIiMTs5ptvxsx45513Um3H+++/z6WXXspRRx1FU1NT1faQyacgIyJSp959912uv/56+vXrxxFHHJFKGxRkRETq1IABA1izZg1z5szhtNNOS6UNCjIiUruWzISrD4KpA4LvS2ZW7annz5/Pscceyw477EBzczOjR49m8eLFoddPnjyZ4cOHs8MOO9Da2sp5553Hn/70p62umTVrFh//+Mfp378/AwcO5LDDDuOxxx7bcv6GG27gwAMPpKmpiUGDBnHMMcewbNmyku1MY4gsX+bKyoiIRLJkJjzwVdjYGdxe91pwG2DEWYk+9bx58zjxxBM59thjueWWW+jfvz8LFiygo6ODkSNHFr3PqlWr+OY3v8nQoUNZvXo13//+9znuuONYunQpffr04ZVXXuGMM87g4osvZsaMGWzYsIFFixaxZk2wh+P8+fO56KKLmDZtGqNGjeLtt99m4cKFrFu3LtHftbcUZESkNj067cMA02VjZ3A84SAzZcoUDj74YObMmbOlpzBmTKld4+HGG2/c8vOmTZsYNWoUra2tLFiwgKOPPprFixez4447MmPGjC3XjRs3bsvPv/3tbxkxYgRTpkzZcmz8+PFx/UqJ0XCZiNSmdSsqOx6T9evX8+STTzJx4sSKhqJmz57NEUccQXNzM9tuuy2trcHO8S++GOxYMnz4cNatW8fEiRN56KGHWL9+/Vb3P+SQQ1i8eDGXXHIJ8+fP5/3334/vl0qQgoxkwv2LOzhy+lz2nvxLjpw+l/sXd6TdJMm65tbKjsfkrbfewt0ZMmRI5Ps89dRTjB8/ntbWVn72s5+xcOFCnnjiCQA2bNgAwH777cfPf/5zli9fzrhx4xg0aBDnnnsuq1evBuCEE07gpptuYv78+Vu2YP7iF7/YLRhljYKMpO7+xR1MuXcpHWs7caBjbSdT7l0aW6BRAKtTx18GfZu2Pta3KTieoIEDB7LNNtvw+uuvR77Pfffdx+DBg7nrrrsYP348hx9+OLvvvnu36z71qU/xm9/8hjfffJMbbriBRx55hK985Stbzk+cOJFFixbxxhtvMGPGDG6++WauuOKKWH6vpCjISOpmzHmBzo2btjrWuXETM+a80OvHTjqASYpGnAWn/BCa9wAs+H7KDxOfj+nfvz+HHXYYt956K1G3r+/s7KRv375bDa/dfvvtodc3Nzdz7rnnctppp/Hss892Oz948GC+8IUvcNRRRxU9nyWa+JfUrVzbWdHxSpQKYBNGtvT68SVlI85KPKgUM336dE444QTGjh3LpEmT6N+/PwsXLqStrY1Pf/rT3a4/8cQTueaaa/ja177GKaecwuOPP85tt9221TXXXnstCxcuZMyYMQwdOpSXXnqJu+++m/PPPx+Ayy+/nDVr1mwZKlu8eDGPPfYY06dPL9nW2bNns379ep5++mkA7rnnHgA+8YlPsOeee8bxcpSkICOpGzqgiY4iAcWBI6fP5dKT9+txQEgygEnjOvroo3n44Yf553/+Zz772c+y3XbbMXLkSCZMmFD0+nHjxnHVVVfxox/9iJ/+9KeMGjWKX/ziF+y7775brhkxYgSzZs3i61//OmvWrGHIkCH87d/+LdOmTQOCoHD11Vdz55138uc//5k999yTqVOncvHFF5ds69/93d/x6quvbrl95plnAnDTTTdxwQUX9PKVKM+idvfqVVtbm7e3t6fdjIbWNaRV2OPo0tS3D1eePrxHgebI6XOLBrCWAU0smHxcxY8nvffcc89xwAEHpN0MKaPcv5OZLXL3tnKPozkZKakak+YTRrZw5enDaRnQVPR8b+ZnLj15P5r69tnqWFPfPlx68n49ejwRqYyCjISq5qT5hJEtLJh8HGGrDno6vJUfwIygB9PTXpGIVE5zMhIqjUnzsPmZoSG9nCgmjGxRUBFJiYKMhKrmpPn9izuYMecFOtZ2YgST/l00vCVSuxRkJFQSvYpiCif+HbYEmpYBTb3KLhORdGlORkJVa9K82LBcV4BZMPk4BZg61OhZrVkX57+PejISquvNfcacF1i5tpOhCfUqtJalsfTt25fOzk769euXdlMkRFeFgjgoyEhJvZ0075prKRWkqjUsJ9mw66670tHRQUtLS2r7zktx7k5nZycdHR3stttusTymgowkpnCupSsFGtgq0Fx68n7dFmNqsr9+7bTTTgCsXLmSjRs3ptwaKdS3b1922223Lf9OvaUgI4mJmgJdrWE5yY6ddtoptjcxyTYFGUlMJXMt1VjLEmXoTkTipSAjsSl8Ex/Qry9vvdt9OCSNuZaoQ3ciEq/MpTCb2Y1mtsrMngk5b2b2QzN72cyWmNlf5Z2baGYv5b4mVq/VUqwEzTsbPqBvn60nddOaa0lyzxoRCZe5IAPcDIwpcX4ssE/uaxLwnwBmtjNwOXAYcChwuZkNTLSlskWxN/GNm53+222bibphSpMWSUfmhsvcfb6Z7VXiklOBWz1YLfSEmQ0wsyHAaOBhd18DYGYPEwSrO5JtsUD4m/W6zo08fflJVW5Nd7WeJq35JKlVmQsyEbQAr+XdXpE7Fna8GzObRNALYtiwYcm0ssFk/U28XJp03G/icT6e5pOkltVikCm2cstLHO9+0P064DoINi2Lr2n1L+zNM+trXUqlScf1Jh5W5LO3QUFbSEstq8UgswLYI+92K7Ayd3x0wfF5VWtVA4jyZpzlIZ2wNOk43sSLFfnszePl03yS1LJaDDKzgC+b2Z0Ek/zr3P11M5sD/GveZP9JwJS0GlmPyr0Z1+q+LVHexMsNfxV7baI+TzlZH4oUKSVzQcbM7iDokQwysxUEGWN9Adz9J8CDwDjgZeBd4HO5c2vM7ArgqdxDTetKApB4JPmJuidzGL2Z98i/7zZmbCpSdbbrTTxKDy7Ka9DToJD1oUiRUjIXZNz9nDLnHfhSyLkbgRuTaJck94m6J3MivZlHKbxvsQCT/yYeZTgt7LUp9niVqoWhSJEwWVwnIxmV1P4yPVko2ZvFlWFDW33Miq7nidKDK/badGWixLE+aMLIFhZMPo4/TP+U9tiRmpK5noxkV1KfqHsyDNebobuwaza784fpn+p2PEoPTr0NkeIUZKQiSUzu92QYrjdDd5XeN+qcSGGg6epVKdBII9NwmaSuJ8NwvRm6q/S+E0a2cOXpw8uWxylWv23KvUu5f3FH2TaJ1Cv1ZCR1PRlq6s3wVE+fr9xja9GkSHfmRTJrGklbW5u3t7en3QypA3tP/mXREhMGRed6RGqZmS1y97Zy12m4TCQmYXM6WjQpjUxBRiQmSaV4i9QyzclIqFosL59mm5XGLNKdgowUleXy8mGBJAttrtX6bSJJUZCRosIypf5+5u+B9AJNqUCi7C6R7FGQkaLCVsVvcufSu9MLNKUCSVibO9Z2cuT0ubEPYcUxNJeFIckstEHql4KMFFWq4OPGzc7UWcsSrZIcplQ5mbA2G2w5HtcQWhxDc9+6fym3P/HH2DY364ksDDFKfVN2mRRVLFMq39rOjSXvn9Tq91JpwmFFKsM2EOuN3hTohOD1uS0vwMTZtkr09vcQKUdBRorqKqXSU0m9eZVKEy5W/iVsqXFv98Dp7d46335gWcWPnQTtuilJ03CZhJowsoVvP7CMt97t3msZ2K9vyfsm9eZVLk24MLvryOlzE9kDp7d76xR7TSt9jDho101JmnoyUtLlpxxI3z621bG+fYzLTzmw5P2SXP1eyd4qSS2QTHLhZTUXb2oBqSRNQUZKmjCyhRlnHLzVENSMMw4uOymclTevqBWUq/24A5qK9wSb+m5T1Qn3pF4fkS4qkKkCmZH0JFNMqbHh7l/cwaV3/56Nmz/8/9d3G2PGmeUDuEgWRC2QqSCjIFNWYZprl4H9+nL5KQfqTbGHFISllkUNMpr4l7KKZYpBMHmtNRU9pxI00ggUZKSsUhlh9V62Rb0Nkd7RxL+UVS4jrF7XVGg7ZZHeU5CRssqt/q/XNRVaDS/Sexouk7K6hoemzlrWrZxMPa+pyOJqeA3fSa1RkJFQhW9oU8cHCzAb5U0ua6vhC9OeO9Z2ploRWyQKBRkpqlh13q/d9TQDmvoydXxjpC1fevJ+3VK30+y5TZ21bKt1NRC9IrZIWhRkGlS5YZewtOW1nY2Ttpy17ZTDKl+Xq4gtkiYFmQYUZQ+RRk5bzqe1LCK9o+yyBhQla6pR05azLKzydbmK2CJpylyQMbMxZvaCmb1sZpOLnL/azJ7Ofb1oZmvzzm3KOzerui2vHVGypho1bTnLeloRWyRNmRouM7M+wI+BE4EVwFNmNsvdn+26xt0vybv+K8DIvIfodPdDqtXeWhUla6priKjYfjL1nLacZeXmiJTeLFmUqSADHAq87O7LAczsTuBU4NmQ688BLq9S2+pG1KyprvkIvXllR9gcUZR5NpE0ZC3ItACv5d1eARxW7EIz2xPYG5ibd/gjZtYOfABMd/f7Q+47CZgEMGzYsBiaXVsqzZrS5Hf2lZpn07+dpClrQcaKHAvbi+Bs4B53z/+fNczdV5rZR4G5ZrbU3V/p9oDu1wHXQVDqv7eNTloSPQkFjvrQ9bdRbPgTlKAh6ctakFkB7JF3uxVYGXLt2cCX8g+4+8rc9+VmNo9gvqZbkKklGgaRMGH7/ORTgoakLWvZZU8B+5jZ3ma2HUEg6ZYlZmb7AQOBhXnHBprZ9rmfBwFHEj6XUzNUpFHChC2Y7aIEDcmCTPVk3P0DM/syMAfoA9zo7svMbBrQ7u5dAecc4E7felvPA4BrzWwzQfCcnp+VVquyWKRRsqHU30BLCgkaShCRYjIVZADc/UHgwYJjlxXcnlrkfo8DwxNtXArSKNKoN4vaEPa30TKgiQWTj6tqWzSsK2GyNlwmBYotikxyGCRzG3UtmQlXHwRTBwTfl8xMpx0ZFOVv4/7FHRw5fS57T/4lR06fm9i/o4Z1JUzmejKytZ4WaexpbyRTqbBLZsIDX4WNuU/r614LbgOMOKu6bcmgKIszq9W70LCuhFGQqQGVphv35s0lU28Wj077MMB02dgZHFeQAUr/bVTzA0PW9t6R7NBwWR3qzdBF2JvC0AFNVRt62WLdisqOy1aq+YGh2sO6UjsUZOpQqTeXcoEi7M3i2P0HV3+uprm1suOylVIfGOI2YWQLV54+nJYBTRhB8sGVpw/XpL9ouKwehQ1dDOjXt+wwWtg4fypzNcdftvWcDEDfpuC4lFXtnT1VRUKKUZCpQ2FvLu5EChTF3iwuuevpos+V6FxN17zLo9OCIbLm1iDAaD4mkqzt7CmNSUGmDoW9ufQmUKQ2sTviLAWVXlDvQtKmIFOnir25hBVSjBIoqj30IiL1QRP/DaQ3GUCa2M2mqmf8iVRIPZkG0tsxeg29ZItKuUgtUJCpQb2pLaZAUT8qyfhTPTpJi4JMjdGnV+kSdbGl/mYkTQoyGVDJp8ywT6/ffmCZPqk2mKgZfz1Z46Sej8RFE/8pq7Tqcdin17fe3ZidyslSFVETOSotL5O5StxS0xRkUlZpnbGo61JUZr3+Rc34q7S8jMr2S5w0XJaySj9lFluvUuljS/2IkshR6RqnTFXilpqnnkzKKv2UWezT64CmvhU9hjSWStc4VbOwptQ/9WRS1pOV9IWfXguzh6I8hjSWSlLXVd1B4qQgk7I4ihiqEKLESX9PEidz9/CTZiOBLwO7AsuAG9z9pYJrDgHudfePJtnQpLS1tXl7e3vazRARqSlmtsjd28pdFzonY2ZtwELgOMCAicASM/tKwaXbA3v2oq0iIlKnSg2XfQd4BDjN3Tea2XbAt4CrzWxf4KteqhskIiINr1SQ+ThwnrtvBHD394HLzOxJ4E6gxczOrUIbRUSkRpVLYbbCA+7+S4IhtCMJejo7J9AukZ5ZMhOuPgimDgi+L5mZdotEGlqpnsyzwPHArwpPuPtTZnZU7tztCbVNpDJLZsIDX4WNuUWD614LboN21yxBdcokSaV6Mr8ELjSz5mIn3f1Fgt7MH5NomEjFHp32YYDpsrEzOC5FqU6ZJK1UkJkB7AH8OewCd38dOByoyfTlrNDuhjFZt6Ky46I6ZZK40OEyd98MrC/3AO6+AXg1zkY1Eu31EaPm1mCIrNhxKUp1yiRpql2WMn2SjCjKhP7xl0HfgvpafZuC41KU6pRJ0jIXZMxsjJm9YGYvm9nkIucvMLPVZvZ07uvzeecmmtlLua+J1W15z+iTZARdE/rrXgP8wwn9wkAz4iw45YfQvAdgwfdTfqhJ/xKi7kkj0lOZql1mZn2AHwMnAiuAp8xslrs/W3DpXe7+5YL77gxcDrQBDizK3fetKjS9x6LubtjQSk3oFwaQEWcpqFRAdcokaZkKMsChwMvuvhzAzO4ETiVIpy7nZOBhd1+Tu+/DwBjgjoTaGgtVvI1AE/qJqqRCs0ilKhous8AeZnaEmfVPoD0tQP7M7YrcsUJ/bWZLzOweM9ujwvtmSqV7fTSksIl7TeiLZF7knoyZfZGgdtnuBMNRnwB+Z2b3AvPd/ZoY2tOtwkDuufI9ANzh7u+Z2UXALXxYxLPcfYMnMZsETAIYNmxYz1sbE32SLOP4y7ZeZAkNPaGvxZNSSyL1ZMzsUuAHwE/p/oY+D/hMTO1ZQbA2p0srsDL/And/093fy938KUGNtUj3zXuM69y9zd3bBg8eHEvD09Aw62sqndCv49IyWjwptSZqT+ZLwGXu/t3c5Hy+F4B9Y2rPU8A+ZrY30AGcDWxVhNPMhuQWgQKMB57L/TwH+FczG5i7fRIwJaZ2ZU7Dra+JOqFf56VlSqW81+W/u9S8qHMyuwOLQs5tBj4SR2Pc/QOCTdLmEASPme6+zMymmdn43GVfNbNlZvZ74KvABbn7rgGuIAhUTwHTupIA6pHW14So89IySnmXWhO1J/MycAzwaJFzRxMt+ysSd38QeLDg2GV5P08hpIfi7jcCN8bVlizTm02IOs9EU8q71JqoPZlrgMlm9i1gn9yxXc3sQuDrwNVJNE7CaaV2iDrPRNPUxHW+AAAfaUlEQVTiSak1kYKMu18P/BPwDWBZ7vCDwL8BU939v5JpnoTRm02IOi8to5R3qTVWyQ7KZrYjMAoYBKwBFrr7uoTaVhVtbW3e3t6edjN6RKmsIZbMDOZg1q0IejDHX1Z80j/qdSLSjZktcve2steVCzJm9hFgFvCv7j4vnuZlRy0HGemFwiw0CHo8qnUmEknUIFN24t/dN5jZJ4DC1GWJkXolVVZJPTQR6bGoE/+zgAlJNqSRaYFdCuo8C00kK6KmMM8BZpjZEIIJ/zcoKNmSSz2WHtACuxRogzNpNCnNQUYNMrflvp+e+yrkaDitx7TmJQWqhyaNJMVKGFGDzN6JtqLBaYFdCrr+Yym7LBaaU8y4FOcgIwUZd3810VY0qK7/mB1rOzG2Hn/Umpcq0AZnQO8DRMPV0UtDb4e6UpyDjBRkzOxj5a4psnullFD4H9NhS6Bp0SdBqZI4AoTmFBMWx1BXinOQUYfLniFkb5Y8mpOpQLH/mF0BZsHk49JplDScOAKE5hQTFsdQV4pzkFGDzLFFju1MUE7/JODi2FrUIPQfU7Igjr9DzSkmLI6hrhTnIKPOyTwWcuo+M/sX4CzgF7G1qgHoP6ZkQRx/h5eevN9WQ26gOcVY9XSoq9g8ziXPJNPGEqIuxizl18CpMTxOQ1GBS8mCOP4OVbQzYT0p+to1j7PuNcA/nMdJYZfYqMNlpXwKWBvD4zSUrv+ASvuUNPX077BYRprmEhPSk6GuDJVNilSF2cyKhb/tgP0J9pf5prtfFXPbqkIFMkUqU5iRBkHvR72XDJk6gOK5WgZT4+kTxFYgM2dXurd4A/Ab4OuNVlJGC8+kkSlluQZkqGxS1In/0Qm3o2Zo4Zk0OmVG1oAMlU2KNPFvZjeaWdHSMma2p5ndGG+zsqvUp7h89y/u4Mjpc9l78i85cvpcVVSWuqGtv2vAiLOCvZGa9wAs+J7SXklRh8suAH4C/KHIuUHAROBvYmpTpkX5FKfeTsZoB8xYKWW5RmSkbFIlKcxhGQIHAatjaEtNiPIpLmpvR6ogQ6mc9UIpy1KJ0J6MmV3Mhyv5HbjfzN4ruOwjwG7AzYm0LoOifIrTmHWGzP5GZlI568mEkS0KKlkQtZeeYm++1HDZs8B/E9Rt/DrBosvXC655H3geaJiPhVHWFWg1f0YsmQmda4qf0w6YUutKFc6ED4NK00B478+weWP369IsK+PuDwMPA5jZn4Hr3V2z15T/FBd1zPpb9y/ljidfY5M7fcw457A9+JcJwxNrd8N5dFr4Oe2AKbUubMHl7G/AB50fniv2QauKvfmoKczfTroh9SRKb+db9y/ltif+uOX2JvcttxVoYlKqt6IdMKXWhf19h/Xeo94/ZpHLypjZKOBCYF+CuZituPuhMbar5pXr7dzxZJGFUsBtT/yRtj131nh3HMIWpDXtrPkYqX1hf9+V3L8Koq6TORGYD7QCnyTIJnsHOBjYhWC/GanAphLlfKbcu7TsuhqtwyEYk776oKCExtUHdc8YCyssOLYmKyBlmv4eUxD29920c/n7VnFhZtQU5mnAvxEUwwT4Z3c/jqBXsxGYF3/T6lsfs9Bz5dKdu9bhdKztxPlwHU5D/ceOkpqcoQVp9Ux/jykJ+/see1X34NNnu1zwqf7/g6jDZR8DvgVsJkhn7g/g7q+a2VTg28CtSTSwXp1z2B5bzckUKpXurNpRRK8yW2pBmhZpxkJ/jykq9fedkb/tqEFmA7CNu7uZvQ78BUFxTIC3CYbRYmFmYwh6TX0IMtqmF5z/OvB54AOCYbu/cfdXc+c2AUtzl/7R3cfH1a44FBbWPPIvdmbBK8Un6UqlO2sdDr3fLTCOfdMF0N9jJmVktT9EHy77PdCVf/soMMXMTjSzYwiG0paG3rMCZtYH+DEwlqD3dI6ZfazgssVAm7uPAO4Bvpt3rtPdD8l9ZS7AFA4p/O6P6/js4cMq3jRKtaMIn7SMOplZqickFdHfo5QSNchcw4dlZb4JrAfmECzQ3BX4UkztORR42d2Xu/v7wJ0U7Lrp7r9293dzN58gxl5UksKGFH79/OqKS3RoV016tltgvjj2TRdAf49SWtR1Mg/m/dxhZh8H/hJoAp7PBYQ4tAD5OXkrgMNKXH8hMDvv9kfMrJ1gKG26u98fU7t6rdSQQqUlOrSrJj3bLTBfhvbbyJKe7JWkv0cppeLtl83MgCHAH9z9g5jbUyzlqmiur5l9FmgDjsk7PMzdV5rZR4G5ZrbU3V8pct9JwCSAYcOG9b7VEcRdaka1o+jduHOx/TYw2OekWJpWi3pTPVx/jxImchVmMxtnZk8SJAG8BozIHf9p7g0/DiuAPfJutwIri7TlBOCfgPHuvqVop7uvzH1fTpBWPbLYk7j7de7e5u5tgwcPjqnppWlIIWNGnAUHn8vWn2scfv9fDVuhWdXDJQlRF2OeD8wiKIY5ia3/Z75IMGwVh6eAfcxsbzPbDjg797z5bRkJXEsQYFblHR9oZtvnfh4EHElQ5DMTVB49g156iG4d5Qae/A8b0u1Y26mFltJjUYfL/gmY4e5TchlgN+WdWwb8QxyNcfcPzOzLBEkFfYAb3X2ZmU0D2t19FjAD2AG4Oxi525KqfABwrZltJgie0909M0EGNKSQOZr830rYkK7BluNJb8DXkzkhybaoQWZPchWZi9gA7BRPc7YkGTxYcOyyvJ9PCLnf44AqS0p0mvzfSrHq4Ub3SdGkFlo21I6yDbQQOOqczGuEzG8QTL6/HE9zGpdqP6Wgt2nQdabYkG5Yhb0kFlo2zJxQg+3WGrUncwNwuZm9AXSlBZuZHQ/8I8GCTOmhhvoElyW9TYOuQ4VDukdOn1u1DfgapnJA1JJIdSJqkLmKIOvrFqDro8bjBPMm17r7DxNoW8NQ7acUZaj8RhZF3YAvDg2zo2yDzQVGGi7zwJcIqi5/maBY5sXAx3LHpRca5hOc1JxqZkU2TJp/b0siVarclhgJq2gxZm5hY7fFjdI7DfMJTmpStbIiG6ZyQLGFwEnNBWagEGxokDGzh4CvuPsLeceOA5509/XVaFyjqOaQhEiWNUSafzXnAjMw/1OqJ3MC0Nx1I7c+5mHgE8DvEm5XQ2mYT3AiEqjWXGAG5n8qrV0Wvp2j9EpDfIITkerKwFqwyLXLRESkxmRgLVi5IFNsLVbY+iwREcmSEWfBKT+E5j0AC76f8sOqpu2XGy6bY2aF5fwfLXIMd981vmaJiEgsUl4LVirIfLtqrRARkboUGmTcXUEmJqosKyKNquKdMaUyqksmIo1M2WUJa5jKsiIiRSjIJEx1yUSkkSnIJCys/pjqklVJysUBRRqdgkzCGqaybBY12OZQWaTN+ERBJmHVLJUuBUoVB0yaelBbkl461nbifJj0okDTWJRdVgWqS5aStIoDZqC8ehZoMz4B9WSknlV7c6guafagMkRJL1VQAz1mBRmpX5UUB4zzP2sGyqtngZJeElYjc44KMlK/ohYHjPs/a1o9qIxR0kvCwnrMs7+RTntCmHtjF1Vua2vz9vb2tJshabr6oJA9N/aAS56p/PEK52Qg6EFVufptFuSXVBrQry/usK5zo8orlbJkZrRdM6cOILQo/uk/TfxvzcwWuXtbuevUk5HG1TVEVizAQOnhrVLDaxkor54VE0a2sGDycVz9mUPYsHEzazs3KtOslEp61aV6xhma/1N2mTSmYr2NQmH/iaNkj6VcXj1rlGkWUamkkcK/p+Mvg3v/tvjjZGj+Tz0ZaUzF/jPnK7V7oLLHKqZMs4gqSRoZcRY07Vz8+gzN/ynISGMq9Umv3PCWsscqpkyziCpNGhl7VerbK5ejICONKfQ/c26yv9RQl7LHKqZMs4gqSbuHmpj/05yMNKbjLyueARblE2Bv7tuguuZdtHlfGV3BIUp2Wf59MhRUCimFOcUUZu2YmbKoqaJx31ekDkRNYc5ckDGzMcC/AX2A6919esH57YFbgY8DbwKfcff/zZ2bAlwIbAK+6u5zyj1fWkGmcMdMCIYPVDxTRGpBTa6TMbM+wI+BscDHgHPM7GMFl10IvOXufwlcDVyVu+/HgLOBA4ExwH/kHi+TtGOmiDSCTAUZ4FDgZXdf7u7vA3cCpxZccypwS+7ne4Djzcxyx+909/fc/Q/Ay7nHyySldIpII8hakGkB8pdfr8gdK3qNu38ArAN2iXhfAMxskpm1m1n76tWrY2p6ZZTSmUE1UNFWpNZkLchYkWOFk0Zh10S5b3DQ/Tp3b3P3tsGDB1fYxHgopTNjaqSibZZo10uJImtBZgWwR97tVmBl2DVmti3QDKyJeN/M0I6ZGaNV/BXRrpcSVdbWyTwF7GNmewMdBBP55xZcMwuYCCwEzgDmurub2Szgv8zsB8BQYB/gt1VreQ/0dMfMwtTnY/cfzK+fX61U6N7QKv6KqBaZRJWpIOPuH5jZl4E5BCnMN7r7MjObBrS7+yzgBuBnZvYyQQ/m7Nx9l5nZTOBZ4APgS+6+qegT1bDC1OeOtZ3c9sQft5zv+kQJ6D97JZpbQ8r9axV/MUpckaiyNlyGuz/o7vu6+1+4+3dyxy7LBRjcfYO7n+nuf+nuh7r78rz7fid3v/3cfXZav0OSin2CLKRU6B6otJxHg1PiSg2rcoJL5oKMlBb1k6I+UVaoBmpAZYkSV2IW5Y0/juCQQoJLpobLpLyhA5roiBBA9ImyBzJeAypLVIssRE/KDUXZnyjKNVFUsl9NTBRkasylJ+/XrRxNIX2ilGroaeJK3eppIIjyxh9XcEghwUXDZTWmWOrzZw8fplRokbT1NA0+yht/XMEhhW0q1JOpQfoEKVmjiuL0PBBEyWyMK/sxhW0q1JMRkV7RwsycnvYSomQ2xpX9mEKCi3oyItIrWpiZ09NeQpSNynqymVmp56tigouCjIj0ihZm5sQZCMIevwazHxVkRKRXwtLqGzKNvieBIK705IzSnIyI9IoWZlaocFHl7G/UdXFW9WREqqEni/RqhBZmVqBYryVMnRRnVZARSVqdD4eA0uojK7aWJkydFGfVcJlI0rRXjXSJ2jupo+KsCjIiSdNeNdIlrHfStHPdFmfVcJlI0rRXjXQJW0sz9qq6CSqF1JMRSVrU1dpV3udDUtCAW0qoJyOStCiL9Oo4OUB1zQrU6KLKnjJ3T7sNqWpra/P29va0myGN7uqDQobU9oBLnql+e2JSuF04BGtoVCm89pnZIndvK3edhstEsqBOkwNK1TWLjYYZM01BRiQLUtjnoxoSr2uWwnbCUhkFGZEsiKuUe8aE1S+Lra6Z1iBlnoKMSBbUadZR4nXN6nSYsZ4ou0wkK+ow6yjxumZag5R5CjIikqhE65qlsJ2wVEbDZSJSu+p0mLGeqCcjIrWtDocZt1Lj20QoyIiIZFUdVILQcJmISFbVQYq2goyISFbVQYq2goyISFbVQSWIzAQZM9vZzB42s5dy3wcWueYQM1toZsvMbImZfSbv3M1m9gczezr3dUh1fwORlKh2V/2qg0oQmQkywGTgUXffB3g0d7vQu8D57n4gMAa4xswG5J2/1N0PyX09nXyTRVKm2l31rQ5StLOUXXYqMDr38y3APOAb+Re4+4t5P680s1XAYGBtdZooUkSaKaalJoZr6I1ISqjxFO0s9WR2c/fXAXLfdy11sZkdCmwHvJJ3+Du5YbSrzWz7EvedZGbtZta+evXqONoujSrtnkQdTAxLfatqkDGzR8zsmSJfp1b4OEOAnwGfc/fNucNTgP2BTwA7U9ALyufu17l7m7u3DR48uIe/jQjpp5jWwcSw1LeqDpe5+wlh58zsDTMb4u6v54LIqpDrdgJ+CXzL3Z/Ie+zXcz++Z2Y3Af8QY9NFiku7J6HaXZJxWRoumwVMzP08Efh54QVmth1wH3Cru99dcG5I7rsBE4Da3bNWakfaPYk6mBiW+palif/pwEwzuxD4I3AmgJm1ARe5++eBs4CjgV3M7ILc/S7IZZLdbmaDAQOeBi6qcvulEWWhJ1HjE8NS38zd025Dqtra2ry9vT3tZkgtq/EChiI9YWaL3L2t3HVZ6smI1Cb1JERCZWlORkRE6oyCjIhUTqVsJCINl4lUQvMvdbHHiVSPejIiUaW9uj8r4liAqp5Qw1CQEYkq7dX9WdHbBagK1g1FQUYkqrRX92dFbxeg1nKwVg+sYgoyIlEltbq/1t64ervHSa0Ga/XAekRBRiSqJDaQqsU3rt6Wsql2KZ64gngt98BSpOwykai63kTjzC7L6n4w5bLowhagRsm+q2YpnmKZcPdOgj8+AZ/+QWWPVas9sJQpyIhUIu7V/Vl84+ppinLU+yURrMMUC+I4tN8Iww6v7DmbW3M9ziLHJZSGy0TSlHYV52J6OixUyf1GnAWXPANT1wbfk+q1hQZrr3yYK4nh0gagICOSpiy+cfW0dxVnryyueZRSwbrSdmlbhR7RcJlImqo5dBRVT4eF4hpOirOiwPGXBXMwFKk235PeooqhVkw9GZG0VWvoKKqe9q7i6pVVOlxXqtcz4ixo+xuCbaZ62a5aSzXPCAUZEdlaT4eF4hpOqmTYLUoK+Kd/AKdf17t21WKqeUZo0zJtWiaSLVcfFDLstkfQ0+vptdVqU4OIummZejIiWVELwzFLZsJVe8PU5uDrqr3jb2clw27VSgHPYqp5jVCQEcmCWhiOWTITfv4l6Fzz4bHONXD/F+NtZyXDbtVKAc9iqnmNUJARyYJaKFny6DTY9H7345s3xt/OqMkQ1UoBz2KqeY1QkBHJgmLj/aWOp6HU0FBaw0bVWruiNTI9pnUyIllgfcA3FT+eFWHrYLrOpaVaa1e0RqZH1JMRyYJiAabU8TQcfxn02a778W36athIQinIiGRB8x6VHU/DiLPg1B9D084fHmvaGSb8hz7hSygNl4lkQTXL3/eGhoykQurJiFRDuTUwmliuHbWwnilD1JMRSVol+6zEHVSibCIm0cVZvLNBqCcjkrS01sDUwgLPWlML65kyRkFGJGlplSTRG2J3vR3qUnmZiinIiCQtrZIkekPcWhw9O5WXqVhmgoyZ7WxmD5vZS7nvA0Ou22RmT+e+ZuUd39vMnszd/y4zK5LQL5KCsJIk+5yU7ASy3hC3FkfPTuVlKpaZIANMBh51932AR3O3i+l090NyX+Pzjl8FXJ27/1vAhck2VySiYpljB58Lv/+vZOdL9Ia4tTh6dsoCrFhm9pMxsxeA0e7+upkNAea5+35FrnvH3XcoOGbAamB3d//AzEYBU9395HLPq/1kJBXV2p9E2WUf0p4wsYq6n0yWUph3c/fXAXKBZteQ6z5iZu3AB8B0d78f2AVY6+4f5K5ZAbQk3mKRnqrWfEnaiyfLBblqBsFaWfBaZ6oaZMzsEWD3Iqf+qYKHGebuK83so8BcM1sKvF3kutAumplNAiYBDBs2rIKnFolJWLHJepovKbempJI1J3EEo67r1bOrqpobLiu4z83AL4D/RsNlUksK32Ah+FQd5/h+2kNl5Yanog5fVeO1korV4vbLs4CJuZ8nAj8vvMDMBprZ9rmfBwFHAs96ECl/DZxR6v4imZH0BHIWFmKWGxKMOmSo9T41LUtzMtOBmWZ2IfBH4EwAM2sDLnL3zwMHANea2WaCADnd3Z/N3f8bwJ1m9i/AYuCGav8CIhVJcr6k1BtztT79lxsSjDpkqPU+NS0zQcbd3wSOL3K8Hfh87ufHgeEh918OHJpkG0VqRhbemMtNtEediG+E+as6lqXhMhGJSxYWYpYbEow6ZKj1PjUtMxP/adHEv9SlepssTzuJQbqpxXUyIhKXekvXTXu9j/SYgoxIvcp/Y+7qCdw7qXzAUa9BYqQgI1LvKl30qE25JEaa+Bepd5WsM9GaFImZgoxIvasknTkLqc9SVxRkROpdJenMWUh9lrqiICNS7ypZZ6I1KRIzBRmReldJnTRtyiUx02JMLcYUEalYLVZhFhGROqMgIyIiiVGQERGRxCjIiIhIYhRkREQkMQoyIiKSGAUZERFJjIKMiIgkRkFGREQSoyAjIiKJafiyMma2Gni1Ck81CPi/KjxPEtT2dNRq22u13aC2V2JPdx9c7qKGDzLVYmbtUer8ZJHano5abXutthvU9iRouExERBKjICMiIolRkKme69JuQC+o7emo1bbXartBbY+d5mRERCQx6smIiEhiFGQSYmZnmtkyM9tsZqEZH2Y2xsxeMLOXzWxyNdsYxsx2NrOHzeyl3PeBIddtMrOnc1+zqt3OgraUfB3NbHszuyt3/kkz26v6rewuQrsvMLPVea/z59NoZzFmdqOZrTKzZ0LOm5n9MPe7LTGzv6p2G4uJ0O7RZrYu7zW/rNptDGNme5jZr83sudz7y8VFrsnW6+7u+krgCzgA2A+YB7SFXNMHeAX4KLAd8HvgYxlo+3eBybmfJwNXhVz3Ttptjfo6Al8EfpL7+Wzgrhpp9wXAv6fd1pD2Hw38FfBMyPlxwGzAgMOBJ9Nuc8R2jwZ+kXY7Q9o2BPir3M87Ai8W+ZvJ1OuunkxC3P05d3+hzGWHAi+7+3J3fx+4Ezg1+daVdSpwS+7nW4AJKbYliiivY/7vdA9wvJlZFdtYTFb//SNx9/nAmhKXnArc6oEngAFmNqQ6rQsXod2Z5e6vu/vvcj//GXgOaCm4LFOvu4JMulqA1/Jur6D7H0wadnP31yH4owZ2DbnuI2bWbmZPmFmagSjK67jlGnf/AFgH7FKV1oWL+u//17lhj3vMbI/qNC0WWf37jmKUmf3ezGab2YFpN6aY3JDvSODJglOZet23TeuJ64GZPQLsXuTUP7n7z6M8RJFjVUn3K9X2Ch5mmLuvNLOPAnPNbKm7vxJPCysS5XVM7bUuIUqbHgDucPf3zOwigt7YcYm3LB5ZfM2j+B1ByZR3zGwccD+wT8pt2oqZ7QD8N/A1d3+78HSRu6T2uivI9IK7n9DLh1gB5H8ybQVW9vIxIynVdjN7w8yGuPvruW72qpDHWJn7vtzM5hF8qkojyER5HbuuWWFm2wLNpD9kUrbd7v5m3s2fAldVoV1xSe3vuzfy37Td/UEz+w8zG+TumahpZmZ9CQLM7e5+b5FLMvW6a7gsXU8B+5jZ3ma2HcGEdKpZWjmzgIm5nycC3XplZjbQzLbP/TwIOBJ4tmot3FqU1zH/dzoDmOu5WdIUlW13wVj6eIIx+FoxCzg/l+10OLCuaxg2y8xs9675OjM7lOB98s3S96qOXLtuAJ5z9x+EXJat1z3tbIl6/QJOI/hE8R7wBjAnd3wo8GDedeMIMkReIRhmy0LbdwEeBV7Kfd85d7wNuD738xHAUoKMqKXAhSm3udvrCEwDxud+/ghwN/Ay8Fvgo2m/zhHbfSWwLPc6/xrYP+0257X9DuB1YGPub/1C4CLgotx5A36c+92WEpJlmcF2fznvNX8COCLtNue1/ZMEQ19LgKdzX+Oy/Lprxb+IiCRGw2UiIpIYBRkREUmMgoyIiCRGQUZERBKjICMiIolRkBHJY2ZTzcyLfD2SwHOdZGZfi/txe8rMTjSzO8zsf3O/89S02yS1Tyv+RbpbB4wpcixuJxEsDL0mgcfuiTHACIK1UWen3BapEwoyIt194EH12ppiZk3u3tmLh7jU3f8+91g1Uw1ask3DZSIVMrNtzGxyblOo98zsRTObWHDNpyzY8G2Vmb2dq1R9Ut75qcDfA3vmDcndnDs3z8zuKXi80blrDsrd3it3+zwzu9XM1hIU0+y6/vO5Ta3eM7NXzewfy/1e7r65Fy+LSFHqyYgUkSuimW+Tf1ge40cEddCmEVTsPRG40czedPdf5K7Zm+BN/3vAZmAsMNvMjnb3BcD1BJV9jyMoQQSwugdN/R5wL3AmsCnX9kuBfyXYfG4e8HHgCjN7193/vQfPIdJjCjIi3e1CUNcq34nAI2b2l8DfAZ9z965N0B7JFbK8HPgFQP6buZltQ1B37ECCOlkL3H2Fmb0OvNfLobkn3P1Lec+1U64d/+Lu384dftjM+gHfMrP/dPdNvXg+kYpouEyku3XAJwq+ujaGOp6gZ3KfmW3b9UUwWX6ImfUBMLNWM7vFzDqADwiC1knAvjG39ZcFt0cB/YG7C9o3F9iNoOy7SNWoJyPS3Qfu3h5ybhDQh/BssyFmtpKg3PqOwGUElZ/XEwyvhe0y2lNvFGkfBFWEi9kDeDXmNoiEUpARqcwagp7JkQQ9mkKrgL8k2MBtrLv/quuEmTVFfI4NwHYFx3YOubawjHrXRmyfpnsAAnghYhtEYqEgI1KZuQQ9mWZ3f7jYBXnB5L28Y3sSBKYleZe+T7DPTaEVwNEFx06M2L6FQCcw1N0Lh9JEqk5BRqQC7v6Cmf0EuNPMvgu0EwSKA4F93f3zwPMEgeL7ZvbPBMNm3wY6Ch7ueWA3M7sAeAb4P3f/X+A+4EIzu5pgzuVY4OSI7VubS4/+t1xgm08w97ovcKy7nxZ239z1n8jd3A74mJmdAax399lRnl+kkIKMSOW+RLCb5d8SzLO8TbD19A0A7v6emZ1OsDvhPQQB5zvAaOCgvMeZSRBAvgsMBm4BLnD3X5rZN4EvAp8n2P76axTZBrsYd/9ubl7oEoK1OBty7b2rzF2PBW7Ku31m7utVYK8ozy1SSDtjiohIYpTCLCIiiVGQERGRxCjIiIhIYhRkREQkMQoyIiKSGAUZERFJjIKMiIgkRkFGREQSoyAjIiKJ+f8fLV3+qHVJ5AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#plot the blob dataset\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(6,6))\n",
    "\n",
    "fontsize = 15\n",
    "\n",
    "#plot both classes as above, but all at once\n",
    "for cat in np.unique(y_moon_train): #loop over unique class IDs\n",
    "    \n",
    "    mask = y_moon_train == cat\n",
    "    \n",
    "    ax.plot(X_moon_train[mask][:,0], X_moon_train[mask][:,1], 'o', label='class {}'.format(cat))\n",
    "\n",
    "\n",
    "\n",
    "ax.legend(fontsize=fontsize)\n",
    "\n",
    "ax.set_xlabel('Feature 1', fontsize=fontsize)\n",
    "ax.set_ylabel('Feature 2', fontsize=fontsize)\n",
    "\n",
    "ax.set_title('Moons', fontsize=fontsize);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <center> \"Real\" Data </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data can come in many different formats and data manipulation and pre-processing is, most of the times, data-specific. Here we see two very simple cases, that are loading \"classic\" datasets that come shipped with many libraries (in this case scikit-learn) and loading datasets from an external file. There are a lot of different ways to do such things, especially loading data from a file, and many times we have to use different ways for different data. What's more, many libraries come with their own utilities to load data in a conveninet way, so feel free to research different methods and adopt the one you feel most comfortable with."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Datasets coming with a library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "#load utility for the simple Iris dataset\n",
    "from sklearn.datasets import load_iris"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "#load the iris dataset\n",
    "iris_data = load_iris()\n",
    "\n",
    "#scikit-learn loading methods return a Bunch object (defined inside scikit-learn)\n",
    "#It beahaves essentially like a python dictionary and contains some useful meta-information on the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".. _iris_dataset:\n",
      "\n",
      "Iris plants dataset\n",
      "--------------------\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      "    :Number of Instances: 150 (50 in each of three classes)\n",
      "    :Number of Attributes: 4 numeric, predictive attributes and the class\n",
      "    :Attribute Information:\n",
      "        - sepal length in cm\n",
      "        - sepal width in cm\n",
      "        - petal length in cm\n",
      "        - petal width in cm\n",
      "        - class:\n",
      "                - Iris-Setosa\n",
      "                - Iris-Versicolour\n",
      "                - Iris-Virginica\n",
      "                \n",
      "    :Summary Statistics:\n",
      "\n",
      "    ============== ==== ==== ======= ===== ====================\n",
      "                    Min  Max   Mean    SD   Class Correlation\n",
      "    ============== ==== ==== ======= ===== ====================\n",
      "    sepal length:   4.3  7.9   5.84   0.83    0.7826\n",
      "    sepal width:    2.0  4.4   3.05   0.43   -0.4194\n",
      "    petal length:   1.0  6.9   3.76   1.76    0.9490  (high!)\n",
      "    petal width:    0.1  2.5   1.20   0.76    0.9565  (high!)\n",
      "    ============== ==== ==== ======= ===== ====================\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "    :Class Distribution: 33.3% for each of 3 classes.\n",
      "    :Creator: R.A. Fisher\n",
      "    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\n",
      "    :Date: July, 1988\n",
      "\n",
      "The famous Iris database, first used by Sir R.A. Fisher. The dataset is taken\n",
      "from Fisher's paper. Note that it's the same as in R, but not as in the UCI\n",
      "Machine Learning Repository, which has two wrong data points.\n",
      "\n",
      "This is perhaps the best known database to be found in the\n",
      "pattern recognition literature.  Fisher's paper is a classic in the field and\n",
      "is referenced frequently to this day.  (See Duda & Hart, for example.)  The\n",
      "data set contains 3 classes of 50 instances each, where each class refers to a\n",
      "type of iris plant.  One class is linearly separable from the other 2; the\n",
      "latter are NOT linearly separable from each other.\n",
      "\n",
      ".. topic:: References\n",
      "\n",
      "   - Fisher, R.A. \"The use of multiple measurements in taxonomic problems\"\n",
      "     Annual Eugenics, 7, Part II, 179-188 (1936); also in \"Contributions to\n",
      "     Mathematical Statistics\" (John Wiley, NY, 1950).\n",
      "   - Duda, R.O., & Hart, P.E. (1973) Pattern Classification and Scene Analysis.\n",
      "     (Q327.D83) John Wiley & Sons.  ISBN 0-471-22361-1.  See page 218.\n",
      "   - Dasarathy, B.V. (1980) \"Nosing Around the Neighborhood: A New System\n",
      "     Structure and Classification Rule for Recognition in Partially Exposed\n",
      "     Environments\".  IEEE Transactions on Pattern Analysis and Machine\n",
      "     Intelligence, Vol. PAMI-2, No. 1, 67-71.\n",
      "   - Gates, G.W. (1972) \"The Reduced Nearest Neighbor Rule\".  IEEE Transactions\n",
      "     on Information Theory, May 1972, 431-433.\n",
      "   - See also: 1988 MLC Proceedings, 54-64.  Cheeseman et al\"s AUTOCLASS II\n",
      "     conceptual clustering system finds 3 classes in the data.\n",
      "   - Many, many more ...\n"
     ]
    }
   ],
   "source": [
    "#dataset description\n",
    "print(iris_data['DESCR'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "#extract data matrix\n",
    "X = iris_data['data']\n",
    "\n",
    "#extract categories\n",
    "y = iris_data['target']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']\n",
      "[[5.1 3.5 1.4 0.2]\n",
      " [4.9 3.  1.4 0.2]\n",
      " [4.7 3.2 1.3 0.2]]\n"
     ]
    }
   ],
   "source": [
    "#print features names\n",
    "print(iris_data.feature_names) #another way in which you can access fields of a Bunch object\n",
    "\n",
    "#show first 3 samples\n",
    "print(X[:3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['setosa' 'versicolor' 'virginica']\n"
     ]
    }
   ],
   "source": [
    "#print the meaning of the categories\n",
    "print(iris_data.target_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Datasets coming from an external file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now load the same iris dataset directly from the .csv in which the data are stored, without relying on the scikit-learn utility function. You can find the .csv file in the 'datasets/' folder. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#open the file in reading mode\n",
    "with open('datasets/iris.csv', 'r') as fh:\n",
    "    \n",
    "    #read first line (header)\n",
    "    features = fh.readline()\n",
    "    \n",
    "    #split the line, make list of names of features\n",
    "    features = features.split(',')\n",
    "    \n",
    "    #remove spaces before and after feature names\n",
    "    features = [f.strip() for f in features]\n",
    "    \n",
    "    #now load the rest of the dataset - feature vectors - into a data matrix\n",
    "    X = np.loadtxt(fh, dtype=float, delimiter=',')\n",
    "    \n",
    "    #file stream gets automatically closed at the end of the 'with' statement "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#remember that the last column contains the labels, we want them into a separate vector\n",
    "y = X[:,-1]\n",
    "\n",
    "#drop last column from data matrix\n",
    "X = X[:,:-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)', 'target']\n"
     ]
    }
   ],
   "source": [
    "print(features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of data matrix: (150, 4)\n",
      "Shape of target vector: (150,)\n"
     ]
    }
   ],
   "source": [
    "print('Shape of data matrix:', X.shape)\n",
    "print('Shape of target vector:', y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <center> **The Perceptron** </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have our data we can start looking into the simplest possible neural network: the **perceptron**. At its core, the perceptron is just a non-linear activation function (the Heaviside theta) applied to a (linear) weighted sum of the data features, coupled with a rough version of SGD for tuning the weights. The schema of the computation is summarized in the figure below (in the figure, the dimensionality of the input vector is called _n_ istead of _D_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![The Perceptron](img/perceptron.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What this means, practically, is that all it takes to build a perceptron in python is a numpy array holding the lalues of the weights (+1 float variable for the bias, which can be embedded into the weight vector as an extra dimension). Then we will need an algorithm that tells us how to update the weights, the Perceptron Learning Algorithm (PLA).\n",
    "\n",
    "The PLA works in two steps, repeated for each sample in the training set. Every time the PLA goes through a complete scan of all the samples and needs to start again, we say it has trained for one **epoch**. The two steps of the algorithm are the following:\n",
    "\n",
    "> Repeat the two steps for a predefined number of epochs or until convergence (when all samples are correctly classified):\n",
    ">\n",
    "> * Predict the label of the sample based on current weights **w**:\n",
    "$$\n",
    "\\hat{y} = \\theta ( \\textbf{x} \\cdot \\textbf{w} + b) = \\theta (\\displaystyle\\sum_{i=0}^{D} x_i w_i + b)\n",
    "$$\n",
    ">\n",
    "> * Update the weights **w** with the following expression:\n",
    "$$\n",
    "w_i \\leftarrow w_i + \\eta (y-\\hat{y}) x_i\\\\\n",
    "b \\leftarrow b + \\eta (y-\\hat{y}) b\n",
    "$$\n",
    "\n",
    "**NOTE:** if the sample is correctly classified there is no update of the weights (the difference between the predicted and actual label is zero). If the sample is incorrectly classified the update step pushes the weights towards a correct classification. The parameter ${\\bf \\eta}$ tunes how strong each update should be and it is called **learning rate**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Perceptron Implementation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we now implement the perceptron model with a bunch of separate functions\n",
    "# the implementation as a class is left as a guided exercise\n",
    "\n",
    "\n",
    "def pla_step(weights, bias, x, y, eta=1):\n",
    "    '''\n",
    "    Performs PLA on a single sample.\n",
    "    \n",
    "    Parameters:\n",
    "    -----------\n",
    "    \n",
    "    weights : array\n",
    "        Current weights of the perceptron.\n",
    "        \n",
    "    bias : float\n",
    "        Current bias term of the perceptron.\n",
    "\n",
    "    x : array\n",
    "        Incoming observation.\n",
    "        \n",
    "    y : int or float\n",
    "        True label of the observation.\n",
    "        \n",
    "    eta : float (default: 1)\n",
    "        Learning rate.\n",
    "        \n",
    "\n",
    "    Returns:\n",
    "    --------\n",
    "        \n",
    "    weights : array\n",
    "        Updated weights.\n",
    "        \n",
    "    bias : float\n",
    "        Updated bias.\n",
    "    \n",
    "    w_updated : bool\n",
    "        Whether the weights were updated or not.\n",
    "    '''\n",
    "    \n",
    "    z_pred = x.dot(weights)+bias  # calculate weighted sum\n",
    "    \n",
    "    y_pred = np.heaviside(z_pred,0) #returns 0 if the argument is exactly 0\n",
    "    \n",
    "    \n",
    "    # if prediction is wrong update weights and bias\n",
    "    if y != y_pred:\n",
    "        \n",
    "        weights += eta*(y-y_pred)*x\n",
    "        bias += eta*(y-y_pred)*1\n",
    "            \n",
    "        w_updated = True\n",
    "        \n",
    "    else:\n",
    "        #right classification, do nothing\n",
    "        w_updated = False\n",
    "        \n",
    "        \n",
    "    return weights, bias, w_updated"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def perceptron_train(X, y, eta=1, max_epochs=10, error_history=True):\n",
    "    '''\n",
    "    Performs full PLA on new perceptron, stops on specified number of epochs or upon convergence (no errors).\n",
    "    Returns weights and bias term of the trained perceptron, plus the number of errors at each epoch (optional).\n",
    "    \n",
    "    Parameters:\n",
    "    -----------\n",
    "\n",
    "    X : 2d array\n",
    "        Data matrix (training set).\n",
    "        \n",
    "    y : array\n",
    "        Label vector for the training set.\n",
    "        \n",
    "    eta : float (default: 1)\n",
    "        Learning rate.\n",
    "        \n",
    "    max_epochs : int (default: 10)\n",
    "        Maximum number of epochs for learning.\n",
    "        \n",
    "    error_history : bool (default: True)\n",
    "        If True also returns number of errors of the perceptron at each training epoch.\n",
    "        \n",
    "\n",
    "    Returns:\n",
    "    --------\n",
    "        \n",
    "    weights : array\n",
    "        Trained weights.\n",
    "        \n",
    "    bias : float\n",
    "        Trained bias.\n",
    "        \n",
    "    history : array (optional)\n",
    "        Number of errors of the perceptron at each trining epoch.\n",
    "    \n",
    "    '''\n",
    "    \n",
    "    \n",
    "    #initialize weights and bias\n",
    "    w = np.zeros(X.shape[1]) #dimension of input data, in our case 2d\n",
    "    b = 0.\n",
    "    \n",
    "    \n",
    "    errors = []\n",
    "    \n",
    "    #create an array with the index of every sample\n",
    "    #we will need it to loop through samples in random order\n",
    "    index = np.arange(X.shape[0]).astype(int)\n",
    "\n",
    "    converged = False\n",
    "    epoch = 0\n",
    "    \n",
    "    #repeat training for given epochs or until convergence\n",
    "    while not converged and epoch < max_epochs:\n",
    "        \n",
    "        #shuffle ordering in which we will get the samples\n",
    "        np.random.shuffle(index)\n",
    "\n",
    "        #keep track of number of updates in this epoch\n",
    "        n_updates = 0\n",
    "\n",
    "        #loop through each sample in the new ordering and train\n",
    "        for idx in index:\n",
    "\n",
    "            w, b, updated = pla_step(w, b, X[idx], y[idx], eta) #training step\n",
    "\n",
    "            n_updates += int(updated)\n",
    "\n",
    "            \n",
    "            \n",
    "        errors.append(n_updates) #update only happen if the prediction was wrong\n",
    "\n",
    "        \n",
    "        #check for convergence\n",
    "        if 0 == n_updates:\n",
    "            converged = True\n",
    "            print('Perceptron converged on epoch {}!'.format(epoch))\n",
    "            \n",
    "        epoch += 1\n",
    "        \n",
    "        \n",
    "    if error_history:        \n",
    "        return w, b, errors\n",
    "    else:\n",
    "        return w, b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def perceptron_predict(weights, bias, X):\n",
    "    '''\n",
    "    Predicts the labels of the given data using parameters from a trained perceptron.\n",
    "    \n",
    "    Parameters:\n",
    "    -----------\n",
    "    \n",
    "    weights : array\n",
    "        Trained weights.\n",
    "        \n",
    "    bias : float\n",
    "        Trained bias.\n",
    "        \n",
    "    X : 2d array\n",
    "        Data matrix (test set).\n",
    "        \n",
    "        \n",
    "    Returns:\n",
    "    --------\n",
    "    y_pred : array\n",
    "        Predicted label vector for X.\n",
    "        \n",
    "    '''\n",
    "    \n",
    "    #fully vectorized code\n",
    "    z_pred = X.dot(w)+b\n",
    "    y_pred = np.heaviside(z_pred,0)\n",
    "    \n",
    "    return y_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Perceptron on BLOBS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Perceptron converged on epoch 1!\n",
      "Number of training errors:\t0 (0.0 %)\n"
     ]
    }
   ],
   "source": [
    "#Train the perceptron on the blobs dataset, with default parameters\n",
    "w, b, training_errors = perceptron_train(X_blob_train, y_blob_train)\n",
    "\n",
    "#number of training errors in last epoch\n",
    "print('Number of training errors:\\t{} ({} %)'.format(training_errors[-1], training_errors[-1]/X_blob_train.shape[0]*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of testing errors:\t0 (0.0 %)\n"
     ]
    }
   ],
   "source": [
    "#evaluate performance of trained perceptron on test set\n",
    "y_pred = perceptron_predict(w, b, X_blob_test)\n",
    "\n",
    "test_errors = np.sum(y_pred != y_blob_test)\n",
    "\n",
    "print('Number of testing errors:\\t{} ({} %)'.format(test_errors, test_errors/X_blob_test.shape[0]*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAGKCAYAAADngI2gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl4VOXZ+PHvnRCSgBC2QCAs7qAsEQEBUbYEwu7ydtG6YGu1tta1tm61qFVLa1+rte+v7ksrrW0tLsgaQPZ9FWVTXEkIIEtASEJInt8fMwmTyeyZmXPOzP25rlzMnDmZuQPh3OfZ7keMMSillFKeUqwOQCmllP1oclBKKdWAJgellFINaHJQSinVgCYHpZRSDWhyUEop1YAmB6UAEXlYRIzH13ER2SIiN3ucM9z9Wq8w33uRiLwV/aiVip0mVgeglI2UAWPcj5sDE4HnReRbY8w/rAtLqfjT5KDUKSeNMas8ni8QkYuBywFNDiqpaLeSUoEdBdL8vSgizUTkzyJSKiIVIrJWREb7OfdmEflCRMpFZKaI5Hq9fr+IfOp+n70iMkdEcqL88ygVEm05KOVBRGr/TzQDJgHDgB8F+JYX3ec9AHwK3ATMFJERxphlHucNBroDdwMZwO+Bd4AB7s+93v0e9wIfA22Bkbi6t5SKO00OSp3SFqjyOvZnY8zffJ0sIucBVwM/NMa87j42F/gQeAgo9Di9PXCxMeZL93lfAstEZIwxZg5wETDPGPP/PL5nehR+JqUiot1KSp1ShutOfgBwCXAHMFlEpvg5fwAgwH9qDxhjatzPL/E6d0NtYnCftxzYhyspAGwCxonIIyJykYikRuHnUSpi2nJQ6pSTxph1Hs+Xi0ga8ISIPOvj/I7At8aY417H9wLNRCTdGFPpPrbPx/fvc78HwCtAC+Bm4DfAARH5K/CwMaY6wp9HqYhpy0GpwLYCTYGzfLy2BzhNRJp5He8AHPdIDODqVvLW3v0eGGNqjDF/MsacB3QF/ohrDOKmRsavVEQ0OSgVWO2Ct699vLYWMMB3ag+IiLifL/M690IR6epx3hBcyWGN95saY742xkzFNcB9fqOiVypC2q2k1ClNRGSQ+3FToB/wa+BdY0ypiPTwPNkYs01E/gn8RURacmq2Ug/gp17vvQ94X0Qe5tRspQ3uwWhE5HngILAK19jHCOAcXLOXlIo7TQ5KnZIFrHQ/rgK+BJ4DHgvwPTfhutA/BLQCtgATvKax4n7f+cDTQDawCNf4gufrNwE/wZU8PgVuMsa8E/mPo1TkRLcJVUop5U3HHJRSSjWgyUEppVQDmhyUUko1oMlBKaVUA5oclFJKNeDYqaxt2rUzXbqdbnUYSik/PvtkJ98ePQpA19xcstu1tTgiBbB+84ffGGOyg53n2OTQpdvpzF6xLviJSqm4O1JWRp/Op64/y2e/Q+dOnSyMSNWSdrlfBj9Lu5WUUjGwqGguVVWu6ud9e/fSxOBAmhyUUlFXNHNG3eNJY3xujKdsTpODUirq+lzYj57nu2oGanJwJseOOSil7Oum2+7kyvyhtEkzdMrRbbCdSFsOSqmoKy0pASC3Y0dcVcyV02hyUEpFXdrhUrKbWh2FagxNDkqpqPGs8pyj3UmOpslBKRU1zz/zFAX983juhRf4ZNdnVoejGkEHpJVSUTNvxrts++hDtn30IQN6duecs860OiQVIW05KKWi4uCBA6xduRwAEWFcQb7FEanG0OSglIqKhXNmUVNTA8DgAf1on93O4ohUY2hyUEpFxbyZ79U91oVvzhfX5CAir4jIPhH5yOPYkyKyXUQ+FJG3RaRVPGNSSjVeZWUli4vm1j2fWDjKwmhUNMS75fAaMMbrWBHQyxjTB9gJ3B/nmJRSjbRqyeK68txnnXE65517jrUBqUaLa3IwxiwBDnodm2eMOel+ugroHM+YlFKN59mlNHF0ga6KTgB2G3P4ETDb34sicrOIrBORdQf2749jWEopf4wxWoU1AdkmOYjIg8BJYJq/c4wxLxhj+htj+rfNDrqRkVIqDrZu+ZDir78CIKtlSy4ZdJHFEalosEVyEJHJwATgGuO5/l4pZXulJcXkuDfzGVcwkrS0NIsjUtFg+QppERkD3AsMM8YctzqeRDd3czHPLdjBvrJy2mdlckt+dwrzcq0OSzlY/phxzJ3xHtu2bee8Lh2sDkdFSVyTg4j8ExgOtBOR3cAUXLOT0oEi9yDWKmPMLfGMK1nM3VzM1BlbqKyqBmBvWTlTZ2wB0AShGkVE+P6YEVaHoaIorsnBGHO1j8MvxzOGZPbcgh11iaFWZVU1zy3YoclBRay0pATtSEo8thhzUPGxr6w8rONKhUL3bkhMmhySSPuszLCOKxVI+fHj/Pj7V/LW9OkYdF1DotHkkERuye9OelpqvWPpaanckt/dooiUky1dOJ/Z777No489zqjv+OoxVk5m+WwlFT+14wo6W0lFg+fCN62llHg0OSSZwrxcv8lAp7mqUNXU1FA0yzM5FFgYjYoFTQ4K0GmuKjyb1q1l/969AGS3a8vAfhdaHJGKNh1zUEDgaa5KefPsUpowuoDU1NQAZysn0uSgAJ3mqsJT5FmFVccbEpImBwXoNFcVuq+/+IJtH7m6HNPT0xk1bKjFEalY0OSgAJ3mqkLnORCdf+kQTjutuYXRqFjRAWkFRD7NVWc4JR+dwpocNDmoOoGmufqiM5yST2VlJZvWral7PkGnsCYsTQ4qYlrIL/mkp6cze+V6NsybwcE9xXR27+OgEo8mBxUxneGUnFpUlPGdgqHk5ORYHYqKIR2QVhHTGU7JSxND4tPkoCKmM5yST2lJidUhqDjRbiUVMS3kl1zen/4Wx48d47K+51gdiooDMcZYHUNE8vr1N7NXrLM6DOWHTnFNLMYYLul1Ll/s+pSMjHSWvf8O/S7oY3VYKgLSLne9MaZ/sPO05ZBg7HBR1imuiWfXzh18setTAFJTU+nZ41yLI1KxpmMOCaT2ory3rBzDqYvy3M3FcY1Di/glHs+Fb6OHDyMjI8PCaFQ8aHJIIHa5KOsU18Qz7/1ThfYmjdFV0clAk0MCsctFWae4JpaD33zDulUrABARxhXkWxyRigdNDgnELhdlneKaWBbMmUVNTQ0Agwf0o312O4sjUvGgySGB2OWiXJiXy30Te9MhKxMBOmRlct/E3joY7VDzZnp2KY22MBIVTzpbKYHYad1BuEX8lD1VVlayuGhu3XOtwpo8NDk4TLCpqpFclO0w/VXZ08olizj27bcAnHXG6Zx3ri6ASxaaHBwkFusHwnlPTSLJp97eDaMLEBELo1HxpMnBQWJRIjvU99SFbcnp9nsfpGu7Vmxcu4Yrxo+1OhwVRzog7SCxmKrq73v3eh23yxoKFV/GGL5z5ZW8N+01hl48yOpwVBxpcnCQWExVDfS9niur7bKGQsVfdlOrI1BW0OTgILGYqhroez1bBeEkprmbi7niqYUMmTKTK55aGPfyHSp60g6XWh2CsoiOOThILKaqFubl8sj0TT5f21tWzhVPLWRfWTktM9NITRGqa05V8fWVmHRsIjEcOniQr7/4nE7pkJPT0epwlAU0OThMLNYPdMjKbDDGUKv2eFl5FWmpQvPMNI6WV/lNTLqvdGKYOf0t7v35T2jfvj0P3vlzbr/5RqtDUnGmyUFxS373enf7/lRVG9o0bcKc+/yvktWxicRQuyp63759OHXPF9U4OuagfJa78CfYRd4u9Z1U5I4fO8ayhfPrnuuq6OSkLQcFNOyuuuKphT67moJd5H21QrTonrMsXTifyspKAHr26M6Zp3ezOCJlBW05KJ8inRmlRfecT/duUKAth4TWmHIXjZkZFWjQXEtw2FtNTQ3zZ79f91y7lJJXXJODiLwCTAD2GWN6uY+1Af4FnA58AXzPGHMonnElolCnlAa6WEd7ZpROc7W/jWvX8M2+fQC0z27HRRf2tTgiZZV4dyu9BozxOnYfsMAYcw6wwP1cNVIo5S587Tn9xLubGTN1XkwWsGkJDvvzLLQ3flQ+qampAc5WiSyuycEYswQ46HX4MuB19+PXgcvjGVOiCmVKqa+LdVW14Uh5VV2ymDpjS9QShE5ztb8i3dhHudlhQLqDMWYPgPvP9hbHkxBCmVIaykU5mnf2Os3V3r76/HO2f/wRAOnp6YwaNtTiiJSV7JAcQiYiN4vIOhFZd2D/fqvDsbVb8ruTmlK/9n5qitSbbRTqRbm2jEZju5rsso2p8i0jM5Of3PEL+px/HgVDL6F582ZWh6QsZIfZSntFpKMxZo+IdAT2+TvRGPMC8AJAXr/+MVm2aefZNOHGliJQ7fXcU6gro+FUGY3GDCLbaRtT1VD7nBxuvuNuHr7pB7Rp08bqcJTF7NByeA+Y7H48GXjXqkB8DdBGs889nrE9t2AHVdX182dVtanXReS9JiHLXVwvmMZ0NRXm5XJLfnfaZ2Wyr6yc5xbsCPj3qxVe46u2CmvTplqnO9nFeyrrP4HhQDsR2Q1MAaYC/xaRG4GvgO/GMyZPdi4aF25soQ7+ek9X9WydtMhM40h5VUjvE6pwtyXVqa/xl5OTY3UIygbimhyMMVf7eSk/nnH4Y+fZNOHG1t5PpdVg4wyeyeKKpxb6TQ6RDiKHk+TsnKwTzdEjRzix+1Orw1A2YoduJduw82yacGOLxuBvoKQY6SByOEnOzsk6kZw8eZJBPc7g6muv4+1//ZOKigqrQ1I2oMnBg51n04QbWzRqHPlLPC0z0yK+cw8nydk5WSeStSuWc/jgQT7eupUX/z6N9PR0q0NSNqDJwYOdi8ZZEZuvhARwpLwq4sHhcJKcnZN1IpnnsfBtYuEoRIJPSlCJzw5TWW0lFjutRUuosc3dXMyfZn9cb7wgksFcz6mn3uMXkQ4OhzOdVae+xp4xxqsKq66KVi7i1F2e8vr1N7NXrLM6jLj444wtvLP+a2qMIUWEy/t14Z6JvX2e6z3Dx1uHrEzevntk2DH4298h0vdT9vDpju0MyzsPgObNm/HNji1kZGRYHJWKJWmXu94Y0z/YedpysLk/ztjC9HVf1T2vMabuua8E4WuGj6dIB3MbMzhs54WFyc6z1VA4YrgmBlVHk4ONzd1cXC8xeHpn/dc+k0Oo23gGu2B7v94yM40yH9Nagw0O61oFe9ONfZQ/mhxsqvai6k+Nn+5Af+sb4NRgrndrxPuC7euCHuj9AtG1CvZ1YP9+1q9eCYCIMK7AFsuNlE3obCWbCtY95I+/GUZZmWnc525p+GqNeJbECOWzszLTSG+SwqPTNwWcuaRrFexrwZxZ1NTUAHDxRf3JbtfW4oiUnWjLwaaCXTxTU4S5m4sb3H0Hm+Ezduq8oJ8Z7LMzm6ZScbImpK6iSFdqq9irt3dDoc5SUvVpcrAJ7z7+QHWNAKprTL2umVAHfX2NG9SqvWAH6poCKD/RsFXhr6vIV+VXXatgD11OP4P22dns27+fiTreoLxocrABX338aalCaopQXeN/qnHdPgvnZDNzc3GjB31rL9jhlPL25KvFEcu1CjoLqnF+M/WP/Oy6q6k5coAe55xtdTjKZjQ52IC/7TqzMtPIaNok4F383rJyv2MIj0zfxHMLdtS7aLb00yLJTEupO6f2T++FdMH46yqKxcJCnQXVeAe2bkBEuKB3L6tDUTakA9IW8N6jwN/F/0h5FW/fPZIpV17gc5A5FN77Ptw1tqfPHeJ+NbFPvWOFebnMuW80LTPTQvqceHcVBZoFpULXu6uW51a+aXKIM1+b9vhTeyfuWVcpEp4XzcK8XH59eV69Gk2/vjzP7932XWN7+qxvdGX/rpbWoNJZUErFlnYrxVmoU1S978Rru2YCtTQC8bxohtPNE2jM4J6wo4genQUVub8+9SSL589jxMUD6XDN92if3c7qkJQNaXKIs0B3th3cW2cGGlz1N/tnfF4uyz/Z7zdxNOaiacdihDoLKnIz/vtvNq9fx9KF8zmvay5XXXmZ1SEpG9LkEGf+7nhDLWAX7E7eV+G9aF807TBLSCu2RmZPcTGb17sKVjZp0oQx+cOtDUjZliaHOIvGHW+gO/lYXzTtNEvIji0au5s/6/26x8MuHkSrrCwLo1F2pskhzuJxxxvLi6bWSnK2olkz6h5PLNSFb8o/TQ4WiNcdbyy6f+IxS8gO3VaJ6PixYyxbOL/uuSYHFYgmhwQVq+6fWM8SslO3VaJZunA+lZWVAPTs0Z0zT+9mcUTKzjQ5OFSwu+tQun98vUft9/p731jPEtJuq9jRvRtUODQ5OJCvu+tHpm9iy1cH6zYACtb94+s9HntnMyniKt1Re8z7rj3WYya6uC02ampqmD/71GC07hWtgtHk4ED+FtJNX/cVvbu2oTAvN2j3j6/3qK4xeL+r9z4PsR4H0MVtsbFx7Rq+2bcPgPbZ7bjowr4WR6TsTstnOFCgu+jaC7mvTX88u3/CuROvbUF4lvzwrNcUTcHiVpH57JOdZGS6EuyE0QWkpOh/fRWYthwcKNB+C7UX/WDdP8H2bPCUIhK3cQBd3BYb3732ei4+/2y2b1hDr/PPtzoc5QCaHBzolvzuPDJ9k8/XPLtfAk2Z9TWwnJoi9cYcwHXX7q8WVKzGAXRxW2xkZGRw3VXfszoM5RCaHOIg2vP2C/Ny2fLVwQb7OITT/eLvDt3XsecW7NBxAIcrLSkhtOLrSrlocoixaM3b95Vgendt06ik4+8O3dcxLXLnfNlNrY5AOYkmhxiLxrx9fwnmvom9QyrW11g6DuBcR8rKePzBexnU+zzGDRlgdTjKQTQ5xFg05u3bYWGYjgM40wfz5vDGS8/zBjBk4ACWzXzH6pCUQ2hyiLFA8/ZDHYvQhWEqUkUeq6JHDx9mYSTKaXSyc4z5m7c/5JzskNcO+Bv41QFhFUhVVRUL586qe64lM1Q4NDnEmOf+z577LS//ZL/friJv0VgYNndzMVc8tZAhU2ZyxVMLY7KATdnL2hXLKTt8GIAuuZ3I69XT4oiUk2i3Uhz46q9/1M86BV9dRd4Dwi0y0xD3ezy3YEfQwWE7VjrVstyxN2/mqS6liYWjEBELo1FOo8nBIuHWEKpNMJFc6GM9oB3uhd6OySrRGGPqjTdooT0VLu1WskikXUWBLvT+xHJAu/ZC7zl28sS7mxkzdZ7fLqxIfgYVnk93bOeLz3YBcFrz5gwfMtjiiJTTaMvBIpGuHYjkQh/LSqe+LvRV1Yaq8irAd6tAZ1/FnufeDYUjh5Genm5hNMqJbJMcROQu4MeAAbYAPzTGVFgbVWxFsnYgkgt9LDfoCeWC7t2FpWW5Y69opu4VrRrHFt1KIpIL3A70N8b0AlKBq6yNyp4i6Y7yN2PKMzFFOpsp1Au6ZxLRstyxVXb4MBvWrAIgJSWFcQX5FkeknMg2LQdcsWSKSBXQDCixOB5birQ7KlArpTEDxL5aJb54V4uN5GdQoclq1YqZy9aw4v3pVBw5RHa7tlaHpBxIjDHBz4oDEbkDeBwoB+YZY67xcc7NwM0AuV269lvzyZfxDTJBXfHUQp/dPC0z05hzX/BZLp6zlVpmpvFt5Umqa+qX/fZuqajYKi0podPxUnJycqwORdmMtMtdb4zpH+w8W7QcRKQ1cBlwBnAY+I+IXGuMecPzPGPMC8ALAHn9+tsjqyUAf5v+HCmvYu7m4rBbJbqGwXpph0tBq7CqRrDFmANQAHxujNlvjKkCpgMXWxxTUgg2tqDTS51LWw2qMWzRcgC+AgaJSDNc3Ur5wDprQ0oOwS7+e8vKGTJlZsgtAF3gZq35s2eSfvww5/XoYXUoyuFs0XIwxqwG3gI24JrGmoK7+0jFVihTUYMVBvSkC9ysY4zhvp/fwlXXXMv4iRP59LPPrQ5JOZgtkgOAMWaKMaaHMaaXMeY6Y0yl1TElg3DWFoRykdcFbtb5ePMm9hTvBuDY8XJO79rF4oiUk9kmOShr+FpzEEiwi3xjyotr5djG8VwVPa5gJE2a2KXXWDmR/vY4TLRnAnmvOWiZmYbBNVPJl2AX+UhXY+tYReN5VmHVvRtUY2lycJBIL6DBEoq/iq/eQrnIR7rAzQ5boTpZye7dbNm4AYAmTZpQOHK4pfEo59Pk4CD+LqC/fXsz4DtBhJNQfL1/rQ5htFIiqRmlYxWNM3/2+3WPhw8ZTFbLlhZGoxKBjjk4iL8LZY0xfmcShTN7yN/7C/D23SNjegevW6E2jhbaU9GmycFBAl0ow73g+zpu5QVai/FF7vixYyz/YEHdc00OKho0OThIsJlFjb3gW3mBDqVyrPJtyYIiKitdM797ndeDM7p1tTgilQh0zMFBai+Uv317MzU+Cib6u+CHOnvI6mqpkYxVqPpTWHWWkooWTQ4OU3vxjNUFXy/QznPT7XfRtllTli1ZontFq6ixTcnucOX1629mr0je8kvxrnyqlVbt7cDWDfTuqoX2VHCOKtmtwhfPO3xdoKZU8gk4IC0ifUXkZRGZISJTReQcH+dcICKfxS5EZTUtpmdvB7ZusDoElYD8JgcR6Q+sBEbimuo+GfhQRG7zOjUd6BazCJXldIGaPR0+dIg9xa61LdqlpKItUMvhcWA+cK4xZgKuBPAk8CcReVZEJB4BKmvN3VyMv39qXaBmrbem/Y3+Z3Xmqmuu5V9vv2t1OCrBBBpz6Adc496ZDWPMCeA3IrIaeBPIFZEfxCFGFUeeA88tMtMoP3HS57RZXaBmvdoprFu3baPsyFGLo1GJJtgiuAa3jMaYmbi6mobgalm0iUFcygK1A897y8rrKrNWVTdMDCkiukDNYmWHD7N62ZK65xNGF1gYjUpEgZLDVlzbdTZgjFkLXAp0AqbFIC5lgUCF9zwZYzQxWGzRvDmcPHkSgP4X5NGpo445qOgKlBxmAjeKSJavF40xO3G1Hr6KRWAq/kIdYNaxBuvpqmgVa4GSw5NAF8BvZ6YxZg8wCDgzynEpC4Ry0dexButVVVXxwbzZdc+10J6KBb/JwRhTY4w5ZoypCfQGxpgKY8yX0Q9NxZuvwnupKUJWZpoWw7ORNcuXUXb4MABdcjuR16unxRGpRKQrpFUdqwvvqdDU3w50tN+pxko1hiYHVY8W3rM3YwzzdWMfFQe6n4NSDvLJ9m188dkuAE5r3pzhQwZbHJFKVJoclHKQJmlpXPvjn5Ddrh1j8oeTnp5udUgqQYVVsttdMqMzrllMm40xx2IVWDDJXrJbJa8DWzdQU1NDt9bNaZXlc6a5Un6FWrI75JaDiPwMKAa+BJYC3d3Hp4vInZEGqpQKX97pnTQxqJgKKTmIyC+Bp4AXOVWltdYi4PtRj0wp1UBpSYnVIagkEepspVuB3xhj/iAi3jvc7wDOjW5YSilvteUysptaHIhKCqF2K+UA6/28VgNkRCccpZQvFRUVXHhGJ+6+4WqmvfU21dXBa2Ap1RihJodPgWF+XhuKq0ifUipGViz+gAP797N4yRKe/9s0UlJ0oqGKrVC7lZ4G/p+InADech9rLyI3AncDN8UiOKWUS5HXwjddFa1iLaTkYIx5SURaA78BHnEfngUcBx42xvwjRvEplfSMMfWSw6Qxoy2MRiWLkMtnGGOeFJHngMFAO+AgsNIYUxar4JRS8PHmTewp3g1A61atGDJwgMURqWQQNDmISAbwHvCEMWYRMC/WQSmlTvHcu2FcwUiaNNGSaCr2go5qGWMqgAGA9xRWpVQc1K/CqoX2VHyEOuXhPeDyWAailGqoZPdutmzcAECTJqkUjhxuaTwqeYTaPp0LPCkiHXENRO8F6hVlMsbMinJsSiW9+bPfr3s8fMjFZLVsaWE0KpmEmhzecP95pfvLm0G7nZSKOu8prErFS6jJ4YyYRgGISCvgJaAXrmTzI2PMylh/rlJ2ZYyh+WmnkZGRTkVFpSYHFVehrnOIxx7RzwBzjDHfEZGmQLM4fKZStiUiPPfGv9i9fjlH9+7mjG5drQ5JJZGQkoOInB/sHGNMxCU0RKQlrjIcN7jf6wRwItL3UypRlJaUkJmZyUVDL7U6FJVkQu1W+givAWgfGjPmcCawH3hVRPJwFfm7w8rNhJSyg7TDpVqFVVki1OQwwsexNsBo99cdUYjjQuA2Y8xqEXkGuA94yPMkEbkZuBkgt4s2sVVyyMnJsToElYRCHXNY7Oelt0XkMeB7wPt+zgnFbmC3MWa1+/lbuJKDdxwvAC+Aa5vQRnyeUrb23J/+yMcfbmLkoAGcfeUEMjMzrQ5JJZlorMP/AJjemDcwxpSKyNci0t0YswPIR8uAqyT2r7+9ys5tW5n+z2l0aZPFpLFabE/FVzSSw3jgcBTe5zZgmnum0mfAD6Pwnko5zhe7drFzm+veKCMjg4JhOhit4i/U2Ur/9nG4KdADOAd4oLGBGGM2Af0b+z5KOV3RrFML3wqGXkKzZtqlpOIv1JZDexrOVqoAlgJ3a+kMpaLHswqr7t2grBLqgPTwGMehlAIOHzrE6mVL6p5PGF1gYTQqmYVUlVVEXhERnyU0RKSbiLwS3bCUSk6L5s2huroagAF9L6BjTgeLI1LJKtSS3TcA2X5eawdMjko0SiW5+tuBai0lZZ1QkwP4XyHdC9fqZqVUI1RVVbFw7qnhOy20p6zkd8xBRO7g1MpnA7wjIpVep2UAHYDXYhKdUklkzfJlHClzbcnetXMufXoGLWmmVMwEGpDeCvwXEOBuXIvd9nidcwLYDvia6qqUCsOWjevrHk8sHIWIWBiNSnZ+k4MxpggoAhCRo8BLxpjieAWmVLK55a57GHZhb3Z9tJl+eX2sDkcluVCnsj4S60CUUtA+O5v8yddaHYZSoZfPEJHBwI3AubjGGuoxxlwUxbiUUkpZKNR1DqOAJUBn4BJcs5O+BfKAtrj2e1BKNcKBrRusDkGpOqFOZX0U1zae493PHzLGjMTViqgCFkU/NKWSw4H9+3nqsUfYum0bvbroojdlD6Emh/OB2UANrmmtzaFub+mHgQdjEZxSyWD+7Jn872MPc9U11/I/N9xkdThKAaEnhwogxRhjcE1nPcvjtSO4upuUUhEomnmq0N7g/v0sjESpU0IdkN4MdMc1tXUBcL+IFONa5/AosCU24SmV2CpAdn9KAAAgAElEQVQqKlhUNLfuuZbMUHYRasvhaU6Vz3gAOAbMxbUwrj1wa/RDUyrxrVj8AeXHjwNwzpln0P2csy2OSCmXUNc5zPJ4XCwi/YCzgUxguzHmRIziUyqh6d4Nyq7C3iZUXGv6OwKfG2NORj8kpZKDMaberm+aHJSdhFyVVUTGichqXIPTXwN93MdfFBFd0qlUmD7atJHSYldFmtatWnHxRbpLrrKPUBfBXQ+8h6vI3s24ivHV2olr5bRSKgyeXUrjR42kSZOwG/JKxUyoLYcHgSeNMZOBN7xe+xjXOgilVBjmzdTxBmVfoSaHbrgrtPpQAbSMTjhKJYeS3bv5aNNGANLS0igcOdzSeJTyFmo79mugL7DQx2v9gU+jFpFSSaBjbi5zVq7n3ddeILX6BC1btLA6JKXqCTU5vAxMEZG9wDvuYyIi+cCvcC2EU0qFSETolA4//cnN9O6aY3U4SjUQanL4PdAFeB2odh9bAaQCzxtj/hyD2JRKeJoYlF2FugjOALeKyFNAPtAOOAgsNMbsjGF8SiWk0pIS0qwOQqkAwpo7Z4zZBeyKUSxKJYUVixfRmkp65mZbHYpSfvmdrSQi80Sku9exkSLSPPZhKZWYqqurueXa71FQOIaJ1/6I0r37rA5JKZ8CTWUtALJqn4hIKq7prN39fodSKqANa1ZzYP9+AL4uKaF9djuLI1LKt5DLZ7hJ8FOUUv547t0wcfQoUlLC/S+oVHzob6ZScVQ081ShvYmFuneDsq9gycGEeEwpFcQXu3axc9tWADIyMigYdqnFESnlX7DZSnNFxLss9wIfxzDGtI9eWEolHs/y3AVDL6FZs0wLo1EqsEDJ4ZG4RaFUEtCNfZST+E0OxhhNDkpFyeFDh1i9bEnd8wmjCyyMRqngdEBaqThYNG8O1dWuyjMD+l5Ax5wOFkekVGCaHJSKg/pdSjpLSdmfbj2lVBxcdcONNK2pYvny5TqFVTmCJgel4mBofgE9O7ahZ+f2iCTZWtIdM2Dl03B0D7ToCIPvhO4TrY5KBWGr5OAu0bEOKDbGTLA6HqWi5cDWDQDJtyJ6xwxY+Bs4WeF6frTE9Rw0Qdic3X5T7wC2WR2EUrGQlHs3rHz6VGKodbLCdVzZmm2Sg4h0BsYDL1kdi1LRcvTIEcqPH7c6DOsc3RPecWUbtkkOwNO4thyt8XeCiNwsIutEZF1tZUul7Oy15/6Pnp3acs/dd7FgyVKrw4m/Fh3DO65swxbJQUQmAPuMMesDnWeMecEY098Y079ttm6UouyvaNYMKisqmLdoCSWle60OJ/4G3wlNMuofa5LhOh4LO2bAa/nw7PmuP3fMCP49yie7DEgPASaJyDggA2gpIm8YY661OC6lIvbNvn1sWL0KcA1EjysYaXFEURLO7KPa4/GYraSD31Fli+RgjLkfuB9ARIYD92hiUE63YPZMXNuvwyUDL6JtmzYWRxQFkVyAu0+Mz8U50OC3Joew2aJbSalENM9zY59EWfhm59lHOvgdVbZLDsaYRbrGQTldRUUFi+fPq3ueMCUz7HwB1sHvqLJdclAqESxftLBuCuu5Z53JuWefZXFEUWLnC3C8B78TnCYHpWIgYfdusPMFuPtEGPkotOgEiOvPkY/qeEOEbDEgrVQiMcYwf1aC7hUdz9lHkYjX4HcS0OSgVJRt2biB0pISANq0bsXFF/W3OKIoc9IFWIv+RUyTg1IxMGzoUFavWc34Ufk0aaL/zeLCOxF0Gwrb39F1DxGS2nnYTpPXr7+ZvWKd1WEo5dOBrRs4q10WZUeO6K5v8eC9/gIAAXxc31p0ghsWxCsy25F2ueuNMUGbs3pLo1SUlZaUkAY0a5ZJs2aZVoeTHHytv/CVGMAe024dQJODUjGQ3dTqCCLg5P75cC74dph26wA6lVWpKHJqN21dt8zREsCc6p93SuG6UC/4dpl26wCaHJSKkm+PHmXI+Wcz9Ze3s2HLx85KFHYuixEKf+svel2l6x4ipN1KSkXJkgVFfPn5Z3z5+Wfs3LGNcaPyrQ4pdHYuixFKd5fd1184kCYHpaLEc1W04xa+tejo7lLycdxK4VSBddL6CwfQbiWloqC6upoFc2bWPZ9U6LCSGXYti+H07i4H05aDUlGwYfUqDn7zDQA57dvT74I+FkcUJjt2y+yY4bs1A+F3dzl5JpZFNDkoFQXeezekpDiwUW5Vt4yvCzec6j7yJZzuLt0hLiKaHJSKgqKZnoX2CiyMxGH8XbibZPhY1OYWbneXHXeIc0BLRpODUo30+a5P+WT7NgAyMzPIH3qpxRE5iL8Lt7/EAOFPR7XbTCyHtGQc2PZVyl48Ww2jhg3VkhnhCPcC3aJT+BfQWGxQtGMGvJYPz57v+jOcxYIOGWTX5KBUIzl6CmukGnNx9OTvAp3eKnqzp6I9E6uxq8nt1pLxQ5ODUo1w8uRJKitP3QVOGJ0E4w3RLLXh78I97IHo7OpW27d/sgIk1XWssSulG3vnb+etVj3omINSjdCkSRNe/Od/OfTJFg4Vf0lOh/ZWhxR70RzgDTaFtjF98N59+6b6VIuh9n0jGRhu7J3/4Dsblhe3w5oSL5oclIqCnrnZ5PTrbXUY8RHtbpFYTaFd/ETgJBZsYNhf4mjsanI7rinxQZODUo2UdrgUnFiiO1J2LbXhaccMqDzs+7XaJBase8hf4ojGnb8DSn3omINSUZCTk2N1CPFj11IbngL1/9cmsUAtoGBdZ9EYD7E5bTkoFaEXn32avZ9uZ/iwofTq0gERsTqk+LBLt0ig8YJAXVy1ScxfC6h2kN2X2vd1wJ1/Y+ke0kpFwBjDxeedxVdffA7A/Olv6uK3ePrgEfjozfrHmmScuoN/Ld/PBd6dwFt0hKyusHs1frcT9SWa+09btEpa95BWKoZ2bttalxhanHYalw4aaHFECcpf3aWP/tXwXM9uH1/jAkBdIjha4r914E80u84csEpak4NSEfBc+DYmfzhNmybTiHScBKq75O9u37PbB04lFklxTWWNiET/zt6O9Z68aHJQKgKeVVgdt3dDvDS22ySSukueM6Y8xwWePT/0z633flHsRvLkgFXSmhyUCtP+vXvZuGY1AKmpqYwbNdLiiGxmxwxY8gRUeEwljaTbJJILpb9uH7+DzwHEcgaWA6YD61RWpcK0YPZMaidyDLloAG1at7Y4Ihup7Qqq8LHGINzicuFeKHtd5T/x+Jp+6y0lzVXTKRrTU4PVnnLAdGBtOSgVpnpdSmOSpNBeqHx1BXkKpzXgd1DZjxFT/L/ma/ptt6Hw5ZLozxYKZbDZLtOBA9DkoFQYysvLWTx/Xt3zSWN0vKGeYBf/cFoDvi6gVcd9t0padArt/XxdfGvHRubd6/qzsRfpUAebbb5WQpODUmFYvmghFeXlAHQ/+yzOOetMiyOymUB9+5F0m3hfQL3vyiN9X3/vF40ppQ4YbA6FjjkoFYYViz+oe6ytBh8C9u2La6C6MXtARLt0RSw23nFISe5gtOWgVBge+t2TjBjQl3XLl3LVFZdZHU74Yr0qt15XkFcL4mS56wsad4ceze6YWNzlO6QkdzCaHJQKg4hw/nnn8f3CEVaHEr54rcqtvXj7LWHhZodFX+lZvqu3pmdF/p4OGGwOhSYHpcJQWlJCmtVBRCreq3JDufsO5w49Fq0ef7USG1tD0eaDzaHQMQelwpB2uJRsp1bKiPdAaSh97OlZoe1F7Wtr0nm/chXga4yKsvCOJxFbtBxEpAvwNyAHqAFeMMY8Y21USp1SWlLC/Fnv0++cbvR26o5v8V6VG8o6hapjp7p1AnVz+Vs/UVuAL9L1CsH+TjxbK+lZrhZFRZlju4rCYZeWw0ngF8aY84BBwK0iEmExFKWib86Md7j35z+hoHAMt9//kNXhRCbeq3JrZxZJqp8TBGqq6h/yN1PIb+vGuEp3e7YoFv4m9JlQgf5OvFsrlYfdaywi+BwHskVyMMbsMcZscD8+CmwDcq2NSqlTijyqsHY/+ywLI2mEeO5gVls+Yt690LSFqzSFp1Aqq3oKp3UTzlTUQH8nwVZ7N3bKq83ZolvJk4icDvQFVvt47WbgZoDcLl3jGpdKXt8ePVpvfcOE0QUWRtNI8Rgo9Z4V5T0bKKMVDH3A93RX8J0IBt/pGmMIVTjjKP7+TqI9oO4wtmg51BKR04D/AncaY454v26MecEY098Y079tdnb8A1RJafH8eZw4cQKAvF7n061LZ4sjsrlQ7rghvG6u7hNdhfUaTCPyM60o1JZGoAJ5obyHwxa2hcM2yUFE0nAlhmnGmOlWx6NUraKZpy4YEwu10F5Qwe6mPafPhtPNNWIKjP59/fN7fT94gvGXAHzNgPIcRwhWydWBC9vCYYtuJXHtzP4ysM0Y85TV8ShVq7q6mgVzZtY91419QhDK3glHS1wX4XC7uXyd3+lC/+sfAi38C7buw3sxW5LNVrJFcgCGANcBW0Rkk/vYA8aYWRbGpBQbVq/i4DffAJDTvj39LuhjcUQOEGqp7aIHXH+Gc4H1txDO33sESgChrPtIgMVskbJFcjDGLKPxaxKVijrPvRsmFo4iJcU2PbH2Fai+kidzMrzV2ZGU/wiUABywG5uV9DddqQDqjzc4eJZSLAQazO0+0bX38m3bAr9HOLN9IqmgGqhCqgN2Y7OSJgel/Pjs00/4ZLvr4paZmUH+0EstjshGgg3megq0EU84d+mRlP/wOagsrnhXPg09Lo/Pug8HskW3klJ21KlzF55+6XWWzXqX1s0zadYs0+qQ7COcIn6D74T5DzZcDS1NwrtLj6QbyGcXl3vx3dES2P6OJgQ/NDko5UdGRgYjL+zF9wf1Iicnx+pwrOFvADicu/jaC++SJ05t8ZneCoY9EN5FOdJ9EgKVEA9WlTbW+1/YmCYHpYJwfGKI9AIXaAA43Lv4aMz6CbRPQig/Y7jdUvHa/8KmNDko5Yej926o1ZgLXKCuI6t2O/OVZHz9jPMfhMVPQKXHmoRwKrC26AhVx+O7/4XN6IC0Uj5s2biBiopy5+7dUCvSPZJ3zPA/DfXontBXNwea0RQtS55o+DPWVLlrOnkMlncbGnoF1qMlp7rAvNUu4Etw2nJQysuJEyf4buEIqk5UMmzwIP79yvO0bNHC6rAiE8kMn9oLpT+1d9rBuopCbbWE2+3leX5Glv+LuKeTFa49H0Y+6vuzXssPvmjPUxJ0LyVscqg8eoTyg/uoOVkV/GQVVylN0shs0570Fi2tDsWnVUuXcPSIq+7jjl2f0eK00yyOqBEimeETqHBeOF1HocxoCrfby/v8UBJDrdoWT6QVWD0lQfdSQiaHyqNHqDiwl9zOuWRkZOIq3aTswBhDRUU5JbuLAWyZIObPql9oz9G/P5GMDQS6UIYz7TOUVku4+1oHq/gaSKCE6C+JprdqWHK8VrD6UQ6XkGMO5Qf30alzLpmZzZz9HzsBiQiZmc3o1DmX8oP7rA6nAWMM8zw29pk0xuFVWCPZ4MfvquJO4d0pB1qdXCvcbq9I908IlhD9rZYe9oD/nez87nCXGBIyOdScrCIjQxcs2VlGRqYtu/x2bP2Yr7/8AoCWLVowdPAgawOKhrpSFltdfwa7wEerrEQo7xNKAgnleDDBEmKgJGqqfX+Pv+PhiMeAfYQSslsJ0BaDzdn138ez1TBm5HCaNnX6dKUIBFpPEO33CbfbK9wd4cC181wosfsbj2jRyc+4TYCyIKGw+TqKhE0OSkXCswrrpDFJvHdDtEpVB3ufcBNR94nhJwc/W1WHLFZrOsIdb4mzhOxWSgZ/e+010lOFb7/91upQKCsr46Yf/ZAObVuT3TqLyddew4EDB6wOK2z7SkvZtHYNAKmpqYwtGGFxRDYTqy6QcLu9wr1jryyLPDaIbNwmFNEeV4kybTmoRrvmqu+zc+cOnnvhJVJSUnjg/nv57pWXs3DxUqtDC8uC2TMxxnWbecnAi2jTurXFEdmInbpA/N3Jp2b4nlkUjf0ZYrHpj833k9DkoBpl1cqVFM2by/wPFnPp0KEAdMrN5ZLBA1kwfz75Bc7ZA6FoVpz3bnBSUTc7dYH464oCa0p6RMqqEiQh0m4lG1u6ZAmj80fQpuVpZLfOYtTI4WzauNHv+Q/efx8X5vWmTcvTOLNrZyZfew2lpaX1zpnx3nsMGtCP1i2a06Ftay4ZPJAlixfXvf7qyy9zQe+eZDXPpFP7dhSMGMbWjz/2+5lz58ymQ4cOdYkBYMBFF3H6GWcwd87syH94C4y7/H8YNnQoGRkZsR9vCGc/BDuwUxdIoK1CY9H9Eys2j1dbDkG8t7GYp4p2sOdwOR1bZXL3qO5M6psb889dvGgR4wpHMWz4CF5+9XWaNW/OyhXLKS4u5oK+fX1+z/59+7j3vgfo2KkT3+zfz9N/+l8KC0ayYfMWUlNT2bVrF1d/7zv8/PY7+N3vn6SiooKNG9Zz6OBBwJWMfv6zW5jyyKMMHDSYI0eOsHrVSsrK/PfZ7ti+ne49ejQ43qPHeezYvj06fxlxcsmIfEb07cnZ2VlkZsZ4KrSd7sRDYZcukGDdW07b89nG8WpyCOC9jcX8+p0tVFS55jOXHC7n1+9sAYh5gnjowfvpk5fHzDlz66Z9Fo4ZE/B7Xnj5lbrH1dXVDBw8mDO7dmbF8uVcOnQomzdupEWLFkz9w5N1540dN67u8dq1a+jdpw+/uu/+umMTJ00K+JmHDh0iK6tVg+OtW7fm888/C/xD2lB2U2KfGMBed+KhsEsXiNOSqoNpt1IATxXtqEsMtSqqqnmqaEdMP/fYsWOsWb2aa6+fHNZ6gDmzZzPskovJbp1Fs6ZNOLNrZwA+2bkTgJ69e1NWVsaNN0ymaN48jh07Vu/78/IuYNPGjdxz910sXbKEEydOhPS5vmI0xth2LYM/aYdLg58ULdFe5BVrdqnC6rSk6mCaHALYc7g8rOPRcujQIYwxdMwJ/UKxbu1a/ufySeTmduaV1//OkuUrWbpiFQAVla47re7du/PW2+/y+eefcdmEcXRq347rr/kB+/fvByC/oIAXX36VZUuXMGrkcDq1b8dtt/6sQRLx1Lp1a8rKGs4QOXz4sM8WhR2Vl5fXzVKK28Y+TtzcPtiU02iOo/hLMnZNqjZe6RwpTQ4BdGzlu3vB3/Foad26NSkpKewpDf1u6N133iY7O5tpb/6LiZMmMXDQIDr4uNCNGz+ehYuXsmf/AZ5/8WUWLpjPXbffVvf6dZMns2rter7es5ff/f5J/v76azzx2G/9fm73Hj18ji3s2OF7LMKO/u/JqfQ/qzO/ffwJNn74UXw+1OaDkRGJdO8Ib4GSjB2TqtMmF4RIk0MAd4/qTkZa/eJaGWmp3D2qe0w/t3nz5lw0cCDT/v63ujvaYMrLy0lLS6vXlfPmP6b5PT8rK4urfvADJl1+Bdu2bW3wenZ2Njf95CcMueRStm1t+HqtwjFjKS0tZfmyZXXH1q9bx+effUbhmLEhxW61eTPfo7SkhP/897/s+uKL+H1wuIu/7C5aXT7BxhXsllSjlRRtRgekA6gddLZittJjT0xl7OgCJo4by49vuplmzZuzetVKLuzXn/ETJjQ4P79gFM8+8zS/uOtOxk+YyKqVK/jHtDfqnfPi88+zetVKRheOoWOnTnz6ySdMf+s/XHPd9QA8+vAUDh48yLBhw2nbrh2bNm1k6ZLFPPa7qX7jHDR4MKNGF/KjG67n93/4I5KSwoP338uQSy5xxBqH4q++4uPNmwBo2rQphSOGWxuQk0VrRlOwJGO3GT4JOg6iySGISX1z45IMvF06dCiz5hbxyJSHuOH6a2natCkX9O3LpMsu93n+2HHjeHzq7/nrX57llZdeZODgwbz93vv06nFu3Tm9+/Th/Rnv8at77ubgwYPkdOzIj358E1MeeRSAfv0H8Odn/sR//vUmR48epWu3bjw05WFuu/2OgLG+8c83+eXdd3Hzj39ETU0N48ZP4Kln/hy9v4wYKpr9ft3j4UMG06KFgzf2sVq0ZjTZZdpsqJwWb4gk1G4Lu8nr19/MXrHO52sHd23j3B7nxTkiFa6d27fR5ixr/52umTiGRUVzAfjL7x/n1htvsDQex4vGqm/vtQzgSjJWdx958vw507Og6phr3+padovXg7TLXW+M6R/sPG05qKT17dGjrFj8Qd3ziYUO39jHDqLR5ROtkuGx4p28Kg+DNHGVBq8os1+8EdLkoJLW4vnz6tZy5PU6n66d4999qPyw27iCJ18D0OYkpDWDm1ZaE1MM6GwllbTqbQdamMR7N6jwJOgAtDdtOaikVF1dzYI5M+ueJ/XGPskkGmMiCToA7U1bDioprV+1kkPuDYk65eRwYV5viyNSMRetxWp2XIgXA5ocVFKqqKjgvF59AJgwOp+UFP2vkPCitVjNjgvxYkC7lVRSGppfwBvvzSbliy201h3fkkM0xwrsPGAeJXq7pJJW2uFSctpn061LZ6tDUfFg16J9NqXJQSW1uFVhVdZLkrGCaNFuJZWUDmzdYHUIKt6csLjORrFpy8Gh/vbaa6SnCt9++62lcZw4cYL7fvVLRg67lKzmmaSn2nuDn0MHDzLh0oH89fkXOHn4G6vDUfFm10q4Niz7bZvkICJjRGSHiHwqIvdZHY8KzfHjx3n15ZfIzGzGoIsvtjqcoD6YO5uNa9fw1+ef56e/1F8zZSHPDYKK7rdd2W9bJAcRSQX+DxgLnA9cLSLnWxuVCkWrVq0o/eYgM+fM5bLLr7A6nKDmzfRYFa0L35RVvFsKptr3eRauurZFcgAuAj41xnxmjDkBvAlcZnFMAKRs+Tdpz/Qi7dFWpD3Ti5Qt/47bZy9dsoTR+SNo0/I0sltnMWrkcDZt3Oj3/Afvv48L83rTpuVpnNm1M5OvvYbS0vr7Is947z0GDehH6xbN6dC2NZcMHsiSxYvrXn/15Ze5oHdPsppn0ql9OwpGDGPrxx8HjNMpe0WfOHGCRfPm1D3XQnvKMr7WXPhi4UwquwxI5wJfezzfDQy0KJY6KVv+Ter7tyNV7j2jy74m9f3bAajp/b2YfvbiRYsYVziKYcNH8PKrr9OseXNWrlhOcXExF/Tt6/N79u/bx733PUDHTp34Zv9+nv7T/1JYMJINm7eQmprKrl27uPp73+Hnt9/B737/JBUVFWzcsJ5DBw8CrmT085/dwpRHHmXgoMEcOXKE1atWUlZWFtOfNV5WLV3C0SNHADi9axd6neeMbUxVAgqlRWDxTCq7JAdft54NNpoQkZuBmwFyu3SNdUykLnz0VGKojaGqnNSFj8Y8OTz04P30yctj5py5dXfmhWPGBPyeF15+pe5xdXU1AwcP5syunVmxfDmXDh3K5o0badGiBVP/8GTdeWPHjat7vHbtGnr36cOv7ru/7tjESZOi9SNZbv6sU4N7k8aMdkyLRyUgf/WZJBVMjc5W8rAb6OLxvDPQ4G/OGPOCMaa/MaZ/2+zs2EdVtju841Fy7Ngx1qxezbXXTw7rAjZn9myGXXIx2a2zaNa0CWd2dS3u+mTnTgB69u5NWVkZN94wmaJ58zh27Fi978/Lu4BNGzdyz913sXTJkrpy1onAGFOvCuvEQvtvYaoSmL81F6N+Z5uZVHZJDmuBc0TkDBFpClwFvBfke2Ivy8/KWX/Ho+TQoUMYY+iYE3p/47q1a/mfyyeRm9uZV17/O0uWr2TpilUAVFS6+ja7d+/OW2+/y+eff8ZlE8bRqX07rr/mB+zfvx+A/IICXnz5VZYtXcKokcPp1L4dt936swZJxIm2f/wRX3/5BQAtW7Rg6OBB1gakkpsD6jPZolvJGHNSRH4OzAVSgVeMMYFHQeOgeuRv6o85ACYtk+qRv4np57Zu3ZqUlBT2lIY+U+Hdd94mOzubaW/+q6618eWXXzY4b9z48YwbP56ysjJmz5zJPXffyV2338Yb/3wTgOsmT+a6yZPZv38/70yfzi9/cRctW7bk8d9Njc4PZ5Gimae6lMaMHE7Tpk2tC0YpsH19Jru0HDDGzDLGnGuMOcsY87jV8YBr0Ll6wp8xWV0wCCarC9UT/hzz8YbmzZtz0cCBTPv73wh1j+/y8nLS0tLqdUO9+Y9pfs/Pysriqh/8gEmXX8G2bVsbvJ6dnc1NP/kJQy65lG1bG77uNDqFVanw2KLlYGc1vb8X82Tgy2NPTGXs6AImjhvLj2+6mWbNm7N61Uou7Nef8RMmNDg/v2AUzz7zNL+4607GT5jIqpUr+Me0N+qd8+Lzz7N61UpGF46hY6dOfPrJJ0x/6z9cc931ADz68BQOHjzIsGHDaduuHZs2bWTpksU8FqTVMGf2bI4fO8aHmzYBMP2ttwDoN2AA3bp1i8ZfR6OUHz/O3hLXEFZqaipjC0ZYHJFS9qfJwaYuHTqUWXOLeGTKQ9xw/bU0bdqUC/r2ZdJll/s8f+y4cTw+9ff89S/P8spLLzJw8GDefu99evU4t+6c3n368P6M9/jVPXdz8OBBcjp25Ec/vokpjzwKQL/+A/jzM3/iP/96k6NHj9K1WzcemvIwt91+R8BYb7/1p/W6sK7+/ncBePHlV7n+hhsa+TfReJnNmrHmky9ZNv0ffLu/lDZaolupoCTUbgu7yevX38xesc7nawd3bePcHufFOSIVrp3bt9HmrPj8O9UW2uvdVauwquQm7XLXG2P6BzvPNmMOSsWaJgalQqfJQSmlVAM65qAS2mvP/z+OHzvGyH59QFsOSoVMk4NKWDU1Nfx56mPs3bOHx4H1C+ZwYV5vq8NSyhEStlvJqQPtySIe/z4fbljP3j2uhYRt27Qmr5dWgVcqVAmZHFKapFFRUR78RGWZiopyUpqkxfQzPGspTRhdQGpqakw/T6lEkpDJIbNNe0p2F1NeflxbEDZjjKG8/Dglu4vJbNM+pp9V5FGFVfduUCo8CTnmkN6iJQDFxTIFGOQAAAgtSURBVCXUnKyyOBrlLaVJGpltO9T9O8XC7i+/ZOuHmwFo2rQpo4cPi9lnKZWIEjI5gCtBxPLio+xt/uz36x6PuORiWrQ4zcJolHKehOxWUspzvGHSGO1SUipcmhxUwjl65AgrFn9Q93zCaE0OSoVLk4NKOIvnz6OqyjXWdEHvnnTtnGtxREo5jyYHlXAWzplV93hSoe7doFQkEnZAWiWvx5/5Py7tl8eqZUu4csJYq8NRypE0OaiEk5mZydBLL+XWa75rdShKOZZj93MQkf1A7Q4z7YBvLAynsZwcv5NjB2fH7+TYwdnxOzn2bsaY7GAnOTY5eBKRdaFsXmFXTo7fybGDs+N3cuzg7PidHHuodEBaKaVUA5oclFJKNZAoyeEFqwNoJCfH7+TYwdnxOzl2cHb8To49JAkx5qCUUiq6EqXloJRSKooSKjmIyG0iskNEPhaRP1gdT7hE5B4RMSLSzupYwiEiT4rIdhH5UETeFpFWVscUjIiMcf+ufCoi91kdTzhEpIuIfCAi29y/63dYHVO4RCRVRDaKyPvBz7YXEWklIm+5f+e3ichgq2OKhYRJDiIyArgM6GOM6Qn80eKQwiIiXYBRwFdWxxKBIqCXMaYPsBO43+J4AhKRVOD/gLHA+cDVIuKkPURPAr8wxpwHDAJudVj8AHcA26wOIkLPAHOMMT2APJz7cwSUMMkB+Ckw1RhTCWCM2WdxPOH6E/ArwHGDQMaYecaYk+6nq4DOVsYTgouAT40xnxljTgBv4rqxcARjzB5jzAb346O4Lk6OqS4oIp2B8cBLVscSLhFpCQwFXgYwxpwwxhy2NqrYSKTkcC5wqYisFpHFIjLA6oBCJSKTgGJjzGarY4mCHwGzrQ4iiFzga4/nu3HQxdWTiJwO9AVWWxtJWJ7GdSNUY3UgETgT2A+86u4We0lEmlsdVCw4qraSiMwHcny89CCun6U1rmb2AODfInKmscl0rCCxPwDYunxooPiNMe+6z3kQV5fHtHjGFgHxccwWvyfhEJHTgP8CdxpjjlgdTyhEZAKwzxizXkSGWx1PBJoAFwK3GWNWi8gzwH3AQ9aGFX2OSg7GmAJ/r4nIT4Hp7mSwRkRqcNU/2R+v+ALxF7uI9AbOADaLCLi6ZDaIyEXGmNI4hhhQoL97ABGZDEwA8u2SkAPYDXTxeN4ZKLEoloiISBquxDDNGDPd6njCMASYJCLjgAygpYi8YYy51uK4QrUb2G2MqW2pvYUrOSScROpWegcYCSAi5wJNcUBhLGPMFmNMe2PM6caY03H98l1op8QQjIiMAe4FJhljjlsdTwjWAueIyBki0hS4CngvyPfYhrjuIl4GthljnrI6nnAYY+43xnR2/65fBSx0UGLA/f/yaxHp7j6UD2y1MKSYcVTLIYhXgFdE5CPgBDDZAXewieIvQDpQ5G79rDLG3GJtSP4ZY06KyM+BuUAq8Iox5mOLwwrHEOA6YIuIbHIfe8AYMyvA96jouQ2Y5r6x+Az4ocXxxISukFZKKdVAInUrKaWUihJNDkoppRrQ5KCUUqoBTQ5KKaUa0OSglFKqAU0OKiGIyMPuirbeX/Nj8FmjReTOaL9vpERklIj8U0S+cP/MD1sdk3K+RFrnoFQZMMbHsWgbDXwHV40gOxgD9AEW4FpYplSjaXJQieSkMWaV1UGES0QyjTHljXiLXxpjfuF+L8dUl1X2pt1KKmmISIqI3Ofe4KdSRHa6a0J5njNeRIpEZJ+IHBGRVSIy2uP1h4FfAN08uq5ec7+2SETe8nq/4e5zermfn+5+fo2I/E1EDgMzPM7/sXsDn0oR+VJEfhXs5zLGOLG6qbI5bTmohCIi3r/T1R5lVJ4FJgOPAhtwba70iogcMMbU7kh2Bq6L9R9xlZQeC8wWkaHGmOW49iA4B1cdryvc3xNJccc/AtOB7wLV7th/CTwB/AFYBPQDfisix40xf4ngM5SKmCYHlUjaAlVex0YB80XkbFwbQv3QGPO6+7X5ItIRmAK8D+B5ERaRFOADoCdwI7DcGLNbRPYAlY3swlpljLnV47NauuN4zBjziPtwkYg0A34tIn81xlQ34vOUCot2K6lEUoZrLw/Pr9rSyvm4WgJvi0iT2i9cg7gXuLcORUQ6i8jrIlKMa2+KKlwD0OdGOdaZXs8HA82B/3jFtxDogP1311MJRlsOKpGcNMas8/NaO1wVWP3NXuooIiW4Sne3AH4DfAocw9UN1T7Kse71ER+Av+qwXYAvoxyDUn5pclDJ4iCulsAQfG9PuQ84G9eWm2ONMXNqXxCRzBA/owLXPiKe2vg517sc8kH3nxNomDgAdoQYg1JRoclBJYuFuFoOWcaYIl8neCSBSo9j3XAllA89Tj2Baxczb7txbT7vaVSI8a0EyoFOxhjvLiel4k6Tg0oKxpgdIvIc8KaI/AFYh+sC3xM41xjzY2A7rgv8/4rIQ7i6lx4Bir3ebjvQQURuAD4CvjHGfAG8DdwoIn/CNaYwAigMMb7D7mmyz7gT0hJcY4LnAiOMMVf4+173+QPcT5sC54vId4BjxpjZoXy+Ug0YY/RLvxz/BTyM6yId6BwB7sTVr1+JawrqYuB6j3MGAGtw3cV/AtwAvAas8zgnA3gVV1eUAV7zeO1+4GvgKPAGMMl9Ti/366e7n0/wE+O1wHr35x/CNaB+d5Cf6wb3e3p/fWH1v4t+OfdLd4JTSinVgE5lVUop1YAmB6WUUg1oclBKKdWAJgellFINaHJQSinVgCYHpZRSDWhyUEop1YAmB6WUUg1oclBKKdXA/we/jgT4R97sZQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#visualize prediction boundary on blobs dataset\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(6,6))\n",
    "\n",
    "fontsize = 15\n",
    "\n",
    "#----------------------------------------------------------------\n",
    "### VISUALIZE BLOBS TRAINIG DATASET (with correct classification)\n",
    "\n",
    "\n",
    "#plot the points of the first class:\n",
    "mask = y_blob_train == 0\n",
    "\n",
    "ax.plot(X_blob_train[mask][:,0], X_blob_train[mask][:,1], 'o', label='class 0') #plot first feature on x-axis and second feature on y-axis\n",
    "\n",
    "\n",
    "#plot the points of the second class:\n",
    "mask = y_blob_train == 1\n",
    "\n",
    "ax.plot(X_blob_train[mask][:,0], X_blob_train[mask][:,1], 'o', label='class 1') #plot first feature on x-axis and second feature on y-axis\n",
    "\n",
    "#----------------------------------------------------------------\n",
    "### Visualize perceptron's prediction field\n",
    "\n",
    "# generate square mesh of points to evaluate, with \"n_pts_side\" ponits per side\n",
    "n_pts_side = 500\n",
    "pad = 1\n",
    "\n",
    "#extrema change based on generated points\n",
    "xx_min = X_blob[:,0].min()-pad\n",
    "xx_max = X_blob[:,0].max()+pad\n",
    "\n",
    "yy_min = X_blob[:,1].min()-pad\n",
    "yy_max = X_blob[:,1].max()+pad\n",
    "\n",
    "xx = np.linspace(xx_min, xx_max, n_pts_side)  \n",
    "yy = np.linspace(yy_min, yy_max, n_pts_side)\n",
    "\n",
    "#plot decision line\n",
    "ax.plot(xx, (-b-w[0]*xx)/w[1], 'k--', linewidth=3)\n",
    "\n",
    "\n",
    "# plot decision field\n",
    "xx, yy = np.meshgrid(xx, yy)\n",
    "\n",
    "zz = perceptron_predict(w, b, np.vstack([xx.ravel(),yy.ravel()]).T) #generate predictions for each point in field\n",
    "zz = zz.reshape(xx.shape)\n",
    "\n",
    "ax.contourf(xx, yy, zz, levels=2, colors=('deepskyblue','k','orangered'), alpha=.1)\n",
    "\n",
    "#----------------------------------------------------------------\n",
    "\n",
    "ax.set_xlim(xx_min, xx_max)\n",
    "ax.set_ylim(yy_min, yy_max)\n",
    "\n",
    "\n",
    "ax.legend(fontsize=fontsize)\n",
    "\n",
    "ax.set_xlabel('Feature 1', fontsize=fontsize)\n",
    "ax.set_ylabel('Feature 2', fontsize=fontsize)\n",
    "\n",
    "ax.set_title('Blobs', fontsize=fontsize);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Perceptron on MOONS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of training errors:\t24 (16.0 %)\n"
     ]
    }
   ],
   "source": [
    "#Train the perceptron on the moons dataset, with default parameters\n",
    "w, b, training_errors = perceptron_train(X_moon_train, y_moon_train)\n",
    "\n",
    "#number of training errors in last epoch\n",
    "print('Number of training errors:\\t{} ({} %)'.format(training_errors[-1], training_errors[-1]/X_moon_train.shape[0]*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of testing errors:\t14 (28.000000000000004 %)\n"
     ]
    }
   ],
   "source": [
    "#evaluate performance of trained perceptron on test set\n",
    "y_pred = perceptron_predict(w, b, X_moon_test)\n",
    "\n",
    "test_errors = np.sum(y_pred != y_moon_test)\n",
    "\n",
    "print('Number of testing errors:\\t{} ({} %)'.format(test_errors, test_errors/X_moon_test.shape[0]*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAGKCAYAAAAmB8cMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8lNXZ//HPlRAgAglrwiruWBFBRZG6oagoimKf1lrbR2yr1qdaqz5t1dqKWtu6Pdaltu5ba2urP7W4UKpSxapYcaGIiruyJQECYUuAJOf3x8yEmWSWe+aefb7v1ysvZu77zMyZIbmvOedc5xxzziEiIpKqslxXQERECpsCiYiI+KJAIiIiviiQiIiILwokIiLiiwKJiIj4okAikgIzu8LMXNjPZjNbZGZnh5WZFDy3d5LP/YKZPZr+WotkRrdcV0CkgDUBxwZv9wKmAXeY2Ubn3J9yVy2R7FIgEUldq3Nuftj9583sy8B0QIFESoa6tkTSawNQEeukme1gZreYWZ2ZtZjZ62Z2TIyyZ5vZZ2bWbGZPm9mwTucvNbOPgs9Tb2Z/N7PBaX4/IgmpRSLig5mF/oZ2AE4EDge+E+chdwXL/RT4CDgLeNrMjnDO/Sus3ERgFHAR0BO4FngCOCD4uqcHn+NiYDEwADiSQBebSFYpkIikbgCwrdOxW5xzD0YrbGZfAr4BfNs590Dw2BzgP8DPgSlhxWuALzvnPg+W+xz4l5kd65z7O3Ag8A/n3O/CHvNYGt6TSNLUtSWSuiYCLYQDgEOAHwIzzGxmjPIHAAY8EjrgnGsP3j+kU9k3Q0EkWO5loIFAAAF4G5hqZlea2YFmVp6G9yOSErVIRFLX6pxbEHb/ZTOrAH5lZrdGKT8E2Oic29zpeD2wg5n1cM5tCR5riPL4huBzANwL9AHOBi4H1pjZ74ErnHNtKb4fkZSoRSKSXu8C3YFdo5xbCfQ2sx06Ha8FNocFEQh0bXVWE3wOnHPtzrnfOOe+BOwI3EBgzOQsn/UXSZoCiUh6hSYfLo1y7nXAAV8NHTAzC97/V6ey+5nZjmHlDiYQSP7d+Umdc0udc9cQGLzfy1ftRVKgri2R1HUzs4OCt7sD+wM/A/7mnKszsz3DCzvn3jOzPwO/NbMqtmdt7Qn8T6fnbgCeMrMr2J619WZwoB0zuwNoBOYTGKs5AtidQBaXSFYpkIikrhp4NXh7G/A5cDtwdZzHnEUgKPwc6AssAk7olPpL8HmfA24CBgEvEBgPCT9/FvA9AoHmI+As59wTqb8dkdSYttoVERE/NEYiIiK+KJCIiIgvCiQiIuKLAomIiPiiQCIiIr6URPpv/4ED3YiRO+W6GiIiBeU/b76x2jk3KFG5kggkI0buxOxXFiQuKCIiHYb1tM8Tl1LXloiI+KRAIiIiviiQiIiILwokIiLiiwKJiIj4UhJZWyKSG1s2rKe5sYH21s5b20uulXWroLJ/DT36VPl+LgUSEcmILRvW07KmnmHDh9GzZyWBPbwkHzjnaGlpZsWy5QC+g4m6tkQkI5obGxg6fBiVlTsoiOQZM6OycgeGDh9Gc2OD7+dTIBGRjGhv3UbPnpW5robE0bNnZVq6HRVIRCRj1BLJb+n6/1EgERERXxRIRETEFwUSEZEUPXj//fQoNzZu3JjrqtDU1MRZ3/k2tQP6MahfNTO+9U3WrFmTlddWIBERKQLfPPXrvPjiC9x+593cfe/9LFjwOl/7yvSsvLbmkYiIFLj5r77Ks/+Yw3P/fJFDDzsMgKHDhnHIxAk8/9xzTD7qqIy+vlokIpL3Zr21nEnXzWXUT59m0nVzmfXW8qy99kvz5nHM5CPoX9WbQf2qOfrISbz91lsxy1926SXsN3YM/at6s8uOw5nxrW9SV1cXUebJWbM46ID96denF7UD+nHIxAnMe/HFjvP33XMP48aMprpXJUNrBnLUEYfz7uLFMV9zzt9nU1tb2xFEAA448EB22nln5vx9dupv3iO1SEQkr816azk/e2IRLdvaAFixrpmfPbEIgBP3HZbR137xhReYOuVoDp90BPfc9wA79OrFq6+8zPLlyxm3775RH7OqoYGLL/kpQ4YOZfWqVdz0m/9jylFH8ubCRZSXl/Pxxx/zjVO+ynnn/5BfX3s9LS0tvPXmG6xtbAQCgeu875/DzCuvYsJBE1m/fj2vzX+VpqammPVc8v77jNpzzy7H99zzSyx5//30fBhxKJCISF678dklHUEkpGVbGzc+uyTjgeTnl13KPmPH8vTf53TMuZhy7LFxH3PnPfd23G5ra2PCxInssuNwXnn5ZQ497DAWvvUWffr04Zrrru8od9zUqR23X3/934zZZx9+csmlHcemnXhi3Ndcu3Yt1dV9uxzv168fn376Sfw3mQbq2hKRvLZyXXNSx9Nl06ZN/Pu11/jW6TOSmrj399mzOfyQLzOoXzU7dO/GLjsOB+DDDz4AYPSYMTQ1NfHdM2bw7D/+waZNmyIeP3bsON5+6y1+dNGFvDRvHlu3bvX0utHq6JzLyqRQBRIRyWtD+kZfZiXW8XRZu3YtzjmGDB7i+TELXn+d/5p+IsOGDefeB/7AvJdf5aVX5gPQsqUFgFGjRvHo43/j008/4aQTpjK0ZiCnf/M0Vq1aBcDko47irnvu418vzePoIycxtGYgPzj3+10CTrh+/frR1LSuy/F169ZFbamkmwKJiOS1i44eRc+K8ohjPSvKuejoURl93X79+lFWVsbKupWeH/O3Jx5n0KBBPPTwX5h24olMOOggagcP7lJu6vHHM/fFl1i5ag133HUPc59/jgvP/0HH+f+eMYP5r7/B0pX1/Pra6/nDA/fzq6t/EfN1R+25Z9SxkCVLoo+dpJsCiYjktRP3HcbV08cwtG8lBgztW8nV08dkfHykV69eHDhhAg/94UGcc54e09zcTEVFRUR30sN/eihm+erqak497TROnH4y7733bpfzgwYN4qzvfY+DDzmU997tej5kyrHHUVdXx8v/+lfHsTcWLODTTz5hyrHHeaq7HxpsF5G8d+K+wzIeOKK5+lfXcNwxRzFt6nGcedbZ7NCrF6/Nf5X99h/P8Sec0KX85KOO5tabb+J/L7yA40+YxvxXX+FPD/0xosxdd9zBa/Nf5ZgpxzJk6FA++vBDHnv0Eb7536cDcNUVM2lsbOTwwycxYOBA3n77LV6a9yJX//qamPU8aOJEjj5mCt8543Suve4GrKyMyy69mIMPOSTjc0hAgUREJKZDDzuMZ+Y8y5Uzf84Zp3+L7t27M27ffTnxpOgzxo+bOpVfXnMtv//trdx7911MmDiRx2c9xd577tFRZsw++/DUk7P4yY8uorGxkcFDhvCdM89i5pVXAbD/+AO45ebf8MhfHmbDhg3sOHIkP595BT84/4dx6/rHPz/Mjy+6kLPP/A7t7e1MPf4Ebrz5lvR9GHGY1yZbIRu7/3g3+5UFua6GSElp/Pg99tjzS7muhiTwwfvv0X/X6P9Pw3raG8658YmeQ2MkIiLiiwKJiIj4okAiIiK+KJCIiIgvCiQiIuJL3gUSM7vXzBrM7J0Y5yeZWZOZvR38uTzbdRQRke3ycR7J/cBvgQfjlHnJOdd1NpCIiGRd3rVInHPzgMZc10NERLzJu0Di0UQzW2hms81sdLQCZna2mS0wswVrgqtqiohI+hViIHkTGOmcGwvcCjwRrZBz7k7n3Hjn3PgBgwZltYIiUhoevP9+epQbGzduzGk9tm7dyiU/+TFHHn4o1b0q6VGe+T1IwhVcIHHOrXfObQzefgaoMLOBOa6WiEjObN68mfvuuZvKyh046MtfzvrrF1wgMbPBFlyj2cwOJPAe1uS2ViIiudO3b1/qVjfy9N/ncNL0k7P++nkXSMzsz8CrwCgzW2Zm3zWzc8zsnGCRrwLvmNlC4BbgVFcKK0+KlLCyRX+l4ua9qbiqLxU3703Zor9m7bVfmjePYyYfQf+q3gzqV83RR07i7bfeiln+sksvYb+xY+hf1ZtddhzOjG99k7q6uogyT86axUEH7E+/Pr2oHdCPQyZOYN6LL3acv++eexg3ZjTVvSoZWjOQo444nHcXL45bz2xsqRtL3qX/Oue+keD8bwmkB4tICShb9FfKnzof2xbco71pKeVPnQ9A+5hTMvraL77wAlOnHM3hk47gnvseYIdevXj1lZdZvnw54/bdN+pjVjU0cPElP2XI0KGsXrWKm37zf0w56kjeXLiI8vJyPv74Y75xylc57/wf8utrr6elpYW33nyDtY2BZNWX5s3jvO+fw8wrr2LCQRNZv349r81/laampoy+Vz/yLpCIiIQrn3vV9iASZNuaKZ97VcYDyc8vu5R9xo7l6b/P6fjGP+XYY+M+5s577u243dbWxoSJE9llx+G88vLLHHrYYSx86y369OnDNddd31HuuKlTO26//vq/GbPPPvzkkks7jk078cR0vaWMyLuuLRGRCE3LkjueJps2beLfr73Gt06fkVS30d9nz+bwQ77MoH7V7NC9G7vsOByADz/4AIDRY8bQ1NTEd8+YwbP/+AebNm2KePzYseN4+623+NFFF/LSvHls3bo1fW8qQxRIRCS/VQ9P7niarF27FuccQwYP8fyYBa+/zn9NP5Fhw4Zz7wN/YN7Lr/LSK/MBaNnSAsCoUaN49PG/8emnn3DSCVMZWjOQ0795GquC890mH3UUd91zH/96aR5HHzmJoTUD+cG53+8ScPKJAomI5LW2Iy/HVVRGHHMVlbQdmdll9vr160dZWRkr61Z6fszfnnicQYMG8dDDf2HaiScy4aCDqB08uEu5qccfz9wXX2LlqjXccdc9zH3+OS48/wcd5/97xgzmv/4GS1fW8+trr+cPD9zPr67+RVreVyYokIhIXmsfcwptJ9yCqx6Bw3DVI2g74ZaMj4/06tWLAydM4KE/PIjXxNDm5mYqKioiusIe/tNDMctXV1dz6mmnceL0k3nvvXe7nB80aBBnfe97HHzIobz3btfz+UKD7SKS99rHnJLxwBHN1b+6huOOOYppU4/jzLPOZodevXht/qvst/94jj+h67qxk486mltvvon/vfACjj9hGvNffYU/PfTHiDJ33XEHr81/lWOmHMuQoUP56MMPeezRR/jmf58OwFVXzKSxsZHDD5/EgIEDefvtt3hp3otc/etr4tb177Nns3nTJv7z9tsAPPboowDsf8ABjBw5Mh0fR0wKJCIiMRx62GE8M+dZrpz5c844/Vt0796dcfvuy4knTY9a/ripU/nlNdfy+9/eyr1338WEiRN5fNZT7L3nHh1lxuyzD089OYuf/OgiGhsbGTxkCN858yxmXnkVAPuPP4Bbbv4Nj/zlYTZs2MCOI0fy85lX8IPzfxi3ruef+z98/vnnHfe/8fWvAXDXPfdx+hln+Pwk4rNSmMs3dv/xbvYrC3JdDZGS0vjxe+yx55dyXQ1J4IP336P/rtH/n4b1tDecc+MTPYfGSERExBcFEhER8UWBREREfFEgERERX5S1JXlnzsLl3P78EhqamqmpruScyaOYMnZYrqslIjEokEhembNwOdc8uYgt29oAqG9q5ponFwEomBQg51xOlzeX+NKVtauuLckrtz+/pCOIhGzZ1sbtzy/JUY0kVWXdKmhpaU5cUHKmpaWZsm4Vvp9HgUTySkNT9AtPrOOSvyr717Bi2XKamzen7ZuvpIdzjubmzaxYtpzK/jW+n09dW5JXaqorqY8SNGqqK6OUlnzWo08VAMuXr6C9dVuOayOdlXWroHJAbcf/kx8KJJJXzpk8KmKMBKBHRTnnTB6Vw1pJqnr0qUrLhUrymwKJ5JXQgLqytkQKhwKJ5J0pY4cpcIgUEA22i4iILwokIiLiiwKJiIj4okAiIiK+KJCIiIgvCiQiIuKLAomIiPiiQCIiIr4okIiIiC8KJCIi4osCiYiI+KJAIiIiviiQiIiILwokIiLiiwKJiIj4ov1IJC/NWbg8rZtbpfv5RGQ7BRLJO3MWLo/Ybre+qZlrnlwEkNLFP93PJyKR1LUleef255dE7NkOsGVbG1c+9jYn3ziXOQuXp+X5bn9+ie+6iogCieShhqbmmOdCrYlkgkms54v3OiLinbq2xJdMjD3UVFdSH+ciH2pNeH2dWM9XU12Zch1FZDu1SCRlobGH+qZmHKm1FqI5Z/IoelSUxy2TTGsi2vP1qCjnnMmjUqqfiERSIJGUZWrsYcrYYRw/dhhlZjHLJNOamDJ2GJdMG0NtdSUG1FZXcsm0MRpoF0kTdW1JyjI19jBn4XKeXricdueink+lNTFl7DAFDpEMUYtEUharVeB37CFaSydErQmR/KMWiaTsnMmjIuZngPfWQrxB+lgtGgMev+jItNRdRNJHgURSFrrwJ5u1lWiCoLKsRAqLAon4ksrYQ7xB+iljh/lq6cSjZVJEMiPvAomZ3QucADQ45/aOct6Am4GpwGbgDOfcm9mtpSQr/CIefQh9e5dWqi2dRK+vZVJEMiPvAglwP/Bb4MEY548Ddg/+TAB+H/xX8lTni3gs4V1X6c6yStQKEpHU5V0gcc7NM7Od4hQ5CXjQOeeA+WbW18yGOOdWZqWCkrR4WVghmZ4gWAjLpKjrTQpV3gUSD4YBS8PuLwseUyDJA9EuhvEu1gZZuWgmGsD3exFPx+PV9SaFqhADSbTpzl263c3sbOBsgGEjdsx0nYTYF8M+lRWsb97WpXxtdWXW0nnjDeCnehEPBY/OASqVIKCuNylkhRhIlgEjwu4PB1Z0LuScuxO4E2Ds/uNjje9KGsW6GPbsVkaPivKksrDS3c0TbwD/5Bvnxl3qJdpjEo37JBsECqHrTSSWQgwks4DzzOxhAoPsTRofyQ+xLnrrm7dx+VfGeQ4MybYQvAad8AH80GOueuztmFlkodeNVg8v4z7JBAHNnZFClneBxMz+DEwCBprZMmAmUAHgnLsdeIZA6u9HBNJ/v52bmkpn8S6GyWRhJdPNk0q3lNcssjKzmPXwEiSSCQKZmjsjkg15F0icc99IcN4B52apOpKEdF0Mk+nmSWVswWsWWawyoZZPvD1Tkn3fmZg7I5IteRdIpHCl62KYTDdPKmMLXrPIog2kE3Y+Xqvm+BTmwWiFYilUCiSSVum4GCbTskllbCHWY6JlkcWqR+g9/uLxhVGXu3/5w1X8KGYNRIqLlpGXvJPMRlSp7H7o9TGJ6jFl7DBcjD1TlG0lpUQtEslLXls2qXSnJfOYRPVQtpUIWKxvVMVk7P7j3exXFuS6GkUnX5f0yGa9omWA9ago1+ZbUhSG9bQ3nHPjE5VTIJGUxEqhraqs4MLjRmflIhotYED0cY1MXtjzNaCK+KVAEkaBJP1OvnFuzPTX8jLjZ9PHZvRiGqsl0KNbWdTlWMrMcM75vtD7CRrZDjgKcOKX10CiMRJJSbzB5LZ2x29mL447IdDvBS7W/JFY6bihzCo/iyH6WVjx/PtfZcGnjR33M70ooxaBlGxS1pakJNFgcrRWAWy/wNUHN7gKXeDmLFye1Ov7yYoKX0crGfEmP8Zzw5OLIoKI33p4kWpdRVKhQCIpiZZC60W6LnCxAll1ZYWneqUSiFJdWPGJN5bGPJepNGEtAinZpEAiKQnNsbBoi/oTuKBHk64LXKy5IBccNzpi7kdZjAqmkp4b6zGJnivahEU/9fAi1bqKpEKBRFI2ZewwLj95HBXlXS/Wk0cPifqYdF3g4k0WnDJ2GI9fdCQvX3k8Pz95bNITFmNJZfIjxA5moefMhFTrKpIKDbaLL1PGDmPRF408tuCLiOOPLfiC5xev5IJOqcDpXOXWy6TFdC6GmOpzTd9/RJfPB2D8zv0zNvCtRSAlm5T+K77FSwWONoejUNJS01nPG55cxBNvLKXdOcrMmL7/CH40bUyaayySXppHEkaBJLMOnvl0zM2hILtb6qaLZqyLeA8kGiMR3xKNbxRippDSZ0W80xiJpCzU9RNvgycozEyhXKXPFkq3n0g4tUgkJeETC+Mp1EyhXKTPRpuseeVjb3NDcEa6SL5SIJG45ixczsk3zuXgmU9z8o1zO2age9muNt4+IvkuF+mzsT7TxxZ8kfTMf5FsUteWxBRvvaZEXTyFOMAeLhfps/E+03h70IvkmgKJxBRvwDnWhk4hhTjA3lm291CP95nWNzVz8o1zNXYieUldWxJTvAHnRGttFeIAe64l6jbzu9ClSKYokBSpWGMbyYg34BxaoiTamlqFOsCea1PGDuMr43f0VFapyJJPFEiKULqWak804Dxl7DBmX3IMM78yLuqaV5K8H00bE/F5xlr8Eoqj+1CKg8ZIilC8sY1kLvBeB5yzPZZQ7EKfZ+gLQSzqPpR8oUBShNI5mU5BInfipVhnsvtQkyIlWQokRShW9k86v8Hm6mJTufiv9HnhKsrXL6OtajgbJl1O8+hTMv66uRAv8Geq+1Bb9EoqFEiKULJLtScbFHJ1salc/FeqnzmfstbABbbb+qVUP3M+QFEGk1hfCGqDyQ6ZCObp6haV0qJAUoSSmUyXSlCIdbG5afbijLZS+rxwVUcQCSlrbabPC1cVZSCJ94UgU8FcW/RKKhRIipTXsY1UgkKsi0pT8zaamrcBmWmllK9fltTxQhfvC8HJN87NSMshG92iUnwUSEpcKkEh0az2kHR3ibRVDafb+qVRjxerWF8IMtVySOcOllI6NI+kxHn9phk+AS7RrPZw6ewS2TDpctq7Rda3vVslGyZdnrbXKBSZWp04NNFU84IkGXFbJGa2L3AeUAMsBu5xzn3Yqcw44DHn3C4Zq6VkTLRvoLGEgkK0Lpfmra2sD7ZgwqWzSyQ0DlIqWVvxZLLloJRvSVbMQGJm44F/ASsJBJEZwA/N7CfOuVvDivYARma0lpIxqQaFzhebWFvTprtLpHn0KSUVOGJlZuVidWKRWGLu2W5mc4BtwMnOuW1m1h34GfBT4PfA+c45Z2YTgFecc976OnJAe7Ynl+Kb6n7lmsiWXl7/H/S5S6Z43bM9XiBZDXzTOTen0/HjgYeBZ4HTgLEokOS1aBekinKjsns3NjRvi3rx0cUp906+cW7MeSShvV4UbCSTvAaSRFlb1vmAc+5pMzsSeAp4DvhlalUUv7xeHKKl+G5rc2yLk6qrfvLc85KZ5WUCoWarS6bFy9p6F5gc7YRz7nXgUGAo8FAG6iUJJLPCr5fMKS1Lnn+8ZGb5DTYi6RAvkDwNfNfMqqOddM59ABwMfJGJikl8yVwcvGZOafZyfvGyb3y6go2IH/ECyfXACGBDrALOuZXAQYBSf7MsmYuD13kfmr2cX7zM6UhXsBHxI+YYiXOuHdiU6Amccy3A5+mslCSWzFIWnVNFqyor2Lillbb27YkWmr2cnxKNVXlJA9Zsdck0LZFSoJK9OESb96EsnuKQjmAj4kfM9N9iUqzpvwoGIpJJ6Ur/lTymFF0RyQcKJFKUSmknxWjUWpVsSiqQmJkBwwlkcy10ziUcjBfJtlLbSbEzTUCUbPO8jLyZfR9YTiBD6yVgVPD4Y2Z2QWaqJ4nMWbick2+cy8Ezn+bkG+dGnZBYauLtpFgKNAFRss1TIDGzHwM3AncBRxK5dMoLwNfTXjNJKJnZ7cWkcvFfqbltb4b8ui81t+1N5eK/RpwvtZ0UO9MERMk2ry2Sc4HLnXMzCbRGwi0B9khXhczsWDNbYmYfmdklUc6fYWarzOzt4M+Z6XrtQlOK3zxD3Vbd1i/FcB3dVuHBJNaOicW8k2I4TUCUbPMaSAYDb8Q41w70TEdlzKwcuA04DtgL+IaZ7RWl6F+cc+OCP3en47ULUSl+8/TSbVXqOyl6me0ukk5eA8lHwOExzh1GYIHHdDgQ+Mg594lzbiuB5epPStNzF51S/ObppduqefQpNE29hdaqETiM1qoRNE29pSQG2kHb5Ur2ec3augn4nZltBR4NHqsxs+8CFwFnpak+w4ClYfeXAROilPsvMzsM+AC40Dm3tHMBMzsbOBtg2Igd01S9/FKKS1+0VQ2n2/ou/91F123lN31Xc4wkmzwFEufc3WbWD7gcuDJ4+BlgM3CFc+5PaapPl/1PgM5T758E/uyc22Jm5wAPEEgA6FznO4E7ITCzPU31yyvJLn1RDHMLNky6PCK1F7p2WxV6+q/Sd6XQeJ5H4py73sxuByYCA4FG4FXnXFMa67OMwByVkOHAik71WBN29y7g2jS+fsHx+s2zWC5OoUAQb7JhvHGUQggkXjarEsknCQOJmfUEZgG/cs69APwjg/V5HdjdzHYmMGflVALb+YbXZ0hw+XqAE4H3MlifolFMF6fm0afEDQiFnv5bikkUUtgSDrYHl4k/AMj4nuzOuVbgPGAOgQDxV+fcYjO7ysxODBY738wWm9lC4HzgjEzXqxiU0sWp0NN/SzGJQgqb16ytWcD0TFYkxDn3jHNuD+fcrs65XwaPXe6cmxW8falzbrRzbqxz7gjn3PvZqFehK6WLk5f030STGnNJ6btSaLyOkcwBrjezIQQG2evpNAjunHsmzXUTD7wOoJdShleicZR8H4zX/iFSaDztR2Jm7QmKOOdcxru+UlXM+5FECw6x5gwUQ9ZWOtTctnfUFOLWqhE0nPtODmokkp/SvR/Jzj7rIxmQ7AC65hYEFPpgvBSuYt3ewOs8Eu3JnodKaQA9nUplUmMq1GrNnHzvUvXDUyCJsd5VBOdcupZJEQ/mLFyOmRGta7IYB9DTycukxkKXSkAolrlG2ZJs66LQ5zfF47Vr6x26zjDvLG/HSIpN6A++PUoQKdYB9HTyMqmxkKUaEIpprlGmpdK6KOYuVa+B5Igox/oDxwR/fpi2GklC0f7gAcrMtDifR4kmNRayVAOCukq9S6V1Ucxdql7HSF6McepxM7saOAV4Km21krhi/WE75xREJOWAUFNdSX2UMuoq7SqZ1sX2LrClOAwL69wpli5Vz1vtxvFPtNR7VpXS5EJJXrK/H6HtmqMFEXWVRud19YTIjdjAcLhgKCmm7Q3SEUiOB9al4XnEI818lniS+f0I3665M+1jEpvXzdOidYEZjrbgnKViCCLgPWsr2voR3YE9gd2Bn6azUqUi1VTqKMy/AAAgAElEQVRLzXyWeJL5/Yg13lZbXcnjF3XZnUGCvCZsFPMAezivg+01dM3aaiGwf/tFWh4leV4ya+IFGk0ulHi8/n5ogD11XhI2inmAPZzXwfZJGa5HyUmUWaOc/vQp1tnE6aAB9swqhTlL4HGMxMzuDe4REu3cSDO7N73VKn6JvgnGCzTiXeXiv9L36XODg52ObuuX0vfpc/Nqtd9c0nhb+kRbUbp59Ck0Tb2F1qoROIy2yv64bpX0nXV23q067YfXwfYzgEExzg0EZqSlNiUkUWaNuhzSo+rZi7G2rRHHrG0rVc9enKMa5ZcpY4dxybQx1FZXYmiAPVWR2VmuY4Ji1ZyLOlrD7ZX9sJYNlLc0RpQphmDieatdYs9s3xtYlYa6lJREy7on6nK44clFPPHGUtqdo8yM6fuP4EfTxmSn8gWkrLkxqeOlSONt/sWaoNjrzXs75o2UR/mdK5YlUmK2SMzsh2b2iZl9QiCIPBG6H/azArgXeDpbFS4Wib4JxutyuOHJRTy24IuOJVLaneOxBV9w/v2vZvttiAixs7As4cpSxZHBFa9F8i7w/wADLiIw8XBlpzJbgfeBwm+b5UC8b4LxUjh/8fjCqI9Z8GkjcxYuL7m9SOINprf37E95S9dvgu09+2e7mgWhmH9PMilWdpbXxxa6mIHEOfcs8CyAmW0A7nbOLc9WxSR2oIm2WGNItPWUijkDLNHieeuPuZbqJ79PmdvW8Zh2q2D9Mdcqm6uTYv49ybRo2Vmdl0OJplgyuDwNtjvnrlQQya3QMhYHz4zfixhtML6YM8DiLZ4HgWDSNO13HVkzrVUjaJr2O4Cog6PFMPCZqmL+Pcm0ztlZrVUj2LTfd7rMfnfl3Wnr2X/772KRLJHiebDdzCYC3wX2AHp2Pu+cOzCN9ZIw0bbUjSVaNlgxZ4B5mTkcbeJYzW17F+3eEKkq5t+TbIj2e7Zt+EEl0er1ukTK0cAzwPPAIcBsoBI4GFgGxFodWNIg1jIWncXK/y/mSWepzhwulaUrwPu4RzH/nuRKMW9XEM7rPJKrgJsJLNAI8HPn3JEEWifbgBfSXzUJifeN0Ev+fzFPOvO6eF5nsQJNe2W/tNUtH4QvyujYPu4xZ2HXnupi/j2RzPLatbUX8DOgnUAqcC8I7OVuZlcAVwIPZqKCEvuboteF9Yp5kcdUdzvcMOly+j59btfJii0bOmYkF4NY4x6/eHwhVz32dsTvQjH/nkhmeQ0kLUCZc86Z2UpgVwILNgKsBwo/fy2PJZq86EUxTzpLpfugefQpVP3jYsrbIlODy9y2ohonidWaDWX+dc7M8vt7ovTh0uQ1kCwERhFIB34euNTMlhOYR3IVsCgz1ROI36LQH27qylrWRj1eTOMksVqz4dK1L3uxpA8rLTx5XgPJTUBo0cafAk8Cc4L3lwEnp7le0km0b4rF8oebK6WwxHe01mw06cjMSnWv+HySaF6SROd1HskzzrnbgreXA/sTaKGMA3Zzzr2RuSpKLMr79yfVgfpC0nkpnjKzqOXSkZlVDOnDieYlSXTJLNoIgJkZMAT41DnXmv4qiVfF8IebS6kO1Bea8NZstDlJ6crMKob04UymhRdzl1kyExKnAjMJtEK6AQcAb5rZXcCLzrk/ZqaKEksx/OHmWqnk+YdkMjMrHUkhuZap7s5i7zLzOiHxdAKr/D4E/A64L+z0BwRmvCuQZFkx/OFK9mUqg68Y0ocztaNhvC6zkgkkwGXA9c65S82snMhAshj4UdprJgkVwx+uFJdCTzPPVHdnsa+k4DWQjCS4EnAULUBVeqojySr0P1yRfJOJ7s5izxD0ukTKUmDfGOfGAx+lpzoiIsWn2DMEvbZI7gFmmlk98ETwmJnZZOAnBCYlSoZo0qFIYSv2DEGvgeRaYATwABAa2X0FKAfucM7dkoG6CZp0KFIsijlD0FMgcc454FwzuxGYDAwEGoG5zrkPMli/klcMs4VFpLglNSHROfcx8HGG6pIxG9dv4JrLL6Nm8GAG1tRSUzuYgbWBf/tUVWExZvvmA0069KeYJ4Fli7pWJZGYgcTM/gH8wDm3JOzYkcBrzrlN2ahcuqxfu4Zbr/tV1HM9e/ZkYE0tgwYPZlBNLcdOm87XZ3w7oszaxkZ69OjBDr16ZaO6ETTpMHXZnARWrAFLXaviRbwWyVFAdehOcP7IswRntGe4XukVXDI7mpaWFpZ98TnLvvgcgF336DqZ7/KLzuexhx9ih169Aq2ZmtrI1k3Y/V1234N+/funreqadJi6bE0CK+ZZy+paFS+SXWsrf/uA4ujXt5rzzvw29atWUdewquPfuoYGNm+OvNDs4Lax5t3IOLn8kw8B2LxpE5998jGffRK7d++XN9/GGd/7fsSx6674ORvWN3UEnlDrZ1DtYAbW1NC9e/eYz6dJh6lLZhKYnxZFMc9aVtdqcoq1ZZpI0os2FqKqPn2Y+ZOLop7buHFTRIDZe89R7LHj4Igy1b0q6dGjB1u2bEn4Wj23booIRNv6Dub//fmPLPv8s5iP6du/f0fLZuZ1N7L32HER50d2W8sdp+3FgEGDKC8vj/Es0pnXSWB+WxTFPGtZXaveRfs96jvrbCqWzWf9lBtzXLvMShRIovUJxe4nKkC9e/eid+9e7LrzTjHLzHnkTzjnWL9hA3X1DRGtmo7WTX0D9atWc+jY0YwJC0R1dXU0NtTHrcO6xkbWNTbywXvvsvbDxaypaGdb38EMHjqU9vZ2pkzYl9bWVsyMAYMGMah2e4tmUG3g31AgOvDgQ+jZs2eaPp3ClmjdpO3fHpd2aWon06Io5lnLsfYzqW9q5uQb55ZM69hLSyNay9Rw9HrzXrYNP6ioWyaJAskcM+u8VPzzUY7hnKtJX7Xyj5lRXVVFdVUVo3bfzfPjamtrues311PXEAg04f/WNaxi1eo1tLe3d5Q/ZOyelBusWlfHmnV1rFu3jtbWwMftnGN1QwOrGxp4L8brPf/GIvYcvXfH/VX19Vx09re3B5/Bgztuh8Z1qvv2zevMtVTFmwTW+dtjNJ1bFLEuJpla6C8fhHetdm6ZlMrAu9cWa6wWqOGKopsznniB5Mqs1aKImRmnfTX2BpJtbW2sXtPYEWBqBw2iW7duhNo0n1sro/ccRV1DA2sao28NG1LRrRsDXEtE19r7777H3Dmz4z6ue/fuDKodzMhdduWROXMjzq1ZtYqPlrzfMa7Tu0+fggo6sSaBRfv22Fl4i8LLxaRY+8ZD67mdfOPcLsGkFAbevY6BxWqZQnF0c8YTM5A45xRIsqC8vJzamkHU1gyKen7kiOG886/AxX3btm00rF5NfcP2Fk1HF1vDKlrbWhm709CIx3/x/jsJ67B161aWL/2CsrIy6lasiDj33DNPcfF53+u437OyMmIeTngCwS677cEhRxyZ7EeQE4n+sDu3KBJdTIp51nJIqQ68ex0D2zDpcvrOOhuL0vtfDN2c8ZTEYHuxqKioYNiQIQwbMsTzYw7Ydxyz/nhf3HGdDRs3AjBkQF+Gbq6LeHzbyk8i7rc0N/PFZ5/yxWefdnmtQ488qksg+csD9/HwA/dGZKqFuthqgrcH1tbSo0cPz+8pHWJ9e3RAW9WILi2KYh5Q96pUB969joE1jz6FimXz6fXmvRHBpFi6OePJu0BiZscCNxNYx+tu59w1nc73AB4ksG/8GuDrzrnPsl3PXHvi3Uauf2kFK9ZvY2hVBT8+dCjT9+o6f6Vm0ECmHXtM3OfatGkz9atW0drayuDBkRlrI0eM4KDx+1G/ajUr6xtoaWmJ+TxVPSq6pE6/teDf/PuVfyV8P9V9+zLje+dy8ZVXRxx/47X5NK1bG5Eu3a2b/1/bWOMaTVNvidqyKOYBda9KdU5TMmNg66fcyLbhBxVtN2cseRVIgpMebwOOBpYBr5vZLOfcu2HFvgusdc7tZmanElhQ8uvZr23uPPFuI5fO+YLm1sC3nuXrt3HpnC8AogaTRHr12oFdeo2Meu60r57cMcbjnGPDxo3UN2yfh9ORQNCwmokH7B+RsQbQXBe9z7izpnXraF5dFxGIBuy1H3fc/H88/dijHcfMjH4DBgRaM+EJBDW1HH70FPYas4+n10t2XKOYB9S9KtU5Tcn+rpRCN2dneRVIgAOBj5xznwCY2cPASUB4IDkJuCJ4+1Hgt2ZmwYUlS8L1L63oCCIhza2O619akVIg8crMqOrTh6o+fdh91108PeaGqy7n+9+ZEZGpFj4htL5hNQ2rV9PW1sY+u+/cEYgWfREIKis/jZz86ZyjcfVqGlev5v3FncZ/zOg/YCCDh24fJ/rv6cdTv3JFl3TpjvtH/IVBtYPp179/3CSCYh9Q96pUN1IrxeCQjHwLJMMIbKIVsgyYEKuMc67VzJqAAcDqrNQwD6xYvy2p47k0csRwRo6I3/3T3t7Omsa19OixfYZ/KKBMPvgg+lf17ghEq9c0Eus7wx59ulERTJsOefftN6iriz+PB6BbeTn/+N4IJg1aGxEk/nDXHVT36xds+Yyl5lsv5/1Cn6nQwoziR74Fkmh/nZ2vGl7KYGZnA2cD7Di8uP4ghlZVsDxK0BhaVZGD2vhXVlbGoIEDop677oqfRdxvbW1l1eo1XRIH6htW8eUJB7JrWNeac47GBCnTHc/b1sYgtwqjnG7rl1L19A9YtWo1l55/XpfA1aNHj8jWTbCFc86FP6JP1fZdp51zBRFwtDCj+JVvgWQZgQ20QoYDK2KUWWZm3QgsLNnY+Ymcc3cCdwKMHze2qLq9fnzo0IgxEoDKbsaPDx0a8zHhg/N9K8txztHU0h53oD4fdevWjSGDaxkyuNZT+Y8XvExdfafutLBxnbqGBuqWf8b6lnYG995+0S9va6HnSzdEbf1s2bIlYqHPkO//708i7n/x6accdcA+EQt7xlyVoLY2ZysSaGFG8SvfAsnrwO5mtjOwHDgVOK1TmVnADOBV4KsENtcqqkCRSOii7yVrC7oOzq9t3n7R8DtQn8/MjOFDhzJ8aOwAC8Cte9G8rZ2enf4aerSs5n++fXrChT4BKisrafniA8Jz2j5e+B9PC30C9BswgHeWR/bOfvzhBzz92KMR83ZCmWsVFelrfZbq/JBcK6YFHvMqkATHPM4D5hBI/73XObfYzK4CFjjnZhHYP/4PZvYRgZbIqbmrce5M36u/5wt/tMH5cNkYqM9rfYZQuaFzwxdqhwzjd5f+usvxjRs3dVnqZsuWLV0y1j5a9JbnKvSvru6SOr1o0RKunXlZ1PL9BgzoaNnU1A5m7PgDOOsHF0SUaWlpoVu3bgnTpUt1fkgiiS70fgJBsW09kFeBBMA59wzwTKdjl4fdbgG+lu16FTIvg/D5OFCfNRMvgLmXQ2tYe6Jbz8DxKHr37sVuvXdmt112jvu006cey9qP3+0Yw+kcfEJp1PWrVrHryBERgWjRF3V8tnBBzOdeu2YNa9es4YP3AgmNq5d/zlcmH9ax2CfAnTffyHVX/Cyw0GfHPJyuXWwnjIQH3mqmtXx74CjW+SFeL/6JLvR+A0GxbT2Qd4FE0i/W4HznMiVr1LTAv6/eBBtWQp8hgSASOp4iM6NvdTV9q6vZM8FCn517Z8fsOJhpRxyCbWvpMq7TsHp1xEKfALsHA9GiL7ZnrS19f1HkQp/vLIr5+kf81ww27X96RNZW03+e49ePvB+5KkFwXKcQF/pM5uKf6ELvNxAU20oJCiQlINrgfLhEA/UlYdQ034HDj2gX5UMnTuDQiZ2z37cv9BmeuTZyeCDFOrxV07PM++vvVVPJ947tywufd+fB/2zgqsfeZt0Tt7Nuyfyo5bt37x6xM+i3/+c8Jh09JaJM3YoV9O7Th169e+dF0Enm4p/oQu83EBTbSgkKJCWg8+B8IWdtSeRCn/uMjl3unpv/j9tvuCZioc9YXWsTxnyJjzd253cL6jq+cGxe3yUZssPWrVtZsWwpK5YFLoaHTT6aPUePiShz6tSj+PD996jcYYeIFQjCM9VCGW17jt6b3n36+P9w4kjm4p/oQu83EBTbSgkKJCUimcF5KR7JLPR58B3vRLRaq8afxLa1K6jctp6Da9oigs/GTZsiHrt77/IuC36uCW7o1rx5c8yFPkMee24eEw45NOLY+d85narq6o4Vpjsy11Jc6DOZi3+0C73DaNntmJjnkwkExbZSggKJiABdEy567XU4EJgB/OiP9404F1roM9TKOWTCgRGTStvb2+lXXcXmzZvjLvQZUrFxTUTW2oae1fy/P/0h7mOq+/btSBq46y+P0a//9i9KW7Zs4YN3FzOodjADBg2ioqIiqYt/tJV8DccO//lTxG6HfgJBMS27okAikglLnkz74H2mJbNiQmihz112ir7YZ1lZGR+9/nLHQp+hLQuiTQitX7Waw8Z+iaqwrq3X33gz6vOGa1q3jqZ16/hoyfu0fPEBa+q6d2StLf3sU46duH9H2f4DB1JTO5jaXoMY2rqUIT2aqRnQj+oDv0q/+gEcGuwiDM/qwsq67C1SanvQeKVAIpJuS56MTCfesCJwH/I6mKSyYkIi4Qt97rHbrp4fN2qP3fnznb+LOa4TWugToG91NfvvtiN1dXUdW1R/tCAydbpjoc+Io/Xw1G0MrHmUhV/URWR1vbK0late3MLg3mXU9jIG97bA7d5Gbe/PcWvWJFzos5QokIik26s3Rc5JgcD9V2/K60CS7IoJmVTVpw+nfuWkmOdDC33WNTSwfkNgY7bBgwd3bFG9+vN+jNnrSwkX+gQY2L8va959k1HP/qyj2+uDNe3M+bgNaIv+oN8NpKKigoE1tRw2+WhuvPPeiNOfffwxdSuWdyQYFONCn+EUSETSbcPK5I7nkUJJyggt9Blrsc8jDj2Y/8x7Dti+0Of2pW6CrZz6QCtn5IjhgbTp5oaOx9dtTLzq0rZt21i5fBn1dSu7bFH9wB2/485bbuy437Nnz8CE0ChbVI/dbzzjxh+QyseQNxRIRNKtz5BAd1a041CQ4yfgfVfOfON5oc+w/7dv7F3B3jVl1G101G+Cug3t1G3pQb3VUrehjfpVq1m/YQMAO/bt1SVjrXll5IKeLS0tLP38M5Z+/lmXlz3r/Au7BJJfXnYx/375Xx3zdKIt/DmwtpbKyvxYxkaBRCTd4i25UqDjJ+nelTMvhf2/jexbxsi+ZYH/tyOvivp/s3lzM/WrVtG9e0WXLapHf2kUXz5wfNyFPkN60dp1i+rX/82C+a8krHJVdTVX3nAzp/z3jIjjz81+GqAjAA2sqaF79+7RniItFEhE0i3ekiv3Ty7I8ZO078qZj62yJJfK2WGHSnYeuWPUcxeffy4Xn39ux/2NGzd12spgVcf2BtMmHdJlwc8NDd66Qdc3NbFt9cqOQDRgr/0A+MUlP+KjJZGpBX3794+6RfX0r5/GsBEjujx3MhRIRDIh1pIrBTp+ktZdOfO5VZahpXJ69+5F79692HXnnTyVf/zBe1i+si4yXTpsXKd+1SrqV62mtbWVA/fanTE7Dg5krQUDSv2K5V2ec11jI+saGzsW+gzZY6/REYHEOceEPXaiqm9fz+9PgUQkmxKNn+SptO7KWaBZbdm02y6JV5dub2+nce06+vTuBWzPWmtvb+fkqVMi9tFZtXpNl4U+Q2q7EzkZdMMGli/9guVLv/BcXwUSkWxKcsn6fJHWOSYF2irLN2VlZQwc0LVbsaysjAduuzniWGihz475OGFda5PG7RWxO+eSDzcmXRcFEpFsytCS9ZmW1jkmBdoqK2ThC32OTVB2t112ZtmiBdQ3rGb/ycd6en4rhV1qx48b6xY8PzvX1RAR6DpGAnGzo/JSPiYLZIANHPaGc258onJqkYhIdhVoq6xDPicL5IgCiYhkX443EvNFyQJdKJCI5LsS6UYpGEoW6CKJzThFJOtC3SgbVgBuezfKkidzXbPSFSspoISTBRRIRFKx5MnALPVb9wr8m6kLe7xuFMmNiRcEkgPCFUAKdyapa0skWdkcbFU3Sv4p9GSBDFAgEUlWNgdbNeciPxVyskAGqGtLJFnZbCWoG0UKgFokIsnKZitB3ShSABRIRJKV7fWy1I0ieU6BRCRZaiWIRFAgEUkk1oTAYgkcmvAoPimQiMRT7Osqpfr+FHwkjLK2ROIp9gmBqby/fJ9tn63JotJBgUQknnSm+ubjBS6V95fPwTXfg1yRUteWSDzpSvXN1y6yVN5fNubR/PNKWPwIuDawchj9NThiZuLHaWXenFCLRCSedE0IzPW3+FitoVjvb+RhsVtPmV608J9XwjsPB4IIBP595+HA8US0pExOKJCIxDNqWmDnvj5DAQv8m8pOfrm8wMXr7on2/vacDu8/Ebt7KNOz7Rc/ktzxcFqZNyfUtSWSSDpSfXO5Zlai7p7O7+/+yYnLh543VtaWn6yuUEvE6/Fw2Z4sKoACiUh25PICl2xryMvxeMHV63hQrGBj5dGDhpVHf71wo6bBijcjx1f2nK7xkQxT15ZINqSriywVyXb3+O0e8jIeFK+7bfTXoj9vrOPhljwZ6JYLH195/wllbWWYWiQi2ZKr2fDxWkNLnoR5v4KWdYHjPfrC7scGLr6ptp68tGjiBZszng/cV9ZWwVAgEcm2bM8KjzWmAfDcZdC+bXvZLetg8aMw+qvw+bzU6uhlPChRsDliprfAEevxXo9LWiiQiGTTkifh2Z+Caw3c37AicB8yH0w6P//9kyODSIhrDQSRUMsgWV7GgzKVfKCNwHJCYyQi2fTir7YHkRDXGjiebfG+pfv5Bu9lPChTKcTaCCwn1CIRyaYt65I7nkmxvr2HzvmRaDwoU0vxa4n/nFAgESlVEy/oOkYCYN2y8w0+U8kHxbTEf4FQIBHJhFgD6j37bs+QCtezb/brGLrYds7aOvynxXEh1lL3WaNAIpJu8SbkHfbTrq2AsorA8VRex++Fsli/vefrIplFSoPtIumWaC7DUb+MHIg+6pfJX9xKZbn0VJfez/UimSVGLRKRdEs0lyEdrYBSmHjnp1Wh+SRZlTctEjPrb2bPmtmHwX/7xSjXZmZvB39mZbueIgnFzHhy6dvQqhQulH5aFVoFOKvyJpAAlwDPO+d2B54P3o+m2Tk3LvhzYvaqJ+JRtLkMIenqgiqFC6WfYKn5JFmVT4HkJOCB4O0HgOk5rItI6iIm5EWRjr76bF4o441TeBnDSHWcw0+wzOUimSUon8ZIap1zKwGccyvNrCZGuZ5mtgBoBa5xzj0RrZCZnQ2cDbDj8GGZqK9IbKFxkFv3AlzX8+nogurWc3vXT6bSduONU0DiMQw/4xx+l94v1oy0PJTVQGJmzwGDo5y6LImn2dE5t8LMdgHmmtki59zHnQs55+4E7gQYP25slL9kkSzIxNpPnS/OAG0tscv7kWicItGAv5+kAM1SLxhZDSTOuaNinTOzejMbEmyNDAEaYjzHiuC/n5jZC8C+QJdAIpIXMrGhVTYztlIZpwg/5zcpQK2KgpBPYySzgBnB2zOAv3UuYGb9zKxH8PZA4GDg3azVUCRZmeirz2bGVrxxCi9jGKWQFCB5FUiuAY42sw+Bo4P3MbPxZnZ3sMyXgAVmthD4J4ExEgUSyW+jpgWWZP/Bu4F/07H/ezLH/Yg3qO9lwF/ZUyUhbwbbnXNrgMlRji8AzgzefgUYk+WqieSXaN1lAFs3B8ZPOgcqP0upeBmniHdO4xwlwZwr/nHo8ePGugXPz851NUTSp/MWuSHdekZ2nUUbmO9cRiQGGzjsDefc+ETl8qlrS0S8GjUNKnboerzzHBWtOSVZoEAiUqi8DLqXwlIqknMKJCKFSllTkicUSEQKlbKmJE/kTdaWiCTJS0aUsqYkCxRIRAqZl5nfmh0uGaauLRER8UWBREREfFEgERERXxRIRETEFwUSERHxRYFERER8USARERFfFEhERMQXBRIREfFFgURERHxRIBEREV8USERExBcFEhER8UWBREREfFEgERERXxRIRETEFwUSERHxRYFERER8USARERFfFEhERMQXBRIREfFFgURERHxRIBEREV8USERExBcFEhER8UWBREREfFEgERERXxRIRETEFwUSERHxRYFERER8USARERFfFEhERMQXBRIREfFFgURERHxRIBEREV8USERExBcFEhER8UWBREREfFEgERERXxRIRETEl7wJJGb2NTNbbGbtZjY+TrljzWyJmX1kZpdks44iItJV3gQS4B3gK8C8WAXMrBy4DTgO2Av4hpntlZ3qiYhINN1yXYEQ59x7AGYWr9iBwEfOuU+CZR8GTgLezXgFRUQkqnxqkXgxDFgadn9Z8JiIiORIVlskZvYcMDjKqcucc3/z8hRRjrkYr3U2cDbAjsMVa0REMiWrgcQ5d5TPp1gGjAi7PxxYEeO17gTuBBg/bmzUYCMiIv4VWtfW68DuZrazmXUHTgVm5bhOIiIlLW8CiZmdbGbLgInA02Y2J3h8qJk9A+CcawXOA+YA7wF/dc4tzlWdRUQkv7K2Hgcej3J8BTA17P4zwDNZrJqIiMSRNy0SEREpTAokIiLiiwKJiIj4Ys4Vf2asma0CPvdQdCCwOsPVSVU+1w3yu36qW+ryuX6qW+q81m+kc25QokIlEUi8MrMFzrmYC0bmUj7XDfK7fqpb6vK5fqpb6tJdP3VtiYiILwokIiLiiwJJpDtzXYE48rlukN/1U91Sl8/1U91Sl9b6aYxERER8UYtERER8KelAksT2vp+Z2SIze9vMFuRZ3XKy9bCZ9TezZ83sw+C//WKUawt+bm+bWUYX2Ez0WZhZDzP7S/D8a2a2Uybrk2TdzjCzVWGf1ZlZrNu9ZtZgZu/EOG9mdkuw7v8xs/3yqG6TzKwp7HO7PIt1G2Fm/zSz94J/qz+MUiYnn53HuqXvs3POlewP8CVgFPACMD5Ouc+AgflWN3MJfQIAAAcQSURBVKAc+BjYBegOLAT2ylL9rgMuCd6+BLg2RrmNWapPws8C+D5we/D2qcBf8qhuZwC/zebvWNhrHwbsB7wT4/xUYDaB/YAOAl7Lo7pNAp7K0ec2BNgveLsP8EGU/9ecfHYe65a2z66kWyTOufecc0tyXY9oPNatY+th59xWILT1cDacBDwQvP0AMD1LrxuLl88ivM6PApMtwd7OWaxbzjjn5gGNcYqcBDzoAuYDfc1sSJ7ULWeccyudc28Gb28gsCJ55130cvLZeaxb2pR0IEmCA/5hZm8Ed17MF7ncerjWObcSAr+0QE2Mcj3NbIGZzTezTAYbL59FRxkX2JKgCRiQwTolUzeA/wp2fzxqZiOinM+VfN/ieqKZLTSz2WY2OhcVCHaT7gu81ulUzj+7OHWDNH12ebOMfKakYXtfgIOdcyvMrAZ41szeD35TynXdPG89nIp49UviaXYMfna7AHPNbJFz7uP01DCCl88io59XHF5e90ngz865LWZ2DoGW05EZr5k3ufrcvHiTwDIeG81sKvAEsHs2K2BmvYH/B1zgnFvf+XSUh2Tts0tQt7R9dkUfSJz/7X1xgT1RcM41mNnjBLoqfAeSNNTN89bDqYhXPzOrN7MhzrmVwaZ6Q4znCH12n5jZCwS+GWUikHj5LEJllplZN6Ca7HSbJKybc25N2N27gGuzUC+vMvp75kf4xdE594yZ/c7MBjrnsrLOlZlVELhQP+SceyxKkZx9donqls7PTl1bCZhZLzPrE7oNHANEzSDJgVxuPTwLmBG8PQPo0oIys35m1iN4eyBwMPBuhurj5bMIr/NXgbkuOOqYYQnr1qnf/EQCfdr5YhZwejAD6SCgKdStmWtmNjg0zmVmBxK4pq2J/6i0vbYB9wDvOedujFEsJ5+dl7ql9bPLRgZBvv4AJxP4xrAFqAfmBI8PBZ4J3t6FQJbNQmAxgW6nvKhb8P5UAhkZH2erbsHXHQA8D3wY/Ld/8Ph44O7g7S8Di4Kf3SLguxmuU5fPArgKODF4uyfwCPAR8G9glyx+Xonq9uvg79dC4J/Anlms25+BlcC24O/cd4FzgHOC5w24LVj3RcTJcMxB3c4L+9zmA1/OYt0OIdBN9R/g7eDP1Hz47DzWLW2fnWa2i4iIL+raEhERXxRIRETEFwUSERHxRYFERER8USARERFfFEikJJnZFWbmovw8l4HXOsbMLkj386bKzI42sz9bYFVrZ2ZX5LpOUtiKfma7SBxNwLFRjqXbMQQmQN6UgedOxbHAPgTm/5ya47pIEVAgkVLW6gIrshYUM6t0zjX7eIofO+f+N/hcebMKsRQudW2JxGBmZWZ2SXBToi1m9oGZzehU5ngLbOzVYGbrg6scHxN2/grgf4GRYd1n9wfPvWBmj3Z6vknBMnsH7+8UvP9NM3vQzNYRWOAxVP5MC2xctMXMPjeznyR6X865dh8fi0gXapFISQsu3hiuzW1f7uFWAmtzXUVgpdSjgXvNbI1z7qlgmZ0JXNhvANqB44DZZnaYc+5l4G4CK6oeSWDZG4BVKVT1BuAx4GtAW7DuPwZ+RWCTsReA/YFfmNlm59xvU3gNkZQokEgpG0BgDadwRwPPmdluwP8A33bOhTbDei64uOJM4CmA8Au2mZURWCdrNIE1oV52zi0zs5XAFp/daPOdc+eGvVZVsB5XO+euDB5+1sx2AH5mZr93zrX5eD0Rz9S1JaWsCTig009o85/JBFoYj5tZt9APgQHqcWZWDmBmw83sATNbDrQSCEzHAHukua5Pd7o/EegFPNKpfnOBWgLLlYtkhVokUspanXMLYpwbSGCv9VhZXEPMbAWBZcL7AJcTWFV4E4GusFg7RqaqPkr9ILB6azQjgM/TXAeRqBRIRKJrJNDCOJhAy6SzBmA3Aht1Heec+3vohJlVenyNFqB7p2P9Y5TtvEx3aEOuE+gaZACWeKyDiG8KJCLRzSXQIql2zj0brUBYwNgSdmwkgeDzn7CiWwnshdLZMuCwTseO9li/V4FmYKhzrnO3l0hWKZCIROGcW2JmtwMPm9l1wAICwWA0sIdz7kzgfQLB4P/M7OcEuriuBJZ3err3gVozO4PA7pqrnXOfAY8D3zWz3xAYAzkCmOKxfuuCqcU3B4PXPAJjnnsARzjnTo712GD5A4J3uwN7mdlXgU3OudleXl8knAKJSGznEtjV8CwC4x7rCWwVfA+Ac26LmX2FwA54jxIIKr8EJgF7hz3PXwkEieuAQcADwBnOuafN7KfA94EzCWxXfAFRti2Oxjl3XXCc5kICc1VagvX9S4KHHgHcF3b/a8Gfz4GdvLy2SDjtkCgiIr4o/VdERHxRIBEREV8USERExBcFEhER8UWBREREfFEgERERXxRIRETEFwUSERHxRYFERER8+f9heHKLmjc0SgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#visualize prediction boundary on moons dataset\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(6,6))\n",
    "\n",
    "fontsize = 15\n",
    "\n",
    "#----------------------------------------------------------------\n",
    "### VISUALIZE BLOBS TRAINIG DATASET (with correct classification)\n",
    "\n",
    "\n",
    "#plot the points of the first class:\n",
    "mask = y_moon_train == 0\n",
    "\n",
    "ax.plot(X_moon_train[mask][:,0], X_moon_train[mask][:,1], 'o', label='class 0') #plot first feature on x-axis and second feature on y-axis\n",
    "\n",
    "\n",
    "#plot the points of the second class:\n",
    "mask = y_moon_train == 1\n",
    "\n",
    "ax.plot(X_moon_train[mask][:,0], X_moon_train[mask][:,1], 'o', label='class 1') #plot first feature on x-axis and second feature on y-axis\n",
    "\n",
    "#----------------------------------------------------------------\n",
    "### Visualize perceptron's prediction field\n",
    "\n",
    "# generate square mesh of points to evaluate, with \"n_pts_side\" ponits per side\n",
    "n_pts_side = 500\n",
    "pad = .5\n",
    "\n",
    "#extrema change based on generated points\n",
    "xx_min = X_moon[:,0].min()-pad\n",
    "xx_max = X_moon[:,0].max()+pad\n",
    "\n",
    "yy_min = X_moon[:,1].min()-pad\n",
    "yy_max = X_moon[:,1].max()+pad\n",
    "\n",
    "xx = np.linspace(xx_min, xx_max, n_pts_side)  \n",
    "yy = np.linspace(yy_min, yy_max, n_pts_side)\n",
    "\n",
    "#plot decision line\n",
    "ax.plot(xx, (-b-w[0]*xx)/w[1], 'k--', linewidth=3)\n",
    "ax.set_ylim(yy_min, yy_max)\n",
    "\n",
    "\n",
    "# plot decision field\n",
    "xx, yy = np.meshgrid(xx, yy)\n",
    "\n",
    "zz = perceptron_predict(w, b, np.vstack([xx.ravel(),yy.ravel()]).T) #generate predictions for each point in field\n",
    "zz = zz.reshape(xx.shape)\n",
    "\n",
    "ax.contourf(xx, yy, zz, levels=2, colors=('deepskyblue','k','orangered'), alpha=.1)\n",
    "\n",
    "#----------------------------------------------------------------\n",
    "\n",
    "ax.set_xlim(xx_min, xx_max)\n",
    "ax.set_ylim(yy_min, yy_max)\n",
    "\n",
    "ax.legend(fontsize=fontsize)\n",
    "\n",
    "ax.set_xlabel('Feature 1', fontsize=fontsize)\n",
    "ax.set_ylabel('Feature 2', fontsize=fontsize)\n",
    "\n",
    "ax.set_title('Blobs', fontsize=fontsize);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <center> **Exercises** </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Play with the code \n",
    "\n",
    "In order to gain a deeper understandig of what is happening in the code, try play a bit with it. \n",
    "\n",
    "* Re-run the perceptron algorithm on the given datasets for a number of times. Does it always find the same solution? \n",
    "* Change the parameters of the learning algorithm: does anything change? Can you reach a higher accuracy an the moons dataset? If so, in which way? If not, why? \n",
    "* Change the parameters in the functions that generate the synthetic datasets, how do they change? How do the results change? \n",
    "\n",
    "Make sure you understand what is happening in every line of the code, including the plotting code. Look into the matplotlib and numpy docs to understand the exact functioning of the functions you may have not seen before. Try to guess what are the values of the variables, then use Jupyter to look at what's inside them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prediction boundaries with test data\n",
    "\n",
    "In the prediction boundary plots only the training data are shown. Replicate the plots showing the test data instead. Show the samples that are correctly classified with the standard 'o' marker and samples classified incorrectly with an 'X' marker."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Perceptron class\n",
    "\n",
    "We implemented the perceptron algorithm with three separate functions. A more elegant and functional way to do this is to create a \"Perceptron\" class that contains both the parameters and the functions that should handle them. Try to implement a working Perceptron class, by following the provided blueprint and reusing the code of the previously implemented functions, with the due modifications."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Perceptron():\n",
    "    \n",
    "    def __init__(self):\n",
    "        \n",
    "        #initialize and store perceptron parameters\n",
    "        self.weights = None  #we don't know the dimension of the incoming data yet\n",
    "        self.bias = 0.\n",
    "        \n",
    "        \n",
    "    def _pla_step(self, x, y): #initial underscore is a standard way to signal \"private\" functions in python\n",
    "        \n",
    "        #use the code from the pla_step function\n",
    "        \n",
    "        #remember now you don't need to pass or return copies of w and b:\n",
    "        #you can access and modify them directly from the function, using the self keyword!\n",
    "        \n",
    "        \n",
    "        \n",
    "        \n",
    "        return w_updated\n",
    "    \n",
    "    \n",
    "    def train(self, X, y, eta=1, max_epochs=10, error_history=True):\n",
    "        \n",
    "        \n",
    "        #initialize weights with the correct dimension, if needed\n",
    "        if None == self.weights:\n",
    "            self.weights = np.zeros(X.shape[0])\n",
    "            \n",
    "        \n",
    "        #from now on it works pretty much as the previous one\n",
    "        #remember now you don't need to pass weight and bias term to _pla_step (note that the name has changed, btw)\n",
    "        \n",
    "        \n",
    "        \n",
    "        \n",
    "        #return errors only if required\n",
    "        if error_history:        \n",
    "            return errors\n",
    "        \n",
    "        \n",
    "    def predict(self, X):\n",
    "    \n",
    "    \n",
    "        return y_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the new perceptron class on the blobs dataset. Does it work in the same way? Try to replicate the decision boundary figure with the new implementataion. Try to train it also on the moon dataset.\n",
    "\n",
    "* Try to guess what happens when you re-train a trained perceptron with the old and the new implementation. Is there any difference?\n",
    "* Go and look at the docs for the scikit-learn (much more complete) implementation of the [Perceptron](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Perceptron.html). Try to wrap your head around it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### TRAIN PERCEPTRON ON THE BLOBS DATASET\n",
    "\n",
    "percpt = Perceptron()\n",
    "\n",
    "#Train the perceptron on the blobs dataset, with default parameters\n",
    "training_errors = percpt.train(X_blob_train, y_blob_train)\n",
    "\n",
    "#number of training errors in last epoch\n",
    "print('Number of training errors:\\t{} ({} %)'.format(training_errors[-1], training_errors[-1]/X_blob_train.shape[0]*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multilabel Perceptron\n",
    "\n",
    "Implement an algorithm based on perceptrons alone that can classify data with more than two labels (three labels is enough for a proof-of concept). Generate a mock dataset with three blobs (look at the docs of the function `make_blobs`). Plot the decision boundaries.\n",
    "\n",
    "**Hint:** You don't need to change anything inside the _Perceptron_ class. Transform the data labels into _one-hot encoding_. Now you can use each sample to update one different perceptron for each of the three components of its label!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}