LinearRegression_vs_KNeighborsRegressor.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sample taken from SciPy 2016 Scikit-learn Tutorial"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regression Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In regression we are trying to predict a continuous output variable -- in contrast to the nominal variables we were predicting in the previous classification examples. \n",
    "\n",
    "Let's start with a simple toy example with one feature dimension (explanatory variable) and one target variable. We will create a dataset out of a sinus curve with some noise:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-3.         -2.93939394 -2.87878788 -2.81818182 -2.75757576 -2.6969697\n",
      " -2.63636364 -2.57575758 -2.51515152 -2.45454545 -2.39393939 -2.33333333\n",
      " -2.27272727 -2.21212121 -2.15151515 -2.09090909 -2.03030303 -1.96969697\n",
      " -1.90909091 -1.84848485 -1.78787879 -1.72727273 -1.66666667 -1.60606061\n",
      " -1.54545455 -1.48484848 -1.42424242 -1.36363636 -1.3030303  -1.24242424\n",
      " -1.18181818 -1.12121212 -1.06060606 -1.         -0.93939394 -0.87878788\n",
      " -0.81818182 -0.75757576 -0.6969697  -0.63636364 -0.57575758 -0.51515152\n",
      " -0.45454545 -0.39393939 -0.33333333 -0.27272727 -0.21212121 -0.15151515\n",
      " -0.09090909 -0.03030303  0.03030303  0.09090909  0.15151515  0.21212121\n",
      "  0.27272727  0.33333333  0.39393939  0.45454545  0.51515152  0.57575758\n",
      "  0.63636364  0.6969697   0.75757576  0.81818182  0.87878788  0.93939394\n",
      "  1.          1.06060606  1.12121212  1.18181818  1.24242424  1.3030303\n",
      "  1.36363636  1.42424242  1.48484848  1.54545455  1.60606061  1.66666667\n",
      "  1.72727273  1.78787879  1.84848485  1.90909091  1.96969697  2.03030303\n",
      "  2.09090909  2.15151515  2.21212121  2.27272727  2.33333333  2.39393939\n",
      "  2.45454545  2.51515152  2.57575758  2.63636364  2.6969697   2.75757576\n",
      "  2.81818182  2.87878788  2.93939394  3.        ]\n"
     ]
    }
   ],
   "source": [
    "x = np.linspace(-3, 3, 100)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "rng = np.random.RandomState(42)\n",
    "y = np.sin(4 * x) + x * np.cos(3 * x) + rng.uniform(size=len(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Try changing y\n",
    "# y = np.sin(4 * x) + np.cos(3 * x) + x + rng.uniform(size=len(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 3.64450382,  4.08915616,  3.62948456,  3.15307313,  2.27668674,\n",
       "        1.76752795,  1.10369318,  1.31030152,  0.43121915, -0.06451328,\n",
       "       -1.31966771, -0.88051264, -1.45041809, -2.40816534, -2.66844872,\n",
       "       -2.78030325, -2.65315189, -2.30840654, -2.16608797, -1.97267743,\n",
       "       -1.23517251, -1.22771656, -0.55477689,  0.05617909,  0.67464417,\n",
       "        1.50086534,  1.35909133,  2.04520943,  2.4078512 ,  2.04888624,\n",
       "        2.69417528,  2.23843399,  2.01638395,  2.69568053,  2.43408616,\n",
       "        1.94232384,  1.06792741,  0.47577371,  0.68422749,  0.08989719,\n",
       "       -0.53191927, -0.40029894, -1.02865898, -0.24006609, -0.89325869,\n",
       "       -0.41094728, -0.60916954, -0.18569649,  0.10348604,  0.03376103,\n",
       "        1.12067805,  1.21835706,  1.64526346,  1.81570796,  1.67136954,\n",
       "        2.07391291,  1.23787899,  1.25903037,  0.94070314,  0.97928783,\n",
       "        0.73893259,  0.27135457,  0.45063592, -0.40656032, -0.85299198,\n",
       "       -0.92575804, -1.60587077, -1.14913537, -1.99335922, -1.09974349,\n",
       "       -1.23019106, -1.61672095, -1.52545288, -0.34395612, -0.00883204,\n",
       "        0.51043299,  1.0814531 ,  0.92096619,  1.72567615,  1.96289447,\n",
       "        3.12701   ,  3.22133112,  3.164061  ,  3.02095248,  3.27469008,\n",
       "        3.17545701,  3.35011063,  2.9204182 ,  2.73763524,  1.81246713,\n",
       "        0.89218011,  0.88314065,  0.31665967, -0.48433237, -0.84056625,\n",
       "       -1.62687251, -2.03168182, -2.4699496 , -3.11302272, -3.16207228])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGeNJREFUeJzt3XuMXFd9B/Dv15sBb3hkqbIqeBNjS0ROIU5tsaKVUrWN\nedg0KQlGKKQtFaWSxR+oBJVQh0g8hezKEg9R/qgFUUGKIEh5gJKgPJRElKgJrBOTB05QCrKSBTXL\nY4HgVbJr//rHzMTj8b1z78w9997z+H4kS97d2Ttndub+7rnn/M7v0MwgIiLxWNd2A0RExC0FdhGR\nyCiwi4hERoFdRCQyCuwiIpFRYBcRiYwCu4hIZBTYRUQio8AuIhKZM9p40rPPPts2bdrUxlOLiATr\n0KFDvzSz2aLHtRLYN23ahIWFhTaeWkQkWCSPlnmchmJERCKjwC4iEhkFdhGRyCiwi4hERoFdRCQy\nrWTFVHXLw4s4cMeT+PnyCjbMTOPqnVtw+fa5tpslIuKF4AL7LQ8v4pqbHsXK6nEAwOLyCq656VEA\nUHAXEYHDoRiSUyQfJnmrq2NmOXDHky8G9b6V1eM4cMeTdT6tiEgwXI6xfwjAEYfHy/Tz5ZWxvi8i\nkhongZ3kOQAuAfAVF8cbZcPM9FjfFxFJjase+xcAfBTACUfHy3X1zi2Y7kyd8r3pzhSu3rml7qcW\nEQlC5cBO8lIAz5rZoYLH7SG5QHJhaWlp4ue7fPsc9u3eirmZaRDA3Mw09u3eqolTEZEemlm1A5D7\nALwXwBqA9QBeCeAmM/uHvN+Zn583FQETERkPyUNmNl/0uMo9djO7xszOMbNNAN4D4J5RQV1EROql\nlaciIpFxukDJzO4DcJ/LY4qIyHjUYxcRiUwwJQVUH0ZEpJwgAvuo+jAAFPBFRAYEEdjz6sN88juP\n4/m1EyoIJiIyIIgx9rw6MMsrqyoIJiIyJIjAPm4dGBUEE5GUBRHY8+rDvOrMTubjVRBMRFIWxBh7\nf7x8eJIUwCmTqoAKgomIBBHYgW5wz5sQVVaMiMhJwQT2PKMCvoikQ2tdTgo+sIuIaC/kUymwi0gw\n8nrlo/ZCVmAXEfHUqF659kI+lQK7iHihaIx8VK98w8w0FjOCeKqpzwrsItK6rN74h284jKtuOIy5\nXpDP630vLq9gZrqDzhSxevzkjnAppz5HFdg1Ky4Slv45m9Xb7ofo/pDLzJkd/ObYauZxlldW0VlH\nvOrMDpaPrSZ//kcT2DUrLhKW4XN2lJXV43jpGesw3ZnKffzqCcOZLzkDD3/8ba6bGpwgSgqUMWr8\nTUT8k3XOjvLblVXs270VcyPGzVOdLB1WObCTXE/yByR/RPJxkp9y0bBxaVZcJCzjnpuG7sXg6p1b\ncoN7qpOlw1z02J8HsMPM/hTANgC7SP65g+OOJe8N1Rst4qdR5yZzvt8fYr34/NnMwoCpTpYOqxzY\nreu53ped3j8b8Su1yKsAqTdaxE955+wXrtiGz1+xLbdXvrJ6HPc+sfTisAwBzM1MY9/urZpP63Ey\neUpyCsAhAK8D8GUze9DFcceRVwFSb7SIn4rO2cu3z2Hz3tsye4k/X14ZWScq9Qw5J4HdzI4D2EZy\nBsDNJC8ws8cGH0NyD4A9ALBx40YXT3saFQQTCUvROTvJwiNlyDnOijGzZQD3AtiV8bODZjZvZvOz\ns7Mun1ZEIjXJEKsy5Nxkxcz2euogOQ3grQCeqHpcEZHLt8+NPZauDDk3QzGvAfC13jj7OgDfMrNb\nHRxXRGTsIVbVjXGTFfOImW03swvN7AIz+7SLhomITCJr+IbojrVftP8e3PLwYjsNa1A0JQXErbJZ\nBalnH4h/BrNtFpdXQJxed2bwcTGKpqSAuNPPKlhcXoHh5Mkw3NMp+ziRpl2+fQ73792BuZnp09Il\nU5hIVWCX05TNKlD2gfgu1YlUBXY5TdmTIdWTRsKRaqkRBXY5TdmTIdWTRsKRaqkRBXY5TdmTIdWT\nRsIxSR58DJQVI6cpW3dH9XkkBCmWGqFZ44UYMT8/bwsLC40/r9RLqY8i9SJ5yMzmix6nHnviXAVj\nFV4S8YfG2BPmMg9dqY8i/lBgT5jLYKzURxF/KLAnzGUwVuqjiD+iDey3PLyIi/bfg817b0um8M+4\nXAZjpT6K+CPKwK4aJuWUDcZlLpKp5guL+CjKrJhRY8cKNCeVyUMfJ9slxXxhER9FGdg1kVdeUTDW\nRVIkPFEOxWgizx1dJEXCE2Vg10SeO7pIlqcJe/FFlEMxqmEyWt5q06zvX71zyylj7IAuklm08lZ8\nUrlWDMlzAXwdwB+juwPVQTP74qjfUa2Y9gwHIKAbqN/1xjnceGjxtO/v270VwKkXyYvPn8W9Tyzp\nojngov33ZG6gPDczjfv37mihRRKjJmvFrAH4VzN7iOQrABwieZeZ/djBscWxvMnQbzz4NI4PXeT7\nk6T3793xYuBWzzSb5iLKU7G4+lUeYzezX5jZQ73//x7AEQB6lzyVF2iGg3re48uWIUhtvFlzEeVo\njUkznE6ektwEYDuABzN+tofkAsmFpaUll08rY8gLNFNkqceX6ZmmePJqwr4cFYtrhrPATvLlAG4E\ncJWZ/W7452Z20MzmzWx+dnbW1dPKmPIC0JV/dm6pwFSmZ5riyauVt+VoyKoZTrJiSHbQDerXm9lN\nLo4p9RiVMTT/2j8qHPsskyUT+8mbN0aslbfFNsxMZ04yNzVklcr4fuXATpIAvgrgiJl9rnqTpG55\nAahMYCqTStr2yVsnTR5X02b6bErvnYse+0UA3gvgUZKHe9/7mJnd7uDYzqRypW5C0QUg5tx3lVio\nps01Jim9d5UDu5l9H0D2zJsnUrpS+yDmBWKxDzPVxYeOVUrvXZQrT4eldKX2RazjzXnDTIbuIqVY\nLmAu+dKxinmIcFiUtWKGpXSllnr08/IXl1dyb09TSOuchC9ZUimlpCYR2LV4RKoYzMsHur3zvOAe\ne1rnJHzpWKWUkprEUEzMk3lSv6we56gKS7oTPJVPQyCxDhEOS6LHntKVWtwbN1DrTvBUKQ2B+CKJ\nHjuQzpV6kA+ZCDHI63HOTHfw/NoJ3QkWiDlLylfJBPbU+JKJEIO8obxPvuMNABSwykixY9UmBfZI\nKcXTnaIep/6e4hsF9kj5kokQC/U4JSRJTJ6mKG8Cr7+QRrnWIvFKPrDHuiFEViZCnxbSiMQt6aGY\nmCcYB8eFszI6NN7eHGUnSdOSDuyxTzD2x4U3770tc0GNxtvr0w/m/RIE/b9/TJ0H8VfSQzGpTDCq\npMJkJh2myypBMEhlB6RuSQf2VAKeVv6Nr8q+rVl3gsNi6zyIX5IO7KkEPJVUKK/fS7/qhsMTVyQs\nE7Rj6zyIX5IcYx+czDpruoP1nXVYPrYa9cSW8rCLDU+mZykbtLMmrPti7Dz0aaLYD8kF9uGTd3ll\nFdOdKXz+im36ANYklJO9zBBKmZ52VgmC/gTqnMevv6qYs8xC4ySwk7wOwKUAnjWzC1wcsy6xZ8L4\nJqSTvag3XrannWrRK51b/nDVY/8vAP8B4OuOjlebVDJhfBHSyT5qCGXcnnbZoa9Q7mbK0LnlDyeB\n3cy+R3KTi2PVzaei/ykI6WTPq+LocqJ5eH7nDy+sYfV4NyHS57uZMnRu+aOxrBiSe0gukFxYWlpq\n6mlPk0omjC9CSimtO3toOIVyeWX1xaDeF3KOe9a5RXQvWD6W64i1nAgA0GzUJl9jHKjbY7+1zBj7\n/Py8LSwsOHneScR0++u7rEwT173gUPQ3wy5CAD/bf0n9DapB3opbwK/3PetzGcIEN8lDZjZf9Ljk\nsmIApf41KdWJxCxlh598vJspq39uZV3EfJpbGbWPbehDYkCigV2apQtpV1F+OxDPsKDvcytF7fDp\nIjQJJ2PsJL8B4H8AbCH5DMl/dnFckZhkjUF31hGvOrMT3Ypg3+dWyrTDl4vQJFxlxVzp4jgiMSsa\nluqPT3/4hsPBD1nlZRj5cjeS1b5hvlyEJqGhmMiENDEcUltdyRuWCmkhVxm+z60M71eQNdHry0Vo\nEs6yYsbRdlZMrELKQAmprU3Iy5iZm5nG/Xt3tNCitITSyVBWTIJCWuUZUlub4PtkY+xim+BPumxv\nbEIKDiG1tQm+TzZKWBTYIxJScAiprU3QimhxSYE9IiEFh5Da2oSQN0OJeWl+qDTGHhHfMxGANDc5\nKWtUxoyv72ls2TyxUFbMAJ9PoBgoE2Z8vv/NlM3TrLJZMRqK6amyebGUMyoTRrL5/jfTJLifFNh7\nfD+BYqAgMD7f/2aaBPeTAnuP7ydQDBQExuf73yzGSfAYJoMV2Ht8P4FiEGMQqEs/uPSXuw/y6W8W\ncjZPlliGZJUV0+N70aIYhJC144PhCVOD35tAxLRqM5YV0QrsPQo6zYgpCNQlbxMIZZrUL5YhWQX2\nAQo64oNYgkuIYtmQW2PsIp7RfE97YpkHUo9dvND04jCfF6Npvqc9sQzJKrBHwOcgVUbTy9J9XwYf\nS3AJVQxDsk5KCpDcBeCLAKYAfMXM9o96vK8lBULk+5LzMppelp73fP3nVBAVXzVWUoDkFIAvA3g7\ngNcDuJLk66seV8qJYcVs05OFo44bat5y02JYxBMzF5OnbwLwlJn91MxeAPBNAJc5OG6rQvngxpBB\n0fRkYdFxQ7swNi2WRTwxcxHY5wA8PfD1M73vBSukD24MGRRNZyJkPd+wkC6MTYvhLjF2jaU7ktxD\ncoHkwtLSUlNPO5GQPrgxpGc1vSx98PnyhHRhbFoMd4mxc5EVswjg3IGvz+l97xRmdhDAQaA7eerg\neWsT0gc3lgyKpjMR+s+XN/kc0oWxabEs4omZi8D+QwDnkdyMbkB/D4C/c3Dc1oT2wY0hPasJo9JC\nQ78wNkl59v6rHNjNbI3kBwHcgW6643Vm9njllrVIH1z/jZu7X5S7rkBeni6G/tPWeDlCX/QTs0ly\n97WFm8SgbB67Vp7mUC/OX5OUVg1p3kSkKgV2Cc4kQTq0eRMf6S42HKruKMGZJHc/hrTQNoW0tkPU\nYxfPZfUSiya3Y85+aavXHMvOQqlQYBdv5WWy7Nu9Fft2b80McDFnv7RZlVJzFGFRYBdvjeol3r93\nR2Ywi7ln2eZrS3WOItR5BY2xi3f6BdjySuuO6iXG3LNs87WlOEcR8ryCAnugQqk+Oa7BkynPqF5i\nDEXR8rT52pqu5+ODkGpGDdNQTIB83wGoiqyTaVBRLzFrYpXo/o0u2n9PMLfSWdpeER36HMW4Qr77\nU489QCH3JIqMOmnK9BKHKzcSQH9tdUi30llS7DW3KeS7P/XYAxRyT6JI3iTdOEv/+z3LrHH60CdS\nU+s1t6ntO6Qq1GMPUMg9iSIuJ+livgBK/UK+Q1KPPUAh9ySKuFxIlGqKnrgT6h2SAnuAYllFmcfV\nyRTzBVBkFAX2QIXak2hS7BdAkTwK7BK1mC+Aoa6KDFVIf28F9jGF9OamIsX3JOa1DD4K7e+trJgx\nhLzEOFapvid5axmuuuFwVCuRfRHa2pFKgZ3ku0k+TvIEycLtmkIX2pubglTfk1Epm6lc3JoUWups\n1R77YwB2A/ieg7Z4L7Q3NwWpvidFKZspXNyaFNrakUqB3cyOmFkyn57Q3tzYZBU+S/U9yVrINSz2\ni1uTQqtuqTH2MYT25sYkbyz94vNnk3xPhmviZIn94tak0Fah0sxGP4C8G8CrM350rZl9u/eY+wB8\nxMwWRhxnD4A9ALBx48Y3Hj16dNI2tyrFDAwf5NVnn+u9Bym/J8MZG0D34jYq8OhzHCaSh8yscD6z\nMLCXfLL7UBDYB83Pz9vCQqmHigAANu+9DVmfVAL42f5Lmm6Od/ICddb3AYx9IRA/lA3symOXIKju\ny2hZC7Hycq/Xd9ZFu32gdFUK7CTfCeBLAGYB3EbysJntdNIyAXBqT+ys6Q5IYPnYanK3z6r7Mr68\nVNC8jUxi2IxEuioFdjO7GcDNjtoiQ4Z7XMsrqy/+zPeVb66p7sv4JsmKSe1zFSsNxXisaJu41G6f\nY677Uoe84auZ6Q6eXzuR+9lK7XMVI6U7ltDWxtFlelzKVZY8eem5n3zHGwpTJfW5Cpt67AXaLP6T\n1+MafoxIlqLhq7ztAwF9rkKnwF5gVC2SugN71oThIE0eSpGi4StNSsdJgb3AqFokdS/yGO5xpZwV\nI/XQpHScnCxQGldIC5TyblWzJqC0yENE6lR2gZImTwvkTUCRSLJcrKSlrcQBqUaBvUBe8Z/lY6uZ\nj1c2gcQi1U1MYqAx9hKyJqAO3PGksgnES67mftpMHJBqFNgnpGwC8VHV9NzBi0Le7JvuSv2vjqmh\nmAmFVp9Z0lBlq8DhoZc8qd+VhjBEpR57BVriLr6pslVgUQkLQHelQBhDVOqxi0SkylaBo4K/7kpP\nCmGfXfXYRSJSZe4nr4TF3Mw07t+7w2k7QxbC3gDqsYtEpMrcj/b0LSeEv5N67CKRmXTuR+UFygnh\n76SSAiIigVBJARGRRGkoxkO+L34QEb9V3cz6AIC/BfACgP8F8E9mtuyiYalqc2MPiZs6DOmoOhRz\nF4ALzOxCAD8BcE31JqWtyspBkTwhrJYMlY8VMCsFdjO708zWel8+AOCc6k1KWwiLHyQ86jDUw9cL\npsvJ0/cD+G7eD0nuIblAcmFpacnh08alyspBkTzqMNTD1wtmYWAneTfJxzL+XTbwmGsBrAG4Pu84\nZnbQzObNbH52dtZN6yMUwuIHCY86DPXw9YJZOHlqZm8Z9XOS7wNwKYA3WxtJ8ZEYnNg6a7qD9Z11\n2ttUnFGZ6Xr4Wl6galbMLgAfBfBXZnbMTZPSM5wJs7yyiunOFD5/xTYFdHEihNWSIfL1gllp5SnJ\npwC8FMCvet96wMw+UPR7Wnl6qrwNs1V8ScR/TaaRll15WqnHbmavq/L70uXrOJ2IFPNxXwatPG3J\n4FV+HYnjGXdObY/TiUiYFNhbMDymnhXUfRinE5EwKbA7Ms44W94WZFMkTphpYktEKlFgd2Dc+i55\nY+cnzPCz/ZfU11ARSYLK9jow7uozLRYRiZMvdWMU2B0YN6tFq0tF4uNT3RgFdgfG7YFX2ZdSRPzk\nU90YjbE7MMnqMx9zX0Vkcj6tR1GP3QH1wEXEp7kz9dgdGeyB91MfP3zD4VNSF7WDjUi8fKobo8Du\nWF7q48LRX+PGQ4va8k4kUj4VWqtUBGxSMRcByyvoNZVTNkCFvkSkrLJFwDTG7ljeRElWUB/1eBGR\nSSmwOzbuRIkWJYmIawrsjmUtPsqjRUkiUgdNnjo2OIGSNdbeN6esGBGpiQJ7Dfqpj5v33oaskXUC\nmjAVkdpoKKZGPi1YEJF0VArsJD9D8hGSh0neSXKDq4bFQMW+RKQNVXvsB8zsQjPbBuBWAB930KZo\nqNSAiLSh6mbWvxv48mVA5pBy0lTsS0SaVnnylORnAfwjgN8CuHjE4/YA2AMAGzdurPq0IiKSo7Ck\nAMm7Abw640fXmtm3Bx53DYD1ZvaJoieNuaSAiAgw3j7IZZUtKeCsVgzJjQBuN7MLih6rwC4iMRsu\nBggAnXXEy9efgeVjqxMH+kZqxZA8b+DLywA8UeV4IiIxyNpNafWE4TfHVhvZNq9qVsx+ko+RfATA\n2wB8yEGbRESCVqa4X53b5lXNinmXq4aIiMRiw8z0yJIifXVVd9XKUxERx8oWA6xrFbpqxYiIODa8\nm9JZ0x384YU1rB4/maxS5yp0BXYRkRoML05scs9jBXYRkQY0uQpdY+wiIpFRYBcRiYwCu4hIZBTY\nRUQio8AuIhIZZ0XAxnpScgnA0Ql//WwAv3TYnDbptfgnltcB6LX4qsprea2ZzRY9qJXAXgXJhTLV\nzUKg1+KfWF4HoNfiqyZei4ZiREQio8AuIhKZEAP7wbYb4JBei39ieR2AXouvan8twY2xi4jIaCH2\n2EVEZIQgAzvJz5B8hORhkneS3NB2myZF8gDJJ3qv52aSM223aRIk303ycZInSAaZvUByF8knST5F\ncm/b7ZkUyetIPkvysbbbUgXJc0neS/LHvc9WsDu0kVxP8gckf9R7LZ+q9flCHIoh+Uoz+13v//8C\n4PVm9oGWmzURkm8DcI+ZrZH8dwAws39ruVljI/knAE4A+E8AHzGzoHYrJzkF4CcA3grgGQA/BHCl\nmf241YZNgORfAngOwNfLbC7vK5KvAfAaM3uI5CsAHAJweaDvCQG8zMyeI9kB8H0AHzKzB+p4viB7\n7P2g3vMyAOFdnXrM7E4zW+t9+QCAc9psz6TM7IiZ1bOBYzPeBOApM/upmb0A4JvobtAeHDP7HoBf\nt92OqszsF2b2UO//vwdwBEAzdW8ds67nel92ev9qi1tBBnYAIPlZkk8D+HsAH2+7PY68H8B3225E\nouYAPD3w9TMINIjEiOQmANsBPNhuSyZHcorkYQDPArjLzGp7Ld4GdpJ3k3ws499lAGBm15rZuQCu\nB/DBdls7WtFr6T3mWgBr6L4eL5V5HSKukXw5gBsBXDV0tx4UMztuZtvQvSt/E8nahsm83UHJzN5S\n8qHXA7gdwCdqbE4lRa+F5PsAXArgzebxpMcY70mIFgGcO/D1Ob3vSYt649E3ArjezG5quz0umNky\nyXsB7AJQywS3tz32UUieN/DlZQCeaKstVZHcBeCjAN5hZsfabk/CfgjgPJKbSb4EwHsAfKflNiWt\nN+H4VQBHzOxzbbenCpKz/Yw3ktPoTtLXFrdCzYq5EcAWdLMwjgL4gJkF2bsi+RSAlwL4Ve9bD4SY\n4UPynQC+BGAWwDKAw2a2s91WjYfk3wD4AoApANeZ2WdbbtJESH4DwF+jW0Xw/wB8wsy+2mqjJkDy\nLwD8N4BH0T3XAeBjZnZ7e62aDMkLAXwN3c/WOgDfMrNP1/Z8IQZ2ERHJF+RQjIiI5FNgFxGJjAK7\niEhkFNhFRCKjwC4iEhkFdhGRyCiwi4hERoFdRCQy/w90LE7xbJ7y7wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f39812db208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x, y, 'o');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Linear Regression\n",
    "=================\n",
    "\n",
    "The first model that we will introduce is the so-called simple linear regression. Here, we want to fit a line to the data, which \n",
    "\n",
    "One of the simplest models again is a linear one, that simply tries to predict the data as lying on a line. One way to find such a line is `LinearRegression` (also known as [*Ordinary Least Squares (OLS)*](https://en.wikipedia.org/wiki/Ordinary_least_squares) regression).\n",
    "The interface for LinearRegression is exactly the same as for the classifiers before, only that ``y`` now contains float values, instead of classes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we remember, the scikit-learn API requires us to provide the target variable (`y`) as a 1-dimensional array; scikit-learn's API expects the samples (`X`) in form a 2-dimensional array -- even though it may only consist of 1 feature. Thus, let us convert the 1-dimensional `x` NumPy array into an `X` array with 2 axes:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before:  (100,)\n",
      "After:  (100, 1)\n"
     ]
    }
   ],
   "source": [
    "print('Before: ', x.shape)\n",
    "X = x[:, np.newaxis]\n",
    "print('After: ', X.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, we start by splitting our dataset into a training (75%) and a test set (25%):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we use the learning algorithm implemented in `LinearRegression` to **fit a regression model to the training data**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "regressor = LinearRegression()\n",
    "regressor.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After fitting to the training data, we paramerterized a linear regression model with the following values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Weight coefficients:  [-0.07995503]\n",
      "y-axis intercept:  0.362327766706\n"
     ]
    }
   ],
   "source": [
    "print('Weight coefficients: ', regressor.coef_)\n",
    "print('y-axis intercept: ', regressor.intercept_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since our regression model is a linear one, the relationship between the target variable (y) and the feature variable (x) is defined as \n",
    "\n",
    "$$y = weight \\times x + \\text{intercept}$$.\n",
    "\n",
    "Plugging in the min and max values into thos equation, we can plot the regression fit to our training data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGhxJREFUeJzt3X2MXFd5BvDn2dlZe20Tm+AVJLGNoQRUZLmhrFKq0lJq\nPlwaSElFBC1UiAoLqRSISgMhkjFEaUGRCOFDoi5EgBSgRnHER4OaBIJSJAxZU8c4JKERVYjTiGxI\nYmJ2k/16+8fMeufj3pk7c8+de865z0+K4p0d3zmz63nPuee85z00M4iISDzGym6AiIi4pcAuIhIZ\nBXYRkcgosIuIREaBXUQkMgrsIiKRUWAXEYmMAruISGQU2EVEIjNexotu3brVdu7cWcZLi4gE6+jR\no4+a2VS/55US2Hfu3ImZmZkyXlpEJFgkH8jyPE3FiIhERoFdRCQyCuwiIpFRYBcRiYwCu4hIZMIM\n7McPAdfuAg5safz/+KGyWyQi4o1S0h1zOX4I+NZ7gMX5xtenHmx8DQC7Ly2vXSIinnA2YidZI/nf\nJL/t6pqJvvvRtaC+anG+8biIiDidinkvgHscXi/ZqZODPS4iUjFOAjvJbQD+AsDnXVyvp83bBntc\nRKRiXI3YPwngcgArjq6Xbs9+oD7Z/lh9svG4iIjkD+wkLwLwiJkd7fO8fSRnSM7Mzs4O/4K7LwVe\n/ylg83YAbPz/9Z/SwqmISBPNLN8FyH8B8DYASwDWAzgLwGEze2va35menjYVARMRGQzJo2Y23e95\nuUfsZnaFmW0zs50A3gzge72CuoiIFCvMDUoiIpLK6QYlM/s+gO+7vKaIiAxGI3YRkcgosIuIRCb8\nwK6CYCIibcIrAtZKBcFERLqEPWJXQTARkS5hB3YVBBMR6RJ2YFdBMBGRLmEHdhUEExHpEnZgV0Ew\nEZEuYWfFAI0grkAuIscPNRInTp1sTMfu2V/Z2BB+YBcRUepzm7CnYkSkWtI2JCr1uY1G7CIShl6j\ncqU+t9GIXUT80K88SK9RuVKf22jELiLlSxqNH94HHH5nI9ttz/4eo/IHgcmzgdoEsLyw9niFU5/j\nGrGrIJhImJJG42ge27k65TL5zPS/P/8YYNYI8Ep9jmjErlVxkXD1mwtfnAfGJxuj8K4OoGllEZjY\nCHzgf923LzDxjNi1Ki4Srixz4fOPt2xITFHRxdJOuQM7yfUkf0zyLpJ3k/yIi4YNTKviIuFKKg/S\nafO2xt33ZSfSg3tFF0s7uRixPw3gz8zs9wBcAGAvyZc5uO5gtCouEq628iAAwPbvdy6Eqk5UT7kD\nuzWcbn5Zb/5nea87MP2iRcK2Oho/cAq45GDvGlCqE9UTzfLHYJI1AEcBvADAZ83sA72ePz09bTMz\nM7lft4tqRYgIEG0sIHnUzKb7Ps9FYG950S0AbgLwD2Z2ouN7+wDsA4AdO3a89IEHHnD2uiIiZ3Rm\nyAGNu/cIRvRZA7vTrBgzewLA7QD2JnzvoJlNm9n01NSUy5cVEVmjDDknWTFTzZE6SE4CeDWAe/Ne\nV0RkKMqQc7JB6RwAX2rOs48BOGRm33ZwXRGRwW3e1tigmPR4ReQO7GZ2HMBLHLRFRCS/PfuT59gr\nlCEXz85TERFAqZCIqVaMiMiqih+ZqRG7iEhkFNhFRCKjwC4iEhkFdhGRyCiwi4hERoFd3NHRhCJe\nULpj1bmqgqejCUW8oRF7la0G41MPArC1YDzMSFuFl0S8ocBeZS6DsQoviXhDgb3KXAZjHU0o4g0F\n9ipzGYx1NKGINxTYq8xlMFbhJRFvKCumylaDrquzISteeEnEFwrsVadgLBIdTcWIiERGgV1EJDIK\n7FWUtvVfJQFEopB7jp3kdgBfBvBsAAbgoJldl/e6UpC0rf+/PALc9ZVsJQFclSEQkUK4GLEvAfhH\nM3sxgJcB+HuSL3ZwXSlC2m7To1/MtgvVZRkCqSbdGRYud2A3s4fN7CfNPz8J4B4A5+W9rhQkbVep\nLWd7ftYyBPrwShINDEbC6Rw7yZ0AXgLgRwnf20dyhuTM7Oysy5eVQaTtKmUt2/OzlCHQh1fSqFjc\nSDgL7CQ3AbgRwPvM7Ded3zezg2Y2bWbTU1NTrl5WBpW22/Slb8+2CzVLGQJ9eCWNisWNhJPATrKO\nRlC/wcwOu7imFCRt6/9Fn8hWEiBLGQJ9eCVN2cXiKjJF6CIrhgC+AOAeM/tE/iZJ4dJ2m2bZhZql\nDMHmbc1pmA6q9Ch79rdnZQGjKxZXocNgXJQU+CMAbwPwU5LHmo99yMxudnBtd5Si506/DqDMD6/4\nzXV9okH0miKMLBbkDuxm9gMAdNCW4lSop/ZCmR9e8ZMPA6sKTRFWowhYhXpqb1SluJgPAct3vgys\nKjRFWI2SAhXqqaUgSYtuSuvMxpcsqQodBlONEXuFemopQNqIc3xSd4JZ+DKwqtAUYTUCuxbzJI+0\nEWfnY6t0J9jOp4FVRaYIqzEVo2PbJI9BA7XuBNtVaArEF9UYsQOV6anbaGHPjbQR5+TZwNK87gT7\nqdAUiC+qE9irxpdMhBikTeX9+ccbf1bA6q+KA6sSKbDHSime7vQbcernKZ5RYI+VL5kIsdCIUwJS\njcXTKiq72JKIlEaBPdZqb8pEEKmsak/FxLzAqEwEfyg7SUaMZjbyF52enraZmZmRv26Xa3elbJzY\nDlx2YvTtkXicCeYPolEjr+VzVp/UPgoZCsmjZjbd73nVnorRAqP0Muw0XVsNGaAtqAM6TUoKV+3A\nrgVGSZOnwFdSqmknDR6kQNUO7FpglE6ro/TD7xy+ImGWoK3BgxSomounrYtZk89sVOmbf1wLW1XX\nuZieJGvQTlq7WaXBgxSseiP2zlvs+cca9T4uOdhYMFVQdy+UlNIsUyhZRtpJd4Krh4ypAJ2MgJMR\nO8nrAVwE4BEz2+XimoXRVvvRCimltN9oPOtIW6mmUjJXUzFfBPAZAF92dL3iKBNmtELqSHtNoWze\nPlhwzlqCQDnuUgAngd3M7iC508W1CudT0f8qCKkjTavi6HLqpHN9Z+E0sLzQ+J7PdzMSlJHNsZPc\nR3KG5Mzs7OyoXrabMmFGK6SU0qIPZEla31kN6quU4y4OjCwrxswOAjgINHaejup1u2j+c7RCO5aw\nyCqOWRZnAT/vZmIX2ZRYNdMdVYJ1dNSRrskasH28m4lZSAv8GVUzsMtoqSNt6JffDvh9NxOrkBb4\nM3Iyx07yqwB+COBFJE+S/DsX1xWJStL6zli9cXaqDlkvT0gL/Bm5yop5i4vriESt37TU6jzv4X3V\nnrIatQgz5TQVE5uQFoFCaqsradNSEc7zBiO0Bf4MqldSIGZ5KhKOWkhtHYVe87xSrKLTXEugEXtM\nQloECqmtoxDhPG9QIlvg14g9JiEFh5DaOgohbeQS7wU1Yv/B/zyK+371JDZM1LBhoobJeg0bJsYx\n2fx6w0St+edxTNZrqI2x7CaPVkiLQCG1dRQinOeV8gQV2G8+8TC+8qNfZn7+uvGxZsBfC/6NzqDj\nsYkaNtTHWzqG1cfHU/9OvebhzU5IwSGkto5CyBu5qrgI7rmgDrNeXF7B3MIy5heWMbew1Pjz4nLz\nscbXa99fxtzi0pk/J/+dtceeXloZqC31Gs/cMbR2CJMT49hQr3U/1tKxrHUy4113Ghsmalg3PgZy\nyLsN3z9knUWwAB1y0o/Pv9Okw0l0WHdhsh5mHVRgL9LyimHx2L+jfvtVGHvyISxuOhcnf/+f8H/b\nX4+5haUznUFiJ7KY9NhapzK3sDxQW8aIZgfQGfiTOoO1qafODuLM9+vtdyxjZU1RKQgMzvef2bW7\nUqbUtjcOrhGnsgb2oKZiilQ78XXUvvO+Mx+gidMP4fk/vALP37ox9wfIzPDU4krXHcPcQsIdxeJy\nW4cw3/F3Hvvt/NpjzU5leWWwznl9faytM2hbm5iotdxxZOlYWjqReg3jvaaolAkzON9/ZloE95IC\n+6oCP0AkMdkMfs/KdaVuZoaF5ZWOzqC7k2i7o1hsmZZq+TuPPPlU12MLy4NNUU3UxtrXLlruGL54\n6iSS7hXs1EncePRkWweRtCg+UcsxRRUq3wOnFsG9pMC+yvcPUAqSWDdew7rxGrZscH/9xeUVzLfd\nRbRPMc0vtncQbZ3I4lon88TcAh7hVjzbumvxP7TyLLz/63f1bUttjNhQT1+7SLzL6Ogk2u4yWhbF\n19c97TR8D5wxLoL7vKaRkQL7Kt8/QCWp18ZQr43hrPX1/Bc7/s9dQcDGJ3HWa67CHb/zSsx1dRIt\nU1UJdxmrHcvpp5cw++TT7R3L4jIGWT5ic12jNUuqK0OqXut+LEPq7YZh1jXOBJcH0TgIu+XN+BQ4\nQ87mSRJJaQcF9lUxjjx8kxAEuGc/ztp9Kc5y/FJmhqeXVs50EHNJnUXGdY3H5xZzr2sMknp7weO3\n4lX3X436ylOr7wbWDO6Lm87D43/4Qdhz34DJ+UU/Um9j2rXp+5pGRgrsq2IbefhqREGAJNbXa1hf\nr+HsjRNOr+1yXWP29NOYW5hre+x7tetQH3uq7TUJw8mVrXj5o9cA3wLwre+e+V6e1NuuDKqOu5Nc\nqbchCnRKtpMCe6uYRh5SmKLXNezArxMfP2/s1/jcW1+aeV3j1NwCHu7oWJ5aHGwxfIzoe5fRmi2V\n1EGkpeFO1mv+dRqRTMkqsIt4hinBhZu3Ye+u5+S69sqKtW/QG3Jd47cLS3j0dL51DaBjXSPlLqM1\nDTdxn0ZKGu5QJUUimZJVYJdq8jnzocDgMjZGbFw3jo3r3H/0O9c1krKn2qeiuu8yVr//8KnFro5l\nacB1jYnVdY2WDqF3Z1DD5MQf4IW7DmD3vddh/fzDWNh4Lh658HIsnPM6bDg1f2ZRfWLcw5IiLbTz\nNAY+Bykf+b6bE9DvNMHC0krfu4zWXeC9O5alMwvhcwvLWBiwpMj4GHvs/E7f7Dc5UcMfn78V52ye\n7P8iCUZaUoDkXgDXAagB+LyZfazX8xXYHQohSPkmbRs80NgKryBaOcsrlukuo7VD6LzLSOtY5hfb\nS4p86R0X4hUvnBqqnSMrKUCyBuCzAF4N4CSAO0l+08x+lvfakkEk6Vkj1SvDIdC85ZGL7I6iNkY8\nY30dz3CxX6PDyorhqaW1DmLrpnXOX6OTi4miCwHcb2a/MLMFAF8DcLGD65br+KHGyO7Alsb/fT2y\nLZL0rJHql+GgI+l607GGAxkbIzZMjGPrpnXYfvYGTE7Uin9NB9c4D0Drfe3J5mPhCukfrk7eGdye\n/Y3pql7UMabT+azeG9nSLsl9JGdIzszOdtcL8UpI/3CTglSA6Vkj1XZ4cQp1jOl0l+g9F4H9IQCt\nn5BtzcfamNlBM5s2s+mpqeEWDkYmpH+4EZ6wPhK7L23UC7/k39QxDkp3id5zkcx6J4DzST4PjYD+\nZgB/7eC65Qlt95l2zGbTa8EvooXAwkWyiSdmuQO7mS2RfDeA/0Qj3fF6M7s7d8vKpH+4/hs0K6Nf\n1T4F8uzUGXpPG5TSRJbOFZVhcvd1hJtEQEfj5aVRnL+Gyd0Pad1EJCe/Cx6IJBkmSGvBTypEgV3C\nM0yQVlqoVIgCu/gtaQfwMEE6lrTQUHZES6m0eCr+6rVIClRvcVsF3ypvpNUdB6XALpkok6Wdfh6j\n51l2XNbArqkY8c/qdENaad2qZrIos2e0QqoZ1UGBXfzS9mFKUdVMFmX2jFZINaM6KLCLX5I+TK0G\nzWSJabFRmT2jFfAdkgK7+KVnLvqAmSwB30oniiWzJxQB3yFp56n4JbUA2xALhDGeLqUd0aMTcM0o\njdjFLy6nGwK+lRYPBHyHpBG7+MVl5cDQyi+LfwK9Q1JgF/+4+jAFfCstkoemYiReAd9Ki+ShEbvE\nLdBb6Uw82xUZvYB+3grsgwrolysR63cilLgV2M9bUzGDiC0vWsKVlsp5+J3hb8TyUWC7UHMFdpJv\nInk3yRWSfQvTBC+wX65ErFfKpgYc7gWWOpt3xH4CwCUA7nDQFv8F9suViPVL2dSAw63AdqHmCuxm\ndo+Z3eeqMd4L7JcrEUvayNVJAw53AqvTozn2QQT2y5WItaVyptCAw53AUmf7ZsWQvA3AcxK+daWZ\nfSPrC5HcB2AfAOzYsSNzA73iclekSF6rqZxpJytpwOFWQKmzTk5QIvl9AO83s0zHIukEJRHH0tJw\nlZ4blawnKCmPXSQGSaPJwHKvxZ1cgZ3kGwF8GsAUgP8geczMXuukZdLQOuKafGbjsfnHNfqS/mIs\nWyyZ5ArsZnYTgJsctUU6dY645h9b+55GX9KP0nMrS1kxPut3TJxylaUXpedWlgK7z7KMrDT6kjRK\nz60sBXafZRlZafQlaQLLvRZ3lBXjs6SDIlpp9CX9BJR7Le5oxO6zzhHX5NmN/zT6EpEeNGL3nUZc\nIjIgjdhFJN3xQ4367ge2qM57QDRiF5Fk2rkaLI3YRWLjapStg2WCpRG7SEzyjrJbS1ggpUCg9k54\nX1xNI3aRmOQZZXee6Zum6nsnAjj7WIFdJCZ56sP0K2EBaO8EEMQUlQK7SEzy1IfpGfy1d+KMAIqr\naY5dJCZJu5WzjrI3b2tOL3Q+vh247IS7NoYu9efkzxSVRuwiMclTH0ZFw7IJ4OekEbtIbIbdrawz\nfbMJ4Ofk5MzTQenMUxGRwWU981RTMSIikVFg95Hqc4hIDrkCO8lrSN5L8jjJm0hucdWwygpg84ME\nSgOGysg7Yr8VwC4z2w3g5wCuyN+kigtg84MESAOG4njYYeYK7GZ2i5ktNb88AsCfRM5QBbD5QQKk\nAUMxPO0wXc6xvwPAd9K+SXIfyRmSM7Ozsw5fNjI6WV6KoAFDMTztMPsGdpK3kTyR8N/FLc+5EsAS\ngBvSrmNmB81s2symp6am3LQ+RgFsfpAAacBQDE87zL4blMzsVb2+T/LtAC4CsMfKSIqPTQCbHyRA\neUoNSDpPywvk2nlKci+AywG8wszm3DRJdM6pOKcBQzE87TDzlhT4DIB1AG4lCQBHzOxduVslIu5p\nwOCepx1mrsBuZi9w1RARkSB52GFq52lZPMx9FZE4qLpjGXT6u4gUSCN2VwYZgXua+yoicdCI3YVB\nR+Ce5r6KSBw0Yndh0BG4NouIxMmTtTMFdhcGHYFrd6lIfDyqG6PA7sKgI/A851KKiJ88WjvTHLsL\nw+w+8zD3VURy8GjtTCN2FzQCFxGP1s4U2F3ZfSlw2QngwBONkfp3P9q9gOLJwoqIFMCjtTNNxbiW\nlvr4yyPAXV/RpiSRWHlUN4ZlVNqdnp62mZmZkb/uSFy7K7mMJ2uALXc/vnl7Y6QvItIHyaNmNt3v\neZqKcS1toSQpqPd6vojIkBTYXRt0oUSbkkTEMQV215IWUNJoU5KIFECB3bW21McelBIpIgVRVkwR\nVjcfHdgCIGlxmlowFZHCaMReJI82LIhIdeQK7CSvInmc5DGSt5A811XDouDRhgURqY68I/ZrzGy3\nmV0A4NsAFLFaqdSAiJQg72HWv2n5ciOSJ5SrTcW+RGTEci+ekrwawN8COAXglT2etw/APgDYsWNH\n3pcVEZEUfUsKkLwNwHMSvnWlmX2j5XlXAFhvZh/u96JRlxQQEQEadaMc143JWlLAWa0YkjsA3Gxm\nu/o9V4FdRKLWWQwQAGoTwMQmYP7xoQP9SGrFkDy/5cuLAdyb53oiIlFIOk1peQGYfwyjODYvb1bM\nx0ieIHkcwGsAvNdBm0REwpaluF+Bx+blzYr5K1cNERGJxuZtyeW7OxVU3VU7T0VEXMtaDLCgXegK\n7CIirnVuTpw8Gxirtz+nwF3oKgImIlKEzs2JBaQ/plFgFxEZhRHuQtdUjIhIZBTYRUQio8AuIhIZ\nBXYRkcgosIuIRMZZEbCBXpScBfDAkH99K4BHHTanTHov/onlfQB6L77K816ea2ZT/Z5USmDPg+RM\nlupmIdB78U8s7wPQe/HVKN6LpmJERCKjwC4iEpkQA/vBshvgkN6Lf2J5H4Dei68Kfy/BzbGLiEhv\nIY7YRUSkhyADO8mrSB4neYzkLSTPLbtNwyJ5Dcl7m+/nJpJbym7TMEi+ieTdJFdIBpm9QHIvyftI\n3k/yg2W3Z1gkryf5CMkTZbclD5LbSd5O8mfNf1vBntBGcj3JH5O8q/lePlLo64U4FUPyLDP7TfPP\n7wHwYjN7V8nNGgrJ1wD4npktkfw4AJjZB0pu1sBI/i6AFQD/CuD9ZhbUaeUkawB+DuDVAE4CuBPA\nW8zsZ6U2bAgk/wTAaQBfznK4vK9IngPgHDP7CclnADgK4C8D/Z0QwEYzO02yDuAHAN5rZkeKeL0g\nR+yrQb1pI4DweqcmM7vFzJaaXx4BUMyRKgUzs3vM7L6y25HDhQDuN7NfmNkCgK+hcUB7cMzsDgCP\nld2OvMzsYTP7SfPPTwK4B8B55bZqONZwuvllvflfYXEryMAOACSvJvkggL8BUMwxJKP3DgDfKbsR\nFXUegNZDKk8i0CASI5I7AbwEwI/KbcnwSNZIHgPwCIBbzayw9+JtYCd5G8kTCf9dDABmdqWZbQdw\nA4B3l9va3vq9l+ZzrgSwhMb78VKW9yHiGslNAG4E8L6Ou/WgmNmymV2Axl35hSQLmybz9gQlM3tV\nxqfeAOBmAB8usDm59HsvJN8O4CIAe8zjRY8BfichegjA9pavtzUfkxI156NvBHCDmR0uuz0umNkT\nJG8HsBdAIQvc3o7YeyF5fsuXFwO4t6y25EVyL4DLAbzBzObKbk+F3QngfJLPIzkB4M0Avllymyqt\nueD4BQD3mNknym5PHiSnVjPeSE6isUhfWNwKNSvmRgAvQiML4wEA7zKzIEdXJO8HsA7Ar5sPHQkx\nw4fkGwF8GsAUgCcAHDOz15bbqsGQfB2ATwKoAbjezK4uuUlDIflVAH+KRhXBXwH4sJl9odRGDYHk\nywH8F4CfovFZB4APmdnN5bVqOCR3A/gSGv+2xgAcMrOPFvZ6IQZ2ERFJF+RUjIiIpFNgFxGJjAK7\niEhkFNhFRCKjwC4iEhkFdhGRyCiwi4hERoFdRCQy/w9PVKqxyoPOBwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f3981256710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "min_pt = X.min() * regressor.coef_[0] + regressor.intercept_\n",
    "max_pt = X.max() * regressor.coef_[0] + regressor.intercept_\n",
    "\n",
    "plt.plot([X.min(), X.max()], [min_pt, max_pt])\n",
    "plt.plot(X_train, y_train, 'o');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar to the estimators for classification in the previous notebook, we use the `predict` method to predict the target variable. And we expect these predicted values to fall onto the line that we plotted previously:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_pred_train = regressor.predict(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f398118ab38>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuUVOW55/Hv001LF6A0EtaJ3AZWTkQiV231RDQzalRM\niFGiEY/JDHPG4CWOmnUkkqNLPTpOyJAVjbeVYExmMgtjNCpeT4xGMlG8RBDEC5KEiBFwJkRpAnah\nDf3MH9UF3dW169K1q2rvXb/PWiy6qnbv/VZ31/Pen23ujoiIJEdTvQsgIiLhUmAXEUkYBXYRkYRR\nYBcRSRgFdhGRhFFgFxFJGAV2EZGEUWAXEUkYBXYRkYQZVI+LfuxjH/MJEybU49IiIrG1evXqv7r7\nqGLH1SWwT5gwgVWrVtXj0iIisWVmb5dynIZiREQSRoFdRCRhFNhFRBJGgV1EJGEU2EVEEqYuq2Iq\ntXzNFpY8sYGtHWlGt6VYeOokzpg5pt7FEhGJhNgF9uVrtvCtB14l3bUXgC0dab71wKsACu4iIoQ4\nFGNmzWa2xsweDeuc+Sx5YsO+oJ6V7trLkic2VPOyIiKxEeYY+2XA+hDPl9fWjnRZz4uINJpQAruZ\njQU+D/wojPMVMrotVdbzIiKNJqwW+83AN4HukM4XaOGpk0i1NPd5LtXSzMJTJ1X70iIisVBxYDez\nOcBf3H11keMWmNkqM1u1bdu2AV/vjJlj+PbcqYxpS2HAmLYU3547VROnIiI9zN0rO4HZt4GvAnuA\nVuAg4AF3/0rQ97S3t7uSgImIlMfMVrt7e7HjKm6xu/u33H2su08A5gFPFwrqIiJSXdp5KiKSMKFu\nUHL33wC/CfOcIiJSHrXYRUQSRoFdRCRhYpcrJpcSgomI9BXrwK6EYCIi/cV6KEYJwURE+ot1YFdC\nMBGR/mId2JUQTESkv1gHdiUEExHpL9aTp9kJUq2KERHZL9aBHTLBXYFcRLT0eb/YB3YRES197kuB\nXURiI6hVXmjpswK7iEhEFWqVa+lzXwrsIhIJxcbIC7XKR7el2JIniDfq0mcFdhGpu3yt8W/8fC2X\n/3wtY3qCfFDre0tHmrZUCy3NRtfe/XeEa+Slz4kK7JoVF4mnfK3xbIjODrkMT7XQke7K+/0d6S5a\nmowRQ1ro6Oxq+M9/YgK7ZsVF4qvYWHi6ay+tLU2kWpr7VQBZXd3OkAMGseaaU6pRxFiJ9c7T3pQQ\nTCS+ShkL7+js4ttzpzKmwLGNOlmaq+LAbmatZvY7M3vFzF43s38No2Dl0qy4SHzlSw+Sa3RbijNm\njmHlohMDg3ujTpbmCqPF/iFwortPB2YAs83sH0I4b1mUEEwkvs6YOaZPa9xyXs+dCFWeqMIqHmN3\ndwd29Txs6fnnwd9RHQtPndRnjB30ixaJk97pQYothFCeqMIsE5crPIlZM7Aa+Hvgdne/stDx7e3t\nvmrVqoqvm0urYkQEkhsLzGy1u7cXPS6MwN7rom3Ag8B/dffXcl5bACwAGD9+/JFvv/12aNcVEcnK\nXSEHmd77t+dOjX1wLzWwh7oqxt07gBXA7DyvLXX3dndvHzVqVJiXFRHZRyvkwlkVM6qnpY6ZpYCT\ngTcrPa+IyEBohVw4LfZDgBVmtg54CXjS3R8N4bwiImXTCrkQAru7r3P3me4+zd2nuPv1YRRMRGQg\ntBQyQSkFRERASyFBgV1EEqjRb5mZmFwxIiKSocAuIpIwCuwiIgmjwC4ikjAK7CIiCaNVMRKapCZe\nEokbBfYGF1Yw1q0JRaJDQzENLBuMt3SkcfYH4+VrtpR9LiVeEokOBfYGFmYwVuIlkehQYG9gYQZj\nJV4SiQ4F9gYWZjBW4iWR6FBgb2BhBuPeNyM2YExbKhF3rBGJI62KaWBhZ8Fr9MRLIlGhwN7gFIxF\nkkdDMSIiCaPALiKSMBqKaUBBu02VEkAkGSoO7GY2Dvgp8HeAA0vd/fuVnleqI2jr/6q33+f+1VtK\nSgmgCkAk2sJose8B/tndXzazA4HVZvaku78RwrklZEG7TX/24jvsde/3/JInNvQJ2soJI5VSw6D6\nKg7s7v4u8G7P1zvNbD0wBlBgj6CgXaW5QT3o+EJpCHIrAH14JZcaBrUR6uSpmU0AZgIv5nltgZmt\nMrNV27ZtC/OyUoagXaXNZiUdX0oagjCTi0myKFlcbYQW2M1sGHA/cLm7/y33dXdf6u7t7t4+atSo\nsC4rZQrabXruMeNK2oVaShoCfXgliJLF1UYogd3MWsgE9WXu/kAY55TqCNr6/9/OmFpSSoBS0hDo\nwytB6p0sbvmaLcxa/DQTFz3GrMVPJ7YXGcaqGAPuAta7+/cqL5JUW9Bu01J2oZaShmB0W4oteYK4\nMj3KwlMn9Rljh9oli2uk8f0wVsXMAr4KvGpma3ue+xd3fzyEc4dGk3nhKVYB1PPDK9EWdn6icpQ6\n8Z8EYayKeRbIP/MWEY1UU0dBPT+8Ek1RaFg10hBhQ+w8baSaOioaJblYFAJW1EWlYdVIQ4QNEdgb\nqaaW6sgXwIFIBKyoi0rDqpGGCBsisDdSTS3hC2pxDh7UFImAFXVRaVg10hBhQwT2RqqpJXxBLc7c\n57LUE+wrSg2rRhkibIi0vbptm1Si3ECtnmBfuh9u7TVEix0ap6buTRN74QhqcY4Y0sLurm71BIto\npCGQqGiYwN5oorISIQmChvKu/cLhgAJWKRqxYVVPCuwJFZWVCElQrMWpn6dEjQJ7QkVlJUJSqMUp\ncdIQk6eNqN7JlkSkfho+sCc125tWIog0roYeiknyBKNWIkSHVidJrTV0YE/6BKPGhesnG8y3dKQx\nMnd5h2Q1HiS6Gjqwa4JRChloSzu3J5h7N9kkNR4kmhp6jF0TjBKkkvu25usJ5lLjQaqpoQO7Jhgl\nV3Yy/fKfrx3wfVtLCdpqPEg1NeRQTO8udtuQFgYPamJHuksTWw0udwgln1KDdr4UBFlqPEi1NVxg\nz/3wbu/sItXSzE3nzFBAr5K4rAopZQillJZ2vhQE2QnUMRF+/5IcoQR2M/sxMAf4i7tPCeOc1ZL0\nlTBRE6clpcVa46W2tLXUVOotrBb7/wRuA34a0vmqRithaitOFWmhIZRyW9qlLjWNS29G4iWUwO7u\nvzWzCWGcq9qilPS/EcSpIg3K4hhm7v7egXx4qoUPPtpD197Mgsgo92YkXmq2KsbMFpjZKjNbtW3b\ntlpdth+thKmtOC0prfYNWXKXUHaku/YF9axSV96IFFKzyVN3XwosBWhvb8/ds1EzGv+srbjdlrCa\nu3VLmZyFaPZmki5pQ2INtyoGtNW+llSR7ldqwI5ibybJ4jTBX6qGDOxSW6pIM4qtb4do92aSKk4T\n/KUKZYzdzH4GPA9MMrPNZvZfwjivSJLkm99paTJGDGnRTdbrKE4T/KUKa1XMuWGcRyTJig1LZcd5\nv/HztQ09ZFVrSVwpp6GYhInTJFCcyhqWoGGpJI7zxkXcJvhL0dBJwJKmkoyEtRanstZCoXFeqa5q\nL3Oth3i22NfdC7++HnZshuFj4aRrYNqX612quovTJFCcyloLSRznjZOkTfDHL7CvuxceuRS6ev7g\nd7wDDyyAB74GqYMzz6W39w/4vSuD1Ij9x5XydUwqjzgFhziVtRaSOM4r9ROrwP781uf547PXk2pt\nInXAEFLupLq7e/53hnT9jZR3kzJo3fEOzY9cuv+be1cG6ff3P1/K1zveyXx/Vr7eQrkVRxUqizgF\nhziVtRaSOM4r9WPutd8E2t7e7qtWrSr7+65//nru+/19JR8/uLubFEbKIbV3Tybod3vfCqGnUkh5\nN0Oyr/U5rtfjwcNJdaVJfdRJS/YiLSmY/o/wyt37K46S9SRzHT6u/N5FnkrFd2xmuw/D3Rlhu9jq\nH+Nm5nHcmRdHrpuZL/d52HlZ4iauk8lxLXccmdlqd28velycAntXdxfpW2aS3rmVtBnpJiNtTT3/\n93psRmef15pIG6SbmvJ/nxkfNpU3jzyod4WQWwH0fpzntVZ3hnTnvNY8mNQp/51UUwuDH/tnrJRK\nooRKxclUH30qj3IrjhDl3uTEHd3kpIgoB05V0LWVyMAO9B9jL2b4uMz/O94peNheYHevoN+572vr\nXyEEVA7Z4zr7VTjlVRpNPcG/f+Wwv6cxZN/XRqp7b55eSP9KZcigwbROPZemdT8r7efXkoIv3NI/\nuAdNXhepMDy9ne0+FIA2dtHBMABG2AdYTOYxai3qgXPW4qfzDqmNaUuxctGJdShRspUa2GM1xg7s\n/+D/+vqeYJ29N00eLalMsICilUEzMNSdoXsd6M5/rkGtfcfes6wZPDi5k9NTaeStHHpXCE29Kon8\nlUrHoEH9XttrFnjtft5/itbRIwN6FXkqhJXXk2pOkxqUIjUoxZAta0j97k5SXbtJHTCI1Afvknrs\nclKbniH16n0MKjCPYcDBtmvf0wez/+vAeYwiw09FV0bFfAVV1FcPaRI8muLXYs8V1Eosc1VMd3o7\n27uHYgZtfEAHQ3GHEU0f0JQ9F/SvICoaY89RYu+iN7dmunxvTm+hV4VQsMeRea2zwGtd5VQaQEu/\nyqFXj6HAa/sqlZZhpPZ8mJnLcN8/79HdTQtgQT/vfL2LvL27noZA7xVUlaySqnLFMXHRY3mbLQa8\ntfjzoV1noNRir63kDsVUSckfoAEOQ/T9+n369TSygQlKH2oaSKVSpHeRq2v4OHZ//XnSe9KZf7e1\n95mv6OxXcfQfhgp6bXeZQ1TNOZPd/SqHQSlSU8/e17tIvfBDUuntfSqHoEql1Z2yqrBiFQyUNo/x\nyVPgD78q2ODo8J5GRs+E+P/Y82VWH3RyJAJn1IeKBiLKcxoK7GWqecujUEtvgKti+h4fUHmUUxHk\nawXfNCV/j6LMCgMyA16791UOARVCvsoh28vI1xs58O/2VUIeNESXh2XnNfKtksqzgirV7aQgM7+R\ne9wBBzGkK53pdfS8FuYW724HM7Cweh0VinIgLFfUKyoF9jJF/Rc6IJX0LoI++PmGN8IajmpJwaBU\n/nmM3oIqkeHj4BuvAeDufHjzVNI7t5BuMjoD5zWC5zwKzYeUNa9Bz9LboB7HQJbe9nqtpfjl8wsa\nvur9N9O7R1HK31AM5zF6i/rQkgL7ACSp5VFVoQxHBVQkUHgoquIx9so50AWkmweRxus+rzGo1Aoh\nX+XQejCpM+7ITIxvWknq/ywh9VF63zkOcPoOUZVSiRerMMrqeWpOo085FNgltsL8gO87rsgKqnIV\nqmBK6XUUsAcCh6BqPa/RVGxeI6fiGJL9evAIUp9bkpnr+POLpJ67ldRHu3PO0TOvUWplkfeYXpv8\nguYrSvm76Xm+e8fmXosoojenocAukquSHkU5gQKq0lMIQ795jYNGkz53WWZe4n+fMYB5jb4Vh5fZ\n28i33yJwL8a+PRt9K5UhAb2RPrc0KVQRF6lUQpvTCBrWKoMCu0gBVR92K3UeI08rM7ORK5saIrP0\nFjKrYizsXkfvIZOgifFcAXMcDnyYDf7Dx5D+Tw9lKowfnUTayLNPI2iyPHfzX+brPWVWGgcU2qfR\nZ7NfziR43kql73DXgOc1egvaAFhAcjcoST+aGyhPTW5qMe3LAx77NeC3Qb/TavQ6sk66pnhPo0AL\n14BWd1oHtTLihGug7ROZFwaPKl5hlLCqqsua6bTuvEE/qHLIN//xfnNTv3N81DTweY2Sexw9r306\nvZuP792b+fn9+vqqTDSH0mI3s9nA98ls4PyRuy8udLxa7OFJ5GqeKgta+QCZ1Q8NXTFWY1VMsUns\nisbYyxBUeVgze31v3t5CZ75cVCVWKkEpRX7wf//CrPTu7MXhuo7S30KtWuxm1gzcDpwMbAZeMrOH\n3f2NSs8txUV9y3kUFdru3vC3pCuxp7F87yyWfHgLW3enGd1apDLskwakQEUw/h/KOKbMyfAiY+zN\nr9zNsK40w4JSilQgO6+RDfoj9/Y6//CxoV4rK4yhmKOBP7r7nwDM7B7gi0CsA3tchjeUq6N8Qbng\ns1QxFjagoaxSKoxyjynUuyil8ihaqeTZ5DcATcCQnsR9feqM3rmsQhZGYB8D9B5A2wwcE8J56yZO\nNxbWDSvKl++mFrlUMQaLTC9xIPMYQd8T9HyYcxohrIopVc0mT81sAbAAYPz48bW67IBE5g+3BLrz\nTvmyv8MlT2wIbLmrYgzWUL3ECibB6ymMFBZbgHG9Ho/tea4Pd1/q7u3u3j5q1KgQLls9cfrDTeId\n1mvhjJljWLnoRG4+Zwaplj4rnlUxFhFU6akyjI4wWuwvAZ80s4lkAvo84B9DOG/dxG14I2l3WK+W\nQvMmcZhPiQr1EqOv4sDu7nvM7BLgCTLLHX/s7q9XXLI60h9u9JU7uV1s3kSBvHSqDKNPO08DxGVV\nTCMayNr9qGftEymFdp5WSK246BrI5Hac5k1EKhVm/n+RmhhIkNaEnzQSBXaJnYEE6YWnTtLqF2kY\nGoqRSMs31zGQye2kTPhp7kdKoclTiaxCk6QQ/yBdLiV8E02eSuwVmiRduejEhgtmcdoRnRRx7SEp\nsEvkZD9MQdv9G3Uli1b21Facckbl0uSpREr2w1Qo+2KjrmTRyp7aKtRDijoFdomUfB+m3spdybJ8\nzRZmLX6aiYseY9bip1m+pl8ao9jQyp7ainMPSUMxEimFPjTl3t0ozl3pfJKysicu4pYzqjcFdomU\noA/TQLb+J3GyUTuiayfOOaM0FCOREuZwQ5y70lJ/cU6JrRa7REqYww1x7kpLNMS1h6TALpET1ocp\nzl1pkUoosEtiabJRGpUCuyRaXLvSpYjrrsi4itPPW4G9THH65UpyJW0pZ9TF7eetVTFl6L0r0tn/\ny43zpheJp6ClnJf/fG3sN2JFUdx2oVYU2M3sbDN73cy6zaxoxrG4i9svV5Kr0JJNNTjCF7els5W2\n2F8D5gK/DaEskRe3X64kV7Elm2pwhCtueXoqCuzuvt7dG+avJ26/XEmufBu5cqnBEZ645enRGHsZ\n4vbLleTqvSsyiBoc4YnbLtSiq2LM7Cng43leusrdHyr1Qma2AFgAMH78+JILGCVaFy1Rkl3KGXRn\nJTU4whWnpbOh3BrPzH4DXOHuJd3vTrfGEwlX0DJcLc9NFt0aT6SB5GtNxm3ttYSnosBuZmcCtwKj\ngMfMbK27nxpKyQTo2xJrG9KCO+xId6n1JUUlMW2xlKaiwO7uDwIPhlQWyZHb4tre2bXvNbW+pBgt\nz21cWhUTYcVuE6e1ylKIluc2LgX2CCulZaXWlwTR8tzGpcAeYaW0rNT6kiBxW3st4dGqmAjLd6OI\n3tT6kmLitPZawqPAHmG5G6K0KkZESqHAHnFqcYlIuRTYRSSQdq7GkwK7iOSlnavxpcAukjBhtbK1\nczW+FNhFEqTSVnbvSiEoPaD2TkR/iErr2EUSpJLbN+be0zdIo++diMO9j9Vil6rr6upi8+bN7N69\nu95FSZTW1lbGjh1LS0vLvucqyQ9TLIUFaO8ExGOISoFdqm7z5s0ceOCBTJgwATOrd3ESwd157733\n2Lx5MxMnTtz3/Oi2FFvyBPFSWtmFgr/1nCNqQw71EIfkahqKkarbvXs3I0eOVFAPkZkxcuTIfr2g\nSvLDBAX/MW0p3lr8eVYuOrHhgzrEI7maArvUhIJ6+PL9TCvJD6OkYaWJw89JQzHSkK677jqGDRvG\nFVdckff15cuXc+ihh/KpT32qxiWr3EB3K+uevqWJw89JgV0iJwpLyZYvX86cOXNiGdgroRQWpYn6\nz0lDMRIp1VxKduONN3LooYdy3HHHsWFDZvnfnXfeyVFHHcX06dP50pe+RGdnJ8899xwPP/wwCxcu\nZMaMGWzcuDHvcSJRpcAeQcvXbGHW4qeZuOgxZi1+OlLrY6utknXYhaxevZp77rmHtWvX8vjjj/PS\nSy8BMHfuXF566SVeeeUVJk+ezF133cWxxx7L6aefzpIlS1i7di2f+MQn8h4nElWV3sx6CfAF4CNg\nI/Cf3b0jjII1qkbPz1GtpWTPPPMMZ555JkOGDAHg9NNPB+C1117j6quvpqOjg127dnHqqfnvxV7q\ncVEWhSEuqY1KW+xPAlPcfRrwe+BblRepsVWrxRoXtV5KNn/+fG677TZeffVVrr322sBNVKUeF1Vx\n2C0ZV1HsYVcU2N39V+6+p+fhC8DYyovU2OKw+aGaqrWU7DOf+QzLly8nnU6zc+dOHnnkEQB27tzJ\nIYccQldXF8uWLdt3/IEHHsjOnTv3PQ46Li4avcFQLVGtMMMcY/8n4N+CXjSzBWa2ysxWbdu2LcTL\nJkscNj9UU7Xu03nEEUdwzjnnMH36dE477TSOOuooAG644QaOOeYYZs2axWGHHbbv+Hnz5rFkyRJm\nzpzJxo0bA4+Li0ZvMFRLVCtMcy+U7gfM7Cng43leusrdH+o55iqgHZjrxU4ItLe3+6pVqwZQ3OTL\nHWOHTIs1zjchXr9+PZMnT653MRKp1J/trMVP5001MKYtxcpFJ1ajaA1h4qLH8iZMM+CtxZ8P/Xpm\nttrd24sdV3Ty1N0/W+RC84E5wEmlBHUpLA6bHyR+8t0YPWq7JeOoktw81VTpqpjZwDeBf+/uWtgb\nkqhvfpD4UYOhOqJaYVa68/Q2YDDwZE/eihfc/cKKSyUioVODIXxRrTArCuzu/vdhFUREJI6iWGEq\nV0ydaLOIiFSLAnsdNPruUhGpLuWKCUk5u8+iuvZVSjds2DAAtm7dyllnnVXw2JtvvrlP0rDPfe5z\ndHQo84ZUjwJ7CMrdfabNIkWsuxdumgLXtWX+X3dvTS67d2/h+33mM3r0aH7xi18UPCY3sD/++OO0\ntbWVfS2RUimwh6DcFnij7y4taN298MilsOMdwDP/P3JpxcF906ZNHHbYYZx33nlMnjyZs846i87O\nTiZMmMCVV17JEUccwX333cfGjRuZPXs2Rx55JMcffzxvvvkmAG+99Raf/vSnmTp1KldffXWf806Z\nMgXIVAxXXHEFU6ZMYdq0adx6663ccsstbN26lRNOOIETTjgBgAkTJvDXv/4VgO9973tMmTKFKVOm\ncPPNN+875+TJk/na177G4YcfzimnnEI6rUo/DqKSN0aBPQTltsDjcGutuvn19dCV83PrSmeer9CG\nDRu4+OKLWb9+PQcddBB33HEHACNHjuTll19m3rx5LFiwgFtvvZXVq1fz3e9+l4svvhiAyy67jIsu\nuohXX32VQw45JO/5ly5dyqZNm1i7di3r1q3jvPPO49JLL2X06NGsWLGCFStW9Dl+9erV/OQnP+HF\nF1/khRde4M4772TNmjUA/OEPf+DrX/86r7/+Om1tbdx///0Vv3+prijljVFgD0G5LfBq5UNJhB2b\ny3u+DOPGjWPWrFkAfOUrX+HZZ58F4JxzzgFg165dPPfcc5x99tnMmDGDCy64gHfffReAlStXcu65\n5wLw1a9+Ne/5n3rqKS644AIGDcqsSTj44IMLlufZZ5/lzDPPZOjQoQwbNoy5c+fyzDPPADBx4kRm\nzJgBwJFHHsmmTZsqeOdSC1GaO9OqmBAMZPdZFNe+RsLwsT3DMHmer1DuzZ+zj4cOHQpAd3c3bW1t\nrF27tqTvr6bBgwfv+7q5uVlDMTEQpbkztdhDoBZ4iE66BlpyejotqczzFfrzn//M888/D8Ddd9/N\ncccd1+f1gw46iIkTJ3LfffcB4O688sorAMyaNYt77rkHIDBt78knn8wPf/hD9uzJZLJ+//33gf4p\ngLOOP/54li9fTmdnJx988AEPPvggxx9/fMXvU+ojSnNnCuwhOWPmGFYuOpG3Fn+ehadOYskTG/pN\noERlYiXSpn0ZvnALDB8HWOb/L9ySeb5CkyZN4vbbb2fy5Mls376diy66qN8xy5Yt46677mL69Okc\nfvjhPPTQQwB8//vf5/bbb2fq1Kls2ZL/93b++eczfvx4pk2bxvTp07n77rsBWLBgAbNnz943eZp1\nxBFHMH/+fI4++miOOeYYzj//fGbOnFnx+5T6iNLcWdG0vdWQ5LS9QWl3v3TkGO5fvSVR6XhLFYW0\nvZs2bWLOnDm89tprdS1H2KLws5X9qr2jPLS0vVKeoAmUn734DntzKtHsxErSA7tIo4jK3JmGYkIW\nNFGSG9SLHS/hmjBhQuJa6yJBFNhDVu5EiTYliUjYFNhDlm8CJYg2JYlINWiMPWS9E+/nu2VW1hil\n6hWRKlFgr4LsBEqhG93qBsIiUi0aiqmiKG1YaHS33HILkydPZsSIESxevBiA5cuX88Ybb9S5ZCLh\nqyiwm9kNZrbOzNaa2a/MbHRYBUuCKG1YaHR33HEHTz75JNu3b2fRokWAArskV6Ut9iXuPs3dZwCP\nApXv+04QpRqIhgsvvJA//elPnHbaadx0001ccsklPPfcczz88MMsXLiQGTNmsHHjxnoXUyQ0ld7M\n+m+9Hg6FvEPKDS0qGxai4ju/+w5vvv9mqOc87ODDuPLoKwNf/8EPfsAvf/lLVqxYwaOPPgrAscce\ny+mnn86cOXOK3gFJJG4qnjw1sxuB/wjsAE4ocNwCYAHA+PHjK72siIgEKBrYzewp4ON5XrrK3R9y\n96uAq8zsW8AlwLX5zuPuS4GlkMkVM/AiS5wValmLJEm188YUUjSwu/tnSzzXMuBxAgK7SNQEpdMV\nqVRuMsAtHWkW/uIVrnv4dXaku6oe6CtdFfPJXg+/CIQ7eCpSRfPmzWPJkiXMnDlTk6cSqnzJALv2\nOh3prprcNq/SMfbFZjYJ6AbeBi6svEgi4cveWm7+/PnMnz8fyNw8Q8sdpRpKSe5Xzeyula6K+VJY\nBRERSYrRbamCKUWyqpXdVTtPRURCVmoywGrtQldgFxEJWe7mxBFDWmhp6nsz9GruQlcSMKkJd8fM\nih8oJavHbS2ldLmbE2u5/FGBXaqutbWV9957j5EjRyq4h8Tdee+992htba13UaREtdyFrsAuVTd2\n7Fg2b97Mtm3b6l2URGltbWXs2LH1LoZEkAK7VF1LSwsTJ06sdzFEGoYmT0VEEkaBXUQkYRTYRUQS\nxuqxZMrwP0WcAAADcElEQVTMtpFJQTAQHwP+GmJx6knvJXqS8j5A7yWqKnkv/87dRxU7qC6BvRJm\ntsrd2+tdjjDovURPUt4H6L1EVS3ei4ZiREQSRoFdRCRh4hjYl9a7ACHSe4mepLwP0HuJqqq/l9iN\nsYuISGFxbLGLiEgBsQzsZnaDma0zs7Vm9iszG13vMg2UmS0xszd73s+DZtZW7zINhJmdbWavm1m3\nmcVy9YKZzTazDWb2RzNbVO/yDJSZ/djM/mJmr9W7LJUws3FmtsLM3uj527qs3mUaKDNrNbPfmdkr\nPe/lX6t6vTgOxZjZQe7+t56vLwU+5e6xvC2fmZ0CPO3ue8zsOwDufmWdi1U2M5tM5haJPwSucPdV\ndS5SWcysGfg9cDKwGXgJONfdY3fvPDP7DLAL+Km7T6l3eQbKzA4BDnH3l83sQGA1cEZMfycGDHX3\nXWbWAjwLXObuL1TjerFssWeDeo+hQPxqpx7u/it339Pz8AUglun63H29u2+odzkqcDTwR3f/k7t/\nBNxD5gbtsePuvwXer3c5KuXu77r7yz1f7wTWA7XJexsyz9jV87Cl51/V4lYsAzuAmd1oZu8A5wHX\n1Ls8Ifkn4N/qXYgGNQZ4p9fjzcQ0iCSRmU0AZgIv1rckA2dmzWa2FvgL8KS7V+29RDawm9lTZvZa\nnn9fBHD3q9x9HLAMuKS+pS2s2HvpOeYqYA+Z9xNJpbwPkbCZ2TDgfuDynN56rLj7XnefQaZXfrSZ\nVW2YLLL52N39syUeugx4HLi2isWpSLH3YmbzgTnASR7hSY8yfidxtAUY1+vx2J7npI56xqPvB5a5\n+wP1Lk8Y3L3DzFYAs4GqTHBHtsVeiJl9stfDLwJv1qsslTKz2cA3gdPdvbPe5WlgLwGfNLOJZnYA\nMA94uM5lamg9E453Aevd/Xv1Lk8lzGxUdsWbmaXITNJXLW7FdVXM/cAkMqsw3gYudPdYtq7M7I/A\nYOC9nqdeiOMKHzM7E7gVGAV0AGvd/dT6lqo8ZvY54GagGfixu99Y5yINiJn9DPgPZLII/j/gWne/\nq66FGgAzOw54BniVzGcd4F/c/fH6lWpgzGwa8L/I/G01Afe6+/VVu14cA7uIiASL5VCMiIgEU2AX\nEUkYBXYRkYRRYBcRSRgFdhGRhFFgFxFJGAV2EZGEUWAXEUmY/w9uoIcgymBqmgAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f398118f240>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(X_train, y_train, 'o', label=\"data\")\n",
    "plt.plot(X_train, y_pred_train, 'o', label=\"prediction\")\n",
    "plt.plot([X.min(), X.max()], [min_pt, max_pt], label='fit')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see in the plot above, the line is able to capture the general slope of the data, but not many details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's try the test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_pred_test = regressor.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHoJJREFUeJzt3Xt0HOWZ5/HvI6lttbptixjPxDdW3kwwDvINBAwYsgsE\nbFjHAQLBWZI97CzxAskxM2fxAAPHMGQzOMdZTMzlJGZMZnYXhoEAAjJsCBfPhEsAy9jYBuMhJk4i\nOZMYGyVuqW231O/+0Rfr0pJa3dXqrurf5xwf9aVU9ZZCnrfe533qLXPOISIiwVFT7gaIiIi3FNhF\nRAJGgV1EJGAU2EVEAkaBXUQkYBTYRUQCRoFdRCRgFNhFRAJGgV1EJGDqynHQ448/3jU1NZXj0CIi\nvrVly5aPnHNTRtquLIG9qamJtra2chxaRMS3zOyX+WynVIyISMAosIuIBIwCu4hIwCiwi4gEjAK7\niEjAlKUqphCtWztY+/xu9nXGmdYYZtXi2VyycHq5myUiUnF8Edhbt3Zwy5M7iCd6AejojHPLkzsA\nFNxFRAbwRSpm7fO7s0E9I57oZe3zu8vUIhGRyuWLwL6vMz6qz0VEqpkvAvu0xvCoPhcRqWa+COyr\nFs8mHKrt91k4VMuqxbPL1CIRkcrli8nTzASpqmJEREbmi8AOqeCuQC4iMjJfpGJERCR/CuwiIgGj\nwC4iEjAK7CIiAaPALiISMArsIiIBo8AuIhIwCuwiIgGjwC4iEjAK7CIiAeObJQVEJHj0ZLTSUGAX\nkbLQk9FKR6kYESkLPRmtdHTFLiJlke+T0ZSuGb2ir9jNrN7M3jKzd8zsXTP7ay8aJiLBls+T0TLp\nmo7OOI5j6ZrWrR1j1Ep/8iIVcwQ4zzk3H1gALDGzP/VgvyISYPk8GU3pmsIUnYpxzjkgln4bSv9z\nxe5XRIItnyej6UH2hfEkx25mtcAW4E+A+51zb+bYZgWwAuCEE07w4rAi4nMjPRltWmOYjhxBXA+y\nH54nVTHOuV7n3AJgBnC6mTXn2GaDc67FOdcyZcoULw4rIgGnB9kXxtNyR+dcJ7AJWOLlfkWkOl2y\ncDp3XTaX6Y1hDJjeGOauy+aqKmYERadizGwKkHDOdZpZGLgA+HbRLRMRQQ+yL4QXOfapwN+n8+w1\nwGPOuR95sF8RESmAF1Ux24GFHrRFREQ8oDtPRSTQqvHOVQV2EQmsal1oTIFdcsr3Kqcar4bEP4a7\nczXI/50qsMsg+V7lVOvVkPhHtd65qmV7ZZB81+fQOh5S6fJZaCyIFNhlkHyvcqr1akj8o1rvXFVg\nl0Hyvcqp1qsh8Y9qvXNVOXYZZNXi2f1y55D7Kiff7UTKqRrvXFVgl0HyWU51NNuJyNiy1HLqY6ul\npcW1tbWN+XFFRPzMzLY451pG2k45dhGRgFFgFxEJGAV2EZGAUWAXEQkYBXYRkYBRYBcRCZjA1rFr\n1UERqVaBDOxaddBb6iRF/CWQqRitOuidTCfZ0RnHcayTbN3aUe6micgQAhnYteqgd9RJivhPIAO7\nVh30jjpJEf8JZGCv1jWYS0GdpIj/BDKwV+sazKWgTlLEfwJZFQPVuQZzKWhp3vypekgqRWADu3hH\nneTIVGIrlaToVIyZzTSzTWb2npm9a2Y3eNEwET9R9ZBUEi+u2HuA/+Gce9vMJgBbzOwF59x7Huxb\nxBdUPSSVpOgrdufcb5xzb6dfHwJ2ARp7SlVR9ZBUEk+rYsysCVgIvJnjuxVm1mZmbfv37/fysCJl\np+ohqSSeBXYziwJPAH/unPvDwO+dcxuccy3OuZYpU6Z4dViRiqASW6kknlTFmFmIVFB/2Dn3pBf7\nFPEbVQ9JpSg6sJuZARuBXc65u4tvkkhlU726VDovUjGLgK8C55nZtvS/iz3Yr0jF0WqX4gdFX7E7\n514FzIO2iFS84erVddUulSKQa8WIlIrq1cUPFNhFRkH16uIHCuwio6B6dfEDLQImMgpa7VL8QIFd\nZJRUry6VTqkYEZGAUWAXEQkYBXYRkYBRYBcRCRgFdhGRgFFgFxEJGAV2EZGAUWAXEQkY3aBUJbSG\nuEj1UGCvApk1xDPLzWbWEAcU3EUCSKmYKjDcGuIiEjwK7FVAa4iLVBcF9iqgNcRFqosCexXQGuIi\n1UWTp1VAa4iLVBcF9iqhNcRFqodSMSIiAaMrdhHxjG6EqwwK7CLiCd0IVzmUihERT+hGuMrhSWA3\ns4fM7HdmttOL/YmI/+hGuMrh1RX73wFLPNqXiPiQboSrHJ4EdufcT4GDXuxLRPxJN8JVjjHLsZvZ\nCjNrM7O2/fv3j9VhRWSMXLJwOnddNpfpjWEMaAyHqA/V8Bf/uI1Fa16mdWtHuZtYNcw5582OzJqA\nHznnmkfatqWlxbW1tXlyXBGpPAMrZCB19X7XZXMrpkLGj6WZZrbFOdcy0naqihERz1V6hUym4+no\njOM4VpoZlFGFAruIeK7SK2QqveMpllfljv8A/AyYbWbtZvbfvNiviPhTpVfIVHrHUyyvqmK+7Jyb\n6pwLOedmOOc2erFfEfGnSq+QqfSOp1hKxYiI5wZWyExvDFfUxGmldzzF0loxIlISlbxUdNCfUaDA\nLiJVqZI7nmIpFSMiEjAK7CIiAeOfwL79MVjXDHc0pn5uf8zb7UVEAsIfOfbtj8GzK0kk4tQB9vtf\nw7MrU9/N+9KQ25NI16SOtH0+x3/pTvh9O0yaAeevLmw/IiJjwB+B/aU7IRHnO584jkcnRmlwjkgy\nSXTznUTanyZSFyE6LkokFEn9e/v/Em2oI5KMprZLJom4JNF/vpNI05lExkWIhqKMqx038rGH6yQy\nbVPAF5EK4o/A/vt2AM6Ox5mQTNJVY3TV1BCrqaErFCWWiLE/vp9YIkbX0S5i9eDCx+Xe11MXZ1/W\n1dQRDaU6hMzP7OtxESJ1ESJtf0c0XEdkfCTVSThHJNlL5IVbiCbiRI52E8FRO9ajAo0iRGQInq3u\nOBqjXt1xXXPqSnmgSTPhLwY/tMmtayb+h3a6aiwV/K0m9Tr6R3Rd9Dd0JbroSnQROxojlojRnehO\ndQqJrkHv4z353WIcTiaJJB1Rq6Hhj07u12E0hBqIhqJEx0VpqGvIji6yncmHPyX60v8kcrSbsHMY\nQCgMn1+fX6pppO0zv6OOQMTX8l3d0R9X7Oevzh3Izl+dc3M7fzUNz66kIRFnSm/y2PafXQ2fWjaq\nQ/esa6b7UDtdlh4hZEYLlvqZGjmkX6c/i9VPpivRxb7Yvn4dRk+yZ+gDTZsMTKbGOSJJR8QliaRT\nTYNGE2//H6LhOhrqo0TTqaaGTKpp1pnZ7bOppkLmHPp2BJnRT/zjwZ2COgyRiuOPwJ4JFPkGkNFu\nP4y681cz8dmVTEzEoTe9GlwoDHVhiOV4aNSkmfC5B3Lu62jv0Wy6qKsnNWLoSnQRe/TL6Q7DiFlN\nusNIdxJ1EWKJGL/t+m22k+gaLtX05LFUU6gmlBoxdB8kOmUSETeBaDI1PxFJJom+ficRd3BQKiry\nqzeJ/vR/ET0ap6EGIvGDZG++HjjH4NUktToIEc/4IxVTbrmCDow+HTKUUaaakuuaOfyHdmLp0UK3\npUcN6VRTtsPIdARbHuozwsiMOFKvD9fkV/EazkxCp0cT0ZpxNDiIHulOvc98l0wSrT+OyOfXD+4w\nQhHCdWHMrP/O800tDRX8B37+6Qvhg5+ok5DAyTcVo8BeDK+uMkebMx/t9sN0HD03bMvOOWTnHn6w\nJN1hWDb91G+uoqaG7j7pqEwqqmdgwM6hxmr6p5VCEaId22hIxNOdQ3quIpkkUt9I9KK16VHEW0Rf\nXUfD0TjR9PehUBjm/2d455H+f4uBCu1wh6LRhZSJArvflLIqxquOoK9JM1M/09s54KiRGglMmk7X\nVx4/NmLIdBiJGLGjMbp7uvuNKrp/8S/HRh/puQqXRycxLjMXkUweSzFlSmH7ji6Sjsj4RiL/6e5s\nxVPfkURDXQO1NbUjHm/EvyUo4EtJKbBLf8V2BH31DWRepKMGdCRJIG5GbNIMuv7rM6kS1o2f61Pm\natnRQ7bs1fq8zowqbBSpprpw7tLXcf1TSdFX7iHSfeDYqCIzwhg/iUiim/qjcbJdktcjBal6waqK\nkeLN+1L+AWbg5PNwVTF9tyv0KnVA1VMNEKmrJ3Leapj071PbjJ+SexRhteB6B3+eloDUKGDSdGL/\n5clsmWtXT1eqw8gxqsi8Phg72G+iu9f1QsQgcvwQR4tQ6xwNSUc0PZKIvHUn0d++nO0kMuWufctg\nc95HEYoQqg2N7u84GvlWPYkv6YpdKsNII4qhUiBjlGN3znGk9wix+1rojv2mT4nrsTmGXJ911dTQ\nNfO0ftVQXYmuvI45rmbc4BFDjg5g4P0R/b5LdyA11mfkku+IbODfK59RnzqMklIqRoKnEqpihupg\n6sIQH6L8dUBlU9Il6U509597GHij3IBRxcCb6DK/e6T3SF7NbqhrODZKOPgLIokj/VNJ7tjrSDJJ\nNDyZyBcfOjYf8cFLRJ7/q+FTTYV0GKPtLKq8g1BgFymVUpe/jkKiN5EK8n3ui8iVVurXYex6OjvC\n6O4zuujNY8K6Njs57VI3xVmISNPZqfWadj9P5MihPhPZfSa1Mx1G9JNE/vurRMZFCO18auS/WSF3\nWQ8UoI5BgV1krPklgOSoenLAYbNjk9ATp9J1xUPHOoXW6+iqod89ENnJ6hktqVHFwZ9nS2HzMd71\nuVkueayaqaEuTHTulan1mt56iGj84+xkdWb+Itrwx0SueSlV1TQw1dRXwKqYFNhFJLdCUib53ESX\n3qaXdFVTrnsgIsfTde7NqZHEP/9Nv7mIfjfRTfhjYkdjHE0ezeuUIqH0on0DSlkjH7xI9HBscFls\nKEK05wiR9H0RDckk0drxjF/6XWz+laP7e44hVcWIr7Ru7Qjsg4UrzmirniC/9ZrS29Qm4kSdI9rb\nC30LlkJhOGc1fCa975fvhY+H6Cyu+RcAEuua6TrUMXhiOjqF2AW3Z+cecqWhPop/RFdtL7FohK4a\nIzko1RQGGvt9Urf1mzS8t77f/Q7ZRfxCw0xkD5i4bgg1EKopYVXTCBTYpexat3Zwy5M7iCdSUaCj\nM84tT+4AUHAvldGUv2a2h+FTF6PtMPLoLELnr6bx2ZU0JuJke4lMB3HiFSO3Oz2KyKaaBqzHlB1V\nZD+rIXbGl/vdRNd5uJOOQx3ZDiPfFV/H147PWcl07fxrOfn4k/PaR6EU2KXs1j6/OxvUM+KJXtY+\nv1uBvZLk0xkUc79EPp3FaHPh6c7DEnHCzhHudRwfGg+1w1Qx/emtw+6yN9lLd0/3sPdE5Kpgih2N\n8W/d/0Yimciv7UXwJLCb2RLgu0At8LfOuTVe7Feqw77O3FdAQ30uAeJ1Z5HrdyH/KqYhlgLvq7am\nlgnjJjBh3ASIFNasUis6sJtZLXA/cAHQDmw2s2ecc+8Vu2+pDtMaw3TkCOLTGsNlaI0EznAdg8+q\nYvLlxRX76cDPnXMfApjZo8AXAAV2ycuqxbP75dgBwqFaVi2eXcZWSeAVMxKocPkVnA5vOtB3ars9\n/Vk/ZrbCzNrMrG3//v0eHFaC4pKF07nrsrlMbwxjwPTGMHddNlf5dZECjdnkqXNuA7ABUnXsY3Vc\n8YdLFk5XIBfxiBdX7B3AzD7vZ6Q/ExGRMvAisG8GPm1ms8xsHLAceMaD/YqISAGKTsU453rM7BvA\n86TKHR9yzr1bdMtERKQgnuTYnXPPAc95sS8RESmOF6kYERGpIArsIiIBo8AuIhIwWgRMRPKipZX9\nQ4FdREakpZX9RYFdAkdXlt7T0sr+osAugaIry9LQ0sr+oslTCZThriylcEMtoayllSuTArsEiq4s\nS2PV4tmEQ7X9PtPSypVLgV0CRVeWpaGllf1FOXaf0gRhbnpoR+loaWX/UGD3IU0QDi1z/ur0pJop\nsPuQSs+GpytLqXbKsfuQJghFZDgK7D6kCUIRGY4Cuw+p9ExEhqMcuw9pglBEhqPA7lOaIBSRoSgV\nIyISMArsIiIBo8AuIhIwCuwiIgGjyVPxPa2bI9KfArv4mtbNERlMqRjxNT1YQ2Swoq7YzewK4A5g\nDnC6c67Ni0aJ5Evr5pRXIpGgvb2dw4cPl7spgVJfX8+MGTMIhUIF/X6xqZidwGXA94vcj8iIcuXS\npzWG6cgRxLVuzthob29nwoQJNDU1YWblbk4gOOc4cOAA7e3tzJo1q6B9FJWKcc7tcs5pzCsll8ml\nd3TGcRzLpZ970hStm1NGhw8fZvLkyQrqHjIzJk+eXNQoaMxy7Ga2wszazKxt//79Y3VYCYihcumb\n3t+vR7aVmYK694r9m46YijGzF4FP5vjqVufc0/keyDm3AdgA0NLS4vJuoQjD59K1bs7oBblE9I47\n7iAajXLjjTfm/L61tZUTTzyRz3zmM2PcsrEzYmB3zn1uLBoiMhzl0r1TzhLRSuhQWltbWbp0aaAD\nu8odxRe0Br13ylUiOtQ8SevWjqL3/a1vfYsTTzyRs88+m927U+fx4IMPctpppzF//ny++MUv0t3d\nzeuvv84zzzzDqlWrWLBgAXv27Mm5nd8VFdjN7FIzawfOBP7JzJ73plki/V2ycLpy6R4pV4loqTqU\nLVu28Oijj7Jt2zaee+45Nm/eDMBll13G5s2beeedd5gzZw4bN27krLPOYtmyZaxdu5Zt27bxqU99\nKud2fldUuaNz7ingKY/aIjIs5dK9MVRaywGL1rxcsvRIqTqUV155hUsvvZSGhgYAli1bBsDOnTu5\n7bbb6OzsJBaLsXjx4py/n+92fqJUjEiVyZXWyvAyPTLQWD+r9+qrr+a+++5jx44d3H777UOWD+a7\nnZ8osItUmb5prVxKlW8v1TzJZz/7WVpbW4nH4xw6dIhnn30WgEOHDjF16lQSiQQPP/xwdvsJEyZw\n6NCh7PuhtvMzBfY8tG7tYNGal5l18z+xaM3LJbmaERlLlyyczms3n8dQ1dKlyLeXap7klFNO4cor\nr2T+/PlcdNFFnHbaaQB885vf5IwzzmDRokWcdNJJ2e2XL1/O2rVrWbhwIXv27BlyOz8z58a+pLyl\npcW1tfljWZmBpWGQusrQxJ0EwaI1L+fMt09vDPPazeeN+Pu7du1izpw5pWha1cv1tzWzLc65lpF+\nV1fsI9DqgRJkKiMNJq3HPgKtHihBlhl1lvumIfGWAvsIdMejBJ3KSINHqZgRDDdU1aSqiFQiXbGP\nYKihKqBHsolIRVJgz0OuoeqiNS8POamqwC4i5aRUTIE0qSoSTNFoFIB9+/Zx+eWXD7vtPffc02/R\nsIsvvpjOzs6Sti8fCuwFGuvbo0UCYftjsK4Z7mhM/dz+2Jgctre3d+SNBpg2bRo//OEPh91mYGB/\n7rnnaGxsHPWxvKbAXiDV/4qM0vbH4NmV8PtfAy7189mVRQf3vXv3ctJJJ3HVVVcxZ84cLr/8crq7\nu2lqauKmm27ilFNO4fHHH2fPnj0sWbKEU089lXPOOYf3338fgF/84heceeaZzJ07l9tuu63ffpub\nm4FUx3DjjTfS3NzMvHnzuPfee1m/fj379u3j3HPP5dxzzwWgqamJjz76CIC7776b5uZmmpubueee\ne7L7nDNnDl/72tc4+eSTufDCC4nHvR/lK7AXSMvIiozSS3dCYkAQS8RTnxdp9+7dXH/99ezatYuJ\nEyfywAMPADB58mTefvttli9fzooVK7j33nvZsmUL3/nOd7j++usBuOGGG7juuuvYsWMHU6dOzbn/\nDRs2sHfvXrZt28b27du56qqrWLlyJdOmTWPTpk1s2rSp3/ZbtmzhBz/4AW+++SZvvPEGDz74IFu3\nbgXggw8+4Otf/zrvvvsujY2NPPHEE0Wf/0CaPC2C6n9FRuH37aP7fBRmzpzJokWLAPjKV77C+vXr\nAbjyyisBiMVivP7661xxxRXZ3zly5AgAr732Wja4fvWrX+Wmm24atP8XX3yRa6+9lrq6VMj8xCc+\nMWx7Xn31VS699FIikQiQWhv+lVdeYdmyZcyaNYsFCxYAcOqpp7J3795CT3tICuwiMjYmzUinYXJ8\nXqSBD3/OvM8E1mQySWNjI9u2bcvr90tp/Pjx2de1tbVKxYiIj52/GkIDigtC4dTnRfrVr37Fz372\nMwAeeeQRzj777H7fT5w4kVmzZvH4448D4JzjnXfeAWDRokU8+uijAEMu23vBBRfw/e9/n56eHgAO\nHjwIDF4COOOcc86htbWV7u5uurq6eOqppzjnnHOKPs98KbCLyNiY9yX4/HqYNBOw1M/Pr099XqTZ\ns2dz//33M2fOHD7++GOuu+66Qds8/PDDbNy4kfnz53PyySfz9NNPA/Dd736X+++/n7lz59LRkfvu\n8WuuuYYTTjiBefPmMX/+fB555BEAVqxYwZIlS7KTpxmnnHIKV199NaeffjpnnHEG11xzDQsXLiz6\nPPOlZXtFpGCVsGzv3r17Wbp0KTt37ixrO7ymZXtFRCRLgV1EfK2pqSlwV+vFUmAXEQkYBXYRkYBR\nYBcRCRgFdhGRgCkqsJvZWjN738y2m9lTZlb+Zc1EpOqsX7+eOXPmcNxxx7FmzRoAWltbee+998rc\nsvIo9or9BaDZOTcP+FfgluKbJCIyOg888AAvvPACH3/8MTfffDOgwF4w59xPnHM96bdvAMUv+iAi\nMgrXXnstH374IRdddBHr1q3jG9/4Bq+//jrPPPMMq1atYsGCBezZs6fczRxTXi4C9mfAPw71pZmt\nAFYAnHDCCR4eVkQqwbff+jbvH3zf032e9ImTuOn0wast9vW9732PH//4x2zatIkf/ehHAJx11lks\nW7aMpUuXjvgUpCAaMbCb2YvAJ3N8datz7un0NrcCPUDuFXQA59wGYAOklhQoqLUiIjKiEQO7c+5z\nw31vZlcDS4HzXTkWnhGRijDSlbWMnaJSMWa2BPhL4D8457pH2l6G1rq1g7XP72ZfZ5xpjWFWLZ6t\nh3iIFGGoJXWrQbFVMfcBE4AXzGybmX3PgzZVndatHdzy5A46OuM4oKMzzi1P7qB1a+4lREVkZMuX\nL2ft2rUsXLiw6iZPtWxvBVi05mU6Ogc/RWV6Y5jXbj6vDC0SyU8lLNsbVFq21+f25Qjqw30uIjIc\nBfYKMK0xPKrPRUSGo8BeAVYtnk04VNvvs3CollWLZ5epRSLiZ17eoCQFylS/qCpG/Mg5h5mVuxmB\nUuzcpwJ7hbhk4XQFcvGd+vp6Dhw4wOTJkxXcPeKc48CBA9TX1xe8DwV2ESnYjBkzaG9vZ//+/eVu\nSqDU19czY0bhS28psItIwUKhELNmzSp3M2QATZ6KiASMAruISMAosIuIBExZlhQws/3ALwv89eOB\njzxsTjnpXCpPUM4DdC6Vqphz+XfOuSkjbVSWwF4MM2vLZ60EP9C5VJ6gnAfoXCrVWJyLUjEiIgGj\nwC4iEjB+DOwbyt0AD+lcKk9QzgN0LpWq5Ofiuxy7iIgMz49X7CIiMgxfBnYz+6aZbU8/ju8nZjat\n3G0qlJmtNbP30+fzlJk1lrtNhTCzK8zsXTNLmpkvqxfMbImZ7Tazn5vZzeVuT6HM7CEz+52Z7Sx3\nW4phZjPNbJOZvZf+b+uGcrepUGZWb2Zvmdk76XP565Iez4+pGDOb6Jz7Q/r1SuAzzrlry9ysgpjZ\nhcDLzrkeM/s2gHPOd497N7M5QBL4PnCjc85Xzz40s1rgX4ELgHZgM/Bl59x7ZW1YAczss0AM+N/O\nueZyt6dQZjYVmOqce9vMJgBbgEt8+r+JARHnXMzMQsCrwA3OuTdKcTxfXrFngnpaBPBf75TmnPuJ\nc64n/fYNoPAl3crIObfLObe73O0owunAz51zHzrnjgKPAl8oc5sK4pz7KXCw3O0olnPuN865t9Ov\nDwG7AF+ube1SYum3ofS/ksUtXwZ2ADP7lpn9GrgKWF3u9njkz4D/V+5GVKnpwK/7vG/Hp0EkiMys\nCVgIvFnelhTOzGrNbBvwO+AF51zJzqViA7uZvWhmO3P8+wKAc+5W59xM4GHgG+Vt7fBGOpf0NrcC\nPaTOpyLlcx4iXjOzKPAE8OcDRuu+4pzrdc4tIDUqP93MSpYmq9j12J1zn8tz04eB54DbS9icoox0\nLmZ2NbAUON9V8KTHKP438aMOYGaf9zPSn0kZpfPRTwAPO+eeLHd7vOCc6zSzTcASoCQT3BV7xT4c\nM/t0n7dfAN4vV1uKZWZLgL8Eljnnusvdniq2Gfi0mc0ys3HAcuCZMrepqqUnHDcCu5xzd5e7PcUw\nsymZijczC5OapC9Z3PJrVcwTwGxSVRi/BK51zvny6srMfg6MBw6kP3rDjxU+ZnYpcC8wBegEtjnn\nFpe3VaNjZhcD9wC1wEPOuW+VuUkFMbN/AP4jqVUEfwvc7pzbWNZGFcDMzgZeAXaQ+v86wF85554r\nX6sKY2bzgL8n9d9WDfCYc+7Okh3Pj4FdRESG5stUjIiIDE2BXUQkYBTYRUQCRoFdRCRgFNhFRAJG\ngV1EJGAU2EVEAkaBXUQkYP4/MD/+s2lsWUQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f398111ad68>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(X_test, y_test, 'o', label=\"data\")\n",
    "plt.plot(X_test, y_pred_test, 'o', label=\"prediction\")\n",
    "plt.plot([X.min(), X.max()], [min_pt, max_pt], label='fit')\n",
    "plt.legend(loc='best');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, scikit-learn provides an easy way to evaluate the prediction quantitatively using the ``score`` method. For regression tasks, this is the R<sup>2</sup> score. Another popular way would be the Mean Squared Error (MSE). As its name implies, the MSE is simply the average squared difference over the predicted and actual target values\n",
    "\n",
    "$$MSE = \\frac{1}{n} \\sum^{n}_{i=1} (\\text{predicted}_i - \\text{true}_i)^2$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.089522023813181306"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regressor.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "KNeighborsRegression\n",
    "=======================\n",
    "As for classification, we can also use a neighbor based method for regression. We can simply take the output of the nearest point, or we could average several nearest points. This method is less popular for regression than for classification, but still a good baseline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsRegressor(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "          metric_params=None, n_jobs=1, n_neighbors=1, p=2,\n",
       "          weights='uniform')"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsRegressor\n",
    "kneighbor_regression = KNeighborsRegressor(n_neighbors=1)\n",
    "kneighbor_regression.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, let us look at the behavior on training and test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuQVdWdL/Dvrx/pJt0+ImKCYtttrg9iKw8Bb4KdDDFq\n+ygEo1GMsRwLKbUmI3euRjLhpvVSVmaKlJdo0AQkjlOA1DiEjrkaSbwylU6MQoMorUgUoaUhM7Rt\nTOiGbvvxu3+cPs2hOfucfc7e5+y11v5+qqiiT+8+Z53Xb6+91m/9lqgqiIjIHSVRN4CIiMLFwE5E\n5BgGdiIixzCwExE5hoGdiMgxDOxERI5hYCcicgwDOxGRYxjYiYgcUxbFg5566qlaW1sbxUMTEVlr\n69atH6rquGzHRRLYa2tr0draGsVDExFZS0Ta/RzHoRgiIscwsBMROYaBnYjIMZGMseeqvasHK1ve\nR/PrB9DTN4CqijLMmXI67mw4G2eNrYq6eZRFf38/Ojo60NvbG3VTnFJZWYkJEyagvLw86qaQYYwP\n7Jt2HcQ9q7ehf3AIA0OJ2vHdfQNYt3kf1m/dj8dvnYpZ550WcSspk46ODpxwwgmora2FiETdHCeo\nKrq6utDR0YG6urqom0OGMXoopr2rB/es3oYj/YMjQT1pYEhxpH8Q96zehvaunohaSH709vZi7Nix\nDOohEhGMHTuWV0GUltE99pUt76N/cAgAsKXiboyTv4z8rlNPwvS+J9A/OIQnW/ZgyZz6qJpJPjCo\nh4+vKXkJLbCLSCmAVgD7VfXaMO6z+fUDIz311KCe+vPAkGLD6/sZ2B3B+RSi4MIcirkXwM4Q7w89\nfQP+jvvE33Fktk27DqJxWQvWbd6H7r4BKI7OpzQua8GmXQdDe6wHH3wQP/zhDz1/39zcjLfffju0\nxyMqplACu4hMAHANgCfDuL+kqgp/FxRVnzJ6RIl8MG0+hYGdbBZWj30ZgO8AGArp/gAAc6acjrKS\nxDhip550zO+SP5eVCOZOOSPMh6UIpM6neEnOp+Tr4YcfxrnnnotLL70Uu3btSjzuypWYPn06Jk2a\nhK9//es4fPgwXnnlFTz33HO4//77MXnyZOzevTvtcUSmChzYReRaAAdVdWuW4xaISKuItHZ2dvq6\n7zsbzkZ5aaKJ0/ueQG3v2pF/0/ueAACUl5ZgfgPTvWyXOp/iJTmfko+tW7di3bp12L59O1544QVs\n2bIFAHD99ddjy5YteOONNzBx4kSsWrUKX/rSlzB79mwsXboU27dvx+c///m0xxGZKowe+0wAs0Vk\nL4B1AL4qIqtHH6SqK1R1mqpOGzcua3EyAMBZY6vw+K1TMaa8dKTnnlRWIhhTXorHb53KSTUHFHo+\npaWlBXPnzsWnP/1pnHjiiZg9ezYAoK2tDQ0NDbjwwguxZs0avPXWW2n/3u9xRCYIHNhV9buqOkFV\nawHcDOBlVb01cMuGzTrvNLy4sAHzZtSguqIMIkB1RRnmzajBiwsbuDjJEVHNp9x+++348Y9/jB07\ndqCpqckzL9zvcUQmMHqBUtJZY6uwZE492h66Ent+cA3aHroSS+bUs6fukNT5FC9B5lO+/OUvo7m5\nGUeOHMGhQ4fwy1/+EgBw6NAhjB8/Hv39/VizZs3I8SeccAIOHTo08rPXcUQmCjWwq+p/hJXDTvGS\nOp/iJch8ytSpU3HTTTdh0qRJuOqqqzB9+nQAwJIlS3DJJZdg5syZOP/880eOv/nmm7F06VJMmTIF\nu3fv9jyOyESimnnCqhCmTZum+W60wQUs9tm5cycmTpyY9bh0dYGARE+9vLSEdYHS8PvakhtEZKuq\nTst2nBVDMUnFXMBCxcf5FKJwWLOyJ3UBy2gDQ4qBocQClhcXNrDnbrHkfApLRBDlz5rAzoJgRET+\nWDMUk0tBMCKiOLMmsLMgGBGRP9YEdhYEIyLyx5rAzoJgZJLq6moAwIEDB3DDDTdkPHbZsmXHFA27\n+uqr8fHHHxe0fXHS3tWDxc07UN+0EXWLnkd900Ysbt4R653VrMljb+/qQeOylrRZMUljykuZFWMg\nW3KtBwcHUVpa6uvY6upqdHd3+zq2trYWra2tOPXUU4M0Ly1bXttCidvaB+fy2FkQLEaWngM8eNLR\nf0vPCXyXe/fuxfnnn49vfvObmDhxIm644QYcPnwYtbW1eOCBBzB16lQ8++yz2L17NxobG3HxxRej\noaEB77zzDgBgz549+OIXv4gLL7wQixcvPuZ+6+sTWViDg4O47777UF9fj4suugiPPfYYHn30URw4\ncACzZs3CrFmzACQC/YcffggAeOSRR1BfX4/6+nosW7Zs5D4nTpyIO++8ExdccAGuuOIKHDlyJPBr\nYLvRPfMvfP9FzH+61Zga/iaxakA6uYDlyZY92PD6fvR8MoCqT5Vh7pQzML+hjkHdFT0HM/+cp127\ndmHVqlWYOXMm7rjjDjz++OMAgLFjx2Lbtm0AgMsuuww/+clPcM455+C1117DPffcg5dffhn33nsv\n7r77btx2221Yvnx52vtfsWIF9u7di+3bt6OsrAwfffQRTjnlFDzyyCPYtGnTcT32rVu34qmnnsJr\nr70GVcUll1yCr3zlK/jMZz6Dd999F8888wxWrlyJb3zjG1i/fj1uvTW02nrWSdczP/zJ0at3pkAf\ny6rADnABC+XvzDPPxMyZMwEAt956Kx599FEAwE033QQA6O7uxiuvvIIbb7xx5G/6+voAAL///e+x\nfv16AMC3vvUtPPDAA8fd/0svvYS77roLZWWJr9Upp5ySsT2/+93vMHfuXFRVJTok119/PVpaWjB7\n9mzU1dVh8uTJAICLL74Ye/fuzfdpWyFTqRAAnosTk7gn8rGsC+xE+RKRtD8nA+vQ0BBOPvlkbN++\n3dffF1JFRcXI/0tLS50eiknXG+/uG8DqVz/A6lc/QGmJYGj4dq+eeSZxTIG2Zow9Hc6GO6rqtMw/\n5+mDDz7AH/7wBwDA2rVrcemllx7z+xNPPBF1dXV49tlnAQCqijfeeAMAMHPmTKxbtw4APMv2Xn75\n5fjpT3+KgYFEIPnoo48AHF8COKmhoQHNzc04fPgwenp6sGHDBjQ0NITwTO2Raa/bpMEhRfI3Xj3z\nTOKYAm1tYGdBMIfd/y7w4F+O/rv/3VDu9rzzzsPy5csxceJE/PnPf8bdd9993DFr1qzBqlWrMGnS\nJFxwwQX4xS9+AQD40Y9+hOXLl+PCCy/E/v3pVzfPnz8fNTU1uOiiizBp0iSsXbsWALBgwQI0NjaO\nTJ4mTZ06FbfffjtmzJiBSy65BPPnz8eUKVNCea62GF0qZG/lLSP/tlQc//54YQr0saxJd0zF1Ee7\nmJCSt3fvXlx77bVoa2uLtB1hM+G1DaK+aSO6h1eV7628xfO45JCLn2NSuRYH/KY7WnmNwoJgRG7w\nWyok+R3v1JM8h19G3x7nFGgrh2JYEIxyVVtb61xv3QV+S4UkTe97ArW9a1HbuzbrsXGu4R84sItI\npYhsFpE3ROQtEXkojIZlwoJg9oliyM91LrymmUqFBFFdURbLnnpSGD32PgBfVdVJACYDaBSR/x7C\n/XpiQTC7VFZWoqury4lAZApVRVdXFyorK6NuSiCpe92m9sa9JkMz3Zb8WQD0DQzGOlMucOTTxLc1\nWTSjfPhfQb/Bc6acjnWb92FgSI8bc4v7bLiJJkyYgI6ODnR2dkbdFKdUVlZiwoQJUTcjkGSpkHtW\nb8MnA4MYHI4cXrnplWUlUAB9A0OexyiA/sGj+fDrNu/D+q37nasbk0koWTEiUgpgK4D/BmC5qh6/\nLC8Fs2KIKFV7Vw+ebNmDZ1v3oXdg6Ljfpxb1ApC28Fc2LsQEv1kxoaY7isjJADYA+Laqto363QIA\nCwCgpqbm4vb29kCPFbeqbkRxkQzymepBjT6mVGRkIZNXplxZiWDejBqrM+UiCezDD/x9AIdV9Yde\nxwTtsSf5+QAQkfuy5cMns2iqK8rQ9tCVRW1bmIqWxy4i4wD0q+rHIjIGwOUA/jno/frBgmBEBDBT\nbrQw0kbGA3h6eJy9BMC/qer/DeF+iYh8qaooG+mxZzwuJplyYWTFvAkgXgUuiMgozJQ7VjxOX0Tk\ntDsbzsb6rfsxMDTomQZZXlqC+Q11RW5ZNKwsKUBElIpbZx6LgZ2InJDcOnPejBpUV5RBJJEFM29G\nTezqxlhZtpeIKI78pjuyx05E5BhOnlIgmTYhjst4JpFpGNhjKKxg7LUJcRyLLhGZhEMxMRPWXrGZ\nNiEeGFIc6R/EPau3xa5cKpEJGNhjJMxg7GcT4uT2hERUXAzsMRJmMOb2hETmYmCPkTCDMYsuEZmL\ngT1GwgzG3J6QyFwM7DESZjDOtAlxHIsuEZmE3akYCbMCHosuEZmLgT1GwgzGqZsQZ9qekIuUiIqP\nQzExEnYFPBZdIjITi4DFxOjVpmWlAtVEFkxZiUAEGBhUlgQgMljR9jwl86Vb+t8/qEh22lUVA4n0\n9rQlAVgPhsgu7LE7rr2rB43LWnCkfzDnvx1TXooHZ38BDz73dsZxdA65kB/sIATnt8ceOLCLyJkA\n/hXAZwEogBWq+qNMf8PAXjyLm3eMZMJsqbj7uEyY6X1PeN5eVgKoAoMZPiJjykvx4sIGnDW2il9c\n8pTuqhFgByFXxRyKGQDwP1V1m4icAGCriPxGVd8O4b4pID+rTb1XoR69zSv4J0sQfHXiaaz0SGml\n1igabWBIMTCUqFGU7CBQcIGzYlT1T6q6bfj/hwDsBMBVKYbwu9o0m0wlCH6+rYOVHskTC8YVX6iT\npyJSC2AKgNfS/G4BgAUAUFNTE+bDUgZVFWXoDim4e+n5ZHAkfTJbz37JnPqCtoXMk0uNokJ9PuI2\nTBhaHruIVANYD2Chqv519O9VdYWqTlPVaePGjQvrYSkLP0v/vW7PBSs9kpeoC8aFtQeBTULpsYtI\nORJBfY2q/jyM+wwqbmdoL35Wm3rdXloigCoGFRlLEIwefvHCSo/x5PeqsRAF4+I6vh/4lRQRAbAK\nwE5VfSR4k4Ljlm1HZVr6XyLAkAKlcmzmSzJTIZnqeKQ/cwmCT5UBhz/Jnk7JSo/x097Vg/EnVeLd\ng90AMncQClEwbvT4flyGCcP4ps0E8C0AO0Rk+/Bt/6iqL4Rw3zmL6xk6k+TS/ydb9mDD6/vR88kA\nqj5VhrlTzsBV9Z/Dr9r+87jb5zfU4ayxVTjtxMqsaWr/b+d/hVZczBa8Iswu2cH6ZODod7HYBeNM\nGN+PgnMLlPzkbZeVCObNqHHqjSyk9q6etCeFZPD3swgqNd/dJukC+PS6z+CV97owOKTMyfbgd2Fc\nWQlQXlpasNesbtHzSL5DeytvOe73tb1rAQAiwJ4fXBP644cttiUF4nqGLqSzxlZhyZx6z9fL1UqP\nXkN6m97pTHt8XK8I0/EzBAIAZ4+rxsrbphXstYpyfD9KzlV3jHoGPq5cq/SYaePvJOZke/PTwQKA\nAx/3FvQEGNcNYdw6TSG+Z2gTZOvZ28RPj5NXhN5M6WDFdUMY53rscT1DJ7V39WBx8w7UN21E3aLn\nUd+0EYubd3DVZ4789ji9xP2K0JQ9ccPeg8AWznVb43qGBpjmGaagpRjifkUY5jaMQWXKCksmALjG\nuU+fqxN52TDNM1x+hvSiDlgmM62D5dIwoR/OBXYgnmfouC7EKBQ/PU4TApap4trBMoVzeexxVd+0\ncaSHmSlft7qiDG0PXVnUttkonw1KmMd+vGxrICg3sc1jz8bVFYOmZCG4IluPs6xE8KXPn4rNez9i\nwMogbkMgpohVYHd5cpFpnuELOqTnaieCzBeboRiXl70DLKVgimQw//fWDvSmbkE1jMM1FASHYkZx\nfXLRtCwEWwXpZacWvfLaJ5YZSlQMzi1Q8pJLDRkbxXUhRpiCbMiQmm6aGtRZdoCiEJvAHofJRdfq\ntRRLe1cP7l23DX/71Ja892312tfTxU4EmS8WQzHtXT0o9bnTj+2Ti8xCyE1y+KQ3Ze4ln6G6TFeE\nXmzuRJDZ7I5iPiS/uIMpQZ0rBgvPhowQr9W6+RT3yqcEge2dCDKX058sry8uJxcLy5a00kwT6l68\netl+003ZiaBicDqw+y32LwAqObkYCptq1uQzfOLVy85WgmB0Z0JVsX5bB1a/2m7k1QzZLZTALiI/\nA3AtgIOqaszgrt/Sq2WlYkSgcYFNaaV+h0/89LL9pJumGtSjG4CbeDVDdgsrK+ZfADSGdF+h8fvF\nHRhSBvWQ2JRWmq1meG3vWtT2rh0J1JmG6jKlm/rhJ/OGyK9Qeuyq+lsRqQ3jvsLEZfbFZ1NaqZ8K\njoD/obp0JQjGlJeib2DomMl7wPyrmTixYaI/V0WLaCKyAMACAKipqSnKY5pU7D8ubDqZ+h0+uW7y\n6fgfl5/r60s+Ot00WeoByDxBy231omHLRH+uirZASVVXqOo0VZ02bty4ojzmnQ1no7w08RSn9z0x\ncmnt9/KacmfT1oR+Vus+9bfTsezmKXn33Jjfbq5MG5bbPjQWfbepgFjsv/hsq1lT6E1ZmN9uLpsm\n+nPl/CcojrspRcnGk2khV+vmmt8OAKefVIn2rh6jXiMX5TLRH8vALiLPAPgbAKeKSAeAJlVdFcZ9\nh8H1ZfamTf5kOpleVf85vND2J3x77etGtLXQcs1vB4D3P+xG47IWa8d3bWHTRH+uYlOP3VXpJn8A\nM+t+29TWsOSzxV6SzfsD2MDG7ST91mOPTXVHF9k0+WNTW8PkJ7+dpX2jYdNEf66cH2N3mU2TPza1\nNWzphqZSL5RdG9+1hW0T/blgj91iNq3ytKmthZCc52l76Ers+cE18Ls21cbxXVu4vDkNe+wWs2ny\nx6a2FoNNC7m8mDZpnw9Xs+bM/dRQVrYEhzhtdOKXv1XRwOknV6K+aaNxgdOlFZsuZs3F41vkwfYe\nhw0lE7jRSXp+xncHhoDdB7tH9lA1JXDaVJo5rmKb7uhC6p2fVLooU+ZyTfWLW3qf92cwEdQzifK1\nSta/GRhSz4nwshLBvBk1TvWCTcB0xwxcSb0zffLHa4Pn1JQ+IFE9Meq2RsFr8/Gzx1WjdPjtNDEV\nMu4T4TaI5VCMS6l3Jk/+cKOT7NKN79Y3bRwZfjExFdLliXDbh2eTYhnYXasRYerkDzc6yV17V4+v\nCXEgusBpy6R9rlyaEI7lUIzLPQ6TZNuhaOQ4ywJAoWzadRCNy1p8Hx/V6+biik1XhmeTYvmNcrXH\nYRobsnZM4ZVpYuLr5uKKTZeGZ4GYBnYGnOJwMQAUip/AkirK183G0szZuDY8G8vAzoBTHC4GgELx\nO9EMmJFBZPKkfT5cG56NZWBnwCkekwKAyRkPuey0ZEoGkamT9vlwbXjWjlYWgEkBJ1cmB6h0TAgA\npmc8+A0s1RVlRr7HtnNteDa2K09t5cKK2WLzuwL2usnj8Q+Xn8fVnDFk+iruJL8rTxnYLWLLh880\nfoImkFgBWzk8fl3sk6Mt761tV4u5sKHTVNSSAiLSKCK7ROQ9EVkUxn0WW3tXDxY370B900bULXoe\n9U0bsbh5h1F5q36W6HPXneP5nZhUILJ8ZdPLQwBH8+zXbd6H7r4BKI4OZzUua8GmXQcja1sYvEo8\nzJtRgxcXNkQe1HMRuMcuIqUA/gjgcgAdALYAmKeqb3v9jWk9dhvO1ICdezSaoG7R80i+q+letyQT\nhjzau3qMnPex5YrCdX577GFMns4A8J6qvj/8wOsAXAfAM7CbxKYSpK6lZBWL34lJE/KVTZhoTse1\nBTyuC2Mo5gwA+1J+7hi+zQo2DW9wiX5+Mi2B98KT47FY0dEuRasVIyILRKRVRFo7OzuL9bBZ2fSB\ndbFGR6G1d/XgUG//yHs8ve8J1PauHRm28sKT47F4tWiXMD69+wGcmfLzhOHbjqGqKwCsABJj7CE8\nbihs+sByxWxuWRmpcyfpuJCvXCyuLeBxXRjvwhYA54hIHRIB/WYA3jNUhrHpAxv3FbO5LDLKNHeS\nFNeTYz5cW8DjusDRSlUHROTvAGwEUArgZ6r6VuCWFYltH1ibV8wGkeskd65FtYB4nBzzxatFu8R+\ngRLTuOyQ68pMv6mhACCCWJwcg7IlLdhlxUx3tFrchzdskWtZVb9zJyLAnh9cE25jHRXXq0UbxT6w\nA/zAmibdBGmu28XZNHfihylL+U3Ns6dj2fGpLgJ+YM3gNUHqVzJQ2zZ3konplSnJPAzsZAw/mSyA\nvzRFVyb7bFoZ7RpTrpLywcBORmjv6sH8p1tHAlimTBY/gdqVuRMu5Y+G7VdJRVt5SuQlWTXw3YPd\nI7dl2x4ulVf1Qxeq9dm0MtoVqVdJqR0CIPFaR1UBNBfssVOk/A6/jFZdUeY5yW3zJfRoNq2MdoUL\nV0kM7BSpTF8iL5nKEtt+CT2aa9k9Nsg1tdZEHIqhSGX6EqXyk8niwiX0aCz8VnwuXCXxNE+RyvYl\nGl2FMVMmiwuX0KO5kt1jExeukthjp0j5rTEPIOv2cC5ONNqwZZ5rXLhKMveUQ7HgZyERAJz72Wqs\nvG1axgDmwiV0OlwZXVwuXCUxsFOk/HyJxpSXZg3qgBuX0F64Mrp4XFgDwaEYilSYQw0uXEL70d7V\ng8XNO1DftBF1i55HfdNGLG7eYdWksOlGr4EAEp8dEaC3fxDfXvu60a957Mv2ZuNSTrTJ2rt6Ag81\nxKEEM0vnFp9Jr7nfsr0M7BmY9IaSPy6/Z35OXABw3eTx+IfLz7P25GUS0zoLfgM7h2I8uJgTHQcu\nlBHwMjqdc2/lLSP/tlTcPXLcc9v/hMZlLdi062BUTXWGn9c8mUJrEvtmkYrExZzouHB1otFPOicA\nKDDS8bB52MkEtq5CDdRjF5EbReQtERkSkayXBzZxMSea7JYtndOGnqRtbE2hDToU0wbgegC/DaEt\nRrH1DSV3+V3MxY5HePy+5qal0AYK7Kq6U1V3hdUYk9j6hpK7MqVzemHHIxhbU2gZlTy4tLUaucFr\nMdfeyls8/0YVqFv0PNN082TrKtSsPXYReUlE2tL8uy6XBxKRBSLSKiKtnZ2d+be4SO5sOBvlpYmX\nZ3rfE6jtXTvyL/kGm/iGkrtSF3OlLuXy6kkmKY6WLma2TG5srdUTSh67iPwHgPtU1VdyOvPYvY1e\nEFVWKlBNjJdWs9dFSHxG/s9v/ojm7Qfy+nvbF2lFIYwFdGEo6gIlVwM7UNw31OtEksqFhTYUjmyf\nF6803bISwbwZNUal55E/RQnsIjIXwGMAxgH4GMB2VU2/tU0KmwJ7sfhdVZjEXhcBx3c8Ur/O6cbe\nk/XtM+1CRebyG9gDTZ6q6gYAG4LcByX43SKOi6Mo1ejFWHWLnoefrhqzZdzGkgKG8LtFHHOUKROm\n6RLAwG4Mvwuijvkb9rpoFFvzrilcPG0bwu8mEcf8DXtdNIqtedcULvbYDeF3VSF7XZSJrXnXFC52\n+Qzhp6eVir0u8hLGHqncYMZu3GjDIMxjJxO4vFmJ7biDkqWOyUtOs/KUO9OTlzB62abtGETHYmAn\nipEwetntXT2Y/3Qr3j3YDYArV71EOUzFrfGIYiKMbRw37TqIxmUtI0Ed4AYz6SRfp3Wb96G7b8DY\nAmsM7ESWC7ovZ+qJwa84rqGwaR9kZsUQWS7ovpx+y1mkiuMaCpv2QWaPnchyQbdx9FvOIu5rKGza\nBzl+p10ix/hdtezVy852YkhWhEyK6xoKm/ZBZo+dyHJB68P4LRwGINYrV20qsBZ9C4gokKD1Yfzs\n7wsA5362GitvmxbLoA7YtQ8yAzuR5ZL1YbLlsXsFZD8nhjHlpbEO6oBdBda4QMkArMtBYQiyjSPL\nCPgT9evElaeWiPqDQu7z23EwZcNm00X5OjGwW4B1OajQ2HEorGJfbRelpICILBWRd0TkTRHZICIn\nB7m/uAm6YpAoE5tWStrI5PICQdMdfwOgXlUvAvBHAN8N3qT4sGnBA9mHHYfCMf2kGSiwq+qvVTWZ\njf8qgAnBmxQfNi14IPuw41A4pp80w1ygdAeAX3n9UkQWiEiriLR2dnaG+LD2smnBA9mHHYfCMf2k\nmTWwi8hLItKW5t91Kcd8D8AAgDVe96OqK1R1mqpOGzduXDittxx3lKdCYsehcEw/aWZ9R1X1a5l+\nLyK3A7gWwGUaRYqNxWxa8ED2sWmlpG2C1ucptKBZMY0AvgNgtqoeDqdJ8cEd5amQ7mw4G+Wlia/4\n9L4nUNu7duRfsiPBjkN+TL/aDpTHLiLvAagA0DV806uqele2v4trHrtXzuvV9ePxq7b/5MIQCh3z\n2AsjqjUoXKBkGH7BKCpcUVoYUXynGdgLLJcVZ1xhSuSmYp80GdgLKNcz9eLmHSOTWNz5nYjyVZSS\nAnGUz4oz03NeiSi49q4eLG7egfqmjahb9DzqmzZicfOOSFafMoE1R/lsaGt6zisRBZPuKj5ZN2b9\n1v1Fn0Njjz1H+fS+uVCEyF0m1o1hYM9RPr1v03NeiSh/JtaNYRcxR/msOOMKUyJ35XIVX6zkCPbY\nc+Sn9w0AfQODWNy8A6+89yFWtrwPr+wjrjAlspuJc2jssefIT+8bAPoHFc+89gFWv/oBSgQYShPX\nx5SX4IaLz+RCESKLmVg3hj32HGWq7zLa4HAwTxfUE4RBnchyJs6hsceeh1nnnYYXFzbgyZY9eGbz\nByPja17pj37TIonIPibOobHHnqezxlZhyZx6VJaXjtzmNXHCRUlE7jKxSisDe0B+J048/56Lkois\nl7yKnzejBtUVZRABqivKMG9GDV5c2FD0An8cignI78SJ599zURKRE5JX8SYMrbLHHlCmiRMA2Ft5\ny3G3cVESERUSu4sBeU2cpAvotb1rj/mZi5KIqBDYYw8odeIkF1yURESFwsAeguTESba89qSyEolk\nQoWI4iHoZtZLRORNEdkuIr8WkdPDaphtzhpbhZtnnOlrocK8GTXsqRNRwQQdY1+qqv8LAETk7wF8\nH0DWzaxdZeJCBSKKn0CBXVX/mvJjFYDi77NnkOR4e7Zt89hbJ3JbLnsiF0LgPU9F5GEAtwH4C4BZ\nqtrpcdyJkfcFAAAEsElEQVQCAAsAoKam5uL29vZAj2sy7gpPFF9eeyILjvZ8q/MM9KFtZi0iLwH4\nXJpffU9Vf5Fy3HcBVKpqU7YHtX0zayKidNq7etC4rAVH+gezHpt6Fe83kcJvYM86FKOqX/P1iMAa\nAC8AyBrYiYhc5GdP5KSBIcXAUGLbvBcXNoR6NR80K+aclB+vA/BOsOYQEdkr225Kxdo2L2ge+z+J\nSJuIvAngCgD3htAmIiIr5VIUsJBVXoNmxXw9rIYQEdku36KAYVd55cpTIqKQZCsK6CXsKq8sAkZE\nFJJMixTTTaYChanyysBORBSSTIsUi7kanUMxREQhOm43peHbR5cILOS2eYFXnuaDC5SIKE7CWo0e\n2srTQmBgJyLKnd/AzqEYIiLHMLATETmGgZ2IyDGRjLGLSCeAXOr2ngrgwwI1p9hcei6AW8+Hz8VM\nLj0XINjzOUtVx2U7KJLAnisRafUzYWADl54L4Nbz4XMxk0vPBSjO8+FQDBGRYxjYiYgcY0tgXxF1\nA0Lk0nMB3Ho+fC5mcum5AEV4PlaMsRMRkX+29NiJiMgnawK7iCwRkTdFZLuI/FpETo+6TfkSkaUi\n8s7w89kgIidH3aZ8iciNIvKWiAyJiJWZCyLSKCK7ROQ9EVkUdXuCEJGfichBEWmLui1BiciZIrJJ\nRN4e/oxZu0ObiFSKyGYReWP4uTxU0MezZShGRE5U1b8O///vAXxBVe+KuFl5EZErALysqgMi8s8A\noKoPRNysvIjIRABDAH4K4D5VtaoIkIiUAvgjgMsBdADYAmCeqr4dacPyJCJfBtAN4F9VtT7q9gQh\nIuMBjFfVbSJyAoCtAObY+N6IiACoUtVuESkH8DsA96rqq4V4PGt67MmgPqwKgB1npDRU9deqmtwL\n61UAE6JsTxCqulNVd0XdjgBmAHhPVd9X1U8ArENiY3YrqepvAXwUdTvCoKp/UtVtw/8/BGAngHB3\npCgSTege/rF8+F/BYpg1gR0ARORhEdkH4JsAvh91e0JyB4BfRd2IGDsDwL6UnztgafBwmYjUApgC\n4LVoW5I/ESkVke0ADgL4jaoW7LkYFdhF5CURaUvz7zoAUNXvqeqZANYA+LtoW5tZtucyfMz3AAwg\n8XyM5ee5EBWKiFQDWA9g4agrd6uo6qCqTkbiCn2GiBRsqMyorfFU9Ws+D10D4AUATQVsTiDZnouI\n3A7gWgCXqeETHTm8LzbaD+DMlJ8nDN9GBhgej14PYI2q/jzq9oRBVT8WkU0AGgEUZJLbqB57JiJy\nTsqP1wF4J6q2BCUijQC+A2C2qh6Ouj0xtwXAOSJSJyKfAnAzgOcibhNhZMJxFYCdqvpI1O0JQkTG\nJbPfRGQMEpP1BYthNmXFrAdwHhIZGO0A7lJVK3tWIvIegAoAXcM3vWpxhs9cAI8BGAfgYwDbVfXK\naFuVGxG5GsAyAKUAfqaqD0fcpLyJyDMA/gaJCoL/BaBJVVdF2qg8icilAFoA7EDiew8A/6iqL0TX\nqvyIyEUAnkbiM1YC4N9U9X8X7PFsCexEROSPNUMxRETkDwM7EZFjGNiJiBzDwE5E5BgGdiIixzCw\nExE5hoGdiMgxDOxERI75/yDKNeBSawMHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f39811e3128>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred_train = kneighbor_regression.predict(X_train)\n",
    "\n",
    "plt.plot(X_train, y_train, 'o', label=\"data\", markersize=10)\n",
    "plt.plot(X_train, y_pred_train, 's', label=\"prediction\", markersize=4)\n",
    "plt.legend(loc='best');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On the training set, we do a perfect job: each point is its own nearest neighbor!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGm9JREFUeJzt3X9wlWV6//H3BQlhE1R+typCAl9FsCuIQWRBWNuli1sG\nB3cd0I0dp2XJip3Vsau71t3quOO0HRxGx5U2UNzOfsHynV0Lurt8KbGloPxwCGzUuGyqIFD8UbOA\nFhKNCVz9IwmE/DjnJOc55zzPcz6vmczw5Dx5zn3Ck+vc576v+7rN3RERkfgYkOsGiIhIsBTYRURi\nRoFdRCRmFNhFRGJGgV1EJGYU2EVEYkaBXUQkZhTYRURiRoFdRCRmCnLxpCNHjvTS0tJcPLWISGTt\n27fvd+4+Ktl5OQnspaWl1NTU5OKpRUQiy8yOpHKehmJERGJGgV1EJGYU2EVEYkaBXUQkZhTYRURi\nRoFdRCRmcpLu2C8rroTGj84fl4yGB9/OXXtEREIqsB67mQ00s1+b2S+DuuYFOgf1no5FRAQIdijm\nPuBAgNcTEZF+CCSwm9kY4E+AfwzieiIi0n9B9difAh4CzgZ0ve5KRic+FhERIIDJUzNbAHzk7vvM\n7MsJzlsGLAMYO3Zs359IE6UiIikJosc+C1hoZoeBDcAfmtm6rie5+2p3L3f38lGjkhYnExGRfko7\nsLv7w+4+xt1LgSXAv7t7RdotExGRftECJRGRmAl0gZK7/wfwH0FeU0RE+kY9dhGRmFFgFxGJmUjU\nimlsbqVqx0HW7T7CyaYWhhUXUjFzHJVzJlBSFImXICKSNaGPio3NrSxatZMjx5tobm1b/3SiqYWq\n7YfYUvchG5fPUnAXEekk9EMxVTsOXhDUOzS3nuXI8SaqdhzMUctERMIp9IF93e4j3YJ6h+bWs6zb\nczTLLRIRCbfQB/aTTS1JHv88Sy0REYmG0Af2YcWFSR4flKWWiIhEQ+gDe8XMcRQV9NzMooIBVNzY\nj4JiIiIxFvrAXjlnAuNGFHcL7kUFAxg3opjKORNy1DIRkXAKfWAvKSpg4/JZVM4dz/CSQZjB8JJB\nVM4dr1RHEZEemLtn/UnLy8u9pqYm688rIhJlZrbP3cuTnRf6HruIiPSNAruISMwosIuIxIxmHkVE\nAFZcCY0fnT8uGR3ZvZbVYxcRgQuDek/HEaLALiISMxqKEZHYyte9HOL7yjrEaNxMRFLX570cSkZ3\njxURFf/AHqNxMxFJXSp7OTwwb+L5B2LU4Yt/YBeRvJTKXg4dgT1uQzZpt9jMBgM7gKL26/3c3R9N\n97oiIulIdS+HOG6/GURWTDPwh+4+BZgKzDezGwO4bjC6jpNFeNxMRFKX6l4Ocdx+M+23IW+rIna6\n/bCw/Sv7lcV6E6NxMxFJXcXMcVRtP9TjcEznvRz6MmQTFYHksZvZQDOrBT4Cqt39tSCuKyLSX6nu\n5RDH7TcDCezufsbdpwJjgBvM7A+6nmNmy8ysxsxqGhoagnhaEZFepbqXQxy33wy8HruZ/TXQ5O5P\n9naO6rGLSFisrK5POGRTOXd8aIZislaP3cxGmdnQ9n9/AZgH/Dbd64qIZEMct98MYijmUmCbmb0B\n7KVtjP2XAVxXRCTj4rj9prbGExEJQDYWOaU6FBO9tyIJjVRv5Lit6hPpKmyLnFS2V/ql40au2n6I\nE00tOOdv5EWrdtLY3Nqn80SiLGyLnBTYpV9SvZHDdsOLZEIqi5yySYFd+iXVGzlsN7xIJoRtkZMC\nu/RLqjdy2G54kUwI2yInBXbpl1Rv5LDd8CKZUDFzXLc8+A6d69JkiwK79EuqN3LYbniRTAjbIicF\ndumXVG/ksN3wIpkQtkVOWqAk/XYuP33PUU42fc6w4kFU3Di29zz2JOeJSGKpLlBSYBcRiQitPJVw\nWXFl9x3gtQmKSEZojF2yo3NQ7+lYRAKjwC4iEjOxHopR8SkRyUexjW5hq7YWd0nfREtGdx9jF5GM\niG1kS6X4VFi2u4q6lN5ENVEqkjWxHWNX8ansUQVHkXCJbWBX8ans0ZuoSLjENrCr+FT26E1UJFxi\nG9hVfCp79CYqEi6xDewqPpU9ehMNRmNzKyur65n2+FbKvv8rpj2+lZXV9do+UPostoE9bNXW4kxv\nounT3rASpLSLgJnZFcBPgd8DHFjt7k8n+hkVAYuRLjVgfsclTG/+e1Vw7KOV1fVUbT/U4yR0UcEA\nKueOV3quZLUIWCvwl+6+38wuAvaZWbW7/yaAa0vYdan5MpJPeLfoTjgD1I6GecpfT0UqmUUK7JKq\ntIdi3P0Dd9/f/u9TwAHg8nSvKzGgQl8pU2aRBCnQz8hmVgpcB7wW5HVF4m5YcSEnEgR3ZRalIQ9L\nRgc2eWpmQ4AXgPvd/X96eHyZmdWYWU1DQ0NQTyu5ppovgVBmUQblYcnoQHrsZlZIW1Bf7+7/0tM5\n7r4aWA1tk6dBPK+EQOeeT089I7lAb8XSKmaMY0vdh91KMyizSPoj7cBuZgasBQ64+8r0mySRFfOP\nt+lKVixt3Z/PYN1rR7Q3rKQtiLtlFnAX8KaZ1bZ/76/cfXMA1xaJjWTF0ta9doQH5k1U9kvQ8rBk\ndNqB3d1fBSyAtojEmlIacySbnyQTTdRmcRI3titPRcJGKY15INFEbRYncTVwJ5IlSmkMWB6mMaZK\nPXaRLFFKY8DyMI0xVeqxi2RJ5ZwJbKn7kPUf38Uo++Tc9xv8Er459P8qpTEOEk3UZnESV4FdJEs6\nKo6W/M0nF3x/lH2iiqNxkWgoKIvDRBqKEcmi3oK3gno/dO3x5kEaY6p0N4lINGmitFfqsYtkm3qa\nkmHqsUsbpY5lj36vkmHqsUsbpY6JxIYCu4hIzCiwS68bJWsDZZFoUmDPcx2lZBv8kgu+3+CXsGjV\nTgV3kQjS5Gme6yglO73177s9VnS8iaodB1VxUCRi1GPPc6mUkhWRaFFgz3MqJSsSPxqKyXMqJRsy\nWk8gAVCPPc+plGzIaD2BBECBPc9VzpnAuBHF3YJ7UcEAxo0oVilZkQjSUEye6yglW7XjIOv2HOVk\n0+cMKx5ExY1jqZwzQVUHJRQam1vb7tHdRzjZ1MKw4kIqZo7TPdoLc/esP2l5ebnX1NRk/XlFQk9j\n7N10rLU4crzpggyujk+V+VTL3sz2uXt5svPy47chEhV5HsR70rHWomtabnPrWY5orUWPAhljN7Pn\nzOwjM6sL4noiIh201qLvgpo8/SdgfkDXEhE5R2st+i6QwO7uO4ATQVxLRKSzYcWFCR93h2mPb2Vl\ndb1qG7VTuqOIhFqitRYdTjS1ULX9UGgL1zU2t7Kyup5pj2+l7Pu/yvgbUdYCu5ktM7MaM6tpaGjI\n1tOKSMT1ttaiq86TqWHSkdVTtf0QJ5pacDL/RpS1wO7uq9293N3LR40ala2nFZGI61hrUTl3PMNL\nEpe4CONkaipZPUHTUIyIhF5JUQEPzJvI/h/Ow5KcG7bJ1Fxk9QSV7vjPwG5gopkdM7M/D+K6IiJd\nJZtMDVvhulxk9QSVFXOHu1/q7oXuPsbd1wZxXRGRrqJWuC4Xb0QaihGRSIla4bpcvBEpsItIpHSd\nTDWD4SWDqJw7PpR1Y3LxRqQiYCIiGXauOmWaFVRTLQKmwC4iEhGpBnYNxYiIxIwCu4hIzCiwi4jE\njAK7iEjMhCsvSESkN9o2MGXqsYtINHQO6j0dyzkK7CIiMaPALiISMwrsIhINJaMTH8s5mjwVkWjQ\nRGnK1GMXEYkZBXYRkZhRYBcRiRkFdhGRmFFgFxGJGWXFSOSc27Rg9xFONrUwrLiQipnj+rxpgUhc\n6a9AIqWxuZVFq3Zy5HgTza1nATjR1ELV9kNsqfswlFujiWSbhmIkUqp2HLwgqHdobj3LkeNNVO04\nmKOWiYRHIF0bM5sPPA0MBP7R3f82iOtKP8W4Ct663Ue6BfUOza1nWbfnKA/Mm5jlVomES9o9djMb\nCDwL3AJMBu4ws8npXlfSEOMqeCebWpI8/nmWWiISXkH02G8A3nH3QwBmtgG4FfhNANcWucCw4kJO\nJAjuw4oHZbE1EogYf8LMlSDG2C8H/qvT8bH274kErmLmOIoKer5tiwoGUHHj2Cy3SNIW40+YuZK1\n9AEzWwYsAxg7NsJ/fFHoXZSM7t7GmKicM4EtdR92m0AtKhjAuBHFVM6ZkMPWiYRDEIH9PeCKTsdj\n2r93AXdfDawGKC8v9wCeNzei0LsI2xtNgEqKCti4fFZbHvueo5xs+pxhxYOouHGs8thF2gXxV7AX\nuNLMymgL6EuAOwO4rkiPSooKeGDeRGW/xEWMP2HmStqB3d1bzewvgH+lLd3xOXd/K+2WiUh+iPEn\nzFwJ5HOru28GNgdxrdBT70JEQk4Dkn2l3kW0JJrsjsJEuEg/KLBLvCWa7I7CRHjMqaBbZug3JyI5\noYJumaMiYCKdlH3/V0x7fCsrq+tpbG7NdXNiTQXdMkeBXeKt6+R25+MujzX4JTjne42LVu1UcM+g\nVAq6Sf/oc47EW6LJ0AffZmV1PVXbDyXsNSpfPjNU0C1z1GOXvKZeY+4MKy5M8rgKuvWXArvkNfUa\nc0cF3TJHQzExo/SxvlEZ4NxRQbfM0V96jORV+lhAi4sqZo7rcYwd1GvMNBV0yxz95mIklfSx2EwE\nBrS4SL3G3FJBt8xQYI8R7Qfadx29Rp68kpKWE+e+31g4HJa/rV6jRJLu2hjRRGD/lBQVQKegDrQF\neQV1iShlxcRIXqWPJVp4JJLn1CWJkbyaCFQVRpFeqcceI5VzJjBuRHG33GBNBKZAnwAkRtRjjxGl\nj6VBnwAkRsw9+/tKl5eXe01NTdafV0Qkysxsn7uXJztPXTjJS1qhK3GmO1jyTl6t0JW8pLs3SNpD\nMxLyaoWu5CVlxQRJe2hGgkr1Styl1WM3s9uBx4BJwA3urhlRCY3extETVXMErdCV6Et3KKYOuA2o\nCqAtIoFJNI4+wOBsgmSwWK3QlbyU1lCMux9w9/qgGhN5WuQSGonG0c2Mgdbzz8Vuha7kpaxNnprZ\nMmAZwNixMf3D0URpaCQaRz9z1hlobUFcpXqDoxTS8EjaYzezl82sroevW/vyRO6+2t3L3b181KhR\n/W+xSAqSVbo8C1TOHc/wkkGYwfCSQVTOHa9Ux37qGPqq2n6IE00tOOeHvhat2kljc2uum5hXkt7B\n7v6VbDREJEipbHmnDR6CoxTScFG6o8SSNkrOLqWQhktagd3MFpnZMWAm8Csz+9dgmiWSHlW6zK5k\nQ18nGj9nZXW9hmSyJN2smI3uPsbdi9z999z9q0E1TCQdHZUuNY6eHck2eQE03p5Fqu7YB5r1F+nZ\nyur6Xjd56ayoYACVc8drvL2fUq3uqDH2FGnWX6R3vQ19daXx9uxQYE9RKrP+Ivmq89BXMirZkHkK\n7CnSrL9IYiVFBTwwbyLD82lT9ZDSwHCKUp311/i75Lu82lQ9pNRjT1GyWf8BhsbfRVCqaRgosKco\n0YKXgQZmpvF3EZRqGgZKd0xRT2Vgoa0X0nLmbMIysMNLBrH/h/Oy0Mpwamlp4dixY3z22We5bkqs\nDB48mDFjxlBYmDyHXOJBm1kHrKMXUrXjIOv2HOVk0+cMKx5ExY1jeebf3kn4s/meBXDs2DEuuugi\nSktLMeulXq70ibtz/Phxjh07RllZWa6bIyGjwN4HHbP+XRdXrNt9JGnBqXz22WefKagHzMwYMWIE\nDQ0NuW6KhJDG2AOgglPJKagHT79T6Y167AGonDOBLXUf9jj+riyAvslW2YbHHnuMIUOG8N3vfrfH\nxzdt2sRVV13F5MmTA3tOkWxRYA9AovF35bGnLtE+pVvqPsxqRsWmTZtYsGCBAns6VlwJjR+dPy4Z\nrV3GskRDMQHpGH/f/8N5vPs3f8L+H87jgXkTFdT7INNlG5544gmuuuoqZs+eTX1921a9a9asYfr0\n6UyZMoWvf/3rNDU1sWvXLl566SUefPBBpk6dysGDB3s8T5LoHNR7OpaMUWCX0Mhk2YZ9+/axYcMG\namtr2bx5M3v37gXgtttuY+/evbz++utMmjSJtWvX8qUvfYmFCxeyYsUKamtrmTBhQo/niYSVupMS\nGsnKNqSTNvrKK6+waNEiiouLAVi4cCEAdXV1/OAHP+Djjz/m9OnTfPWrPW8pkOp5ImGgHruERrKy\nDZlIG7377rv58Y9/zJtvvsmjjz7a6yKqVM+TTkpGJz6WjFFgl9DIZNronDlz2LRpE59++imnTp3i\nF7/4BQCnTp3i0ksvpaWlhfXr1587/6KLLuLUqVPnjns7TxJ48G147JPzX5o4zRoFdgmNTBaPmjZt\nGosXL2bKlCnccsstTJ8+HYAf/ehHzJgxg1mzZnH11VefO3/JkiWsWLGC6667joMHD/Z6nkgYqVaM\nZNyBAweYNGlSSueey2NX2mhK+vK7lehTrRiJpN7KNohI6jQUIyISM2kFdjNbYWa/NbM3zGyjmQ0N\nqmEiItI/6Q7FVAMPu3urmf0d8DDwvfSbJSKRpDICoZBWj93dt7p7x75ve4Ax6TdJRCJLZQRCIcgx\n9j8D/n9vD5rZMjOrMbMa1ZAWEcmcpIHdzF42s7oevm7tdM4jQCvQ68oNd1/t7uXuXj5q1KhgWi+S\nI0OGDAHg/fff5xvf+EbCc5966qkLioZ97Wtf4+OPP85o+yS/JR1jd/evJHrczO4GFgB/5LlIio+p\nbNUll/POnDnDwIED+/Qzl112GT//+c8TnvPUU09RUVFxrk7N5s2b+93G0CsZ3X2MXbIurQhhZvOB\nh4C57q46pgEJU13ynMjABNzhw4eZP38+119/Pfv37+eaa67hpz/9KZMnT2bx4sVUV1fz0EMPMX36\ndO69914aGhooLi5mzZo1XH311bz77rvceeednD59mltvvfWC6y5YsIC6ujrOnDnD9773PbZs2cKA\nAQP41re+hbvz/vvvc/PNNzNy5Ei2bdtGaWkpNTU1jBw5kpUrV/Lcc88BsHTpUu6//34OHz7MLbfc\nwuzZs9m1axeXX345L774Il/4whfS+h1khSZKQyHdMfYfAxcB1WZWa2b/EECb8l6m65KHXoYm4Orr\n61m+fDkHDhzg4osvZtWqVQCMGDGC/fv3s2TJEpYtW8YzzzzDvn37ePLJJ1m+fDkA9913H/fccw9v\nvvkml156aY/XX716NYcPH6a2tpY33niDb37zm3znO9/hsssuY9u2bWzbtu2C8/ft28dPfvITXnvt\nNfbs2cOaNWv49a9/DcDbb7/Nvffey1tvvcXQoUN54YUXAvkdSH5INyvm/7j7Fe4+tf3r20E1LJ9l\nsi55PrviiiuYNWsWABUVFbz66qsALF68GIDTp0+za9cubr/9dqZOnUplZSUffPABADt37uSOO+4A\n4K677urx+i+//DKVlZUUFLR9mho+fHjC9rz66qssWrSIkpIShgwZwm233cYrr7wCQFlZGVOnTgXg\n+uuv5/Dhw2m8csk3Mf48H12ZrEuez7pu/txxXFJSAsDZs2cZOnQotbW1Kf18JhUVFZ3798CBA/n0\n00+z9twSfSopEEK5qEseKhmq43306FF2794NwPPPP8/s2bMvePziiy+mrKyMn/3sZwC4O6+//joA\ns2bNYsOGDQC9lu2dN28eVVVVtLa2Le04ceIE0L0EcIebbrqJTZs20dTURGNjIxs3buSmm24K4JVK\nvlNgD6GOuuR7i+7h8OA7z33tLbon7brkkZChOt4TJ07k2WefZdKkSZw8eZJ77rmn2znr169n7dq1\nTJkyhWuuuYYXX3wRgKeffppnn32WL37xi7z33ns9Xn/p0qWMHTuWa6+9lilTpvD8888DsGzZMubP\nn8/NN998wfnTpk3j7rvv5oYbbmDGjBksXbqU6667LpDXKvlNZXtDqCMrZusnC7s99seXvBS5rJgw\nlJbtnL0SJ2H43Ur2pFq2Vz32ECopKmDj8lk9Pha1oC4i2afAHlK9BW8F9f4pLS2NXW9dpDcK7GGm\nzYBFpB/U/QuzGK3ic/espgvmA1XwkN6oxy4ZN3jwYI4fP65AFCB35/jx4wwePDjXTZEQUo9dMm7M\nmDEcO3YMlWsO1uDBgxkzRlsgSHcK7JJxhYWFlJWV5boZInlDQzEiIjGjwC4iEjMK7CIiMZOTkgJm\n1gAc6eePjwR+F2BzwkavL9ri/Pri/NogGq9vnLsn3Vs0J4E9HWZWk0qthKjS64u2OL++OL82iNfr\n01CMiEjMKLCLiMRMFAP76lw3IMP0+qItzq8vzq8NYvT6IjfGLiIiiUWxxy4iIglEMrCb2Y/M7A0z\nqzWzrWZ2Wa7bFCQzW2Fmv21/jRvNbGiu2xQkM7vdzN4ys7NmFossBDObb2b1ZvaOmX0/1+0Jkpk9\nZ2YfmVksC9qb2RVmts3MftN+X96X6zalK5KBHVjh7te6+1Tgl8Bf57pBAasG/sDdrwX+E3g4x+0J\nWh1wG7Aj1w0JgpkNBJ4FbgEmA3eY2eTctipQ/wTMz3UjMqgV+Et3nwzcCNwb9f+/SAZ2d/+fTocl\nQKwmCtx9q7u3th/uAWJVws/dD7h7fa7bEaAbgHfc/ZC7fw5sAG7NcZsC4+47gBO5bkemuPsH7r6/\n/d+ngAPA5bltVXoiW93RzJ4A/hT4BLg5yelR9mfA/8t1IyShy4H/6nR8DJiRo7ZIGsysFLgOeC23\nLUlPaAO7mb0M/H4PDz3i7i+6+yPAI2b2MPAXwKNZbWCakr2+9nMeoe1j4vpsti0Iqbw+kTAxsyHA\nC8D9XUYFIie0gd3dv5LiqeuBzUQssCd7fWZ2N7AA+COPYE5qH/7/4uA94IpOx2PavycRYWaFtAX1\n9e7+L7luT7oiOcZuZld2OrwV+G2u2pIJZjYfeAhY6O5NuW6PJLUXuNLMysxsELAEeCnHbZIUWdtm\nvGuBA+6+MtftCUIkFyiZ2QvAROAsbVUiv+3usekhmdk7QBFwvP1be9z92zlsUqDMbBHwDDAK+Bio\ndfev5rZV6TGzrwFPAQOB59z9iRw3KTBm9s/Al2mrfvjfwKPuvjanjQqQmc0GXgHepC2mAPyVu2/O\nXavSE8nALiIivYvkUIyIiPROgV1EJGYU2EVEYkaBXUQkZhTYRURiRoFdRCRmFNhFRGJGgV1EJGb+\nF0mBsMPAw/aTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f39815f6358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred_test = kneighbor_regression.predict(X_test)\n",
    "\n",
    "plt.plot(X_test, y_test, 'o', label=\"data\", markersize=8)\n",
    "plt.plot(X_test, y_pred_test, 's', label=\"prediction\", markersize=4)\n",
    "plt.legend(loc='best');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On the test set, we also do a better job of capturing the variation, but our estimates look much messier than before.\n",
    "Let us look at the R<sup>2</sup> score:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.84934338190269976"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kneighbor_regression.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Much better than before! Here, the linear model was not a good fit for our problem; it was lacking in complexity and thus under-fit our data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Exercise\n",
    "=========\n",
    "Change the definition of y with a more complex formula"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Exercise\n",
    "=========\n",
    "Compare the KNeighborsRegressor and LinearRegression on the boston housing dataset. You can load the dataset using ``sklearn.datasets.load_boston``. You can learn about the dataset by reading the ``DESCR`` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Code here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}