07a-CDMS_NNRegressor.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8fa7657c",
   "metadata": {},
   "source": [
    "## Introduction to Neural Networks: Multi-layer Perceptrons\n",
    "\n",
    "In this notebook, we will start looking at a Neural Network based model for the regression problem on the CDMS dataset. The problem remains the same- we want to predict the impact location of a potential dark matter candidate on the detector prototype based on the phonon resonance timing measurements obtained from its different channels. The simplest form of a neural net is called a multi-layer perceptron (MLP). A fully connected MLP, also called a Dense Network sometimes, follows a simple construction. The architecture is described in terms of *layers*. The very first layer is composed of the input values and the very last layer is essentially the desired output. Each numerical component of a layer is called a *node*, so the number of nodes in the input layer is essentially the same as the dimension of the input feature space. Each layer between the input and output layers is called a hidden layer. Each hidden layer is constructed as a linear combination of the outputs of the previous layer and its output is transformed via a non-linear activation (like a sigmoid function). If a NN with $N$ hidden layers is designed approximate the following functional relationship between inputs $x$ and output $y$ such that\n",
    "\\begin{equation}\n",
    "y = f(x)\n",
    "\\end{equation}\n",
    "then the hidden layers are defined as\n",
    "\\begin{equation}\n",
    "h_{i+1} = \\sigma (W_{i+1} h_i + b_{i+1})\n",
    "\\end{equation}\n",
    "where $W_i, b_i$ respective represent the weights and biases of the $i$-th hidden layer, $\\sigma$ represents a non-linear activation function, and the index $i$ runs from $0$ to $N-1$. The initial and final layers are given as\n",
    "\\begin{equation}\n",
    "h_0 \\equiv x \\\\\n",
    "h_{N+1} \\equiv y\n",
    "\\end{equation}\n",
    "\n",
    "Our job is the determine the right set of model parameters, in this case the $W_i, b_i$ values, so that the NN can determine the functional relationship between the input and the output as closely as apossible. How this is done will be explained in greater details later. For the time being, we will look at data processing and model building. We will use the `PyTorch` libraries for creating our MLP network. Let's start by importing the necessary libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "31da1c8a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.model_selection import *\n",
    "import torch\n",
    "from torch import nn\n",
    "from torch import optim\n",
    "from torch.utils.data import DataLoader\n",
    "import pandas as pd\n",
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96ccc13d",
   "metadata": {},
   "source": [
    "Now we will load our data. PyTorch has a number of facilities that allow memor efficient data processing and loading. However, given it is a simple and small dataset, we'll stick to importing the data via `pandas` and storing them as `NumPy` arrays. Also note that in the following code block, we don't split the features and outputs right away. Additionally, the data standardization is only applied to the input features while the output is kept unnormalized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2a8cf63a",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Get the dataset as numpy array\n",
    "\n",
    "data = pd.read_csv('data/CDMS_Dataset.csv', sep=',')\n",
    "\n",
    "features = list(data.columns)[1:]\n",
    "xy_data = data.values[:,1:]\n",
    "n_features = len(features) - 1\n",
    "\n",
    "def split_data(split_type='random', labels = []):\n",
    "    if split_type == 'random':\n",
    "        XY_train, XY_test  = train_test_split(xy_data, test_size=0.25)\n",
    "    if split_type == 'label':\n",
    "        XY_train = data.loc[~data[features[-1]].isin(labels)].values[:, 1:]\n",
    "        XY_test = data.loc[data[features[-1]].isin(labels)].values[:, 1:]\n",
    "        \n",
    "    XY_train, XY_valid = train_test_split(XY_train, test_size=0.2, stratify=XY_train[:, -1])\n",
    "    x_mean = np.mean(XY_train[:,:-1], axis = 0)\n",
    "    x_std  = np.std(XY_train[:,:-1], axis = 0)\n",
    "    XY_train[:,:-1] = (XY_train[:,:-1] - x_mean)/x_std\n",
    "    XY_test[:,:-1]  = (XY_test[:,:-1] - x_mean)/x_std\n",
    "    XY_valid[:,:-1]  = (XY_valid[:,:-1] - x_mean)/x_std\n",
    "    \n",
    "    return XY_train, XY_valid, XY_test, x_mean, x_std"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de88b900",
   "metadata": {},
   "source": [
    "You can see we have generated three separate outputs here. \n",
    "\n",
    "- The `XY_test` dataset is associated with a subset of the data corresponding to a certain subset of impact locations in the dataset which the neural network never learns about during training. The performance of the NN on this dataset will shed light on its ability to interpolate for new impact locations. \n",
    "- On the other hand, the `XY_valid` is called the validation dataset- they are obtained from the same sample space as the training data. The performance of the NN on this dataset can be used to assure that the NN doesn't overtrain- i.e. so precisely tunes its parameters to correctly predict the outputs for the training dataset that it fails to generalize for datasets beyond the training subset\n",
    "- And finally, there is the training dataset `XY_train`. This is the dataset that we will use to train the NN, i.e. the NN's model parameters will be optimized to predict the outputs for this subset of the data as accurately as possible (without overtraining).\n",
    "\n",
    "Now, we will create a class to construct the network. It will be inherited from PyTorch's `nn.Module` class. While this class can typically include user defined custom methods, defining an inheritance class from `nn.Module` definitely requires defining the `forward` method- which tells what the output of the model will be when some data is given as input to it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e30e54e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Defining a class for a generic MLP\n",
    "\n",
    "class Network(nn.Module):\n",
    "    def __init__(self, Layers, device, activation, \n",
    "                 end_in_sigmoid = False,\n",
    "                 add_dropout = False,\n",
    "                 add_batchnorm = False):\n",
    "        super(Network, self).__init__()\n",
    "        self.Layers = Layers\n",
    "        self.device = device\n",
    "        self.activation = activation\n",
    "        self.end_in_sigmoid = end_in_sigmoid\n",
    "        self.add_dropout = add_dropout,\n",
    "        self.add_batchnorm = add_batchnorm,\n",
    "        self.nn = self.build_model().to(device)\n",
    "        \n",
    "    def build_model(self):\n",
    "        Seq = nn.Sequential()\n",
    "        for ii in range(len(self.Layers)-1):\n",
    "            this_module = nn.Linear(self.Layers[ii], self.Layers[ii+1])\n",
    "            Seq.add_module(\"Linear\" + str(ii), this_module)\n",
    "            if not (ii == len(self.Layers)-2):\n",
    "                if self.add_batchnorm[0]:\n",
    "                    Seq.add_module(\"BatchNorm\" + str(ii), nn.BatchNorm1d(self.Layers[ii+1]))\n",
    "                Seq.add_module(\"Activation\" + str(ii), self.activation)\n",
    "                if self.add_dropout[0]:\n",
    "                    Seq.add_module(\"Dropout\" +str(ii), nn.Dropout(p=0.5, inplace=False))\n",
    "            if self.end_in_sigmoid:\n",
    "                Seq.add_module(\"Sigmoid\" + str(ii), nn.Sigmoid())\n",
    "        return Seq\n",
    "    \n",
    "    def forward(self, X):\n",
    "        X = X.to(self.device)\n",
    "        return self.nn(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52774c4b",
   "metadata": {},
   "source": [
    "Now we want to create the functionality of training the NN, i.e. determining its parameters. Below, we create a custom class for our regression problem which constructs our NN and also provides a `train()` method, that can be used to train the NN. The training is done minimizing a loss function. In this case we are going to use the mean squared error (MSE) loss, which determines the averaged squared difference between true values of the desired output and the NN-evaluated values.\n",
    "\n",
    "\\begin{equation}\n",
    "\\mathcal{L}_{MSE} = \\frac{1}{N}\\sum_{i=1}^N \\left(y_i - f(x_i)\\right)^2\n",
    "\\end{equation}\n",
    "\n",
    "During each iteration, the parameters receive a correction based on how large their impact is on the loss function. \n",
    "\n",
    "\\begin{equation}\n",
    "\\theta_i^{(m+1)} = \\theta_i^{(m)} - \\eta \\frac{\\partial \\mathcal{L}}{\\partial \\theta}\n",
    "\\end{equation}\n",
    "Here $\\theta$ represents any of the model parameters and $\\eta$ is called learning rate, which is one of the hyperparameters of the model. This method of obtaining correction for model parameters is called the method of Gradient Descent. However, we typically don't use the entire dataset to perform gradient descent. During each iteration over the entire dataset, we divide the dataset into smaller subsets called *minibatches*. We choose these subsets so that each of them has substantially lower number of data entries but still has just enough data to represent the underlying physical probability distribution of the training dataset. This method is called **stochastic gradient descent**, which is typically much faster than the naive gradient descent method. The size of the minibatch is yet another hyperparameter of the model, and so is the number of iterations (also called epochs) used to train the model.\n",
    "\n",
    "AS you cans see, the `train()` method takes three arguments here. `n_epoch` and `batch_size` represent the hyperparameters number of epochs and minibatch size as described above. The other one is `lossfn` which is the loss function to be used to train the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ecee55b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "class CDMS_Regressor:\n",
    "    def __init__(self, Layers, XY_train, XY_valid, XY_test, device, activation=nn.Sigmoid()):\n",
    "        self.XY_train = torch.tensor(XY_train, dtype = torch.float32, device = device)\n",
    "        self.XY_test  = torch.tensor(XY_test,  dtype = torch.float32, device = device)\n",
    "        self.XY_valid = torch.tensor(XY_valid, dtype = torch.float32, device = device)\n",
    "        self.device = device\n",
    "        self.MLP = Network(Layers, device, activation, \n",
    "                           end_in_sigmoid = False,\n",
    "                           add_dropout = True,\n",
    "                           add_batchnorm = True)\n",
    "        \n",
    "    def predict(self,X):\n",
    "        return self.MLP.forward(X).to(self.device)\n",
    "    \n",
    "    def Train(self, n_epochs, batch_size, lossfn):\n",
    "        Train_Losses = []\n",
    "        Validation_Losses = []\n",
    "        Test_Losses = []\n",
    "        params = list(self.MLP.nn.parameters())\n",
    "        optimizer = optim.Adam(params, lr=1e-3)\n",
    "        for ii in range(n_epochs):\n",
    "            self.MLP.train()\n",
    "            DL = DataLoader(self.XY_train, batch_size=batch_size, shuffle=True, drop_last=True)\n",
    "            for xy in DL:\n",
    "                optimizer.zero_grad()\n",
    "                y_pred = self.predict(xy[:,:-1]).reshape(-1)\n",
    "                loss = torch.sqrt(lossfn(y_pred, xy[:,-1].reshape(-1)))\n",
    "                loss.backward()\n",
    "                optimizer.step()\n",
    "                \n",
    "            self.MLP.eval()    \n",
    "            with torch.no_grad():\n",
    "                train_loss = np.sqrt(float(lossfn(self.predict(self.XY_train[:,:-1]).reshape(-1), \n",
    "                                                  self.XY_train[:,-1])))\n",
    "                test_loss =  np.sqrt(float(lossfn(self.predict(self.XY_test[:,:-1]).reshape(-1), \n",
    "                                                  self.XY_test[:,-1])))\n",
    "                val_loss =   np.sqrt(float(lossfn(self.predict(self.XY_valid[:,:-1]).reshape(-1), \n",
    "                                                  self.XY_valid[:,-1])))\n",
    "                Train_Losses.append(train_loss)\n",
    "                Test_Losses.append(test_loss)\n",
    "                Validation_Losses.append(val_loss)\n",
    "                \n",
    "                if ii % int(n_epochs/10) == 0 or ii == n_epochs - 1:\n",
    "                    print(\"Epoch {0}/{1}, Training Loss = {2:0.4f}, Validation Loss = {3:0.4f}, Test Loss = {4:0.4f}\".format(ii, n_epochs, train_loss, val_loss, test_loss))\n",
    "            \n",
    "        \n",
    "        return Train_Losses, Test_Losses, Validation_Losses\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a35a8cf2",
   "metadata": {},
   "source": [
    "Now we perform the actual fitting. We need to decide how many hidden layers we want and how many nodes we want for each layer. These details again are additional hyperparameters of the model. In this problem, we will use a simple network with one hidden layer and use 32 nodes for the hidden layer. The activation function we are going to use is called `LeakyReLU` function, defined as-\n",
    "\n",
    "\\begin{equation}\n",
    "\\mathrm{LeakyReLU} (x) = \\begin{cases}\n",
    "x; \\quad  x \\ge 0 \\\\\n",
    "mx; \\quad x < 0\n",
    "\\end{cases}\n",
    "\\end{equation}\n",
    "where $m$ is generally a small positive number. Let's now train the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "71342ae7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0/100, Training Loss = 23.6176, Validation Loss = 23.6023, Test Loss = 30.2627\n",
      "Epoch 10/100, Training Loss = 7.0905, Validation Loss = 7.0100, Test Loss = 11.0949\n",
      "Epoch 20/100, Training Loss = 1.9805, Validation Loss = 1.9554, Test Loss = 2.2737\n",
      "Epoch 30/100, Training Loss = 1.8398, Validation Loss = 1.8276, Test Loss = 2.2254\n",
      "Epoch 40/100, Training Loss = 1.7918, Validation Loss = 1.7665, Test Loss = 2.3288\n",
      "Epoch 50/100, Training Loss = 1.6203, Validation Loss = 1.5915, Test Loss = 1.9770\n",
      "Epoch 60/100, Training Loss = 1.6804, Validation Loss = 1.6503, Test Loss = 2.0657\n",
      "Epoch 70/100, Training Loss = 1.5778, Validation Loss = 1.5412, Test Loss = 1.7220\n",
      "Epoch 80/100, Training Loss = 1.7213, Validation Loss = 1.6898, Test Loss = 2.2021\n",
      "Epoch 90/100, Training Loss = 1.7891, Validation Loss = 1.7545, Test Loss = 2.3751\n",
      "Epoch 99/100, Training Loss = 1.5932, Validation Loss = 1.5629, Test Loss = 2.0547\n"
     ]
    }
   ],
   "source": [
    "NhiddenLayers = 1\n",
    "NNodes = 32\n",
    "Layers = [n_features] + NhiddenLayers*[NNodes] + [1]\n",
    "XY_train, XY_valid, XY_test,  x_mean, x_std = split_data('label', [-12.502, -29.5, -41.9])\n",
    "lossfn = nn.MSELoss()\n",
    "device = torch.device('cpu')\n",
    "activation = nn.LeakyReLU(negative_slope=0.01)\n",
    "n_epochs = 100\n",
    "batch_size = 128\n",
    "\n",
    "NNreg = CDMS_Regressor(Layers = Layers, \n",
    "                      XY_train = XY_train, \n",
    "                      XY_test = XY_test, \n",
    "                      XY_valid = XY_valid,\n",
    "                      device = device, \n",
    "                      activation=activation)\n",
    "\n",
    "Train_Losses, Test_Losses, Validation_Losses = NNreg.Train(n_epochs = n_epochs,\n",
    "                                                           batch_size = batch_size,\n",
    "                                                           lossfn = lossfn)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9022a36f",
   "metadata": {},
   "source": [
    "You can try to change different hyperparameters of the model and see how the model's performace changes. Try using different batch sizes, adding more hidden layers, chaning the activation function. Also, in the model description, there are a couple of flag variables called `add_dropout` and `add_batchnorm`. I would strongly suggest you to try changing them and see how the behavior and the performance of the model changes.\n",
    "\n",
    "Now, we will evaluate the performance of the model. First, let's check the evolution of the loss function for training, validation, and test datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "411edca8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f3990132c10>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAxIUlEQVR4nO3deXyU1dnw8d+VkJCQHYgkEARccSkujb7uRcW0QESlvriAItqqVV6i9mkrj09t5HlbbKul2L51KaCASGuRiiZiodSthaJoeVBRgbJDgLAFSIRs1/vH3DPOJDOTScgsmbm+n08+mTlzL+fMct33nDn3dURVMcYYkziSol0BY4wxkWWB3xhjEowFfmOMSTAW+I0xJsFY4DfGmATTLdoVCEXv3r114MCB0a6GMcZ0KR9++OFeVc1vWd4lAv/AgQNZtWpVtKthjDFdiohs8VduXT3GGJNgLPAbY0yCscBvjDEJxgK/McYkmC7x464x5ivNzc1s376d2traaFfFRFlKSgonnHAC2dnZ7VovbgN/5cZKpn80nV21uyjIKKDs/DJGnjQy2tUy5rjt3bsXEeH0008nKcm+tCcqVeXLL79kx44dAO0K/nH5rqncWEn58nKqaqtQlKraKsqXl1O5sTLaVTPmuB08eJA+ffpY0E9wIkKPHj3o168fe/bsade6YXvniEiaiLwvIv8jIp+KyGNOeU8RWSoi653/eZ297+kfTedo01GfsqNNR5n+0fTO3pUxEdfU1ERKSkq0q2FiRHp6Og0NDe1aJ5ynDMeAq1T1HOBc4FsichHwMLBMVU8Fljn3O9Wu2l3tKjemqxGRaFfBxIiOvBfCFvjV5YhzN8X5U+A6YLZTPhu4vrP3XZBR0K5yY4xJJGHtJBSRZBFZDewBlqrqSqCPqlYBOP9PCLDu3SKySkRWVVdXt2u/ZeeXkZac5lOWlpxG2fllHWiFMSbShg8fzuzZs9tesJ3LGpewjupR1SbgXBHJBf4sIme3Y93ngOcAiouL2zU/pHv0jo3qMSZyMjMzPbfr6uro3r07ycnJADz77LOMHTs25G0tXrw4LMu2x9tvv824cePYvn17WLYfTREZzqmqB0XkbeBbwG4RKVTVKhEpxPVtoNONPGmkJ9C7h3ZOfm+yHQRMwlJVn/7glveP15EjRzy3Bw4cyIwZMxg2bFir5RobG+nWLW5HkncJ4RzVk++c6SMi6cAw4HPgNWC8s9h4YFG46gA2tNMYgGlL1zGlYi2qri/PqsqUirVMW7ou7Pt+++23KSoq4uc//zkFBQVMmDCBAwcOUFpaSn5+Pnl5eZSWlvqcWQ8dOpQZM2YA8MILL3DZZZfxH//xH+Tl5TFo0CCfs/z2LLtp0yauuOIKsrKyGDZsGPfffz/jxo1rd5s+++wzhg4dSm5uLmeddRavvfaa57E33niDM888k6ysLPr168cTTzwBuK6/KC0tJTc3l549e3L55ZfT3Nzc7n13hnD28RcCb4nIGuADXH38FcDjwDUish64xrkfNja00yQ6VeXQ0Qae/8dmT/CfUrGW5/+xmUNHGzwHg3DatWsX+/fvZ8uWLTz33HM0NzczYcIEtmzZwtatW0lPT2fixIkB11+5ciWnn346e/fu5Yc//CF33XVXwHoHW/bWW2/lwgsvZN++fZSXlzN37tx2t6WhoYFrr72WkpIS9uzZw29+8xvGjh3LF198AcBdd93Fs88+y+HDh/nkk0+46qqrAHjyyScpKiqiurqa3bt387Of/Sxqo7PC9n1LVdcA5/kp3wdcHa79tmRDO02iExEeLT0TgOf/sZnn/7EZgAmXDuTR0jMjEnySkpJ47LHH6N69O+Aae/7tb3/b8/gjjzzClVdeGXD9AQMG8N3vfheA8ePHc99997F7924KClqP1Au0bH19PR988AHLli0jNTWVyy67jFGjRrW7Lf/85z85cuQIDz/8MElJSVx11VWUlpYyf/58ysvLSUlJYe3atZxzzjnk5eWRl+e6VCklJYWqqiq2bNnCKaecwuWXX97ufXeWuL/0z4Z2GuMb/N0iFfQB8vPzSUv7aqRdXV0d99xzDwMGDCA7O5srrriCgwcP0tTU5Hd97wDfo0cPwPc3hVCW3blzJz179vSUAfTv37/dbdm5cyf9+/f3uXJ6wIABntQJr7zyCm+88QYDBgzgG9/4BitWrADgBz/4AaeccgolJSWcdNJJPP54WDs7gor7wG9DO435qk/fm3eff7i1PMA8+eSTfPHFF6xcuZJDhw7x7rvveuoZLoWFhezfv5+6ujpP2bZt29q9nb59+7Jt2zaf/vmtW7fSr18/AC644AIWLVrEnj17uP766xkzZgwAWVlZPPnkk2zcuJHXX3+dX/3qVyxbtuw4W9UxcR/4R540kvJLyinMKEQQCjMKKb+k3Eb1mITh3ac/4dKBbJo6ggmXDvTp84+0w4cPk56eTm5uLvv37+exxx4L+z4HDBhAcXEx5eXl1NfXs2LFCl5//fU21zt69KjP34UXXkhGRga/+MUvaGho4O233+b111/n5ptvpr6+nnnz5lFTU0NKSgrZ2dmeIa0VFRVs2LABVfWUux+LtIQYU+U9tNOYRCMiZKel+PTpu7t9stNSovID4wMPPMCtt95K79696du3L9///vd59dVXw77fefPmcccdd9CrVy8uvPBCbrrppoDdSwA7duwgPT3dp2z9+vW89tpr3HfffUydOpV+/foxZ84cBg8eTH19PXPnzmXixIk0NTVx+umn8+KLL3rWmzhxItXV1eTl5XHfffcxdOjQcDY3IInG0b69iouL1SZbN8bls88+44wzzmj3euEex98V3XTTTQwePDgi3zjCKdB7QkQ+VNXiluVx39XTUuXGSkoWlDBk9hBKFpTYeH6TMFoG+UQM+h988AH//ve/aW5u5s0332TRokVcf/310a5WxCVEV4+b+2Iu97h+98VcgHUFGZMAdu3axejRo9m3bx9FRUU8/fTTnHdeq1HncS+hAn+wi7ks8BsT/6699lquvfbaaFcj6hKqq8cu5jLGmAQL/HYxlzHGJFjgt4u5jDEmwfr4LU+/McYkWOAHu5jLGGMSqqvHGBO7RIQNGzYAcO+99/Lf//3fIS3bXvPmzaOkpKRD68YLC/zGmE7xzW9+k0cffbRV+aJFiygoKKCxsTHkbT3zzDP8+Mc/Pu46bd68GRHx2ffYsWNZsmTJcW+7JfeEM11BQgd+u4rXmM5zxx13MHfu3FZJ3+bOncvYsWNtusUYkrCB36ZkNAlnzcsw7Wwoz3X9X/Nyp27++uuvZ//+/bz33nuesgMHDlBRUcHtt9/O+++/z8UXX0xubi6FhYVMnDiR+vp6v9u64447+K//+i/P/V/+8pcUFhbSt29fZs2a5bNsZWUl5513HtnZ2fTv35/y8nLPY1dccQUAubm5ZGZmsmLFCs/0jG7Lly/nggsuICcnhwsuuIDly5d7Hhs6dCg//vGPufTSS8nKyqKkpIS9e/e2+7mJtakaEzbw25SMJqGseRlenwQ12wB1/X99UqcG//T0dMaMGcOcOXM8ZS+//DKDBw/mnHPOITk5mWnTprF3715WrFjBsmXL+N3vftfmdt98802eeOIJli5dyvr16/nrX//q83hGRgZz5szh4MGDVFZW8vTTT3syfbrz/B88eJAjR45w8cUX+6y7f/9+Ro4cyaRJk9i3bx8PPfQQI0eOZN++fZ5lXnrpJZ5//nn27NlDfX29JzCHKhanakzYwG9X8ZqEsmwKNHzpW9bwpau8E40fP54//elPfPmla19z5sxh/PjxAHz961/noosuolu3bgwcOJB77rmHd955p81tvvzyy0yYMIGzzz6bjIwMnzN6cJ2Vf+1rXyMpKYkhQ4Zwyy23hLRdcH1bOPXUU7ntttvo1q0bt9xyC4MHD/bJ0z9hwgROO+00z4Ft9erVoT0ZDu+pGlNTU32magQ8UzUeOnSIvLw8zj//fE+5e6rGlJQULr/8cgv8x8uu4jUJpWZ7+8o76LLLLiM/P59FixaxceNGPvjgA2699VYA1q1bR2lpKQUFBWRnZ/Of//mfIXWbuKc6dBswYIDP4ytXruTKK68kPz+fnJwcnnnmmZC7Y3bu3Nlqe97TKELrqRwDTfnYVv1jaarGhA38dhWvSSg5AUabBCo/Drfffjtz5sxh7ty5lJSU0KdPHwC+973vMXjwYNavX8+hQ4f42c9+FtLsX4WFhT5TJG7dutXn8VtvvZVRo0axbds2ampquPfeez3bbesMuW/fvmzZssWnzHsaxc4Qi1M1JmzgtykZTUK5+lFI8Z1JipR0V3knu/322/nrX//K73//e083D7imW8zOziYzM5PPP/+cp59+OqTtjRkzhhdeeIG1a9dSV1fXatKUw4cP07NnT9LS0nj//fd56aWXPI/l5+eTlJTExo0b/W57xIgRrFu3jpdeeonGxkb++Mc/snbtWkpLSzvQcpeuMFVjwgZ+cAX/JTcuYc34NSy5cYkFfRO/hoyBa5+CnP6AuP5f+5SrvJMNHDiQSy65hNraWkaNGuUpf+KJJ3jppZfIysriu9/9LjfddFNI2xs+fDgPPPAAV111Faeccornx0+33/3udzz66KNkZWUxZcoUzxkzuLpmHnnkES699FJyc3P55z//6bNur169qKio4Mknn6RXr1784he/oKKigt69e3eo7e6pGr3/tm3bxmuvvcbixYvp3bs39913n2eqRnANdx04cCDZ2dk888wzPlM1Dhs2jMzMTC6++OJOnarRpl40povp6NSLJn7Z1IvGGGOCClvgF5H+IvKWiHwmIp+KSJlTXi4iO0RktfM3Ilx1MMYY01o4r6FuBL6vqh+JSBbwoYgsdR6bpqrtuwrCGGNMpwhb4FfVKqDKuX1YRD4DOm+MlDHGmA6JSB+/iAwEzgNWOkUTRWSNiMwSkbwA69wtIqtEZFV1dXXY62gJ24wxiSLsgV9EMoFXgAdU9RDwNHAycC6ubwRP+ltPVZ9T1WJVLc7Pzw9rHS1hmzEmkYQ18ItICq6gP09VFwKo6m5VbVLVZuD3wIXhrEMoLGGbMSaRhHNUjwAzgc9U9Vde5YVei90AfBKuOoTKErYZYxJJOEf1XArcBnwsIqudsv8EbhGRcwEFNgP3hLEOISnIKKCqtspvuTHGxJuwnfGr6t9VVVR1iKqe6/y9oaq3qerXnPJRzuifqLKEbcYcv8zMTM9fUlIS6enpnvvz5s1r9/aGDh3KjBkzAj7ub1pFE5q4nQtNVX0y87W8782do2f6R9PZVbuLgowCys4vs9w9xrSDd7rigQMHMmPGDIYNGxbFGplA4jJlw7Sl61g4exrqTDOn085m4expTFu6LuA6lrDNxLtoDVlubm7m8ccf5+STT6ZXr16MGTOG/fv3A65MluPGjaNXr17k5uZywQUXsHv3bh555BHee+89Jk6cSGZmJhMnTmzXPnfu3MmoUaPo2bMnp5xyCr///e89j73//vsUFxeTnZ1Nnz59eOihh4LWJR7F3Rm/qnLijgqGb5qKiGs+T6nZxvCDU1kMqD7YabPYGNNVuIcsu0evuYcsA2E/yXnqqad49dVXeeedd8jPz2fSpEncf//9zJ8/n9mzZ1NTU8O2bdvo3r07q1evJj09nZ/+9Kf84x//YNy4cXznO99p9z5vueUWzjrrLHbu3Mnnn3/ONddcw0knncTVV19NWVkZZWVl3HbbbRw5coRPPnGNLwlUl3gUd2f8IsLoA7PoIb6TOPeQekYfmGVB3ySkaA5ZfvbZZ/npT39KUVER3bt3p7y8nAULFtDY2EhKSgr79u1jw4YNJCcn8/Wvf53s7Ozj2t+2bdv4+9//zs9//nPS0tI499xz+c53vsPcuXMB15SGGzZsYO/evWRmZnLRRRd5yju7LrEq7gI/gASYTi5QuTHxLppDlrds2cINN9xAbm4uubm5nHHGGSQnJ7N7925uu+02vvnNb3LzzTfTt29ffvjDH9LQ0HBc+9u5cyc9e/YkKyvLU+Y91eHMmTNZt24dgwcP5oILLqCiogIgLHWJVXEZ+DXAdHKByo2Jd9GcY7p///4sXryYgwcPev6OHj1Kv379SElJ4Sc/+Qlr165l+fLlVFRUMGfOHKDtaRMD6du3L/v37+fw4cOeMu+pDk899VTmz5/Pnj17+NGPfsSNN95IbW1t0LrEm7gL/KrKwrw7qdNUn/I6TWVh3p2oakjzfBoTT6I5ZPnee+/lkUce8cxtW11dzaJFiwB46623+Pjjj2lqaiI7O5uUlBTP9IJ9+vQJOGWit2PHjvlMddivXz8uueQSJk+ezNGjR1mzZg0zZ85k7NixALz44otUV1eTlJREbm4uAMnJyUHrEm/iLvCLCFv7lbJ40GQ0pz+KUJuUjXZLZ/TmKfDrr7U5wgcsaZuJL9GcY7qsrIxRo0ZRUlJCVlYWF110EStXuvI17tq1ixtvvJHs7GzOOOMMvvGNbzBu3DjPegsWLCAvL49JkyYF3H5mZqbPVId/+9vfmD9/Pps3b6Zv377ccMMNPPbYY1xzzTUAvPnmm5x11llkZmZSVlbGH/7wB9LS0oLWJd7E7dSL7nYtnD2N4Zum+vzYW6epLB40mdHj/Y/waTkCAlxnRzYZu4kFNvWiacmmXnSISIdH+FjSNmNMPIvbwO/WkRE+lrTNGBPP4j7wd2SETzRHQBhjTLjFdeAPZYSPP5a0zRgTz+IuZYM3zwgfcPXp12xHc4pYnHcnW/uVWtI202UFSzpoEktzc3O714nbUT3evD8k7vZ637cPkOlKNm3aRFZWFr169bL3bgJTVRoaGti9e7crR9mJJ7ZaJtConrg+43dzfzimLV3HiTsqGH1gFjhn/wuds/8HrzktyrU0JjRFRUVs376d6urqaFfFRFm3bt3Iycmhd+/e7VsvTPWJOZa108SLlJQUBg0aFO1qmC4srn/c9WZZO40xxiVhAj9Y1k5jjIEEC/yWtdMYYxIo8Hd0TD9YwjZjTHxJmB93OzqmP5pT1hljTDgkxDh+by3H7bc1jr9kQQlVtVWtygszClly45JOqZMxxoRDwmXnDCRY0Pd3ELSEbcaYeJMwXT0teV/MJUEu5irIKPB7xm8J24wxXVXYzvhFpL+IvCUin4nIpyJS5pT3FJGlIrLe+Z8XrjoE4nMxV802QF0Xc22ayok7KnzO/C1hmzEm3oSzq6cR+L6qngFcBNwvImcCDwPLVPVUYJlzP6LaczFXNKesM8aYcAhbV4+qVgFVzu3DIvIZ0A+4DhjqLDYbeBv4UbjqEUh7LuYaedJIC/TGmLgRkR93RWQgcB6wEujjHBTcB4cTIlGHluxiLmNMogp74BeRTOAV4AFVPdSO9e4WkVUisqqzsxAez8VcxhjT1YU18ItICq6gP09VFzrFu0Wk0Hm8ENjjb11VfU5Vi1W1OD8/v7Pr5bqYa9BkNKc/IGhOfxYPmhz0Yi5jjIkHYbuAS1zRczawX1Uf8Cr/JbBPVR8XkYeBnqr6w2Db6swLuLy192IuY4zpSjp8AZeInCwi3Z3bQ0VkkojkhrDPS4HbgKtEZLXzNwJ4HLhGRNYD1zj3o6JlkLegb4xJBKGM6nkFKBaRU4CZwGvAS8CIYCup6t+BQJH06vZUMpZUbqy0uXiNMV1aKIG/WVUbReQG4Neq+hsR+Ve4KxZpoXT7WMI2Y0w8CCXwN4jILcB44FqnLCV8VYq8UNM3TP9ouifoux1tOsr0j6Zb4DfGdBmhjOqZAFwM/FRVN4nIIODF8FYrctqTvsESthlj4kGbgV9V16rqJFWd7+TVyVLVqP0g29nak74hUGI2S9hmjOlKQhnV87aIZItIT+B/gOdF5Ffhr1rkhJq+wRK2GWPiQShdPTnOFbejgedV9evAsPBWK7JCTd9gCduMMfEglB93uzlX2I4BHglzfSLOnb5h+MGpPt09dZrK4rw7Gd1idI8lbDPGdHWhnPFPAf4C/FtVPxCRk4D14a1W5Fj6BmNMokm4OXcDsfQNxph4czwpG4pE5M8iskdEdovIKyISd7mLLX2DMSZRhNLV8zyuNA19cU2k8rpTZowxpgsKJfDnq+rzqtro/L0AdG6e5BjUsgusK3SJGWNMKEIZ1bNXRMYB8537twD7wlel6As1hYMxxnRFoZzx34lrKOcuXHPo3ogrjUNcak8KB2OM6YpCSdmwVVVHqWq+qp6gqtcDk8JftehoTwoHY4zpijo69eKYTq1FjAk1hUPlxkpKFpQwZPYQShaUULmxMhLVM8aY49LRwB/Xp72hpHBw5+avqq1CUU9ufgv+xphYFzDwi0jPAH+9iOPA707hUKepPuV1msrCvDs9ffzBcvMbY0wsCzaq50NA8R/k6/2UxQVPCgfwGdWz2BnV4+7jt9z8xpiuKmDgV9VBkaxILHnwmtNQfRCRhwDXka9lsraCjAKqaqtarWu5+Y0xsa6jffxxr60UDpab3xjTVYVyAZfxw52aefpH09lVu4uCjALKzi+zlM3GmJhngf84WG5+Y0xXFGxUz1Vetwe1eGx0OCsVayxvjzEmngQ7438CON+5/YrXbYD/AhaGq1KxxPL2GGPiTbAfdyXAbX/345Ll7THGxKNggV8D3PZ3vxURmeVM3vKJV1m5iOwQkdXO34h21jeiLG+PMSYeBevqOUlEXsN1du++jXM/lDH+LwC/Bea0KJ+mqk+0t6LREmreHmOM6SqCBf7rvG63DNRtBm5VfVdEBnakUrFEc4qcbh4/5VGojzHGHK+AXT2q+o73H7AcOAR85tzvqIkissbpCsoLtJCI3C0iq0RkVXV19XHsruNCzdtjjDFdSbDhnM+IyFnO7Rzgf3B12/xLRG7p4P6eBk4GzsU1qcuTgRZU1edUtVhVi/PzozPToydvz6DJaE5/QNCc/iweNNknb48xxnQlEuisVUQ+VVV34H8AGKqq14tIAbBYVc9rc+Ourp4KVT27PY+1VFxcrKtWrWprsbDRFnl6Wt43xphYJCIfqmpxy/Jgo3q8h7JcA7wKoKodTj8pIoVed28APgm0bCxpK2+PMcZ0JcF+3D0oIqXADuBS4C4AEekGpLe1YRGZDwwFeovIduAnwFARORfXcNDNwD3HUXdjjDEdECzw3wM8BRQAD3id6V8NtDnNlKr6+x1gZrtraIwxplMFy8e/DviWn/K/AH8JZ6ViWaD+/sqNlZap0xjTJQQM/CLyVLAVVXVS51cntgXK2/NWXjYrDz/rmYrRPf8uYMHfGBNzgnX13Ivrx9eXgZ0kSH6eQHzy9jgpHKRmG8MPTmUaJ3NU/M+/a4HfGBNrggX+QuB/AzcBjcAfgVdU9UAkKhZr3Hl7xE/enkMcwd9x0ebfNcbEomBX7u5T1WdU9UrgDiAX+FREbotQ3WJOoPw8BY1N/stt/l1jTAxqc85dETkfeAAYBywGPgxznWKW5hT5LZ90LNnm3zXGdBnBUjY8JiIfAg8B7wDFqnqXqq6NWO1iSLC8Pccy7uQnF/+EwoxCBKEwo5DyS8qtf98YE5OC9fH/GNgInOP8/cwZxiiAquqQ8Fcvdnjy9oDPqJ7F7tm4Tj6N0pNLo11NY4xpU7BcPQOCraiqW8JSIz+inavHm+XtMcZ0FYFy9QS7gMtvYBeRZOBmIGKBP5ZY3h5jTFcXrI8/W0Qmi8hvRaREXP4Pru6fMZGrojHGmM4UrI9/LnAAWAF8B/gBkApcp6qrw181Y4wx4RB0zl1V/RqAiMwA9gInqurhiNTMGGNMWAQbx9/gvqGqTcAmC/q+Wv4wblMxGmO6gmBn/OeIyCHntgDpzn33cM7ssNcuhk1buo5DRxt4tPRMRARVZUrFWrLTUnjwmtOiXT1jjAko2Kie5EhWpCtRVQ4dbWDfihep+fgVcur3UJN6AvuOfBsuHmdDPI0xMS3YGb8JQER4dMCnNHw0k9T6YwDk1u/mie4zSRlwHs4c9cYYE5PazNVj/JNlU0jVYz5lqXoMWTYlSjUyxpjQWODvIA2QqVNrtlO5sZKSBSUMmT2EkgUlVG5sc6ZKY4yJGOvq6QBVpSb1BHLrd7d67E85BfxyebnNxmWMiVl2xt8BIsLyAfdTL919yuulO7/pmecJ+m7u2biMMSYWWODvoBFjy0i54beQ0x8QyOlPyg2/pYZav8vbbFzGmFhhXT3HQYaMgSFfpS0SoGDdDKpqq1ota7NxGWNihZ3xd7Ky88tsNi5jTEyzM/5O5v4Bd/pH09lVu4uCjALKzi+zH3aNMTEjbIFfRGYBpcAeVT3bKesJ/BEYCGwGxqjqgXDVIVpGnjTSAr0xJmaFs6vnBeBbLcoeBpap6qnAMue+McaYCApb4FfVd4H9LYqvA2Y7t2cD14dr/8YYY/yL9I+7fVS1CsD5f0KgBUXkbhFZJSKrqqurI1ZBY4yJdzE7qkdVn1PVYlUtzs/Pj3Z1jDEmbkQ68O8WkUIA5/+eCO8/bGxSFmNMVxHp4ZyvAeOBx53/iyK8/7CYtnQdJ+6oYPSBWUjNdjSniIV5d7K1X6lNymKMiTnhHM45HxgK9BaR7cBPcAX8l0XkLmAr8L/Dtf9IUVVO3FHB8E1TEakHQGq2MfzgVBYDFf8+jaf+9ZSN6TfGxIywBX5VvSXAQ1eHa5/RICKuM30n6Lv1kHq6187isRXplqnTGBNTYvbH3a5EAuTmf6p7k2XqNMbEHAv8nUBzivyW7+rmf9piy9RpjIkmC/zHSVVZmHcndZrqU16nqWST6Xcdy9RpjIkmC/zHSUTY2q+UxYMmo05ufs3pz+JBkzk37zuWqdMYE3OkK4w3Ly4u1lWrVkW7GkGpKiLS6n7lxkrL1GmMiQoR+VBVi1uWW1rmTuId9L3vW6ZOY0yssa4eY4xJMBb4jTEmwVjgN8aYBGOB3xhjEowF/jCwTJ3GmFhmgb+TTVu6jikVaz3BXlWZUrGWaUvXRblmxhjjYsM5O5GqcuhoA/tWvEjNx6+QU7+HmtQT2Hfk23DxuFZj/Y0xJhos8HciEeHRAZ/S8NFMUuuPAZBbv5snus8kZcB5iJwV5RoaY4x19XQ6WTaFVD3mU5aqx5BlU6JUI2OM8WWBv5NpgBTNWrOdyo2VlCwoYcjsIZQsKKFyY2WEa2eMMdbV06lUlZrUE8it393qsT/lFPDL5eU2KYsxJursjL8TiQjLB9xPvXT3Ka+X7vymZ55NymKMiQkW+DvZiLFlpNzwW3BSNJPTn5QbfksNtX6Xt0lZjDGRZl09YSBDxsCQMcBX6ZkL1s2gqraq1bI2KYsxJtIs8IfRtKXrOHFHBaMPzKKscT/l+T056jWO3yZlMcZEgwX+MFFVTtxRwfBNUxGpx/XzrfLrvFx2detGYWahTcpijIkKC/xhIiKMPjALkXpP2cjaOkbW1qE5/ZE7lkSxdsaYRGY/7oaRBBjTH6jcGGMiwQJ/GGlOUbvKjTEmEqIS+EVks4h8LCKrRSS2Z1HvIFVlYd6d1GmqT3mdprIw705L1WyMiZpo9vFfqap7o7j/sBIRtvYrZTG4+vprtqM5RSzOu5Ot/UotS6cxJmrsx90wevCa01B9EJGHABBgtKVmNsZEWbT6+BVYIiIfisjd/hYQkbtFZJWIrKquro5w9TpPyyBvQd8YE23RCvyXqur5wHDgfhG5ouUCqvqcqharanF+fn7kaxgG3v36lqnTGBMtUQn8qrrT+b8H+DNwYTTqEUneUzJWbqykfHk5VbVVKOrJ1GnB3xgTCREP/CKSISJZ7ttACfBJpOsRST5TMk49nel/+w/L1GmMiZpo/LjbB/iz09fdDXhJVd+MQj0ipuWUjLu69fe7nGXqNMZEQsQDv6puBM6J9H6jzXtKxoLGJqpSWj/1lqnTGBMJduVuhHhPyVh24CBpzc0+j1umTmNMpNg4/ghoOSXjyNo6AKbn5bKrWzIFmX0tU6cxJmLsjD8C/E3JOLK2jood+3g85bssuXGJBX1jTMRY4I+QllMyqjMl44ixru4dy91jjIkU6+qJIPeUjNOWruPQ0QYe/dqZAFT8u4L/Xv4kdc37KMwosG4fY0xYWeCPMJ8x/R+/wt9TjlDeuyfHklypHNwXcwEW/I0xYWFdPRHmHtP/RPeZ5Nbv5qm8HE/Qd7OLuYwx4WSBPwq8x/Tv6pbsdxm7mCuxWO4mE0nW1RMFWrMd9zl+oIu5eiT3imylTNS4cze503hYd58JNzvjjzD3mH43fxdzoVDbtJeSBSVU/LsiwjU8fnb26qut52P6R9NDzt1kz23bOvIchbJOPD330hWGERYXF+uqVfEzQ+Mb86YzbMNPPd09lRk9mJ6XS5W728c7Z78CAjmpOYgINcdqKMgo4PJ+l/PejvfYVbuLAq+RQJUbK5n+0fRW5eHm3m9VbVWrx9KS0yi/pDxoPbzrnZ2a7dPWttoWrjZ3xnZbns1D6+djyOwhKK0/h4Iw9fKpPs9LXWMdDc0NAbeV6EJ5vkNZx60wo9BzRX17txtqfdt63x8PEflQVYtblVvgjw5d8zKybArUbPOUlRT19dvtczzcb06gw4HVm7836sFjB9usR2FGIUtuXOL3sWAfPHcbrjvlOhZtWBRwGX9aHiwDfZDa2yZ3MAjlQObvQOjehvv5KFlQ4ne5nNQcjjUda7PN7vp0NIAczwH1eA+O7V2/rboGe769n6OCjAKuKLqCd7e/G3Adt7TkNNK6pfl9TwR77tt6HUJ53x/vgcUCf4zS8lzEOdsbMrA/GoYZulIlk6Skxg4HVncADSXAByZ8PH6NT0lbH9Zw8Q7cbX34AvF3QHV/uKH12aE/Ps+r883Oo+X9TuDvQOivroHeDy0DUUfOrr2Fun5b3yY7clIQTe35PCVJEqra4W8AFvhjkKpSM/V0Tw6fcJzxOzvy7T7yuwydHmi8iQoI7fqWEAlJkkSzNre9YCR4fxYjMEVnWnIa2pzCMT0c8jre31QuevFKapv2tlomI7k3/xz3VsBujPZ8q4LQDqKJoCPfACzwxyjv/v7KjB6U9+7J0aQ4+809lAOPASBJleZIPlftPuALUy/7GU/96ymqjlT5XzfMJxGJLFiXqT+BAr8N54yyEWPL0DWF6LIpjKjZRrPCb3oG+KHXW6wHU+eEIgkiG8hiWQivWVvfPbo1N5OpysGkpM55/du7CVUmvzfZtV6gdWPl5e7IZ8R9IhxsvSh+9qo66fqeODu17JpkyBjkwU/49aUfUH/CD/jL4STWbN7G49X7KGxoRFTJaWwit6kJUaWwoZGbDh2msKHR9SZs+a3NX1kkOPstbGjk8ep9fLx5m5+xKm1vw93WgG0I1ubObncnbTetuZnclsN2/Qj4gXSe1/+7dz/vbd1BYWPTcdWnw4IF/BgS0vPd4jVNa27m8ep9ns9doNc8LEGzxWc8KcC+c7RHpyR0tK6eGKOqiEirIZ+tl/vqpMM9HHRXt2QKGpsoO3DQNTzU3+8FquQ0NSNC4LPGts56nMfd26lJSvLs1z3XgFt7frdIa26mfO9+zzb8dX25lwFatXlkbZ3Pc5Ht3c5g7cHVxaLgWadlm3yG3LbnbE+VQmc7QNCuvLTmZq47fIRFWZl+2+z93PrtFmzxuoTS7pZ1Dfh+aPM3ohDOlDuy7/buq63n21m3sLGJK+rqeLdHj1bvIbdA7z9/r5G/z0So779QX9+05mZ+UieU3h/6FOXWx98FfTXkczuangeAfHmAQ9378FnmJVzY+AHUbKcuKQtESG86xJfO7b+lNTAlQNBsb2ANFAyDaXY+x5UZPXgs2O8WXh9Ef9v1d1Bra9/+BLtWwt8HL9h2AgWTULbr78Dk/byOqK3jjSBtbuuAH+z5y27jgJ/T2MSxJGn1fjgq0mZQLmxoBOjQ4ISAwbSlNoJ6W893R94/gdZv73bbet39ret/H19C+cGQ62+BP864vxm4b4MrAZz79q//up6DO6azovkddiVBn6Zm7j14jBsO7/McHNKbaqjs0YPf9PzqzfV/9h+ktK6OuqRszzIoeOeRcwd10vOoq292HXCSs+iR2g358gCa04+FeXextV8pD3wymjca94X0pndvV1vsr1XbaXuZYKIRDNxta6vKNWSRosfoIfWeMs/znVPE+90u5Iwjy8k6tqvDz0HAs8m9+1HnN6aQvj22WBdJ4rFeuSGfCQf9VgXtOoh29Pn21ux1UI3V3izN6Y88aGf8JohQDg4n7qhg9IGZSM0O34A97NQ2l3nwmtM8+/Del/f+Fs6exvBNU30CmTfvoLYw7y5ePnYxY7qvYPSBmeDMU+z9IayX7jyTXUa/vPSv6pSe2+oAxJcHvL4JtT54eTumyTR2y6RH0yG/3558bwffVsC25d7F5v11fO/Q9IDPhW/bZiE12wM+39OWrnNel1kBv/UFq2vLoHnv/iNsSr6bgT0zGH3Q97kP9i2n0FlX8x90LV89jWd6ZgYNxi0PWP7K/QX1EbUhnpCE+nzjvO6Nh3zf+31Ww7LHXN+0Q6hrS+6IGkpdv1rmq9fN/T5MaaojVRo969RpKosHTWb0+Ad9PmvBWOA3fvkL2C3fVKEsE0igANWj6RCEcBDRNS/DsilIzXbIKUKvfhQZMiZgnfwd7LwPXtTsaFGHIhbm3snWoq/qAL4HyEAHS+9ttfzg+mvbV+s6Qd05YPVoOoSE0LZgr11IB3Y/B0jXN7QiFubd6Tng+z73rgBYEeCbofuAvbVfKUCbr7X3gd37RKLlAb9lYHUHvRtufwARCemEpK3nu/mqH5N0zk0Bn2fv967rIOx6nr6qa+vtklMEXq+j39fBzwlWy/fbtKXrOHF7BaMP+u7b3b5QWeA3URMoQLV8rKvUoa2AG2y7x3MQ7YhQDpBt1aN1APR/wG65nUDPR1t1+mp/7kDpP+iF+6Sls56/jtajM94rFviNMR0WKwcs0z6BAr+N4zfGtKll0A13EI70/hKNBX5jjEkwUQn8IvItEflCRDaIyMPRqIMxxiSqiAd+EUkG/h8wHDgTuEVEzox0PYwxJlFF44z/QmCDqm5U1XrgD8B1UaiHMcYkpGhk5+wHbPO6vx34Xy0XEpG7gbudu0dE5It27KM30DpRePxLxHYnYpshMdudiG2G42v3AH+F0Qj8/n6ebzWmVFWfA57r0A5EVvkbwhTvErHdidhmSMx2J2KbITztjkZXz3agv9f9ImBnFOphjDEJKRqB/wPgVBEZJCKpwM3Aa1GohzHGJKSId/WoaqOITAT+AiQDs1T1007eTYe6iOJAIrY7EdsMidnuRGwzhKHdXSJlgzHGmM5jV+4aY0yCscBvjDEJJu4CfyKkgxCR/iLyloh8JiKfikiZU95TRJaKyHrnf16069rZRCRZRP4lIhXO/URoc66ILBCRz53X/OJ4b7eIPOi8tz8RkfkikhaPbRaRWSKyR0Q+8SoL2E4RmezEti9E5Jsd3W9cBf4ESgfRCHxfVc8ALgLud9r5MLBMVU8Fljn3400Z8JnX/URo83TgTVUdDJyDq/1x224R6QdMAopV9Wxcg0BuJj7b/ALwrRZlftvpfMZvBs5y1vmdE/PaLa4CPwmSDkJVq1T1I+f2YVyBoB+uts52FpsNXB+VCoaJiBQBI4EZXsXx3uZs4ApgJoCq1qvqQeK83bhGHKaLSDegB65rfeKuzar6LrC/RXGgdl4H/EFVj6nqJmADrpjXbvEW+P2lg+gXpbpEhIgMBM4DVgJ9VLUKXAcH4IQoVi0cfg38EGj2Kov3Np8EVAPPO11cM0Qkgzhut6ruAJ4AtgJVQI2qLiGO29xCoHZ2WnyLt8AfUjqIeCEimcArwAOqeija9QknESkF9qjqh9GuS4R1A84HnlbV84Ba4qOLIyCnT/s6YBDQF8gQkXHRrVVM6LT4Fm+BP2HSQYhICq6gP09VFzrFu0Wk0Hm8ENgTrfqFwaXAKBHZjKsL7yoReZH4bjO43tPbVXWlc38BrgNBPLd7GLBJVatVtQFYCFxCfLfZW6B2dlp8i7fAnxDpIMQ1D91M4DNV/ZXXQ68B453b44FFka5buKjqZFUtUtWBuF7Xv6nqOOK4zQCqugvYJiKnO0VXA2uJ73ZvBS4SkR7Oe/1qXL9jxXObvQVq52vAzSLSXUQGAacC73doD6oaV3/ACGAd8G/gkWjXJ0xtvAzXV7w1wGrnbwTQC9cogPXO/57RrmuY2j8UqHBux32bgXOBVc7r/SqQF+/tBh4DPgc+AeYC3eOxzcB8XL9jNOA6o78rWDuBR5zY9gUwvKP7tZQNxhiTYOKtq8cYY0wbLPAbY0yCscBvjDEJxgK/McYkGAv8xhiTYCzwm4QmIk0istrrr9OuihWRgd5ZF42JFRGfetGYGPOlqp4b7UoYE0l2xm+MHyKyWUR+LiLvO3+nOOUDRGSZiKxx/p/olPcRkT+LyP84f5c4m0oWkd87ueWXiEi6s/wkEVnrbOcPUWqmSVAW+E2iS2/R1XOT12OHVPVC4Le4MoPi3J6jqkOAecBTTvlTwDuqeg6uXDqfOuWnAv9PVc8CDgLfdsofBs5ztnNveJpmjH925a5JaCJyRFUz/ZRvBq5S1Y1OQrxdqtpLRPYChara4JRXqWpvEakGilT1mNc2BgJL1TWhBiLyIyBFVf+viLwJHMGVguFVVT0S5qYa42Fn/MYEpgFuB1rGn2Net5v46ne1kbhmi/s68KEz4YgxEWGB35jAbvL6v8K5vRxXdlCAscDfndvLgO+BZ17g7EAbFZEkoL+qvoVrYplcoNW3DmPCxc4yTKJLF5HVXvffVFX3kM7uIrIS1wnSLU7ZJGCWiPwA18xYE5zyMuA5EbkL15n993BlXfQnGXhRRHJwTa4xTV3TKRoTEdbHb4wfTh9/sarujXZdjOls1tVjjDEJxs74jTEmwdgZvzHGJBgL/MYYk2As8BtjTIKxwG+MMQnGAr8xxiSY/w/T8N1QfV/Y2wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(np.arange(1,n_epochs+1), Train_Losses, marker='x', label=\"Training Loss\")\n",
    "plt.scatter(np.arange(1,n_epochs+1), Validation_Losses, label=\"Validation Loss\")\n",
    "plt.scatter(np.arange(1,n_epochs+1), Test_Losses, label=\"Test Loss\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"RMSE Loss\")\n",
    "plt.legend(fontsize=12)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82d2a042",
   "metadata": {},
   "source": [
    "And finally, we will check the performance of the model against the true data for different datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "0f8ae14a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f3983ff8cd0>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAA3e0lEQVR4nO3dd3xUVfr48c9JL5Q0MYSAARdDSwxFRGywIOAGEQsC6iqyuyjghmVXFCyQpViIXySIriu/VXddl6ICglEDKAiKqNSEFjokJKEEQkmBZHJ+f0wyJGEmhZnJlDzv14tXuM/cuffkQp45Offc5yitNUIIIdyTh6MbIIQQwn4kyQshhBuTJC+EEG5MkrwQQrgxSfJCCOHGvBzdgMrCwsJ0VFSUo5shhBAuZcuWLae11teZe82pknxUVBSbN292dDOEEMKlKKWOWnpNhmuEEMKNSZIXQgg3JkleCCHcmFONyZtTUlJCVlYWxcXFjm6KqCc/Pz8iIyPx9vZ2dFOEaLScPslnZWXRtGlToqKiUEo5ujmijrTW5OXlkZWVRdu2bR3dHCEaLbsP1yilBimlMpRSB5RSk+v7/uLiYkJDQyXBuxilFKGhofIbmBAOZtckr5TyBN4B7gU6ASOVUp2u4Ti2bppoAPLvJoTj2bsn3xM4oLU+pLW+DCwC7rfzOYUQwmWUlpby97//nd27d9vl+PZO8q2AzErbWeUxE6XUGKXUZqXU5lOnTtm5OfWXl5dHXFwccXFxhIeH06pVK9P25cuXa33/unXr2LhxY53OFRUVxenTp2vc59VXX63TsYQQzu+DDz7A29ubxMRE3nvvPbucw943Xs39vl5llRKt9fvA+wA9evRwuhVMQkND2b59OwCJiYk0adKE5557rs7vX7duHU2aNKF37942ac+rr77Kiy++aJNjCSEc48iRI1UmJPTv35+5c+fa5Vz27slnAa0rbUcC2fY84fJtx7n99e9oOzmF21//juXbjtv8HFu2bOHuu++me/fuDBw4kJycHADmzZtHp06diI2NZcSIERw5coT33nuPt956i7i4ODZs2FDlOHl5eQwYMICuXbvy9NNPU3mVrqFDh9K9e3c6d+7M+++/D8DkyZMpKioiLi6Oxx57zOJ+QgjnVFZWxj333FMlwR8+fJjVq1fj4WGfdKzsufyfUsoL2Af0A44DvwKPaq13mdu/R48eunrtmj179tCxY8c6nW/5tuNMWZpOUYnBFPP39uS1B2MY2rVVDe+sm8TERAIDA1m2bBlffPEF1113HYsXLyY1NZUPPviAiIgIDh8+jK+vL/n5+QQFBdXY+09ISCAsLIypU6eSkpLC4MGDOXXqFGFhYZw5c4aQkBCKioq45ZZb+P777wkNDaVJkyZcvHjRdAxL+zmL+vz7CeHOFi5cyKOPPmraXrBgAX/84x+J+bAzVJ6koDXpT5lNkRYppbZorXuYe82uwzVa61Kl1LNAKuAJfGApwdtCUmpGlQQPUFRiICk1wyZJHuDSpUvs3LmTe+65BwCDwUDLli0BiI2N5bHHHmPo0KEMHTq01mOtX7+epUuXAhAfH09wcLDptXnz5rFs2TIAMjMz2b9/v9nkXdf9hBCOcfz4cSIjI03bvXr1YsOGDXh5eV1J8NVmosV82Lneid4Suz8MpbX+CvjK3ucByM4vqlf8Wmit6dy5Mz/99NNVr6WkpLB+/XpWrFjBjBkz2LWr9n8kc9MM161bx5o1a/jpp58ICAigT58+Zueb13U/IUQdpS2Bb6fDuSxoHgn9pkLsI9d0KK01Dz74IMuXLzfFMjIyuOmmm67sZCbBX7VtJbeqXRMR5F+v+LXw9fXl1KlTpiRfUlLCrl27KCsrIzMzk759+zJ79mzy8/O5ePEiTZs25cKFC2aPddddd/HJJ58A8PXXX3P27FkAzp07R3BwMAEBAezdu5dNmzaZ3uPt7U1JSUmt+wkh6iltCTO/n8zNwRATFcnNwTDz+8nGxF9Py5cvx8PDw5Tgk5OT0VpXTfANxK2S/KSB0fh7e1aJ+Xt7MmlgtM3O4eHhwWeffcYLL7zAzTffTFxcHBs3bsRgMPD4448TExND165dmThxIkFBQdx3330sW7bM7I3XadOmsX79erp168aqVato06YNAIMGDaK0tJTY2FheeeUVevXqZXrPmDFjTMNCNe0nhKifmT+8wuKmAZSV967LlGJx0wBm/vBKnY9x8uRJlFI88MADAHTp0oVLly6RkJBgr2bXyq43XuvL2huvYLz5mpSaQXZ+ERFB/kwaGG2z8XhRf3LjVbiKmz/qYkzw1XhozY5RO2t8r9aaJ554gv/+97+mWHp6Ol26dKnxfTEfdTE/PKM16bWcszKH3Xh1hKFdW0lSF0LUW1k94xVSU1MZNGiQafu1115j8uR6l+myG7dL8kII0RDOnDlTZSbbDTfcwN69e/Hz83Ngq67mVmPyQgjREMaOHVslwW/evJkjR444XYIHSfJCCFFn69atQyllqjPzyiuvoLWme/fuDm6ZZTJcI4QQtTh//jzh4eEUFRmfuQkNDeXo0aMEBgZadVxvrSkxc+PV24YTYqQnL4QQNchdnEvz5s1NCf7HH3/k9OnTVid4gBltH4TqCV1rY9xGJMnXwppSw5s3b67T/FhbVaisrk+fPlSfklrd3LlzKSwstMv5hXBlhQcK2TlqJ6e/yQNg4sSJaK1t+vMa32cGr0c9QEuDRmlNS4Pm9agHiO8zw2bnkOGaWtRWari0tBQvL/OXsUePHvToYXbqahV1rTdvD3PnzuXxxx8nICDAYW0QbsyGZQIaStmlMjKey8BwwVgHS3kpzp4+S/Pmze1yvvg+M2ya1Ktzv5582hJ4qwskBhm/XsMjybUZNWoUf/3rX+nbty8vvPACv/zyC71796Zr16707t2bjIwMwHiTZvDgwYDxA2L06NH06dOHdu3aMW/ePNPxmjRpYtq/T58+PPzww3To0IHHHnvMVH74q6++okOHDtxxxx0kJCSYjltZUVERI0aMIDY2luHDh5t+vQTjbIAePXrQuXNnpk2bBhiLm2VnZ9O3b1/69u1rcT8hrknaEliZAOcyAW38ujLBLj+TtnJi2Ql2P73blOCjXoii84JOdkvwgN1zlnv15Cv+U5WUJ7eK/1Rg897Dvn37WLNmDZ6enpw/f57169fj5eXFmjVrePHFF/n888+ves/evXtZu3YtFy5cIDo6mrFjx+Lt7V1ln23btrFr1y4iIiK4/fbb+fHHH+nRowdPP/0069evp23btowcOdJsm/7xj38QEBBAWloaaWlpdOvWzfTarFmzCAkJwWAw0K9fP9LS0khISGDOnDmsXbuWsLAwi/vFxsba8MqJRuPb6Vd+FiuUFBnjTtab37ZtGzsrVX0M7hNMq1HlD1XasypAA+Qs9+rJ1/SfysaGDRuGp6exTs65c+cYNmwYXbp0YeLEiRarT8bHx+Pr60tYWBgtWrTgxIkTV+3Ts2dPIiMj8fDwIC4ujiNHjrB3717atWtnWmjAUpJfv349jz/+OGAse1w5OS9ZsoRu3brRtWtXdu3aZXE9ybruJ0StzmXWL+4AxcXFtGvXrkqHqMP8DlcSvL01QM5yryR/Lqt+cStUvrP+yiuv0LdvX3bu3MnKlSstlvv19fU1/d3T05PS0tI67VOf+kLmShcfPnyYN998k2+//Za0tDTi4+PNtrGu+wlRJ8qzfvEGNnv2bPz9/Tl8+DAAN0xsQ5ePuuDVpAEHOBogZ7lXkm8eWb+4jZw7d45WrYyf/B999JHNj9+hQwcOHTrEkSNHAFi8eLHZ/SqXLt65cydpaWmAcY5vYGAgzZs358SJE3z99dem91QuhVzTfkLUmzbUL95Adu3ahVKKF154AYBHH32UsrIymsY2bfjGNEDOcq8x+X5Tq45vAXj7G+N29Pzzz/Pkk08yZ84cfvvb39r8+P7+/rz77rsMGjSIsLAwevbsaXa/sWPH8tRTTxEbG0tcXJxpv5tvvpmuXbvSuXNn2rVrx+233256z5gxY7j33ntp2bIla9eutbifEPXm4QNlZqYZe/g0fFswrv3QvXt30tPTTbHc3Fyuv/56h7QHaJCc5Xalhl1xylZdXLx4kSZNmqC1Zvz48bRv356JEyc6ulm1klLDjVhiDTNSEs81XDuAd955h2effda0/fnnn/Pgg1UfOOr+YWcum1lM26esjC02WorPLBvkrEZVapjYR9wiqVe3YMEC/v3vf3P58mW6du3K008/7egmCeH09u/fX2U1piFDhrB8+XKz966mnz7Di9eFVqkp76E100+fsW8j7Zyz3C/Ju6mJEye6RM9dCGdgMBi48847q6zFnJmZWWVB7eriCy/BqTySg4PI9fIkvNTAhLP5xrgLc68br0KIRu/DDz/Ey8vLlOD/+9//orWuMcED0H0U8QWFrMrKJu1IJquysokvKITuo+zfaDuSnrwQwm5mhgTxabOmlGHsUQ47f4GXz+Tb5VxHjx4lKirKtP3b3/6W1atX42FmnN2swXOMX7d8ZJwBpDyNCb4i7qIkyQsh7GJmSBCLmzU1rWFaBsZt4GUbnqesrIxBgwaxevVqU+zQoUOmhwfrZfAcl0/q1clwjRDCLpZUSvAmShnjNrJo0SI8PT1NCX7BggVora8twbspSfK16NOnD6mpqVVic+fOZdy4cTW+p2Iq6O9+9zvy8/Ov2icxMZE333yzxnMvX768SlmBqVOnsmbNmnq0vm6kJLGwB0uTs20xaTs7OxullKnER8+ePSkpKeGPf/yjDY7uXiTJ12LkyJEsWrSoSmzRokUW68dU99VXXxEUFHRN566e5KdPn07//v2v6VjWkiQvnIHWmgcffND0hDkYC//9/PPPFkt+N3Zul+RTDqUw4LMBxP47lgGfDSDlUIpVx3v44Yf58ssvuXTJOI3qyJEjZGdnc8cdd9SpLG9UVBSnT58GjBUeo6Oj6d+/v6kcMRh/xbzlllu4+eabeeihhygsLGTjxo2sWLGCSZMmERcXx8GDBxk1ahSfffYZAN9++y1du3YlJiaG0aNHm9oXFRXFtGnT6NatGzExMezdu/eqNklJYuGKvvjiCzw8PFi2bBkAycnJaK2Jjo52cMucm1sl+ZRDKSRuTCSnIAeNJqcgh8SNiVYl+tDQUHr27Mk333wDGHvxw4cPRynFrFmz2Lx5M2lpaXz//femWjHmbNmyhUWLFrFt2zaWLl3Kr7/+anrtwQcf5Ndff2XHjh107NiRf/3rX/Tu3ZshQ4aQlJTE9u3bufHGG037FxcXM2rUKBYvXkx6ejqlpaX84x//ML0eFhbG1q1bGTt2rNkhocoliV966SW2bNlies3c95SQkEBERARr165l7dq1FvcT7mP5tuPc/vp3tJ2cwu2vf8fybccd1pZTp06hlGLo0KEAdOrUiUuXLtVp1TXhZkk+eWsyxYaqVROLDcUkb0226riVh2wqD9XUpyzvhg0beOCBBwgICKBZs2YMGTLE9NrOnTu58847iYmJ4ZNPPrFYqrhCRkYGbdu2NT3J9+STT7J+/XrT6xWPa3fv3t1U1KwyKUksarJ823EmfbaD4/lFaOB4fhGTPtvR4Ilea80TTzxBixYtTLG0tDR27dqFj49j6t+4IrdK8rkFufWK19XQoUP59ttv2bp1K0VFRXTr1u2ayvKae5QajCtNzZ8/n/T0dKZNm1brcWqrN1RRrthSOWNLbZGSxALg7yt3UWKo+n+sxKD5+0o71m+pJjU1FQ8PDz7++GMAXn31VbTWxMTENFgb3IVbJfnwwPB6xeuqSZMm9OnTh9GjR5t68fUty3vXXXexbNkyioqKuHDhAitXrjS9duHCBVq2bElJSYmpVDBULQNcWYcOHThy5AgHDhwA4OOPP+buu++u8/cjJYlFTc4WltQrbtNznz2LUopBgwYB0KZNGwoLC5kyZYrdz+2u3CrJT+g2AT9PvyoxP08/JnSbYPWxR44cyY4dOxgxYgRQtXzv6NGjay3L261bN4YPH05cXBwPPfQQd955p+m1GTNmcOutt3LPPffQoUMHU3zEiBEkJSXRtWtXDh48eOV78vPjww8/ZNiwYcTExODh4cEzzzxT5+9l7NixXLx4kdjYWGbPnm22JHH176miJHHfvn3r/b0LURfjx48nJCTEtL1582aOHj2Kv7+/A1vl+tyu1HDKoRSStyaTW5BLeGA4E7pNIL5dvK2bKupISg27nqjJlicqHHm97j9LMR91ufphKACtSR+107T5/fff06dPH9P2yy+/zIwZM+p8HtHISg3Ht4uXpC6EC7hw4QLh4eGm5y9CQkI4evQoTZo0cXDL3ItbDdcIIVzD888/T7NmzUwJ/ocffiAvL08SvB1YleSVUsOUUruUUmVKqR7VXpuilDqglMpQSg205jzONKQk6k7+3UR1hQcL2fnULpKSkgD4y1/+gtZa7uvYkbXDNTuBB4F/Vg4qpToBI4DOQASwRil1k9b1X8HXz8+PvLw8QkNDLU5BFM5Ha01eXh5+fn617yzcXtmlMjImZWA4b0wBPj4+nDx5kubNa1giUNiEVUlea70HzM65vh9YpLW+BBxWSh0AegI/Vd+xNpGRkWRlZXHq1ClrmiocwM/Pr/aFGoTbO7n8JCeXnzRtR026gcOzjziuQY2MvW68tgI2VdrOKo9dRSk1BhgDxjmx1Xl7e0vZUCFcUNGxIg5OO2TaDr47mFZPtQIZxmtQtSZ5pdQawNzTRC9prb+w9DYzMbP/slrr94H3wTiFsrb2CCGc26VLl+jUqROHDl1J8B3md8CridtN5nMJtV51rfW11LbNAlpX2o4Esq/hOEIIF/Lmm28yadIk0/YNf7mBpnG2WyRE1J+9PlpXAP9TSs3BeOO1PfCLnc4lhHCw3bt307lzZ9P2iBEjSB+Qhqrr+qrCbqydQvmAUioLuA1IUUqlAmitdwFLgN3AN8D4a5lZI4RwbiUlJcTFxVVJ8Lm5uSxcuFBmwzkJq5K81nqZ1jpSa+2rtb5eaz2w0muztNY3aq2jtdZSwUoIN/Puu+/i4+PDjh07APjss8/QWnP99dc7uGWiMrkTIoSolwMHDtC+fXvT9uDBg1mxYoX03J2UJHkhRJ3oMgN33nknP/zwgymWmZkpz0I4ObkrIoSo1cX0NRxLut+U4D/++GO01pLgXYD05IUQFpWeP8nxf4w2bffp04c1a9bg6elZ63ubl2nOeV49hNO8TB6HaUjSkxdCXEXrMk4smVYlwUc8/f9Yu3ZtnRI8wJS8s3iVlVWJeZWVMSXvrE3bKmomPXkhRBUFezZwesUbpu2Qgc/SNG5QvY8T7xUCp/NIDg4i18uT8FIDE87mE+8VasvmilpIkhdCAJCTk0NERIRp2ye8PeG/fxPlUbee+1X6TSV+ZQLxWZUedvf2h4FTrWypqA8ZrhGikdNaM2zYsCoJPuKP/6Dlk29de4IHiH0E7psHzVsDyvj1vnnGuGgw0pMXohFbuXIlQ4YMMW2/9dZbzM1tX8M76in2EUnqDiZJXohG6NSpU7Ro0cK03aFDB3bs2IGPjw9za1jIW7geGa4RohHRWvPUU09VSfA7duxgz549+Pj4OLBlwl4kyQvRSKxevRoPDw8++ugjAGbNmoXWmtjYWMc2TNiVDNcIUZO0JfDtdDiXBc0jod9Ulxtjzs/PJzg42LQdGRnJvn378Pf3d2CrREORnrwQlqQtgZUJcC4T0MavKxOMcVsc+60ukBhk/GqLY5rx5z//uUqC/+WXX8jMzJQE34hIkhfCkm+nQ0lR1VhJkTFuDXt+eJTbsGEDSinmz58PwJQpU9Bac8stt9jsHMI1yHCNEJacy6pzPOVQCslbk8ktyCU8MJwJ3SYQ3y7e/Ptr+vCwcijowoULtGrVigsXLgAQFBREZmYmTZo0seq4wnVJT14IS/yD6xRPOZRC4sZEcgpy0GhyCnJI3JhIyiELUxHr8eFRH5MnT6ZZs2amBL9hwwbOnj0rCb6RkyQvhJWStyZTbCiuEis2FJO8Ndn8G+r44VFXv/zyC0op3njDWG8mISEBrTV33HHHNR1PuBcZrhHCkiIL1RKrxXMLcszuZiluK4WFhfzmN78hJ8d4Hi8vL06dOkVQUJBdzytci/TkhbCkjj3ucIP5+uiW4hSdISUwgAGREcRGtWZAZAQpgQFQdKbOTZsxYwaBgYGmBL969WpKSkokwYurSE9eCCtNyDtDYlgwxR5X+kx+ZWVMsFA3PSUwgFeuC6WkfE3UHG8vXrnOWH7Xwq1akx07dhAXF2fa/sMf/sCCBQtsur6qr5cHl0rLzMaF65EkL4QllnrW1eL1rZv+emiwKcFXKFGK10ODLSb5S5cuERMTw/79+02xU6dOERYWVudvp67MJfia4sK5yUezENbyDqxXPN/D/I+dpficOXPw8/MzJfiVK1eitbZLggfwtPBbgaW4cG7SkxfuwU7lB1ICA67uoRcUVt2n6Bgvh4VQWp6kc7y9eDksBE4fq3X4pSZ79uyhU6dOpu3hw4ezcOFCmw7NmGPQ5u8lWIoL5yZJXri+iidIKx4wqniCFKxK9CmBASSGhZjG2nO8vUgMCwGqjp2/FhJsSvAVSj08eC3E8vBLTUpKSrj11lvZtm2bKZaTk0N4ePg1HK3+gvy9yS8qMRsXrkeGa4Trs1P5geTgoCo3UwGKPTxIDg6qEjvnaf7HyFK8Ju+99x4+Pj6mBL9kyRK01g2W4AEs/aIgozWuSXrywvXZ6QnSHC/zS99Zilvj0slL7H9+P2MZC0B8fDwrVqzAw8I4vT3lF17di68pLpybJHnh+ppHlhf7MhO3ggdgbj6JLdOuLtMcfv0whfuujPMfO3aM1q1b2/As9dPcwnBNcxmucUkyXCNcX7+p4F2tdK63vzFuBUsTBm01kfDsj2fZNXqXKcG3+mMrtNYOTfAgwzXuRnrywvVV3Fx1kcU9LueVsO+5fabtgOgA2r7Q1mmSqAzXuBdJ8sI9xD7itEm9QllZGYMHD2bf11cSfPvZ7fFt4WvccJIpihFB/hzPLzIbF65HhmuEW0g5lMKAzwYQ++9YBnw2wHKZXwf59NNP8fT05OuvvwYg4skIunzU5UqCdyKTBkbj71315rK/tyeTBkY7qEXCGtKTFy6vop57RbnfinrugOWFOxpIbm4uLVu2NG13796d4rFFKCeuAzO0aysAklIzyM4vIiLIn0kDo01x4VokyQuXV1M9d0clea01me9m0vKpKwl+9+7ddOzYkZiPujikTfUxtGsrSepuwnm7E0LUUW5Bbr3i9nZ++3l2PbWL87+eB4y1Z7TWdOzY0SHtEY2bVT15pVQScB9wGTgIPKW1zi9/bQrwB8AAJGitU61rqhDmhQeGk2NmgY7wwIZ7ShSg9EIpe/+817TtE+7D+SPn8fV1vnF30XhY25NfDXTRWscC+4ApAEqpTsAIoDMwCHhXKWX7xwSFACZ0m4Cfp1+VmJ+nHxO6TWiQ82utOfvj2SoJ/sbpN3LTa+0lwQuHsyrJa61Xaa1Lyzc3ARWPGN4PLNJaX9JaHwYOAD2tOZcQlsS3iyfR0ISWJaUorWlZUkqioUmDjMcfOnSIo3OPcXzBcQBaPNiCLh91wb+NTDcUzsGWN15HA4vL/94KY9KvkFUeu4pSagwwBqBNmzY2bI5oNP49hPjDW6tVfMyGfw+BJ1fY5ZTFxcW88cYbvPbaa5SoEsKHhxN6TyjKy0meaBKiXK1JXim1BjA3uPmS1vqL8n1eAkqBTyreZmZ/s096aK3fB94H6NGjh3M8DSJcy+Hv6xe30oW0C3SZ2YWDBw8yfPhwtt+6De8QH7ucSwhr1Zrktdb9a3pdKfUkMBjop7Xpkb0soHIBjkgg+1obKURt6rK4h7Uun75Mzv9yuLD1AtHR0axevZr+/fu7xJRI0XhZO7tmEPACcLfWuvJP1Argf0qpOUAE0B74xZpzCWFJXRf3uFZlpWXkfZPHyRUnAbj+oRbs+GSH3FQVLsHaMfn5gC+wunxJsk1a62e01ruUUkuA3RiHccZrrQ1WnksIs2pa3MPaJH9x90Wy/5PN5dzLNOvejPCR4fiEeluV4HsVFbHJ379qWUet6VV0db0YIaxlVZLXWv+mhtdmAbOsOb4QdWGPxT2OHz/OsXczOf/reXxa+HDDX2+gaWxT44tWFhJbcOI0f7o+zJjoy/UqKmLBidNWHVcIc6SsgRCVlJSUMG/ePBITEykoLqDF0BaE/S4MDx/bPhwuCV00FEnyQpRbv34948ePZ+fOnfzud79j/9378b1ext2Fa5PaNaLRy83N5fe//z133303Fy5cYPny5Xz55Zf4trDTtEjvwPrFhbCCJHnRaJWWlvL2228THR3N4sWLefHFF9m9ezf3338/yp7LNHlZ+O3AUlwIK8hwjWiUfvrpJ8aNG8f27dvp378/8+fPJzq6gRbFKDpbv7gQVpCevGhUTp8+zR/+8Ad69+7NyZMnWbx4MatWrWq4BA/gH0xKYAADIiOIjWrNgMgIUgIDwD+44dogGg1J8qJRKCsr45///Cc33XQT//nPf3juuefYu3cvjzzyiH2HZsxI8TM+rJXj7YVWyvTwVoqf/GItbE+SvHB7W7ZsoVevXjzzzDPExMSwfft2kpKSaNq0qU2OH1RWVq94cqCX+Ye3AiXJC9uTJC/clqHAwLhx47jllls4duwYH3/8MevWraNz5842Pc/AiwVXPyCltTFuRq6Fh7QsxYWwhnQdhNvRZZr8H/PJXZxLRuE+/vznPzN9+nSaN29ul/OtDwioWqIAQClj3IxwnyBySs6ZjQtha9KTF26l6FgRh187zPF/Hcfneh+2bNlCcnLyNSV4/zLz5Quqx+vbM5/Qawp+yrtKzE95M6HXlHq3UYjaSE9euAVDoYGTy06S920enoGetBrdiqDbmxMXF3fNx/RFY65kmG+1pRGal5WR73l1Qm9uYUy+YsWq5K3J5BbkEh4YzoRuExpkJSvR+EiSFy5Na03+T/nkLj5B6flSQvqE0OKhFng18bK6kNg5D/O/6FaPWzpNTaePbxfv1El9+bbjJKVmkJ1fRESQP5MGRjO0q9nF3YSTkyQvXNbu3bsZP348WeuO49/WnzYT2hDQzvw4+LUIN2hyzCznF26omr3Pe5r/MLAUd3bLtx1nytJ0ikqM1cGP5xcxZWk6gCR6F+Sa/wtFo3bx4kWef/55br75Znbs2EHEEy1p90o7myZ4gAlnzuJXbcjFr6yMCWeqPplaPenXFnd2SakZpgRfoajEQFJqhoNaJKwhSV64DK01n332GR07diQpKYknnniCjIwMQvqGoDxs/0BT/GVIPH2GliWlKK1pWVJK4ukzxF+uul9dPwxcRXa++cVLLMWFc5MkL1zCvn37GDRoEMOGDSM0NJQff/yRf/3rX1x33XX2O2mJhTViq8XjvULMfxh4hdivbXYUEeRfr7hwbpLkhVMrLCzk5ZdfJiYmhk2bNpGcnMzmzZvp3bu33c+dEuhvvvxAYLVk128qeFYrS+zpY4y7oEkDo/H3rjpbyN/bk0kDG7C+j7AZufEq6qyhZ1ysWLGChIQEjh49yuOPP05SUhLh4eF2O191dV07NqVJIIlhoRTrEqBiIfFQaBJok4XEG1rFv6nMrnEPkuRFnTTkjItDhw4xYcIEvvzySzp16sS6deu4++67bXqOuqjrQ07JW5NNCb5CsS4heWuyU0+TrMnQrq0kqbsJGa4RddIQMy6Ki4uZPn06nTt3Zu3atSQlJbF9+3aHJHiA8FJDneK5BTlm97MUF6IhSZIXdWLvGRfffPMNXbp0Ydq0aQwZMoS9e/fy3HPP4e3tXfub7WRCQan5WTMFpVVi4aXmn2y1FBeiIUmSF3XS3N98srUUr6tjx47x0EMPce+99+Lp6cmqVatYvHgxkZGRVh3XFuKLS83PmimumuQnnDljYQrlmYZsrhBmyZi8qJMLxSX1itfm8uXLzJkzhxkzZqC1ZtasWfztb3/D19eJ1jktOks8mviC6lMpq/72UvF6cnAQuV6ehJcamHA238z7hGh4kuRFnVh6ePNaHur87rvvGD9+PHv37mXo0KG89dZbREVFWdU+u/APhiIzvfHqy/T5hxBfcObqpO7vmvPkhXuR4RrRYLKzsxk5ciT9+vXj8uXLfPnllyxbtsw5E3x93PsGeFQbtvLwNsaFcDBJ8sLuSkpKmDNnDtHR0Sxbtoxp06axc+dO4uNtM72wvsvv1VmRhbIE1eOxj8DQd6F5a0AZvw591xgXwsFkuEbY1YYNGxg3bhw7d+7k3nvv5e233+bGG2+06TmupdRvnTSPhHOZ5uPVxT4iSV04JenJC7s4ceIETz75JHfddRfnz59n2bJlpKSk2DzBA5w3s2BHTfE66zcVvKuVMPD2d9lyBaJxkiQvbMpgMDB//nyio6NZuHAhU6ZMYffu3QwdOhRVfR1UGwm3sEyfpXidxT4C982rOgxz3zzpsQuXIsM1wmY2bdrEuHHj2LZtG/369WP+/Pl06NDB7uedkHeGxLDgKnVm/MrKmJBng1K/MgwjXJz05IXVTp8+zZ/+9Cduu+02Tpw4waJFi1i9enWDJHhwv1K/QtiS9OTFNdO6jIs7VhEd/QTnzp3jb3/7G9OmTaNp06YN25B+U4lfmUB8VvaVmLc/DLR+7DzlUIosuC1cmiR5cU0u5R7gzKp3uZyzj7vuuot33nmHLl26OKYxFcMp306Hc1nG2S/9plo9zJJyKIXEjYkUG4oByCnIIXFjIoAkeuEyJMmLejEUXyR//cdc3PYVHgHNCY3/K+tWvmm3m6p1Zoex8+StyaYEX6HYUOzSJYRF42NVkldKzQDuB8qAk8AorXV2+WtTgD8ABiBBa51qZVuFA2ldRsHO7zi77kPKii7QtPtggu54DA+/Jo5P8HaSW5Bbr7gQzsjannyS1voVAKVUAjAVeEYp1QkYAXQGIoA1SqmbtNbmC3QLp5aWlsaJTyZz6fhufCKiCX1kOj7X236+uzXsMXYe7t2MnJJzZuNCuAqrZtdorc9X2gwEKiYm3w8s0lpf0lofBg4APa05l1v48q/w9xBIbG78+uVfHd2iGp0/f56JEyfSrVs3Ss5kETIogfDHk5wywSduTCSnIAeNNo2dpxxKseq4E06dMF9C+NQJq44rREOyekxeKTULeAI4B/QtD7cCNlXaLas8Zu79Y4AxAG3atLG2Oc7ry7/C5n9d2daGK9uD5zimTRZorVm4cCF/+9vfOHHiBGPGjCHFtw+e/g08a6aO7DV2Hn/2JFwOkBLCwqXV2pNXSq1RSu008+d+AK31S1rr1sAnwLMVbzNzKLOPH2qt39da99Ba97juuuuu9ftwfls+ql/cQXbv3k2/fv147LHHaNWqFT///DPvvfee0yZ4sO/ye/EFhazKyibtSCarsrIlwQuXU2tPXmvdv47H+h+QAkzD2HNvXem1SCDb3JucStoSm0/DM7F0O8JJblNcvHiRGTNmMGfOHJo0acK7777LmDFj8LS2/ksDCDdocjyv7leEX0ux+8r8QyzUk5eHrITrsGpMXinVvtLmEGBv+d9XACOUUr5KqbZAe+AXa85ld2lLYGVCedVBbfy6MsEYd2Naaz7//HM6duzI7Nmz+f3vf8++ffsYO3asSyR4MJY1MDt2nmfl8nv3vgGePlVjnj5SJ164FGvLGrxePnSTBgwAJgBorXcBS4DdwDfAeKefWfPtdCiptih1SZExbiMpgQEMiIwgNqo1AyIjSAkMsNmxr8X+/fsZNGgQDz/8MCEhIfzwww988MEHuNqwmd3KGsQ+Ave/U7VA2f3vSC0b4VKsuvGqtX6ohtdmAbOsOX6DOpdVv3g9pQQGkBgWYiqilePtRWKYMQk19GM1hYWFvPbaa8yePRtfX1/mzp3L+PHj8fJy0Wfj7FjWQAqUCVcnBcoqmFsIoqZ4PSWHBFWpkghQ7OFBckiQTY5fVytXrqRz587MnDmTYcOGkZGRwYQJE1w3wYOUBBaiBi78k21j/aYax+ArD9nYcIGIHAvj25bitnb48GESEhL48ssv6dSpE2vXrqVPnz4Ncu4GIT1uIcySnnyF2Efg5kdBlSdd5WncdvHEUVxczIwZM0yJffbs2Wzfvt29ErwQwiK36Mm/vDydhT9nYtAaT6UYeWtrZg6Nqd9B0pbAjv9dmdKoDcbtNr1cNtGnpqby7LPPcuDAAYYNG8acOXOIjLTN8JMQwjW4fE/+5eXp/HfTMQzlqzYbtOa/m47x8vL0+h2oAWbXNJTMzEweeughBg0ahFKK1NRUlixZIgleiEbI5ZP8wp8zLcfTlsBbXSAxyPi1pjnvdp5d0xAuX77MG2+8QYcOHfj666+ZOXMm6enpDBgwwNFNE0I4iMsn+YoefHXxagMpayYxoKmB2KhIBjQ1kLJmkuVEb+fZNfb23XffERcXx+TJk+nfvz+7d+/mpZdewtfX1ybHv/1G83POLcWFEM7B5ZO8p4Va5rcEfU5icBNyvL3QShnnpQc3IWWDheGXflONs2kqs+HsGnvJzs5m5MiR9OvXj+LiYlauXMkXX3xBVFSUTc/zyZ9uo32LwCqx9i0C+eRPt9n0PEII23L5JD/y1tZm4x+HeJqfl+5r4cFbF5trXVpayltvvUWHDh1YtmwZU6dOZdeuXQwePNgu51u+7TgHTxZUiR08WcDybcftcj4hhG24/OyamUNjOHzqIj8evFKn5PYbQ0j3Mj//PNdCHHCZudY//PAD48aNIz09nUGDBvH222/zm9/8xq7nnLI0jbJqsbLy+NCuZqtICyGcgMv35JdvO87WY1VX79l67BzhpeZ77JbiruDEiRM8+eST3HnnneTn57N06VK++uoruyd4gKKS6im+5rgQwjm4fJJPSs2gqKRq4i4qMXBnYSFUvymrNXcV1lAP3ElXbjIYDLzzzjtER0ezcOFCJk+ezJ49e3jggQfcdn1VIYRtuPxwTXZ+kdn4hoAAqJ4AlWJ9gIXKj066ctPPP//MuHHj2Lp1K7/97W9555136NChQ4O3Q2F+1Rf5iBHCubl8Tz4iyN9s3NLYu8UxeSdbuSkvL48xY8Zw2223kZuby6JFi1izZo1DEjyAn7f5/yqW4kII5+DyP6GTBkbjXW1VIG9PVf8xeSdZuamsrIwFCxZw00038cEHHzBx4kT27t3L8OHDHTo0U2xh7N1SXAjhHFw+yQNXjyNoSDibb361oLP5Ddas+tq6dSu9e/dmzJgxdO7cmW3btvF///d/NG3q+PVVm/t71ysuhHAOLp/kk1IzKCmrmuVLyjTxBYXmVwtywoWYz549y7PPPsstt9zC4cOH+c9//sP3339PTEw9i6zZUYnBfI/dUlwI4Rxc/sbrcQs3XtEQX1DolEm9gtaa/B/yiX4+mry8PMaNG8eMGTMICgpydNOuUnDZ/LCVpbgQwjm4fJKvycyQID5t1pQyjL+yDDt/gZfP5FvcPyUwgOTgIHK9PAkvNTDhbL7dPiSKM4vJ/jibwn2F3HrrrXzzzTd069bNLucSQjRebpvkZ4YGsaRZU9M0yjJgcTPj2PbLZvZPCQxgSlgIutIarFPssAarocjAyeUnyVudh2eAJxFPRbDx/23Ew8O5R86C/L3JLyoxGxdCOC/nzixWqJzgTZQyJfrq/h56JcFX0B4e/D3UNlUWtdbkb8pn/5T95K3KI/iuYNq/3p6Qu4KdPsEDJA7pjLdHtVlMHorEIZ0d1CIhRF24bU++voo8zE9PtBSvjz179nAk6SgFewrwu8GPNgltCGhX/lCWhVLJzqaiPk1SagbZ+UVEBPkzaWC01K0RwslJkrejgoICZsyYwZw5czB4G2j5REtC+oSgbPDB4QhDu7aSpC6Ei5Ekbwdaa5YuXcrEiRPJzMxk1KhRbOr6E17NZfxaCNGwnH8w2MXs37+fe++9l4cffpjg4GA2bNjAhx9+iFcz+TwVQjQ8SfI2UlRUxNSpU+nSpQsbN25k7ty5bNmyhTvuuMPRTRNCNGLSvbSBlStXkpCQwJEjR3j00Ud58803admypaObZXPLtx2XG69CuBjpyVvh8OHDDBkyhCFDhuDv7893333HJ5984rYJfsrSdI7nF6ExPmk8ZWm6LP8nhJOTJH8NLl26xMyZM+nUqRPfffcdb7zxBtu3b6dv376ObprdWFqcJSk1w0EtEkLUhQzX1FNqairPPvssBw4c4OGHH2bOnDm0bm1+MXF3YmlxFktxIYRzkJ58HWVmZvLwww8zaNAglFJ88803fPrpp3VO8JZmxrvKjHlLi7NYigshnIMk+VqUlZYxe/ZsOnbsSEpKCjNmzCA9PZ2BAwfW6ziWnmt1jeddjYuz+HtXXVXL39uTSQOjHdQiIURdyHBNDS7uuUjOxzm8kP0CQ4YMITk5maioqGs6lgfGImnm4q5AyhoI4ZokyZtRkl9C7qJczm06h3eYNytXrmTw4MFWHdPS0hqutOSGlDUQwvVIkq9EGzR5a/I4uewkulRz3ZDruC4+zOoED9Cy1ECO99WXu6WlNWeFEMIGbDJaoJR6TimllVJhlWJTlFIHlFIZSqn6DWA7QMH+Qg4mHiR3YS4B7QP4zazfcP2D1+PhY5sBlQmXPM2vOXvJ08I7hBDCelb35JVSrYF7gGOVYp2AEUBnIAJYo5S6SWvtdN3WkydP8vzzz3P434fxDvGm9bOtada9Gap6LXorxd8wAPYsvHrlqY4jbXoeIYSozBbDNW8BzwNfVIrdDyzSWl8CDiulDgA9gZ9scD6bMBgM/POf/+Sll17i4sWLxA5oTumDEXj4VepZa02vIhvNA9+/yvyas/tX2eb4QghhhlVjEUqpIcBxrfWOai+1AjIrbWeVx8wdY4xSarNSavOpU6esaU6d/fLLL9x6662MHz+ebt26kZaWxo7bNL3LLhkX8Sj/06uoiAUnTtvmpOey6hcXQggbqLUnr5RaA4Sbeekl4EVggLm3mYmZnRKutX4feB+gR48edp02XnqxlBOfnqDX+l6Eh4ezcOFChg8fXj40oywkdBsN23gHQEmB+bgQQthJrUlea93fXFwpFQO0BXaUj19HAluVUj0x9twrPwoaCWRb3dprpMs0Zzec5cSnJzAUGpj4l4kkJibSrFmzKzv1GA2b/3X1m3uMtk0jSi0M+1iKCyGEDVzzmLzWOh1oUbGtlDoC9NBan1ZKrQD+p5Sag/HGa3vgFyvbek2KjhaR/Z9sig4WEXBTABGPt2TOK3Ou3nFweWzLR6ANoDyh+6grcWtpCzPiLcWFEMIG7DJPXmu9Sym1BNgNlALjHTGzRmtN9r+zuXz6Mq3+1Iqg3kE1D74MnmO7pF6d8jR+eJiLCyGEndgsyWuto6ptzwJm2er410IpReSYSLyaeuEZ6FnRMMc0pvso88NB3Uc1dEuEEI2I2z7xGlRWRr6nJ77hvlfFHcLew0FCCGGG2yb550+fZVqLUEoqPdTkrTWT8/Id1yh7DgcJIYQZrlIEsd66Xgxgxqk8WpaUorSmZUkpM07lEe8V4uimCSFEg3Hbnvzs0keYd/lD4rMqzdz09oeBUx3XKCGEaGAu35P39zb/Laz2vAvumwfNWwPK+PW+eRD7SMM2UAghHMjle/KvPRjLXxdvr1KX3aM8TmwrSepCiEbN5ZO8rFgkhBCWuXySB1mxSAghLHH5MXkhhBCWSZIXQgg3JkleCCHcmCR5IYRwY5LkhRDCjbnF7Jrl247LFEohhDDD5ZP88m3HmbI0naISY6324/lFTFmaDiCJXgjR6Ln8cE1SaoYpwVcoKjGQlJrhoBYJIYTzcPkkn51vfo1US3EhhGhMXD7JRwT51ysuhBCNicsn+UkDo/H3rrpOqr+3J5MGRjuoRUII4Txc/sarFCgTQgjLXD7JgxQoE0IIS1x+uEYIIYRlkuSFEMKNSZIXQgg3JkleCCHcmCR5IYRwY0pr7eg2mCilTgFHG/i0YcDpBj6nq5FrVDu5RjWT61M7a67RDVrr68y94FRJ3hGUUpu11j0c3Q5nJteodnKNaibXp3b2ukYyXCOEEG5MkrwQQrgxSfLwvqMb4ALkGtVOrlHN5PrUzi7XqNGPyQshhDuTnrwQQrgxSfJCCOHGGn2SV0o9p5TSSqmwSrEpSqkDSqkMpdRAR7bPkZRSM5RSaUqp7UqpVUqpiEqvNfprpJRKUkrtLb9Gy5RSQZVea/TXB0ApNUwptUspVaaU6lHtNblGgFJqUPk1OKCUmmzzE2itG+0foDWQivEBrLDyWCdgB+ALtAUOAp6ObquDrk+zSn9PAN6Ta1Tl+gwAvMr//gbwhlyfq65RRyAaWAf0qBSXa2S8Dp7l33s7wKf8mnSy5Tkae0/+LeB5oPLd5/uBRVrrS1rrw8ABoKcjGudoWuvzlTYDuXKd5BoBWutVWuvS8s1NQGT53+X6lNNa79FaZ5h5Sa6RUU/ggNb6kNb6MrAI47WxmUab5JVSQ4DjWusd1V5qBWRW2s4qjzVKSqlZSqlM4DFganlYrtHVRgNfl/9drk/t5BoZ2f06uMXKUJYopdYA4WZeegl4EeOv21e9zUzMbeeZ1nSNtNZfaK1fAl5SSk0BngWm0YiuUW3Xp3yfl4BS4JOKt5nZ3y2vD9TtGpl7m5mY216jGtj9Orh1ktda9zcXV0rFYBwH3KGUAuOv2VuVUj0xfpK2rrR7JJBt56Y6jKVrZMb/gBSMSb7RXKParo9S6klgMNBPlw+y0oiuD9Tr/1Bljeoa1cDu16FRDtdordO11i201lFa6yiMF7qb1joXWAGMUEr5KqXaAu2BXxzYXIdRSrWvtDkE2Fv+d7lGGGdFAC8AQ7TWhZVekutTO7lGRr8C7ZVSbZVSPsAIjNfGZty6J38ttNa7lFJLgN0YfwUfr7U2OLhZjvK6UioaKMM4A+kZkGtUyXyMs0NWl/9GuElr/YxcnyuUUg8AbwPXASlKqe1a64FyjYy01qVKqWcxzvLzBD7QWu+y5TmkrIEQQrixRjlcI4QQjYUkeSGEcGOS5IUQwo1JkhdCCDcmSV4IIdyYJHkhhHBjkuSFEMKN/X/SiJuH8Q/ZKgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_orig = NNreg.XY_test[:,-1].cpu().detach().numpy()\n",
    "y_pred = NNreg.predict(NNreg.XY_test[:,:-1]).reshape(-1).cpu().detach().numpy()\n",
    "plt.scatter(y_orig, y_pred, label='Test data')\n",
    "y_orig_2 = NNreg.XY_train[:,-1].cpu().detach().numpy()\n",
    "y_pred_2 = NNreg.predict(NNreg.XY_train[:,:-1]).reshape(-1).cpu().detach().numpy()\n",
    "plt.scatter(y_orig_2, y_pred_2, label='Training data')\n",
    "y_orig_2 = NNreg.XY_valid[:,-1].cpu().detach().numpy()\n",
    "y_pred_2 = NNreg.predict(NNreg.XY_valid[:,:-1]).reshape(-1).cpu().detach().numpy()\n",
    "plt.scatter(y_orig_2, y_pred_2, label='Validation data')\n",
    "plt.plot(y_orig_2, y_orig_2, color='k')\n",
    "plt.legend()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}