From 6ce67fbdf6200a8f203a5893d67b2f4b3ffa84ad Mon Sep 17 00:00:00 2001 From: Steffen Schneider Date: Wed, 24 May 2023 03:15:37 +0200 Subject: [PATCH 1/2] Add notebook (incl solution) for week 11 --- week11/week11.ipynb | 3174 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 3174 insertions(+) create mode 100644 week11/week11.ipynb diff --git a/week11/week11.ipynb b/week11/week11.ipynb new file mode 100644 index 0000000..aa4630d --- /dev/null +++ b/week11/week11.ipynb @@ -0,0 +1,3174 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "source": [ + "# Continual learning: Overcoming catastophic forgetting with memory replay\n", + "\n", + "In this exercise class we'll implement an experiment to measure catastrophic forgetting in a neural network trained on MNIST. We will then fix/reduce the catastrophic forgetting by implementing a simple memory replay strategy." + ], + "metadata": { + "id": "lDTvzuBtNg1X" + } + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Mzy1CzlGQvAk" + }, + "source": [ + "The following figure highlights the setup of the dataset and is taken from [van den Ven & Tolias, 2019](https://arxiv.org/pdf/1904.07734.pdf):\n", + "\n", + "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABGsAAAFKCAIAAAAlg0X+AAAAIGNIUk0AAHomAACAhAAA+gAAAIDoAAB1MAAA6mAAADqYAAAXcJy6UTwAAAAGYktHRAD/AP8A/6C9p5MAAAAQY2FOdgAAB4AAAAdYAAAB+wAABExSeSo6AACAAElEQVR42uydd0AUV/f378xsY2Fh6U26FEERe+9iwW7UmEQTY2Jij9FETTW2mGhMfEwxMYmx1xh7A7soVkREkCK9L51l2b7vH+f33HcfQLI2pJzPX8MwM7t759xy7j33exiDwUAQBEEQBEEQBEEQE2CxCBAEQRAEQRAEQUwEPSgEQRAEQRAEQRBTQQ8KQRAEQRAEQRDEVNCDQhAEQRAEQRAEMRX0oBAEQRAEQRAEQUwFPSgEQRAEQRAEQRBTQQ8KQRAEQRAEQRDEVNCDQhAEQRAEQRAEMRX0oBAEQRAEQRAEQUwFPSgEQRAEQRAEQRBTQQ8KQRAEQRAEQRDEVNCDQhAEQRAEQRAEMRX0oBAEQRAEQRAEQUwFPSgEQRAEQRAEQRCTMTwJWFxIwxAQEGB4ctauXYtFhzQMW7ZseVL7lMlkAQEBWHRIAxAQECCTyZ7URP/8808sOqRhWLt27VP08tiEIg3Gv1oj70mfyDCMQCBgWVy8Ql4IOp1Oo9E8yxM4jhMIBFiSyAtCpVLp9fpneYJQKMQmFHlB6PV6lUr1LE9gWVYoFGJJIi8ItVqt0+me+naGYfh8PsdxWJLIi0Cv16vValMWjZ7YgxIKhevXr2/Xrh2WMvIiOHv27KpVq57lCR07dly/fj2WJPIi0Ov1M2fOfPjw4bM8ZM2aNZ07d8bCRF4Et2/fXrhw4bM8wc/P79dff0UnH3lBLFq06NatW099O8MwS5cuHTx4MJYk8iK4f//+okWLlErlv175xB4Uy7Lt27fv1asXljLyIsjKynrGJ0il0j59+mBJIi8CnU5nYWHxjA9p27YtmijygjCl468fCwuLPn36oAeFvCCkUukzPsHf3x+bUOQFwbKsia0fNpEIgiAIgiAIgiCmgh4UgiAIgiAIgiCIqaAHhSAIgiAIgiAIYiroQSEIgiAIgiAIgpgKelAIgiAIgiAIgiCmgh4UgiAIgiAIgiCIqaAHhSAIgiAIgiAIYiroQSEIgiAIgiAIgpgKelAIgiAIgiAIgiCmgh4UgiAIgiAIgiCIqfCa5a/S6XQGg+FJ72IYhuO4Bvh6Wq32RX+cRqNRq9UCgYDP56OVN0LABp4UlmVZ9oXPeuj1er1eTwjhOI5hmOdeN9VqtcFgEAgEPF7zbH+aAdQGnpSGeae0hX/uH2cwGFQqlU6nw8azkfN0TWjD9PIGg0Gn05EX32JDRWiYfgFpGBNt4F4ejedZaIYjGJVK9emnnyYkJDzpjV5eXhs3bnzRzWtKSsr8+fMJIT169Pjiiy9eUL398ssvL168+OGHH06aNAmtvLGRk5Pz0UcflZeXP+mNI0eOnD179ov+eocOHfrzzz8JIbNmzRo1atRzeaZOp4uNjT127Njdu3dLSkr0er1EIgkJCRk+fHjXrl2FQiFaRaPin3/+2bJly1Pc+OOPP/r4+LzQ76ZUKpcsWZKcnAy2+ryMp7y8PDw8/NSpU+np6SqVytLSsnPnzuPGjWvfvn3DzKwhpqPRaF599VWlUvmkN/bq1euzzz570V/v/Pnz69evJ4RMnjz5zTfffEGfkpGR8fHHH8vl8gkTJkyfPh2tolEhl8snT578FPNQw4cPnzdv3ov+ekePHt28eTMhZMaMGePGjXv2B1ZXVx88eLCsrKyea+zt7V999dXm9JaboQel0+mioqKioqKe9Mbg4GC9Xv+iO8vy8vJTp04RQl7cqPHIkSM//fSTXC7Py8trfu+3GSCXy8+dOyeTyZ70Ri8vrwb4eqmpqWCiw4cPfy4PLC4uXrZs2c6dO2s4jadOndqwYcOYMWO+/fZbd3d3NIzGA7WBJ+Up5gWeFK1We+3atdu3b8Pxc2lIY2Ji5syZc+PGDVg6AE6fPv3jjz9OnTr1q6++srW1RatoPOh0uoiICLlc/qQ3isXiBvh6OTk5UH06duz4gj5CrVYvWbLkwIEDhJB27dqhSTQ21Gr16dOnjdsTE3Fzc2uAr5eeng4mOnjw4OfywNzc3AULFhQXF9dzTYcOHdCDauwwDCMWiy0sLGqcNxgM1dXVer2eYRhzc/PaN4rF4uces9TwXLx48YMPPniKrgVpMFiWNTc3r66urnFep9MplUqDwcBxnJmZWe0bRSJRk/uxZWVlb7755unTp/V6vVAo7Nq1q5+fH8uyKSkpV69era6u3rdvX25u7v79+x0dHdE2GgkCgaB2E0oIUSqVEJoiEonqjKBriss1cXFxY8eOzcjIYBjGw8Ojd+/eNjY2mZmZ58+fLy8v/+WXX3Jzc7dt21ZngSAvBYZh6nwder1eoVCAHdbZhNZ5simyffv2gwcPoiU0WliWtbCwqO1B6XQ66PqbUy9PCMnIyCgtLW1pb7kZelAikWjfvn0ajabG+dTU1LFjx8pkMpFIFBUVZWdnV7MseLymvjHj1KlT06dPz8/Pb2l23LTw9va+ceNG7fX948ePz5w5U6fTtWvXrs4VgIaZQH2+/Oc//zlz5oxer3d3d//rr7969OgB3YZSqbxx48a0adPS09OvXLmyZs2a77//HgOyGwkzZsyYPHly7fNTp049e/YsIWT58uV1hifVblcbOSqVasGCBZmZmSzLTpw48fvvv3d2dmYYRqvV3rt374033khMTDxy5Mhff/3VAKE1iIkIhcKYmJjau51v3bo1evRoQoifn9/hw4ctLS1rXNBEh6c1ePTo0ZIlS55umw3SMFhZWT18+LD2+fDw8LfeeosQEhQUdPr06dqz9k3UyY+NjYUhzdy5c7t3717nNdbW1s3sLTfPNag6Iy4qKyvp+MzBwcHBwaE5/WqlUrl+/fqVK1eqVKrm906bGRzH1Wl+Uqn0/6olj+fk5NQMfmlBQcGuXbt0Oh2Px9u7d2+PHj3ov0QiUb9+/fbs2RMaGiqXy7du3frxxx+7urqieTQGzM3N61yopyFzlpaWzcNEL1y4cPXqVYPB0LZt282bN9MxN4/H69Sp06+//jp8+HClUvnbb7/NmDGjeYy/mwd1LlnTrp/H4zk6OlpZWTW/H65UKt95553S0lKWZZ9O7gVpABiGqbOFpF4E9PLNIO4JuHfvHvzqd999t3379i3kLeOMb5NHq9WePn26T58+X3zxBWyAbjnmizRy7t279+jRI0LI4MGD65yX6tKlC8Rhl5eX37x5E0sMaWCOHj2qVCoZhnn//fdrL1l06NAhICCAEJKQkFB/iD+CNAAGg+Hbb7+NjIw0MzObMmUKFgjSGFAoFLDg5uzs7OLi0nJ+OKoJ10Sv1+fm5kZHR9+/f7+wsFCtVkskEk9Pzx49erRp0+Zxc5AlJSWxsbE3btzIzc3V6/UODg4hISEdO3Z80jn11NRUiCXl8/lt2rQxRU73zp07Y8aMUavVDMP4+/uvXbs2OTkZ5gOQZolarU5NTb19+3ZiYmJpaanBYLC2tg4ICOjevbuXl1edG1EMBkNeXt7Nmzfv3r1bXFzM4/FatWrVtWvX4OBguvBlCjqdLj4+Xq1WE0IkEomvr2/9U2gZGRlWVlYKhaJr1651XslxXOvWreE4NzcXX27zoLS0ND4+Pjo6OjMzUy6XC4VCZ2fnjh07dujQ4XFhfiqVKikpKSoqKiUlpaqqSiKR+Pn59ejRw8fHRyAQPNFHp6amwrGHh8e/RhWOHj1aKpU+ePBg4MCBtf/L5/MhqEav11dWVuKbbR5oNJr09PTo6GhwjHU6nVQq9fX17datm6+v7+O63YKCgjt37ty5c6egoIDjOFdX106dOrVv3/6JIlf1ev3Dhw9hJ4yZmVlAQMAThS5funRpw4YNOp1u5syZnTt33r59O77NZolarU5LS7tz505CQgLt5f39/bt16+bj41NPL3/79u3o6GiZTCYQCFxdXbt06RIcHPxE4XNarTYxMRGELs3NzWEKqX7kcjlMlfr5+UkkkpbzmtCD+h8yMzPXrVv3zz//5OXl1YixFolEgwcPXrVqVY0VHp1Od/jw4a+//vrevXvGuwYZhnFzc3vnnXcWLVpUZzxMbc6dO/fWW2/l5OTweLzFixcvW7bMlLs0Go1Go7GxsXnjjTc++eQTZ2dn0FFFmiU3b95cs2bNuXPnaoznGIaxsbF59dVXv/zyyxrxLQqFYuPGjZs2bcrKyjK2ao7jgoODFy9ePGHCBFN2AOp0up9++mnp0qVKpdLW1vann37y9fWt/5YJEyZ069attLT0cfpCBoOhqKgIjlHTvBmgUqm2bt36yy+/xMfH19inwXGcv7//4sWLp0yZUmMEkJKSsmTJkjNnzlRVVRmbtIWFxfDhw5ctWxYYGGjKpxcXF7/22msREREMw/Tu3Xvbtm3/OrodNmzYsGHDHvffkpKSnJwcQoiFhQUqnTQP7t+/v3r16oiIiJKSkhr/srKyGjVq1Jo1a1q1amV8XqvVbtq0aePGjampqcaBcxzHBQQEzJ8/f9q0aSb6+du3b583b55cLpdIJOvWrTNleEopKChYvHhxWVlZSEjIV199dfz4cXybzQ+DwXDr1q1vvvnm/PnzFRUVNQaiUql00qRJq1atsre3Nz5fXV0NvXxmZmaNXr5t27YfffTR66+/boqvrtPpNm3atHTpUoVCYWlp+euvv5piotnZ2dCPBwYGtqhQZ/Sg/j+pqamvvvrqnTt3DAaDUCj08fGxsrLSaDQ5OTn5+flKpfLEiRNpaWknT540Vl4+ePDgjBkzKioqWJb18vKCwNa8vLyMjIzMzMzly5fn5OT88ssv/ypRdenSpbfffhvcp08++eTLL780UdbC0tJy1qxZ7777bkhISLOJqUXq5PTp09OnT8/Ly2MYxsrKytPT09zcXKFQpKenl5WVFRcXb9q0qaioaNu2bbQV02q1y5Yt+89//qPRaAQCQevWra2trbVabVpamkwmu3v37rvvvltVVfXOO+/U/9HQsH722WdKpdLa2nrz5s3jx4//1y9sbW1d/+xXSUnJtWvXCCEsy2L0aVNHq9UuXrz4l19+0Wq1LMu6ubk5OztzHFdcXJyWlqbRaOLj42fNmiUUCo1lKrKzs1955ZX79+8TQmxtbb28vEQiUWVlZWJiYmVl5f79++Pj448ePfqvUv5FRUVvvfXW2bNnGYbp27fv9u3bn1EiX6vV/vbbb9nZ2YSQESNG1I7xQ5ocELJBvWIvLy+JRFJdXZ2ZmVlcXFxeXr5z586MjIyzZ88ae0TffffdF198odVq+Xx+69at7e3t9Xp9RkZGbm7ugwcP5s+fX1FRsWjRon/tf3fv3g3uk6Wl5ffffz9t2jTTF6A0Gs3XX399584dS0vLn376qUXN9Lcozp8/P2XKFNADs7S09Pb2FovFCoUCxO7Kysp+//334uLi3bt3UxPV6/UrVqz47rvvtFqtQCDw8fGxtbWFhdbCwsJ79+69++67SqXy3Xffrf+jDQbD5s2bwX2ytrb+9ddfTUwoeu/ePXDbOnbsmJaWduLEiWvXrhUVFVlaWgYEBISFhXXv3r15ykQZngRCiFgsjoyMNDRBkpKSYBLRzMysoKCgxn81Gs3UqVOhBezbt+/NmzeLi4vlcnlFRUV2dvbmzZthIMgwzNq1a+ldpaWl4KBLpdKtW7cWFBTI5fKqqqqCgoLffvsNtrGam5tfvnyZ3gJpTAghY8eOpSejoqJg0ovP569YsUKtVpv+u3Q6HSQmp3z33XfwERs2bGhyr2nXrl0sywYEBDzFvWvXriWEhIaGNkX7NBgMBw4cAE+7c+fOtf9bUFAAeT8Yhpk9e3ZSUlJZWVlVVVVZWVlKSsrChQvB5RYIBFFRUfSu69evQ1/r6+t79erV4uLiqqqqysrKjIyMBQsWgMH7+/vDoqtxMRJCNm7cSG3s119/BflgS0vL48ePP6+f/J///Aca1vbt28N8WyNHq9V27tyZELJly5YnvVcmk0FzER4e3kRNdMSIEWAbmzZtqtOAYSHR0dFxx44deXl5lZWVcrlcJpNdvHiR6oh07txZq9XSuz788EOWZVmWnTFjxqNHjyoqKhQKRWlpaVRUVEhICBj8okWL9Ho9XF9ZWQmvgBAil8vhZHFx8ejRo1mWZRimf//+OTk5z/iW79y58+abb8LP8fDwiI+PbyrvKDw8nBASEBAgk8me9F5Ipd25c+cafUpT4erVq2AY7dq1Kysrq/FflUoFlsNx3Ouvvx4XF1dSUlJVVVVeXp6amrps2TKIFmFZ1riJS05Oht7f09Pz5MmTMpmsqqpKLpdnZWV9+eWXYCG2tra5ubn0lm3btsHX+Oyzz+jJvXv3wpBAIpFs27bNuAqYwuHDh83MzBiGWblyJdSF3bt3w6csXry4ab2m0NBQQojxUMp0IO5x165dTbQJPXr0KLy1jh070jaNUl5e3qZNGzDRGTNmJCYm0l4+OTl58eLF4DWJxeKLFy/Su2JjY2HONCAg4Pz588XFxQqForKyMj09feHChXCLq6urcY344Ycf4GusX7+envz9998haNnGxuaff/6p/fUex4IFC6DiTJgwwdXVtcZUgoWFxZQpU56iOXpZREZGgu7xv16JHtT/kZiYCK2nm5tbRkZG7ds3btwI1hAWFmZc0GAr8+fPr2Fter3+q6++YhiGYZglS5bQ87U9qJiYGGdnZxj7fv3110/kPtUJelBNtG2t34Patm0bOBuDBw/WaDQ1/qtWqydOnAjvfd26dfT8qlWr4OShQ4dq3KLVagcMGACN9cmTJ2sUo7EHtWXLFmhQpFLpqVOnntfvjY6OhqEJy7Jbt25tEu8IPajHeVBarXbQoEGEEB6PV+fbjI+PhxaY47iioiI4WVVVFRQUBB18dXV1jVvu3r0LzXJQUFB5eTmcrO1BlZeXjxs3DmpH37596cOfgpSUlH79+rm7u0N6QIZhevToAcLZTQX0oB7nQZ04cQL2OHXv3p2aE0Wn082ePRs89o8++oie/+233xiGYVn2l19+qXGLWq1+44034BO3b99Oz9f2oA4ePAhtnVgs3rt3r+ljUyA3N9fT05MQ0r9/f/rN0YNqitTvQe3evRuGlEOHDqXTQ8ZtLGSkZRhmzZo19Pw333wDJ/fv31/jFpVKNXz4cGiWjxw5Qs/X9qC2b98Ovby1tfWxY8dMN1GdTgctPzSY4DK1bdvWx8eHbimEhrSwsLBJvCPTPSiM4vs/srKyfH19s7KypkyZUmfsx4gRI+bPn08IKSsrU6lUMPMEXTWsota4nmEYUP339PTs0KHD4z43JiZmxIgReXl5ZmZmK1euhOlYfB1IbfLz8319fQsKChYtWlQ7wpPP5w8ZMgRS1BcWFtLz9Lh28hCO47744ovQ0FBvb+/g4ODHfe727dtnzpypVqsdHR13795d54b7pyA5OXn8+PGlpaUMw0ydOrXO7ENIE6K0tNTCwsLV1dXS0vL111+vfYGvr6+fn19BQYFOp5PJZCA8XVVVBSlQDQZD7VY0MDDw66+/ZlnWz8/vcWlSKioqpk2bdvjwYUJIaGjo33///Szhdnl5edeuXaPpBPl8vpWVVW5ublBQUFPPFojk5ub6+fnl5eXNnDmztpGwLDtkyJDffvtNp9MZN6F0o2Zt++Tz+QsWLPD39/fx8enZs2edH2owGI4cOTJ9+vTy8nJzc/M9e/aMGjXqib62Xq+fM2dORkaGo6PjDz/8gNGkzZi8vDw/P7/CwsI5c+bU3j/PcdywYcMOHDig1+sLCgroeWqutRP4CgSCJUuW9O7d28fHp1OnTo8z0Z07d86cObO6utre3n737t0gkGsiGo0GYrAJIe7u7suXL3/11VdhhFxQUPDjjz9u3LhRLpdfv3597ty5u3btak4NKXYJ/8egQYPu3r2rVqsfl2CBOtM6nY5e4+npyePxtFrttm3bvLy8Jk2a5ODgQINTPT0961eDiI6OnjBhQm5urpmZ2Zo1a+bPn48bmZDHsXjx4sWLF1dVVdU5lDQYDFSJAbTygNatWzMMYzAYPvroI5FI1L17dxsbG+qlDxgwAJah6sRgMOzZs+e9995Tq9Wurq47duyo5+InIi4ubtKkSenp6bBlZd26dSgj0dSxs7MDN6aysrJONbM6TdTKysre3j4tLS0/P3/69OmfffaZl5cXxIsSQgQCAcxbPQ65XD5v3rxDhw4xDDNixIjt27c/4/iSz+fPnz/f3d29rKzs3LlzN2/ePH369NmzZz/44IOvv/76iVQBkcbGu+++CxtCHjdNyXEcx3E6nc44raK3tzfHcVqt9uuvv7a2th40aJCtrS0dBXbu3JmuiNZp88ePHwf3SSqVbtu27UndJ0LIn3/+eeTIER6P99lnn0FcK9JcWbhw4cKFC6urqx/XhNImiM7yEEJAz9ZgMHz22WcCgaB37952dnbUyPv169evX7/HfaJer9+9e/ecOXMUCoW9vf3+/fv79+//RN+5srLS3t6ez+eD9wVRiICTk9Pq1atbt24Nk7DHjx+/dOkSLFg1D9CD+h+odep0uvLy8tLS0qKiopSUlOjoaBoeoNfrqQfl5+c3cODA8PDwysrKRYsWrVmzpmvXroMGDerfvz8IUdTzWUlJSa+//npaWhohxNLScty4ceg+If8KnZdSq9VlZWWlpaX5+fmJiYnXr1+/cuUKtV56fVhY2Pr16zMyMjIyMsaMGePh4dGjR48hQ4Z069bN09OzftmcY8eO3bhxAwYTbdq0eVyi8ScCIm2mT5+enJzMMEyvXr127txZQ1YIadLAvjuDwVBVVVVaWlpaWpqRkQHJHm7cuFHDRAUCwdSpU6Ojo7Va7b59+44cORIcHNynT5+hQ4cGBQU5OzvX3yrOnDkT3DaGYcaOHfvsOe+7devWrVs3OP7000937Njx0UcflZSU/PDDD35+fu+99x6+36YObfQ0Gk1ZWVlZWVlBQUFiYuKtW7euXLkCA1PjJnTo0KGenp4pKSm5ublTpkxxdXXt1q1baGho3759PTw8INrncURGRv7222+QocTb2/tJx6aEkPv373/xxRcGgyEsLGzGjBn4+loCdJKU9vIFBQUPHz68efMmbKqvYaJjxoxZuXJlXl5eamrqhAkTPDw8unfvPnTo0O7du3t4eDxu9R44efLk3bt3QdrX39//cUup9WBnZ3f//v3S0lKtVuvg4FD7gmnTpv39998nT55UKBT//PMPelDNFoVCERkZeerUqXv37hUWFspkspKSktrhT8aG/sMPP7z99tu3b9/W6/VFRUUnT548efKkWCz28/MbNGjQK6+80qVLlzpXLePj4+lxYWHhl19++fvvv5uSAAppyRQVFUVERISHhyclJRUVFclksvLy8noy0/v4+GzYsGH+/PnZ2dl6vT4tLS0tLW337t02NjaBgYFjxowZNWqUn59fnePUs2fP0uPLly9v3779/ffff5Yvr9fr//nnnw8++ABSPw0cOHDbtm1PmjMNaeTcv3//+PHj169fB4lbmUymVCoN/6vJa8w777yTmJi4efNmtVqtVCpv3rx58+bNDRs2eHh4dOnSZeLEiQMGDLCxsanzXnCfwLRWr149cODAf5XsMx0ej/fWW2+VlpYuXrxYp9OtWbPm7bffxia6qVNWVnb27Nnw8PCHDx/KZDKZTFZaWlpPE2ptbb1ly5Y333wzPT3dYDBkZ2dnZ2f/888/lpaWgYGBYWFho0ePbtu2bZ3rWpcvX6bHd+/e/emnnz799FPTvyrMzBYWFrq4uHz33XctSie6JQO9/NmzZ5OSkgoLCwsLCysqKuoxURcXl82bN8+ePRsSlqSnp6enp+/bt8/a2jooKGjEiBGjRo16XEKICxcu0OPr16//+eefs2bNetIvDMlU6vnva6+9dvLkSULIvXv36C6YZgB6UP+fhISEBQsWXL58GVKJ0Xdvb28fFBTUrVu3b7/9tvZdgYGBx44d++effw4cOHD9+nWI6VcoFDExMTExMb/99tvo0aP/85//1JmWRCwWf/TRR5s2bZLJZPv37x8+fDhsE0SQOjl+/Pgnn3ySkJBgPP/EcZy7u3vHjh0tLCx27txZ+66xY8cGBQXt2rXr8OHDDx48gBmBkpKSyMjIq1evfvfdd3Pnzv3444/rbNScnZ1nzpwJ+pCrVq3q16/fEyUwMUav12/cuPHLL7+srKxkGGbixImbNm2qp9lFmhwKhWLFihV//PFHcXGx8XmxWOzr69urV68LFy4kJCTUuMvMzGz9+vUjR47csWPH+fPn8/PzYXdyampqamrqwYMHg4ODv//++zoDURiGgbCoo0ePpqWlff7559u3b//X1BGmw7Ls5MmTf/rpp7S0tKysrAcPHmAYVZPm6tWr8+bNu3//vvHEKMuyrq6u7du3Bw3J2nOmffr0uXjx4u7duw8fPhwTEwNqT+Xl5VFRUdevX//hhx+mTZu2atWqOif7HRwcZsyYsXHjxsrKyjVr1owYMcL0tA1bt269cOGCwWDo1KlTZGRkZGSk8X/pim5cXNzWrVsJIa6uriDSgDRdTp8+vXjx4vj4+Nq9fEhIiJWV1a5du2p7UyNHjvT399+9e/eRI0fAvA0GQ0lJyZUrVyIjI7/77rvZs2d/8cUXdc7mu7m5TZky5YcfflAqlatWrerfv79xJN5zgT4QpFbRg2pu5OXlTZo06cGDBwaDQSQS9ejRo0OHDsHBwe3atXN1dbWysiorK6vTgyKEODg4zJw5c9q0aXl5eZcuXYqIiLh06RJom8rl8r1794LOSQ3bNTc337Bhw/Tp0x0cHObOnVtdXb1s2bKePXs+Lvco0sK5cOHCtGnTiouLYb6nb9++YJ9t27a1t7e3srI6cOBAnR4UIcTX1/err75auHBhWlrauXPnzp07d/XqVdB0KigoWLlypVQqnTt3bo273Nzc/v777w4dOsTHx+/bty87O/vTTz/du3fvU+wG0el0S5cu3bhxo1qt5jhuzpw5q1evpttdkGaAXq9funTpL7/8otPpQE6zR48ebdu2DQ4O9vPzs7S0tLCwGDJkSG0PihAiEAiGDh06ePDgoqKimJiY8PDwCxcuxMbG6nQ6rVYbHR09derUS5cu1V5fmjBhwpYtW7Kysm7dupWXl7d///4xY8aYmMPERJydnV1cXNLS0gwGQ0ZGBnpQTZfY2Njx48fDtnsrK6tevXqFhIS0a9cuODjYwcHB0tLy/Pnzu3btqvNeDw+PTz75ZN68eZmZmRcuXDh79uylS5dKS0thnLphwwaJRPLVV1/VuMvOzm7Xrl0goPfzzz/L5fK5c+dGRESYuJr06NEjcOeOHj1KNdxqA5EvhJBhw4ahB9WkiYqKmjx5cnl5OSHExsamd+/eYKLQy1taWv7zzz+PM1FfX99ly5YtXLgwIyMDlrBoL19UVLRmzRobG5sPPvigxl3u7u579uzp2rVrenr63r178/Lyli5dum/fvue74EldJthq2Hxe2BNp/JHmq2b+0UcfQSBTUFBQbGxsbbVo2vF37ty5tspkDW3HqqqqS5cujRs3DmxFKpXevXsX/kvVzEeNGgV6kRqNBqZXGYaZPn06qpmjmnltNXO5XN63b194rePGjSsqKqqdTuTnn3+GC2bPnl3/B2m12tLS0gMHDtDZ0JCQECr+S9XMqc7po0ePINaOZdk///zzSX+XRqN5//33IcrFzMxs7dq1z27kLwtUMwfbqK1mfuXKFXCJRSLRli1blEplDT1clUpFg+whcXk9qFSqR48eff7551QZYuXKlfAvYzVzGMIaDIaNGzdC3XFxcYFVLFOQy+W7du36/PPP586dW89lvXr1AuM/fPhw439HqGZOHqNmHhYWBoLLgwcPzszMrN2EHjhwAIYBr7zyyr+2A+Xl5UeOHOnduzfcIpVKVSoV/JeqmVOd8by8PH9/f4ZhOI77+uuvTfw5H374Ifd4aNwgy7JwxjjVSmMG1czJY9TMu3btCkPBcePG5eXl1TZR0NYnhLz33nv/aqKlpaX79+/v1KkT3OLn56dQKOC/VM3822+/hTM5OTmwi4nH49WZ7u9x3L17d+fOnT/88EM9rfqZM2fg4/r37197dN3YQDXzJ0OtVp8+fdpgMDAM8/XXX0Pe0hqkpKTAgbGSRExMzN27d9PT02fNmuXk5ESbM7FY3Ldv3169evXs2fPmzZvl5eW1Jy85jgOz5vF4Gzdu7NevX1lZ2fbt28PCwl555RV8KYgxubm5t27dIoRYWlr+/PPPoAT9OBOlUSgajebq1atxcXEqlWr+/Pl0CwfHcVKpdMKECd26dWvbtm1FRUVmZmZZWVkN7RN6vbe397Jly+bMmaPRaD7++ONBgwZ5eHiY+M3hlj/++EOv11tbW//00091Sl0jTZ1z587J5XJCyJgxY95+++3aF5SVlclkshomWlBQEBkZ+fDhwy5dugwZMoReLBAIvL29V65caW1tvWjRIkJIXFwcNNF1mug777xz/PjxiIiIvLy8BQsW7Ny505SZTqVSuXz58qSkJD6fP2/ePD8/v9rX5Ofn5+fnE0IYhvH29sYX3UTJzs6GEZ5EIvnuu+/qjPV49OiRwWAwtk+9Xh8VFRUXF5eTk/Pll1/SQBKO4ywtLUePHt2lS5fevXunpqZCL+/r61unfTo5OX3zzTevvfaaUqlct27doEGDYKxcP++++249utKXLl2C2a7x48dDjUNJniZNTk7OnTt3CCEODg7ffPMNHVIaU7uXNxgMEB2tUCg++OADGiECvfzEiRO7d+/eoUOH4uLi4uLinJwcEO6jUJN2cXFZt27d22+/rdVqV6xY0bdv38dtnarBjh07vv/+e0LIvHnzOnbsWOc1Fy9eJIQwDPM4XYAmCnpQ/2eCJSUlYEx1Dg11Oh2dVYJpA7hr/fr1O3fuZBimTZs2tRPacBxHZ+7r33/crl27jz766Msvv9RqtR999FGPHj1cXFzwvSCUqqoq2J4HS/m1L5DJZMePH6fmCgeVlZVz5syJj4+3sLAYP3587SAoqVQqlUorKioEAkH9I8433njj2LFjx48fLykpmT9//t9//23KlnqDwfD7779v2rRJp9NZWVlt2bJl7Nix+DabJXTvk4+PT50XXLhwAaRHjU305s2bkydP1mq1o0ePDg0Nra1oQhvk+kPnxWLx6tWro6Oji4qKDh06tHfvXprqtB5sbGxCQkKSkpK0Wu0ff/xBV1+NDfjo0aMZGRmEEE9PTxOHFEgjpLKyEnT2hEJhq1atal9QVlb2zz//1LBPg8Ewf/786Ohoc3PzESNGUJ1GilQqhYknhmHqN9ExY8a88soru3fvLi0tXbJkyeHDh+tX6yWEBAYG1mNyEOtFCPH29g4LC8NX3NShTahUKq1zkrSkpASy4pL/1eJ7++23MzMz7ezshg0bVnsBwNHREULyOI6rPwL/9ddfP3DgwPHjx/Py8j755JM9e/bULzUJ9O/fHzyo06dP5+bm1h675ufnb9++nRBiYWExevTo5vTKMHkrAcOCt67Vai9dumT4X9kojUbzyy+/0LVXtVoNDTHDMGPGjIFkO+vWrcvKyqrx2MTERNj66eDgUP/+e4ZhZs+eDWrR6enpn3zyieHx0lVIC8TKygocp+zs7No7SSoqKhYuXJicnAx/Uukza2tryOAkl8uXLVtmnOQEOHXqVHZ2NiGkTZs2dTbZFLFYvGLFCoiDPXXqFJ1QqJ+kpKSvv/5arVYzDLN8+fJhw4YpH0/tbIBIE4JO6l++fLm2pd28efOzzz6j86aguEMI6dSpE2Qwj4iIoFMAFIVCsXv3bkIIwzA0XOpxdO7ced68eRzHqVSqr776inpr9cAwzLRp08zMzCCA7dy5c8b/NRgMFy9eXL58uVarZRhm0aJFzSqCv4Vha2sLSg9VVVVRUVE1/iuXy5cvXw4rAMb2yXHc6NGjGYZRKBRr1qyhTgvl4sWLsCzg5eVVp2NmbGxr166FJvTKlSubNm2qR10NaYG4uLjA+kxeXl5cXFyN/5aVlX300Ud0DYqaKCRyIIQUFxd/9913EAhgzKFDh8A38/b2rl/5lsfjffPNNxDLd+LEiR07dpjytQcOHAhLrykpKatXr6ZfDCgqKpo5c2Zubi7DMMOHD689B9GkQQ/q/+wGTNBgMKxcufLnn39+9OhRcXFxZmZmeHj4W2+9BdHMtP2lYn1Dhw7t0aMHISQ6OnrUqFFbtmy5e/duUlJSdHT077//DvtVQM3pXzV2ra2tV65cCZNSe/bsoSq9CEIIcXZ2hs0YKpXqrbfeOn78eHZ2dnFxcXJy8r59+0aOHLlr1y5zc3Nof0tLS8EbYRjm/fffh2CAXbt2TZo06ciRI3FxcYmJiVFRUStXrpw5c6ZerzczM5s9e3b9WSMIISEhIQsWLODxeBqNZuXKlYmJifVfD6PSnJwc+HPLli296sVErwxpnAwePBjSMUVFRc2ZMyc6OlomkxUUFERHR69cuXLMmDGpqamQKooQUlRUBAcuLi7vvPMOj8errq6GjLqRkZGJiYkPHjw4efLklClTjhw5Qghp27atKauXCxcuhGDpR48eLV++3Di19OMIDQ19/fXXWZYtKSmZMmXK6tWrb9++nZycfP369c8//3zSpEmgvD927Ni33noL33LTxcHBoW/fvgzDVFdXf/jhh3v27MnIyCguLk5NTT106ND48eN/+uknoVAITaixmOQ777zj5uZmMBiOHz8+fvz4/fv3x8bGJiUl3bx5c926dTNmzKisrGRZdsmSJY9L1EtxcXH5+uuvWZbV6XTr16+Pjo7G94JQ7OzsYEt8RUXF3LlzDx06lJWVVVxcnJKSsnfv3jFjxmzfvt3c3BwmkmgTSgiZO3eunZ2dwWDYvXv3q6++evjwYdrLr1ixYv78+Uqlks/nL1y48F/ngAIDA5csWQJ5pb/44gvqsNWDubn5t99+a25ubjAYNm/ePHny5FOnTiUlJcXFxe3cuXPEiBHHjh0zGAw+Pj6rV69ubtkgnmh/FWm+ShJFRUW9e/eGMuE4zsHBwd3d3dnZGVY/HR0dt23bNmzYMEKISCS6deuW8S46Gj3PsqyVlRUIo0FjCjsC6XZng5GSxNixY2vv/Fu4cCHc2Lp1a1DzewpQSaIp2qehXiUJg8Fw69YtmECCcBEXFxd3d3d7e3sej8cwTMeOHQ8dOgQhTz4+PpWVlXAXZByn4SJ8Pt/GxsbOzo42xCKRaMWKFcabO2ks08aNG2t8B7lcDlMGDMNMmDABFrseR1lZWZ1bCh/HqlWrGv87QiUJeFm1txrrdLoVK1ZAB8kwjJWVlZubW6tWraysrMBi58yZ8/3334PVLV261Nio3njjDdpgisViOzs7GxsbGnDi5eV19epVer2xkkRtUZ/IyEgIphKJRHv27DHlR1VUVLz22mvwxRiGsbS0tLe3p0KRLMtOnDixsLCwqbwjVJIgj1GSSE5OpjvZBAKBk5OTu7u7o6MjGC0kJoEL7O3tjd/4mTNnpFIp3Mjj8aytre3t7SUSCd3JvGDBAuPGkE4GffbZZzW+Q3V19ciRI+G//fv3r6qqeuofC8uzxEivoqmAShLkMUoS9+7dc3Z2hguEQqGzs7O7u7uDgwM49h06dLh06RJEo/j5+RlX8H379tFensfj1ejleTzesmXLjHUpqJIE1YuilJeXQ+gKIWTEiBG11Sxqo9PpNm3aRL+AQCCwt7e3sbGhDpuvr+/t27ebyjsyXUmiBa1BcRxnZWUFUcu154psbW0PHjw4b948S0tLnU5XWFiYmZmZl5dnaWm5YMGC69evv/nmmyNHjpRKpSKRCJRDgZCQkIiIiPnz59vZ2en1+vLyckhyajAYAgMDf/nllz179tDGF0wZNp+Ym5vX/oZLly7t0KGDVCotLi6uHZRvIiKRCD6i2YjutxD4fD68ODpVb0znzp1Pnjw5atQoiFPKzc3NzMwsKiry8fFZv379xYsXR40a1bt3b6lUWlJSArITMCKcPHlyRETE+PHjBQKBRqMpKSkpKiqqqqpiWXbQoEFHjx799NNPjTd31mM/5ubmGzdutLOzs7KyunTpUv0rpQqFoqKiQmoymC+y8WNubv4422BZ9pNPPvn99999fHygG87KysrOzlYoFMOHDz99+vSGDRuGDx/u5eUllUrDw8MhFhqe+eeff27atKlt27ZgNkVFRSUlJRqNxtbWdt68eRcuXKAifmDSEokEvkbtuL6ePXvOnTsXzGnt2rUFBQX/+qMkEsnWrVv//PPPoKAgQkhFRYVMJpPL5SzLhoSE/PXXX9u2bcM9+k0C2r1S98aY1q1bnz17dsKECQKBQK1W5+fnZ2ZmFhYWtmrVavXq1RcvXhw5cmRYWJhUKtXr9cb5xIcMGRIZGfnaa69ZWFiAxJlMJqusrOQ4rlevXvv27Vu7dq1xjRAIBPA1ai/si0SiNWvWQC24d+/esyy81/MpSKOl/l4+ODg4IiJi1KhRfD5fpVLl5eVlZmbKZDJvb+/169efO3eub9++MBA17uUJIRMnTjx79uwrr7wiFou1Wi3t5Xk8Xv/+/Y8dO/bFF18YL0AJhcLHdbuWlparV692c3OTSqXXr1/fv3//v/4olmXfe++98PDw8ePHi0QitVotk8lKSkp0Op29vf2iRYuuXLnSqVOn5vc2GcOT7LeBCcLw8HAIKGpy0LDjelbbq6qq4uLiIGWEp6env7+/8d47eAIoota4UaPRJCUlwYjBwsLCz8/Pzc2tzg+q5yHGX7L+71kP4BzX8xGNmd27d0+dOtXPz6/OvDH1s27dusWLF4eGhsIsbLM0UYPBUFJSEhsbW15ebmZm5uvr6+HhQVvGel69wWBQKBTx8fEymUytVtvY2AQFBdnY2NS2kH+1H/ol/9XAnijQv0mYq06n6969++3bt7ds2VKn4lw9FBUV9enT5+HDh+Hh4U00bYspbYtOp0tJSUlLS1MqlXZ2dm3btoVlqH+1cL1en5+fn5SUVF5ezrKsu7t7mzZt6tz6XH81oV+SPGETqtfr09LSEhMTy8vLLS0tg4KCPDw8mlwTGhERMWTIkICAgCtXrtSZyb0etmzZ8s4773Tu3PnGjRtP1/u8dExpQsvKyuLi4kpKSvh8vo+Pj6+vL724fguvrq5++PBhfn6+Uqm0trZu06aNg4PDC21CTfmxTa6jHzJkSERExNq1az/++OMnvbdNmzZJSUk7duxoupqupphocXHxgwcPSktLRSKRr6+vl5eXKSYKvXxCQkJBQQHMQMEO5wYzUYPBIJfL7927V1hYyLKsh4dHYGBgk5vKv3r16pAhQ0D5vf4rW5YWnyldgrm5eT173ep5Ap/PDwoKglnMZ/kaz95vNUXHCTHx7TMMY2trSxfZTX/1DMOYm5t36dLl2e3HdBNtooMw5KltgxDCcZy/v7+/v/+TmgTLsi4uLqbIkNZvV0/dALIs6+Pj8zgtQaRJYEoTam1t3adPn6cwHjMzsw4dOjx7NXleDSM2sM3VROmeqCeyLujlaZBzw5soBAjQHTHN/1WiNSMIgiAIgiAIgpgIelAIgiAIgiAIgiCmgh4UgiAIgiAIgiCIqaAHhSAIgiAIgiAIYiroQSEIgiAIgiAIgpgKelAIgiAIgiAIgiCmgh4UgiAIgiAIgiCIqaAHhSAIgiAIgiAIYiroQSEIgiAIgiAIgpgKelAIgiAIgiAIgiCmgh4UgiAIgiAIgiCIqaAHhSAIgiAIgiAIYiroQSEIgiAIgiAIgpgK77k8JTc3V6lUYmkipiMSiVxcXBrs4x49esQwDBY7Yjr29vYSiaRhPkutVufm5ur1eix2xEQ4jvPw8GiwjysqKqqoqMBiR0xHKBS6uro22MelpqZimSNPhFQqtbGxeerbn48HJZfL/fz88GUgppOcnNyQH6fT6dBEkSeioKCgwTyo6upqT09PLHPkiTAYDA02MSSTydq0aYNljphOUVFRQ35cfn5+z549sdgR0ykoKHiW2zGKD0EQBEEQBEEQxFR4z/FZBw4cWLx4MZYpIWTr1q39+vUzPnPt2rU33ngDS4YQsmrVqpdVFAEBASqVCl8BISQiIqJ169ZYf02svw1DUVFRly5dsPwJIW+++eby5ctrnGzTpg2Giz+u/jYM9+7dGzt2LJY/IeSTTz557733jM8UFxd37twZS4YQMnXq1BUrVryUjx47duy9e/fwFbi6ukZGRtY4OWfOnJMnT2LhEEKWLl36/vvvP+NDnqcHJZfL09PT8cUQQqqrq2ucUSqVWDjUTl7WR2dmZtZ+NS0TjUaD9df0+tsw6PV6fAVASUlJnfVXoVBg4dRZfxsGtVqNJgrU3hWG9bf++tsw5OXl4Vt4HDKZDAvncfX3KcAoPgRBEARBEARBEFNBDwpBEARBEARBEMRU0INCEARBEARBEAQxFfSgEARBEARBEARBTAU9KARBEARBEARBEFNBDwpBEARBEARBEMRU0INCEARBEARBEAQxFR4WAYIgCIIgjRAej2dmZkYIYRgGDjQaDc3YplardTodlhKCIA0PelAIgiAIgjRGvLy8hg0bRggRiUTDhw8nhCQnJ58+fRr+e/v27ezsbEKIwWDAskIQpCHBKD4EQRAEQRAEQRBTaSZrUCzLCoVCQoi1tbW1tTWcFIvFLMsSQjiOgzMKhaKsrIwQUllZCWEASqVSr9e3wBfftWtXJycnQsiVK1c0Gg0hRKfT0dAIBHkWxGKxpaUlHJubm/N4PEKIp6cnwzCEkJycHLA0nU5XIwKHYRipVArHrq6uQqFQr9enp6fDmYqKitzcXEKIRqNpmdUWQVoIIpFIIBAQQhwdHdu0aUMIEQgEdnZ2hJCqqqrWrVvDZUlJSdCq4BoUgiANTDPxoGjb2qlTp44dO8JJNzc3cKtEIhGcyc7Ojo6OJoQkJSXBUCw3N1etVrfAFz9v3jyIiBg6dGh5eTkhpLq6OicnB6sE8uw4ODjAoIcQ4uXlZWFhQQiZNm0azGUcPnwYLE2hUFCnHYZBLMuGhITAmfHjx9va2mo0mm3btsGZBw8eHDlyhBBSWVmpVCqxnBGkuSKVSm1sbAghwcHB48ePh8YBztjb25ubm8NlsbGxSUlJhBCcUkEQpIHBKD4EQRAEQRAEQRBTQQ8KQRAEQRAEQRDEVJpkFJ9EIoGdFT4+Pp6enoQQW1vbTp06EUJcXFycnZ3hMnNzc4gaovugfH19g4ODCSGZmZlFRUWEkF27dslkMkKITqdrCTFs5ubmVlZWUCYqlYoQUl1dDZFU8CeCmAifz4eaNWDAANixwHEcyA2rVCoanhcbG6vVagkhGRkZEKr36NEjuVwOlQ7+Rf4bxUcIefjwIRzExcVBFR4zZgyccXBwgF1SZ8+ejYmJwVeANBgSiQSsXa/XQ+C3Vqul1os8d6RSqZubGyHEyckJwoBpE5Gbm3vixAk4zszMxLeAIMhLoUl6UCKRCLY2eXp6dunShRDi4uISGhpKCBGLxWKxmF7JMEydG0zz8vJKS0sJIZcvX4YL1Gp1S/CgRCIRjEE5joNN/HQcgFk1kCdrO3g8Pp9PCGnXrh1sS+DxeOCfp6enJyYmwmXZ2dkVFRWEkHv37sEZlUpVz6aFR48ewUF6erpAIBCJRPPnz4czlpaWULsfPHiAHhTSkJiZmcG0nU6ngy7DYDDg2P3FYW5ubmtrSwixsrKiO5mB8vLy2NhYOC4tLUUNCQRBXgoYxYcgCIIgCIIgCGIqTWkNimVZWMd/5513AgICCCG+vr4+Pj7EKHwoPz+/oKAArj948GBJSYnBYKDxaa1btx48eDAhxN3d3dvbmxDy7bffQkhGXl7ekCFDmv37dnNz69mzJyHEx8fHwcGBEGJubg5q5qD8jrwgOnbsCCbKsiyEAykUClicUalUaWlpTe4XKZVKqFl//vknGA/DMHCg0WhopaMrTnTdycQ54+LiYoZh+Hw+1eLz9/cHAcny8nJ3d3c4uXPnzqqqKjSwF4GPjw+0k4MGDXJ0dIQ3e+7cOUJIXl5eZGQkvM3msQjAsiwkeBAKhVSLPzg42MXFhRAyYsQIiCWrqKgIDw8nhDx69OjOnTtg4VlZWWgtz5cOHTqMGDGCEGJmZgYLzgqF4p9//iGEpKenQ8kTQiorK1tg4fB4PLouJxAIoNVdtGgR9CxisRiM9sSJE2CrhYWFGKXfMG0IvAsvLy8Yo9JEO4QQumRtZ2dHTz4OuVwOetGEkPj4+KbbwggEAmhOLS0tP/zwQzjZvXt3GA6VlpZCwP/KlStLSkoIIUqlMjU1lTSR/ARNyYNiGAY8KDs7O1dXV0KIo6MjiJjr9Xoak6ZQKOD6nJycwsJCg8FAhY/Nzc1hA4bBYIAGqFWrVvCeIEKj2SMUCiUSCSFEJBJBCBbHcdDsogf1QjE3N4cINFrgDMOA79pEoUPn4uLiF/F8GlYKDSshRKVSQVtsY2Njb28PJ+kuR+S5IxKJICyzVatW0ORqNBpQlJbL5XRfSvPwoBiGge18QqGQhoLb2trCxlofHx+wvdLSUph7KikpqRFdhjxHxGIxWBohBMZYVVVV2dnZhJDCwkLay7fM4HM6DUcI4fF4cNyqVSs4sLCw8PDwIEYjdezcG6wNgVaRbpdgWRZcBUII7e7t7e3/tekQiUR0ZlAgEDTdpGcsy8JQUygU0nnPwMBAaGOLiorgZ0okkhqpTZrE78V6hSAIgiAIgiAIYipNaeFFKBSCbyoUCmFmhU7D6PV6WKQuLy8vLCyEk1VVVUql0jiKTy6Xg/Jeq1at4CT1cRmGoVJLzXhnKp0PMBgMMHun1+trBFkhFDqlBDPxhBCdTgczSVqtFgrQzMyMxrBB2dJCNsbe3h4mXej0oZmZGcx5V1dXg1kSQioqKsD86Ib1Fo7BYKBTcbQicxxXu4SRx1mvMU9UzWmwEG1yWZa1trYmhJSVlYH1Gi/yN/Xigp9mZmZGVzilUikE79EYco7j4IxUKoXFKKVSWTuWTK1W03a1HsGJ+lVVWiA8Hg/Kmc/nQ2CIcTMIlqZWq+nJltZI0lUmkNkghIhEohrr8FQu0szMDAJTS0tLqRE26cCHRgVd2QPFI2IU2mNvbw8rqCzL0uUm+gqsrKz+NYqPZVkImCKE2NjYgPIZDGiNL9PpdPCuGxt8Ph+KwsLCAozQ0tLSeOUNfggdeNvZ2UHFr6qqysjIgGreOH/a/7RXTchee/ToASbbs2fP9u3bG/+roqLi/v37hJC//vpr586dj3vCvXv3Dh8+TAiZP38+SCRTGTHa1igUCjDWZom9vX27du3AfGHUXlFRAd0/Nqy1sbKyghiJzz//HM4UFBSApaWlpeXn5xNChg8fDqGkfD4fytbW1tbf37/Go5ydnWFAQMe1Go0GGojKyspLly7BZd988w2MErKzs3F7DyFEp9MdOXIEjjmOmzFjBiHExcUFihqKHUupTiwsLMDJ4fP51OTKysqI0bxJ/Xh7ew8YMIAQ0rNnT2geCSG9e/cmhFy/fh1C87Va7fXr15tBcUkkkuXLlxNCnJycOnbs+LjLRCLR22+/TQhRq9Xg0iuVSiogSYmLi4Po0/z8fLqfoTbnz59/QUGwTRQPDw+IevLz8/Pz8yNGPXJ1dfXVq1eNS76lIRQKIaZ07Nix77zzDpw0NzeHnoVWQ4VCERcXRwgJCwtbtGgRIWTevHmwl0yn00H/hTwjHMdBv08I6dy5c9++fQkhXbp0gRdkbW0N0yvPhcuXLyckJMABuFXQsBNCsrOz4c1qNJrGMBdDZ5ratWvXunVrQkiHDh2mT59OjGbfAPCgpFIphDtu2bIFzmdlZX3yySeEkJSUlNu3bzdyM8AoPgRBEARBEARBEFNpSmtQXbt2hfU+0EJ4CmjoWn5+PuSr8fX1palsQDslPz+/Ga9BSSSSVq1aEUIYhoElDpVKBfN5mNukNl5eXhYWFhzHgeQjIcTW1hZieFxdXWERr0OHDjCJwuPxIAWkRCIxTkoG1I6nouvXQqEQXgohRCqVwn5KWOBCiNFmcY1GA4VTUVFBKykGQT0Of39/WDhycHCA1XuFQgEBEnl5eZmZmXDZ01V8oVAID28Ga9eWlpYwPwq/CKozUFxcDO2kTCYDO+Q4DiQltFot/HaNRgP52Y3R6/WwOurm5gbNQp3weLzy8nLjM1qtFoy8srISPloul+fl5TVvW6VKTj4+PlD+9vb2NOAcNEsrKytpLuMWVZFp2AKtdLa2tnQUJBQKoR+Bqg0GCZYDinyEkPbt24PRajQaWBpVqVRUoQeb0CcCAvAsLCxg3YkQ0qZNG1gvtbe3h+HBvwbpAVSNyThon24KoMGZdnZ2oImqVCohRIXWl7y8PKgvOTk5tE/My8t7We9UIpFAtfXz8+vcuTMhxMvLC1bMjIdAWVlZ0JxKJBJYcxaJRPDDhUJhUFAQIUStVsPKG/zwxqkZ05Q8qPnz58O7AZ/nKaA5EGNjY2ExtG/fvhBPLBaLX3nlFULIrVu3Hjx40Fwrv6ura48ePQghiYmJ0OtXVFRA/4TUZvDgwe7u7hzHDRs2rEaTV1BQAAFRnp6eVGznyeoejwftoEgk6tWrF5z09vYGmamsrKwaQ6sWCx0wVVdXQyeRm5tL9d8xE/TjGDduXJ8+fQghXbt2hZazuLgY4kXPnDlz4MABuEwulz9FGVpZWXXq1IkQolQqaZhlE8Xd3V0oFNra2sIvMt5pkJycnJ6eTgi5ePEiVEyRSBQYGEj+bQeOmZkZjHGDg4Mh7XudyGQyjUZj/CiqYvzw4UNQ9X306NHRo0ebt63SKaehQ4d6enoSQtq2bQuD0dLSUkh2T73Zlgbd9imVSjt06EAICQgIoJNu1HgiIyNh3KxSqaCddHNzg5jbadOmwb+qq6tho3hhYeHNmzfhxkYSANZUsLKyYlnWzc3tyy+/hDO2trY0yPmJoNHUCoWCNsLgb9AMPYSQwMBAaHNCQ0MZhjFuLoqLi6GBOn36NM0yf+LEiZe1N9XNzQ3G52PGjJk4ceLjLouKioIReFBQEMQ9uri4gAcllUpfe+01QohEIrl37x5cn5eXR+U3GxUYxYcgCIIgCIIgCGIqjX0NytbWlk7/m5ubw7Jm7QwwOp0OPFQTQ0rUajVcT6deeDyel5cXMVoKb97QdK44+VQPPj4+/v7+NQLw4E+JRAJzRbUziSkUChogUVVVVUNPRiAQwC0SiaT2ZtOAgAAIqrx7926zD90xEVr+NLDBxcWFzsMJhUKYu0JLBuzs7KBA7O3tQVOOFqBYLG7bti0h5P79+9Rua8eXmgINKKJJPJoWlpaWsMRBCJk6daq1tbWZmRmYUHp6+sWLF+Ffd+7cgXDHrKwsWAvlOK62bkRteDwevIVLly7VMz/t7+9vYWFhPKmsVqthdjYtLQ3SH9UOEWx+FRxC6AkhgYGBsAZFk0G1NOhe/Pbt20MHQZc4rKysunXrRghxdnamQm2RkZEgRvLgwQMwJK1WC53IhQsXwGiDg4Mh1kuv18MEf3p6OpWSS0pKgr6mdevWoJamVqvhxqqqKjA/g8HQkhtYjuNocb322mtmZmbW1taQZJwQUjsIRaPRUM1eGkty69atGpcpFApYLEpJSYH1VT6fP3r0aHjpVC2pNnRQwTAMxGd6eXnRF3TmzJkGLh8XFxfQ1XjjjTfAaKGjIUbrbNXV1ceOHYOThw4dgt97+/ZtqOkzZ86kJgqCE97e3jQ25+zZs41zDaqxe1BSqXTUqFFwLBaLwYhrh09QyXITI6RpE0P+O4Dg8/kQrU5FQpsr8HtVKhVKvf0r7u7uvr6+tUuPEGJhYUFHYDVQqVRUUp8mjKOIxWIIknZ2dq7tQfn4+EDL+9Shqs3VYomRELyDgwPttAQCAcdxLbyDN8ba2hq8I1tbW2jNaN9vZmYG8fqOjo5Uw/DpPCiBQADuWePs2P4ViURCJctHjx5Nd4wQQnJzc0GylRASExMDIWQviMmTJxtPBxgDGeEJIc0+mpdlWbrR1NfXFzyoFgvHcVAlg4KC2rRpQwjx8PAAcVexWAwHCoWCelBXr14Fl/7Ro0c1DOnWrVvg/4jFYsixy3HciBEjCCGJiYmgpUkIKSkpAQ/Kw8MD8nZUVVXBAEkmk0FAIHpQdMppxIgRVlZWIpGIavHVRqfTUaHOgoICOHnu3Lkal5WVlcFEdlRUFMy6mpmZgYSdh4dHPR4UXSpgGAbGIa1ataLfsPas7ovGwcEBRkqjR4+G+ks7HWo5CoXixIkTcDI8PBzGRY6OjhDtPGHCBBq+CN2Wu7s7hFWDJdejaPoy266W3FQhCIIgCIIgCII8EehBIQiCIAiCIAiCmEojjeKj+cgCAgJATgqARWq6VJ2QkJCUlEQISUlJ2bVrF3lyDWgqrUb+KwckkUho2i+5XN788szC701MTHz48CHBRLr1cvjw4Zs3b3Ic99lnn8EZrVYLIch37twB2ysoKIAyVKlUN27cgAMaeFNdXV2jhN3d3WGROjQ0tHbiXXt7e9DkoSnzEBpYa21tDRELMpmMCkhqNBpjKdgWCw1rfOuttyAENCgoCGJyni5Or1nCcRzE0Pr5+dEgGb1eL5fLq6urT548SQi5f/8+zXD9oiWtTp069biQG41GA0HpzVVtkmVZMFqBQDBlyhQ4WXv7U0VFBbS0LUSbtF27dhBhO378+O7du0P5QHeg0Wgg9O7UqVN79uyB65OTkyEgqnYbmJWVBSFkDx8+hGeyLNu+fXswKtoxtWrVCpT9xo0bB42tq6srHFy9evWXX34hhKjVair11gIxlivMzMy0sLCQSCS1o+yoZFx4ePjBgwehAaGDUhDvNUan00Ht1mq1sAMNdliRx4TxFxUVabVapVL51VdfwRmZTAaKnSqViu5haRh1ZZZl4TsTQoKDg/v160eMYsjpTrC4uDjQfa2qqjp16hRcT2O/6RhboVDASRpebm1tDbLmhJDH7Zh46TRSD4rH40HbKhaL68n+pFQqwSiLi4shkPdZ9vZQmQrapdFQzuYH7oMyhbKyMir8QIGOSi6XQ4h5Xl4e+FRKpRJ6erVaTctWpVLVGADx+Xxo6SorK2t/Ip/PB6trxrb3RDAMYxzeTVVkcNdTjVIyTh4CbaaZmVnDB8Q3lYKiauPUnLRaLWR4KykpoZtMXjQtOWMBfRccx9EtJbUtVq/Xw5aSGpI8zRWxWAz+ko2NDd2qR0sMCqG4uBjGzXD8uJKho1jaH7EsC2bP5/NBgYAYJTCkOyddXV1BYzo1NRWG8rXlu1oa1EFVq9VqtbrOqWeNRgO2WlRUBJvTjPdB1QNN6sWyLFSBOgtcq9XCHC7N5kdTm77E+gtGC7N1PB6PTthBiSkUCtjCRNXLatRuGCDp9foaCyTUC2jM5tck+1c65UxzGmq12hql/3QGQYyy1yEINFiEEKo7QpO6KpVK6LeMVXdgWG+8sPmkBkknpZr0ogqPx6MzSbWhzo9Op6tdYrWhrSf1KulKIL0X16Boq8VxHJRY7XaMdlfGq3Yts+ioKgnFYDC0zIStLxHqQdWeMDJONgpvpP73YvwEav9wkrYzpCks6PH5fGg8af2lg0utVgvz9Gq1mv6iJ6q/dFs/7WjgJHwWy7LGAgDkfwe4aK7Uuuq0WPK/I1LatdXzKDrapBO1xq4UvYw21xDSUl1dTd/dy30vtAmlnY7xd6YDpHrqL22HaVEYO2BPZ+QNSSP1oIRCISgbWlpa0lx7lMuXL8Nk4bVr10Agsqys7BklX3k8nqurKyHE09MTBHAIIfHx8fBBSMskPDwc2ko6h5eVlQWhekql8kUEQF69ehUCh0CgtokyZcqUTz/9FI5rt32ZmZnghZ47d+7y5cuEkIqKiscpRPN4vPHjx8MxjW+JiIg4dOgQnCwvL0f3SSAQgIITIWTcuHE0DrkGBQUFf//9NyHk0qVLNIF9y0xJ3L59e0ihTgixsLBgGEahUHz77bct3JAaEjs7u4EDB4L1QqJYY0pLS2GV/u7duxCiX7+hhoSEQEyRubl5586dCSG+vr5du3YlhBw+fDguLg4uO3jwYCMPXP/oo4+g/vr6+sJokiYQj4+PnzNnzrM83GAw3L9/v8bJ4cOHe3t7E0K6d+8OsulU66+goACub+GJy0UiEUT/EkJCQ0Otra3r9KAiIiJiY2PBaKnUYT04OTmBJPrcuXNBjpLH41EJb8q9e/cgbe7SpUtzcnIMBsPLSphrjIODQ2hoKByPGjVq8ODBxv89f/48jJTi4+OptGltunfvDvKw/v7+NbQNKyoqUlJS4LjRKr5ipBCCIAiCIAiCIIipoAeFIAiCIAiCIAhiKk1yH1ReXh5spMvNzYU0ZM8uikC3q1ZVVRnLfDWP11x7xbmsrAwkYjC+uR4gmplhGFhDJ4QUFRWBVbygcqusrIQItya9GUMul9M8pK6urhCISyuptbU1bGIODAyEYpTJZGCier0e4lUIIWZmZizLCgQCWOUnhEilUtiTWlhYSKMc0YBNR6lUQlyETCarHWLu6urKMIxxSKSNjQ28u2aja6LX66H+0uh88t9NCDRKB2kYBAIBKO7WuWcyMzMTxBKSk5OhSayzprdu3RqCf7p16wb7zoVCIUS0Ojs7g2qCn58fFQ1Tq9XQqsMzCSFFRUUQq19cXEz3u75Erl27BspjZWVlEOKVmZkJQc4ZGRnP61MYhqFyrw4ODpAFlSpVPHr0CBrwxMREONPCm1m6q4cQcuvWLdhdYiwTDTg6OkI8JH1TxuVMpWtcXFzAIO3t7cF6XV1d4YBlWYif1Gg0VGsqLi4uISGBECKXyxuP8KxAIAD7JEZKsPTrFRQUJCcnE0JA4+1xWFpawkPoNiq6/am0tJRG8TVa2bMm6UHduXMHhDXT0tKoJskzotfrwRkrLCzMzs6Gk7SRbdIwDEO7KGqmaWlp9+7dI7htul5otPGFCxca5hNzc3Mh5LcxBDo/NQUFBbBBkRDi4eEBTSR1jXx9faH/8Pf3Hzt2LPwL9ueo1erNmzfDZU5OTkKhUCQSjRs3Ds7k5+ffvn2bEBIXF/eyBIiaNGVlZadPnyaElJaW1q74PXr0YFnWuHsOCAiAMW6z0dbX6/VQs5RKJa1iEolEJBI16RrXFJFIJMHBweQxQluRkZFHjhyBxgS65joZMWJE3759CSEDBgygW6ZrCPpBIwPMmDED2hnay1+5cgWa92vXrpkim/aiWbt2LWx/Cg0NBccmPT0dVF6f44CEz+eDeBohJDg4GDa08Hg86H1Onz4dHh5OCMnLy0PNXvJf/T04/vnnnwUCQatWrWp7UB07dvT19YVyg9QIHMfRcg4ICIA3O2bMGBcXF6gC4Fa1a9cOLtPpdNBRlpWVgQdCCNm7d++VK1cIIQqFovFs+pVIJCEhIXBMRSPVajV8w/v37x8/fpz82wjTw8MDHkInsKiiT1paGlU/f0aZgxcHRvEhCIIgCIIgCIKYCmYL+T+ovAlVoSHNZX3GzMysd+/ecAxLzIQQpVL5uDR8yAsC5p9at24NYo/0XUAuGjguKCiA99IY4kmemrS0tBMnTsCxra2tl5cXIcTf3x9KICsrCyKpbGxsIIFmq1atwsLCiFFiQUKIi4sLhFfRELKCggKYGX1eK88tDbFY3LZtW0KIpaUl1evj8/nwXnr27Fkjis/Z2RnWD+lLofB4PJoAhM79V1dXN5X2pLCwkC5jtmnTBpKWjhw5khBSXl5OdbRkMhnOwb8UautocxwHerne3t4g4kcI6datm4eHB5gorGUZDAZoTmlynqysLJp3q3379hzH8fl8EJ0jhAQHB8MSa2ZmJsRNQaTfy/rhVPUuISEBgpbLy8tBOfO5bCuAyu7g4DBs2DBq/xBWnZCQAKOg+Ph4WABpsMRoTYjMzEyO4xQKBXRGhBBPT08INTc3Nwcj7NSp09SpUwkhQqEQMmsRQqj8aXBwMKxB0VzJDMPAAqNarT5//jwhpKCg4ObNm3A9jaVsEq1rjVRDdcZ/2tjYQEE5OTlB+RhH8UEVoJljSCOWgkQP6v8DNqpSqZpH8B7FzMwMRF0JIW5ubnCg0WgwaqUhoekOPD09QbqXvgti5KvTbJ5Neg9ednY2DZIJCAiAPUsdOnSAVjIhIQH6EoiFIISIxWLoUXQ6HRwQQpydnaF3oW1rcXHx1atXCSE0yzvypE2Bv78/IcTf35/qR9Ohp4+PTw0Pqh54PB7En7AsS4Pga+ePbrSUlJTQzY1Q1wQCwYABAwghoBcM/6qqqkIPqgGoMwdjjYw61IPq1q3bggUL4KRQKISYPdrA0gkptVoNbemjR4+oSxwcHAwpaOimFJZlIQbp2LFjYBLUB3sp0BEnzZn7Iorazs5u0KBBcMbHxwdKIyMjAwImHz169Bz3XDUz8vLywOG5du0aLVLwoGg25Hbt2kGraGZmBh4+IQRcI0JIQEAA7eYAuVwO40+FQnH9+nVCSFZW1rlz55puKRnPC9dGKpVCQdnb20OguHEoL/XB6Dxyo92Gh1F8CIIgCIIgCIIgpoJrUP+HVquFSZdGu2XtKYDZJj6fD14+IQREfpCGB5TlCCH29vYwkwo5owkher2eznNXVlbCvGmzUfjIzMyEOf7c3FyYLRaJRLa2tsRIwMcYaqI8Ho/jOL1eTzfUZmVlgU4m3dSLADqdjgo3paWlwRKfo6Ojubm58WUikQjmSl1dXekcPJ/Pr3Mrf/1wHAdvimEYGuNX50pC40Qmk1G7KikpYVlWrVbDxn1zc3Na+ziOg0iqkpKSGstrer0eZkb1en2jTfjYVKhn5ZPH40GMmVgs7tSpEyHEx8fHWBwS7i0sLITXoVQqYS++XC6HipCcnAxqe4SQy5cv8/l8+igAqgnEDJNmpDxZG5ZlQb0jICCAxjHqdDoon8TERNDSqEe6A9HpdAzD6HQ6ujCi1WpplwRtqYWFBcQ/CwQC2gjXtqvq6mpYZklISIAGXKlUQvhGcxqIUmiP3759e+g+nJycoPugfYdGowEdyKKiItquYhTfU0KX5mscP/fnK5VK0P0A0ZvmAY/HYxhGIpF0794dzjg5OTWhUU5zwsnJCZatO3fuTMMnAK1WS8PSaCx+s4HK6XTt2hUGKH369IGwGZFIZFy7CSEcx7m7uxvfrlKpdu/eDcfx8fE08gqpYUI06ubo0aPQZ48dOxaEoSgODg7Tpk17Lp8oFothe5tcLqdCTMXFxU3F87979+7du3fheMKECa6urnw+H0TbVCoV3f5x/vx5cLQuXrxYw01Sq9Uw+qEa8ciLQCKR+Pj4EELs7OyWL19OCOHz+bU35t24cQPmoQoLC//66y+wRppTgXLs2DGWZb28vOAaQohUKoVYYhsbG5hW0Gg0zTV0UyAQLFy4kBDi7OxMpeTi4+OhLmzduhUb2H+F7rKjHpRSqYSpPYlEAppynp6eMB1jDJWbo65UUVERdP0bN26ENkSr1YJIcvOD4zi6JWzJkiVwbG1tTbfRAlVVVRcvXiSExMTENP5wfYziQxAEQRAEQRAEMZXGvgZlMBhqr+/Tk0+tTMIwDJ0GoI+qsW+1OWE804/iey8F2L5M6op0otmcm6v5AWq1mobZULEWKA2GYWB5yvjnQw01GAw0xoxlWZp4Fy2qBrTotFotzJLSMLMXsXpPG0zjaJYmar2wmlRb/I0QwnEcWKZAIKgRSUKL1GAw0NllugSn1+uxpX0WaB8N0nnkf7M80bKl5qdSqaAVpe2M6a0EHQM847ii0cJxHJgrhEYTQowzv9EgNLRY0zEemppoObQjowVO00xRM25CXRvtAur84bVzlHMcR8+8iC7ppdCU9kHBOikhpLi4GIJEn1o0r127diDwDQHWYNMgfVNYWNhsKrmFhQXLsra2th07dmw2P6qJMnz4cNCPpso8lJycHFA+JYQ04w0V8+fPhwOxWAyDIR8fH5B09/T0nDBhAvnfhtjPz8/MzEyn040ePRrOuLm5QYB+SkpK/ZnOWyB07L5z507oqj09PSGcz9nZ+Sl2OtVPcXExRP4oFIoHDx7AyaYixFfbMkGcfe3atYQQJycn2mDSbM4ff/wxvR76fuMovkePHsG/tm/fDkOiiIgIKkeJPAXe3t7QRzs4OLRr144QIhAIQEBfrVZTdfJ9+/bFxMQQQvbs2WOK9DboTVtYWBhvCYaYPSpi3vwciVdeeQWio4VCIbS0KpWKNqF79uyB5BOocfpE0L1PWq3WlKZvzJgxcPDTTz/BVr2cnBzYeJacnNzkAkflcvn9+/fhmI5qaHjtokWL3nzzTWIka87j8WAXLiFEKpXS3AM1Hsvj8ezs7EgT2bSPUXwIgiAIgiAIgiCmgh4UgiAIgiAIgiCIqTTSKD4vL68RI0YQQkAREoDlTkJIZmYmRNw9qe6TUCiEMIDu3btDeAaN4lOpVJDIDIWSkeeIi4sLBO+NGzcO1GOpHI0xNAYA7LNO6LYWGj7URKGpnBMTE0H6SSgUUsk+yuzZs52dnVmWpSFVUqkUjs+cORMdHQ0nIyIiqqur69ww2TKh7eThw4chvm7atGmgd8RxHG3xHmeEMpnszJkzcMbLywsUpdzd3eEJxoUM1+t0uiYavEcpLS1lGKa0tHTJkiWEELFYTIWebWxsIBLS2toawk4EAgGEntrb24NGHMdxQUFBcP3HH38MlbRTp06ZmZmEkOvXr1++fBnN8knp2bNn+/btCSE8Ho+qdUHwXkxMzI4dO+BMTEwMCO7VE/8slUqpjPLChQthGADNMiHk4cOHUE0ePnwIgW1N3Z45jgOj7dWrV48ePQghQ4YMgaEU3Xzy6NGjX3/9FY6joqJgQIWDn8dBt+ASQrp16wYZYt544w04Y2tra2NjQ/53q15tQGKOEHLy5ElQ+NRoNBDhRvvEJkRRUdHJkyfh2MnJCQysVatWsNnJyckJcpaQ/27uMs57oVAooCvhOA4KVigUgnGamZlBtveHDx82/kJopB6UWCwGhVzazJH/Zo4Ha3s6g6Nb2aRSKaRIohZvMBhoaDWCPC9EIhGE8zo6OsKwrM4kSBTYM10ndKd7U08VRbefKhQKOu6pHYJfUFAgEAiMt5+amZlBGcbHx9NNJnSTNHpQAB0GyWQyMDalUgmjfx6PV89OZbAupVJJNaCtrKygndTr9c04CwKVJ4ZhjVAopF6oo6Mj+Jz29vY0mxkM6Pl8PnRJDMPQkH164OXlBWXenNJjNCTW1tbGvT8YYVlZGSGkvLyc5vLKycmhuZ4eh7H6uYeHh5mZmVgspi2tSqWCbdUKhaJ5uBAsy4Kt2tvbt27dGn41VGS6+7+6uhq24hBCSkpKmuIIviEx1h6ztrYWCAS2trZ0MlQsFkOqkvqB7GSEkKKiomaQ7kmj0RQXF8NxZWUlqBIYDAboKYRCYQ0lCWMUCkVtlSOa0QSa3PpHSo2lrmHdQBAEQRAEQRAEMZFGugZFJ1Ger4SUjY1N586dCSGOjo6Pk5ZuTvj4+PD5fOMUpdXV1TDbVF1dDROoOHP/QgkJCQkJCSGEWFtbwzxobZO2tbX96KOP4LieSVCtVgsrNkVFRXSOPD8/X6VSGQwGCMMghJSUlNDV2ibNuXPnJBIJx3E0xbC3t/fAgQMJIR06dKBxVnfu3CkrK9NoNBA0hVBSU1NhpnPDhg0w486ybO1UpBSYFJTL5fHx8XBGp9PBdKCHhwfEBVGapfQzrMIZS71pNBqYSc3NzYVJaB6PB5P3EokkPDycECIWi6GOE0JeffVVmI328/ODKWqZTAa5MhUKBa2kCJQtrCnVH/uk1WohaLmqqmrPnj3EKPoXzPVxN9L1/5CQEDc3Nzjp4+MjFAp1Ol1cXByciYqKioiIgDfVPArW2dl51KhRhJCuXbsGBwcTQszNzaGeajQa+LHx8fF0dRSjbx4HXXrq3r07hOwSQoYPHy4UCsViMV1wrseAtVotFdk7f/48HIDZN3W0Wi3Vx758+TLI5L755pvQNUskElirp/VXr9dTpesjR45AsXTq1Anqpr+/P7ScTatnabweVO0UEM+OVCqFBsXe3v65y/s2Qjw8PIRCofHGG7VaDUavUqmaerR3kyAwMBAG/VKp9HGDVxsbm/fee+9fH6VWq6HlpfsACSGxsbGVlZUGg4G6GZWVlc3Dg7p27RohhOM4+nP69u07ZMgQQkhQUBBUZELIn3/+KZPJlEolelA1yMrKggOqNv6kODs7g/5sC9kgQUeZ1ORotaoHCwuL/v37w/G4ceNgHODt7Q1jr/T09NjYWEJIUVERelDG6HQ6KN76e3mdTgcBQqWlpYcPHyaEyGQyGmhaD0KhEHanBAUF0Y1qHh4eAoGgoqICNNAJITExMZGRkaQZZZJwdHSEDBA+Pj5eXl7kf9NnXblyhRCSmppKvVDkcdBEhR06dOjXrx+cHDlyZD3xabWtlzr5169fhxdhSqvS+NHr9dQ5vH37Nmxb6tu3L5QY3b5I669Go8nIyIDrDxw4ABGAdHe3j4+PKZGQjQ2M4kMQBEEQBEEQBDGVppRR96mhSj5SqdTV1ZUQQiNSMjIyYIUxNze3+f1wPp/P5/ONZ/gUCgUspFZXV+Ma1POFZVmQoxEKhVTxzNnZWSKRkOcRL8owDEzSWFpa0hi26upqUKKjW4ElEglMeuXl5TWD7aoGg4H+CroB18rKilZhb29va2truVx+48YNNMIGQ6/Xw0JN81jwfPa6SYsFplSp9kZVVRUsHTePiefnCF035vP5VFyUz+fTLfuAQqGA1dSSkhKIFHpc5B4UOI/HAxEwV1fXNm3aEEJ8fX2h34fXoVQqi4uLaTLQ3NxckBKpR2SlSdCuXTsoAT8/P1h8E4lEsOiRmZkJ6wBKpRJiUDF5bj1wHAdKZpaWlhDC4+npSXVNanfltNbrdDoqMQdRVAzD0AUrFxcX+C8ETTSnEtNoNPCL7t+/D5XU3t5eKpUSQtRqNVRYjUYD8cyEkLKyMljybeqyWM3fg2IYxsXFBY4DAwNDQ0MJIfBqCSHHjx+HpqRZhgJLJBIaCw7k5+ffvn2bEFJUVNSkFbEbIQKBAIJ5nJ2dO3XqBCc7dOgAYb71iOyZCJ/PB7uVSqU0t3ffvn0JIQaDgY4qLl26BE3Yvn37akuENzn0ev3du3fh2NLS8t69e1CRqQc1adIktVpdUFAAeySQhoFGp8DIrCXDMAyt3VqtFlxKlUoFg6rc3FyIb8H2tgbFxcWnT58mhAiFQipTZmtrW8ODys/Ph/1m5eXl9cSjUk1OiUTSp08fQkivXr0mTZoE7QaNoI6Li9Pr9ampqRs2bIAzKpWqSb8aqma2cOFC2Jvg7OxMg5zBCE+fPg2Ok0ql+ueffwjuf64XsVgMJtS+ffuwsDBCiFQqpT1O7aBTWuvlcjm4BBYWFpCYRCAQ2NnZwWUQz08IycjIoDuImgdUVpfuuaX7oDQaDfQRWq229kJFU+8+MIoPQRAEQRAEQRDEVBrpGlSdchzGsvEwTVV/Gk24nsY+EUJ4PB5NHQNzM1qtFuYMmvpiYp2wLEsTlgE0pxBOQT0j1Ajp3KdQKKTyJ7TM6wnee16voMZHwEsnzUhn0rji1/5Ruv+CNtmQ6PX6ZtxyPmkLUHuFmba0er2++YkWPhdo963X66lUyVOH0tHGgcpQUbEo+i4IIWq1GgJQ6eto6u+FJnvlOA5+Mu19qO0Zj3PQGk2BakHTYWRt66XHOp0ODFilUkE5CwQCOGAYhtphszE5Eys1jWeGg2b5qxupB6VSqSB2nM/nU9sNDAyE4x49ekCyrbS0tHoEuLy9vUUikUgkAgEfQoiZmRkEsxYWFsJ2oP3790NuvmY5Ahs/fryVlZVIJKJlmJKScuTIEYJh0M8AFGbnzp0DAgIIIdOnT4cwUY7jQPiImOa96HQ6WPo3VgV9Uuzs7CCZN+y2IoRA1AEhJDY2lqYMb9Kl7evrC8cBAQEQnWJlZUVL+MyZM+Xl5c0sKKLxk52d/eeff5KW50ExDAO9j7+/P4T62Nvbz5s3D/5rbm4Olnnp0iXom65fv26KcFwLRK1W5+XlEUJYlp01axac/M9//gNJYCk8Hg8C0eu3tMDAQNhw4uPjs337duN/3blzh+pSLlu2DML2ms22tEmTJkHCkldffbWGwnBsbCxs9/r5558TExPR5EzE3t7+888/J4RIpVJjKWNjjPcenzhxYt++fYSQ5ORkGLh26dIFtPucnJwmTpwIl1ER82Y830e3wzSDPdimgFF8CIIgCIIgCIIgpoIeFIIgCIIgCIIgiKk00ii+jIwMyJzt4+MzZswYOGkwGP41OKpVq1ZUt3TOnDlOTk4cx4G2KQCqOxcvXrx+/TohJD09vZmp8EmlUqre7uvrK5VKWZalEah5eXmgxddssgc2MK1aterevTshZOTIkT179iSEODg4UN3S2tfn5uZCgFlsbCzYXk5ODqzmKxQKsD2VSgWhpE9Bnz59QLh2xIgRcCYrKws+KCEhoWHKJCwsDKSuSkpKLl68+Lwea21tzePx+Hz+zJkzqT1DXS4vL6fZzY8ePSqTyVrsbpwXARWbMjc3h7DnlpB/vH68vb2hmltbW4PCm4eHR0hICBQXxPURQk6ePAmhuVu2bImPjyfNVOX1+WIwGEApjhCSn58PqSAsLCygI3N2dobIZIVCAZZZXFxMY9J69uzp6elJCHF1dYX/0u4vNzf30aNHhJCdO3feunULTqamptLdQU0OlmUhoDEoKKhLly5w8pVXXoExD+2AMjIyQOQwMjIScjw0y2QtLw46aKQSjsZER0frdLqioqLffvuNGi2UsI+Pj7e3NyEkMDAQzJJqoIPd0pzdWMjNg0bqQUHGBkIIFYI0EZFIRJXKW7du7ebmZqwkodPpYKRVWloKgdFKpbKZxaTy+Xxa7c3NzakEJ6BWq3HHyLMgEokgF5O7u3uNeP06UavV4KwWFBSAcGdaWhqECMvlctDwra6urkelt34cHR0dHBwIIfS1ymQy+ESaL/xFY29vT0ctz9eSBQKBQCCAKH/4IKjLNM05/N6CggLcGP28oAIJhBAejwcFXkNgugUiFothm41UKoWNeW5ubh4eHjUuKyoqgj3lWVlZ6enpaE4mQreUqNVqGF9SSQmhUAjJeZRKJWRxyMvLo+rnvr6+gYGB0DiAB0VtVa1Ww1xVdnZ2amoqnGzS6skMw8BvlEql1Pbc3NxqeFBKpTI7O5sQkpGRQX848kTlDO1ebeFyQkhlZaVWqy0uLo6NjaVGBXYlEAhAxNzCwgJmVWgyKEII1S/B3qrZgFF8CIIgCIIgCIIgptJI16BownutVkvXiGgkyfLly2Ep6datW5Bt09raul27doQQFxcXmj8XlCgNBgPNl/fgwYOjR48SQk6dOnXnzp1m+UaNp5CpvCnyvPD19V24cCH539X5GmRlZUGMKCFk8+bNUVFRhJDq6uoXMfME0RqEkBryUw2Jm5sbKAG2atUKJoljYmKuXr0K/62qqvpXhWKpVAoie0KhcP369XDS1tYW4qYgTJEQotFoQEHr+PHjly9fhpPNL7/7y4XjOFrg7du3HzZsWAv54QzDwFy+SCSitXv+/PkDBgwghNjZ2cGENJXrlclkEBGdmZn5wQcfwPX5+fk4wfykGAwGqqk7Z84cmLx///33R48eTYziSszNzceOHfu4h9A43qKiop9//pkQcufOHWge1Wp184g0sbW1HT9+PCFk4MCBcGDM6tWrwfaSkpJ2796NdvXUqFSquLg4QoijoyMsOGs0GtqLff/995WVlQaDwc3NDc60atUKZHhff/31Nm3aGD9Ko9HQgPO4uDh4QbiHgpKZmRkTE0MI6dWrF0SysCwLza+fn9/gwYPhsujo6JKSkkb4/RupB1U/NPNDPQfkMdljmj1YAg1me1ga/1ofTSyo+uvvEz0KwQbkeZnx406iNTb8q3ku1zRLE0VeulliK/GijbzRFiNG8SEIgiAIgiAIgphK41WSgK2ihYWFSUlJcDIgIAA8URqZZmtrC4InEokENvAZa6dUVlZCOmS6TT85ORnEeZqxPpKFhQXVHoTi0uv1dPssTf2OPB1yuRz25np4eECMGUWv14OcQ3p6+r179+BkcXExBJA049ietLQ0WH8XCoUQB+Xt7U1jHgoKCiCMlp5RqVRwbGZmBhE7NjY2HTt2JITw+XwqSkGtNyUlhdZoiIhITEyk6lLNODsh8ryg/QLHcRCMZ2lpSVV2BAIB7NGHHNkCgQAUzwghTk5OsKdcLpeDQapUKgg5KygogN7EOI4UQ/iekaqqKojHe/ToEQRJ2tjYgGYPy7IgKVHjeujU0tLSIPK/pKQExgwFBQXNo+0Vi8VgkE5OTiD1ZmNjQ3+UTqeD48zMTDigYWPIUwNbIepcjQ8KCoIwPDoQtbe3h8S7NYS7CCF6vZ7G7MnlcnhB2GdRRCIRbAEwXmWCY5FIRIeyNcZajYdG6kGVlJRA1GNJScnWrVvh5Jo1a2ihw0GHDh06dOjwuIdkZGRUV1erVKqlS5fCmaKiIjoaa654enrSSFwYK2g0GlDmISit+8xkZ2dD9vFBgwZRPVlArVZHR0cTQm7cuPHjjz/CyX/dAtQMOHjwIPQ3Pj4+K1asIIT06tULFAsJITdv3oS6TL33/Px8GHF6enrCDIiDg0OvXr0IIQaDge5nkMvler1ep9Pt3LkTzqSnp1+7do0QIpPJUFISMR1qjXSPU4cOHWDPHvmvjJtIJBo5ciR5zIA7Li4ODLiwsHDTpk2EkMzMzPv372PZPl/ozMiePXvCw8MJIUFBQRMnToR3V3tXXk5ODkibbty4EXq3yspKug21eeDi4tK2bVtCiKen5+TJk8n/jtQVCgWMyE+cOAEHdOM38nQwDAODTDpwN95SvmDBAoPBQJXlCSGQdaP2QwghGo0mPz8fzlAXF6E4ODj4+PgQQqhiNsMwUNSOjo6dO3eGkzExMbQYGxUYxYcgCIIgCIIgCGIqjV1JQi6Xg5QZIWTbtm0w1d2nTx9YzRcIBCC3r1Ao8vLyCCFFRUWQSIoQEhUVJZfLtVotXYFpCRIoSqUS9MrIf+MYy8rKzpw5A2doSCTydJSWloKKY2lpKbVMQKfTgRFmZ2e3qKkmjUYD820FBQWwQOfu7g4BUYQQrVYLlbRNmzY0DgqW5qRSKahs6fV6mM7X6XQ0gUlcXJxCodDr9TQbZllZGSxn4SQrYjosy4KqGyHE0tISFLScnZ1piIhYLGZZlmVZmOasqqqSyWTwr5ycHFjZoGtQcrkcWlFcz3+hVFZW1gh24vF4tCmgFBcXQ7ceHx8PK9vNJlKdx+PBZLyDgwM0py4uLrDuwePx6Fr98ePHYeEOFu0JBok9MwaDAayozkTtEGdOc3MRoxRkKpUKCv/hw4cQyU+bC4Ihvk+IjY0NLL0SQpycnKBxhgxvjaiSNvJClMvlELdjbKmurq6wxiqRSKgHBSHpCQkJ1F7Dw8PLy8sNBgPNvtcSqO1BFRYWUg8K8+s9I9SDaq5q+E8B7WYKCgr27NlDCAkKCqJj0Pbt20PcVMeOHaH+mpmZ1chUmJube/HiRXgUHBBCjh492qJqLvKC4Dhu3LhxcGxnZ1dDbpii0WgSEhIIITKZjG6dvXHjRkZGBjHyoPR6fZ3jKuT5IpfL5XI5IaSwsBCkpVsaAoEAYsMcHR0ha7CdnZ2lpSUhRKvVGntQ0BlBcSHPDvWgqC9qvEuHBu/VRqVSwWa8O3fuwPYThUKRlpaGRfoU2Nratm/fHo5dXFxg92lFRUWj2hmBUXwIgiAIgiAIgiCmgh4UgiAIgiAIgiCIqTSljLqxsbGwlrpgwQKavRgChbVabVVVFSGkurqaKneXl5e3wHCLhw8f0lXjV199leM4tVqdlZUFZ3ADCdIApKamUkXd8+fPQyXduHEj1F+WZWskyFOr1aCtZzAYqMgequ29FLRaLc3+HhcXd/bsWUJIx44dbWxsmu4vWrJkCRyLRCLIe1EbvV4Pqm5KpZJG28vlcmgzFQoFbGPAzQxIwzBu3LigoCBCSNu2bbt27UoIYVkW9npdu3YtIiICLrt161ZOTg4W13MkPz//gw8+IIS4urrCK/Dw8LCzs4P/jh492jhrDiEkOzsbYn3PnDmTnp5OCElOToZIYL1ej4Oup0OlUtF2uKysDI4bm7hxU/KgaJjvw4cP0bweR1VVFTiThBA6EkKQhsR4IoNuiEKaBDUE5UGYp0lPRRkMBro5FlTL67xMr9fDDlKNRkPzOyHIy8LJyQmEnlu1agXDdzocLy0tpXlZKioqcIz+fFGr1bAlsrKyEraisSxL28Dacyh0rJ+WlpaYmEgIyc/Px21pz4hxT2S88a9RgVF8CIIgCIIgCIIgptKU1qAQBEGQFw1VoEpJSTl//jwhJCQkxMHBgRCi1+shZCgrKwuyRDSJnNGNTQMXQf6VrKwsyJwrkUiCg4MJIbGxsZBRt6ysDCJOkRcEaPGlpqaCfPGRI0eoZPmSJUtqRKHrdDrUkX86jhw5cvXqVUKIl5cX5D3nOA5ynOTl5V25cgUuy87ObpxR/ehBIQiCIAiCNHZwM97LLXlCiF6vr+FBGQwGfCPPXrBNEYziQxAEQRAEQRAEMRVcg0IQBEHqIDExEWL27t+/D/Kn5L96nuXl5ZAkHidfEeRFcPHixZs3bxJCjhw5sm7dOkJIZWVlbm4uIQTStiINCW3o6pQ0wGbw6cjLy4N41BkzZggEAkIIwzA8Ho8QUlVVRaOvIWK8EYIeFIIgCFIHlZWVIE+Xl5eHpYEgDQlNCIE0KtBZeo4olUoQPoXJgiYHRvEhCIIgCIIgCIKYCnpQCIIgCIIgCIIgpoIeFIIgCIIgCIIgiKmgB4UgCIIgCIIgCGIqz1NJwtHRsXfv3limhBBra+saZ6ysrLBwAMib9lLo2bMnKIkhZmZmWH9Nr78NA5/Px1cA+Pj41Fl/YdsxUrv+NgwWFhZoooCrqyvW3yeqvw1D+/btQdWthePs7Fz7ZEBAAJro4+rvU/A8PaiwsLCwsDB8MXXSqVMnml8ZeVmcPXsWCwHrb6PF2toaW4l6iIiIwEJ4ubRp0wZN9HFIpVIsnJfO5s2bsRAex4oVK7AQniMYxYcgCIIgCIIgCGIqz2cNysnJSSaTYWkipuPg4NCQH2djY4MmijwRUqm0wT7L3Nwc7RN5IhiGsbOza7CPc3d3RxNFngg+n9+QH+fv748mijwREonkWW5/Ph6UpaUlvgmkMdOQQw0EeVJ4PJ69vT2WA9JoMTc3Nzc3x3JAGi22trZYCEhDglF8CIIgCIIgCIIgpoIeFIIgCIIgCIIgiKnwmsS31Gq1jx49Ki0tFQqFXl5eLMtqtVorKyuO4/AVNjYMBkNFRYVer5dKpQzDtJBfXVVVlZKSUl1dLZFIvL29lUolwzANuZEGMR2tVltZWclxXIsKPy4oKMjOztbpdPb29s7OzgqFQigUYlxW46S6urq6ulosFotEohbyk/V6fUZGhkwmY1nW3d3d3NxcpVJZWFigMnXjpKKiQqfTSSQSHq9pDCOfHZVK9ejRo4qKCjMzMx8fH51Op9frraysWBaXIhoder2+vLycZVlLS8sXNxBtAi++vLx8xowZffv2HTRo0MCBA0+ePPn2229369YtOTn52R+uVCovX75sMBjQ4J4XxcXFr7zySr9+/crKylrIT05MTBwyZEj//v0HDRo0ZsyYy5cvh4aGPi9lcJlMdufOHbSr58itW7f69Onz/vvvt5yffPDgwd69ew8YMGDgwIHz588/ePBg9+7dn5ey7YMHD7Kzs9GuniO//vprt27dtmzZ0kJ+r1qt/uKLL3r37j1w4MCBAwf++eefn332Wbdu3cLDw5/94VqtNioqSqFQoF09L7Ra7YwZM/r06RMbG9tCfnJ+fv7kyZP79es3aNCgIUOGnDt37pVXXhk0aFBeXt6zP7yioiIqKgrt6jmSm5s7aNCgiRMnlpeXv7hPaQKTB1u3bt2xY4dIJBo4cKCZmZm3t3dWVlZKSsqzp0bNzs6ePn26Tqc7c+ZMy5lHedFotdr09PS8vDydTtdCfvKiRYuioqJatWrVtWtXW1tbc3Pz1NRUoVD47E++cOHCzJkz33jjjU6dOqFpPS8UCkVKSsrLSpvb8FRUVMycObOkpKR9+/Y+Pj6dOnWqrq5OSUkpKCh49oevX7/++++/37VrV6tWrdC0nhfFxcUpKSklJSUt5PdeuHBh3bp1LMv26tVLKpWGhIRs27YtJSWlsrLy2Sv7u+++Gx8fHx4eLhaL0bSeCwaDAYZh1dXVLeQnr1+//siRIzY2NqGhoUKh0M3NLSMjo7CwUKPRPOOT4+Pj3377bT8/v+7du7ecsJ0XjUajSU1NraqqeqED0SbgNly9elWn0y1btmzhwoVQdRcvXlxQUPDsHXZaWtqlS5e6deuG1oY8NRUVFbGxsRzHHTp0qEOHDgaDoby8fO3atc8lxPTy5ctJSUm4Roo8C3FxcSUlJb6+vhcuXADx1sTExB9//DEgIODZH37w4MGWM9BHXhDXrl3TaDTvvPPOpk2boOXkOK5Pnz6dO3d+xifL5fLDhw+7ublhISNPjUqlunXrlsFg+OmnnyZOnEgIUavVy5Ytq66ufnYBwLi4uJs3b3p7e2M5NzmagAcFs1CBgYF0SDphwoTn8mSDwYBjU+QZUSqVWq2WZdk2bdpAPLSNjc277777vEwUSxh5RiorKw0GQ6tWraysrOBMmzZt2rRpgyaKNBLkcjkhxN/fn2YQGjJkyJAhQ56LfaKJIs+IVqtVKpWEkKCgIBiImpmZTZky5bk8HO2z6dKoPaiYmJiYmJicnBxCyPnz52Uymb+/f9euXU+dOlVUVDRmzBhra2uDwRAREVFcXBwWFnb58uXjx4+7uLi8/vrrvr6+Wq32/v37p0+ffvTokUAg8PPzGzx4cJs2baACnD9/PiIiwmAwFBQUbNu2zcHBYdSoUS/rlxoMhvT09DNnzsTGxmq1WldX10GDBnXu3LnGNuKqqqqrV69evHixsLDQ2tp6wIABffv2tbCwqPG0/Pz8s2fP3rhxQ6FQuLu7Dx8+vGPHjsZhiiqV6vbt22fPns3KyjIzM+vatWtoaKiTkxO9IDMz8/z58yEhIa1btz579mxkZGR5eXnr1q1HjRoVEBBgvG/SYDAkJSUdOXIkKSnJ2tp65MiRLWcqRa/XnzlzJikpSaFQGAyG3bt383i8AQMGWFlZnTx5kuO4V199lRAil8tPnDhhaWnZp0+fLVu2xMXFdejQYfLkydbW1gqF4sqVKxcuXJDJZBKJpEOHDoMHD3Z1dSWEVFdXnzhxIiYmBirC1q1bg4ODO3bs+LJ+rFarvXfv3pkzZ9LS0vh8vr+/f2hoqL+/f42ltoKCgrNnz966dUsul3t4eAwbNiwkJKRGXkW9Xp+SknLy5Mn4+HiGYYKCgkaOHOnl5WUcwFBWVnbp0qXIyMiSkhIHB4e+ffv26dPH2NSjoqISExNHjhyp0+kOHTp0//59QkjHjh3DwsKcnZ2NP06tVt+6dev48eMymczLy2v8+PEtp8eqrKw8ceLErVu3CCGFhYVbt24ViUSjRo3Kzc29evVq69ate/fuTQjJyMi4ePFi+/btra2tf//999LS0tDQ0JEjR/J4vIKCgoiIiJs3byoUCnt7+z59+vTu3RsUOHJycs6dOyeTyfR6/enTp9PT04cPH+7o6PiyfqxcLo+MjITm0crKqlOnTgMHDnRxcanR0iYnJ586dSohIQFsLywsrIbtQQsZHR196tSpnJwcc3PzHj16DBkypMZkc3Z29unTp+/evavRaDw9PYcMGdK+fXtq6lqt9sSJEyqVavz48Q8fPjx69Gh6erpEIunfv//AgQNrCHhUVlaePn368uXLGo2mY8eOY8eObSH2Sf4bCfLgwQNCSHR09NatW52cnIYOHXr58uW0tLR+/fp5eXkRQq5fv56cnDxo0KC0tLS9e/eamZm9+uqrnTp10uv1ycnJp0+fjo+PJ4R4enoOGjQoJCQE9Cdu374NMSwVFRV79+61t7efOHHiS4zYLywsPH369K1bt6qrqx0cHHr37k1rE0Wj0dy6dSs8PJza3uDBg2svdJSVlV24cCEyMrKsrMze3n7w4MF9+vQxDh3X6XRxcXGnTp1KTU3l8/lt27YdOnSoce9cUlJy8uRJNze3Hj16REZGRkREFBUVOTs7jxo1qkOHDjVKKTs7+8iRI7GxsbCfYsCAAS3HRK9evXr//n3I1Xvs2LE7d+507NjRz8/vxIkTSqVyzJgxFhYWOp3u+PHjOp1u8ODBx48fv3jxoo+Pz5QpU1xdXVUq1Z07dyIiIjIzM8VicWBg4ODBg318fFiW1ev1p06dunTpElSErVu3enh4DBw48GX9UoPB8PDhw9OnTz98+BBq0+DBg2v34OXl5RcvXoyMjCwtLXVwcAgNDe3Ro0dtzZusrCxoIdVqtY+Pz6hRowIDA41HjwqFIioq6vz58wUFBRKJpGfPnoMHDzaOrk9ISLhx40bv3r3t7OxOnjwJ3VBQUNCoUaNqjDN1Ot39+/ePHTuWkZHh4OAwfvz4BtLxMjwJhBCxWBwZGWloEFasWMHj8aDEeTwej8ebNWuWQqHo0qULISQmJsZgMKhUqtDQUFtb29WrV0OUM8MwX375pUajWb58uYWFBZ/Pt7e3t7Oz4zhOKpWuXLlSr9frdLrx48fzeDyGYRiG4fF4Xbt2Nbw8jh496uLiwnGctbW1o6OjQCAQCASvvfZaSUkJvSYjI2PEiBE8Hk8kEjk6OorFYo7jhg8f/ujRI3qNXq8/efJkmzZtGIaRSCSOjo48Hs/c3PzTTz9VKpVwTUFBwdSpU0UiESTxtLCwYBimTZs2J06c0Ov1cM2hQ4cIIfPmzXvllVd4PJ5YLIYqZG9vv2PHDnqZVqv9448/HB0dGYaxsbGxsrISiUTTpk1zc3MTi8UymewpimLXrl0sywYEBDzFvWvXriWEhIaGNsxbU6vVAwcONLYiHo936NChuLg4a2trJycnuCw1NdXFxaVPnz4zZsyASicUCi9evFhSUjJ27FiWZc3MzBwdHa2srBiG8fT0PH36tMFgyMnJad26NQ1o4fF4n3/++cuyT4hYMDc3p7WJZVlra+tvvvlGo9HQy86dO9euXTtqe3w+38LC4sMPP4Q1EEClUn377bf29vYMw9ja2tra2jIM4+rqun//fmpXN2/e7NmzJ8dxIpHIwcFBKBTyeLyhQ4cmJibS58ycOZMQsmnTpnbt2rEsa2FhwXEcwzBt27aNjY2ll1VWVs6bN8/MzIzjOHt7eyjqmTNnCoXC3r17P0VRaLVaiCzasmXLk94rk8kgcC48PLxhXlxycrKLiwuYKMuyPB7PyckpMzPzjz/+YBjmrbfegssOHjzIcdz8+fNp0JSHh0dpaendu3cDAwMZhrG0tHRycoJi7N+/f2pqqsFgOHPmjFgsNjb+S5cuvSwTLSwsHDt2LI/HM65NPj4+Z86cMTbjjRs3Ojg4sCwLtseyrIuLy19//UVtD17Tm2++CVbn4OAgkUgYhunUqVNCQgK1gR07dnh6ekLJQOdiYWExf/78iooKanjt2rVr1arVpk2brK2t+Xw+lBXHcaNHj5bL5fTj0tLSBg0axDCMSCSyt7fn8/khISEQKbRy5cqnKApQXwgICHiKFvjPP/8khHTu3Fmn0zXMi9u3b59AIIBeHhq6wYMHq9VqmH7avXs3XDZr1iyRSLRq1Rnh2EsAAIAASURBVCqaeHrixIl6vf6vv/6yt7fnOM7Gxgb6O7FYPGvWLCjhpUuXGrfPTk5OVVVVL8tEb926BSNIKysrR0dHMzMzlmX79OmTk5NDryktLX333XeFQiGfz6e2FxISEh0dbfyoe/fudevWjWEYsHaw1alTp5aXl8MFcrl8yZIlEokETB2qg7Oz8x9//EFN/c6dOxYWFmPHjp0/f75QKBSJREKhkGEYc3PzVatWGRtAeHg4zDJYWlra2NhwHDdhwoT27dsLhcKnGw2GhoYSQtauXfsU98Ic7q5duxrsxc2YMQOsiA5Ev/7668LCwtatW1taWqalpRkMhqqqqrZt27Zu3Xrp0qXgx3Ict3nz5qqqqnnz5kHZOjg4QJvj4OCwefNmvV6vVCqhpyOEQPsME3wviz/++MPa2prjOFtbWwcHB2hO58yZU11dTa+Jj4/v1asXy7JisdjR0VEkEvH5/Ndee62goMD4UTt27HB3d2cYxsrKysHBAYbf33//vVarhQtgQMvn8wUCgb29PTSPnTp1ioqKog/5z3/+Qwj56quvBg4cyHGcubk5vAg3N7eTJ0/SyzQazTfffCOVSsHaJRKJRCKZO3euRCLx8/MrKip60nKIjIwEb+Jfr2zUHlR2dvaNGze6d+9OCNmwYcONGzfS0tLq9KCEQqGVlVW/fv2WL1/+1ltvRUdHnz9/3tzc3M/P78yZM4mJiQ8fPvztt99sbGwEAgE0RklJSb/++it0V9euXYuLi3tZVltRUeHi4iKVSn/77bcHDx4kJyeHh4cHBARwHPf999/DNQqFol+/fizLjhgx4uLFi8nJyTdv3pw8eTLLsoMGDSotLYXLHjx4ACPOefPmRUdHJycn79+/383NTSAQwIBPq9WOGTOGYZiOHTsePnw4MTHx7t278+fPFwgETk5O165dg+eAB2Vtbe3h4fHTTz9FR0dfvnx5woQJDMN4eXnl5eXBZRcvXrS0tJRIJGvWrHnw4EFsbOynn35qbm4OtavZe1B6vT4+Pv7UqVO2trZ8Pv/KlSs3btwoKSmp04OSSqVisXjq1KlffPHFe++9V1pa+tVXX7EsO3r06Bs3biQnJ8fGxi5YsIDH47Vp00Yul6vV6piYGIgGnDFjxo0bN7Kysl6WiZ47d04kEgUGBp48eRJq06ZNm6RSqUQiuX37NlwTGxvr6upqZma2cOHCu3fvJicnHz9+vFOnThzHffrpp7Td/PPPP4VCob29/Y8//piQkBAfH79q1Spzc3M7O7v4+HiDwZCenu7r68tx3KRJk65evZqUlHT+/PmhQ4cyDNOjRw/aGoIHZWNj07Nnz8OHD8fExBw8eLB9+/aEkPHjx9Puf9WqVTwez8vL6++//05MTLx69eq4ceP4fD7DMC3Bg1IqlXfu3NmwYQPDMN27d79x48adO3dUKlWdHpS9vb2jo+OCBQsWLVq0evVqnU7Xq1cvoVD45ZdfxsbGpqSkXL9+feDAgQzDvPvuuwaDoays7ObNm+3atRMKhZs2bbpx4wb1Hxqezz//nGXZV1555ebNm8nJyffu3Zs3bx6fzw8MDKTd/59//gkjzt9++y0hISEhIeGXX35xcnISi8W0P1ar1dOmTWNZNjg4+Pjx40lJSbdv337ttdc4juvWrRvMFxw+fNjS0lIsFn/xxRexsbEPHz7cvXu3n58fwzBz586F54AHJRKJJBLJ22+/fenSpejo6I0bN9rZ2TEM88svv8Bl1dXVw4YNYxgmNDT0ypUriYmJhw8fbtu2LawAtAQPqri4+MaNG6+99hoh5IMPPrhx40ZCQoJer6/tQcHArn379p9//vk777xz/PjxtLQ0Ozs7JyennTt3xsfHJycnHzx40MPDg8fjwY2ZmZmnTp0SCoXu7u6nT5++c+dOg/2uGqhUqj59+rAsu3Llyvv37ycnJ0dGRg4YMIBhmPfff5/a3jvvvMOybJcuXU6ePJmUlHT37t2FCxcKhUJfX1/a/hcUFEAs2aRJk65du5acnBwRERESEsJx3NKlS6Fj+vjjjzmOc3d33759e0JCQmxs7Ndffw1Ge+DAAXgOeFBSqdTOzm7lypU3b968fv36/PnzYbBLfba0tDRnZ2ehULhgwYKYmJj4+Pgff/xRKpVyHNdCPKjU1NQLFy4EBQURQvbu3Xvjxo2cnJw6PSixWGxpaTlixIjly5e/9tprWVlZu3fv5vP53bt3v3jxYlJSUnx8/DfffGNhYWFjY5Oenq7X6+Pi4lavXk0IGTJkyPXr12Hb80shNTVVKpU6Ozvv3bs3ISEhKSnpn3/+cXV1FQgE+/bto10YtE5vvPFGVFRUcnLylStXhg4dyrLsq6++Slvay5cvm5ubW1hYfPnll/fu3UtKStqyZYu1tbWFhQW0tHK5vGfPngzD9OvXD4boN2/efPPNNxmGad269cOHD+E54EHZ2NgEBgZu3br17t27Z8+ehTW6Ll26lJWVwWX79+8HB/WXX35JSEi4e/furFmzYF6mRXtQwNChQwkhp06dgj/r9KAIId26dSsuLjYYDCDSv3LlSkLImjVr6HP0ev1nn33Wvn37/fv3w5lLly7x+fw+ffoYz6M3PNeuXYP6YzwPeuLEiYCAgCVLlsCff/31F8MwvXr1Ml6VqqysHD58OMdxdDBHB9wqlYpetm/fPpjrqqqqCg8PZ1nWw8PDeDpfrVYvWLCAEDJmzBi4ETwoc3NzY0c/Ly/P39+fYZjz588bDAaNRjNu3DiGYVauXEm7Ja1W+/HHH4OdNHsPCsjPz3dychIIBHSVr04PihAyf/58cCTA3nr37s1xHPVAwJgHDRrUt29f2oIsW7YM5mAMLxX4GuvWraNndDrd0qVLQ0JCDh48CO99ypQpDMN8/PHHxrUpISHB3d3d0tISOoaysrJ27drx+fydO3car2SCsPgnn3wCfT8hZOTIkXQ+1WAwFBQUQIP7ww8/wBnwoIwHFgaD4ezZs2Kx2NraGprynJwcT09PMzOzCxcu0GtKSkp69epFCGkJHhRw6tQphmGGDh1Kz9TpQQkEAnibYKKlpaUCgcDT0xOCVIGUlBQ/P78pU6bQufzu3buLRCLjEm54VCpV3759CSE1lh+HDBnSvXv3zMxMg8FQWFgYEBAgFAqPHj1qfO/u3btFIlHHjh3BIK9fvy6RSOzt7Y1byNLS0nbt2nEcd+HChYqKip49exJCvvnmG+Ph+I0bN+zs7AQCwb179wz/9aBgqUStVhtXJYZhxowZQ9t5kUjk5eVl3LDHxMRAKEtL8KCADz/8kBCyfv16+LNODwqWRpOTk+ECnU538OBBQsibb75p3HVu2bIlKCiIjs7z8/NFIpGfn1+NOfIGJjk52d7e3s3NzbhgY2Ji2rVrN2nSJPjzwoULFhYWrq6uxqtSGo1m9uzZLMsuXrwYznz33Xcsy/bv3994Pe3u3bt8Pt/Dw6OwsDA2NtbW1lYkEtEpUSixn376iWXZdu3awUwHeFAcx/3888+0ADUaTZ8+fYyd/I8//phhmClTphib8ebNmyGYoiV4UFCdu3btSgh58OABnKnTgyKEjBo1CmIuoB+EwJOdO3ca9yDvvvtux44dL168CGf27t1LCJk8ebKxGTc8e/bsYVl2+vTpxic3b94cFBREp/JXrFjBsuzYsWONO4XS0tL27duLRCI6XBwyZAjLsl9++SWdOTUYDOvXr6fzm7///jvHcW3bts3Pz6cXKJXKiRMnwpwCFAV4UHZ2diDjAaSlpdnY2Jibm0Nrr9frIc4QlvXoo9544w1CyIv2oJpPIrDRo0fb2NjAYigsHRJC9u3bBztJILrviy++uHr16vjx4xvVN4evevfu3UOHDqnVajg5dOjQmzdvgh8IP4Rl2Xfffdc4SNTCwmLatGk6ne7o0aPgNx49elQikUyfPt04C+Hw4cP37Nnzww8/CIXCgwcP6vX6SZMm+fn50Qv4fP6cOXOsra0vX75snNygdevWMC4BHBwcWrdubTAYiouLCSEVFRXnzp1zcHCYNGkSjW3lOG7atGnwIpAavPnmm7BeDxPMVlZWer3+xx9/zM3NhQsEAsGRI0dOnz5t/HYaj4keOHDg7t27BoMBatlXX30VGRk5ZswYQohcLj9+/Litre1bb71lHEAfEBAwePDgiooKGNUlJiY+ePAgICAgLCyM7jzhOO7jjz/ev3//O++8o1arjx07JhAI5s6da7w9wMHBYdasWQzD/PPPP8bipEOGDDHW5AwICLCzsysrKwOF2ejo6IyMjK5du8I6NmBtbT19+nTMgVgbOzs7uhcU4jf4fH5BQcHmzZthoz8hxMfH5/bt23/88UejEobm8XiwR27Dhg35+fm0efznn3/Onz8PFnLnzp2kpKTOnTsPGjTI+N6xY8e2atXq/v37EPp/9uxZuVw+ZswYX19feo1UKv3xxx+PHj0aHBycnp5+584dNze3t956y9iKunbtGhYWptFo/vnnH+PnT5061XgXQdeuXfl8flFREfx56tQppVI5depU44a9ffv2MG+I1KB3796tW7cmhEBUKrRLly9fjoiIoM3C1KlTr1+/DnOCjQexWCwQCPLy8v744w+amSo4OPjq1avbt2+HP48ePVpVVTVlyhTjzXs8Hu+9994jhBw/fhyGB0eOHNHr9bNnzzaugyEhITt27Ni5c6elpeX58+dLSkpCQ0ON2z2GYd577z0XF5eUlBTjBINisXjy5Mm0NebxeF26dDEYDGCisL2Hz+e/++67xmb8+uuvY/aCOpk4cSK0RbSXJ4Rs3bo1KSkJLuA47scff7xy5UqfPn0a1TeHzLPnzp27cOECrU3Tpk27fv36vHnzCCF6vf7AgQMcx82ePdvMzMy4eZwyZYpSqTx+/DghpLCw8Nq1a1Kp9O233zbeJv3222/v3Lnzq6++gn7cYDC89957xvtmhULh0qVLGYaBneH0fHBwsHE2F0dHx9atWysUCqhHqampEP8yfvx4asZCoXD27NkNsOOx+SRBCgwMNP5z9OjRP/7447179zp37uzt7T18+PDhw4f36NHjheYnfjoCAgKGDx9++vTpiRMn2tnZDRo0KCwsLDQ0FMLr4Zq7d+8yDHP69Om4uDjjeyGjS2pqqlwuVygUMpnMzc3Nx8fH+BqJRDJp0iQ4fvjwIY/HM25YAScnp9atW9++fTsrK8vDw4NaqvGOZ5Zl4U8Ynqanp1dWVnp6eoLyAcXV1bVVq1YpKSnYmBojEAiMx2SEkFmzZl28eHH79u179+7t0KFDWFjY8OHD27ZtW3tH5kvnlVde+fHHH2/dutWlSxdvb++wsLCwsLDu3btDmD5YYHl5uVQq/eWXX4zbVkII9ByQePHBgwd6vd7Hx6dGLiYfHx8wWrlc/ujRI3t7+9o+ZHBwsKWl5YMHD3Q6Ha0XNbaTwv4BmOcjhNy/f99gMAQGBtYo0nbt2tXYyo8QQlq3bm08SBIKhbNmzfr+++8XLly4bNmyPn36QCvq5ubW2LLnwezS5cuX//rrr507d3bu3DksLGzYsGFBQUH01cfExOj1+qKioi+//LLG7UqlUqfTPXjwoE2bNmAzHTt2rNFN9OvXDw6uXLmiUqkCAgJqb+7v2bPnjh07QNSEUsNEzc3NGYaBJtRgMEB7Xlshplu3bvv27UObrEGNXr5bt269evW6du3a8OHDQX8iLCysf//+NjY2jW2KxMXFZeLEiT/++OOsWbMWL17cr1+/sLCwIUOGGNcmCKuJiYn56KOPjO8FaykqKsrJyZFKpZC9urbOO6zaEUIghKF2ciE+n9+lS5fDhw8nJyf3798fTlpZWdWY8YSEB/CheXl5hYWFfD4fFlQpZmZm7du3P3v2LNpkjRKu0XO98cYbO3fuPHfuXGBgYGBg4PDhw8PCwjp16lRbAOylAxIs0dHRgwcPdnFxGTJkyPDhwwcMGGBtbQ21KTc3F5aM9u/ff+bMGeN7YbwXHx+v0+mSkpLUarWLi4unp6fxNdbW1rAuZDAYkpKSWJatnUnIy8vLxcUlLy9PJpM5ODjQJtTYkmGTCO3l4+Li9Hq9h4dHjQbZ09PTWB3tBdF8PCio9hQPD48jR46sXLny/PnzaWlpGzdu3Lhxo4uLy7Rp02CvTuP55rAkvWbNmr///jsvL2/v3r179uyxsLAYN27csmXLfHx8qquroY8/efJk7Y7BysqKZVmdTgdRoQKBoJ7p4bKyMo7japQV1Hxzc3ODwVBRUUFPPi4nLKxCwMcJhcIal5mZmWHiwtrA9jDjMyNGjNi3b98333wD6SCuX7++bNmytm3bfvLJJ5MmTXou6aSeF+7u7kePHl21atWlS5fS0tL+85//QG2aMWPGhx9+aGlpCTvxKisrd+3aVft2Kysr6I8hOzjV1K5NVVWVRqMRCoU13DCwK6FQWFpaqtfrTTdRQkgNqSs4U0NcCKndhBJCVq9e7eTktGXLloyMjBMnTsBs9ODBg7/66iuIaWk8jB07dteuXd99911cXNz169evXbv25ZdftmvXbsmSJa+++irLsqWlpYSQjIyMP/74o/btlpaWkBsUGsB6dJzAqEBVpca/YCuzcRP6OBMF+9RqtTCNWttEnz3JTEswUQsLi/37969evfrYsWMgNfnXX39ZW1u/+uqrX3zxRQ0ZxpfON99806pVqz/++CMrK+vYsWNHjx4VCAQDBgxYs2ZNhw4d1Gp1VVUVIeTq1avXr1+v/cPNzMyUSmV1dTW0pfW0otDM1hkJAvLFxib6uAk7MFFISAriQMb/ZRim5WQkNx2BQFCjvoeEhBw+fHj16tXXr1+Pj4+/f//+unXrPD09586dO3v27EY1WyoUCo8dO7Z8+fITJ04UFhZu2bJly5YtUqn0jTfe+Pzzz52cnORyuUaj0el0Bw4cqH07DETVanVFRYVer6/HPqurq9VqNcMwtZtZHo9naWlZWFhIo7GgVOsxUWiQa3+cUCiUSCQvNJ0uaU4eVO2VpTZt2uzevRvkKM6dO3flypXk5OSvv/66qqpqw4YNjerLW1tbr1279tNPP71z586lS5fOnTt3//79HTt2pKenQ2Aex3Fisfivv/6qsY5BbcXKygoibbRarVqtfpyLKBaL9Xp97TziNN2B6b4lXY9Sq9XGc9I6nQ7mBpB/ZcSIEcOHD09ISIiKigLN6AcPHkyfPl0kEo0bN65RfdW2bdvu3bs3Kyvr1q1bERERly9fTklJWbFiRVVV1TfffAM9gY+Pz44dO+ocMkJbCTZTTxp7kUjEcRwYVY1/qVQqjUYDij1PZKIwLjFGo9EYu2EIULtgBQLBokWLZs+eHRMTc+XKlYiIiJiYmFOnTiUmJoaHh9dY637pX3706NEjR46Mj4+PiooKDw+/efPm/fv33377bdjbDT75uHHjli5dWuftsJYOsz80zqo2cIFKpdJqtTXW4kD/zfQmlMfjweCgtomqVCo0SFNM1MXF5eeff16+fPnt27fPnz9//vz5hISEX3/9NS8vb8+ePbUnYl4iQqEQalN0dPSVK1fOnj0bExMTHh6emJh45coVFxcX8MnXrFljHDxP4TjOy8urqqoKrE6hUDwuWh5MlEbeGgPZNU2f4qQNskKhqDFIrd1EI3XSrVu3o0ePpqamXr9+/ezZs9euXXv06NHHH3+s1+trLDa+dJydnX/99VeZTHbr1q0LFy5cvHjxwYMHv/zyS25u7p49ewQCAciHnDp1qs5WTiwWC4VCWGavp5cXCARgw7XbPZ1OV1VVxefzTZ/ifFwvr9PpNBrNi16Lbp6bAQwGQ1ZW1vXr16uqqlq1avXKK6/88ssvV69eXb16tUAg2Lt3r6ExJYQpLi6+c+dOdna2VCodNGjQihUrLl68+Pfff9vY2Ny4cSMlJYXjOA8PD2iwgv8XNzc3lUoFuqiOjo4WFhYVFRV0Xw0gl8tnzZo1c+bM4uLi1q1ba7VaSJ1hTGlpaWZmppmZmenzdrBHXyaTQSQhpaSkhEb5I49Do9EkJSXdvHmTZdmgoKB333133759V69enTBhglKprLGV4qXXpszMzJs3b1ZWVrq5uY0fP37Tpk1RUVGrVq1iGAZi9z08PAQCQWVlpa2tbQ0TtbCw0Ov10GfDcnxWVlaNEerDhw8nTZq0evVqPp/fqlWrkpKSrKysGl8jNTW1oqLCw8PD9NW5gIAAhmGSk5NrdPZZWVn1tO8IbTdiY2MTExPNzMx69OixePHi8PDwiIgIPz+/tLQ00L9pPLUJZEUNBkPbtm1nzJhx4MAB2PKqUqn+/vtvQgj4e8XFxf7+/jVMVKfTQVIB8t+gu8TExBofceDAgUmTJh06dMjLy4vH46Wnp9dYayKEgIaE6QnxGIaBmJ8asdnkv7GvSP0UFBTcunVLJpPZ2dkNGzZs7dq1kZGRW7ZskUgkERERxlspXjqVlZWxsbFJSUlmZma9evVaunTpmTNnzp496+3tnZWVdfnyZZZlIeqpoqKihn36+flVV1eDbLRUKgXHqbaJLl269O23305ISICtYjWiSQkhBoPh/v37IDhh4td2dna2srLSarU1Pk6r1T569AgtsH70en1aWtrNmzeVSqW3t/frr7++ZcuWq1evLlq0iBACAhKkrnmBl0J+fv7t27dlMpm9vX1YWNi6deuuXLmyfft2Ho939erVjIwMGGFCa1nDRO3t7WF+k2VZd3d3Ho8nk8lgwzylrKxs8uTJS5YsgQED5Cur8R0KCwvz8vJsbW1Nz+bk7+9PCMnMzKzRIJeVlRUWFr7oQmu2HtTixYsHDhwIO9sAS0vLIUOGmJmZqdVqmIHmOI7juBe9zPev7N27d8CAAVQ0ghDC5/N79eoFuj2wnjN8+HCNRrN161ZYKaL1c+3atQMGDFi+fLlerxcKhf369SsrKzt58qTx8x8+fLhjx47Lly8zDBMWFgZhrLD0STl69GheXl67du1Mjxy1trbu2bNnXl7eiRMnjM9fuHCh9vAXqUFJScm4ceNCQ0ONpTtcXFxAqZO+ZZiJebkmqtPpPv7444EDB546dapGbRKJRFCb7O3tO3funJ+fD2Ju9LLKysrp06f3798fjCQoKMjFxeX+/fv37t0z/ogTJ078/fffycnJQqFw4MCBCoViz549xr9apVLt3LlTq9VCAi4Tv3mXLl0cHR2joqIgXyf9OX///TdOoP4rN27cGDBgwHvvvQdRQ4QQhmGCg4PbtWsH8ndwEl7Hy13TKywsHD9+/IgRI4xbnlatWsFmD/iq3bp1s7W1vXbt2t27d43vjYuLGzFixKBBg6Am9u/fXyAQHD9+3Lg/1ul0W7duPXjwYFVVlbe3t7+/P4jsGz8nNzf3+PHjoEtu+jcfOnQon8/ftWuXccNeXl4OgoFohPXz008/DRw48Oeff6ZnRCJRv379YDwHDQjLshzHgXDWS/yqV69eHThwIE0JSAjhOC4kJKRNmzYGgwGao9DQUB6Pt337dlgpouzZs2fgwIHTp0/XarUcx0FGgd27dxtfU1JS8vvvvx87dozjuL59+1pYWJw4cQJ2TFHOnTuXkpJib28fEhJi4tcWCASQnmvHjh3G5+/duwdbW5F6qK6ufv/99wcNGnTz5k160tbWNjQ0lM/ng+4x+W8vDzpyL/Hb/uc//xkwYMCvv/5qXJvCwsLMzc1hIGpubt63b1+tVvv7778bN/h6vX7x4sUDBgyAmujp6enr61taWmo8YCCEREVFHThwICoqSiAQDBs2jBCyY8eOGovtf/31l1qtfqKBaJs2bUCSF9SqAIPBcPjw4Rr16EXQPD0olmUHDBigVqs///zzW7duwcnS0tINGzZUVlaGhobCNDZM6iQlJV2+fNl4jNXAQAu1f//+bdu2wWBFq9Vu3bo1Pj7ez88Ppk5nzZrl4OBw5syZjz76CKbWtFrtli1bfvrpJ4PBMHHiRPhFH374IZ/PX7du3d69e8H1Sk5OXrBgQXV19ZtvvmljYzNu3Ljg4ODY2NiZM2fCaEOn0+3cufOLL74AAbTaeyHqaVsXLlzI5/NXrFhx4MABqP/h4eGffvrpS3dKGz8ODg5t27aVy+WzZ8/OycmBk8nJyb/++ivHcUOGDIEzINVw+fLlO3fuvCy/lMfjDRgwQKFQrFix4saNG3CypKTkhx9+UCgUffr0sbCwAGPg8Xhr1qz5/fffoVksKSlZunRpZGSkjY0NtJiOjo7Tp0+vqqqaM2cODGQhB/T3339vZmb23nvvcRw3d+5ca2vrv/76a+3atbA0X1ZW9sUXXxw5cgSS4Zr+zT08PKZOnVpZWfnOO+/AdJdSqdywYUNjW4VunHTr1k0kEt24ceObb76hMRKnT58+d+6cra0t3QRsaWmpVqtPnTp19+5d2IDR8Dg6OgYFBeXn5y9atIhOSSQkJIA2N9iej4/PG2+8UVVVNW3atCtXrsA1Dx8+nD17dn5+fs+ePb28vAghffv27d27N5yHlra6unrdunXnz5/38vIaO3asjY3NzJkzGYb54IMPjh8/DoaUkpIyffr0jIyMnj17PpEHFRYW1qVLlwcPHsycOROW7gsLC/9fe3cdEEX2BwD8zWzSsXSXdCmgiIiIpCAmtp7tnV1nnX13dudZp2eceQY2dgd2AgIS0rB0bM7vj6dz64LcgnDu/fx+/rlzmZ3dffPem/edV2PHjn379i3kwH8UHBwsEAi2bt164sQJ3KSrrq7evn17RkaGt7c33nsXb3acm5sbFxeHVxP5Kl/V09NTS0vr9u3by5Ytwx3gUqn01KlTN2/e1NDQwPsrREdHe3p6JiUlDR48OC0tDR9z7ty5n376SSAQ9O7dGw+QHjFihL6+Pl5fFwfe+fn533//fUlJSVRUVIsWLXx8fCIiIvh8fr9+/d68eYPPc/nyZbxI9PDhwxs0QwzPdN21a9fatWtxxY4Hx9IPVsDnqKmp+fn5VVZWTp8+nR74k5OTs379eqFQiDeCQwipq6sTBPHs2bM7d+58xc7nkJCQmpqazZs3nzlzBheTmpqa5cuXl5aWOjg44Opx8uTJ6urq+/fvX7RoEa7ta2pqVq5ciffFxstcEwQxY8YMkUg0Z86cuLg4fKqnT5/iFsKwYcNUVFSGDBliYWFx9erVqVOn4npPIBBs3Lhx3bp1GhoaEydOVPw5KZPJnDlzplAonD59+qVLl/CzksOHDy9duvTfuMs3aJV0pMT7QV2+fFn2XRUVFTExMSRJcrlcOzs7d3d3Ho/HYDAcHBzw9iB4vU4nJyeCINhsNl6w4WutxL98+XImk8lisUxNTT09PU1MTJhMJo/Hw3do7OLFi4aGhiRJ6ujo4N3u8YrDy5cvp7eYkEgka9euxcsQW1hYuLq6qqmpMRiM/v374z0KKIp6/fo13lFBU1PTxcWFPs8vv/xC7+SD94Pq0qWL3PfEa/rRWzGIxeIlS5bgQNTa2rpFixYcDqd169Z2dnawH5TcflA6Ojr0JaBfb9GiBd7o3cXFxcnJCa820bt3b/pUt2/fVlFRIQiCy+UOGTLka+XP8vLyHj160KXJw8MDr8/j7u6Od8PA21/gzMBkMo2MjDw8PPCKWMbGxngDMay0tLR///4MBoPL5To6Otra2rLZbBUVlbVr19LZ+ODBg3hndAMDAzc3N7yPu7GxMV0JUB/3g8JPEGgFBQX0YC38SklJCf7mqqqqzs7OeH/AqKgoDQ0N2A9Kbj+oqKgouTeePHkSVyD6+voeHh5WVlb4Yq1bt47eeePXX38lCAJf0H95kxZZiYmJeE9bDQ0NNzc3R0dHXJr69+9Pb47H5/N79uyJh/Lb2Ng4OzvjY1q3bi27LUliYqKbmxtJktra2m5ubsbGxkwm09DQkN5dp6amZurUqWw2m81mW1paOjk5qaur441Q09PT6SKDly/D+xfRrl27xuFw8ILR2Js3b+zt7UmS1NXVdXV11dHRUVFRwSvzwn5QcvtBbdiwQfZdIpFo+vTpeDqZhYWFp6engYEBg8EwMTG5ffs2fbHwftwsFktFRUX2Qv/Ljh49ivdfMjAw8PT0xCOfWSzWli1b6GNevnyJN11UV1d3cnKysbHBG4P+8MMPsvvs0bOjjY2N3dzc8BImbdu2zc/Pxwfk5OQEBgbi5XMdHBxsbGw4HA5espK+ueD9oPAOJbLw7n9z5syhXzl8+LC6ujqTyTQ1NXV0dFRVVbWwsGjfvj3sByW3H5SamhoeSyxb5+N9k1VVVR0cHNzc3PCKC35+fvQ+RbiFQBAEh8Np06bN18qfUql00qRJJEnias3T09PQ0JDBYJiamt67d48+7NChQzgb4xH7RkZGeHGyPXv20MeIxeKZM2figmljY+Po6IgbpRMnTqR3Fbt3756JiQlu0Lq6uhobG+Pz4A4ufAzeD2rMmDGy37OmpgYPLrh58yZ+RSAQTJ48mclk4r2n8X2qffv25ubmzb0fFGPBggWKh1sLFy5ksViDBg2ysLD41yJjPp9vYWERFhaGHykhhIqLi21tbcPDw/ESRsXFxZaWlmFhYXp6evS72Gx2t27d7O3tRSJReXm5RCJxdHQcPXr05s2b6RXoVVVVO3ToIBaLNTU1XV1dg4KCvtbEUz8/v8DAQFwOq6urTUxMevfuvXnzZtllx/E4Wg0Njerq6rKyMm1t7cjIyPXr1/fu3ZseR0sQhK+vL24qlZaWCoVCLy+vefPm/fTTT/SqL/r6+oMGDTI0NKyqqiotLdXS0urWrdvGjRvxilX4GFzJtmvXTm7FrZKSEmNj45CQEDzrmiRJf3//Dh06VFdXFxUVaWlpDR06dO3atRRF2dnZRUREfG4RlXq8ePHi+PHjPB5v3LhxDX3vnTt3Ll68aGtrO2jQoH/t2lEUVVxc7OrqGhkZiRMQz4b09PTEXUlSqRQ3p8LCwmSfrOjo6AwcOFBDQwMvX4PbVfPmzZs/fz59mLm5ubu7u0gkMjQ09PDwoJdU/pfh0tSiRQuxWFxeXi4Wi52dnceMGbNu3Tq6tx1nhvDwcAaDUVlZWVlZaW1tPXjw4N9++83Dw4M+FYfD6dq1q5ubm1Ao5PP5HA4nJCRk/fr1vXr1orOxq6trz549ORxORUVFRUWFlZXV8OHDt27d2rJlS/o8ZWVlenp6QUFBskumUhRVUlLi5OQUFRWFh0ZwudwePXrY2tqWlpaWlJTY2NjMnTt37NixAoHA29sbP/pt6OXesWNHdnZ2165dZb+PIqqqqnbt2lVYWDho0KB/cxkGgUAgkUj8/PzojqPq6moWi+Xv74/H8+An2X5+fnLl3cHBoWvXrgwGo6ampqKiAvclrlu3rmfPnvTFatOmjZaWFpPJtLKyatu2LQ4R/308Hq9v3746OjoCgaCsrIzL5fr6+i5YsGDWrFn0pGQcmeCJT/im4OnpOWXKlDVr1sgOu+fxeAMHDtTT06uoqCgtLTUwMOjfv//27dtdXFzwAUwmMzQ0tH379lKptLS0VCKRtGzZcsaMGatXr6bX0MPVgr29PR4GQ58cr5LSpk0bejcYPT29gQMHcjickpISgUDQpk2bTZs2tW3blsViBQYG1rl0UP1SU1P37t2rp6c3bNiwhi6L+uTJk9jYWLzM5r85PaOsrExHRycoKAjPIiMIAt9rgoOD8b2mtLQUl3f8LJyuczp16tS6dWtcx9bU1FhZWQ0cOHDLli14e1P0sQsdIaSmpubo6NipU6evtYics7Mz3j2vurq6srKSx+OFh4dv2rQJv4gZGBgMGTJET09PIBCUl5fjQYnLli3DTVvZgonv1+Xl5dXV1Q4ODhMnTtywYQO9qKO6uvrAgQOtra1ramrwzSUoKGjlypWTJk2iby54KUhvb288dJxWUVGhoaERGBiIZ5gghFxcXLp27SoSifh8PovFio6O3r17t7q6Om4MNGL7x71796ampoaEhDSiBt60aVNRUVGPHj3kFlhvVrg429raRkRE4HE6uInl5OQUFhaG2418Pt/BwSEkJER2aU1VVdWYmBhzc3ORSFRaWooHQk+ZMmXlypX0yhw6Ojpt27YVi8Xa2tr29vbR0dFfJX/ip2xeXl5SqbSiokIgEFhaWg4cOHDbtm2ytbqLi0ufPn04HE51dXV5ebmhoWGPHj22bt0qu88eLph4yF9JSYlUKvXz81u6dOmECRPoacxmZmaDBw/W0NCoqqoqKyszMDCIiYnZvn17SEgIXfNUVVVxOJyAgAC5a11SUmJlZRUWFobzHoPBCAkJ8fHxKS8v5/P5+vr648ePX7x4sUAgcHV1xQOzG5QUmZmZe/fuFYlE/xwfNSgyQ1+jDwp8U/5zfVDgm/Jf7IMC35T/Yh8U+Kb85/qgwDdF8T6o/895UAAAAAAAAADQHCCCAgAAAAAAAABFQQQFAAAAAAAAAIqCCAoAAAAAAAAAFAURFAAAAAAAAAAoCiIoAAAAAAAAAFAURFAAAAAAAAAAoCiIoAAAAAAAAABAURBBAQAAAAAAAICiIIICAAAAAAAAAEX9lyIogUDA5/OrqqrgstVWXV3N5/Orq6shKb6iiooKPp8vEokgKWorKysrLi4Wi8WQFF8LRVElJSXFxcUURUFq1E4cPp9fUlICifMViUQiPp9fXl4OSVGbUCjk8/kVFRWQFF9RVVUVn8+vqamBpKitsrKSz+cLBIJv5Pf+lyKoLVu28Hi8mTNnQjatbcmSJTweb9myZZAUX4tYLO7fvz+Px7tw4QKkRm0hISEmJiY3b96EpPha3r9/7+HhYW1tXVxcDKkhJz8/38TEpHXr1vn5+ZAaX8vFixeNjIy6d+8OSVHbsWPHeDzeiBEjpFIppMbXMnfuXB6Pt2HDBkiK2saNG8fj8fbt2/eN/F4YxQcAAAAAAAAAioIICgAAAAAAAAAUBREUAAAAAAAAACiKqVTfRiwWX79+/a+//nrz5g1Jkm5ubv3792/dunU9b3n9+nVsbGx8fHxRURGLxbK1te3SpUtYWBiT+fdPy8jI+PPPP+/cuVNaWqqjo9O2bdu+fftaWlrKfu7p06dPnTqVmprKZDLt7OyioqLCw8MZDIZSJc758+dPnjyZnJzMYDA8PT0HDx7s7u5ez1uePn166tSpR48eFRcXczgce3v76Ojo0NBQ2WNSU1P37t374MGDyspKPT29du3a9enTx8TEhD5AKBTGxsaeOnUqLS2Ny+U6ODhER0cHBQWR5LcYfldUVBw/fvzs2bNZWVkaGhp+fn6DBg2ysLCo56rduHHj/Pnzr1+/Lisr09DQ8PDwiImJadmyJX0MRVHx8fGHDx9+9uyZWCw2NTUNCQnp1q2blpYWfUxxcfHhw4cvX76cm5urrq7u5eXVo0cP2ZMog5KSkqNHj168eDE7O1tHRycgIGDAgAHGxsb1JM7FixcvXryYkJBQWVmpoaHh5eXVp08fZ2dn2cS5efPmX3/99erVK6lUam5uHhYW1r17dxUVFfqYwsLC/fv3X79+vaioSFNT08vLq3fv3rIn+aZkZWXt3bv39u3bpaWlRkZGYWFhffv2VVNT+9zx5eXlZ86cuXLlSlpamlAo1NPT8/Pz69u3r1wlcObMGVxDstlse3v7qKio4OBg2Wr27du3Bw4cuH//fkVFhYGBgb+/f0xMjOxJlEFKSsrBgwdv375dWVlpamoaGRnZo0cP2bwkp7i4+PTp09euXcvIyBCJRAYGBh06dOjdu7e+vj59THV19cmTJ8+ePZuens7lcp2cnLp27dqhQwfZGvLVq1cHDx58+PBhdXW1gYFBx44dY2Ji9PT0vs0sGh8ff+jQIVzd2draxsTEhIaGEgTxueMzMzNPnDhx+/bt/Px8giDMzc2Dg4N79uwpe+H4fP7BgwcvX75cWFioqanp6enZu3dvNzc3uZrk6NGjL1++RAhZWlqGhoZ27dpVVVVVqRLnzp07Bw8efPXqFUVRDg4Offv2DQgIqCdxkpOTT58+fffu3by8PCaTaWVlFR4eHhUVxeVy6WNyc3MPHjyIa0gdHR1vb++YmBhHR0fZqvjq1avHjx9PSEggSdLa2joiIqJz586yJ/l2CIXCc+fOxcbGpqSksFgsb2/vgQMHuri41J+lz5w58/Tp0+LiYtxM6tGjR4cOHWQvXGJi4p9//hkfH19VVWVgYNCuXbt+/foZGBjQBwgEgr/++uvcuXMZGRkqKiqOjo5du3YNDAys5+p/lcQ5fvz4mTNn0tPTVVRUfHx8vvvuOzs7u88dT1HU3bt3z5079/Tp07KyMlVVVWdn5549e7Zt21b2dz1//vzgwYOPHj0SCoVGRkYdOnSIiYnh8Xj0ARUVFX/99df58+ezs7NVVVXd3Ny6d+/epk2br9wQpRoCIaSqqnrr1i2qGVRVVY0aNQrXifr6+np6egRBaGlpbd68WSqVUhS1Zs0ahND48ePpt2zfvl1bW5sgCE1NTXNzcxUVFYIguFzulClTJBIJPubZs2eWlpYEQaipqZmbm6upqREEYWZmdvXqVfo8I0eOZLPZDAbDwMDAyMiIJEkOhzNkyBCRSEQph6qqqpiYGA6HQ5KkgYGBrq4uQRC6urp79+7FB8ydOxchNH/+fPoty5cv19LSwmlobm6Oq0Iul7to0SL6mFu3bpmamhIEoa6ubm5urqqqihPn6dOn+IDq6uqBAweyWCwmk2lkZGRgYECSJJfLlb0KTWv//v0kSTo6OjbivcuXL0cIhYSENNN3S01N9ff3ZzKZDAbD1NRUXV2dIAhra+t79+5RFCUSibp06YIQOnXqFD6+pqZm+PDhXC6XIAh9fX1TU1Mmk0kQBI/HO3r0KH3aHTt2aGlpkSSpo6Njbm6Oz9+2bdusrCx8QH5+fps2bRgMBofDMTU11dHRIQhCR0dn69atSpI/KYp6/fp1y5YtGQwGi8UyMzNTVVUlSdLJyen58+f4gNatW3O53CtXrtCJ07NnTzpLGxsbMxgMgiAMDAzi4uLo065ZswafSldXFycgk8ns1KkTn8/HB2RmZrq5ueFsaWZmhvO8rq7u4cOHm+NnisVib29vhNDvv//e0PcWFBTgVovsD2xa586ds7CwIAhCRUXF1NQUV2sdOnTAS8xlZGRYWFhoaWkVFRXh49PS0tq2bYtT1dTUVF9fnyAIkiQdHBxSU1PpK0XXkIaGhkZGRria/f7774VCIT7m2rVrZmZmdE2ioqJCkqStre2jR4+UJ4seOnTIxMSEIAhVVVUzMzMmk8lisbp27VpaWkpRVG5uLofDadGiRW5uLj4+MTHR3d2dwWCw2WxTU1N8OydJ0svLKzMzk64hY2JiZGtIfP7p06eLxWJ8zNmzZ/HtDN+ncJ53dnZ+8+ZNc/zMuLg4hJCjo2NBQUFD37tz506EkLe3N30DbXKLFi1SV1dHCOno6OC7LZfLnTZtGv7EM2fOsFisTp060cefP3/e3NwcZ2lzc3NNTU2EEJPJjI6OplP4/fv3rVu3xjduMzMz3CrQ0tLas2cPfZ5ly5bhGpvH49E1SceOHcvLy5UkfwqFwunTp+MvqaurS+cl+pZ94MABhFCfPn3oq3P48GFcHunmDUKIzWYPHTpUIBDgY96+fUvXkDgB8f3o2LFjdJ02b948XGb19PRMTU1JkmSz2T169CgrK2uOXxoSEoIQWr58eSPe6+joSJLk/v37m+kqlJSUdO/enc1mkyRpZGSE77YGBgZHjhzBB0yZMkXuy+MsTRCEtra2mZkZh8NBCKmpqckec+3aNVwJaGho4PYYQRC2trYvX76ka5IePXrgbGlsbIyrYhUVlRkzZihPFVpQUBASEsJisRgMhrGxMb7bGhoaXrp0CR8wZMgQhNCOHTvoLP3jjz/Kljs2m40TZ+fOnfRpjx07Rjf4cQ3JYDCcnJzevXuHD+Dz+Z07d8Y1tomJCY/Hw3l+6dKlODpoWrdu3cIPVv7xSCWKoBYtWoSzVFxcnEAgEAgEe/bs0dDQ0NTUxLdhuQgqKSlJTU1NU1Nz3759VVVVIpGooqJizZo1TCZTW1v72bNn+LDg4GA2m7148eLKykqRSFRWVjZ+/HiCINq3b4+rmAsXLrBYLE9Pz6SkJKFQKBQKb9++bWRkxOFwzpw5oyQZd+LEiQRBuLi43Lp1SyAQ1NTUbN26lcViGRoa4tuwXAR1+/ZtTU1NTU3N06dPCwQCkUhUXFy8cOFCFoulo6ODb/9CodDX15ckya1bt1ZXV4tEosLCwqFDh5IkGR4eTlfZTCbT19f3/fv3QqFQIBCcO3dOX1+fJMmHDx82xy9V2giqsrISPygNDQ19+/YtTtKJEycyGAw3NzecA+UiqB07dhAE4ezs/ODBA6FQKBKJ3r9/P3DgQIIgfHx88J07Ly9PX19fX1///Pnz+ErhOI0giDlz5uDzTJ8+nSTJYcOGFRUViUQigUCwceNGFotlZWVFt+S+rqqqKi8vL5Ike/bs+e7dO5FIVFRUNHToUIIgOnbsWFFRQdWKoJYtW4Yboy9evMCJk5GRER0djRDq0KEDfniRnp6uqqpqbGx869YtfExSUpKHhwdJkqtWrcLnGTFiBEmSEydOLCsrE4lElZWVS5YswT3JzXH7V+YIKikpycTEhCTJGTNm4FX1k5OT8QPsH374gaoVQQmFwv79+xME0atXr/fv34tEIqFQ+PDhQ09PT4TQ5MmT8WljY2OZTKaPj09ycjKuIS9dumRiYsJgMK5fv46Lhp+fH4fD2bJlC65JCgoKvvvuO4IgunXrRkdZX9eTJ090dXU5HM6iRYtKS0tFItGrV698fHzogiYXQUml0tDQUFzucnNzceLcunXL3t6eIAi6Ubt3716CIIKCgtLT03HinDp1SkdHR11dPT4+HmcYKysrdXX1ffv21dTU4MTp2rUrQRCDBg1qjl+qzBHUwYMHWSwWj8fbv39/VVWVUCi8evWqoaEhh8PBLVS5CCo3N9fBwYHBYCxbtqyiokIkElVVVR06dEhHR4fJZJ4+fRofNmHCBIIgRo4cSVcCq1at4nK5BgYGlZWVFEW9fPmSx+MZGxvfu3cP1ySvXr1q2bIlQRDr169XhvxJUdTmzZtZLJaenl5sbCxuAp08eRLfba9du0bViqDS09NNTU1VVFQ2btyImzeVlZX08zj8XE8ikeCSOH36dJw45eXlCxcuZDKZrq6uuIZ8/PgxHqLy+PFjnDiPHj1ycHAgSZJ+RNu0lDmCGjFiBEEQLVu2vH//vlAorKmpWbVqFYvFMjU1TUlJoWpFUFeuXFFRUZG9g/P5/J9++okgCFNT0/fv31MUJRKJXF1dORzOrl276Bqyb9++BEH07NmTLnoMBqNjx45ZWVm4rRUbG6utra2qqko/0f66pFJpr169SJJs27bts2fPhEJhRUUFzkvW1tb5+flUrQjq2LFj+KHG9evXcdbKz88fN24cfhiHbw2FhYVOTk4qKipHjx6lm0ldunQhCGLEiBH4PKtWrSIIomvXrvn5+SKRqKamZu/evZqammpqaunp6U3+S/97ERSfzzc0NFRXV798+bLsBZs2bZqamtrChQupWhHU+vXr9fT0Jk6cKHseoVDo7++PEIqNjcWvaGpqGhgYyDY0KysrQ0JCRo0aVVhYSFHUkiVLEEILFiyQPc+mTZvCw8OPHz+uDBn3/fv3WlpadCRJGzp0qLq6Or4ByEVQCxcu5PF4cj9KIpE4OjpyOBycyJmZmSYmJjo6OrJdbbm5ucHBwSNHjsQv/vjjjwghuXvMkiVLIiIi6KcOTUtpI6i4uDgmk2lvb5+Xl0e/WFFR4efnp6mpefPmTbkISiqV9uzZU19fXy4XpaSk8Hg8fX397OxsiqLu3LmDEAoODpZtsty/f79Dhw5LlizBD1kDAwNVVFRke02FQuHIkSN79uz59u1bZciiR44cQQh5eXnhvYboQu3u7s7j8fANQDaCkkqlQUFB+vr6sj+Koqjk5GT8GAUn8smTJxkMRq9evWSPuXDhQlBQ0Lp16/A/3d3dORwO/SQPn7xHjx4DBgxojrpVmSOo6dOn4waW7IuvXr3S09OztbWtqKiQi6BycnKcnJxatGghl1AHDhwgCKJTp0748d78+fMRQsuWLZM9Zu3ateHh4bgJm52dzePx5IpGenp6RETE+PHjcRP265JKpSNHjsS3D7rjgqKoW7du8Xg8Dw+P6upquQgqOzvbyMjIzc0NtwxomzdvRgjReXL8+PEIoW3btsl+1s8//xwZGYlbvVlZWTieofsEcOK0b99+xowZNTU1zVFTKWcEJRKJvLy8GAwG3cDCtm/frqKi0q9fP6pWBHX27FlDQ8MuXbrIXjWKovDTmaVLl+J/+vr6IoSePHlCHyCRSAYMGNCrV6+0tDSKok6cOIEQ6tevn+wT6zNnznTq1GnDhg3KUIUWFxe7u7uTJCl3v1i6dKmamtqYMWOoWhHUgQMH9PT0Bg0aJJs4Uqm0W7dudJ4sKyvz8vJSV1d//fo1fUxZWVn//v179+6Nc8iuXbsQQvgjaIcOHQoODt69e3dz/FiljaAyMzNxEEsPncB1fv/+/dXV1XG+lYugZsyYoauru2LFCrkKx9TUVE1N7fbt27i84+4p2bZWXl6ev7//hAkTcM3www8/IIRke2Yoipo7d25UVNSNGzeUIYvev39fXV1dT08Ph4WyV1NLSws/AZGLoEaPHq2rqytX3ouKikxMTFgsFo5Inz59qqWl1apVK9nK8Pnz56GhodOmTcP/7NOnD0LoxIkTsueZMmVKly5dHj9+3OS/9L8XQeFK38fHR+6Okp2dfefOHXxjlougampq3r17h6MgOtfm5uaGhYUhhA4ePIhfdHV1ZTKZQ4YMefjwoWzzjrZ371488PfgwYPp6enKM3KP9ueffzIYjMDAQLnX09PT7927h1NALoKqqqpKSUmR/b0SiSQ3N9fHx4fNZp89exZXo46OjgRBTJgw4enTp3goi5x169aRJNmiRYvY2Nj379/L3caag9JGUNOmTUMI0UWa9uLFi0ePHuHno3J9UKWlpW/fvq2qqqIPFgqFT548wUMpcFSfnJzM5XI1NTWXLVuWkJBQXV1d+6MHDx5MEERwcPDVq1fz8vKao9v6C+EGzeLFi+Vef/LkydOnT3EKyPVBlZSUJCQkyBY3gUDw5s0bBoNhaWmJ6+iHDx+y2WwdHZ2NGze+ffu2zuZmZGQkSZLR0dG3b9+WrQ2aidJGUNXV1XjK6MWLF+X+dOPGjYSEBLFYLBdBSaXS/Pz8lJQU2eZyZWXl/v37mUymn58ffh13pdrZ2R05ciQzM7N2JcDn8x0dHVksFq5Jmmnkz5coLS11cnLicDj4wTytpqbmzp07CQkJ+N4h1weVm5uL7/G08vLyrVu3IoS6dOmCE2fFihUIITc3t5MnT75//7524FFVVaWpqcnlcmfMmPHixYt/YcyY0kZQr1690tbW1tbWlkuEsrKy69evZ2RkULUiKJFIlJmZSY9npvMbbm7S97uYmBiEUGho6K1btwoKCmrXkHfu3FFRUdHW1t68eXNKSkpzBK5fKD4+XkVFpXbPeUFBwa1bt3AKyEVQQqEwPT2dHnSKM21hYWH//v0RQvgZk0Ag6Ny5M47579y5Qw/flcswJEkaGxvv3LkzNTX1X+g0VtoICuf/qKgouddTU1Pv37+PU08ugqqsrExOTpZtPonF4pycHBcXFy6Xix8RlpWVmZqaslisqVOnPn/+vM4acvHixQRBuLm5nT59Oisr619oazXUihUrCIIYOHCg3OuJiYnx8fF4oLhcBFVRUZGUlCT7EE0kEmVlZZmbm7NYLPzcMy0tzczMjMVizZ8///Xr13jEihz8KL9169ZxcXE5OTnNN8YYUzyCUpaVJJKSkhBCLVq0wENIacbGxp+bic7hcKysrLKzs69du5aUlJSQkJCUlJScnJyamooQorecmz179ogRI3bv3n3o0CEHB4e2bdtGRET4+vrSU4Gjo6Nbt2597969/v37Gxsbe3p6BgcHd+rUydnZWUlWkkhOTpZIJK6urnKvW1hYfG4NAxUVFRsbm8zMzMePHyckJCQmJr59+zY5Ofndu3cEQeDE0dDQmDBhwtSpU9evX79jxw4nJyd/f/+wsLA2bdro6uri8/Ts2XPPnj2PHz/u1q2bmZlZq1atQkJCAgMDcS327UwtlUqlycnJuKkk9yf6uojFYrk/4V7mlJSUlJQUfBWSkpLevn2Lp0JSFIUQsrS0HD58+NatW2fMmLF06VJXV9egoKCQkBBPT0969v+4cePi4uIuXbp0/fp1a2trHx+fzp07+/v717OCxb9ffkmSrD3XFo8Hq5OWlpa6unpiYuK7d+9ev36NC29SUpJUKsUNIIRQy5Yte/bsefDgwfHjx/N4PDc3t44dO4aGhnp4eNBTnKdOnXr37t3Y2Njz58/b2dm1adMmPDy8Xbt2pqam307+RAjx+fyCggKSJGtXFO3bt6/zLXg6hLq6+rNnz1JTU1+/fp2YmJiSkpKUlCSRSOgqNCoqytfX9969e7179zY1NW3ZsmVoaCiuBPBKEtra2mPHjp0+fTquSZydnXFN0rp1a7om+bqqq6szMjL09PSMjIxkX+dwOG3btv1c4hgaGlZWVj569CglJeXNmzeJiYk4i+IKAWfRvn37/vHHHy9evOjevbu5uXnLli0jIiI6dOhgZ2eHbx8qKio//vjjzz//vGzZss2bNzs7O7dv3z4iIsLLy0t2qZhvQWZmZlVVlbOzM54HRdPQ0AgICKjzLUwm08zMjM/n37lzh77Lp6SkvH37Fsnc5cePH3/z5s24uLirV6/a2dm1bt06LCzM39/f3Nycroi6d+9+6NChsWPH8ng8d3f3oKCg4OBg2Zrk60pLS6uurrazs5P7Pnp6ep9bcYTFYllYWOTn59+6devt27e4FsW3Gzpx2Gz22LFj79+/f/To0djYWHt7e19f37CwMD8/P3qVl7Zt23bu3Pns2bMjRowwMDDw8PAIDg4OCgrCA8++qSyamJiIEKp9I7O2tra2tq7zLaqqqra2tmlpafHx8XRb6+3bt2lpaUwmE9cSGhoaU6dOnTlz5qpVq7Zu3Srb1tLW1sbnGTBgwP79+1++fBkdHW1hYdGqVavQ0NAOHTo4ODgoyUoSiYmJFEXVbgLZ29t/7i1qamp2dnbv3r3DVWhCQkJycjJuAtENUQsLi6FDhy5btmzRokVr1qxxcXEJCAgIDQ319vbGkx4RQkOGDDlx4kR8fHx4eLi1tbW3t3dYWFj79u1tbW2/cuI0KDJDzdYHtWzZMoTQqFGj6jlGrg9KJBItXbrU2tqaxWLhmsLW1rZHjx54Da4///wTHyaRSC5cuNCrVy884RJXyk5OTr///jsdyObm5s6bN8/T0xPPckMI6erq9uvXT+7R19eCB+fMnDmznmPk+qAqKyvnz59vaWmJmzhcLtfe3r5v375WVlYsFosePi4SiU6dOtWlSxe6jmaxWM7OzvSkSYqi0tPTZ82ahbvyEEIEQeCRA3V26H055eyDEgqFuG/zr7/++twxtfug8vLyhg4dipffQAipq6u7u7v37dtXV1dXXV0dP3PFz2m2bdvWsWNHDQ0NulIOCAi4e/cuPkAqlT5+/HjEiBHW1tb4VCRJWlhYLFiwQLaD6yvCZaf+rhW5PqisrKzevXvzeDz8i/DyWXhSk7m5OT3stqysbP369f7+/vSSWWpqaiEhIfRER4qi7t69O2jQIDzdHCeOlZXVihUrmqM/WWn7oNLS0kxNTblcbp2PmbHaK0k8ePAgKCgITy7HAVW7du26d++OZz/SNWRmZubcuXM9PT3pSoDH4w0YMIAeticUCk+cOBEdHY0nBOMK2d3d/ciRI8rQZZqdnY0HGtRTpddeSeLGjRt+fn54GjQOqAICAnAZj4yMpB8Sp6am/vjjjy4uLnTiGBgY0KPEKYoSCAQHDhyIiIjAs9IRQhwOx9vbG48FaHJK2wd1/PhxJpPp7+9fzzFyfVASiWTv3r10U57JZFpYWEREROC4d+7cufQb4+Pjhw0bhleNomvIRYsW0ZeppKRk3bp17dq1o1fwU1dXDw4OxqvefXW7d+9GCPXo0aOelJfrgxKJRL/99hsemU8QBJ6OEh0djddoXbNmDZ2Gt27dGjRoEF7rBSHEYDBsbW3XrFlD15AFBQXLly9v06YNjt/wnP7o6OhmGiWutH1QY8eORQj98ssv9Rwj1wdVUVExc+ZMvDYJbms5Ojr279/f1NSUw+HQ9zuRSHT06NHIyEh6fTkWi+Xh4SE7Mi0tLW3KlClOTk50TaKvrz9q1CjcvfPV9evXDyG0adOmeo6R64Pi8/kTJkzAk2Zxw8bZ2fm7777D8xjpoZLV1dV//vlnaGgo/VCJy+V6e3vLThV5/fr1hAkT8PQ8XMCNjIwmTZrUHKPE/3uj+DZt2oQQ6t+/fz3HyEVQmzZtYrFY2traU6ZMOXnyZFJSUkFBQU1NDb7D0REUnX2zs7PPnj07ceJEvPCipqam3ByM0tLS58+fr127tnPnzrhK6t27d3N3Fyri559/RgiNGzeunmPkIqiff/6ZyWTyeLzZs2efOXMmOTm5sLBQLBb7+vrKRlCYUCjEy8WOGTPGysoKr/MmN7q0uLj48ePHy5cvDw4Oxjez+sPdRlPOCEoikXTv3h0h9Mcff3zumDrnQREE4eDgsGTJksuXL6enp5eUlLx//97CwkI2gsKqqqpSU1P37NnTv39/XMk6OTnJtoEkEklBQcGNGzfmzJmDl21gsVibN29WhrrV39+fwWCcPHmynmNkIyiBQBASEoIHLaxatQqP4SktLeXz+Xi1YrkVMiorKxMTE3fu3NmnTx/c3G/durVsDC8Wi/Py8i5fvjxjxgx3d3e8WJxcJdAklDaCys7OtrGxIUkSz6+rk1wE9f79e7wuQnBw8LZt2+Lj47OzsysrK8+fP89gMGQjKKykpOTZs2erVq0KCwvDa0kNGjRINkASCATv378/fvz46NGj8XYRWlpaL168+Or5My8vT11d3dTUFM+KqZNcBJWRkWFoaMhkMjt37rxz585Hjx5lZ2fjZQzkIihc2IuLix89erRs2bLAwEAWi0WS5IQJE2QTsKamJj09/ciRIyNGjMADK0xNTZOSkpr8xyptBHXhwgUOh+Pp6VnPMXIR1JkzZzQ1Ndls9rBhww4fPvzq1au8vDyBQIDHVMtGULiGzMvLu3bt2uzZs/G6oEwmc9euXbLHVFRUJCYmbt++PSYmBjfX3N3dlWHE1F9//YUQCg8Pr2cQnVwEdejQIS6Xq66uPnr06KNHjyYkJOTn5wuFwmHDhslGULI15KVLl6ZNm+bi4kIQBJvNlptzVV5e/ubNm82bN3ft2hWvWhwaGtocD+mUNoKaOXMmQqj+5e/kIqhZs2bhdRHmzJlz9uzZlJSUoqIiiUTi5uYmG0HRNWRGRsaxY8e+//57HNAaGhrK1ZB8Pj8+Pn7p0qUdO3bENcmkSZO+ev6kKGrUqFGo1oRYObIRlFgsHjt2LIPBMDc3X7hw4cWLF9+9e8fn82tqavDDfdnJZriGTEtLO3DgwLBhw3ANaWBgkJOTI1vNFhUV3bt3b9GiRX5+fkwmkyRJ2cWlm8p/L4K6dOkSXqBMboDyo0ePXFxcJk2aJBQKZSMoqVSKR9n99ttvcrdw3L7Zt28fri6vX78u114pLS3F/Ql4gYqEhIRjx47JzoEWiURnzpxBCFlZWclev6/l4MGDeEliudevXbvm7OyM+6ZkI6iioiLcESfblYQztKWlJYvFwstslJSUXL16FU93pjMon8/H6/SvX79eKpW+evXq0KFDssPWhULhrl278PK+zfFjlXYeFB6JW3se1Lp16zw9Pf/44w+5COrdu3cMBoPH48nFoklJSRoaGvQaMjk5OWfPnpVdCEEikbx+/RqvSX337l2xWHzv3r0TJ07I3lmrqqqmTp2KEOrevbsyPOMfPnx4nfOgFi5c6OXlhZfNlY2gbt++raqqqq2tLRcp4a1azM3NceJkZmaePn0aDx6g8/DLly/pJTprampw4siepLq6eujQoQihoUOHNvkvVdoIqqam5nPzoHr16hUQEPDs2TO5CGrbtm0EQfj6+so949y1axeOUfFYvjdv3hw9epRePh5XAgcOHOByufr6+iKRqKSk5MqVK7LTnfEMKz8/P4SQMgT5paWlLi4ubDZbbh6UUCiMjIz09/fH80lkI6ilS5cihIKCguSmJq5atQohFBERgZvdz58/P3HihGziCASCnTt34uW8SktLi4qK4uLiZD9XKpVmZ2e7uroSBEHP121CShtBvXnzRkdHR0tLS26qA56R269fv5KSErkIqlevXggh2e1JsL59+yKEZs+ejRP8wYMHJ0+elA2Eqqurx44dSxAEXlglPT391KlTsj0qYrH44cOHurq6TCazOeLYhnr48KGKioqtra3cJJm3b996enriZTZkIyiRSIS3dvzll19kE0cikYSHhyOEVq5cidPh9u3b58+fl02c8vLyQYMGIYTwEp2pqanHjx+XfaKHt4fS0tJSUVFpjiaQ0kZQeFGNyMhIudfPnj3r4uKCW4yyEVRhYaGVlRVC6Ny5c7LHSyQSHo/H4XBwLwqfz798+fLNmzdlKwG80hJBEHg81IsXLw4ePChb2wgEgt9++w03h5ThUT5eEK/2PKg///zT3d0dR+yyEVR6erqxsTGLxZILGXJycoyMjJhMJl5iqqCgIC4uDq9cSqcefrrHYDCOHz8uFoufPHly/Phx2e4mgUCwZMkSkiR9fX2b/JcqHkEpy1QWb29vY2Pj169fX79+nX5RKpXireXwJjOyxwuFwuLiYgaDITdI9NGjR8+ePUMfJ6WkpqaGhoaOHDmysLCQPkZDQ8PGxoYgCNyxOGvWrF69ev3555/0AXihT9xRqAyzfQICArS0tB49ehQfHy87/HLv3r1v3rypvVdmRUVFeXl57eG8Z86cwY0DnDiPHz/u3LnziBEjBAIBPgD3PuHZNQwGQyqVTpgwoU+fPsePH6dPwmKx7O3t8Z5F/2+DoOsVERHBZrNPnjyZk5Mjm9QHDx589uyZoaGh3PH5+fkSiQTvACObpQ8fPlxeXo4rRITQoUOHOnfujEeb4GNIkqR3PCBJUiqVhoeHDxky5MmTJ/R5VFRUHBwc8GVShsTBoeORI0eKioroF/l8Pt6/snbi4LW2dXR0ZHdclUqlv//+O/4fPEJ6+/btXbp0wU0BfAyDwbC2tsbdUCRJFhQUREdHDxgwIC0tjT4PHrOqPInz7+BwOLhbb9u2bRKJhH79zZs3cXFxCQkJtSeG4drAzMyMHm6OEKqsrDx06BBFUSKRCCEkkUhmzpzZq1evw4cPy1YCdnZ2eNcOhNDTp0+Dg4MnTpxYXFxM1yR4YxkluQoaGhqBgYFCofCPP/6Qna/44MGDmzdv4ilStRMHIWRrays7L6WsrAwvO0k/thg9enRMTMz58+fpY9hstpOTEy7IBEHcvHkzNDR0ypQpNTU1somDc/43lUVbtGjh4OBQUVGxb98+2dePHDmSmJhYWloqmw8RQmKxGN+45abdpqSkXLt2DX28y+fn58fExPTu3RvP/5GtBOi7/LZt26Kjo1evXk1PncIj2XBPizJcBTs7Oycnp5SUlAsXLsi+fvz48efPn9ee7CEWi/Pz82snztOnTx89eiSbOJ07dx4yZAieH46pq6vb2toihPBEnZUrV3bv3h3vLkgnjqWlpbq6Ot6j79vJoqGhoVwu9969e7J3W7FYvH///tevX9PD7GmlpaXV1dUIIbkN3I8fP87n86VSKa6K7927FxERMW7cuNptLXwjoyhqxIgRffv2PXfunFxNoiT5EyEUGBiooaFx9uzZzMxM+kWJRPLHH3+8fv2antBFq6qqKi0tVVdXx0EmRlHUiRMncDsBZ9FLly6Fh4dPnz6dThy8rgnea5jBYFRUVAwfPrxXr1737t2TTRy8z8FXTpwGRWaoOfeDWrVqFZPJtLW1jY2Nzc7OzszM3Lhxo4aGhomJCV6IU24UH24udO/e/d27d8XFxTk5OceOHbO0tMRjzOjFjgMDA0mS7NWr1/Pnz3Nzc7Oysg4dOqStra2rq4uj3tjYWDabbWBgcODAgffv3+fl5b1+/bpLly4kSdKr0X91c+bMwftBXbx4MScnJzMzc+XKlTi58KZjsn1QVVVVeEn3AQMGvH//vri4ODs7e+/evSYmJngzR7zPYGVlpa+vL0EQ3333XUJCQl5e3vv373fu3Kmurq6vr48f1+3evZvNZhsbG586dSo7Ozs3N/fJkyfBwcF4f4nm+KVK2wdVVVXVo0cPPLDhyZMnubm5SUlJo0aNwt2DeKcX2T4oPp+vpaXFZrN/+eWX3NxcPp+fkZHx66+/amhosFgsgiDwRl7p6ekGBgYqKioLFix49+5dXl7eu3fvZs+ejXfgwZ0DkydPJgiiTZs2N2/ezMnJycnJuXLlSosWLdhsdnMMVGuE6urqDh064P2gnj17lpubm5CQgPca6tGjB36uJtsHlZiYyOPxWCzWpk2b8vPz+Xx+amrqvHnzuFwuXkwWF/k3b95oaWmpqqquWrUqPT09Ly8vNTUV7/3SsWNHPLwEL1QYGBgYHx+fm5ubnZ194cIFvE1Kc+znpsyrmaemptra2pIkOX369NTU1Nzc3Hv37vn5+ZEkOW/ePKrWKL5Tp04xmUx9ff1Tp04VFhYWFha+ePFiwIABXC4Xty9xnH/48GEOh2NsbHz48OGsrKy8vLxnz55FRkYSBIGHFpeVleG94YcNG/bmzZu8vLzMzMytW7dqamrq6+vTO/N+XS9fvsQzE+bPn5+Wlpabm3v9+vWWLVsymczVq1dTtUbx4Y2eTE1Nr1y5UlRUVFhY+OzZs+joaDzA29/fH2e/7du3MxgMGxsbfNvKzc19/PhxQEAASZJ4jFlVVRWerDt+/PikpCScOGvWrOFwOLa2tnJDeZuEMu8HdfLkSRUVFT09vV27dr1//z47O/vYsWMGBgYaGhoXLlygao3iw4Xd29v71atXxcXFeKSut7c3vgqjR4/Gh33//ff4afSDBw9yc3NzcnLi4uJsbW1ZLBYeqPbkyRM9PT0VFZW1a9dmZGTk5eW9fft2zJgx/zgv69+0e/duvIfV0aNHs7KysrKy/vjjDx6Pp6amhjcyke2Dwhs9IYSCgoISEhKKi4tzc3NxVwluAs2aNYuiKKFQiHcgDAkJefjwIa4hT548aWZmJjsiQE1NTUdH57fffsOJk5CQgHeR6tmzp+wq/E1FmfeDmjJlCkmSLVu2vHr1Km5r4Y00nZyc8Aqxsn1QVVVVeNbZqFGjsrKyiouLs7Kydu3apa+vj7MonjhdWVnp5ubGYDBGjx6dmJhI15BsNtvc3BzXkFu2bGEymVZWVmfPnsU1yaNHj9q1a8dgMOS2pflapFIp3i/Uz8/vzp07ubm57969mzFjBpPJbNmyJe6Hl+2DKigowA/xf/zxx+zs7OLi4oyMjA0bNvB4PDabzWQycfbDm2qQJDl16tSUlJS8vLz09PRly5apqKiYmZnhJtDixYvxHruXL1/OycnJzc29ffs2nsvQHPu5/Sd31K2urv7xxx81NDQYDIaZmRnerdzCwoLeOVsugnrw4AEeSGpgYODm5oY35O7Tp89PP/2EEBo5ciR+TPjmzRvcVaqqqmplZYV3nNTX19+yZQs+QCqVzpgxQ1VVFU9Ns7a2VlNTYzAY4eHh9cwo+JdVVVWNGjVKVVWVxWKZm5vj/arx7sP4ALl5UBcuXDAxMSEIwsjIyN3d3dTUVFNTc9iwYRMmTMAZGh8WHx+Pm3Tq6urW1taGhoYkSRoYGNDt8urq6vHjx+Pdyk1NTa2srFRUVFgsVpcuXZppdqPSRlAURWVkZHTu3JnFYnG5XCsrK7xxoa+vb0JCAlXXShIbNmzAj/GsrKxcXV319PQMDAwWLFiAR1/QYywPHDiAlwjT1dW1sbHR0dHB69rhrSRwTRQREcFkMtlsNl6Akclkqqur//jjj3Wufv5VvH37NjAwkMlkqqqqWltb44LcqVMneltxuZUkfvnlF7ql7uLioqura2RktHTp0vbt27PZ7PPnz+PDtm3bpquri5cusLGxwWneqlUrepPB7Ozs9u3bkyTJ4XAsLS3NzMyYTKaGhsbChQubY3qDMkdQFEVdv34dP7bU1ta2tLTkcrkcDmfw4MG4uS8XQVVVVQ0aNIjJZHI4HCcnJ0dHRw0NDScnp02bNpmbm3O5XNwEr6mpmTZtGq4hTUxMrK2tcUUUGRlJL7pw584d/Llqamo2Nja4JjExMfnzzz+VZ/H906dP29nZEQShq6traWnJZrNVVVXHjx+PE0cugqquro6KiiJJUkVFxdnZ2cHBQVVV1d3d/bffftPX17ewsMCjm6qrq0eOHIlzsqmpqbW1NZfLZbPZffr0oZfruHLlCl7hQFNT09raGtfeVlZW9c8bbDRljqAoilq3bh1eP8bIyMjU1JTBYOjo6OBB41StCCo5OblVq1Z4YQM3NzcbGxs1NbXg4ODVq1ezWKz27dvj75mdnR0aGspgMNhstqWlpbm5OZPJVFNTmz59Oj5AKpVu3rwZd+zr6elZW1vjmsTT01NJVpKgKEooFP7666+4/jcxMcFtFQMDA3oql9w8qJcvX+L9SHR1dd3d3a2srFRVVbt06fLrr78yGIzo6GicpCkpKe3bt2cwGBwOx9raGqe5trb24sWL8UoSYrF4xYoVWlpauDVlY2ODa29/f/9mGt+ozBFUaWnp0KFDcTvHwsICL43j6OhIz3eQmwcVGxuLS7SJiYmbm5uxsbGWltaoUaO+//579HGqCEVRd+/exav5aWhoWFtbGxgY4Ac0dDOgqqpqxIgRuGPf1NQU194sFqtPnz517jTzVfD5/JiYGDabjQsavjW7urrSW7HJrSSxb98+fIy5ubm7u7uhoaGuru7UqVP79u1LkiS9FdvZs2dxDamlpWVjY4PT3NLSkr5RlpSU9OvXD8dd5ubmuPbmcrnDhg1rjnl6/73VzHG3++LFi6Ojo0+dOpWUlMRgMLy8vHr16tWiRQt8QPv27ZcvX96qVSv8Tx8fn8uXL//5558vXryQSqV2dnYRERH+/v6FhYVaWlpGRkYikYjNZjs6OsbFxZ06derOnTsFBQVqamoeHh5RUVH0mowEQfz66694Nc+3b99KJBJjY+NOnTqFhYXV7rT9WvC+43369Dlz5kxqaiqTyWzdunWfPn3o9azxt23Xrh3dGX3+/PnDhw+/fPmSIIjIyMjOnTu3adMmMzPTzMyMHs/j7e19+fLlEydO3L9/n8/na2houLu7d+3aFY9CwRdl5cqV3bp1w5+LELKwsAgJCQkKCqLXRvt2mJubHzx48OLFixcuXMjJydHS0goMDOzWrRte+IHBYAwdOrR9+/Z0h/4PP/zg6up6/PjxtLQ0DofTr1+/rl27Ojs737lzJzg4mF7ouU+fPu7u7rGxsc+ePauoqNDT0/P39w8PD6cvk56e3sGDBy9cuHD16tWsrCwWi+Xg4NC5c+fWrVvLjW79iuzs7I4fP37u3LkrV67k5ubyeLzg4OCoqCi6c3/8+PEFBQV4HReE0MyZM729vU+cOJGVlaWmpvbdd99FR0c7ODj4+Pg8evSIXu945MiRPj4+p06devnyZXV1tZ6eXmBgYHh4OO7iRwgZGxvHxsaeOXPmxo0bOTk5eCmkyMjI1q1bf1PjT7CAgIDLly8fO3bs7t27eBOSqKgoevUXbW3t2bNnCwQCXHhVVFQ2b94cERERFxdXWFjI4/H8/f0jIyONjY21tbWzsrLKysr09PQ4HM7ixYujoqJwDSmVSs3NzfF6x3QN2bZt2wsXLpw6dQrvmqKlpeXl5RUZGYlHUilJ4kRGRrq4uJw8efL+/fsVFRXR0dHR0dEBAQF4CVYNDY3FixdramriH8Xlcvft23fs2LGrV6+WlJTo6el16NAhMjJSU1NTVVUVb8uGD9uwYUOvXr0uXLiAHxZER0eHh4cHBATQNWTHjh2vXr0aGxsbHx9fWlqqpaXl6+vbuXNnGxubby1/IoQmTJjQoUOHEydOvHz5UiwWOzo6xsTE0Ld1JyenpUuXmpmZ4X/iMSmHDx++e/euQCAwNzcPDQ0NCgrCqyOyWKzq6mo1NTVjY+MjR46cP3/+ypUrOTk5eIQPvuXhEW4EQXz//fd+fn4nT558+fJlTU2NgYFBhw4dwsLC6Jrkq2OxWDNnzgwNDT1+/Dh+Kufm5hYTE0OPxvf09Fy+fDl+YI8QcnFxOXv27OHDhx8+fCgUCq2trcPDwzt06FBRUYGf3FEURRCEjY0NrpmvX7+el5fH5XJdXFwiIyPxYhv4zjV58uSAgIDTp0+/efNGIBAYGxvjTTWUZDeCf5OmpuaWLVv69et35syZtLQ0Npvt5+fXs2dPejR+ly5djIyMAgMD6X/GxcUdPHgwISEBB6747pOcnGxjY2NlZYWvgq+v77Vr106ePPngwYOSkhJNTU1vb+8uXbrQN0QVFZVNmzb17NnzwoULeFC6paVlWFhYx44dlWTBfYSQjo7Onj17Ll26dP78+ffv36uqqvr5+eGVrvEBffr0cXZ2xjNyEUL9+vWzs7M7evRoUlISm83u0aNHVFSUp6fnixcvvLy86OZrRETE5cuXY2NjHz58WFZWpqOj06ZNm8jISHr4n5aW1o4dO/r373/x4sX09HT84BXvWPCVF9xvUGSGmrMPCgBKufugAFDyPigAlLwPCgBl7oMC4L+3kgQAAAAAAAAAKD+IoAAAAAAAAABAURBBAQAAAAAAAICiIIICAAAAAAAAAEVBBAUAAAAAAAAAioIICgAAAAAAAAAUBREUAAAAAAAAACgKIigAAAAAAAAAUBREUAAAAAAAAACgKIigAAAAAAAAAEBREEEBAAAAAAAAgKIgggIAAAAAAAAARUEEBQAAAAAAAACKgggKAAAAAAAAABQFERQAAAAAAAAAKAoiKAAAAAAAAABQFERQAAAAAAAAAKAoiKAAAAAAAAAAQFHMhr5BKpU+e/ZMKpVC2oHmkJiY+IVnKCkpuXnzJqQkaA5SqbSiouILT/Ly5UsulwuJCZrDy5cvv/AMFRUVN2/eJEl4wAqaRUlJyReeITExEe7yoJm8ePFCwRinwRGUQCCYMmUK1K2gmUgkEoqivuQMjx8/DgsLg5QEzUQgEHzhGWbNmgVVKGgmX/58MykpKSIiAlISNBOhUPglb6coaunSpStWrICUBM1BKpUqmEUbHEFRFPXlDQgAmo9EIqmuroZ0AEoLqlCgzKRSKVShQGlRFPWFMRgATYL4wuf9AAAAAAAAAPDtgJEkAAAAAAAAAKAoiKAAAAAAAAAAQFEQQQEAAAAAAACAoiCCAgAAAAAAAABFQQQFAAAAAAAAAIqCCAoAAAAAAAAAFAURFAAAAAAAAAAoCiIoAAAAAAAAAFAURFAAAAAAAAAAoCiIoAAAAAAAAABAURBBAQAAAAAAAICiIIICAAAAAAAAAEVBBAUAAAAAAAAAioIICgAAAAAAAAAUBREUAAAAAAAAACgKIigAAAAAAAAAUBREUAAAAAAAAACgKIigAAAAAAAAAEBREEEBAAAAAAAAgKIgggIAAAAAAAAARUEEBQAAAAAAAACKgggKAAAAAAAAABQFERQAAAAAAAAAKAoiKAAAAAAAAABQFERQAAAAAAAAAKAoiKAAAAAAAAAAQFEQQQEAAAAAAACAoiCCAgAAAAAAAABFQQQFAAAAAAAAAIqCCAoAAAAAAAAAFAURFAAAAAAAAAAoCiIoAAAAAAAAAFAURFAAAAAAAAAAoCiIoAAAAAAAAABAURBBAQAAAAAAAICiIIICAAAAAAAAAEUxFixYAKkA/h+UP93909Rf/3yKHH1d9FjfYgpI+E9jr+ZbtDBs7l8vrcp98+Dq+VNnrz1Nzi6htE2MNJgIIYTECbF7nqm62Wh90aMZacW7W0e2rT2W5xbgrElAPlY+gpz4E9s3rP9t244/Dp+7/iRNZOTkYMD9epdKWpF+L3bvls13OZ18LZhQLShdkVZ6grxncQe3bThV7BngqPHN1TnKlhk/X5wBUCoUAMpA9HBtH383c20VLqch1Dx/eoDf/3ieJ5tACBGqASvfir+x1JPwnx2c18tdl8kJ3ZLXrJ8kSDuzqIerDpNACBEEQRAEIpgaFj5dx/yydfemyR2MjIacrG7kj8iNP7R25tCIlqaqJIEQy+fnV9/adfwv5OOyx1sHuZm2HPTrtu0rxwYYMQiEEKHiOOZMoeTfzvW58Uc3Lxjbu6OrkSpJIESo9zpQ2YzJKX6zc3iwr4vZJ5WUqv24SxWKF5+7szzUZOovroq2uZt/z5X3KNHD9QM6eppp/n1qFeOwdS9rPnum20u7tfu0wuSqaJu5+Q/+7WXh2Z86B3jZG2nKVaZcVV1Ld7/g/mvjG1+ka65OtFflNIpm5La8fzuP1K8m4/afa2YNj/Kx1GQSCCGW37Jv7t6hNBQozgAoF4jvgXJgek08eHOipOD2L72iFt0okSKEEMO888yZPe1UZZ5+UhKRsKa8KDvlyZXY0/ff10iqqgQIIYSk5aUVUgohhETlZZXUN5Nu0pKXx9b9unjjX08LRRRC7Gb9MMGbrX1CxsdmSTTdB6xcNKG7v5uROPXRnTtXjqxf89uck5sRQgSHm1uOELcRZ6fKSyq5+jrC/Nwq6bdzAeWvp3LnY2nW4TE9xh02mP9412wXBkI9vVQ7Ra18Uln9Njb28erOIZx/9csUF5SpWnu6J149XS2lECKaOTkZjsN2XByGpPz7y2I6z7nClyKEEEret+7grI7DTRTpoyk5s/mPl5UCCf4XqeU/5/Sxef76DIQQQuP3XRlf9XrPDz2/35tQTSGEcuJm9p/tem1VkE4dp2L7zTh+a4ak6Pbinl0WXC+WkryOC08emtUOn+yXMxG/IGHW2WmRPTc+q6EQQojkBS85d3iajw75RUWaqiwvrxEIpARD09K7vZ+bjZGuGpfNJKU5V7btulMoRQgxHbpPjXFh4bq6MDPp8e1bz3NqKESVV1QpV3YWFvNFui1cLW+crBB/s1WO0tQt/1ycAVAyEEQC5VK8q4sKrjvrfwYlKbwyxYPLtB575cO/35+aHd26pV/vxVfzletBZ3ORFN3eMnX4kFE/zpvZ200dpxm7GfugRC+XtVMjEGKYDTicI5fEFa/2jfHRJQmEWC3nPRN9wacIbk62ZyL0DfRBiUV1/TylzscVl8faMhHTbtINAX29Ev4YFeDhHbPmQcXX+lb5W8M4xGcqDEWSs+4LUZ+yPd3U6AYewW4577EiWV6StilEZoQYwQnfVlhHMUtcH6L9McohmGYxe97V+/UqTg83ZSCGzbgrtfurxK9+9vkwcpHQ7HWgvAmKdPXR/jokoeE16fR7wSfl9vZU+w8PZDlddpV9cqrylzt6W7EIls8vrxVO6YZflMarPjvClIHQ/20f1L+Zls1bnAFQKrCSBFAyXH197Q+tDAZZT/4keR1nTgpVF4mEH/5tGvXryfuPbx+aFaj/bWRrUs1twK87dm1dvnDJ1ilBKs39yE54e9u2+5UUYlh0HR5tJJfEas4DNpw/MtldhZDm52RLvuRXaWqqfwNPH6UZ+4aMP1BRx+9X4nxcEbfryDsxIrR0demvxnYYvPX60/jDk3zUvtbX0tDTU/tclvnH5PzshagPW0+PDnIQJXy+c/2Zkn98k/jJjm3Xqk0c7DXwWwl1Pb060oxp6WL/sQZElPj9X+MHLn1YT9+NSksPewbBcfVqWbsDkKGv/7HLidTR12c1QZGW1lQLGM4/bFwaaapwh7e6y9B1i7rwkKCmSrGenkZdlMZj6unr/N/eM/7ltGze4gyAUoEICigZgsNmKVh/8oKCvRgi0TebVBwNjQ+NJq62VnNHUJJ3dx9kiBFCpIa2JqOOA0jdoIWbxjiR/Jyc6i/KAN/AlRO8XD9s0l9Z/7GRQ8KnNx/wpQghkiCU6iqRZGNvZI29EBwOi1D1DvDGpU6S/df6ve/+4blB+fktu18yvEaM8P4Y6bDZn41BCI69u6MqgRBC0tI7iwZMPp0v/dyP19LVZhIaury6Rs7+/REEm8MmmqBIC2oEpHvvQd4NG69JGkX3C+EJqmqkSlk6SPL/tdb5L9Y0jS/OAPzLeRWSAPx3c6/F+Etpv0dBQiDU/IGHpJhfQiGEkPjtrRvpdbcW1XzHjvZn52fnSJQtdZRKybU5A2dfKZL+1763MC0t6//qyn7ZhSAt+43vakAihBBVcWPz5vuCeg6WZh3ccjRHK2LMUFuGIidnuU3btzQEd5tRwqQdI4ZtSxR/puSz2WySxWY3uAZoTJGWCiRsp+BO9g2eQa3RvpOfOhL9c0v+v1o6oKYB4FsDERT4TxJXlVYIIRn+RQxdXTy2iKq+tWLK70l1pj7DsmuvAElJAURQnyVI/H344LXPq/97E9elFWVlov+f+fZffiEI7egJgx1YBEKIEiftWXe84LNNVfHLnVsvVVr1H9/LUMFAh2A5jf1j+1B7HBhJ885OGzDvZulnjkUIEQ2/lzemSJNGo44/WNqu4SvWkIbD/3q4KpDz/1o6oKYB4JsDERT4Lyo9MsRr7NmaT18U5j+L3Ti9X8ew+Xc/E1yVvz62ZFSXNg7mBrr6ZrbuHXr+sOj3SwnFnz7brYnfMWnEkO8GfzB8+dXyj3+5tWYE/frgwYOHzjueKddmqsq4tXfRsE6hc28JEapJPPZTD28bY0Mr3+F/vPnkO1UknV47fXjPYB8HU31DC4c2USN/OfKytNnTreLhxgGt7Wzco+dfyG3gg0mGVYC/DX7yLMk5Nb7zgC1P6vi+pOWY2IcrAupsYFWlXtw85/u+4W2dzAxN7Tz8QnpPWHnydfk/fK7g3YV1E7u3dbIy5OmZOvj2+HHPs8+mk6KpKua/ubhzwYiwlt03vJMghCR5d3fO7BvgZKavZ2zXKmLc1nv0Y1tJ3v29C4eHeztYGPAMrVt1nrDzUUndKVf6+uTqyX07ebs6O9mZGZnZenTsO3XLzVzZzCXJOj8/OuSH45liCiEkfvr76A8ZaebBFEkD8nHjUrIWacmLYyun9Ovk5WBpzOMZWTp4BfWbvPLYS7kfKH6ybdx3gwcPHjLuj1cShBCSpJ2YM4QuH8sulyjyYcL0y+sn9Gjv4eLh7dvGp23YgKkrdu1eueZkxifRtrjoxakN02LahS2MFyIkeHdx3cQefk5WhnqGls7tY37cfje/wcF5Xcmp2IX4Z+zWP4wJwNP2pAWn1u9M/MybK69u3vWM9BkxJkC1AWcnjbqu2zfXD8+4oiofrxw89lBmE/YnfHmRblKKXhQFCloDc12dhaPo+qYfxwyXqe+/Gzpy3PTdT8T/+EMalYkVvHMoWGYVz+AKnvALKqBGfQIA/wmwmAZQLjUXv7fA41wIrd6H6l6Lp/ryGBvDQcc/bFJS8e76nl9GR3oYcgiEEGLaTbwuqP2eyufb+tirskxD5h17klGY+zpudT880YBgaZnYOrq4urq6urX98TxFUZSk5MWW7ib4W3yyup2khp9y87f+LfBMLVar+c/xGlXiwuexm2YN6mivzSIQQgzrcVfKXv7WzYye0cWwHHP5w1JZktyrv0Za6bSInLL69wOHD+xYMTHCVpVAiODaxmx90bjFh6qPD9Ija31beZK09R3x3ArEtJ98U9DADxEnbw7/ewUBRGo49V55LVuhdffEOVd+ibJW0/Ec8Muf11+m52QnXFkZbcokEMG16bH5qeyPFj2f34qFEEIsn59fpJ/+0U+PQZBsFQ7zY1KSvE5rX9T6VMVSVfBs96TegY56H0Y8sfyWJZa82j+urQGLYGka6Gt8+BCSF7TmhYiqSTw8yd+IzdE2Mrcw1vpwLUn98E0JcktbSYpuLgk35zAMOs49nVxBUZS4+NkfQ5y5BMEy674j8eOXlbw/v+anmdMHeOE2MdO+69SZM2fOnDlz9uqz6WLF8nEDUrJegtS/JrczYqvZd59/8E5yPj8/5f7Rxf3cNElEsIzaTT6a+vdHi14f/nnWzJkzZ4wJwSWTYRQwfMbMD+bsuFv+j59W/mBZJwOWUejPlzMEFEVR1SknZnYwZBAst9nxIoqiKPH7m78vGt2llTGXJBBCTIdpN5OPT2rDY3zaY0OwTKPWPan4TO7/dPGuepLzHy6EApXU5bFWTPxpxSeGmH4Yl8ew+eFiXQsSSnJ2ddNh6PXclyeRKaqINB52urr+k1MUJU7b19v8Y+YndTqueCr/nsrDfbWZZqPj6to6quT3qA+9PswWMgsoNkGRrpWh6lmLTyGKXBQFC5qiuY7O4Q9/cmN9qBA+WYtP8HKRNwsRTP3WI9ade1P0DwnTmEzckDuH4mVWwQzegEqgkRVQYz6h7uIMgBKCCAoomX+MoETZl+f4azP0PkZQolcnNm7Z8duiHi3Yn215SopOj27BJki9Hnv+XrRX9GZ1Ry0SIYRIXe8+Y8eNGzf+x11PPvyxOnao4WdiEsHtaQ7MTyOo4geHf9u+ZeHHr8CwGvrrJP82g5bu2j4rzJxNIILlNvuBiKIoquT2/LbaKp7TrvP/XjxYnLwhRItECBEsh/GXShqRaIpFUOKUle1xcxIxrMddrWnwx4jTDgyyk51vQTB5rQatvJRe76kkuWfGuaky9INXPZZpX4oTlrTFe51yPOY8kGmv0xGUc/cBrS1aDllzPrFETImKE0/MaMfDl0u/3+HiTz5C0VSV8LPScjLuLu2El95i2nSM8nYLmbTl3NP35WKKEqT9NdKZSyCEGMYxi5d2c3Lvu/J8cpmEoihKlHV+qpcagRBimI04K9tQluQdHmDKIBDTfsotmYz3oWlN8nDLuXYGl2toKpCPG5aS9V3HfX0tWQTTcsDBDNkGo4R/c04bDQIhgmUR80eqWD77rPDHV8ZjzuOGtLJFTxd6cQmVoA3pMukgyT7Qx4RlP/W2gKIoSpB4cc+urX8XIBO/jm5mbj1mbP7ryoPHd8/unNPN/sP6XATL7vuz/Dpz/ydNrn9Ozs9dCEUqKZkgR/R4rufH6Fo7asf7Wmumi18t9uUyW0y4Vv1JUVUwgqIoqvja9JYfVycjuC7jz3+6d/EXRFCNLdJ1+OII6p8vSsMKmgK5jj607ghKkLy3n42aaej88xkKZffGZGKF7xyNKLP1Z/DGVAINq4AaV81ABAX+OyCCAkpGJoJi6tm3biOrtXdLJwsdNoEQIukI6gPxq5+9WZ9peYqezG/FJhChErXzk7ZXWewwUwZCiGC3XSLXryC4McmOWXdMQn/S3xHUhz+8WdyGhRBCBFvdZVQsvp2Lcx8c/G3bqdflFEVRlTcmO7AY5sNOyYVJFedGmjMQQohQC1qX2vDNOxSLoCiq5PaKHp4WZk5hM0+9b9xmQ6J3xyf7fvpUlSC1nHstjvtMK0P8bnsUjyS1QzelfPq7RC8WeX/oy2u/6t3fL3+MoAimaee1j2X7N0RP57fCF99gyEmZi9/QVJVkrA1k4+ao+4TzubIJIcnZ2UWLQAghgmM/8sSne+QU7++pTSKEGFb0Y2GKoqiSvd3VCYQQqf+d7LeSZKzryEYIkbxPc+o/NNw/n48bmpKfI07cEKJDIkIz/LeMWplA9GJxW1UCIURqdVz96e49jY2gRC8WerEQodX70KcdNKLnC3zsPwnkRc8XfLz2FjG7E2Ub8ZL8c+NduPjKsFvNfyKqI/fX1eSqp1poogiKkrzfHvVhbfO6toaqvDrejqnSbtkb8adFVfEIiqIEbzZ3NvzQV0QwTLrvTJa5Ml8UQTWmSNfpX4igGlTQFM91dUdQZY/Xd7Uyajv1ZFoDu+Qak4n/8c7RqDJbX1o25oQKVkCSL6tmIIIC/x0wDwooL0KFZ/IpYyNjYwOtOledIvT0dD8zSVuSeOrMCyGFEEdd85OZzBpB3UINSIQo0ctHT2pqnbGeb1bnqwx9vQ/binADJy+KxMt0MQx9+oweGeWkjpA0Y9+SXW8lWu3CO2h9+k413/beagRCiKq6e+ZCfrMNENfym/bXk/TM1+eXRJk2ruwzrbqtvv4wdmF3R82PKwBT0tLXR3/q7NV+4pHkWsuRlcYt/vVcEdKLHDXA5tNFyJjOI+eO9TZgc80j+oaZ1fFJnqOXj2upLvuKU0d/cwZCSFqckkzP42p4qpKaGvhZMGnYLjrQUDYhSD2vlnhqCMOyy5CIT/fIUbW1MyERQpLshNf8v68Rx8REn0EggmViZiy7RBmbxUQIIWn5+0y+4lf0s/m48Sn5qYpLq1ddKZYSXL/u3WpnAqbr6EnR+iRC0tIbq1ecbYqpeZLCAr4UUeXXDx79ZBYP06lHFydKNseQmpoaHx7Fdx/fz162rJL64b+uGWrNRAhRwucH9t7/0uRsOqRp3wl9cVO19tZQ0oITmw6k6USNHWrPaPxHsB1H79o5yoFDIIQoSfaJSQMW3WuyHX4aWqS/mgYVNMVzXW3SvEs/dY5cQ0w5e2FltGUDlx1sSCYWKnjnaPIy25gTKloBkV+nmgHg3wcRFFBeqm0n7T8m63js2cv3Et492tjFqFbOJQnyc5vUiNPT8Y6QNeVln946VXx8XFkEQpSguKjsy+OWj9+A5FnZaNf6htKCuJM3SqVI/HB1t05yemx4gif4UsJXj58p+RZXbKvOc489fXFh+aBWvI8zNChxwb0N/dt3WXb/k7thUez2IxkSgtMqoL1GrdQyil7zILOsJO3kGJe6Wim1Nx1iGBrq4dcqyz5erkal6ue3MyL19LQ/90eGjrYmXruspKjw79zCDVp58fyurfuvnF3gI/s7hCLxhw+vacB6WJ/Lx1+SkrKqrh2OzZAgxDB3dqlzG1HdsO5BOjhQPHv0WhNkF5aNvTWbQNK8k+MjBm+4k0fPZGe6Tf395/acui45g1X7d2h0HNbXiYUQQuJ3d299YXI2KfWgcaN8uB+2hjq6fk8q/RMlyX/8dqbEZsD47l+4PTJpELnqwMKAD6tKlD9YMmji8Zyme9LSgCL99TSooDUg131KkPznyE79Yx3XXj4yvqVGI75mAzIxvaBF/XeOJi+zjTlhwyqgBn5CpXLkMAAahglJAP5r1FxGzOi/vmeB4u9QVcFPb8VvX78RIr+/l5UitU2MNQlUgDjauppN+jihjlZbzZ3rD6opxDBr3a17W7Vaf+4xGP+XYWX5X9jekWMRPG3P/YEj1k8cNf9IYgWFEKLEuZd+6v6d3s2/hn/Y9Kbm3rV7ZVJEaJuafiZ52ZyGbM3J+tgmkVLSZkpVBuPz+YDJxB9PCWo+2TSYa9vpO1v6X9Li12f2bPv90NXHbz607KgvX1C4qVJS/OrOg0IpQohQU9eo+0QabVq7Mg9eFyJp0aN7CHX90q9OmvUd13vljT0Z4vJX+ycGxu0ZNHf1Lz+0N2YipKqr04ATMd0C/QyXvXgvQdKsNKUqDAyn4eMiVz/4q1CKqMobW7bcH73Cj4MQEtzbtuMe4bfkBz/ul3+Iassf92553n7wgXQRRYmSd48a4mF/aoJL0y2Sp1CR/qoaUNAal+sqX27qtWHThdK2yw53s26W1tEnmThNjOzkk5Vo/jLbmBM2rAJq6Ce8Ql1bK1WJBkCh4gxJAP572B5+PjqnFY4zOO7tfLQ3vyuSit9dOPNssZ/P340OcXWVkEKEautAP5Vm/tKSvNT0UilCDG2Pnj+Ms1OC9kgT1B9GAVMOxQcFDus5+a9UAYUQJck5NWPq3ohjQ0xIhKR5b1OKpAgxSILRTFGhMqWqtPTN6e3rNu69I2kzaPzPp6bf7Gc95mLT7FrWZCkpzs3BaylTVVWVdfdhkIYtbHTJ67lSJC3Ma4ovT+p3X3tgYUaP+dfyxJSo4OHvE4NO7u4/Z/WScR1MGnQHYtq1sCbRewmiKpXsoTVp0HPioEUn1ySJESVO+mPtsem+/fRJfuymfSk6XRYOsW+afEma992870VSxJKHFRSSFl6cNeAnt2sr2vybRVopKFTQGpPrqKJH59OyRVLpzQWDF7S5vLi9VjPUmTKZuEKhRytNXmYbccIGVkAN/YQ8BMB/EIziA/9F6j33vtnVVeHnuro9Zv/YTptESPR6xy973tEDOqQFZ2NvVyAVl9E/fWfZ3GWBqqysQgghaXZ6hhj9H9Hw/OHAlT2DP6wmhaT8czv2p+L/LS+roBBC0tLCgpr/61SVFtzdOKyNvfeYc7o/HLl7eduP3Vx1mjKaa7qUFArxaEZpccHnNoAltbXxc2OC2VS/Qdtv9tlbB6cGmrJxZ3DR4z1TglsGTj2R1pCLxqC/mJa2spUCrt+Y7/3x9Dpp4ekNOxMkktR9W2L5toPGd+M13cdo+S/cv7qLMQNP73u+9rvR+zObftpknUVaKbbJblBBa3CuIywGbl7V1ZiBd98afySrGWakymZibcWehjR5mW34CRtaATXwE1jKVpwBUAREUOBbwPb48a+Tv3axVqHyT02OGrjk0M3EzHcPjy4YMOUvQauxe04u66jR7N+BVMfrF0iznz7MVIrWiOLEL5aGhS178dnWLtOy98bfRtl9GOMmevHwEf7JmprqBEKIqnnx8KmwWb6ZMqSqIGH3YP/gCX+881x84czi7vZqzfAzmyolSX19vCS8tPhtwvvPJBj5YSQjqW/SdD+Ba9dzxeXnd3ZO7GjBxQ3a/Ntr+oaPP634wilSoVBEIYQIhqml0pURht2Qcd3x/Eyq+sG2jXG3tm+/hdqN/t6X06Sfw7Qf/vvvPzh9SMPMo+MG/5bYiImTDS/SX39JiUYUtIbmOob5d1u2DLFjEZQobf8P361/2eS/WjYTK7ZKRZOX2UacsIEVUEM/wVjpijMAipRNSALwbeR0/YCZx8+vDDUxsTcqOr96Qu/uQ38+VRrw69WXtzb0svk3RrOSeiaGqgRClOjR8aNJn23sSzNPrN79XMk6qZhmhuzb65YcK/h8Y1e9XddOHzYhRhIRbtKRPAszTRIhJE47feRW1efeWpUY//y/m6rCJ8sGjfkzqYrRYuj8H1y4qFk0WUqyXb3ccE+J+PW9e3UvESAtKS6TIoQYBj5tm+C7S4tePXuHn1uTul5D1155+fjwrBBzNoEQJUjaOW+TwtdFmpOTTyGEGLa+7b5yiaAQQvLT23S6TBjigld1lqTvHT14e4Ju9JjvGjGytK6Tf5IZwlcc+CUQb4UrLXlw7UllwyfaNbxIS9HX1bCC1uhcRxh2XbNzsocKgaRFV2YP/OlqcdP+DNlMbKpQ3mjyMtuIEzagAuJLG/4JDgiA/yCIoIDyoZrjpMLkA6NHnm1/+Hn85bjr9588e3jt5B/r5nznZ/xPM7Epivq07UAJBI3qBVDxaf1h4b/4DQsOZ9fZIBEnbhk9/4W2hbJNUFSzsTEqOLZ845PPP5JlaOl8mGLMMLW2/vCT/QK8VQiEkCRt36LNr+pqs0jzz0wbtf5Vo7/Z105V4d1dfzypphBi2Dg4suUyTv1vlUgU7zRrspTU7tTZX4tACFFlt85dqattI81LfFskRYhhFtnDvwlSSPxq55Q1d2QyjoZTr8Xnbu7uY8FAiBK/fhCv4FpvUv6jJyliRLCcuvdu1ZTXsCEXgn4HJRLUyPX9sL2+HxOEl2uUVmRmlrUYQvYQcgAADrRJREFUPD5at+H1W90n/wTXffK+rYNsWF8wJ67BRZr9T/Vk0xYs+YvSwIL2JblOI2DR7vnttUlEVT9f990Pf2Y0Yef2J5lYwVVAvrjMyqdlI06oeAUkbMQnNNODJwCaF0RQQMlQVdU1H26IIpFQ8buyVCqRUgghREk+/I8Mybv9Q0J/SO2xaVY7XcXyPKmh+WFSQ0ZK8idtGWnR85fp+IYkEn+29VHHF2dY9ujZDt+Esv+aMmxNrVt4TcKeYd1+FQ6f1UW7wammYPtFknvvzw1rd15ObeBkGqadnQUpeLxy+OzLn9vbqCwx4b0YIUQwbTp38/6QigbdB0fqkQghaemN+X3GHH33aYJJ868siBmfED0rRua1jwGrVFr7On5sCfz9p+ZLVcWStPJtcpYEIYSkOZmfzMSSFuTzqQ/fWvZ1gs3GDV9paTFfomg+bkxK1p2xTfpNGWjLJBCSFp75/VDtSTTSvLhLT0SI0PAfPzlEdjsuJP6Y+p/+IAXK5vsr5x4L5QpDzOxhniyEEEtFlaNQAZLmnjx+s5IidcMmjJJve1KNqBb+4ULUmzXKKyopqrykRC71SIsBE2LMP2wIruY/+vvWtX6atKryY/0mFAobcnL569hr475ZPhr1x1B/fwQlFHxamza4SNff3Keqqz7+rs/9MEV8/qI0uKA1Jtd9xPGYtmtllBEDUeLMI2MHr3rauMmHDcvEn83KjSyzn0/LRpxQ8QrIiPyyaqa+4izOurV3/bo/bmQ0z5hwABoGIiigZKqLij6sUESJ+QWFCg8doT5MdUVUda0ljvhHZ00+9K78xYGfF289EHvh6q17D5++TEhJz84vLq+pu+3EbNHKXZdECInf7py99O6HNoYw6+rqId02pRlwCYSQJPPp0zyZLyit+dBIocpKSuoKoWy+m/5dCxaBEJLkxk0P9e81d9fFR0lZuRlv4i/untfHr/2IONv5m8Y0fOWumtLSDzuhiOobbVN5Y0anwIETJo8M8x+8P7sho3JIno01j6Sqnq8bOGjjszo286x6uH7DhVIKEQyjbvOmtfvYOiH1e86djnexoape7ejr3SZm1qYjF28/uHft9N6lozr5RG/lTlo/3vnv3iFpZRX+KVRlebncV6TKyvFHUzWV9ApPDU9VaVVlFU4tqVQ+EaiqKgGFEELS8tJaTdiamg8NKZFI/PFvKnp6eD606MWORXs+7D9alX5t89jvt76SEAghaX52jhghaXFRiRQhxDA1M8IjvV7duoHztyDpyPpDiZJ683HDU/Jz1AIXbv7ekUMgquzSqiVxcoOUqh6s33SpguAF/7J5vOMnGZEqLS3HqVZWUtqwiEOadGDzKbnxYqSGhhqBCM32YQGq8sdLch7Hyy0MIi2K+3llXBmhH/7r2qGfLvsirqjAjXdxTbVQ8WrhHy5EPSTv3iSVSyVpSW9rNeQ0QscPb8khECKNuo0dbFOrJEuLCopFH+q3isLCygadXI5623kH1nUzqWdtNElh4cdMLC0uyBc1RZH+DFERvaOelJ+f19hem89flAYXNMVznVQkEn+sECR0tTJ06+ZBNiwCSUtuzI0ZdSCtoWOAG5CJ/+nO0bgyW18Gb/gJG1oBNbaaqac4l54fHxj03cRJQ4MDvj9RgAD46igAlEjFs5XB9AZ8hLrv3Nt8iUJvlGT/0V0Pr+zD8pr3pOaTv2Vtjfjss1qCwdWx8oqesPVu/qefVH33J08V/C6CY+DUrmNASxtdrprz8COJRwZ+6Mki2PpOQcN3PBdRFEVJco4MMMM3A6b92Ev8Or9n8c2F7Xl1PrggWCahSxT9uZ8QZ+zubvBhtI3liDMlnztO9Gx+K9bH2+F3J6sb9Blvfm3z4b2Eqk3n2Qef5FV/+KqS4pdHZgebsghEMPQC5l+X/wni1INDHVWIupJe13f25aJPDhcnr+qAjyVUA1YmiD75/m/XdFTHf+O2W5Ygbmyq8o8NwtM7CJWO6959+m0LD/Uz/JCNPH+K/zQb5e7tgT+E0AjdnPHxfcWxwy0+NGIJprZtm44dvOx4aqad5p05PBa3oAk163bhQe7uo09XUxRFiR7OccfPhkndVr0nThvTy8cxeMWzmn/Mxw1LyXrVvN0/zFmNQATTOHjR5eyPyVz2dFsfOzbTqNOvN2udTPBmRQDuliVYLX96UKnwZwmuT7RjEkzTrpufV/z9atndOd6qDN3A5TK/UZK+JpCNfw9TzSZq7qGnRfgiVySfmNXBgMHgtZ9/o1YJKT7//YcFD5j2E66WK1wt1H8h6qmjXv/ex5JJIETqdlx8v9bXkeTs7qZLslxm3BXUrqWKbs1po/7xApLagYvjSxp08jrOeHGSmwrBMBsdV+t7S4pv/9Ralf40vfC1z8ubpkjXUvlqY6QBXW9zPSdfyhM37g7w+YvSsIKmeK6jxBnbu+CbDsNk0DHZervk8jh7nLUIlRa91t0tVOBHNSITK3TnaESZ/YcM3vATNrQCakQ1U09xFtz50fFDfMaw+P7i/2kTCPyXQAQFlIPo4fr+Hb3tdNmfVs8EU9Pcwz984sH0z93DJfmXVv4wMMrH9O+KnWDpuYT0GbUsLo8+e/rpeV1aaNTX5UqouE48/2l9XpO4/3tvvY9zDQgWz6Pviut5Eqr62EAem+ccMnTultjH2dUUJU4/PrtngL2uzLQEQtXMJ2rg97+cSq910y17sX9qWAst5t/fl1SzCZu6+2FhA6MnccLheeOG9wv3MGTLnEvT1r/74FHjpu2Mr9WCq7w1y1OVQIhgWQ46nNOwT6s41FvboN241b+tnjOii7e5BpNgaZrYe7R0tdHjkgQiWDyPXvNPptTd/BTn3d44vJ2pKknQl0jHvd/ySxmyX7H67vphka2MOTLX0cA9fNDCU1kSSfaZnwdHeBj8ncAE29AzauL+v8MoRVJVkn959ZgBoU7a9GN7Qs3Kv9fw2Qdeiyj+1dWje3dsoSXzHfU9Ir+b8Nu9aqrsxprhXX0t1OiTk+q2HWJGzDuaLKYoSdH1n0PNuR+zCdfEd9j6mzkiiqq+O9frY8Rn3XX1vRL6a5yb0urD5xAMbafea+7xJfXmY0nDUlIxJS8Ozu7uYcglSQ1Ln9Cu3SMDnAx5th1Hrr+e/Wng+njnpGExQU4y2ZsgNW3adR04ctKmW2X/+DmC65PcLD39WjubGdq07TZk4pwFcycP7mitbeQzfEt8sWwupBufTPvRm1f3c9FmcnQsnT1cLLRZHEPPngtOpX6SvUQv904b1KW1uerf34uhbd+x1/CZ+18JFKkW6rwQ9dVR6/p1aGmlLVvMWVpWrTr2X/9QNsUqr4z3CF2fKlvwJe8PTQpv526mIddhRLC0rVoFxqy6p/DJa6t5sTbEwEI2gio5N69rsL+npRar1qdZtwoIG7g+/suL9IfflXFoYnj7VjY68p/EUDNxbtMxetGVmoZmy89flAYVNEVynaTo8tLBkb622nStQZBa9h1jhk3YcreSKr+zdfKANoYMmZuRtkPHPrOPvqs3jmpQJm7YnUPxMqtwBm/gCRtRASn8CfUVZ3xkyaWJTlwCIYJjP/pUITSawFdHUFSzTNsHQKmUP9s+bsJZ10WLAiRpGVm5eQX8ktKKaoFAIBTWVJYWZiU9uRP/tspnybMbMxw+GVMgrUh/eDP+XbW2nZePh6UWEyGEpLmvXkhsPUy/aPprVdbTB09Tcqs4pvauro5WOpx/JR2kJW8un38qcOwU4WnQsLGCkoyLux6ZDun+YZRGVfajG7depGXlFlQgdX1zJ9+gQE/jf0gQSUnKw/iX6cWksVOrls6m6s2w/e3XSVWEUE3e60fxr7Kl+s6+fi4G9ASH8uSblx9mM63aBvlafDJWTZDz5OqtxAo1c/e2be11GjqauglTUlKc8vDx64z8SqaOia27t7uJatNflPx8iZ6BBonEpZmvnz9PzChjmTq1bOn6oTjJ5M6MtZ1aTL4mREyHaTeer/CpTLp362lGJYtn7tjSx8mAjZrBl10I5SDNu3/2rUmEv/m/XqSbST0XReGCpnCua+qL0fyZuEFlVpEM3uBKoMEVUBNVM9KiF+fjXpFuoaGuujAFBXx1EEGB/3vSvIuzu/bbZ7n+/oH+pp+rdqUFF8cH9C/5JXN/T1gXCICvUlI/bXy2ZUOSAMjEAADlBGE8+D8nfrW+X58Vj02HTO9tWk92J/W9WrYw0NODEgEAAAAAAOoD7UXw/01wad3KG8XSOlZd+1TNix2H8iN7+8ITQwAAAAAAUB+IoMD/N3FZaaUUIdHrLWOnHE6oqOsQQc69XZM79zvXftXcdpxvLX0AUB70cw6JSASpASATAwCUFxOSAPxfU4sYM8r9zIpnlSXxG/q675/btoN/SycLA00VUlxdwc9OS3oZf/dJkdWApXtPjGipAekFwFcjzi8owjv5FObkCBGCDmEAmRgAoKQYCxYsgFQA/8c4VkHd2+sUvn7+JrtcWF2UnvDk/s1rVy5dunzl2q17rwo0vPvN3Lhr/bhAM1hAAoCvRFya+er+mS2LVp1MLpciRAkykvM0zfRVScRU01KBB30AMjEAQMnAWnzgGyHMf3H14vVHb9LySkUsTZ6+sbW7b2D7VubqMJIVgK9Lknlm1bpLOZJafyC1fEfO6eMIrU8AmRgAoFwgggIAAAAAAAAARcHzdwAAAAAAAABQFERQAAAAAAAAAKAoiKAAAAAAAAAAQFEQQQEAAAAAAACAoiCCAgAAAAAAAABFQQQFAAAAAAAAAIqCCAoAAAAAAAAAFAURFAAAAAAAAAAoCiIoAAAAAAAAAFAURFAAAAAAAAAAoCiIoAAAAAAAAABAURBBAQAAAAAAAICiIIICAAAAAAAAAEVBBAUAAAAAAAAAioIICgAAAAAAAAAUBREUAAAAAAAAACgKIigAAAAAAAAAUBREUAAAAAAAAACgKIigAAAAAAAAAEBREEEBAAAAAAAAgKIgggIAAAAAAAAARUEEBQAAAAAAAACKgggKAAAAAAAAABT1P5BHjOTR9pl9AAAAAElFTkSuQmCC)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "MN_GMVuyPD9D" + }, + "outputs": [], + "source": [ + "\"\"\"\n", + "Code adapted from the torchvision MNIST example:\n", + "https://github.com/pytorch/examples/blob/main/mnist/main.py\n", + "\n", + "BSD 3-Clause License\n", + "\n", + "Copyright (c) 2017, \n", + "All rights reserved.\n", + "\n", + "Redistribution and use in source and binary forms, with or without\n", + "modification, are permitted provided that the following conditions are met:\n", + "\n", + "* Redistributions of source code must retain the above copyright notice, this\n", + " list of conditions and the following disclaimer.\n", + "\n", + "* Redistributions in binary form must reproduce the above copyright notice,\n", + " this list of conditions and the following disclaimer in the documentation\n", + " and/or other materials provided with the distribution.\n", + "\n", + "* Neither the name of the copyright holder nor the names of its\n", + " contributors may be used to endorse or promote products derived from\n", + " this software without specific prior written permission.\n", + "\n", + "THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\n", + "AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n", + "IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\n", + "DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE\n", + "FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n", + "DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\n", + "SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n", + "CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\n", + "OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n", + "OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n", + "\"\"\"\n", + "\n", + "import argparse\n", + "import torch\n", + "import torch.nn as nn\n", + "import torch.nn.functional as F\n", + "import torch.optim as optim\n", + "from torchvision import datasets, transforms\n", + "from torch.optim.lr_scheduler import StepLR" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "id": "Bu5GtxDyPMW5" + }, + "outputs": [], + "source": [ + "# Network implementation -- bonus exercise: Modify the network architecture,\n", + "# and study the effect on the training results.\n", + "\n", + "class Net(nn.Module):\n", + "\n", + " def __init__(self):\n", + " super(Net, self).__init__()\n", + " self.conv1 = nn.Conv2d(1, 32, 3, 1)\n", + " self.conv2 = nn.Conv2d(32, 64, 3, 1)\n", + " self.dropout1 = nn.Dropout(0.25)\n", + " self.dropout2 = nn.Dropout(0.5)\n", + " self.fc1 = nn.Linear(9216, 128)\n", + " self.fc2 = nn.Linear(128, 10)\n", + "\n", + " def forward(self, x):\n", + " x = self.conv1(x)\n", + " x = F.relu(x)\n", + " x = self.conv2(x)\n", + " x = F.relu(x)\n", + " x = F.max_pool2d(x, 2)\n", + " x = self.dropout1(x)\n", + " x = torch.flatten(x, 1)\n", + " x = self.fc1(x)\n", + " x = F.relu(x)\n", + " x = self.dropout2(x)\n", + " x = self.fc2(x)\n", + " output = F.log_softmax(x, dim=1)\n", + " return output" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "id": "FljvS0QcPPTg" + }, + "outputs": [], + "source": [ + "# Exercise: Adapt the following function and store part of the images\n", + "# presented during training in a replay buffer. Replay these images\n", + "# during the training step.\n", + "\n", + "def train(args, model, device, train_loader, optimizer, epoch, buffer = None):\n", + " model.train()\n", + "\n", + " for batch_idx, (data, target) in enumerate(train_loader):\n", + "\n", + " ### START SOLUTION ###\n", + " # We append images and labels from the first training batch\n", + " # to our buffer. You can extend this strategy based on how\n", + " # many images you choose to store in the buffer.\n", + " if buffer is not None and batch_idx == 0:\n", + " if buffer is not None:\n", + " images, targets = next(iter(train_loader))\n", + " buffer.add(images, targets)\n", + " ### END SOLUTION ###\n", + "\n", + "\n", + " data, target = data.to(device), target.to(device)\n", + "\n", + " optimizer.zero_grad()\n", + " output = model(data)\n", + " loss = F.nll_loss(output, target)\n", + " loss.backward()\n", + " optimizer.step()\n", + "\n", + " ### START SOLUTION ###\n", + " # A simple strategy for overcoming forgetting is to retrieve images\n", + " # from the buffer (here: one image per class) and perform a gradient\n", + " # step on these images along with every incoming new batch.\n", + " if buffer is not None and len(buffer) > 0:\n", + " replayed_images, replayed_targets = buffer.get()\n", + " replayed_images = replayed_images.to(device)\n", + " replayed_targets = replayed_targets.to(device)\n", + " optimizer.zero_grad()\n", + " output = model(replayed_images)\n", + " loss = F.nll_loss(output, replayed_targets)\n", + " loss.backward()\n", + " optimizer.step()\n", + " ### END SOLUTION ###\n", + "\n", + " if batch_idx % args.log_interval == 0:\n", + " print('Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n", + " epoch, batch_idx * len(data), len(train_loader.dataset),\n", + " 100. * batch_idx / len(train_loader), loss.item()))\n", + " if args.dry_run:\n", + " break" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "id": "QXPgWNCRPRd4" + }, + "outputs": [], + "source": [ + "# The test routine was adapted from the original implementation and now computes\n", + "# the classification probabilities per class instead of an average. This allows to\n", + "# later assess the effect of catastrophic forgetting. No adaptations in this function\n", + "# are required for the exercise.\n", + "\n", + "import collections\n", + "\n", + "def test(model, device, test_loader):\n", + " model.eval()\n", + " test_loss = 0\n", + " correct_by_class = collections.Counter()\n", + " count_by_class = collections.Counter()\n", + " with torch.no_grad():\n", + " for data, target in test_loader:\n", + " data, target = data.to(device), target.to(device)\n", + " output = model(data)\n", + " test_loss += F.nll_loss(output, target, reduction='sum').item() # sum up batch loss\n", + " pred = output.argmax(dim=1, keepdim=True) # get the index of the max log-probability\n", + " correct = pred.eq(target.view_as(pred)).float()\n", + " for class_ in range(10):\n", + " idc = (target == class_)\n", + " correct_by_class[class_] += correct[idc].sum().item()\n", + " count_by_class[class_] += idc.sum().item()\n", + "\n", + " test_loss /= len(test_loader.dataset)\n", + " test_acc = correct / len(test_loader.dataset)\n", + "\n", + " print(f'\\nTest set: Average loss: {test_loss:.4f}')\n", + " result = {}\n", + " for class_ in range(10):\n", + " acc = correct_by_class[class_] / count_by_class[class_]\n", + " result[class_] = acc\n", + " print(\n", + " f\"Class {class_} accuracy: \"\n", + " f\"{correct_by_class[class_]}/{count_by_class[class_]}\"\n", + " f\"({acc*100:.0f}%)\"\n", + " )\n", + "\n", + " return result" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "id": "UsdJlkLkRRbp" + }, + "outputs": [], + "source": [ + "# Exercise: Adapt the dataset class \"MNISTContinualLearning\" for the experiment\n", + "# outlined in the introduction text. The class needs to support indexing of the\n", + "# dataset based on the provided list of classes.\n", + "\n", + "def identity(x):\n", + " return\n", + "\n", + "class MNISTContinualLearning(datasets.MNIST):\n", + "\n", + " def __init__(self, *args, classes=list(range(10)), **kwargs):\n", + "\n", + " # This inherits from the base dataset\n", + " super().__init__(*args, **kwargs)\n", + "\n", + " if len(classes) < 2:\n", + " raise ValueError(f\"Need at least two classes, but got {len(classes)}\")\n", + "\n", + " # Add code for filtering the dataset here. You need to adapt\n", + " # the \"data\" and \"targets\" attribute of the dataset. The \"data\"\n", + " # attribute stores the images as a numpy array, while the \"targets\"\n", + " # attributes stores the labels as a numpy array.\n", + " # You can override the existing attributes.\n", + " #\n", + " # self.data = ...\n", + " # self.targets = ...\n", + " \n", + " ### START SOLUTION ###\n", + " idc = None\n", + " for class_ in classes:\n", + " if idc is None:\n", + " idc = self.targets == class_\n", + " idc |= self.targets == class_\n", + "\n", + " self.data = self.data[idc]\n", + " self.targets = self.targets[idc]\n", + " ### END SOLUTION ###\n", + "\n", + "\n", + "def test_dataset():\n", + " transform=transforms.Compose([\n", + " transforms.ToTensor(),\n", + " transforms.Normalize((0.1307,), (0.3081,))\n", + " ])\n", + " dataset1 = MNISTContinualLearning('../data', train=True, download=True, transform=transform, classes = [0, 5])\n", + " assert len(dataset1) == 11344\n", + "\n", + "test_dataset()" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "id": "EDwtthksNUpw" + }, + "outputs": [], + "source": [ + "# To implement memory replay, add additional code (functions or classes)\n", + "# to this cell. One possible solution is to implement a \"Buffer\" class\n", + "# that allows to add images into memory, and allows to retrieve the stored\n", + "# images and labels for training the model.\n", + "#\n", + "# The buffer is typically memory constrained, and there are multiple ways\n", + "# to efficiently compress the individual elements present in the buffer.\n", + "# Think about different ways of reducing the storage required by your buffer\n", + "# class, and explore which of them is most effective at mitigating catastophic\n", + "# forgetting.\n", + "\n", + "\n", + "### START SOLUTION ###\n", + "\n", + "class Buffer(nn.Module):\n", + "\n", + " def __init__(self):\n", + " self.buffer = {}\n", + " \n", + " def add(self, images, targets):\n", + " for class_ in targets.unique():\n", + " idc = targets == class_\n", + " self.buffer[class_] = images[idc][0]\n", + "\n", + " def get(self):\n", + " assert len(self) > 0\n", + " keys = list(self.buffer.keys())\n", + " targets = torch.tensor(keys)\n", + " images = torch.stack([self.buffer[k] for k in keys], dim = 0)\n", + " assert len(targets) == len(images)\n", + " return images, targets\n", + "\n", + " def __len__(self):\n", + " return sum(len(v) if v is not None else 0 for v in self.buffer.values())\n", + " \n", + "### END SOLUTION ###" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "id": "u2xvlBoZPTyx" + }, + "outputs": [], + "source": [ + "# Adapt the main training loop to work with the functions you defined above.\n", + "\n", + "def train_model(args, model, phase, replay_buffer = [], history = [], buffer = None):\n", + " use_cuda = not args.no_cuda and torch.cuda.is_available()\n", + " device = torch.device(\"cuda\" if use_cuda else \"cpu\")\n", + "\n", + " torch.manual_seed(args.seed)\n", + "\n", + " train_kwargs = {'batch_size': args.batch_size}\n", + " test_kwargs = {'batch_size': args.test_batch_size}\n", + " if use_cuda:\n", + " cuda_kwargs = {'num_workers': 1,\n", + " 'pin_memory': True,\n", + " 'shuffle': True}\n", + " train_kwargs.update(cuda_kwargs)\n", + " test_kwargs.update(cuda_kwargs)\n", + "\n", + " transform=transforms.Compose([\n", + " transforms.ToTensor(),\n", + " transforms.Normalize((0.1307,), (0.3081,))\n", + " ])\n", + " \n", + " # You need to replace the original MNIST dataset here by the continual\n", + " # learning dataset we implemented in the previous cell. Make sure to\n", + " # pass the list of classes selected for training in args.train_classes.\n", + " \n", + " ### START SOLUTION ###\n", + " # We added the revised MNIST class here, which takes the same arguments\n", + " # as the original class, but additionally takes a \"classes\" argument which\n", + " # specifies the subselection of classes to consider during this training\n", + " # phase.\n", + " dataset1 = MNISTContinualLearning('../data', train=True, download=True,\n", + " transform=transform, classes=args.train_classes)\n", + " ### END SOLUTION ###\n", + " dataset2 = datasets.MNIST('../data', train=False, download=True,\n", + " transform=transform)\n", + " train_loader = torch.utils.data.DataLoader(dataset1,**train_kwargs)\n", + " test_loader = torch.utils.data.DataLoader(dataset2, **test_kwargs)\n", + "\n", + " model = model.to(device)\n", + " optimizer = optim.Adadelta(model.parameters(), lr=args.lr)\n", + " scheduler = StepLR(optimizer, step_size=1, gamma=args.gamma)\n", + "\n", + " for epoch in range(1, args.epochs + 1):\n", + " ### START SOLUTION ###\n", + " # We additionally pass the \"buffer\" class here to collect training\n", + " # images for later memory replay.\n", + " # \n", + " # Original content:\n", + " # train(args, model, device, train_loader, optimizer, epoch)\n", + " train(args, model, device, train_loader, optimizer, epoch, buffer)\n", + " ### END SOLUTION ###\n", + " test_results = test(model, device, test_loader)\n", + " test_results[\"phase\"] = phase\n", + " history.append(test_results)\n", + " scheduler.step()\n", + "\n", + " return history\n" + ] + }, + { + "cell_type": "code", + "source": [ + "# We now define the experiment setup.\n", + "# Without modifications to the code, the following will simply train an MNIST\n", + "# network multiple times, and reach an accuracy of >99% on the test set.\n", + "#\n", + "# The result will be modified in two steps:\n", + "#\n", + "# -- first, after implementing the MNISTContinualLearning dataset, you will be\n", + "# able to observe catastrophic forgetting: Training on a new task (specified)\n", + "# by the \"phase\" and \"config.train_classes\" variables will make the network\n", + "# forget the previously learned tasks, and you will observe a performance drop.\n", + "#\n", + "# -- second, after implementing the catastrophic forgetting network, you will\n", + "# fix the catastrophic forgetting my implementing a simple memory buffer. This\n", + "# buffer will keep some of the images seen in each individual training phase,\n", + "# and add them to the training in each subsequent phase. This task is open-ended\n", + "# and different strategies exist. They will differ in terms of memory efficiency\n", + "# and performance.\n", + "\n", + "# You should adapt the train config to each experiment setup and especially test the\n", + "# effects of taking different values for the number of epochs (per phase of the training)\n", + "# and the learning rate used.\n", + "config = argparse.Namespace(\n", + " batch_size=64, \n", + " test_batch_size=1000, \n", + " epochs=5, \n", + " lr=0.001,\n", + " gamma=0.7,\n", + " no_cuda=False,\n", + " dry_run=False,\n", + " seed=1,\n", + " log_interval=10,\n", + " save_model=False\n", + ")\n", + "\n", + "def train_regular_mnist():\n", + " model = Net()\n", + " history = []\n", + "\n", + " config.train_classes = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n", + " train_model(config, model, phase = \"baseline\", history = history)\n", + " return history\n", + "\n", + "history_regular_mnist = train_regular_mnist()\n", + "history_regular_mnist" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "SK-GNifLPN2S", + "outputId": "9cd9a6a2-9b54-46c8-a072-fcd66b995364" + }, + "execution_count": 14, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Train Epoch: 1 [0/60000 (0%)]\tLoss: 2.295775\n", + "Train Epoch: 1 [640/60000 (1%)]\tLoss: 2.290210\n", + "Train Epoch: 1 [1280/60000 (2%)]\tLoss: 2.310344\n", + "Train Epoch: 1 [1920/60000 (3%)]\tLoss: 2.292868\n", + "Train Epoch: 1 [2560/60000 (4%)]\tLoss: 2.290216\n", + "Train Epoch: 1 [3200/60000 (5%)]\tLoss: 2.276424\n", + "Train Epoch: 1 [3840/60000 (6%)]\tLoss: 2.281419\n", + "Train Epoch: 1 [4480/60000 (7%)]\tLoss: 2.276124\n", + "Train Epoch: 1 [5120/60000 (9%)]\tLoss: 2.267236\n", + "Train Epoch: 1 [5760/60000 (10%)]\tLoss: 2.294219\n", + "Train Epoch: 1 [6400/60000 (11%)]\tLoss: 2.254271\n", + "Train Epoch: 1 [7040/60000 (12%)]\tLoss: 2.264095\n", + "Train Epoch: 1 [7680/60000 (13%)]\tLoss: 2.256358\n", + "Train Epoch: 1 [8320/60000 (14%)]\tLoss: 2.274228\n", + "Train Epoch: 1 [8960/60000 (15%)]\tLoss: 2.269273\n", + "Train Epoch: 1 [9600/60000 (16%)]\tLoss: 2.240185\n", + "Train Epoch: 1 [10240/60000 (17%)]\tLoss: 2.248815\n", + "Train Epoch: 1 [10880/60000 (18%)]\tLoss: 2.223281\n", + "Train Epoch: 1 [11520/60000 (19%)]\tLoss: 2.222255\n", + "Train Epoch: 1 [12160/60000 (20%)]\tLoss: 2.233015\n", + "Train Epoch: 1 [12800/60000 (21%)]\tLoss: 2.211506\n", + "Train Epoch: 1 [13440/60000 (22%)]\tLoss: 2.234103\n", + "Train Epoch: 1 [14080/60000 (23%)]\tLoss: 2.225921\n", + "Train Epoch: 1 [14720/60000 (25%)]\tLoss: 2.202645\n", + "Train Epoch: 1 [15360/60000 (26%)]\tLoss: 2.196408\n", + "Train Epoch: 1 [16000/60000 (27%)]\tLoss: 2.202403\n", + "Train Epoch: 1 [16640/60000 (28%)]\tLoss: 2.209469\n", + "Train Epoch: 1 [17280/60000 (29%)]\tLoss: 2.180980\n", + "Train Epoch: 1 [17920/60000 (30%)]\tLoss: 2.153812\n", + "Train Epoch: 1 [18560/60000 (31%)]\tLoss: 2.150105\n", + "Train Epoch: 1 [19200/60000 (32%)]\tLoss: 2.158034\n", + "Train Epoch: 1 [19840/60000 (33%)]\tLoss: 2.172502\n", + "Train Epoch: 1 [20480/60000 (34%)]\tLoss: 2.093013\n", + "Train Epoch: 1 [21120/60000 (35%)]\tLoss: 2.130392\n", + "Train Epoch: 1 [21760/60000 (36%)]\tLoss: 2.172241\n", + "Train Epoch: 1 [22400/60000 (37%)]\tLoss: 2.162427\n", + "Train Epoch: 1 [23040/60000 (38%)]\tLoss: 2.168904\n", + "Train Epoch: 1 [23680/60000 (39%)]\tLoss: 2.179659\n", + "Train Epoch: 1 [24320/60000 (41%)]\tLoss: 2.163229\n", + "Train Epoch: 1 [24960/60000 (42%)]\tLoss: 2.126366\n", + "Train Epoch: 1 [25600/60000 (43%)]\tLoss: 2.138455\n", + "Train Epoch: 1 [26240/60000 (44%)]\tLoss: 2.113500\n", + "Train Epoch: 1 [26880/60000 (45%)]\tLoss: 2.067201\n", + "Train Epoch: 1 [27520/60000 (46%)]\tLoss: 2.114285\n", + "Train Epoch: 1 [28160/60000 (47%)]\tLoss: 2.036888\n", + "Train Epoch: 1 [28800/60000 (48%)]\tLoss: 2.068524\n", + "Train Epoch: 1 [29440/60000 (49%)]\tLoss: 2.097640\n", + "Train Epoch: 1 [30080/60000 (50%)]\tLoss: 2.028418\n", + "Train Epoch: 1 [30720/60000 (51%)]\tLoss: 2.031548\n", + "Train Epoch: 1 [31360/60000 (52%)]\tLoss: 2.051078\n", + "Train Epoch: 1 [32000/60000 (53%)]\tLoss: 2.028303\n", + "Train Epoch: 1 [32640/60000 (54%)]\tLoss: 1.983975\n", + "Train Epoch: 1 [33280/60000 (55%)]\tLoss: 2.043919\n", + "Train Epoch: 1 [33920/60000 (57%)]\tLoss: 1.980050\n", + "Train Epoch: 1 [34560/60000 (58%)]\tLoss: 1.970208\n", + "Train Epoch: 1 [35200/60000 (59%)]\tLoss: 2.048699\n", + "Train Epoch: 1 [35840/60000 (60%)]\tLoss: 1.956716\n", + "Train Epoch: 1 [36480/60000 (61%)]\tLoss: 1.987072\n", + "Train Epoch: 1 [37120/60000 (62%)]\tLoss: 1.984343\n", + "Train Epoch: 1 [37760/60000 (63%)]\tLoss: 1.947972\n", + "Train Epoch: 1 [38400/60000 (64%)]\tLoss: 1.938421\n", + "Train Epoch: 1 [39040/60000 (65%)]\tLoss: 1.919820\n", + "Train Epoch: 1 [39680/60000 (66%)]\tLoss: 1.931831\n", + "Train Epoch: 1 [40320/60000 (67%)]\tLoss: 1.953799\n", + "Train Epoch: 1 [40960/60000 (68%)]\tLoss: 1.883609\n", + "Train Epoch: 1 [41600/60000 (69%)]\tLoss: 1.876115\n", + "Train Epoch: 1 [42240/60000 (70%)]\tLoss: 1.870510\n", + "Train Epoch: 1 [42880/60000 (71%)]\tLoss: 1.910781\n", + "Train Epoch: 1 [43520/60000 (72%)]\tLoss: 1.950781\n", + "Train Epoch: 1 [44160/60000 (74%)]\tLoss: 1.985367\n", + "Train Epoch: 1 [44800/60000 (75%)]\tLoss: 1.823831\n", + "Train Epoch: 1 [45440/60000 (76%)]\tLoss: 1.870280\n", + "Train Epoch: 1 [46080/60000 (77%)]\tLoss: 1.794647\n", + "Train Epoch: 1 [46720/60000 (78%)]\tLoss: 1.864569\n", + "Train Epoch: 1 [47360/60000 (79%)]\tLoss: 1.871643\n", + "Train Epoch: 1 [48000/60000 (80%)]\tLoss: 1.794408\n", + "Train Epoch: 1 [48640/60000 (81%)]\tLoss: 1.836012\n", + "Train Epoch: 1 [49280/60000 (82%)]\tLoss: 1.784632\n", + "Train Epoch: 1 [49920/60000 (83%)]\tLoss: 1.775053\n", + "Train Epoch: 1 [50560/60000 (84%)]\tLoss: 1.691660\n", + "Train Epoch: 1 [51200/60000 (85%)]\tLoss: 1.818502\n", + "Train Epoch: 1 [51840/60000 (86%)]\tLoss: 1.792987\n", + "Train Epoch: 1 [52480/60000 (87%)]\tLoss: 1.763221\n", + "Train Epoch: 1 [53120/60000 (88%)]\tLoss: 1.779346\n", + "Train Epoch: 1 [53760/60000 (90%)]\tLoss: 1.678108\n", + "Train Epoch: 1 [54400/60000 (91%)]\tLoss: 1.780516\n", + "Train Epoch: 1 [55040/60000 (92%)]\tLoss: 1.650490\n", + "Train Epoch: 1 [55680/60000 (93%)]\tLoss: 1.704799\n", + "Train Epoch: 1 [56320/60000 (94%)]\tLoss: 1.720001\n", + "Train Epoch: 1 [56960/60000 (95%)]\tLoss: 1.716193\n", + "Train Epoch: 1 [57600/60000 (96%)]\tLoss: 1.665386\n", + "Train Epoch: 1 [58240/60000 (97%)]\tLoss: 1.669093\n", + "Train Epoch: 1 [58880/60000 (98%)]\tLoss: 1.669740\n", + "Train Epoch: 1 [59520/60000 (99%)]\tLoss: 1.576309\n", + "\n", + "Test set: Average loss: 1.5640\n", + "Class 0 accuracy: 942.0/980(96%)\n", + "Class 1 accuracy: 1049.0/1135(92%)\n", + "Class 2 accuracy: 530.0/1032(51%)\n", + "Class 3 accuracy: 928.0/1010(92%)\n", + "Class 4 accuracy: 747.0/982(76%)\n", + "Class 5 accuracy: 299.0/892(34%)\n", + "Class 6 accuracy: 880.0/958(92%)\n", + "Class 7 accuracy: 851.0/1028(83%)\n", + "Class 8 accuracy: 744.0/974(76%)\n", + "Class 9 accuracy: 788.0/1009(78%)\n", + "Train Epoch: 2 [0/60000 (0%)]\tLoss: 1.627061\n", + "Train Epoch: 2 [640/60000 (1%)]\tLoss: 1.545597\n", + "Train Epoch: 2 [1280/60000 (2%)]\tLoss: 1.698457\n", + "Train Epoch: 2 [1920/60000 (3%)]\tLoss: 1.572205\n", + "Train Epoch: 2 [2560/60000 (4%)]\tLoss: 1.636262\n", + "Train Epoch: 2 [3200/60000 (5%)]\tLoss: 1.641547\n", + "Train Epoch: 2 [3840/60000 (6%)]\tLoss: 1.528854\n", + "Train Epoch: 2 [4480/60000 (7%)]\tLoss: 1.606497\n", + "Train Epoch: 2 [5120/60000 (9%)]\tLoss: 1.548691\n", + "Train Epoch: 2 [5760/60000 (10%)]\tLoss: 1.562155\n", + "Train Epoch: 2 [6400/60000 (11%)]\tLoss: 1.498272\n", + "Train Epoch: 2 [7040/60000 (12%)]\tLoss: 1.519710\n", + "Train Epoch: 2 [7680/60000 (13%)]\tLoss: 1.605046\n", + "Train Epoch: 2 [8320/60000 (14%)]\tLoss: 1.567077\n", + "Train Epoch: 2 [8960/60000 (15%)]\tLoss: 1.434826\n", + "Train Epoch: 2 [9600/60000 (16%)]\tLoss: 1.478333\n", + "Train Epoch: 2 [10240/60000 (17%)]\tLoss: 1.479111\n", + "Train Epoch: 2 [10880/60000 (18%)]\tLoss: 1.503707\n", + "Train Epoch: 2 [11520/60000 (19%)]\tLoss: 1.373165\n", + "Train Epoch: 2 [12160/60000 (20%)]\tLoss: 1.460236\n", + "Train Epoch: 2 [12800/60000 (21%)]\tLoss: 1.565628\n", + "Train Epoch: 2 [13440/60000 (22%)]\tLoss: 1.565041\n", + "Train Epoch: 2 [14080/60000 (23%)]\tLoss: 1.426577\n", + "Train Epoch: 2 [14720/60000 (25%)]\tLoss: 1.531582\n", + "Train Epoch: 2 [15360/60000 (26%)]\tLoss: 1.427804\n", + "Train Epoch: 2 [16000/60000 (27%)]\tLoss: 1.453471\n", + "Train Epoch: 2 [16640/60000 (28%)]\tLoss: 1.528445\n", + "Train Epoch: 2 [17280/60000 (29%)]\tLoss: 1.515689\n", + "Train Epoch: 2 [17920/60000 (30%)]\tLoss: 1.517196\n", + "Train Epoch: 2 [18560/60000 (31%)]\tLoss: 1.446620\n", + "Train Epoch: 2 [19200/60000 (32%)]\tLoss: 1.496210\n", + "Train Epoch: 2 [19840/60000 (33%)]\tLoss: 1.423505\n", + "Train Epoch: 2 [20480/60000 (34%)]\tLoss: 1.415124\n", + "Train Epoch: 2 [21120/60000 (35%)]\tLoss: 1.544892\n", + "Train Epoch: 2 [21760/60000 (36%)]\tLoss: 1.340283\n", + "Train Epoch: 2 [22400/60000 (37%)]\tLoss: 1.363816\n", + "Train Epoch: 2 [23040/60000 (38%)]\tLoss: 1.382646\n", + "Train Epoch: 2 [23680/60000 (39%)]\tLoss: 1.264206\n", + "Train Epoch: 2 [24320/60000 (41%)]\tLoss: 1.332618\n", + "Train Epoch: 2 [24960/60000 (42%)]\tLoss: 1.358470\n", + "Train Epoch: 2 [25600/60000 (43%)]\tLoss: 1.344868\n", + "Train Epoch: 2 [26240/60000 (44%)]\tLoss: 1.327641\n", + "Train Epoch: 2 [26880/60000 (45%)]\tLoss: 1.382141\n", + "Train Epoch: 2 [27520/60000 (46%)]\tLoss: 1.403416\n", + "Train Epoch: 2 [28160/60000 (47%)]\tLoss: 1.420001\n", + "Train Epoch: 2 [28800/60000 (48%)]\tLoss: 1.414284\n", + "Train Epoch: 2 [29440/60000 (49%)]\tLoss: 1.367697\n", + "Train Epoch: 2 [30080/60000 (50%)]\tLoss: 1.498880\n", + "Train Epoch: 2 [30720/60000 (51%)]\tLoss: 1.278797\n", + "Train Epoch: 2 [31360/60000 (52%)]\tLoss: 1.333852\n", + "Train Epoch: 2 [32000/60000 (53%)]\tLoss: 1.370776\n", + "Train Epoch: 2 [32640/60000 (54%)]\tLoss: 1.062860\n", + "Train Epoch: 2 [33280/60000 (55%)]\tLoss: 1.245440\n", + "Train Epoch: 2 [33920/60000 (57%)]\tLoss: 1.152776\n", + "Train Epoch: 2 [34560/60000 (58%)]\tLoss: 1.344226\n", + "Train Epoch: 2 [35200/60000 (59%)]\tLoss: 1.242468\n", + "Train Epoch: 2 [35840/60000 (60%)]\tLoss: 1.353402\n", + "Train Epoch: 2 [36480/60000 (61%)]\tLoss: 1.166169\n", + "Train Epoch: 2 [37120/60000 (62%)]\tLoss: 1.165925\n", + "Train Epoch: 2 [37760/60000 (63%)]\tLoss: 1.237298\n", + "Train Epoch: 2 [38400/60000 (64%)]\tLoss: 1.209161\n", + "Train Epoch: 2 [39040/60000 (65%)]\tLoss: 1.284919\n", + "Train Epoch: 2 [39680/60000 (66%)]\tLoss: 1.301399\n", + "Train Epoch: 2 [40320/60000 (67%)]\tLoss: 1.257363\n", + "Train Epoch: 2 [40960/60000 (68%)]\tLoss: 1.199494\n", + "Train Epoch: 2 [41600/60000 (69%)]\tLoss: 1.113902\n", + "Train Epoch: 2 [42240/60000 (70%)]\tLoss: 1.155495\n", + "Train Epoch: 2 [42880/60000 (71%)]\tLoss: 1.227993\n", + "Train Epoch: 2 [43520/60000 (72%)]\tLoss: 1.231888\n", + "Train Epoch: 2 [44160/60000 (74%)]\tLoss: 1.373095\n", + "Train Epoch: 2 [44800/60000 (75%)]\tLoss: 1.248195\n", + "Train Epoch: 2 [45440/60000 (76%)]\tLoss: 1.127467\n", + "Train Epoch: 2 [46080/60000 (77%)]\tLoss: 1.045080\n", + "Train Epoch: 2 [46720/60000 (78%)]\tLoss: 1.206602\n", + "Train Epoch: 2 [47360/60000 (79%)]\tLoss: 1.128123\n", + "Train Epoch: 2 [48000/60000 (80%)]\tLoss: 1.348677\n", + "Train Epoch: 2 [48640/60000 (81%)]\tLoss: 1.197632\n", + "Train Epoch: 2 [49280/60000 (82%)]\tLoss: 1.165728\n", + "Train Epoch: 2 [49920/60000 (83%)]\tLoss: 1.209910\n", + "Train Epoch: 2 [50560/60000 (84%)]\tLoss: 1.137662\n", + "Train Epoch: 2 [51200/60000 (85%)]\tLoss: 1.109170\n", + "Train Epoch: 2 [51840/60000 (86%)]\tLoss: 1.038135\n", + "Train Epoch: 2 [52480/60000 (87%)]\tLoss: 1.092770\n", + "Train Epoch: 2 [53120/60000 (88%)]\tLoss: 1.205910\n", + "Train Epoch: 2 [53760/60000 (90%)]\tLoss: 0.988055\n", + "Train Epoch: 2 [54400/60000 (91%)]\tLoss: 1.205869\n", + "Train Epoch: 2 [55040/60000 (92%)]\tLoss: 1.149507\n", + "Train Epoch: 2 [55680/60000 (93%)]\tLoss: 1.059419\n", + "Train Epoch: 2 [56320/60000 (94%)]\tLoss: 1.130637\n", + "Train Epoch: 2 [56960/60000 (95%)]\tLoss: 1.067291\n", + "Train Epoch: 2 [57600/60000 (96%)]\tLoss: 1.102901\n", + "Train Epoch: 2 [58240/60000 (97%)]\tLoss: 1.082644\n", + "Train Epoch: 2 [58880/60000 (98%)]\tLoss: 1.176209\n", + "Train Epoch: 2 [59520/60000 (99%)]\tLoss: 1.100892\n", + "\n", + "Test set: Average loss: 0.9323\n", + "Class 0 accuracy: 937.0/980(96%)\n", + "Class 1 accuracy: 1083.0/1135(95%)\n", + "Class 2 accuracy: 748.0/1032(72%)\n", + "Class 3 accuracy: 923.0/1010(91%)\n", + "Class 4 accuracy: 826.0/982(84%)\n", + "Class 5 accuracy: 470.0/892(53%)\n", + "Class 6 accuracy: 886.0/958(92%)\n", + "Class 7 accuracy: 898.0/1028(87%)\n", + "Class 8 accuracy: 732.0/974(75%)\n", + "Class 9 accuracy: 844.0/1009(84%)\n", + "Train Epoch: 3 [0/60000 (0%)]\tLoss: 1.117987\n", + "Train Epoch: 3 [640/60000 (1%)]\tLoss: 1.044462\n", + "Train Epoch: 3 [1280/60000 (2%)]\tLoss: 1.027436\n", + "Train Epoch: 3 [1920/60000 (3%)]\tLoss: 1.230957\n", + "Train Epoch: 3 [2560/60000 (4%)]\tLoss: 1.004591\n", + "Train Epoch: 3 [3200/60000 (5%)]\tLoss: 1.060180\n", + "Train Epoch: 3 [3840/60000 (6%)]\tLoss: 0.902412\n", + "Train Epoch: 3 [4480/60000 (7%)]\tLoss: 0.957223\n", + "Train Epoch: 3 [5120/60000 (9%)]\tLoss: 1.091123\n", + "Train Epoch: 3 [5760/60000 (10%)]\tLoss: 1.175083\n", + "Train Epoch: 3 [6400/60000 (11%)]\tLoss: 0.969085\n", + "Train Epoch: 3 [7040/60000 (12%)]\tLoss: 1.044883\n", + "Train Epoch: 3 [7680/60000 (13%)]\tLoss: 1.010503\n", + "Train Epoch: 3 [8320/60000 (14%)]\tLoss: 1.037116\n", + "Train Epoch: 3 [8960/60000 (15%)]\tLoss: 0.968517\n", + "Train Epoch: 3 [9600/60000 (16%)]\tLoss: 1.037061\n", + "Train Epoch: 3 [10240/60000 (17%)]\tLoss: 0.992225\n", + "Train Epoch: 3 [10880/60000 (18%)]\tLoss: 1.091760\n", + "Train Epoch: 3 [11520/60000 (19%)]\tLoss: 1.100666\n", + "Train Epoch: 3 [12160/60000 (20%)]\tLoss: 0.852405\n", + "Train Epoch: 3 [12800/60000 (21%)]\tLoss: 0.957409\n", + "Train Epoch: 3 [13440/60000 (22%)]\tLoss: 0.959758\n", + "Train Epoch: 3 [14080/60000 (23%)]\tLoss: 0.998838\n", + "Train Epoch: 3 [14720/60000 (25%)]\tLoss: 1.045380\n", + "Train Epoch: 3 [15360/60000 (26%)]\tLoss: 0.842424\n", + "Train Epoch: 3 [16000/60000 (27%)]\tLoss: 1.003876\n", + "Train Epoch: 3 [16640/60000 (28%)]\tLoss: 0.948943\n", + "Train Epoch: 3 [17280/60000 (29%)]\tLoss: 1.090439\n", + "Train Epoch: 3 [17920/60000 (30%)]\tLoss: 1.087915\n", + "Train Epoch: 3 [18560/60000 (31%)]\tLoss: 1.145274\n", + "Train Epoch: 3 [19200/60000 (32%)]\tLoss: 1.217145\n", + "Train Epoch: 3 [19840/60000 (33%)]\tLoss: 1.085176\n", + "Train Epoch: 3 [20480/60000 (34%)]\tLoss: 1.083079\n", + "Train Epoch: 3 [21120/60000 (35%)]\tLoss: 0.999794\n", + "Train Epoch: 3 [21760/60000 (36%)]\tLoss: 0.911105\n", + "Train Epoch: 3 [22400/60000 (37%)]\tLoss: 1.273139\n", + "Train Epoch: 3 [23040/60000 (38%)]\tLoss: 1.048052\n", + "Train Epoch: 3 [23680/60000 (39%)]\tLoss: 0.818689\n", + "Train Epoch: 3 [24320/60000 (41%)]\tLoss: 0.858526\n", + "Train Epoch: 3 [24960/60000 (42%)]\tLoss: 0.982630\n", + "Train Epoch: 3 [25600/60000 (43%)]\tLoss: 1.042941\n", + "Train Epoch: 3 [26240/60000 (44%)]\tLoss: 0.818404\n", + "Train Epoch: 3 [26880/60000 (45%)]\tLoss: 0.947276\n", + "Train Epoch: 3 [27520/60000 (46%)]\tLoss: 0.990826\n", + "Train Epoch: 3 [28160/60000 (47%)]\tLoss: 0.956242\n", + "Train Epoch: 3 [28800/60000 (48%)]\tLoss: 1.052988\n", + "Train Epoch: 3 [29440/60000 (49%)]\tLoss: 1.157221\n", + "Train Epoch: 3 [30080/60000 (50%)]\tLoss: 0.913521\n", + "Train Epoch: 3 [30720/60000 (51%)]\tLoss: 1.009682\n", + "Train Epoch: 3 [31360/60000 (52%)]\tLoss: 0.770679\n", + "Train Epoch: 3 [32000/60000 (53%)]\tLoss: 0.990214\n", + "Train Epoch: 3 [32640/60000 (54%)]\tLoss: 0.937079\n", + "Train Epoch: 3 [33280/60000 (55%)]\tLoss: 0.872119\n", + "Train Epoch: 3 [33920/60000 (57%)]\tLoss: 0.840864\n", + "Train Epoch: 3 [34560/60000 (58%)]\tLoss: 1.002336\n", + "Train Epoch: 3 [35200/60000 (59%)]\tLoss: 0.915164\n", + "Train Epoch: 3 [35840/60000 (60%)]\tLoss: 1.012089\n", + "Train Epoch: 3 [36480/60000 (61%)]\tLoss: 0.939280\n", + "Train Epoch: 3 [37120/60000 (62%)]\tLoss: 1.060541\n", + "Train Epoch: 3 [37760/60000 (63%)]\tLoss: 0.937920\n", + "Train Epoch: 3 [38400/60000 (64%)]\tLoss: 1.006945\n", + "Train Epoch: 3 [39040/60000 (65%)]\tLoss: 1.078756\n", + "Train Epoch: 3 [39680/60000 (66%)]\tLoss: 1.017594\n", + "Train Epoch: 3 [40320/60000 (67%)]\tLoss: 1.033234\n", + "Train Epoch: 3 [40960/60000 (68%)]\tLoss: 1.230211\n", + "Train Epoch: 3 [41600/60000 (69%)]\tLoss: 0.684275\n", + "Train Epoch: 3 [42240/60000 (70%)]\tLoss: 1.083214\n", + "Train Epoch: 3 [42880/60000 (71%)]\tLoss: 0.852170\n", + "Train Epoch: 3 [43520/60000 (72%)]\tLoss: 0.995445\n", + "Train Epoch: 3 [44160/60000 (74%)]\tLoss: 0.790355\n", + "Train Epoch: 3 [44800/60000 (75%)]\tLoss: 0.920374\n", + "Train Epoch: 3 [45440/60000 (76%)]\tLoss: 1.090557\n", + "Train Epoch: 3 [46080/60000 (77%)]\tLoss: 0.755298\n", + "Train Epoch: 3 [46720/60000 (78%)]\tLoss: 1.101079\n", + "Train Epoch: 3 [47360/60000 (79%)]\tLoss: 0.719922\n", + "Train Epoch: 3 [48000/60000 (80%)]\tLoss: 0.805579\n", + "Train Epoch: 3 [48640/60000 (81%)]\tLoss: 0.866616\n", + "Train Epoch: 3 [49280/60000 (82%)]\tLoss: 0.777545\n", + "Train Epoch: 3 [49920/60000 (83%)]\tLoss: 0.800337\n", + "Train Epoch: 3 [50560/60000 (84%)]\tLoss: 0.921025\n", + "Train Epoch: 3 [51200/60000 (85%)]\tLoss: 0.961804\n", + "Train Epoch: 3 [51840/60000 (86%)]\tLoss: 0.882696\n", + "Train Epoch: 3 [52480/60000 (87%)]\tLoss: 0.849839\n", + "Train Epoch: 3 [53120/60000 (88%)]\tLoss: 0.893887\n", + "Train Epoch: 3 [53760/60000 (90%)]\tLoss: 0.815971\n", + "Train Epoch: 3 [54400/60000 (91%)]\tLoss: 1.109305\n", + "Train Epoch: 3 [55040/60000 (92%)]\tLoss: 0.864345\n", + "Train Epoch: 3 [55680/60000 (93%)]\tLoss: 0.969730\n", + "Train Epoch: 3 [56320/60000 (94%)]\tLoss: 0.959234\n", + "Train Epoch: 3 [56960/60000 (95%)]\tLoss: 0.954617\n", + "Train Epoch: 3 [57600/60000 (96%)]\tLoss: 0.790892\n", + "Train Epoch: 3 [58240/60000 (97%)]\tLoss: 1.002821\n", + "Train Epoch: 3 [58880/60000 (98%)]\tLoss: 0.799075\n", + "Train Epoch: 3 [59520/60000 (99%)]\tLoss: 0.859415\n", + "\n", + "Test set: Average loss: 0.7067\n", + "Class 0 accuracy: 942.0/980(96%)\n", + "Class 1 accuracy: 1085.0/1135(96%)\n", + "Class 2 accuracy: 815.0/1032(79%)\n", + "Class 3 accuracy: 909.0/1010(90%)\n", + "Class 4 accuracy: 844.0/982(86%)\n", + "Class 5 accuracy: 572.0/892(64%)\n", + "Class 6 accuracy: 881.0/958(92%)\n", + "Class 7 accuracy: 895.0/1028(87%)\n", + "Class 8 accuracy: 773.0/974(79%)\n", + "Class 9 accuracy: 861.0/1009(85%)\n", + "Train Epoch: 4 [0/60000 (0%)]\tLoss: 0.921860\n", + "Train Epoch: 4 [640/60000 (1%)]\tLoss: 1.006901\n", + "Train Epoch: 4 [1280/60000 (2%)]\tLoss: 0.829744\n", + "Train Epoch: 4 [1920/60000 (3%)]\tLoss: 0.958730\n", + "Train Epoch: 4 [2560/60000 (4%)]\tLoss: 0.908768\n", + "Train Epoch: 4 [3200/60000 (5%)]\tLoss: 0.809689\n", + "Train Epoch: 4 [3840/60000 (6%)]\tLoss: 0.864438\n", + "Train Epoch: 4 [4480/60000 (7%)]\tLoss: 0.871907\n", + "Train Epoch: 4 [5120/60000 (9%)]\tLoss: 0.879927\n", + "Train Epoch: 4 [5760/60000 (10%)]\tLoss: 0.958960\n", + "Train Epoch: 4 [6400/60000 (11%)]\tLoss: 0.866098\n", + "Train Epoch: 4 [7040/60000 (12%)]\tLoss: 1.041393\n", + "Train Epoch: 4 [7680/60000 (13%)]\tLoss: 0.813019\n", + "Train Epoch: 4 [8320/60000 (14%)]\tLoss: 0.767784\n", + "Train Epoch: 4 [8960/60000 (15%)]\tLoss: 0.933339\n", + "Train Epoch: 4 [9600/60000 (16%)]\tLoss: 0.821877\n", + "Train Epoch: 4 [10240/60000 (17%)]\tLoss: 0.826119\n", + "Train Epoch: 4 [10880/60000 (18%)]\tLoss: 0.772280\n", + "Train Epoch: 4 [11520/60000 (19%)]\tLoss: 0.868618\n", + "Train Epoch: 4 [12160/60000 (20%)]\tLoss: 1.034466\n", + "Train Epoch: 4 [12800/60000 (21%)]\tLoss: 0.872086\n", + "Train Epoch: 4 [13440/60000 (22%)]\tLoss: 0.912373\n", + "Train Epoch: 4 [14080/60000 (23%)]\tLoss: 0.870299\n", + "Train Epoch: 4 [14720/60000 (25%)]\tLoss: 0.731594\n", + "Train Epoch: 4 [15360/60000 (26%)]\tLoss: 1.079396\n", + "Train Epoch: 4 [16000/60000 (27%)]\tLoss: 0.882126\n", + "Train Epoch: 4 [16640/60000 (28%)]\tLoss: 0.682331\n", + "Train Epoch: 4 [17280/60000 (29%)]\tLoss: 1.029123\n", + "Train Epoch: 4 [17920/60000 (30%)]\tLoss: 0.596465\n", + "Train Epoch: 4 [18560/60000 (31%)]\tLoss: 0.813218\n", + "Train Epoch: 4 [19200/60000 (32%)]\tLoss: 0.747696\n", + "Train Epoch: 4 [19840/60000 (33%)]\tLoss: 0.880849\n", + "Train Epoch: 4 [20480/60000 (34%)]\tLoss: 0.734443\n", + "Train Epoch: 4 [21120/60000 (35%)]\tLoss: 0.840509\n", + "Train Epoch: 4 [21760/60000 (36%)]\tLoss: 0.745489\n", + "Train Epoch: 4 [22400/60000 (37%)]\tLoss: 0.841733\n", + "Train Epoch: 4 [23040/60000 (38%)]\tLoss: 0.920951\n", + "Train Epoch: 4 [23680/60000 (39%)]\tLoss: 0.924071\n", + "Train Epoch: 4 [24320/60000 (41%)]\tLoss: 0.906279\n", + "Train Epoch: 4 [24960/60000 (42%)]\tLoss: 0.867031\n", + "Train Epoch: 4 [25600/60000 (43%)]\tLoss: 0.946203\n", + "Train Epoch: 4 [26240/60000 (44%)]\tLoss: 0.989663\n", + "Train Epoch: 4 [26880/60000 (45%)]\tLoss: 0.839466\n", + "Train Epoch: 4 [27520/60000 (46%)]\tLoss: 0.843587\n", + "Train Epoch: 4 [28160/60000 (47%)]\tLoss: 0.737129\n", + "Train Epoch: 4 [28800/60000 (48%)]\tLoss: 0.807360\n", + "Train Epoch: 4 [29440/60000 (49%)]\tLoss: 0.823521\n", + "Train Epoch: 4 [30080/60000 (50%)]\tLoss: 0.841337\n", + "Train Epoch: 4 [30720/60000 (51%)]\tLoss: 0.999043\n", + "Train Epoch: 4 [31360/60000 (52%)]\tLoss: 0.962988\n", + "Train Epoch: 4 [32000/60000 (53%)]\tLoss: 1.040999\n", + "Train Epoch: 4 [32640/60000 (54%)]\tLoss: 0.750595\n", + "Train Epoch: 4 [33280/60000 (55%)]\tLoss: 0.904500\n", + "Train Epoch: 4 [33920/60000 (57%)]\tLoss: 0.716458\n", + "Train Epoch: 4 [34560/60000 (58%)]\tLoss: 0.917832\n", + "Train Epoch: 4 [35200/60000 (59%)]\tLoss: 0.834955\n", + "Train Epoch: 4 [35840/60000 (60%)]\tLoss: 0.833779\n", + "Train Epoch: 4 [36480/60000 (61%)]\tLoss: 0.798480\n", + "Train Epoch: 4 [37120/60000 (62%)]\tLoss: 0.718178\n", + "Train Epoch: 4 [37760/60000 (63%)]\tLoss: 0.774633\n", + "Train Epoch: 4 [38400/60000 (64%)]\tLoss: 0.640932\n", + "Train Epoch: 4 [39040/60000 (65%)]\tLoss: 0.761126\n", + "Train Epoch: 4 [39680/60000 (66%)]\tLoss: 0.567800\n", + "Train Epoch: 4 [40320/60000 (67%)]\tLoss: 0.731927\n", + "Train Epoch: 4 [40960/60000 (68%)]\tLoss: 0.972790\n", + "Train Epoch: 4 [41600/60000 (69%)]\tLoss: 0.913961\n", + "Train Epoch: 4 [42240/60000 (70%)]\tLoss: 0.630956\n", + "Train Epoch: 4 [42880/60000 (71%)]\tLoss: 0.928092\n", + "Train Epoch: 4 [43520/60000 (72%)]\tLoss: 0.749575\n", + "Train Epoch: 4 [44160/60000 (74%)]\tLoss: 0.970667\n", + "Train Epoch: 4 [44800/60000 (75%)]\tLoss: 0.804512\n", + "Train Epoch: 4 [45440/60000 (76%)]\tLoss: 0.832241\n", + "Train Epoch: 4 [46080/60000 (77%)]\tLoss: 0.851197\n", + "Train Epoch: 4 [46720/60000 (78%)]\tLoss: 0.918072\n", + "Train Epoch: 4 [47360/60000 (79%)]\tLoss: 0.886158\n", + "Train Epoch: 4 [48000/60000 (80%)]\tLoss: 0.803933\n", + "Train Epoch: 4 [48640/60000 (81%)]\tLoss: 0.711099\n", + "Train Epoch: 4 [49280/60000 (82%)]\tLoss: 0.786826\n", + "Train Epoch: 4 [49920/60000 (83%)]\tLoss: 0.834988\n", + "Train Epoch: 4 [50560/60000 (84%)]\tLoss: 0.902848\n", + "Train Epoch: 4 [51200/60000 (85%)]\tLoss: 0.838035\n", + "Train Epoch: 4 [51840/60000 (86%)]\tLoss: 0.853425\n", + "Train Epoch: 4 [52480/60000 (87%)]\tLoss: 0.843906\n", + "Train Epoch: 4 [53120/60000 (88%)]\tLoss: 0.782411\n", + "Train Epoch: 4 [53760/60000 (90%)]\tLoss: 0.752052\n", + "Train Epoch: 4 [54400/60000 (91%)]\tLoss: 0.738311\n", + "Train Epoch: 4 [55040/60000 (92%)]\tLoss: 0.929952\n", + "Train Epoch: 4 [55680/60000 (93%)]\tLoss: 0.858274\n", + "Train Epoch: 4 [56320/60000 (94%)]\tLoss: 1.006905\n", + "Train Epoch: 4 [56960/60000 (95%)]\tLoss: 0.824607\n", + "Train Epoch: 4 [57600/60000 (96%)]\tLoss: 0.852737\n", + "Train Epoch: 4 [58240/60000 (97%)]\tLoss: 0.789701\n", + "Train Epoch: 4 [58880/60000 (98%)]\tLoss: 0.821994\n", + "Train Epoch: 4 [59520/60000 (99%)]\tLoss: 0.840377\n", + "\n", + "Test set: Average loss: 0.6137\n", + "Class 0 accuracy: 947.0/980(97%)\n", + "Class 1 accuracy: 1097.0/1135(97%)\n", + "Class 2 accuracy: 840.0/1032(81%)\n", + "Class 3 accuracy: 912.0/1010(90%)\n", + "Class 4 accuracy: 852.0/982(87%)\n", + "Class 5 accuracy: 602.0/892(67%)\n", + "Class 6 accuracy: 879.0/958(92%)\n", + "Class 7 accuracy: 904.0/1028(88%)\n", + "Class 8 accuracy: 770.0/974(79%)\n", + "Class 9 accuracy: 866.0/1009(86%)\n", + "Train Epoch: 5 [0/60000 (0%)]\tLoss: 0.841241\n", + "Train Epoch: 5 [640/60000 (1%)]\tLoss: 0.963371\n", + "Train Epoch: 5 [1280/60000 (2%)]\tLoss: 0.714024\n", + "Train Epoch: 5 [1920/60000 (3%)]\tLoss: 0.760713\n", + "Train Epoch: 5 [2560/60000 (4%)]\tLoss: 0.709700\n", + "Train Epoch: 5 [3200/60000 (5%)]\tLoss: 0.959411\n", + "Train Epoch: 5 [3840/60000 (6%)]\tLoss: 0.823688\n", + "Train Epoch: 5 [4480/60000 (7%)]\tLoss: 0.688386\n", + "Train Epoch: 5 [5120/60000 (9%)]\tLoss: 0.836027\n", + "Train Epoch: 5 [5760/60000 (10%)]\tLoss: 0.814876\n", + "Train Epoch: 5 [6400/60000 (11%)]\tLoss: 0.915517\n", + "Train Epoch: 5 [7040/60000 (12%)]\tLoss: 0.930052\n", + "Train Epoch: 5 [7680/60000 (13%)]\tLoss: 0.713963\n", + "Train Epoch: 5 [8320/60000 (14%)]\tLoss: 0.611171\n", + "Train Epoch: 5 [8960/60000 (15%)]\tLoss: 0.753952\n", + "Train Epoch: 5 [9600/60000 (16%)]\tLoss: 0.753516\n", + "Train Epoch: 5 [10240/60000 (17%)]\tLoss: 0.638662\n", + "Train Epoch: 5 [10880/60000 (18%)]\tLoss: 0.851477\n", + "Train Epoch: 5 [11520/60000 (19%)]\tLoss: 0.748764\n", + "Train Epoch: 5 [12160/60000 (20%)]\tLoss: 0.717946\n", + "Train Epoch: 5 [12800/60000 (21%)]\tLoss: 0.648801\n", + "Train Epoch: 5 [13440/60000 (22%)]\tLoss: 0.817798\n", + "Train Epoch: 5 [14080/60000 (23%)]\tLoss: 1.011557\n", + "Train Epoch: 5 [14720/60000 (25%)]\tLoss: 0.979450\n", + "Train Epoch: 5 [15360/60000 (26%)]\tLoss: 0.741829\n", + "Train Epoch: 5 [16000/60000 (27%)]\tLoss: 0.929305\n", + "Train Epoch: 5 [16640/60000 (28%)]\tLoss: 0.898180\n", + "Train Epoch: 5 [17280/60000 (29%)]\tLoss: 0.682718\n", + "Train Epoch: 5 [17920/60000 (30%)]\tLoss: 0.841480\n", + "Train Epoch: 5 [18560/60000 (31%)]\tLoss: 0.895796\n", + "Train Epoch: 5 [19200/60000 (32%)]\tLoss: 0.707863\n", + "Train Epoch: 5 [19840/60000 (33%)]\tLoss: 0.851275\n", + "Train Epoch: 5 [20480/60000 (34%)]\tLoss: 0.855291\n", + "Train Epoch: 5 [21120/60000 (35%)]\tLoss: 0.851695\n", + "Train Epoch: 5 [21760/60000 (36%)]\tLoss: 0.684048\n", + "Train Epoch: 5 [22400/60000 (37%)]\tLoss: 0.917995\n", + "Train Epoch: 5 [23040/60000 (38%)]\tLoss: 0.765770\n", + "Train Epoch: 5 [23680/60000 (39%)]\tLoss: 0.647655\n", + "Train Epoch: 5 [24320/60000 (41%)]\tLoss: 0.739678\n", + "Train Epoch: 5 [24960/60000 (42%)]\tLoss: 0.672344\n", + "Train Epoch: 5 [25600/60000 (43%)]\tLoss: 0.766927\n", + "Train Epoch: 5 [26240/60000 (44%)]\tLoss: 0.738036\n", + "Train Epoch: 5 [26880/60000 (45%)]\tLoss: 0.852725\n", + "Train Epoch: 5 [27520/60000 (46%)]\tLoss: 0.632756\n", + "Train Epoch: 5 [28160/60000 (47%)]\tLoss: 0.689774\n", + "Train Epoch: 5 [28800/60000 (48%)]\tLoss: 0.674076\n", + "Train Epoch: 5 [29440/60000 (49%)]\tLoss: 0.755838\n", + "Train Epoch: 5 [30080/60000 (50%)]\tLoss: 0.786167\n", + "Train Epoch: 5 [30720/60000 (51%)]\tLoss: 0.804736\n", + "Train Epoch: 5 [31360/60000 (52%)]\tLoss: 0.723398\n", + "Train Epoch: 5 [32000/60000 (53%)]\tLoss: 0.685933\n", + "Train Epoch: 5 [32640/60000 (54%)]\tLoss: 0.808149\n", + "Train Epoch: 5 [33280/60000 (55%)]\tLoss: 0.944384\n", + "Train Epoch: 5 [33920/60000 (57%)]\tLoss: 0.638669\n", + "Train Epoch: 5 [34560/60000 (58%)]\tLoss: 0.683750\n", + "Train Epoch: 5 [35200/60000 (59%)]\tLoss: 0.702595\n", + "Train Epoch: 5 [35840/60000 (60%)]\tLoss: 0.742202\n", + "Train Epoch: 5 [36480/60000 (61%)]\tLoss: 0.761243\n", + "Train Epoch: 5 [37120/60000 (62%)]\tLoss: 0.768657\n", + "Train Epoch: 5 [37760/60000 (63%)]\tLoss: 0.824368\n", + "Train Epoch: 5 [38400/60000 (64%)]\tLoss: 0.657427\n", + "Train Epoch: 5 [39040/60000 (65%)]\tLoss: 0.724466\n", + "Train Epoch: 5 [39680/60000 (66%)]\tLoss: 0.800387\n", + "Train Epoch: 5 [40320/60000 (67%)]\tLoss: 0.812061\n", + "Train Epoch: 5 [40960/60000 (68%)]\tLoss: 0.811920\n", + "Train Epoch: 5 [41600/60000 (69%)]\tLoss: 0.888286\n", + "Train Epoch: 5 [42240/60000 (70%)]\tLoss: 0.691121\n", + "Train Epoch: 5 [42880/60000 (71%)]\tLoss: 0.749742\n", + "Train Epoch: 5 [43520/60000 (72%)]\tLoss: 0.944218\n", + "Train Epoch: 5 [44160/60000 (74%)]\tLoss: 0.698245\n", + "Train Epoch: 5 [44800/60000 (75%)]\tLoss: 0.796792\n", + "Train Epoch: 5 [45440/60000 (76%)]\tLoss: 0.653151\n", + "Train Epoch: 5 [46080/60000 (77%)]\tLoss: 0.813349\n", + "Train Epoch: 5 [46720/60000 (78%)]\tLoss: 0.600790\n", + "Train Epoch: 5 [47360/60000 (79%)]\tLoss: 0.848091\n", + "Train Epoch: 5 [48000/60000 (80%)]\tLoss: 0.636887\n", + "Train Epoch: 5 [48640/60000 (81%)]\tLoss: 0.767468\n", + "Train Epoch: 5 [49280/60000 (82%)]\tLoss: 0.713464\n", + "Train Epoch: 5 [49920/60000 (83%)]\tLoss: 0.635464\n", + "Train Epoch: 5 [50560/60000 (84%)]\tLoss: 0.722847\n", + "Train Epoch: 5 [51200/60000 (85%)]\tLoss: 0.924447\n", + "Train Epoch: 5 [51840/60000 (86%)]\tLoss: 0.738732\n", + "Train Epoch: 5 [52480/60000 (87%)]\tLoss: 0.732151\n", + "Train Epoch: 5 [53120/60000 (88%)]\tLoss: 0.769458\n", + "Train Epoch: 5 [53760/60000 (90%)]\tLoss: 0.761817\n", + "Train Epoch: 5 [54400/60000 (91%)]\tLoss: 0.766024\n", + "Train Epoch: 5 [55040/60000 (92%)]\tLoss: 0.820377\n", + "Train Epoch: 5 [55680/60000 (93%)]\tLoss: 0.713448\n", + "Train Epoch: 5 [56320/60000 (94%)]\tLoss: 0.753411\n", + "Train Epoch: 5 [56960/60000 (95%)]\tLoss: 0.758932\n", + "Train Epoch: 5 [57600/60000 (96%)]\tLoss: 0.690840\n", + "Train Epoch: 5 [58240/60000 (97%)]\tLoss: 0.855213\n", + "Train Epoch: 5 [58880/60000 (98%)]\tLoss: 0.847026\n", + "Train Epoch: 5 [59520/60000 (99%)]\tLoss: 0.714162\n", + "\n", + "Test set: Average loss: 0.5663\n", + "Class 0 accuracy: 949.0/980(97%)\n", + "Class 1 accuracy: 1095.0/1135(96%)\n", + "Class 2 accuracy: 841.0/1032(81%)\n", + "Class 3 accuracy: 909.0/1010(90%)\n", + "Class 4 accuracy: 861.0/982(88%)\n", + "Class 5 accuracy: 630.0/892(71%)\n", + "Class 6 accuracy: 881.0/958(92%)\n", + "Class 7 accuracy: 906.0/1028(88%)\n", + "Class 8 accuracy: 798.0/974(82%)\n", + "Class 9 accuracy: 869.0/1009(86%)\n" + ] + }, + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[{0: 0.9612244897959183,\n", + " 1: 0.9242290748898678,\n", + " 2: 0.5135658914728682,\n", + " 3: 0.9188118811881189,\n", + " 4: 0.7606924643584522,\n", + " 5: 0.3352017937219731,\n", + " 6: 0.918580375782881,\n", + " 7: 0.8278210116731517,\n", + " 8: 0.7638603696098563,\n", + " 9: 0.7809712586719524,\n", + " 'phase': 'baseline'},\n", + " {0: 0.9561224489795919,\n", + " 1: 0.9541850220264317,\n", + " 2: 0.7248062015503876,\n", + " 3: 0.9138613861386139,\n", + " 4: 0.8411405295315683,\n", + " 5: 0.5269058295964125,\n", + " 6: 0.9248434237995825,\n", + " 7: 0.8735408560311284,\n", + " 8: 0.7515400410677618,\n", + " 9: 0.8364717542120912,\n", + " 'phase': 'baseline'},\n", + " {0: 0.9612244897959183,\n", + " 1: 0.9559471365638766,\n", + " 2: 0.7897286821705426,\n", + " 3: 0.9,\n", + " 4: 0.8594704684317719,\n", + " 5: 0.6412556053811659,\n", + " 6: 0.9196242171189979,\n", + " 7: 0.8706225680933852,\n", + " 8: 0.7936344969199178,\n", + " 9: 0.8533201189296333,\n", + " 'phase': 'baseline'},\n", + " {0: 0.9663265306122449,\n", + " 1: 0.9665198237885463,\n", + " 2: 0.813953488372093,\n", + " 3: 0.902970297029703,\n", + " 4: 0.8676171079429735,\n", + " 5: 0.6748878923766816,\n", + " 6: 0.9175365344467641,\n", + " 7: 0.8793774319066148,\n", + " 8: 0.7905544147843943,\n", + " 9: 0.8582755203171457,\n", + " 'phase': 'baseline'},\n", + " {0: 0.9683673469387755,\n", + " 1: 0.9647577092511013,\n", + " 2: 0.814922480620155,\n", + " 3: 0.9,\n", + " 4: 0.8767820773930753,\n", + " 5: 0.7062780269058296,\n", + " 6: 0.9196242171189979,\n", + " 7: 0.8813229571984436,\n", + " 8: 0.8193018480492813,\n", + " 9: 0.8612487611496531,\n", + " 'phase': 'baseline'}]" + ] + }, + "metadata": {}, + "execution_count": 14 + } + ] + }, + { + "cell_type": "code", + "source": [ + "# Exercise 1: Implement an experiment setup that measures catastrophic forgetting while\n", + "# training in a task-incremental way on digits (0,1), (2,3), ..., (8, 9). After finishing\n", + "# training, measure the performance on the first task again.\n", + "#\n", + "# Implementation requires to fill in code further up in this notebook. You can visualize\n", + "# the result using the code further below.\n", + "\n", + "### START SOLUTION ###\n", + "# In this first part, we implement an experiment setup to measure catastrophic forgetting.\n", + "# We will subsequently train on pairs of classes, (0,1), then (2,3), etc., and in the end\n", + "# revisit the first task to measure if there was any forward transfer while training on the\n", + "# other tasks.\n", + "\n", + "def train_catastrophic_forgetting():\n", + " model = Net()\n", + " history = []\n", + "\n", + " config.train_classes = [0, 1]\n", + " train_model(config, model, phase = \"0_1\", history = history)\n", + " \n", + " config.train_classes = [2, 3]\n", + " train_model(config, model, phase = \"2_3\", history = history)\n", + "\n", + " config.train_classes = [4, 5]\n", + " train_model(config, model, phase = \"4_5\", history = history)\n", + "\n", + " config.train_classes = [6, 7]\n", + " train_model(config, model, phase = \"6_7\", history = history)\n", + "\n", + " config.train_classes = [8, 9]\n", + " train_model(config, model, phase = \"8_9\", history = history)\n", + " \n", + " config.train_classes = [0, 1]\n", + " train_model(config, model, phase = \"0_1_again\", history = history)\n", + "\n", + " return history\n", + "\n", + "### END SOLUTION ###\n", + " \n", + "history_catastrophic_forgetting = train_catastrophic_forgetting()\n", + "history_catastrophic_forgetting" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "OnATcaveP3Fi", + "outputId": "3864a59d-b604-491d-8697-f4f7d622c1e7" + }, + "execution_count": 15, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Train Epoch: 1 [0/12665 (0%)]\tLoss: 2.311233\n", + "Train Epoch: 1 [640/12665 (5%)]\tLoss: 2.246531\n", + "Train Epoch: 1 [1280/12665 (10%)]\tLoss: 2.186137\n", + "Train Epoch: 1 [1920/12665 (15%)]\tLoss: 2.105181\n", + "Train Epoch: 1 [2560/12665 (20%)]\tLoss: 2.048626\n", + "Train Epoch: 1 [3200/12665 (25%)]\tLoss: 1.975536\n", + "Train Epoch: 1 [3840/12665 (30%)]\tLoss: 1.913516\n", + "Train Epoch: 1 [4480/12665 (35%)]\tLoss: 1.818088\n", + "Train Epoch: 1 [5120/12665 (40%)]\tLoss: 1.744223\n", + "Train Epoch: 1 [5760/12665 (45%)]\tLoss: 1.696749\n", + "Train Epoch: 1 [6400/12665 (51%)]\tLoss: 1.615581\n", + "Train Epoch: 1 [7040/12665 (56%)]\tLoss: 1.508412\n", + "Train Epoch: 1 [7680/12665 (61%)]\tLoss: 1.426383\n", + "Train Epoch: 1 [8320/12665 (66%)]\tLoss: 1.385702\n", + "Train Epoch: 1 [8960/12665 (71%)]\tLoss: 1.271088\n", + "Train Epoch: 1 [9600/12665 (76%)]\tLoss: 1.157527\n", + "Train Epoch: 1 [10240/12665 (81%)]\tLoss: 1.164531\n", + "Train Epoch: 1 [10880/12665 (86%)]\tLoss: 1.056471\n", + "Train Epoch: 1 [11520/12665 (91%)]\tLoss: 1.015163\n", + "Train Epoch: 1 [12160/12665 (96%)]\tLoss: 0.924486\n", + "\n", + "Test set: Average loss: 2.6919\n", + "Class 0 accuracy: 972.0/980(99%)\n", + "Class 1 accuracy: 1134.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 2 [0/12665 (0%)]\tLoss: 0.725303\n", + "Train Epoch: 2 [640/12665 (5%)]\tLoss: 0.845168\n", + "Train Epoch: 2 [1280/12665 (10%)]\tLoss: 0.872068\n", + "Train Epoch: 2 [1920/12665 (15%)]\tLoss: 0.699184\n", + "Train Epoch: 2 [2560/12665 (20%)]\tLoss: 0.702631\n", + "Train Epoch: 2 [3200/12665 (25%)]\tLoss: 0.649261\n", + "Train Epoch: 2 [3840/12665 (30%)]\tLoss: 0.638805\n", + "Train Epoch: 2 [4480/12665 (35%)]\tLoss: 0.584503\n", + "Train Epoch: 2 [5120/12665 (40%)]\tLoss: 0.612078\n", + "Train Epoch: 2 [5760/12665 (45%)]\tLoss: 0.522527\n", + "Train Epoch: 2 [6400/12665 (51%)]\tLoss: 0.505232\n", + "Train Epoch: 2 [7040/12665 (56%)]\tLoss: 0.500317\n", + "Train Epoch: 2 [7680/12665 (61%)]\tLoss: 0.420275\n", + "Train Epoch: 2 [8320/12665 (66%)]\tLoss: 0.444307\n", + "Train Epoch: 2 [8960/12665 (71%)]\tLoss: 0.431154\n", + "Train Epoch: 2 [9600/12665 (76%)]\tLoss: 0.426933\n", + "Train Epoch: 2 [10240/12665 (81%)]\tLoss: 0.428767\n", + "Train Epoch: 2 [10880/12665 (86%)]\tLoss: 0.397589\n", + "Train Epoch: 2 [11520/12665 (91%)]\tLoss: 0.332724\n", + "Train Epoch: 2 [12160/12665 (96%)]\tLoss: 0.391286\n", + "\n", + "Test set: Average loss: 3.6493\n", + "Class 0 accuracy: 970.0/980(99%)\n", + "Class 1 accuracy: 1134.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 3 [0/12665 (0%)]\tLoss: 0.367619\n", + "Train Epoch: 3 [640/12665 (5%)]\tLoss: 0.309399\n", + "Train Epoch: 3 [1280/12665 (10%)]\tLoss: 0.296202\n", + "Train Epoch: 3 [1920/12665 (15%)]\tLoss: 0.340773\n", + "Train Epoch: 3 [2560/12665 (20%)]\tLoss: 0.293039\n", + "Train Epoch: 3 [3200/12665 (25%)]\tLoss: 0.288575\n", + "Train Epoch: 3 [3840/12665 (30%)]\tLoss: 0.285356\n", + "Train Epoch: 3 [4480/12665 (35%)]\tLoss: 0.283475\n", + "Train Epoch: 3 [5120/12665 (40%)]\tLoss: 0.285720\n", + "Train Epoch: 3 [5760/12665 (45%)]\tLoss: 0.245976\n", + "Train Epoch: 3 [6400/12665 (51%)]\tLoss: 0.249808\n", + "Train Epoch: 3 [7040/12665 (56%)]\tLoss: 0.239022\n", + "Train Epoch: 3 [7680/12665 (61%)]\tLoss: 0.268774\n", + "Train Epoch: 3 [8320/12665 (66%)]\tLoss: 0.223297\n", + "Train Epoch: 3 [8960/12665 (71%)]\tLoss: 0.236870\n", + "Train Epoch: 3 [9600/12665 (76%)]\tLoss: 0.222967\n", + "Train Epoch: 3 [10240/12665 (81%)]\tLoss: 0.242883\n", + "Train Epoch: 3 [10880/12665 (86%)]\tLoss: 0.208523\n", + "Train Epoch: 3 [11520/12665 (91%)]\tLoss: 0.191585\n", + "Train Epoch: 3 [12160/12665 (96%)]\tLoss: 0.206234\n", + "\n", + "Test set: Average loss: 4.2921\n", + "Class 0 accuracy: 971.0/980(99%)\n", + "Class 1 accuracy: 1134.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 4 [0/12665 (0%)]\tLoss: 0.222183\n", + "Train Epoch: 4 [640/12665 (5%)]\tLoss: 0.175392\n", + "Train Epoch: 4 [1280/12665 (10%)]\tLoss: 0.230008\n", + "Train Epoch: 4 [1920/12665 (15%)]\tLoss: 0.246189\n", + "Train Epoch: 4 [2560/12665 (20%)]\tLoss: 0.224284\n", + "Train Epoch: 4 [3200/12665 (25%)]\tLoss: 0.172705\n", + "Train Epoch: 4 [3840/12665 (30%)]\tLoss: 0.188962\n", + "Train Epoch: 4 [4480/12665 (35%)]\tLoss: 0.212664\n", + "Train Epoch: 4 [5120/12665 (40%)]\tLoss: 0.197847\n", + "Train Epoch: 4 [5760/12665 (45%)]\tLoss: 0.161129\n", + "Train Epoch: 4 [6400/12665 (51%)]\tLoss: 0.160840\n", + "Train Epoch: 4 [7040/12665 (56%)]\tLoss: 0.193788\n", + "Train Epoch: 4 [7680/12665 (61%)]\tLoss: 0.150140\n", + "Train Epoch: 4 [8320/12665 (66%)]\tLoss: 0.193050\n", + "Train Epoch: 4 [8960/12665 (71%)]\tLoss: 0.173765\n", + "Train Epoch: 4 [9600/12665 (76%)]\tLoss: 0.170651\n", + "Train Epoch: 4 [10240/12665 (81%)]\tLoss: 0.170728\n", + "Train Epoch: 4 [10880/12665 (86%)]\tLoss: 0.179045\n", + "Train Epoch: 4 [11520/12665 (91%)]\tLoss: 0.156496\n", + "Train Epoch: 4 [12160/12665 (96%)]\tLoss: 0.174930\n", + "\n", + "Test set: Average loss: 4.6719\n", + "Class 0 accuracy: 972.0/980(99%)\n", + "Class 1 accuracy: 1134.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 5 [0/12665 (0%)]\tLoss: 0.150109\n", + "Train Epoch: 5 [640/12665 (5%)]\tLoss: 0.133148\n", + "Train Epoch: 5 [1280/12665 (10%)]\tLoss: 0.143406\n", + "Train Epoch: 5 [1920/12665 (15%)]\tLoss: 0.210281\n", + "Train Epoch: 5 [2560/12665 (20%)]\tLoss: 0.129901\n", + "Train Epoch: 5 [3200/12665 (25%)]\tLoss: 0.140156\n", + "Train Epoch: 5 [3840/12665 (30%)]\tLoss: 0.127928\n", + "Train Epoch: 5 [4480/12665 (35%)]\tLoss: 0.153185\n", + "Train Epoch: 5 [5120/12665 (40%)]\tLoss: 0.132688\n", + "Train Epoch: 5 [5760/12665 (45%)]\tLoss: 0.174982\n", + "Train Epoch: 5 [6400/12665 (51%)]\tLoss: 0.174245\n", + "Train Epoch: 5 [7040/12665 (56%)]\tLoss: 0.147878\n", + "Train Epoch: 5 [7680/12665 (61%)]\tLoss: 0.105936\n", + "Train Epoch: 5 [8320/12665 (66%)]\tLoss: 0.139824\n", + "Train Epoch: 5 [8960/12665 (71%)]\tLoss: 0.128497\n", + "Train Epoch: 5 [9600/12665 (76%)]\tLoss: 0.102555\n", + "Train Epoch: 5 [10240/12665 (81%)]\tLoss: 0.149713\n", + "Train Epoch: 5 [10880/12665 (86%)]\tLoss: 0.204445\n", + "Train Epoch: 5 [11520/12665 (91%)]\tLoss: 0.173781\n", + "Train Epoch: 5 [12160/12665 (96%)]\tLoss: 0.104449\n", + "\n", + "Test set: Average loss: 4.9040\n", + "Class 0 accuracy: 972.0/980(99%)\n", + "Class 1 accuracy: 1134.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 1 [0/12089 (0%)]\tLoss: 7.002198\n", + "Train Epoch: 1 [640/12089 (5%)]\tLoss: 6.786501\n", + "Train Epoch: 1 [1280/12089 (11%)]\tLoss: 6.051014\n", + "Train Epoch: 1 [1920/12089 (16%)]\tLoss: 6.422165\n", + "Train Epoch: 1 [2560/12089 (21%)]\tLoss: 5.702563\n", + "Train Epoch: 1 [3200/12089 (26%)]\tLoss: 5.380751\n", + "Train Epoch: 1 [3840/12089 (32%)]\tLoss: 4.846919\n", + "Train Epoch: 1 [4480/12089 (37%)]\tLoss: 4.511275\n", + "Train Epoch: 1 [5120/12089 (42%)]\tLoss: 4.290347\n", + "Train Epoch: 1 [5760/12089 (48%)]\tLoss: 3.839983\n", + "Train Epoch: 1 [6400/12089 (53%)]\tLoss: 3.078847\n", + "Train Epoch: 1 [7040/12089 (58%)]\tLoss: 2.839584\n", + "Train Epoch: 1 [7680/12089 (63%)]\tLoss: 2.448348\n", + "Train Epoch: 1 [8320/12089 (69%)]\tLoss: 2.193519\n", + "Train Epoch: 1 [8960/12089 (74%)]\tLoss: 1.988640\n", + "Train Epoch: 1 [9600/12089 (79%)]\tLoss: 1.651685\n", + "Train Epoch: 1 [10240/12089 (85%)]\tLoss: 1.662965\n", + "Train Epoch: 1 [10880/12089 (90%)]\tLoss: 1.390365\n", + "Train Epoch: 1 [11520/12089 (95%)]\tLoss: 1.214716\n", + "\n", + "Test set: Average loss: 2.9396\n", + "Class 0 accuracy: 554.0/980(57%)\n", + "Class 1 accuracy: 1033.0/1135(91%)\n", + "Class 2 accuracy: 387.0/1032(38%)\n", + "Class 3 accuracy: 1008.0/1010(100%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 2 [0/12089 (0%)]\tLoss: 1.242918\n", + "Train Epoch: 2 [640/12089 (5%)]\tLoss: 1.003339\n", + "Train Epoch: 2 [1280/12089 (11%)]\tLoss: 0.984789\n", + "Train Epoch: 2 [1920/12089 (16%)]\tLoss: 0.937961\n", + "Train Epoch: 2 [2560/12089 (21%)]\tLoss: 0.883540\n", + "Train Epoch: 2 [3200/12089 (26%)]\tLoss: 0.840368\n", + "Train Epoch: 2 [3840/12089 (32%)]\tLoss: 0.937714\n", + "Train Epoch: 2 [4480/12089 (37%)]\tLoss: 0.909657\n", + "Train Epoch: 2 [5120/12089 (42%)]\tLoss: 0.840488\n", + "Train Epoch: 2 [5760/12089 (48%)]\tLoss: 0.763511\n", + "Train Epoch: 2 [6400/12089 (53%)]\tLoss: 0.767492\n", + "Train Epoch: 2 [7040/12089 (58%)]\tLoss: 0.721862\n", + "Train Epoch: 2 [7680/12089 (63%)]\tLoss: 0.698475\n", + "Train Epoch: 2 [8320/12089 (69%)]\tLoss: 0.630833\n", + "Train Epoch: 2 [8960/12089 (74%)]\tLoss: 0.732636\n", + "Train Epoch: 2 [9600/12089 (79%)]\tLoss: 0.686765\n", + "Train Epoch: 2 [10240/12089 (85%)]\tLoss: 0.647560\n", + "Train Epoch: 2 [10880/12089 (90%)]\tLoss: 0.584798\n", + "Train Epoch: 2 [11520/12089 (95%)]\tLoss: 0.605022\n", + "\n", + "Test set: Average loss: 3.8344\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 889.0/1032(86%)\n", + "Class 3 accuracy: 985.0/1010(98%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 3 [0/12089 (0%)]\tLoss: 0.633021\n", + "Train Epoch: 3 [640/12089 (5%)]\tLoss: 0.661086\n", + "Train Epoch: 3 [1280/12089 (11%)]\tLoss: 0.531795\n", + "Train Epoch: 3 [1920/12089 (16%)]\tLoss: 0.595388\n", + "Train Epoch: 3 [2560/12089 (21%)]\tLoss: 0.484488\n", + "Train Epoch: 3 [3200/12089 (26%)]\tLoss: 0.610725\n", + "Train Epoch: 3 [3840/12089 (32%)]\tLoss: 0.539676\n", + "Train Epoch: 3 [4480/12089 (37%)]\tLoss: 0.555348\n", + "Train Epoch: 3 [5120/12089 (42%)]\tLoss: 0.475156\n", + "Train Epoch: 3 [5760/12089 (48%)]\tLoss: 0.532515\n", + "Train Epoch: 3 [6400/12089 (53%)]\tLoss: 0.561437\n", + "Train Epoch: 3 [7040/12089 (58%)]\tLoss: 0.513701\n", + "Train Epoch: 3 [7680/12089 (63%)]\tLoss: 0.469366\n", + "Train Epoch: 3 [8320/12089 (69%)]\tLoss: 0.517332\n", + "Train Epoch: 3 [8960/12089 (74%)]\tLoss: 0.443565\n", + "Train Epoch: 3 [9600/12089 (79%)]\tLoss: 0.427636\n", + "Train Epoch: 3 [10240/12089 (85%)]\tLoss: 0.427255\n", + "Train Epoch: 3 [10880/12089 (90%)]\tLoss: 0.556653\n", + "Train Epoch: 3 [11520/12089 (95%)]\tLoss: 0.481399\n", + "\n", + "Test set: Average loss: 4.3145\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 945.0/1032(92%)\n", + "Class 3 accuracy: 967.0/1010(96%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 4 [0/12089 (0%)]\tLoss: 0.505337\n", + "Train Epoch: 4 [640/12089 (5%)]\tLoss: 0.464085\n", + "Train Epoch: 4 [1280/12089 (11%)]\tLoss: 0.518291\n", + "Train Epoch: 4 [1920/12089 (16%)]\tLoss: 0.452366\n", + "Train Epoch: 4 [2560/12089 (21%)]\tLoss: 0.405918\n", + "Train Epoch: 4 [3200/12089 (26%)]\tLoss: 0.502257\n", + "Train Epoch: 4 [3840/12089 (32%)]\tLoss: 0.448973\n", + "Train Epoch: 4 [4480/12089 (37%)]\tLoss: 0.445046\n", + "Train Epoch: 4 [5120/12089 (42%)]\tLoss: 0.495061\n", + "Train Epoch: 4 [5760/12089 (48%)]\tLoss: 0.456786\n", + "Train Epoch: 4 [6400/12089 (53%)]\tLoss: 0.371738\n", + "Train Epoch: 4 [7040/12089 (58%)]\tLoss: 0.468638\n", + "Train Epoch: 4 [7680/12089 (63%)]\tLoss: 0.495899\n", + "Train Epoch: 4 [8320/12089 (69%)]\tLoss: 0.453645\n", + "Train Epoch: 4 [8960/12089 (74%)]\tLoss: 0.479238\n", + "Train Epoch: 4 [9600/12089 (79%)]\tLoss: 0.458163\n", + "Train Epoch: 4 [10240/12089 (85%)]\tLoss: 0.446410\n", + "Train Epoch: 4 [10880/12089 (90%)]\tLoss: 0.441910\n", + "Train Epoch: 4 [11520/12089 (95%)]\tLoss: 0.430284\n", + "\n", + "Test set: Average loss: 4.5708\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 952.0/1032(92%)\n", + "Class 3 accuracy: 969.0/1010(96%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 5 [0/12089 (0%)]\tLoss: 0.405303\n", + "Train Epoch: 5 [640/12089 (5%)]\tLoss: 0.396980\n", + "Train Epoch: 5 [1280/12089 (11%)]\tLoss: 0.421218\n", + "Train Epoch: 5 [1920/12089 (16%)]\tLoss: 0.459768\n", + "Train Epoch: 5 [2560/12089 (21%)]\tLoss: 0.354826\n", + "Train Epoch: 5 [3200/12089 (26%)]\tLoss: 0.372944\n", + "Train Epoch: 5 [3840/12089 (32%)]\tLoss: 0.394097\n", + "Train Epoch: 5 [4480/12089 (37%)]\tLoss: 0.444731\n", + "Train Epoch: 5 [5120/12089 (42%)]\tLoss: 0.470775\n", + "Train Epoch: 5 [5760/12089 (48%)]\tLoss: 0.414834\n", + "Train Epoch: 5 [6400/12089 (53%)]\tLoss: 0.451856\n", + "Train Epoch: 5 [7040/12089 (58%)]\tLoss: 0.390931\n", + "Train Epoch: 5 [7680/12089 (63%)]\tLoss: 0.392495\n", + "Train Epoch: 5 [8320/12089 (69%)]\tLoss: 0.341354\n", + "Train Epoch: 5 [8960/12089 (74%)]\tLoss: 0.355052\n", + "Train Epoch: 5 [9600/12089 (79%)]\tLoss: 0.414375\n", + "Train Epoch: 5 [10240/12089 (85%)]\tLoss: 0.419718\n", + "Train Epoch: 5 [10880/12089 (90%)]\tLoss: 0.367832\n", + "Train Epoch: 5 [11520/12089 (95%)]\tLoss: 0.358470\n", + "\n", + "Test set: Average loss: 4.7194\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 953.0/1032(92%)\n", + "Class 3 accuracy: 970.0/1010(96%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 1 [0/11263 (0%)]\tLoss: 6.261249\n", + "Train Epoch: 1 [640/11263 (6%)]\tLoss: 5.999681\n", + "Train Epoch: 1 [1280/11263 (11%)]\tLoss: 5.607321\n", + "Train Epoch: 1 [1920/11263 (17%)]\tLoss: 5.542882\n", + "Train Epoch: 1 [2560/11263 (23%)]\tLoss: 4.963273\n", + "Train Epoch: 1 [3200/11263 (28%)]\tLoss: 4.750457\n", + "Train Epoch: 1 [3840/11263 (34%)]\tLoss: 4.334918\n", + "Train Epoch: 1 [4480/11263 (40%)]\tLoss: 3.912877\n", + "Train Epoch: 1 [5120/11263 (45%)]\tLoss: 3.827431\n", + "Train Epoch: 1 [5760/11263 (51%)]\tLoss: 3.393839\n", + "Train Epoch: 1 [6400/11263 (57%)]\tLoss: 3.033727\n", + "Train Epoch: 1 [7040/11263 (62%)]\tLoss: 2.671916\n", + "Train Epoch: 1 [7680/11263 (68%)]\tLoss: 2.473807\n", + "Train Epoch: 1 [8320/11263 (74%)]\tLoss: 2.285081\n", + "Train Epoch: 1 [8960/11263 (80%)]\tLoss: 1.972579\n", + "Train Epoch: 1 [9600/11263 (85%)]\tLoss: 1.953343\n", + "Train Epoch: 1 [10240/11263 (91%)]\tLoss: 1.653112\n", + "Train Epoch: 1 [10880/11263 (97%)]\tLoss: 1.666049\n", + "\n", + "Test set: Average loss: 2.6135\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 242.0/1135(21%)\n", + "Class 2 accuracy: 341.0/1032(33%)\n", + "Class 3 accuracy: 431.0/1010(43%)\n", + "Class 4 accuracy: 982.0/982(100%)\n", + "Class 5 accuracy: 11.0/892(1%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 2 [0/11263 (0%)]\tLoss: 1.265681\n", + "Train Epoch: 2 [640/11263 (6%)]\tLoss: 1.242094\n", + "Train Epoch: 2 [1280/11263 (11%)]\tLoss: 1.308110\n", + "Train Epoch: 2 [1920/11263 (17%)]\tLoss: 1.304853\n", + "Train Epoch: 2 [2560/11263 (23%)]\tLoss: 1.242684\n", + "Train Epoch: 2 [3200/11263 (28%)]\tLoss: 1.155575\n", + "Train Epoch: 2 [3840/11263 (34%)]\tLoss: 1.025319\n", + "Train Epoch: 2 [4480/11263 (40%)]\tLoss: 1.151237\n", + "Train Epoch: 2 [5120/11263 (45%)]\tLoss: 1.071076\n", + "Train Epoch: 2 [5760/11263 (51%)]\tLoss: 0.978908\n", + "Train Epoch: 2 [6400/11263 (57%)]\tLoss: 0.869457\n", + "Train Epoch: 2 [7040/11263 (62%)]\tLoss: 0.841398\n", + "Train Epoch: 2 [7680/11263 (68%)]\tLoss: 0.893406\n", + "Train Epoch: 2 [8320/11263 (74%)]\tLoss: 0.835585\n", + "Train Epoch: 2 [8960/11263 (80%)]\tLoss: 0.868128\n", + "Train Epoch: 2 [9600/11263 (85%)]\tLoss: 0.763401\n", + "Train Epoch: 2 [10240/11263 (91%)]\tLoss: 0.905359\n", + "Train Epoch: 2 [10880/11263 (97%)]\tLoss: 0.842730\n", + "\n", + "Test set: Average loss: 3.4385\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 979.0/982(100%)\n", + "Class 5 accuracy: 808.0/892(91%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 3 [0/11263 (0%)]\tLoss: 0.688606\n", + "Train Epoch: 3 [640/11263 (6%)]\tLoss: 0.747074\n", + "Train Epoch: 3 [1280/11263 (11%)]\tLoss: 0.699562\n", + "Train Epoch: 3 [1920/11263 (17%)]\tLoss: 0.687192\n", + "Train Epoch: 3 [2560/11263 (23%)]\tLoss: 0.694984\n", + "Train Epoch: 3 [3200/11263 (28%)]\tLoss: 0.643804\n", + "Train Epoch: 3 [3840/11263 (34%)]\tLoss: 0.637270\n", + "Train Epoch: 3 [4480/11263 (40%)]\tLoss: 0.776547\n", + "Train Epoch: 3 [5120/11263 (45%)]\tLoss: 0.624326\n", + "Train Epoch: 3 [5760/11263 (51%)]\tLoss: 0.763693\n", + "Train Epoch: 3 [6400/11263 (57%)]\tLoss: 0.619411\n", + "Train Epoch: 3 [7040/11263 (62%)]\tLoss: 0.672091\n", + "Train Epoch: 3 [7680/11263 (68%)]\tLoss: 0.640160\n", + "Train Epoch: 3 [8320/11263 (74%)]\tLoss: 0.594804\n", + "Train Epoch: 3 [8960/11263 (80%)]\tLoss: 0.712096\n", + "Train Epoch: 3 [9600/11263 (85%)]\tLoss: 0.584561\n", + "Train Epoch: 3 [10240/11263 (91%)]\tLoss: 0.490905\n", + "Train Epoch: 3 [10880/11263 (97%)]\tLoss: 0.511629\n", + "\n", + "Test set: Average loss: 3.9981\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 975.0/982(99%)\n", + "Class 5 accuracy: 837.0/892(94%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 4 [0/11263 (0%)]\tLoss: 0.560350\n", + "Train Epoch: 4 [640/11263 (6%)]\tLoss: 0.549644\n", + "Train Epoch: 4 [1280/11263 (11%)]\tLoss: 0.548100\n", + "Train Epoch: 4 [1920/11263 (17%)]\tLoss: 0.523234\n", + "Train Epoch: 4 [2560/11263 (23%)]\tLoss: 0.563891\n", + "Train Epoch: 4 [3200/11263 (28%)]\tLoss: 0.462694\n", + "Train Epoch: 4 [3840/11263 (34%)]\tLoss: 0.562608\n", + "Train Epoch: 4 [4480/11263 (40%)]\tLoss: 0.446004\n", + "Train Epoch: 4 [5120/11263 (45%)]\tLoss: 0.511273\n", + "Train Epoch: 4 [5760/11263 (51%)]\tLoss: 0.548768\n", + "Train Epoch: 4 [6400/11263 (57%)]\tLoss: 0.529422\n", + "Train Epoch: 4 [7040/11263 (62%)]\tLoss: 0.544079\n", + "Train Epoch: 4 [7680/11263 (68%)]\tLoss: 0.428724\n", + "Train Epoch: 4 [8320/11263 (74%)]\tLoss: 0.600822\n", + "Train Epoch: 4 [8960/11263 (80%)]\tLoss: 0.474362\n", + "Train Epoch: 4 [9600/11263 (85%)]\tLoss: 0.595750\n", + "Train Epoch: 4 [10240/11263 (91%)]\tLoss: 0.544630\n", + "Train Epoch: 4 [10880/11263 (97%)]\tLoss: 0.506346\n", + "\n", + "Test set: Average loss: 4.3219\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 974.0/982(99%)\n", + "Class 5 accuracy: 841.0/892(94%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 5 [0/11263 (0%)]\tLoss: 0.355252\n", + "Train Epoch: 5 [640/11263 (6%)]\tLoss: 0.517941\n", + "Train Epoch: 5 [1280/11263 (11%)]\tLoss: 0.526465\n", + "Train Epoch: 5 [1920/11263 (17%)]\tLoss: 0.523074\n", + "Train Epoch: 5 [2560/11263 (23%)]\tLoss: 0.433639\n", + "Train Epoch: 5 [3200/11263 (28%)]\tLoss: 0.434077\n", + "Train Epoch: 5 [3840/11263 (34%)]\tLoss: 0.403323\n", + "Train Epoch: 5 [4480/11263 (40%)]\tLoss: 0.505131\n", + "Train Epoch: 5 [5120/11263 (45%)]\tLoss: 0.355989\n", + "Train Epoch: 5 [5760/11263 (51%)]\tLoss: 0.490564\n", + "Train Epoch: 5 [6400/11263 (57%)]\tLoss: 0.399952\n", + "Train Epoch: 5 [7040/11263 (62%)]\tLoss: 0.436878\n", + "Train Epoch: 5 [7680/11263 (68%)]\tLoss: 0.491910\n", + "Train Epoch: 5 [8320/11263 (74%)]\tLoss: 0.429957\n", + "Train Epoch: 5 [8960/11263 (80%)]\tLoss: 0.495502\n", + "Train Epoch: 5 [9600/11263 (85%)]\tLoss: 0.446051\n", + "Train Epoch: 5 [10240/11263 (91%)]\tLoss: 0.364511\n", + "Train Epoch: 5 [10880/11263 (97%)]\tLoss: 0.447680\n", + "\n", + "Test set: Average loss: 4.5150\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 974.0/982(99%)\n", + "Class 5 accuracy: 843.0/892(95%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 1 [0/12183 (0%)]\tLoss: 7.209232\n", + "Train Epoch: 1 [640/12183 (5%)]\tLoss: 7.299924\n", + "Train Epoch: 1 [1280/12183 (10%)]\tLoss: 6.381386\n", + "Train Epoch: 1 [1920/12183 (16%)]\tLoss: 6.263927\n", + "Train Epoch: 1 [2560/12183 (21%)]\tLoss: 5.731481\n", + "Train Epoch: 1 [3200/12183 (26%)]\tLoss: 5.564906\n", + "Train Epoch: 1 [3840/12183 (31%)]\tLoss: 4.826056\n", + "Train Epoch: 1 [4480/12183 (37%)]\tLoss: 4.433992\n", + "Train Epoch: 1 [5120/12183 (42%)]\tLoss: 4.331242\n", + "Train Epoch: 1 [5760/12183 (47%)]\tLoss: 3.656218\n", + "Train Epoch: 1 [6400/12183 (52%)]\tLoss: 3.236688\n", + "Train Epoch: 1 [7040/12183 (58%)]\tLoss: 2.764972\n", + "Train Epoch: 1 [7680/12183 (63%)]\tLoss: 2.587098\n", + "Train Epoch: 1 [8320/12183 (68%)]\tLoss: 2.090186\n", + "Train Epoch: 1 [8960/12183 (73%)]\tLoss: 2.049698\n", + "Train Epoch: 1 [9600/12183 (79%)]\tLoss: 1.641774\n", + "Train Epoch: 1 [10240/12183 (84%)]\tLoss: 1.447641\n", + "Train Epoch: 1 [10880/12183 (89%)]\tLoss: 1.294460\n", + "Train Epoch: 1 [11520/12183 (94%)]\tLoss: 1.226808\n", + "Train Epoch: 1 [4370/12183 (99%)]\tLoss: 0.847535\n", + "\n", + "Test set: Average loss: 2.5262\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 1.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 957.0/958(100%)\n", + "Class 7 accuracy: 795.0/1028(77%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 2 [0/12183 (0%)]\tLoss: 1.063643\n", + "Train Epoch: 2 [640/12183 (5%)]\tLoss: 0.945048\n", + "Train Epoch: 2 [1280/12183 (10%)]\tLoss: 0.910895\n", + "Train Epoch: 2 [1920/12183 (16%)]\tLoss: 0.803437\n", + "Train Epoch: 2 [2560/12183 (21%)]\tLoss: 0.714711\n", + "Train Epoch: 2 [3200/12183 (26%)]\tLoss: 0.731389\n", + "Train Epoch: 2 [3840/12183 (31%)]\tLoss: 0.697865\n", + "Train Epoch: 2 [4480/12183 (37%)]\tLoss: 0.518302\n", + "Train Epoch: 2 [5120/12183 (42%)]\tLoss: 0.506918\n", + "Train Epoch: 2 [5760/12183 (47%)]\tLoss: 0.638351\n", + "Train Epoch: 2 [6400/12183 (52%)]\tLoss: 0.607182\n", + "Train Epoch: 2 [7040/12183 (58%)]\tLoss: 0.582088\n", + "Train Epoch: 2 [7680/12183 (63%)]\tLoss: 0.526564\n", + "Train Epoch: 2 [8320/12183 (68%)]\tLoss: 0.453763\n", + "Train Epoch: 2 [8960/12183 (73%)]\tLoss: 0.463578\n", + "Train Epoch: 2 [9600/12183 (79%)]\tLoss: 0.505179\n", + "Train Epoch: 2 [10240/12183 (84%)]\tLoss: 0.390394\n", + "Train Epoch: 2 [10880/12183 (89%)]\tLoss: 0.431393\n", + "Train Epoch: 2 [11520/12183 (94%)]\tLoss: 0.422808\n", + "Train Epoch: 2 [4370/12183 (99%)]\tLoss: 0.379329\n", + "\n", + "Test set: Average loss: 3.8944\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 954.0/958(100%)\n", + "Class 7 accuracy: 1005.0/1028(98%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 3 [0/12183 (0%)]\tLoss: 0.411217\n", + "Train Epoch: 3 [640/12183 (5%)]\tLoss: 0.430715\n", + "Train Epoch: 3 [1280/12183 (10%)]\tLoss: 0.321807\n", + "Train Epoch: 3 [1920/12183 (16%)]\tLoss: 0.305609\n", + "Train Epoch: 3 [2560/12183 (21%)]\tLoss: 0.329529\n", + "Train Epoch: 3 [3200/12183 (26%)]\tLoss: 0.344710\n", + "Train Epoch: 3 [3840/12183 (31%)]\tLoss: 0.351153\n", + "Train Epoch: 3 [4480/12183 (37%)]\tLoss: 0.355009\n", + "Train Epoch: 3 [5120/12183 (42%)]\tLoss: 0.316889\n", + "Train Epoch: 3 [5760/12183 (47%)]\tLoss: 0.268590\n", + "Train Epoch: 3 [6400/12183 (52%)]\tLoss: 0.269210\n", + "Train Epoch: 3 [7040/12183 (58%)]\tLoss: 0.272690\n", + "Train Epoch: 3 [7680/12183 (63%)]\tLoss: 0.299527\n", + "Train Epoch: 3 [8320/12183 (68%)]\tLoss: 0.294239\n", + "Train Epoch: 3 [8960/12183 (73%)]\tLoss: 0.256061\n", + "Train Epoch: 3 [9600/12183 (79%)]\tLoss: 0.263354\n", + "Train Epoch: 3 [10240/12183 (84%)]\tLoss: 0.266107\n", + "Train Epoch: 3 [10880/12183 (89%)]\tLoss: 0.301182\n", + "Train Epoch: 3 [11520/12183 (94%)]\tLoss: 0.221701\n", + "Train Epoch: 3 [4370/12183 (99%)]\tLoss: 0.256709\n", + "\n", + "Test set: Average loss: 4.6037\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 954.0/958(100%)\n", + "Class 7 accuracy: 1012.0/1028(98%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 4 [0/12183 (0%)]\tLoss: 0.200790\n", + "Train Epoch: 4 [640/12183 (5%)]\tLoss: 0.200424\n", + "Train Epoch: 4 [1280/12183 (10%)]\tLoss: 0.190599\n", + "Train Epoch: 4 [1920/12183 (16%)]\tLoss: 0.205490\n", + "Train Epoch: 4 [2560/12183 (21%)]\tLoss: 0.245161\n", + "Train Epoch: 4 [3200/12183 (26%)]\tLoss: 0.154059\n", + "Train Epoch: 4 [3840/12183 (31%)]\tLoss: 0.181401\n", + "Train Epoch: 4 [4480/12183 (37%)]\tLoss: 0.283187\n", + "Train Epoch: 4 [5120/12183 (42%)]\tLoss: 0.176126\n", + "Train Epoch: 4 [5760/12183 (47%)]\tLoss: 0.218665\n", + "Train Epoch: 4 [6400/12183 (52%)]\tLoss: 0.294239\n", + "Train Epoch: 4 [7040/12183 (58%)]\tLoss: 0.213903\n", + "Train Epoch: 4 [7680/12183 (63%)]\tLoss: 0.233052\n", + "Train Epoch: 4 [8320/12183 (68%)]\tLoss: 0.161038\n", + "Train Epoch: 4 [8960/12183 (73%)]\tLoss: 0.211654\n", + "Train Epoch: 4 [9600/12183 (79%)]\tLoss: 0.192449\n", + "Train Epoch: 4 [10240/12183 (84%)]\tLoss: 0.161328\n", + "Train Epoch: 4 [10880/12183 (89%)]\tLoss: 0.197734\n", + "Train Epoch: 4 [11520/12183 (94%)]\tLoss: 0.191102\n", + "Train Epoch: 4 [4370/12183 (99%)]\tLoss: 0.376056\n", + "\n", + "Test set: Average loss: 4.9524\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 955.0/958(100%)\n", + "Class 7 accuracy: 1013.0/1028(99%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 5 [0/12183 (0%)]\tLoss: 0.250694\n", + "Train Epoch: 5 [640/12183 (5%)]\tLoss: 0.137549\n", + "Train Epoch: 5 [1280/12183 (10%)]\tLoss: 0.179946\n", + "Train Epoch: 5 [1920/12183 (16%)]\tLoss: 0.172115\n", + "Train Epoch: 5 [2560/12183 (21%)]\tLoss: 0.216046\n", + "Train Epoch: 5 [3200/12183 (26%)]\tLoss: 0.137594\n", + "Train Epoch: 5 [3840/12183 (31%)]\tLoss: 0.167268\n", + "Train Epoch: 5 [4480/12183 (37%)]\tLoss: 0.215348\n", + "Train Epoch: 5 [5120/12183 (42%)]\tLoss: 0.190677\n", + "Train Epoch: 5 [5760/12183 (47%)]\tLoss: 0.143424\n", + "Train Epoch: 5 [6400/12183 (52%)]\tLoss: 0.176460\n", + "Train Epoch: 5 [7040/12183 (58%)]\tLoss: 0.115466\n", + "Train Epoch: 5 [7680/12183 (63%)]\tLoss: 0.123880\n", + "Train Epoch: 5 [8320/12183 (68%)]\tLoss: 0.172899\n", + "Train Epoch: 5 [8960/12183 (73%)]\tLoss: 0.121368\n", + "Train Epoch: 5 [9600/12183 (79%)]\tLoss: 0.117190\n", + "Train Epoch: 5 [10240/12183 (84%)]\tLoss: 0.193983\n", + "Train Epoch: 5 [10880/12183 (89%)]\tLoss: 0.137996\n", + "Train Epoch: 5 [11520/12183 (94%)]\tLoss: 0.165303\n", + "Train Epoch: 5 [4370/12183 (99%)]\tLoss: 0.120738\n", + "\n", + "Test set: Average loss: 5.1628\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 955.0/958(100%)\n", + "Class 7 accuracy: 1014.0/1028(99%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 1 [0/11800 (0%)]\tLoss: 8.108945\n", + "Train Epoch: 1 [640/11800 (5%)]\tLoss: 7.738531\n", + "Train Epoch: 1 [1280/11800 (11%)]\tLoss: 6.640255\n", + "Train Epoch: 1 [1920/11800 (16%)]\tLoss: 6.513710\n", + "Train Epoch: 1 [2560/11800 (22%)]\tLoss: 5.460023\n", + "Train Epoch: 1 [3200/11800 (27%)]\tLoss: 5.351851\n", + "Train Epoch: 1 [3840/11800 (32%)]\tLoss: 4.810606\n", + "Train Epoch: 1 [4480/11800 (38%)]\tLoss: 4.409184\n", + "Train Epoch: 1 [5120/11800 (43%)]\tLoss: 3.466255\n", + "Train Epoch: 1 [5760/11800 (49%)]\tLoss: 3.398129\n", + "Train Epoch: 1 [6400/11800 (54%)]\tLoss: 2.812559\n", + "Train Epoch: 1 [7040/11800 (59%)]\tLoss: 2.539098\n", + "Train Epoch: 1 [7680/11800 (65%)]\tLoss: 1.932615\n", + "Train Epoch: 1 [8320/11800 (70%)]\tLoss: 1.948097\n", + "Train Epoch: 1 [8960/11800 (76%)]\tLoss: 1.529168\n", + "Train Epoch: 1 [9600/11800 (81%)]\tLoss: 1.568914\n", + "Train Epoch: 1 [10240/11800 (86%)]\tLoss: 1.332329\n", + "Train Epoch: 1 [10880/11800 (92%)]\tLoss: 1.441239\n", + "Train Epoch: 1 [11520/11800 (97%)]\tLoss: 1.161227\n", + "\n", + "Test set: Average loss: 2.4228\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 485.0/958(51%)\n", + "Class 7 accuracy: 345.0/1028(34%)\n", + "Class 8 accuracy: 962.0/974(99%)\n", + "Class 9 accuracy: 735.0/1009(73%)\n", + "Train Epoch: 2 [0/11800 (0%)]\tLoss: 1.033281\n", + "Train Epoch: 2 [640/11800 (5%)]\tLoss: 1.074798\n", + "Train Epoch: 2 [1280/11800 (11%)]\tLoss: 1.087005\n", + "Train Epoch: 2 [1920/11800 (16%)]\tLoss: 1.221566\n", + "Train Epoch: 2 [2560/11800 (22%)]\tLoss: 1.020146\n", + "Train Epoch: 2 [3200/11800 (27%)]\tLoss: 1.010463\n", + "Train Epoch: 2 [3840/11800 (32%)]\tLoss: 0.871487\n", + "Train Epoch: 2 [4480/11800 (38%)]\tLoss: 0.892453\n", + "Train Epoch: 2 [5120/11800 (43%)]\tLoss: 0.761051\n", + "Train Epoch: 2 [5760/11800 (49%)]\tLoss: 0.883746\n", + "Train Epoch: 2 [6400/11800 (54%)]\tLoss: 0.881131\n", + "Train Epoch: 2 [7040/11800 (59%)]\tLoss: 0.761468\n", + "Train Epoch: 2 [7680/11800 (65%)]\tLoss: 0.702959\n", + "Train Epoch: 2 [8320/11800 (70%)]\tLoss: 0.575911\n", + "Train Epoch: 2 [8960/11800 (76%)]\tLoss: 0.668820\n", + "Train Epoch: 2 [9600/11800 (81%)]\tLoss: 0.780607\n", + "Train Epoch: 2 [10240/11800 (86%)]\tLoss: 0.565945\n", + "Train Epoch: 2 [10880/11800 (92%)]\tLoss: 0.687550\n", + "Train Epoch: 2 [11520/11800 (97%)]\tLoss: 0.551138\n", + "\n", + "Test set: Average loss: 3.3994\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 940.0/974(97%)\n", + "Class 9 accuracy: 914.0/1009(91%)\n", + "Train Epoch: 3 [0/11800 (0%)]\tLoss: 0.637666\n", + "Train Epoch: 3 [640/11800 (5%)]\tLoss: 0.512363\n", + "Train Epoch: 3 [1280/11800 (11%)]\tLoss: 0.587236\n", + "Train Epoch: 3 [1920/11800 (16%)]\tLoss: 0.496453\n", + "Train Epoch: 3 [2560/11800 (22%)]\tLoss: 0.617932\n", + "Train Epoch: 3 [3200/11800 (27%)]\tLoss: 0.538651\n", + "Train Epoch: 3 [3840/11800 (32%)]\tLoss: 0.471803\n", + "Train Epoch: 3 [4480/11800 (38%)]\tLoss: 0.569236\n", + "Train Epoch: 3 [5120/11800 (43%)]\tLoss: 0.473018\n", + "Train Epoch: 3 [5760/11800 (49%)]\tLoss: 0.661038\n", + "Train Epoch: 3 [6400/11800 (54%)]\tLoss: 0.436601\n", + "Train Epoch: 3 [7040/11800 (59%)]\tLoss: 0.551020\n", + "Train Epoch: 3 [7680/11800 (65%)]\tLoss: 0.422065\n", + "Train Epoch: 3 [8320/11800 (70%)]\tLoss: 0.436460\n", + "Train Epoch: 3 [8960/11800 (76%)]\tLoss: 0.521548\n", + "Train Epoch: 3 [9600/11800 (81%)]\tLoss: 0.582636\n", + "Train Epoch: 3 [10240/11800 (86%)]\tLoss: 0.460395\n", + "Train Epoch: 3 [10880/11800 (92%)]\tLoss: 0.637835\n", + "Train Epoch: 3 [11520/11800 (97%)]\tLoss: 0.514315\n", + "\n", + "Test set: Average loss: 3.9214\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 934.0/974(96%)\n", + "Class 9 accuracy: 939.0/1009(93%)\n", + "Train Epoch: 4 [0/11800 (0%)]\tLoss: 0.498162\n", + "Train Epoch: 4 [640/11800 (5%)]\tLoss: 0.536717\n", + "Train Epoch: 4 [1280/11800 (11%)]\tLoss: 0.436568\n", + "Train Epoch: 4 [1920/11800 (16%)]\tLoss: 0.471787\n", + "Train Epoch: 4 [2560/11800 (22%)]\tLoss: 0.497887\n", + "Train Epoch: 4 [3200/11800 (27%)]\tLoss: 0.444460\n", + "Train Epoch: 4 [3840/11800 (32%)]\tLoss: 0.448744\n", + "Train Epoch: 4 [4480/11800 (38%)]\tLoss: 0.372841\n", + "Train Epoch: 4 [5120/11800 (43%)]\tLoss: 0.465649\n", + "Train Epoch: 4 [5760/11800 (49%)]\tLoss: 0.490277\n", + "Train Epoch: 4 [6400/11800 (54%)]\tLoss: 0.463720\n", + "Train Epoch: 4 [7040/11800 (59%)]\tLoss: 0.355945\n", + "Train Epoch: 4 [7680/11800 (65%)]\tLoss: 0.438429\n", + "Train Epoch: 4 [8320/11800 (70%)]\tLoss: 0.508788\n", + "Train Epoch: 4 [8960/11800 (76%)]\tLoss: 0.413552\n", + "Train Epoch: 4 [9600/11800 (81%)]\tLoss: 0.395993\n", + "Train Epoch: 4 [10240/11800 (86%)]\tLoss: 0.410553\n", + "Train Epoch: 4 [10880/11800 (92%)]\tLoss: 0.409377\n", + "Train Epoch: 4 [11520/11800 (97%)]\tLoss: 0.400534\n", + "\n", + "Test set: Average loss: 4.2109\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 930.0/974(95%)\n", + "Class 9 accuracy: 948.0/1009(94%)\n", + "Train Epoch: 5 [0/11800 (0%)]\tLoss: 0.441779\n", + "Train Epoch: 5 [640/11800 (5%)]\tLoss: 0.349478\n", + "Train Epoch: 5 [1280/11800 (11%)]\tLoss: 0.347695\n", + "Train Epoch: 5 [1920/11800 (16%)]\tLoss: 0.374920\n", + "Train Epoch: 5 [2560/11800 (22%)]\tLoss: 0.371782\n", + "Train Epoch: 5 [3200/11800 (27%)]\tLoss: 0.405380\n", + "Train Epoch: 5 [3840/11800 (32%)]\tLoss: 0.493990\n", + "Train Epoch: 5 [4480/11800 (38%)]\tLoss: 0.383761\n", + "Train Epoch: 5 [5120/11800 (43%)]\tLoss: 0.361125\n", + "Train Epoch: 5 [5760/11800 (49%)]\tLoss: 0.377500\n", + "Train Epoch: 5 [6400/11800 (54%)]\tLoss: 0.411204\n", + "Train Epoch: 5 [7040/11800 (59%)]\tLoss: 0.467779\n", + "Train Epoch: 5 [7680/11800 (65%)]\tLoss: 0.445416\n", + "Train Epoch: 5 [8320/11800 (70%)]\tLoss: 0.303449\n", + "Train Epoch: 5 [8960/11800 (76%)]\tLoss: 0.319432\n", + "Train Epoch: 5 [9600/11800 (81%)]\tLoss: 0.331827\n", + "Train Epoch: 5 [10240/11800 (86%)]\tLoss: 0.377502\n", + "Train Epoch: 5 [10880/11800 (92%)]\tLoss: 0.406466\n", + "Train Epoch: 5 [11520/11800 (97%)]\tLoss: 0.405720\n", + "\n", + "Test set: Average loss: 4.3858\n", + "Class 0 accuracy: 0.0/980(0%)\n", + "Class 1 accuracy: 0.0/1135(0%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 929.0/974(95%)\n", + "Class 9 accuracy: 950.0/1009(94%)\n", + "Train Epoch: 1 [0/12665 (0%)]\tLoss: 5.571240\n", + "Train Epoch: 1 [640/12665 (5%)]\tLoss: 5.058652\n", + "Train Epoch: 1 [1280/12665 (10%)]\tLoss: 4.223537\n", + "Train Epoch: 1 [1920/12665 (15%)]\tLoss: 3.769430\n", + "Train Epoch: 1 [2560/12665 (20%)]\tLoss: 3.397155\n", + "Train Epoch: 1 [3200/12665 (25%)]\tLoss: 3.066763\n", + "Train Epoch: 1 [3840/12665 (30%)]\tLoss: 2.407099\n", + "Train Epoch: 1 [4480/12665 (35%)]\tLoss: 2.510794\n", + "Train Epoch: 1 [5120/12665 (40%)]\tLoss: 1.850332\n", + "Train Epoch: 1 [5760/12665 (45%)]\tLoss: 1.656529\n", + "Train Epoch: 1 [6400/12665 (51%)]\tLoss: 1.517869\n", + "Train Epoch: 1 [7040/12665 (56%)]\tLoss: 1.028974\n", + "Train Epoch: 1 [7680/12665 (61%)]\tLoss: 0.833506\n", + "Train Epoch: 1 [8320/12665 (66%)]\tLoss: 0.765792\n", + "Train Epoch: 1 [8960/12665 (71%)]\tLoss: 0.600499\n", + "Train Epoch: 1 [9600/12665 (76%)]\tLoss: 0.491542\n", + "Train Epoch: 1 [10240/12665 (81%)]\tLoss: 0.525203\n", + "Train Epoch: 1 [10880/12665 (86%)]\tLoss: 0.473181\n", + "Train Epoch: 1 [11520/12665 (91%)]\tLoss: 0.445249\n", + "Train Epoch: 1 [12160/12665 (96%)]\tLoss: 0.329707\n", + "\n", + "Test set: Average loss: 2.6586\n", + "Class 0 accuracy: 974.0/980(99%)\n", + "Class 1 accuracy: 1134.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 616.0/1009(61%)\n", + "Train Epoch: 2 [0/12665 (0%)]\tLoss: 0.174450\n", + "Train Epoch: 2 [640/12665 (5%)]\tLoss: 0.286020\n", + "Train Epoch: 2 [1280/12665 (10%)]\tLoss: 0.380822\n", + "Train Epoch: 2 [1920/12665 (15%)]\tLoss: 0.200067\n", + "Train Epoch: 2 [2560/12665 (20%)]\tLoss: 0.233174\n", + "Train Epoch: 2 [3200/12665 (25%)]\tLoss: 0.215359\n", + "Train Epoch: 2 [3840/12665 (30%)]\tLoss: 0.261502\n", + "Train Epoch: 2 [4480/12665 (35%)]\tLoss: 0.193090\n", + "Train Epoch: 2 [5120/12665 (40%)]\tLoss: 0.201911\n", + "Train Epoch: 2 [5760/12665 (45%)]\tLoss: 0.123239\n", + "Train Epoch: 2 [6400/12665 (51%)]\tLoss: 0.151434\n", + "Train Epoch: 2 [7040/12665 (56%)]\tLoss: 0.161058\n", + "Train Epoch: 2 [7680/12665 (61%)]\tLoss: 0.121602\n", + "Train Epoch: 2 [8320/12665 (66%)]\tLoss: 0.152282\n", + "Train Epoch: 2 [8960/12665 (71%)]\tLoss: 0.138010\n", + "Train Epoch: 2 [9600/12665 (76%)]\tLoss: 0.116535\n", + "Train Epoch: 2 [10240/12665 (81%)]\tLoss: 0.148996\n", + "Train Epoch: 2 [10880/12665 (86%)]\tLoss: 0.094655\n", + "Train Epoch: 2 [11520/12665 (91%)]\tLoss: 0.093630\n", + "Train Epoch: 2 [12160/12665 (96%)]\tLoss: 0.100545\n", + "\n", + "Test set: Average loss: 3.7860\n", + "Class 0 accuracy: 974.0/980(99%)\n", + "Class 1 accuracy: 1134.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 3 [0/12665 (0%)]\tLoss: 0.113853\n", + "Train Epoch: 3 [640/12665 (5%)]\tLoss: 0.061060\n", + "Train Epoch: 3 [1280/12665 (10%)]\tLoss: 0.073625\n", + "Train Epoch: 3 [1920/12665 (15%)]\tLoss: 0.089106\n", + "Train Epoch: 3 [2560/12665 (20%)]\tLoss: 0.100265\n", + "Train Epoch: 3 [3200/12665 (25%)]\tLoss: 0.086965\n", + "Train Epoch: 3 [3840/12665 (30%)]\tLoss: 0.078005\n", + "Train Epoch: 3 [4480/12665 (35%)]\tLoss: 0.089567\n", + "Train Epoch: 3 [5120/12665 (40%)]\tLoss: 0.083796\n", + "Train Epoch: 3 [5760/12665 (45%)]\tLoss: 0.091500\n", + "Train Epoch: 3 [6400/12665 (51%)]\tLoss: 0.071154\n", + "Train Epoch: 3 [7040/12665 (56%)]\tLoss: 0.063725\n", + "Train Epoch: 3 [7680/12665 (61%)]\tLoss: 0.081232\n", + "Train Epoch: 3 [8320/12665 (66%)]\tLoss: 0.063744\n", + "Train Epoch: 3 [8960/12665 (71%)]\tLoss: 0.044772\n", + "Train Epoch: 3 [9600/12665 (76%)]\tLoss: 0.056600\n", + "Train Epoch: 3 [10240/12665 (81%)]\tLoss: 0.075639\n", + "Train Epoch: 3 [10880/12665 (86%)]\tLoss: 0.059448\n", + "Train Epoch: 3 [11520/12665 (91%)]\tLoss: 0.051801\n", + "Train Epoch: 3 [12160/12665 (96%)]\tLoss: 0.045583\n", + "\n", + "Test set: Average loss: 4.3795\n", + "Class 0 accuracy: 974.0/980(99%)\n", + "Class 1 accuracy: 1134.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 4 [0/12665 (0%)]\tLoss: 0.073423\n", + "Train Epoch: 4 [640/12665 (5%)]\tLoss: 0.081250\n", + "Train Epoch: 4 [1280/12665 (10%)]\tLoss: 0.107188\n", + "Train Epoch: 4 [1920/12665 (15%)]\tLoss: 0.108589\n", + "Train Epoch: 4 [2560/12665 (20%)]\tLoss: 0.074006\n", + "Train Epoch: 4 [3200/12665 (25%)]\tLoss: 0.038409\n", + "Train Epoch: 4 [3840/12665 (30%)]\tLoss: 0.065204\n", + "Train Epoch: 4 [4480/12665 (35%)]\tLoss: 0.083654\n", + "Train Epoch: 4 [5120/12665 (40%)]\tLoss: 0.073944\n", + "Train Epoch: 4 [5760/12665 (45%)]\tLoss: 0.048630\n", + "Train Epoch: 4 [6400/12665 (51%)]\tLoss: 0.041222\n", + "Train Epoch: 4 [7040/12665 (56%)]\tLoss: 0.055742\n", + "Train Epoch: 4 [7680/12665 (61%)]\tLoss: 0.048848\n", + "Train Epoch: 4 [8320/12665 (66%)]\tLoss: 0.099426\n", + "Train Epoch: 4 [8960/12665 (71%)]\tLoss: 0.050589\n", + "Train Epoch: 4 [9600/12665 (76%)]\tLoss: 0.039493\n", + "Train Epoch: 4 [10240/12665 (81%)]\tLoss: 0.059188\n", + "Train Epoch: 4 [10880/12665 (86%)]\tLoss: 0.071380\n", + "Train Epoch: 4 [11520/12665 (91%)]\tLoss: 0.095400\n", + "Train Epoch: 4 [12160/12665 (96%)]\tLoss: 0.052493\n", + "\n", + "Test set: Average loss: 4.6909\n", + "Class 0 accuracy: 975.0/980(99%)\n", + "Class 1 accuracy: 1134.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 5 [0/12665 (0%)]\tLoss: 0.042433\n", + "Train Epoch: 5 [640/12665 (5%)]\tLoss: 0.045415\n", + "Train Epoch: 5 [1280/12665 (10%)]\tLoss: 0.045671\n", + "Train Epoch: 5 [1920/12665 (15%)]\tLoss: 0.116742\n", + "Train Epoch: 5 [2560/12665 (20%)]\tLoss: 0.042136\n", + "Train Epoch: 5 [3200/12665 (25%)]\tLoss: 0.053295\n", + "Train Epoch: 5 [3840/12665 (30%)]\tLoss: 0.035618\n", + "Train Epoch: 5 [4480/12665 (35%)]\tLoss: 0.055716\n", + "Train Epoch: 5 [5120/12665 (40%)]\tLoss: 0.043886\n", + "Train Epoch: 5 [5760/12665 (45%)]\tLoss: 0.052791\n", + "Train Epoch: 5 [6400/12665 (51%)]\tLoss: 0.079626\n", + "Train Epoch: 5 [7040/12665 (56%)]\tLoss: 0.044485\n", + "Train Epoch: 5 [7680/12665 (61%)]\tLoss: 0.027041\n", + "Train Epoch: 5 [8320/12665 (66%)]\tLoss: 0.041513\n", + "Train Epoch: 5 [8960/12665 (71%)]\tLoss: 0.034053\n", + "Train Epoch: 5 [9600/12665 (76%)]\tLoss: 0.028694\n", + "Train Epoch: 5 [10240/12665 (81%)]\tLoss: 0.054827\n", + "Train Epoch: 5 [10880/12665 (86%)]\tLoss: 0.182031\n", + "Train Epoch: 5 [11520/12665 (91%)]\tLoss: 0.068930\n", + "Train Epoch: 5 [12160/12665 (96%)]\tLoss: 0.022576\n", + "\n", + "Test set: Average loss: 4.8702\n", + "Class 0 accuracy: 976.0/980(100%)\n", + "Class 1 accuracy: 1134.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n" + ] + } + ] + }, + { + "cell_type": "code", + "source": [ + "# Exercise 2: Implementing the previous exercise should have shown that the network\n", + "# suffers from catastrophic forgetting. To fix this, implement a memory replay buffer\n", + "# and use this buffer to counteract the catastrophic forgetting.\n", + "#\n", + "# Implementation requires adaptation of the cells above, plus the addition of a new\n", + "# experiment setup below.\n", + "\n", + "\n", + "### START SOLUTION ###\n", + "# The experiment setup is identical to the one above, but now we initialize and use\n", + "# a replay buffer. Right now, the buffer only stores a single image for each class.\n", + "# As a bonus exercise, explore how different variants of the replay buffer affect the\n", + "# resulting model performance.\n", + "\n", + "def train_memory_replay():\n", + " model = Net()\n", + " history = []\n", + " buffer = Buffer()\n", + "\n", + " config.train_classes = [0, 1]\n", + " train_model(config, model, phase = \"0_1\", history = history, buffer = buffer)\n", + " \n", + " config.train_classes = [2, 3]\n", + " train_model(config, model, phase = \"2_3\", history = history, buffer = buffer)\n", + "\n", + " config.train_classes = [4, 5]\n", + " train_model(config, model, phase = \"4_5\", history = history, buffer = buffer)\n", + "\n", + " config.train_classes = [6, 7]\n", + " train_model(config, model, phase = \"6_7\", history = history, buffer = buffer)\n", + "\n", + " config.train_classes = [8, 9]\n", + " train_model(config, model, phase = \"8_9\", history = history, buffer = buffer)\n", + " \n", + " config.train_classes = [0, 1]\n", + " train_model(config, model, phase = \"0_1_again\", history = history, buffer = buffer)\n", + "\n", + " return history\n", + "\n", + "### END SOLUTION ###\n", + "\n", + "history_memory_replay = train_memory_replay()\n", + "history_memory_replay" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "LtWvHl_tQCpg", + "outputId": "e4913ffb-5f2b-474f-d24a-bde139af833c" + }, + "execution_count": 16, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Train Epoch: 1 [0/12665 (0%)]\tLoss: 2.295792\n", + "Train Epoch: 1 [640/12665 (5%)]\tLoss: 2.208695\n", + "Train Epoch: 1 [1280/12665 (10%)]\tLoss: 2.106677\n", + "Train Epoch: 1 [1920/12665 (15%)]\tLoss: 1.961109\n", + "Train Epoch: 1 [2560/12665 (20%)]\tLoss: 1.898052\n", + "Train Epoch: 1 [3200/12665 (25%)]\tLoss: 1.824553\n", + "Train Epoch: 1 [3840/12665 (30%)]\tLoss: 1.719246\n", + "Train Epoch: 1 [4480/12665 (35%)]\tLoss: 1.623634\n", + "Train Epoch: 1 [5120/12665 (40%)]\tLoss: 1.617715\n", + "Train Epoch: 1 [5760/12665 (45%)]\tLoss: 1.727536\n", + "Train Epoch: 1 [6400/12665 (51%)]\tLoss: 1.507114\n", + "Train Epoch: 1 [7040/12665 (56%)]\tLoss: 1.447730\n", + "Train Epoch: 1 [7680/12665 (61%)]\tLoss: 1.549878\n", + "Train Epoch: 1 [8320/12665 (66%)]\tLoss: 1.058249\n", + "Train Epoch: 1 [8960/12665 (71%)]\tLoss: 1.447155\n", + "Train Epoch: 1 [9600/12665 (76%)]\tLoss: 1.273197\n", + "Train Epoch: 1 [10240/12665 (81%)]\tLoss: 1.260120\n", + "Train Epoch: 1 [10880/12665 (86%)]\tLoss: 1.187015\n", + "Train Epoch: 1 [11520/12665 (91%)]\tLoss: 1.091164\n", + "Train Epoch: 1 [12160/12665 (96%)]\tLoss: 0.971557\n", + "\n", + "Test set: Average loss: 2.7060\n", + "Class 0 accuracy: 972.0/980(99%)\n", + "Class 1 accuracy: 1134.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 2 [0/12665 (0%)]\tLoss: 0.952087\n", + "Train Epoch: 2 [640/12665 (5%)]\tLoss: 0.869477\n", + "Train Epoch: 2 [1280/12665 (10%)]\tLoss: 0.858798\n", + "Train Epoch: 2 [1920/12665 (15%)]\tLoss: 0.809786\n", + "Train Epoch: 2 [2560/12665 (20%)]\tLoss: 0.799570\n", + "Train Epoch: 2 [3200/12665 (25%)]\tLoss: 0.890806\n", + "Train Epoch: 2 [3840/12665 (30%)]\tLoss: 0.805940\n", + "Train Epoch: 2 [4480/12665 (35%)]\tLoss: 1.059435\n", + "Train Epoch: 2 [5120/12665 (40%)]\tLoss: 0.478667\n", + "Train Epoch: 2 [5760/12665 (45%)]\tLoss: 0.673640\n", + "Train Epoch: 2 [6400/12665 (51%)]\tLoss: 0.538314\n", + "Train Epoch: 2 [7040/12665 (56%)]\tLoss: 0.710136\n", + "Train Epoch: 2 [7680/12665 (61%)]\tLoss: 0.516661\n", + "Train Epoch: 2 [8320/12665 (66%)]\tLoss: 0.602048\n", + "Train Epoch: 2 [8960/12665 (71%)]\tLoss: 0.384920\n", + "Train Epoch: 2 [9600/12665 (76%)]\tLoss: 0.435023\n", + "Train Epoch: 2 [10240/12665 (81%)]\tLoss: 0.372631\n", + "Train Epoch: 2 [10880/12665 (86%)]\tLoss: 0.426314\n", + "Train Epoch: 2 [11520/12665 (91%)]\tLoss: 0.539168\n", + "Train Epoch: 2 [12160/12665 (96%)]\tLoss: 0.321303\n", + "\n", + "Test set: Average loss: 3.8263\n", + "Class 0 accuracy: 966.0/980(99%)\n", + "Class 1 accuracy: 1135.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 3 [0/12665 (0%)]\tLoss: 0.401120\n", + "Train Epoch: 3 [640/12665 (5%)]\tLoss: 0.263190\n", + "Train Epoch: 3 [1280/12665 (10%)]\tLoss: 0.398854\n", + "Train Epoch: 3 [1920/12665 (15%)]\tLoss: 0.229336\n", + "Train Epoch: 3 [2560/12665 (20%)]\tLoss: 0.240626\n", + "Train Epoch: 3 [3200/12665 (25%)]\tLoss: 0.239559\n", + "Train Epoch: 3 [3840/12665 (30%)]\tLoss: 0.418891\n", + "Train Epoch: 3 [4480/12665 (35%)]\tLoss: 0.177813\n", + "Train Epoch: 3 [5120/12665 (40%)]\tLoss: 0.226682\n", + "Train Epoch: 3 [5760/12665 (45%)]\tLoss: 0.215379\n", + "Train Epoch: 3 [6400/12665 (51%)]\tLoss: 0.211677\n", + "Train Epoch: 3 [7040/12665 (56%)]\tLoss: 0.224589\n", + "Train Epoch: 3 [7680/12665 (61%)]\tLoss: 0.253302\n", + "Train Epoch: 3 [8320/12665 (66%)]\tLoss: 0.212234\n", + "Train Epoch: 3 [8960/12665 (71%)]\tLoss: 0.230823\n", + "Train Epoch: 3 [9600/12665 (76%)]\tLoss: 0.164656\n", + "Train Epoch: 3 [10240/12665 (81%)]\tLoss: 0.154504\n", + "Train Epoch: 3 [10880/12665 (86%)]\tLoss: 0.100245\n", + "Train Epoch: 3 [11520/12665 (91%)]\tLoss: 0.156521\n", + "Train Epoch: 3 [12160/12665 (96%)]\tLoss: 0.232217\n", + "\n", + "Test set: Average loss: 4.6647\n", + "Class 0 accuracy: 963.0/980(98%)\n", + "Class 1 accuracy: 1135.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 4 [0/12665 (0%)]\tLoss: 0.131072\n", + "Train Epoch: 4 [640/12665 (5%)]\tLoss: 0.227836\n", + "Train Epoch: 4 [1280/12665 (10%)]\tLoss: 0.161914\n", + "Train Epoch: 4 [1920/12665 (15%)]\tLoss: 0.182439\n", + "Train Epoch: 4 [2560/12665 (20%)]\tLoss: 0.123449\n", + "Train Epoch: 4 [3200/12665 (25%)]\tLoss: 0.118936\n", + "Train Epoch: 4 [3840/12665 (30%)]\tLoss: 0.132258\n", + "Train Epoch: 4 [4480/12665 (35%)]\tLoss: 0.191232\n", + "Train Epoch: 4 [5120/12665 (40%)]\tLoss: 0.192155\n", + "Train Epoch: 4 [5760/12665 (45%)]\tLoss: 0.122639\n", + "Train Epoch: 4 [6400/12665 (51%)]\tLoss: 0.138319\n", + "Train Epoch: 4 [7040/12665 (56%)]\tLoss: 0.106367\n", + "Train Epoch: 4 [7680/12665 (61%)]\tLoss: 0.078942\n", + "Train Epoch: 4 [8320/12665 (66%)]\tLoss: 0.103141\n", + "Train Epoch: 4 [8960/12665 (71%)]\tLoss: 0.129387\n", + "Train Epoch: 4 [9600/12665 (76%)]\tLoss: 0.117389\n", + "Train Epoch: 4 [10240/12665 (81%)]\tLoss: 0.164451\n", + "Train Epoch: 4 [10880/12665 (86%)]\tLoss: 0.216510\n", + "Train Epoch: 4 [11520/12665 (91%)]\tLoss: 0.110041\n", + "Train Epoch: 4 [12160/12665 (96%)]\tLoss: 0.104683\n", + "\n", + "Test set: Average loss: 5.1577\n", + "Class 0 accuracy: 966.0/980(99%)\n", + "Class 1 accuracy: 1135.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 5 [0/12665 (0%)]\tLoss: 0.077715\n", + "Train Epoch: 5 [640/12665 (5%)]\tLoss: 0.105138\n", + "Train Epoch: 5 [1280/12665 (10%)]\tLoss: 0.124953\n", + "Train Epoch: 5 [1920/12665 (15%)]\tLoss: 0.080610\n", + "Train Epoch: 5 [2560/12665 (20%)]\tLoss: 0.123617\n", + "Train Epoch: 5 [3200/12665 (25%)]\tLoss: 0.112670\n", + "Train Epoch: 5 [3840/12665 (30%)]\tLoss: 0.093631\n", + "Train Epoch: 5 [4480/12665 (35%)]\tLoss: 0.122231\n", + "Train Epoch: 5 [5120/12665 (40%)]\tLoss: 0.093652\n", + "Train Epoch: 5 [5760/12665 (45%)]\tLoss: 0.167812\n", + "Train Epoch: 5 [6400/12665 (51%)]\tLoss: 0.117981\n", + "Train Epoch: 5 [7040/12665 (56%)]\tLoss: 0.066692\n", + "Train Epoch: 5 [7680/12665 (61%)]\tLoss: 0.080171\n", + "Train Epoch: 5 [8320/12665 (66%)]\tLoss: 0.092669\n", + "Train Epoch: 5 [8960/12665 (71%)]\tLoss: 0.146656\n", + "Train Epoch: 5 [9600/12665 (76%)]\tLoss: 0.101289\n", + "Train Epoch: 5 [10240/12665 (81%)]\tLoss: 0.089997\n", + "Train Epoch: 5 [10880/12665 (86%)]\tLoss: 0.108752\n", + "Train Epoch: 5 [11520/12665 (91%)]\tLoss: 0.078000\n", + "Train Epoch: 5 [12160/12665 (96%)]\tLoss: 0.046221\n", + "\n", + "Test set: Average loss: 5.4729\n", + "Class 0 accuracy: 968.0/980(99%)\n", + "Class 1 accuracy: 1135.0/1135(100%)\n", + "Class 2 accuracy: 0.0/1032(0%)\n", + "Class 3 accuracy: 0.0/1010(0%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 1 [0/12089 (0%)]\tLoss: 1.166433\n", + "Train Epoch: 1 [640/12089 (5%)]\tLoss: 1.153999\n", + "Train Epoch: 1 [1280/12089 (11%)]\tLoss: 1.238186\n", + "Train Epoch: 1 [1920/12089 (16%)]\tLoss: 1.274190\n", + "Train Epoch: 1 [2560/12089 (21%)]\tLoss: 1.001455\n", + "Train Epoch: 1 [3200/12089 (26%)]\tLoss: 0.980768\n", + "Train Epoch: 1 [3840/12089 (32%)]\tLoss: 0.890982\n", + "Train Epoch: 1 [4480/12089 (37%)]\tLoss: 0.845770\n", + "Train Epoch: 1 [5120/12089 (42%)]\tLoss: 0.915158\n", + "Train Epoch: 1 [5760/12089 (48%)]\tLoss: 0.577478\n", + "Train Epoch: 1 [6400/12089 (53%)]\tLoss: 0.558826\n", + "Train Epoch: 1 [7040/12089 (58%)]\tLoss: 0.697927\n", + "Train Epoch: 1 [7680/12089 (63%)]\tLoss: 0.780273\n", + "Train Epoch: 1 [8320/12089 (69%)]\tLoss: 0.586348\n", + "Train Epoch: 1 [8960/12089 (74%)]\tLoss: 0.634719\n", + "Train Epoch: 1 [9600/12089 (79%)]\tLoss: 0.753965\n", + "Train Epoch: 1 [10240/12089 (85%)]\tLoss: 0.886030\n", + "Train Epoch: 1 [10880/12089 (90%)]\tLoss: 0.913329\n", + "Train Epoch: 1 [11520/12089 (95%)]\tLoss: 0.726805\n", + "\n", + "Test set: Average loss: 2.9830\n", + "Class 0 accuracy: 966.0/980(99%)\n", + "Class 1 accuracy: 1132.0/1135(100%)\n", + "Class 2 accuracy: 109.0/1032(11%)\n", + "Class 3 accuracy: 668.0/1010(66%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 2 [0/12089 (0%)]\tLoss: 0.903527\n", + "Train Epoch: 2 [640/12089 (5%)]\tLoss: 0.690564\n", + "Train Epoch: 2 [1280/12089 (11%)]\tLoss: 1.054734\n", + "Train Epoch: 2 [1920/12089 (16%)]\tLoss: 0.668291\n", + "Train Epoch: 2 [2560/12089 (21%)]\tLoss: 0.846006\n", + "Train Epoch: 2 [3200/12089 (26%)]\tLoss: 0.862174\n", + "Train Epoch: 2 [3840/12089 (32%)]\tLoss: 1.078590\n", + "Train Epoch: 2 [4480/12089 (37%)]\tLoss: 0.812564\n", + "Train Epoch: 2 [5120/12089 (42%)]\tLoss: 0.751745\n", + "Train Epoch: 2 [5760/12089 (48%)]\tLoss: 0.819927\n", + "Train Epoch: 2 [6400/12089 (53%)]\tLoss: 0.641764\n", + "Train Epoch: 2 [7040/12089 (58%)]\tLoss: 0.941192\n", + "Train Epoch: 2 [7680/12089 (63%)]\tLoss: 1.113382\n", + "Train Epoch: 2 [8320/12089 (69%)]\tLoss: 0.745287\n", + "Train Epoch: 2 [8960/12089 (74%)]\tLoss: 0.845461\n", + "Train Epoch: 2 [9600/12089 (79%)]\tLoss: 0.675010\n", + "Train Epoch: 2 [10240/12089 (85%)]\tLoss: 0.772856\n", + "Train Epoch: 2 [10880/12089 (90%)]\tLoss: 0.668837\n", + "Train Epoch: 2 [11520/12089 (95%)]\tLoss: 0.776711\n", + "\n", + "Test set: Average loss: 3.1608\n", + "Class 0 accuracy: 828.0/980(84%)\n", + "Class 1 accuracy: 1091.0/1135(96%)\n", + "Class 2 accuracy: 898.0/1032(87%)\n", + "Class 3 accuracy: 983.0/1010(97%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 3 [0/12089 (0%)]\tLoss: 0.733399\n", + "Train Epoch: 3 [640/12089 (5%)]\tLoss: 0.793913\n", + "Train Epoch: 3 [1280/12089 (11%)]\tLoss: 0.580512\n", + "Train Epoch: 3 [1920/12089 (16%)]\tLoss: 0.477269\n", + "Train Epoch: 3 [2560/12089 (21%)]\tLoss: 0.833255\n", + "Train Epoch: 3 [3200/12089 (26%)]\tLoss: 0.636208\n", + "Train Epoch: 3 [3840/12089 (32%)]\tLoss: 0.573351\n", + "Train Epoch: 3 [4480/12089 (37%)]\tLoss: 0.656650\n", + "Train Epoch: 3 [5120/12089 (42%)]\tLoss: 0.692221\n", + "Train Epoch: 3 [5760/12089 (48%)]\tLoss: 0.723047\n", + "Train Epoch: 3 [6400/12089 (53%)]\tLoss: 0.626788\n", + "Train Epoch: 3 [7040/12089 (58%)]\tLoss: 0.557750\n", + "Train Epoch: 3 [7680/12089 (63%)]\tLoss: 0.643362\n", + "Train Epoch: 3 [8320/12089 (69%)]\tLoss: 0.725737\n", + "Train Epoch: 3 [8960/12089 (74%)]\tLoss: 0.868968\n", + "Train Epoch: 3 [9600/12089 (79%)]\tLoss: 0.720436\n", + "Train Epoch: 3 [10240/12089 (85%)]\tLoss: 0.723306\n", + "Train Epoch: 3 [10880/12089 (90%)]\tLoss: 0.624521\n", + "Train Epoch: 3 [11520/12089 (95%)]\tLoss: 0.507009\n", + "\n", + "Test set: Average loss: 3.3136\n", + "Class 0 accuracy: 780.0/980(80%)\n", + "Class 1 accuracy: 1067.0/1135(94%)\n", + "Class 2 accuracy: 917.0/1032(89%)\n", + "Class 3 accuracy: 983.0/1010(97%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 4 [0/12089 (0%)]\tLoss: 0.624655\n", + "Train Epoch: 4 [640/12089 (5%)]\tLoss: 0.452378\n", + "Train Epoch: 4 [1280/12089 (11%)]\tLoss: 0.457449\n", + "Train Epoch: 4 [1920/12089 (16%)]\tLoss: 0.554645\n", + "Train Epoch: 4 [2560/12089 (21%)]\tLoss: 0.621985\n", + "Train Epoch: 4 [3200/12089 (26%)]\tLoss: 0.527864\n", + "Train Epoch: 4 [3840/12089 (32%)]\tLoss: 0.577068\n", + "Train Epoch: 4 [4480/12089 (37%)]\tLoss: 0.514906\n", + "Train Epoch: 4 [5120/12089 (42%)]\tLoss: 0.581334\n", + "Train Epoch: 4 [5760/12089 (48%)]\tLoss: 0.331880\n", + "Train Epoch: 4 [6400/12089 (53%)]\tLoss: 0.424212\n", + "Train Epoch: 4 [7040/12089 (58%)]\tLoss: 0.527507\n", + "Train Epoch: 4 [7680/12089 (63%)]\tLoss: 0.685347\n", + "Train Epoch: 4 [8320/12089 (69%)]\tLoss: 0.525770\n", + "Train Epoch: 4 [8960/12089 (74%)]\tLoss: 0.417910\n", + "Train Epoch: 4 [9600/12089 (79%)]\tLoss: 0.731371\n", + "Train Epoch: 4 [10240/12089 (85%)]\tLoss: 0.528346\n", + "Train Epoch: 4 [10880/12089 (90%)]\tLoss: 0.454048\n", + "Train Epoch: 4 [11520/12089 (95%)]\tLoss: 0.373003\n", + "\n", + "Test set: Average loss: 3.4222\n", + "Class 0 accuracy: 754.0/980(77%)\n", + "Class 1 accuracy: 1055.0/1135(93%)\n", + "Class 2 accuracy: 918.0/1032(89%)\n", + "Class 3 accuracy: 986.0/1010(98%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 5 [0/12089 (0%)]\tLoss: 0.546961\n", + "Train Epoch: 5 [640/12089 (5%)]\tLoss: 0.382696\n", + "Train Epoch: 5 [1280/12089 (11%)]\tLoss: 0.616167\n", + "Train Epoch: 5 [1920/12089 (16%)]\tLoss: 0.551108\n", + "Train Epoch: 5 [2560/12089 (21%)]\tLoss: 0.671682\n", + "Train Epoch: 5 [3200/12089 (26%)]\tLoss: 0.434367\n", + "Train Epoch: 5 [3840/12089 (32%)]\tLoss: 0.589797\n", + "Train Epoch: 5 [4480/12089 (37%)]\tLoss: 0.462255\n", + "Train Epoch: 5 [5120/12089 (42%)]\tLoss: 0.427646\n", + "Train Epoch: 5 [5760/12089 (48%)]\tLoss: 0.393138\n", + "Train Epoch: 5 [6400/12089 (53%)]\tLoss: 0.374952\n", + "Train Epoch: 5 [7040/12089 (58%)]\tLoss: 0.392871\n", + "Train Epoch: 5 [7680/12089 (63%)]\tLoss: 0.494981\n", + "Train Epoch: 5 [8320/12089 (69%)]\tLoss: 0.474602\n", + "Train Epoch: 5 [8960/12089 (74%)]\tLoss: 0.418477\n", + "Train Epoch: 5 [9600/12089 (79%)]\tLoss: 0.402891\n", + "Train Epoch: 5 [10240/12089 (85%)]\tLoss: 0.545055\n", + "Train Epoch: 5 [10880/12089 (90%)]\tLoss: 0.465828\n", + "Train Epoch: 5 [11520/12089 (95%)]\tLoss: 0.378968\n", + "\n", + "Test set: Average loss: 3.4964\n", + "Class 0 accuracy: 743.0/980(76%)\n", + "Class 1 accuracy: 1053.0/1135(93%)\n", + "Class 2 accuracy: 928.0/1032(90%)\n", + "Class 3 accuracy: 986.0/1010(98%)\n", + "Class 4 accuracy: 0.0/982(0%)\n", + "Class 5 accuracy: 0.0/892(0%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 1 [0/11263 (0%)]\tLoss: 0.850781\n", + "Train Epoch: 1 [640/11263 (6%)]\tLoss: 0.798292\n", + "Train Epoch: 1 [1280/11263 (11%)]\tLoss: 0.834117\n", + "Train Epoch: 1 [1920/11263 (17%)]\tLoss: 0.725765\n", + "Train Epoch: 1 [2560/11263 (23%)]\tLoss: 0.676635\n", + "Train Epoch: 1 [3200/11263 (28%)]\tLoss: 0.758110\n", + "Train Epoch: 1 [3840/11263 (34%)]\tLoss: 0.689677\n", + "Train Epoch: 1 [4480/11263 (40%)]\tLoss: 0.686142\n", + "Train Epoch: 1 [5120/11263 (45%)]\tLoss: 0.819930\n", + "Train Epoch: 1 [5760/11263 (51%)]\tLoss: 0.822826\n", + "Train Epoch: 1 [6400/11263 (57%)]\tLoss: 0.597154\n", + "Train Epoch: 1 [7040/11263 (62%)]\tLoss: 0.712456\n", + "Train Epoch: 1 [7680/11263 (68%)]\tLoss: 0.668146\n", + "Train Epoch: 1 [8320/11263 (74%)]\tLoss: 0.641806\n", + "Train Epoch: 1 [8960/11263 (80%)]\tLoss: 0.607866\n", + "Train Epoch: 1 [9600/11263 (85%)]\tLoss: 0.837627\n", + "Train Epoch: 1 [10240/11263 (91%)]\tLoss: 0.767775\n", + "Train Epoch: 1 [10880/11263 (97%)]\tLoss: 0.710244\n", + "\n", + "Test set: Average loss: 2.3788\n", + "Class 0 accuracy: 936.0/980(96%)\n", + "Class 1 accuracy: 1123.0/1135(99%)\n", + "Class 2 accuracy: 808.0/1032(78%)\n", + "Class 3 accuracy: 926.0/1010(92%)\n", + "Class 4 accuracy: 790.0/982(80%)\n", + "Class 5 accuracy: 360.0/892(40%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 2 [0/11263 (0%)]\tLoss: 0.896705\n", + "Train Epoch: 2 [640/11263 (6%)]\tLoss: 0.905773\n", + "Train Epoch: 2 [1280/11263 (11%)]\tLoss: 0.899539\n", + "Train Epoch: 2 [1920/11263 (17%)]\tLoss: 0.838611\n", + "Train Epoch: 2 [2560/11263 (23%)]\tLoss: 0.892302\n", + "Train Epoch: 2 [3200/11263 (28%)]\tLoss: 1.038975\n", + "Train Epoch: 2 [3840/11263 (34%)]\tLoss: 0.937163\n", + "Train Epoch: 2 [4480/11263 (40%)]\tLoss: 0.748620\n", + "Train Epoch: 2 [5120/11263 (45%)]\tLoss: 0.863919\n", + "Train Epoch: 2 [5760/11263 (51%)]\tLoss: 1.029196\n", + "Train Epoch: 2 [6400/11263 (57%)]\tLoss: 0.845210\n", + "Train Epoch: 2 [7040/11263 (62%)]\tLoss: 0.758227\n", + "Train Epoch: 2 [7680/11263 (68%)]\tLoss: 0.620243\n", + "Train Epoch: 2 [8320/11263 (74%)]\tLoss: 0.842112\n", + "Train Epoch: 2 [8960/11263 (80%)]\tLoss: 0.743856\n", + "Train Epoch: 2 [9600/11263 (85%)]\tLoss: 0.656504\n", + "Train Epoch: 2 [10240/11263 (91%)]\tLoss: 0.817470\n", + "Train Epoch: 2 [10880/11263 (97%)]\tLoss: 0.658012\n", + "\n", + "Test set: Average loss: 2.4907\n", + "Class 0 accuracy: 789.0/980(81%)\n", + "Class 1 accuracy: 1107.0/1135(98%)\n", + "Class 2 accuracy: 670.0/1032(65%)\n", + "Class 3 accuracy: 515.0/1010(51%)\n", + "Class 4 accuracy: 942.0/982(96%)\n", + "Class 5 accuracy: 766.0/892(86%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 3 [0/11263 (0%)]\tLoss: 0.754302\n", + "Train Epoch: 3 [640/11263 (6%)]\tLoss: 0.660190\n", + "Train Epoch: 3 [1280/11263 (11%)]\tLoss: 0.655176\n", + "Train Epoch: 3 [1920/11263 (17%)]\tLoss: 0.696439\n", + "Train Epoch: 3 [2560/11263 (23%)]\tLoss: 0.786622\n", + "Train Epoch: 3 [3200/11263 (28%)]\tLoss: 0.699489\n", + "Train Epoch: 3 [3840/11263 (34%)]\tLoss: 0.776751\n", + "Train Epoch: 3 [4480/11263 (40%)]\tLoss: 0.755700\n", + "Train Epoch: 3 [5120/11263 (45%)]\tLoss: 0.776476\n", + "Train Epoch: 3 [5760/11263 (51%)]\tLoss: 0.718861\n", + "Train Epoch: 3 [6400/11263 (57%)]\tLoss: 0.644046\n", + "Train Epoch: 3 [7040/11263 (62%)]\tLoss: 0.879339\n", + "Train Epoch: 3 [7680/11263 (68%)]\tLoss: 0.587297\n", + "Train Epoch: 3 [8320/11263 (74%)]\tLoss: 0.763207\n", + "Train Epoch: 3 [8960/11263 (80%)]\tLoss: 0.761769\n", + "Train Epoch: 3 [9600/11263 (85%)]\tLoss: 0.656073\n", + "Train Epoch: 3 [10240/11263 (91%)]\tLoss: 0.616851\n", + "Train Epoch: 3 [10880/11263 (97%)]\tLoss: 0.573584\n", + "\n", + "Test set: Average loss: 2.5840\n", + "Class 0 accuracy: 754.0/980(77%)\n", + "Class 1 accuracy: 1089.0/1135(96%)\n", + "Class 2 accuracy: 640.0/1032(62%)\n", + "Class 3 accuracy: 371.0/1010(37%)\n", + "Class 4 accuracy: 960.0/982(98%)\n", + "Class 5 accuracy: 800.0/892(90%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 4 [0/11263 (0%)]\tLoss: 0.648252\n", + "Train Epoch: 4 [640/11263 (6%)]\tLoss: 0.596721\n", + "Train Epoch: 4 [1280/11263 (11%)]\tLoss: 0.558993\n", + "Train Epoch: 4 [1920/11263 (17%)]\tLoss: 0.492765\n", + "Train Epoch: 4 [2560/11263 (23%)]\tLoss: 0.646996\n", + "Train Epoch: 4 [3200/11263 (28%)]\tLoss: 0.737724\n", + "Train Epoch: 4 [3840/11263 (34%)]\tLoss: 0.578851\n", + "Train Epoch: 4 [4480/11263 (40%)]\tLoss: 0.646620\n", + "Train Epoch: 4 [5120/11263 (45%)]\tLoss: 0.652254\n", + "Train Epoch: 4 [5760/11263 (51%)]\tLoss: 0.631144\n", + "Train Epoch: 4 [6400/11263 (57%)]\tLoss: 0.702145\n", + "Train Epoch: 4 [7040/11263 (62%)]\tLoss: 0.601468\n", + "Train Epoch: 4 [7680/11263 (68%)]\tLoss: 0.567884\n", + "Train Epoch: 4 [8320/11263 (74%)]\tLoss: 0.487138\n", + "Train Epoch: 4 [8960/11263 (80%)]\tLoss: 0.673586\n", + "Train Epoch: 4 [9600/11263 (85%)]\tLoss: 0.492252\n", + "Train Epoch: 4 [10240/11263 (91%)]\tLoss: 0.645415\n", + "Train Epoch: 4 [10880/11263 (97%)]\tLoss: 0.587828\n", + "\n", + "Test set: Average loss: 2.6582\n", + "Class 0 accuracy: 731.0/980(75%)\n", + "Class 1 accuracy: 1086.0/1135(96%)\n", + "Class 2 accuracy: 631.0/1032(61%)\n", + "Class 3 accuracy: 293.0/1010(29%)\n", + "Class 4 accuracy: 959.0/982(98%)\n", + "Class 5 accuracy: 817.0/892(92%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 5 [0/11263 (0%)]\tLoss: 0.693316\n", + "Train Epoch: 5 [640/11263 (6%)]\tLoss: 0.579783\n", + "Train Epoch: 5 [1280/11263 (11%)]\tLoss: 0.606642\n", + "Train Epoch: 5 [1920/11263 (17%)]\tLoss: 0.578930\n", + "Train Epoch: 5 [2560/11263 (23%)]\tLoss: 0.593126\n", + "Train Epoch: 5 [3200/11263 (28%)]\tLoss: 0.568712\n", + "Train Epoch: 5 [3840/11263 (34%)]\tLoss: 0.730927\n", + "Train Epoch: 5 [4480/11263 (40%)]\tLoss: 0.517850\n", + "Train Epoch: 5 [5120/11263 (45%)]\tLoss: 0.595436\n", + "Train Epoch: 5 [5760/11263 (51%)]\tLoss: 0.623922\n", + "Train Epoch: 5 [6400/11263 (57%)]\tLoss: 0.477650\n", + "Train Epoch: 5 [7040/11263 (62%)]\tLoss: 0.672610\n", + "Train Epoch: 5 [7680/11263 (68%)]\tLoss: 0.543774\n", + "Train Epoch: 5 [8320/11263 (74%)]\tLoss: 0.580296\n", + "Train Epoch: 5 [8960/11263 (80%)]\tLoss: 0.583141\n", + "Train Epoch: 5 [9600/11263 (85%)]\tLoss: 0.448098\n", + "Train Epoch: 5 [10240/11263 (91%)]\tLoss: 0.625687\n", + "Train Epoch: 5 [10880/11263 (97%)]\tLoss: 0.533021\n", + "\n", + "Test set: Average loss: 2.7075\n", + "Class 0 accuracy: 719.0/980(73%)\n", + "Class 1 accuracy: 1080.0/1135(95%)\n", + "Class 2 accuracy: 629.0/1032(61%)\n", + "Class 3 accuracy: 287.0/1010(28%)\n", + "Class 4 accuracy: 959.0/982(98%)\n", + "Class 5 accuracy: 834.0/892(93%)\n", + "Class 6 accuracy: 0.0/958(0%)\n", + "Class 7 accuracy: 0.0/1028(0%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 1 [0/12183 (0%)]\tLoss: 0.851070\n", + "Train Epoch: 1 [640/12183 (5%)]\tLoss: 0.703527\n", + "Train Epoch: 1 [1280/12183 (10%)]\tLoss: 0.862511\n", + "Train Epoch: 1 [1920/12183 (16%)]\tLoss: 0.664779\n", + "Train Epoch: 1 [2560/12183 (21%)]\tLoss: 0.622195\n", + "Train Epoch: 1 [3200/12183 (26%)]\tLoss: 0.686788\n", + "Train Epoch: 1 [3840/12183 (31%)]\tLoss: 0.766821\n", + "Train Epoch: 1 [4480/12183 (37%)]\tLoss: 0.635133\n", + "Train Epoch: 1 [5120/12183 (42%)]\tLoss: 0.673985\n", + "Train Epoch: 1 [5760/12183 (47%)]\tLoss: 0.736285\n", + "Train Epoch: 1 [6400/12183 (52%)]\tLoss: 0.763070\n", + "Train Epoch: 1 [7040/12183 (58%)]\tLoss: 0.914076\n", + "Train Epoch: 1 [7680/12183 (63%)]\tLoss: 0.897798\n", + "Train Epoch: 1 [8320/12183 (68%)]\tLoss: 0.769690\n", + "Train Epoch: 1 [8960/12183 (73%)]\tLoss: 0.705697\n", + "Train Epoch: 1 [9600/12183 (79%)]\tLoss: 0.890666\n", + "Train Epoch: 1 [10240/12183 (84%)]\tLoss: 0.964875\n", + "Train Epoch: 1 [10880/12183 (89%)]\tLoss: 0.846083\n", + "Train Epoch: 1 [11520/12183 (94%)]\tLoss: 0.977361\n", + "Train Epoch: 1 [4370/12183 (99%)]\tLoss: 1.044579\n", + "\n", + "Test set: Average loss: 1.6683\n", + "Class 0 accuracy: 839.0/980(86%)\n", + "Class 1 accuracy: 1086.0/1135(96%)\n", + "Class 2 accuracy: 547.0/1032(53%)\n", + "Class 3 accuracy: 854.0/1010(85%)\n", + "Class 4 accuracy: 673.0/982(69%)\n", + "Class 5 accuracy: 311.0/892(35%)\n", + "Class 6 accuracy: 878.0/958(92%)\n", + "Class 7 accuracy: 904.0/1028(88%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 2 [0/12183 (0%)]\tLoss: 0.782327\n", + "Train Epoch: 2 [640/12183 (5%)]\tLoss: 0.875331\n", + "Train Epoch: 2 [1280/12183 (10%)]\tLoss: 0.926851\n", + "Train Epoch: 2 [1920/12183 (16%)]\tLoss: 0.874468\n", + "Train Epoch: 2 [2560/12183 (21%)]\tLoss: 0.925109\n", + "Train Epoch: 2 [3200/12183 (26%)]\tLoss: 0.954436\n", + "Train Epoch: 2 [3840/12183 (31%)]\tLoss: 0.942288\n", + "Train Epoch: 2 [4480/12183 (37%)]\tLoss: 0.940796\n", + "Train Epoch: 2 [5120/12183 (42%)]\tLoss: 0.857594\n", + "Train Epoch: 2 [5760/12183 (47%)]\tLoss: 1.031697\n", + "Train Epoch: 2 [6400/12183 (52%)]\tLoss: 0.929081\n", + "Train Epoch: 2 [7040/12183 (58%)]\tLoss: 0.983720\n", + "Train Epoch: 2 [7680/12183 (63%)]\tLoss: 0.876552\n", + "Train Epoch: 2 [8320/12183 (68%)]\tLoss: 0.934435\n", + "Train Epoch: 2 [8960/12183 (73%)]\tLoss: 1.003677\n", + "Train Epoch: 2 [9600/12183 (79%)]\tLoss: 0.931277\n", + "Train Epoch: 2 [10240/12183 (84%)]\tLoss: 1.008535\n", + "Train Epoch: 2 [10880/12183 (89%)]\tLoss: 0.810847\n", + "Train Epoch: 2 [11520/12183 (94%)]\tLoss: 0.702577\n", + "Train Epoch: 2 [4370/12183 (99%)]\tLoss: 0.964019\n", + "\n", + "Test set: Average loss: 1.6425\n", + "Class 0 accuracy: 804.0/980(82%)\n", + "Class 1 accuracy: 1065.0/1135(94%)\n", + "Class 2 accuracy: 527.0/1032(51%)\n", + "Class 3 accuracy: 774.0/1010(77%)\n", + "Class 4 accuracy: 629.0/982(64%)\n", + "Class 5 accuracy: 347.0/892(39%)\n", + "Class 6 accuracy: 931.0/958(97%)\n", + "Class 7 accuracy: 983.0/1028(96%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 3 [0/12183 (0%)]\tLoss: 0.658189\n", + "Train Epoch: 3 [640/12183 (5%)]\tLoss: 0.624558\n", + "Train Epoch: 3 [1280/12183 (10%)]\tLoss: 0.811082\n", + "Train Epoch: 3 [1920/12183 (16%)]\tLoss: 0.653154\n", + "Train Epoch: 3 [2560/12183 (21%)]\tLoss: 0.784420\n", + "Train Epoch: 3 [3200/12183 (26%)]\tLoss: 0.620821\n", + "Train Epoch: 3 [3840/12183 (31%)]\tLoss: 0.743391\n", + "Train Epoch: 3 [4480/12183 (37%)]\tLoss: 0.694466\n", + "Train Epoch: 3 [5120/12183 (42%)]\tLoss: 0.580736\n", + "Train Epoch: 3 [5760/12183 (47%)]\tLoss: 0.673027\n", + "Train Epoch: 3 [6400/12183 (52%)]\tLoss: 0.545014\n", + "Train Epoch: 3 [7040/12183 (58%)]\tLoss: 0.924152\n", + "Train Epoch: 3 [7680/12183 (63%)]\tLoss: 0.730344\n", + "Train Epoch: 3 [8320/12183 (68%)]\tLoss: 0.724853\n", + "Train Epoch: 3 [8960/12183 (73%)]\tLoss: 0.699868\n", + "Train Epoch: 3 [9600/12183 (79%)]\tLoss: 0.697215\n", + "Train Epoch: 3 [10240/12183 (84%)]\tLoss: 0.783108\n", + "Train Epoch: 3 [10880/12183 (89%)]\tLoss: 0.672869\n", + "Train Epoch: 3 [11520/12183 (94%)]\tLoss: 0.625879\n", + "Train Epoch: 3 [4370/12183 (99%)]\tLoss: 0.598810\n", + "\n", + "Test set: Average loss: 1.6302\n", + "Class 0 accuracy: 797.0/980(81%)\n", + "Class 1 accuracy: 1062.0/1135(94%)\n", + "Class 2 accuracy: 526.0/1032(51%)\n", + "Class 3 accuracy: 762.0/1010(75%)\n", + "Class 4 accuracy: 657.0/982(67%)\n", + "Class 5 accuracy: 415.0/892(47%)\n", + "Class 6 accuracy: 938.0/958(98%)\n", + "Class 7 accuracy: 992.0/1028(96%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 4 [0/12183 (0%)]\tLoss: 0.686760\n", + "Train Epoch: 4 [640/12183 (5%)]\tLoss: 0.678613\n", + "Train Epoch: 4 [1280/12183 (10%)]\tLoss: 0.649789\n", + "Train Epoch: 4 [1920/12183 (16%)]\tLoss: 0.512429\n", + "Train Epoch: 4 [2560/12183 (21%)]\tLoss: 0.689496\n", + "Train Epoch: 4 [3200/12183 (26%)]\tLoss: 0.608655\n", + "Train Epoch: 4 [3840/12183 (31%)]\tLoss: 0.553349\n", + "Train Epoch: 4 [4480/12183 (37%)]\tLoss: 0.538118\n", + "Train Epoch: 4 [5120/12183 (42%)]\tLoss: 0.461560\n", + "Train Epoch: 4 [5760/12183 (47%)]\tLoss: 0.588525\n", + "Train Epoch: 4 [6400/12183 (52%)]\tLoss: 0.508286\n", + "Train Epoch: 4 [7040/12183 (58%)]\tLoss: 0.540536\n", + "Train Epoch: 4 [7680/12183 (63%)]\tLoss: 0.479708\n", + "Train Epoch: 4 [8320/12183 (68%)]\tLoss: 0.501095\n", + "Train Epoch: 4 [8960/12183 (73%)]\tLoss: 0.469510\n", + "Train Epoch: 4 [9600/12183 (79%)]\tLoss: 0.497756\n", + "Train Epoch: 4 [10240/12183 (84%)]\tLoss: 0.523196\n", + "Train Epoch: 4 [10880/12183 (89%)]\tLoss: 0.555491\n", + "Train Epoch: 4 [11520/12183 (94%)]\tLoss: 0.515540\n", + "Train Epoch: 4 [4370/12183 (99%)]\tLoss: 0.572875\n", + "\n", + "Test set: Average loss: 1.6347\n", + "Class 0 accuracy: 788.0/980(80%)\n", + "Class 1 accuracy: 1063.0/1135(94%)\n", + "Class 2 accuracy: 534.0/1032(52%)\n", + "Class 3 accuracy: 769.0/1010(76%)\n", + "Class 4 accuracy: 672.0/982(68%)\n", + "Class 5 accuracy: 440.0/892(49%)\n", + "Class 6 accuracy: 940.0/958(98%)\n", + "Class 7 accuracy: 996.0/1028(97%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 5 [0/12183 (0%)]\tLoss: 0.595143\n", + "Train Epoch: 5 [640/12183 (5%)]\tLoss: 0.550319\n", + "Train Epoch: 5 [1280/12183 (10%)]\tLoss: 0.548414\n", + "Train Epoch: 5 [1920/12183 (16%)]\tLoss: 0.511655\n", + "Train Epoch: 5 [2560/12183 (21%)]\tLoss: 0.401490\n", + "Train Epoch: 5 [3200/12183 (26%)]\tLoss: 0.400202\n", + "Train Epoch: 5 [3840/12183 (31%)]\tLoss: 0.397857\n", + "Train Epoch: 5 [4480/12183 (37%)]\tLoss: 0.479915\n", + "Train Epoch: 5 [5120/12183 (42%)]\tLoss: 0.408378\n", + "Train Epoch: 5 [5760/12183 (47%)]\tLoss: 0.431881\n", + "Train Epoch: 5 [6400/12183 (52%)]\tLoss: 0.429216\n", + "Train Epoch: 5 [7040/12183 (58%)]\tLoss: 0.347318\n", + "Train Epoch: 5 [7680/12183 (63%)]\tLoss: 0.549904\n", + "Train Epoch: 5 [8320/12183 (68%)]\tLoss: 0.484814\n", + "Train Epoch: 5 [8960/12183 (73%)]\tLoss: 0.430466\n", + "Train Epoch: 5 [9600/12183 (79%)]\tLoss: 0.517198\n", + "Train Epoch: 5 [10240/12183 (84%)]\tLoss: 0.395947\n", + "Train Epoch: 5 [10880/12183 (89%)]\tLoss: 0.551409\n", + "Train Epoch: 5 [11520/12183 (94%)]\tLoss: 0.495534\n", + "Train Epoch: 5 [4370/12183 (99%)]\tLoss: 0.495679\n", + "\n", + "Test set: Average loss: 1.6534\n", + "Class 0 accuracy: 781.0/980(80%)\n", + "Class 1 accuracy: 1059.0/1135(93%)\n", + "Class 2 accuracy: 530.0/1032(51%)\n", + "Class 3 accuracy: 763.0/1010(76%)\n", + "Class 4 accuracy: 672.0/982(68%)\n", + "Class 5 accuracy: 444.0/892(50%)\n", + "Class 6 accuracy: 940.0/958(98%)\n", + "Class 7 accuracy: 1001.0/1028(97%)\n", + "Class 8 accuracy: 0.0/974(0%)\n", + "Class 9 accuracy: 0.0/1009(0%)\n", + "Train Epoch: 1 [0/11800 (0%)]\tLoss: 0.848253\n", + "Train Epoch: 1 [640/11800 (5%)]\tLoss: 0.637744\n", + "Train Epoch: 1 [1280/11800 (11%)]\tLoss: 0.628570\n", + "Train Epoch: 1 [1920/11800 (16%)]\tLoss: 0.461403\n", + "Train Epoch: 1 [2560/11800 (22%)]\tLoss: 0.513085\n", + "Train Epoch: 1 [3200/11800 (27%)]\tLoss: 0.590126\n", + "Train Epoch: 1 [3840/11800 (32%)]\tLoss: 0.472829\n", + "Train Epoch: 1 [4480/11800 (38%)]\tLoss: 0.592700\n", + "Train Epoch: 1 [5120/11800 (43%)]\tLoss: 0.617516\n", + "Train Epoch: 1 [5760/11800 (49%)]\tLoss: 0.555048\n", + "Train Epoch: 1 [6400/11800 (54%)]\tLoss: 0.743809\n", + "Train Epoch: 1 [7040/11800 (59%)]\tLoss: 0.765445\n", + "Train Epoch: 1 [7680/11800 (65%)]\tLoss: 0.768416\n", + "Train Epoch: 1 [8320/11800 (70%)]\tLoss: 0.777279\n", + "Train Epoch: 1 [8960/11800 (76%)]\tLoss: 0.842107\n", + "Train Epoch: 1 [9600/11800 (81%)]\tLoss: 0.796940\n", + "Train Epoch: 1 [10240/11800 (86%)]\tLoss: 0.601572\n", + "Train Epoch: 1 [10880/11800 (92%)]\tLoss: 0.841399\n", + "Train Epoch: 1 [11520/11800 (97%)]\tLoss: 0.846165\n", + "\n", + "Test set: Average loss: 0.9925\n", + "Class 0 accuracy: 835.0/980(85%)\n", + "Class 1 accuracy: 1050.0/1135(93%)\n", + "Class 2 accuracy: 638.0/1032(62%)\n", + "Class 3 accuracy: 738.0/1010(73%)\n", + "Class 4 accuracy: 548.0/982(56%)\n", + "Class 5 accuracy: 170.0/892(19%)\n", + "Class 6 accuracy: 779.0/958(81%)\n", + "Class 7 accuracy: 776.0/1028(75%)\n", + "Class 8 accuracy: 908.0/974(93%)\n", + "Class 9 accuracy: 801.0/1009(79%)\n", + "Train Epoch: 2 [0/11800 (0%)]\tLoss: 0.883257\n", + "Train Epoch: 2 [640/11800 (5%)]\tLoss: 0.857211\n", + "Train Epoch: 2 [1280/11800 (11%)]\tLoss: 0.841614\n", + "Train Epoch: 2 [1920/11800 (16%)]\tLoss: 0.790240\n", + "Train Epoch: 2 [2560/11800 (22%)]\tLoss: 0.950122\n", + "Train Epoch: 2 [3200/11800 (27%)]\tLoss: 0.835133\n", + "Train Epoch: 2 [3840/11800 (32%)]\tLoss: 0.882796\n", + "Train Epoch: 2 [4480/11800 (38%)]\tLoss: 0.810285\n", + "Train Epoch: 2 [5120/11800 (43%)]\tLoss: 0.799711\n", + "Train Epoch: 2 [5760/11800 (49%)]\tLoss: 0.834429\n", + "Train Epoch: 2 [6400/11800 (54%)]\tLoss: 0.839871\n", + "Train Epoch: 2 [7040/11800 (59%)]\tLoss: 0.947290\n", + "Train Epoch: 2 [7680/11800 (65%)]\tLoss: 0.710232\n", + "Train Epoch: 2 [8320/11800 (70%)]\tLoss: 0.779843\n", + "Train Epoch: 2 [8960/11800 (76%)]\tLoss: 0.733478\n", + "Train Epoch: 2 [9600/11800 (81%)]\tLoss: 0.938040\n", + "Train Epoch: 2 [10240/11800 (86%)]\tLoss: 0.839119\n", + "Train Epoch: 2 [10880/11800 (92%)]\tLoss: 0.696384\n", + "Train Epoch: 2 [11520/11800 (97%)]\tLoss: 0.752226\n", + "\n", + "Test set: Average loss: 0.9674\n", + "Class 0 accuracy: 825.0/980(84%)\n", + "Class 1 accuracy: 1004.0/1135(88%)\n", + "Class 2 accuracy: 599.0/1032(58%)\n", + "Class 3 accuracy: 615.0/1010(61%)\n", + "Class 4 accuracy: 332.0/982(34%)\n", + "Class 5 accuracy: 42.0/892(5%)\n", + "Class 6 accuracy: 762.0/958(80%)\n", + "Class 7 accuracy: 623.0/1028(61%)\n", + "Class 8 accuracy: 943.0/974(97%)\n", + "Class 9 accuracy: 923.0/1009(91%)\n", + "Train Epoch: 3 [0/11800 (0%)]\tLoss: 0.708711\n", + "Train Epoch: 3 [640/11800 (5%)]\tLoss: 0.783265\n", + "Train Epoch: 3 [1280/11800 (11%)]\tLoss: 0.810831\n", + "Train Epoch: 3 [1920/11800 (16%)]\tLoss: 0.753713\n", + "Train Epoch: 3 [2560/11800 (22%)]\tLoss: 0.696868\n", + "Train Epoch: 3 [3200/11800 (27%)]\tLoss: 0.572866\n", + "Train Epoch: 3 [3840/11800 (32%)]\tLoss: 0.757275\n", + "Train Epoch: 3 [4480/11800 (38%)]\tLoss: 0.732344\n", + "Train Epoch: 3 [5120/11800 (43%)]\tLoss: 0.671541\n", + "Train Epoch: 3 [5760/11800 (49%)]\tLoss: 0.724425\n", + "Train Epoch: 3 [6400/11800 (54%)]\tLoss: 0.719062\n", + "Train Epoch: 3 [7040/11800 (59%)]\tLoss: 0.712015\n", + "Train Epoch: 3 [7680/11800 (65%)]\tLoss: 0.619695\n", + "Train Epoch: 3 [8320/11800 (70%)]\tLoss: 0.781576\n", + "Train Epoch: 3 [8960/11800 (76%)]\tLoss: 0.707311\n", + "Train Epoch: 3 [9600/11800 (81%)]\tLoss: 0.700315\n", + "Train Epoch: 3 [10240/11800 (86%)]\tLoss: 0.579610\n", + "Train Epoch: 3 [10880/11800 (92%)]\tLoss: 0.836607\n", + "Train Epoch: 3 [11520/11800 (97%)]\tLoss: 0.599106\n", + "\n", + "Test set: Average loss: 0.9365\n", + "Class 0 accuracy: 817.0/980(83%)\n", + "Class 1 accuracy: 996.0/1135(88%)\n", + "Class 2 accuracy: 598.0/1032(58%)\n", + "Class 3 accuracy: 612.0/1010(61%)\n", + "Class 4 accuracy: 291.0/982(30%)\n", + "Class 5 accuracy: 58.0/892(7%)\n", + "Class 6 accuracy: 772.0/958(81%)\n", + "Class 7 accuracy: 574.0/1028(56%)\n", + "Class 8 accuracy: 941.0/974(97%)\n", + "Class 9 accuracy: 948.0/1009(94%)\n", + "Train Epoch: 4 [0/11800 (0%)]\tLoss: 0.764324\n", + "Train Epoch: 4 [640/11800 (5%)]\tLoss: 0.750924\n", + "Train Epoch: 4 [1280/11800 (11%)]\tLoss: 0.597976\n", + "Train Epoch: 4 [1920/11800 (16%)]\tLoss: 0.711546\n", + "Train Epoch: 4 [2560/11800 (22%)]\tLoss: 0.744140\n", + "Train Epoch: 4 [3200/11800 (27%)]\tLoss: 0.756678\n", + "Train Epoch: 4 [3840/11800 (32%)]\tLoss: 0.566600\n", + "Train Epoch: 4 [4480/11800 (38%)]\tLoss: 0.728696\n", + "Train Epoch: 4 [5120/11800 (43%)]\tLoss: 0.720701\n", + "Train Epoch: 4 [5760/11800 (49%)]\tLoss: 0.566305\n", + "Train Epoch: 4 [6400/11800 (54%)]\tLoss: 0.689037\n", + "Train Epoch: 4 [7040/11800 (59%)]\tLoss: 0.678461\n", + "Train Epoch: 4 [7680/11800 (65%)]\tLoss: 0.550391\n", + "Train Epoch: 4 [8320/11800 (70%)]\tLoss: 0.694139\n", + "Train Epoch: 4 [8960/11800 (76%)]\tLoss: 0.700623\n", + "Train Epoch: 4 [9600/11800 (81%)]\tLoss: 0.623185\n", + "Train Epoch: 4 [10240/11800 (86%)]\tLoss: 0.696449\n", + "Train Epoch: 4 [10880/11800 (92%)]\tLoss: 0.732011\n", + "Train Epoch: 4 [11520/11800 (97%)]\tLoss: 0.734793\n", + "\n", + "Test set: Average loss: 0.9336\n", + "Class 0 accuracy: 809.0/980(83%)\n", + "Class 1 accuracy: 986.0/1135(87%)\n", + "Class 2 accuracy: 593.0/1032(57%)\n", + "Class 3 accuracy: 575.0/1010(57%)\n", + "Class 4 accuracy: 283.0/982(29%)\n", + "Class 5 accuracy: 60.0/892(7%)\n", + "Class 6 accuracy: 772.0/958(81%)\n", + "Class 7 accuracy: 550.0/1028(54%)\n", + "Class 8 accuracy: 943.0/974(97%)\n", + "Class 9 accuracy: 956.0/1009(95%)\n", + "Train Epoch: 5 [0/11800 (0%)]\tLoss: 0.598379\n", + "Train Epoch: 5 [640/11800 (5%)]\tLoss: 0.805639\n", + "Train Epoch: 5 [1280/11800 (11%)]\tLoss: 0.687005\n", + "Train Epoch: 5 [1920/11800 (16%)]\tLoss: 0.553637\n", + "Train Epoch: 5 [2560/11800 (22%)]\tLoss: 0.630970\n", + "Train Epoch: 5 [3200/11800 (27%)]\tLoss: 0.618934\n", + "Train Epoch: 5 [3840/11800 (32%)]\tLoss: 0.695465\n", + "Train Epoch: 5 [4480/11800 (38%)]\tLoss: 0.617699\n", + "Train Epoch: 5 [5120/11800 (43%)]\tLoss: 0.745449\n", + "Train Epoch: 5 [5760/11800 (49%)]\tLoss: 0.529404\n", + "Train Epoch: 5 [6400/11800 (54%)]\tLoss: 0.633640\n", + "Train Epoch: 5 [7040/11800 (59%)]\tLoss: 0.788254\n", + "Train Epoch: 5 [7680/11800 (65%)]\tLoss: 0.565721\n", + "Train Epoch: 5 [8320/11800 (70%)]\tLoss: 0.707190\n", + "Train Epoch: 5 [8960/11800 (76%)]\tLoss: 0.583630\n", + "Train Epoch: 5 [9600/11800 (81%)]\tLoss: 0.549434\n", + "Train Epoch: 5 [10240/11800 (86%)]\tLoss: 0.546574\n", + "Train Epoch: 5 [10880/11800 (92%)]\tLoss: 0.536187\n", + "Train Epoch: 5 [11520/11800 (97%)]\tLoss: 0.731755\n", + "\n", + "Test set: Average loss: 0.9307\n", + "Class 0 accuracy: 807.0/980(82%)\n", + "Class 1 accuracy: 977.0/1135(86%)\n", + "Class 2 accuracy: 588.0/1032(57%)\n", + "Class 3 accuracy: 548.0/1010(54%)\n", + "Class 4 accuracy: 283.0/982(29%)\n", + "Class 5 accuracy: 70.0/892(8%)\n", + "Class 6 accuracy: 772.0/958(81%)\n", + "Class 7 accuracy: 546.0/1028(53%)\n", + "Class 8 accuracy: 945.0/974(97%)\n", + "Class 9 accuracy: 958.0/1009(95%)\n", + "Train Epoch: 1 [0/12665 (0%)]\tLoss: 0.606930\n", + "Train Epoch: 1 [640/12665 (5%)]\tLoss: 0.566195\n", + "Train Epoch: 1 [1280/12665 (10%)]\tLoss: 0.488845\n", + "Train Epoch: 1 [1920/12665 (15%)]\tLoss: 0.440726\n", + "Train Epoch: 1 [2560/12665 (20%)]\tLoss: 0.412488\n", + "Train Epoch: 1 [3200/12665 (25%)]\tLoss: 0.525131\n", + "Train Epoch: 1 [3840/12665 (30%)]\tLoss: 0.394614\n", + "Train Epoch: 1 [4480/12665 (35%)]\tLoss: 0.449948\n", + "Train Epoch: 1 [5120/12665 (40%)]\tLoss: 0.455428\n", + "Train Epoch: 1 [5760/12665 (45%)]\tLoss: 0.374721\n", + "Train Epoch: 1 [6400/12665 (51%)]\tLoss: 0.441609\n", + "Train Epoch: 1 [7040/12665 (56%)]\tLoss: 0.499441\n", + "Train Epoch: 1 [7680/12665 (61%)]\tLoss: 0.482373\n", + "Train Epoch: 1 [8320/12665 (66%)]\tLoss: 0.431031\n", + "Train Epoch: 1 [8960/12665 (71%)]\tLoss: 0.455554\n", + "Train Epoch: 1 [9600/12665 (76%)]\tLoss: 0.415438\n", + "Train Epoch: 1 [10240/12665 (81%)]\tLoss: 0.355638\n", + "Train Epoch: 1 [10880/12665 (86%)]\tLoss: 0.477118\n", + "Train Epoch: 1 [11520/12665 (91%)]\tLoss: 0.512437\n", + "Train Epoch: 1 [12160/12665 (96%)]\tLoss: 0.351764\n", + "\n", + "Test set: Average loss: 0.8019\n", + "Class 0 accuracy: 968.0/980(99%)\n", + "Class 1 accuracy: 1130.0/1135(100%)\n", + "Class 2 accuracy: 604.0/1032(59%)\n", + "Class 3 accuracy: 753.0/1010(75%)\n", + "Class 4 accuracy: 673.0/982(69%)\n", + "Class 5 accuracy: 496.0/892(56%)\n", + "Class 6 accuracy: 695.0/958(73%)\n", + "Class 7 accuracy: 825.0/1028(80%)\n", + "Class 8 accuracy: 589.0/974(60%)\n", + "Class 9 accuracy: 751.0/1009(74%)\n", + "Train Epoch: 2 [0/12665 (0%)]\tLoss: 0.382221\n", + "Train Epoch: 2 [640/12665 (5%)]\tLoss: 0.359024\n", + "Train Epoch: 2 [1280/12665 (10%)]\tLoss: 0.443718\n", + "Train Epoch: 2 [1920/12665 (15%)]\tLoss: 0.352842\n", + "Train Epoch: 2 [2560/12665 (20%)]\tLoss: 0.390380\n", + "Train Epoch: 2 [3200/12665 (25%)]\tLoss: 0.450553\n", + "Train Epoch: 2 [3840/12665 (30%)]\tLoss: 0.285430\n", + "Train Epoch: 2 [4480/12665 (35%)]\tLoss: 0.317991\n", + "Train Epoch: 2 [5120/12665 (40%)]\tLoss: 0.304995\n", + "Train Epoch: 2 [5760/12665 (45%)]\tLoss: 0.402834\n", + "Train Epoch: 2 [6400/12665 (51%)]\tLoss: 0.410913\n", + "Train Epoch: 2 [7040/12665 (56%)]\tLoss: 0.323035\n", + "Train Epoch: 2 [7680/12665 (61%)]\tLoss: 0.359033\n", + "Train Epoch: 2 [8320/12665 (66%)]\tLoss: 0.332048\n", + "Train Epoch: 2 [8960/12665 (71%)]\tLoss: 0.344597\n", + "Train Epoch: 2 [9600/12665 (76%)]\tLoss: 0.363583\n", + "Train Epoch: 2 [10240/12665 (81%)]\tLoss: 0.267385\n", + "Train Epoch: 2 [10880/12665 (86%)]\tLoss: 0.303813\n", + "Train Epoch: 2 [11520/12665 (91%)]\tLoss: 0.394272\n", + "Train Epoch: 2 [12160/12665 (96%)]\tLoss: 0.275770\n", + "\n", + "Test set: Average loss: 0.7998\n", + "Class 0 accuracy: 970.0/980(99%)\n", + "Class 1 accuracy: 1130.0/1135(100%)\n", + "Class 2 accuracy: 586.0/1032(57%)\n", + "Class 3 accuracy: 743.0/1010(74%)\n", + "Class 4 accuracy: 675.0/982(69%)\n", + "Class 5 accuracy: 498.0/892(56%)\n", + "Class 6 accuracy: 686.0/958(72%)\n", + "Class 7 accuracy: 819.0/1028(80%)\n", + "Class 8 accuracy: 607.0/974(62%)\n", + "Class 9 accuracy: 756.0/1009(75%)\n", + "Train Epoch: 3 [0/12665 (0%)]\tLoss: 0.361247\n", + "Train Epoch: 3 [640/12665 (5%)]\tLoss: 0.295900\n", + "Train Epoch: 3 [1280/12665 (10%)]\tLoss: 0.298250\n", + "Train Epoch: 3 [1920/12665 (15%)]\tLoss: 0.288731\n", + "Train Epoch: 3 [2560/12665 (20%)]\tLoss: 0.304051\n", + "Train Epoch: 3 [3200/12665 (25%)]\tLoss: 0.301107\n", + "Train Epoch: 3 [3840/12665 (30%)]\tLoss: 0.264046\n", + "Train Epoch: 3 [4480/12665 (35%)]\tLoss: 0.236352\n", + "Train Epoch: 3 [5120/12665 (40%)]\tLoss: 0.251096\n", + "Train Epoch: 3 [5760/12665 (45%)]\tLoss: 0.369783\n", + "Train Epoch: 3 [6400/12665 (51%)]\tLoss: 0.356867\n", + "Train Epoch: 3 [7040/12665 (56%)]\tLoss: 0.247389\n", + "Train Epoch: 3 [7680/12665 (61%)]\tLoss: 0.264304\n", + "Train Epoch: 3 [8320/12665 (66%)]\tLoss: 0.243768\n", + "Train Epoch: 3 [8960/12665 (71%)]\tLoss: 0.328787\n", + "Train Epoch: 3 [9600/12665 (76%)]\tLoss: 0.294538\n", + "Train Epoch: 3 [10240/12665 (81%)]\tLoss: 0.300040\n", + "Train Epoch: 3 [10880/12665 (86%)]\tLoss: 0.258787\n", + "Train Epoch: 3 [11520/12665 (91%)]\tLoss: 0.283641\n", + "Train Epoch: 3 [12160/12665 (96%)]\tLoss: 0.338280\n", + "\n", + "Test set: Average loss: 0.7930\n", + "Class 0 accuracy: 971.0/980(99%)\n", + "Class 1 accuracy: 1130.0/1135(100%)\n", + "Class 2 accuracy: 589.0/1032(57%)\n", + "Class 3 accuracy: 741.0/1010(73%)\n", + "Class 4 accuracy: 671.0/982(68%)\n", + "Class 5 accuracy: 516.0/892(58%)\n", + "Class 6 accuracy: 685.0/958(72%)\n", + "Class 7 accuracy: 813.0/1028(79%)\n", + "Class 8 accuracy: 608.0/974(62%)\n", + "Class 9 accuracy: 755.0/1009(75%)\n", + "Train Epoch: 4 [0/12665 (0%)]\tLoss: 0.236698\n", + "Train Epoch: 4 [640/12665 (5%)]\tLoss: 0.221445\n", + "Train Epoch: 4 [1280/12665 (10%)]\tLoss: 0.236138\n", + "Train Epoch: 4 [1920/12665 (15%)]\tLoss: 0.257844\n", + "Train Epoch: 4 [2560/12665 (20%)]\tLoss: 0.233735\n", + "Train Epoch: 4 [3200/12665 (25%)]\tLoss: 0.222089\n", + "Train Epoch: 4 [3840/12665 (30%)]\tLoss: 0.307528\n", + "Train Epoch: 4 [4480/12665 (35%)]\tLoss: 0.285662\n", + "Train Epoch: 4 [5120/12665 (40%)]\tLoss: 0.240001\n", + "Train Epoch: 4 [5760/12665 (45%)]\tLoss: 0.292137\n", + "Train Epoch: 4 [6400/12665 (51%)]\tLoss: 0.290430\n", + "Train Epoch: 4 [7040/12665 (56%)]\tLoss: 0.299295\n", + "Train Epoch: 4 [7680/12665 (61%)]\tLoss: 0.255210\n", + "Train Epoch: 4 [8320/12665 (66%)]\tLoss: 0.203961\n", + "Train Epoch: 4 [8960/12665 (71%)]\tLoss: 0.238729\n", + "Train Epoch: 4 [9600/12665 (76%)]\tLoss: 0.249784\n", + "Train Epoch: 4 [10240/12665 (81%)]\tLoss: 0.193929\n", + "Train Epoch: 4 [10880/12665 (86%)]\tLoss: 0.198174\n", + "Train Epoch: 4 [11520/12665 (91%)]\tLoss: 0.270540\n", + "Train Epoch: 4 [12160/12665 (96%)]\tLoss: 0.217733\n", + "\n", + "Test set: Average loss: 0.8017\n", + "Class 0 accuracy: 971.0/980(99%)\n", + "Class 1 accuracy: 1130.0/1135(100%)\n", + "Class 2 accuracy: 575.0/1032(56%)\n", + "Class 3 accuracy: 736.0/1010(73%)\n", + "Class 4 accuracy: 671.0/982(68%)\n", + "Class 5 accuracy: 511.0/892(57%)\n", + "Class 6 accuracy: 682.0/958(71%)\n", + "Class 7 accuracy: 814.0/1028(79%)\n", + "Class 8 accuracy: 607.0/974(62%)\n", + "Class 9 accuracy: 750.0/1009(74%)\n", + "Train Epoch: 5 [0/12665 (0%)]\tLoss: 0.226164\n", + "Train Epoch: 5 [640/12665 (5%)]\tLoss: 0.318618\n", + "Train Epoch: 5 [1280/12665 (10%)]\tLoss: 0.238432\n", + "Train Epoch: 5 [1920/12665 (15%)]\tLoss: 0.262093\n", + "Train Epoch: 5 [2560/12665 (20%)]\tLoss: 0.286501\n", + "Train Epoch: 5 [3200/12665 (25%)]\tLoss: 0.219213\n", + "Train Epoch: 5 [3840/12665 (30%)]\tLoss: 0.250186\n", + "Train Epoch: 5 [4480/12665 (35%)]\tLoss: 0.270830\n", + "Train Epoch: 5 [5120/12665 (40%)]\tLoss: 0.149153\n", + "Train Epoch: 5 [5760/12665 (45%)]\tLoss: 0.206908\n", + "Train Epoch: 5 [6400/12665 (51%)]\tLoss: 0.237038\n", + "Train Epoch: 5 [7040/12665 (56%)]\tLoss: 0.289587\n", + "Train Epoch: 5 [7680/12665 (61%)]\tLoss: 0.243903\n", + "Train Epoch: 5 [8320/12665 (66%)]\tLoss: 0.234777\n", + "Train Epoch: 5 [8960/12665 (71%)]\tLoss: 0.194367\n", + "Train Epoch: 5 [9600/12665 (76%)]\tLoss: 0.248726\n", + "Train Epoch: 5 [10240/12665 (81%)]\tLoss: 0.233232\n", + "Train Epoch: 5 [10880/12665 (86%)]\tLoss: 0.258024\n", + "Train Epoch: 5 [11520/12665 (91%)]\tLoss: 0.280726\n", + "Train Epoch: 5 [12160/12665 (96%)]\tLoss: 0.206879\n", + "\n", + "Test set: Average loss: 0.8039\n", + "Class 0 accuracy: 971.0/980(99%)\n", + "Class 1 accuracy: 1130.0/1135(100%)\n", + "Class 2 accuracy: 575.0/1032(56%)\n", + "Class 3 accuracy: 739.0/1010(73%)\n", + "Class 4 accuracy: 672.0/982(68%)\n", + "Class 5 accuracy: 520.0/892(58%)\n", + "Class 6 accuracy: 676.0/958(71%)\n", + "Class 7 accuracy: 812.0/1028(79%)\n", + "Class 8 accuracy: 597.0/974(61%)\n", + "Class 9 accuracy: 746.0/1009(74%)\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8Yad6wstNUpx" + }, + "source": [ + "# Analysis of the results" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "id": "gimHfUXdW4_K" + }, + "outputs": [], + "source": [ + "# The following helper code takes the logs and converts them into a dataframe\n", + "# for easier reading. You can also store the result as a CSV or HDF file by\n", + "# using the .to_csv and .to_hdf methods from pandas for later reading.\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import pandas as pd\n", + "\n", + "def format_results(history):\n", + " logs = pd.DataFrame(history).round(2)\n", + " logs.index.name = 'epoch'\n", + " logs = logs.reset_index(drop = False)\n", + " return logs\n", + "\n", + "#display(format_results(history_regular_mnist).head())\n", + "#display(format_results(history_catastrophic_forgetting).head())\n", + "#display(format_results(history_memory_replay).head())" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "jR6eRKn4WguU" + }, + "outputs": [], + "source": [ + "# Finally, we plot the results and optionally compare the three different training setups.\n", + "# Try to adapt and extend the plotting function to the needs of your experimental setup.\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import seaborn as sns\n", + "\n", + "fig, axes = plt.subplots(10, 2,figsize = (5, 7), sharex = True)\n", + "\n", + "# Iterate over the different training setups for comparison\n", + "for log_id, logs in enumerate([history_catastrophic_forgetting, history_memory_replay]):\n", + " # Format the logs into a dataframe\n", + " logs = format_results(logs)\n", + " # Iterate through the 10 different MNIST classes (0...9). We will plot one subpanel\n", + " # for each of them, showing the accuracies for that particular class over the course\n", + " # of the entire training.\n", + " for class_ in range(10):\n", + " # Get the correct subpanel of the plot\n", + " ax = axes[class_, log_id]\n", + " \n", + " # Draw a line plot: The x axis will be the epoch, the y axis will be the accuarcy\n", + " # for predicting a particular class.\n", + " ax.plot(logs.epoch, logs[class_], color = 'black')\n", + " \n", + " # Finally, we will optimize the plot a bit and remove unneeded lines\n", + " ax.set_ylim([0, 1])\n", + " sns.despine(bottom = True, trim = True)\n", + "\n", + "axes[-1].set_xlabel(\"Epochs\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "source": [ + "Here are some example outputs:\n", + "\n", + "| With catastrophic forgetting | Training setup using a regularizer |\n", + "|-|-|\n", + "|![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAfgAAAMNCAAAAABRgzkgAAAAIGNIUk0AAHomAACAhAAA+gAAAIDoAAB1MAAA6mAAADqYAAAXcJy6UTwAAAACYktHRAD/h4/MvwAAABBjYU52AAAPAAAABDgAAAmIAAABKWSQUlwAAC9eSURBVHja7Z17XJRV/sc/w20G1Bm8IEqieC9vSCikpqJiiMpF0NQ1l/D2s9RSdtvNLYXabe2y66UkLctsa/MSJgOappKXUldTI81bYpg3UDAZFBhA5vn94Y0ZgWHmOfMw53m+75fLq5md833OOe/5nnmu56gEEErEhbqAxBMkniDxBIknSDwhL/F7o/xU6dQryhNfEphKfaII3MxfRkZSlyhSfHXKywGhooWKOkmO1LFzt0in03m3vGn5dqHKXlpcpv62yu2WdvfvR4wyfn4SUOz/0NseLxuNZXf/lRmNZVX13tb1994ksdY4UKD7w52uvdu9Gk+NxvPOH42ne13jb09G4tXqGt/WLrKzSeljPlzoRWatoEfU+9VfCw76qZXwOD6q/Y3PSKwVBD1izN5w1C6Whfhb2dnIzb7gkE25zsEyughshTM5HhGSbMhC/OGgICQFLXTMtqY0PrWD1NZNBoY0aQjxYYIgCMIax2xLNwVLSa018dHSbEhV9+BbrDNo2W0sp4twuivJrYMCX+GCf0NkvGPpNBrvkty62CIESeNd4qtzc7HmBtmtA71UI73E4of0KP2Y7NZO2XaLgzm5iFfNxXu3yW+tfFvaprcsxeMPzS/oyW+tZCBaqmtiEov3nElHdLVjypTsJ17awzkAV9rdPhxMimvmh5DGhWp5Zjz8nsYyMlwLGRghlXfpb7Z8EevySHFt4iUb6aUXH9K/ciUprpHcY66j5CseL2KFkSTXRCaebCZj8XH+BetIck1IOdI3gHi32VhKl+VroGgPouQsHtM8f9pDmh9m2+3HOstafLMEOqKrCUlHeslP4ADAqW6qnA4k2oJKH8O+/rLOeDwWISwn0ZZ8Z2gZKuHmGuRp2bn4qJhMW6DHaFe5i3+q6801ZNocQdqf+IYR7/Ii3jWRazN+Pq8Jl714/NH73BZybUYGwhvJX3yj6XRE95B4SUf6BjmcA3ChQ9WxniDuk+eHK63ln/FoG0cpb8ZmhErqvcEmP3oRn+eQ7gdIPdI3mPj+/csHnSTf9yjZqRTxqo098gYfJeN32WFs310Z4tFqd5/CoftJ+R0kvK+6ocWjedZAw/Ascg4AVZulHukbcmZL7baI0lGZZB3AwQLvgcoRDy/9mPK49aQdyMBIdwWJh3rDM7cn0lOUDXAw19CTGLt9+n/CNDqTc/aU2whliYfLij9h7htKv/cyA2G6hhafGqAJPSTh9lXvvIZX5yvcfAOM9Jbi1yclHw2MuCal+YX/wltzFH11/vr3Ut5XXbP4xdMTu630Wi1pFf70gSp1ipKnS/ja1CtA8o2aT2lacWQ+4BJ+QNo6zGiU8Gm+gp+d3irZ/Ce1ii+s8gXgexrAnWnLpbkncpLXhG++UXDKN4R4K9OW1zT3VnlKOYMNm0UZUy7YhzHZKDCATRj7owQ7tntrwvwOnAqvtFggoUh/L+NrWqiAzV05zhRFhk2yGsU84z2CswBTVj8AgFqr1ep8aIEKeWIxX31SQp+QpSWJ1C9KEz++YGF+722+1C+yxzXF/HXIvAXT21grE+bGYstOFEWGTbIWRUVzFCgTWmKUxBMkniDxBIknSDxB4gkST5B4gsQTJJ4g8QSJJ0g8QeIJEk+QeMIx4vdG+anSqVeUJ74kMJX6RBFY3JEXGUldosiMJxSa8dWp7REqQubiF70GAA89glXyip2bEn44APhPmdKWer0Ovh+otfdBpvH9bPjww/fVqzbFPsj4Yn+W05b//NFnv0MVMT1K6rm9OOIv70z6XIrt1PEbr9ZqtWwnq++x9PIXQ4Rt8W3+epYM14JU8+FYZPytHAQtHtLs/mjsgIUKcj7+5CoweHq8hiw/zJlH3QukmQHL/An9XQCAhPuvDTAIzKnYNNIFaJcvEA/xDoZLsyGLoT5MEARBWOPY75p77Jbzr/n+tgZEg430DbUmDbBqRu8fybMlhb6m8+0k2VKDncCJd8s+TaIt2WLqLY33hhPfLAJrSbQl0s1x2XCnbCdiLT2bb4HxGwWIj/E8S2vSWLCrxO9x2YtvHIV1pNocCZemacCrcxOwTvTcxVUn5fRzIeWK0g0oPlJ7aZ/YGK90XyEj8UevNBqiAPGaONH79UXL8ZaM5r3OwAjJzmM35I0YE/FlpbgIq0pw4Sv5iNdLuGBBQ4of6lP4ragAle+iJxbLxvtvP7mMVIR4t3Eix/qNl1pu9jh4ADIhEwNaKEI8JmKTUURxYQmebzsJS+QiXtKlaRrsIg0AmAIuboyzv/j+AeoLLY/3cjkXIAvvBp/KM12UkfEuE0SN9YvxTEv0DDe9J4+E/6byUem8N/Dt1ROx+abdhXM3YS6AefjoJuSAtIuQNaz43l2MersLv2ca3gPAiK7FqyEDKrcoSLxqov1jffFHmAcALvOwrEoG4r8vavGEYsRjIrZft7PoxzcfjQAATG6Wqwf/ZGC0q3LEdw26vdG+klXvYt6dynvNlMMRnaCXdp3Rhn52zu6xPv1888l3/3OW+/eHuRd/Mlc9XEnix2PPZbsKLsFMz7v/6TdBBimfgfDGShLfdoCwwZ5yh/a5z7r/Yh42XOJfvLQrSjf4Y9J2jvVLMLH1/RdBg28v59x7/kGMVpb4ca4/nLO91MUv7xzL3U/5D0v4Fr9F6OunLPEth9lz693yqrDe1V6O7nTjU77FS7xP7wwzYthzvv7WB2YJD9cXsdTU4C0RQekO5Ykf43HiuK1l1hg6mf8iPut9dgvP4nca2/VUmnjvkTanvGkZ5ppXvPEMvo/oMhAj8ZQzTjD50USss/Ee6c053gkWb8123ZXNr3dTptQjvTOIH90o95BtJZZghuXJDv9xWMqv+EPXdIOUJ94rxsax/sfdbnMeenMevsjjVnwGIqWeFsgZ5rmbiPU2XVddgnEPL3gdMqDyfY7FSz3SN+w9d3epaHUja2j9P57XrvJQ34ff3ji2xQVPPr2f6+RW4K3AjPcYa9M5nNTKATV4R2xA4edO0Bj7En6w1N4fEp8aoAk9JHUlJiKtot4fLl2BpJred30BSzh9grIBRnrLWa/Weaw+Md37qmNnvXqI262RWe8Pr0T72zX+H4Ym2MrlTFfXXfGr5Bu1yPjF0xO7rfSS+uZF16frv19vWooXar5FSTuN05M4W6t6tpd8o+Y7dxVeabFAQtH9e9gk2bkDDj7R6KV6fvTqiiaXaqnS+Y6mv/K4e7f58Cv/kHyj5pMYF1b5AvC9Mx1VeTlQLE0tQjr8mlLvD0+r7asYEJf2FpcpjxjpN1nHXv0inU7nX8P75SnlDDZsFkV1zoZfp8W11uVLe3/wjMlGBj+b9kfp69jurYk6hvra5qtnM/w7UxQZNslqFPOM9wjOAkxZd+Y9V2u1Wp0PrVMgTywWKkhK6BOytCRRmX2hZPHjCxbm997mS/0ie1xTzF+HzFswvY21MmFuLLbsRFFk2CRrUVQ0ragyoeXHSDxB4gkST5B4gsQTJJ4g8QSJJ0g8QeIJEk+QeILEEySeIPEEiSccI35vlJ8qnXpFeeJLAlOpTxSBxR15kZHUJYrMeEKhGV+d2h6hImSe8Yt0Op13S3ms8ETYIH6+wWC4SD0kU+oY6tXqBqpT5U+/mD3mUWU0GsuMZXf/6N7pTdqYi7+VA+RmN2vbYPUpOrBv/8HSOj8Ssb8jeRONxSNUu4cAQMKae68lmgoFACD8um//vhMCAO9AD7M6emo09/9pFmZ32teSxDEWb4lU4k2H9u3bfxUAOg0YMODRug4y8/qf77NL0nV7SLzjmLgOgEfwgAH9rSfzLwMKIzKlnvqVxDuE7RFuIwcMCNbU79MHh5b+cQ2dXpCB+Kqg43NtmaJuS0zVy4vInSic4pTtp8e9X7Xl86NW4c13yR334ktexavNbSqR+AbmbnCCmpN4Ufw7L2C2jUXmzxIm7yJ7fIvPextv2nqSULUsviL2GOnjWnxySejTNhdy/XxQ8YjfyB/H4n/+GP+249hMo++RF3GdBPIr/i+m+AH2lPPe6n9mdCkZ5FX8jq1ub9pXss22pv8bf5sU8im+6s+Y1cnOst0yNZufo3n6+BT/2THdArsLD1jn8tEH5JBH8aWv2HruxoyYt5HC+cLxChW/+IrN527MeKHD1WUkkT/x+W9ikUZMAPfX8fYNssid+JSSkPHiIkzsaXibLNpBg16WPdnT9N2TImNkRnuea00e+cr4l0xxYr1jdL+yN0gjXxm/c7jbyc6io+we4n5G+vX6KOPtp+rPeF68d4Q9VZlMHnnK+E+f1eW0YBDnSB/VsR5kkpuML30Fr7DwjuB4YQEIbsQvudxuDptIf3dJP0gmeRF/Vey5mwc8loC/kUlOxAsv3eo7nlWwZI9vs0glF+Kr/u8z/JvZttvNxN/o8iwP4iv/uMrl44Hs4r3S6FA6uXR+8eXjvnD7YgrDgC3n4dUqkuns4kui9OpN45mG/FPTk/8lmU4u3jBiR6Mto9nG9H4ZyRVk06nFXx/2vW7HMNZRZ7c+/yHZdGbxeYOPtNjVj3lYr4X4B92E5cTifxt0wm9PkAMCT+lwlR6gdV7xvwzMCfiumyMie9BNWKLEpwZoQg85bGvHB13s+l0Hx8Se2LOIbsKyW/z6pOSjgRHXHLSxHwZfDdzbxlEteQPL8kmoneIXT0/sttJrtWO2tXfYjdBdjpupbHS/sn+Q0PpiPsFhxZH5gEv4gbpKlC61c1Nl/y4Ly2jiuKao/jnkw5ZuCnY5srctnxaqcxn7BUF4KUQQBEEwGgyGizAIFhSKqFmp4FCeUnYSr7Klr6zMXu3/8NvNjclGO8Vs8XwQpjylnEFbzaN8Y+83xv4mOVOUabChe83vuavwSosFEor0QO3z1bO5D8+ZosiwSVajmGe8R3AWYMq6c2JNrdVqdT40kaA8sdgZSkroE7K0JJH6RWnixxcszO+9zZf6Rfa4ppi/Dpm3YLq1UyyuYSwOmpwpigybZC2Kiu5VUya0/BiJJ0g8QeIJEk+QeILEEySeIPEEiSdIPEHiCRJPkHiCxBMkniDxhGPE743yU6VTryhPfElgKvWJIrC4Iy8ykrpEkRlPKDTjq1PbI1SEzDN+kU6n82550zHb/Vmljh7Ys00jlc24biNrjs34+UlAsb+DtpuB4RkAUHGj6MaNG0U3DEajscxoLLvzp44nPQXT4hGkzaHi1WoHbjcD0QAAD18bn9f6tePO39qRN9FYDPW3srORm33B4ZvNPwg757bsMFT4hLQxF384KAhJQQsdvtnNCPGzs+hUfELzFYvHYqgPk+hRunsjvR2M0V3IUvicJw7IeIko3WG/eM9J+Ji8cSp+hzHA/gXDpiL9OonjU3wGou0/L/R47wqam55P8VWZ9o/0AKbiY3qqn0vxhwp0g0QUn6Q+doTM8Sg+AyPdRRRvGke7d7yKjxZVfiq+KCV1/InPOekm7nT7kPbFG0kdf+IzMNhbXKUTaaznU3y0yAjPqvbkkDvexF//HlEiQ/hHYDUIzsRvrerZXmyMqVhzm+RxJj4DMaJjRLfIoxtxOBNfvk30Tzzg8QyN9byJ33OzdbD4KFOReZXscSVejygGG+0RcvszsseTeCGDwUgPulLDnfjsS15DWcSZ4HX6AOnjSHwGnvJkEUc7js7ecSY+hk2gqVh/EwQv4i8dVY1iE+nJLiUbyB834jPR34dNJNUUGus5Eq9ns08PAAmuB06RQE7EF3/LTnyrUZTy3IjfXtm5K7NgU/GfCjLIh/gMxLB73n5kq4ItZJAL8be3sBvpAbcEGus5Eb//9+b9GIabgq2XSSEP4jMwyo1huC5Pmj4lhRyIF/QsR3oAU7HaRA6dX/zpHA+2DziPa3LuC3Lo/OIzMLQJ04CN/ojJ0efIovOLj2Ec8e0kt8zur5aQR5uxspp0sc6gZbata62Ei21YN+DUCzvh/69xNBufE2f8FiGYuXc8tv2rdhfHD/uZVIoTnxqgCT3koG1lMN6nv4NqzKkUza7eLxaRTBHi1yclHw2MuOaQTZVtd4h4wDP5VFzVu10+piM7WxDMCJklCFV+i+6/NsAgsCIT/ibBUWx/FOh7UCDqi/mJtIoj8wGX8DrvYiyzd0b7LWImvrHG8GPvpfwQOrmXknP4KZtab/Y1uIz9giC8FCIIgiAYDQbDxYczvtD+mm136Hc4L0Hhg/cqW3rLyuzVNcxh3NyYbLRTzfBqYcpTyhm0tXqUVmvs/s7Y3yRnijLNlu41P46v8EqLBRKK9EDt89WzObZ3pigybJLVKOYZ7xGcBZiy7lw6VWu1Wp0PnRmRJxZXSZMS+oQsLUmkflGa+PEFC/N7b/NVZl8oCtcU89ch8xZMt3Ze1TWMxd0UzhRFhk2yFkVFj5wqE1p+jMQTJJ4g8QSJJ0g8QeIJEk+QeILEEySeIPEEiSdIPEHiCRJPkHjCMeL3Rvmp0qlXlCe+JDBVmf2gOCzuyIuMpC5RZMYTCs346tT2CBUh84xfpNPpvFs69yoQz6jspZ3C7yt/+L561abYBxlf7M9w8iP2VLQ0JDat/f/WeGo0nvf+qKt/xyseKznUV9Hi6xjq1Wqnr/0eg+8qV7tKjtqQpmzxFkP9rexs5GZf4KX26Yi2zzvGIk3ZY73FUL97CAAkrLn3muk8d8wR/C9vGWlf0Vsty44GUcbfJ0wQBEFYw0nlj1xuZO/ihY0jkabojOf6OD4dkRp7y47Fl4oe6zkXH2t32VHqs4qeDZNn8TknXEfaXVgboeyxnmfxeoQ1tb/0WBLPK2JGeiDK/eRJEs8j1/aJmv7eezg2knge2Sw87i+mvLLHeo7FixvpgRi3Y7+QeP64tV2k+GZDlTzW8yt+e3mHHuIiKHqs51e8XvQytbEuR38l8bxxO1PkSA/4hCl4rOdW/Hc3WvQXG0PJYz234tMRJXrmzzGqQ7+ReL4Q9AwWL2w1EF+ReL746TfP4eKjKHis51V8OiK8xEeJw36lLkDPr/hYBlEe6a/YsZ5T8ed/chnFIo5yx3pOxesxsAWLOPH4Lp/EcwSbkR5oGyJsIvH8cH0vq5XoFTvW8yl+i6lXezaR4rG7gMRzA6uRHujwuCmdxPNC2TfMxCt2rOdS/M7Str3Zic+6TuI5IV30pfgHdA6syiDxfFCVwW6kV+xYz6P4/YVNB7IUv6OIxHOBHqPc2UV7tHtlJonnASGd5Uiv1LHeytqyzjgxws891YWN2ca75swT/VDG3yUdw1l6R/eu5VuUl/EcitezOk9/F5Uix3r+hvqLbVV5vkwjZgdpCho7WSslz/jUAE3oIaeucQb6s/WOwI7GzYrLeAvx65OSjwZGXHPmGjPepwegGouJnce/tVNRp24thvrQvsth8p/z8r3XzjfUF/nc/qUz45i/jjoNAAh4/PHgx1sqQ7z5QwkVR+YDLuEH6ipRtqpBK3zydjfW3tHh1PUfjxw9mnP+/FfAI8FBzfh0Oay73eILq3wB+N75/peXA8UPlyh9sYHbF+OAmM3Dw4Gi7KNHjp65fJnXizarbBEPoTqXsV8QhJdCBEEQhGQAgEGwxJhsFMTjTFFk2CSrUcx/4yu80mKBhCL9vYyvab56Nr/7zhRFhk2yGsV8r94jOAswZfUDAKi1Wq3Oh9YpkCcWT5wmJfQJWVqSSP2iNPHjCxbm997mq8y+UBSuKeavQ+YtmN7GWpkwNxZbdqIoMmyStSgqhS/Nolho+TEST5B4gsQTJJ4g8QSJJ0g8QeIJEk+QeILEEySeIPEEiSdIPEHiCceI3xvlp0qnXlGe+JLAVOoTRWBxR15kJHWJIjOeUGjGV6e2R6gImWf8Ip1O593ypvzaLBiL8q8o/bbyh++rV22KfZDxxf7ON91Z7Zh+LiozlhmNd/+UC+a67/8fAPC3N5Qtvo6hXq3mrC3nn91T/w9//LoriX/ArRwgN7tZWw5bInwy96YmQOOp0Xje+aNRm/+Oae6+7anReML36oEnlZ3y5o/L7wIAJNx/bahhYgTnJD8aGJBT749PxlxOGuYgOJzStEY2zSj0+Puf6j9662Pbnlf08Yo8juMNz8YV9vrhLzb8aj/ldeGookd6WYj/tuenLvMP9bKliOdI5S4kLRfxZXOHXey49582HoPEYaOiD+X5F3/48WWYmT3A1mKjPM6cIvH8UvnaE6dbf73C9jmItcOVPdZzLv5C/5Sq8T/bdUkxDhtJPLfMOdx07Tr7ZiCNdsn+lcRzSm4m9k6ws2yLwVDoQtIyEL9CeKqH3YXjFf0jz/WZu9I2NzJH2136chtc9qOM55EvbrQXcavYI08gXbkZz7N4YTlmibm2GqfksZ7nof67QZ6XxKwpcK6T69XmlPH8sRzPiFpLoqNCF5LmXfzljZgtLoKSx3qOxX9QNaiXuAhx2C7De0nlLr78A7EJj+5dKr4m8byRdu2RWJEhVAo+X8+v+PcwU/Qq8nH4uozE88UPBz1miA7Sx79kB4nni+V4WvyakAoe63kVf20d5jAIE4eMShLPEx9V9A1hEGaAT9FuEs8Rt1cwSXi4xir1HA6n4vWXfJ5mEigem6pIPD+8hxlsHukcort6gMRzw/E9rjPZRPKIUuhYz6f45RjThlGoOHylyAcruBR/43PRp+nvE+H1myIfouNS/CelPQexiuUVqcyxnkfxVamYze4RZ4VelOdR/LZfvSexizbK/bQSH6LjUfx7mNKIXTRduCLP13Mo/pdvVM+zjKfMBys4FJ+KkR1Zxot2+TGXxKcGaEIPOXWNb65hc5r+Pj6DlPgQnYX49UnJRwMjrjlzjT8r7jycbURFXpS3eKAitO9ymPznvHzvtfM9UCF0P7XsBbYhL/njSmuliTef4LDiyHzAJbzOyxbGTxq0wldONUpgHLJN6MFX+srA5eBudosvrPIF4HsawN25bB8uUfJ8A7cvQcc6YtzBTz4B/6yyRbyV2av9H367uTHZyGBmRfujVF9DozylnEGP/aWhm8QmyjRbOsb8N77CKy0WSCjS38v4muarZ/O770xRZNgkq1HMM94jOAswZfUDAKi1Wq3Oh9YpkCcWs1cnJfQJWVqSSP2iNPHjCxbm997mS/0ie1xTzF+HzFsw3drNLa5hbiy27ERRZNgka1FUSl+bRanQ8mMkniDxBIknSDxB4gkST5B4gsQTJJ4g8QSJJ0g8QeIJEk+QeILEE44RvzfKT5VOvaI88SWBqdQnisDijrzISGV2g+IznlBoxlentkeoCJmLX/QaADjz2rKsiP+qh6+xzGgsM5YZbSza+2tOn6x/+L561abYBxlf7K8A8cYWJYeD7/63UG4sM5rqW/K3idfabX1MbhmvVnPZItvZVeL3+P2vvUbjXf+SHfdHnh2Q8SSPjbbYubuVnY3c7AsKMX6XDETbuyPTcf8TN8K5nEHHQvzhoCAkBS1UlHchA9F2F26RFVM+bhmHreZ5NWlGHOnTuFDEr1rVC+8j6R3uDovpOB4ZiBCzN+O6/E0snmjkrdUkHnoRIz0AqP76mfuGiBuctZqG+t8CXK62EBkjK66429a2lPFckYkBYr1j2Hd+J5/4icRzRYbIkR4A0Ot/3fMG7iTxHGHYzUI8/L8ffDPyCxLPD99UPtqFRRzvbybcnlpI4rmByUgPAOr/Bhk/JvG8ULmFlXi4zMH7t0k8J3xf1OIJVrEmNL+wmcRzQgZGu7KK5TkN75F4PhD0zEZ6AM+5fHuSxHPByVw1w3VO2kVjOYnnggyEN2YYbg7+YyDxfIiPZhluSLeSNSSeA/IPYjTLeKrZSDXx0XRli98s9PVjGnCy9ux2Eu/8MB7pgcaJvOzeKfp6fGlz40+92IY820V1tiMPbVd0xu80tuvJOGTnEQIfj50qWnwGYpg/IDYHq0tIvHNjymT9Ew9gREfD5yTeuTl4TTeIfYfOwnIelntRsvgMRLqzj5ro9fMeEu/k4qMdENV7MhdHdAo+nMvp7Fbg7YC4x3u55vpTxjsvmRjsCO/oGVa10vlbr2DxjhnpAczBh87/RJVyxf/+HaIcEznav3ADiXdatlb1bO+YyG7P4T2nP6JTrni9o0Z6YJr68CES76SUb3OceJ8Jzn/XpWLF77nZuo/Dgs/Ghqsk3jnJQJTj2t7nicoPSbxTImQ4bqQHMBsrK/kSnxqgCXX6HRMG/HTRa6gDw4/zvbKJK/Hrk5KPBkZck7/4DDzl6cDwHjOc/YS9hfjF0xO7rfRaLX/xeoeO9MBMt++ce4YM85ktK47MB1zCD9RVwvhfGXg3HlWNcugG/OI2zI+XuFFPdrXl00J1LmO/IAgvhQiCIAhGg8FwEQbBAo4e/q+L/oJj+U76Jq2ypX517NUv0ul0NVxebG5MNjLomIaOsq96m8pTyhl0vFmUJ6Vv0jRbWmR+Pb7CKy0WSCjSA7XPV8/mGr0zRZFhk6xGMc94j+AswJTVDwCg1mq1Oh9ap0CeWExbnpTQJ2RpSSL1i9LEjy9YmN97my/1i+xxTTF/HTJvwfQ21sqEubHYshNFkWGTrEVR8XAPOMEemtKUxBMkniDxBIknSDxB4gkST5B4gsQTJJ4g8QSJJ0g8QeIJEk+QeMIx4vdG+anSqVeUJ74kkI+5lwmxWNyRFxlJXaLIjCcUmvHVqe0RKkLmGb9Ip9N5t7yprP7IeauvygaaFvHa0Ifvq1dtin2Q8cX+cl6TxpLTaRuzAZeBHepbIOtCSjKnba1jqFerlZTqwom0tBOA69CxsS3rXejLp5e86C0L8bdygNzsZm2VpByAkJ228QzgHj42prkt5eK7n1jGacpbDPW7hwBAwpp7r+W9/Ng9row4DnhEjI1qamvJL5/Wnecz5S0yPkyRj9ItPa4ZOXaUPd9wflNe0QsO3sUUcDHN3omKuE15OoEDHLjYZKS9ZeO7G5Zx2WgSD6xFrN2THbokY0kRieeS219iov2l47sblpJ4Lvn2WvNwER2YjKU8pjyJx1qME7PuYHwPLlOexJd/JWak5zblSfzW4keeFBUgjsuUJ/FrMV5cJ/CZ8ooXfytT3EjPa8orXry+rFOw2D7kMeUVL34dJoq+x4jHlFf6ufrfW1We6CY6Sto47fmmfDVc6Rm/sbKXeO+I61nMW8orXfxa0bt2wJ1f+RskniOu7MYEFnHGcJfyChf/pfBEAJNu5C7lFS6ezUjPY8ore6/+144ul1uxCbVxLF879srO+HUYwsg7dymvdPETWYXi7Vde0eJPHHePYxaMs5RXtPi1GMHuZ9klGe9kk3geENayG+kBjIkoi75K4jngh1+9WC4p7bKuy8W4cl4ar2Tx6xDdiGU87wzd/pm8PIqkYPFV69mcrn1A1w0ua5aQeGfnuyveIxiHfGoxXtpK4p2ctYhjPgPAC1NNE06TeKemIo3pPv0dVO8/WRz1O4l3Znb+7juEfVSPjW1zxt8m8U7MWoxzdUDYlhmNdiaReOelNN0BIz0ABH6G91aReKdly612/RwTeczf8fxeEu+srMUER03d+Mr42/HneROfGqAJPaQA74avHTTSA1CtDi6MdvaJIS3Er09KPhoYcU3+4tPLH+3lsOBe+lbHJ5u4Er94emK3lV6r5S9+LYMHaGrnkXS1foFzd4D5dGcVR+YDLuEH6ipRvkEG3it3Om6kB4DQjyb/U91e4kY90dmWTwvVuYz9giC8FCIIgiAYDQbDRRgECwrlkfLBgmP5q/RNWmVL/azMXu3/8NvNjclGBh3T0FEOm41iKSyuo5tFeVP6Jk2zpUXmt1dXeKXFAglFeqD2+erZ3HLtTFFk2CSrUcwz3iM4CzBl3TmzodZqtTofWqdAnljMZZuU0CdkaUki9YvSxI8vWJjfe5sv9YvscU0xfx0yb8H0NtbKhLmx2LITRZFhk6xFUSlynnKC5sAh8QSJJ0g8QeIJEk+QeILEEySeIPEEiSdIPEHiCRJPkHiCxBMknnCQ+L1Rfqp06hXliS8JTKU+UQQWd+RFRlKXKDLjCYVmfHVqe4SKkLn4Ra8BgMwXHGTI1pHmr9WeGk21f2Z/3M3SqbGb9LWtY5Pzk4BifxJaX/R4+tkbRTdu3PmfAeX1fgLX/5i3M4lXq0HUH1MGEqvNjVtluGU0lhnLjHf+mf+x+EpcTH+2ocXfygFys5u1JY82czivSfWZMl2bNavjS1JV/dU/Xt8ovXiLvfrDQUFIClpIGm1HjxH1HiJd3KszDtuLG1p8mCAIgrCGNNojPsbOkt27VGxpaPGEvZw74TrSzqKqeKSReH4TfrDdK1qNxdYSEs+t+Bi7ywYFlG0j8XxS+L0I8ap4bCTxfLLFFNjO/tLx2Cz1umUkng1iRnog1O/mDhLPI2XfiBLvEif5WE/imZBV2iZITPl46CtJPIdkIFrU5euBPjd2k3j+MGWKGukB1zFSn8Mh8Sw4lK8NExchHulVJJ479Ij0EBdhSNNr35N4DsXHiIzgHi3xfj2JZ8DZU26i706Ox1eSrmJD4pkkfJi32BjDG1+WdPUvEs9EfIzoGJooacd6Ei+egv2IFh8lHhulnFCaxItnsymIwU2KIzxzfyTxXMFipAcaRUo61pN40ZRuZyJe4rGexItmZ1m7QBZxRnucOUniOUIv8gLNPbTDpRzrSbxYqjLZjPTAWBLPEwcLdIPYRIp2O5ZD4rlBj5HubCI1GyJhypN48eJjWIWS8mZbK8uPMVq7VcacedS9kFUXXW0tnG8nUb0p40Un/BBmqeE7EF9JVW8SL1p8DLtgEo71NNSL42pr4WIbZtEu+asut6aM54HNQjA772jzhLBJooqTeHEwHemlHOtpqBdFSQvjT70Yxsvt4JrfgjLe+dlhDOjJMl77oCq9NDUn8aLQI4btBJCSTY5hKT41QBMq6U1/XFO1me1PPBCPrKKGEL8+KfloYMQ1Ulo/DhQ2Hcg24qPdKjMbQvzi6YndVnqtJqX1Q49RrGcjlerarHm9K47MB1zCD9RVojydhN9jI+uRHoh//euBd2e89dRoPDUe9d+H6NvBbvGFVb4AfE/fMVwO1DDx3q0JJPw+HhGsI/bsdczeh+hW2SK+jr36RTqdroY5jJsbk42CeJwpiv1hyptUHwtTGExkozpkd5Om2VIX8xM4FV5psUBCkf5extc0Xz2bkzrOFEWGTbIaxTzjPYKzAFNWPwCAWqvV6nxonQJ5YrFTmpTQJ2RpSSL1i9LEjy9YmN97my/1i+xxTTF/HTJvwXRr1xldw1gcvDpTFBk2yVoUlZRPaBLOA12kIfEEiSdIPEHiCRJPkHiCxBMkniDxBIknSDxB4gkST5B4gsQTJJ5wjPi9UX6qdOoV5YkvCUylPlEEFnfkRUYqsxsUn/GEQjO+OrU9QkXIPOMX6XQ675Y3qY8cjFBhaoCtPnxfvWpT7IOML/anWa+YyL1uKDMay4wP/lR7YeqypZPkNapjqFerSZg4qgqu5OXl5eXlXcmvc3X4X0bubyF13SzE38oBcrObtQUhkjP/Opp3tdoY3tjTU6O5/0/jeXfWC42nxrNk2NmYnZ4S189iqN89BAAS1tx7TRMc2seJN9YJAFxatm7d2q9169atW9W12vTJ/oZx6yQ+vqKZLR3Asb9vFBCd6O/nU8/HH3dFVP75HRLPOUf/ng7Ev9rbljKfT0bq85LWkk7gMOZQVHC6avzxNJu845l/YE6mpPUk8UzZPyJ0s8szJ9b1sLXg36aZJhwm8ZyyJ3zAN67Pnv7sMduLqt6PKB19XsK60m88MyrHZsAt8eUOdhYvHnjssX1NKeP5Q5/h/lzOh/Z6h/brNqfGlEtWWxLPjJX4y/tiFg97ZEuTPVMkm5mGxLPilyzVdHERem10++JVEs8bH2CU2MUCh3+If64i8XxRtgYzRQdJXIjntpF4rkj7ve0I8VFS/lg1LpvE88RKzHAVH0W1auitUb+TeH44tt9tCos4Hhu7XvkXiecp4WPZrA3q/RbeLSDxvHDzMzzHKFR0cMnbJJ4X1t7qMoRRKNXrSM0n8XwgrMT/MbsNPTK07C3HV5ku0rDgUKj6cnNm0bZHqH/1o4zngZV4mp13DB9QvogyngduPFK2vx/DeLuGepx19I3OlPEM+E9ZrydYxhsSVvFPR9eZxItHWImZbJ8wfA0fnyfxTs/e040msY04KPz2P0i807MSk1jvCL2ONTkk3sm5tpHBBVkL+kVW/Z3EOzmrK0ODmAd9DZ+fIfFOjekD9gkP9I0yve7QatNxvFi2RXpf9mIf9sfHVce7U8Y7MSuR4ADvCIoTXqOMd2IutTOdfMwRgY/3wk+9KOOdlo9MYQ7xjp5PI8WB9Sbx4qhc5YhdOwBAsmrTURLvrGy+4jPGQaG7/QHJJN5ZWYmpHo6KvdBl8yES75yc266a4bDgXSY7MOUtxacGaEId9zWTHR9iRHvHRV/gum2/ROLXJyUfDYy4RkbrR/lqh+3aAUDHRCx0VGyL4/jQvsth8p/z8r3XdBxfJ19MapPr5sD457tU7h7smNDm1a44Mh9wCT9Q57d8Cwm/xxJMd6R3BExd+fJL9f50kE2/OkJ1LmO/IAgvhQiCIAhGg8FwEQbBgkLy/QDXS4JDuWDLIcMqWyJbmb3a/+G3mxuTjQya5ExR7A9z+5HqY2EKi5lMzKL4l9tQmWm21MX8N77CKy0WSCjSA7XPV8/md9+ZosiwSVajmGe8R3AWYMq6c6uwWqvV6nxonQJ5YrFvkpTQJ2RpSSL1i9LEjy9YmN97my/1i+xxTTF/HTJvwfQ21sqEsTiGcaYoMmyStSgqySZWI5wKukhD4gkST5D4h2By4TZFpVKpHhUT4d7yx8LC1p7hZ8VGeValUqnsnqduUd8mLWPPADDOat44/qrYKGEqlUpl91W/Fb202n5brdbFZvGMLtx2z8vL+15MgHvLH7/97sqDjSKMIqNgRF5e3lp767Jn1v92VD5VAszL/HLPlTixUTA9Ly/P7gmQ2rx55PDQmBNW62Lr2emQWYJQ5bdI5Mnx5EAGZ9ixSRBMrd4RhCL1WnFRhIQYsZW5hj1CkfuXgnAKB8RFEQa/KLprmn5krS62ZnzFkXCrF27rw1m/DpMuMPipys0PB3Shouuzu2XX566LCWBAMxypDAcebXtAXBTgvy16zC8VUZeqdSX9rNXF1nMFhVW+AHxPi+zn0DVd814b+HMT0eLz4QvAV+z8YCPi2p/7W+QB+yclNc0d0AP5Ht7iKmOaO6AH8Id2fsf+euYre4Mc72dsvKlbtpW6OPQ+gtqJBHqFttswFU7CBKBnr467h9kdYNbP3zOoxp0oM4CerYed62hnkK7ZhrSEPdY+ZetQ38L1KoCrrRi00rsLg2f/W4FZfTq0sL8+szfvagO0qigSVZk7UQAAobC7Mh6dghcFLrNWF1vFm124Fcetcwxmf23fKgsoPsiiPpeu21sfYfamb9sDCHbPAs5c6CcuCgAgG6I6x1RutS627i+uU685OcM7X+Re55925+4Lb3FNRISbP/6IxT/+JrzprT8W075MXJSbfz6Qu/PxzvbezfOcbndeXl6pIMxs++3hfv1ERsl5/XCuvsMgezvm5T25x15WbbdWF5vFC++19Qj5n9jDjfGtPR4ZnyMmwi4AQIJgWuCrHnZGZJTSp3zc2023+9t8J4U+EYSy55t6jckTGeXCoGbqTi8Z7K3MlHYePsO2W60LXZ1TKHSunsQTJJ4g8QSJJ0g8QeIJEk+QeO5QpZN4OfKsStRddfLDTSkNHfEJADUJV1rGQ92qVatWTaFaEenZIQ3A8aGezWfcArC6u7r1bACFY7w6ZwA3Jvl4dv6ExMuOBfE/TZpwCiURTX/4cudsYMWsGcczOgF47eljIyf9jgUnt55a0UIBHSEogwTXRo0aNXpDwExBEEKfEz5seksQtrjkC36v3L0o+qog3MJWISpRGR2imN/4ISsANAP6AeiXjVOBjYABpjOqK/fus+sFNNJew3PxR5+K7S///lCM+Eadanzb8/5/uQNQmRD529c7hs36l+z7Q3G/8f8D8L/H8NhPJcA+l65NArIsPuCT8PnSDynjZUN5PgC3Fviyz5P/PfQxJiUnpBTMmeyLlJktI2/um3P/cwuDu5dvfkz+/aEY8dtaA+h6Gq+te7712m7w+ubFvl7xi4EE45I/txj74HMe8897Dlwn//5Q2j13qk2xympwbdC5ehJPKAm6vZoyniDxBIknSDwhI/4fVfrR8b+kkkUAAAAASUVORK5CYII=) |![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAcMAAAJtCAAAAABvE0QBAAAAIGNIUk0AAHomAACAhAAA+gAAAIDoAAB1MAAA6mAAADqYAAAXcJy6UTwAAAACYktHRAD/h4/MvwAAABBjYU52AAAPAAAABDgAAAmJAAABQRqZkoYAADKWSURBVHja7Z13XBRHG8d/x9GOKgcoKqBBxIIdRdEk9i7WxB5NMzFqNFFjSV5bmokaTTTFaBITOxpFA/ZYEyug2PUEiYpSlKMIHEe5ef+gHcidu3sst7s3v08+hr2bZ2d2vjezu1OeR3YJVCKXjNA6ELusaBVQhlSUIRVlSEUZUoZUlCEVZUhFGVKGVJQhFWVIRRlShlSUIRVlSFUlw1Oh9WR7Sg9OtLPz/53WkNgY5rT+oezvhIHdYz94+xCtIsHrmbUYsvChxX/M3XcNGJ1x0LBtdF6Ap4xWodllbfCbs70A9P2g+ECrBUi+R0ViS3ejVkBAQEBAYydakYJkmFwHQJ0sjQIAli4BgEyXCik8G9zPuHABAOoFBAQEBPjZVD6H7qEqTqN37B4Q4Ga5dX3mAluLJv1NY1hB82cCWT6VPlwLTbxKpVKpHj96dAKA3C8gICAgoJ4VgDSVSqVS3dE8cyaPgICAgAB/heUhTO2hZWsyxkSGXikAUlyKK9vOrso0ihYtACD9jkp1+44q586dfQAcGjdMUalLTt+osV7bJSmqxCdPzgCAb8DgaRZ2L/1d6/siS5POjFIZZhiyH8CRECYncQsOBkAeqVQqlepu7uXLALwDAgKaBDSsfP6cOyqVSnU74/79vz3GWBRC8gsWvsXLmSs+l2bHoe3K7krf+Q83IqHF1DePTd/Xt+zLLNdK98OqVfCfKqFOgL+j0etJU21a63XL1ZIYnujulMTTsx/R13EAwEQysSsh5HgbW78Nel9mIpNUn/IC8D6xJI3BOzydmcVafYbtkKmO9rKKamc5zTCtXn50ED+nNt94ac8xuslFlsNwU35bnhCac8z7G5eo9RaDkKzDJL7ObUaGdT/H/BRLYXjmpsNYCTLElLYZcyyF4XqM4u0p3HzPNAAudCInuloEwox6mjMhfJ3crHPAwe/ivXyLYLhF06ITbyc37zz+l543V3Ewezp6tbgQkvWYxN/IonkZuq3Ap/fYm60Im3FSVAyjL9uN5+/sZl5P89rLuTNYGz1ZBbyRLSaG6/CqUrIMZT9a741ga7T8aSvfhLkiQvh0G38vh+ZniMBZeD+HnUnSGnz5G348Kh6G23OavCRhhljge+8LdhZLNZ0G9JyCN7NEw3Adn080qDRvYVTVO29Rpj2wucEm/T1bHCXk6QuYJJYZi0uwSeXz/OZfIzx4UMEUNo6OPsvv3gNw2oD1Ylk3uR7DPHnNwOztkCQosIl5apUcpwkhhExH/XRRNMNsFxzhNQMBrNVvuAAz/9IxTb2kaEDxKpOl/g9niqIZ7szy68FvDuZvh0QbCDT+MZtR2qsyxJT8+a8MEWJoh53xJb8ZCIEhSZnjCig/fsgg6XCMKPt7FuqmCR/hNcgf8ZuDWectypW94du7sBkzs/Vz0sW0l11rXnqgaXdr/Kaa7RcL4/U6fWtfOwYmH3w3bDfPpRJCOySEkMJdXQD03FdkNFV/jNc7OmeF8BptUwcDKtSdlV+/6d8f/s9okTVu2M9zqQTDkBBybpQcaPpzruEU/0Iep388D7Uf1xzBhKGAvXu5Spdf2rUYPu+3uwaMNsO30IIYEvLfLBfAY2Gyga91XSu92OcFYlRNEcxdYg/5hxn6xUk+tf6jIc1Kdpk0n3OqoAqzrljCd8mExZCQzJUNANs3r1b55RHY3q/4SbQcO1jm8GjJ7qfsy6Xb+wLQ/VpVXxXEH/hu6otyAMpx29SVvrwNqweWxpCQgh0dAfQ5qHu2HjtieuXPFsAjhdXpbzYAbPuuSWBXKFV/oP52nZEU6m3j3ADIuy6/qZ/sIwzkvcYE8lxaUWdX7tYh8MNx9hU/jhjsEO9VKWl+hytt+wQEMN7MeiZU3UB+F0DgoEGdGG76Qs4X3+TbzPrkeSvtC89GRt4A4Odb/ll09t7BvNdXRaTfN7ALPl/y96oAe+8PNDXeDgkhhNyd4QTUXlJhqLioNeY+m/KSLQDAtcP4T7dffG4nGW6Pjo91N5a9bKDrq7Ib3eEN9L3NrODxq/vYVqjfBgU13A7DJqzt+O3O27UBYOubv3VWvT56pRnaIQBk/PJdIuz6DBpYv/STnSNdEqqYDb99SKVS3S+5jKbTXje2WWftVN2gMAcAUB+KPJAOWDHYxaLLRsNvBzOfPHr6z1O9oxBf/uuqAtHgqYQU1VtKCCFkag9CyMwu5mmHhBCSv609ALRbeKGIEEIKm2KxwbS5V//88vXOHgDc5hsc7tF9AkwqbxYFpz5qxqiK7BblEiGrQjvMd/hzKDAxYy8AbJ1yOPjuwNc+BlC8Hz/LpybbIQCQK5GR5wlQZ+Cg3k4bJyoTnpd/2rZv42Ez+sO2VX1Z8M7vWLKgYnt6xGQNQW2h77HTB/oQZwghHwUXH31nY43JJd8sAoAabofFSvn9FWcAtn288TWD5IXhLwHoHvHs2MnTfpCvF3R74irDDI/XWX9lt8+nxd/kZWZmZqTqzFNG7ZEZfgDq5DBLfmGMHAj4qVLq5CAoRDHNYSJDrTycEDJhMCGEkBdnE0I2KYoEUUzdzeXDDjBOff8jV0A5a0tU+aiKyg8e56SJkFR4RbINOjoUuqPTAAC5VgDkEEY8KFnTpixS+yxbsOHbhG8A1AkICGgS4Gd3YeCTFyoNWEtIFYhut/v9xju1kslr8whZ5Lzt7uFGI8X64yzc9fbLpQMCVn4KtEuWaCus1A4x6vHC5DYH6+C+FfA/2f8eeoZ+Idbfpnz4cCDrzm2VSqV6ehd9dzpLtRVaROw8kqLK72oj3eszZU2UdrFWFMYyr5d72VRPzoK8ZFPaoUmDb+I0FmSpqR9h8YsypAypzC/5YlOMu1lbmrEQS03jcotftC+VNMMKgRKoRMlQP1AClYBl5Cbbvz/z02RcvMnWB2JgT0ur6/Pn2VoE9DORob60Whhy6p11MTomOo7DJU1dJeExzCqU2V3D1mRMdTJcugTAosWV8V2KiY5RAQAatmbpKV+z94erO2tbEsMjGk+2XQ9DF2/GZ/nDS/7Iy8zMzMyr9O3A4vVFDYZ/eegJ+1mvv5zhE81twuxu4Ag1EZ1ex8yamD80qCq70VrEp31Q+yAPbj/L0AtDVC+ue42DZeG469dj9waKrBnq9mMAT6dm1g6r1H8ppv1+MgYBH3JYBr0YLg3huEtkzfACnLX8nNnIu0V2bCwSYu8bTtDAxNuZ694FWNX3CVuzM59ibVSPnBELdRCT9qG3bY23w9JACXxqlyMaxrIzyXwB4wgp+BAYlCGmdtgev/J0Zpj5yq42gmI7K4uJaJBBCCEb7dHkpngQJgFJEmVI1H2BuSx2O2+H1T/Ff0X7wPkv0TDcgCC+Tm12hqRwLtCH8V7Ze7WwoPTvlJeBT1mvYT4/a5PGDJf5ChZKlyEh2xRwXMbsma2wKzrmlx3lTwWGZ7HK6/EkGaD8sMY74XwXnJcyQ3LtRaDZMSYpl8KpgmOMX23R/DLzbSBFPyuBUB8AXbdWHrEouJnI436SY/DkbdeDIBgS3e+ewJjn+4mKssaGip+cqwcoe8/7M4FJ/UcHAy1PkcLIUCvAY7aqrHVf++P9zg5AnYEL/3rIzxXOwgTeak8YDAlJn2oF55X5xhNlN8arlVk9GlA8cu7e5+Nd/xkFmfaeDM7fFo8p3F9UH0CP7drC6xund3EAADjKAQB1By2OqH7nXE0RxlvdCWctxsWp59DyB6M+k99ZX//Ks7u5tddiomOuFgCARyO9MQuHNu2Dyo91f8x5gnHL65YN2O1ft59Aqc0BAMd2Qe2DArSXo2Oib+gAoJ6v3l7TZitN3kR6t5H8SS3eqk4g7ZAQUvSLO/Caka0t4ZAZvGnmRf30dptnB39du8/efkdHCLkUAjQ/XtHm3oK6gEOXGRtv6L3bZJ9ePSGw0uhVR5M3z65GV/4qTlBrotQfryMuS4Y2qNp/waNWaXO+Nmqfd/WR3lHaxejLeQBQq137pz/rnBZPf2a+svCssqm8ihPlxOqNAGZPV3c6ZOIu9n6Hln3EW7UJbF1b1JRowD0oqH2Qb2WQun5H2p1lOeJYcCMmOuayFgBGfVOfa6Fie6SHHDQJYo679npzS2GIol/WXym+tQW1D2rvrQdy1UzFxaZcTllwPTrm8eReJhTqUs/0zgdN2fsWMbjhXf486wtwfan2WnR0zNVCAFA6lH+cVLT2XXMV6WLPjC4HTIA4+eep3/NXOqGuEc67Gh0Tfa3COqthu2RmK05Mr4wXD3COqk18E/f3569wgl7nrblTWH4gb27OJVTRvTJf2s8V4pXWijQFf2WzhoClaCWYorQ/0vufgfs4QtyHHjwipGv1marDYZdTg3K42e7DQD6LRvfMMNb5PlndIh05GKbV1t3j0/MebYeM1fGQ84nQXA6Gh3QteHWeSBkyV6dDzsdDn7C3289vV0oZslHIQadjTdaxXU5XdJAyFJA6n2ilfrdTNDuj82luIbyWijJkpaCYb52jgt9Ts7HZh778vsFRhuxkPeP2OLK2yW8sOlSe3yzouwUXnZh6AyE/tmGYOtFHlsJvCEvaDtmrW+wKp7NB0zOYpd6PjvwipAy5yGbWrVG6NU03Mtr6zHtXSvtSrvp72m00ePutus9Lp1XmXmxLGQpT+SuXpUM++N3exruyw33rJfI8Z0b7Uq6ynfdwY5ei8H7+S1OMJduHAXxPe9J2aJKurduYCeuh7/Yw1BZI4/jwoZShsJW74+dzQKO3Gup95teqbLrwdlPbNCeei0AZmq4rP2/OqviJvEVQ+/Yt7QGsnNX7MN/5U4bVoZyw3Xq+ZwpvPgYA65ZB7YNmn/x2BmUoQpEHMTHRMSWzVKrGlKFoQd6PiY6JVoec4T0nypBfkI887HjPxBJiI1SjMVtbWX07/ktNYyOIv9R0nEb8ogwpQyrzi8ZGEH+p6buF+EX7UsqQijKkMl1C3n9YoFbnaHJzNbma3FxNbn6H4e6UV1US2jMNeXTrSdqTtLS0J2lplSblIO81aqgbRSZkhoW3Y2Nj9R3DQObq5OCgcFA4OCgcHIr2xwI2fUYNdhVMiSlDPWVfib0Uey0PAOT+ddw93N3dPdzd3d0q+v9R7dhxFbDtPzJUujG2RctwzgoCAE6t27RpE2h05/qNHWG3APsBH75I0QmKYbaHtm7btm3a+DF5SCbXdoTdgfzbqTLzF5wyLNOOUf4qNkTI5WXb8M4aW/OXXBgSwvvhbgxn1ahkbbYsl63r80QARaftsFh5ntnnOrI1ihz79IUIsYULkm47/Du7fgfWRoPO+iV0iqD8BMJwN4ZxKEXghW7ZQ76msy6CYFi4F8O52LkfnkzmTcijCAXA8JTa4yVOhjY//SDf3C2JpRW5fyJDYgzNP+a9G0O4FmJKk1fPd9gbxDR5wc3Y2NjYdNj2HTnYxezXXX0y+3OpzufRPu6xHeNCbynmeJYNqipsDbykZF+JjY29ng8A1nUeAnYDRg5ygkRkdobnQpwfm7AUOnPMAeaJXdu0adOmmd31sDAVoBg4aoADZVgd+mjFmK2m2Bd9H1M+xWg4mW+bNm3blDnsJ1d2hMUDjqEjXyhPQvKLz5Kryc3V6Dm/hby0lTsoHFzqu3Id5MtLzDbwjZW9g4PCvsJ5dXma3FyNcyMxMCT+d3e+Ul0nK9IYomj7TMdJLu7Y8R+XTJx8fHy8fXx8fBj2xQUPHxTreeNKxb8UuzxNriY3t/h5e/wmMTC83Mb+sbluTCRqR+RTfdAOJVOVCgeFvt/pQk15A81IL/u4Vl39ZzH7MlsHRUFu+fKDlOSyGlbUMtCEi/JyC6r43Skchv0gBoaLPh2yB2JSTuKDBw8SHzx4kMXUws7b28fHx8fHx81IL1z6O9Hklf0Y5EwzMDfDltc2vgZRKutBip7TNpJX3vRybcubpMLTx1PifjFUTaxT6RIZE2XmcZpw9KAIRc5wN7exUip9mbcvfeAre+RFIYi6He5BF4pQ5AxpV1odMmtf+thLl9CQMhB1O9yrC6IIRc6QdqXVInP2pZmeBTebCqcqaDvkoH0FzShCkTOkXWn1yIx9aa5nbkw7SkDU7fBwboO2wqkIypCL2G6zoBIcw/wIejsUO8MTGV6daf2Lm+EuDKWOVXhn+END+44X+Mq4aA/tSvlnGDZz0cXWfVN5yvhMaq1utPqrRUa2Oqyc9AbW7vttnsEE6fkmZLwNg21o9fPMMD9mPmDV6ywAQKsFYFdpTf34/SZlTbvSapLhvvRJUR0AdZIBAEtdXV1dl1ZKITNJ7frS2ue7HVbQ/JkAKm9tiaTVJ/B26CFPAZBSvN7FzsXFxaUyQxobQRjGRsa8Owavgc53muFnGhobQRjGRvrSmRPbB3+b84bkuh7JyQjDUY8XJrc5WIfWkYgZYto0Wj9iEI2NIP5S09gI4peR8dJTofVke2gNiZphTusfLKcexCwjHXT//uV/a7UAyfegaydExlBfS5cAQCZP3pXi/9G7K+s0xcrTaDT63tisOo0Job+hqmT8mUYWPrS8HWb58MMw+7NVBcxSNhgzpiXFyJUhYOJQkUGR7bMfoaOHXp4KhUKhsFcoFAqFnd7tOnvvnmyg+ZgxjUBVqQ6NCeF6B5nIJNWuyy8DfhHM0ubsGGoLIHjVI0KlT8msDNOny6H4TMPC4LfeVoBVj78puXIZ6Uuz49B2ZXelL299qe73eY/xyje+7KySd247C5uDPWgXyqAvPQ4AmMhbO4wKBpoe4WJ5dyicY7lme7rtiNWXiyynL62g6mX4eJIMziu03Iw1XVE3gZtpbiMAcBu8IqqAMjRNhc2B17g/m6S3QJMnnCz/h3qf93UCAOd+X57WEgmIxZh3td4Pj/aqFWFKzKbEzg86HeXgQvZm64Jdw1F46eSpfzIA2Ncuf5FR6I93yBXlqiX0RZRmaoevY7JpJ7juhlD2vWHRSxikK+kJYr8b4cmoipzeOq2j7fAZaeo8/cfE0Gmne+W9vY7tqM1vbzncaKD3A45P1xvZK9JLWFD6oUajBdD0zddMd4Z0Nar8pBoolW5KpVKpdFO6yrXqdLVarU5Xq9VZtmXt317h30XA7TAMDU1+NAy3wiKWJqlKLGd/6z71ugMgH7wn36TiFiw0NEckszcEZ5SQ2+HgiE8+N/kka9/Dz++wspi4sXUUl5tb1o7fzgJ1JrzRjHNhH4z7By/VLmthCpJe3PbU6hwAMjdlSbt0ydeUt9Zu84TbDh9b40Y1nGYBrPawSX8MsnNc87oxuzaAkLXx3G6Ne5Rw3lL1V9rkeLVJnZJ5GP6IdtVxGt1bsD/NPHleAKaYkFv+nsFyAN5jf77JlqNmGhB0h/Ak8zDsjJXVcp6CgVAyb9CL4ZVhWn5JX3e2AYDar6xhM9RzqzUwi79XUbPcD+82skqsWy1nyul53vdsPWZpb7fKDxtpco65506eOpcHwO2lLvWUSqWb2/Om0ckfU3M9NvYHbzILw88W9j5cTeV/0kXlM+MNJZMbf8/j/fdVzwyyNurkqdM5pUcuSqVS6d64ZcvaVZ49672t6LGpHviTORiSpqo/JlTXBSR0fQD7sVOf76xo40TF9Reqr+IKLp2MSVOr1XohEjxatmzRMrBS3Pfo0fHyT+fKwaPMwTC6gyKl+iLc52z7/jLQadorxqNGpTV98tVcHiqwMEOtVqvVqTeuxRVXZcOWgbXKv368usB3G9/+P8zwTDMDY6r1nq77d4wN4PnxfWOJ3kSLfMKrcqI2zOz97G1+uJrwLDO0w8L6qZEDq/mXmPzL2oewGjyth6Eb3qmuOF0j7nDSrl29odE77jGe91VcZmB4sL/Ho+qfCSjc+8NxoF5T/0b+/o0cK3+b3+bmu2shUZkhDukWjOZhMsd6xIgbP/7x6NExAPDyb+Tv39BdWav0WWL5zdpLpYrQDO0wu07u2U48XU32lbj4uLg4ddkHtYqHIZ0XabeOoQyrjeGW8Y3u8HyLSI+Pi4+Le6DWC4zX+5B0FxfXfF+6Gbzf5d3atwcAFJTNDZCvJbw+vMbbYUr9IlVj6danOVTjrgvDioIpQpEz3IzxtNbFzVAVJR9Fa13cDLegb21a66JmSDZjHK10cTM8d9dxCK10cTPcjOGOtNJFzbAgjHalYmd4KK1OT1rn4ma4GWOsBVoREmVY7bERsvbSF/yaZVj9sRHC85rQQGvVL1NiI8Q8YZnZOv6nLChDfT0TGyGrcoqF7GMjjKU1XpMMS2Ij3AJQ6uurkhqxDl840I/WeE0yrKD5MwGSX2lR6GpafYIQi9gIrp6VbmY0NoIwjGlsBPGXmsZGEL9obARJM6SxEUQiGhtB/KU28kxzanlMkr4PWiqBivrVF78Y+tWnEidDfdHYCOJnyG9sBFOU8XuCgW+KNOW+2JxWSnjiUgCxEUzRve/WZzNLuWKWpbdDOzshFj7mmx1FCAw18FxmVe6B1H7NqtnJX1sJ8RpqjqEApTu44jjQc3ZfJjfpb7zmrkj9Rei+ZKufYXYckBCr9BVkubVbvrkB+ehZDKcwZXPqvLXxyQ6Jrm017DKDb7/6JijtCy/AedZ9NjaRCnR8IpQLEL+/NlP1qF0KvD9425Wd1dmB6U0PCbNXMU2ivM9vTGmw6e4slggR8q/3rc7XKUNhaDv+N57D80nzM80fvniaMhSCbl+2Hs7J0Oefzhm9IihDASgMfZTcLJVHBuUN+40yNLvIdnDeDu4Q/kbRW92+uqSTEkMRPpdebWWb6srZmnyyFECdPn37eNZwucmXUXrhOW2ZDE14tZYow/99MTTcFPv4g4eO5QCydv36dip+MtJlqdVqdXq2XlQC/XglDb4Kqo5yr5nO2mTMVmkyJI3jt4028RzaM4cOXgbg0t1RrVarM57TtVpN/9R0n7lnXy6c6p9XNpOSz8Sm35fSZBjTXpHqVA3nSTp86HBa6YGDUql0c9KL06U3Blm0dht8vh9sYnap7R6+GsbP/Kv4GH60YmRYNZ2q6OJJKzelUql0szea7uCUBAxfXd+UrAr7HG96wZmnKjGDL2jTHD/7YleNZ5ozVw7n7wtNOMN8OF7nq3SiY3gGzrlmyPZyR6Aj5/i1ZA+wnbeyie79cDuGKMyQbavTP7icD5qby806biJm8OjiTGTtsLAuIsyU9cMRQMMDnLriVujMY9RasbXDf5Nq9TFT1vX+/Mvnv/6hN9k3kylXau+w5a9gYmO4HcNtzZZ56I2Z1pEtp7B1M7H+D6vt9fksl7j60gJPHDJrAW4NAZyXatiYXLDFV7yWSWQMD8PD3GHtj7cDfLcwj5z3xBdD+A0JLbK+dDteMfdKvG5RG73vj+v0D8PkRePu+//B7/p4cTHM3w3zeyG2eu32505RLw+/wyj1Z4cUu1z5LZC4xtoiQ+s+kEMASlm0XmczKVBRPplkoDEcCSUbX+O7NKK6H47HdLPfkkt0jemusMm8F0VU7VBT52nNBE9jpCNbs0pnGzWGJ6/67eF9m4OoGO4e4ZsgyBs4KcgzUI+yGqgyUe23CMNIYT6DyWxtzZi7mJ5LcyIF8FQqQImJYURuoyARFZcyrEJhGEU3k4ubYeZ+jBZPaSnDqrQ3v1kLCkzcDGlXKnqGaYfpUyl7htUeG8EkhRe2bkp5sWRY/bERTFIYbYYGZMyPcIfvofN5v8yP8DNjbTvv1mBBixbo4qlH9yplSmyE3/fXaFE7UIRsGT4/NkKvGo0oav0epcWWYQVVGRvhQ1p9ghCNjSD+UtPYCOIvNY2NIH7R2AiSZkhjI4hENDaC+EvNYk0UlUAlVd+6lCGVmGQJoc5JrppI0W+p1BnqUhKT0tLS0tLUaWlpWiBspHQZSuyZpig5MTHxQWJi4sNC/Y+tdIFXpbuOQyoMi+7dUanuqO6X+VmzqlvPw91d6e7u7u7uXuiTfaAfZShYPY1R3VHdiS/xfyav5+Pt7e3j7e1V4S4xa2XPvylDoepR8EMAgJ1/QOOAAD+vqrcn3vcruthW5FcqWYZk2F7P9gEBjQO8je8tHbd13GbKUJjaOdLmIpOVwxeD5Ak+EmUo8nf8tGmYz2jxd7seRd/RdihITdzY/CKzfbYHBjg/cBX1tUq0HR7cKPuV4Vbpfs2frpNoOxQ1w6fvYkYnhmlls/FdvpgvVqIMP7nf8HPGicd6PQwT88VKk+Hp77GeeTQ8u+lYIc3JUhE/0+S1vfUGm5gxat+cw71pOxSUPr/l9Q2b9Mq3sIK2Q0EptkPhLnahu/5rpLvcyowlLnisd1CoydXkaor/Mbj6t+04RicWiu8stipohxFsbUZhghlLnN6INfUx1e3rS1haNrfWTS+WNtEdrBO8zVbi1/+Q6Y3pyh0UCgeFwkHhoFDYGZrbbDNGyu1QZY8N7K26Yo7ZSrwHVqfN7nNPSNL1ONn7EPuZ+chQlweMtjzkppau40hLU/tuqoa76JPA1Dlf81MZImX482SHay9wQB94a6WxHXfZJSs5EtUVPnbZ3dPUApORfwZG21OG5Ups/vTbGVwMf5nkE191JOHMiD23H2SWHdqXrONQurs7TT9l89t4E0u8fYz1Ob4clYmSIQnd1+lfTv6E8xqkbhn77McZf+08nA8ALj7exUs5XPQ6au3EMHw5z6Q1VUmB6YsX8VcfItRW2HINgPUZ2lZ2cq/eMMAGQLOFh24YcJRcNBuYbIo/f91AtMvnrTrEyPCxB5ZwtX2iwFH947Rf+1kDCFx0zajZahlCs7mX+FfYXuOvPsTIcBxacI+eNBX9CSGEZN04/OuSSd2sAbRccuO5ZrvtEZzCNc//nLGMx/oQ4f1w3yCrs8GcreMbk4/UiYkPSj21tH7l1SZM7M6EqhsdaMwpS13vY11O8hgOQHwMswITZ5kydv3KruL/1/L29vb2eTGAqd3t/gkeEZ245Pj9+w6X/XmsEfExnPKT31UHE+wTFjh6+3h712cd2DVlULT99iHsM7zTWvPDFF6rRGw3w5PAMTNlnT0QsjWsQzcVhqBXEa/lEhvD3MaYZLbMCyYBLX/VsDP6Gi73+C2W2BjOQ9108+Wu+9oRqL0omYXJVVsug/NSZhgjxx6zFkC9zAewfeMK0/TatgjV8VwmcT3TFATHjjT34rSC3avOAz0/7P+cZSxFGo0mV7NplfK6F88lEhfDpR8rbwjA6dHZVbt0aDJjgkNW6QSVOl0v4hPR5mo0uaWLWfnfgSwqhrdba/kPRMdI99asz4JdYZHxVHYOCsWIr3kvjJgY6rr+22+/ULZkP93w3V1A4V4yQeVmXYmdg+GYiJbM8MepjtcbCKc4RfGOSoUQCiIihvcDs9dQD3JVSDyxEcjk7C5TBFMakTAUWGyErQdsf6FeraoSw9gIWi0Au0pb/ZbF1mRJD6Z/8THlVaUMjzDIwwkhEwYTQghZBACLKqUYULMlbZ1PqKoSw9gI82cCqLzj9p0+NYlQPsyGtjiW7fAhzhBCPgo2zD9vUR73H484jQVZasP3w3yHP4cCEzP2GsRP/eoLw9jwk55t0FFAdzSEdlVCF42NIGmGNDaCSGTMr37whwsmGd2uR/3qC8KY+tUXv4yMXp0KrSfbQ2tI1AxzWv9A60cMMtJB9+9Pq0fsDPWl1QIk30OCfs11eZpS6Qwk8ekq7OtmyLA4lqwJwwwCVGrUhaiL6Uz88HX8soeQL8T4c6ksfGh5O8zykQzDpzFRF6LulR7ZKOwVCoXC3sDvmZzORc8vOoq+HdrZCfcSWCpv05kLNwkAWbMOwe29DLMrU/KXa48eHfJZS5G3Q8DEIVvh6PMFAHw7BHcIYn4195b8oZONXewvzCsywjA7Dm1Xdlf6Soxhq6sTX+nAegTx1qIdsH5rQX1BXpLhGavjAICJZceZyJTApHcc5GmcDC8OAOxmpgrwkliMtUmjHX4zu8dRjqb/fvwPYKcokx2TJVrNXuV91sDiGL54mvsiVXL4kxjWRlY9xg6rRRlWo1LqkvsmxCohT3I1Go2meFwgj4FBwdHzgO2AsYN4XBFuaQzXv9M+qmZzjN++7TrgNHRsL77WdFkawwEHzLBK9eq2bf8B7q++pHcDtVIoiocWFDamjuRZGMMsz/wbzcyQLzm3LczQinkrwz5ohzMKq2IJ8YD1dCA/oKk58pWFhKw8Hpag90lRyUA7oMvJMWSWzejkFsYwHMPMNQdh3buqwAxEqzHik51Zv2dZDLX7MVRYJZLZ27uZeg7L2kl07GndYOldlWUxDMdQCV6wRTEs2iu0rpQyZKtzqa7dKENxKxyDbClDUYvskWRXalEMr8Xb9aMMxa1w9HGiDMUtiXallsTwv0tWoZSh2JvhS56UodgZDpXmhVkOw8f/UIZiV4SubUPKUNySbFdqOQyzD2MYZShuHdL6taAMxS0zrsKgDKtH+ZGS7UothuHJzNqdpHptlsIwHEPkUr02C2Go2yPdrtRSGEYlOQvaLQJvDIXlV98khWOAdDwKsGAoML/6poiES7grZepXHxD1npmbzW0fS8q3TgUZXqufHzMfsOp1FkCJf5rKKSafEstVZqKndBGCoV/9Yj9RlfTgpniuc7x0ETLdMzN/JkDyK4Wy/mqOaC7TtbWEGTKMUUJjI4gyNgI6Bq+BznfaPIMJaGwEYRhTv/riF/WrL2mGmEYjRopCxmIjPN+YxkYQgrGRZ5pTy2OS9H1fUglU1K+++EX96kuaob6k61ffchhK0a9+BekqxkYgBZpyNXhb2BFQLdOvfgm3qyfPqEs4GQ2RELKtgQTaoYT86herKPbkyX/SK1aFfl1YO5S5C7bZdLbNhqHiZygpFV48efKfLABOL77sXe7a2eC6t5ljzg+bttxesNdjeX71704/mQ3A5aWu3doy/AUXfLIcbcMaC/aSLM2vPpkEuA1eGVPIymi/B5y2CPWKLM4ne1Hdx1tGs1+R+XDcSby52lFsfak0GZ7s5pbC5VWh6LNPSfMwU7ZOFS47q3dkoyyVo6lv3Rb3TLMbQzi97ckXdx13o8PqtzlXeM6YiKq/sFG6GOoXBn5DGT4r3W4M52jaPXbiwXeOftWQm3XqoCj7JR7lx9r0NLVarVanFRSkpBgyasfo1JbWl57v5PSY81uC7puPC+UjP2rLwVTV/657REgVX5BctTrTkJUnM//xzB9/JPFcOgejTDE/3xtAr0M6tnZn3OF3m69rsiw/wiC7MMIU++DDF8fK/+7bdnMBK7PwHmkdzgbwd1UW1Q4vw/6pqef4b4Yj4LMyi7nFGhkGZfN3URbGcCGGVMNZ0r6oDdSa94hZ6qLZwOQCHi/Kwp5pWl77Y0J1nCdv0woV5C/4+zf2929o3Ddx3uthWDqXz5lXy2KoamKd6lY9p9L9tfxM8V9WDfz9/RvVVirdqnpRSx96yuY3fnd7WBbDr+b3OVR9Z3t4Jy4uLi6uPNKPa+nIS/lkyJOBN1129+T3qizrHd/Ep9JKql+/GwCSHBcXF3c3TZ2JzMyEKlId4Dugt0W1w/sNZEn8rVsvzFAXKy2ndBWHFgja4833ZVlUO9yNl3jcemDt4VH5I11enhv/68gsimH1dqUMZOXgUBO5WBDC5NPS9KxgUX71SbCPFK/Lkhhyn3aiDAUi9XHKUOyKKGwp3KVplCEj1fhTKWVY3Xp6WKJdqQUx3K9tLFGH3pbDcDeGS3TrncUw1OyT6u3QchgezvFpTxmKW9LtSi0nNsJfku1KLYbhiYzanSlDcWsXhtHYCOJW0R6pvuBbDsPTqbW6WyRDCcVG2I3BwvZPwhNDKcVG2C3dp1LTYiMUELFcZUxnx8cKyTI0JTbC0P3iuc6B0kVopC8tiY2QDABY6urqKua1KLbvShehSbERdhSK5jLthOshqBpkcEcUjY0gllLT2AjiLzWNjSB+0dgIkmZIYyOIRDQ2gvhLTWMjiF80NoL4RWMjSJqhvmhsBPEzlHxsBK4qiaGQW7oDP1ej0eTpBVmQjXqb9zJYcmwEDlq3rKj8QKfRaIqea7JttEDaoeRiI3CTblHysx/al4ZRcCj5115v9dWJdW+1DBQGQyoAwMVkp8PlT/IyhUKhUNgbfUgYeffv4VE8d19GGGbHAQmx5bERqBCBviHsLORbg1Sv7+L3WdDI+2F027aY2XYhJVeuSAxia+L5p234cn5LZXF+9U3RQ29Zcm3WVuvetTrSw0ztkKqy9qEje4SY9LpudCJlKBBFIpSDlezHNo9f0VKGgpDmb/a3QwBQ7HI7P5MyFISOaXy4ubH02yz7cRNlKARFIJTjS8KAhXj3MmVofpFIbl0pACzsrxmeThmaXbEPHTjvnbLa3PDuBB1laG5Fojf3lcbKXXaRX/JUMDpeylgRnN4sStXupzcX+rcy0JDsFQoF9711dJyGqZLr4lFdU07w7jqjX8tLJz4U5X/4MgoPRtshU+1DB5MQYnXWEQPfFOXlAUXZ2c98MTKMMqxOmfBUWiy7bYa/I3mV1gMU/9mE0YkpQ4bKO2zS7fA5kilM2B9Jn0sZ6kRu/TYCLRplyFARGCTUVX2UITMRk2+HlKG5dfW+oqdQy0YZMlMkegrWKwNlyEwRfD6VUoY1odTzGEgZilv7Sbv6lKG4JeCnUsqQmbSHBHw7pAwZ6VS2VzvKUNyKwCABVxRlyEBCHqShDJnpRoJdL8pQ3IpED0fKUNwS8iANZchIT84KeJCGMmSkA7rWgt5ISxk+X8J+KrWY2AimKP+gsG+HFhIbwST9m1W7g6ALaGRd28pJb2Dtvt8M+xG+nWkRDDdioLDvOKbERpgpotgIJknYt0MjDEtiI9wCUOrrq5JqN7AMhE0F7nzQlNgIGyykGQpdhnt6D3kKgBQvAICdi4uLq2elBZbaxSZ4ChCnsSBLTWMjiL/UNDaC+EVjI0iaIY2NIBLR2AjiL7VMNHEoqQyJzltQhlSUIZXpovvxTVdOYmJiUnauRlPyn35sBHT/oTbf+dNnGs4it+4lPkxMTEzMMJrMaxPf6xopQ646Pie69E8nH+96LgqHEudADvqxER5PuSGb85kNryWhDLnp2tz9sA/w9q7v7e3tbWwUNHfmzwje5sdvj0DFXg/etIL11GRmiXfWgvNWPktD2yF7ZX69Kg8jvgxgmv7euNN4fY0TbYeCkfZbd+DFM2xMChZaoXEMbyWiDNmpaLsf0GSPjqXZCW/YrNTxVCbal7J7lHkjGl5L3mT/Vp329h4MWK23mMVKoaiu8RXKkNWNp/VVpzkzOe2BIms/rLyYwk7hUPw2YjDsV+eplGF161I7u3jO/jGuvhHD1mTMViap6FgbG23EYO4uTlpG6x8V5ZUNzmnyDJkw819K2yELFdZPjRDgemE6b8FCh1M9+wqwWJQhC23EWBsBFov2pcyV4aWNEaKbGtoOmetPbWBbIZaLMmSujZggSHfQtC9lrLuNZA8E6TyRtkPG2oyewvR/SRkyFdmECcIsGe1LmepsZ8dkJ0GWjLZDptqIEcJESNshU2m9Mv4WaHQE2g4ZKjLDu5tAi0YZMtRGjJcLtGi0L2Wmx/UKrzen7VDUCisMEipCypChNgr15RC0L2Wom82tH9YWauFoO2SkTegvWISUISPpBDvOBtC+lJmO9ayVZE/boai1ESOFi5C2QybKqZPzbxfhFo+2Qwbak+PXWcDFowwZSKiLMEpE+9Ln66GvLt5PwOWj7fD52qp7UcgIaWyE54v8IeSXQ9DYCAwUe93uVUEX0Jgf4Q7fQ+fzvmE/wgcfWATDyL9e3SHoArKIjQA7u4op1lhKbARhd6WsYiMsWlwxRSchbiDhQU2kExsBlZohFlhIMxS6WMRGcKnMkMZGEIYxjY0g/lLT2AjiF42NIGmGNDaCSERjI4i/1HTeQvyi8xaUIRVlSEUZUpnG0IQ54sUymUzWlIPhqdB6sj0AyMK6il53uNm+LpPJZP1YZry0g3PtobcB5E11dxqRws22m0wmk01ml/FPrVxcQg4Yy9cEhibNEQcmJSX9y8Eup/UPAIBlq9eed+ybx8kW/ZKSkraxzPjk1HNHCvrkAB9G7Dz5aDg3W0xKSkpaxi5j769ionsMuW4sX+4uiIOnElJUbykn20WtTXBfHU6Izms5IRl227jYkolDOOacipMkw2YnITdxlost6TqDY85uvxjJl3s7zI/pVT5HzFp36vmNu8+9D0hI7gW4duSY+YnaTd5L42CXCSViCnoBTX3PcrEFtni0mJ/LOt+i7TkhRvLlPnBQYY6YrTr+3iRpyUvXnLlmnow6AOokczLuN/yF+I/7n2W9NVv3QZcWSLatxSVn3QddWgBjG9S7Mvf2bnamV0PynMKbxxrO10x+hPsDrTo22PGWWTIfDbRs1egEay8XU6/9yznPYtt3gJZ1e8Y3YmXaJDbzz4knjSTg3pdWmCPmoloBcZxtvWBi5n4erDOfFnncG/DKz+CQc7FtcRcEljnb+gctbf2dkXy5M7QNOgrojoZwPkF2fF3Oti94HQWyznPPPDGNZeZkWvixFwAE2RwFbt8P4WILAIgFh8vWaY3ly/35cLvd7zfeqZXMyXbWiYTTvTxS2Rs+vXQJKy/dI1/V2ntlyAsaLrZPZ59N+Ltd4zx2Gb/neiIpKSmXkMm+x6JDQjjZxn0anbDX72V2Gc87mXBlnuywkXxNiTOzxtc2+Bw301F1beuPiuNgeBwAMJHoFtSx63mbk21uH0+bBpPY/vaKf/EbCNFMcXMYlsTJ9v7LSjv/jzLZZfxmA1vPnoeN5UvnnsQvOl5KGVJRhlSUIRVlSBlSUYZUlCGVBTOU7aEMBa7XZVyWzIhVEo1D2m8Dnt3zKlVJtC+18/Ly8nKD7Kf+Cr8/AVztoXB/JxvAb4F2dacBeDLMofFfQPo4T0XjDZShkLVgxOVxo28ip69b1M6/pwE/TX3n6l/+AJaMvDJgnBoLbhy4+ZOH2K+SSFET5Y6Ojo5fEEwmhHR8j6xzyyZkn1UyqfdJyWzQ/wjJxgES+oYErlai98PuPwFQAiEAQmJxs7Uj0EV3W/aodBFNK8DRJRXvjbjYZ6iQfSIykUQZOvpX+bGi7C8bADId+t/bf6Tn1BXivliJ3w/PATjXDM0u5wCnrZo4NzxaKYHnxM3frqPtUIjSJgOw9sDO9i9uufArxi2auPjx+6/VweLJtfs/Pf1+WbqFQYHayGaUoRB1sC6AJrewZPuUutuaw+HQjA4OI1YCE/NWzfZ4pTyd7fz/FC9tF/nFSns9jSx8qJQvr0R0zJsypDK/6NpE2g6pKEMqypCKMqQMqShDKsqQijKkDKkoQyrKkIoypAypKEMqypCKMqQMqYSj/wNAP3OI1dlP3wAAAABJRU5ErkJggg==) |" + ], + "metadata": { + "id": "5RolXP_mTxwp" + } + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "T4", + "provenance": [] + }, + "gpuClass": "standard", + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.3" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file From 950fa9eb836f89568f77b0283d6848741916b9fb Mon Sep 17 00:00:00 2001 From: Steffen Schneider Date: Wed, 24 May 2023 03:31:26 +0200 Subject: [PATCH 2/2] Generate a better result plot --- week11/week11.ipynb | 4246 ++++++++++++++++++++----------------------- 1 file changed, 1935 insertions(+), 2311 deletions(-) diff --git a/week11/week11.ipynb b/week11/week11.ipynb index aa4630d..5b175ec 100644 --- a/week11/week11.ipynb +++ b/week11/week11.ipynb @@ -24,7 +24,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 1, "metadata": { "id": "MN_GMVuyPD9D" }, @@ -76,7 +76,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 2, "metadata": { "id": "Bu5GtxDyPMW5" }, @@ -114,7 +114,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 3, "metadata": { "id": "FljvS0QcPPTg" }, @@ -173,7 +173,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 4, "metadata": { "id": "QXPgWNCRPRd4" }, @@ -222,7 +222,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 5, "metadata": { "id": "UsdJlkLkRRbp" }, @@ -279,7 +279,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 6, "metadata": { "id": "EDwtthksNUpw" }, @@ -325,7 +325,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 7, "metadata": { "id": "u2xvlBoZPTyx" }, @@ -418,8 +418,8 @@ "config = argparse.Namespace(\n", " batch_size=64, \n", " test_batch_size=1000, \n", - " epochs=5, \n", - " lr=0.001,\n", + " epochs=3, \n", + " lr=0.01,\n", " gamma=0.7,\n", " no_cuda=False,\n", " dry_run=False,\n", @@ -436,619 +436,15 @@ " train_model(config, model, phase = \"baseline\", history = history)\n", " return history\n", "\n", - "history_regular_mnist = train_regular_mnist()\n", - "history_regular_mnist" + "# Uncomment to train\n", + "#history_regular_mnist = train_regular_mnist()\n", + "#history_regular_mnist" ], "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "SK-GNifLPN2S", - "outputId": "9cd9a6a2-9b54-46c8-a072-fcd66b995364" + "id": "SK-GNifLPN2S" }, - "execution_count": 14, - "outputs": [ - { - "output_type": "stream", - "name": "stdout", - "text": [ - "Train Epoch: 1 [0/60000 (0%)]\tLoss: 2.295775\n", - "Train Epoch: 1 [640/60000 (1%)]\tLoss: 2.290210\n", - "Train Epoch: 1 [1280/60000 (2%)]\tLoss: 2.310344\n", - "Train Epoch: 1 [1920/60000 (3%)]\tLoss: 2.292868\n", - "Train Epoch: 1 [2560/60000 (4%)]\tLoss: 2.290216\n", - "Train Epoch: 1 [3200/60000 (5%)]\tLoss: 2.276424\n", - "Train Epoch: 1 [3840/60000 (6%)]\tLoss: 2.281419\n", - "Train Epoch: 1 [4480/60000 (7%)]\tLoss: 2.276124\n", - "Train Epoch: 1 [5120/60000 (9%)]\tLoss: 2.267236\n", - "Train Epoch: 1 [5760/60000 (10%)]\tLoss: 2.294219\n", - "Train Epoch: 1 [6400/60000 (11%)]\tLoss: 2.254271\n", - "Train Epoch: 1 [7040/60000 (12%)]\tLoss: 2.264095\n", - "Train Epoch: 1 [7680/60000 (13%)]\tLoss: 2.256358\n", - "Train Epoch: 1 [8320/60000 (14%)]\tLoss: 2.274228\n", - "Train Epoch: 1 [8960/60000 (15%)]\tLoss: 2.269273\n", - "Train Epoch: 1 [9600/60000 (16%)]\tLoss: 2.240185\n", - "Train Epoch: 1 [10240/60000 (17%)]\tLoss: 2.248815\n", - "Train Epoch: 1 [10880/60000 (18%)]\tLoss: 2.223281\n", - "Train Epoch: 1 [11520/60000 (19%)]\tLoss: 2.222255\n", - "Train Epoch: 1 [12160/60000 (20%)]\tLoss: 2.233015\n", - "Train Epoch: 1 [12800/60000 (21%)]\tLoss: 2.211506\n", - "Train Epoch: 1 [13440/60000 (22%)]\tLoss: 2.234103\n", - "Train Epoch: 1 [14080/60000 (23%)]\tLoss: 2.225921\n", - "Train Epoch: 1 [14720/60000 (25%)]\tLoss: 2.202645\n", - "Train Epoch: 1 [15360/60000 (26%)]\tLoss: 2.196408\n", - "Train Epoch: 1 [16000/60000 (27%)]\tLoss: 2.202403\n", - "Train Epoch: 1 [16640/60000 (28%)]\tLoss: 2.209469\n", - "Train Epoch: 1 [17280/60000 (29%)]\tLoss: 2.180980\n", - "Train Epoch: 1 [17920/60000 (30%)]\tLoss: 2.153812\n", - "Train Epoch: 1 [18560/60000 (31%)]\tLoss: 2.150105\n", - "Train Epoch: 1 [19200/60000 (32%)]\tLoss: 2.158034\n", - "Train Epoch: 1 [19840/60000 (33%)]\tLoss: 2.172502\n", - "Train Epoch: 1 [20480/60000 (34%)]\tLoss: 2.093013\n", - "Train Epoch: 1 [21120/60000 (35%)]\tLoss: 2.130392\n", - "Train Epoch: 1 [21760/60000 (36%)]\tLoss: 2.172241\n", - "Train Epoch: 1 [22400/60000 (37%)]\tLoss: 2.162427\n", - "Train Epoch: 1 [23040/60000 (38%)]\tLoss: 2.168904\n", - "Train Epoch: 1 [23680/60000 (39%)]\tLoss: 2.179659\n", - "Train Epoch: 1 [24320/60000 (41%)]\tLoss: 2.163229\n", - "Train Epoch: 1 [24960/60000 (42%)]\tLoss: 2.126366\n", - "Train Epoch: 1 [25600/60000 (43%)]\tLoss: 2.138455\n", - "Train Epoch: 1 [26240/60000 (44%)]\tLoss: 2.113500\n", - "Train Epoch: 1 [26880/60000 (45%)]\tLoss: 2.067201\n", - "Train Epoch: 1 [27520/60000 (46%)]\tLoss: 2.114285\n", - "Train Epoch: 1 [28160/60000 (47%)]\tLoss: 2.036888\n", - "Train Epoch: 1 [28800/60000 (48%)]\tLoss: 2.068524\n", - "Train Epoch: 1 [29440/60000 (49%)]\tLoss: 2.097640\n", - "Train Epoch: 1 [30080/60000 (50%)]\tLoss: 2.028418\n", - "Train Epoch: 1 [30720/60000 (51%)]\tLoss: 2.031548\n", - "Train Epoch: 1 [31360/60000 (52%)]\tLoss: 2.051078\n", - "Train Epoch: 1 [32000/60000 (53%)]\tLoss: 2.028303\n", - "Train Epoch: 1 [32640/60000 (54%)]\tLoss: 1.983975\n", - "Train Epoch: 1 [33280/60000 (55%)]\tLoss: 2.043919\n", - "Train Epoch: 1 [33920/60000 (57%)]\tLoss: 1.980050\n", - "Train Epoch: 1 [34560/60000 (58%)]\tLoss: 1.970208\n", - "Train Epoch: 1 [35200/60000 (59%)]\tLoss: 2.048699\n", - "Train Epoch: 1 [35840/60000 (60%)]\tLoss: 1.956716\n", - "Train Epoch: 1 [36480/60000 (61%)]\tLoss: 1.987072\n", - "Train Epoch: 1 [37120/60000 (62%)]\tLoss: 1.984343\n", - "Train Epoch: 1 [37760/60000 (63%)]\tLoss: 1.947972\n", - "Train Epoch: 1 [38400/60000 (64%)]\tLoss: 1.938421\n", - "Train Epoch: 1 [39040/60000 (65%)]\tLoss: 1.919820\n", - "Train Epoch: 1 [39680/60000 (66%)]\tLoss: 1.931831\n", - "Train Epoch: 1 [40320/60000 (67%)]\tLoss: 1.953799\n", - "Train Epoch: 1 [40960/60000 (68%)]\tLoss: 1.883609\n", - "Train Epoch: 1 [41600/60000 (69%)]\tLoss: 1.876115\n", - "Train Epoch: 1 [42240/60000 (70%)]\tLoss: 1.870510\n", - "Train Epoch: 1 [42880/60000 (71%)]\tLoss: 1.910781\n", - "Train Epoch: 1 [43520/60000 (72%)]\tLoss: 1.950781\n", - "Train Epoch: 1 [44160/60000 (74%)]\tLoss: 1.985367\n", - "Train Epoch: 1 [44800/60000 (75%)]\tLoss: 1.823831\n", - "Train Epoch: 1 [45440/60000 (76%)]\tLoss: 1.870280\n", - "Train Epoch: 1 [46080/60000 (77%)]\tLoss: 1.794647\n", - "Train Epoch: 1 [46720/60000 (78%)]\tLoss: 1.864569\n", - "Train Epoch: 1 [47360/60000 (79%)]\tLoss: 1.871643\n", - "Train Epoch: 1 [48000/60000 (80%)]\tLoss: 1.794408\n", - "Train Epoch: 1 [48640/60000 (81%)]\tLoss: 1.836012\n", - "Train Epoch: 1 [49280/60000 (82%)]\tLoss: 1.784632\n", - "Train Epoch: 1 [49920/60000 (83%)]\tLoss: 1.775053\n", - "Train Epoch: 1 [50560/60000 (84%)]\tLoss: 1.691660\n", - "Train Epoch: 1 [51200/60000 (85%)]\tLoss: 1.818502\n", - "Train Epoch: 1 [51840/60000 (86%)]\tLoss: 1.792987\n", - "Train Epoch: 1 [52480/60000 (87%)]\tLoss: 1.763221\n", - "Train Epoch: 1 [53120/60000 (88%)]\tLoss: 1.779346\n", - "Train Epoch: 1 [53760/60000 (90%)]\tLoss: 1.678108\n", - "Train Epoch: 1 [54400/60000 (91%)]\tLoss: 1.780516\n", - "Train Epoch: 1 [55040/60000 (92%)]\tLoss: 1.650490\n", - "Train Epoch: 1 [55680/60000 (93%)]\tLoss: 1.704799\n", - "Train Epoch: 1 [56320/60000 (94%)]\tLoss: 1.720001\n", - "Train Epoch: 1 [56960/60000 (95%)]\tLoss: 1.716193\n", - "Train Epoch: 1 [57600/60000 (96%)]\tLoss: 1.665386\n", - "Train Epoch: 1 [58240/60000 (97%)]\tLoss: 1.669093\n", - "Train Epoch: 1 [58880/60000 (98%)]\tLoss: 1.669740\n", - "Train Epoch: 1 [59520/60000 (99%)]\tLoss: 1.576309\n", - "\n", - "Test set: Average loss: 1.5640\n", - "Class 0 accuracy: 942.0/980(96%)\n", - "Class 1 accuracy: 1049.0/1135(92%)\n", - "Class 2 accuracy: 530.0/1032(51%)\n", - "Class 3 accuracy: 928.0/1010(92%)\n", - "Class 4 accuracy: 747.0/982(76%)\n", - "Class 5 accuracy: 299.0/892(34%)\n", - "Class 6 accuracy: 880.0/958(92%)\n", - "Class 7 accuracy: 851.0/1028(83%)\n", - "Class 8 accuracy: 744.0/974(76%)\n", - "Class 9 accuracy: 788.0/1009(78%)\n", - "Train Epoch: 2 [0/60000 (0%)]\tLoss: 1.627061\n", - "Train Epoch: 2 [640/60000 (1%)]\tLoss: 1.545597\n", - "Train Epoch: 2 [1280/60000 (2%)]\tLoss: 1.698457\n", - "Train Epoch: 2 [1920/60000 (3%)]\tLoss: 1.572205\n", - "Train Epoch: 2 [2560/60000 (4%)]\tLoss: 1.636262\n", - "Train Epoch: 2 [3200/60000 (5%)]\tLoss: 1.641547\n", - "Train Epoch: 2 [3840/60000 (6%)]\tLoss: 1.528854\n", - "Train Epoch: 2 [4480/60000 (7%)]\tLoss: 1.606497\n", - "Train Epoch: 2 [5120/60000 (9%)]\tLoss: 1.548691\n", - "Train Epoch: 2 [5760/60000 (10%)]\tLoss: 1.562155\n", - "Train Epoch: 2 [6400/60000 (11%)]\tLoss: 1.498272\n", - "Train Epoch: 2 [7040/60000 (12%)]\tLoss: 1.519710\n", - "Train Epoch: 2 [7680/60000 (13%)]\tLoss: 1.605046\n", - "Train Epoch: 2 [8320/60000 (14%)]\tLoss: 1.567077\n", - "Train Epoch: 2 [8960/60000 (15%)]\tLoss: 1.434826\n", - "Train Epoch: 2 [9600/60000 (16%)]\tLoss: 1.478333\n", - "Train Epoch: 2 [10240/60000 (17%)]\tLoss: 1.479111\n", - "Train Epoch: 2 [10880/60000 (18%)]\tLoss: 1.503707\n", - "Train Epoch: 2 [11520/60000 (19%)]\tLoss: 1.373165\n", - "Train Epoch: 2 [12160/60000 (20%)]\tLoss: 1.460236\n", - "Train Epoch: 2 [12800/60000 (21%)]\tLoss: 1.565628\n", - "Train Epoch: 2 [13440/60000 (22%)]\tLoss: 1.565041\n", - "Train Epoch: 2 [14080/60000 (23%)]\tLoss: 1.426577\n", - "Train Epoch: 2 [14720/60000 (25%)]\tLoss: 1.531582\n", - "Train Epoch: 2 [15360/60000 (26%)]\tLoss: 1.427804\n", - "Train Epoch: 2 [16000/60000 (27%)]\tLoss: 1.453471\n", - "Train Epoch: 2 [16640/60000 (28%)]\tLoss: 1.528445\n", - "Train Epoch: 2 [17280/60000 (29%)]\tLoss: 1.515689\n", - "Train Epoch: 2 [17920/60000 (30%)]\tLoss: 1.517196\n", - "Train Epoch: 2 [18560/60000 (31%)]\tLoss: 1.446620\n", - "Train Epoch: 2 [19200/60000 (32%)]\tLoss: 1.496210\n", - "Train Epoch: 2 [19840/60000 (33%)]\tLoss: 1.423505\n", - "Train Epoch: 2 [20480/60000 (34%)]\tLoss: 1.415124\n", - "Train Epoch: 2 [21120/60000 (35%)]\tLoss: 1.544892\n", - "Train Epoch: 2 [21760/60000 (36%)]\tLoss: 1.340283\n", - "Train Epoch: 2 [22400/60000 (37%)]\tLoss: 1.363816\n", - "Train Epoch: 2 [23040/60000 (38%)]\tLoss: 1.382646\n", - "Train Epoch: 2 [23680/60000 (39%)]\tLoss: 1.264206\n", - "Train Epoch: 2 [24320/60000 (41%)]\tLoss: 1.332618\n", - "Train Epoch: 2 [24960/60000 (42%)]\tLoss: 1.358470\n", - "Train Epoch: 2 [25600/60000 (43%)]\tLoss: 1.344868\n", - "Train Epoch: 2 [26240/60000 (44%)]\tLoss: 1.327641\n", - "Train Epoch: 2 [26880/60000 (45%)]\tLoss: 1.382141\n", - "Train Epoch: 2 [27520/60000 (46%)]\tLoss: 1.403416\n", - "Train Epoch: 2 [28160/60000 (47%)]\tLoss: 1.420001\n", - "Train Epoch: 2 [28800/60000 (48%)]\tLoss: 1.414284\n", - "Train Epoch: 2 [29440/60000 (49%)]\tLoss: 1.367697\n", - "Train Epoch: 2 [30080/60000 (50%)]\tLoss: 1.498880\n", - "Train Epoch: 2 [30720/60000 (51%)]\tLoss: 1.278797\n", - "Train Epoch: 2 [31360/60000 (52%)]\tLoss: 1.333852\n", - "Train Epoch: 2 [32000/60000 (53%)]\tLoss: 1.370776\n", - "Train Epoch: 2 [32640/60000 (54%)]\tLoss: 1.062860\n", - "Train Epoch: 2 [33280/60000 (55%)]\tLoss: 1.245440\n", - "Train Epoch: 2 [33920/60000 (57%)]\tLoss: 1.152776\n", - "Train Epoch: 2 [34560/60000 (58%)]\tLoss: 1.344226\n", - "Train Epoch: 2 [35200/60000 (59%)]\tLoss: 1.242468\n", - "Train Epoch: 2 [35840/60000 (60%)]\tLoss: 1.353402\n", - "Train Epoch: 2 [36480/60000 (61%)]\tLoss: 1.166169\n", - "Train Epoch: 2 [37120/60000 (62%)]\tLoss: 1.165925\n", - "Train Epoch: 2 [37760/60000 (63%)]\tLoss: 1.237298\n", - "Train Epoch: 2 [38400/60000 (64%)]\tLoss: 1.209161\n", - "Train Epoch: 2 [39040/60000 (65%)]\tLoss: 1.284919\n", - "Train Epoch: 2 [39680/60000 (66%)]\tLoss: 1.301399\n", - "Train Epoch: 2 [40320/60000 (67%)]\tLoss: 1.257363\n", - "Train Epoch: 2 [40960/60000 (68%)]\tLoss: 1.199494\n", - "Train Epoch: 2 [41600/60000 (69%)]\tLoss: 1.113902\n", - "Train Epoch: 2 [42240/60000 (70%)]\tLoss: 1.155495\n", - "Train Epoch: 2 [42880/60000 (71%)]\tLoss: 1.227993\n", - "Train Epoch: 2 [43520/60000 (72%)]\tLoss: 1.231888\n", - "Train Epoch: 2 [44160/60000 (74%)]\tLoss: 1.373095\n", - "Train Epoch: 2 [44800/60000 (75%)]\tLoss: 1.248195\n", - "Train Epoch: 2 [45440/60000 (76%)]\tLoss: 1.127467\n", - "Train Epoch: 2 [46080/60000 (77%)]\tLoss: 1.045080\n", - "Train Epoch: 2 [46720/60000 (78%)]\tLoss: 1.206602\n", - "Train Epoch: 2 [47360/60000 (79%)]\tLoss: 1.128123\n", - "Train Epoch: 2 [48000/60000 (80%)]\tLoss: 1.348677\n", - "Train Epoch: 2 [48640/60000 (81%)]\tLoss: 1.197632\n", - "Train Epoch: 2 [49280/60000 (82%)]\tLoss: 1.165728\n", - "Train Epoch: 2 [49920/60000 (83%)]\tLoss: 1.209910\n", - "Train Epoch: 2 [50560/60000 (84%)]\tLoss: 1.137662\n", - "Train Epoch: 2 [51200/60000 (85%)]\tLoss: 1.109170\n", - "Train Epoch: 2 [51840/60000 (86%)]\tLoss: 1.038135\n", - "Train Epoch: 2 [52480/60000 (87%)]\tLoss: 1.092770\n", - "Train Epoch: 2 [53120/60000 (88%)]\tLoss: 1.205910\n", - "Train Epoch: 2 [53760/60000 (90%)]\tLoss: 0.988055\n", - "Train Epoch: 2 [54400/60000 (91%)]\tLoss: 1.205869\n", - "Train Epoch: 2 [55040/60000 (92%)]\tLoss: 1.149507\n", - "Train Epoch: 2 [55680/60000 (93%)]\tLoss: 1.059419\n", - "Train Epoch: 2 [56320/60000 (94%)]\tLoss: 1.130637\n", - "Train Epoch: 2 [56960/60000 (95%)]\tLoss: 1.067291\n", - "Train Epoch: 2 [57600/60000 (96%)]\tLoss: 1.102901\n", - "Train Epoch: 2 [58240/60000 (97%)]\tLoss: 1.082644\n", - "Train Epoch: 2 [58880/60000 (98%)]\tLoss: 1.176209\n", - "Train Epoch: 2 [59520/60000 (99%)]\tLoss: 1.100892\n", - "\n", - "Test set: Average loss: 0.9323\n", - "Class 0 accuracy: 937.0/980(96%)\n", - "Class 1 accuracy: 1083.0/1135(95%)\n", - "Class 2 accuracy: 748.0/1032(72%)\n", - "Class 3 accuracy: 923.0/1010(91%)\n", - "Class 4 accuracy: 826.0/982(84%)\n", - "Class 5 accuracy: 470.0/892(53%)\n", - "Class 6 accuracy: 886.0/958(92%)\n", - "Class 7 accuracy: 898.0/1028(87%)\n", - "Class 8 accuracy: 732.0/974(75%)\n", - "Class 9 accuracy: 844.0/1009(84%)\n", - "Train Epoch: 3 [0/60000 (0%)]\tLoss: 1.117987\n", - "Train Epoch: 3 [640/60000 (1%)]\tLoss: 1.044462\n", - "Train Epoch: 3 [1280/60000 (2%)]\tLoss: 1.027436\n", - "Train Epoch: 3 [1920/60000 (3%)]\tLoss: 1.230957\n", - "Train Epoch: 3 [2560/60000 (4%)]\tLoss: 1.004591\n", - "Train Epoch: 3 [3200/60000 (5%)]\tLoss: 1.060180\n", - "Train Epoch: 3 [3840/60000 (6%)]\tLoss: 0.902412\n", - "Train Epoch: 3 [4480/60000 (7%)]\tLoss: 0.957223\n", - "Train Epoch: 3 [5120/60000 (9%)]\tLoss: 1.091123\n", - "Train Epoch: 3 [5760/60000 (10%)]\tLoss: 1.175083\n", - "Train Epoch: 3 [6400/60000 (11%)]\tLoss: 0.969085\n", - "Train Epoch: 3 [7040/60000 (12%)]\tLoss: 1.044883\n", - "Train Epoch: 3 [7680/60000 (13%)]\tLoss: 1.010503\n", - "Train Epoch: 3 [8320/60000 (14%)]\tLoss: 1.037116\n", - "Train Epoch: 3 [8960/60000 (15%)]\tLoss: 0.968517\n", - "Train Epoch: 3 [9600/60000 (16%)]\tLoss: 1.037061\n", - "Train Epoch: 3 [10240/60000 (17%)]\tLoss: 0.992225\n", - "Train Epoch: 3 [10880/60000 (18%)]\tLoss: 1.091760\n", - "Train Epoch: 3 [11520/60000 (19%)]\tLoss: 1.100666\n", - "Train Epoch: 3 [12160/60000 (20%)]\tLoss: 0.852405\n", - "Train Epoch: 3 [12800/60000 (21%)]\tLoss: 0.957409\n", - "Train Epoch: 3 [13440/60000 (22%)]\tLoss: 0.959758\n", - "Train Epoch: 3 [14080/60000 (23%)]\tLoss: 0.998838\n", - "Train Epoch: 3 [14720/60000 (25%)]\tLoss: 1.045380\n", - "Train Epoch: 3 [15360/60000 (26%)]\tLoss: 0.842424\n", - "Train Epoch: 3 [16000/60000 (27%)]\tLoss: 1.003876\n", - "Train Epoch: 3 [16640/60000 (28%)]\tLoss: 0.948943\n", - "Train Epoch: 3 [17280/60000 (29%)]\tLoss: 1.090439\n", - "Train Epoch: 3 [17920/60000 (30%)]\tLoss: 1.087915\n", - "Train Epoch: 3 [18560/60000 (31%)]\tLoss: 1.145274\n", - "Train Epoch: 3 [19200/60000 (32%)]\tLoss: 1.217145\n", - "Train Epoch: 3 [19840/60000 (33%)]\tLoss: 1.085176\n", - "Train Epoch: 3 [20480/60000 (34%)]\tLoss: 1.083079\n", - "Train Epoch: 3 [21120/60000 (35%)]\tLoss: 0.999794\n", - "Train Epoch: 3 [21760/60000 (36%)]\tLoss: 0.911105\n", - "Train Epoch: 3 [22400/60000 (37%)]\tLoss: 1.273139\n", - "Train Epoch: 3 [23040/60000 (38%)]\tLoss: 1.048052\n", - "Train Epoch: 3 [23680/60000 (39%)]\tLoss: 0.818689\n", - "Train Epoch: 3 [24320/60000 (41%)]\tLoss: 0.858526\n", - "Train Epoch: 3 [24960/60000 (42%)]\tLoss: 0.982630\n", - "Train Epoch: 3 [25600/60000 (43%)]\tLoss: 1.042941\n", - "Train Epoch: 3 [26240/60000 (44%)]\tLoss: 0.818404\n", - "Train Epoch: 3 [26880/60000 (45%)]\tLoss: 0.947276\n", - "Train Epoch: 3 [27520/60000 (46%)]\tLoss: 0.990826\n", - "Train Epoch: 3 [28160/60000 (47%)]\tLoss: 0.956242\n", - "Train Epoch: 3 [28800/60000 (48%)]\tLoss: 1.052988\n", - "Train Epoch: 3 [29440/60000 (49%)]\tLoss: 1.157221\n", - "Train Epoch: 3 [30080/60000 (50%)]\tLoss: 0.913521\n", - "Train Epoch: 3 [30720/60000 (51%)]\tLoss: 1.009682\n", - "Train Epoch: 3 [31360/60000 (52%)]\tLoss: 0.770679\n", - "Train Epoch: 3 [32000/60000 (53%)]\tLoss: 0.990214\n", - "Train Epoch: 3 [32640/60000 (54%)]\tLoss: 0.937079\n", - "Train Epoch: 3 [33280/60000 (55%)]\tLoss: 0.872119\n", - "Train Epoch: 3 [33920/60000 (57%)]\tLoss: 0.840864\n", - "Train Epoch: 3 [34560/60000 (58%)]\tLoss: 1.002336\n", - "Train Epoch: 3 [35200/60000 (59%)]\tLoss: 0.915164\n", - "Train Epoch: 3 [35840/60000 (60%)]\tLoss: 1.012089\n", - "Train Epoch: 3 [36480/60000 (61%)]\tLoss: 0.939280\n", - "Train Epoch: 3 [37120/60000 (62%)]\tLoss: 1.060541\n", - "Train Epoch: 3 [37760/60000 (63%)]\tLoss: 0.937920\n", - "Train Epoch: 3 [38400/60000 (64%)]\tLoss: 1.006945\n", - "Train Epoch: 3 [39040/60000 (65%)]\tLoss: 1.078756\n", - "Train Epoch: 3 [39680/60000 (66%)]\tLoss: 1.017594\n", - "Train Epoch: 3 [40320/60000 (67%)]\tLoss: 1.033234\n", - "Train Epoch: 3 [40960/60000 (68%)]\tLoss: 1.230211\n", - "Train Epoch: 3 [41600/60000 (69%)]\tLoss: 0.684275\n", - "Train Epoch: 3 [42240/60000 (70%)]\tLoss: 1.083214\n", - "Train Epoch: 3 [42880/60000 (71%)]\tLoss: 0.852170\n", - "Train Epoch: 3 [43520/60000 (72%)]\tLoss: 0.995445\n", - "Train Epoch: 3 [44160/60000 (74%)]\tLoss: 0.790355\n", - "Train Epoch: 3 [44800/60000 (75%)]\tLoss: 0.920374\n", - "Train Epoch: 3 [45440/60000 (76%)]\tLoss: 1.090557\n", - "Train Epoch: 3 [46080/60000 (77%)]\tLoss: 0.755298\n", - "Train Epoch: 3 [46720/60000 (78%)]\tLoss: 1.101079\n", - "Train Epoch: 3 [47360/60000 (79%)]\tLoss: 0.719922\n", - "Train Epoch: 3 [48000/60000 (80%)]\tLoss: 0.805579\n", - "Train Epoch: 3 [48640/60000 (81%)]\tLoss: 0.866616\n", - "Train Epoch: 3 [49280/60000 (82%)]\tLoss: 0.777545\n", - "Train Epoch: 3 [49920/60000 (83%)]\tLoss: 0.800337\n", - "Train Epoch: 3 [50560/60000 (84%)]\tLoss: 0.921025\n", - "Train Epoch: 3 [51200/60000 (85%)]\tLoss: 0.961804\n", - "Train Epoch: 3 [51840/60000 (86%)]\tLoss: 0.882696\n", - "Train Epoch: 3 [52480/60000 (87%)]\tLoss: 0.849839\n", - "Train Epoch: 3 [53120/60000 (88%)]\tLoss: 0.893887\n", - "Train Epoch: 3 [53760/60000 (90%)]\tLoss: 0.815971\n", - "Train Epoch: 3 [54400/60000 (91%)]\tLoss: 1.109305\n", - "Train Epoch: 3 [55040/60000 (92%)]\tLoss: 0.864345\n", - "Train Epoch: 3 [55680/60000 (93%)]\tLoss: 0.969730\n", - "Train Epoch: 3 [56320/60000 (94%)]\tLoss: 0.959234\n", - "Train Epoch: 3 [56960/60000 (95%)]\tLoss: 0.954617\n", - "Train Epoch: 3 [57600/60000 (96%)]\tLoss: 0.790892\n", - "Train Epoch: 3 [58240/60000 (97%)]\tLoss: 1.002821\n", - "Train Epoch: 3 [58880/60000 (98%)]\tLoss: 0.799075\n", - "Train Epoch: 3 [59520/60000 (99%)]\tLoss: 0.859415\n", - "\n", - "Test set: Average loss: 0.7067\n", - "Class 0 accuracy: 942.0/980(96%)\n", - "Class 1 accuracy: 1085.0/1135(96%)\n", - "Class 2 accuracy: 815.0/1032(79%)\n", - "Class 3 accuracy: 909.0/1010(90%)\n", - "Class 4 accuracy: 844.0/982(86%)\n", - "Class 5 accuracy: 572.0/892(64%)\n", - "Class 6 accuracy: 881.0/958(92%)\n", - "Class 7 accuracy: 895.0/1028(87%)\n", - "Class 8 accuracy: 773.0/974(79%)\n", - "Class 9 accuracy: 861.0/1009(85%)\n", - "Train Epoch: 4 [0/60000 (0%)]\tLoss: 0.921860\n", - "Train Epoch: 4 [640/60000 (1%)]\tLoss: 1.006901\n", - "Train Epoch: 4 [1280/60000 (2%)]\tLoss: 0.829744\n", - "Train Epoch: 4 [1920/60000 (3%)]\tLoss: 0.958730\n", - "Train Epoch: 4 [2560/60000 (4%)]\tLoss: 0.908768\n", - "Train Epoch: 4 [3200/60000 (5%)]\tLoss: 0.809689\n", - "Train Epoch: 4 [3840/60000 (6%)]\tLoss: 0.864438\n", - "Train Epoch: 4 [4480/60000 (7%)]\tLoss: 0.871907\n", - "Train Epoch: 4 [5120/60000 (9%)]\tLoss: 0.879927\n", - "Train Epoch: 4 [5760/60000 (10%)]\tLoss: 0.958960\n", - "Train Epoch: 4 [6400/60000 (11%)]\tLoss: 0.866098\n", - "Train Epoch: 4 [7040/60000 (12%)]\tLoss: 1.041393\n", - "Train Epoch: 4 [7680/60000 (13%)]\tLoss: 0.813019\n", - "Train Epoch: 4 [8320/60000 (14%)]\tLoss: 0.767784\n", - "Train Epoch: 4 [8960/60000 (15%)]\tLoss: 0.933339\n", - "Train Epoch: 4 [9600/60000 (16%)]\tLoss: 0.821877\n", - "Train Epoch: 4 [10240/60000 (17%)]\tLoss: 0.826119\n", - "Train Epoch: 4 [10880/60000 (18%)]\tLoss: 0.772280\n", - "Train Epoch: 4 [11520/60000 (19%)]\tLoss: 0.868618\n", - "Train Epoch: 4 [12160/60000 (20%)]\tLoss: 1.034466\n", - "Train Epoch: 4 [12800/60000 (21%)]\tLoss: 0.872086\n", - "Train Epoch: 4 [13440/60000 (22%)]\tLoss: 0.912373\n", - "Train Epoch: 4 [14080/60000 (23%)]\tLoss: 0.870299\n", - "Train Epoch: 4 [14720/60000 (25%)]\tLoss: 0.731594\n", - "Train Epoch: 4 [15360/60000 (26%)]\tLoss: 1.079396\n", - "Train Epoch: 4 [16000/60000 (27%)]\tLoss: 0.882126\n", - "Train Epoch: 4 [16640/60000 (28%)]\tLoss: 0.682331\n", - "Train Epoch: 4 [17280/60000 (29%)]\tLoss: 1.029123\n", - "Train Epoch: 4 [17920/60000 (30%)]\tLoss: 0.596465\n", - "Train Epoch: 4 [18560/60000 (31%)]\tLoss: 0.813218\n", - "Train Epoch: 4 [19200/60000 (32%)]\tLoss: 0.747696\n", - "Train Epoch: 4 [19840/60000 (33%)]\tLoss: 0.880849\n", - "Train Epoch: 4 [20480/60000 (34%)]\tLoss: 0.734443\n", - "Train Epoch: 4 [21120/60000 (35%)]\tLoss: 0.840509\n", - "Train Epoch: 4 [21760/60000 (36%)]\tLoss: 0.745489\n", - "Train Epoch: 4 [22400/60000 (37%)]\tLoss: 0.841733\n", - "Train Epoch: 4 [23040/60000 (38%)]\tLoss: 0.920951\n", - "Train Epoch: 4 [23680/60000 (39%)]\tLoss: 0.924071\n", - "Train Epoch: 4 [24320/60000 (41%)]\tLoss: 0.906279\n", - "Train Epoch: 4 [24960/60000 (42%)]\tLoss: 0.867031\n", - "Train Epoch: 4 [25600/60000 (43%)]\tLoss: 0.946203\n", - "Train Epoch: 4 [26240/60000 (44%)]\tLoss: 0.989663\n", - "Train Epoch: 4 [26880/60000 (45%)]\tLoss: 0.839466\n", - "Train Epoch: 4 [27520/60000 (46%)]\tLoss: 0.843587\n", - "Train Epoch: 4 [28160/60000 (47%)]\tLoss: 0.737129\n", - "Train Epoch: 4 [28800/60000 (48%)]\tLoss: 0.807360\n", - "Train Epoch: 4 [29440/60000 (49%)]\tLoss: 0.823521\n", - "Train Epoch: 4 [30080/60000 (50%)]\tLoss: 0.841337\n", - "Train Epoch: 4 [30720/60000 (51%)]\tLoss: 0.999043\n", - "Train Epoch: 4 [31360/60000 (52%)]\tLoss: 0.962988\n", - "Train Epoch: 4 [32000/60000 (53%)]\tLoss: 1.040999\n", - "Train Epoch: 4 [32640/60000 (54%)]\tLoss: 0.750595\n", - "Train Epoch: 4 [33280/60000 (55%)]\tLoss: 0.904500\n", - "Train Epoch: 4 [33920/60000 (57%)]\tLoss: 0.716458\n", - "Train Epoch: 4 [34560/60000 (58%)]\tLoss: 0.917832\n", - "Train Epoch: 4 [35200/60000 (59%)]\tLoss: 0.834955\n", - "Train Epoch: 4 [35840/60000 (60%)]\tLoss: 0.833779\n", - "Train Epoch: 4 [36480/60000 (61%)]\tLoss: 0.798480\n", - "Train Epoch: 4 [37120/60000 (62%)]\tLoss: 0.718178\n", - "Train Epoch: 4 [37760/60000 (63%)]\tLoss: 0.774633\n", - "Train Epoch: 4 [38400/60000 (64%)]\tLoss: 0.640932\n", - "Train Epoch: 4 [39040/60000 (65%)]\tLoss: 0.761126\n", - "Train Epoch: 4 [39680/60000 (66%)]\tLoss: 0.567800\n", - "Train Epoch: 4 [40320/60000 (67%)]\tLoss: 0.731927\n", - "Train Epoch: 4 [40960/60000 (68%)]\tLoss: 0.972790\n", - "Train Epoch: 4 [41600/60000 (69%)]\tLoss: 0.913961\n", - "Train Epoch: 4 [42240/60000 (70%)]\tLoss: 0.630956\n", - "Train Epoch: 4 [42880/60000 (71%)]\tLoss: 0.928092\n", - "Train Epoch: 4 [43520/60000 (72%)]\tLoss: 0.749575\n", - "Train Epoch: 4 [44160/60000 (74%)]\tLoss: 0.970667\n", - "Train Epoch: 4 [44800/60000 (75%)]\tLoss: 0.804512\n", - "Train Epoch: 4 [45440/60000 (76%)]\tLoss: 0.832241\n", - "Train Epoch: 4 [46080/60000 (77%)]\tLoss: 0.851197\n", - "Train Epoch: 4 [46720/60000 (78%)]\tLoss: 0.918072\n", - "Train Epoch: 4 [47360/60000 (79%)]\tLoss: 0.886158\n", - "Train Epoch: 4 [48000/60000 (80%)]\tLoss: 0.803933\n", - "Train Epoch: 4 [48640/60000 (81%)]\tLoss: 0.711099\n", - "Train Epoch: 4 [49280/60000 (82%)]\tLoss: 0.786826\n", - "Train Epoch: 4 [49920/60000 (83%)]\tLoss: 0.834988\n", - "Train Epoch: 4 [50560/60000 (84%)]\tLoss: 0.902848\n", - "Train Epoch: 4 [51200/60000 (85%)]\tLoss: 0.838035\n", - "Train Epoch: 4 [51840/60000 (86%)]\tLoss: 0.853425\n", - "Train Epoch: 4 [52480/60000 (87%)]\tLoss: 0.843906\n", - "Train Epoch: 4 [53120/60000 (88%)]\tLoss: 0.782411\n", - "Train Epoch: 4 [53760/60000 (90%)]\tLoss: 0.752052\n", - "Train Epoch: 4 [54400/60000 (91%)]\tLoss: 0.738311\n", - "Train Epoch: 4 [55040/60000 (92%)]\tLoss: 0.929952\n", - "Train Epoch: 4 [55680/60000 (93%)]\tLoss: 0.858274\n", - "Train Epoch: 4 [56320/60000 (94%)]\tLoss: 1.006905\n", - "Train Epoch: 4 [56960/60000 (95%)]\tLoss: 0.824607\n", - "Train Epoch: 4 [57600/60000 (96%)]\tLoss: 0.852737\n", - "Train Epoch: 4 [58240/60000 (97%)]\tLoss: 0.789701\n", - "Train Epoch: 4 [58880/60000 (98%)]\tLoss: 0.821994\n", - "Train Epoch: 4 [59520/60000 (99%)]\tLoss: 0.840377\n", - "\n", - "Test set: Average loss: 0.6137\n", - "Class 0 accuracy: 947.0/980(97%)\n", - "Class 1 accuracy: 1097.0/1135(97%)\n", - "Class 2 accuracy: 840.0/1032(81%)\n", - "Class 3 accuracy: 912.0/1010(90%)\n", - "Class 4 accuracy: 852.0/982(87%)\n", - "Class 5 accuracy: 602.0/892(67%)\n", - "Class 6 accuracy: 879.0/958(92%)\n", - "Class 7 accuracy: 904.0/1028(88%)\n", - "Class 8 accuracy: 770.0/974(79%)\n", - "Class 9 accuracy: 866.0/1009(86%)\n", - "Train Epoch: 5 [0/60000 (0%)]\tLoss: 0.841241\n", - "Train Epoch: 5 [640/60000 (1%)]\tLoss: 0.963371\n", - "Train Epoch: 5 [1280/60000 (2%)]\tLoss: 0.714024\n", - "Train Epoch: 5 [1920/60000 (3%)]\tLoss: 0.760713\n", - "Train Epoch: 5 [2560/60000 (4%)]\tLoss: 0.709700\n", - "Train Epoch: 5 [3200/60000 (5%)]\tLoss: 0.959411\n", - "Train Epoch: 5 [3840/60000 (6%)]\tLoss: 0.823688\n", - "Train Epoch: 5 [4480/60000 (7%)]\tLoss: 0.688386\n", - "Train Epoch: 5 [5120/60000 (9%)]\tLoss: 0.836027\n", - "Train Epoch: 5 [5760/60000 (10%)]\tLoss: 0.814876\n", - "Train Epoch: 5 [6400/60000 (11%)]\tLoss: 0.915517\n", - "Train Epoch: 5 [7040/60000 (12%)]\tLoss: 0.930052\n", - "Train Epoch: 5 [7680/60000 (13%)]\tLoss: 0.713963\n", - "Train Epoch: 5 [8320/60000 (14%)]\tLoss: 0.611171\n", - "Train Epoch: 5 [8960/60000 (15%)]\tLoss: 0.753952\n", - "Train Epoch: 5 [9600/60000 (16%)]\tLoss: 0.753516\n", - "Train Epoch: 5 [10240/60000 (17%)]\tLoss: 0.638662\n", - "Train Epoch: 5 [10880/60000 (18%)]\tLoss: 0.851477\n", - "Train Epoch: 5 [11520/60000 (19%)]\tLoss: 0.748764\n", - "Train Epoch: 5 [12160/60000 (20%)]\tLoss: 0.717946\n", - "Train Epoch: 5 [12800/60000 (21%)]\tLoss: 0.648801\n", - "Train Epoch: 5 [13440/60000 (22%)]\tLoss: 0.817798\n", - "Train Epoch: 5 [14080/60000 (23%)]\tLoss: 1.011557\n", - "Train Epoch: 5 [14720/60000 (25%)]\tLoss: 0.979450\n", - "Train Epoch: 5 [15360/60000 (26%)]\tLoss: 0.741829\n", - "Train Epoch: 5 [16000/60000 (27%)]\tLoss: 0.929305\n", - "Train Epoch: 5 [16640/60000 (28%)]\tLoss: 0.898180\n", - "Train Epoch: 5 [17280/60000 (29%)]\tLoss: 0.682718\n", - "Train Epoch: 5 [17920/60000 (30%)]\tLoss: 0.841480\n", - "Train Epoch: 5 [18560/60000 (31%)]\tLoss: 0.895796\n", - "Train Epoch: 5 [19200/60000 (32%)]\tLoss: 0.707863\n", - "Train Epoch: 5 [19840/60000 (33%)]\tLoss: 0.851275\n", - "Train Epoch: 5 [20480/60000 (34%)]\tLoss: 0.855291\n", - "Train Epoch: 5 [21120/60000 (35%)]\tLoss: 0.851695\n", - "Train Epoch: 5 [21760/60000 (36%)]\tLoss: 0.684048\n", - "Train Epoch: 5 [22400/60000 (37%)]\tLoss: 0.917995\n", - "Train Epoch: 5 [23040/60000 (38%)]\tLoss: 0.765770\n", - "Train Epoch: 5 [23680/60000 (39%)]\tLoss: 0.647655\n", - "Train Epoch: 5 [24320/60000 (41%)]\tLoss: 0.739678\n", - "Train Epoch: 5 [24960/60000 (42%)]\tLoss: 0.672344\n", - "Train Epoch: 5 [25600/60000 (43%)]\tLoss: 0.766927\n", - "Train Epoch: 5 [26240/60000 (44%)]\tLoss: 0.738036\n", - "Train Epoch: 5 [26880/60000 (45%)]\tLoss: 0.852725\n", - "Train Epoch: 5 [27520/60000 (46%)]\tLoss: 0.632756\n", - "Train Epoch: 5 [28160/60000 (47%)]\tLoss: 0.689774\n", - "Train Epoch: 5 [28800/60000 (48%)]\tLoss: 0.674076\n", - "Train Epoch: 5 [29440/60000 (49%)]\tLoss: 0.755838\n", - "Train Epoch: 5 [30080/60000 (50%)]\tLoss: 0.786167\n", - "Train Epoch: 5 [30720/60000 (51%)]\tLoss: 0.804736\n", - "Train Epoch: 5 [31360/60000 (52%)]\tLoss: 0.723398\n", - "Train Epoch: 5 [32000/60000 (53%)]\tLoss: 0.685933\n", - "Train Epoch: 5 [32640/60000 (54%)]\tLoss: 0.808149\n", - "Train Epoch: 5 [33280/60000 (55%)]\tLoss: 0.944384\n", - "Train Epoch: 5 [33920/60000 (57%)]\tLoss: 0.638669\n", - "Train Epoch: 5 [34560/60000 (58%)]\tLoss: 0.683750\n", - "Train Epoch: 5 [35200/60000 (59%)]\tLoss: 0.702595\n", - "Train Epoch: 5 [35840/60000 (60%)]\tLoss: 0.742202\n", - "Train Epoch: 5 [36480/60000 (61%)]\tLoss: 0.761243\n", - "Train Epoch: 5 [37120/60000 (62%)]\tLoss: 0.768657\n", - "Train Epoch: 5 [37760/60000 (63%)]\tLoss: 0.824368\n", - "Train Epoch: 5 [38400/60000 (64%)]\tLoss: 0.657427\n", - "Train Epoch: 5 [39040/60000 (65%)]\tLoss: 0.724466\n", - "Train Epoch: 5 [39680/60000 (66%)]\tLoss: 0.800387\n", - "Train Epoch: 5 [40320/60000 (67%)]\tLoss: 0.812061\n", - "Train Epoch: 5 [40960/60000 (68%)]\tLoss: 0.811920\n", - "Train Epoch: 5 [41600/60000 (69%)]\tLoss: 0.888286\n", - "Train Epoch: 5 [42240/60000 (70%)]\tLoss: 0.691121\n", - "Train Epoch: 5 [42880/60000 (71%)]\tLoss: 0.749742\n", - "Train Epoch: 5 [43520/60000 (72%)]\tLoss: 0.944218\n", - "Train Epoch: 5 [44160/60000 (74%)]\tLoss: 0.698245\n", - "Train Epoch: 5 [44800/60000 (75%)]\tLoss: 0.796792\n", - "Train Epoch: 5 [45440/60000 (76%)]\tLoss: 0.653151\n", - "Train Epoch: 5 [46080/60000 (77%)]\tLoss: 0.813349\n", - "Train Epoch: 5 [46720/60000 (78%)]\tLoss: 0.600790\n", - "Train Epoch: 5 [47360/60000 (79%)]\tLoss: 0.848091\n", - "Train Epoch: 5 [48000/60000 (80%)]\tLoss: 0.636887\n", - "Train Epoch: 5 [48640/60000 (81%)]\tLoss: 0.767468\n", - "Train Epoch: 5 [49280/60000 (82%)]\tLoss: 0.713464\n", - "Train Epoch: 5 [49920/60000 (83%)]\tLoss: 0.635464\n", - "Train Epoch: 5 [50560/60000 (84%)]\tLoss: 0.722847\n", - "Train Epoch: 5 [51200/60000 (85%)]\tLoss: 0.924447\n", - "Train Epoch: 5 [51840/60000 (86%)]\tLoss: 0.738732\n", - "Train Epoch: 5 [52480/60000 (87%)]\tLoss: 0.732151\n", - "Train Epoch: 5 [53120/60000 (88%)]\tLoss: 0.769458\n", - "Train Epoch: 5 [53760/60000 (90%)]\tLoss: 0.761817\n", - "Train Epoch: 5 [54400/60000 (91%)]\tLoss: 0.766024\n", - "Train Epoch: 5 [55040/60000 (92%)]\tLoss: 0.820377\n", - "Train Epoch: 5 [55680/60000 (93%)]\tLoss: 0.713448\n", - "Train Epoch: 5 [56320/60000 (94%)]\tLoss: 0.753411\n", - "Train Epoch: 5 [56960/60000 (95%)]\tLoss: 0.758932\n", - "Train Epoch: 5 [57600/60000 (96%)]\tLoss: 0.690840\n", - "Train Epoch: 5 [58240/60000 (97%)]\tLoss: 0.855213\n", - "Train Epoch: 5 [58880/60000 (98%)]\tLoss: 0.847026\n", - "Train Epoch: 5 [59520/60000 (99%)]\tLoss: 0.714162\n", - "\n", - "Test set: Average loss: 0.5663\n", - "Class 0 accuracy: 949.0/980(97%)\n", - "Class 1 accuracy: 1095.0/1135(96%)\n", - "Class 2 accuracy: 841.0/1032(81%)\n", - "Class 3 accuracy: 909.0/1010(90%)\n", - "Class 4 accuracy: 861.0/982(88%)\n", - "Class 5 accuracy: 630.0/892(71%)\n", - "Class 6 accuracy: 881.0/958(92%)\n", - "Class 7 accuracy: 906.0/1028(88%)\n", - "Class 8 accuracy: 798.0/974(82%)\n", - "Class 9 accuracy: 869.0/1009(86%)\n" - ] - }, - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "[{0: 0.9612244897959183,\n", - " 1: 0.9242290748898678,\n", - " 2: 0.5135658914728682,\n", - " 3: 0.9188118811881189,\n", - " 4: 0.7606924643584522,\n", - " 5: 0.3352017937219731,\n", - " 6: 0.918580375782881,\n", - " 7: 0.8278210116731517,\n", - " 8: 0.7638603696098563,\n", - " 9: 0.7809712586719524,\n", - " 'phase': 'baseline'},\n", - " {0: 0.9561224489795919,\n", - " 1: 0.9541850220264317,\n", - " 2: 0.7248062015503876,\n", - " 3: 0.9138613861386139,\n", - " 4: 0.8411405295315683,\n", - " 5: 0.5269058295964125,\n", - " 6: 0.9248434237995825,\n", - " 7: 0.8735408560311284,\n", - " 8: 0.7515400410677618,\n", - " 9: 0.8364717542120912,\n", - " 'phase': 'baseline'},\n", - " {0: 0.9612244897959183,\n", - " 1: 0.9559471365638766,\n", - " 2: 0.7897286821705426,\n", - " 3: 0.9,\n", - " 4: 0.8594704684317719,\n", - " 5: 0.6412556053811659,\n", - " 6: 0.9196242171189979,\n", - " 7: 0.8706225680933852,\n", - " 8: 0.7936344969199178,\n", - " 9: 0.8533201189296333,\n", - " 'phase': 'baseline'},\n", - " {0: 0.9663265306122449,\n", - " 1: 0.9665198237885463,\n", - " 2: 0.813953488372093,\n", - " 3: 0.902970297029703,\n", - " 4: 0.8676171079429735,\n", - " 5: 0.6748878923766816,\n", - " 6: 0.9175365344467641,\n", - " 7: 0.8793774319066148,\n", - " 8: 0.7905544147843943,\n", - " 9: 0.8582755203171457,\n", - " 'phase': 'baseline'},\n", - " {0: 0.9683673469387755,\n", - " 1: 0.9647577092511013,\n", - " 2: 0.814922480620155,\n", - " 3: 0.9,\n", - " 4: 0.8767820773930753,\n", - " 5: 0.7062780269058296,\n", - " 6: 0.9196242171189979,\n", - " 7: 0.8813229571984436,\n", - " 8: 0.8193018480492813,\n", - " 9: 0.8612487611496531,\n", - " 'phase': 'baseline'}]" - ] - }, - "metadata": {}, - "execution_count": 14 - } - ] + "execution_count": 18, + "outputs": [] }, { "cell_type": "code", @@ -1096,41 +492,41 @@ "history_catastrophic_forgetting" ], "metadata": { + "id": "OnATcaveP3Fi", "colab": { "base_uri": "https://localhost:8080/" }, - "id": "OnATcaveP3Fi", - "outputId": "3864a59d-b604-491d-8697-f4f7d622c1e7" + "outputId": "6e9bc36f-8e10-405a-8297-e104141e58cc" }, - "execution_count": 15, + "execution_count": 19, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ - "Train Epoch: 1 [0/12665 (0%)]\tLoss: 2.311233\n", - "Train Epoch: 1 [640/12665 (5%)]\tLoss: 2.246531\n", - "Train Epoch: 1 [1280/12665 (10%)]\tLoss: 2.186137\n", - "Train Epoch: 1 [1920/12665 (15%)]\tLoss: 2.105181\n", - "Train Epoch: 1 [2560/12665 (20%)]\tLoss: 2.048626\n", - "Train Epoch: 1 [3200/12665 (25%)]\tLoss: 1.975536\n", - "Train Epoch: 1 [3840/12665 (30%)]\tLoss: 1.913516\n", - "Train Epoch: 1 [4480/12665 (35%)]\tLoss: 1.818088\n", - "Train Epoch: 1 [5120/12665 (40%)]\tLoss: 1.744223\n", - "Train Epoch: 1 [5760/12665 (45%)]\tLoss: 1.696749\n", - "Train Epoch: 1 [6400/12665 (51%)]\tLoss: 1.615581\n", - "Train Epoch: 1 [7040/12665 (56%)]\tLoss: 1.508412\n", - "Train Epoch: 1 [7680/12665 (61%)]\tLoss: 1.426383\n", - "Train Epoch: 1 [8320/12665 (66%)]\tLoss: 1.385702\n", - "Train Epoch: 1 [8960/12665 (71%)]\tLoss: 1.271088\n", - "Train Epoch: 1 [9600/12665 (76%)]\tLoss: 1.157527\n", - "Train Epoch: 1 [10240/12665 (81%)]\tLoss: 1.164531\n", - "Train Epoch: 1 [10880/12665 (86%)]\tLoss: 1.056471\n", - "Train Epoch: 1 [11520/12665 (91%)]\tLoss: 1.015163\n", - "Train Epoch: 1 [12160/12665 (96%)]\tLoss: 0.924486\n", - "\n", - "Test set: Average loss: 2.6919\n", - "Class 0 accuracy: 972.0/980(99%)\n", + "Train Epoch: 1 [0/12665 (0%)]\tLoss: 2.295613\n", + "Train Epoch: 1 [640/12665 (5%)]\tLoss: 1.647854\n", + "Train Epoch: 1 [1280/12665 (10%)]\tLoss: 1.039609\n", + "Train Epoch: 1 [1920/12665 (15%)]\tLoss: 0.593714\n", + "Train Epoch: 1 [2560/12665 (20%)]\tLoss: 0.321345\n", + "Train Epoch: 1 [3200/12665 (25%)]\tLoss: 0.211062\n", + "Train Epoch: 1 [3840/12665 (30%)]\tLoss: 0.124204\n", + "Train Epoch: 1 [4480/12665 (35%)]\tLoss: 0.082856\n", + "Train Epoch: 1 [5120/12665 (40%)]\tLoss: 0.075962\n", + "Train Epoch: 1 [5760/12665 (45%)]\tLoss: 0.071156\n", + "Train Epoch: 1 [6400/12665 (51%)]\tLoss: 0.117059\n", + "Train Epoch: 1 [7040/12665 (56%)]\tLoss: 0.059726\n", + "Train Epoch: 1 [7680/12665 (61%)]\tLoss: 0.032035\n", + "Train Epoch: 1 [8320/12665 (66%)]\tLoss: 0.033078\n", + "Train Epoch: 1 [8960/12665 (71%)]\tLoss: 0.030728\n", + "Train Epoch: 1 [9600/12665 (76%)]\tLoss: 0.039060\n", + "Train Epoch: 1 [10240/12665 (81%)]\tLoss: 0.035301\n", + "Train Epoch: 1 [10880/12665 (86%)]\tLoss: 0.041941\n", + "Train Epoch: 1 [11520/12665 (91%)]\tLoss: 0.040506\n", + "Train Epoch: 1 [12160/12665 (96%)]\tLoss: 0.014235\n", + "\n", + "Test set: Average loss: 7.1276\n", + "Class 0 accuracy: 977.0/980(100%)\n", "Class 1 accuracy: 1134.0/1135(100%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", @@ -1140,29 +536,29 @@ "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 2 [0/12665 (0%)]\tLoss: 0.725303\n", - "Train Epoch: 2 [640/12665 (5%)]\tLoss: 0.845168\n", - "Train Epoch: 2 [1280/12665 (10%)]\tLoss: 0.872068\n", - "Train Epoch: 2 [1920/12665 (15%)]\tLoss: 0.699184\n", - "Train Epoch: 2 [2560/12665 (20%)]\tLoss: 0.702631\n", - "Train Epoch: 2 [3200/12665 (25%)]\tLoss: 0.649261\n", - "Train Epoch: 2 [3840/12665 (30%)]\tLoss: 0.638805\n", - "Train Epoch: 2 [4480/12665 (35%)]\tLoss: 0.584503\n", - "Train Epoch: 2 [5120/12665 (40%)]\tLoss: 0.612078\n", - "Train Epoch: 2 [5760/12665 (45%)]\tLoss: 0.522527\n", - "Train Epoch: 2 [6400/12665 (51%)]\tLoss: 0.505232\n", - "Train Epoch: 2 [7040/12665 (56%)]\tLoss: 0.500317\n", - "Train Epoch: 2 [7680/12665 (61%)]\tLoss: 0.420275\n", - "Train Epoch: 2 [8320/12665 (66%)]\tLoss: 0.444307\n", - "Train Epoch: 2 [8960/12665 (71%)]\tLoss: 0.431154\n", - "Train Epoch: 2 [9600/12665 (76%)]\tLoss: 0.426933\n", - "Train Epoch: 2 [10240/12665 (81%)]\tLoss: 0.428767\n", - "Train Epoch: 2 [10880/12665 (86%)]\tLoss: 0.397589\n", - "Train Epoch: 2 [11520/12665 (91%)]\tLoss: 0.332724\n", - "Train Epoch: 2 [12160/12665 (96%)]\tLoss: 0.391286\n", - "\n", - "Test set: Average loss: 3.6493\n", - "Class 0 accuracy: 970.0/980(99%)\n", + "Train Epoch: 2 [0/12665 (0%)]\tLoss: 0.007759\n", + "Train Epoch: 2 [640/12665 (5%)]\tLoss: 0.029978\n", + "Train Epoch: 2 [1280/12665 (10%)]\tLoss: 0.017544\n", + "Train Epoch: 2 [1920/12665 (15%)]\tLoss: 0.022188\n", + "Train Epoch: 2 [2560/12665 (20%)]\tLoss: 0.009434\n", + "Train Epoch: 2 [3200/12665 (25%)]\tLoss: 0.024381\n", + "Train Epoch: 2 [3840/12665 (30%)]\tLoss: 0.086738\n", + "Train Epoch: 2 [4480/12665 (35%)]\tLoss: 0.020443\n", + "Train Epoch: 2 [5120/12665 (40%)]\tLoss: 0.015766\n", + "Train Epoch: 2 [5760/12665 (45%)]\tLoss: 0.006633\n", + "Train Epoch: 2 [6400/12665 (51%)]\tLoss: 0.007944\n", + "Train Epoch: 2 [7040/12665 (56%)]\tLoss: 0.008917\n", + "Train Epoch: 2 [7680/12665 (61%)]\tLoss: 0.004392\n", + "Train Epoch: 2 [8320/12665 (66%)]\tLoss: 0.008756\n", + "Train Epoch: 2 [8960/12665 (71%)]\tLoss: 0.056707\n", + "Train Epoch: 2 [9600/12665 (76%)]\tLoss: 0.005973\n", + "Train Epoch: 2 [10240/12665 (81%)]\tLoss: 0.008020\n", + "Train Epoch: 2 [10880/12665 (86%)]\tLoss: 0.009416\n", + "Train Epoch: 2 [11520/12665 (91%)]\tLoss: 0.017660\n", + "Train Epoch: 2 [12160/12665 (96%)]\tLoss: 0.010692\n", + "\n", + "Test set: Average loss: 8.1468\n", + "Class 0 accuracy: 977.0/980(100%)\n", "Class 1 accuracy: 1134.0/1135(100%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", @@ -1172,29 +568,29 @@ "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 3 [0/12665 (0%)]\tLoss: 0.367619\n", - "Train Epoch: 3 [640/12665 (5%)]\tLoss: 0.309399\n", - "Train Epoch: 3 [1280/12665 (10%)]\tLoss: 0.296202\n", - "Train Epoch: 3 [1920/12665 (15%)]\tLoss: 0.340773\n", - "Train Epoch: 3 [2560/12665 (20%)]\tLoss: 0.293039\n", - "Train Epoch: 3 [3200/12665 (25%)]\tLoss: 0.288575\n", - "Train Epoch: 3 [3840/12665 (30%)]\tLoss: 0.285356\n", - "Train Epoch: 3 [4480/12665 (35%)]\tLoss: 0.283475\n", - "Train Epoch: 3 [5120/12665 (40%)]\tLoss: 0.285720\n", - "Train Epoch: 3 [5760/12665 (45%)]\tLoss: 0.245976\n", - "Train Epoch: 3 [6400/12665 (51%)]\tLoss: 0.249808\n", - "Train Epoch: 3 [7040/12665 (56%)]\tLoss: 0.239022\n", - "Train Epoch: 3 [7680/12665 (61%)]\tLoss: 0.268774\n", - "Train Epoch: 3 [8320/12665 (66%)]\tLoss: 0.223297\n", - "Train Epoch: 3 [8960/12665 (71%)]\tLoss: 0.236870\n", - "Train Epoch: 3 [9600/12665 (76%)]\tLoss: 0.222967\n", - "Train Epoch: 3 [10240/12665 (81%)]\tLoss: 0.242883\n", - "Train Epoch: 3 [10880/12665 (86%)]\tLoss: 0.208523\n", - "Train Epoch: 3 [11520/12665 (91%)]\tLoss: 0.191585\n", - "Train Epoch: 3 [12160/12665 (96%)]\tLoss: 0.206234\n", - "\n", - "Test set: Average loss: 4.2921\n", - "Class 0 accuracy: 971.0/980(99%)\n", + "Train Epoch: 3 [0/12665 (0%)]\tLoss: 0.016127\n", + "Train Epoch: 3 [640/12665 (5%)]\tLoss: 0.018997\n", + "Train Epoch: 3 [1280/12665 (10%)]\tLoss: 0.004281\n", + "Train Epoch: 3 [1920/12665 (15%)]\tLoss: 0.004884\n", + "Train Epoch: 3 [2560/12665 (20%)]\tLoss: 0.006038\n", + "Train Epoch: 3 [3200/12665 (25%)]\tLoss: 0.012006\n", + "Train Epoch: 3 [3840/12665 (30%)]\tLoss: 0.009289\n", + "Train Epoch: 3 [4480/12665 (35%)]\tLoss: 0.002473\n", + "Train Epoch: 3 [5120/12665 (40%)]\tLoss: 0.002702\n", + "Train Epoch: 3 [5760/12665 (45%)]\tLoss: 0.012154\n", + "Train Epoch: 3 [6400/12665 (51%)]\tLoss: 0.018935\n", + "Train Epoch: 3 [7040/12665 (56%)]\tLoss: 0.004237\n", + "Train Epoch: 3 [7680/12665 (61%)]\tLoss: 0.003406\n", + "Train Epoch: 3 [8320/12665 (66%)]\tLoss: 0.008378\n", + "Train Epoch: 3 [8960/12665 (71%)]\tLoss: 0.003461\n", + "Train Epoch: 3 [9600/12665 (76%)]\tLoss: 0.018804\n", + "Train Epoch: 3 [10240/12665 (81%)]\tLoss: 0.015821\n", + "Train Epoch: 3 [10880/12665 (86%)]\tLoss: 0.003116\n", + "Train Epoch: 3 [11520/12665 (91%)]\tLoss: 0.002931\n", + "Train Epoch: 3 [12160/12665 (96%)]\tLoss: 0.004047\n", + "\n", + "Test set: Average loss: 8.5887\n", + "Class 0 accuracy: 977.0/980(100%)\n", "Class 1 accuracy: 1134.0/1135(100%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", @@ -1204,493 +600,243 @@ "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 4 [0/12665 (0%)]\tLoss: 0.222183\n", - "Train Epoch: 4 [640/12665 (5%)]\tLoss: 0.175392\n", - "Train Epoch: 4 [1280/12665 (10%)]\tLoss: 0.230008\n", - "Train Epoch: 4 [1920/12665 (15%)]\tLoss: 0.246189\n", - "Train Epoch: 4 [2560/12665 (20%)]\tLoss: 0.224284\n", - "Train Epoch: 4 [3200/12665 (25%)]\tLoss: 0.172705\n", - "Train Epoch: 4 [3840/12665 (30%)]\tLoss: 0.188962\n", - "Train Epoch: 4 [4480/12665 (35%)]\tLoss: 0.212664\n", - "Train Epoch: 4 [5120/12665 (40%)]\tLoss: 0.197847\n", - "Train Epoch: 4 [5760/12665 (45%)]\tLoss: 0.161129\n", - "Train Epoch: 4 [6400/12665 (51%)]\tLoss: 0.160840\n", - "Train Epoch: 4 [7040/12665 (56%)]\tLoss: 0.193788\n", - "Train Epoch: 4 [7680/12665 (61%)]\tLoss: 0.150140\n", - "Train Epoch: 4 [8320/12665 (66%)]\tLoss: 0.193050\n", - "Train Epoch: 4 [8960/12665 (71%)]\tLoss: 0.173765\n", - "Train Epoch: 4 [9600/12665 (76%)]\tLoss: 0.170651\n", - "Train Epoch: 4 [10240/12665 (81%)]\tLoss: 0.170728\n", - "Train Epoch: 4 [10880/12665 (86%)]\tLoss: 0.179045\n", - "Train Epoch: 4 [11520/12665 (91%)]\tLoss: 0.156496\n", - "Train Epoch: 4 [12160/12665 (96%)]\tLoss: 0.174930\n", - "\n", - "Test set: Average loss: 4.6719\n", - "Class 0 accuracy: 972.0/980(99%)\n", - "Class 1 accuracy: 1134.0/1135(100%)\n", - "Class 2 accuracy: 0.0/1032(0%)\n", - "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 5 [0/12665 (0%)]\tLoss: 0.150109\n", - "Train Epoch: 5 [640/12665 (5%)]\tLoss: 0.133148\n", - "Train Epoch: 5 [1280/12665 (10%)]\tLoss: 0.143406\n", - "Train Epoch: 5 [1920/12665 (15%)]\tLoss: 0.210281\n", - "Train Epoch: 5 [2560/12665 (20%)]\tLoss: 0.129901\n", - "Train Epoch: 5 [3200/12665 (25%)]\tLoss: 0.140156\n", - "Train Epoch: 5 [3840/12665 (30%)]\tLoss: 0.127928\n", - "Train Epoch: 5 [4480/12665 (35%)]\tLoss: 0.153185\n", - "Train Epoch: 5 [5120/12665 (40%)]\tLoss: 0.132688\n", - "Train Epoch: 5 [5760/12665 (45%)]\tLoss: 0.174982\n", - "Train Epoch: 5 [6400/12665 (51%)]\tLoss: 0.174245\n", - "Train Epoch: 5 [7040/12665 (56%)]\tLoss: 0.147878\n", - "Train Epoch: 5 [7680/12665 (61%)]\tLoss: 0.105936\n", - "Train Epoch: 5 [8320/12665 (66%)]\tLoss: 0.139824\n", - "Train Epoch: 5 [8960/12665 (71%)]\tLoss: 0.128497\n", - "Train Epoch: 5 [9600/12665 (76%)]\tLoss: 0.102555\n", - "Train Epoch: 5 [10240/12665 (81%)]\tLoss: 0.149713\n", - "Train Epoch: 5 [10880/12665 (86%)]\tLoss: 0.204445\n", - "Train Epoch: 5 [11520/12665 (91%)]\tLoss: 0.173781\n", - "Train Epoch: 5 [12160/12665 (96%)]\tLoss: 0.104449\n", - "\n", - "Test set: Average loss: 4.9040\n", - "Class 0 accuracy: 972.0/980(99%)\n", - "Class 1 accuracy: 1134.0/1135(100%)\n", - "Class 2 accuracy: 0.0/1032(0%)\n", - "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 1 [0/12089 (0%)]\tLoss: 7.002198\n", - "Train Epoch: 1 [640/12089 (5%)]\tLoss: 6.786501\n", - "Train Epoch: 1 [1280/12089 (11%)]\tLoss: 6.051014\n", - "Train Epoch: 1 [1920/12089 (16%)]\tLoss: 6.422165\n", - "Train Epoch: 1 [2560/12089 (21%)]\tLoss: 5.702563\n", - "Train Epoch: 1 [3200/12089 (26%)]\tLoss: 5.380751\n", - "Train Epoch: 1 [3840/12089 (32%)]\tLoss: 4.846919\n", - "Train Epoch: 1 [4480/12089 (37%)]\tLoss: 4.511275\n", - "Train Epoch: 1 [5120/12089 (42%)]\tLoss: 4.290347\n", - "Train Epoch: 1 [5760/12089 (48%)]\tLoss: 3.839983\n", - "Train Epoch: 1 [6400/12089 (53%)]\tLoss: 3.078847\n", - "Train Epoch: 1 [7040/12089 (58%)]\tLoss: 2.839584\n", - "Train Epoch: 1 [7680/12089 (63%)]\tLoss: 2.448348\n", - "Train Epoch: 1 [8320/12089 (69%)]\tLoss: 2.193519\n", - "Train Epoch: 1 [8960/12089 (74%)]\tLoss: 1.988640\n", - "Train Epoch: 1 [9600/12089 (79%)]\tLoss: 1.651685\n", - "Train Epoch: 1 [10240/12089 (85%)]\tLoss: 1.662965\n", - "Train Epoch: 1 [10880/12089 (90%)]\tLoss: 1.390365\n", - "Train Epoch: 1 [11520/12089 (95%)]\tLoss: 1.214716\n", - "\n", - "Test set: Average loss: 2.9396\n", - "Class 0 accuracy: 554.0/980(57%)\n", - "Class 1 accuracy: 1033.0/1135(91%)\n", - "Class 2 accuracy: 387.0/1032(38%)\n", - "Class 3 accuracy: 1008.0/1010(100%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 2 [0/12089 (0%)]\tLoss: 1.242918\n", - "Train Epoch: 2 [640/12089 (5%)]\tLoss: 1.003339\n", - "Train Epoch: 2 [1280/12089 (11%)]\tLoss: 0.984789\n", - "Train Epoch: 2 [1920/12089 (16%)]\tLoss: 0.937961\n", - "Train Epoch: 2 [2560/12089 (21%)]\tLoss: 0.883540\n", - "Train Epoch: 2 [3200/12089 (26%)]\tLoss: 0.840368\n", - "Train Epoch: 2 [3840/12089 (32%)]\tLoss: 0.937714\n", - "Train Epoch: 2 [4480/12089 (37%)]\tLoss: 0.909657\n", - "Train Epoch: 2 [5120/12089 (42%)]\tLoss: 0.840488\n", - "Train Epoch: 2 [5760/12089 (48%)]\tLoss: 0.763511\n", - "Train Epoch: 2 [6400/12089 (53%)]\tLoss: 0.767492\n", - "Train Epoch: 2 [7040/12089 (58%)]\tLoss: 0.721862\n", - "Train Epoch: 2 [7680/12089 (63%)]\tLoss: 0.698475\n", - "Train Epoch: 2 [8320/12089 (69%)]\tLoss: 0.630833\n", - "Train Epoch: 2 [8960/12089 (74%)]\tLoss: 0.732636\n", - "Train Epoch: 2 [9600/12089 (79%)]\tLoss: 0.686765\n", - "Train Epoch: 2 [10240/12089 (85%)]\tLoss: 0.647560\n", - "Train Epoch: 2 [10880/12089 (90%)]\tLoss: 0.584798\n", - "Train Epoch: 2 [11520/12089 (95%)]\tLoss: 0.605022\n", - "\n", - "Test set: Average loss: 3.8344\n", + "Train Epoch: 1 [0/12089 (0%)]\tLoss: 10.857133\n", + "Train Epoch: 1 [640/12089 (5%)]\tLoss: 6.280286\n", + "Train Epoch: 1 [1280/12089 (11%)]\tLoss: 2.301530\n", + "Train Epoch: 1 [1920/12089 (16%)]\tLoss: 0.905669\n", + "Train Epoch: 1 [2560/12089 (21%)]\tLoss: 0.688727\n", + "Train Epoch: 1 [3200/12089 (26%)]\tLoss: 0.598405\n", + "Train Epoch: 1 [3840/12089 (32%)]\tLoss: 0.462726\n", + "Train Epoch: 1 [4480/12089 (37%)]\tLoss: 0.345897\n", + "Train Epoch: 1 [5120/12089 (42%)]\tLoss: 0.380626\n", + "Train Epoch: 1 [5760/12089 (48%)]\tLoss: 0.280428\n", + "Train Epoch: 1 [6400/12089 (53%)]\tLoss: 0.216660\n", + "Train Epoch: 1 [7040/12089 (58%)]\tLoss: 0.167003\n", + "Train Epoch: 1 [7680/12089 (63%)]\tLoss: 0.275916\n", + "Train Epoch: 1 [8320/12089 (69%)]\tLoss: 0.326823\n", + "Train Epoch: 1 [8960/12089 (74%)]\tLoss: 0.258519\n", + "Train Epoch: 1 [9600/12089 (79%)]\tLoss: 0.233105\n", + "Train Epoch: 1 [10240/12089 (85%)]\tLoss: 0.235713\n", + "Train Epoch: 1 [10880/12089 (90%)]\tLoss: 0.265890\n", + "Train Epoch: 1 [11520/12089 (95%)]\tLoss: 0.262226\n", + "\n", + "Test set: Average loss: 6.2382\n", "Class 0 accuracy: 0.0/980(0%)\n", "Class 1 accuracy: 0.0/1135(0%)\n", - "Class 2 accuracy: 889.0/1032(86%)\n", - "Class 3 accuracy: 985.0/1010(98%)\n", + "Class 2 accuracy: 992.0/1032(96%)\n", + "Class 3 accuracy: 987.0/1010(98%)\n", "Class 4 accuracy: 0.0/982(0%)\n", "Class 5 accuracy: 0.0/892(0%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 3 [0/12089 (0%)]\tLoss: 0.633021\n", - "Train Epoch: 3 [640/12089 (5%)]\tLoss: 0.661086\n", - "Train Epoch: 3 [1280/12089 (11%)]\tLoss: 0.531795\n", - "Train Epoch: 3 [1920/12089 (16%)]\tLoss: 0.595388\n", - "Train Epoch: 3 [2560/12089 (21%)]\tLoss: 0.484488\n", - "Train Epoch: 3 [3200/12089 (26%)]\tLoss: 0.610725\n", - "Train Epoch: 3 [3840/12089 (32%)]\tLoss: 0.539676\n", - "Train Epoch: 3 [4480/12089 (37%)]\tLoss: 0.555348\n", - "Train Epoch: 3 [5120/12089 (42%)]\tLoss: 0.475156\n", - "Train Epoch: 3 [5760/12089 (48%)]\tLoss: 0.532515\n", - "Train Epoch: 3 [6400/12089 (53%)]\tLoss: 0.561437\n", - "Train Epoch: 3 [7040/12089 (58%)]\tLoss: 0.513701\n", - "Train Epoch: 3 [7680/12089 (63%)]\tLoss: 0.469366\n", - "Train Epoch: 3 [8320/12089 (69%)]\tLoss: 0.517332\n", - "Train Epoch: 3 [8960/12089 (74%)]\tLoss: 0.443565\n", - "Train Epoch: 3 [9600/12089 (79%)]\tLoss: 0.427636\n", - "Train Epoch: 3 [10240/12089 (85%)]\tLoss: 0.427255\n", - "Train Epoch: 3 [10880/12089 (90%)]\tLoss: 0.556653\n", - "Train Epoch: 3 [11520/12089 (95%)]\tLoss: 0.481399\n", - "\n", - "Test set: Average loss: 4.3145\n", + "Train Epoch: 2 [0/12089 (0%)]\tLoss: 0.099378\n", + "Train Epoch: 2 [640/12089 (5%)]\tLoss: 0.116401\n", + "Train Epoch: 2 [1280/12089 (11%)]\tLoss: 0.155221\n", + "Train Epoch: 2 [1920/12089 (16%)]\tLoss: 0.109446\n", + "Train Epoch: 2 [2560/12089 (21%)]\tLoss: 0.104514\n", + "Train Epoch: 2 [3200/12089 (26%)]\tLoss: 0.219575\n", + "Train Epoch: 2 [3840/12089 (32%)]\tLoss: 0.151700\n", + "Train Epoch: 2 [4480/12089 (37%)]\tLoss: 0.162770\n", + "Train Epoch: 2 [5120/12089 (42%)]\tLoss: 0.152628\n", + "Train Epoch: 2 [5760/12089 (48%)]\tLoss: 0.186006\n", + "Train Epoch: 2 [6400/12089 (53%)]\tLoss: 0.095261\n", + "Train Epoch: 2 [7040/12089 (58%)]\tLoss: 0.152865\n", + "Train Epoch: 2 [7680/12089 (63%)]\tLoss: 0.046197\n", + "Train Epoch: 2 [8320/12089 (69%)]\tLoss: 0.068629\n", + "Train Epoch: 2 [8960/12089 (74%)]\tLoss: 0.133037\n", + "Train Epoch: 2 [9600/12089 (79%)]\tLoss: 0.106119\n", + "Train Epoch: 2 [10240/12089 (85%)]\tLoss: 0.073823\n", + "Train Epoch: 2 [10880/12089 (90%)]\tLoss: 0.117314\n", + "Train Epoch: 2 [11520/12089 (95%)]\tLoss: 0.144778\n", + "\n", + "Test set: Average loss: 7.2467\n", "Class 0 accuracy: 0.0/980(0%)\n", "Class 1 accuracy: 0.0/1135(0%)\n", - "Class 2 accuracy: 945.0/1032(92%)\n", - "Class 3 accuracy: 967.0/1010(96%)\n", + "Class 2 accuracy: 998.0/1032(97%)\n", + "Class 3 accuracy: 994.0/1010(98%)\n", "Class 4 accuracy: 0.0/982(0%)\n", "Class 5 accuracy: 0.0/892(0%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 4 [0/12089 (0%)]\tLoss: 0.505337\n", - "Train Epoch: 4 [640/12089 (5%)]\tLoss: 0.464085\n", - "Train Epoch: 4 [1280/12089 (11%)]\tLoss: 0.518291\n", - "Train Epoch: 4 [1920/12089 (16%)]\tLoss: 0.452366\n", - "Train Epoch: 4 [2560/12089 (21%)]\tLoss: 0.405918\n", - "Train Epoch: 4 [3200/12089 (26%)]\tLoss: 0.502257\n", - "Train Epoch: 4 [3840/12089 (32%)]\tLoss: 0.448973\n", - "Train Epoch: 4 [4480/12089 (37%)]\tLoss: 0.445046\n", - "Train Epoch: 4 [5120/12089 (42%)]\tLoss: 0.495061\n", - "Train Epoch: 4 [5760/12089 (48%)]\tLoss: 0.456786\n", - "Train Epoch: 4 [6400/12089 (53%)]\tLoss: 0.371738\n", - "Train Epoch: 4 [7040/12089 (58%)]\tLoss: 0.468638\n", - "Train Epoch: 4 [7680/12089 (63%)]\tLoss: 0.495899\n", - "Train Epoch: 4 [8320/12089 (69%)]\tLoss: 0.453645\n", - "Train Epoch: 4 [8960/12089 (74%)]\tLoss: 0.479238\n", - "Train Epoch: 4 [9600/12089 (79%)]\tLoss: 0.458163\n", - "Train Epoch: 4 [10240/12089 (85%)]\tLoss: 0.446410\n", - "Train Epoch: 4 [10880/12089 (90%)]\tLoss: 0.441910\n", - "Train Epoch: 4 [11520/12089 (95%)]\tLoss: 0.430284\n", - "\n", - "Test set: Average loss: 4.5708\n", + "Train Epoch: 3 [0/12089 (0%)]\tLoss: 0.102792\n", + "Train Epoch: 3 [640/12089 (5%)]\tLoss: 0.209812\n", + "Train Epoch: 3 [1280/12089 (11%)]\tLoss: 0.079453\n", + "Train Epoch: 3 [1920/12089 (16%)]\tLoss: 0.146311\n", + "Train Epoch: 3 [2560/12089 (21%)]\tLoss: 0.057011\n", + "Train Epoch: 3 [3200/12089 (26%)]\tLoss: 0.062497\n", + "Train Epoch: 3 [3840/12089 (32%)]\tLoss: 0.114005\n", + "Train Epoch: 3 [4480/12089 (37%)]\tLoss: 0.089880\n", + "Train Epoch: 3 [5120/12089 (42%)]\tLoss: 0.057159\n", + "Train Epoch: 3 [5760/12089 (48%)]\tLoss: 0.117361\n", + "Train Epoch: 3 [6400/12089 (53%)]\tLoss: 0.098925\n", + "Train Epoch: 3 [7040/12089 (58%)]\tLoss: 0.069072\n", + "Train Epoch: 3 [7680/12089 (63%)]\tLoss: 0.111345\n", + "Train Epoch: 3 [8320/12089 (69%)]\tLoss: 0.131783\n", + "Train Epoch: 3 [8960/12089 (74%)]\tLoss: 0.150885\n", + "Train Epoch: 3 [9600/12089 (79%)]\tLoss: 0.079096\n", + "Train Epoch: 3 [10240/12089 (85%)]\tLoss: 0.082054\n", + "Train Epoch: 3 [10880/12089 (90%)]\tLoss: 0.129982\n", + "Train Epoch: 3 [11520/12089 (95%)]\tLoss: 0.066340\n", + "\n", + "Test set: Average loss: 7.6576\n", "Class 0 accuracy: 0.0/980(0%)\n", "Class 1 accuracy: 0.0/1135(0%)\n", - "Class 2 accuracy: 952.0/1032(92%)\n", - "Class 3 accuracy: 969.0/1010(96%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 5 [0/12089 (0%)]\tLoss: 0.405303\n", - "Train Epoch: 5 [640/12089 (5%)]\tLoss: 0.396980\n", - "Train Epoch: 5 [1280/12089 (11%)]\tLoss: 0.421218\n", - "Train Epoch: 5 [1920/12089 (16%)]\tLoss: 0.459768\n", - "Train Epoch: 5 [2560/12089 (21%)]\tLoss: 0.354826\n", - "Train Epoch: 5 [3200/12089 (26%)]\tLoss: 0.372944\n", - "Train Epoch: 5 [3840/12089 (32%)]\tLoss: 0.394097\n", - "Train Epoch: 5 [4480/12089 (37%)]\tLoss: 0.444731\n", - "Train Epoch: 5 [5120/12089 (42%)]\tLoss: 0.470775\n", - "Train Epoch: 5 [5760/12089 (48%)]\tLoss: 0.414834\n", - "Train Epoch: 5 [6400/12089 (53%)]\tLoss: 0.451856\n", - "Train Epoch: 5 [7040/12089 (58%)]\tLoss: 0.390931\n", - "Train Epoch: 5 [7680/12089 (63%)]\tLoss: 0.392495\n", - "Train Epoch: 5 [8320/12089 (69%)]\tLoss: 0.341354\n", - "Train Epoch: 5 [8960/12089 (74%)]\tLoss: 0.355052\n", - "Train Epoch: 5 [9600/12089 (79%)]\tLoss: 0.414375\n", - "Train Epoch: 5 [10240/12089 (85%)]\tLoss: 0.419718\n", - "Train Epoch: 5 [10880/12089 (90%)]\tLoss: 0.367832\n", - "Train Epoch: 5 [11520/12089 (95%)]\tLoss: 0.358470\n", - "\n", - "Test set: Average loss: 4.7194\n", - "Class 0 accuracy: 0.0/980(0%)\n", - "Class 1 accuracy: 0.0/1135(0%)\n", - "Class 2 accuracy: 953.0/1032(92%)\n", - "Class 3 accuracy: 970.0/1010(96%)\n", + "Class 2 accuracy: 1007.0/1032(98%)\n", + "Class 3 accuracy: 985.0/1010(98%)\n", "Class 4 accuracy: 0.0/982(0%)\n", "Class 5 accuracy: 0.0/892(0%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 1 [0/11263 (0%)]\tLoss: 6.261249\n", - "Train Epoch: 1 [640/11263 (6%)]\tLoss: 5.999681\n", - "Train Epoch: 1 [1280/11263 (11%)]\tLoss: 5.607321\n", - "Train Epoch: 1 [1920/11263 (17%)]\tLoss: 5.542882\n", - "Train Epoch: 1 [2560/11263 (23%)]\tLoss: 4.963273\n", - "Train Epoch: 1 [3200/11263 (28%)]\tLoss: 4.750457\n", - "Train Epoch: 1 [3840/11263 (34%)]\tLoss: 4.334918\n", - "Train Epoch: 1 [4480/11263 (40%)]\tLoss: 3.912877\n", - "Train Epoch: 1 [5120/11263 (45%)]\tLoss: 3.827431\n", - "Train Epoch: 1 [5760/11263 (51%)]\tLoss: 3.393839\n", - "Train Epoch: 1 [6400/11263 (57%)]\tLoss: 3.033727\n", - "Train Epoch: 1 [7040/11263 (62%)]\tLoss: 2.671916\n", - "Train Epoch: 1 [7680/11263 (68%)]\tLoss: 2.473807\n", - "Train Epoch: 1 [8320/11263 (74%)]\tLoss: 2.285081\n", - "Train Epoch: 1 [8960/11263 (80%)]\tLoss: 1.972579\n", - "Train Epoch: 1 [9600/11263 (85%)]\tLoss: 1.953343\n", - "Train Epoch: 1 [10240/11263 (91%)]\tLoss: 1.653112\n", - "Train Epoch: 1 [10880/11263 (97%)]\tLoss: 1.666049\n", - "\n", - "Test set: Average loss: 2.6135\n", - "Class 0 accuracy: 0.0/980(0%)\n", - "Class 1 accuracy: 242.0/1135(21%)\n", - "Class 2 accuracy: 341.0/1032(33%)\n", - "Class 3 accuracy: 431.0/1010(43%)\n", - "Class 4 accuracy: 982.0/982(100%)\n", - "Class 5 accuracy: 11.0/892(1%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 2 [0/11263 (0%)]\tLoss: 1.265681\n", - "Train Epoch: 2 [640/11263 (6%)]\tLoss: 1.242094\n", - "Train Epoch: 2 [1280/11263 (11%)]\tLoss: 1.308110\n", - "Train Epoch: 2 [1920/11263 (17%)]\tLoss: 1.304853\n", - "Train Epoch: 2 [2560/11263 (23%)]\tLoss: 1.242684\n", - "Train Epoch: 2 [3200/11263 (28%)]\tLoss: 1.155575\n", - "Train Epoch: 2 [3840/11263 (34%)]\tLoss: 1.025319\n", - "Train Epoch: 2 [4480/11263 (40%)]\tLoss: 1.151237\n", - "Train Epoch: 2 [5120/11263 (45%)]\tLoss: 1.071076\n", - "Train Epoch: 2 [5760/11263 (51%)]\tLoss: 0.978908\n", - "Train Epoch: 2 [6400/11263 (57%)]\tLoss: 0.869457\n", - "Train Epoch: 2 [7040/11263 (62%)]\tLoss: 0.841398\n", - "Train Epoch: 2 [7680/11263 (68%)]\tLoss: 0.893406\n", - "Train Epoch: 2 [8320/11263 (74%)]\tLoss: 0.835585\n", - "Train Epoch: 2 [8960/11263 (80%)]\tLoss: 0.868128\n", - "Train Epoch: 2 [9600/11263 (85%)]\tLoss: 0.763401\n", - "Train Epoch: 2 [10240/11263 (91%)]\tLoss: 0.905359\n", - "Train Epoch: 2 [10880/11263 (97%)]\tLoss: 0.842730\n", - "\n", - "Test set: Average loss: 3.4385\n", + "Train Epoch: 1 [0/11263 (0%)]\tLoss: 10.835564\n", + "Train Epoch: 1 [640/11263 (6%)]\tLoss: 5.729345\n", + "Train Epoch: 1 [1280/11263 (11%)]\tLoss: 2.635014\n", + "Train Epoch: 1 [1920/11263 (17%)]\tLoss: 1.219099\n", + "Train Epoch: 1 [2560/11263 (23%)]\tLoss: 0.913658\n", + "Train Epoch: 1 [3200/11263 (28%)]\tLoss: 0.655098\n", + "Train Epoch: 1 [3840/11263 (34%)]\tLoss: 0.466011\n", + "Train Epoch: 1 [4480/11263 (40%)]\tLoss: 0.539344\n", + "Train Epoch: 1 [5120/11263 (45%)]\tLoss: 0.249651\n", + "Train Epoch: 1 [5760/11263 (51%)]\tLoss: 0.209131\n", + "Train Epoch: 1 [6400/11263 (57%)]\tLoss: 0.223062\n", + "Train Epoch: 1 [7040/11263 (62%)]\tLoss: 0.228752\n", + "Train Epoch: 1 [7680/11263 (68%)]\tLoss: 0.198413\n", + "Train Epoch: 1 [8320/11263 (74%)]\tLoss: 0.215570\n", + "Train Epoch: 1 [8960/11263 (80%)]\tLoss: 0.272983\n", + "Train Epoch: 1 [9600/11263 (85%)]\tLoss: 0.155609\n", + "Train Epoch: 1 [10240/11263 (91%)]\tLoss: 0.087590\n", + "Train Epoch: 1 [10880/11263 (97%)]\tLoss: 0.132957\n", + "\n", + "Test set: Average loss: 7.1306\n", "Class 0 accuracy: 0.0/980(0%)\n", "Class 1 accuracy: 0.0/1135(0%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 979.0/982(100%)\n", - "Class 5 accuracy: 808.0/892(91%)\n", + "Class 4 accuracy: 977.0/982(99%)\n", + "Class 5 accuracy: 866.0/892(97%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 3 [0/11263 (0%)]\tLoss: 0.688606\n", - "Train Epoch: 3 [640/11263 (6%)]\tLoss: 0.747074\n", - "Train Epoch: 3 [1280/11263 (11%)]\tLoss: 0.699562\n", - "Train Epoch: 3 [1920/11263 (17%)]\tLoss: 0.687192\n", - "Train Epoch: 3 [2560/11263 (23%)]\tLoss: 0.694984\n", - "Train Epoch: 3 [3200/11263 (28%)]\tLoss: 0.643804\n", - "Train Epoch: 3 [3840/11263 (34%)]\tLoss: 0.637270\n", - "Train Epoch: 3 [4480/11263 (40%)]\tLoss: 0.776547\n", - "Train Epoch: 3 [5120/11263 (45%)]\tLoss: 0.624326\n", - "Train Epoch: 3 [5760/11263 (51%)]\tLoss: 0.763693\n", - "Train Epoch: 3 [6400/11263 (57%)]\tLoss: 0.619411\n", - "Train Epoch: 3 [7040/11263 (62%)]\tLoss: 0.672091\n", - "Train Epoch: 3 [7680/11263 (68%)]\tLoss: 0.640160\n", - "Train Epoch: 3 [8320/11263 (74%)]\tLoss: 0.594804\n", - "Train Epoch: 3 [8960/11263 (80%)]\tLoss: 0.712096\n", - "Train Epoch: 3 [9600/11263 (85%)]\tLoss: 0.584561\n", - "Train Epoch: 3 [10240/11263 (91%)]\tLoss: 0.490905\n", - "Train Epoch: 3 [10880/11263 (97%)]\tLoss: 0.511629\n", - "\n", - "Test set: Average loss: 3.9981\n", + "Train Epoch: 2 [0/11263 (0%)]\tLoss: 0.130704\n", + "Train Epoch: 2 [640/11263 (6%)]\tLoss: 0.123305\n", + "Train Epoch: 2 [1280/11263 (11%)]\tLoss: 0.092606\n", + "Train Epoch: 2 [1920/11263 (17%)]\tLoss: 0.108018\n", + "Train Epoch: 2 [2560/11263 (23%)]\tLoss: 0.059542\n", + "Train Epoch: 2 [3200/11263 (28%)]\tLoss: 0.051525\n", + "Train Epoch: 2 [3840/11263 (34%)]\tLoss: 0.105634\n", + "Train Epoch: 2 [4480/11263 (40%)]\tLoss: 0.075492\n", + "Train Epoch: 2 [5120/11263 (45%)]\tLoss: 0.056117\n", + "Train Epoch: 2 [5760/11263 (51%)]\tLoss: 0.119122\n", + "Train Epoch: 2 [6400/11263 (57%)]\tLoss: 0.098337\n", + "Train Epoch: 2 [7040/11263 (62%)]\tLoss: 0.049512\n", + "Train Epoch: 2 [7680/11263 (68%)]\tLoss: 0.070381\n", + "Train Epoch: 2 [8320/11263 (74%)]\tLoss: 0.070122\n", + "Train Epoch: 2 [8960/11263 (80%)]\tLoss: 0.074574\n", + "Train Epoch: 2 [9600/11263 (85%)]\tLoss: 0.085832\n", + "Train Epoch: 2 [10240/11263 (91%)]\tLoss: 0.194243\n", + "Train Epoch: 2 [10880/11263 (97%)]\tLoss: 0.052719\n", + "\n", + "Test set: Average loss: 8.4438\n", "Class 0 accuracy: 0.0/980(0%)\n", "Class 1 accuracy: 0.0/1135(0%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 975.0/982(99%)\n", - "Class 5 accuracy: 837.0/892(94%)\n", + "Class 4 accuracy: 978.0/982(100%)\n", + "Class 5 accuracy: 872.0/892(98%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 4 [0/11263 (0%)]\tLoss: 0.560350\n", - "Train Epoch: 4 [640/11263 (6%)]\tLoss: 0.549644\n", - "Train Epoch: 4 [1280/11263 (11%)]\tLoss: 0.548100\n", - "Train Epoch: 4 [1920/11263 (17%)]\tLoss: 0.523234\n", - "Train Epoch: 4 [2560/11263 (23%)]\tLoss: 0.563891\n", - "Train Epoch: 4 [3200/11263 (28%)]\tLoss: 0.462694\n", - "Train Epoch: 4 [3840/11263 (34%)]\tLoss: 0.562608\n", - "Train Epoch: 4 [4480/11263 (40%)]\tLoss: 0.446004\n", - "Train Epoch: 4 [5120/11263 (45%)]\tLoss: 0.511273\n", - "Train Epoch: 4 [5760/11263 (51%)]\tLoss: 0.548768\n", - "Train Epoch: 4 [6400/11263 (57%)]\tLoss: 0.529422\n", - "Train Epoch: 4 [7040/11263 (62%)]\tLoss: 0.544079\n", - "Train Epoch: 4 [7680/11263 (68%)]\tLoss: 0.428724\n", - "Train Epoch: 4 [8320/11263 (74%)]\tLoss: 0.600822\n", - "Train Epoch: 4 [8960/11263 (80%)]\tLoss: 0.474362\n", - "Train Epoch: 4 [9600/11263 (85%)]\tLoss: 0.595750\n", - "Train Epoch: 4 [10240/11263 (91%)]\tLoss: 0.544630\n", - "Train Epoch: 4 [10880/11263 (97%)]\tLoss: 0.506346\n", - "\n", - "Test set: Average loss: 4.3219\n", + "Train Epoch: 3 [0/11263 (0%)]\tLoss: 0.031168\n", + "Train Epoch: 3 [640/11263 (6%)]\tLoss: 0.133256\n", + "Train Epoch: 3 [1280/11263 (11%)]\tLoss: 0.064342\n", + "Train Epoch: 3 [1920/11263 (17%)]\tLoss: 0.081790\n", + "Train Epoch: 3 [2560/11263 (23%)]\tLoss: 0.055577\n", + "Train Epoch: 3 [3200/11263 (28%)]\tLoss: 0.031944\n", + "Train Epoch: 3 [3840/11263 (34%)]\tLoss: 0.079894\n", + "Train Epoch: 3 [4480/11263 (40%)]\tLoss: 0.054432\n", + "Train Epoch: 3 [5120/11263 (45%)]\tLoss: 0.032932\n", + "Train Epoch: 3 [5760/11263 (51%)]\tLoss: 0.054515\n", + "Train Epoch: 3 [6400/11263 (57%)]\tLoss: 0.015534\n", + "Train Epoch: 3 [7040/11263 (62%)]\tLoss: 0.073943\n", + "Train Epoch: 3 [7680/11263 (68%)]\tLoss: 0.051291\n", + "Train Epoch: 3 [8320/11263 (74%)]\tLoss: 0.071271\n", + "Train Epoch: 3 [8960/11263 (80%)]\tLoss: 0.078468\n", + "Train Epoch: 3 [9600/11263 (85%)]\tLoss: 0.074505\n", + "Train Epoch: 3 [10240/11263 (91%)]\tLoss: 0.041854\n", + "Train Epoch: 3 [10880/11263 (97%)]\tLoss: 0.051243\n", + "\n", + "Test set: Average loss: 9.0650\n", "Class 0 accuracy: 0.0/980(0%)\n", "Class 1 accuracy: 0.0/1135(0%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 974.0/982(99%)\n", - "Class 5 accuracy: 841.0/892(94%)\n", + "Class 4 accuracy: 978.0/982(100%)\n", + "Class 5 accuracy: 879.0/892(99%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 5 [0/11263 (0%)]\tLoss: 0.355252\n", - "Train Epoch: 5 [640/11263 (6%)]\tLoss: 0.517941\n", - "Train Epoch: 5 [1280/11263 (11%)]\tLoss: 0.526465\n", - "Train Epoch: 5 [1920/11263 (17%)]\tLoss: 0.523074\n", - "Train Epoch: 5 [2560/11263 (23%)]\tLoss: 0.433639\n", - "Train Epoch: 5 [3200/11263 (28%)]\tLoss: 0.434077\n", - "Train Epoch: 5 [3840/11263 (34%)]\tLoss: 0.403323\n", - "Train Epoch: 5 [4480/11263 (40%)]\tLoss: 0.505131\n", - "Train Epoch: 5 [5120/11263 (45%)]\tLoss: 0.355989\n", - "Train Epoch: 5 [5760/11263 (51%)]\tLoss: 0.490564\n", - "Train Epoch: 5 [6400/11263 (57%)]\tLoss: 0.399952\n", - "Train Epoch: 5 [7040/11263 (62%)]\tLoss: 0.436878\n", - "Train Epoch: 5 [7680/11263 (68%)]\tLoss: 0.491910\n", - "Train Epoch: 5 [8320/11263 (74%)]\tLoss: 0.429957\n", - "Train Epoch: 5 [8960/11263 (80%)]\tLoss: 0.495502\n", - "Train Epoch: 5 [9600/11263 (85%)]\tLoss: 0.446051\n", - "Train Epoch: 5 [10240/11263 (91%)]\tLoss: 0.364511\n", - "Train Epoch: 5 [10880/11263 (97%)]\tLoss: 0.447680\n", - "\n", - "Test set: Average loss: 4.5150\n", - "Class 0 accuracy: 0.0/980(0%)\n", - "Class 1 accuracy: 0.0/1135(0%)\n", - "Class 2 accuracy: 0.0/1032(0%)\n", - "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 974.0/982(99%)\n", - "Class 5 accuracy: 843.0/892(95%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 1 [0/12183 (0%)]\tLoss: 7.209232\n", - "Train Epoch: 1 [640/12183 (5%)]\tLoss: 7.299924\n", - "Train Epoch: 1 [1280/12183 (10%)]\tLoss: 6.381386\n", - "Train Epoch: 1 [1920/12183 (16%)]\tLoss: 6.263927\n", - "Train Epoch: 1 [2560/12183 (21%)]\tLoss: 5.731481\n", - "Train Epoch: 1 [3200/12183 (26%)]\tLoss: 5.564906\n", - "Train Epoch: 1 [3840/12183 (31%)]\tLoss: 4.826056\n", - "Train Epoch: 1 [4480/12183 (37%)]\tLoss: 4.433992\n", - "Train Epoch: 1 [5120/12183 (42%)]\tLoss: 4.331242\n", - "Train Epoch: 1 [5760/12183 (47%)]\tLoss: 3.656218\n", - "Train Epoch: 1 [6400/12183 (52%)]\tLoss: 3.236688\n", - "Train Epoch: 1 [7040/12183 (58%)]\tLoss: 2.764972\n", - "Train Epoch: 1 [7680/12183 (63%)]\tLoss: 2.587098\n", - "Train Epoch: 1 [8320/12183 (68%)]\tLoss: 2.090186\n", - "Train Epoch: 1 [8960/12183 (73%)]\tLoss: 2.049698\n", - "Train Epoch: 1 [9600/12183 (79%)]\tLoss: 1.641774\n", - "Train Epoch: 1 [10240/12183 (84%)]\tLoss: 1.447641\n", - "Train Epoch: 1 [10880/12183 (89%)]\tLoss: 1.294460\n", - "Train Epoch: 1 [11520/12183 (94%)]\tLoss: 1.226808\n", - "Train Epoch: 1 [4370/12183 (99%)]\tLoss: 0.847535\n", - "\n", - "Test set: Average loss: 2.5262\n", - "Class 0 accuracy: 0.0/980(0%)\n", - "Class 1 accuracy: 0.0/1135(0%)\n", - "Class 2 accuracy: 0.0/1032(0%)\n", - "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 1.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 957.0/958(100%)\n", - "Class 7 accuracy: 795.0/1028(77%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 2 [0/12183 (0%)]\tLoss: 1.063643\n", - "Train Epoch: 2 [640/12183 (5%)]\tLoss: 0.945048\n", - "Train Epoch: 2 [1280/12183 (10%)]\tLoss: 0.910895\n", - "Train Epoch: 2 [1920/12183 (16%)]\tLoss: 0.803437\n", - "Train Epoch: 2 [2560/12183 (21%)]\tLoss: 0.714711\n", - "Train Epoch: 2 [3200/12183 (26%)]\tLoss: 0.731389\n", - "Train Epoch: 2 [3840/12183 (31%)]\tLoss: 0.697865\n", - "Train Epoch: 2 [4480/12183 (37%)]\tLoss: 0.518302\n", - "Train Epoch: 2 [5120/12183 (42%)]\tLoss: 0.506918\n", - "Train Epoch: 2 [5760/12183 (47%)]\tLoss: 0.638351\n", - "Train Epoch: 2 [6400/12183 (52%)]\tLoss: 0.607182\n", - "Train Epoch: 2 [7040/12183 (58%)]\tLoss: 0.582088\n", - "Train Epoch: 2 [7680/12183 (63%)]\tLoss: 0.526564\n", - "Train Epoch: 2 [8320/12183 (68%)]\tLoss: 0.453763\n", - "Train Epoch: 2 [8960/12183 (73%)]\tLoss: 0.463578\n", - "Train Epoch: 2 [9600/12183 (79%)]\tLoss: 0.505179\n", - "Train Epoch: 2 [10240/12183 (84%)]\tLoss: 0.390394\n", - "Train Epoch: 2 [10880/12183 (89%)]\tLoss: 0.431393\n", - "Train Epoch: 2 [11520/12183 (94%)]\tLoss: 0.422808\n", - "Train Epoch: 2 [4370/12183 (99%)]\tLoss: 0.379329\n", - "\n", - "Test set: Average loss: 3.8944\n", - "Class 0 accuracy: 0.0/980(0%)\n", - "Class 1 accuracy: 0.0/1135(0%)\n", - "Class 2 accuracy: 0.0/1032(0%)\n", - "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 954.0/958(100%)\n", - "Class 7 accuracy: 1005.0/1028(98%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 3 [0/12183 (0%)]\tLoss: 0.411217\n", - "Train Epoch: 3 [640/12183 (5%)]\tLoss: 0.430715\n", - "Train Epoch: 3 [1280/12183 (10%)]\tLoss: 0.321807\n", - "Train Epoch: 3 [1920/12183 (16%)]\tLoss: 0.305609\n", - "Train Epoch: 3 [2560/12183 (21%)]\tLoss: 0.329529\n", - "Train Epoch: 3 [3200/12183 (26%)]\tLoss: 0.344710\n", - "Train Epoch: 3 [3840/12183 (31%)]\tLoss: 0.351153\n", - "Train Epoch: 3 [4480/12183 (37%)]\tLoss: 0.355009\n", - "Train Epoch: 3 [5120/12183 (42%)]\tLoss: 0.316889\n", - "Train Epoch: 3 [5760/12183 (47%)]\tLoss: 0.268590\n", - "Train Epoch: 3 [6400/12183 (52%)]\tLoss: 0.269210\n", - "Train Epoch: 3 [7040/12183 (58%)]\tLoss: 0.272690\n", - "Train Epoch: 3 [7680/12183 (63%)]\tLoss: 0.299527\n", - "Train Epoch: 3 [8320/12183 (68%)]\tLoss: 0.294239\n", - "Train Epoch: 3 [8960/12183 (73%)]\tLoss: 0.256061\n", - "Train Epoch: 3 [9600/12183 (79%)]\tLoss: 0.263354\n", - "Train Epoch: 3 [10240/12183 (84%)]\tLoss: 0.266107\n", - "Train Epoch: 3 [10880/12183 (89%)]\tLoss: 0.301182\n", - "Train Epoch: 3 [11520/12183 (94%)]\tLoss: 0.221701\n", - "Train Epoch: 3 [4370/12183 (99%)]\tLoss: 0.256709\n", - "\n", - "Test set: Average loss: 4.6037\n", + "Train Epoch: 1 [0/12183 (0%)]\tLoss: 12.640953\n", + "Train Epoch: 1 [640/12183 (5%)]\tLoss: 6.948784\n", + "Train Epoch: 1 [1280/12183 (10%)]\tLoss: 2.239110\n", + "Train Epoch: 1 [1920/12183 (16%)]\tLoss: 0.789898\n", + "Train Epoch: 1 [2560/12183 (21%)]\tLoss: 0.433876\n", + "Train Epoch: 1 [3200/12183 (26%)]\tLoss: 0.327010\n", + "Train Epoch: 1 [3840/12183 (31%)]\tLoss: 0.240610\n", + "Train Epoch: 1 [4480/12183 (37%)]\tLoss: 0.189328\n", + "Train Epoch: 1 [5120/12183 (42%)]\tLoss: 0.189419\n", + "Train Epoch: 1 [5760/12183 (47%)]\tLoss: 0.063147\n", + "Train Epoch: 1 [6400/12183 (52%)]\tLoss: 0.061185\n", + "Train Epoch: 1 [7040/12183 (58%)]\tLoss: 0.115183\n", + "Train Epoch: 1 [7680/12183 (63%)]\tLoss: 0.030512\n", + "Train Epoch: 1 [8320/12183 (68%)]\tLoss: 0.037145\n", + "Train Epoch: 1 [8960/12183 (73%)]\tLoss: 0.089052\n", + "Train Epoch: 1 [9600/12183 (79%)]\tLoss: 0.044766\n", + "Train Epoch: 1 [10240/12183 (84%)]\tLoss: 0.027388\n", + "Train Epoch: 1 [10880/12183 (89%)]\tLoss: 0.041345\n", + "Train Epoch: 1 [11520/12183 (94%)]\tLoss: 0.022225\n", + "Train Epoch: 1 [4370/12183 (99%)]\tLoss: 0.004926\n", + "\n", + "Test set: Average loss: 7.6796\n", "Class 0 accuracy: 0.0/980(0%)\n", "Class 1 accuracy: 0.0/1135(0%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", "Class 4 accuracy: 0.0/982(0%)\n", "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 954.0/958(100%)\n", - "Class 7 accuracy: 1012.0/1028(98%)\n", + "Class 6 accuracy: 955.0/958(100%)\n", + "Class 7 accuracy: 1017.0/1028(99%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 4 [0/12183 (0%)]\tLoss: 0.200790\n", - "Train Epoch: 4 [640/12183 (5%)]\tLoss: 0.200424\n", - "Train Epoch: 4 [1280/12183 (10%)]\tLoss: 0.190599\n", - "Train Epoch: 4 [1920/12183 (16%)]\tLoss: 0.205490\n", - "Train Epoch: 4 [2560/12183 (21%)]\tLoss: 0.245161\n", - "Train Epoch: 4 [3200/12183 (26%)]\tLoss: 0.154059\n", - "Train Epoch: 4 [3840/12183 (31%)]\tLoss: 0.181401\n", - "Train Epoch: 4 [4480/12183 (37%)]\tLoss: 0.283187\n", - "Train Epoch: 4 [5120/12183 (42%)]\tLoss: 0.176126\n", - "Train Epoch: 4 [5760/12183 (47%)]\tLoss: 0.218665\n", - "Train Epoch: 4 [6400/12183 (52%)]\tLoss: 0.294239\n", - "Train Epoch: 4 [7040/12183 (58%)]\tLoss: 0.213903\n", - "Train Epoch: 4 [7680/12183 (63%)]\tLoss: 0.233052\n", - "Train Epoch: 4 [8320/12183 (68%)]\tLoss: 0.161038\n", - "Train Epoch: 4 [8960/12183 (73%)]\tLoss: 0.211654\n", - "Train Epoch: 4 [9600/12183 (79%)]\tLoss: 0.192449\n", - "Train Epoch: 4 [10240/12183 (84%)]\tLoss: 0.161328\n", - "Train Epoch: 4 [10880/12183 (89%)]\tLoss: 0.197734\n", - "Train Epoch: 4 [11520/12183 (94%)]\tLoss: 0.191102\n", - "Train Epoch: 4 [4370/12183 (99%)]\tLoss: 0.376056\n", - "\n", - "Test set: Average loss: 4.9524\n", + "Train Epoch: 2 [0/12183 (0%)]\tLoss: 0.056296\n", + "Train Epoch: 2 [640/12183 (5%)]\tLoss: 0.026770\n", + "Train Epoch: 2 [1280/12183 (10%)]\tLoss: 0.014056\n", + "Train Epoch: 2 [1920/12183 (16%)]\tLoss: 0.026122\n", + "Train Epoch: 2 [2560/12183 (21%)]\tLoss: 0.025161\n", + "Train Epoch: 2 [3200/12183 (26%)]\tLoss: 0.033366\n", + "Train Epoch: 2 [3840/12183 (31%)]\tLoss: 0.011129\n", + "Train Epoch: 2 [4480/12183 (37%)]\tLoss: 0.034492\n", + "Train Epoch: 2 [5120/12183 (42%)]\tLoss: 0.009984\n", + "Train Epoch: 2 [5760/12183 (47%)]\tLoss: 0.035312\n", + "Train Epoch: 2 [6400/12183 (52%)]\tLoss: 0.028434\n", + "Train Epoch: 2 [7040/12183 (58%)]\tLoss: 0.012921\n", + "Train Epoch: 2 [7680/12183 (63%)]\tLoss: 0.009203\n", + "Train Epoch: 2 [8320/12183 (68%)]\tLoss: 0.034064\n", + "Train Epoch: 2 [8960/12183 (73%)]\tLoss: 0.018005\n", + "Train Epoch: 2 [9600/12183 (79%)]\tLoss: 0.007107\n", + "Train Epoch: 2 [10240/12183 (84%)]\tLoss: 0.003789\n", + "Train Epoch: 2 [10880/12183 (89%)]\tLoss: 0.056796\n", + "Train Epoch: 2 [11520/12183 (94%)]\tLoss: 0.008388\n", + "Train Epoch: 2 [4370/12183 (99%)]\tLoss: 0.008049\n", + "\n", + "Test set: Average loss: 8.7098\n", "Class 0 accuracy: 0.0/980(0%)\n", "Class 1 accuracy: 0.0/1135(0%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", @@ -1698,31 +844,31 @@ "Class 4 accuracy: 0.0/982(0%)\n", "Class 5 accuracy: 0.0/892(0%)\n", "Class 6 accuracy: 955.0/958(100%)\n", - "Class 7 accuracy: 1013.0/1028(99%)\n", + "Class 7 accuracy: 1021.0/1028(99%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 5 [0/12183 (0%)]\tLoss: 0.250694\n", - "Train Epoch: 5 [640/12183 (5%)]\tLoss: 0.137549\n", - "Train Epoch: 5 [1280/12183 (10%)]\tLoss: 0.179946\n", - "Train Epoch: 5 [1920/12183 (16%)]\tLoss: 0.172115\n", - "Train Epoch: 5 [2560/12183 (21%)]\tLoss: 0.216046\n", - "Train Epoch: 5 [3200/12183 (26%)]\tLoss: 0.137594\n", - "Train Epoch: 5 [3840/12183 (31%)]\tLoss: 0.167268\n", - "Train Epoch: 5 [4480/12183 (37%)]\tLoss: 0.215348\n", - "Train Epoch: 5 [5120/12183 (42%)]\tLoss: 0.190677\n", - "Train Epoch: 5 [5760/12183 (47%)]\tLoss: 0.143424\n", - "Train Epoch: 5 [6400/12183 (52%)]\tLoss: 0.176460\n", - "Train Epoch: 5 [7040/12183 (58%)]\tLoss: 0.115466\n", - "Train Epoch: 5 [7680/12183 (63%)]\tLoss: 0.123880\n", - "Train Epoch: 5 [8320/12183 (68%)]\tLoss: 0.172899\n", - "Train Epoch: 5 [8960/12183 (73%)]\tLoss: 0.121368\n", - "Train Epoch: 5 [9600/12183 (79%)]\tLoss: 0.117190\n", - "Train Epoch: 5 [10240/12183 (84%)]\tLoss: 0.193983\n", - "Train Epoch: 5 [10880/12183 (89%)]\tLoss: 0.137996\n", - "Train Epoch: 5 [11520/12183 (94%)]\tLoss: 0.165303\n", - "Train Epoch: 5 [4370/12183 (99%)]\tLoss: 0.120738\n", - "\n", - "Test set: Average loss: 5.1628\n", + "Train Epoch: 3 [0/12183 (0%)]\tLoss: 0.031829\n", + "Train Epoch: 3 [640/12183 (5%)]\tLoss: 0.026151\n", + "Train Epoch: 3 [1280/12183 (10%)]\tLoss: 0.013025\n", + "Train Epoch: 3 [1920/12183 (16%)]\tLoss: 0.015742\n", + "Train Epoch: 3 [2560/12183 (21%)]\tLoss: 0.036039\n", + "Train Epoch: 3 [3200/12183 (26%)]\tLoss: 0.019850\n", + "Train Epoch: 3 [3840/12183 (31%)]\tLoss: 0.021591\n", + "Train Epoch: 3 [4480/12183 (37%)]\tLoss: 0.011566\n", + "Train Epoch: 3 [5120/12183 (42%)]\tLoss: 0.012161\n", + "Train Epoch: 3 [5760/12183 (47%)]\tLoss: 0.005265\n", + "Train Epoch: 3 [6400/12183 (52%)]\tLoss: 0.005189\n", + "Train Epoch: 3 [7040/12183 (58%)]\tLoss: 0.023125\n", + "Train Epoch: 3 [7680/12183 (63%)]\tLoss: 0.058819\n", + "Train Epoch: 3 [8320/12183 (68%)]\tLoss: 0.027194\n", + "Train Epoch: 3 [8960/12183 (73%)]\tLoss: 0.010728\n", + "Train Epoch: 3 [9600/12183 (79%)]\tLoss: 0.002767\n", + "Train Epoch: 3 [10240/12183 (84%)]\tLoss: 0.006029\n", + "Train Epoch: 3 [10880/12183 (89%)]\tLoss: 0.090709\n", + "Train Epoch: 3 [11520/12183 (94%)]\tLoss: 0.003533\n", + "Train Epoch: 3 [4370/12183 (99%)]\tLoss: 0.005167\n", + "\n", + "Test set: Average loss: 9.1796\n", "Class 0 accuracy: 0.0/980(0%)\n", "Class 1 accuracy: 0.0/1135(0%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", @@ -1730,61 +876,30 @@ "Class 4 accuracy: 0.0/982(0%)\n", "Class 5 accuracy: 0.0/892(0%)\n", "Class 6 accuracy: 955.0/958(100%)\n", - "Class 7 accuracy: 1014.0/1028(99%)\n", + "Class 7 accuracy: 1021.0/1028(99%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 1 [0/11800 (0%)]\tLoss: 8.108945\n", - "Train Epoch: 1 [640/11800 (5%)]\tLoss: 7.738531\n", - "Train Epoch: 1 [1280/11800 (11%)]\tLoss: 6.640255\n", - "Train Epoch: 1 [1920/11800 (16%)]\tLoss: 6.513710\n", - "Train Epoch: 1 [2560/11800 (22%)]\tLoss: 5.460023\n", - "Train Epoch: 1 [3200/11800 (27%)]\tLoss: 5.351851\n", - "Train Epoch: 1 [3840/11800 (32%)]\tLoss: 4.810606\n", - "Train Epoch: 1 [4480/11800 (38%)]\tLoss: 4.409184\n", - "Train Epoch: 1 [5120/11800 (43%)]\tLoss: 3.466255\n", - "Train Epoch: 1 [5760/11800 (49%)]\tLoss: 3.398129\n", - "Train Epoch: 1 [6400/11800 (54%)]\tLoss: 2.812559\n", - "Train Epoch: 1 [7040/11800 (59%)]\tLoss: 2.539098\n", - "Train Epoch: 1 [7680/11800 (65%)]\tLoss: 1.932615\n", - "Train Epoch: 1 [8320/11800 (70%)]\tLoss: 1.948097\n", - "Train Epoch: 1 [8960/11800 (76%)]\tLoss: 1.529168\n", - "Train Epoch: 1 [9600/11800 (81%)]\tLoss: 1.568914\n", - "Train Epoch: 1 [10240/11800 (86%)]\tLoss: 1.332329\n", - "Train Epoch: 1 [10880/11800 (92%)]\tLoss: 1.441239\n", - "Train Epoch: 1 [11520/11800 (97%)]\tLoss: 1.161227\n", - "\n", - "Test set: Average loss: 2.4228\n", - "Class 0 accuracy: 0.0/980(0%)\n", - "Class 1 accuracy: 0.0/1135(0%)\n", - "Class 2 accuracy: 0.0/1032(0%)\n", - "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 485.0/958(51%)\n", - "Class 7 accuracy: 345.0/1028(34%)\n", - "Class 8 accuracy: 962.0/974(99%)\n", - "Class 9 accuracy: 735.0/1009(73%)\n", - "Train Epoch: 2 [0/11800 (0%)]\tLoss: 1.033281\n", - "Train Epoch: 2 [640/11800 (5%)]\tLoss: 1.074798\n", - "Train Epoch: 2 [1280/11800 (11%)]\tLoss: 1.087005\n", - "Train Epoch: 2 [1920/11800 (16%)]\tLoss: 1.221566\n", - "Train Epoch: 2 [2560/11800 (22%)]\tLoss: 1.020146\n", - "Train Epoch: 2 [3200/11800 (27%)]\tLoss: 1.010463\n", - "Train Epoch: 2 [3840/11800 (32%)]\tLoss: 0.871487\n", - "Train Epoch: 2 [4480/11800 (38%)]\tLoss: 0.892453\n", - "Train Epoch: 2 [5120/11800 (43%)]\tLoss: 0.761051\n", - "Train Epoch: 2 [5760/11800 (49%)]\tLoss: 0.883746\n", - "Train Epoch: 2 [6400/11800 (54%)]\tLoss: 0.881131\n", - "Train Epoch: 2 [7040/11800 (59%)]\tLoss: 0.761468\n", - "Train Epoch: 2 [7680/11800 (65%)]\tLoss: 0.702959\n", - "Train Epoch: 2 [8320/11800 (70%)]\tLoss: 0.575911\n", - "Train Epoch: 2 [8960/11800 (76%)]\tLoss: 0.668820\n", - "Train Epoch: 2 [9600/11800 (81%)]\tLoss: 0.780607\n", - "Train Epoch: 2 [10240/11800 (86%)]\tLoss: 0.565945\n", - "Train Epoch: 2 [10880/11800 (92%)]\tLoss: 0.687550\n", - "Train Epoch: 2 [11520/11800 (97%)]\tLoss: 0.551138\n", - "\n", - "Test set: Average loss: 3.3994\n", + "Train Epoch: 1 [0/11800 (0%)]\tLoss: 14.120369\n", + "Train Epoch: 1 [640/11800 (5%)]\tLoss: 7.705572\n", + "Train Epoch: 1 [1280/11800 (11%)]\tLoss: 2.772657\n", + "Train Epoch: 1 [1920/11800 (16%)]\tLoss: 1.523421\n", + "Train Epoch: 1 [2560/11800 (22%)]\tLoss: 0.632249\n", + "Train Epoch: 1 [3200/11800 (27%)]\tLoss: 0.823988\n", + "Train Epoch: 1 [3840/11800 (32%)]\tLoss: 0.649929\n", + "Train Epoch: 1 [4480/11800 (38%)]\tLoss: 0.382820\n", + "Train Epoch: 1 [5120/11800 (43%)]\tLoss: 0.287871\n", + "Train Epoch: 1 [5760/11800 (49%)]\tLoss: 0.270225\n", + "Train Epoch: 1 [6400/11800 (54%)]\tLoss: 0.250052\n", + "Train Epoch: 1 [7040/11800 (59%)]\tLoss: 0.436681\n", + "Train Epoch: 1 [7680/11800 (65%)]\tLoss: 0.205961\n", + "Train Epoch: 1 [8320/11800 (70%)]\tLoss: 0.226867\n", + "Train Epoch: 1 [8960/11800 (76%)]\tLoss: 0.178260\n", + "Train Epoch: 1 [9600/11800 (81%)]\tLoss: 0.159317\n", + "Train Epoch: 1 [10240/11800 (86%)]\tLoss: 0.402756\n", + "Train Epoch: 1 [10880/11800 (92%)]\tLoss: 0.158059\n", + "Train Epoch: 1 [11520/11800 (97%)]\tLoss: 0.195788\n", + "\n", + "Test set: Average loss: 5.9943\n", "Class 0 accuracy: 0.0/980(0%)\n", "Class 1 accuracy: 0.0/1135(0%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", @@ -1793,60 +908,29 @@ "Class 5 accuracy: 0.0/892(0%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 940.0/974(97%)\n", - "Class 9 accuracy: 914.0/1009(91%)\n", - "Train Epoch: 3 [0/11800 (0%)]\tLoss: 0.637666\n", - "Train Epoch: 3 [640/11800 (5%)]\tLoss: 0.512363\n", - "Train Epoch: 3 [1280/11800 (11%)]\tLoss: 0.587236\n", - "Train Epoch: 3 [1920/11800 (16%)]\tLoss: 0.496453\n", - "Train Epoch: 3 [2560/11800 (22%)]\tLoss: 0.617932\n", - "Train Epoch: 3 [3200/11800 (27%)]\tLoss: 0.538651\n", - "Train Epoch: 3 [3840/11800 (32%)]\tLoss: 0.471803\n", - "Train Epoch: 3 [4480/11800 (38%)]\tLoss: 0.569236\n", - "Train Epoch: 3 [5120/11800 (43%)]\tLoss: 0.473018\n", - "Train Epoch: 3 [5760/11800 (49%)]\tLoss: 0.661038\n", - "Train Epoch: 3 [6400/11800 (54%)]\tLoss: 0.436601\n", - "Train Epoch: 3 [7040/11800 (59%)]\tLoss: 0.551020\n", - "Train Epoch: 3 [7680/11800 (65%)]\tLoss: 0.422065\n", - "Train Epoch: 3 [8320/11800 (70%)]\tLoss: 0.436460\n", - "Train Epoch: 3 [8960/11800 (76%)]\tLoss: 0.521548\n", - "Train Epoch: 3 [9600/11800 (81%)]\tLoss: 0.582636\n", - "Train Epoch: 3 [10240/11800 (86%)]\tLoss: 0.460395\n", - "Train Epoch: 3 [10880/11800 (92%)]\tLoss: 0.637835\n", - "Train Epoch: 3 [11520/11800 (97%)]\tLoss: 0.514315\n", - "\n", - "Test set: Average loss: 3.9214\n", - "Class 0 accuracy: 0.0/980(0%)\n", - "Class 1 accuracy: 0.0/1135(0%)\n", - "Class 2 accuracy: 0.0/1032(0%)\n", - "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 934.0/974(96%)\n", - "Class 9 accuracy: 939.0/1009(93%)\n", - "Train Epoch: 4 [0/11800 (0%)]\tLoss: 0.498162\n", - "Train Epoch: 4 [640/11800 (5%)]\tLoss: 0.536717\n", - "Train Epoch: 4 [1280/11800 (11%)]\tLoss: 0.436568\n", - "Train Epoch: 4 [1920/11800 (16%)]\tLoss: 0.471787\n", - "Train Epoch: 4 [2560/11800 (22%)]\tLoss: 0.497887\n", - "Train Epoch: 4 [3200/11800 (27%)]\tLoss: 0.444460\n", - "Train Epoch: 4 [3840/11800 (32%)]\tLoss: 0.448744\n", - "Train Epoch: 4 [4480/11800 (38%)]\tLoss: 0.372841\n", - "Train Epoch: 4 [5120/11800 (43%)]\tLoss: 0.465649\n", - "Train Epoch: 4 [5760/11800 (49%)]\tLoss: 0.490277\n", - "Train Epoch: 4 [6400/11800 (54%)]\tLoss: 0.463720\n", - "Train Epoch: 4 [7040/11800 (59%)]\tLoss: 0.355945\n", - "Train Epoch: 4 [7680/11800 (65%)]\tLoss: 0.438429\n", - "Train Epoch: 4 [8320/11800 (70%)]\tLoss: 0.508788\n", - "Train Epoch: 4 [8960/11800 (76%)]\tLoss: 0.413552\n", - "Train Epoch: 4 [9600/11800 (81%)]\tLoss: 0.395993\n", - "Train Epoch: 4 [10240/11800 (86%)]\tLoss: 0.410553\n", - "Train Epoch: 4 [10880/11800 (92%)]\tLoss: 0.409377\n", - "Train Epoch: 4 [11520/11800 (97%)]\tLoss: 0.400534\n", - "\n", - "Test set: Average loss: 4.2109\n", + "Class 8 accuracy: 943.0/974(97%)\n", + "Class 9 accuracy: 974.0/1009(97%)\n", + "Train Epoch: 2 [0/11800 (0%)]\tLoss: 0.157916\n", + "Train Epoch: 2 [640/11800 (5%)]\tLoss: 0.187881\n", + "Train Epoch: 2 [1280/11800 (11%)]\tLoss: 0.283773\n", + "Train Epoch: 2 [1920/11800 (16%)]\tLoss: 0.135888\n", + "Train Epoch: 2 [2560/11800 (22%)]\tLoss: 0.211785\n", + "Train Epoch: 2 [3200/11800 (27%)]\tLoss: 0.238753\n", + "Train Epoch: 2 [3840/11800 (32%)]\tLoss: 0.121974\n", + "Train Epoch: 2 [4480/11800 (38%)]\tLoss: 0.074522\n", + "Train Epoch: 2 [5120/11800 (43%)]\tLoss: 0.239114\n", + "Train Epoch: 2 [5760/11800 (49%)]\tLoss: 0.129581\n", + "Train Epoch: 2 [6400/11800 (54%)]\tLoss: 0.055795\n", + "Train Epoch: 2 [7040/11800 (59%)]\tLoss: 0.349395\n", + "Train Epoch: 2 [7680/11800 (65%)]\tLoss: 0.076350\n", + "Train Epoch: 2 [8320/11800 (70%)]\tLoss: 0.072271\n", + "Train Epoch: 2 [8960/11800 (76%)]\tLoss: 0.103121\n", + "Train Epoch: 2 [9600/11800 (81%)]\tLoss: 0.343085\n", + "Train Epoch: 2 [10240/11800 (86%)]\tLoss: 0.095953\n", + "Train Epoch: 2 [10880/11800 (92%)]\tLoss: 0.095618\n", + "Train Epoch: 2 [11520/11800 (97%)]\tLoss: 0.114838\n", + "\n", + "Test set: Average loss: 6.6632\n", "Class 0 accuracy: 0.0/980(0%)\n", "Class 1 accuracy: 0.0/1135(0%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", @@ -1855,29 +939,29 @@ "Class 5 accuracy: 0.0/892(0%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 930.0/974(95%)\n", - "Class 9 accuracy: 948.0/1009(94%)\n", - "Train Epoch: 5 [0/11800 (0%)]\tLoss: 0.441779\n", - "Train Epoch: 5 [640/11800 (5%)]\tLoss: 0.349478\n", - "Train Epoch: 5 [1280/11800 (11%)]\tLoss: 0.347695\n", - "Train Epoch: 5 [1920/11800 (16%)]\tLoss: 0.374920\n", - "Train Epoch: 5 [2560/11800 (22%)]\tLoss: 0.371782\n", - "Train Epoch: 5 [3200/11800 (27%)]\tLoss: 0.405380\n", - "Train Epoch: 5 [3840/11800 (32%)]\tLoss: 0.493990\n", - "Train Epoch: 5 [4480/11800 (38%)]\tLoss: 0.383761\n", - "Train Epoch: 5 [5120/11800 (43%)]\tLoss: 0.361125\n", - "Train Epoch: 5 [5760/11800 (49%)]\tLoss: 0.377500\n", - "Train Epoch: 5 [6400/11800 (54%)]\tLoss: 0.411204\n", - "Train Epoch: 5 [7040/11800 (59%)]\tLoss: 0.467779\n", - "Train Epoch: 5 [7680/11800 (65%)]\tLoss: 0.445416\n", - "Train Epoch: 5 [8320/11800 (70%)]\tLoss: 0.303449\n", - "Train Epoch: 5 [8960/11800 (76%)]\tLoss: 0.319432\n", - "Train Epoch: 5 [9600/11800 (81%)]\tLoss: 0.331827\n", - "Train Epoch: 5 [10240/11800 (86%)]\tLoss: 0.377502\n", - "Train Epoch: 5 [10880/11800 (92%)]\tLoss: 0.406466\n", - "Train Epoch: 5 [11520/11800 (97%)]\tLoss: 0.405720\n", - "\n", - "Test set: Average loss: 4.3858\n", + "Class 8 accuracy: 967.0/974(99%)\n", + "Class 9 accuracy: 965.0/1009(96%)\n", + "Train Epoch: 3 [0/11800 (0%)]\tLoss: 0.126834\n", + "Train Epoch: 3 [640/11800 (5%)]\tLoss: 0.187850\n", + "Train Epoch: 3 [1280/11800 (11%)]\tLoss: 0.102141\n", + "Train Epoch: 3 [1920/11800 (16%)]\tLoss: 0.036415\n", + "Train Epoch: 3 [2560/11800 (22%)]\tLoss: 0.165435\n", + "Train Epoch: 3 [3200/11800 (27%)]\tLoss: 0.109672\n", + "Train Epoch: 3 [3840/11800 (32%)]\tLoss: 0.070627\n", + "Train Epoch: 3 [4480/11800 (38%)]\tLoss: 0.047646\n", + "Train Epoch: 3 [5120/11800 (43%)]\tLoss: 0.051905\n", + "Train Epoch: 3 [5760/11800 (49%)]\tLoss: 0.264070\n", + "Train Epoch: 3 [6400/11800 (54%)]\tLoss: 0.211393\n", + "Train Epoch: 3 [7040/11800 (59%)]\tLoss: 0.062177\n", + "Train Epoch: 3 [7680/11800 (65%)]\tLoss: 0.033032\n", + "Train Epoch: 3 [8320/11800 (70%)]\tLoss: 0.068230\n", + "Train Epoch: 3 [8960/11800 (76%)]\tLoss: 0.071904\n", + "Train Epoch: 3 [9600/11800 (81%)]\tLoss: 0.102863\n", + "Train Epoch: 3 [10240/11800 (86%)]\tLoss: 0.046137\n", + "Train Epoch: 3 [10880/11800 (92%)]\tLoss: 0.120950\n", + "Train Epoch: 3 [11520/11800 (97%)]\tLoss: 0.101907\n", + "\n", + "Test set: Average loss: 6.9242\n", "Class 0 accuracy: 0.0/980(0%)\n", "Class 1 accuracy: 0.0/1135(0%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", @@ -1886,95 +970,31 @@ "Class 5 accuracy: 0.0/892(0%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 929.0/974(95%)\n", - "Class 9 accuracy: 950.0/1009(94%)\n", - "Train Epoch: 1 [0/12665 (0%)]\tLoss: 5.571240\n", - "Train Epoch: 1 [640/12665 (5%)]\tLoss: 5.058652\n", - "Train Epoch: 1 [1280/12665 (10%)]\tLoss: 4.223537\n", - "Train Epoch: 1 [1920/12665 (15%)]\tLoss: 3.769430\n", - "Train Epoch: 1 [2560/12665 (20%)]\tLoss: 3.397155\n", - "Train Epoch: 1 [3200/12665 (25%)]\tLoss: 3.066763\n", - "Train Epoch: 1 [3840/12665 (30%)]\tLoss: 2.407099\n", - "Train Epoch: 1 [4480/12665 (35%)]\tLoss: 2.510794\n", - "Train Epoch: 1 [5120/12665 (40%)]\tLoss: 1.850332\n", - "Train Epoch: 1 [5760/12665 (45%)]\tLoss: 1.656529\n", - "Train Epoch: 1 [6400/12665 (51%)]\tLoss: 1.517869\n", - "Train Epoch: 1 [7040/12665 (56%)]\tLoss: 1.028974\n", - "Train Epoch: 1 [7680/12665 (61%)]\tLoss: 0.833506\n", - "Train Epoch: 1 [8320/12665 (66%)]\tLoss: 0.765792\n", - "Train Epoch: 1 [8960/12665 (71%)]\tLoss: 0.600499\n", - "Train Epoch: 1 [9600/12665 (76%)]\tLoss: 0.491542\n", - "Train Epoch: 1 [10240/12665 (81%)]\tLoss: 0.525203\n", - "Train Epoch: 1 [10880/12665 (86%)]\tLoss: 0.473181\n", - "Train Epoch: 1 [11520/12665 (91%)]\tLoss: 0.445249\n", - "Train Epoch: 1 [12160/12665 (96%)]\tLoss: 0.329707\n", - "\n", - "Test set: Average loss: 2.6586\n", - "Class 0 accuracy: 974.0/980(99%)\n", - "Class 1 accuracy: 1134.0/1135(100%)\n", - "Class 2 accuracy: 0.0/1032(0%)\n", - "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 616.0/1009(61%)\n", - "Train Epoch: 2 [0/12665 (0%)]\tLoss: 0.174450\n", - "Train Epoch: 2 [640/12665 (5%)]\tLoss: 0.286020\n", - "Train Epoch: 2 [1280/12665 (10%)]\tLoss: 0.380822\n", - "Train Epoch: 2 [1920/12665 (15%)]\tLoss: 0.200067\n", - "Train Epoch: 2 [2560/12665 (20%)]\tLoss: 0.233174\n", - "Train Epoch: 2 [3200/12665 (25%)]\tLoss: 0.215359\n", - "Train Epoch: 2 [3840/12665 (30%)]\tLoss: 0.261502\n", - "Train Epoch: 2 [4480/12665 (35%)]\tLoss: 0.193090\n", - "Train Epoch: 2 [5120/12665 (40%)]\tLoss: 0.201911\n", - "Train Epoch: 2 [5760/12665 (45%)]\tLoss: 0.123239\n", - "Train Epoch: 2 [6400/12665 (51%)]\tLoss: 0.151434\n", - "Train Epoch: 2 [7040/12665 (56%)]\tLoss: 0.161058\n", - "Train Epoch: 2 [7680/12665 (61%)]\tLoss: 0.121602\n", - "Train Epoch: 2 [8320/12665 (66%)]\tLoss: 0.152282\n", - "Train Epoch: 2 [8960/12665 (71%)]\tLoss: 0.138010\n", - "Train Epoch: 2 [9600/12665 (76%)]\tLoss: 0.116535\n", - "Train Epoch: 2 [10240/12665 (81%)]\tLoss: 0.148996\n", - "Train Epoch: 2 [10880/12665 (86%)]\tLoss: 0.094655\n", - "Train Epoch: 2 [11520/12665 (91%)]\tLoss: 0.093630\n", - "Train Epoch: 2 [12160/12665 (96%)]\tLoss: 0.100545\n", - "\n", - "Test set: Average loss: 3.7860\n", - "Class 0 accuracy: 974.0/980(99%)\n", - "Class 1 accuracy: 1134.0/1135(100%)\n", - "Class 2 accuracy: 0.0/1032(0%)\n", - "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 3 [0/12665 (0%)]\tLoss: 0.113853\n", - "Train Epoch: 3 [640/12665 (5%)]\tLoss: 0.061060\n", - "Train Epoch: 3 [1280/12665 (10%)]\tLoss: 0.073625\n", - "Train Epoch: 3 [1920/12665 (15%)]\tLoss: 0.089106\n", - "Train Epoch: 3 [2560/12665 (20%)]\tLoss: 0.100265\n", - "Train Epoch: 3 [3200/12665 (25%)]\tLoss: 0.086965\n", - "Train Epoch: 3 [3840/12665 (30%)]\tLoss: 0.078005\n", - "Train Epoch: 3 [4480/12665 (35%)]\tLoss: 0.089567\n", - "Train Epoch: 3 [5120/12665 (40%)]\tLoss: 0.083796\n", - "Train Epoch: 3 [5760/12665 (45%)]\tLoss: 0.091500\n", - "Train Epoch: 3 [6400/12665 (51%)]\tLoss: 0.071154\n", - "Train Epoch: 3 [7040/12665 (56%)]\tLoss: 0.063725\n", - "Train Epoch: 3 [7680/12665 (61%)]\tLoss: 0.081232\n", - "Train Epoch: 3 [8320/12665 (66%)]\tLoss: 0.063744\n", - "Train Epoch: 3 [8960/12665 (71%)]\tLoss: 0.044772\n", - "Train Epoch: 3 [9600/12665 (76%)]\tLoss: 0.056600\n", - "Train Epoch: 3 [10240/12665 (81%)]\tLoss: 0.075639\n", - "Train Epoch: 3 [10880/12665 (86%)]\tLoss: 0.059448\n", - "Train Epoch: 3 [11520/12665 (91%)]\tLoss: 0.051801\n", - "Train Epoch: 3 [12160/12665 (96%)]\tLoss: 0.045583\n", - "\n", - "Test set: Average loss: 4.3795\n", - "Class 0 accuracy: 974.0/980(99%)\n", + "Class 8 accuracy: 952.0/974(98%)\n", + "Class 9 accuracy: 978.0/1009(97%)\n", + "Train Epoch: 1 [0/12665 (0%)]\tLoss: 8.722752\n", + "Train Epoch: 1 [640/12665 (5%)]\tLoss: 3.160372\n", + "Train Epoch: 1 [1280/12665 (10%)]\tLoss: 0.653003\n", + "Train Epoch: 1 [1920/12665 (15%)]\tLoss: 0.256268\n", + "Train Epoch: 1 [2560/12665 (20%)]\tLoss: 0.064630\n", + "Train Epoch: 1 [3200/12665 (25%)]\tLoss: 0.089074\n", + "Train Epoch: 1 [3840/12665 (30%)]\tLoss: 0.022682\n", + "Train Epoch: 1 [4480/12665 (35%)]\tLoss: 0.014132\n", + "Train Epoch: 1 [5120/12665 (40%)]\tLoss: 0.011369\n", + "Train Epoch: 1 [5760/12665 (45%)]\tLoss: 0.032196\n", + "Train Epoch: 1 [6400/12665 (51%)]\tLoss: 0.087249\n", + "Train Epoch: 1 [7040/12665 (56%)]\tLoss: 0.010759\n", + "Train Epoch: 1 [7680/12665 (61%)]\tLoss: 0.009488\n", + "Train Epoch: 1 [8320/12665 (66%)]\tLoss: 0.009431\n", + "Train Epoch: 1 [8960/12665 (71%)]\tLoss: 0.017381\n", + "Train Epoch: 1 [9600/12665 (76%)]\tLoss: 0.014684\n", + "Train Epoch: 1 [10240/12665 (81%)]\tLoss: 0.007916\n", + "Train Epoch: 1 [10880/12665 (86%)]\tLoss: 0.008834\n", + "Train Epoch: 1 [11520/12665 (91%)]\tLoss: 0.082196\n", + "Train Epoch: 1 [12160/12665 (96%)]\tLoss: 0.002452\n", + "\n", + "Test set: Average loss: 7.1540\n", + "Class 0 accuracy: 977.0/980(100%)\n", "Class 1 accuracy: 1134.0/1135(100%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", @@ -1984,29 +1004,29 @@ "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 4 [0/12665 (0%)]\tLoss: 0.073423\n", - "Train Epoch: 4 [640/12665 (5%)]\tLoss: 0.081250\n", - "Train Epoch: 4 [1280/12665 (10%)]\tLoss: 0.107188\n", - "Train Epoch: 4 [1920/12665 (15%)]\tLoss: 0.108589\n", - "Train Epoch: 4 [2560/12665 (20%)]\tLoss: 0.074006\n", - "Train Epoch: 4 [3200/12665 (25%)]\tLoss: 0.038409\n", - "Train Epoch: 4 [3840/12665 (30%)]\tLoss: 0.065204\n", - "Train Epoch: 4 [4480/12665 (35%)]\tLoss: 0.083654\n", - "Train Epoch: 4 [5120/12665 (40%)]\tLoss: 0.073944\n", - "Train Epoch: 4 [5760/12665 (45%)]\tLoss: 0.048630\n", - "Train Epoch: 4 [6400/12665 (51%)]\tLoss: 0.041222\n", - "Train Epoch: 4 [7040/12665 (56%)]\tLoss: 0.055742\n", - "Train Epoch: 4 [7680/12665 (61%)]\tLoss: 0.048848\n", - "Train Epoch: 4 [8320/12665 (66%)]\tLoss: 0.099426\n", - "Train Epoch: 4 [8960/12665 (71%)]\tLoss: 0.050589\n", - "Train Epoch: 4 [9600/12665 (76%)]\tLoss: 0.039493\n", - "Train Epoch: 4 [10240/12665 (81%)]\tLoss: 0.059188\n", - "Train Epoch: 4 [10880/12665 (86%)]\tLoss: 0.071380\n", - "Train Epoch: 4 [11520/12665 (91%)]\tLoss: 0.095400\n", - "Train Epoch: 4 [12160/12665 (96%)]\tLoss: 0.052493\n", - "\n", - "Test set: Average loss: 4.6909\n", - "Class 0 accuracy: 975.0/980(99%)\n", + "Train Epoch: 2 [0/12665 (0%)]\tLoss: 0.000673\n", + "Train Epoch: 2 [640/12665 (5%)]\tLoss: 0.029478\n", + "Train Epoch: 2 [1280/12665 (10%)]\tLoss: 0.002406\n", + "Train Epoch: 2 [1920/12665 (15%)]\tLoss: 0.008619\n", + "Train Epoch: 2 [2560/12665 (20%)]\tLoss: 0.001777\n", + "Train Epoch: 2 [3200/12665 (25%)]\tLoss: 0.031666\n", + "Train Epoch: 2 [3840/12665 (30%)]\tLoss: 0.133443\n", + "Train Epoch: 2 [4480/12665 (35%)]\tLoss: 0.006141\n", + "Train Epoch: 2 [5120/12665 (40%)]\tLoss: 0.006000\n", + "Train Epoch: 2 [5760/12665 (45%)]\tLoss: 0.003495\n", + "Train Epoch: 2 [6400/12665 (51%)]\tLoss: 0.000851\n", + "Train Epoch: 2 [7040/12665 (56%)]\tLoss: 0.001280\n", + "Train Epoch: 2 [7680/12665 (61%)]\tLoss: 0.000972\n", + "Train Epoch: 2 [8320/12665 (66%)]\tLoss: 0.001776\n", + "Train Epoch: 2 [8960/12665 (71%)]\tLoss: 0.067486\n", + "Train Epoch: 2 [9600/12665 (76%)]\tLoss: 0.001030\n", + "Train Epoch: 2 [10240/12665 (81%)]\tLoss: 0.001892\n", + "Train Epoch: 2 [10880/12665 (86%)]\tLoss: 0.001493\n", + "Train Epoch: 2 [11520/12665 (91%)]\tLoss: 0.047196\n", + "Train Epoch: 2 [12160/12665 (96%)]\tLoss: 0.004952\n", + "\n", + "Test set: Average loss: 7.8296\n", + "Class 0 accuracy: 977.0/980(100%)\n", "Class 1 accuracy: 1134.0/1135(100%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", @@ -2016,29 +1036,29 @@ "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 5 [0/12665 (0%)]\tLoss: 0.042433\n", - "Train Epoch: 5 [640/12665 (5%)]\tLoss: 0.045415\n", - "Train Epoch: 5 [1280/12665 (10%)]\tLoss: 0.045671\n", - "Train Epoch: 5 [1920/12665 (15%)]\tLoss: 0.116742\n", - "Train Epoch: 5 [2560/12665 (20%)]\tLoss: 0.042136\n", - "Train Epoch: 5 [3200/12665 (25%)]\tLoss: 0.053295\n", - "Train Epoch: 5 [3840/12665 (30%)]\tLoss: 0.035618\n", - "Train Epoch: 5 [4480/12665 (35%)]\tLoss: 0.055716\n", - "Train Epoch: 5 [5120/12665 (40%)]\tLoss: 0.043886\n", - "Train Epoch: 5 [5760/12665 (45%)]\tLoss: 0.052791\n", - "Train Epoch: 5 [6400/12665 (51%)]\tLoss: 0.079626\n", - "Train Epoch: 5 [7040/12665 (56%)]\tLoss: 0.044485\n", - "Train Epoch: 5 [7680/12665 (61%)]\tLoss: 0.027041\n", - "Train Epoch: 5 [8320/12665 (66%)]\tLoss: 0.041513\n", - "Train Epoch: 5 [8960/12665 (71%)]\tLoss: 0.034053\n", - "Train Epoch: 5 [9600/12665 (76%)]\tLoss: 0.028694\n", - "Train Epoch: 5 [10240/12665 (81%)]\tLoss: 0.054827\n", - "Train Epoch: 5 [10880/12665 (86%)]\tLoss: 0.182031\n", - "Train Epoch: 5 [11520/12665 (91%)]\tLoss: 0.068930\n", - "Train Epoch: 5 [12160/12665 (96%)]\tLoss: 0.022576\n", - "\n", - "Test set: Average loss: 4.8702\n", - "Class 0 accuracy: 976.0/980(100%)\n", + "Train Epoch: 3 [0/12665 (0%)]\tLoss: 0.007902\n", + "Train Epoch: 3 [640/12665 (5%)]\tLoss: 0.003385\n", + "Train Epoch: 3 [1280/12665 (10%)]\tLoss: 0.001230\n", + "Train Epoch: 3 [1920/12665 (15%)]\tLoss: 0.001810\n", + "Train Epoch: 3 [2560/12665 (20%)]\tLoss: 0.001579\n", + "Train Epoch: 3 [3200/12665 (25%)]\tLoss: 0.028920\n", + "Train Epoch: 3 [3840/12665 (30%)]\tLoss: 0.004439\n", + "Train Epoch: 3 [4480/12665 (35%)]\tLoss: 0.001913\n", + "Train Epoch: 3 [5120/12665 (40%)]\tLoss: 0.000407\n", + "Train Epoch: 3 [5760/12665 (45%)]\tLoss: 0.004402\n", + "Train Epoch: 3 [6400/12665 (51%)]\tLoss: 0.009100\n", + "Train Epoch: 3 [7040/12665 (56%)]\tLoss: 0.000980\n", + "Train Epoch: 3 [7680/12665 (61%)]\tLoss: 0.000387\n", + "Train Epoch: 3 [8320/12665 (66%)]\tLoss: 0.004606\n", + "Train Epoch: 3 [8960/12665 (71%)]\tLoss: 0.001144\n", + "Train Epoch: 3 [9600/12665 (76%)]\tLoss: 0.007978\n", + "Train Epoch: 3 [10240/12665 (81%)]\tLoss: 0.011006\n", + "Train Epoch: 3 [10880/12665 (86%)]\tLoss: 0.001300\n", + "Train Epoch: 3 [11520/12665 (91%)]\tLoss: 0.000758\n", + "Train Epoch: 3 [12160/12665 (96%)]\tLoss: 0.001379\n", + "\n", + "Test set: Average loss: 8.1397\n", + "Class 0 accuracy: 978.0/980(100%)\n", "Class 1 accuracy: 1134.0/1135(100%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", @@ -2049,6 +1069,213 @@ "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n" ] + }, + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[{0: 0.996938775510204,\n", + " 1: 0.9991189427312775,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '0_1'},\n", + " {0: 0.996938775510204,\n", + " 1: 0.9991189427312775,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '0_1'},\n", + " {0: 0.996938775510204,\n", + " 1: 0.9991189427312775,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '0_1'},\n", + " {0: 0.0,\n", + " 1: 0.0,\n", + " 2: 0.9612403100775194,\n", + " 3: 0.9772277227722772,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '2_3'},\n", + " {0: 0.0,\n", + " 1: 0.0,\n", + " 2: 0.9670542635658915,\n", + " 3: 0.9841584158415841,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '2_3'},\n", + " {0: 0.0,\n", + " 1: 0.0,\n", + " 2: 0.9757751937984496,\n", + " 3: 0.9752475247524752,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '2_3'},\n", + " {0: 0.0,\n", + " 1: 0.0,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.994908350305499,\n", + " 5: 0.9708520179372198,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '4_5'},\n", + " {0: 0.0,\n", + " 1: 0.0,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.9959266802443992,\n", + " 5: 0.9775784753363229,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '4_5'},\n", + " {0: 0.0,\n", + " 1: 0.0,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.9959266802443992,\n", + " 5: 0.9854260089686099,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '4_5'},\n", + " {0: 0.0,\n", + " 1: 0.0,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.9968684759916493,\n", + " 7: 0.9892996108949417,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '6_7'},\n", + " {0: 0.0,\n", + " 1: 0.0,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.9968684759916493,\n", + " 7: 0.9931906614785992,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '6_7'},\n", + " {0: 0.0,\n", + " 1: 0.0,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.9968684759916493,\n", + " 7: 0.9931906614785992,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '6_7'},\n", + " {0: 0.0,\n", + " 1: 0.0,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.9681724845995893,\n", + " 9: 0.9653121902874133,\n", + " 'phase': '8_9'},\n", + " {0: 0.0,\n", + " 1: 0.0,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.9928131416837782,\n", + " 9: 0.956392467789891,\n", + " 'phase': '8_9'},\n", + " {0: 0.0,\n", + " 1: 0.0,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.9774127310061602,\n", + " 9: 0.9692765113974232,\n", + " 'phase': '8_9'},\n", + " {0: 0.996938775510204,\n", + " 1: 0.9991189427312775,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '0_1_again'},\n", + " {0: 0.996938775510204,\n", + " 1: 0.9991189427312775,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '0_1_again'},\n", + " {0: 0.9979591836734694,\n", + " 1: 0.9991189427312775,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '0_1_again'}]" + ] + }, + "metadata": {}, + "execution_count": 19 } ] }, @@ -2100,41 +1327,41 @@ "history_memory_replay" ], "metadata": { + "id": "LtWvHl_tQCpg", "colab": { "base_uri": "https://localhost:8080/" }, - "id": "LtWvHl_tQCpg", - "outputId": "e4913ffb-5f2b-474f-d24a-bde139af833c" + "outputId": "6705c5fe-9838-4da5-a562-77d7c9a8c6d8" }, - "execution_count": 16, + "execution_count": 20, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ - "Train Epoch: 1 [0/12665 (0%)]\tLoss: 2.295792\n", - "Train Epoch: 1 [640/12665 (5%)]\tLoss: 2.208695\n", - "Train Epoch: 1 [1280/12665 (10%)]\tLoss: 2.106677\n", - "Train Epoch: 1 [1920/12665 (15%)]\tLoss: 1.961109\n", - "Train Epoch: 1 [2560/12665 (20%)]\tLoss: 1.898052\n", - "Train Epoch: 1 [3200/12665 (25%)]\tLoss: 1.824553\n", - "Train Epoch: 1 [3840/12665 (30%)]\tLoss: 1.719246\n", - "Train Epoch: 1 [4480/12665 (35%)]\tLoss: 1.623634\n", - "Train Epoch: 1 [5120/12665 (40%)]\tLoss: 1.617715\n", - "Train Epoch: 1 [5760/12665 (45%)]\tLoss: 1.727536\n", - "Train Epoch: 1 [6400/12665 (51%)]\tLoss: 1.507114\n", - "Train Epoch: 1 [7040/12665 (56%)]\tLoss: 1.447730\n", - "Train Epoch: 1 [7680/12665 (61%)]\tLoss: 1.549878\n", - "Train Epoch: 1 [8320/12665 (66%)]\tLoss: 1.058249\n", - "Train Epoch: 1 [8960/12665 (71%)]\tLoss: 1.447155\n", - "Train Epoch: 1 [9600/12665 (76%)]\tLoss: 1.273197\n", - "Train Epoch: 1 [10240/12665 (81%)]\tLoss: 1.260120\n", - "Train Epoch: 1 [10880/12665 (86%)]\tLoss: 1.187015\n", - "Train Epoch: 1 [11520/12665 (91%)]\tLoss: 1.091164\n", - "Train Epoch: 1 [12160/12665 (96%)]\tLoss: 0.971557\n", - "\n", - "Test set: Average loss: 2.7060\n", - "Class 0 accuracy: 972.0/980(99%)\n", + "Train Epoch: 1 [0/12665 (0%)]\tLoss: 2.196357\n", + "Train Epoch: 1 [640/12665 (5%)]\tLoss: 1.335955\n", + "Train Epoch: 1 [1280/12665 (10%)]\tLoss: 0.555878\n", + "Train Epoch: 1 [1920/12665 (15%)]\tLoss: 0.146986\n", + "Train Epoch: 1 [2560/12665 (20%)]\tLoss: 0.082966\n", + "Train Epoch: 1 [3200/12665 (25%)]\tLoss: 0.054255\n", + "Train Epoch: 1 [3840/12665 (30%)]\tLoss: 0.011100\n", + "Train Epoch: 1 [4480/12665 (35%)]\tLoss: 0.078007\n", + "Train Epoch: 1 [5120/12665 (40%)]\tLoss: 0.005687\n", + "Train Epoch: 1 [5760/12665 (45%)]\tLoss: 0.004260\n", + "Train Epoch: 1 [6400/12665 (51%)]\tLoss: 0.002415\n", + "Train Epoch: 1 [7040/12665 (56%)]\tLoss: 0.005845\n", + "Train Epoch: 1 [7680/12665 (61%)]\tLoss: 0.005518\n", + "Train Epoch: 1 [8320/12665 (66%)]\tLoss: 0.001475\n", + "Train Epoch: 1 [8960/12665 (71%)]\tLoss: 0.002208\n", + "Train Epoch: 1 [9600/12665 (76%)]\tLoss: 0.000701\n", + "Train Epoch: 1 [10240/12665 (81%)]\tLoss: 0.000352\n", + "Train Epoch: 1 [10880/12665 (86%)]\tLoss: 0.000833\n", + "Train Epoch: 1 [11520/12665 (91%)]\tLoss: 0.002580\n", + "Train Epoch: 1 [12160/12665 (96%)]\tLoss: 0.002562\n", + "\n", + "Test set: Average loss: 7.8370\n", + "Class 0 accuracy: 976.0/980(100%)\n", "Class 1 accuracy: 1134.0/1135(100%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", @@ -2144,93 +1371,29 @@ "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 2 [0/12665 (0%)]\tLoss: 0.952087\n", - "Train Epoch: 2 [640/12665 (5%)]\tLoss: 0.869477\n", - "Train Epoch: 2 [1280/12665 (10%)]\tLoss: 0.858798\n", - "Train Epoch: 2 [1920/12665 (15%)]\tLoss: 0.809786\n", - "Train Epoch: 2 [2560/12665 (20%)]\tLoss: 0.799570\n", - "Train Epoch: 2 [3200/12665 (25%)]\tLoss: 0.890806\n", - "Train Epoch: 2 [3840/12665 (30%)]\tLoss: 0.805940\n", - "Train Epoch: 2 [4480/12665 (35%)]\tLoss: 1.059435\n", - "Train Epoch: 2 [5120/12665 (40%)]\tLoss: 0.478667\n", - "Train Epoch: 2 [5760/12665 (45%)]\tLoss: 0.673640\n", - "Train Epoch: 2 [6400/12665 (51%)]\tLoss: 0.538314\n", - "Train Epoch: 2 [7040/12665 (56%)]\tLoss: 0.710136\n", - "Train Epoch: 2 [7680/12665 (61%)]\tLoss: 0.516661\n", - "Train Epoch: 2 [8320/12665 (66%)]\tLoss: 0.602048\n", - "Train Epoch: 2 [8960/12665 (71%)]\tLoss: 0.384920\n", - "Train Epoch: 2 [9600/12665 (76%)]\tLoss: 0.435023\n", - "Train Epoch: 2 [10240/12665 (81%)]\tLoss: 0.372631\n", - "Train Epoch: 2 [10880/12665 (86%)]\tLoss: 0.426314\n", - "Train Epoch: 2 [11520/12665 (91%)]\tLoss: 0.539168\n", - "Train Epoch: 2 [12160/12665 (96%)]\tLoss: 0.321303\n", - "\n", - "Test set: Average loss: 3.8263\n", - "Class 0 accuracy: 966.0/980(99%)\n", - "Class 1 accuracy: 1135.0/1135(100%)\n", - "Class 2 accuracy: 0.0/1032(0%)\n", - "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 3 [0/12665 (0%)]\tLoss: 0.401120\n", - "Train Epoch: 3 [640/12665 (5%)]\tLoss: 0.263190\n", - "Train Epoch: 3 [1280/12665 (10%)]\tLoss: 0.398854\n", - "Train Epoch: 3 [1920/12665 (15%)]\tLoss: 0.229336\n", - "Train Epoch: 3 [2560/12665 (20%)]\tLoss: 0.240626\n", - "Train Epoch: 3 [3200/12665 (25%)]\tLoss: 0.239559\n", - "Train Epoch: 3 [3840/12665 (30%)]\tLoss: 0.418891\n", - "Train Epoch: 3 [4480/12665 (35%)]\tLoss: 0.177813\n", - "Train Epoch: 3 [5120/12665 (40%)]\tLoss: 0.226682\n", - "Train Epoch: 3 [5760/12665 (45%)]\tLoss: 0.215379\n", - "Train Epoch: 3 [6400/12665 (51%)]\tLoss: 0.211677\n", - "Train Epoch: 3 [7040/12665 (56%)]\tLoss: 0.224589\n", - "Train Epoch: 3 [7680/12665 (61%)]\tLoss: 0.253302\n", - "Train Epoch: 3 [8320/12665 (66%)]\tLoss: 0.212234\n", - "Train Epoch: 3 [8960/12665 (71%)]\tLoss: 0.230823\n", - "Train Epoch: 3 [9600/12665 (76%)]\tLoss: 0.164656\n", - "Train Epoch: 3 [10240/12665 (81%)]\tLoss: 0.154504\n", - "Train Epoch: 3 [10880/12665 (86%)]\tLoss: 0.100245\n", - "Train Epoch: 3 [11520/12665 (91%)]\tLoss: 0.156521\n", - "Train Epoch: 3 [12160/12665 (96%)]\tLoss: 0.232217\n", - "\n", - "Test set: Average loss: 4.6647\n", - "Class 0 accuracy: 963.0/980(98%)\n", - "Class 1 accuracy: 1135.0/1135(100%)\n", - "Class 2 accuracy: 0.0/1032(0%)\n", - "Class 3 accuracy: 0.0/1010(0%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 4 [0/12665 (0%)]\tLoss: 0.131072\n", - "Train Epoch: 4 [640/12665 (5%)]\tLoss: 0.227836\n", - "Train Epoch: 4 [1280/12665 (10%)]\tLoss: 0.161914\n", - "Train Epoch: 4 [1920/12665 (15%)]\tLoss: 0.182439\n", - "Train Epoch: 4 [2560/12665 (20%)]\tLoss: 0.123449\n", - "Train Epoch: 4 [3200/12665 (25%)]\tLoss: 0.118936\n", - "Train Epoch: 4 [3840/12665 (30%)]\tLoss: 0.132258\n", - "Train Epoch: 4 [4480/12665 (35%)]\tLoss: 0.191232\n", - "Train Epoch: 4 [5120/12665 (40%)]\tLoss: 0.192155\n", - "Train Epoch: 4 [5760/12665 (45%)]\tLoss: 0.122639\n", - "Train Epoch: 4 [6400/12665 (51%)]\tLoss: 0.138319\n", - "Train Epoch: 4 [7040/12665 (56%)]\tLoss: 0.106367\n", - "Train Epoch: 4 [7680/12665 (61%)]\tLoss: 0.078942\n", - "Train Epoch: 4 [8320/12665 (66%)]\tLoss: 0.103141\n", - "Train Epoch: 4 [8960/12665 (71%)]\tLoss: 0.129387\n", - "Train Epoch: 4 [9600/12665 (76%)]\tLoss: 0.117389\n", - "Train Epoch: 4 [10240/12665 (81%)]\tLoss: 0.164451\n", - "Train Epoch: 4 [10880/12665 (86%)]\tLoss: 0.216510\n", - "Train Epoch: 4 [11520/12665 (91%)]\tLoss: 0.110041\n", - "Train Epoch: 4 [12160/12665 (96%)]\tLoss: 0.104683\n", - "\n", - "Test set: Average loss: 5.1577\n", - "Class 0 accuracy: 966.0/980(99%)\n", + "Train Epoch: 2 [0/12665 (0%)]\tLoss: 0.221930\n", + "Train Epoch: 2 [640/12665 (5%)]\tLoss: 0.027592\n", + "Train Epoch: 2 [1280/12665 (10%)]\tLoss: 0.065451\n", + "Train Epoch: 2 [1920/12665 (15%)]\tLoss: 0.015034\n", + "Train Epoch: 2 [2560/12665 (20%)]\tLoss: 0.046994\n", + "Train Epoch: 2 [3200/12665 (25%)]\tLoss: 0.008316\n", + "Train Epoch: 2 [3840/12665 (30%)]\tLoss: 0.021686\n", + "Train Epoch: 2 [4480/12665 (35%)]\tLoss: 0.024137\n", + "Train Epoch: 2 [5120/12665 (40%)]\tLoss: 0.002627\n", + "Train Epoch: 2 [5760/12665 (45%)]\tLoss: 0.042552\n", + "Train Epoch: 2 [6400/12665 (51%)]\tLoss: 0.026033\n", + "Train Epoch: 2 [7040/12665 (56%)]\tLoss: 0.002244\n", + "Train Epoch: 2 [7680/12665 (61%)]\tLoss: 0.003471\n", + "Train Epoch: 2 [8320/12665 (66%)]\tLoss: 0.041315\n", + "Train Epoch: 2 [8960/12665 (71%)]\tLoss: 0.008438\n", + "Train Epoch: 2 [9600/12665 (76%)]\tLoss: 0.000700\n", + "Train Epoch: 2 [10240/12665 (81%)]\tLoss: 0.000417\n", + "Train Epoch: 2 [10880/12665 (86%)]\tLoss: 0.002115\n", + "Train Epoch: 2 [11520/12665 (91%)]\tLoss: 0.008961\n", + "Train Epoch: 2 [12160/12665 (96%)]\tLoss: 0.002133\n", + "\n", + "Test set: Average loss: 8.7829\n", + "Class 0 accuracy: 976.0/980(100%)\n", "Class 1 accuracy: 1135.0/1135(100%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", @@ -2240,29 +1403,29 @@ "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 5 [0/12665 (0%)]\tLoss: 0.077715\n", - "Train Epoch: 5 [640/12665 (5%)]\tLoss: 0.105138\n", - "Train Epoch: 5 [1280/12665 (10%)]\tLoss: 0.124953\n", - "Train Epoch: 5 [1920/12665 (15%)]\tLoss: 0.080610\n", - "Train Epoch: 5 [2560/12665 (20%)]\tLoss: 0.123617\n", - "Train Epoch: 5 [3200/12665 (25%)]\tLoss: 0.112670\n", - "Train Epoch: 5 [3840/12665 (30%)]\tLoss: 0.093631\n", - "Train Epoch: 5 [4480/12665 (35%)]\tLoss: 0.122231\n", - "Train Epoch: 5 [5120/12665 (40%)]\tLoss: 0.093652\n", - "Train Epoch: 5 [5760/12665 (45%)]\tLoss: 0.167812\n", - "Train Epoch: 5 [6400/12665 (51%)]\tLoss: 0.117981\n", - "Train Epoch: 5 [7040/12665 (56%)]\tLoss: 0.066692\n", - "Train Epoch: 5 [7680/12665 (61%)]\tLoss: 0.080171\n", - "Train Epoch: 5 [8320/12665 (66%)]\tLoss: 0.092669\n", - "Train Epoch: 5 [8960/12665 (71%)]\tLoss: 0.146656\n", - "Train Epoch: 5 [9600/12665 (76%)]\tLoss: 0.101289\n", - "Train Epoch: 5 [10240/12665 (81%)]\tLoss: 0.089997\n", - "Train Epoch: 5 [10880/12665 (86%)]\tLoss: 0.108752\n", - "Train Epoch: 5 [11520/12665 (91%)]\tLoss: 0.078000\n", - "Train Epoch: 5 [12160/12665 (96%)]\tLoss: 0.046221\n", - "\n", - "Test set: Average loss: 5.4729\n", - "Class 0 accuracy: 968.0/980(99%)\n", + "Train Epoch: 3 [0/12665 (0%)]\tLoss: 0.000570\n", + "Train Epoch: 3 [640/12665 (5%)]\tLoss: 0.011992\n", + "Train Epoch: 3 [1280/12665 (10%)]\tLoss: 0.000725\n", + "Train Epoch: 3 [1920/12665 (15%)]\tLoss: 0.000835\n", + "Train Epoch: 3 [2560/12665 (20%)]\tLoss: 0.000185\n", + "Train Epoch: 3 [3200/12665 (25%)]\tLoss: 0.001194\n", + "Train Epoch: 3 [3840/12665 (30%)]\tLoss: 0.000802\n", + "Train Epoch: 3 [4480/12665 (35%)]\tLoss: 0.001044\n", + "Train Epoch: 3 [5120/12665 (40%)]\tLoss: 0.001886\n", + "Train Epoch: 3 [5760/12665 (45%)]\tLoss: 0.001518\n", + "Train Epoch: 3 [6400/12665 (51%)]\tLoss: 0.001240\n", + "Train Epoch: 3 [7040/12665 (56%)]\tLoss: 0.005406\n", + "Train Epoch: 3 [7680/12665 (61%)]\tLoss: 0.004714\n", + "Train Epoch: 3 [8320/12665 (66%)]\tLoss: 0.000870\n", + "Train Epoch: 3 [8960/12665 (71%)]\tLoss: 0.001174\n", + "Train Epoch: 3 [9600/12665 (76%)]\tLoss: 0.000833\n", + "Train Epoch: 3 [10240/12665 (81%)]\tLoss: 0.006711\n", + "Train Epoch: 3 [10880/12665 (86%)]\tLoss: 0.006384\n", + "Train Epoch: 3 [11520/12665 (91%)]\tLoss: 0.006488\n", + "Train Epoch: 3 [12160/12665 (96%)]\tLoss: 0.000901\n", + "\n", + "Test set: Average loss: 9.2011\n", + "Class 0 accuracy: 977.0/980(100%)\n", "Class 1 accuracy: 1135.0/1135(100%)\n", "Class 2 accuracy: 0.0/1032(0%)\n", "Class 3 accuracy: 0.0/1010(0%)\n", @@ -2272,61 +1435,30 @@ "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 1 [0/12089 (0%)]\tLoss: 1.166433\n", - "Train Epoch: 1 [640/12089 (5%)]\tLoss: 1.153999\n", - "Train Epoch: 1 [1280/12089 (11%)]\tLoss: 1.238186\n", - "Train Epoch: 1 [1920/12089 (16%)]\tLoss: 1.274190\n", - "Train Epoch: 1 [2560/12089 (21%)]\tLoss: 1.001455\n", - "Train Epoch: 1 [3200/12089 (26%)]\tLoss: 0.980768\n", - "Train Epoch: 1 [3840/12089 (32%)]\tLoss: 0.890982\n", - "Train Epoch: 1 [4480/12089 (37%)]\tLoss: 0.845770\n", - "Train Epoch: 1 [5120/12089 (42%)]\tLoss: 0.915158\n", - "Train Epoch: 1 [5760/12089 (48%)]\tLoss: 0.577478\n", - "Train Epoch: 1 [6400/12089 (53%)]\tLoss: 0.558826\n", - "Train Epoch: 1 [7040/12089 (58%)]\tLoss: 0.697927\n", - "Train Epoch: 1 [7680/12089 (63%)]\tLoss: 0.780273\n", - "Train Epoch: 1 [8320/12089 (69%)]\tLoss: 0.586348\n", - "Train Epoch: 1 [8960/12089 (74%)]\tLoss: 0.634719\n", - "Train Epoch: 1 [9600/12089 (79%)]\tLoss: 0.753965\n", - "Train Epoch: 1 [10240/12089 (85%)]\tLoss: 0.886030\n", - "Train Epoch: 1 [10880/12089 (90%)]\tLoss: 0.913329\n", - "Train Epoch: 1 [11520/12089 (95%)]\tLoss: 0.726805\n", - "\n", - "Test set: Average loss: 2.9830\n", - "Class 0 accuracy: 966.0/980(99%)\n", - "Class 1 accuracy: 1132.0/1135(100%)\n", - "Class 2 accuracy: 109.0/1032(11%)\n", - "Class 3 accuracy: 668.0/1010(66%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 2 [0/12089 (0%)]\tLoss: 0.903527\n", - "Train Epoch: 2 [640/12089 (5%)]\tLoss: 0.690564\n", - "Train Epoch: 2 [1280/12089 (11%)]\tLoss: 1.054734\n", - "Train Epoch: 2 [1920/12089 (16%)]\tLoss: 0.668291\n", - "Train Epoch: 2 [2560/12089 (21%)]\tLoss: 0.846006\n", - "Train Epoch: 2 [3200/12089 (26%)]\tLoss: 0.862174\n", - "Train Epoch: 2 [3840/12089 (32%)]\tLoss: 1.078590\n", - "Train Epoch: 2 [4480/12089 (37%)]\tLoss: 0.812564\n", - "Train Epoch: 2 [5120/12089 (42%)]\tLoss: 0.751745\n", - "Train Epoch: 2 [5760/12089 (48%)]\tLoss: 0.819927\n", - "Train Epoch: 2 [6400/12089 (53%)]\tLoss: 0.641764\n", - "Train Epoch: 2 [7040/12089 (58%)]\tLoss: 0.941192\n", - "Train Epoch: 2 [7680/12089 (63%)]\tLoss: 1.113382\n", - "Train Epoch: 2 [8320/12089 (69%)]\tLoss: 0.745287\n", - "Train Epoch: 2 [8960/12089 (74%)]\tLoss: 0.845461\n", - "Train Epoch: 2 [9600/12089 (79%)]\tLoss: 0.675010\n", - "Train Epoch: 2 [10240/12089 (85%)]\tLoss: 0.772856\n", - "Train Epoch: 2 [10880/12089 (90%)]\tLoss: 0.668837\n", - "Train Epoch: 2 [11520/12089 (95%)]\tLoss: 0.776711\n", - "\n", - "Test set: Average loss: 3.1608\n", - "Class 0 accuracy: 828.0/980(84%)\n", - "Class 1 accuracy: 1091.0/1135(96%)\n", - "Class 2 accuracy: 898.0/1032(87%)\n", + "Train Epoch: 1 [0/12089 (0%)]\tLoss: 2.805099\n", + "Train Epoch: 1 [640/12089 (5%)]\tLoss: 1.660801\n", + "Train Epoch: 1 [1280/12089 (11%)]\tLoss: 0.437137\n", + "Train Epoch: 1 [1920/12089 (16%)]\tLoss: 0.808282\n", + "Train Epoch: 1 [2560/12089 (21%)]\tLoss: 0.285198\n", + "Train Epoch: 1 [3200/12089 (26%)]\tLoss: 0.129722\n", + "Train Epoch: 1 [3840/12089 (32%)]\tLoss: 0.145888\n", + "Train Epoch: 1 [4480/12089 (37%)]\tLoss: 0.343596\n", + "Train Epoch: 1 [5120/12089 (42%)]\tLoss: 0.078071\n", + "Train Epoch: 1 [5760/12089 (48%)]\tLoss: 0.098522\n", + "Train Epoch: 1 [6400/12089 (53%)]\tLoss: 0.089895\n", + "Train Epoch: 1 [7040/12089 (58%)]\tLoss: 0.067733\n", + "Train Epoch: 1 [7680/12089 (63%)]\tLoss: 0.046314\n", + "Train Epoch: 1 [8320/12089 (69%)]\tLoss: 0.028943\n", + "Train Epoch: 1 [8960/12089 (74%)]\tLoss: 0.082591\n", + "Train Epoch: 1 [9600/12089 (79%)]\tLoss: 0.024760\n", + "Train Epoch: 1 [10240/12089 (85%)]\tLoss: 0.050374\n", + "Train Epoch: 1 [10880/12089 (90%)]\tLoss: 0.020318\n", + "Train Epoch: 1 [11520/12089 (95%)]\tLoss: 0.025210\n", + "\n", + "Test set: Average loss: 5.2845\n", + "Class 0 accuracy: 829.0/980(85%)\n", + "Class 1 accuracy: 1046.0/1135(92%)\n", + "Class 2 accuracy: 991.0/1032(96%)\n", "Class 3 accuracy: 983.0/1010(97%)\n", "Class 4 accuracy: 0.0/982(0%)\n", "Class 5 accuracy: 0.0/892(0%)\n", @@ -2334,725 +1466,651 @@ "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 3 [0/12089 (0%)]\tLoss: 0.733399\n", - "Train Epoch: 3 [640/12089 (5%)]\tLoss: 0.793913\n", - "Train Epoch: 3 [1280/12089 (11%)]\tLoss: 0.580512\n", - "Train Epoch: 3 [1920/12089 (16%)]\tLoss: 0.477269\n", - "Train Epoch: 3 [2560/12089 (21%)]\tLoss: 0.833255\n", - "Train Epoch: 3 [3200/12089 (26%)]\tLoss: 0.636208\n", - "Train Epoch: 3 [3840/12089 (32%)]\tLoss: 0.573351\n", - "Train Epoch: 3 [4480/12089 (37%)]\tLoss: 0.656650\n", - "Train Epoch: 3 [5120/12089 (42%)]\tLoss: 0.692221\n", - "Train Epoch: 3 [5760/12089 (48%)]\tLoss: 0.723047\n", - "Train Epoch: 3 [6400/12089 (53%)]\tLoss: 0.626788\n", - "Train Epoch: 3 [7040/12089 (58%)]\tLoss: 0.557750\n", - "Train Epoch: 3 [7680/12089 (63%)]\tLoss: 0.643362\n", - "Train Epoch: 3 [8320/12089 (69%)]\tLoss: 0.725737\n", - "Train Epoch: 3 [8960/12089 (74%)]\tLoss: 0.868968\n", - "Train Epoch: 3 [9600/12089 (79%)]\tLoss: 0.720436\n", - "Train Epoch: 3 [10240/12089 (85%)]\tLoss: 0.723306\n", - "Train Epoch: 3 [10880/12089 (90%)]\tLoss: 0.624521\n", - "Train Epoch: 3 [11520/12089 (95%)]\tLoss: 0.507009\n", - "\n", - "Test set: Average loss: 3.3136\n", - "Class 0 accuracy: 780.0/980(80%)\n", - "Class 1 accuracy: 1067.0/1135(94%)\n", - "Class 2 accuracy: 917.0/1032(89%)\n", - "Class 3 accuracy: 983.0/1010(97%)\n", + "Train Epoch: 2 [0/12089 (0%)]\tLoss: 0.091748\n", + "Train Epoch: 2 [640/12089 (5%)]\tLoss: 0.038480\n", + "Train Epoch: 2 [1280/12089 (11%)]\tLoss: 0.027258\n", + "Train Epoch: 2 [1920/12089 (16%)]\tLoss: 0.040072\n", + "Train Epoch: 2 [2560/12089 (21%)]\tLoss: 0.024616\n", + "Train Epoch: 2 [3200/12089 (26%)]\tLoss: 0.021542\n", + "Train Epoch: 2 [3840/12089 (32%)]\tLoss: 0.016757\n", + "Train Epoch: 2 [4480/12089 (37%)]\tLoss: 0.029563\n", + "Train Epoch: 2 [5120/12089 (42%)]\tLoss: 0.010378\n", + "Train Epoch: 2 [5760/12089 (48%)]\tLoss: 0.020319\n", + "Train Epoch: 2 [6400/12089 (53%)]\tLoss: 0.019660\n", + "Train Epoch: 2 [7040/12089 (58%)]\tLoss: 0.006761\n", + "Train Epoch: 2 [7680/12089 (63%)]\tLoss: 0.026263\n", + "Train Epoch: 2 [8320/12089 (69%)]\tLoss: 0.008666\n", + "Train Epoch: 2 [8960/12089 (74%)]\tLoss: 0.013218\n", + "Train Epoch: 2 [9600/12089 (79%)]\tLoss: 0.002491\n", + "Train Epoch: 2 [10240/12089 (85%)]\tLoss: 0.010161\n", + "Train Epoch: 2 [10880/12089 (90%)]\tLoss: 0.019670\n", + "Train Epoch: 2 [11520/12089 (95%)]\tLoss: 0.012468\n", + "\n", + "Test set: Average loss: 6.1435\n", + "Class 0 accuracy: 764.0/980(78%)\n", + "Class 1 accuracy: 999.0/1135(88%)\n", + "Class 2 accuracy: 1000.0/1032(97%)\n", + "Class 3 accuracy: 988.0/1010(98%)\n", "Class 4 accuracy: 0.0/982(0%)\n", "Class 5 accuracy: 0.0/892(0%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 4 [0/12089 (0%)]\tLoss: 0.624655\n", - "Train Epoch: 4 [640/12089 (5%)]\tLoss: 0.452378\n", - "Train Epoch: 4 [1280/12089 (11%)]\tLoss: 0.457449\n", - "Train Epoch: 4 [1920/12089 (16%)]\tLoss: 0.554645\n", - "Train Epoch: 4 [2560/12089 (21%)]\tLoss: 0.621985\n", - "Train Epoch: 4 [3200/12089 (26%)]\tLoss: 0.527864\n", - "Train Epoch: 4 [3840/12089 (32%)]\tLoss: 0.577068\n", - "Train Epoch: 4 [4480/12089 (37%)]\tLoss: 0.514906\n", - "Train Epoch: 4 [5120/12089 (42%)]\tLoss: 0.581334\n", - "Train Epoch: 4 [5760/12089 (48%)]\tLoss: 0.331880\n", - "Train Epoch: 4 [6400/12089 (53%)]\tLoss: 0.424212\n", - "Train Epoch: 4 [7040/12089 (58%)]\tLoss: 0.527507\n", - "Train Epoch: 4 [7680/12089 (63%)]\tLoss: 0.685347\n", - "Train Epoch: 4 [8320/12089 (69%)]\tLoss: 0.525770\n", - "Train Epoch: 4 [8960/12089 (74%)]\tLoss: 0.417910\n", - "Train Epoch: 4 [9600/12089 (79%)]\tLoss: 0.731371\n", - "Train Epoch: 4 [10240/12089 (85%)]\tLoss: 0.528346\n", - "Train Epoch: 4 [10880/12089 (90%)]\tLoss: 0.454048\n", - "Train Epoch: 4 [11520/12089 (95%)]\tLoss: 0.373003\n", - "\n", - "Test set: Average loss: 3.4222\n", - "Class 0 accuracy: 754.0/980(77%)\n", - "Class 1 accuracy: 1055.0/1135(93%)\n", - "Class 2 accuracy: 918.0/1032(89%)\n", - "Class 3 accuracy: 986.0/1010(98%)\n", + "Train Epoch: 3 [0/12089 (0%)]\tLoss: 0.028908\n", + "Train Epoch: 3 [640/12089 (5%)]\tLoss: 0.017523\n", + "Train Epoch: 3 [1280/12089 (11%)]\tLoss: 0.012286\n", + "Train Epoch: 3 [1920/12089 (16%)]\tLoss: 0.029993\n", + "Train Epoch: 3 [2560/12089 (21%)]\tLoss: 0.014532\n", + "Train Epoch: 3 [3200/12089 (26%)]\tLoss: 0.023068\n", + "Train Epoch: 3 [3840/12089 (32%)]\tLoss: 0.007645\n", + "Train Epoch: 3 [4480/12089 (37%)]\tLoss: 0.016837\n", + "Train Epoch: 3 [5120/12089 (42%)]\tLoss: 0.004771\n", + "Train Epoch: 3 [5760/12089 (48%)]\tLoss: 0.004505\n", + "Train Epoch: 3 [6400/12089 (53%)]\tLoss: 0.085267\n", + "Train Epoch: 3 [7040/12089 (58%)]\tLoss: 0.007249\n", + "Train Epoch: 3 [7680/12089 (63%)]\tLoss: 0.005857\n", + "Train Epoch: 3 [8320/12089 (69%)]\tLoss: 0.036750\n", + "Train Epoch: 3 [8960/12089 (74%)]\tLoss: 0.035684\n", + "Train Epoch: 3 [9600/12089 (79%)]\tLoss: 0.005433\n", + "Train Epoch: 3 [10240/12089 (85%)]\tLoss: 0.006491\n", + "Train Epoch: 3 [10880/12089 (90%)]\tLoss: 0.026140\n", + "Train Epoch: 3 [11520/12089 (95%)]\tLoss: 0.007269\n", + "\n", + "Test set: Average loss: 6.5537\n", + "Class 0 accuracy: 723.0/980(74%)\n", + "Class 1 accuracy: 968.0/1135(85%)\n", + "Class 2 accuracy: 999.0/1032(97%)\n", + "Class 3 accuracy: 992.0/1010(98%)\n", "Class 4 accuracy: 0.0/982(0%)\n", "Class 5 accuracy: 0.0/892(0%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 5 [0/12089 (0%)]\tLoss: 0.546961\n", - "Train Epoch: 5 [640/12089 (5%)]\tLoss: 0.382696\n", - "Train Epoch: 5 [1280/12089 (11%)]\tLoss: 0.616167\n", - "Train Epoch: 5 [1920/12089 (16%)]\tLoss: 0.551108\n", - "Train Epoch: 5 [2560/12089 (21%)]\tLoss: 0.671682\n", - "Train Epoch: 5 [3200/12089 (26%)]\tLoss: 0.434367\n", - "Train Epoch: 5 [3840/12089 (32%)]\tLoss: 0.589797\n", - "Train Epoch: 5 [4480/12089 (37%)]\tLoss: 0.462255\n", - "Train Epoch: 5 [5120/12089 (42%)]\tLoss: 0.427646\n", - "Train Epoch: 5 [5760/12089 (48%)]\tLoss: 0.393138\n", - "Train Epoch: 5 [6400/12089 (53%)]\tLoss: 0.374952\n", - "Train Epoch: 5 [7040/12089 (58%)]\tLoss: 0.392871\n", - "Train Epoch: 5 [7680/12089 (63%)]\tLoss: 0.494981\n", - "Train Epoch: 5 [8320/12089 (69%)]\tLoss: 0.474602\n", - "Train Epoch: 5 [8960/12089 (74%)]\tLoss: 0.418477\n", - "Train Epoch: 5 [9600/12089 (79%)]\tLoss: 0.402891\n", - "Train Epoch: 5 [10240/12089 (85%)]\tLoss: 0.545055\n", - "Train Epoch: 5 [10880/12089 (90%)]\tLoss: 0.465828\n", - "Train Epoch: 5 [11520/12089 (95%)]\tLoss: 0.378968\n", - "\n", - "Test set: Average loss: 3.4964\n", - "Class 0 accuracy: 743.0/980(76%)\n", - "Class 1 accuracy: 1053.0/1135(93%)\n", - "Class 2 accuracy: 928.0/1032(90%)\n", - "Class 3 accuracy: 986.0/1010(98%)\n", - "Class 4 accuracy: 0.0/982(0%)\n", - "Class 5 accuracy: 0.0/892(0%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 1 [0/11263 (0%)]\tLoss: 0.850781\n", - "Train Epoch: 1 [640/11263 (6%)]\tLoss: 0.798292\n", - "Train Epoch: 1 [1280/11263 (11%)]\tLoss: 0.834117\n", - "Train Epoch: 1 [1920/11263 (17%)]\tLoss: 0.725765\n", - "Train Epoch: 1 [2560/11263 (23%)]\tLoss: 0.676635\n", - "Train Epoch: 1 [3200/11263 (28%)]\tLoss: 0.758110\n", - "Train Epoch: 1 [3840/11263 (34%)]\tLoss: 0.689677\n", - "Train Epoch: 1 [4480/11263 (40%)]\tLoss: 0.686142\n", - "Train Epoch: 1 [5120/11263 (45%)]\tLoss: 0.819930\n", - "Train Epoch: 1 [5760/11263 (51%)]\tLoss: 0.822826\n", - "Train Epoch: 1 [6400/11263 (57%)]\tLoss: 0.597154\n", - "Train Epoch: 1 [7040/11263 (62%)]\tLoss: 0.712456\n", - "Train Epoch: 1 [7680/11263 (68%)]\tLoss: 0.668146\n", - "Train Epoch: 1 [8320/11263 (74%)]\tLoss: 0.641806\n", - "Train Epoch: 1 [8960/11263 (80%)]\tLoss: 0.607866\n", - "Train Epoch: 1 [9600/11263 (85%)]\tLoss: 0.837627\n", - "Train Epoch: 1 [10240/11263 (91%)]\tLoss: 0.767775\n", - "Train Epoch: 1 [10880/11263 (97%)]\tLoss: 0.710244\n", - "\n", - "Test set: Average loss: 2.3788\n", - "Class 0 accuracy: 936.0/980(96%)\n", - "Class 1 accuracy: 1123.0/1135(99%)\n", - "Class 2 accuracy: 808.0/1032(78%)\n", - "Class 3 accuracy: 926.0/1010(92%)\n", - "Class 4 accuracy: 790.0/982(80%)\n", - "Class 5 accuracy: 360.0/892(40%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 2 [0/11263 (0%)]\tLoss: 0.896705\n", - "Train Epoch: 2 [640/11263 (6%)]\tLoss: 0.905773\n", - "Train Epoch: 2 [1280/11263 (11%)]\tLoss: 0.899539\n", - "Train Epoch: 2 [1920/11263 (17%)]\tLoss: 0.838611\n", - "Train Epoch: 2 [2560/11263 (23%)]\tLoss: 0.892302\n", - "Train Epoch: 2 [3200/11263 (28%)]\tLoss: 1.038975\n", - "Train Epoch: 2 [3840/11263 (34%)]\tLoss: 0.937163\n", - "Train Epoch: 2 [4480/11263 (40%)]\tLoss: 0.748620\n", - "Train Epoch: 2 [5120/11263 (45%)]\tLoss: 0.863919\n", - "Train Epoch: 2 [5760/11263 (51%)]\tLoss: 1.029196\n", - "Train Epoch: 2 [6400/11263 (57%)]\tLoss: 0.845210\n", - "Train Epoch: 2 [7040/11263 (62%)]\tLoss: 0.758227\n", - "Train Epoch: 2 [7680/11263 (68%)]\tLoss: 0.620243\n", - "Train Epoch: 2 [8320/11263 (74%)]\tLoss: 0.842112\n", - "Train Epoch: 2 [8960/11263 (80%)]\tLoss: 0.743856\n", - "Train Epoch: 2 [9600/11263 (85%)]\tLoss: 0.656504\n", - "Train Epoch: 2 [10240/11263 (91%)]\tLoss: 0.817470\n", - "Train Epoch: 2 [10880/11263 (97%)]\tLoss: 0.658012\n", - "\n", - "Test set: Average loss: 2.4907\n", - "Class 0 accuracy: 789.0/980(81%)\n", - "Class 1 accuracy: 1107.0/1135(98%)\n", - "Class 2 accuracy: 670.0/1032(65%)\n", - "Class 3 accuracy: 515.0/1010(51%)\n", - "Class 4 accuracy: 942.0/982(96%)\n", - "Class 5 accuracy: 766.0/892(86%)\n", - "Class 6 accuracy: 0.0/958(0%)\n", - "Class 7 accuracy: 0.0/1028(0%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 3 [0/11263 (0%)]\tLoss: 0.754302\n", - "Train Epoch: 3 [640/11263 (6%)]\tLoss: 0.660190\n", - "Train Epoch: 3 [1280/11263 (11%)]\tLoss: 0.655176\n", - "Train Epoch: 3 [1920/11263 (17%)]\tLoss: 0.696439\n", - "Train Epoch: 3 [2560/11263 (23%)]\tLoss: 0.786622\n", - "Train Epoch: 3 [3200/11263 (28%)]\tLoss: 0.699489\n", - "Train Epoch: 3 [3840/11263 (34%)]\tLoss: 0.776751\n", - "Train Epoch: 3 [4480/11263 (40%)]\tLoss: 0.755700\n", - "Train Epoch: 3 [5120/11263 (45%)]\tLoss: 0.776476\n", - "Train Epoch: 3 [5760/11263 (51%)]\tLoss: 0.718861\n", - "Train Epoch: 3 [6400/11263 (57%)]\tLoss: 0.644046\n", - "Train Epoch: 3 [7040/11263 (62%)]\tLoss: 0.879339\n", - "Train Epoch: 3 [7680/11263 (68%)]\tLoss: 0.587297\n", - "Train Epoch: 3 [8320/11263 (74%)]\tLoss: 0.763207\n", - "Train Epoch: 3 [8960/11263 (80%)]\tLoss: 0.761769\n", - "Train Epoch: 3 [9600/11263 (85%)]\tLoss: 0.656073\n", - "Train Epoch: 3 [10240/11263 (91%)]\tLoss: 0.616851\n", - "Train Epoch: 3 [10880/11263 (97%)]\tLoss: 0.573584\n", - "\n", - "Test set: Average loss: 2.5840\n", - "Class 0 accuracy: 754.0/980(77%)\n", - "Class 1 accuracy: 1089.0/1135(96%)\n", - "Class 2 accuracy: 640.0/1032(62%)\n", - "Class 3 accuracy: 371.0/1010(37%)\n", - "Class 4 accuracy: 960.0/982(98%)\n", - "Class 5 accuracy: 800.0/892(90%)\n", + "Train Epoch: 1 [0/11263 (0%)]\tLoss: 1.635322\n", + "Train Epoch: 1 [640/11263 (6%)]\tLoss: 0.442154\n", + "Train Epoch: 1 [1280/11263 (11%)]\tLoss: 0.249078\n", + "Train Epoch: 1 [1920/11263 (17%)]\tLoss: 0.270407\n", + "Train Epoch: 1 [2560/11263 (23%)]\tLoss: 0.180369\n", + "Train Epoch: 1 [3200/11263 (28%)]\tLoss: 0.088140\n", + "Train Epoch: 1 [3840/11263 (34%)]\tLoss: 0.171120\n", + "Train Epoch: 1 [4480/11263 (40%)]\tLoss: 0.263797\n", + "Train Epoch: 1 [5120/11263 (45%)]\tLoss: 0.111911\n", + "Train Epoch: 1 [5760/11263 (51%)]\tLoss: 0.117878\n", + "Train Epoch: 1 [6400/11263 (57%)]\tLoss: 0.033867\n", + "Train Epoch: 1 [7040/11263 (62%)]\tLoss: 0.058040\n", + "Train Epoch: 1 [7680/11263 (68%)]\tLoss: 0.034263\n", + "Train Epoch: 1 [8320/11263 (74%)]\tLoss: 0.082758\n", + "Train Epoch: 1 [8960/11263 (80%)]\tLoss: 0.041592\n", + "Train Epoch: 1 [9600/11263 (85%)]\tLoss: 0.035437\n", + "Train Epoch: 1 [10240/11263 (91%)]\tLoss: 0.015268\n", + "Train Epoch: 1 [10880/11263 (97%)]\tLoss: 0.029453\n", + "\n", + "Test set: Average loss: 4.5940\n", + "Class 0 accuracy: 760.0/980(78%)\n", + "Class 1 accuracy: 986.0/1135(87%)\n", + "Class 2 accuracy: 664.0/1032(64%)\n", + "Class 3 accuracy: 367.0/1010(36%)\n", + "Class 4 accuracy: 979.0/982(100%)\n", + "Class 5 accuracy: 864.0/892(97%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 4 [0/11263 (0%)]\tLoss: 0.648252\n", - "Train Epoch: 4 [640/11263 (6%)]\tLoss: 0.596721\n", - "Train Epoch: 4 [1280/11263 (11%)]\tLoss: 0.558993\n", - "Train Epoch: 4 [1920/11263 (17%)]\tLoss: 0.492765\n", - "Train Epoch: 4 [2560/11263 (23%)]\tLoss: 0.646996\n", - "Train Epoch: 4 [3200/11263 (28%)]\tLoss: 0.737724\n", - "Train Epoch: 4 [3840/11263 (34%)]\tLoss: 0.578851\n", - "Train Epoch: 4 [4480/11263 (40%)]\tLoss: 0.646620\n", - "Train Epoch: 4 [5120/11263 (45%)]\tLoss: 0.652254\n", - "Train Epoch: 4 [5760/11263 (51%)]\tLoss: 0.631144\n", - "Train Epoch: 4 [6400/11263 (57%)]\tLoss: 0.702145\n", - "Train Epoch: 4 [7040/11263 (62%)]\tLoss: 0.601468\n", - "Train Epoch: 4 [7680/11263 (68%)]\tLoss: 0.567884\n", - "Train Epoch: 4 [8320/11263 (74%)]\tLoss: 0.487138\n", - "Train Epoch: 4 [8960/11263 (80%)]\tLoss: 0.673586\n", - "Train Epoch: 4 [9600/11263 (85%)]\tLoss: 0.492252\n", - "Train Epoch: 4 [10240/11263 (91%)]\tLoss: 0.645415\n", - "Train Epoch: 4 [10880/11263 (97%)]\tLoss: 0.587828\n", - "\n", - "Test set: Average loss: 2.6582\n", - "Class 0 accuracy: 731.0/980(75%)\n", - "Class 1 accuracy: 1086.0/1135(96%)\n", - "Class 2 accuracy: 631.0/1032(61%)\n", - "Class 3 accuracy: 293.0/1010(29%)\n", - "Class 4 accuracy: 959.0/982(98%)\n", - "Class 5 accuracy: 817.0/892(92%)\n", + "Train Epoch: 2 [0/11263 (0%)]\tLoss: 0.008444\n", + "Train Epoch: 2 [640/11263 (6%)]\tLoss: 0.020623\n", + "Train Epoch: 2 [1280/11263 (11%)]\tLoss: 0.028893\n", + "Train Epoch: 2 [1920/11263 (17%)]\tLoss: 0.025949\n", + "Train Epoch: 2 [2560/11263 (23%)]\tLoss: 0.027629\n", + "Train Epoch: 2 [3200/11263 (28%)]\tLoss: 0.012673\n", + "Train Epoch: 2 [3840/11263 (34%)]\tLoss: 0.132138\n", + "Train Epoch: 2 [4480/11263 (40%)]\tLoss: 0.019918\n", + "Train Epoch: 2 [5120/11263 (45%)]\tLoss: 0.040663\n", + "Train Epoch: 2 [5760/11263 (51%)]\tLoss: 0.072586\n", + "Train Epoch: 2 [6400/11263 (57%)]\tLoss: 0.027573\n", + "Train Epoch: 2 [7040/11263 (62%)]\tLoss: 0.010655\n", + "Train Epoch: 2 [7680/11263 (68%)]\tLoss: 0.007804\n", + "Train Epoch: 2 [8320/11263 (74%)]\tLoss: 0.021280\n", + "Train Epoch: 2 [8960/11263 (80%)]\tLoss: 0.038740\n", + "Train Epoch: 2 [9600/11263 (85%)]\tLoss: 0.005795\n", + "Train Epoch: 2 [10240/11263 (91%)]\tLoss: 0.005783\n", + "Train Epoch: 2 [10880/11263 (97%)]\tLoss: 0.049235\n", + "\n", + "Test set: Average loss: 5.5267\n", + "Class 0 accuracy: 647.0/980(66%)\n", + "Class 1 accuracy: 970.0/1135(85%)\n", + "Class 2 accuracy: 616.0/1032(60%)\n", + "Class 3 accuracy: 240.0/1010(24%)\n", + "Class 4 accuracy: 979.0/982(100%)\n", + "Class 5 accuracy: 876.0/892(98%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 5 [0/11263 (0%)]\tLoss: 0.693316\n", - "Train Epoch: 5 [640/11263 (6%)]\tLoss: 0.579783\n", - "Train Epoch: 5 [1280/11263 (11%)]\tLoss: 0.606642\n", - "Train Epoch: 5 [1920/11263 (17%)]\tLoss: 0.578930\n", - "Train Epoch: 5 [2560/11263 (23%)]\tLoss: 0.593126\n", - "Train Epoch: 5 [3200/11263 (28%)]\tLoss: 0.568712\n", - "Train Epoch: 5 [3840/11263 (34%)]\tLoss: 0.730927\n", - "Train Epoch: 5 [4480/11263 (40%)]\tLoss: 0.517850\n", - "Train Epoch: 5 [5120/11263 (45%)]\tLoss: 0.595436\n", - "Train Epoch: 5 [5760/11263 (51%)]\tLoss: 0.623922\n", - "Train Epoch: 5 [6400/11263 (57%)]\tLoss: 0.477650\n", - "Train Epoch: 5 [7040/11263 (62%)]\tLoss: 0.672610\n", - "Train Epoch: 5 [7680/11263 (68%)]\tLoss: 0.543774\n", - "Train Epoch: 5 [8320/11263 (74%)]\tLoss: 0.580296\n", - "Train Epoch: 5 [8960/11263 (80%)]\tLoss: 0.583141\n", - "Train Epoch: 5 [9600/11263 (85%)]\tLoss: 0.448098\n", - "Train Epoch: 5 [10240/11263 (91%)]\tLoss: 0.625687\n", - "Train Epoch: 5 [10880/11263 (97%)]\tLoss: 0.533021\n", - "\n", - "Test set: Average loss: 2.7075\n", - "Class 0 accuracy: 719.0/980(73%)\n", - "Class 1 accuracy: 1080.0/1135(95%)\n", - "Class 2 accuracy: 629.0/1032(61%)\n", - "Class 3 accuracy: 287.0/1010(28%)\n", - "Class 4 accuracy: 959.0/982(98%)\n", - "Class 5 accuracy: 834.0/892(93%)\n", + "Train Epoch: 3 [0/11263 (0%)]\tLoss: 0.060683\n", + "Train Epoch: 3 [640/11263 (6%)]\tLoss: 0.049512\n", + "Train Epoch: 3 [1280/11263 (11%)]\tLoss: 0.034698\n", + "Train Epoch: 3 [1920/11263 (17%)]\tLoss: 0.015581\n", + "Train Epoch: 3 [2560/11263 (23%)]\tLoss: 0.097252\n", + "Train Epoch: 3 [3200/11263 (28%)]\tLoss: 0.022429\n", + "Train Epoch: 3 [3840/11263 (34%)]\tLoss: 0.021434\n", + "Train Epoch: 3 [4480/11263 (40%)]\tLoss: 0.030935\n", + "Train Epoch: 3 [5120/11263 (45%)]\tLoss: 0.055945\n", + "Train Epoch: 3 [5760/11263 (51%)]\tLoss: 0.032630\n", + "Train Epoch: 3 [6400/11263 (57%)]\tLoss: 0.010632\n", + "Train Epoch: 3 [7040/11263 (62%)]\tLoss: 0.014181\n", + "Train Epoch: 3 [7680/11263 (68%)]\tLoss: 0.011640\n", + "Train Epoch: 3 [8320/11263 (74%)]\tLoss: 0.022853\n", + "Train Epoch: 3 [8960/11263 (80%)]\tLoss: 0.011952\n", + "Train Epoch: 3 [9600/11263 (85%)]\tLoss: 0.029158\n", + "Train Epoch: 3 [10240/11263 (91%)]\tLoss: 0.005775\n", + "Train Epoch: 3 [10880/11263 (97%)]\tLoss: 0.044570\n", + "\n", + "Test set: Average loss: 6.0346\n", + "Class 0 accuracy: 662.0/980(68%)\n", + "Class 1 accuracy: 972.0/1135(86%)\n", + "Class 2 accuracy: 558.0/1032(54%)\n", + "Class 3 accuracy: 228.0/1010(23%)\n", + "Class 4 accuracy: 978.0/982(100%)\n", + "Class 5 accuracy: 883.0/892(99%)\n", "Class 6 accuracy: 0.0/958(0%)\n", "Class 7 accuracy: 0.0/1028(0%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 1 [0/12183 (0%)]\tLoss: 0.851070\n", - "Train Epoch: 1 [640/12183 (5%)]\tLoss: 0.703527\n", - "Train Epoch: 1 [1280/12183 (10%)]\tLoss: 0.862511\n", - "Train Epoch: 1 [1920/12183 (16%)]\tLoss: 0.664779\n", - "Train Epoch: 1 [2560/12183 (21%)]\tLoss: 0.622195\n", - "Train Epoch: 1 [3200/12183 (26%)]\tLoss: 0.686788\n", - "Train Epoch: 1 [3840/12183 (31%)]\tLoss: 0.766821\n", - "Train Epoch: 1 [4480/12183 (37%)]\tLoss: 0.635133\n", - "Train Epoch: 1 [5120/12183 (42%)]\tLoss: 0.673985\n", - "Train Epoch: 1 [5760/12183 (47%)]\tLoss: 0.736285\n", - "Train Epoch: 1 [6400/12183 (52%)]\tLoss: 0.763070\n", - "Train Epoch: 1 [7040/12183 (58%)]\tLoss: 0.914076\n", - "Train Epoch: 1 [7680/12183 (63%)]\tLoss: 0.897798\n", - "Train Epoch: 1 [8320/12183 (68%)]\tLoss: 0.769690\n", - "Train Epoch: 1 [8960/12183 (73%)]\tLoss: 0.705697\n", - "Train Epoch: 1 [9600/12183 (79%)]\tLoss: 0.890666\n", - "Train Epoch: 1 [10240/12183 (84%)]\tLoss: 0.964875\n", - "Train Epoch: 1 [10880/12183 (89%)]\tLoss: 0.846083\n", - "Train Epoch: 1 [11520/12183 (94%)]\tLoss: 0.977361\n", - "Train Epoch: 1 [4370/12183 (99%)]\tLoss: 1.044579\n", - "\n", - "Test set: Average loss: 1.6683\n", - "Class 0 accuracy: 839.0/980(86%)\n", - "Class 1 accuracy: 1086.0/1135(96%)\n", - "Class 2 accuracy: 547.0/1032(53%)\n", - "Class 3 accuracy: 854.0/1010(85%)\n", - "Class 4 accuracy: 673.0/982(69%)\n", - "Class 5 accuracy: 311.0/892(35%)\n", - "Class 6 accuracy: 878.0/958(92%)\n", - "Class 7 accuracy: 904.0/1028(88%)\n", + "Train Epoch: 1 [0/12183 (0%)]\tLoss: 1.379911\n", + "Train Epoch: 1 [640/12183 (5%)]\tLoss: 0.125989\n", + "Train Epoch: 1 [1280/12183 (10%)]\tLoss: 0.938824\n", + "Train Epoch: 1 [1920/12183 (16%)]\tLoss: 0.376103\n", + "Train Epoch: 1 [2560/12183 (21%)]\tLoss: 0.235958\n", + "Train Epoch: 1 [3200/12183 (26%)]\tLoss: 0.146971\n", + "Train Epoch: 1 [3840/12183 (31%)]\tLoss: 0.261188\n", + "Train Epoch: 1 [4480/12183 (37%)]\tLoss: 0.162076\n", + "Train Epoch: 1 [5120/12183 (42%)]\tLoss: 0.064259\n", + "Train Epoch: 1 [5760/12183 (47%)]\tLoss: 0.127866\n", + "Train Epoch: 1 [6400/12183 (52%)]\tLoss: 0.153409\n", + "Train Epoch: 1 [7040/12183 (58%)]\tLoss: 0.117668\n", + "Train Epoch: 1 [7680/12183 (63%)]\tLoss: 0.058572\n", + "Train Epoch: 1 [8320/12183 (68%)]\tLoss: 0.084697\n", + "Train Epoch: 1 [8960/12183 (73%)]\tLoss: 0.034420\n", + "Train Epoch: 1 [9600/12183 (79%)]\tLoss: 0.017252\n", + "Train Epoch: 1 [10240/12183 (84%)]\tLoss: 0.020837\n", + "Train Epoch: 1 [10880/12183 (89%)]\tLoss: 0.022506\n", + "Train Epoch: 1 [11520/12183 (94%)]\tLoss: 0.073931\n", + "Train Epoch: 1 [4370/12183 (99%)]\tLoss: 0.032285\n", + "\n", + "Test set: Average loss: 2.6226\n", + "Class 0 accuracy: 668.0/980(68%)\n", + "Class 1 accuracy: 975.0/1135(86%)\n", + "Class 2 accuracy: 475.0/1032(46%)\n", + "Class 3 accuracy: 489.0/1010(48%)\n", + "Class 4 accuracy: 691.0/982(70%)\n", + "Class 5 accuracy: 564.0/892(63%)\n", + "Class 6 accuracy: 948.0/958(99%)\n", + "Class 7 accuracy: 1003.0/1028(98%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 2 [0/12183 (0%)]\tLoss: 0.782327\n", - "Train Epoch: 2 [640/12183 (5%)]\tLoss: 0.875331\n", - "Train Epoch: 2 [1280/12183 (10%)]\tLoss: 0.926851\n", - "Train Epoch: 2 [1920/12183 (16%)]\tLoss: 0.874468\n", - "Train Epoch: 2 [2560/12183 (21%)]\tLoss: 0.925109\n", - "Train Epoch: 2 [3200/12183 (26%)]\tLoss: 0.954436\n", - "Train Epoch: 2 [3840/12183 (31%)]\tLoss: 0.942288\n", - "Train Epoch: 2 [4480/12183 (37%)]\tLoss: 0.940796\n", - "Train Epoch: 2 [5120/12183 (42%)]\tLoss: 0.857594\n", - "Train Epoch: 2 [5760/12183 (47%)]\tLoss: 1.031697\n", - "Train Epoch: 2 [6400/12183 (52%)]\tLoss: 0.929081\n", - "Train Epoch: 2 [7040/12183 (58%)]\tLoss: 0.983720\n", - "Train Epoch: 2 [7680/12183 (63%)]\tLoss: 0.876552\n", - "Train Epoch: 2 [8320/12183 (68%)]\tLoss: 0.934435\n", - "Train Epoch: 2 [8960/12183 (73%)]\tLoss: 1.003677\n", - "Train Epoch: 2 [9600/12183 (79%)]\tLoss: 0.931277\n", - "Train Epoch: 2 [10240/12183 (84%)]\tLoss: 1.008535\n", - "Train Epoch: 2 [10880/12183 (89%)]\tLoss: 0.810847\n", - "Train Epoch: 2 [11520/12183 (94%)]\tLoss: 0.702577\n", - "Train Epoch: 2 [4370/12183 (99%)]\tLoss: 0.964019\n", - "\n", - "Test set: Average loss: 1.6425\n", - "Class 0 accuracy: 804.0/980(82%)\n", - "Class 1 accuracy: 1065.0/1135(94%)\n", - "Class 2 accuracy: 527.0/1032(51%)\n", - "Class 3 accuracy: 774.0/1010(77%)\n", - "Class 4 accuracy: 629.0/982(64%)\n", - "Class 5 accuracy: 347.0/892(39%)\n", - "Class 6 accuracy: 931.0/958(97%)\n", - "Class 7 accuracy: 983.0/1028(96%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 3 [0/12183 (0%)]\tLoss: 0.658189\n", - "Train Epoch: 3 [640/12183 (5%)]\tLoss: 0.624558\n", - "Train Epoch: 3 [1280/12183 (10%)]\tLoss: 0.811082\n", - "Train Epoch: 3 [1920/12183 (16%)]\tLoss: 0.653154\n", - "Train Epoch: 3 [2560/12183 (21%)]\tLoss: 0.784420\n", - "Train Epoch: 3 [3200/12183 (26%)]\tLoss: 0.620821\n", - "Train Epoch: 3 [3840/12183 (31%)]\tLoss: 0.743391\n", - "Train Epoch: 3 [4480/12183 (37%)]\tLoss: 0.694466\n", - "Train Epoch: 3 [5120/12183 (42%)]\tLoss: 0.580736\n", - "Train Epoch: 3 [5760/12183 (47%)]\tLoss: 0.673027\n", - "Train Epoch: 3 [6400/12183 (52%)]\tLoss: 0.545014\n", - "Train Epoch: 3 [7040/12183 (58%)]\tLoss: 0.924152\n", - "Train Epoch: 3 [7680/12183 (63%)]\tLoss: 0.730344\n", - "Train Epoch: 3 [8320/12183 (68%)]\tLoss: 0.724853\n", - "Train Epoch: 3 [8960/12183 (73%)]\tLoss: 0.699868\n", - "Train Epoch: 3 [9600/12183 (79%)]\tLoss: 0.697215\n", - "Train Epoch: 3 [10240/12183 (84%)]\tLoss: 0.783108\n", - "Train Epoch: 3 [10880/12183 (89%)]\tLoss: 0.672869\n", - "Train Epoch: 3 [11520/12183 (94%)]\tLoss: 0.625879\n", - "Train Epoch: 3 [4370/12183 (99%)]\tLoss: 0.598810\n", - "\n", - "Test set: Average loss: 1.6302\n", - "Class 0 accuracy: 797.0/980(81%)\n", - "Class 1 accuracy: 1062.0/1135(94%)\n", - "Class 2 accuracy: 526.0/1032(51%)\n", - "Class 3 accuracy: 762.0/1010(75%)\n", - "Class 4 accuracy: 657.0/982(67%)\n", - "Class 5 accuracy: 415.0/892(47%)\n", - "Class 6 accuracy: 938.0/958(98%)\n", - "Class 7 accuracy: 992.0/1028(96%)\n", - "Class 8 accuracy: 0.0/974(0%)\n", - "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 4 [0/12183 (0%)]\tLoss: 0.686760\n", - "Train Epoch: 4 [640/12183 (5%)]\tLoss: 0.678613\n", - "Train Epoch: 4 [1280/12183 (10%)]\tLoss: 0.649789\n", - "Train Epoch: 4 [1920/12183 (16%)]\tLoss: 0.512429\n", - "Train Epoch: 4 [2560/12183 (21%)]\tLoss: 0.689496\n", - "Train Epoch: 4 [3200/12183 (26%)]\tLoss: 0.608655\n", - "Train Epoch: 4 [3840/12183 (31%)]\tLoss: 0.553349\n", - "Train Epoch: 4 [4480/12183 (37%)]\tLoss: 0.538118\n", - "Train Epoch: 4 [5120/12183 (42%)]\tLoss: 0.461560\n", - "Train Epoch: 4 [5760/12183 (47%)]\tLoss: 0.588525\n", - "Train Epoch: 4 [6400/12183 (52%)]\tLoss: 0.508286\n", - "Train Epoch: 4 [7040/12183 (58%)]\tLoss: 0.540536\n", - "Train Epoch: 4 [7680/12183 (63%)]\tLoss: 0.479708\n", - "Train Epoch: 4 [8320/12183 (68%)]\tLoss: 0.501095\n", - "Train Epoch: 4 [8960/12183 (73%)]\tLoss: 0.469510\n", - "Train Epoch: 4 [9600/12183 (79%)]\tLoss: 0.497756\n", - "Train Epoch: 4 [10240/12183 (84%)]\tLoss: 0.523196\n", - "Train Epoch: 4 [10880/12183 (89%)]\tLoss: 0.555491\n", - "Train Epoch: 4 [11520/12183 (94%)]\tLoss: 0.515540\n", - "Train Epoch: 4 [4370/12183 (99%)]\tLoss: 0.572875\n", - "\n", - "Test set: Average loss: 1.6347\n", - "Class 0 accuracy: 788.0/980(80%)\n", - "Class 1 accuracy: 1063.0/1135(94%)\n", - "Class 2 accuracy: 534.0/1032(52%)\n", - "Class 3 accuracy: 769.0/1010(76%)\n", - "Class 4 accuracy: 672.0/982(68%)\n", - "Class 5 accuracy: 440.0/892(49%)\n", - "Class 6 accuracy: 940.0/958(98%)\n", - "Class 7 accuracy: 996.0/1028(97%)\n", + "Train Epoch: 2 [0/12183 (0%)]\tLoss: 0.045590\n", + "Train Epoch: 2 [640/12183 (5%)]\tLoss: 0.091010\n", + "Train Epoch: 2 [1280/12183 (10%)]\tLoss: 0.021004\n", + "Train Epoch: 2 [1920/12183 (16%)]\tLoss: 0.036341\n", + "Train Epoch: 2 [2560/12183 (21%)]\tLoss: 0.021020\n", + "Train Epoch: 2 [3200/12183 (26%)]\tLoss: 0.036480\n", + "Train Epoch: 2 [3840/12183 (31%)]\tLoss: 0.043549\n", + "Train Epoch: 2 [4480/12183 (37%)]\tLoss: 0.029598\n", + "Train Epoch: 2 [5120/12183 (42%)]\tLoss: 0.053440\n", + "Train Epoch: 2 [5760/12183 (47%)]\tLoss: 0.016025\n", + "Train Epoch: 2 [6400/12183 (52%)]\tLoss: 0.022595\n", + "Train Epoch: 2 [7040/12183 (58%)]\tLoss: 0.008219\n", + "Train Epoch: 2 [7680/12183 (63%)]\tLoss: 0.047070\n", + "Train Epoch: 2 [8320/12183 (68%)]\tLoss: 0.021060\n", + "Train Epoch: 2 [8960/12183 (73%)]\tLoss: 0.024627\n", + "Train Epoch: 2 [9600/12183 (79%)]\tLoss: 0.031155\n", + "Train Epoch: 2 [10240/12183 (84%)]\tLoss: 0.045095\n", + "Train Epoch: 2 [10880/12183 (89%)]\tLoss: 0.015965\n", + "Train Epoch: 2 [11520/12183 (94%)]\tLoss: 0.025968\n", + "Train Epoch: 2 [4370/12183 (99%)]\tLoss: 0.043935\n", + "\n", + "Test set: Average loss: 3.1688\n", + "Class 0 accuracy: 682.0/980(70%)\n", + "Class 1 accuracy: 969.0/1135(85%)\n", + "Class 2 accuracy: 480.0/1032(47%)\n", + "Class 3 accuracy: 471.0/1010(47%)\n", + "Class 4 accuracy: 635.0/982(65%)\n", + "Class 5 accuracy: 484.0/892(54%)\n", + "Class 6 accuracy: 950.0/958(99%)\n", + "Class 7 accuracy: 1013.0/1028(99%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 5 [0/12183 (0%)]\tLoss: 0.595143\n", - "Train Epoch: 5 [640/12183 (5%)]\tLoss: 0.550319\n", - "Train Epoch: 5 [1280/12183 (10%)]\tLoss: 0.548414\n", - "Train Epoch: 5 [1920/12183 (16%)]\tLoss: 0.511655\n", - "Train Epoch: 5 [2560/12183 (21%)]\tLoss: 0.401490\n", - "Train Epoch: 5 [3200/12183 (26%)]\tLoss: 0.400202\n", - "Train Epoch: 5 [3840/12183 (31%)]\tLoss: 0.397857\n", - "Train Epoch: 5 [4480/12183 (37%)]\tLoss: 0.479915\n", - "Train Epoch: 5 [5120/12183 (42%)]\tLoss: 0.408378\n", - "Train Epoch: 5 [5760/12183 (47%)]\tLoss: 0.431881\n", - "Train Epoch: 5 [6400/12183 (52%)]\tLoss: 0.429216\n", - "Train Epoch: 5 [7040/12183 (58%)]\tLoss: 0.347318\n", - "Train Epoch: 5 [7680/12183 (63%)]\tLoss: 0.549904\n", - "Train Epoch: 5 [8320/12183 (68%)]\tLoss: 0.484814\n", - "Train Epoch: 5 [8960/12183 (73%)]\tLoss: 0.430466\n", - "Train Epoch: 5 [9600/12183 (79%)]\tLoss: 0.517198\n", - "Train Epoch: 5 [10240/12183 (84%)]\tLoss: 0.395947\n", - "Train Epoch: 5 [10880/12183 (89%)]\tLoss: 0.551409\n", - "Train Epoch: 5 [11520/12183 (94%)]\tLoss: 0.495534\n", - "Train Epoch: 5 [4370/12183 (99%)]\tLoss: 0.495679\n", - "\n", - "Test set: Average loss: 1.6534\n", - "Class 0 accuracy: 781.0/980(80%)\n", - "Class 1 accuracy: 1059.0/1135(93%)\n", - "Class 2 accuracy: 530.0/1032(51%)\n", - "Class 3 accuracy: 763.0/1010(76%)\n", - "Class 4 accuracy: 672.0/982(68%)\n", - "Class 5 accuracy: 444.0/892(50%)\n", - "Class 6 accuracy: 940.0/958(98%)\n", - "Class 7 accuracy: 1001.0/1028(97%)\n", + "Train Epoch: 3 [0/12183 (0%)]\tLoss: 0.019614\n", + "Train Epoch: 3 [640/12183 (5%)]\tLoss: 0.059921\n", + "Train Epoch: 3 [1280/12183 (10%)]\tLoss: 0.018621\n", + "Train Epoch: 3 [1920/12183 (16%)]\tLoss: 0.011714\n", + "Train Epoch: 3 [2560/12183 (21%)]\tLoss: 0.014679\n", + "Train Epoch: 3 [3200/12183 (26%)]\tLoss: 0.012920\n", + "Train Epoch: 3 [3840/12183 (31%)]\tLoss: 0.022788\n", + "Train Epoch: 3 [4480/12183 (37%)]\tLoss: 0.016748\n", + "Train Epoch: 3 [5120/12183 (42%)]\tLoss: 0.028296\n", + "Train Epoch: 3 [5760/12183 (47%)]\tLoss: 0.021148\n", + "Train Epoch: 3 [6400/12183 (52%)]\tLoss: 0.008148\n", + "Train Epoch: 3 [7040/12183 (58%)]\tLoss: 0.011808\n", + "Train Epoch: 3 [7680/12183 (63%)]\tLoss: 0.009274\n", + "Train Epoch: 3 [8320/12183 (68%)]\tLoss: 0.011613\n", + "Train Epoch: 3 [8960/12183 (73%)]\tLoss: 0.013502\n", + "Train Epoch: 3 [9600/12183 (79%)]\tLoss: 0.022795\n", + "Train Epoch: 3 [10240/12183 (84%)]\tLoss: 0.013261\n", + "Train Epoch: 3 [10880/12183 (89%)]\tLoss: 0.026117\n", + "Train Epoch: 3 [11520/12183 (94%)]\tLoss: 0.004542\n", + "Train Epoch: 3 [4370/12183 (99%)]\tLoss: 0.002898\n", + "\n", + "Test set: Average loss: 3.5287\n", + "Class 0 accuracy: 636.0/980(65%)\n", + "Class 1 accuracy: 955.0/1135(84%)\n", + "Class 2 accuracy: 451.0/1032(44%)\n", + "Class 3 accuracy: 432.0/1010(43%)\n", + "Class 4 accuracy: 562.0/982(57%)\n", + "Class 5 accuracy: 458.0/892(51%)\n", + "Class 6 accuracy: 952.0/958(99%)\n", + "Class 7 accuracy: 1018.0/1028(99%)\n", "Class 8 accuracy: 0.0/974(0%)\n", "Class 9 accuracy: 0.0/1009(0%)\n", - "Train Epoch: 1 [0/11800 (0%)]\tLoss: 0.848253\n", - "Train Epoch: 1 [640/11800 (5%)]\tLoss: 0.637744\n", - "Train Epoch: 1 [1280/11800 (11%)]\tLoss: 0.628570\n", - "Train Epoch: 1 [1920/11800 (16%)]\tLoss: 0.461403\n", - "Train Epoch: 1 [2560/11800 (22%)]\tLoss: 0.513085\n", - "Train Epoch: 1 [3200/11800 (27%)]\tLoss: 0.590126\n", - "Train Epoch: 1 [3840/11800 (32%)]\tLoss: 0.472829\n", - "Train Epoch: 1 [4480/11800 (38%)]\tLoss: 0.592700\n", - "Train Epoch: 1 [5120/11800 (43%)]\tLoss: 0.617516\n", - "Train Epoch: 1 [5760/11800 (49%)]\tLoss: 0.555048\n", - "Train Epoch: 1 [6400/11800 (54%)]\tLoss: 0.743809\n", - "Train Epoch: 1 [7040/11800 (59%)]\tLoss: 0.765445\n", - "Train Epoch: 1 [7680/11800 (65%)]\tLoss: 0.768416\n", - "Train Epoch: 1 [8320/11800 (70%)]\tLoss: 0.777279\n", - "Train Epoch: 1 [8960/11800 (76%)]\tLoss: 0.842107\n", - "Train Epoch: 1 [9600/11800 (81%)]\tLoss: 0.796940\n", - "Train Epoch: 1 [10240/11800 (86%)]\tLoss: 0.601572\n", - "Train Epoch: 1 [10880/11800 (92%)]\tLoss: 0.841399\n", - "Train Epoch: 1 [11520/11800 (97%)]\tLoss: 0.846165\n", - "\n", - "Test set: Average loss: 0.9925\n", - "Class 0 accuracy: 835.0/980(85%)\n", - "Class 1 accuracy: 1050.0/1135(93%)\n", - "Class 2 accuracy: 638.0/1032(62%)\n", - "Class 3 accuracy: 738.0/1010(73%)\n", - "Class 4 accuracy: 548.0/982(56%)\n", - "Class 5 accuracy: 170.0/892(19%)\n", - "Class 6 accuracy: 779.0/958(81%)\n", - "Class 7 accuracy: 776.0/1028(75%)\n", - "Class 8 accuracy: 908.0/974(93%)\n", - "Class 9 accuracy: 801.0/1009(79%)\n", - "Train Epoch: 2 [0/11800 (0%)]\tLoss: 0.883257\n", - "Train Epoch: 2 [640/11800 (5%)]\tLoss: 0.857211\n", - "Train Epoch: 2 [1280/11800 (11%)]\tLoss: 0.841614\n", - "Train Epoch: 2 [1920/11800 (16%)]\tLoss: 0.790240\n", - "Train Epoch: 2 [2560/11800 (22%)]\tLoss: 0.950122\n", - "Train Epoch: 2 [3200/11800 (27%)]\tLoss: 0.835133\n", - "Train Epoch: 2 [3840/11800 (32%)]\tLoss: 0.882796\n", - "Train Epoch: 2 [4480/11800 (38%)]\tLoss: 0.810285\n", - "Train Epoch: 2 [5120/11800 (43%)]\tLoss: 0.799711\n", - "Train Epoch: 2 [5760/11800 (49%)]\tLoss: 0.834429\n", - "Train Epoch: 2 [6400/11800 (54%)]\tLoss: 0.839871\n", - "Train Epoch: 2 [7040/11800 (59%)]\tLoss: 0.947290\n", - "Train Epoch: 2 [7680/11800 (65%)]\tLoss: 0.710232\n", - "Train Epoch: 2 [8320/11800 (70%)]\tLoss: 0.779843\n", - "Train Epoch: 2 [8960/11800 (76%)]\tLoss: 0.733478\n", - "Train Epoch: 2 [9600/11800 (81%)]\tLoss: 0.938040\n", - "Train Epoch: 2 [10240/11800 (86%)]\tLoss: 0.839119\n", - "Train Epoch: 2 [10880/11800 (92%)]\tLoss: 0.696384\n", - "Train Epoch: 2 [11520/11800 (97%)]\tLoss: 0.752226\n", - "\n", - "Test set: Average loss: 0.9674\n", - "Class 0 accuracy: 825.0/980(84%)\n", - "Class 1 accuracy: 1004.0/1135(88%)\n", - "Class 2 accuracy: 599.0/1032(58%)\n", - "Class 3 accuracy: 615.0/1010(61%)\n", - "Class 4 accuracy: 332.0/982(34%)\n", - "Class 5 accuracy: 42.0/892(5%)\n", - "Class 6 accuracy: 762.0/958(80%)\n", - "Class 7 accuracy: 623.0/1028(61%)\n", - "Class 8 accuracy: 943.0/974(97%)\n", - "Class 9 accuracy: 923.0/1009(91%)\n", - "Train Epoch: 3 [0/11800 (0%)]\tLoss: 0.708711\n", - "Train Epoch: 3 [640/11800 (5%)]\tLoss: 0.783265\n", - "Train Epoch: 3 [1280/11800 (11%)]\tLoss: 0.810831\n", - "Train Epoch: 3 [1920/11800 (16%)]\tLoss: 0.753713\n", - "Train Epoch: 3 [2560/11800 (22%)]\tLoss: 0.696868\n", - "Train Epoch: 3 [3200/11800 (27%)]\tLoss: 0.572866\n", - "Train Epoch: 3 [3840/11800 (32%)]\tLoss: 0.757275\n", - "Train Epoch: 3 [4480/11800 (38%)]\tLoss: 0.732344\n", - "Train Epoch: 3 [5120/11800 (43%)]\tLoss: 0.671541\n", - "Train Epoch: 3 [5760/11800 (49%)]\tLoss: 0.724425\n", - "Train Epoch: 3 [6400/11800 (54%)]\tLoss: 0.719062\n", - "Train Epoch: 3 [7040/11800 (59%)]\tLoss: 0.712015\n", - "Train Epoch: 3 [7680/11800 (65%)]\tLoss: 0.619695\n", - "Train Epoch: 3 [8320/11800 (70%)]\tLoss: 0.781576\n", - "Train Epoch: 3 [8960/11800 (76%)]\tLoss: 0.707311\n", - "Train Epoch: 3 [9600/11800 (81%)]\tLoss: 0.700315\n", - "Train Epoch: 3 [10240/11800 (86%)]\tLoss: 0.579610\n", - "Train Epoch: 3 [10880/11800 (92%)]\tLoss: 0.836607\n", - "Train Epoch: 3 [11520/11800 (97%)]\tLoss: 0.599106\n", - "\n", - "Test set: Average loss: 0.9365\n", - "Class 0 accuracy: 817.0/980(83%)\n", - "Class 1 accuracy: 996.0/1135(88%)\n", - "Class 2 accuracy: 598.0/1032(58%)\n", - "Class 3 accuracy: 612.0/1010(61%)\n", - "Class 4 accuracy: 291.0/982(30%)\n", - "Class 5 accuracy: 58.0/892(7%)\n", - "Class 6 accuracy: 772.0/958(81%)\n", - "Class 7 accuracy: 574.0/1028(56%)\n", - "Class 8 accuracy: 941.0/974(97%)\n", - "Class 9 accuracy: 948.0/1009(94%)\n", - "Train Epoch: 4 [0/11800 (0%)]\tLoss: 0.764324\n", - "Train Epoch: 4 [640/11800 (5%)]\tLoss: 0.750924\n", - "Train Epoch: 4 [1280/11800 (11%)]\tLoss: 0.597976\n", - "Train Epoch: 4 [1920/11800 (16%)]\tLoss: 0.711546\n", - "Train Epoch: 4 [2560/11800 (22%)]\tLoss: 0.744140\n", - "Train Epoch: 4 [3200/11800 (27%)]\tLoss: 0.756678\n", - "Train Epoch: 4 [3840/11800 (32%)]\tLoss: 0.566600\n", - "Train Epoch: 4 [4480/11800 (38%)]\tLoss: 0.728696\n", - "Train Epoch: 4 [5120/11800 (43%)]\tLoss: 0.720701\n", - "Train Epoch: 4 [5760/11800 (49%)]\tLoss: 0.566305\n", - "Train Epoch: 4 [6400/11800 (54%)]\tLoss: 0.689037\n", - "Train Epoch: 4 [7040/11800 (59%)]\tLoss: 0.678461\n", - "Train Epoch: 4 [7680/11800 (65%)]\tLoss: 0.550391\n", - "Train Epoch: 4 [8320/11800 (70%)]\tLoss: 0.694139\n", - "Train Epoch: 4 [8960/11800 (76%)]\tLoss: 0.700623\n", - "Train Epoch: 4 [9600/11800 (81%)]\tLoss: 0.623185\n", - "Train Epoch: 4 [10240/11800 (86%)]\tLoss: 0.696449\n", - "Train Epoch: 4 [10880/11800 (92%)]\tLoss: 0.732011\n", - "Train Epoch: 4 [11520/11800 (97%)]\tLoss: 0.734793\n", - "\n", - "Test set: Average loss: 0.9336\n", - "Class 0 accuracy: 809.0/980(83%)\n", - "Class 1 accuracy: 986.0/1135(87%)\n", - "Class 2 accuracy: 593.0/1032(57%)\n", - "Class 3 accuracy: 575.0/1010(57%)\n", - "Class 4 accuracy: 283.0/982(29%)\n", - "Class 5 accuracy: 60.0/892(7%)\n", - "Class 6 accuracy: 772.0/958(81%)\n", - "Class 7 accuracy: 550.0/1028(54%)\n", - "Class 8 accuracy: 943.0/974(97%)\n", - "Class 9 accuracy: 956.0/1009(95%)\n", - "Train Epoch: 5 [0/11800 (0%)]\tLoss: 0.598379\n", - "Train Epoch: 5 [640/11800 (5%)]\tLoss: 0.805639\n", - "Train Epoch: 5 [1280/11800 (11%)]\tLoss: 0.687005\n", - "Train Epoch: 5 [1920/11800 (16%)]\tLoss: 0.553637\n", - "Train Epoch: 5 [2560/11800 (22%)]\tLoss: 0.630970\n", - "Train Epoch: 5 [3200/11800 (27%)]\tLoss: 0.618934\n", - "Train Epoch: 5 [3840/11800 (32%)]\tLoss: 0.695465\n", - "Train Epoch: 5 [4480/11800 (38%)]\tLoss: 0.617699\n", - "Train Epoch: 5 [5120/11800 (43%)]\tLoss: 0.745449\n", - "Train Epoch: 5 [5760/11800 (49%)]\tLoss: 0.529404\n", - "Train Epoch: 5 [6400/11800 (54%)]\tLoss: 0.633640\n", - "Train Epoch: 5 [7040/11800 (59%)]\tLoss: 0.788254\n", - "Train Epoch: 5 [7680/11800 (65%)]\tLoss: 0.565721\n", - "Train Epoch: 5 [8320/11800 (70%)]\tLoss: 0.707190\n", - "Train Epoch: 5 [8960/11800 (76%)]\tLoss: 0.583630\n", - "Train Epoch: 5 [9600/11800 (81%)]\tLoss: 0.549434\n", - "Train Epoch: 5 [10240/11800 (86%)]\tLoss: 0.546574\n", - "Train Epoch: 5 [10880/11800 (92%)]\tLoss: 0.536187\n", - "Train Epoch: 5 [11520/11800 (97%)]\tLoss: 0.731755\n", - "\n", - "Test set: Average loss: 0.9307\n", - "Class 0 accuracy: 807.0/980(82%)\n", - "Class 1 accuracy: 977.0/1135(86%)\n", - "Class 2 accuracy: 588.0/1032(57%)\n", - "Class 3 accuracy: 548.0/1010(54%)\n", - "Class 4 accuracy: 283.0/982(29%)\n", - "Class 5 accuracy: 70.0/892(8%)\n", - "Class 6 accuracy: 772.0/958(81%)\n", - "Class 7 accuracy: 546.0/1028(53%)\n", + "Train Epoch: 1 [0/11800 (0%)]\tLoss: 0.473740\n", + "Train Epoch: 1 [640/11800 (5%)]\tLoss: 0.210103\n", + "Train Epoch: 1 [1280/11800 (11%)]\tLoss: 0.418402\n", + "Train Epoch: 1 [1920/11800 (16%)]\tLoss: 0.530582\n", + "Train Epoch: 1 [2560/11800 (22%)]\tLoss: 0.304206\n", + "Train Epoch: 1 [3200/11800 (27%)]\tLoss: 0.122092\n", + "Train Epoch: 1 [3840/11800 (32%)]\tLoss: 0.183575\n", + "Train Epoch: 1 [4480/11800 (38%)]\tLoss: 0.260152\n", + "Train Epoch: 1 [5120/11800 (43%)]\tLoss: 0.277376\n", + "Train Epoch: 1 [5760/11800 (49%)]\tLoss: 0.212683\n", + "Train Epoch: 1 [6400/11800 (54%)]\tLoss: 0.163834\n", + "Train Epoch: 1 [7040/11800 (59%)]\tLoss: 0.088070\n", + "Train Epoch: 1 [7680/11800 (65%)]\tLoss: 0.092493\n", + "Train Epoch: 1 [8320/11800 (70%)]\tLoss: 0.169504\n", + "Train Epoch: 1 [8960/11800 (76%)]\tLoss: 0.130592\n", + "Train Epoch: 1 [9600/11800 (81%)]\tLoss: 0.043165\n", + "Train Epoch: 1 [10240/11800 (86%)]\tLoss: 0.051642\n", + "Train Epoch: 1 [10880/11800 (92%)]\tLoss: 0.081378\n", + "Train Epoch: 1 [11520/11800 (97%)]\tLoss: 0.073074\n", + "\n", + "Test set: Average loss: 1.1909\n", + "Class 0 accuracy: 710.0/980(72%)\n", + "Class 1 accuracy: 937.0/1135(83%)\n", + "Class 2 accuracy: 525.0/1032(51%)\n", + "Class 3 accuracy: 409.0/1010(40%)\n", + "Class 4 accuracy: 368.0/982(37%)\n", + "Class 5 accuracy: 359.0/892(40%)\n", + "Class 6 accuracy: 782.0/958(82%)\n", + "Class 7 accuracy: 587.0/1028(57%)\n", "Class 8 accuracy: 945.0/974(97%)\n", - "Class 9 accuracy: 958.0/1009(95%)\n", - "Train Epoch: 1 [0/12665 (0%)]\tLoss: 0.606930\n", - "Train Epoch: 1 [640/12665 (5%)]\tLoss: 0.566195\n", - "Train Epoch: 1 [1280/12665 (10%)]\tLoss: 0.488845\n", - "Train Epoch: 1 [1920/12665 (15%)]\tLoss: 0.440726\n", - "Train Epoch: 1 [2560/12665 (20%)]\tLoss: 0.412488\n", - "Train Epoch: 1 [3200/12665 (25%)]\tLoss: 0.525131\n", - "Train Epoch: 1 [3840/12665 (30%)]\tLoss: 0.394614\n", - "Train Epoch: 1 [4480/12665 (35%)]\tLoss: 0.449948\n", - "Train Epoch: 1 [5120/12665 (40%)]\tLoss: 0.455428\n", - "Train Epoch: 1 [5760/12665 (45%)]\tLoss: 0.374721\n", - "Train Epoch: 1 [6400/12665 (51%)]\tLoss: 0.441609\n", - "Train Epoch: 1 [7040/12665 (56%)]\tLoss: 0.499441\n", - "Train Epoch: 1 [7680/12665 (61%)]\tLoss: 0.482373\n", - "Train Epoch: 1 [8320/12665 (66%)]\tLoss: 0.431031\n", - "Train Epoch: 1 [8960/12665 (71%)]\tLoss: 0.455554\n", - "Train Epoch: 1 [9600/12665 (76%)]\tLoss: 0.415438\n", - "Train Epoch: 1 [10240/12665 (81%)]\tLoss: 0.355638\n", - "Train Epoch: 1 [10880/12665 (86%)]\tLoss: 0.477118\n", - "Train Epoch: 1 [11520/12665 (91%)]\tLoss: 0.512437\n", - "Train Epoch: 1 [12160/12665 (96%)]\tLoss: 0.351764\n", - "\n", - "Test set: Average loss: 0.8019\n", - "Class 0 accuracy: 968.0/980(99%)\n", - "Class 1 accuracy: 1130.0/1135(100%)\n", - "Class 2 accuracy: 604.0/1032(59%)\n", - "Class 3 accuracy: 753.0/1010(75%)\n", - "Class 4 accuracy: 673.0/982(69%)\n", - "Class 5 accuracy: 496.0/892(56%)\n", - "Class 6 accuracy: 695.0/958(73%)\n", - "Class 7 accuracy: 825.0/1028(80%)\n", - "Class 8 accuracy: 589.0/974(60%)\n", - "Class 9 accuracy: 751.0/1009(74%)\n", - "Train Epoch: 2 [0/12665 (0%)]\tLoss: 0.382221\n", - "Train Epoch: 2 [640/12665 (5%)]\tLoss: 0.359024\n", - "Train Epoch: 2 [1280/12665 (10%)]\tLoss: 0.443718\n", - "Train Epoch: 2 [1920/12665 (15%)]\tLoss: 0.352842\n", - "Train Epoch: 2 [2560/12665 (20%)]\tLoss: 0.390380\n", - "Train Epoch: 2 [3200/12665 (25%)]\tLoss: 0.450553\n", - "Train Epoch: 2 [3840/12665 (30%)]\tLoss: 0.285430\n", - "Train Epoch: 2 [4480/12665 (35%)]\tLoss: 0.317991\n", - "Train Epoch: 2 [5120/12665 (40%)]\tLoss: 0.304995\n", - "Train Epoch: 2 [5760/12665 (45%)]\tLoss: 0.402834\n", - "Train Epoch: 2 [6400/12665 (51%)]\tLoss: 0.410913\n", - "Train Epoch: 2 [7040/12665 (56%)]\tLoss: 0.323035\n", - "Train Epoch: 2 [7680/12665 (61%)]\tLoss: 0.359033\n", - "Train Epoch: 2 [8320/12665 (66%)]\tLoss: 0.332048\n", - "Train Epoch: 2 [8960/12665 (71%)]\tLoss: 0.344597\n", - "Train Epoch: 2 [9600/12665 (76%)]\tLoss: 0.363583\n", - "Train Epoch: 2 [10240/12665 (81%)]\tLoss: 0.267385\n", - "Train Epoch: 2 [10880/12665 (86%)]\tLoss: 0.303813\n", - "Train Epoch: 2 [11520/12665 (91%)]\tLoss: 0.394272\n", - "Train Epoch: 2 [12160/12665 (96%)]\tLoss: 0.275770\n", - "\n", - "Test set: Average loss: 0.7998\n", - "Class 0 accuracy: 970.0/980(99%)\n", - "Class 1 accuracy: 1130.0/1135(100%)\n", - "Class 2 accuracy: 586.0/1032(57%)\n", - "Class 3 accuracy: 743.0/1010(74%)\n", - "Class 4 accuracy: 675.0/982(69%)\n", - "Class 5 accuracy: 498.0/892(56%)\n", - "Class 6 accuracy: 686.0/958(72%)\n", - "Class 7 accuracy: 819.0/1028(80%)\n", - "Class 8 accuracy: 607.0/974(62%)\n", - "Class 9 accuracy: 756.0/1009(75%)\n", - "Train Epoch: 3 [0/12665 (0%)]\tLoss: 0.361247\n", - "Train Epoch: 3 [640/12665 (5%)]\tLoss: 0.295900\n", - "Train Epoch: 3 [1280/12665 (10%)]\tLoss: 0.298250\n", - "Train Epoch: 3 [1920/12665 (15%)]\tLoss: 0.288731\n", - "Train Epoch: 3 [2560/12665 (20%)]\tLoss: 0.304051\n", - "Train Epoch: 3 [3200/12665 (25%)]\tLoss: 0.301107\n", - "Train Epoch: 3 [3840/12665 (30%)]\tLoss: 0.264046\n", - "Train Epoch: 3 [4480/12665 (35%)]\tLoss: 0.236352\n", - "Train Epoch: 3 [5120/12665 (40%)]\tLoss: 0.251096\n", - "Train Epoch: 3 [5760/12665 (45%)]\tLoss: 0.369783\n", - "Train Epoch: 3 [6400/12665 (51%)]\tLoss: 0.356867\n", - "Train Epoch: 3 [7040/12665 (56%)]\tLoss: 0.247389\n", - "Train Epoch: 3 [7680/12665 (61%)]\tLoss: 0.264304\n", - "Train Epoch: 3 [8320/12665 (66%)]\tLoss: 0.243768\n", - "Train Epoch: 3 [8960/12665 (71%)]\tLoss: 0.328787\n", - "Train Epoch: 3 [9600/12665 (76%)]\tLoss: 0.294538\n", - "Train Epoch: 3 [10240/12665 (81%)]\tLoss: 0.300040\n", - "Train Epoch: 3 [10880/12665 (86%)]\tLoss: 0.258787\n", - "Train Epoch: 3 [11520/12665 (91%)]\tLoss: 0.283641\n", - "Train Epoch: 3 [12160/12665 (96%)]\tLoss: 0.338280\n", - "\n", - "Test set: Average loss: 0.7930\n", - "Class 0 accuracy: 971.0/980(99%)\n", - "Class 1 accuracy: 1130.0/1135(100%)\n", - "Class 2 accuracy: 589.0/1032(57%)\n", - "Class 3 accuracy: 741.0/1010(73%)\n", - "Class 4 accuracy: 671.0/982(68%)\n", - "Class 5 accuracy: 516.0/892(58%)\n", - "Class 6 accuracy: 685.0/958(72%)\n", - "Class 7 accuracy: 813.0/1028(79%)\n", - "Class 8 accuracy: 608.0/974(62%)\n", - "Class 9 accuracy: 755.0/1009(75%)\n", - "Train Epoch: 4 [0/12665 (0%)]\tLoss: 0.236698\n", - "Train Epoch: 4 [640/12665 (5%)]\tLoss: 0.221445\n", - "Train Epoch: 4 [1280/12665 (10%)]\tLoss: 0.236138\n", - "Train Epoch: 4 [1920/12665 (15%)]\tLoss: 0.257844\n", - "Train Epoch: 4 [2560/12665 (20%)]\tLoss: 0.233735\n", - "Train Epoch: 4 [3200/12665 (25%)]\tLoss: 0.222089\n", - "Train Epoch: 4 [3840/12665 (30%)]\tLoss: 0.307528\n", - "Train Epoch: 4 [4480/12665 (35%)]\tLoss: 0.285662\n", - "Train Epoch: 4 [5120/12665 (40%)]\tLoss: 0.240001\n", - "Train Epoch: 4 [5760/12665 (45%)]\tLoss: 0.292137\n", - "Train Epoch: 4 [6400/12665 (51%)]\tLoss: 0.290430\n", - "Train Epoch: 4 [7040/12665 (56%)]\tLoss: 0.299295\n", - "Train Epoch: 4 [7680/12665 (61%)]\tLoss: 0.255210\n", - "Train Epoch: 4 [8320/12665 (66%)]\tLoss: 0.203961\n", - "Train Epoch: 4 [8960/12665 (71%)]\tLoss: 0.238729\n", - "Train Epoch: 4 [9600/12665 (76%)]\tLoss: 0.249784\n", - "Train Epoch: 4 [10240/12665 (81%)]\tLoss: 0.193929\n", - "Train Epoch: 4 [10880/12665 (86%)]\tLoss: 0.198174\n", - "Train Epoch: 4 [11520/12665 (91%)]\tLoss: 0.270540\n", - "Train Epoch: 4 [12160/12665 (96%)]\tLoss: 0.217733\n", - "\n", - "Test set: Average loss: 0.8017\n", - "Class 0 accuracy: 971.0/980(99%)\n", - "Class 1 accuracy: 1130.0/1135(100%)\n", - "Class 2 accuracy: 575.0/1032(56%)\n", - "Class 3 accuracy: 736.0/1010(73%)\n", - "Class 4 accuracy: 671.0/982(68%)\n", - "Class 5 accuracy: 511.0/892(57%)\n", - "Class 6 accuracy: 682.0/958(71%)\n", - "Class 7 accuracy: 814.0/1028(79%)\n", - "Class 8 accuracy: 607.0/974(62%)\n", - "Class 9 accuracy: 750.0/1009(74%)\n", - "Train Epoch: 5 [0/12665 (0%)]\tLoss: 0.226164\n", - "Train Epoch: 5 [640/12665 (5%)]\tLoss: 0.318618\n", - "Train Epoch: 5 [1280/12665 (10%)]\tLoss: 0.238432\n", - "Train Epoch: 5 [1920/12665 (15%)]\tLoss: 0.262093\n", - "Train Epoch: 5 [2560/12665 (20%)]\tLoss: 0.286501\n", - "Train Epoch: 5 [3200/12665 (25%)]\tLoss: 0.219213\n", - "Train Epoch: 5 [3840/12665 (30%)]\tLoss: 0.250186\n", - "Train Epoch: 5 [4480/12665 (35%)]\tLoss: 0.270830\n", - "Train Epoch: 5 [5120/12665 (40%)]\tLoss: 0.149153\n", - "Train Epoch: 5 [5760/12665 (45%)]\tLoss: 0.206908\n", - "Train Epoch: 5 [6400/12665 (51%)]\tLoss: 0.237038\n", - "Train Epoch: 5 [7040/12665 (56%)]\tLoss: 0.289587\n", - "Train Epoch: 5 [7680/12665 (61%)]\tLoss: 0.243903\n", - "Train Epoch: 5 [8320/12665 (66%)]\tLoss: 0.234777\n", - "Train Epoch: 5 [8960/12665 (71%)]\tLoss: 0.194367\n", - "Train Epoch: 5 [9600/12665 (76%)]\tLoss: 0.248726\n", - "Train Epoch: 5 [10240/12665 (81%)]\tLoss: 0.233232\n", - "Train Epoch: 5 [10880/12665 (86%)]\tLoss: 0.258024\n", - "Train Epoch: 5 [11520/12665 (91%)]\tLoss: 0.280726\n", - "Train Epoch: 5 [12160/12665 (96%)]\tLoss: 0.206879\n", - "\n", - "Test set: Average loss: 0.8039\n", - "Class 0 accuracy: 971.0/980(99%)\n", - "Class 1 accuracy: 1130.0/1135(100%)\n", - "Class 2 accuracy: 575.0/1032(56%)\n", - "Class 3 accuracy: 739.0/1010(73%)\n", - "Class 4 accuracy: 672.0/982(68%)\n", - "Class 5 accuracy: 520.0/892(58%)\n", - "Class 6 accuracy: 676.0/958(71%)\n", - "Class 7 accuracy: 812.0/1028(79%)\n", - "Class 8 accuracy: 597.0/974(61%)\n", - "Class 9 accuracy: 746.0/1009(74%)\n" + "Class 9 accuracy: 974.0/1009(97%)\n", + "Train Epoch: 2 [0/11800 (0%)]\tLoss: 0.106367\n", + "Train Epoch: 2 [640/11800 (5%)]\tLoss: 0.110765\n", + "Train Epoch: 2 [1280/11800 (11%)]\tLoss: 0.168906\n", + "Train Epoch: 2 [1920/11800 (16%)]\tLoss: 0.130259\n", + "Train Epoch: 2 [2560/11800 (22%)]\tLoss: 0.099700\n", + "Train Epoch: 2 [3200/11800 (27%)]\tLoss: 0.045566\n", + "Train Epoch: 2 [3840/11800 (32%)]\tLoss: 0.032498\n", + "Train Epoch: 2 [4480/11800 (38%)]\tLoss: 0.127420\n", + "Train Epoch: 2 [5120/11800 (43%)]\tLoss: 0.127244\n", + "Train Epoch: 2 [5760/11800 (49%)]\tLoss: 0.123305\n", + "Train Epoch: 2 [6400/11800 (54%)]\tLoss: 0.033301\n", + "Train Epoch: 2 [7040/11800 (59%)]\tLoss: 0.050663\n", + "Train Epoch: 2 [7680/11800 (65%)]\tLoss: 0.029956\n", + "Train Epoch: 2 [8320/11800 (70%)]\tLoss: 0.016801\n", + "Train Epoch: 2 [8960/11800 (76%)]\tLoss: 0.067689\n", + "Train Epoch: 2 [9600/11800 (81%)]\tLoss: 0.044518\n", + "Train Epoch: 2 [10240/11800 (86%)]\tLoss: 0.073524\n", + "Train Epoch: 2 [10880/11800 (92%)]\tLoss: 0.035689\n", + "Train Epoch: 2 [11520/11800 (97%)]\tLoss: 0.084134\n", + "\n", + "Test set: Average loss: 1.5479\n", + "Class 0 accuracy: 684.0/980(70%)\n", + "Class 1 accuracy: 893.0/1135(79%)\n", + "Class 2 accuracy: 470.0/1032(46%)\n", + "Class 3 accuracy: 346.0/1010(34%)\n", + "Class 4 accuracy: 312.0/982(32%)\n", + "Class 5 accuracy: 298.0/892(33%)\n", + "Class 6 accuracy: 758.0/958(79%)\n", + "Class 7 accuracy: 583.0/1028(57%)\n", + "Class 8 accuracy: 953.0/974(98%)\n", + "Class 9 accuracy: 977.0/1009(97%)\n", + "Train Epoch: 3 [0/11800 (0%)]\tLoss: 0.106828\n", + "Train Epoch: 3 [640/11800 (5%)]\tLoss: 0.064151\n", + "Train Epoch: 3 [1280/11800 (11%)]\tLoss: 0.064141\n", + "Train Epoch: 3 [1920/11800 (16%)]\tLoss: 0.046738\n", + "Train Epoch: 3 [2560/11800 (22%)]\tLoss: 0.060965\n", + "Train Epoch: 3 [3200/11800 (27%)]\tLoss: 0.041177\n", + "Train Epoch: 3 [3840/11800 (32%)]\tLoss: 0.015125\n", + "Train Epoch: 3 [4480/11800 (38%)]\tLoss: 0.036628\n", + "Train Epoch: 3 [5120/11800 (43%)]\tLoss: 0.040213\n", + "Train Epoch: 3 [5760/11800 (49%)]\tLoss: 0.038948\n", + "Train Epoch: 3 [6400/11800 (54%)]\tLoss: 0.065069\n", + "Train Epoch: 3 [7040/11800 (59%)]\tLoss: 0.060684\n", + "Train Epoch: 3 [7680/11800 (65%)]\tLoss: 0.051811\n", + "Train Epoch: 3 [8320/11800 (70%)]\tLoss: 0.096974\n", + "Train Epoch: 3 [8960/11800 (76%)]\tLoss: 0.014142\n", + "Train Epoch: 3 [9600/11800 (81%)]\tLoss: 0.068916\n", + "Train Epoch: 3 [10240/11800 (86%)]\tLoss: 0.047880\n", + "Train Epoch: 3 [10880/11800 (92%)]\tLoss: 0.007572\n", + "Train Epoch: 3 [11520/11800 (97%)]\tLoss: 0.025779\n", + "\n", + "Test set: Average loss: 1.6151\n", + "Class 0 accuracy: 647.0/980(66%)\n", + "Class 1 accuracy: 885.0/1135(78%)\n", + "Class 2 accuracy: 492.0/1032(48%)\n", + "Class 3 accuracy: 379.0/1010(38%)\n", + "Class 4 accuracy: 323.0/982(33%)\n", + "Class 5 accuracy: 310.0/892(35%)\n", + "Class 6 accuracy: 736.0/958(77%)\n", + "Class 7 accuracy: 546.0/1028(53%)\n", + "Class 8 accuracy: 961.0/974(99%)\n", + "Class 9 accuracy: 976.0/1009(97%)\n", + "Train Epoch: 1 [0/12665 (0%)]\tLoss: 0.030852\n", + "Train Epoch: 1 [640/12665 (5%)]\tLoss: 0.051628\n", + "Train Epoch: 1 [1280/12665 (10%)]\tLoss: 0.097404\n", + "Train Epoch: 1 [1920/12665 (15%)]\tLoss: 0.066156\n", + "Train Epoch: 1 [2560/12665 (20%)]\tLoss: 0.085003\n", + "Train Epoch: 1 [3200/12665 (25%)]\tLoss: 0.022351\n", + "Train Epoch: 1 [3840/12665 (30%)]\tLoss: 0.047033\n", + "Train Epoch: 1 [4480/12665 (35%)]\tLoss: 0.021085\n", + "Train Epoch: 1 [5120/12665 (40%)]\tLoss: 0.038875\n", + "Train Epoch: 1 [5760/12665 (45%)]\tLoss: 0.035716\n", + "Train Epoch: 1 [6400/12665 (51%)]\tLoss: 0.033066\n", + "Train Epoch: 1 [7040/12665 (56%)]\tLoss: 0.020775\n", + "Train Epoch: 1 [7680/12665 (61%)]\tLoss: 0.027079\n", + "Train Epoch: 1 [8320/12665 (66%)]\tLoss: 0.071041\n", + "Train Epoch: 1 [8960/12665 (71%)]\tLoss: 0.030695\n", + "Train Epoch: 1 [9600/12665 (76%)]\tLoss: 0.038757\n", + "Train Epoch: 1 [10240/12665 (81%)]\tLoss: 0.003501\n", + "Train Epoch: 1 [10880/12665 (86%)]\tLoss: 0.011310\n", + "Train Epoch: 1 [11520/12665 (91%)]\tLoss: 0.057821\n", + "Train Epoch: 1 [12160/12665 (96%)]\tLoss: 0.016945\n", + "\n", + "Test set: Average loss: 1.3637\n", + "Class 0 accuracy: 977.0/980(100%)\n", + "Class 1 accuracy: 1133.0/1135(100%)\n", + "Class 2 accuracy: 574.0/1032(56%)\n", + "Class 3 accuracy: 561.0/1010(56%)\n", + "Class 4 accuracy: 494.0/982(50%)\n", + "Class 5 accuracy: 328.0/892(37%)\n", + "Class 6 accuracy: 656.0/958(68%)\n", + "Class 7 accuracy: 753.0/1028(73%)\n", + "Class 8 accuracy: 701.0/974(72%)\n", + "Class 9 accuracy: 866.0/1009(86%)\n", + "Train Epoch: 2 [0/12665 (0%)]\tLoss: 0.016473\n", + "Train Epoch: 2 [640/12665 (5%)]\tLoss: 0.007508\n", + "Train Epoch: 2 [1280/12665 (10%)]\tLoss: 0.037633\n", + "Train Epoch: 2 [1920/12665 (15%)]\tLoss: 0.026398\n", + "Train Epoch: 2 [2560/12665 (20%)]\tLoss: 0.004491\n", + "Train Epoch: 2 [3200/12665 (25%)]\tLoss: 0.002447\n", + "Train Epoch: 2 [3840/12665 (30%)]\tLoss: 0.007396\n", + "Train Epoch: 2 [4480/12665 (35%)]\tLoss: 0.022243\n", + "Train Epoch: 2 [5120/12665 (40%)]\tLoss: 0.032471\n", + "Train Epoch: 2 [5760/12665 (45%)]\tLoss: 0.012960\n", + "Train Epoch: 2 [6400/12665 (51%)]\tLoss: 0.009143\n", + "Train Epoch: 2 [7040/12665 (56%)]\tLoss: 0.005618\n", + "Train Epoch: 2 [7680/12665 (61%)]\tLoss: 0.016858\n", + "Train Epoch: 2 [8320/12665 (66%)]\tLoss: 0.015019\n", + "Train Epoch: 2 [8960/12665 (71%)]\tLoss: 0.042326\n", + "Train Epoch: 2 [9600/12665 (76%)]\tLoss: 0.005700\n", + "Train Epoch: 2 [10240/12665 (81%)]\tLoss: 0.020611\n", + "Train Epoch: 2 [10880/12665 (86%)]\tLoss: 0.011131\n", + "Train Epoch: 2 [11520/12665 (91%)]\tLoss: 0.019555\n", + "Train Epoch: 2 [12160/12665 (96%)]\tLoss: 0.015460\n", + "\n", + "Test set: Average loss: 1.5375\n", + "Class 0 accuracy: 977.0/980(100%)\n", + "Class 1 accuracy: 1133.0/1135(100%)\n", + "Class 2 accuracy: 500.0/1032(48%)\n", + "Class 3 accuracy: 503.0/1010(50%)\n", + "Class 4 accuracy: 489.0/982(50%)\n", + "Class 5 accuracy: 362.0/892(41%)\n", + "Class 6 accuracy: 639.0/958(67%)\n", + "Class 7 accuracy: 746.0/1028(73%)\n", + "Class 8 accuracy: 657.0/974(67%)\n", + "Class 9 accuracy: 847.0/1009(84%)\n", + "Train Epoch: 3 [0/12665 (0%)]\tLoss: 0.003884\n", + "Train Epoch: 3 [640/12665 (5%)]\tLoss: 0.024282\n", + "Train Epoch: 3 [1280/12665 (10%)]\tLoss: 0.007995\n", + "Train Epoch: 3 [1920/12665 (15%)]\tLoss: 0.006038\n", + "Train Epoch: 3 [2560/12665 (20%)]\tLoss: 0.007882\n", + "Train Epoch: 3 [3200/12665 (25%)]\tLoss: 0.008968\n", + "Train Epoch: 3 [3840/12665 (30%)]\tLoss: 0.007985\n", + "Train Epoch: 3 [4480/12665 (35%)]\tLoss: 0.017382\n", + "Train Epoch: 3 [5120/12665 (40%)]\tLoss: 0.006549\n", + "Train Epoch: 3 [5760/12665 (45%)]\tLoss: 0.006111\n", + "Train Epoch: 3 [6400/12665 (51%)]\tLoss: 0.016712\n", + "Train Epoch: 3 [7040/12665 (56%)]\tLoss: 0.009508\n", + "Train Epoch: 3 [7680/12665 (61%)]\tLoss: 0.013591\n", + "Train Epoch: 3 [8320/12665 (66%)]\tLoss: 0.002323\n", + "Train Epoch: 3 [8960/12665 (71%)]\tLoss: 0.003972\n", + "Train Epoch: 3 [9600/12665 (76%)]\tLoss: 0.002240\n", + "Train Epoch: 3 [10240/12665 (81%)]\tLoss: 0.012233\n", + "Train Epoch: 3 [10880/12665 (86%)]\tLoss: 0.046749\n", + "Train Epoch: 3 [11520/12665 (91%)]\tLoss: 0.006683\n", + "Train Epoch: 3 [12160/12665 (96%)]\tLoss: 0.006137\n", + "\n", + "Test set: Average loss: 1.5670\n", + "Class 0 accuracy: 976.0/980(100%)\n", + "Class 1 accuracy: 1134.0/1135(100%)\n", + "Class 2 accuracy: 509.0/1032(49%)\n", + "Class 3 accuracy: 520.0/1010(51%)\n", + "Class 4 accuracy: 496.0/982(51%)\n", + "Class 5 accuracy: 355.0/892(40%)\n", + "Class 6 accuracy: 625.0/958(65%)\n", + "Class 7 accuracy: 743.0/1028(72%)\n", + "Class 8 accuracy: 654.0/974(67%)\n", + "Class 9 accuracy: 854.0/1009(85%)\n" ] + }, + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[{0: 0.9959183673469387,\n", + " 1: 0.9991189427312775,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '0_1'},\n", + " {0: 0.9959183673469387,\n", + " 1: 1.0,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '0_1'},\n", + " {0: 0.996938775510204,\n", + " 1: 1.0,\n", + " 2: 0.0,\n", + " 3: 0.0,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '0_1'},\n", + " {0: 0.8459183673469388,\n", + " 1: 0.9215859030837005,\n", + " 2: 0.9602713178294574,\n", + " 3: 0.9732673267326732,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '2_3'},\n", + " {0: 0.7795918367346939,\n", + " 1: 0.8801762114537445,\n", + " 2: 0.9689922480620154,\n", + " 3: 0.9782178217821782,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '2_3'},\n", + " {0: 0.7377551020408163,\n", + " 1: 0.852863436123348,\n", + " 2: 0.9680232558139535,\n", + " 3: 0.9821782178217822,\n", + " 4: 0.0,\n", + " 5: 0.0,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '2_3'},\n", + " {0: 0.7755102040816326,\n", + " 1: 0.8687224669603524,\n", + " 2: 0.6434108527131783,\n", + " 3: 0.36336633663366336,\n", + " 4: 0.9969450101832994,\n", + " 5: 0.968609865470852,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '4_5'},\n", + " {0: 0.6602040816326531,\n", + " 1: 0.8546255506607929,\n", + " 2: 0.5968992248062015,\n", + " 3: 0.2376237623762376,\n", + " 4: 0.9969450101832994,\n", + " 5: 0.9820627802690582,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '4_5'},\n", + " {0: 0.6755102040816326,\n", + " 1: 0.8563876651982378,\n", + " 2: 0.5406976744186046,\n", + " 3: 0.22574257425742575,\n", + " 4: 0.9959266802443992,\n", + " 5: 0.9899103139013453,\n", + " 6: 0.0,\n", + " 7: 0.0,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '4_5'},\n", + " {0: 0.6816326530612244,\n", + " 1: 0.8590308370044053,\n", + " 2: 0.46027131782945735,\n", + " 3: 0.48415841584158414,\n", + " 4: 0.7036659877800407,\n", + " 5: 0.6322869955156951,\n", + " 6: 0.9895615866388309,\n", + " 7: 0.97568093385214,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '6_7'},\n", + " {0: 0.6959183673469388,\n", + " 1: 0.8537444933920705,\n", + " 2: 0.46511627906976744,\n", + " 3: 0.46633663366336636,\n", + " 4: 0.6466395112016293,\n", + " 5: 0.5426008968609866,\n", + " 6: 0.9916492693110647,\n", + " 7: 0.9854085603112841,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '6_7'},\n", + " {0: 0.6489795918367347,\n", + " 1: 0.8414096916299559,\n", + " 2: 0.437015503875969,\n", + " 3: 0.4277227722772277,\n", + " 4: 0.5723014256619144,\n", + " 5: 0.5134529147982063,\n", + " 6: 0.9937369519832986,\n", + " 7: 0.9902723735408561,\n", + " 8: 0.0,\n", + " 9: 0.0,\n", + " 'phase': '6_7'},\n", + " {0: 0.7244897959183674,\n", + " 1: 0.8255506607929516,\n", + " 2: 0.5087209302325582,\n", + " 3: 0.404950495049505,\n", + " 4: 0.37474541751527496,\n", + " 5: 0.4024663677130045,\n", + " 6: 0.8162839248434238,\n", + " 7: 0.5710116731517509,\n", + " 8: 0.9702258726899384,\n", + " 9: 0.9653121902874133,\n", + " 'phase': '8_9'},\n", + " {0: 0.6979591836734694,\n", + " 1: 0.786784140969163,\n", + " 2: 0.45542635658914726,\n", + " 3: 0.3425742574257426,\n", + " 4: 0.31771894093686354,\n", + " 5: 0.33408071748878926,\n", + " 6: 0.791231732776618,\n", + " 7: 0.5671206225680934,\n", + " 8: 0.9784394250513347,\n", + " 9: 0.9682854311199207,\n", + " 'phase': '8_9'},\n", + " {0: 0.6602040816326531,\n", + " 1: 0.7797356828193832,\n", + " 2: 0.47674418604651164,\n", + " 3: 0.37524752475247525,\n", + " 4: 0.3289205702647658,\n", + " 5: 0.3475336322869955,\n", + " 6: 0.7682672233820459,\n", + " 7: 0.5311284046692607,\n", + " 8: 0.9866529774127311,\n", + " 9: 0.9672943508424182,\n", + " 'phase': '8_9'},\n", + " {0: 0.996938775510204,\n", + " 1: 0.9982378854625551,\n", + " 2: 0.5562015503875969,\n", + " 3: 0.5554455445544555,\n", + " 4: 0.5030549898167006,\n", + " 5: 0.36771300448430494,\n", + " 6: 0.6847599164926931,\n", + " 7: 0.7324902723735408,\n", + " 8: 0.7197125256673511,\n", + " 9: 0.8582755203171457,\n", + " 'phase': '0_1_again'},\n", + " {0: 0.996938775510204,\n", + " 1: 0.9982378854625551,\n", + " 2: 0.4844961240310077,\n", + " 3: 0.498019801980198,\n", + " 4: 0.4979633401221996,\n", + " 5: 0.40582959641255606,\n", + " 6: 0.6670146137787056,\n", + " 7: 0.72568093385214,\n", + " 8: 0.6745379876796714,\n", + " 9: 0.8394449950445986,\n", + " 'phase': '0_1_again'},\n", + " {0: 0.9959183673469387,\n", + " 1: 0.9991189427312775,\n", + " 2: 0.4932170542635659,\n", + " 3: 0.5148514851485149,\n", + " 4: 0.505091649694501,\n", + " 5: 0.39798206278026904,\n", + " 6: 0.6524008350730689,\n", + " 7: 0.7227626459143969,\n", + " 8: 0.6714579055441479,\n", + " 9: 0.846382556987116,\n", + " 'phase': '0_1_again'}]" + ] + }, + "metadata": {}, + "execution_count": 20 } ] }, @@ -3067,11 +2125,571 @@ }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 11, "metadata": { - "id": "gimHfUXdW4_K" + "id": "gimHfUXdW4_K", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 459 + }, + "outputId": "3a07ff95-27df-468d-f571-414c99f1ca1e" }, - "outputs": [], + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + " epoch 0 1 2 3 4 5 6 7 8 9 phase\n", + "0 0 0.96 0.97 0.82 0.85 0.59 0.35 0.86 0.85 0.52 0.82 baseline" + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epoch0123456789phase
000.960.970.820.850.590.350.860.850.520.82baseline
\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + " \n", + "
\n", + "
\n", + " " + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + " epoch 0 1 2 3 4 5 6 7 8 9 phase\n", + "0 0 1.0 1.0 0.00 0.0 0.00 0.00 0.0 0.00 0.0 0.00 0_1\n", + "1 1 0.0 0.0 0.14 1.0 0.00 0.00 0.0 0.00 0.0 0.00 2_3\n", + "2 2 0.0 0.0 0.00 0.0 0.99 0.89 0.0 0.00 0.0 0.00 4_5\n", + "3 3 0.0 0.0 0.00 0.0 0.00 0.00 1.0 0.93 0.0 0.00 6_7\n", + "4 4 0.0 0.0 0.00 0.0 0.00 0.00 0.0 0.00 1.0 0.52 8_9" + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epoch0123456789phase
001.01.00.000.00.000.000.00.000.00.000_1
110.00.00.141.00.000.000.00.000.00.002_3
220.00.00.000.00.990.890.00.000.00.004_5
330.00.00.000.00.000.001.00.930.00.006_7
440.00.00.000.00.000.000.00.001.00.528_9
\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + " \n", + "
\n", + "
\n", + " " + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + " epoch 0 1 2 3 4 5 6 7 8 9 phase\n", + "0 0 1.00 0.99 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0_1\n", + "1 1 0.04 0.51 0.23 1.00 0.00 0.00 0.00 0.00 0.00 0.00 2_3\n", + "2 2 0.71 0.64 0.06 0.00 0.98 0.85 0.00 0.00 0.00 0.00 4_5\n", + "3 3 0.23 0.53 0.00 0.01 0.00 0.00 0.99 0.96 0.00 0.00 6_7\n", + "4 4 0.44 0.55 0.00 0.00 0.00 0.00 0.19 0.00 0.98 0.88 8_9" + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epoch0123456789phase
001.000.990.000.000.000.000.000.000.000.000_1
110.040.510.231.000.000.000.000.000.000.002_3
220.710.640.060.000.980.850.000.000.000.004_5
330.230.530.000.010.000.000.990.960.000.006_7
440.440.550.000.000.000.000.190.000.980.888_9
\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + " \n", + "
\n", + "
\n", + " " + ] + }, + "metadata": {} + } + ], "source": [ "# The following helper code takes the logs and converts them into a dataframe\n", "# for easier reading. You can also store the result as a CSV or HDF file by\n", @@ -3087,17 +2705,33 @@ " return logs\n", "\n", "#display(format_results(history_regular_mnist).head())\n", - "#display(format_results(history_catastrophic_forgetting).head())\n", - "#display(format_results(history_memory_replay).head())" + "display(format_results(history_catastrophic_forgetting).head())\n", + "display(format_results(history_memory_replay).head())" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 25, "metadata": { - "id": "jR6eRKn4WguU" + "id": "jR6eRKn4WguU", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 807 + }, + "outputId": "a20c835b-1f62-4b6c-cd45-745f122a0041" }, - "outputs": [], + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk0AAAMWCAYAAADsxLLYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAADNIUlEQVR4nOzdd1hUZ/4+/ntoQ1FQmopix464QcEuKIho2FhiyzeriW5Mc1NM8kmyWVsaiZtC1phNMyZmN8YWjSbSRJSosQvYxYoNpQ59gJnn90d+MyuieAZm5ky5X9d1LmFg5twHyDvvec5znqMQQggQERERUaMc5A5AREREZA3YNBERERFJwKaJiIiISAI2TUREREQSsGkiIiIikoBNExEREZEEbJqIiIiIJGDTRERERCQBmyYiIiIiCdg0EREREUkga9OUkZGBuLg4BAQEQKFQYPPmzXLGISI7x5pERI2RtWmqqKhASEgIVqxYIWcMIiIArElE1DgnOXceGxuL2NhYOSMQEemxJhFRY2RtmgylVquhVqv1nwshUFNTA19fXygUChmTEZE9Yk0isi9WNRE8Pj4eXl5e+q1Vq1bw9/dHWVlZo8/TarVQKBRm2x588EEz/USI7q+wsBDOzs5m+/ufNm2a3IdsNk2tSYsWLTJrTWrbti0+//xz1NbWmuknQ3R3VVVVGDRokFn//k+ePGm0/FY10vT6669jwYIF+s9LS0sRGBgo6bkvv/yyqWLplZeX4/PPP8eOHTtQXV0NV1dXk++T6H62b9+Ouro6dOjQATNmzDD5/vr372/yfViKptakYcOGmaUmabVabN68GRcuXMDTTz+Njz76CO+++y6mTJnCkTAyOyEE/vrXv+LQoUPw8fHBY489Zpa/Qx8fH+O9mLAQAMSmTZsMeo5KpRIAhEqlMk0oA2m1WhEQECAAiNTUVLnjEAkhhHj88ccFAPHSSy/JHcWq2EJNEkIItVotli9fLvz8/AQAAUCEhYWJ9PR0uaORnXn//fcFAOHk5GS1f3+ST885ODjA0dFR/y81pFAoMHbsWABAcnKyzGmI/nhnp/tbjImJkTmN8bAeSefi4oL58+fj/PnzWLx4MTw8PHDgwAFERkZi/PjxyM7Oljsi2YFff/0Vr732GgDgk08+QUREhLyBmkhy03Tx4kVcuHBB/68xlJeXIzMzE5mZmfp9ZGZmIjc31yivL4dx48YBAJKSkmROQgQcP34c169fh5ubG0aMGCF3HKMxRT0CbLMm6bRs2RJLlizB+fPn8eyzz8LJyQmJiYkYMGAAZs+ejcuXL8sdkWzUqVOn8Mgjj0AIgXnz5uHpp5+WO1LTyTnMlZ6erh8uvn2bPXu2pOdb4lB4QUGBUCgUAoC4evWq3HHIzv3zn/8UAERsbKzcUayCLdake8nJyRHTp0/XH6OLi4tYsGCBKCgokDsa2ZCioiLRvXt3AUCMGDFCqNVquSM1i0IIIQxpsg4cOIDff/8deXl5AIC2bdtiyJAhCAsLM0YPZ5DS0lJ4eXlBpVLB09PT7Pu/l/DwcBw4cADffPMNHn/8cbnjkB2Ljo7G9u3bkZCQgOeff17uOEZnSfUIsNya1JhDhw7h1VdfxY4dOwAAnp6emDdvHqKiojBs2DC0aNFC5oRkrerq6jB+/HikpqaiY8eOOHjwIPz9/eWO1TxSu6ubN2+K4cOHC4VCITp16iTCwsJEWFiY6NSpk1AoFGL48OHi5s2bpmvv7sJS39UtXLhQABDTp0+XOwrZsYqKCqFUKgUAcerUKbnjGJUl1iMhLLcm3Y9WqxXJycliwIAB9UbYnJycxJAhQ8Trr78uUlJSRHl5udxRyYq8+OKLAoBwd3cXR48elTuOUUgeaXr44Ydx/fp1rFq1Cj179qz3tTNnzmDOnDkICAjA+vXrjdrUNcZS39Xt2bMHw4cPh7e3N27dusWJqiSLxMREjB8/Hh07dsSlS5ds6hJzS6xHgOXWJKm0Wi02bdqEX375Benp6Q3mOTk7OyMsLAwRERGIjIzEkCFD4O7uLlNasmTfffcdHnvsMQDA+vXr8fDDD8sbyEgkN00tW7ZERkYG/vSnP93164cPH0ZERMR9F3UzJkstUHV1dfD19YVKpcK+ffsQHh4udySyQ88//zz+9a9/Yd68efjiiy/kjmNUlliPAMutSU116dIl7Ny5E+np6UhPT8eVK1fqfd3Z2Rnh4eGIjIzEgAEDEBQUhG7duhm1kaqursb58+eRk5ODyspKTJgwAV5eXkZ7fTK+ffv2YdSoUaipqcGiRYuwdOlSuSMZjeTFLZVKJUpLS+/59bKyMiiVSqOEsnZOTk6IiorCxo0bkZyczKaJZGGLSw3osB6ZR+fOnfHYY4/hsccegxACFy9erNdEXbt2Dbt378bu3bvrPa9Dhw4ICgpC9+7dERQUpN+6det210V/1Wo1Lly4gJycnHrbuXPncOXKFdz+3r5FixZ47LHHMH/+/AajjCS/a9euYdKkSaipqcGkSZOwePFiuSMZl9TzeM8884zo1KmT+Omnn+qdr1epVOKnn34SnTt3FvPnzzfy2cPGWfL8gS+//FIAEEOHDpU7CtmhS5cuCQDC0dFRlJSUyB3H6CyxHun2b6k1ydi0Wq04d+6c+Oqrr8Rf/vIXMWjQINGqVau7Xn2o2xQKhQgMDBSjR48Ws2fPFlFRUfp5aI09z9PTU4SGhoqePXvWe3zcuHEiMTFRaDQauX8cJISorKwUAwcOFABEv379RFlZmdyRjE7y6Tm1Wo0XXngB33zzDerq6uDi4gIAqKmpgZOTE+bOnYuPP/7YrO/uLHkoPDc3F506dYKjoyMKCgrQqlUruSORHfnyyy/x5JNPYtiwYQ1GAWyBJdYjwLJrkjkIIVBUVHTXEaOcnByoVKp7Prdly5Z3HZ3q3r07/Pz8oFAoIIRAWloaPvnkE/z666/6EaiePXvib3/7G2bPns2r/fDHSOuePXuwc+dO3LhxA61bt4aPjw+8vb3h7e2t/1j3r6enZ7PnPAoh8Oijj+KHH36Aj48PDh48iC5duhjpiCyHwUsOlJaW4tChQ7h58yaAPy7xDQ0NlaVAWHqB6t27N06fPo0NGzZgypQpcschOzJlyhT89NNPePPNN7Fw4UK545iMJdUjXR5LrklyEkKgoKBA30hdvXoVAQEB+ubI39/foP9xnzt3Dp9++im++eYb/dw1T09PzJ07F/Pnz0fXrl1NdSiNqqmpQVFRkX4rLCy868dlZWXo1q0b+vfvj/79+6Nfv35NbvjKy8v1TVJ6ejoOHToEjUYj+fmOjo76hsrb2xvt27ev17hK+f0sW7YMr776KhwdHZGamorIyMgmHYulM7hpsiSWXqBefPFFJCQk4K9//Su++uorueOQnaitrYWvry9KS0tx4MABDBo0SO5IdsPSa5ItKisrw7fffovly5cjJycHwB+3tIqLi8Pzzz+PyMhIaLVaqFQqfdNyt0bm9sfUarVBGYQQUKlUKCoqQnl5eZOP5fYmSrd17doVDg71b95RWVlZr0k6ePAg6urq6n1Ply5dEBERgZ49e6KkpKTBcer+raqqkpStZcuWdx0FDAoKwsGDBxEXFwchBFasWIFnnnmmyT8DSye5aVq9enW9z2fNmmWSQIaw9AKVlJSE2NhYBAYG4vLlyzZ1yTdZrt27d2PEiBHw8fHBzZs3bXLJC0usR4Dl1yRbptVqkZycjE8++aTevT9btGiBiooKmHN8QKFQoHXr1vc8Hebt7Q03NzecPXsW2dnZyM7Oxo0bN+76Wh4eHujXrx/69+8Pb29v7N69GwcOHEBtbW297+vUqRMiIyMRERGBiIgIdOrUSVLWqqqqeg1kYWEhcnNz651ezc3NlfTzmzdvHj7//HOb/n+d5Kbp9qE2hUKhXz1WTpZeoCorK+Ht7Q21Wo2TJ0+id+/eckciO7Bw4UK8/fbbmDFjBtasWSN3HJOwxHoEWH5NshenT5/G8uXL8d1336GiokL/eIsWLe7ZxNzZ0BjK09NT/zpeXl4Gv1nJz8/HsWPH9E1UdnY2jh8/fs9Rr8DAQH2TFBkZic6dOxucWarq6mpcvHixwVy1nJwc/TIUo0aNQkpKin5+oa3i6TkTGzt2LFJTU/Hxxx/jhRdekDsO2YGwsDAcPHgQq1at0i8uR+ZhDTXJnpSVleHKlSv6Zsja/odeV1eHc+fO6ZuoW7duYfDgwYiIiECXLl0sYkSnqqoKV69eRadOnazu59sUbJpM7MMPP8TLL7+McePGITExUe44ZOMKCgrg7+8PIQSuXbuGgIAAuSPZFWuoSUTUdA73/xZqDt3Cgjt37pQ84Y6oqVJTUyGEQHBwMBsmIiIjY9NkYn379kX79u1RXV2N3377Te44ZON0E2DHjRsncxIiItvDpsnEFAqFfrTp9is6iIxNCIGUlBQAtnnrFCIiubFpMgM2TWQOx44dw40bN+Du7o7hw4fLHYeIyOYY3DStWrUK69evb/D4+vXr8d133xkllK2JioqCg4MDTpw40eAu4UTGkpSUBACIiIiwm5vVsh4RkTkZ3DTFx8fD19e3weP+/v549913jRLK1nh7e+tXZdadPiEyNt1Ipj2dmmM9IiJzMrhpys3NvetN+Dp16oTc3FyjhLJFuom5PEVHplBRUaG/Ma89TQJnPSIiczK4afL390d2dnaDx7OysuDj42OUULZI9+5/+/btBt1IkUiKnTt3oqamBp07d0ZQUJDcccyG9YiIzMngpmnmzJl47rnnkJ6eDo1GA41Ggx07duD555/HjBkzTJHRJgwaNAitWrVCcXExDh48KHccsjG3n5qzhFWCzYX1iIjMycnQJ7z11lu4dOkSxowZAyenP56u1Woxa9YsziFohJOTE6KiorBhwwYkJSVh8ODBckciG6KbBG5P85kA1iMiMq8m30YlJycHmZmZcHNzQ3BwsOQ7KhuTtd2y4Ouvv8YTTzyBwYMH4/fff5c7DtmIixcvomvXrnB0dERhYSG8vLzkjmR2llCPAOurSURkGINHmnSCgoLsau6EMehGAQ4cOIDi4mK0bt1a5kRkC3Sn5oYOHWqXDRPAekRE5mHwnKYpU6bg/fffb/D4smXLMHXqVKOEslWBgYHo06cPtFottm/fLnccshH2uNSADusREZmTwU1TRkYGxo8f3+Dx2NhYZGRkGCWULePq4GRMtbW1SEtLA2CfTRPrERGZk8FNU3l5OVxcXBo87uzsjNLSUqOEsmW6/7ElJSWhidPJiPR+//13lJWVwdfXFw888IDcccyO9YiIzMngpik4OBhr165t8PiPP/6IPn36GCWULRs5ciRcXV1x7do1nDx5Uu44ZOV0I5bR0dFwcLC/W0myHhGRORk8EXzhwoWYPHkyzp8/j9GjRwMA0tLSsGbNmrveA4rqc3Nzw6hRo5CcnIzk5GT07dtX7khkxXRNkz2tAn471iMiMieD35rGxcVh8+bNOHfuHJ555hm89NJLuHr1KrZv346JEyeaIKLt4bwmMoZbt27h8OHDAICxY8fKnEYerEdEZE5NXqfpbo4fP45+/foZ6+Xuy1rXRDl58iT69u0LpVKJoqIiuLu7yx2JrNB///tfPProowgJCUFmZqbccSyOuesRYL01iYikafYkiLKyMnz55ZcICwtDSEiIMTLZvN69e6NDhw5Qq9W8woeazJ6XGrgX1iMiMqUmN00ZGRmYNWsW2rVrhw8++ACjR4/Gvn37jJnNZikUCp6io2bRarVISUkBwKYJYD0iIvMwaCJ4Xl4evv32W6xcuRKlpaWYNm0a1Go1Nm/ezCtVDDRu3DisXLmSTRM1SXZ2Nm7evAkPDw8MGzZM7jiyYD0iInOTPNIUFxeHnj17Ijs7GwkJCbh+/TqWL19uymw2bcyYMXBwcMCpU6eQm5srdxyyMrpmOzIyEkqlUuY05sd6RERykNw0JSYmYu7cuVi6dCkmTJgAR0dHo4VYsWIFOnfuDFdXV4SHh+PAgQNGe21L1bp1a4SHhwPgKToyXFJSEgD7PTXHekREcpDcNO3evRtlZWUIDQ1FeHg4Pv30UxQUFDQ7wNq1a7FgwQIsXrwYR44cQUhICGJiYnDr1q1mv7al47wmaory8nLs2bMHgP02TaxHRCQLYaDy8nKxcuVKMWzYMOHs7CwcHBxEQkKCKC0tNfSlhBBChIWFiWeffVb/uUajEQEBASI+Pv6+z1WpVAKAUKlUTdq33H7//XcBQHh5eYna2lq545CV2LJliwAgunTpIrRardxxZGVJ9UgI669JRNS4Zq3TdObMGaxcuRLff/89SkpKEB0djS1btkh+fk1NDdzd3bFhw4Z6C9HNnj0bJSUl+Pnnnxt9vrWviaLRaODn54fi4mI8+eST8PHxkTsSWYGMjAzs3r0bTz31FP7973/LHcdiyF2PAOuvSUTUOINvo3K7nj17YtmyZYiPj8fWrVvxzTffGPT8goICaDQatGnTpt7jbdq0wenTpxt8v1qthlqt1n9u7TfkdHR0RExMDH788Ud88cUXcschKxMbGyt3BIti7noE2F5NIqLGGeUOn46Ojpg4caJB7+qaIj4+Hl5eXvotMDBQ8nPVajWWLFlSr8BZgjVr1kAIYfKturoaixcvRnV1tVn2Z2mbLR7/n//8Z8l/Z5b6928K5qpHgG3WJHPh8fP4rfH4jXobFUMZOhx+57s6IQRqamrg6+sLhULR6L7sfdicx8/jt+fjl6Ipp+dYk5qOx8/jt8bjN8pIU1O5uLggNDQUaWlp+se0Wi3S0tIwZMiQBt+vVCrh6emp37y8vODn53ff4kREdD+G1iOANYnI3jRrTpMxLFiwALNnz8bAgQMRFhaGhIQEVFRU4PHHH5c7GhHZGdYjImqM7E3T9OnTkZ+fj0WLFiEvLw8DBgxAUlJSg8mYRESmxnpERI2RvWkCgPnz52P+/Pkm3YdSqcTixYvt8pYTAI+fx2/fx28Ic9QjgL8THj+P3xqPX9aJ4ERERETWQtaJ4ERERETWgk0TERERkQRsmoiIiIgkYNNEREREJAGbJiIiIiIJ2DQRERERScCmiYiIiEgCNk1EREREErBpIiIiIpKATRMRERGRBGyaiIiIiCRg00REREQkAZsmIiIiIgnYNBERERFJwKaJiIiISAI2TUREREQSsGkiIiIikkDWpikjIwNxcXEICAiAQqHA5s2b5YxDRHaONYmIGiNr01RRUYGQkBCsWLFCzhhERABYk4iocU5y7jw2NhaxsbFyRiAi0mNNIqLGcE4TERERkQSyjjQZSq1WQ61W6z8XQqCmpga+vr5QKBQyJiMie8SaRGRfrKppio+Px9KlSxs8rlKp4Onpec/nabVaLFiwwJTRAADl5eVYuXIl3N3dUVRUBKVSafJ9Et3P+vXrMW3aNLRv3x4PP/ywyfc3YMAAPPbYYybfjyVoak3atm0bUlJSTBkNwB9NXE5ODtLT01FdXa1/3MXFBSNHjkRsbCzGjx+Pnj17sskjkxJC4LPPPsMLL7yAuro69O3bF1FRUWbZ92uvvYa2bdsa5bUUQggh5Rtzc3Prfd6xY0ejBNAHUSiwadMmTJw48Z7fc+e7utLSUgQGBt63QJmLEALt27fHjRs3sH37dowZM0buSESYO3cuvvnmGyxYsAAffvih3HGMwtT1CLCNmqRTVVWFnTt3Ytu2bdi2bRsuXLhQ7+tdunTRN1CRkZFwd3eXKSnZoqqqKjz11FNYvXo1AGDGjBn4+uuv4eHhIXMyw0keaercuTMUCgWEEFAoFNBoNKbMdVdKpdKiR28UCgXGjh2L7777DsnJyWyaSHZCCCQnJwMAYmJiZE5jPJZQjwDLr0k6bm5u+knu//rXv5CTk4Nt27YhMTERO3fuxMWLF/HZZ5/hs88+g1KpRGhoKIKCgvRb9+7dERQUhJYtW8p9KGRlLl26hMmTJ+Po0aNwdHTEsmXL8OKLL1rtyKbkpkmr1Rp95+Xl5Th37pz+84sXLyIzMxPe3t4meedoDjExMfqmadmyZXLHITt38uRJXLt2Da6urhgxYoTccYzGFPUIsM2adCeFQoEePXqgR48eeOGFF1BRUYH09HT9KNTly5exd+9e7N27t8Fz27Rp06CR0n3cokULGY6GLNn27dsxY8YMFBYWws/PD2vXrkVkZKTcsZpF8uk5U9i5c+ddf4CzZ8/Gt99+e9/nl5aWwsvLy6KGwgsKCuDv7w8hBK5du4aAgAC5I5Ed++ijj/DSSy8hJiYGSUlJcsexeLZYkwwhhMCZM2eQnZ2NnJycelt+fn6jz/Xx8UGHDh3Qvn37ev/e/rGnp2ejIwxCCJSXl6OwsBBFRUX6f3UfV1ZWonXr1vD29oa3tzd8fHzq/ducUT8hBNRqNaqqqpo0cuni4gI3Nzc4Ozs3OYOtEELgn//8J15//XVotVoMHDgQP/30EwIDA+WO1mxGmwheXFyMrVu3YtasWZKfExERARl7NpPw9fXFwIEDcfDgQaSkpNjNhFiyTLpGady4cTInMa+m1CPANmuSIRQKBXr16oVevXo1+JpKpcK5c+caNFM5OTkoLCzUb1lZWfd8fQ8PD30T1aZNG1RUVDRokOrq6pqc38PDo14T5e3tDTc3N1RVVaGyshJVVVX67c7Pq6qqjPK7d3R0hLu7O9zc3Optdz4m5XukPM/R0bHZmY2prKwMc+bMwYYNGwAAc+bMwYoVK+Dq6ipzMuMw2khTVlYWHnjgAbPOLbDUd3ULFy7E22+/jRkzZmDNmjVyxyE7VVlZCW9vb6jVapw8eRK9e/eWO5LZyFGPAMutSaZWUlKCK1eu4Nq1a7h69ar+39s/Li4ulvx6SqWyXuOj+9jNzQ0qlarBCFRxcbHJTtlaOmdn50YbK29v73qnUoOCguDj42OSOUVnz57FpEmTcPLkSTg7O2P58uWYN2+e1c5fuhvJI02lpaWNfr2srKzZYWxFTEwM3n77baSmpkKj0VjcOwGyDxkZGVCr1QgMDLzryIE1Yz2yLK1atUKrVq0QHBx8z++prKys11TdvHkTLVu2bDAy5OPjAzc3N4P+R6vVaqFSqeo1Urp/1Wp1oyM5d3vcycmwkzC69bkaG8Uy5PG7fd/tj91+xWZtbS1UKhVUKpXkvK1atWrQSOnmpvn4+Bh07DpbtmzBX/7yF5SWliIgIAAbN27E4MGDm/RalkzyX0arVq3uey7alrrJ5ggPD4enpycKCwtx5MgRDBo0SO5IZIduv2rO1v7bZD2yPu7u7vr/ORubg4MDWrdujdatW6Nbt25Gf30pdA2XOWi1WsmN161bt+qdSr169SpKSkpw6NAhHDp0qMFrt27dGm3btjXoNOHly5fxySefAABGjBiBdevWGW1dJEsjuWlq2bIl3njjDYSHh9/16zk5OXjyySeNFsyaOTs7Y8yYMdi0aROSk5PZNJEsbHGpAR3WI7JnDg4O8PDwaNI6R5WVlbhw4UKDeWnnzp3DtWvXUFxcbNCp1Ns999xz+OCDD2x6MrzkpumBBx4AAIwaNequX2/VqpVdT6C807hx47Bp0yYkJSXhH//4h9xxyM7k5ubi1KlTcHR0NNuqu+bEekTUNO7u7ujXrx/69evX4GsVFRU4f/48ioqKDDqFWFtbi6lTp2Lq1KkyHJF5SW6aHnnkEVRVVd3z623btsXixYuNEsoW6N7d79u3DyqVCl5eXjInInuiG2UKDw9Hq1at5A1jAqxHRMbn4eGB/v37yx3Dosm6TlNzWfqVKr169cKZM2ewceNGTJ48We44ZEcefvhhbNy4EUuXLsWiRYvkjmM3LL0mEVHzOMgdwJbpRpt07/qJzKGurg7bt28HYJvzmYiI5CL59JzuRns6hi4aZ49iYmLwr3/9C8nJybyah8zmwIEDUKlU8Pb2xsCBA+WOYxKsR0QkB8lN06pVq/QfKxQKFikJRo0aBRcXF1y+fBlnzpyxubVyyDLpVgGPioqy2TXCWI+ISA6Sm6b09HRT5rBJHh4eGDlyJLZv347k5GQ2TWQWutPBtnzrFNYjIpID5zSZGOc1kTkVFhbi4MGDAICxY8fKnIaIyLawaTIxXdO0c+dOVFdXy5yGbN327dshhEC/fv3Qvn17ueMQEdkUNk0m1q9fPwQEBKCqqgq7d++WOw7ZOFteBZyISG5smkxMoVDoT5PoJugSmYIQgk0TEZEJsWkyA92EXM5rIlM6fvw4rl+/Djc3N4wYMULuOERENsfgpmnVqlVYv359g8fXr1+P7777ziihbE1UVBQUCgWOHz+Oa9euyR2HbJSuKY+IiICrq6vMacyD9YiIzMngpik+Ph6+vr4NHvf398e7775rlFC2xsfHB4MGDQIApKSkyJyGbJU9nppjPSIiczK4acrNzUWXLl0aPN6pUyfk5uYaJZQt4tIDZEoVFRXIyMgAYF9NE+sREZmTwU2Tv78/srOzGzyelZUFHx8fo4SyRbr/kaWkpECj0cichmzNrl27UFNTg44dO6Jnz55yxzEb1iMiMieDm6aZM2fiueeeQ3p6OjQaDTQaDXbs2IHnn38eM2bMMEVGmxAeHg4vLy8UFxfj0KFDcschG3P7qTl7usch6xERmZPk26jovPXWW7h06RLGjBkDJ6c/nq7VajFr1izOIWiEk5MToqKisHHjRiQnJyM8PFzuSGRD7OHWKXfDekRE5qQQQoimPDEnJweZmZlwc3NDcHAwOnXqZOxs91VaWgovLy+oVCp4enqaff+G+uqrrzBv3jwMHToUe/bskTsO2YhLly6hS5cucHR0RGFhIby8vOSOZHaWUI8A66tJRGQYg0eadIKCghAUFGTMLDZPN69p3759KC4uRuvWrWVORLZAN8o0ePBgu2yYANYjIjIPg+c0TZkyBe+//36Dx5ctW4apU6caJZSt6tixI3r16gWtVou0tDS545CNsMelBnRYj4jInAxumjIyMjB+/PgGj8fGxuoveaZ749IDZEy1tbX6BtwemybWIyIyJ4ObpvLycri4uDR43NnZGaWlpUYJZctuv6VKE6eTEent378fpaWl8PHxQWhoqNxxzI71iIjMyeCmKTg4GGvXrm3w+I8//og+ffoYJZQtGzlyJJRKJa5cuYLTp0/LHYesnG7EMjo6Go6OjjKnMT/WIyIyJ4Mngi9cuBCTJ0/G+fPnMXr0aABAWloa1qxZc9d7QFF97u7uGDlyJFJTU5GUlITevXvLHYmsWFJSEgD7PDUHsB4RkXkZPNIUFxeHzZs349y5c3jmmWfw0ksv4erVq9i+fTsmTpxogoi2h/OayBgKCgpw+PBhAMDYsWNlTiMP1iMiMqcmr9N0N8ePH0e/fv2M9XL3Za1rohw/fhzBwcFwdXVFUVER3Nzc5I5EVmjNmjV45JFHEBwcfNdbidg7c9cjwHprEhFJY/BI053Kysrw5ZdfIiwsDCEhIcbIZPP69u2L9u3bo7q6Gr/99pvccchK2esq4I1hPSIiU2py05SRkYFZs2ahXbt2+OCDDzB69Gjs27fPmNlslkKh4Ck6ahYhhF2vz3Qn1iMiMgeDmqa8vDy89957CAoKwtSpU+Hl5QW1Wo3Nmzfjvffew6BBg0yV0+bo/kenm8hLZIjs7Gzk5eXB3d0dw4cPlzuOLFiPiMjcJDdNcXFx6NmzJ7Kzs5GQkIDr169j+fLlpsxm06KiouDg4ICTJ0/iypUrcschK6MbZYqIiIBSqZQ5jfmxHhGRHCQ3TYmJiZg7dy6WLl2KCRMmGHVNmBUrVqBz585wdXVFeHg4Dhw4YLTXtlTe3t76d8IpKSkypyFrY++n5liPiEgOkpum3bt3o6ysDKGhoQgPD8enn36KgoKCZgdYu3YtFixYgMWLF+PIkSMICQlBTEwMbt261ezXtnSc10RNUVFRgd27dwOw36aJ9YiIZCEMVF5eLlauXCmGDRsmnJ2dhYODg0hISBClpaWGvpQQQoiwsDDx7LPP6j/XaDQiICBAxMfH3/e5KpVKABAqlapJ+5bb3r17BQDRunVrUVtbK3ccshK//PKLACA6d+4stFqt3HFkZUn1SAjrr0lE1LhmrdN05swZrFy5Et9//z1KSkoQHR2NLVu2SH5+TU0N3N3dsWHDhnoL0c2ePRslJSX4+eefG32+ta+JUldXBz8/P5SUlGD+/Pnw9fWVOxJZgfT0dOzatQtPPvkkPv/8c7njWAy56xFg/TWJiBpn8G1UbtezZ08sW7YM8fHx2Lp1K7755huDnl9QUACNRoM2bdrUe7xNmzZ3vS+bWq2GWq3Wf27tN+R0cnLC2LFjsW7dOnz66adyxyErw/WZ6jN3PQJsryYRUeOavbglADg6OmLixIkGvatrivj4eHh5eem3wMBAyc9Vq9VYsmRJvQJnCdauXQshhMm36upqLF68GNXV1WbZn6Vttnj8htwmxFL//k3BXPUIsM2aZC48fh6/NR6/UW+jYihDh8PvfFcnhEBNTQ18fX2hUCga3Ze9D5vz+Hn89nz8UjTl9BxrUtPx+Hn81nj8RhlpaioXFxeEhoYiLS1N/5hWq0VaWhqGDBnS4PuVSiU8PT31m5eXF/z8/O5bnIiI7sfQegSwJhHZm2bNaTKGBQsWYPbs2Rg4cCDCwsKQkJCAiooKPP7443JHIyI7w3pERI2RvWmaPn068vPzsWjRIuTl5WHAgAFISkpqMBmTiMjUWI+IqDGyN00AMH/+fMyfP9+k+1AqlVi8eLFd3nIC4PHz+O37+A1hjnoE8HfC4+fxW+PxyzoRnIiIiMhayDoRnIiIiMhasGkiIiIikoBNExEREZEEbJqIiIiIJGDTRERERCQBmyYiIiIiCdg0EREREUnApomIiIhIAjZNRERERBKwaSIiIiKSgE0TERERkQRsmoiIiIgkYNNEREREJAGbJiIiIiIJ2DQRERERScCmiYiIiEgCNk1EREREEsjaNGVkZCAuLg4BAQFQKBTYvHmznHGIyM6xJhFRY2RtmioqKhASEoIVK1bIGYOICABrEhE1zknOncfGxiI2NlbOCEREeqxJRNQYzmkiIiIikkDWkSZDqdVqqNVq/edCCNTU1MDX1xcKhULGZERkj1iTiOyLVTVN8fHxWLp0aYPHVSoVPD09ZUjUfCdOnMB7772HkpISVFVVoaqqCpWVlfqPb39Mq9U2e3+XLl1Cp06djJCciGytJgkh8P777+PAgQP3rEO6j29vFptCoVBg0aJFWLJkiXHCE5mBQggh5A4B/PEf0KZNmzBx4sR7fs+d7+pKS0sRGBhotQUKAB566CFs2bLFoOcoFAq4ubnBzc0N7u7u+o9v3+58/Ndff8W5c+fw73//G0899ZSJjobIdthjTTp69CgeeOABg5/n7OwMR0dHg55TXV0NAFi1ahUee+wxg/dJJAfJI00ODg5QKBQQQkChUECj0Zgy110plUoolUqz79dU1Go1tm/fDuCPd6wdO3ZstBnSPebi4mLw0L+fnx/+8Y9/ICkpiU0TWT1LqEeA7dWkbdu2AQDCw8Px9NNPN1qHbt8MbZgAYNGiRXjrrbcwb948dO3aFSNHjjT24RAZneSm6eLFi0bfeXl5Oc6dO1dvH5mZmfD29kbHjh2Nvj9Lk5GRgcrKSrRr1w6vvvqqSedAxMbG4h//+AfS0tJQU1MDFxcXk+2LyNRMUY8A1iRd0/T4449j9uzZJt3XkiVLcObMGaxbtw6TJk3C/v370b17d5Puk6jZhIzS09MFgAbb7NmzJT1fpVIJAEKlUpk2qIm88MILAoCYM2eOyfel0WhEmzZtBACRlpZm8v0RWSN7rkkFBQXCwcFBABCXL182yz4rKyvFoEGDBADRq1cvUVxcbJb9EjWVwUsO3GsyslarRW5urkGvFRERASFEg+3bb781NJZV0r2rGz9+vMn35eDggJiYGABAUlKSyfdHZA7GrEeAfdeklJQUaLVa9OvXz2yjam5ubvj555/RoUMHnD59GtOmTUNtba1Z9k3UFJKbptLSUkybNg0eHh5o06YNFi1aVG8eQX5+Prp06WKSkLbo3LlzOHv2LJycnBAdHW2WfeoW7UtMTDTL/ohMhfXI+H799VcAwIQJE8y633bt2mHr1q3w8PBAamoqnnvuOQjLuD6JqAHJTdPChQuRlZWF77//Hu+88w5Wr16Nhx56CDU1Nfrv4R+6dLpRphEjRpjtKpvo6Gg4ODjg+PHjuHLliln2SWQKrEfGpdFo9CPQ5hj5vtOAAQPwww8/QKFQ4PPPP8fy5cvNnoFICslN0+bNm/HFF1/g4Ycfxl//+lccOnQI+fn5iIuL019yy8XcpDPnqTkdHx8fhIWFAQCSk5PNtl8iY2M9Mq6DBw+isLAQXl5eGDJkiCwZ/vznP2PZsmUAgBdffJEj4mSRJDdN+fn59RZF9PX1xfbt21FWVobx48ejsrLSJAFtUUVFBXbu3AnA/O/qeIqObAHrkXHp3sSNHTsWzs7OsuV46aWXMHfuXGi1WkyfPh3Hjx+XLQvR3Uhumjp27IhTp07Ve6xly5ZISUlBVVUVJk2aZPRwtio9PR1qtRqdOnVC7969zbrvcePGAQC2b9/OCZdktViPjEuOke+7USgU+OyzzxAREYGysjI8+OCDuHXrlqyZiG4nuWkaO3YsVq1a1eDxFi1aIDk5Ga6urkYNZstuL1DmPoUwcOBA+Pr6orS0FL///rtZ901kLKxHxpOXl4fDhw8D+N+bKjm5uLhg48aN6N69Oy5fvoyJEyfqVw8nkpvkpmnp0qX3vEdQy5YtkZqaih07dhgrl80SQsh2lQpQf+kBnqIja8V6ZDy6OjBw4EC0bdtW5jR/8Pb2xi+//IJWrVrh999/x9y5czmxnyyC5KapdevW6Nu37z2/3rJlS4waNcoooWzZyZMnkZubC6VSicjISFky6N5Nsmkia8V6ZDyWcmruTj179sSGDRvg5OSEH374Ae+8847ckYik30Zl9erV9T6fNWuW0cPYA12BioyMhLu7uywZYmJioFAokJWVhevXryMgIECWHERNxXpkHLW1tUhJSQFgeU0TAIwZMwYrVqzAk08+iYULF6JHjx6YNm2a3LHIjklumm6fP6BQKFikmsgS3tX5+flh4MCBOHjwIJKTk/H444/LloWoKViPjGPv3r0oLS2Fr68vBg4cKHecu5o3bx5Onz6Njz/+GI8++ig+/vhjBAUF1du6d+8OLy8vuaOSHZDcNKWnp5syh11QqVTYvXs3gP9d+i+XcePG4eDBg0hMTGTTRFaH9cg4dG/ixo0bB0dHR5nT3Ns///lPXLp0CZs2bcK+ffuwb9++Bt/j5+fXoJEKCgpCjx490KJFCxlSky2S3DRR823fvh11dXXo0aOH7Hfzjo2NxVtvvYXU1FTU1dXByYl/CkT2xhJGvqVwdHTExo0bkZ2djTNnzuDcuXPIycnRb7du3UJ+fj7y8/Oxd+/ees91cHBA3759ER4ejvDwcAwePBi9e/e26CaRLBf/T2lGcl41d6ewsDC0bt0axcXF2L9/P4YNGyZ3JCIyo9zcXBw/frzeFbWWTKFQICQkBCEhIQ2+VlpaipycnAbNVE5ODgoKCnDs2DEcO3YMX3/9NYA/lqYYNGiQvokKDw+3mCsH7UFVVRVOnTqFnj17wsPDQ+44BmHTZCZarVZ/tZolvKtzdHTE2LFjsXbtWiQmJrJpIrIzulGmIUOGwNvbW+Y0zePp6YnQ0FCEhoY2+Nr169exf/9+7N+/H/v27cOhQ4dQXl6O9PT0eqd5O3bsqG+iRowYgdDQUDg4SL7AnO7j/Pnz2LZtGxITE5Geno7q6moolUqMGjUK48ePR2xsLIKCgiz+9kcKYcWLX5SWlsLLywsqlcpsN71tqiNHjiA0NBQeHh4oLCyEUqmUOxK+++47PPbYYwgNDcWhQ4fkjkNk9aypJv35z3/G1q1b8c477+Dvf/+73HHMpq6uDidPnqzXSJ08ebLBOlB+fn4YN24cxo8fj7Fjx1p9Y2lu1dXVyMjIQGJiIrZt24azZ8/W+3qLFi1QXl5e77Fu3brpG6iIiAi4ubmZM7IkbJrM5O2338bChQvx0EMPYfPmzXLHAfDHSsDt2rXTf9ymTRuZExFZN2upSdXV1fDx8UFlZSWOHj2KAQMGyB1JVqWlpTh06BD279+P33//HTt37kRZWZn+6w4ODhg8eLD+f+gDBgyw2FGogoICZGRkYOfOndi5cyeKi4vRt29f9O/fX7/16tULLi4uRt/3pUuXkJiYiMTERKSlpdW7B6STkxOGDx+O2NhYjB8/Hn379sXp06f1TVVGRka9W3u5urpi9OjR+u/v2rWrwXm0Wi2qqqrg5uZmtN+XwU3TqlWr0KJFC0ydOrXe4+vXr0dlZSVmz55tlGBSWEuBAoChQ4fi999/xxdffIF58+bJHUfvgQcewNGjR/Hdd9/xsm2yOpZUjwDrqUkpKSmIiYlBu3btcO3aNYs/JWJuNTU12Lt3L7Zt24Zt27bhxIkT9b7etm1b/ShUdHQ0WrVqJel1tVotqqurUVVVhdraWvj4+DT7BslFRUXIyMhAeno6du7ciezs7Ps+x8nJCb17967XSIWEhKBt27Z3/VsQQqC8vByFhYUoKipCUVGR/uPCwkLk5eUhPT0dJ0+erPe8du3aYfz48Rg/fjyioqIa/W+irKwMO3bs0J/Cu3LlSr2v9+jRAyNGjIAQAlVVVfW2ysrKu35eU1MDADh79iyCgoKk/Djvy+CmqUePHvjiiy8arGa9a9cuzJs3D2fOnDFKMCmspUAVFBTA398fQgjk5uYiMDBQ7kh6b7zxBt59913MnDkTP/zwg9xxiAxiSfUIsJ6a9MILL+CTTz7B3Llz9ZOj6d5yc3P1IyJpaWmoqKjQf83R0RFDhgyBn59fo/8jr6qqanAPPYVCgbZt26J9+/bo0KEDOnTooP/49n9vXwi5uLhYP5KUnp6O7OzsBqcW+/bti4iICERGRqJdu3Y4fvw4srOz9ZtKpbrrcfr4+KB///5o2bJlvQapqKhI0g3eHR0dMXToUP3oUP/+/ZvUkAshcOLECX0DtXv3btTV1Rn8OjpZWVno379/k59/O4ObJldXV5w+fRqdO3eu9/ilS5fQu3dvVFVVGSWYFNZSoP773//i0UcfRf/+/ZGVlSV3nHp+++03jBw5Et7e3rh16xYvwyWrYkn1CLCemhQUFIRz587hp59+wqRJk+SOY1XUajV+++03fRN1+vRpk++zdevW6NChAxQKBY4dO9agSerduzciIyMRERGBUaNGwd/f/56vJYTAlStX9A1UVlYWsrOzcfbsWWi12kZzKJVK+Pj4wNvbu8G/oaGhiI6ORuvWrY1yzLdTqVRIS0tDVlYWXF1d4ebm1mBzd3dv9HFjnZ4z+Oo5f39/ZGdnNyhSWVlZ8PHxMUooW2PJa6EMGTIEXl5eKCoqwsGDBzF48GC5IxFJxnpkON2l+c7OzhgzZozccayOUqlEVFQUoqKi8OGHH+LixYvYsWMHampq7vs/8dsfc3BwQH5+Pq5du4arV6/q/73z44qKChQXF6O4uFifoWfPnvomKSIiwqD5qAqFAh07dkTHjh3x4IMP6h+vqqrCyZMncezYMdTW1sLb27teU+Tt7S3brb+8vLwwefJkTJ48WZb9387gpmnmzJl47rnn0LJlS4wcORLAH0Phzz//PGbMmGH0gNZOo9EgKSkJgGU2TU5OToiOjsaGDRuQlJTEpomsCuuR4XRv4kaMGGHRo2HWokuXLpg7d26TntumTRu0adMGDzzwwF2/LoRAaWmpvomqrKxEeHi4/gIeY3Jzc7vnsg30PwY3TW+99RYuXbqEMWPG6FeR1mq1mDVrFt59912jB7R2Bw4cQFFREby8vDBkyBC549zVuHHjsGHDBiQmJmLJkiVyxyGSjPXIcJY88k31KRQKeHl5wcvLC3369JE7DqEZSw7k5OQgMzMTbm5uCA4ORqdOnYyd7b6sYf7AwoUL8fbbb2PatGlYu3at3HHu6tq1a/rz5Tdv3oSfn5/ckYgMYgn1CLD8mlRRUQFvb2/U1NTg5MmT6N27t9yRiKxKk1cE190UkRpnDe/q2rdvj+DgYBw7dgypqal45JFH5I5EZBDWI2l0c286d+6MXr16yR2HyOoYPJ18ypQpeP/99xs8vmzZsgZrpdi7Gzdu4MiRIwD+uEGuJdPl093qhcgasB4Z5vb7X3JtJiLDGdw0ZWRk3HXUJDY2FhkZGUYJZSt0DcigQYMavQTUEuiapuTk5PtedkpkKViPpBNCWMXIN5ElM7hpKi8vv+vy687OzigtLTVKKFthTQVq6NChaNGiBfLz8/WjY0SWjvVIuhMnTuDKlStwdXVFRESE3HGIrJLBTVNwcPBdJzT/+OOPnN1/m9raWqSkpACwjqbJxcUFUVFRAHiKjqwH65F0ujdxkZGRsq23Q2TtDJ4IvnDhQkyePBnnz5/H6NGjAQBpaWlYs2YN1q9fb/SA1mrPnj0oKyuDn58fBg4cKHccSWJjY7F582YkJiZi4cKFcschui/WI+msaeSbyFIZ3DTFxcVh8+bNePfdd7Fhwwa4ubmhf//+2L59O0aNGmWKjFZJV6DGjRtnsXfDvtO4ceMAAPv370dRURG8vb1lTkTUONYjaVQqFXbv3g2ATRNRczRpyYEJEyZgwoQJDR4/fvw4+vXr1+xQtuD2q1SsRceOHdGnTx+cPHkSqampmD59utyRiO6L9ej+UlJSoNFo0KtXL3Tt2lXuOERWq9lDIGVlZfjyyy8RFhaGkJAQY2SyepcuXcLJkyfh4OCAsWPHyh3HIFx6gKwZ69Hd8dQckXE0uWnKyMjArFmz0K5dO3zwwQcYPXo09u3bZ8xsVkvXcAwdOtQkd3w2Jd0puqSkJC49QFaD9ejetFqtviaxaSJqHoNOz+Xl5eHbb7/FypUrUVpaimnTpkGtVmPz5s28UuU21vyubsSIEXB3d8fNmzeRlZWFP/3pT3JHIror1iNpjh49ips3b6JFixYYPny43HGIrJrkkaa4uDj07NkT2dnZSEhIwPXr17F8+XJTZrNK1dXVSEtLA2CdTZNSqdRfhZSUlCRzGqK7Yz2STvcmLioqCkqlUuY0RNZNctOUmJiIuXPnYunSpZgwYQIcHR2NFmLFihXo3LkzXF1dER4ejgMHDhjttc1t165dqKqqQvv27dG/f3+54zQJ5zWRpWM9ks6aR76JLI3kpmn37t0oKytDaGgowsPD8emnn6KgoKDZAdauXYsFCxZg8eLFOHLkCEJCQhATE4Nbt241+7XloLtqbvz48VZ7byfdvKa9e/eipKRE3jBEd8F6JE1+fj72798PgE0TkVEIA5WXl4uVK1eKYcOGCWdnZ+Hg4CASEhJEaWmpoS8lhBAiLCxMPPvss/rPNRqNCAgIEPHx8fd9rkqlEgCESqVq0r6NTavViq5duwoAYtOmTXLHaZYePXoIAGLDhg1yRyG6J0uqR0JYXk36/vvvBQAREhIidxQim6AQQoimNlxnzpzBypUr8f3336OkpATR0dHYsmWL5OfX1NTA3d0dGzZswMSJE/WPz549GyUlJfj5558bfX5paSm8vLygUqng6el5z+8TQiA+Pl5yrqaqqKjAu+++C2dnZxQWFqJly5Ym36epvPDCC/jkk08wfPhw/ek6Iil69eqFyZMnm32/ctcjQHpNysjI0C82aUpbtmzB/v378frrr+Pdd981+f6IbJ4xOq+6ujqxadMmERcXZ9Dzrl27JgCIvXv31nv8lVdeEWFhYQ2+v7q6WqhUKv125coVSe/qNBqNAGC2bcyYMQb9HCxRcnKyWX9m3Gxnmzp1qqx/u+aqR0I0vSYtXLjQrL+T3377zaCfBRHdnVHu7+Ho6IiJEyca9K6uKeLj4+Hl5aXfAgMDJT3PwcEB1dXVWLx4MaqrqyGEMOm2fft2k/4cmkKtVmPJkiVQq9WSvn/s2LEm/zmZczPn798SN3Me/7p160z819w4c9UjoOk16c033zTr78QSlxowtCbZGh6/dR5/s07PNZehw+FqtbreD1gIgZqaGvj6+t530rXUYXNbxePn8dvz8UvRlNNzrElNx+Pn8Vvj8ct6J1kXFxeEhobq1zUC/li9Ni0tDUOGDGnw/UqlEp6envrNy8sLfn5+VnuVGhFZDkPrEcCaRGRvmnTDXmNasGABZs+ejYEDByIsLAwJCQmoqKjA448/Lnc0IrIzrEdE1BjZm6bp06cjPz8fixYtQl5eHgYMGICkpCS0adNG7mhEZGdYj4ioMbI3TQAwf/58zJ8/36T7UCqVWLx4sd3eRoDHz+O35+M3hDnqEcDfCY+fx2+Nxy/rRHAiIiIiayHrRHAiIiIia8GmiYiIiEgCNk1EREREErBpIiIiIpKATRMRERGRBGyaiIiIiCRg00REREQkAZsmIiIiIgnYNBERERFJwKaJiIiISAI2TUREREQSsGkiIiIikoBNExEREZEEbJqIiIiIJGDTRERERCQBmyYiIiIiCdg0EREREUkga9OUkZGBuLg4BAQEQKFQYPPmzXLGISI7x5pERI2RtWmqqKhASEgIVqxYIWcMIiIArElE1DgnOXceGxuL2NhYOSMQEemxJhFRYziniYiIiEgCWUeaDKVWq6FWq/WfCyFQU1MDX19fKBQKGZMRkT1iTSKyL1Y10hQfHw8vLy/91qpVK/j7+6OsrEzuaE32yCOPQKFQmGVzcHBAbm6u3IdMZDNsrSaVl5ejY8eOZqtJU6ZMkfuQiQyiEEIIuUMAgEKhwKZNmzBx4sR7fs+d7+pKS0sRGBgIlUoFT09PM6Q0rrq6Ovj7+6O4uBhPPPEEfHx8TLavDRs24Ny5c/jiiy8wb948k+2HyFbYY03avHkzJk2aBAcHB3h6esLd3R1ubm71tjsfc3d3h6urKxwdHSXvp6amBh999BEUCgVycnLQrVs3Ex4VkfFIPj3n4OAAhUIBIQQUCgU0Go0pc92VUqmEUqk0+35N5cCBAyguLkbr1q3x2WefwcnJdGdLPTw8sHDhQiQmJrJpIqtnCfUIsL2atG3bNgDA/Pnz8cknn5h0X6dOnUJiYiISEhKwfPlyk+6LyFgkn567ePEiLly4oP/XGMrLy5GZmYnMzEz9PjIzM+3mFFJSUhIAIDo62qQNEwCMGzcOAJCWloaamhqT7ovI1ExRjwD7rklCCCQmJgKAWa4gfOmllwAA33zzDYqKiky+PyKjEDJKT08XABpss2fPlvR8lUolAAiVSmXaoCYycOBAAUCsWrXK5PvSaDTC399fABDp6ekm3x+RNbLnmpSdnS0ACDc3N1FZWWny/Wm1WtG/f38BQMTHx5t8f0TGYNBEcCEELl68iLq6OgB/nJdeu3YtVq9ejYKCAoMbtoiICAghGmzffvutwa9lbW7duoVDhw4BAGJiYky+PwcHB/1+dO8miayZsesRYN81SVcXIiMj4ebmZvL9KRQK/WjT8uXLOQJOVkFy03TmzBl06dIF3bt3R+/evXHx4kUMHToUc+fOxdNPP43evXsjJyfHlFltSkpKCgBgwIABaNeunVn2qTtFpzstSGStWI+Mz5yn5nRmzJiBgIAAXL9+HT/++KPZ9kvUVJKbpldffRUhISHIzMzEgw8+iAkTJqBDhw4oLi5GUVERhgwZgjfffNOUWW2KHAVq7NixUCgUyM7OxrVr18y2XyJjYz0yrtLSUuzevRsAMH78eLPt18XFBX/7298AAB9++CGEZVzMTXRvUs/j+fn5iaNHjwohhCgvLxcKhUL89ttv+q/v2bNHdOzY0RinDCWz1vkDdXV1wsfHRwAQGRkZZt13eHi4ACC+/vprs+6XyJgssR4JYb01aePGjQKA6NGjh9n3XVRUJDw8PAQAkZqaavb9ExlC8khTeXk5vL29Afxx+bqHh0e900qBgYG4efOmMfs5m3X48GEUFhbC09MTgwcPNuu+eYqObAHrkXHplhow5yiTTuvWrTFnzhwAf4w2EVkyyU1TQEBAvctuly1bBn9/f/3n+fn5aN26tXHT2Sjdqbno6Gg4Ozubdd+604Gpqan6CbRE1ob1yHiEEPo3UXLdrPj555+HQqFAUlISTpw4IUsGIikkN01RUVE4ffq0/vOnn34aLVu21H+ekpKCBx54wLjpbJQc85l0Bg4cCB8fH6hUKvz+++9m3z+RMbAeGc+xY8dw7do1uLu7Y+TIkbJk6NatGyZNmgQA+Pjjj2XJQCSF0W6jcvHiRbi6uprtSjDgj8mLXl5eVnXLgsLCQvj5+UEIgStXrqBDhw5mz/DII49gzZo1+Pvf/4533nnH7PsnMjU56hFgnTXpvffew+uvv44HH3wQW7dulS3H3r17MWzYMLi4uCA3Nxdt2rSRLQvRvRjthr1dunQxe4GyRikpKRBCIDg4WJaGCfjfCBfXayJbxXoknZwj37cbOnQoBg8ejJqaGqxYsULWLET3IvneHatXr673+axZs4wexh5YQoEaO3YsAODo0aPIy8tD27ZtZctC1BSsR8ZRUlKCPXv2AJC/aQL+uLXK1KlT8dlnn+G1116Du7u73JGI6pHcNK1atUr/sUKhYJFqAq1Wi+TkZAD/u4pNDm3atEFoaCgOHz6M5ORkzJ49W7YsRE3BemQc27dvh0ajQa9evdClSxe542DSpEno0qULLl68iNWrV+Opp56SOxJRPZKbpvT0dFPmsAtHjx7FrVu30KJFCwwbNkzWLLGxsTh8+DASExPZNJHVYT0yDksY+b6do6Mjnn/+ebzwwgv4+OOPMW/ePDg4GG0WCVGz8a/RjHQFKioqCi4uLrJm0Y10paSkcOkBIjskhNDXJDnWZ7qXOXPmwMvLC2fPnsWvv/4qdxyietg0mZFuLRQ5T83phIeHo1WrViguLsbBgwfljkNEZpaVlYUbN27Aw8MDI0aMkDuOXsuWLfHkk08C4GKXtkqr1eLs2bMoKyuTO4rB2DSZSXFxsX5dJEsYCndyctJPCOdVdET2R7cK+OjRo6FUKmVOU9/f/vY3ODk5YdeuXTh8+LDccciIzpw5g1GjRqFnz57w8vJCv3798Ne//hVfffUVsrOzodFo5I7YKDZNZpKamgqtVos+ffqgY8eOcscB8L8RLzZNRPbHEk/N6XTo0AEzZswAwNEmW1FbW4v4+HiEhIRg9+7dcHR0hBACJ06cwMqVKzFv3jyEhITAy8sLkZGReO2117Bp0yZcv35d7uj1GG1xSzlY00Jyc+bMwapVq7BgwQKLKQI3btxAQEAAAODmzZv1bkNBRIazlppUXFwMPz8/aDQaXLp0CZ06dZI7UgOZmZn405/+BEdHR1y4cMFi3myS4Y4cOYK5c+ciMzMTABATE4MvvvgCSqUS+/fvx/79+7Fv3z4cPHgQ5eXlDZ7foUMHDB48GP369UNdXR2qqqr0W2VlZb3P7/bYkSNH0K1bN6Mci8FN06pVq9CiRQtMnTq13uPr169HZWWlWa/EspYCJYRA+/btcePGDaSmpiIqKkruSHoDBgxAVlYWvv/+ezz66KNyxyEyiCXVI8B6atK6deswffp09OnTx6Lv9TZmzBjs2LEDL7/8Mv75z3/KHeeuqqqq8MUXXyA/Px9ubm5wc3ODu7u7/uN7PdaiRQv4+PhAoVDIfQgmU1VVhaVLl+KDDz6ARqOBt7c3EhIS8Oijj971uDUaDU6dOqVvovbv348TJ05Aq9U2K0d2djaCg4Ob9Ro6kpcc0ImPj8cXX3zR4HF/f3/MmzePl6/fhW7Cpbu7u0VNuAT+mF+VlZWFxMRENk1kdViPmsbSlhq4lwULFmDHjh348ssvsXDhQotrRIuKihAXF4e9e/c26fkeHh7o3r07unfvjqCgoHpbmzZtrLqhysjIwF//+lfk5OQAAKZNm4Z//etfjd4ex9HREf369UO/fv0wd+5cAEBZWRkOHz6Mffv24fz583B1dW3QkN6rUdU9bsxRSoObptzc3LsugtapU6d6dx2n/9FdNWeJEy5jY2Px3nvvITk5GRqNBo6OjnJHIpKM9chwWq1WX5MsvWmKjY1Fr169cPr0aaxcuRIvvvii3JH0cnNzMW7cOJw6dQqtWrXCo48+iurq6vueKrp9q6ioQFZWFrKyshq8fosWLRo0U927d0eHDh0QEBAg+7I191JaWopXX30Vn3/+OQAgICAAn332GR566KEmvV7Lli0RERGBiIgII6ZsOoObJn9/f2RnZ6Nz5871Hs/KyoKPj4+xctkUS35XN2TIEHh6eqKwsBCHDx9GWFiY3JGIJGM9MlxmZiby8vLQokULDB8+XO44jXJwcMCCBQswb948fPLJJ/qr6uR2/PhxjBs3DteuXUP79u2RlJSEfv36GfQaNTU1uHTpEnJycupt586dw+XLl1FeXo7MzEz9PKA7+fv7o0OHDujQoQPat2/f4OP27dujRYsWRjha6X755Rc8/fTTuHr1KgDgiSeewLJly9CqVSuz5jApYaD/+7//E506dRI7duwQdXV1oq6uTqSlpYlOnTqJl156ydCXaxaVSiUACJVKZdb9GqKkpEQ4OjoKAOL8+fNyx7mryZMnCwBiyZIlckchMogl1SMhrKMmvf322wKAeOihh+SOIklVVZXw8/MTAMSPP/4odxyxa9cu4eXlJQCIPn36iNzcXKPvo7q6Wpw6dUps2bJFfPjhh+Lpp58WUVFRokuXLsLFxUUAkLR5eXmJP/3pT+LJJ58U33zzjTh58qTQaDRGz3vr1i0xc+ZM/X67desmduzYYfT9WAKDmya1Wi2mTZsmFAqFcHZ2Fs7OzsLR0VE8/vjjQq1WmyLjPVlDgdq4caMAIHr06CF3lHv66quvBAAxePBguaMQGcSS6pEQ1lGThg4dKgCIL774Qu4oki1ZskQAEIMGDRJarVa2HBs3bhRKpVIAEMOHDxeFhYVmz6DVasWtW7fE0aNHxdatW8Xnn38u/vGPf4jHHntMREdHi969e4uWLVs22khFRUWJN954Q2zdulXcunVL8r6Li4vFgQMHxA8//CCWLl0qHn30UREeHi5atGghAAgHBwfx8ssvi4qKChP+BOTV5CUHcnJykJmZCTc3NwQHB8tyyao1XKnyxBNP4Ouvv8bzzz+PhIQEuePc1dWrVxEYGAiFQoH8/Hye1iCrYwn1CLD8mlRUVAQ/Pz9otVrk5uYiMDBQ7kiS5OfnIzAwEGq1Go899hhefPFF9O/f36wZ/v3vf+PZZ5+FEAIPPfQQ1qxZAzc3N7NmMERpaSmuXr2KkydP6i/rP3ToEKqqqhp8b5cuXRAeHo7w8HAMHjwYzs7O+lOFt586LCgouOf+goODsXLlSgwaNMiUhyU7rtNkQkIIBAYG4tq1a0hMTLSI26fcS3BwMI4fP44ffvgBM2fOlDsOkVWy9Jq0du1azJgxA3379sXx48fljmOQhQsX4u2339Z/HhERgeeeew5//vOfTXoBixACixYt0u973rx5WLFihUXMrTJUbW0tjh8/Xm9tpNOnTxv0Gm3btr3rBPXg4GD7uJDI0KGpyZMni/fee6/B4++//754+OGHmzfuZSBLHwo/duyYACBcXV1FZWWl3HEa9fLLLwsAYtasWXJHIZLMkuqREJZfk2bNmiUAiFdeeUXuKAbTarVi9+7dYtq0afp5ogBE586dxT//+U9RVFRk9H3W1taKOXPm6Pe1dOlSWU8PmkJxcbFISUkRb731lnjwwQeFv7+/8Pf3F8OGDROPPfaYePvtt8XatWvFkSNHRGlpqdxxZWdw0+Tr6yuys7MbPJ6dnS38/f2NEkoqSy9Qy5YtEwBEbGys3FHuKy0tTQAQ/v7+JpkoSGQKllSPhLDsmqTRaIS/v78AYPWTdHNzc8Xrr78ufHx89A2Nu7u7eOqpp8TJkyeNso+Kigrx4IMP6ufqfPnll0Z5XbJuBt97rry8/K7rQzg7O6O0tLQpg102S7fUgCWfltMZPnw4PDw8cOvWLRw9elTuOESSsB5Jd+TIEdy6dQstW7bEsGHD5I7TLIGBgXj33Xdx5coVfP311wgODkZlZSU+//xz9OnTB2PHjsUvv/zS5JWkCwsLMWbMGPzyyy9wdXXFpk2b8MQTTxj5KMgaGXxSNjg4GGvXrsWiRYvqPf7jjz+iT58+Rgtm7crKyrB7924Alrk+051cXFwwZswYbNmyBUlJSQgNDZU7EtF9sR5Jp3sTFxUVZbELIxrKzc0Nc+fOxZw5c7Br1y7861//ws8//4zU1FSkpqaiW7dumDFjBlq1anXPlaPv/FylUuGhhx7CmTNn0Lp1a2zdutXqm0wyHoObpoULF2Ly5Mk4f/48Ro8eDQBIS0vDmjVrsH79eqMHtFY7duxAbW0tunXrhqCgILnjSBIbG4stW7YgMTERb7zxhtxxiO6L9Ug6S15kt7kUCoV+1ehLly5hxYoV+Prrr3H+/Hm88847TXrNwMBAJCUlsfmmepp09dyvv/6Kd999V3+Jb//+/bF48WKMGjXKFBnvyZKvVHn66afx+eef49lnn8Wnn34qdxxJLl26hC5dusDBwQEFBQVo3bq13JGI7stS6hFguTWpsLAQfn5+EELgypUr6NChg9yRTK6iogL/+c9/cPjw4bvewuRetzcRQiAsLAwbN260i58TGcaoSw4cP37c4KXkm8NSC5QQAl26dMHly5fxyy+/YMKECXJHkqx37944ffo01q1b1+DO8UTWxNz1CLDcmrRmzRo88sgjCA4ORnZ2ttxxLJYQAjU1NRZ3j1CyHAZPBL9TWVkZvvzyS4SFhSEkJMQYmaze6dOncfnyZSiVSou5yaBUuqF73VA+kTVhPbq7bdu2AQDGjx8vcxLLplAo2DBRo5rcNGVkZGDWrFlo164dPvjgA4wePRr79u0zZjarpbuD+MiRI+Hh4SFzGsPomqakpCQYcRCSyKRYj+5Nq9UiOTkZgG3OZyIyJ4Mmgufl5eHbb7/FypUrUVpaimnTpkGtVmPz5s2cLHcba55wOWLECLi7u+PGjRvIzs7mu3WyWKxH0hw+fBj5+fnw9PTE0KFD5Y5DZNUkjzTFxcWhZ8+eyM7ORkJCAq5fv47ly5ebMptVqqiowK5duwBYZ9Pk6uqKyMhIADxFR5aL9Ug63am56OhoODs7y5yGyLpJbpoSExMxd+5cLF26FBMmTDDqPWZWrFiBzp07w9XVFeHh4Thw4IDRXtvcdu7ciZqaGnTq1Ak9e/aUO06T6Bbj1J1mJLI0rEfSWfPIN5Glkdw07d69G2VlZQgNDUV4eDg+/fTTRu94LNXatWuxYMECLF68GEeOHEFISAhiYmJw69atZr+2HG4vUAqFQuY0TaMrrnv27OGqymSRWI+kyc/P1zd91nBnAiKLZ+h9V8rLy8XKlSvFsGHDhLOzs3BwcBAJCQlNvpFfWFiYePbZZ/WfazQaERAQIOLj4+/7XEu7z5NWqxVdu3YVAMTPP/8sd5xmCQoKEgDExo0b5Y5CdE+WVI+EsLya9J///EcAECEhIXJHIbIJzVqn6cyZM1i5ciW+//57lJSUIDo6Glu2bJH8/JqaGri7u2PDhg2YOHGi/vHZs2ejpKQEP//8c6PPl7omihACH374oeRcTVVeXo6lS5fC2dkZhYWFaNmypcn3aSrPPfccli9fjlGjRuHBBx+UOw5ZkR49euDPf/6z2fcrdz0CpNekPXv24Pfff5ecrak2bdqEvXv34rXXXkN8fLzJ90dk84zRedXV1YlNmzaJuLg4g5537do1AUDs3bu33uOvvPKKCAsLa/D91dXVQqVS6bcrV65Ielen0Wj0d8I2xzZ69GiDfg6WKDEx0aw/M262s02dOlXWv11z1SMhml6TFi5caNbfSUZGhkE/CyK6u2YvbgkAjo6OmDhxokHv6poiPj4eXl5e+i0wMFDS8xwcHFBdXY3FixejuroaQgiTbmlpaSb9OTSFWq3GkiVLoFarJX3/uHHjTP5zMudmzt+/JW7mPP5169aZ+K+5ceaqR0DTa9Kbb75p1t/JiBEjTPyTMJyhNcnW8Pit8/iNehsVQxk6HK5Wq+v9gIX4Y8l7X1/f+066ttTbG5gLj5/Hb8/HL0VTTs+xJjUdj5/Hb43Hb5SRpqZycXFBaGhovZEZrVaLtLQ0DBkypMH3K5VKeHp66jcvLy/4+flZ7VVqRGQ5DK1HAGsSkb0xaEVwU1iwYAFmz56NgQMHIiwsDAkJCaioqMDjjz8udzQisjOsR0TUGNmbpunTpyM/Px+LFi1CXl4eBgwYgKSkJLRp00buaERkZ1iPiKgxsjdNADB//nzMnz/fpPtQKpVYvHix3d7BmsfP47fn4zeEOeoRwN8Jj5/Hb43HL+tEcCIiIiJrIetEcCIiIiJrwaaJiIiISAI2TUREREQSsGkiIiIikoBNExEREZEEbJqIiIiIJGDTRERERCQBmyYiIiIiCdg0EREREUnApomIiIhIAjZNRERERBKwaSIiIiKSgE0TERERkQRsmoiIiIgkYNNEREREJAGbJiIiIiIJ2DQRERERSSBr05SRkYG4uDgEBARAoVBg8+bNcsYhIjvHmkREjZG1aaqoqEBISAhWrFghZwwiIgCsSUTUOCc5dx4bG4vY2Fg5IxAR6bEmEVFjOKeJiIiISAJZR5oMpVaroVar9Z8LIVBTUwNfX18oFAoZkxGRPWJNIrIvVtU0xcfHY+nSpQ0eV6lU8PT0lCGR7Vq1ahXmzJlT7zF3d3e4ubnpt7t97urqCgcH6QOYubm52LlzJ4YPH47ffvvN2IdBZFKsSeZRUVEBb29v1NTUYPr06VAqlSbb14ULF7B79264ubkhJSUFw4cPN9m+yPoohBBC7hAAoFAosGnTJkycOPGe33Pnu7rS0lIEBgayQJnA9OnTsW7dOrz++utYtGgRlEqlSd45X7p0CV26dIGjoyMKCwvh5eVl9H0QNQVrkuXYtm0bJkyYgI4dO+LSpUsmHcWrra3FpEmT8Ouvv8LLywu7du1CSEiIyfZH1kXykICDgwMcHR31/8pBqVTC09Oz3kbGV1dXh9TUVADAn//8Z7i6upqsSHXu3Bk9evSARqPBjh07TLIPsj2WUI8A1iRzSU5OBgDExMSY/LSns7Mz1q1bh+HDh0OlUiEmJgbnzp0z6T7Jekg+PXfx4kWj77y8vLzeH+PFixeRmZkJb29vdOzY0ej7I2kOHjyI4uJitG7dGoMGDTL5/mJiYnD27FkkJydj0qRJJt8fWT9T1COANclS6ZqmcePGmWV/7u7u2Lp1KyIiIpCVlYXo6Gjs3r0b7du3N8v+yYIJGaWnpwsADbbZs2dLer5KpRIAhEqlMm1QO7No0SIBQEyfPt0s+/v1118FANG5c2eh1WrNsk+iu2FNsjyXLl0SAISjo6MoKSkx677z8vJE9+7dBQDRt29fUVhYaNb9k+Vp9pIDo0ePxuXLl5v03IiICAghGmzffvttc2NRMyQlJQEw37u6UaNGwcXFBZcuXUJOTo5Z9km2qTn1CGBNskS6UabBgwebfc5jmzZtkJqaioCAAJw4cQLjx49HeXm5WTOQZZF8em7Lli13fTwjIwO//PILAgMDAfwxB4asV0FBAQ4ePAgAGDt2rFn26eHhgREjRiAtLQ1JSUno0aOHWfZL1ov1yH7o3sTFxMTIsv/OnTsjNTUVI0aMwP79+zFp0iT88ssvJr2CjyyX5KvnHBwcoFAo0Ni3KxQKaDQao4W7n9LSUnh5efFKFSNas2YNHnnkEfTv3x9ZWVlm2+8///lP/N///R/Gjx+PX3/91Wz7JetkifUIYE0yttraWvj6+qK0tBT79+9HWFiYbFkOHDiA0aNHo6KiAg8//DB+/PFHWS9CIHlIPj0XExOD2NhY5OXlQavV6jdHR0ccP34cWq3W7AWKjM/cp+Z0dO8id+7cWe8SbqK7YT2yD/v370dpaSl8fHwQGhoqa5awsDBs3rwZLi4u2LBhA5566qlGm3ayTZKbpsTERIwZMwYDBw7EL7/8YspMJBOtVmv2q1R0goOD0a5dO1RWVmL37t1m3TdZH9Yj+6CrR9HR0RYxqhMVFYU1a9bAwcEBX3/9NV577TW5I5GZGTQR/MUXX8SWLVvw6quv4sknn0RlZaWpcpEMsrKycPPmTXh4eGDYsGFm3bdCodCPNukKJVFjWI9s3+3rM1mKyZMn48svvwQALFu2DMuWLZM5EZmTwVfPDRgwAIcOHYJCocCAAQM4PGlDdKfmxowZAxcXF7PvX1cYdTmI7of1yHYVFBTg0KFDAMx3UYpUc+fO1TdLr776Kr766iuZE5G5NOnec25ubvj888+xZcsWpKenw9fX19i5SAZyzWfSiY6OhkKhwLFjx3D9+nUEBATIkoOsC+uRbUpNTYUQAsHBwRZZC1555RUUFhbi/fffx5NPPolTp04hLCwM/fv3R48ePeDkZFW3diWJLObec03BK1WMR6VSwdfXF3V1dTh//jy6du0qS46wsDAcPHgQq1atwmOPPSZLBqKmYk0ynsceewzfffcdXnnlFYs9BSaEwJNPPtlgpEmpVKJPnz7o379/vc3f31+mpGQsklvh1atX1/t81qxZRg9D8tmxYwfq6urQo0cP2Rom4I9RroMHDyI5OZlNE90T65FtE0IgJSUFgGXNZ7qTQqHAv//9b4wYMQJ79uzBsWPHkJ2djfLychw9ehRHjx6t9/1t2rSp10RFRkbq1xQj6yB5pCkyMvJ/T1IoLOLmqnxXZzxPPfUUvvjiCzz33HP45JNPZMuxZ88eDB8+HD4+Prh586ZFXDFDlscS6xHAmmQs2dnZCAkJgbu7O4qKiqxqIUmtVovLly8jOzu73paTk3PXOXfBwcGIjY3F+PHjMXToUDg7O8uQmqTi6TmCEAJdunTB5cuXsW3bNsTGxsqWpa6uDr6+vlCpVLIvZkdkKNYk41i2bBleffVVm1rstqKiAidPntQ3UQcOHMCBAweg1Wr13+Pp6Yno6GiMHz8e48aNs8i5XPaOM9UIZ86cweXLl6FUKjFq1ChZszg5OWHMmDH46aefkJyczKaJyA5Z4lIDzeXh4YFBgwZh0KBB+scKCwuRkpKCxMREJCYmoqCgABs3bsTGjRsB/HF16Pjx4zF+/HiEh4dzcrkFaPYNe8n66a6aGzlyJNzd3WVOA67XRGTHKioq9AvcynUlr7n4+Phg5syZWL16NW7evIkDBw5gyZIlCAsLg0KhQGZmJt59910MHz4c/v7+ePTRR3Hp0iW5Y9s1Nk0k+1IDd9I1Tfv27YNKpZI5DRGZ086dO1FTU4POnTsjKChI7jhm4+DggEGDBmHx4sXYv38/bt68ie+//x4zZ86Et7c3iouL8d///hchISH48ccf5Y5rt9g02bmqqirs2rULgOU0TZ06dUKvXr2g0WiQlpYmdxwiMqPbT80pFAqZ08jHz88Pjz76KH744QfcunULu3fvxpAhQ1BaWoqZM2fi8ccfR1lZmdwx7Q5PkNq5Xbt2obq6GoGBgejdu7fccfRiYmJw+vRpJCUlYfLkyXLHISIz0Y1829J8puZydHTEsGHDkJGRgTfffBPvvPMOvv32W+zevRs//PBDvXlS1qCgoAAbNmxAVVWVQc/TaDSoqqqqt1VWVt73sf379xttKR2Dm6ZVq1ahRYsWmDp1ar3H169fj8rKSsyePdsowcg8bj81Z0nv6mJiYvDJJ58gOTkZQgiLykaWg/XItly8eBE5OTlwcnLC6NGj5Y5jcZycnPDmm28iKioKjz76KM6dO4ehQ4fi7bffxiuvvAIHB8s+eaTVarFq1Sr83//9H4qKisy234qKCqO9lsFNU3x8PL744osGj/v7+2PevHksUlbG0uYz6YwaNQpKpRK5ubk4c+YMevXqJXckskCsR7ZFd2puyJAh8PLykjmN5Ro5ciSysrIwb948bNiwAa+99hpSUlKwevVqtG/fXu54d5WdnY2nn34ae/fuBQD07dsXAwYMMOg1HBwc4Obm1mBzd3dv9PFu3boZ70CEgZRKpbh48WKDxy9evChcXV0NfblmUalUAoBQqVRm3a+tuHDhggAgHB0dRUlJidxxGoiKihIAREJCgtxRyEJZUj0SgjWpuSZOnCgAiLffflvuKFZBq9WKlStXCnd3dwFAeHt7i02bNskdq57S0lLx4osvCkdHRwFAtGjRQnz00UeitrZW7mhNYvBYnr+/P7Kzsxs8npWVBR8fn+Z1cGRWund1Q4cOtch3dbrRLy49QPfCemQ7amtr9Rd+cD6TNAqFAnPmzMGRI0fwwAMPoKioCJMmTcLTTz+NyspKWbMJIbB+/Xr06tULH3/8MTQaDaZOnYpTp07hxRdftNo1pwxummbOnInnnnsO6enp0Gg00Gg02LFjB55//nnMmDHDFBnJRCz11JyOrnDu3LkT1dXVMqchS8R6ZDv27duHsrIy+Pr64oEHHpA7jlXp2bMnfv/9d7zyyisAgM8//xwDBw5EVlaWLHnOnTuH2NhYTJs2DdevX0e3bt2QmJiIdevWoUOHDrJkMhpDh6bUarWYNm2aUCgUwtnZWTg7OwtHR0fx+OOPC7VabYLBsHvjUHjTqdVq0aJFCwFAHD58WO44d6XVakX79u0FAJGSkiJ3HLJAllSPhGBNao6///3vAoB45JFH5I5i1VJSUkTbtm0FAOHi4iKWLl0qdu/ebZa/yaqqKrFkyRKhVCr1+1+8eLGorKw0+b7Npcn3nsvJyUFmZibc3NwQHByMTp06Ga2Rk4r3eWq6nTt3IjIyEv7+/rhx44bFXnUxZ84crFq1Ci+99BI++OADueOQhbKEegSwJjXHwIEDcfjwYXz33XeYNWuW3HGsWn5+PubMmYNffvml3uOdO3dG//79623du3c3yo3RU1JS8Oyzz+LcuXMAgLFjx+LTTz+1uQVKecNeO/Xaa6/h/fffx6OPPorvv/9e7jj3tHbtWsyYMQP9+vXDsWPH5I5D1CjWpKbJz89HmzZtIITAjRs30LZtW7kjWT0hBL755hv89NNPyM7OxtWrV+/6fW5ubujbty9CQkL0jVTr1q0brH3U2JpIFy5cQGJiIgAgICAACQkJePjhh21yqRiDZ2JNmTIFYWFhePXVV+s9vmzZMhw8eBDr1683WjgyHd3kakudz6QTFRUFhUKB48eP49q1axZ7OS3Jg/XINqSmpkIIgZCQEDZMRqJQKDB37lzMnTsXAFBUVIRjx44hOztbvx07dgxVVVU4dOgQDh061Kz9OTo64rnnnsOSJUts+g2DwU1TRkYGlixZ0uDx2NhYfPjhh8bIRCZ248YNZGZmQqFQYOzYsXLHaZSPjw/CwsKwf/9+pKSk4PHHH5c7ElkQ1iPbcPutU8g0vL29MWrUKIwaNUr/mEajwYULF/RNVFZWFrKzs1FZWXnP9Y/u9piHhwcmTJiA4OBgGY/QPAxumsrLy+Hi4tLgcWdnZ5SWlholFJlWSkoKACA0NBR+fn4yp7m/mJgY7N+/H0lJSWyaqB7WI+un1WqtZuTb1jg6OiIoKAhBQUGYMmWK3HGsgsGzf4ODg7F27doGj//444/o06ePUUKRaVn6UgN30r37TE1NhUajkTkNWRLWI+uXnZ2NmzdvwsPDA8OGDZM7DlGjDB5pWrhwISZPnozz58/r7w2UlpaGNWvWcP6AFdBoNPqRJmtpmsLCwuDl5YXi4mIcOnQI4eHhckciC8F6ZP10o0yRkZF3HTUksiQGjzTFxcVh8+bNOHfuHJ555hm89NJLuHr1KrZv346JEyeaICIZ06FDh1BUVAQvLy+raT6cnJwQFRUFgKuDU32sR9aP85nImjRpcZ4JEyZgz549qKioQEFBAXbs2IFRo0bh+PHjxs5HRqY7NRcdHW1Vy9jzlip0L6xH1qu8vBy7d+8GwKaJrEOzVzQsKyvDl19+ibCwMISEhBgjE5mQtc1n0tEV1H379qG4uFjmNGSpWI+sS3p6Ompra9G1a1d0795d7jhE99XkpikjIwOzZs1Cu3bt8MEHH2D06NHYt2+fMbORkRUWFuLAgQMArO9dXWBgIHr37g2tVqu/qSeRDuuRdbr91JwtLoRItseg8zN5eXn49ttvsXLlSpSWlmLatGlQq9XYvHkzr1SxAtu3b4dWq0W/fv2s8qaJMTExOHXqFJKTk/Hwww/LHYdkxnpk/TifiayN5JGmuLg49OzZE9nZ2UhISMD169exfPlyU2YjI7PWU3M6usKanJwMK777DxkB65H1u3DhAs6dOwcnJydERkbKHYdIEslNU2JiIubOnYulS5diwoQJRrnBn86KFSvQuXNnuLq6Ijw8XH8KiYxHCKFvmqz1Xd2oUaPg6uqKK1eu4PTp03LHIRmxHlk/3SjT0KFDbfq2G2RbJDdNu3fvRllZGUJDQxEeHo5PP/0UBQUFzQ6wdu1aLFiwAIsXL8aRI0cQEhKCmJgY3Lp1q9mvTf+TnZ2NvLw8uLu7Y/jw4XLHaRI3NzeMHDkSwP9Gzcg+sR5ZP2sf+SY7JQxUXl4uVq5cKYYNGyacnZ2Fg4ODSEhIEKWlpYa+lBBCiLCwMPHss8/qP9doNCIgIEDEx8ff97kqlUoAECqVqkn7tifvvfeeACAmTJggd5Rm+fDDDwUAERMTI3cUsgCWVI+EYE2SSq1WixYtWggA4vDhw3LHIZJMIUTTJ4ecOXMGK1euxPfff4+SkhJER0djy5Ytkp9fU1MDd3d3bNiwod5CdLNnz0ZJSQl+/vnnRp9fWloKLy8vqFSqRod3hRB2P9/h66+/xrFjx7B8+XLMnz9f7jhNduLECfTr1w+urq547733eMWNBerevTvGjx9v9v3KXY8A6TVp3759dn3a79q1a1i2bBn8/PyQl5cHB4dmr35DZB7G6Lzq6urEpk2bRFxcnEHPu3btmgAg9u7dW+/xV155RYSFhTX4/urqaqFSqfTblStXJL2r02g0AgA3QOTk5Bj0O7I0Wq1WdOjQQfafI7d7b1OnTpX1b8Rc9UiIptekhQsXyv57soTt//2//2fQ74hIbkZp7x0dHTFx4kSD3tU1RXx8PLy8vPRbYGCgpOc5ODiguroaixcvRnV1NYQQdrfpjl/qz8xSKRQKXLlypcnHb++/f3Mc/7p162T9GzFXPQKaXpPefPNN/k1WV6N79+5Qq9Um/i1ZJrVajSVLlvD4rez4m3V6rrkMHQ5Xq9X1fsBCCNTU1MDX1/e+p2mkDpvbKh4/j9+ej1+KppyeY01qOh4/j98aj1/WE8kuLi4IDQ2tt8KzbsXnIUOGNPh+pVIJT09P/ebl5QU/Pz/OayGiZjO0HgGsSUT2RvY7ti5YsACzZ8/GwIEDERYWhoSEBFRUVODxxx+XOxoR2RnWIyJqjOxN0/Tp05Gfn49FixYhLy8PAwYMQFJSEtq0aSN3NCKyM6xHRNQY2ZsmAJg/f77JL4NXKpVYvHgxlEqlSfdjqXj8PH57Pn5DmKMeAfyd8Ph5/NZ4/LJOBCciIiKyFlxRjIiIiEgCNk1EREREErBpIiIiIpKATRMRERGRBGyaiIiIiCRg00REREQkAZsmIiIiIgnYNBERERFJwKaJiIiISAI2TUREREQSsGkiIiIikoBNExEREZEEbJqIiIiIJGDTRERERCQBmyYiIiIiCdg0EREREUnApomIiIhIAlmbpoyMDMTFxSEgIAAKhQKbN2+WMw4R2TnWJCJqjKxNU0VFBUJCQrBixQo5YxARAWBNIqLGOcm589jYWMTGxsoZgYhIjzWJiBrDOU1EREREEsg60mQotVoNtVqt/1wIgZqaGvj6+kKhUMiYjIjsEWsSkX2xqpGm+Ph4eHl56bdWrVrB398fZWVlckezObdu3UK7du2gUCjMss2ePVvuQyYyGGuS+bz55ptmq0cKhQJ/+9vf5D5kskAKIYSQOwQAKBQKbNq0CRMnTrzn99z5rq60tBSBgYFQqVTw9PQ0Q0r78cUXX+Cpp54CALi4uMDNzU2/ubu71/v8zu32r9/5vXd+7fjx4/h//+//oU2bNrh+/TocHKyqjycbxppkOYQQCAgIQF5eHhwcHCTXn/s9frf6lJWVhUceeQQA8MMPP2DmzJkyHz1ZEsmn5xwcHKBQKCCEgEKhgEajMWWuu1IqlVAqlWbfrz3atm0bAODtt9/GG2+8YbL99OzZE/PmzcPNmzeRnZ2NAQMGmGxfZDssoR4BrEnmkpmZiby8PHh4eKCwsNCkP/M+ffrgxIkTeOedd/DXv/4V/fv3R9++fU22P7IukpumixcvGn3n5eXlOHfuXL19ZGZmwtvbGx07djT6/kgatVqNtLQ0AMD48eNNui+lUonRo0dj69atSEpKYtNEkpiiHgGsSZYqMTERADBmzBizNKlLly7Fvn37kJaWhilTpuDgwYNo2bKlyfdLVkDIKD09XQBosM2ePVvS81UqlQAgVCqVaYPamdTUVAFAtG3bVmi1WpPvb8WKFQKAGDVqlMn3RdQY1iTLNGzYMAFA/Pvf/zbbPm/duiXat28vAIipU6eapRaS5WvW1XMXL17EuXPn0K5dO/Tr18/g50dEREBYxpQquo3uXV1sbKxZrgAaN24cAGDPnj0oLS3lXBBqkubWI4A1yRIVFxfj999/BwCzrqHl5+eH9evXY+TIkVi/fj2GDRuG559/3mz7J8skedbtM888g/LycgBAVVUVHn74YXTv3h0xMTEICQnB6NGj9V8n63Z702QOXbt2RVBQEOrq6rBjxw6z7JOsG+uR/UhNTYVWq0WfPn3QqVMns+57yJAh+OijjwAAL7/8Mvbs2WPW/ZPlkdw0ffHFF6isrAQAvPXWW9i/fz+2b9+O8vJyZGRkIDc3F++8847JgpJ5XLp0CadOnYKjoyOio6PNtl/daFNSUpLZ9knWi/XIfuguSjH1/Mp7mT9/PmbMmIG6ujpMmzYNN2/elCUHWQbJTdPtQ9Zbt27FsmXLEBkZCXd3dwwbNgwfffQRfvrpJ5OEJPPRjTINHToUrVq1Mtt+b2+aeHqE7of1yD5otVr9Gym5bm+jUCjw1VdfoXfv3rh+/TpmzpyJuro6WbKQ/AxaFEc3vyUvLw/9+/ev97WQkBBcuXLFeMlIFrp3deYuUKNGjYJSqcTly5dx5swZs+6brBPrke07evQobt68iRYtWmD48OGy5WjRogU2btwIDw8PpKenY9GiRbJlIXkZ1DQtXLgQCxYsgIODA65fv17va4WFhfDw8DBqODKv6upq/Zwicw+Fe3h4YOTIkQB4io6kYT2yfbqR76ioKLi4uMiapXfv3li5ciWAP1aC37Jli6x5SB6Sm6aRI0fizJkzOHr0KPr06YPLly/X+/q2bdu4AJiV++2331BZWYmAgIAG79zNISYmBgCbJro/1iP7YO6LUu5n+vTp+ivoZs2ahfPnz8uciMzNaLdRuXDhAlxcXNChQwdjvJwkpaWl8PLy4i0LjOTFF19EQkIC5syZo39HZU4nTpxAv3794OrqiqKiIri5uZk9A9kGOeoRwJpkTEVFRfDz84NWq0Vubi4CAwPljgQAqKmpQWRkJPbu3YuQkBD8/vvvrFV2xGg3+uratavZCxQZl+5dnVxXqfTp0wcdOnRAdXU1MjIyZMlAtoH1yPqlpKRAq9WiX79+FtMwAX/ci3PdunXw8/NDVlYW5s+fL3ckMiPJi1uuXr263uezZs0yehiSz4ULF3DmzBk4OTkhKipKlgwKhQLjxo3D119/jaSkJP3pOqI7sR7ZPrkuSpGiffv2WLNmDcaOHYtvvvkGQ4cOxdy5c+WORWYg+fRcZGTk/56kUFjEIoQcCjeeFStWYP78+Rg1ahR27twpW46NGzfi4YcfRq9evXDq1CnZcpBls8R6BLAmGYtWq0Xbtm2Rn5+P9PR0REREyB3pruLj4/H3v/8dSqUSW7duRVRUlFnuokDyMdqcJjmwQBnPhAkTsG3bNrz33nt49dVXZctRUlICX19faDQaXLx4EZ07d5YtC5GhWJOM4+DBgwgLC0PLli1RWFgIZ2dnuSPdlVarxcSJE7F161YAQGBgICZPnowpU6Zg6NChcHR0lDkhGZvR5jSR9aqurkZ6ejoA+YfCW7VqhSFDhgAAkpOTZc1CRPK4fakBS22YAMDBwQGrV6/GrFmz4OHhgStXruCTTz7ByJEj0b59ezz99NPYvn07amtr5Y5KRsKmibBr1y5UVVWhffv2CA4OljsOb6lCZOfkvijFEK1atcJ3332H/Px8bN68GX/5y1/g5eWFmzdv4vPPP0d0dDTatm2LOXPm4JdffoFarZY7MjUDmyaqN+HSEs7H65qmtLQ01NTUyJyGiMypoKAA+/fvB/C/WmAN3Nzc8NBDD2H16tW4desWkpKS8MQTT8DPzw9FRUVYtWoV4uLi4Ofnh0ceeQTr16/HyZMnUVpaKnd0MgDnNBF69OiBnJwc/PTTT5g0aZLccepNAt25cydGjRoldyQiSViTmu+HH37A//t//w/9+/dHVlaW3HGara6uDrt378bGjRvx008/NVi9HvjjNi0dOnRAhw4d0L59+7v+6+vrCwcHjnPITfKSA2Sbzp07h5ycHDg5OWHMmDFyxwHwxzyBmJgY/Oc//0FSUhKbJiI7YslLDTSFk5MTIiIiEBERgU8++QT79+/Hxo0bkZqaitzcXJSUlKC8vBynT5/G6dOn7/k6zs7O6Nq1K2bOnIk5c+ZY1NpV9sTgkaZVq1ahRYsWmDp1ar3H169fj8rKSsyePduoARvDd3XNt3z5cjz33HOIiIjQTwa3BP/973/x6KOPYsCAATh69KjccchCWVI9AliTmkur1aJNmzYoKCiwm1HmiooKXLt2DVevXtX/e+fHt27dwu3/q3ZwcMC4cePwxBNPYMKECRY9Wd7mCAMFBQWJHTt2NHh8586dokePHoa+XLOoVCoBQKhUKrPu15bExsYKAGLZsmVyR6nn5s2bAoAAIK5fvy53HLJQllSPhGBNaq79+/cLAMLT01PU1NTIHcdiqNVqcenSJfHf//5XRERE6GsjANGuXTvx97//XVy4cEHumHbB4BOkubm56NKlS4PHO3XqhNzc3Ob0b2RmVVVVFrPUwJ38/f0RGhoK4I/bKRDdDeuRbdGdmouOjuboyW1cXFzQqVMnPPLII0hPT8eZM2fwyiuvwM/PDzdu3MC7776Lrl27YuzYsVi/fj0voDEhg5smf39/ZGdnN3g8KysLPj4+RglF5rFz505UV1ejQ4cOFnlHeC49QPfDemRbdEsNWNqbOEvTo0cPLFu2DFevXsX69esxduxYAEBqaiqmTZuGDh064JVXXsHZs2dlTmocVVVVOH78OFJTU/Hbb7/h0KFDOHHiBC5cuIC8vDyoVCrU1NTUO4VpKgZPBJ85cyaee+45tGzZEiNHjgTwxzo/zz//PGbMmGH0gGQ6und148ePt4ilBu40btw4vPPOO0hNTYVGo+HqutQA65HtyM/Px8GDBwGwaZLKxcUFDz/8MB5++GFcvHgRK1euxDfffIMbN27ggw8+wAcffIAhQ4YgKioKkZGRGDx4MNzc3OSOfVfV1dW4cOECcnJyGmxXr16V9BoODg5wd3eHm5ubfnN3d8fatWsRFBRklJwGTwSvqanBX/7yF6xfvx5OTn/0XFqtFrNmzcLnn38OFxcXowSTgpMum04Ige7du+PChQvYtGkTJk6cKHekBurq6uDr6wuVSoX9+/cjLCxM7khkYSypHgGsSc3xn//8B3/5y18QEhKCzMxMueNYrbq6Omzbtg1fffUVtm3bBq1Wq/+aUqnE4MGDERERgcjISISHh8PV1dXkeUpKSlBYWIiioiL9v7du3cL58+f1jdGVK1caHSlq1aoV2rdvj9raWlRVVaGyshJVVVWoqqq67wjT6dOn0bNnT6McT5PXacrJyUFmZibc3NwQHByMTp06GSWQIVigmu7s2bPo2bMnnJ2dUVhYiJYtW8od6a4efvhhbNy4EUuXLsWiRYvkjkMWyhLqEcCa1ByPPPII1qxZg9dffx3vvvuu3HFswtWrV5GUlIT09HSkp6fjxo0b9b6uVCoxZMgQREZGIiIiAuHh4VAqlXd9LY1Gg5KSknqNz70+vv1flUolOa+npyeCgoL0W/fu3fUf+/j43PWMiBACNTU19ZqoO5uqkSNHwt3d3bAf3j1wcUs79cknn+CFF17A6NGjkZaWJnece/r666/xxBNPYMiQIdi7d6/ccYgaxZrUNBqNBv7+/igqKkJGRgZGjBghdySbI4RATk4Odu7cifT0dOzcuRN5eXn1vsfV1RVDhw5F27ZtGzRCJSUlzZoz5OXlBW9vb/j4+MDb2xu+vr7o0qVLvQbJz8/PIqeK3M7gOU1TpkxBWFgYXn311XqPL1u2DAcPHsT69euNFo5Mx1omXMbExAAA9u/fj6KiInh7e8uciCwJ65FtOHjwIIqKiuDl5aW/YTcZl0KhQI8ePdCjRw/MmzcPQgicOXOmXhN169Yt7Nixo9HXadmyZb3mx8fHB61bt4aPj0+9x27/t3Xr1vrT59bO4JEmPz8/7Nixo8GNXY8dO4aoqCjcvHnTqAEbw3d1TVNZWQlvb2+o1WqcOHECffr0kTtSo/r164cTJ05g7dq1mDZtmtxxyIJYUj0CWJOaatGiRXjrrbcwdepUrFu3Tu44dkkIgdOnT2PXrl36/0fc3vjoNntfCsLg1q+8vPyukyudnZ1540ErkZ6eDrVajY4dO6J3795yx7mvcePG4cSJE0hKSmLTRPWwHtkGaxn5tmUKhQK9e/e2iv8nyMngdZqCg4Oxdu3aBo//+OOPFj9iQX+w9KUG7nT7ek1WPAWPTID1yPrdvHkThw4dAvC//9aJLJXBI00LFy7E5MmTcf78eYwePRoAkJaWhjVr1nD+gBUQQljdu7rhw4fD3d0dN27cwLFjx9C/f3+5I5GFYD2yfsnJyQCAP/3pT2jXrp3MaYgaZ/BIU1xcHDZv3oxz587hmWeewUsvvYSrV69i+/btFrnWD9V39uxZXLx4ES4uLvr/yVg6V1dXREZGAuDq4FQf65H1s7Y3cWTfmjSdfcKECZgwYUKDx48fP45+/fo1OxSZju7U3MiRI9GiRQuZ00gXExODX3/9FUlJSfi///s/ueOQBWE9sl51dXX6kabx48fLnIbo/gweabpTWVkZvvzyS4SFhSEkJMQYmciErPVdnW6uw+7du1FWViZzGrJUrEfW5cCBAyguLkarVq0QHh4udxyi+2py05SRkYFZs2ahXbt2+OCDDzB69Gjs27fPmNnIyMrLy7Fr1y4A1veurnv37ujatStqa2uRnp4udxyyMKxH1kk38h0TE2Mz6/iQbTPorzQvLw/ffvstVq5cidLSUkybNg1qtRqbN2/mlSpWID09HTU1NejcubPR7sNjLgqFAuPGjcNnn32G5ORk/PnPf5Y7EsmM9cj6WevIN9kvySNNcXFx6NmzJ7Kzs5GQkIDr169j+fLlpsxGRqZ7VxcbG2sVSw3cSXeKLjExkUsP2DnWI+uXl5eHI0eOAOBSA2Q9JDdNiYmJmDt3LpYuXYoJEybA0dHRaCFWrFiBzp07w9XVFeHh4Thw4IDRXpv+cPtSA9Z2ak4nMjISzs7OuHjxIs6dOyd3HJIR65H1010JGxoaijZt2sichkgayU2TbgJuaGgowsPD8emnn6KgoKDZAdauXYsFCxZg8eLFOHLkCEJCQhATE4Nbt241+7Xpf06fPo3Lly/DxcVFf/m+tWnRooX+Rp5cesC+sR5ZP56aI6skDFReXi5Wrlwphg0bJpydnYWDg4NISEgQpaWlhr6UEEKIsLAw8eyzz+o/12g0IiAgQMTHx9/3uSqVSgAQKpWqSfu2Jx988IEAIKKjo+WO0izLli0TAMT48ePljkIWwJLqkRCsSVLV1taKVq1aCQBiz549cschkszgG/be7syZM1i5ciW+//57lJSUIDo6Glu2bJH8/JqaGri7u2PDhg31FqKbPXs2SkpK8PPPPzf6fKk3xxRC4PPPP5ecyxZ9+eWXyMzMxMcff4wXXnhB7jhNplsR3M3NDR988IFVzs2ydd26dcPYsWPNvl+56xEgvSYdOHAAhw8flpzN1ly7dg3vvPMOvL29cevWLaOeXiUyKWN0XnV1dWLTpk0iLi7OoOddu3ZNABB79+6t9/grr7wiwsLCGnx/dXW1UKlU+u3KlSuS3tVpNBoBgBsgTp8+bdDvyNJotVrRvn172X+O3O69TZ06Vda/EXPVIyGaXpMWLlwo++/JErYZM2YY9DsikluzF7cEAEdHR0ycONGgd3VNER8fDy8vL/0WGBgo6XkODg6orq7G4sWLUV1dDSGE3W264+/cubNJf0emplAocPXq1SYfv73//s1x/OvWrZP1b8Rc9Qhoek168803+TdZXY2ePXtCrVab+LdkmdRqNZYsWcLjt7Ljb9bpueYydDhcrVbX+wELIVBTUwNfX9/7nqaROmxuq3j8PH57Pn4pmnJ6jjWp6Xj8PH5rPH6jjDQ1lYuLC0JDQ5GWlqZ/TKvVIi0tDUOGDGnw/UqlEp6envrNy8sLfn5+nNdCRM1maD0CWJOI7I3s69YvWLAAs2fPxsCBAxEWFoaEhARUVFTg8ccflzsaEdkZ1iMiaozsTdP06dORn5+PRYsWIS8vDwMGDEBSUhIXOyMis2M9IqLGyN40AcD8+fMxf/58k+5DqVRi8eLFUCqVJt2PpeLx8/jt+fgNYY56BPB3wuPn8Vvj8cs6EZyIiIjIWsg6EZyIiIjIWrBpIiIiIpKATRMRERGRBGyaiIiIiCRg00REREQkAZsmIiIiIgnYNBERERFJwKaJiIiISAI2TUREREQSsGkiIiIikoBNExEREZEEbJqIiIiIJGDTRERERCQBmyYiIiIiCdg0EREREUnApomIiIhIAjZNRERERBLI2jRlZGQgLi4OAQEBUCgU2Lx5s5xxiMjOsSYRUWNkbZoqKioQEhKCFStWyBmDiAgAaxIRNc5Jzp3HxsYiNjZWzghERHqsSUTUGM5pIiIiIpJA1pEmQ6nVaqjVav3nQgjU1NTA19cXCoVCxmREZI9Yk4jsi1U1TfHx8Vi6dGmDx1UqFTw9PWVIROZ2/fp1tG/fHgqFAhMnTjTp/5i2bt2K2tpanDlzBj169DDZfsh6sSbRe++9h9dff92s+xw0aBD+/e9/IzQ01Kz7JQOaptzc3Hqfd+zY0ehh7uf111/HggUL9J+XlpYiMDDQ7DlIPsnJyQD+KBo//fSTSfc1evRopKenIzk5mU2ThbGEegSwJhGQlJQEAHjnnXcwffp0k+5r27Zt+Mc//oGDBw8iLCwMzzzzDN566y20atXKpPul/5HcNHXu3BkKhQJCCCgUCmg0GlPmuiulUgmlUmn2/ZLl0DVNMTExJt9XTEyMvmn629/+ZvL9kXSWUI8A1iR7V1paij179gAAZsyYga5du5p0f3/729/w8MMP4+WXX8YPP/yATz/9FOvXr8eHH36IRx55hKeEzUDyRHCtVguNRqP/1xjKy8uRmZmJzMxMAMDFixeRmZnZ4F0kEQBoNBqkpqYCMF/TBADp6en15q2Q/ExRjwDWJDLMjh07UFdXh6CgIJM3TDrt2rXDf//7X2zfvh09e/bEzZs38eijj2LMmDE4ffq0WTLYNSGj9PR0AaDBNnv2bEnPV6lUAoBQqVSmDUoWYf/+/QKA8PLyErW1tSbfn0ajEW3atBEARFpamsn3R/JjTSJDPPnkkwKA+Nvf/ibL/qurq8Xbb78tXF1dBQDh7Ows/v73v4uKigpZ8tgDySNNarUatbW1+s/Pnz+PN954A3/5y1/wj3/8AxcvXjS4YYuIiIAQosH27bffGvxaZPt0p+aioqLg5GT6axgcHBz0o026fZNlMEU9AliTSDohhH4+07hx42TJoFQq8cYbb+DkyZOYMGECamtr8e6776Jv37745ZdfZMlk6yQ3TTExMfj5558BAHv27NH/Umpra7Ft2zb069cPv//+u8mCEukKlDlOzeno9qXbN1kG1iOS25kzZ3D58mUolUqMGjVK1ixdunTB1q1bsWnTJgQGBuLSpUuIi4vDxIkTcfbsWWi1Wlnz2RKFEEJI+UYvLy8cOnQIQUFBiIiIwAMPPICPPvpI//WFCxciPT0du3fvNlnYO5WWlsLLy4uX99qBkpIS+Pr6QqPR4PLly2a7Wio/Px9t2rSBEALXr19Hu3btzLJfapwl1iOANcmeJCQk4MUXX0R0dDRSUlLkjqNXUVGBN998Ex999BHq6uoAAE5OTmjfvj3at2+PDh06oEOHDg0+bteuHVxcXGROb/kkn+PQaDT6CZenT5/GJ598Uu/rjz32GBISEowajkgnLS0NGo0GvXr1Muvl5X5+fnjggQdw+PBhpKSkYPbs2WbbN90b6xHJTe5Tc/fi4eGB999/H7NmzcLzzz+vn6x++fJlXL58+Z7PUygU8Pf3R4cOHdCmTRv4+PjA29sb3t7e+o/vfMzT09PurtiT3DSFh4dj69at6NWrF7p164asrCyEhITov56ZmQlvb2+ThCQy51IDdxo3bhwOHz6M5ORkNk0WgvWI5FRVVYVdu3YBsLymSadv377Yvn07amtrkZeXh2vXruHq1au4evVqg4+vXbuGmpoa3Lx5Ezdv3pS8D0dHR7Ru3Ro+Pj4IDw/HlClTMHbsWLi6uprwyOQluWl6++23ERsbi4qKCsycORMvvfQScnJy0Lt3b5w5cwb/+te/zL4qKtkHIYS+aZKjQMXExOCdd95BSkoKNBoNHB0dzZ6B6mM9Ijnt2rUL1dXVCAwMRO/eveWO0yhnZ2cEBgY2uuiqVqtFYWGhvpHKz89HYWEhioqKUFRUpP/49n+rqqqg0WhQUFCAgoICnDlzBqtXr0aLFi0wYcIETJkyBbGxsWjRooUZj9YMDLnUbu/evWLw4MFCoVDU29q3by8SEhKMdkmfVLy81z6cPHlSABBKpVKWS2lrampEy5YtBQBx4MABs++f7s7S6pEQrEn24vnnnxcAxBNPPCF3FNlUVlaKa9euiezsbJGUlCSee+450aFDh3pLdbi6uoqHHnpIrF69WhQXF8sd2SgkTwS/XX5+Pi5cuACtVot27dqhc+fORmzjpOOkS/tgCRMuJ02ahM2bN+Ott97CP/7xD1ky0N1ZSj0CWJPsRa9evXDmzBls3LgRkydPljuOxdBqtTh48CB++uknbNy4EefPn9d/zdnZGWPGjMGUKVPw0EMPwc/PT8akTdekpslSsEDZh9jYWCQlJeGDDz7ASy+9JEuGzz//HE8//TSGDx+O3377TZYMZPlYk2zfxYsX0bVrVzg6OqKwsBBeXl5yR7JIQghkZ2dj48aN2LhxI06ePKn/moODA4YNG4b27dvD3d0dbm5u+q2xzz08PPQT0Vu1agUHB8mrJhmN5KZp9erV9T6fNWuWSQIZggXK9lVXV8Pb2xtVVVU4duwY+vXrJ0sOFkrLYon1CGBNsge6N1AjRoxARkaG3HGsxunTp/UjUEeOHGn26ykUCv0k9Ptd5TdkyBCj/fcouWmKjIysF3bHjh1GCdAcLFC2LzU1FWPHjkX79u1x5coVWS9v7dmzJ86ePYuffvoJkyZNki0HWWY9AliT7IHuVP3bb7+NN954Q+44VunixYtIT09HaWkpqqqqUFlZiaqqqrtut3+tvLwcRUVFKC8vN2h/WVlZ6N+/v1GyS756Lj093Sg7JDLE7auAy70eSExMDM6ePYukpCQ2TTJjPSI51NTUIC0tDYDlLjVgDbp06YIuXbo0+fk1NTUoLi6+61V9t/+r+9jX19do2U1/Ay+iZpBzfaY7xcTEYPny5UhOToYQQvYmjojM6/fff0dZWRn8/Pzwpz/9Se44dsvFxQVt2rRBmzZtzL5v88+iIpLo6tWrOHHiBBwcHBAVFSV3HERERMDFxQWXL1/G2bNn5Y5DRGZ2+8i3HJOQSX78rZPF0i0vMGjQIItY3dnDwwMjRowA8L8RMCKyH5Z66xQyHzZNZLEs6dScji4LmyYi+3Ljxg1kZmZCoVBg7NixcschmbBpIouk0WiQmpoKwLLe1emapp07d6K6ulrmNERkLrqR79DQUKtdmJGaz+CmadWqVVi/fn2Dx9evX4/vvvvOKKGIDh48iOLiYrRq1QqDBg2SO45ecHAw2rVrh8rKSuzevVvuOHaP9YjMhafmCGhC0xQfH3/Xy/f8/f3x7rvvGiUUke70V1RUFJycLOciT4VCwVN0FoT1iMxBo9HoR5rYNNk3g5um3Nzcu66v0KlTJ+Tm5holFJElzmfSYdNkOViPyBwOHTqEoqIieHl5ITw8XO44JCODmyZ/f39kZ2c3eDwrKws+Pj5GCUX2rbi4GPv37wdgmU1TdHQ0FAoFjh07huvXr8sdx66xHpE56E7NRUdHW9TIN5mfwU3TzJkz8dxzzyE9PR0ajQYajQY7duzA888/jxkzZpgiI9mZtLQ0aLVa9OnTB4GBgXLHacDHxwcDBw4E8L/JoSQP1iMyB85nIh2DW+a33noLly5dwpgxY/Qdt1arxaxZsziHgIzi9gXkLFVMTAwOHjyIpKQkPPbYY3LHsVusR2RqhYWFOHDgAADLrklkHpJv2HunnJwcZGZmws3NDcHBwejUqZOxs90Xb45pe4QQ6NixI65evYqkpCSLLVJ79uzB8OHD4e3tjVu3bsHR0VHuSHbNEuoRwJpki9auXYsZM2agb9++OH78uNxxSGZNPjkbFBSEoKAgY2YhwqlTp3D16lW4urpi5MiRcse5p/DwcHh5eaGoqAiHDx9GWFiY3JHsGusRmYrugg+emiOgCXOapkyZgvfff7/B48uWLcPUqVONEorsl65AjRw5Em5ubjKnuTcnJyeMGTMGAK+ikxPrEZmSEILzmageg5umjIwMjB8/vsHjsbGxyMjIMEoosl/W9K6OSw/Ij/WITOnYsWO4ceMG3N3dMXz4cLnjkAUwuGkqLy+Hi4tLg8ednZ1RWlpqlFBkn6qqqrBr1y4A1jHhUpdx3759KCkpkTeMnWI9IlPSjTJFRkbC1dVV5jRkCQxumoKDg7F27doGj//444/o06ePUUKRfcrIyEB1dTU6dOiA3r17yx3nvjp16oRevXpBo9EgLS1N7jh2ifWITImn5uhOBk8EX7hwISZPnozz589j9OjRAP5YV2fNmjV3vQcUkVS3rwKuUChkTiNNTEwMTp8+jeTkZEyZMkXuOHaH9YhMpaysTH9/STZNpGPwSFNcXBw2b96Mc+fO4ZlnnsFLL72Eq1evYvv27Zg4caIJIpK9sORbp9zL7fOamrh6BzUD6xGZSnp6Ompra9GtWzd0795d7jhkIZq8TtPdHD9+HP369TPWy90X10SxHVeuXEHHjh3h4OCAgoICtG7dWu5IklRWVsLb2xtqtRqnTp1Cr1695I5E/z9z1yOANcmWPPPMM/j3v/+NZ599Fp9++qnccchCGDzSdKeysjJ8+eWXCAsLQ0hIiDEykR3SjTKFh4dbTcMEAO7u7vr1pHTzH0g+rEdkDEIIJCYmAuCpOaqvyU1TRkYGZs2ahXbt2uGDDz7A6NGjsW/fPmNmIztijafmdLj0gPxYj8iYcnJycOnSJbi4uCAiIkLuOGRBDJoInpeXh2+//RYrV65EaWkppk2bBrVajc2bN/NKFWqyuro6bN++HYD1Nk0vv/wydu3aherqal6abCasR2QqulHj4cOHo0WLFjKnIUsieaQpLi4OPXv2RHZ2NhISEnD9+nUsX77clNnIThw8eBAlJSVo3bo1Bg0aJHccg/Xt2xft27dHVVUVfvvtN7nj2AXWIzIlLjVA9yK5aUpMTMTcuXOxdOlSTJgwwag3KF2xYgU6d+4MV1dXhIeH6+8oTfZBd1orKirKKm98q1AoMHbsWAA8RWcurEdkKlVVVdi5cycANk3UkOSmaffu3SgrK0NoaCjCw8Px6aefoqCgoNkB1q5diwULFmDx4sU4cuQIQkJCEBMTg1u3bjX7tck6WNOtU+5Fl51Nk3mwHpGp/Pbbb6iqqkJAQIDZr74kKyAMVF5eLlauXCmGDRsmnJ2dhYODg0hISBClpaWGvpQQQoiwsDDx7LPP6j/XaDQiICBAxMfH3/e5KpVKABAqlapJ+yb5FRYWCgcHBwFAXLlyRe44TWYrx2FtLKkeCcGaZAsWLFggAIg5c+bIHYUsULPWaTpz5gxWrlyJ77//HiUlJYiOjsaWLVskP7+mpgbu7u7YsGFDvYXoZs+ejZKSEvz888+NPl/qmihCCHzzzTeSc5H5HD9+HAkJCejbty+OHz8ud5xmGTx4MPbv34+nnnoKAwcOlDuObLp06aJfnduc5K5HgPSadPjwYWRmZkrORubzzjvv4OLFi1i3bh2mTp0qdxyyNMbovOrq6sSmTZtEXFycQc+7du2aACD27t1b7/FXXnlFhIWFNfj+6upqoVKp9NuVK1ckvavTaDQCADcL3hYsWGDQ344lWrx4sew/R0vYpk6dKuvvwVz1SIim16SFCxfK/nvidu/NwcFBFBUVGfT3Q/ah2YtbAoCjoyMmTpxo0Lu6poiPj4eXl5d+CwwMlPQ8BwcHVFdXY/HixaiuroYQwu42Sz/+Dz/80KR/O2q1GkuWLIFarTbZPpYsWSL7z9ESfv/r1q0z2c9YCnPVI6DpNenNN9+0+P8mbelv0tBNo9GYfJFdc9QkS2atx2/U26gYytDhcLVaXe8HLIRATU0NfH1973uDV3u/vQGPn8dvz8cvRVNOz7EmNR2Pn8dvjcdvlJGmpnJxcUFoaCjS0tL0j2m1WqSlpWHIkCENvl+pVMLT01O/eXl5wc/P777FiYjofgytRwBrEpG9MWhFcFNYsGABZs+ejYEDByIsLAwJCQmoqKjA448/Lnc0IrIzrEdE1BjZm6bp06cjPz8fixYtQl5eHgYMGICkpCS0adNG7mhEZGdYj4ioMbI3TQAwf/58zJ8/36T7UCqVWLx4MZRKpUn3Y6l4/Dx+ez5+Q/x/7d17WFTXuT/w7wzCMKCgCCikihrvimhRCMZ4ARXQ0JBaL6mNxKaxMWrS4y8mTVslxkRiano4MVZjSk2TNMZLquY0ggqiiJdYjwU0GoNXVMS7M1xkRob1+yPPTCV4mYGZWXvY38/z7EdmMzPrXTC8vnvttdd2Rz4C+Dth/9l/T+y/1IngRERERJ5C6kRwIiIiIk/BoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOwgtWgqKChASkoKwsPDodFosGnTJpnhEJHKMScR0f1ILZqqq6sRFRWF5cuXywyDiAgAcxIR3V8rmY0nJycjOTlZZghERDbMSUR0P5zTRERERGQHqSNNjjKZTDCZTLbHQgiYzWYEBwdDo9FIjIyI1Ig5iUhdPGqkKSMjA4GBgbatbdu2CA0NRWVlpezQyE3MZjMCAgKg0Wjcsul0OlRXV8vuNikUcxJ98cUXbstHGo0GgwYNwu3bt2V3W7U8aqTptddew9y5c22PjUYjOnXqJDEicrd9+/ahsrISISEh2LVrl0uP5pOSknD27Fns3LkT48ePd1k75LmYk+if//wnAOCRRx5BXFwc9Ho9/Pz8oNfrbdsPH+v1evj6+kKrtX/corq6GomJiSgqKsLbb7+N+fPnu6pLdB92F01arRYajQZCCGg0GlgsFlfGdVc6nQ46nc7t7ZJy5OTkAAASExPRp08fl7aVnJyMlStXIicnh0WTwighHwHMSWpXX19vy0lvvvkmEhISXNresmXL8POf/xyLFi1CamoqIiMjXdoeNWZ30XT69GmnN15VVYUTJ040aKOoqAhBQUHo3Lmz09sjz2dNUElJSS5vKykpyVY0kbK4Ih8BzEnkmOLiYlRUVMDf3x/Dhg1zeXtTpkzB2rVrsXnzZjzzzDPYv38/vL29Xd4u3UFIlJ+fLwA02tLS0ux6vcFgEACEwWBwbaCkCOXl5QKA0Gg04vLlyy5vz2g0ilatWgkAorS01OXtkXzMSeSIxYsXCwDiJz/5idvaLC8vF+3atRMAxFtvveW2dul7dp9Q/eKLL1BTU+PMeg0jR46EEKLR9tFHHzm1HWoZtm3bBgCIjo5GSEiIy9tr06aN7ehx69atLm+P7OeKfAQwJ5FjsrOzAbhn5NsqLCwM//M//wMAWLhwIb755hu3tU0OXD03ceJEhIWFYcaMGfj6669dGRPRXbnz1JyVtS2eolMW5iOS7ebNm9i7dy8A9+YkAPjFL36Bxx9/HGazGdOnT0ddXZ1b21czh5YcePnll3Hw4EHExcWhf//+yMzMxLVr11wVG5GNxWKxjTTJKJp27NjRYD0eko/5iGTKy8uDxWJBr1690LVrV7e2rdFosHLlSgQGBuJf//oX/vSnP7m1fTVzqGj69a9/jUOHDuFf//oXhg8fjoULF+Khhx7CpEmTsH37dlfFSISDBw/i+vXrCAwMRGxsrNvaHTBgADp27IiamhoUFha6rV16MOYjksl6ak7WbXceeughZGZmAgAWLFiAb7/9VkocatOkxS2jo6Px5z//GRcvXsSHH36IK1euICkpye3VNqmH9fTY6NGj0aqV+5YX02g0SExMbBADKQvzEbmbEMKWD2TeqzAtLQ3JyckwmUyYPn26tKU31MTuouluiwj6+vri6aefRn5+Po4fP46f//znTg2OyErGfCYrzmtSHuYjkunIkSO4cOEC9Ho9hg8fLi0OjUaDDz74AAEBAdi/f79t5IlcRyOEEPY8UavVoqKiAqGhoa6OyW5GoxGBgYEwGAwICAiQHQ65yPXr1xESEoL6+nqUlZW5fcXla9euISQkBEIInDt3Dj/60Y/c2j41psR8BDAnqcU777yDV199FePGjcNXX30lOxz85S9/wXPPPQdfX18UFxejZ8+eskO6J4vFgj/+8Y9YunSpw1fAajSaB660frfV2GfMmIEOHTo4JX6HFrcMDg52SqNEjsjNzUV9fT369esn5RYV7du3R0xMDL7++mts27YNv/zlL90eAzXEfEQyyZ7P9EPPPvss1q1bh+3bt+OXv/wldu3aBS8vL9lhNXL+/HlMmzYN+fn5TX6Ppiw1MmHCBPcXTREREU5pkMhRMk/NWSUlJeHrr79GTk4OiyYFYD4iWSorK20XhcjMSXfSaDT4y1/+gn79+mHPnj1YtmwZfvOb38gOq4GNGzfiV7/6Fa5fvw5/f39kZmZi9OjRDr2HxWJBbW0tbt26ZdtqamoaPL7bPmceYNl9eu7jjz9u8HjatGlOC6KpOBTe8gkh8NBDD+HixYvYvn27w39kzrJ//37ExcWhbdu2uHLlilsno1NjSsxHAHOSGmzatAlPPvkkunfvjtLSUtnhNPDBBx/g+eefh16vR0lJCbp37y47JNTU1GDu3Ln44IMPAHx/4cZnn32m6FOI92N35l+9erXta41Go5gkRS3b4cOHcfHiRfj5+bnl3k73MmTIELRr1w43btzAgQMHMHToUGmxEPMRyaOEq+buZcaMGVi3bh127NiBZ599Fvn5+dBqm3SRvFMUFxdjypQptuUQ5s2bhzfffBM+Pj7SYmouu4um5pyDJGoqa4IaNWoUfH19pcXh5eWFsWPHYu3atcjJyWHRJBnzEckghJBy6xR7WU/TRUZGoqCgAH/+858xe/Zst8chhMB7772HV155BWazGWFhYfj444+lnSlwJnklKJEdlDCfyYpLDxCp27Fjx1BWVgadToeRI0fKDueuunbtiiVLlgAAXn31VZw6dcqt7V+6dAnjx4/Hb37zG5jNZqSkpKC4uLhFFEyAAyNNRO6mtAmX1kUuDx48iCtXrrjlpsFEpBzWA6YRI0bAz89PcjT3NnPmTKxfvx67du1Camoq4uLiHHp9UFAQevTogR49eqB79+7o2LHjXddG+6GcnBykpaXh8uXL8PX1xbvvvouZM2fa9VpPwaKJFCs/Px+3b9/Gww8/rIgJjWFhYYiKikJxcTG2b9/OxROJVEZpSw3ci1arRVZWFgYMGIDDhw/j8OHDzXq/1q1bo3v37g0KKevXoaGhMJvN+O1vf2tbXLN///5Ys2YN+vfv74TeKAuLJlIsJZ2as0pKSkJxcTFycnJYNBGpSFVVFQoKCgAov2gCgIcffhi5ubnIzc116HVCCFy6dAmlpaUoLS1FWVkZqqqqUFRUhKKiokbPb9OmDfz9/VFRUQEAmDNnDpYsWQK9Xu+MbigOiyZSpDsnXFpPiylBYmIilixZgq1bt6K+vl7qlSlE5D47d+6E2WxGly5dPOZy+bi4OIdPzf2QyWTC6dOnbUXUiRMnGhRUlZWVqKysRHBwMFavXo3HH3/cSdErk8NF0+rVq9G6dWtMnDixwf7169ejpqYGaWlpTguO1Ku0tBRnzpyBt7c3Ro0aJTscm0cffRT+/v64fPkyioqK8OMf/1h2SKrGfETucuepuZY0R+dBdDodevfujd69ezf6Xm1tLU6fPo3z588jOjoaQUFBEiJ0L4cPkzMyMu66umZoaCgWL17slKCIrKfmHnvsMbRu3VpyNP/h4+ODhIQEALyKTgmYj8gd7hz59oRTc+7i6+uLPn36YMyYMaoomIAmFE1lZWXo2rVro/0REREoKytzSlBEW7duBaCs+UxW1pisMZI8zEfkDqWlpTh9+jR8fHwUNfJN7udw0RQaGoqSkpJG+4uLi9G+fXunBEXqVltba1u8UIlFk3WO1d69e2EwGCRHo27MR+QO1lEmpY18k/s5XDQ99dRTePHFF5Gfnw+LxQKLxYIdO3bgpZdewpQpU1wRI6nM7t27cevWLYSHhyvyktVu3bqhZ8+eqKurw44dO2SHo2rMR+QOSr51CrmXwxPBFy1ahDNnziAhIcF209L6+npMmzaNcwjIKe5cakCpEy6TkpLw3XffIScnB08++aTscFSL+Yhc7datW9i5cycAZY58k3tphBCiKS8sLS1FUVER9Ho9IiMjERER4ezYHoh3FG+Z+vXrh6NHj2LdunWNropSiuzsbIwbNw6dO3fGmTNnFFvcqYUS8hHAnNQSWf/WO3XqhLNnz/JvXeWavE6TdTVQImcqKyvD0aNHodVqFX2vohEjRkCn06GsrAzffvst+vTpIzskVWM+Ile589QcCyZyeE7ThAkTbDcDvNM777yj2FEB8hzWK9IeeeQRtGvXTnI09+bn54cRI0YA4NIDMjEfkatZJ4Hz1BwBTSiaCgoKMG7cuEb7k5OTbUvMEzWVEm+dci/WGFk0ycN8RK508uRJlJaWolWrVrb12UjdHC6aqqqq4OPj02i/t7c3jEajU4Iidbp9+7btPklKunXKvVhj3LVrF2pqaiRHo07MR+RK1gOiRx99lHPUCEATiqbIyEisXbu20f7PP/8cffv2dUpQpE779++H0WhE+/btER0dLTucB+rTpw86deoEk8mEXbt2yQ5HlZiPyJW4Cjj9kMMTwefPn4+f/vSnOHnyJOLj4wEAeXl5WLNmDdavX+/0AEk9rEd1Y8eOhZeXl+RoHkyj0SApKQkffvghcnJymFglYD4iV6mtrbWtw8a/bbJyeKQpJSUFmzZtwokTJ/DCCy/g//2//4fz588jNzcXqampLgiR1ELJt065F95SRS7mI3KVOxfZjYyMlB0OKUST12m6myNHjrh1BWeuidJyXL58GR06dAAAXLx4ER07dpQckX0MBgPat28Pi8WCU6dO3fU+aCSHu/MRwJzUksydOxf//d//jV/+8pfIysqSHQ4phMMjTT9UWVmJVatWISYmBlFRUc6IiVRo27ZtAIBBgwZ5TMEEAIGBgRg6dCgAjjYpAfMROQtvnUJ30+SiqaCgANOmTUNYWBiWLl2K+Ph47N+/35mxkYp40lIDP8SlB+RjPiJnOnv2LI4dOwYvLy9FL7JL7ufQRPCKigp89NFHyMrKgtFoxKRJk2AymbBp0yZeqUJNVl9f75HzmaySkpLw+9//Hnl5eTCbzXe9BJ6cj/mIXMV61VxcXBzatm0rNxhSFLtHmlJSUtCrVy+UlJQgMzMT5eXlWLZsmStjI5U4dOgQrl69ijZt2iAuLk52OA4bOHAgQkNDUVVVhb1798oORxWYj8iVeGqO7sXuoik7OxvPPvssFi5ciPHjxzv1kvDly5ejS5cu8PX1RWxsLA4cOOC09yblsyao0aNHw9vbW3I0jtNqtbaFLnmKzj2Yj8hVzGYz8vLyAHjmyDe5lt1FU2FhISorKxEdHY3Y2Fi8//77uHr1arMDWLt2LebOnYv09HQcOnQIUVFRSExMxOXLl5v93uQZPHk+kxXnNbkX8xG5SmFhIaqqqtChQwcMHDhQdjikNMJBVVVVIisrSzz66KPC29tbaLVakZmZKYxGo6NvJYQQIiYmRsyaNcv22GKxiPDwcJGRkfHA1xoMBgFAGAyGJrVN8l2/fl1otVoBQJw5c0Z2OE12+fJlodFoBABx4cIF2eGohpLykRDMSS3BvHnzBAAxbdo02aGQAjVrnabjx48jKysLn3zyCW7evIkxY8bgyy+/tPv1ZrMZfn5+2LBhQ4OF6NLS0nDz5k1s3rz5vq+3d00UIQQ+/fRTu+Mi9zl8+DD++Mc/onfv3jh27JjscJplyJAhOHjwIObMmYMhQ4bIDkeaiIgIDB8+3O3tys5HgP05qaioCIcPH7Y7NnKfN954AydOnMCaNWswZcoU2eGQ0jij8qqrqxMbN24UKSkpDr3uwoULAoDYu3dvg/3z5s0TMTExjZ5fW1srDAaDbTt37pxdR3UWi0UA4Kbg7aWXXnLos6NEf/jDH6T/HJWwTZw4UervwV35SIim56T58+dL/z1xu/em1WrF1atXHfr8kDo0e3FLAPDy8kJqaqpDR3VNkZGRgcDAQNvWqVMnu16n1WpRW1uL9PR01NbWQgihuk3p/c/MzHTpZ8dkMuH111+HyWRyWRuLFi2S/nNUwu9/3bp1LvsZ28Nd+Qhoek564403FP832ZI+k45uFosF7du3d+lnxx05Sck8tf9OvY2KoxwdDjeZTA1+wEIImM1mBAcHQ6PR3Lcttd/egP1n/9Xcf3s05fQcc1LTsf/svyf23ykjTU3l4+OD6Oho2+WdwPcLHebl5d11vR6dToeAgADbFhgYiJCQkAcmJyKiB3E0HwHMSURq49CK4K4wd+5cpKWlYfDgwYiJiUFmZiaqq6sxffp02aERkcowHxHR/UgvmiZPnowrV65gwYIFqKiowMCBA5GTk2O74z0RkbswHxHR/UgvmgBg9uzZmD17tkvb0Ol0SE9Ph06nc2k7SsX+s/9q7r8j3JGPAP5O2H/23xP7L3UiOBEREZGnkDoRnIiIiMhTsGgiIiIisgOLJiIiIiI7sGgiIiIisgOLJiIiIiI7sGgiIiIisgOLJiIiIiI7sGgiIiIisgOLJiIiIiI7sGgiIiIisgOLJiIiIiI7sGgiIiIisgOLJiIiIiI7sGgiIiIisgOLJiIiIiI7sGgiIiIisgOLJiIiIiI7SC2aCgoKkJKSgvDwcGg0GmzatElmOESkcsxJRHQ/Uoum6upqREVFYfny5TLDICICwJxERPfXSmbjycnJSE5OlhkCEZENcxIR3Q/nNBERERHZQepIk6NMJhNMJpPtsRACZrMZwcHB0Gg0EiMjIjViTiJSF48aacrIyEBgYKBta9u2LUJDQ1FZWSk7NCIcPHgQQUFB0Gg0btny8vJkd1n1mJNIyaZNm+a2fKTRaPDCCy+gvr5edrddyu6RprKysgaPO3fu7PRgHuS1117D3LlzbY+NRiM6derk9jiI7iYrKws3btwAAOh0OrRv3x5BQUG2f+/8+s5/27RpA7PZjFu3btm2mpqau35969YtFBYWori4GP/7v/+LhIQEyb2WQwn5CGBOIuW6ePEiPvnkkwb7WrVqBb1eD71eDz8/P9vX99pn73MOHjyIl156CStWrIDZbMYHH3wALy8vST13LbuLpi5dukCj0UAIAY1GA4vF4sq47kqn00Gn07m9XaIHEUJgy5YtAICNGzfiiSeecNnpmS+++AI/+9nPkJ2djczMTJe0oXRKyEcAcxIpV05ODgBg0KBB2LlzJ/R6Pby9vV3SVlxcHNq1a4e0tDRkZWXBZDJh9erVaNXKo2YA2cXuHrliyK2qqgonTpywPT59+jSKiooQFBQk7ciRqCmOHTuGsrIy6HQ6jBkzxqXzWRISEuDl5YXvvvsOp06dQrdu3VzWllK56hQAcxK1FNnZ2QCAxx9/HAEBAS5v7xe/+AV0Oh1+/vOf49NPP4XJZMLf//53lxVqskid03Tw4EEMGjQIgwYNAgDMnTsXgwYNwoIFC2SGReQw6yjTiBEj4O/v79K22rZti6FDhwL4z9EkOQdzErUEdXV12LZtGwC4dQmNiRMnYsOGDfD29sb69esxceLEBhdKtAQOFU3FxcX461//ilOnTgEAvvnmG7zwwgt4/vnnsXXrVocbHzlyJIQQjbaPPvrI4fciksl6VDdu3Di3tGdNhNZ21cjZ+QhgTqKWYd++fTAYDAgKCkJMTIxb237iiSewefNm6HQ6bN68GU8++SRu3brl1hhcye6i6R//+Aeio6PxyiuvICoqCrm5uRg2bBhKS0tx5swZjB8/Hp999pkrYyVSpMrKSuzevRuA+47qkpKSAAA7duxocUdy9mA+Iro368FUYmKilAnZycnJ+Oc//wm9Xo/s7Gz85Cc/QXV1tdvjcAlhpx//+MfizTffFEIIsWbNGtG2bVvxxhtv2L6/dOlSMXDgQHvfzikMBoMAIAwGg1vbJbrTxo0bBQDx8MMPi/r6ere0WV9fLzp27CgAiO3bt7ulTSVRYj4SgjmJlCEqKkoAEJ988onUOHbt2iVat24tAIjhw4cLo9EoNR5nsHuk6fjx45g6dSoAYPLkyaiurkZqaqrt+08++WSDCZREamE9qktOTnbbgoYajcY22qTGeU3MR0R3d+HCBRQXF0Oj0SAxMVFqLMOHD8e2bdsQEBCAgoICJCYmwmAwSI2puewumtq0aYNr164BAG7evIm6ujrbYwC4du0aWrdu7fwIiRRM3LHUgLvvWWYtmtQ4r4n5iOjurAdRgwcPRkhIiORovl+OIC8vD+3atcO+ffswevRoXL9+XXZYTWZ30TR69GjMmjULf//735GWloaxY8fitddew7fffovjx49j3rx5GDZsmCtjJVKcb775BufPn4evry9Gjhzp1rbHjBkDrVaLo0ePNlrssaVjPiK6O3dflGKPwYMHY8eOHQgODsbBgwcRHx+PK1euyA6rSewumpYuXYqAgAA8//zzMJvNWLt2LQYPHoy+ffuib9++KC8vx9tvv+3KWIkUx5qgRo4cCT8/P7e2HRQUhEceeQSA+k7RMR8RNXb79m1s374dgPtHvh9k4MCB2LlzJzp06IDi4mKMHDkS//jHP3DixAmPuvWKRgghmvMGp06dQk1NDXr37u321T+NRiMCAwNhMBjcsngX0Q+NGjUKO3fuxHvvvYc5c+a4vf1FixZhwYIFSE1NxcaNG93evtLIzEcAcxLJtWvXLowcORLBwcGoqKhQ5K1Mjh8/jvj4eJSXl9v2+fn5ITIyEgMGDLBtkZGRaNeuncRI767ZRZNMTFAkk9FoRPv27VFXV4fS0lJ0797d7TEcPHgQQ4YMQevWrXHt2jX4+Pi4PQb6D+Ykkum3v/0tlixZgqlTp+LTTz+VHc49nTlzBosXL8ahQ4dw5MiRey6b0qlTpwaFVM+ePZu0wniPHj3g6+vb3LABOHAblY8//rjB42nTpjklACJPlZeXh7q6OnTv3l1KwQQAP/7xjxESEoIrV65g7969bp9XJQvzEVFjd17Jq2RdunTBqlWrAHy/evmJEydQUlLSYDt79izOnTuHc+fO4auvvmpWe9988w369u3rjNDtL5pWr15t+1qj0TBJkepZr5qTOeFSq9UiMTERn376KbKzs1VTNDEfETV0/vx5lJSUKGKpAUe0atUKvXv3Ru/evTFp0iTb/ps3b+LIkSMNCqnTp0+jKSfHnHmakqfniJpACIFOnTrhwoULyM7Otl3+L8Nnn32GqVOnIjIyEiUlJdLiIOYkkucvf/kLnnvuOcTGxmL//v2yw2mxpN6wl8hTHT58GBcuXIBer8eIESOkxjJ27FhoNBpbTESkPkoY+VYDFk1ETWCdOzBq1Cjo9XqpsQQHB2PIkCEA1Lf0ABEBZrMZubm5AJQ/n8nTsWgiagKlLSBnTZQsmojUZ+/evaisrERISAiio6Nlh9OisWgicpDBYEBhYSEA5RzVWedUbd++HXV1dZKjISJ3sp6aS0pKglbL/9ZdiT9dIgfl5ubCYrGgZ8+e6Natm+xwAABDhgxB+/btYTAYsG/fPtnhEJEbecpSAy2Bw0XT6tWrsX79+kb7169fj7/97W9OCYpIyZQ44dLLywtjx44FoK5TdMxHpHbnzp3DkSNHoNVqbTmAXMfhoikjIwPBwcGN9oeGhmLx4sVOCYpIqYQQtqJEaUd11lN01qNONWA+IrWz/r3Hxsaiffv2kqNp+RwumsrKytC1a9dG+yMiIlR3p3VSn5KSEpSXl8PPzw/Dhw+XHU4D1gXt/v3vf6OiokJyNO7BfERqp7SLUlo6h4um0NDQuy6gV1xczCqXWjzrqbn4+Hin3cvIWTp06GC7cmbr1q2So3EP5iNSMy414H4OF01PPfUUXnzxReTn58NiscBisWDHjh146aWXMGXKFFfESKQYSp9wqbZTdMxHpGaFhYWoqqpCaGgoBg0aJDscVbD73nNWixYtwpkzZ5CQkIBWrb5/eX19PaZNm8Y5BNSi3bx5E3v37gWg3KIpOTkZb731FrZt2waLxeLUey4pEfMRqdmdB3FcasA9mnzvudLSUhQVFUGv1yMyMhIRERHOju2BeJ8ncqf169dj0qRJ6N27N44dOyY7nLuqq6tDSEiIrcCLi4uTHZJbKCEfAcxJ5F79+vXD0aNH8fnnn2Py5Mmyw1EFh0earHr06IEePXo4MxYiRfOECZetWrXCmDFjsH79emRnZ6umaGI+IrU5e/Ysjh49yqUG3Mzh8bwJEyZgyZIljfa/8847mDhxolOCIlKa+vp6xc9nslLTLVWYj0itrPkoLi4O7dq1kxyNejhcNBUUFNz1SDs5ORkFBQVOCYpIaYqLi1FRUQF/f3889thjssO5L+tk8IMHD+LKlSuSo3Et5iNSK08Y+W6JHC6aqqqq4OPj02i/t7c3jEajU4IiUhprgkpISIBOp5Mczf2FhYUhKioKQogWv/QA8xGpkclkQl5eHgDlj3y3NA4XTZGRkVi7dm2j/Z9//jn69u3rlKCIlMa6PpOnJCi1nKJjPiI12r17N6qrq9GxY0cMHDhQdjiq4vBE8Pnz5+OnP/0pTp48ifj4eABAXl4e1qxZc9d7QBF5uhs3bthugutJRdPbb7+NrVu3or6+vsVejsx8RGp05/xKjUYjORp1cbhoSklJwaZNm7B48WJs2LABer0eAwYMQG5uLkaMGOGKGImk2rZtG+rr69G3b19pl7I7Ki4uDgEBAbh69SoOHjyImJgY2SG5BPMRqZGnjXy3JE1acmD8+PEYP358o/1HjhxB//79mx0UkZJ44oRLb29vjB49Gv/4xz+Qk5PTYosmgPmI1OXMmTP49ttv4eXlhTFjxsgOR3WaPWZfWVmJVatWISYmBlFRUc6IiUgx6uvrbfOCPO2ozhqvWm6pAjAfUctn/XseOnQo2rZtKzcYFWpy0VRQUIBp06YhLCwMS5cuRXx8PPbv3+/M2Iik+/e//41Lly6hdevWGDZsmOxwHGJdeuDAgQO4du2a5Ghci/mI1IKn5uRy6PRcRUUFPvroI2RlZcFoNGLSpEkwmUzYtGkTr1ShFsl6VDd69Oi7XtquZD/60Y/Qv39/HDlyBNu3b29xN7BlPiK1qa2txY4dOwB41nSBlsTukaaUlBT06tULJSUlyMzMRHl5OZYtW+bK2Iik8/SjupZ6io75iNSooKAANTU1CA8Px4ABA2SHo0p2F03Z2dl49tlnsXDhQowfP96pd09fvnw5unTpAl9fX8TGxuLAgQNOe2+iprp+/Tq+/vprAJ5bNFlP0VmXHmgpmI9IjbjUgHx2F02FhYWorKxEdHQ0YmNj8f777+Pq1avNDmDt2rWYO3cu0tPTcejQIURFRSExMRGXL19u9nsTNYd1qYH+/fujU6dOssNpkmHDhqF169a4dOkSioqKZIfjNMxHpEaecv/LFk04qKqqSmRlZYlHH31UeHt7C61WKzIzM4XRaHT0rYQQQsTExIhZs2bZHlssFhEeHi4yMjIe+FqDwSAACIPB0KS2ie7n6aefFgDEK6+8IjuUZnniiScEAPHmm2/KDsXplJSPhGBOItc5efKkACBatWolbt68KTsc1dIIIURTC67jx48jKysLn3zyCW7evIkxY8bgyy+/tPv1ZrMZfn5+2LBhA1JTU23709LScPPmTWzevPm+rzcajQgMDITBYEBAQMA9nyeEwLp16+yOiwgA5syZgytXriA/Px8jR46UHU6TrVy5EjNnzkRUVBRee+01l7fXqVMnDB061OXt/JDsfATYn5MOHz6Mo0eP2h0b0a5du7BixQqMGDECO3fulB2Oejmj8qqrqxMbN24UKSkpDr3uwoULAoDYu3dvg/3z5s0TMTExjZ5fW1srDAaDbTt37pxdR3UWi0UA4MbN4a1NmzbCbDY79LlWmjNnzrj1ZzZx4kSp/XVXPhKi6Tlp/vz50j/b3Dxze/vttx36XJNzOeWGVF5eXkhNTXXoqK4pMjIyEBgYaNvsnWei1WpRW1uL9PR01NbWQgihuo39b1r/jUYjvL29Xfq5drWIiAi3/v5lj+q6Kx8BTc9Jb7zxBv8m2f8m9f/VV1918afaPUwmE15//XWYTCbZoTikWafnmsvR4XCTydTgByyEgNlsRnBw8AOvJLB32LylYv/ZfzX33x5NOT3HnNR07D/774n9l3rrcx8fH0RHRyMvL8+2r76+Hnl5eYiLi2v0fJ1Oh4CAANsWGBiIkJAQXnpJRM3maD4CmJOI1KZJN+x1prlz5yItLQ2DBw9GTEwMMjMzUV1djenTp8sOjYhUhvmIiO5HetE0efJkXLlyBQsWLEBFRQUGDhyInJwcdOjQQXZoRKQyzEdEdD/SiyYAmD17NmbPnu3SNnQ6HdLT06HT6VzajlKx/+y/mvvvCHfkI4C/E/af/ffE/kudCE5ERETkKaROBCciIiLyFCyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDiyaiIiIiOzAoomIiIjIDlKLpoKCAqSkpCA8PBwajQabNm2SGQ4RqRxzEhHdj9Siqbq6GlFRUVi+fLnMMIiIADAnEdH9tZLZeHJyMpKTk2WGQERkw5xERPfDOU1EREREdpA60uQok8kEk8lkeyyEgNlsRnBwMDQajcTIiEiNmJOI1MWjiqaMjAwsXLiw0X6DwYCAgAAJERH9R1FREcaPH4+amhro9Xro9Xr4+fnZvr7XYx8fH9TW1uLWrVu4desWampqbF/f63FYWBh27dqFDh06yO62qjEnkZL96le/wmeffebydjQaDeLi4jBjxgw88cQT0Ol0Lm9TFo0QQtjzxLKysgaPO3fu7NxANBps3LgRqamp93zOD4/qjEYjOnXqxARFivDCCy9gxYoVbmvvvffew5w5c9zWnpK4Oh8BzEnk2SoqKhAWFub2doODg5GWlobnnnsOvXr1cnv7rmb3SFOXLl2g0WgghIBGo4HFYnFlXHel0+ladAVLnksIgezsbADABx98gCFDhtx3tOjOzWQywdfX974jUndu69atQ0ZGBrKzs1VbNCkhHwHMSaRcW7duBQBERUVh8+bNLm2rqqoKa9euxV//+ldcuHAB7777Lt5991089thjmDFjBiZMmAC9Xu/SGNzF7qKpvr7e6Y1XVVXhxIkTtsenT59GUVERgoKCXHLkSOQqx48fx5kzZ+Dj44OpU6fC39/fZW21atUKGRkZyM/Px61bt1pMMnKEK/IRwJxELYf1IO7xxx9HRESEy9t74403sGDBAmRnZ+PDDz/EV199hd27d2P37t2YM2cOnn76aTz33HOIjIx0eSwuJSTKz88XABptaWlpdr3eYDAIAMJgMLg2UKIHePfddwUAMWbMGJe3VV9fLzp16iQAiC1btri8PTVhTqKW4Pbt26Jt27YCgNizZ4+UGM6fPy8WLVokIiIiGvwtPfLIIyIrK0tUVVVJiau57J7TZLVjxw4UFhbi4sWL0Gq16NatG37yk5+gR48ezqniHGA0GhEYGMj5AyTdmDFjkJubiz/96U/4r//6L5e39+tf/xqrVq3CnDlz8N5777m8PaVSUj4CmJNIGfbs2YNhw4ahXbt2uHLlCry8vKTFYrFYkJubiw8//BCbN29GXV0dAMDf3x8REREICgpC+/btERQUdM+vrf/6+fnBbDY3muLwoItnfvWrX6Fdu3bO6ZC91dWlS5dETEyM0Gq1olWrVkKr1Yro6GjRsWNH4eXlJebNm+e60u4eeFRHSlBZWSl8fHwEAHHs2DG3tLlx40YBQDz88MNuaU9plJiPhGBOImX43e9+JwCIKVOmyA6lgYqKCvH222+L7t2733VE90GbRqNp0uuOHj3qtD7YPafpxRdfRHh4OG7cuAGdToeXX34ZRqMRBw8exI4dOzBp0iQ89NBDeOmll5pVxBF5mh07dsBsNqNr165uu1okISEB3t7eOHnyJEpLS6WNrMjCfER0b9b5TEpb3b5Dhw549dVXMW/ePBw9ehSXL1/G9evXce3aNVy/fr3B1z/cd/v2bYg7ToxpNBq7l3Vx6hxTe6urgIAAceTIEdvjqqoq4e3tbTui+uSTT0SvXr2cVs3Zg0d1pATPP/+8ACBeeOEFt7YbHx8vAIjMzEy3tqsESsxHQjAnkXzl5eW2EZaKigrZ4ThFfX29qKysFOXl5eLGjRuitrZW1NfXS4nF7tuo6HS6BivcarVaWCwW2/nJoUOH4syZM86r5og8gBACW7ZsAQCMGzfOrW1b27MeVaoJ8xHR3eXk5AAABg8e3GIWv9VoNGjdujXCwsLQtm3bRn//7mR30TRs2DAsWLAA1dXVuH37Nn73u9+hW7duCAoKAgBcuXLFeROtiDzEsWPHUFZWBp1Oh1GjRrm1bevQ+86dO1FTU+PWtmVjPiK6O6Wemmsp7J7TtHTpUowdOxZt27aFRqOBv78/1q9fb/v+sWPH8Mwzz7giRiLFso4yjRgxAn5+fm5tu0+fPujcuTPKysqQn5+P8ePHu7V9mZiPiBqrq6vDtm3bALh/5Fst7C6aunXrhpKSEhQWFsJsNuORRx5BcHCw7ftMUKRG1qM6GQlKo9Fg3LhxWLlyJbKzs1VVNDEfETW2b98+GAwGtG/fHkOGDJEdTovk0A17/fz8MHbsWFfFQuRRKisrsXv3bgDyhsKTk5OxcuVKbNmyxXZLEbVgPiJqyHoQN3bsWKlrM7VkdhdNH3/8cYPH06ZNc3owRJ4kLy8Pt2/fxsMPPyztkv/4+Hj4+Pjg9OnT+O6771rkDTLvhvmIqDFZF6Woid1F0+rVq21fazQaJilSvTsnXMoa4WndujWGDx+O3NxcbNmyRTVFE/MRUUMXLlxAcXExNBoNEhMTZYfTYtldNOXn57syDiKPInOpgR8aN24ccnNzkZ2d7ZZbuCgB8xFRQ9alBoYMGYKQkBDJ0bRcdi85QET/8c033+D8+fPw9fXFyJEjpcZinU+1a9cuVFdXS42FiOTgUgPuwaKJqAmsCWrkyJHQ6/VSY+nVqxe6dOkCs9mMHTt2SI2FiNzv9u3b2L59OwD5I98tHYsmoiZQyqk54D9LDwDqXB2cSO327t0Lo9GI4OBgDB48WHY4LRqLJiIHGY1GFBYWAlDOULg1DuvSA0SkHtaDpcTERGi1/G/dlfjTJXJQbm4u6urq0KNHD3Tv3l12OACAUaNGQafT4ezZs/j2229lh0NEbqSkke+WzuGiafXq1Q1uV2C1fv16/O1vf3NKUERKpsQJl/7+/hgxYgQAdZ2iYz4itTt//jwOHz4MjUbDxV7dwOGiKSMjo8HtCqxCQ0OxePFipwRFpFRCCEUWTUDDU3RqwXxEamddaiA2NvaufwvkXA4XTWVlZejatWuj/RERESgrK3NKUERKdfjwYVy4cAF6vd42sqMU1qH5goICVFVVSY7GPZiPSO2sB0lKO4hrqRwumkJDQ1FSUtJof3FxMdq3b++UoIiUyjrKNGrUKOlLDfxQjx490K1bN9y+fRt5eXmyw3EL5iNSM7PZjNzcXAAsmtzF4aLpqaeewosvvoj8/HxYLBZYLBbs2LEDL730EqZMmeKKGIkUQ8kTLtW49ADzEanZnj17UFlZiZCQEERHR8sORxXsvo2K1aJFi3DmzBkkJCSgVavvX15fX49p06ZxDgG1aAaDAXv27AGg3KO65ORkvP/++7alB2TdE89dmI9IzawHR0lJSVxqwE00oomLupSWlqKoqAh6vR6RkZGIiIhwdmwPZDQaERgYCIPBgICAALe3T+ryxRdf4Gc/+xl69uyJ48ePyw7nrmpqahAUFASTyYQjR46gX79+skNyCyXkI4A5idwrMjISR44cwZo1aziy6iYOjzRZ9ejRAz169HBmLESKpuRTc1Z+fn4YNWoUcnJykJ2drZqiifmI1ObcuXM4cuQItFotlxpwI4fH8yZMmIAlS5Y02v/OO+9g4sSJTgmKSGmEELZLe5V6as5KTUsPMB+RWllPzcXGxiIoKEhyNOrhcNFUUFBw1yPt5ORkFBQUOCUoIqUpKSlBeXk5/Pz8MHz4cNnh3Jf177OwsBBGo1FyNK7FfERqZS2alDzy3RI5XDRVVVXBx8en0X5vb+8Wn6BJvayjNvHx8fD19ZUczf11794d3bt3V8XSA8xHpEZcakAeh4umyMhIrF27ttH+zz//HH379nVKUERK42lHdWpZeoD5iNSosLAQVVVV6NChAwYNGiQ7HFVxeCL4/Pnz8dOf/hQnT55EfHw8ACAvLw9r1qy56z2giDzdzZs3sXfvXgCec1SXnJyM9957D9nZ2S166QHmI1Ij68g3lxpwP4eLppSUFGzatAmLFy/Ghg0boNfrMWDAAOTm5iruthJEzrB9+3ZYLBb07t0bXbp0kR2OXUaMGAFfX1+cP38eR44cQWRkpOyQXIL5iNRIqfe/VIMmLTkwfvx4jB8/vtH+I0eOoH///s0OikhJPO3UHADo9XrEx8djy5YtyM7ObrFFE8B8ROpy9uxZHD16lEsNSNLscb3KykqsWrUKMTExiIqKckZMRIpRX1/vsUd1alp6wIr5iFo6az6Ki4tDu3btJEejPk0umgoKCjBt2jSEhYVh6dKliI+Px/79+50ZG5F0xcXFqKiogL+/Px577DHZ4TjEOjK2Z88eGAwGydG4FvMRqYUnjny3JA6dnquoqMBHH32ErKwsGI1GTJo0CSaTCZs2beKVKtQiWUdpEhISoNPpJEfjmG7duqFnz5747rvvkJubiwkTJsgOyamYj0htTCaTbRkRTxv5binsHmlKSUlBr169UFJSgszMTJSXl2PZsmWujI1IOk89NWdljbulLT3AfERqtHv3blRXV6Njx44YOHCg7HBUye6iKTs7G88++ywWLlyI8ePHw8vLy2lBLF++HF26dIGvry9iY2Nx4MABp703UVPduHED+/btA+C5RdOd6zU18d7cisR8RGp050FcS11GROnsLpoKCwtRWVmJ6OhoxMbG4v3338fVq1ebHcDatWsxd+5cpKen49ChQ4iKikJiYiIuX77c7Pcmao5t27ahvr4effv2RUREhOxwmmT48OHw8/NDeXk5SkpKZIfjNMxHpEbW6QKeehDXIggHVVVViaysLPHoo48Kb29vodVqRWZmpjAajY6+lRBCiJiYGDFr1izbY4vFIsLDw0VGRsYDX2swGAQAYTAYmtQ20f2kpaUJAOLll1+WHUqzPP744wKAXX9TnkZJ+UgI5iRynVOnTgkAwsvLS9y4cUN2OKqlEaLpY/bHjx9HVlYWPvnkE9y8eRNjxozBl19+affrzWYz/Pz8sGHDBqSmptr2p6Wl4ebNm9i8efN9X280GhEYGAiDwYCAgIB7Pk8IgY0bN9odFxEAzJw5E5cvX0ZeXp5ttWlP9Oc//xmzZs3CoEGD8Ic//MHl7T300EOIjY11eTs/JDsfAfbnpKNHj+Lbb7+1OzainTt3YtmyZXjsscd4M2qZnFF51dXViY0bN4qUlBSHXnfhwgUBQOzdu7fB/nnz5omYmJhGz6+trRUGg8G2nTt3zq6jOovFIgBw4+bw1rp1a2EymRz6XCuN9QjVXdvEiROl9tdd+UiIpuek+fPnS/9sc/PMbfHixQ59rsm5nHLTGi8vL6Smpjp0VNcUGRkZCAwMtG2dOnWy63VarRa1tbVIT09HbW0thBCq29j/pvW/srISPj4+Lv1cu1rXrl3d+vtft26d1P66Kx8BTc9Jb7zxBv8m2f8m9f+1115z8afaPUwmE15//XWYTCbZoTikWafnmsvR4XCTydTgByyEgNlsRnBw8AOvJLB32LylYv/ZfzX33x5NOT3HnNR07D/774n9l3p7ZB8fH0RHR9sW6wK+v21FXl4e4uLiGj1fp9MhICDAtgUGBiIkJISXXhJRszmajwDmJCK1adINe51p7ty5SEtLw+DBgxETE4PMzExUV1dj+vTpskMjIpVhPiKi+5FeNE2ePBlXrlzBggULUFFRgYEDByInJwcdOnSQHRoRqQzzERHdj/SiCQBmz56N2bNnu7QNnU6H9PR0j7t/mLOw/+y/mvvvCHfkI4C/E/af/ffE/kudCE5ERETkKaROBCciIiLyFCyaiIiIiOzAoomIiIjIDqopmpYvX44uXbrA19cXsbGxOHDggOyQ3OL111+HRqNpsPXu3Vt2WC5TUFCAlJQUhIeHQ6PRYNOmTQ2+L4TAggULEBYWBr1ej9GjR6O0tFROsC7woP4/88wzjT4PSUlJcoJVMeYj5iOA+cgT85Eqiqa1a9di7ty5SE9Px6FDhxAVFYXExERcvnxZdmhu0a9fP1y8eNG2FRYWyg7JZaqrqxEVFYXly5ff9fvvvPMO3nvvPaxcuRJff/01/P39kZiYiNraWjdH6hoP6j8AJCUlNfg8rFmzxo0REvMR85EV85EH5iOhAjExMWLWrFm2xxaLRYSHh4uMjAyJUblHenq6iIqKkh2GFADExo0bbY/r6+tFx44dxR//+Efbvps3bwqdTifWrFkjIULX+mH/hRAiLS1NPPHEE1Lioe8xH0XJDkMK5qOWkY9a/EiT2WzG//3f/2H06NG2fVqtFqNHj8a+ffskRuY+paWlCA8PR7du3TB16lSUlZXJDkmK06dPo6KiosFnITAwELGxsar5LADAzp07ERoail69emHmzJm4du2a7JBUg/mI+ciK+eh7npaPWnzRdPXqVVgslkYr+nbo0AEVFRWSonKf2NhYfPTRR8jJycGKFStw+vRpPPbYY6isrJQdmttZf99q/SwA3w+Ff/zxx8jLy8OSJUuwa9cuJCcnw2KxyA5NFZiPmI+smI88Mx8pYkVwcp3k5GTb1wMGDEBsbCwiIiKwbt06PPvssxIjIxmmTJli+zoyMhIDBgzAww8/jJ07dyIhIUFiZKQGzEd0J0/MRy1+pCk4OBheXl64dOlSg/2XLl1Cx44dJUUlT9u2bdGzZ0+cOHFCdihuZ/1987PwH926dUNwcLAqPw8yMB81xHzEfHQnT8hHLb5o8vHxQXR0NPLy8mz76uvrkZeXh7i4OImRyVFVVYWTJ08iLCxMdihu17VrV3Ts2LHBZ8FoNOLrr79W5WcBAM6fP49r166p8vMgA/NRQ8xHzEd38oR8pIrTc3PnzkVaWhoGDx6MmJgYZGZmorq6GtOnT5cdmsu9/PLLSElJQUREBMrLy5Geng4vLy889dRTskNziaqqqgZHKadPn0ZRURGCgoLQuXNn/OY3v8Gbb76JHj16oGvXrpg/fz7Cw8ORmpoqL2gnul//g4KCsHDhQkyYMAEdO3bEyZMn8corr6B79+5ITEyUGLW6MB8xHzEfeXA+kn35nrssW7ZMdO7cWfj4+IiYmBixf/9+2SG5xeTJk0VYWJjw8fERDz30kJg8ebI4ceKE7LBcJj8/XwBotKWlpQkhvr/Md/78+aJDhw5Cp9OJhIQEcfz4cblBO9H9+l9TUyPGjh0rQkJChLe3t4iIiBDPPfecqKiokB226jAfMR8JwXzkiflII4QQ7i3TiIiIiDxPi5/TREREROQMLJqIiIiI7MCiiYiIiMgOLJqIiIiI7MCiiYiIiMgOLJqIiIiI7MCiiYiIiMgOLJqIiIiI7MCiiTyeRqPBpk2bZIdBRASAOaklY9FEzfLMM89Ao9E02pKSkmSHRkQqxJxErqSKG/aSayUlJWH16tUN9ul0OknREJHaMSeRq3CkiZpNp9OhY8eODbZ27doB+H6YesWKFUhOToZer0e3bt2wYcOGBq8/fPgw4uPjodfr0b59e8yYMQNVVVUNnvPXv/4V/fr1g06nQ1hYGGbPnt3g+1evXsWTTz4JPz8/9OjRA19++aXtezdu3MDUqVMREhICvV6PHj16NEqoRNRyMCeRq7BoIpebP38+JkyYgOLiYkydOhVTpkzBsWPHAADV1dVITExEu3bt8K9//Qvr169Hbm5ugwS0YsUKzJo1CzNmzMDhw4fx5Zdfonv37g3aWLhwISZNmoSSkhKMGzcOU6dOxfXr123tHz16FNnZ2Th27BhWrFiB4OBg9/0AiEhRmJOoyQRRM6SlpQkvLy/h7+/fYHvrrbeEEEIAEM8//3yD18TGxoqZM2cKIYRYtWqVaNeunaiqqrJ9/6uvvhJarVZUVFQIIYQIDw8Xv//97+8ZAwDxhz/8wfa4qqpKABDZ2dlCCCFSUlLE9OnTndNhIlI05iRyJc5pomYbNWoUVqxY0WBfUFCQ7eu4uLgG34uLi0NRUREA4NixY4iKioK/v7/t+48++ijq6+tx/PhxaDQalJeXIyEh4b4xDBgwwPa1v78/AgICcPnyZQDAzJkzMWHCBBw6dAhjx45Famoqhg4d2qS+EpHyMSeRq7Boombz9/dvNDTtLHq93q7neXt7N3is0WhQX18PAEhOTsbZs2exZcsWbN++HQkJCZg1axaWLl3q9HiJSD7mJHIVzmkil9u/f3+jx3369AEA9OnTB8XFxaiurrZ9f8+ePdBqtejVqxfatGmDLl26IC8vr1kxhISEIC0tDZ9++ikyMzOxatWqZr0fEXku5iRqKo40UbOZTCZUVFQ02NeqVSvbxMb169dj8ODBGDZsGP7+97/jwIEDyMrKAgBMnToV6enpSEtLw+uvv44rV65gzpw5ePrpp9GhQwcAwOuvv47nn38eoaGhSE5ORmVlJfbs2YM5c+bYFd+CBQsQHR2Nfv36wWQy4Z///KctQRJRy8OcRC4je1IVeba0tDQBoNHWq1cvIcT3EyKXL18uxowZI3Q6nejSpYtYu3Ztg/coKSkRo0aNEr6+viIoKEg899xzorKyssFzVq5cKXr16iW8vb1FWFiYmDNnju17AMTGjRsbPD8wMFCsXr1aCCHEokWLRJ8+fYRerxdBQUHiiSeeEKdOnXL+D4OIpGNOIlfSCCGEjGKN1EGj0WDjxo1ITU2VHQoREXMSNQvnNBERERHZgUUTERERkR14eo6IiIjIDhxpIiIiIrIDiyYiIiIiO7BoIiIiIrIDiyYiIiIiO7BoIiIiIrIDiyYiIiIiO7BoIiIiIrIDiyYiIiIiO7BoIiIiIrLD/wcVA8j8ELjXKQAAAABJRU5ErkJggg==\n" + }, + "metadata": {} + } + ], "source": [ "# Finally, we plot the results and optionally compare the three different training setups.\n", "# Try to adapt and extend the plotting function to the needs of your experimental setup.\n", @@ -3105,12 +2739,13 @@ "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "\n", - "fig, axes = plt.subplots(10, 2,figsize = (5, 7), sharex = True)\n", + "fig, axes = plt.subplots(10, 2, figsize = (6, 8), sharex = True)\n", "\n", "# Iterate over the different training setups for comparison\n", "for log_id, logs in enumerate([history_catastrophic_forgetting, history_memory_replay]):\n", " # Format the logs into a dataframe\n", " logs = format_results(logs)\n", + "\n", " # Iterate through the 10 different MNIST classes (0...9). We will plot one subpanel\n", " # for each of them, showing the accuracies for that particular class over the course\n", " # of the entire training.\n", @@ -3124,24 +2759,13 @@ " \n", " # Finally, we will optimize the plot a bit and remove unneeded lines\n", " ax.set_ylim([0, 1])\n", + " ax.set_ylabel(f\"Acc '{class_}'\")\n", " sns.despine(bottom = True, trim = True)\n", "\n", - "axes[-1].set_xlabel(\"Epochs\")\n", + " ax.set_xlabel(\"Epochs\")\n", + "plt.tight_layout()\n", "plt.show()" ] - }, - { - "cell_type": "markdown", - "source": [ - "Here are some example outputs:\n", - "\n", - "| With catastrophic forgetting | Training setup using a regularizer |\n", - "|-|-|\n", - "|![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAfgAAAMNCAAAAABRgzkgAAAAIGNIUk0AAHomAACAhAAA+gAAAIDoAAB1MAAA6mAAADqYAAAXcJy6UTwAAAACYktHRAD/h4/MvwAAABBjYU52AAAPAAAABDgAAAmIAAABKWSQUlwAAC9eSURBVHja7Z17XJRV/sc/w20G1Bm8IEqieC9vSCikpqJiiMpF0NQ1l/D2s9RSdtvNLYXabe2y66UkLctsa/MSJgOappKXUldTI81bYpg3UDAZFBhA5vn94Y0ZgWHmOfMw53m+75fLq5md833OOe/5nnmu56gEEErEhbqAxBMkniDxBIknSDwhL/F7o/xU6dQryhNfEphKfaII3MxfRkZSlyhSfHXKywGhooWKOkmO1LFzt0in03m3vGn5dqHKXlpcpv62yu2WdvfvR4wyfn4SUOz/0NseLxuNZXf/lRmNZVX13tb1994ksdY4UKD7w52uvdu9Gk+NxvPOH42ne13jb09G4tXqGt/WLrKzSeljPlzoRWatoEfU+9VfCw76qZXwOD6q/Y3PSKwVBD1izN5w1C6Whfhb2dnIzb7gkE25zsEyughshTM5HhGSbMhC/OGgICQFLXTMtqY0PrWD1NZNBoY0aQjxYYIgCMIax2xLNwVLSa018dHSbEhV9+BbrDNo2W0sp4twuivJrYMCX+GCf0NkvGPpNBrvkty62CIESeNd4qtzc7HmBtmtA71UI73E4of0KP2Y7NZO2XaLgzm5iFfNxXu3yW+tfFvaprcsxeMPzS/oyW+tZCBaqmtiEov3nElHdLVjypTsJ17awzkAV9rdPhxMimvmh5DGhWp5Zjz8nsYyMlwLGRghlXfpb7Z8EevySHFt4iUb6aUXH9K/ciUprpHcY66j5CseL2KFkSTXRCaebCZj8XH+BetIck1IOdI3gHi32VhKl+VroGgPouQsHtM8f9pDmh9m2+3HOstafLMEOqKrCUlHeslP4ADAqW6qnA4k2oJKH8O+/rLOeDwWISwn0ZZ8Z2gZKuHmGuRp2bn4qJhMW6DHaFe5i3+q6801ZNocQdqf+IYR7/Ii3jWRazN+Pq8Jl714/NH73BZybUYGwhvJX3yj6XRE95B4SUf6BjmcA3ChQ9WxniDuk+eHK63ln/FoG0cpb8ZmhErqvcEmP3oRn+eQ7gdIPdI3mPj+/csHnSTf9yjZqRTxqo098gYfJeN32WFs310Z4tFqd5/CoftJ+R0kvK+6ocWjedZAw/Ascg4AVZulHukbcmZL7baI0lGZZB3AwQLvgcoRDy/9mPK49aQdyMBIdwWJh3rDM7cn0lOUDXAw19CTGLt9+n/CNDqTc/aU2whliYfLij9h7htKv/cyA2G6hhafGqAJPSTh9lXvvIZX5yvcfAOM9Jbi1yclHw2MuCal+YX/wltzFH11/vr3Ut5XXbP4xdMTu630Wi1pFf70gSp1ipKnS/ja1CtA8o2aT2lacWQ+4BJ+QNo6zGiU8Gm+gp+d3irZ/Ce1ii+s8gXgexrAnWnLpbkncpLXhG++UXDKN4R4K9OW1zT3VnlKOYMNm0UZUy7YhzHZKDCATRj7owQ7tntrwvwOnAqvtFggoUh/L+NrWqiAzV05zhRFhk2yGsU84z2CswBTVj8AgFqr1ep8aIEKeWIxX31SQp+QpSWJ1C9KEz++YGF+722+1C+yxzXF/HXIvAXT21grE+bGYstOFEWGTbIWRUVzFCgTWmKUxBMkniDxBIknSDxB4gkST5B4gsQTJJ4g8QSJJ0g8QeIJEk+QeMIx4vdG+anSqVeUJ74kMJX6RBFY3JEXGUldosiMJxSa8dWp7REqQubiF70GAA89glXyip2bEn44APhPmdKWer0Ovh+otfdBpvH9bPjww/fVqzbFPsj4Yn+W05b//NFnv0MVMT1K6rm9OOIv70z6XIrt1PEbr9ZqtWwnq++x9PIXQ4Rt8W3+epYM14JU8+FYZPytHAQtHtLs/mjsgIUKcj7+5CoweHq8hiw/zJlH3QukmQHL/An9XQCAhPuvDTAIzKnYNNIFaJcvEA/xDoZLsyGLoT5MEARBWOPY75p77Jbzr/n+tgZEg430DbUmDbBqRu8fybMlhb6m8+0k2VKDncCJd8s+TaIt2WLqLY33hhPfLAJrSbQl0s1x2XCnbCdiLT2bb4HxGwWIj/E8S2vSWLCrxO9x2YtvHIV1pNocCZemacCrcxOwTvTcxVUn5fRzIeWK0g0oPlJ7aZ/YGK90XyEj8UevNBqiAPGaONH79UXL8ZaM5r3OwAjJzmM35I0YE/FlpbgIq0pw4Sv5iNdLuGBBQ4of6lP4ragAle+iJxbLxvtvP7mMVIR4t3Eix/qNl1pu9jh4ADIhEwNaKEI8JmKTUURxYQmebzsJS+QiXtKlaRrsIg0AmAIuboyzv/j+AeoLLY/3cjkXIAvvBp/KM12UkfEuE0SN9YvxTEv0DDe9J4+E/6byUem8N/Dt1ROx+abdhXM3YS6AefjoJuSAtIuQNaz43l2MersLv2ca3gPAiK7FqyEDKrcoSLxqov1jffFHmAcALvOwrEoG4r8vavGEYsRjIrZft7PoxzcfjQAATG6Wqwf/ZGC0q3LEdw26vdG+klXvYt6dynvNlMMRnaCXdp3Rhn52zu6xPv1888l3/3OW+/eHuRd/Mlc9XEnix2PPZbsKLsFMz7v/6TdBBimfgfDGShLfdoCwwZ5yh/a5z7r/Yh42XOJfvLQrSjf4Y9J2jvVLMLH1/RdBg28v59x7/kGMVpb4ca4/nLO91MUv7xzL3U/5D0v4Fr9F6OunLPEth9lz693yqrDe1V6O7nTjU77FS7xP7wwzYthzvv7WB2YJD9cXsdTU4C0RQekO5Ykf43HiuK1l1hg6mf8iPut9dgvP4nca2/VUmnjvkTanvGkZ5ppXvPEMvo/oMhAj8ZQzTjD50USss/Ee6c053gkWb8123ZXNr3dTptQjvTOIH90o95BtJZZghuXJDv9xWMqv+EPXdIOUJ94rxsax/sfdbnMeenMevsjjVnwGIqWeFsgZ5rmbiPU2XVddgnEPL3gdMqDyfY7FSz3SN+w9d3epaHUja2j9P57XrvJQ34ff3ji2xQVPPr2f6+RW4K3AjPcYa9M5nNTKATV4R2xA4edO0Bj7En6w1N4fEp8aoAk9JHUlJiKtot4fLl2BpJred30BSzh9grIBRnrLWa/Weaw+Md37qmNnvXqI262RWe8Pr0T72zX+H4Ym2MrlTFfXXfGr5Bu1yPjF0xO7rfSS+uZF16frv19vWooXar5FSTuN05M4W6t6tpd8o+Y7dxVeabFAQtH9e9gk2bkDDj7R6KV6fvTqiiaXaqnS+Y6mv/K4e7f58Cv/kHyj5pMYF1b5AvC9Mx1VeTlQLE0tQjr8mlLvD0+r7asYEJf2FpcpjxjpN1nHXv0inU7nX8P75SnlDDZsFkV1zoZfp8W11uVLe3/wjMlGBj+b9kfp69jurYk6hvra5qtnM/w7UxQZNslqFPOM9wjOAkxZd+Y9V2u1Wp0PrVMgTywWKkhK6BOytCRRmX2hZPHjCxbm997mS/0ie1xTzF+HzFswvY21MmFuLLbsRFFk2CRrUVQ0ragyoeXHSDxB4gkST5B4gsQTJJ4g8QSJJ0g8QeIJEk+QeILEEySeIPEEiSccI35vlJ8qnXpFeeJLAlOpTxSBxR15kZHUJYrMeEKhGV+d2h6hImSe8Yt0Op13S3ms8ETYIH6+wWC4SD0kU+oY6tXqBqpT5U+/mD3mUWU0GsuMZXf/6N7pTdqYi7+VA+RmN2vbYPUpOrBv/8HSOj8Ssb8jeRONxSNUu4cAQMKae68lmgoFACD8um//vhMCAO9AD7M6emo09/9pFmZ32teSxDEWb4lU4k2H9u3bfxUAOg0YMODRug4y8/qf77NL0nV7SLzjmLgOgEfwgAH9rSfzLwMKIzKlnvqVxDuE7RFuIwcMCNbU79MHh5b+cQ2dXpCB+Kqg43NtmaJuS0zVy4vInSic4pTtp8e9X7Xl86NW4c13yR334ktexavNbSqR+AbmbnCCmpN4Ufw7L2C2jUXmzxIm7yJ7fIvPextv2nqSULUsviL2GOnjWnxySejTNhdy/XxQ8YjfyB/H4n/+GP+249hMo++RF3GdBPIr/i+m+AH2lPPe6n9mdCkZ5FX8jq1ub9pXss22pv8bf5sU8im+6s+Y1cnOst0yNZufo3n6+BT/2THdArsLD1jn8tEH5JBH8aWv2HruxoyYt5HC+cLxChW/+IrN527MeKHD1WUkkT/x+W9ikUZMAPfX8fYNssid+JSSkPHiIkzsaXibLNpBg16WPdnT9N2TImNkRnuea00e+cr4l0xxYr1jdL+yN0gjXxm/c7jbyc6io+we4n5G+vX6KOPtp+rPeF68d4Q9VZlMHnnK+E+f1eW0YBDnSB/VsR5kkpuML30Fr7DwjuB4YQEIbsQvudxuDptIf3dJP0gmeRF/Vey5mwc8loC/kUlOxAsv3eo7nlWwZI9vs0glF+Kr/u8z/JvZttvNxN/o8iwP4iv/uMrl44Hs4r3S6FA6uXR+8eXjvnD7YgrDgC3n4dUqkuns4kui9OpN45mG/FPTk/8lmU4u3jBiR6Mto9nG9H4ZyRVk06nFXx/2vW7HMNZRZ7c+/yHZdGbxeYOPtNjVj3lYr4X4B92E5cTifxt0wm9PkAMCT+lwlR6gdV7xvwzMCfiumyMie9BNWKLEpwZoQg85bGvHB13s+l0Hx8Se2LOIbsKyW/z6pOSjgRHXHLSxHwZfDdzbxlEteQPL8kmoneIXT0/sttJrtWO2tXfYjdBdjpupbHS/sn+Q0PpiPsFhxZH5gEv4gbpKlC61c1Nl/y4Ly2jiuKao/jnkw5ZuCnY5srctnxaqcxn7BUF4KUQQBEEwGgyGizAIFhSKqFmp4FCeUnYSr7Klr6zMXu3/8NvNjclGO8Vs8XwQpjylnEFbzaN8Y+83xv4mOVOUabChe83vuavwSosFEor0QO3z1bO5D8+ZosiwSVajmGe8R3AWYMq6c2JNrdVqdT40kaA8sdgZSkroE7K0JJH6RWnixxcszO+9zZf6Rfa4ppi/Dpm3YLq1UyyuYSwOmpwpigybZC2Kiu5VUya0/BiJJ0g8QeIJEk+QeILEEySeIPEEiSdIPEHiCRJPkHiCxBMkniDxhGPE743yU6VTryhPfElgKvWJIrC4Iy8ykrpEkRlPKDTjq1PbI1SEzDN+kU6n82550zHb/Vmljh7Ys00jlc24biNrjs34+UlAsb+DtpuB4RkAUHGj6MaNG0U3DEajscxoLLvzp44nPQXT4hGkzaHi1WoHbjcD0QAAD18bn9f6tePO39qRN9FYDPW3srORm33B4ZvNPwg757bsMFT4hLQxF384KAhJQQsdvtnNCPGzs+hUfELzFYvHYqgPk+hRunsjvR2M0V3IUvicJw7IeIko3WG/eM9J+Ji8cSp+hzHA/gXDpiL9OonjU3wGou0/L/R47wqam55P8VWZ9o/0AKbiY3qqn0vxhwp0g0QUn6Q+doTM8Sg+AyPdRRRvGke7d7yKjxZVfiq+KCV1/InPOekm7nT7kPbFG0kdf+IzMNhbXKUTaaznU3y0yAjPqvbkkDvexF//HlEiQ/hHYDUIzsRvrerZXmyMqVhzm+RxJj4DMaJjRLfIoxtxOBNfvk30Tzzg8QyN9byJ33OzdbD4KFOReZXscSVejygGG+0RcvszsseTeCGDwUgPulLDnfjsS15DWcSZ4HX6AOnjSHwGnvJkEUc7js7ecSY+hk2gqVh/EwQv4i8dVY1iE+nJLiUbyB834jPR34dNJNUUGus5Eq9ns08PAAmuB06RQE7EF3/LTnyrUZTy3IjfXtm5K7NgU/GfCjLIh/gMxLB73n5kq4ItZJAL8be3sBvpAbcEGus5Eb//9+b9GIabgq2XSSEP4jMwyo1huC5Pmj4lhRyIF/QsR3oAU7HaRA6dX/zpHA+2DziPa3LuC3Lo/OIzMLQJ04CN/ojJ0efIovOLj2Ec8e0kt8zur5aQR5uxspp0sc6gZbata62Ei21YN+DUCzvh/69xNBufE2f8FiGYuXc8tv2rdhfHD/uZVIoTnxqgCT3koG1lMN6nv4NqzKkUza7eLxaRTBHi1yclHw2MuOaQTZVtd4h4wDP5VFzVu10+piM7WxDMCJklCFV+i+6/NsAgsCIT/ibBUWx/FOh7UCDqi/mJtIoj8wGX8DrvYiyzd0b7LWImvrHG8GPvpfwQOrmXknP4KZtab/Y1uIz9giC8FCIIgiAYDQbDxYczvtD+mm136Hc4L0Hhg/cqW3rLyuzVNcxh3NyYbLRTzfBqYcpTyhm0tXqUVmvs/s7Y3yRnijLNlu41P46v8EqLBRKK9EDt89WzObZ3pigybJLVKOYZ7xGcBZiy7lw6VWu1Wp0PnRmRJxZXSZMS+oQsLUmkflGa+PEFC/N7b/NVZl8oCtcU89ch8xZMt3Ze1TWMxd0UzhRFhk2yFkVFj5wqE1p+jMQTJJ4g8QSJJ0g8QeIJEk+QeILEEySeIPEEiSdIPEHiCRJPkHjCMeL3Rvmp0qlXlCe+JDBVmf2gOCzuyIuMpC5RZMYTCs346tT2CBUh84xfpNPpvFs69yoQz6jspZ3C7yt/+L561abYBxlf7M9w8iP2VLQ0JDat/f/WeGo0nvf+qKt/xyseKznUV9Hi6xjq1Wqnr/0eg+8qV7tKjtqQpmzxFkP9rexs5GZf4KX26Yi2zzvGIk3ZY73FUL97CAAkrLn3muk8d8wR/C9vGWlf0Vsty44GUcbfJ0wQBEFYw0nlj1xuZO/ihY0jkabojOf6OD4dkRp7y47Fl4oe6zkXH2t32VHqs4qeDZNn8TknXEfaXVgboeyxnmfxeoQ1tb/0WBLPK2JGeiDK/eRJEs8j1/aJmv7eezg2knge2Sw87i+mvLLHeo7FixvpgRi3Y7+QeP64tV2k+GZDlTzW8yt+e3mHHuIiKHqs51e8XvQytbEuR38l8bxxO1PkSA/4hCl4rOdW/Hc3WvQXG0PJYz234tMRJXrmzzGqQ7+ReL4Q9AwWL2w1EF+ReL746TfP4eKjKHis51V8OiK8xEeJw36lLkDPr/hYBlEe6a/YsZ5T8ed/chnFIo5yx3pOxesxsAWLOPH4Lp/EcwSbkR5oGyJsIvH8cH0vq5XoFTvW8yl+i6lXezaR4rG7gMRzA6uRHujwuCmdxPNC2TfMxCt2rOdS/M7Str3Zic+6TuI5IV30pfgHdA6syiDxfFCVwW6kV+xYz6P4/YVNB7IUv6OIxHOBHqPc2UV7tHtlJonnASGd5Uiv1LHeytqyzjgxws891YWN2ca75swT/VDG3yUdw1l6R/eu5VuUl/EcitezOk9/F5Uix3r+hvqLbVV5vkwjZgdpCho7WSslz/jUAE3oIaeucQb6s/WOwI7GzYrLeAvx65OSjwZGXHPmGjPepwegGouJnce/tVNRp24thvrQvsth8p/z8r3XzjfUF/nc/qUz45i/jjoNAAh4/PHgx1sqQ7z5QwkVR+YDLuEH6ipRtqpBK3zydjfW3tHh1PUfjxw9mnP+/FfAI8FBzfh0Oay73eILq3wB+N75/peXA8UPlyh9sYHbF+OAmM3Dw4Gi7KNHjp65fJnXizarbBEPoTqXsV8QhJdCBEEQhGQAgEGwxJhsFMTjTFFk2CSrUcx/4yu80mKBhCL9vYyvab56Nr/7zhRFhk2yGsV8r94jOAswZfUDAKi1Wq3Oh9YpkCcWT5wmJfQJWVqSSP2iNPHjCxbm997mq8y+UBSuKeavQ+YtmN7GWpkwNxZbdqIoMmyStSgqhS/Nolho+TEST5B4gsQTJJ4g8QSJJ0g8QeIJEk+QeILEEySeIPEEiSdIPEHiCceI3xvlp0qnXlGe+JLAVOoTRWBxR15kJHWJIjOeUGjGV6e2R6gImWf8Ip1O593ypvzaLBiL8q8o/bbyh++rV22KfZDxxf7ON91Z7Zh+LiozlhmNd/+UC+a67/8fAPC3N5Qtvo6hXq3mrC3nn91T/w9//LoriX/ArRwgN7tZWw5bInwy96YmQOOp0Xje+aNRm/+Oae6+7anReML36oEnlZ3y5o/L7wIAJNx/bahhYgTnJD8aGJBT749PxlxOGuYgOJzStEY2zSj0+Puf6j9662Pbnlf08Yo8juMNz8YV9vrhLzb8aj/ldeGookd6WYj/tuenLvMP9bKliOdI5S4kLRfxZXOHXey49582HoPEYaOiD+X5F3/48WWYmT3A1mKjPM6cIvH8UvnaE6dbf73C9jmItcOVPdZzLv5C/5Sq8T/bdUkxDhtJPLfMOdx07Tr7ZiCNdsn+lcRzSm4m9k6ws2yLwVDoQtIyEL9CeKqH3YXjFf0jz/WZu9I2NzJH2136chtc9qOM55EvbrQXcavYI08gXbkZz7N4YTlmibm2GqfksZ7nof67QZ6XxKwpcK6T69XmlPH8sRzPiFpLoqNCF5LmXfzljZgtLoKSx3qOxX9QNaiXuAhx2C7De0nlLr78A7EJj+5dKr4m8byRdu2RWJEhVAo+X8+v+PcwU/Qq8nH4uozE88UPBz1miA7Sx79kB4nni+V4WvyakAoe63kVf20d5jAIE4eMShLPEx9V9A1hEGaAT9FuEs8Rt1cwSXi4xir1HA6n4vWXfJ5mEigem6pIPD+8hxlsHukcort6gMRzw/E9rjPZRPKIUuhYz6f45RjThlGoOHylyAcruBR/43PRp+nvE+H1myIfouNS/CelPQexiuUVqcyxnkfxVamYze4RZ4VelOdR/LZfvSexizbK/bQSH6LjUfx7mNKIXTRduCLP13Mo/pdvVM+zjKfMBys4FJ+KkR1Zxot2+TGXxKcGaEIPOXWNb65hc5r+Pj6DlPgQnYX49UnJRwMjrjlzjT8r7jycbURFXpS3eKAitO9ymPznvHzvtfM9UCF0P7XsBbYhL/njSmuliTef4LDiyHzAJbzOyxbGTxq0wldONUpgHLJN6MFX+srA5eBudosvrPIF4HsawN25bB8uUfJ8A7cvQcc6YtzBTz4B/6yyRbyV2av9H367uTHZyGBmRfujVF9DozylnEGP/aWhm8QmyjRbOsb8N77CKy0WSCjS38v4muarZ/O770xRZNgkq1HMM94jOAswZfUDAKi1Wq3Oh9YpkCcWs1cnJfQJWVqSSP2iNPHjCxbm997mS/0ie1xTzF+HzFsw3drNLa5hbiy27ERRZNgka1FUSl+bRanQ8mMkniDxBIknSDxB4gkST5B4gsQTJJ4g8QSJJ0g8QeIJEk+QeILEE44RvzfKT5VOvaI88SWBqdQnisDijrzISGV2g+IznlBoxlentkeoCJmLX/QaADjz2rKsiP+qh6+xzGgsM5YZbSza+2tOn6x/+L561abYBxlf7K8A8cYWJYeD7/63UG4sM5rqW/K3idfabX1MbhmvVnPZItvZVeL3+P2vvUbjXf+SHfdHnh2Q8SSPjbbYubuVnY3c7AsKMX6XDETbuyPTcf8TN8K5nEHHQvzhoCAkBS1UlHchA9F2F26RFVM+bhmHreZ5NWlGHOnTuFDEr1rVC+8j6R3uDovpOB4ZiBCzN+O6/E0snmjkrdUkHnoRIz0AqP76mfuGiBuctZqG+t8CXK62EBkjK66429a2lPFckYkBYr1j2Hd+J5/4icRzRYbIkR4A0Ot/3fMG7iTxHGHYzUI8/L8ffDPyCxLPD99UPtqFRRzvbybcnlpI4rmByUgPAOr/Bhk/JvG8ULmFlXi4zMH7t0k8J3xf1OIJVrEmNL+wmcRzQgZGu7KK5TkN75F4PhD0zEZ6AM+5fHuSxHPByVw1w3VO2kVjOYnnggyEN2YYbg7+YyDxfIiPZhluSLeSNSSeA/IPYjTLeKrZSDXx0XRli98s9PVjGnCy9ux2Eu/8MB7pgcaJvOzeKfp6fGlz40+92IY820V1tiMPbVd0xu80tuvJOGTnEQIfj50qWnwGYpg/IDYHq0tIvHNjymT9Ew9gREfD5yTeuTl4TTeIfYfOwnIelntRsvgMRLqzj5ro9fMeEu/k4qMdENV7MhdHdAo+nMvp7Fbg7YC4x3u55vpTxjsvmRjsCO/oGVa10vlbr2DxjhnpAczBh87/RJVyxf/+HaIcEznav3ADiXdatlb1bO+YyG7P4T2nP6JTrni9o0Z6YJr68CES76SUb3OceJ8Jzn/XpWLF77nZuo/Dgs/Ghqsk3jnJQJTj2t7nicoPSbxTImQ4bqQHMBsrK/kSnxqgCXX6HRMG/HTRa6gDw4/zvbKJK/Hrk5KPBkZck7/4DDzl6cDwHjOc/YS9hfjF0xO7rfRaLX/xeoeO9MBMt++ce4YM85ktK47MB1zCD9RVwvhfGXg3HlWNcugG/OI2zI+XuFFPdrXl00J1LmO/IAgvhQiCIAhGg8FwEQbBAo4e/q+L/oJj+U76Jq2ypX517NUv0ul0NVxebG5MNjLomIaOsq96m8pTyhl0vFmUJ6Vv0jRbWmR+Pb7CKy0WSCjSA7XPV8/mGr0zRZFhk6xGMc94j+AswJTVDwCg1mq1Oh9ap0CeWExbnpTQJ2RpSSL1i9LEjy9YmN97my/1i+xxTTF/HTJvwfQ21sqEubHYshNFkWGTrEVR8XAPOMEemtKUxBMkniDxBIknSDxB4gkST5B4gsQTJJ4g8QSJJ0g8QeIJEk+QeMIx4vdG+anSqVeUJ74kkI+5lwmxWNyRFxlJXaLIjCcUmvHVqe0RKkLmGb9Ip9N5t7yprP7IeauvygaaFvHa0Ifvq1dtin2Q8cX+cl6TxpLTaRuzAZeBHepbIOtCSjKnba1jqFerlZTqwom0tBOA69CxsS3rXejLp5e86C0L8bdygNzsZm2VpByAkJ228QzgHj42prkt5eK7n1jGacpbDPW7hwBAwpp7r+W9/Ng9row4DnhEjI1qamvJL5/Wnecz5S0yPkyRj9ItPa4ZOXaUPd9wflNe0QsO3sUUcDHN3omKuE15OoEDHLjYZKS9ZeO7G5Zx2WgSD6xFrN2THbokY0kRieeS219iov2l47sblpJ4Lvn2WvNwER2YjKU8pjyJx1qME7PuYHwPLlOexJd/JWak5zblSfzW4keeFBUgjsuUJ/FrMV5cJ/CZ8ooXfytT3EjPa8orXry+rFOw2D7kMeUVL34dJoq+x4jHlFf6ufrfW1We6CY6Sto47fmmfDVc6Rm/sbKXeO+I61nMW8orXfxa0bt2wJ1f+RskniOu7MYEFnHGcJfyChf/pfBEAJNu5C7lFS6ezUjPY8ore6/+144ul1uxCbVxLF879srO+HUYwsg7dymvdPETWYXi7Vde0eJPHHePYxaMs5RXtPi1GMHuZ9klGe9kk3geENayG+kBjIkoi75K4jngh1+9WC4p7bKuy8W4cl4ar2Tx6xDdiGU87wzd/pm8PIqkYPFV69mcrn1A1w0ua5aQeGfnuyveIxiHfGoxXtpK4p2ctYhjPgPAC1NNE06TeKemIo3pPv0dVO8/WRz1O4l3Znb+7juEfVSPjW1zxt8m8U7MWoxzdUDYlhmNdiaReOelNN0BIz0ABH6G91aReKdly612/RwTeczf8fxeEu+srMUER03d+Mr42/HneROfGqAJPaQA74avHTTSA1CtDi6MdvaJIS3Er09KPhoYcU3+4tPLH+3lsOBe+lbHJ5u4Er94emK3lV6r5S9+LYMHaGrnkXS1foFzd4D5dGcVR+YDLuEH6ipRvkEG3it3Om6kB4DQjyb/U91e4kY90dmWTwvVuYz9giC8FCIIgiAYDQbDRRgECwrlkfLBgmP5q/RNWmVL/azMXu3/8NvNjclGBh3T0FEOm41iKSyuo5tFeVP6Jk2zpUXmt1dXeKXFAglFeqD2+erZ3HLtTFFk2CSrUcwz3iM4CzBl3TmzodZqtTofWqdAnljMZZuU0CdkaUki9YvSxI8vWJjfe5sv9YvscU0xfx0yb8H0NtbKhLmx2LITRZFhk6xFUSlynnKC5sAh8QSJJ0g8QeIJEk+QeILEEySeIPEEiSdIPEHiCRJPkHiCxBMknnCQ+L1Rfqp06hXliS8JTKU+UQQWd+RFRlKXKDLjCYVmfHVqe4SKkLn4Ra8BgMwXHGTI1pHmr9WeGk21f2Z/3M3SqbGb9LWtY5Pzk4BifxJaX/R4+tkbRTdu3PmfAeX1fgLX/5i3M4lXq0HUH1MGEqvNjVtluGU0lhnLjHf+mf+x+EpcTH+2ocXfygFys5u1JY82czivSfWZMl2bNavjS1JV/dU/Xt8ovXiLvfrDQUFIClpIGm1HjxH1HiJd3KszDtuLG1p8mCAIgrCGNNojPsbOkt27VGxpaPGEvZw74TrSzqKqeKSReH4TfrDdK1qNxdYSEs+t+Bi7ywYFlG0j8XxS+L0I8ap4bCTxfLLFFNjO/tLx2Cz1umUkng1iRnog1O/mDhLPI2XfiBLvEif5WE/imZBV2iZITPl46CtJPIdkIFrU5euBPjd2k3j+MGWKGukB1zFSn8Mh8Sw4lK8NExchHulVJJ479Ij0EBdhSNNr35N4DsXHiIzgHi3xfj2JZ8DZU26i706Ox1eSrmJD4pkkfJi32BjDG1+WdPUvEs9EfIzoGJooacd6Ei+egv2IFh8lHhulnFCaxItnsymIwU2KIzxzfyTxXMFipAcaRUo61pN40ZRuZyJe4rGexItmZ1m7QBZxRnucOUniOUIv8gLNPbTDpRzrSbxYqjLZjPTAWBLPEwcLdIPYRIp2O5ZD4rlBj5HubCI1GyJhypN48eJjWIWS8mZbK8uPMVq7VcacedS9kFUXXW0tnG8nUb0p40Un/BBmqeE7EF9JVW8SL1p8DLtgEo71NNSL42pr4WIbZtEu+asut6aM54HNQjA772jzhLBJooqTeHEwHemlHOtpqBdFSQvjT70Yxsvt4JrfgjLe+dlhDOjJMl77oCq9NDUn8aLQI4btBJCSTY5hKT41QBMq6U1/XFO1me1PPBCPrKKGEL8+KfloYMQ1Ulo/DhQ2Hcg24qPdKjMbQvzi6YndVnqtJqX1Q49RrGcjlerarHm9K47MB1zCD9RVojydhN9jI+uRHoh//euBd2e89dRoPDUe9d+H6NvBbvGFVb4AfE/fMVwO1DDx3q0JJPw+HhGsI/bsdczeh+hW2SK+jr36RTqdroY5jJsbk42CeJwpiv1hyptUHwtTGExkozpkd5Om2VIX8xM4FV5psUBCkf5extc0Xz2bkzrOFEWGTbIaxTzjPYKzAFNWPwCAWqvV6nxonQJ5YrFTmpTQJ2RpSSL1i9LEjy9YmN97my/1i+xxTTF/HTJvwXRr1xldw1gcvDpTFBk2yVoUlZRPaBLOA12kIfEEiSdIPEHiCRJPkHiCxBMkniDxBIknSDxB4gkST5B4gsQTJJ5wjPi9UX6qdOoV5YkvCUylPlEEFnfkRUYqsxsUn/GEQjO+OrU9QkXIPOMX6XQ675Y3qY8cjFBhaoCtPnxfvWpT7IOML/anWa+YyL1uKDMay4wP/lR7YeqypZPkNapjqFerSZg4qgqu5OXl5eXlXcmvc3X4X0bubyF13SzE38oBcrObtQUhkjP/Opp3tdoY3tjTU6O5/0/jeXfWC42nxrNk2NmYnZ4S189iqN89BAAS1tx7TRMc2seJN9YJAFxatm7d2q9169atW9W12vTJ/oZx6yQ+vqKZLR3Asb9vFBCd6O/nU8/HH3dFVP75HRLPOUf/ng7Ev9rbljKfT0bq85LWkk7gMOZQVHC6avzxNJu845l/YE6mpPUk8UzZPyJ0s8szJ9b1sLXg36aZJhwm8ZyyJ3zAN67Pnv7sMduLqt6PKB19XsK60m88MyrHZsAt8eUOdhYvHnjssX1NKeP5Q5/h/lzOh/Z6h/brNqfGlEtWWxLPjJX4y/tiFg97ZEuTPVMkm5mGxLPilyzVdHERem10++JVEs8bH2CU2MUCh3+If64i8XxRtgYzRQdJXIjntpF4rkj7ve0I8VFS/lg1LpvE88RKzHAVH0W1auitUb+TeH44tt9tCos4Hhu7XvkXiecp4WPZrA3q/RbeLSDxvHDzMzzHKFR0cMnbJJ4X1t7qMoRRKNXrSM0n8XwgrMT/MbsNPTK07C3HV5ku0rDgUKj6cnNm0bZHqH/1o4zngZV4mp13DB9QvogyngduPFK2vx/DeLuGepx19I3OlPEM+E9ZrydYxhsSVvFPR9eZxItHWImZbJ8wfA0fnyfxTs/e040msY04KPz2P0i807MSk1jvCL2ONTkk3sm5tpHBBVkL+kVW/Z3EOzmrK0ODmAd9DZ+fIfFOjekD9gkP9I0yve7QatNxvFi2RXpf9mIf9sfHVce7U8Y7MSuR4ADvCIoTXqOMd2IutTOdfMwRgY/3wk+9KOOdlo9MYQ7xjp5PI8WB9Sbx4qhc5YhdOwBAsmrTURLvrGy+4jPGQaG7/QHJJN5ZWYmpHo6KvdBl8yES75yc266a4bDgXSY7MOUtxacGaEId9zWTHR9iRHvHRV/gum2/ROLXJyUfDYy4RkbrR/lqh+3aAUDHRCx0VGyL4/jQvsth8p/z8r3XdBxfJ19MapPr5sD457tU7h7smNDm1a44Mh9wCT9Q57d8Cwm/xxJMd6R3BExd+fJL9f50kE2/OkJ1LmO/IAgvhQiCIAhGg8FwEQbBgkLy/QDXS4JDuWDLIcMqWyJbmb3a/+G3mxuTjQya5ExR7A9z+5HqY2EKi5lMzKL4l9tQmWm21MX8N77CKy0WSCjSA7XPV8/md9+ZosiwSVajmGe8R3AWYMq6c6uwWqvV6nxonQJ5YrFvkpTQJ2RpSSL1i9LEjy9YmN97my/1i+xxTTF/HTJvwfQ21sqEsTiGcaYoMmyStSgqySZWI5wKukhD4gkST5D4h2By4TZFpVKpHhUT4d7yx8LC1p7hZ8VGeValUqnsnqduUd8mLWPPADDOat44/qrYKGEqlUpl91W/Fb202n5brdbFZvGMLtx2z8vL+15MgHvLH7/97sqDjSKMIqNgRF5e3lp767Jn1v92VD5VAszL/HLPlTixUTA9Ly/P7gmQ2rx55PDQmBNW62Lr2emQWYJQ5bdI5Mnx5EAGZ9ixSRBMrd4RhCL1WnFRhIQYsZW5hj1CkfuXgnAKB8RFEQa/KLprmn5krS62ZnzFkXCrF27rw1m/DpMuMPipys0PB3Shouuzu2XX566LCWBAMxypDAcebXtAXBTgvy16zC8VUZeqdSX9rNXF1nMFhVW+AHxPi+zn0DVd814b+HMT0eLz4QvAV+z8YCPi2p/7W+QB+yclNc0d0AP5Ht7iKmOaO6AH8Id2fsf+euYre4Mc72dsvKlbtpW6OPQ+gtqJBHqFttswFU7CBKBnr467h9kdYNbP3zOoxp0oM4CerYed62hnkK7ZhrSEPdY+ZetQ38L1KoCrrRi00rsLg2f/W4FZfTq0sL8+szfvagO0qigSVZk7UQAAobC7Mh6dghcFLrNWF1vFm124Fcetcwxmf23fKgsoPsiiPpeu21sfYfamb9sDCHbPAs5c6CcuCgAgG6I6x1RutS627i+uU685OcM7X+Re55925+4Lb3FNRISbP/6IxT/+JrzprT8W075MXJSbfz6Qu/PxzvbezfOcbndeXl6pIMxs++3hfv1ERsl5/XCuvsMgezvm5T25x15WbbdWF5vFC++19Qj5n9jDjfGtPR4ZnyMmwi4AQIJgWuCrHnZGZJTSp3zc2023+9t8J4U+EYSy55t6jckTGeXCoGbqTi8Z7K3MlHYePsO2W60LXZ1TKHSunsQTJJ4g8QSJJ0g8QeIJEk+QeO5QpZN4OfKsStRddfLDTSkNHfEJADUJV1rGQ92qVatWTaFaEenZIQ3A8aGezWfcArC6u7r1bACFY7w6ZwA3Jvl4dv6ExMuOBfE/TZpwCiURTX/4cudsYMWsGcczOgF47eljIyf9jgUnt55a0UIBHSEogwTXRo0aNXpDwExBEEKfEz5seksQtrjkC36v3L0o+qog3MJWISpRGR2imN/4ISsANAP6AeiXjVOBjYABpjOqK/fus+sFNNJew3PxR5+K7S///lCM+Eadanzb8/5/uQNQmRD529c7hs36l+z7Q3G/8f8D8L/H8NhPJcA+l65NArIsPuCT8PnSDynjZUN5PgC3Fviyz5P/PfQxJiUnpBTMmeyLlJktI2/um3P/cwuDu5dvfkz+/aEY8dtaA+h6Gq+te7712m7w+ubFvl7xi4EE45I/txj74HMe8897Dlwn//5Q2j13qk2xympwbdC5ehJPKAm6vZoyniDxBIknSDwhI/4fVfrR8b+kkkUAAAAASUVORK5CYII=) |![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAcMAAAJtCAAAAABvE0QBAAAAIGNIUk0AAHomAACAhAAA+gAAAIDoAAB1MAAA6mAAADqYAAAXcJy6UTwAAAACYktHRAD/h4/MvwAAABBjYU52AAAPAAAABDgAAAmJAAABQRqZkoYAADKWSURBVHja7Z13XBRHG8d/x9GOKgcoKqBBxIIdRdEk9i7WxB5NMzFqNFFjSV5bmokaTTTFaBITOxpFA/ZYEyug2PUEiYpSlKMIHEe5ef+gHcidu3sst7s3v08+hr2bZ2d2vjezu1OeR3YJVCKXjNA6ELusaBVQhlSUIRVlSEUZUoZUlCEVZUhFGVKGVJQhFWVIRRlShlSUIRVlSFUlw1Oh9WR7Sg9OtLPz/53WkNgY5rT+oezvhIHdYz94+xCtIsHrmbUYsvChxX/M3XcNGJ1x0LBtdF6Ap4xWodllbfCbs70A9P2g+ECrBUi+R0ViS3ejVkBAQEBAYydakYJkmFwHQJ0sjQIAli4BgEyXCik8G9zPuHABAOoFBAQEBPjZVD6H7qEqTqN37B4Q4Ga5dX3mAluLJv1NY1hB82cCWT6VPlwLTbxKpVKpHj96dAKA3C8gICAgoJ4VgDSVSqVS3dE8cyaPgICAgAB/heUhTO2hZWsyxkSGXikAUlyKK9vOrso0ihYtACD9jkp1+44q586dfQAcGjdMUalLTt+osV7bJSmqxCdPzgCAb8DgaRZ2L/1d6/siS5POjFIZZhiyH8CRECYncQsOBkAeqVQqlepu7uXLALwDAgKaBDSsfP6cOyqVSnU74/79vz3GWBRC8gsWvsXLmSs+l2bHoe3K7krf+Q83IqHF1DePTd/Xt+zLLNdK98OqVfCfKqFOgL+j0etJU21a63XL1ZIYnujulMTTsx/R13EAwEQysSsh5HgbW78Nel9mIpNUn/IC8D6xJI3BOzydmcVafYbtkKmO9rKKamc5zTCtXn50ED+nNt94ac8xuslFlsNwU35bnhCac8z7G5eo9RaDkKzDJL7ObUaGdT/H/BRLYXjmpsNYCTLElLYZcyyF4XqM4u0p3HzPNAAudCInuloEwox6mjMhfJ3crHPAwe/ivXyLYLhF06ITbyc37zz+l543V3Ewezp6tbgQkvWYxN/IonkZuq3Ap/fYm60Im3FSVAyjL9uN5+/sZl5P89rLuTNYGz1ZBbyRLSaG6/CqUrIMZT9a741ga7T8aSvfhLkiQvh0G38vh+ZniMBZeD+HnUnSGnz5G348Kh6G23OavCRhhljge+8LdhZLNZ0G9JyCN7NEw3Adn080qDRvYVTVO29Rpj2wucEm/T1bHCXk6QuYJJYZi0uwSeXz/OZfIzx4UMEUNo6OPsvv3gNw2oD1Ylk3uR7DPHnNwOztkCQosIl5apUcpwkhhExH/XRRNMNsFxzhNQMBrNVvuAAz/9IxTb2kaEDxKpOl/g9niqIZ7szy68FvDuZvh0QbCDT+MZtR2qsyxJT8+a8MEWJoh53xJb8ZCIEhSZnjCig/fsgg6XCMKPt7FuqmCR/hNcgf8ZuDWectypW94du7sBkzs/Vz0sW0l11rXnqgaXdr/Kaa7RcL4/U6fWtfOwYmH3w3bDfPpRJCOySEkMJdXQD03FdkNFV/jNc7OmeF8BptUwcDKtSdlV+/6d8f/s9okTVu2M9zqQTDkBBybpQcaPpzruEU/0Iep388D7Uf1xzBhKGAvXu5Spdf2rUYPu+3uwaMNsO30IIYEvLfLBfAY2Gyga91XSu92OcFYlRNEcxdYg/5hxn6xUk+tf6jIc1Kdpk0n3OqoAqzrljCd8mExZCQzJUNANs3r1b55RHY3q/4SbQcO1jm8GjJ7qfsy6Xb+wLQ/VpVXxXEH/hu6otyAMpx29SVvrwNqweWxpCQgh0dAfQ5qHu2HjtieuXPFsAjhdXpbzYAbPuuSWBXKFV/oP52nZEU6m3j3ADIuy6/qZ/sIwzkvcYE8lxaUWdX7tYh8MNx9hU/jhjsEO9VKWl+hytt+wQEMN7MeiZU3UB+F0DgoEGdGG76Qs4X3+TbzPrkeSvtC89GRt4A4Odb/ll09t7BvNdXRaTfN7ALPl/y96oAe+8PNDXeDgkhhNyd4QTUXlJhqLioNeY+m/KSLQDAtcP4T7dffG4nGW6Pjo91N5a9bKDrq7Ib3eEN9L3NrODxq/vYVqjfBgU13A7DJqzt+O3O27UBYOubv3VWvT56pRnaIQBk/PJdIuz6DBpYv/STnSNdEqqYDb99SKVS3S+5jKbTXje2WWftVN2gMAcAUB+KPJAOWDHYxaLLRsNvBzOfPHr6z1O9oxBf/uuqAtHgqYQU1VtKCCFkag9CyMwu5mmHhBCSv609ALRbeKGIEEIKm2KxwbS5V//88vXOHgDc5hsc7tF9AkwqbxYFpz5qxqiK7BblEiGrQjvMd/hzKDAxYy8AbJ1yOPjuwNc+BlC8Hz/LpybbIQCQK5GR5wlQZ+Cg3k4bJyoTnpd/2rZv42Ez+sO2VX1Z8M7vWLKgYnt6xGQNQW2h77HTB/oQZwghHwUXH31nY43JJd8sAoAabofFSvn9FWcAtn288TWD5IXhLwHoHvHs2MnTfpCvF3R74irDDI/XWX9lt8+nxd/kZWZmZqTqzFNG7ZEZfgDq5DBLfmGMHAj4qVLq5CAoRDHNYSJDrTycEDJhMCGEkBdnE0I2KYoEUUzdzeXDDjBOff8jV0A5a0tU+aiKyg8e56SJkFR4RbINOjoUuqPTAAC5VgDkEEY8KFnTpixS+yxbsOHbhG8A1AkICGgS4Gd3YeCTFyoNWEtIFYhut/v9xju1kslr8whZ5Lzt7uFGI8X64yzc9fbLpQMCVn4KtEuWaCus1A4x6vHC5DYH6+C+FfA/2f8eeoZ+Idbfpnz4cCDrzm2VSqV6ehd9dzpLtRVaROw8kqLK72oj3eszZU2UdrFWFMYyr5d72VRPzoK8ZFPaoUmDb+I0FmSpqR9h8YsypAypzC/5YlOMu1lbmrEQS03jcotftC+VNMMKgRKoRMlQP1AClYBl5Cbbvz/z02RcvMnWB2JgT0ur6/Pn2VoE9DORob60Whhy6p11MTomOo7DJU1dJeExzCqU2V3D1mRMdTJcugTAosWV8V2KiY5RAQAatmbpKV+z94erO2tbEsMjGk+2XQ9DF2/GZ/nDS/7Iy8zMzMyr9O3A4vVFDYZ/eegJ+1mvv5zhE81twuxu4Ag1EZ1ex8yamD80qCq70VrEp31Q+yAPbj/L0AtDVC+ue42DZeG469dj9waKrBnq9mMAT6dm1g6r1H8ppv1+MgYBH3JYBr0YLg3huEtkzfACnLX8nNnIu0V2bCwSYu8bTtDAxNuZ694FWNX3CVuzM59ibVSPnBELdRCT9qG3bY23w9JACXxqlyMaxrIzyXwB4wgp+BAYlCGmdtgev/J0Zpj5yq42gmI7K4uJaJBBCCEb7dHkpngQJgFJEmVI1H2BuSx2O2+H1T/Ff0X7wPkv0TDcgCC+Tm12hqRwLtCH8V7Ze7WwoPTvlJeBT1mvYT4/a5PGDJf5ChZKlyEh2xRwXMbsma2wKzrmlx3lTwWGZ7HK6/EkGaD8sMY74XwXnJcyQ3LtRaDZMSYpl8KpgmOMX23R/DLzbSBFPyuBUB8AXbdWHrEouJnI436SY/DkbdeDIBgS3e+ewJjn+4mKssaGip+cqwcoe8/7M4FJ/UcHAy1PkcLIUCvAY7aqrHVf++P9zg5AnYEL/3rIzxXOwgTeak8YDAlJn2oF55X5xhNlN8arlVk9GlA8cu7e5+Nd/xkFmfaeDM7fFo8p3F9UH0CP7drC6xund3EAADjKAQB1By2OqH7nXE0RxlvdCWctxsWp59DyB6M+k99ZX//Ks7u5tddiomOuFgCARyO9MQuHNu2Dyo91f8x5gnHL65YN2O1ft59Aqc0BAMd2Qe2DArSXo2Oib+gAoJ6v3l7TZitN3kR6t5H8SS3eqk4g7ZAQUvSLO/Caka0t4ZAZvGnmRf30dptnB39du8/efkdHCLkUAjQ/XtHm3oK6gEOXGRtv6L3bZJ9ePSGw0uhVR5M3z65GV/4qTlBrotQfryMuS4Y2qNp/waNWaXO+Nmqfd/WR3lHaxejLeQBQq137pz/rnBZPf2a+svCssqm8ihPlxOqNAGZPV3c6ZOIu9n6Hln3EW7UJbF1b1JRowD0oqH2Qb2WQun5H2p1lOeJYcCMmOuayFgBGfVOfa6Fie6SHHDQJYo679npzS2GIol/WXym+tQW1D2rvrQdy1UzFxaZcTllwPTrm8eReJhTqUs/0zgdN2fsWMbjhXf486wtwfan2WnR0zNVCAFA6lH+cVLT2XXMV6WLPjC4HTIA4+eep3/NXOqGuEc67Gh0Tfa3COqthu2RmK05Mr4wXD3COqk18E/f3569wgl7nrblTWH4gb27OJVTRvTJf2s8V4pXWijQFf2WzhoClaCWYorQ/0vufgfs4QtyHHjwipGv1marDYZdTg3K42e7DQD6LRvfMMNb5PlndIh05GKbV1t3j0/MebYeM1fGQ84nQXA6Gh3QteHWeSBkyV6dDzsdDn7C3289vV0oZslHIQadjTdaxXU5XdJAyFJA6n2ilfrdTNDuj82luIbyWijJkpaCYb52jgt9Ts7HZh778vsFRhuxkPeP2OLK2yW8sOlSe3yzouwUXnZh6AyE/tmGYOtFHlsJvCEvaDtmrW+wKp7NB0zOYpd6PjvwipAy5yGbWrVG6NU03Mtr6zHtXSvtSrvp72m00ePutus9Lp1XmXmxLGQpT+SuXpUM++N3exruyw33rJfI8Z0b7Uq6ynfdwY5ei8H7+S1OMJduHAXxPe9J2aJKurduYCeuh7/Yw1BZI4/jwoZShsJW74+dzQKO3Gup95teqbLrwdlPbNCeei0AZmq4rP2/OqviJvEVQ+/Yt7QGsnNX7MN/5U4bVoZyw3Xq+ZwpvPgYA65ZB7YNmn/x2BmUoQpEHMTHRMSWzVKrGlKFoQd6PiY6JVoec4T0nypBfkI887HjPxBJiI1SjMVtbWX07/ktNYyOIv9R0nEb8ogwpQyrzi8ZGEH+p6buF+EX7UsqQijKkMl1C3n9YoFbnaHJzNbma3FxNbn6H4e6UV1US2jMNeXTrSdqTtLS0J2lplSblIO81aqgbRSZkhoW3Y2Nj9R3DQObq5OCgcFA4OCgcHIr2xwI2fUYNdhVMiSlDPWVfib0Uey0PAOT+ddw93N3dPdzd3d0q+v9R7dhxFbDtPzJUujG2RctwzgoCAE6t27RpE2h05/qNHWG3APsBH75I0QmKYbaHtm7btm3a+DF5SCbXdoTdgfzbqTLzF5wyLNOOUf4qNkTI5WXb8M4aW/OXXBgSwvvhbgxn1ahkbbYsl63r80QARaftsFh5ntnnOrI1ihz79IUIsYULkm47/Du7fgfWRoPO+iV0iqD8BMJwN4ZxKEXghW7ZQ76msy6CYFi4F8O52LkfnkzmTcijCAXA8JTa4yVOhjY//SDf3C2JpRW5fyJDYgzNP+a9G0O4FmJKk1fPd9gbxDR5wc3Y2NjYdNj2HTnYxezXXX0y+3OpzufRPu6xHeNCbynmeJYNqipsDbykZF+JjY29ng8A1nUeAnYDRg5ygkRkdobnQpwfm7AUOnPMAeaJXdu0adOmmd31sDAVoBg4aoADZVgd+mjFmK2m2Bd9H1M+xWg4mW+bNm3blDnsJ1d2hMUDjqEjXyhPQvKLz5Kryc3V6Dm/hby0lTsoHFzqu3Id5MtLzDbwjZW9g4PCvsJ5dXma3FyNcyMxMCT+d3e+Ul0nK9IYomj7TMdJLu7Y8R+XTJx8fHy8fXx8fBj2xQUPHxTreeNKxb8UuzxNriY3t/h5e/wmMTC83Mb+sbluTCRqR+RTfdAOJVOVCgeFvt/pQk15A81IL/u4Vl39ZzH7MlsHRUFu+fKDlOSyGlbUMtCEi/JyC6r43Skchv0gBoaLPh2yB2JSTuKDBw8SHzx4kMXUws7b28fHx8fHx81IL1z6O9Hklf0Y5EwzMDfDltc2vgZRKutBip7TNpJX3vRybcubpMLTx1PifjFUTaxT6RIZE2XmcZpw9KAIRc5wN7exUip9mbcvfeAre+RFIYi6He5BF4pQ5AxpV1odMmtf+thLl9CQMhB1O9yrC6IIRc6QdqXVInP2pZmeBTebCqcqaDvkoH0FzShCkTOkXWn1yIx9aa5nbkw7SkDU7fBwboO2wqkIypCL2G6zoBIcw/wIejsUO8MTGV6daf2Lm+EuDKWOVXhn+END+44X+Mq4aA/tSvlnGDZz0cXWfVN5yvhMaq1utPqrRUa2Oqyc9AbW7vttnsEE6fkmZLwNg21o9fPMMD9mPmDV6ywAQKsFYFdpTf34/SZlTbvSapLhvvRJUR0AdZIBAEtdXV1dl1ZKITNJ7frS2ue7HVbQ/JkAKm9tiaTVJ/B26CFPAZBSvN7FzsXFxaUyQxobQRjGRsa8Owavgc53muFnGhobQRjGRvrSmRPbB3+b84bkuh7JyQjDUY8XJrc5WIfWkYgZYto0Wj9iEI2NIP5S09gI4peR8dJTofVke2gNiZphTusfLKcexCwjHXT//uV/a7UAyfegaydExlBfS5cAQCZP3pXi/9G7K+s0xcrTaDT63tisOo0Job+hqmT8mUYWPrS8HWb58MMw+7NVBcxSNhgzpiXFyJUhYOJQkUGR7bMfoaOHXp4KhUKhsFcoFAqFnd7tOnvvnmyg+ZgxjUBVqQ6NCeF6B5nIJNWuyy8DfhHM0ubsGGoLIHjVI0KlT8msDNOny6H4TMPC4LfeVoBVj78puXIZ6Uuz49B2ZXelL299qe73eY/xyje+7KySd247C5uDPWgXyqAvPQ4AmMhbO4wKBpoe4WJ5dyicY7lme7rtiNWXiyynL62g6mX4eJIMziu03Iw1XVE3gZtpbiMAcBu8IqqAMjRNhc2B17g/m6S3QJMnnCz/h3qf93UCAOd+X57WEgmIxZh3td4Pj/aqFWFKzKbEzg86HeXgQvZm64Jdw1F46eSpfzIA2Ncuf5FR6I93yBXlqiX0RZRmaoevY7JpJ7juhlD2vWHRSxikK+kJYr8b4cmoipzeOq2j7fAZaeo8/cfE0Gmne+W9vY7tqM1vbzncaKD3A45P1xvZK9JLWFD6oUajBdD0zddMd4Z0Nar8pBoolW5KpVKpdFO6yrXqdLVarU5Xq9VZtmXt317h30XA7TAMDU1+NAy3wiKWJqlKLGd/6z71ugMgH7wn36TiFiw0NEckszcEZ5SQ2+HgiE8+N/kka9/Dz++wspi4sXUUl5tb1o7fzgJ1JrzRjHNhH4z7By/VLmthCpJe3PbU6hwAMjdlSbt0ydeUt9Zu84TbDh9b40Y1nGYBrPawSX8MsnNc87oxuzaAkLXx3G6Ne5Rw3lL1V9rkeLVJnZJ5GP6IdtVxGt1bsD/NPHleAKaYkFv+nsFyAN5jf77JlqNmGhB0h/Ak8zDsjJXVcp6CgVAyb9CL4ZVhWn5JX3e2AYDar6xhM9RzqzUwi79XUbPcD+82skqsWy1nyul53vdsPWZpb7fKDxtpco65506eOpcHwO2lLvWUSqWb2/Om0ckfU3M9NvYHbzILw88W9j5cTeV/0kXlM+MNJZMbf8/j/fdVzwyyNurkqdM5pUcuSqVS6d64ZcvaVZ49672t6LGpHviTORiSpqo/JlTXBSR0fQD7sVOf76xo40TF9Reqr+IKLp2MSVOr1XohEjxatmzRMrBS3Pfo0fHyT+fKwaPMwTC6gyKl+iLc52z7/jLQadorxqNGpTV98tVcHiqwMEOtVqvVqTeuxRVXZcOWgbXKv368usB3G9/+P8zwTDMDY6r1nq77d4wN4PnxfWOJ3kSLfMKrcqI2zOz97G1+uJrwLDO0w8L6qZEDq/mXmPzL2oewGjyth6Eb3qmuOF0j7nDSrl29odE77jGe91VcZmB4sL/Ho+qfCSjc+8NxoF5T/0b+/o0cK3+b3+bmu2shUZkhDukWjOZhMsd6xIgbP/7x6NExAPDyb+Tv39BdWav0WWL5zdpLpYrQDO0wu07u2U48XU32lbj4uLg4ddkHtYqHIZ0XabeOoQyrjeGW8Y3u8HyLSI+Pi4+Le6DWC4zX+5B0FxfXfF+6Gbzf5d3atwcAFJTNDZCvJbw+vMbbYUr9IlVj6danOVTjrgvDioIpQpEz3IzxtNbFzVAVJR9Fa13cDLegb21a66JmSDZjHK10cTM8d9dxCK10cTPcjOGOtNJFzbAgjHalYmd4KK1OT1rn4ma4GWOsBVoREmVY7bERsvbSF/yaZVj9sRHC85rQQGvVL1NiI8Q8YZnZOv6nLChDfT0TGyGrcoqF7GMjjKU1XpMMS2Ij3AJQ6uurkhqxDl840I/WeE0yrKD5MwGSX2lR6GpafYIQi9gIrp6VbmY0NoIwjGlsBPGXmsZGEL9obARJM6SxEUQiGhtB/KU28kxzanlMkr4PWiqBivrVF78Y+tWnEidDfdHYCOJnyG9sBFOU8XuCgW+KNOW+2JxWSnjiUgCxEUzRve/WZzNLuWKWpbdDOzshFj7mmx1FCAw18FxmVe6B1H7NqtnJX1sJ8RpqjqEApTu44jjQc3ZfJjfpb7zmrkj9Rei+ZKufYXYckBCr9BVkubVbvrkB+ehZDKcwZXPqvLXxyQ6Jrm017DKDb7/6JijtCy/AedZ9NjaRCnR8IpQLEL+/NlP1qF0KvD9425Wd1dmB6U0PCbNXMU2ivM9vTGmw6e4slggR8q/3rc7XKUNhaDv+N57D80nzM80fvniaMhSCbl+2Hs7J0Oefzhm9IihDASgMfZTcLJVHBuUN+40yNLvIdnDeDu4Q/kbRW92+uqSTEkMRPpdebWWb6srZmnyyFECdPn37eNZwucmXUXrhOW2ZDE14tZYow/99MTTcFPv4g4eO5QCydv36dip+MtJlqdVqdXq2XlQC/XglDb4Kqo5yr5nO2mTMVmkyJI3jt4028RzaM4cOXgbg0t1RrVarM57TtVpN/9R0n7lnXy6c6p9XNpOSz8Sm35fSZBjTXpHqVA3nSTp86HBa6YGDUql0c9KL06U3Blm0dht8vh9sYnap7R6+GsbP/Kv4GH60YmRYNZ2q6OJJKzelUql0szea7uCUBAxfXd+UrAr7HG96wZmnKjGDL2jTHD/7YleNZ5ozVw7n7wtNOMN8OF7nq3SiY3gGzrlmyPZyR6Aj5/i1ZA+wnbeyie79cDuGKMyQbavTP7icD5qby806biJm8OjiTGTtsLAuIsyU9cMRQMMDnLriVujMY9RasbXDf5Nq9TFT1vX+/Mvnv/6hN9k3kylXau+w5a9gYmO4HcNtzZZ56I2Z1pEtp7B1M7H+D6vt9fksl7j60gJPHDJrAW4NAZyXatiYXLDFV7yWSWQMD8PD3GHtj7cDfLcwj5z3xBdD+A0JLbK+dDteMfdKvG5RG73vj+v0D8PkRePu+//B7/p4cTHM3w3zeyG2eu32505RLw+/wyj1Z4cUu1z5LZC4xtoiQ+s+kEMASlm0XmczKVBRPplkoDEcCSUbX+O7NKK6H47HdLPfkkt0jemusMm8F0VU7VBT52nNBE9jpCNbs0pnGzWGJ6/67eF9m4OoGO4e4ZsgyBs4KcgzUI+yGqgyUe23CMNIYT6DyWxtzZi7mJ5LcyIF8FQqQImJYURuoyARFZcyrEJhGEU3k4ubYeZ+jBZPaSnDqrQ3v1kLCkzcDGlXKnqGaYfpUyl7htUeG8EkhRe2bkp5sWRY/bERTFIYbYYGZMyPcIfvofN5v8yP8DNjbTvv1mBBixbo4qlH9yplSmyE3/fXaFE7UIRsGT4/NkKvGo0oav0epcWWYQVVGRvhQ1p9ghCNjSD+UtPYCOIvNY2NIH7R2AiSZkhjI4hENDaC+EvNYk0UlUAlVd+6lCGVmGQJoc5JrppI0W+p1BnqUhKT0tLS0tLUaWlpWiBspHQZSuyZpig5MTHxQWJi4sNC/Y+tdIFXpbuOQyoMi+7dUanuqO6X+VmzqlvPw91d6e7u7u7uXuiTfaAfZShYPY1R3VHdiS/xfyav5+Pt7e3j7e1V4S4xa2XPvylDoepR8EMAgJ1/QOOAAD+vqrcn3vcruthW5FcqWYZk2F7P9gEBjQO8je8tHbd13GbKUJjaOdLmIpOVwxeD5Ak+EmUo8nf8tGmYz2jxd7seRd/RdihITdzY/CKzfbYHBjg/cBX1tUq0HR7cKPuV4Vbpfs2frpNoOxQ1w6fvYkYnhmlls/FdvpgvVqIMP7nf8HPGicd6PQwT88VKk+Hp77GeeTQ8u+lYIc3JUhE/0+S1vfUGm5gxat+cw71pOxSUPr/l9Q2b9Mq3sIK2Q0EptkPhLnahu/5rpLvcyowlLnisd1CoydXkaor/Mbj6t+04RicWiu8stipohxFsbUZhghlLnN6INfUx1e3rS1haNrfWTS+WNtEdrBO8zVbi1/+Q6Y3pyh0UCgeFwkHhoFDYGZrbbDNGyu1QZY8N7K26Yo7ZSrwHVqfN7nNPSNL1ONn7EPuZ+chQlweMtjzkppau40hLU/tuqoa76JPA1Dlf81MZImX482SHay9wQB94a6WxHXfZJSs5EtUVPnbZ3dPUApORfwZG21OG5Ups/vTbGVwMf5nkE191JOHMiD23H2SWHdqXrONQurs7TT9l89t4E0u8fYz1Ob4clYmSIQnd1+lfTv6E8xqkbhn77McZf+08nA8ALj7exUs5XPQ6au3EMHw5z6Q1VUmB6YsX8VcfItRW2HINgPUZ2lZ2cq/eMMAGQLOFh24YcJRcNBuYbIo/f91AtMvnrTrEyPCxB5ZwtX2iwFH947Rf+1kDCFx0zajZahlCs7mX+FfYXuOvPsTIcBxacI+eNBX9CSGEZN04/OuSSd2sAbRccuO5ZrvtEZzCNc//nLGMx/oQ4f1w3yCrs8GcreMbk4/UiYkPSj21tH7l1SZM7M6EqhsdaMwpS13vY11O8hgOQHwMswITZ5kydv3KruL/1/L29vb2eTGAqd3t/gkeEZ245Pj9+w6X/XmsEfExnPKT31UHE+wTFjh6+3h712cd2DVlULT99iHsM7zTWvPDFF6rRGw3w5PAMTNlnT0QsjWsQzcVhqBXEa/lEhvD3MaYZLbMCyYBLX/VsDP6Gi73+C2W2BjOQ9108+Wu+9oRqL0omYXJVVsug/NSZhgjxx6zFkC9zAewfeMK0/TatgjV8VwmcT3TFATHjjT34rSC3avOAz0/7P+cZSxFGo0mV7NplfK6F88lEhfDpR8rbwjA6dHZVbt0aDJjgkNW6QSVOl0v4hPR5mo0uaWLWfnfgSwqhrdba/kPRMdI99asz4JdYZHxVHYOCsWIr3kvjJgY6rr+22+/ULZkP93w3V1A4V4yQeVmXYmdg+GYiJbM8MepjtcbCKc4RfGOSoUQCiIihvcDs9dQD3JVSDyxEcjk7C5TBFMakTAUWGyErQdsf6FeraoSw9gIWi0Au0pb/ZbF1mRJD6Z/8THlVaUMjzDIwwkhEwYTQghZBACLKqUYULMlbZ1PqKoSw9gI82cCqLzj9p0+NYlQPsyGtjiW7fAhzhBCPgo2zD9vUR73H484jQVZasP3w3yHP4cCEzP2GsRP/eoLw9jwk55t0FFAdzSEdlVCF42NIGmGNDaCSGTMr37whwsmGd2uR/3qC8KY+tUXv4yMXp0KrSfbQ2tI1AxzWv9A60cMMtJB9+9Pq0fsDPWl1QIk30OCfs11eZpS6Qwk8ekq7OtmyLA4lqwJwwwCVGrUhaiL6Uz88HX8soeQL8T4c6ksfGh5O8zykQzDpzFRF6LulR7ZKOwVCoXC3sDvmZzORc8vOoq+HdrZCfcSWCpv05kLNwkAWbMOwe29DLMrU/KXa48eHfJZS5G3Q8DEIVvh6PMFAHw7BHcIYn4195b8oZONXewvzCsywjA7Dm1Xdlf6Soxhq6sTX+nAegTx1qIdsH5rQX1BXpLhGavjAICJZceZyJTApHcc5GmcDC8OAOxmpgrwkliMtUmjHX4zu8dRjqb/fvwPYKcokx2TJVrNXuV91sDiGL54mvsiVXL4kxjWRlY9xg6rRRlWo1LqkvsmxCohT3I1Go2meFwgj4FBwdHzgO2AsYN4XBFuaQzXv9M+qmZzjN++7TrgNHRsL77WdFkawwEHzLBK9eq2bf8B7q++pHcDtVIoiocWFDamjuRZGMMsz/wbzcyQLzm3LczQinkrwz5ohzMKq2IJ8YD1dCA/oKk58pWFhKw8Hpag90lRyUA7oMvJMWSWzejkFsYwHMPMNQdh3buqwAxEqzHik51Zv2dZDLX7MVRYJZLZ27uZeg7L2kl07GndYOldlWUxDMdQCV6wRTEs2iu0rpQyZKtzqa7dKENxKxyDbClDUYvskWRXalEMr8Xb9aMMxa1w9HGiDMUtiXallsTwv0tWoZSh2JvhS56UodgZDpXmhVkOw8f/UIZiV4SubUPKUNySbFdqOQyzD2MYZShuHdL6taAMxS0zrsKgDKtH+ZGS7UothuHJzNqdpHptlsIwHEPkUr02C2Go2yPdrtRSGEYlOQvaLQJvDIXlV98khWOAdDwKsGAoML/6poiES7grZepXHxD1npmbzW0fS8q3TgUZXqufHzMfsOp1FkCJf5rKKSafEstVZqKndBGCoV/9Yj9RlfTgpniuc7x0ETLdMzN/JkDyK4Wy/mqOaC7TtbWEGTKMUUJjI4gyNgI6Bq+BznfaPIMJaGwEYRhTv/riF/WrL2mGmEYjRopCxmIjPN+YxkYQgrGRZ5pTy2OS9H1fUglU1K+++EX96kuaob6k61ffchhK0a9+BekqxkYgBZpyNXhb2BFQLdOvfgm3qyfPqEs4GQ2RELKtgQTaoYT86herKPbkyX/SK1aFfl1YO5S5C7bZdLbNhqHiZygpFV48efKfLABOL77sXe7a2eC6t5ljzg+bttxesNdjeX71704/mQ3A5aWu3doy/AUXfLIcbcMaC/aSLM2vPpkEuA1eGVPIymi/B5y2CPWKLM4ne1Hdx1tGs1+R+XDcSby52lFsfak0GZ7s5pbC5VWh6LNPSfMwU7ZOFS47q3dkoyyVo6lv3Rb3TLMbQzi97ckXdx13o8PqtzlXeM6YiKq/sFG6GOoXBn5DGT4r3W4M52jaPXbiwXeOftWQm3XqoCj7JR7lx9r0NLVarVanFRSkpBgyasfo1JbWl57v5PSY81uC7puPC+UjP2rLwVTV/657REgVX5BctTrTkJUnM//xzB9/JPFcOgejTDE/3xtAr0M6tnZn3OF3m69rsiw/wiC7MMIU++DDF8fK/+7bdnMBK7PwHmkdzgbwd1UW1Q4vw/6pqef4b4Yj4LMyi7nFGhkGZfN3URbGcCGGVMNZ0r6oDdSa94hZ6qLZwOQCHi/Kwp5pWl77Y0J1nCdv0woV5C/4+zf2929o3Ddx3uthWDqXz5lXy2KoamKd6lY9p9L9tfxM8V9WDfz9/RvVVirdqnpRSx96yuY3fnd7WBbDr+b3OVR9Z3t4Jy4uLi6uPNKPa+nIS/lkyJOBN1129+T3qizrHd/Ep9JKql+/GwCSHBcXF3c3TZ2JzMyEKlId4Dugt0W1w/sNZEn8rVsvzFAXKy2ndBWHFgja4833ZVlUO9yNl3jcemDt4VH5I11enhv/68gsimH1dqUMZOXgUBO5WBDC5NPS9KxgUX71SbCPFK/Lkhhyn3aiDAUi9XHKUOyKKGwp3KVplCEj1fhTKWVY3Xp6WKJdqQUx3K9tLFGH3pbDcDeGS3TrncUw1OyT6u3QchgezvFpTxmKW9LtSi0nNsJfku1KLYbhiYzanSlDcWsXhtHYCOJW0R6pvuBbDsPTqbW6WyRDCcVG2I3BwvZPwhNDKcVG2C3dp1LTYiMUELFcZUxnx8cKyTI0JTbC0P3iuc6B0kVopC8tiY2QDABY6urqKua1KLbvShehSbERdhSK5jLthOshqBpkcEcUjY0gllLT2AjiLzWNjSB+0dgIkmZIYyOIRDQ2gvhLTWMjiF80NoL4RWMjSJqhvmhsBPEzlHxsBK4qiaGQW7oDP1ej0eTpBVmQjXqb9zJYcmwEDlq3rKj8QKfRaIqea7JttEDaoeRiI3CTblHysx/al4ZRcCj5115v9dWJdW+1DBQGQyoAwMVkp8PlT/IyhUKhUNgbfUgYeffv4VE8d19GGGbHAQmx5bERqBCBviHsLORbg1Sv7+L3WdDI+2F027aY2XYhJVeuSAxia+L5p234cn5LZXF+9U3RQ29Zcm3WVuvetTrSw0ztkKqy9qEje4SY9LpudCJlKBBFIpSDlezHNo9f0VKGgpDmb/a3QwBQ7HI7P5MyFISOaXy4ubH02yz7cRNlKARFIJTjS8KAhXj3MmVofpFIbl0pACzsrxmeThmaXbEPHTjvnbLa3PDuBB1laG5Fojf3lcbKXXaRX/JUMDpeylgRnN4sStXupzcX+rcy0JDsFQoF9711dJyGqZLr4lFdU07w7jqjX8tLJz4U5X/4MgoPRtshU+1DB5MQYnXWEQPfFOXlAUXZ2c98MTKMMqxOmfBUWiy7bYa/I3mV1gMU/9mE0YkpQ4bKO2zS7fA5kilM2B9Jn0sZ6kRu/TYCLRplyFARGCTUVX2UITMRk2+HlKG5dfW+oqdQy0YZMlMkegrWKwNlyEwRfD6VUoY1odTzGEgZilv7Sbv6lKG4JeCnUsqQmbSHBHw7pAwZ6VS2VzvKUNyKwCABVxRlyEBCHqShDJnpRoJdL8pQ3IpED0fKUNwS8iANZchIT84KeJCGMmSkA7rWgt5ISxk+X8J+KrWY2AimKP+gsG+HFhIbwST9m1W7g6ALaGRd28pJb2Dtvt8M+xG+nWkRDDdioLDvOKbERpgpotgIJknYt0MjDEtiI9wCUOrrq5JqN7AMhE0F7nzQlNgIGyykGQpdhnt6D3kKgBQvAICdi4uLq2elBZbaxSZ4ChCnsSBLTWMjiL/UNDaC+EVjI0iaIY2NIBLR2AjiL7VMNHEoqQyJzltQhlSUIZXpovvxTVdOYmJiUnauRlPyn35sBHT/oTbf+dNnGs4it+4lPkxMTEzMMJrMaxPf6xopQ646Pie69E8nH+96LgqHEudADvqxER5PuSGb85kNryWhDLnp2tz9sA/w9q7v7e3tbWwUNHfmzwje5sdvj0DFXg/etIL11GRmiXfWgvNWPktD2yF7ZX69Kg8jvgxgmv7euNN4fY0TbYeCkfZbd+DFM2xMChZaoXEMbyWiDNmpaLsf0GSPjqXZCW/YrNTxVCbal7J7lHkjGl5L3mT/Vp329h4MWK23mMVKoaiu8RXKkNWNp/VVpzkzOe2BIms/rLyYwk7hUPw2YjDsV+eplGF161I7u3jO/jGuvhHD1mTMViap6FgbG23EYO4uTlpG6x8V5ZUNzmnyDJkw819K2yELFdZPjRDgemE6b8FCh1M9+wqwWJQhC23EWBsBFov2pcyV4aWNEaKbGtoOmetPbWBbIZaLMmSujZggSHfQtC9lrLuNZA8E6TyRtkPG2oyewvR/SRkyFdmECcIsGe1LmepsZ8dkJ0GWjLZDptqIEcJESNshU2m9Mv4WaHQE2g4ZKjLDu5tAi0YZMtRGjJcLtGi0L2Wmx/UKrzen7VDUCisMEipCypChNgr15RC0L2Wom82tH9YWauFoO2SkTegvWISUISPpBDvOBtC+lJmO9ayVZE/boai1ESOFi5C2QybKqZPzbxfhFo+2Qwbak+PXWcDFowwZSKiLMEpE+9Ln66GvLt5PwOWj7fD52qp7UcgIaWyE54v8IeSXQ9DYCAwUe93uVUEX0Jgf4Q7fQ+fzvmE/wgcfWATDyL9e3SHoArKIjQA7u4op1lhKbARhd6WsYiMsWlwxRSchbiDhQU2kExsBlZohFlhIMxS6WMRGcKnMkMZGEIYxjY0g/lLT2AjiF42NIGmGNDaCSERjI4i/1HTeQvyi8xaUIRVlSEUZUpnG0IQ54sUymUzWlIPhqdB6sj0AyMK6il53uNm+LpPJZP1YZry0g3PtobcB5E11dxqRws22m0wmk01ml/FPrVxcQg4Yy9cEhibNEQcmJSX9y8Eup/UPAIBlq9eed+ybx8kW/ZKSkraxzPjk1HNHCvrkAB9G7Dz5aDg3W0xKSkpaxi5j769ionsMuW4sX+4uiIOnElJUbykn20WtTXBfHU6Izms5IRl227jYkolDOOacipMkw2YnITdxlost6TqDY85uvxjJl3s7zI/pVT5HzFp36vmNu8+9D0hI7gW4duSY+YnaTd5L42CXCSViCnoBTX3PcrEFtni0mJ/LOt+i7TkhRvLlPnBQYY6YrTr+3iRpyUvXnLlmnow6AOokczLuN/yF+I/7n2W9NVv3QZcWSLatxSVn3QddWgBjG9S7Mvf2bnamV0PynMKbxxrO10x+hPsDrTo22PGWWTIfDbRs1egEay8XU6/9yznPYtt3gJZ1e8Y3YmXaJDbzz4knjSTg3pdWmCPmoloBcZxtvWBi5n4erDOfFnncG/DKz+CQc7FtcRcEljnb+gctbf2dkXy5M7QNOgrojoZwPkF2fF3Oti94HQWyznPPPDGNZeZkWvixFwAE2RwFbt8P4WILAIgFh8vWaY3ly/35cLvd7zfeqZXMyXbWiYTTvTxS2Rs+vXQJKy/dI1/V2ntlyAsaLrZPZ59N+Ltd4zx2Gb/neiIpKSmXkMm+x6JDQjjZxn0anbDX72V2Gc87mXBlnuywkXxNiTOzxtc2+Bw301F1beuPiuNgeBwAMJHoFtSx63mbk21uH0+bBpPY/vaKf/EbCNFMcXMYlsTJ9v7LSjv/jzLZZfxmA1vPnoeN5UvnnsQvOl5KGVJRhlSUIRVlSBlSUYZUlCGVBTOU7aEMBa7XZVyWzIhVEo1D2m8Dnt3zKlVJtC+18/Ly8nKD7Kf+Cr8/AVztoXB/JxvAb4F2dacBeDLMofFfQPo4T0XjDZShkLVgxOVxo28ip69b1M6/pwE/TX3n6l/+AJaMvDJgnBoLbhy4+ZOH2K+SSFET5Y6Ojo5fEEwmhHR8j6xzyyZkn1UyqfdJyWzQ/wjJxgES+oYErlai98PuPwFQAiEAQmJxs7Uj0EV3W/aodBFNK8DRJRXvjbjYZ6iQfSIykUQZOvpX+bGi7C8bADId+t/bf6Tn1BXivliJ3w/PATjXDM0u5wCnrZo4NzxaKYHnxM3frqPtUIjSJgOw9sDO9i9uufArxi2auPjx+6/VweLJtfs/Pf1+WbqFQYHayGaUoRB1sC6AJrewZPuUutuaw+HQjA4OI1YCE/NWzfZ4pTyd7fz/FC9tF/nFSns9jSx8qJQvr0R0zJsypDK/6NpE2g6pKEMqypCKMqQMqShDKsqQijKkDKkoQyrKkIoypAypKEMqypCKMqQMqYSj/wNAP3OI1dlP3wAAAABJRU5ErkJggg==) |" - ], - "metadata": { - "id": "5RolXP_mTxwp" - } } ], "metadata": {