Machine_Learning_Assignment_NLP_Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sentimental Analysis Part 1¶\n",
    "Predicting Review Score of Amazon Foods through Summary, Text and other features \n",
    "\n",
    "<b>Name</b>: Bharathvaj Devarajan<br>\n",
    "<b>Student No</b>: 16212388<br>\n",
    "<b>Module</b>: CA684 - Machine Learning<br>\n",
    "<b>Aim</b> The first part of this project is to combine the Numerical and Text features to create a Pipeline for predicting the review score,\n",
    "I have also done Hyper Parameter tuning using GridSearch CV to find the best hyper parameters for modelling "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Pandas, numpy and other helpful libraries\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import string\n",
    "import sqlite3\n",
    "import numpy as np\n",
    "# Scikit learn libraries for Preprocessing,modelling,evaluating, Count Vectorization and X_train X_test split\n",
    "from sklearn.cross_validation import train_test_split\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "from sklearn.metrics import roc_curve, auc,mean_squared_error\n",
    "from sklearn.preprocessing import FunctionTransformer\n",
    "from sklearn.base import BaseEstimator, TransformerMixin\n",
    "from sklearn.pipeline import FeatureUnion,Pipeline\n",
    "from sklearn.preprocessing import Imputer,MaxAbsScaler\n",
    "from sklearn.linear_model import SGDRegressor, ElasticNet,Ridge,Lasso,LinearRegression\n",
    "from sklearn.model_selection import GridSearchCV \n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "# Natural Language Text Processing Libraries for NLP functions\n",
    "from nltk.stem.porter import PorterStemmer\n",
    "from nltk.stem.wordnet import WordNetLemmatizer \n",
    "from nltk import word_tokenize\n",
    "from nltk.corpus import stopwords\n",
    "import nltk\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first step is to establish a database connection and read the data to a panda dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "con = sqlite3.connect('/home/bharath/Downloads/db/amazon_db.sqlite')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As the Score is scaled between 1 to 5 , the scores with 3 represent neutral sentiments,so those amazon_df are omitted from this analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "amazon_df = pd.read_sql_query(\"\"\"\n",
    "SELECT *\n",
    "FROM Reviews\n",
    "WHERE Score != 3\n",
    "\"\"\", con)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "amazon_df = amazon_df.dropna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>ProductId</th>\n",
       "      <th>UserId</th>\n",
       "      <th>ProfileName</th>\n",
       "      <th>HelpfulnessNumerator</th>\n",
       "      <th>HelpfulnessDenominator</th>\n",
       "      <th>Score</th>\n",
       "      <th>Time</th>\n",
       "      <th>Summary</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>B001E4KFG0</td>\n",
       "      <td>A3SGXH7AUHU8GW</td>\n",
       "      <td>delmartian</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1303862400</td>\n",
       "      <td>Good Quality Dog Food</td>\n",
       "      <td>I have bought several of the Vitality canned d...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id   ProductId          UserId ProfileName  HelpfulnessNumerator  \\\n",
       "0   1  B001E4KFG0  A3SGXH7AUHU8GW  delmartian                     1   \n",
       "\n",
       "   HelpfulnessDenominator  Score        Time                Summary  \\\n",
       "0                       1      5  1303862400  Good Quality Dog Food   \n",
       "\n",
       "                                                Text  \n",
       "0  I have bought several of the Vitality canned d...  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "amazon_df.head(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Helper functions to extract Text and Numeric features using Function Transformer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "get_text = FunctionTransformer(lambda x: x[['Summary','Text']], validate=False)\n",
    "get_numeric = FunctionTransformer(lambda x: x[['HelpfulnessNumerator','HelpfulnessDenominator']],\n",
    "                                       validate=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the class for doing transformation based on the extracted features of Function Transformer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class ItemSelector(BaseEstimator, TransformerMixin):\n",
    "    \n",
    "    def __init__(self, key):\n",
    "        self.key = key\n",
    "\n",
    "    def fit(self, x, y=None):\n",
    "        return self\n",
    "\n",
    "    def transform(self, data_dict):\n",
    "        return data_dict[self.key]\n",
    "\n",
    "class NoFitMixin:\n",
    "    def fit(self, X, y=None):\n",
    "        return self\n",
    "\n",
    "class SelectTransform(TransformerMixin, NoFitMixin, BaseEstimator):\n",
    "    def __init__(self, func, copy=True):\n",
    "        self.func = func\n",
    "        self.copy = copy\n",
    "\n",
    "    def transform(self, X):\n",
    "        X_ = X if not self.copy else X.copy()\n",
    "        return self.func(X_)\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Data Preprocessing</b> "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Drop duplicate score-text values\n",
    "amazon_df = amazon_df.drop_duplicates(subset = ['Summary', 'Text', 'Score'])\n",
    "\n",
    "#Create text - remove line breaks\n",
    "amazon_df['Summary'] = amazon_df['Summary'].str.replace('<br />', ' ')\n",
    "amazon_df['Text'] = amazon_df['Text'].str.replace('<br />', ' ')\n",
    "\n",
    "#Replace nan with \"\"\n",
    "amazon_df['Summary'] = amazon_df['Summary'].fillna(value = \"\")\n",
    "amazon_df['Text'] = amazon_df['Text'].fillna(value = \"\")\n",
    "\n",
    "#Extract target variable\n",
    "score = amazon_df['Score']\n",
    "\n",
    "#Remove score from amazon_df\n",
    "amazon_df = amazon_df.drop('Score', axis = 'columns')\n",
    "\n",
    "#Count the number of words\n",
    "amazon_df['summary_count'] = amazon_df['Summary'].str.split().apply(len)\n",
    "amazon_df['text_count'] = amazon_df['Text'].str.split().apply(len)\n",
    "amazon_df = amazon_df.assign(all_words_count = amazon_df['summary_count'] + amazon_df['text_count'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Creating the X_train X_test split and Pipeline</b><br> \n",
    "The pipeline module of scikit-learn allows you to chain transformers and estimators together in such a way that you can use them as a single unit. This comes in very handy when you need to jump through a few hoops of data extraction, transformation, normalization, and finally train your model (or use it to generate predictions)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this analysis , the following sequence of steps will be carried out by our Pipeline<br>\n",
    "1. <b>Feature Union</b>, In which the text and Numerical features will be combined<br> \n",
    "2. <b>Scaling</b>, Using Max absoulte Scaler, to ensure scaled input<br>\n",
    "3. <b>Modelling</b>, SGDRegressor is just a place holder, we will be using a variety of models using Grid Search<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Stratified split will ensure equal proportions of 0's and 1's in both datasets\n",
    "X_train, X_test, y_train, y_test = train_test_split(amazon_df, score, test_size= 0.2,\n",
    "                                                        stratify = score, random_state = 44)\n",
    "\n",
    "# Pipeline\n",
    "pipeline = Pipeline([\n",
    "    ('union', FeatureUnion([\n",
    "        \n",
    "        ('summary', Pipeline([\n",
    "            ('selector', ItemSelector(key = 'Summary')),\n",
    "            ('count_vectorizer', CountVectorizer(min_df = 0.0005, max_df = 1.0)),\n",
    "        ])),\n",
    "        \n",
    "        ('text', Pipeline([\n",
    "            ('selector', ItemSelector(key = 'Text')),\n",
    "            ('count_vectorizer', CountVectorizer(min_df = 0.001, max_df = 1.0)),\n",
    "        ])),\n",
    "        \n",
    "        ('numerical', Pipeline([\n",
    "            ('select', SelectTransform(lambda X: X.select_dtypes(exclude=['object']))),\n",
    "        ])),\n",
    "    ])),\n",
    "    \n",
    "    ('scaler',  MaxAbsScaler()),\n",
    "    \n",
    "        ('model', SGDRegressor())\n",
    "])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initializing the Parameter grid for GridSearch CV,\n",
    "We pass a series of values based on our intituiton to GridSearchCV, which will train the model by chronologically passing these arguments one after the other, and ultimately returns the model and its parameters with the best fit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I have commented out most of the model parameters due to compute power restriction, these models take hours to train and my computer usually hangs after a while"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "param_grid = [\n",
    "    {'model' : [SGDRegressor(loss = 'squared_loss', n_iter = 3, random_state = 44,\n",
    "                             verbose = 3)],\n",
    "        #'model__l1_ratio' : [0, 0.5, 1],\n",
    "        #'model__alpha' : [0.0001, 0.1, 1, 10],\n",
    "        #'union__summary__count_vectorizer__ngram_range': [(1, 1), (1, 2)],\n",
    "        #'union__text__count_vectorizer__ngram_range': [(1, 1), (1, 2)]\n",
    "    },\n",
    "    {'model' : [RandomForestRegressor(n_estimators = 2, random_state = 44, verbose = 3, n_jobs = -1)],\n",
    "        #'model__max_depth' : [3],\n",
    "        #'scaler' : [None],\n",
    "        #'union__summary__count_vectorizer__ngram_range': [(1, 1), (1, 2)],\n",
    "        #'union__text__count_vectorizer__ngram_range': [(1, 1), (1, 2)]\n",
    "    },\n",
    "    {'model' : [LinearRegression(n_jobs = -1)],\n",
    "        #'union__summary__count_vectorizer__ngram_range': [(1, 1), (1, 2)],\n",
    "        #'union__text__count_vectorizer__ngram_range': [(1, 1), (1, 2)]\n",
    "    },\n",
    "    {'model' : [Ridge(random_state = 44)],\n",
    "        #'model__alpha' : [0.1, 1, 10],\n",
    "        #'union__summary__count_vectorizer__ngram_range': [(1, 1), (1, 2)],\n",
    "        #'union__text__count_vectorizer__ngram_range': [(1, 1), (1, 2)]\n",
    "    },\n",
    "    {'model' : [Lasso(random_state = 44)],\n",
    "        #'model__alpha' : [0.1, 1, 10],\n",
    "        #'union__summary__count_vectorizer__ngram_range': [(1, 1), (1, 2)],\n",
    "        #'union__text__count_vectorizer__ngram_range': [(1, 1), (1, 2)]\n",
    "    },\n",
    "    {'model' : [ElasticNet(random_state = 44)],\n",
    "        #'model__l1_ratio' : [0.1, 0.5, 0.7, 0.9],          \n",
    "        #'model__alpha' : [0.1, 1, 10],\n",
    "        #'union__summary__count_vectorizer__ngram_range': [(1, 1), (1, 2)],\n",
    "        #'union__text__count_vectorizer__ngram_range': [(1, 1), (1, 2)]\n",
    "    }\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Creating the Grid using GridSearchCV , we pass the pipeline , parameter grid, number of cross validations and method for scoring "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "grid = GridSearchCV(pipeline, param_grid,cv=5,scoring = 'neg_mean_squared_error')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-- Epoch 1\n",
      "Norm: 7.58, NNZs: 4597, Bias: 0.056325, T: 233703, Avg. loss: 0.699441\n",
      "Total training time: 0.20 seconds.\n",
      "-- Epoch 2\n",
      "Norm: 9.57, NNZs: 4597, Bias: 0.063065, T: 467406, Avg. loss: 0.632610\n",
      "Total training time: 0.37 seconds.\n",
      "-- Epoch 3\n",
      "Norm: 10.88, NNZs: 4597, Bias: 0.068787, T: 701109, Avg. loss: 0.595685\n",
      "Total training time: 0.54 seconds.\n",
      "-- Epoch 1\n",
      "Norm: 7.70, NNZs: 4610, Bias: 0.056438, T: 233703, Avg. loss: 0.692351\n",
      "Total training time: 0.18 seconds.\n",
      "-- Epoch 2\n",
      "Norm: 9.62, NNZs: 4610, Bias: 0.062845, T: 467406, Avg. loss: 0.625480\n",
      "Total training time: 0.36 seconds.\n",
      "-- Epoch 3\n",
      "Norm: 10.86, NNZs: 4610, Bias: 0.068666, T: 701109, Avg. loss: 0.589448\n",
      "Total training time: 0.48 seconds.\n",
      "-- Epoch 1\n",
      "Norm: 7.67, NNZs: 4579, Bias: 0.056347, T: 233703, Avg. loss: 0.696580\n",
      "Total training time: 0.11 seconds.\n",
      "-- Epoch 2\n",
      "Norm: 9.66, NNZs: 4579, Bias: 0.062755, T: 467406, Avg. loss: 0.628572\n",
      "Total training time: 0.22 seconds.\n",
      "-- Epoch 3\n",
      "Norm: 10.96, NNZs: 4579, Bias: 0.068761, T: 701109, Avg. loss: 0.591363\n",
      "Total training time: 0.34 seconds.\n",
      "-- Epoch 1\n",
      "Norm: 7.62, NNZs: 4595, Bias: 0.056116, T: 233703, Avg. loss: 0.697730\n",
      "Total training time: 0.14 seconds.\n",
      "-- Epoch 2\n",
      "Norm: 9.61, NNZs: 4595, Bias: 0.062409, T: 467406, Avg. loss: 0.630318\n",
      "Total training time: 0.27 seconds.\n",
      "-- Epoch 3\n",
      "Norm: 10.93, NNZs: 4595, Bias: 0.068630, T: 701109, Avg. loss: 0.593220\n",
      "Total training time: 0.39 seconds.\n",
      "-- Epoch 1\n",
      "Norm: 7.55, NNZs: 4597, Bias: 0.055598, T: 233704, Avg. loss: 0.699486\n",
      "Total training time: 0.13 seconds.\n",
      "-- Epoch 2\n",
      "Norm: 9.53, NNZs: 4597, Bias: 0.062487, T: 467408, Avg. loss: 0.633168\n",
      "Total training time: 0.25 seconds.\n",
      "-- Epoch 3\n",
      "Norm: 10.85, NNZs: 4597, Bias: 0.068524, T: 701112, Avg. loss: 0.596417\n",
      "Total training time: 0.37 seconds.\n",
      "building tree 1 of 2building tree 2 of 2\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   2 out of   2 | elapsed: 26.3min remaining:    0.0s\n",
      "[Parallel(n_jobs=-1)]: Done   2 out of   2 | elapsed: 26.3min finished\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.2s remaining:    0.0s\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.2s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 2 of 2building tree 1 of 2\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   2 out of   2 | elapsed: 28.0min remaining:    0.0s\n",
      "[Parallel(n_jobs=-1)]: Done   2 out of   2 | elapsed: 28.0min finished\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.3s remaining:    0.0s\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.5s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 2 of 2\n",
      "building tree 1 of 2\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   2 out of   2 | elapsed: 27.0min remaining:    0.0s\n",
      "[Parallel(n_jobs=-1)]: Done   2 out of   2 | elapsed: 27.0min finished\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.2s remaining:    0.0s\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.2s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 2building tree 2 of 2\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   2 out of   2 | elapsed: 24.4min remaining:    0.0s\n",
      "[Parallel(n_jobs=-1)]: Done   2 out of   2 | elapsed: 24.4min finished\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.1s remaining:    0.0s\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.1s finished\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.2s remaining:    0.0s\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.2s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 2 of 2building tree 1 of 2\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   2 out of   2 | elapsed: 26.2min remaining:    0.0s\n",
      "[Parallel(n_jobs=-1)]: Done   2 out of   2 | elapsed: 26.2min finished\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    1.9s remaining:    0.0s\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    2.7s finished\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:   16.8s remaining:    0.0s\n",
      "[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:   17.6s finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=Pipeline(steps=[('union', FeatureUnion(n_jobs=1,\n",
       "       transformer_list=[('summary', Pipeline(steps=[('selector', ItemSelector(key='Summary')), ('count_vectorizer', CountVectorizer(analyzer='word', binary=False, decode_error='strict',\n",
       "        dtype=<class 'numpy.int64'>, encoding='utf-8', input='content',\n",
       " ..., penalty='l2', power_t=0.25,\n",
       "       random_state=None, shuffle=True, verbose=0, warm_start=False))]),\n",
       "       fit_params={}, iid=True, n_jobs=1,\n",
       "       param_grid=[{'model': [SGDRegressor(alpha=0.0001, average=False, epsilon=0.1, eta0=0.01,\n",
       "       fit_intercept=True, l1_ratio=0.15, learning_rate='invscaling',\n",
       "       loss='squared_loss', n_iter=3, penalty='l2', power_t=0.25,\n",
       "       random_state=44, shuffle=True, verbose=3, warm_start=False)]}, {'mod...False, precompute=False,\n",
       "      random_state=44, selection='cyclic', tol=0.0001, warm_start=False)]}],\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score=True,\n",
       "       scoring='neg_mean_squared_error', verbose=0)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have our model, we can see the best Score and return the model with the best parameter settings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The best score is -0.751477963212\n",
      "[mean: -1.01724, std: 0.01187, params: {'model': SGDRegressor(alpha=0.0001, average=False, epsilon=0.1, eta0=0.01,\n",
      "       fit_intercept=True, l1_ratio=0.15, learning_rate='invscaling',\n",
      "       loss='squared_loss', n_iter=3, penalty='l2', power_t=0.25,\n",
      "       random_state=44, shuffle=True, verbose=3, warm_start=False)}, mean: -1.23880, std: 0.01421, params: {'model': RandomForestRegressor(bootstrap=True, criterion='mse', max_depth=None,\n",
      "           max_features='auto', max_leaf_nodes=None,\n",
      "           min_impurity_split=1e-07, min_samples_leaf=1,\n",
      "           min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
      "           n_estimators=2, n_jobs=-1, oob_score=False, random_state=44,\n",
      "           verbose=3, warm_start=False)}, mean: -0.75148, std: 0.00687, params: {'model': LinearRegression(copy_X=True, fit_intercept=True, n_jobs=-1, normalize=False)}, mean: -0.76456, std: 0.00978, params: {'model': Ridge(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=None,\n",
      "   normalize=False, random_state=44, solver='auto', tol=0.001)}, mean: -1.74154, std: 0.02006, params: {'model': Lasso(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=1000,\n",
      "   normalize=False, positive=False, precompute=False, random_state=44,\n",
      "   selection='cyclic', tol=0.0001, warm_start=False)}, mean: -1.74154, std: 0.02006, params: {'model': ElasticNet(alpha=1.0, copy_X=True, fit_intercept=True, l1_ratio=0.5,\n",
      "      max_iter=1000, normalize=False, positive=False, precompute=False,\n",
      "      random_state=44, selection='cyclic', tol=0.0001, warm_start=False)}]\n",
      "The best model parameters are: {'model': LinearRegression(copy_X=True, fit_intercept=True, n_jobs=-1, normalize=False)}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/bharath/anaconda3/lib/python3.6/site-packages/sklearn/model_selection/_search.py:667: DeprecationWarning: The grid_scores_ attribute was deprecated in version 0.18 in favor of the more elaborate cv_results_ attribute. The grid_scores_ attribute will not be available from 0.20\n",
      "  DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "scores = grid.grid_scores_\n",
    "print(\"The best score is %s\" % grid.best_score_)\n",
    "print(scores)\n",
    "print(\"The best model parameters are: %s\" % grid.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Model Evaluation</b>\n",
    "Mean Squared Error is taken as the metrics for calculating the accuracy of our model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.743957953959\n"
     ]
    }
   ],
   "source": [
    "X_test_preds = grid.predict(X_test)\n",
    "\n",
    "score = mean_squared_error(y_test, X_test_preds)\n",
    "print(score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Standard Deviation is calculated to Check the Model fit\n",
    "std = float(np.std(y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model fits well, Score:0.743957953959\n"
     ]
    }
   ],
   "source": [
    "if score < std*0.5:\n",
    "    print(\"Poor Model Score: \"+str(score))\n",
    "else:\n",
    "    print(\"Model fits well, Score:\"+str(score))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}