Updating Westeros tutorials and adding new ones

RELEASE_NOTES.md

@@ -1,6 +1,8 @@
 
 # Next Release
 
+- [#232](https://github.com/iiasa/message_ix/pull/232): Adding a Westeros tutorial for modelling seasonality and updating the existing ones
+
 # v1.2.0
 
 MESSAGEix 1.2.0 adds an option to set the commodity balance to strict equality,
@@ -32,6 +34,7 @@ other improvements. See the ixmp release notes for further details.
 - [#138](https://github.com/iiasa/message_ix/pull/138): Update documentation and tutorials.
 - [#131](https://github.com/iiasa/message_ix/pull/131): Update clone function argument `scen` to `scenario` with planned deprecation of the former.
 
+
 # v1.1.0
 
 ## Warnings

tests/test_tutorials.py

@@ -26,7 +26,8 @@
 #    b. Expected objective value.
 tutorials = [
     (('westeros', 'westeros_baseline'),
-     [('solve-objective-value', 207544.09375)]),
+     [('solve-objective-value', 238193.291167)]),
+
     # on Python 2:
     # 'solve-objective-value', 187445.953125),
     (('westeros', 'westeros_emissions_bounds'), []),

tutorial/README.rst

@@ -62,19 +62,19 @@ Westeros Electrified
 This tutorial demonstrates how to model a very simple energy system, and then
 uses it to illustrate a range of framework features.
 
-1. `Build the baseline model <https://github.com/iiasa/message_ix/blob/v1.1.0/tutorial/westeros/westeros_baseline.ipynb>`_.
-2. `Introduce emissions <https://github.com/iiasa/message_ix/blob/v1.1.0/tutorial/westeros/westeros_emissions_bounds.ipynb>`_ and a bound on the emissions.
-3. `Limit emissions using a tax <https://github.com/iiasa/message_ix/blob/v1.1.0/tutorial/westeros/westeros_emissions_taxes.ipynb>`_ instead of a bound.
-4. `Represent both coal and wind electricity <https://github.com/iiasa/message_ix/blob/v1.1.0/tutorial/westeros/westeros_firm_capacity.ipynb>`_, using a “firm capacity” formulation: each generation technology can supply some firm capacity, but the variable, renewable technology (wind) supplies less than coal.
-5. Represent coal and wind electricity using a different, `“flexibility requirement” formulation <https://github.com/iiasa/message_ix/blob/v1.1.0/tutorial/westeros/westeros_flexible_generation.ipynb>`_, wherein wind *requires* and coal *supplies* flexibility.
+1. `Build the baseline model <https://github.com/iiasa/message_ix/blob/master/tutorial/westeros/westeros_baseline.ipynb>`_.
+2. `Introduce emissions <https://github.com/iiasa/message_ix/blob/master/tutorial/westeros/westeros_emissions_bounds.ipynb>`_ and a bound on the emissions.
+3. `Limit emissions using a tax <https://github.com/iiasa/message_ix/blob/master/tutorial/westeros/westeros_emissions_taxes.ipynb>`_ instead of a bound.
+4. `Represent both coal and wind electricity <https://github.com/iiasa/message_ix/blob/master/tutorial/westeros/westeros_firm_capacity.ipynb>`_, using a “firm capacity” formulation: each generation technology can supply some firm capacity, but the variable, renewable technology (wind) supplies less than coal.
+5. Represent coal and wind electricity using a different, `“flexibility requirement” formulation <https://github.com/iiasa/message_ix/blob/master/tutorial/westeros/westeros_flexible_generation.ipynb>`_, wherein wind *requires* and coal *supplies* flexibility.
 
 Austrian energy system
 ----------------------
 
 This tutorial demonstrates a stylized representation of a national electricity
 sector model, with several fossil and renewable power plant types.
 
-1. Prepare the base model version, in `Python <https://github.com/iiasa/message_ix/blob/v1.1.0/tutorial/Austrian_energy_system/austria.ipynb>`__ or in `R <https://github.com/iiasa/message_ix/blob/v1.1.0/tutorial/Austrian_energy_system/austria_reticulate.ipynb>`__.
-2. Plot results, in `Python <https://github.com/iiasa/message_ix/blob/v1.1.0/tutorial/Austrian_energy_system/austria_load_scenario.ipynb>`__ or in `R <https://github.com/iiasa/message_ix/blob/v1.1.0/tutorial/Austrian_energy_system/austria_load_scenario_R.ipynb>`__.
-3. `Run a single policy scenario <https://github.com/iiasa/message_ix/blob/v1.1.0/tutorial/Austrian_energy_system/austria_single_policy.ipynb>`_.
-4. Run multiple policy scenarios. This tutorial has two notebooks: `an introduction with some exercises <https://github.com/iiasa/message_ix/blob/v1.1.0/tutorial/Austrian_energy_system/austria_multiple_policies.ipynb>`_ and `completed code for the exercises <https://github.com/iiasa/message_ix/blob/v1.1.0/tutorial/Austrian_energy_system/austria_multiple_policies-answers.ipynb>`_.
+1. Prepare the base model version, in `Python <https://github.com/iiasa/message_ix/blob/master/tutorial/Austrian_energy_system/austria.ipynb>`__ or in `R <https://github.com/iiasa/message_ix/blob/master/tutorial/Austrian_energy_system/austria_reticulate.ipynb>`__.
+2. Plot results, in `Python <https://github.com/iiasa/message_ix/blob/master/tutorial/Austrian_energy_system/austria_load_scenario.ipynb>`__ or in `R <https://github.com/iiasa/message_ix/blob/master/tutorial/Austrian_energy_system/austria_load_scenario_R.ipynb>`__.
+3. `Run a single policy scenario <https://github.com/iiasa/message_ix/blob/master/tutorial/Austrian_energy_system/austria_single_policy.ipynb>`_.
+4. Run multiple policy scenarios. This tutorial has two notebooks: `an introduction with some exercises <https://github.com/iiasa/message_ix/blob/master/tutorial/Austrian_energy_system/austria_multiple_policies.ipynb>`_ and `completed code for the exercises <https://github.com/iiasa/message_ix/blob/master/tutorial/Austrian_energy_system/austria_multiple_policies-answers.ipynb>`_.

tutorial/utils/plotting.py

@@ -79,7 +79,7 @@ def plot_capacity(self, baseyear=False, subset=None):
         df = self.model_data('CAP', baseyear=baseyear, subset=subset)
         df.plot.bar(stacked=True)
         plt.title('{} Energy System Capacity'.format(self.country.title()))
-        plt.ylabel('GWa')
+        plt.ylabel('GW')
         plt.xlabel('Year')
         plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5))
 
@@ -91,7 +91,7 @@ def plot_new_capacity(self, baseyear=False, subset=None):
         df = pd.concat([h, m]) if not h.empty else m
         df.plot.bar(stacked=True)
         plt.title('{} Energy System New Capcity'.format(self.country.title()))
-        plt.ylabel('GWa')
+        plt.ylabel('GW')
         plt.xlabel('Year')
         plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5))
 

tutorial/westeros/westeros_baseline.ipynb

@@ -12,12 +12,14 @@
     "\n",
     "### *Integrated Assessment Modeling for the 21st Century*\n",
     "\n",
-    "Please refer to the [user guidelines](https://github.com/iiasa/message_ix/blob/master/NOTICE.rst)\n",
-    "for additional information on using *MESSAGEix*, including the recommended citation and how to name new models.\n",
-    "\n",
     "This tutorial is based on a presentation by Matthew Gidden ([@gidden](https://github.com/gidden))\n",
     "for a summer school at the the **Centre National de la Recherche Scientifique (CNRS)**\n",
-    "on *Integrated Assessment Modeling* in June 2018."
+    "on *Integrated Assessment Modeling* in June 2018.\n",
+    "\n",
+    "For information on how to install `MESSAGEix`, please refer to [Installation page](https://message.iiasa.ac.at/en/stable/getting_started.html) and for getting `MESSAGEix` tutorials, please follow the steps mentioned in [Tutorials](https://message.iiasa.ac.at/en/stable/tutorials.html).\n",
+    "\n",
+    "Please refer to the [user guidelines](https://github.com/iiasa/message_ix/blob/master/NOTICE.rst)\n",
+    "for additional information on using *MESSAGEix*, including the recommended citation and how to name new models."
    ]
   },
   {
@@ -50,25 +52,6 @@
     "<img src='_static/doc_page.png'>"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "source": [
-    "## Simple installation\n",
-    "\n",
-    "When working with Python, you can easily install `MESSAGEix` yourself and get all the tutorials:\n",
-    "\n",
-    "```shell\n",
-    "$ conda install -c conda-forge message-ix # install the MESSAGEix package and all dependencies\n",
-    "\n",
-    "$ messageix-dl # download all tutorials to your current directory\n",
-    "```"
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {
@@ -94,20 +77,20 @@
    "source": [
     "## MESSAGEix: the mathematical paradigm\n",
     "\n",
-    "At its core, *MESSAGEix* is an optimization problem:\n",
+    "At its core, *MESSAGEix* is an optimization model:\n",
     "\n",
     "> $\\min \\quad ~c^T \\cdot x$  \n",
     "> $~s.t. \\quad A \\cdot x \\leq b$\n",
     "\n",
     "More explicitly, the model...\n",
     "- optimizes an **objective function**, nominally minimizing total **system costs**\n",
-    "- under a system of **constraints** equations (inequalities or equality conditions)\n",
+    "- under a system of **constraints** (inequalities or equality conditions)\n",
     "\n",
     "The mathematical implementation includes a number of features that make it particularly geared towards the modelling of *energy-water-land systems* in the context of *climate change mitigation and sustainable development*.\n",
     "\n",
     "Throughout this document, the mathematical formulation follows the convention that\n",
     "- decision **VARIABLES** ($x$) are capitalized\n",
-    "- **parameters** ($A$, $b$) are lower case"
+    "- input **parameters** ($A$, $b$) are lower case"
    ]
   },
   {
@@ -118,7 +101,7 @@
     }
    },
    "source": [
-    "## MESSAGEix: connected to the *ix modeling platform*\n",
+    "## MESSAGEix: connected to the *ix modeling platform (ixmp)*\n",
     "\n",
     "The *modeling platform for integrated and cross-cutting analysis* (ixmp) provides a powerful framework for working with scenarios, including a database infrastucture for data version control and interfaces to scientific programming languages.\n",
     "\n",
@@ -135,7 +118,7 @@
    "source": [
     "## Ready, steady, go!\n",
     "\n",
-    "First, we import all the packages we need."
+    "First, we import all the packages we need. We import a utility function called *make_df*, which can be used to wrap the input data into dataframes that can be saved in model parameters."
    ]
   },
   {
@@ -165,7 +148,7 @@
     }
    },
    "source": [
-    "The `Platform` is your connection to a database for storing model input data and scenario results."
+    "The *MESSAGEix* model is built using the *ixmp* `Platform`. The `Platform` is your connection to a database for storing model input data and scenario results."
    ]
   },
   {
@@ -189,7 +172,7 @@
     }
    },
    "source": [
-    "Once connected, we create a new `Scenario` to build our model."
+    "Once connected, we create a new `Scenario` to build our model. A `Scenario` instance will contain all the model input data and results."
    ]
   },
   {
@@ -227,7 +210,7 @@
     }
    },
    "source": [
-    "The model horizon will span 3 decades. Let's assume that we're far in the future after the events of A Song of Ice and Fire (which occur ~300 years after Aegon the conqueror).\n",
+    "The model horizon will span 3 decades (690-720). Let's assume that we're far in the future after the events of A Song of Ice and Fire (which occur ~300 years after Aegon the conqueror).\n",
     "\n",
     "| Math Notation | Model Meaning                |\n",
     "|---------------|------------------------------|\n",
@@ -289,7 +272,7 @@
     }
    },
    "source": [
-    "And we fill in the energy system's `commodities`, `levels`, `technologies`, and `mode` (defining how certain technologies operate). \n",
+    "And we fill in the energy system's `commodities`, `levels`, `technologies`, and `modes` (i.e., modes of operation of technologies). This information defines how certain technologies operate. \n",
     "\n",
     "\n",
     "| Math Notation | Model Meaning|\n",
@@ -363,13 +346,13 @@
     }
    },
    "source": [
-    "The `COMMODITY_BALANCE` equation ensures that `demand` for each `commodity` is met at each `level` in the energy system.\n",
+    "The `COMMODITY_BALANCE_GT` and `COMMODITY_BALANCE_LT` equations ensure that `demand` for each `commodity` is met at each `level` in the energy system.\n",
     "\n",
     "$\\sum_{\\substack{n^L,t,m \\\\ y^V \\leq y}} \\text{output}_{n^L,t,y^V,y,m,n,c,l} \\cdot \\text{ACT}_{n^L,t,y^V,y,m}$\n",
     "$- \\sum_{\\substack{n^L,t,m, \\\\ y^V \\leq y}} \\text{input}_{n^L,t,y^V,y,m,n,c,l} \\cdot \\text{ACT}_{n^L,t,m,y}$  \n",
     "$\\geq \\text{demand}_{n,c,l,y} \\quad \\forall \\ l \\in L$\n",
     "\n",
-    "While `demand` must be met, it can also be *exceeded* allowing the model to plan for future periods of demand for storable commodities.\n"
+    "While `demand` must be met, supply can *exceed* demand allowing the model to plan for meeting demand in future periods by storing storable commodities.\n"
    ]
   },
   {
@@ -461,6 +444,7 @@
    },
    "outputs": [],
    "source": [
+    "# We use add_par for adding data to a MESSAGEix parameter\n",
     "scenario.add_par(\"demand\", light_demand)"
    ]
   },
@@ -472,7 +456,7 @@
     }
    },
    "source": [
-    "In order to add values for the reference energy system, we define some common keys.\n",
+    "In order to define the input and output commodites of each technology, we define some common keys.\n",
     "\n",
     "- **Input** quantities require `_origin` keys that specify where the inputs are *received from*.\n",
     "- **Output** quantities require `_dest` keys that specify where the outputs are *transferred to*."
@@ -517,7 +501,7 @@
     "- receives *input* in the form of the \"electricity\" *commodity* at the \"final [energy]\" *level*, and\n",
     "- *outputs* the commodity \"light\" at the \"useful [energy]\" level.\n",
     "\n",
-    "Because there are no other commodities or forms of useful energy in Westeros, we specify that 100% of the input and output are from/to these commodities/levels."
+    "The `value` in the input and output parameter is used to represent the effiecieny of a technology (efficiency = output/input). For example, input of 1.0 and output of 1.0 for a technology shows that the efficiency of that technology is 100% in converting input commodity to output commodity."
    ]
   },
   {
@@ -547,7 +531,7 @@
     }
    },
    "source": [
-    "Next, the electrical `grid`, which…\n",
+    "Next, we parameterize the electrical `grid`, which…\n",
     "\n",
     "- receives electricity at the \"secondary\" energy level.\n",
     "- also outputs electricity, but at the \"final\" energy level (to be used by the light bulb).\n",
@@ -626,7 +610,7 @@
    "source": [
     "The model has a number of \"reality\" constraints, which relate built *capacity* (`CAP`) to available power, or the *activity* (`ACT`) of that technology.\n",
     "\n",
-    "The **capacity constraint** limits the total time a each technology can operate.\n",
+    "The **capacity constraint** limits the activity of a technology to the installed capacity multiplied by a capacity factor. Capacity factor or is the fraction of installed capacity that can be active in a certain period (here the sub-annual time step *h*).\n",
     "\n",
     "$$\\sum_{m} \\text{ACT}_{n,t,y^V,y,m,h}\n",
     "    \\leq \\text{duration_time}_{h} \\cdot \\text{capacity_factor}_{n,t,y^V,y,h} \\cdot \\text{CAP}_{n,t,y^V,y}\n",
@@ -675,7 +659,7 @@
    "source": [
     "capacity_factor = {\n",
     "    'coal_ppl': 1,\n",
-    "    'wind_ppl': 1,\n",
+    "    'wind_ppl': 0.36,\n",
     "    'bulb': 1, \n",
     "}\n",
     "\n",
@@ -753,7 +737,7 @@
    "source": [
     "## Technological Diffusion and Contraction\n",
     "\n",
-    "We know from historical precedent that energy systems can not be transformed instantaneously. Therefore, we use a family of constraints on activity and capacity."
+    "We know from historical precedent that energy systems can not be transformed instantaneously. Therefore, we use a family of dynamic constraints on activity and capacity. These constraints define the upper and lower limit of the domain of activity and capacity over time based on their value in the previous time step, an initial value, and growth/decline rates."
    ]
   },
   {
@@ -827,9 +811,9 @@
     }
    },
    "source": [
-    "## Defining an Energy Mix\n",
+    "## Defining an Energy Mix (Model Calibration)\n",
     "\n",
-    "To model the transition of an energy system, one must start with the existing system which are defined by the parameters `historical_activity` and `historical_capacity`. These parameters define the energy mix before the model horizon.  \n",
+    "To model the transition of an energy system, one must start with the existing system which are defined by the parameters `historical_activity` and `historical_new_capacity`. These parameters define the energy mix before the model horizon.  \n",
     "\n",
     "We begin by defining a few key values:\n",
     "\n",
@@ -1010,8 +994,11 @@
     "base_inv_cost = {\n",
     "    'node_loc': country,\n",
     "    'year_vtg': model_horizon,\n",
-    "    'unit': 'USD/GWa',\n",
-    "}"
+    "    'unit': 'USD/kW',\n",
+    "}\n",
+    "\n",
+    "# Adding a new unit to the library\n",
+    "mp.add_unit('USD/kW')     "
    ]
   },
   {
@@ -1065,7 +1052,7 @@
     "    'node_loc': country,\n",
     "    'year_vtg': vintage_years,\n",
     "    'year_act': act_years,\n",
-    "    'unit': 'USD/GWa',\n",
+    "    'unit': 'USD/kW',\n",
     "}"
    ]
   },
@@ -1121,7 +1108,7 @@
     "    'year_act': act_years,\n",
     "    'mode': 'standard',\n",
     "    'time': 'year',\n",
-    "    'unit': 'USD/GWa',\n",
+    "    'unit': 'USD/kWa',\n",
     "}"
    ]
   },
@@ -1135,7 +1122,7 @@
    },
    "outputs": [],
    "source": [
-    "# in $ / MWh\n",
+    "# in $ / kWa\n",
     "costs = {\n",
     "    'coal_ppl': 30,\n",
     "    'grid': 50,\n",
@@ -1173,7 +1160,7 @@
     "## Time to Solve the Model\n",
     "\n",
     "First, we *commit* the model structure and input data (sets and parameters).\n",
-    "In the `ixmp` backend, this creates a new model version in the database, which is assigned a number automatically:"
+    "In the `ixmp` backend, this creates a new model version in the database, which is assigned a version number automatically:"
    ]
   },
   {
@@ -1259,7 +1246,7 @@
    "source": [
     "## Plotting Results\n",
     "\n",
-    "We make use of some custom code; see `tools.py` in the tutorial directory."
+    "We make use of some custom code for plotting the results; see `tools.py` in the tutorial directory."
    ]
   },
   {
@@ -1312,7 +1299,7 @@
    "source": [
     "### Capacity\n",
     "\n",
-    "Given how many new plants are built, how many are actually used?"
+    "How much capacity of each plant is installed in each period?"
    ]
   },
   {
@@ -1338,9 +1325,9 @@
    "source": [
     "### Electricity Price\n",
     "\n",
-    "And how much does the electricity cost? These prices are taken from the **dual variables** of the solution and are given the name **shadow prices**. They reflect the marginal price of electricity, taken from the most expensive producer.  \n",
+    "And how much does the electricity cost? These prices are in fact **shadow prices** taken from the **dual variables** of the model solution. They reflect the marginal cost of electricity generation (i.e., the additional cost of the system for supplying one more unit of electricity), which is in fact the marginal cost of the most expensive generator.  \n",
     "\n",
-    "Note that the price drop when the most expensive technology is no longer in the system."
+    "Note the price drop when the most expensive technology is no longer in the system."
    ]
   },
   {