Assignment 2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SYDE556/750 Assignment 2: Spiking Neurons\n",
    "\n",
    "- Due Date: Feb 15th: Assignment #2 (due at midnight) \n",
    "- Total marks: 20 (20% of final grade)\n",
    "- Late penalty: 1 mark per day\n",
    "- It is recommended that you Python.\n",
    "- *Do not use or refer to any code from Nengo*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1) Generating a random input signal\n",
    "\n",
    "### 1.1) Gaussian white noise\n",
    "\n",
    "Create a function called that generates a randomly varying $x(t)$ signal chosen from a white noise distribution.  Call it 'generate_signal' and ensure that it returns $x(t)$ and $X(\\omega)$.\n",
    "\n",
    "The inputs to the function are:\n",
    "\n",
    "- ``T``: the length of the signal in seconds\n",
    "- ``dt``: the time step in seconds\n",
    "- ``rms``: the root mean square power level of the signal.  That is, the resulting signal should have $\\sqrt{{1 \\over T} \\int{x(t)^2}dt}=rms$\n",
    "- ``limit``: the maximum frequency for the signal (in Hz)\n",
    "- ``seed``: the random number seed to use (so we can regenerate the same signal again)\n",
    "\n",
    "<ol style=\"list-style:lower-alpha outside none;\" start=\"1\">\n",
    "<li> [1 mark] Plot $x(t)$ for three randomly generated signals with ``limit`` at 5, 10, and 20Hz.  For each of these, ``T``=1, ``dt``=0.001, and ``rms``=0.5.</li>\n",
    "\n",
    "<li> [1 mark] Plot the average $|X(\\omega)|$ (the norm of the Fourier coefficients) over 100 signals generated with  ``T``=1, ``dt``=0.001, ``rms``=0.5, and ``limit``=10 (each of these 100 signals should have a different ``seed``).  The plot should have the x-axis labeled ($\\omega$ in radians) and the average $|X|$ value for that $\\omega$ on the y-axis.</li>\n",
    "</ol>\n",
    "\n",
    "Notes:\n",
    "\n",
    "- To do Fourier transforms in MATLAB, see [here](http://www.mathworks.com/help/matlab/ref/fft.html)\n",
    "- To do Fourier transforms in Python, see [here](http://docs.scipy.org/doc/numpy/reference/routines.fft.html)    \n",
    "- In both cases, the transform takes you from $t$ to $\\omega$ (or back the other way).  Importantly, $\\omega$ is frequency in *radians*, not in Hz.  \n",
    "- $\\Delta \\omega$ will be $2 \\pi / T$\n",
    "- To keep the signal real, $X(\\omega)=X(-\\omega)^*$ (the complex conjugate: the real parts are equal, and the imaginary parts switch sign)\n",
    "- When randomly generating $X(\\omega)$ values, sample them from a Normal distribution $N(\\mu=0,\\sigma=1)$.  Remember that these are complex numbers, so sample twice from the distribution; once for the real component and once for the imaginary.\n",
    "- To implement the ``limit``, set all $X(\\omega)$ components with frequencies above the limit to 0\n",
    "- To implement the ``rms``, generate the signal, compute its RMS power ($\\sqrt{{1 \\over T} \\int{x(t)^2}dt}=rms$) and rescale so it has the desired power."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2) Gaussian power spectrum noise\n",
    "\n",
    "Create a modified version of your function from question 1.1 that produces noise with a different power spectrum.  Instead of having the $X(\\omega)$ values be 0 outside of some limit and sampled from $N(\\mu=0,\\sigma=1)$ inside that limit, we want a smooth drop-off of power as the frequency increases.  In particular, instead of the ``limit``, we sample from $N(\\mu=0,\\sigma=e^{-{\\omega^2/(2*b^2)}})$ where $b$ (in radians) is the new ``bandwidth`` parameter that replaces the ``limit`` parameter.\n",
    "\n",
    "<ol style=\"list-style:lower-alpha outside none;\" start=\"1\">\n",
    "<li>[1 mark] Plot $x(t)$ for three randomly generated signals with ``bandwidth`` at 5, 10, and 20Hz.  For each of these, ``T``=1, ``dt``=0.001, and ``rms``=0.5.</li>\n",
    "\n",
    "<li>[1 mark] Plot the average $|X(\\omega)|$ (the norm of the Fourier coefficients) over 100 signals generated with  ``T``=1, ``dt``=0.001, ``rms``=0.5, and ``bandwidth``=10 (each of these 100 signals should have a different ``seed``).</li>\n",
    "</ol>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2) Simulating a Spiking Neuron\n",
    "\n",
    "Write a program to simulate a single Leaky-Integrate and Fire neuron.  The core equation is $ {{dV} \\over {dt}} = {1 \\over {\\tau_{RC}}} (J - V)$ (to simplify life, this is normalized so that $R$=1, the resting voltage is 0 and the firing voltage is 1).  This equation can be simulated numerically by taking  small time steps (Euler's method).  When the voltage reaches the threshold $1$, the neuron will spike and then reset its voltage to $0$ for the next $\\tau_{ref}$ amount of time (to plot this, place a dot or line at that time).  Also, if the voltage goes below zero at any time, reset it back to zero.  For this question, $\\tau_{RC}$=0.02 and $\\tau_{ref}$=0.002\n",
    "\n",
    "Since we want to do inputs in terms of $x$, we need to do $J = \\alpha e \\cdot x + J^{bias}$.  For this neuron, set $e$ to $+1$ and find $\\alpha$ and $J^{bias}$ such that the firing rate when $x=0$ is 40Hz and when $x=1$ it is 150Hz.  To find these $\\alpha$ and $J^{bias}$ values, use the approximation for the LIF neuron $a(J)={1 \\over {\\tau_{ref}-\\tau_{RC}ln(1-{1 \\over J})}}$.\n",
    "\n",
    "<ol style=\"list-style:lower-alpha outside none;\" start=\"1\">\n",
    "<li> [1 mark] Plot the spike output for a constant input of $x=0$ over 1 second.  Report the number of spikes.  Do the same thing for $x=1$.  Use ``dt``=0.001 for the simulation.</li>\n",
    "\n",
    "<li> [1 mark] Does the observed number of spikes in the previous part match the expected number of spikes for $x=0$ and $x=1$?  Why or why not?  What aspects of the simulation would affect this accuracy?</li>\n",
    "\n",
    "<li> [1 mark] Plot the spike output for $x(t)$ generated using your function from part 1.1.  Use ``T``=1, ``dt``=0.001, ``rms``=0.5, and ``limit``=30.  Overlay on this plot $x(t)$.</li>\n",
    "\n",
    "<li> [1 mark] Using the same $x(t)$ signal as in part (c), plot the neuron's voltage over time for the first 0.2 seconds, along with the spikes over the same time.</li>\n",
    "\n",
    "\n",
    "<li> BONUS: How could you improve this simulation (in terms of how closely the model matches actual equation) without significantly increasing the computation time?  0.5 marks for having a good idea. Up to 1 mark for actually implementing it and showing that it works.</li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3) Simulating Two Spiking Neurons\n",
    "\n",
    "Write a program that simulates two neurons.  The two neurons have exactly the same parameters, except for one of them $e=1$ and for the other $e=-1$.  Other than that, use exactly the same settings as in question 2.\n",
    "\n",
    "<ol style=\"list-style:lower-alpha outside none;\" start=\"1\">\n",
    "\n",
    "<li>[0.5 marks] Plot $x(t)$ and the spiking output for $x(t)=0$ (both neurons should spike at ~40 spikes per second).</li>\n",
    "<li>[0.5 marks] Plot $x(t)$ and the spiking output for $x(t)=1$ (one neuron should spike at ~150 spikes per second, and the other should not spike at all).</li>\n",
    "<li>[1 mark] Plot $x(t)$ and the spiking output for $x(t)={1 \\over 2}sin(10\\pi t)$ (a sine wave at 5Hz).</li>\n",
    "<li>[1 mark] Plot $x(t)$ and the spiking output for a random signal generated with your function for question 1.1 with ``T``=2, ``dt``=0.001, ``rms``=0.5, and ``limit``=5.</li>\n",
    "</ol>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4) Computing an Optimal Filter\n",
    "\n",
    "Compute the optimal filter for decoding pairs of spikes.  Instead of implementing this yourself, here is an implementation in <a href=\"files/assignment2/optimal_filter.py\">Python</a> and an implementation in <a href=\"files/assignment2/optimal_filter.m\">Matlab</a>.  \n",
    "\n",
    "<ol style=\"list-style:lower-alpha outside none;\" start=\"1\">\n",
    "<li>[1 mark] Document the code and connect it with the code you wrote for part (3) so that it uses the signal used in 3.d.  Comments should be filled in where there are ``#`` signs (Python) or ``%`` signs (Matlab).  Replace the ``'???'`` labels in the code with the correct labels.  Note that you can use the generated plots for the next few parts of this question.</li>\n",
    "<li>[1 mark] Plot the time and frequency plots for the optimal filter for the signal you generated in question 3.d.</li>\n",
    "<li>[1 marks] Plot the $x(t)$ signal, the spikes, and the decoded $\\hat{x}(t)$ value for the signal in question 3.d.</li>\n",
    "<li>[1 marks] Plot the $|X(\\omega)|$ power spectrum, $|R(\\omega)|$ spike response spectrum, and the $|\\hat{X}(\\omega)|$ power spectrum for the signal in question 3.d. How do these relate to the optimal filter?</li>\n",
    "<li>[1 mark] Generate $h(t)$ time plots for the optimal filter for different ``limit`` values of 2Hz, 10Hz, and 30Hz.  Describe the effects on the time plot of the optimal filter as the ``limit`` increases.  Why does this happen?</li>\n",
    "</ol>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5) Using Post-Synaptic Currents as a Filter\n",
    "\n",
    "Instead of using the optimal filter from the previous question, now we will use the post-synaptic current instead.  This is of the form $h(t)=t^n e^{-t/\\tau}$ normalized to area 1.\n",
    "\n",
    "<ol style=\"list-style:lower-alpha outside none;\" start=\"1\">\n",
    "<li>[1 mark] Plot the normalized $h(t)$ for $n$=0, 1, and 2 with $\\tau$=0.007 seconds.  What two things do you expect increasing $n$ will do to $\\hat{x}(t)$?</li>\n",
    "\n",
    "<li>[1 mark] Plot the normalized $h(t)$ for $\\tau$=0.002, 0.005, 0.01, and 0.02 seconds with $n$=0.  What two things do you expect increasing $\\tau$ will do to $\\hat{x}(t)$?</li>\n",
    "\n",
    "<li>[1 mark] Decode $\\hat{x}(t)$ from the spikes generated in question 3.d using an $h(t)$ with $n$=0 and $\\tau$=0.007.  Do this by generating the spikes, filtering them with $h(t)$, and using that as your activity matrix $A$ to compute your decoders.  Plot the time and frequency plots for this $h(t)$.  Plot the $x(t)$ signal, the spikes, and the decoded $\\hat{x}(t)$ value.</li>\n",
    "\n",
    "<li>[1 mark] Use the same decoder and $h(t)$ as in part (c), but generate a new $x(t)$ with ``limit``=2Hz.  Plot the $x(t)$ signal, the spikes, and the decoded $\\hat{x}(t)$ value.  How do these decodings compare?</li>\n",
    "</ol>\n",
    "\n",
    "Notes:\n",
    "\n",
    "- Do not use in-built convolution functions in c) or d).  Instead, implement this in the time domain by placing the filter when spikes occur, and then summing.  This simulates a post-synaptic current occuring on the arrival of a spike from the pre-synaptic neuron."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}