trial_one.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sensical Trial 1\n",
    "\n",
    "**Hypothesis:**  Patterns of cognitive gaps will follow coronavirus events timeline; the greatest number of tweets containing emotional states of uncertainty, anxiety and confusion will fall in line with significant pandemic events identified by leading health organizations and new sources. Cognitive gaps with be greatest and the onset of the pandemic and decrease over time.\n",
    "\n",
    "## steps\n",
    "\n",
    "0. Configure the Python Twitter client with the developer credentials.\n",
    "1. **Download** about 5k tweets randomly sampled from the start of the [#c19](c19.ai) pandemic to now.\n",
    "2. Run all of these through some **sentiment analysis** function to get emotional states of interest: `['anxiety', 'confusion', 'uncertainty', 'other']`.  We may add scores to each of the emotions of interest, and we may additionally conduct something akin to [PageRank](https://en.wikipedia.org/wiki/PageRank) to determine the authenticity/social rank associated with the source of the tweet.  Or more simply, we may extract their job title.\n",
    "3. Build a basic time-based **visualization** showing how those categories change over time. If the tweets have geolocation enabled, we can build a visual view for how those emotions propagate over time as well.\n",
    "\n",
    "## todo\n",
    "\n",
    "- [x] Get tweet IDs and repository downloaded.\n",
    "- [ ] Configure `twarc` below with developer credentials for using the Twitter API.\n",
    "- [ ] \"Hydrate\" around 5,000 tweets between January and May, inclusive.\n",
    "- [ ] Buildup semantic analysis function to annotate each tweet, constructing a dictionary of `tweet_id -> (semantic_category, timestamp)`.\n",
    "- [ ] Using visualization packages such as `seaborn`, buildup about 3 different time-based visualizations "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/ec2-user/anaconda3/envs/pytorch_p36/bin/python\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "print(sys.executable)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: twarc in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (1.8.3)\n",
      "Requirement already satisfied: pytest in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from twarc) (3.5.1)\n",
      "Requirement already satisfied: requests-oauthlib in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from twarc) (1.3.0)\n",
      "Requirement already satisfied: python-dateutil in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from twarc) (2.7.3)\n",
      "Requirement already satisfied: py>=1.5.0 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from pytest->twarc) (1.5.3)\n",
      "Requirement already satisfied: six>=1.10.0 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from pytest->twarc) (1.11.0)\n",
      "Requirement already satisfied: setuptools in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from pytest->twarc) (39.1.0)\n",
      "Requirement already satisfied: attrs>=17.4.0 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from pytest->twarc) (18.1.0)\n",
      "Requirement already satisfied: more-itertools>=4.0.0 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from pytest->twarc) (4.1.0)\n",
      "Requirement already satisfied: pluggy<0.7,>=0.5 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from pytest->twarc) (0.6.0)\n",
      "Requirement already satisfied: oauthlib>=3.0.0 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from requests-oauthlib->twarc) (3.1.0)\n",
      "Requirement already satisfied: requests>=2.0.0 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from requests-oauthlib->twarc) (2.20.0)\n",
      "Requirement already satisfied: idna<2.8,>=2.5 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from requests>=2.0.0->requests-oauthlib->twarc) (2.6)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from requests>=2.0.0->requests-oauthlib->twarc) (3.0.4)\n",
      "Requirement already satisfied: urllib3<1.25,>=1.21.1 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from requests>=2.0.0->requests-oauthlib->twarc) (1.23)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (from requests>=2.0.0->requests-oauthlib->twarc) (2019.11.28)\n",
      "\u001b[31mfastai 1.0.60 requires nvidia-ml-py3, which is not installed.\u001b[0m\n",
      "\u001b[33mYou are using pip version 10.0.1, however version 20.2b1 is available.\n",
      "You should consider upgrading via the 'pip install --upgrade pip' command.\u001b[0m\n",
      "Requirement already satisfied: tqdm in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages (4.42.1)\n",
      "\u001b[31mfastai 1.0.60 requires nvidia-ml-py3, which is not installed.\u001b[0m\n",
      "\u001b[33mYou are using pip version 10.0.1, however version 20.2b1 is available.\n",
      "You should consider upgrading via the 'pip install --upgrade pip' command.\u001b[0m\n",
      "\n",
      "Twarc needs to know a few things before it can talk to Twitter on your behalf.\n",
      "\n",
      "\n",
      "Please enter your Twitter application credentials from apps.twitter.com:\n",
      "\n",
      "consumer key: ^C\n",
      "python3: can't open file 'hydrate.py': [Errno 2] No such file or directory\n"
     ]
    }
   ],
   "source": [
    "# Install dependencies\n",
    "import sys\n",
    "!{sys.executable} -m pip install twarc\n",
    "!{sys.executable} -m pip install tqdm\n",
    "\n",
    "import seaborn\n",
    "import tqdm\n",
    "import twarc\n",
    "\n",
    "# Hydrate tweets\n",
    "!cd \"COVID-19-TweetIDs/\"\n",
    "!python3 hydrate.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def download_twitter_ids(ids: list) -> str:\n",
    "    \"\"\"\"\"\"\n",
    "    pass"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_pytorch_p36",
   "language": "python",
   "name": "conda_pytorch_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}