AA228-CS238-Student

AA228/CS238: Decision Making under Uncertainty, Winter 2023, Stanford University.

This repository provides starter code and data for Projects 1 and 2.

Project 1: Bayesian Structure Learning

LaTeX Overleaf template: click the link, go to "Menu", and "Copy Project" (make sure you're signed into Overleaf). Note this is an optional template, you're free to use your own (or not even LaTeX).

project1/
├── data                    # CSV data files to apply structured learning
│   ├── small.csv               # Titanic dataset¹
│   ├── medium.csv              # Wine dataset²
│   └── large.csv               # Secret dataset
├── example                 # Helpful examples
│   ├── example.gph             # Example graph (3 parents, 1 child each)
│   ├── example.csv             # Example data generated from "example.gph"
│   ├── example.score           # Bayesian score of the "examples.gph" given the data "examples.csv"
│   ├── examples.pdf            # Visualized "examples.gph" as a TikZ graph
│   └── titanicexample.pdf      # Simple example network using "small.csv"
├── project1.jl             # Starter code in Julia (optional, meant to help)
└── project1.py             # Starter code in Python (optional, meant to help)

Notes:

• The starter code is there to help, but you're free to use any language.
• Use example.gph to validate your Bayesian scoring algorithm, not your structure learning algorithm.

¹https://cran.r-project.org/web/packages/titanic/titanic.pdf
²https://archive.ics.uci.edu/ml/datasets/Wine+Quality

Graph Plotting

Here are some resources for plotting graphs in Julia, Python, and MATLAB.

• Julia:
  • TikzGraphs.jl
  • GraphPlot.jl
  • Graphs.jl
• Python:
  • NetworkX
• MATLAB:
  • GraphPlot

Here's a minimum working example using TikzGraphs.jl in Julia:

using Graphs  # for DiGraph and add_edge!
using TikzGraphs   # for TikZ plot output
using TikzPictures # to save TikZ as PDF

g = DiGraph(2) # create a directed graph
add_edge!(g, 1, 2) # add edge from node 1 to node 2

p = plot(g, ["First", "Second"]) # create TikZ plot with labels
save(PDF("graph.pdf"), p) # save TikZ as PDF

Project 2: Reinforcement Learning

LaTeX Overleaf template: click the link, go to "Menu", and "Copy Project" (make sure you're signed into Overleaf). Note this is an optional template, you're free to use your own (or not even LaTeX).

project2/
└── data                      # CSV data files of (s,a,r,sp)
    ├── small.csv                 # 10x10 grid world
    ├── medium.csv                # MountainCarContinuous-v0
    └── large.csv                 # Secret RL problem

Note: no starter code provided for Project 2.

Contact

Please post on Ed with any questions regarding this code, the data, and the projects in general. We'd be happy to help!