An interactive visualization of the utilization and success of chess openings over time. A final project for 6.859, Spring 2021, by Justin Lim and Matthew Tung.
A complete exposition of this project can be found here. A teaser video demonstrating our visualization can be found here.
Run the following command to gather our dataset of games from the lichess.org open database.
bash get_data.sh
It downloads the complete dataset of games across several years, unpacks it, preprocesses it using preprocess.py, and then writes a compressed version of the first 50,000 games in each month into the datafiles folder. Note that some of these downloads are very large (up to 30GB), and the unpacked versions can take up to 200GB of disk space.
For the actual deployed version of our visualization, we take the first 2,000 games from each of these preprocessed files. These are obtained by running the following command:
bash reduce_data.sh
This command writes files of the form lichess_db_standard_rated_2016-09-cleaned-small.csv to the datafiles/ folder, and this is what is used in script.py.
Visualization here: https://6859-sp21.github.io/final-project-howgoodisyourchessopening-fp/
As an extension of our A4 project, we decided to add features on top of our initial visualization in a similar fashion. Justin Lim, for both the A4 and final project components, primarily contributed to preprocssing the lichess data and displaying it with the stacked bar chart and the treemap via D3. He reflected this contribution in our writeup as well as formatting the paper and doing most of the research and providing context to chess, as a chess player himself. Matthew mainly contributed to the interactive chessboard and making sure all of the encodings interacted with one another and updated accordingly, exploring and implementing the PGN method that could be extracted from Justin's data and visualizations. He also implemented the filtering method for updating the graphs by date, Elo, and PGN, also implementing the reset and most common move functions for the chessboard. These contributions were written by him in the writeup and he also was the front for the video presentation as well as the formatting and CSS of the site itself.
Overall, we thought it would be really cool and fun to explore the open source lichess.org database and see if we could find anything cool. As a game with virutally infinite possibilities, we decided to focus on the openings and make a visualization that is engaging and could potentially be used by someone with prior chess experience (Justin) and someone with none (Matthew). We wanted the data to be given important context and be explorable across multiple dimensions. We split up the work accordingly and created, Your Check Mate, a site that can hopefully help "open" a users' eye to the world of chess!