data_dynamos_exercise

There are two ways to setup and run the code in this repository:

1. Setup Directly on the Machine: Install all the Software directly on your machine and run the code.
2. Setup Using Docker: Use a pre-built Docker image containing all the Software already and run the code inside Docker container.

Setup Directly on the Machine

Prerequisites

Ensure you have Git installed on your machine. If not, you can download it from git-scm.com.

Navigate to the directory where you want to clone the project. For example:

cd path/to/your/directory

Clone the repository using this command:

git clone https://github.com/Data-Dynamos/data_dynamos_exercise.git
cd data_dynamos_exercise

Run the scripts for ontime setup

Based on the shell you are using run one of the below scripts:

For Bash :

./scripts/streamlit_snowflake_bash_setup.sh  

For Zsh:

./scripts/streamlit_snowflake_zsh_setup.sh  

To start streamlit instance - run

./scripts/streamlit_snowflake_startup.sh

Setup Using Docker

Prerequisites

• Brew (Use homebrew for mac package manager - https://brew.sh/)
• Docker (Use colima for docker - https://github.com/abiosoft/colima)
• Once Docker is installed, start the docker instance:

colima start
docker ps

Usage

Note: The code will be mounted inside the container so whatever changes you make in your local machine will be synced to the container.

Data Transformations

• Start the container

docker run --rm -it -v $PWD:/opt/data_dynamos_exercise quay.io/data-dynamos/data_dynamos_exercise bash

• Run the code inside the container

export SNOWFLAKE_ACCOUNT='<your-snowflake-account-name>'
export SNOWFLAKE_DATABASE='<your-database-name>'
export SNOWFLAKE_USER='<your-user-name>'
export SNOWFLAKE_PASSWORD='<your-password>'

cd data_transformation
dbt debug # should connect to snowflake successfully

Data Visualizations

• Start the container

docker run -p 8501:8501 --rm -it -v $PWD:/opt/data_dynamos_exercise quay.io/data-dynamos/data_dynamos_exercise bash

• Run the code inside the container

./scripts/streamlit_snowflake_startup.sh