Skip to content

Commit

Permalink
Dockerfile build process
Browse files Browse the repository at this point in the history
  • Loading branch information
relferreira committed Dec 22, 2022
1 parent 9213657 commit 69d73cf
Show file tree
Hide file tree
Showing 2 changed files with 67 additions and 11 deletions.
64 changes: 60 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,63 @@
FROM metabase/metabase:v0.45.1
#Copied from https://github.com/dacort/metabase-athena-driver/blob/d7572cd99551ea998a011f8f00a1e39c1eaa59b8/Dockerfile
ARG METABASE_VERSION=v0.43.4

ENV MB_DB_CONNECTION_TIMEOUT_MS=60000
FROM clojure:openjdk-11-tools-deps-slim-buster AS stg_base

COPY ./target/sparksql-databricks.metabase-driver.jar /plugins/
# Reequirements for building the driver
RUN apt-get update && \
apt-get install -y \
curl \
make \
unzip \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

RUN chmod 744 /plugins/sparksql-databricks.metabase-driver.jar
# Set our base workdir
WORKDIR /build

# We need to retrieve metabase source
# Due to how ARG and FROM interact, we need to re-use the same ARG
# Ref: https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
ARG METABASE_VERSION
RUN curl -Lo - https://github.com/metabase/metabase/archive/refs/tags/${METABASE_VERSION}.tar.gz | tar -xz \
&& mv metabase-* metabase

# Driver source goes inside metabase source so we can use their build scripts
WORKDIR /build/driver

# Copy our project assets over
COPY deps.edn ./
COPY project.clj ./
COPY src/ ./src
COPY resources/ ./resources

# Then prep our Metabase dependencies
# We need to build java deps and then spark-sql deps
# Ref: https://github.com/metabase/metabase/wiki/Migrating-from-Leiningen-to-tools.deps#preparing-dependencies
WORKDIR /build/metabase
RUN --mount=type=cache,target=/root/.m2/repository \
clojure -X:deps prep

WORKDIR /build/metabase/modules/drivers
RUN --mount=type=cache,target=/root/.m2/repository \
clojure -X:deps prep
WORKDIR /build/driver


# Now build the driver
FROM stg_base as stg_build
RUN --mount=type=cache,target=/root/.m2/repository \
clojure -X:dev:build :project-dir "\"$(pwd)\""

# We create an export stage to make it easy to export the driver
FROM scratch as stg_export
COPY --from=stg_build /build/driver/target/sparksql-databricks.metabase-driver.jar /

# Now we can run Metabase with our built driver
FROM metabase/metabase:${METABASE_VERSION} AS stg_runner

# A metabase user/group is manually added in https://github.com/metabase/metabase/blob/master/bin/docker/run_metabase.sh
# Make the UID and GID match
COPY --chown=2000:2000 --from=stg_build \
/build/driver/target/sparksql-databricks.metabase-driver.jar \
/plugins/sparksql-databricks.metabase-driver.jar
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Metabase Driver: Spark Databricks

**Credits**: This repository is only an updated version of the work of Fernando Goncalves and Rajesh Kumar Ravi.
**Credits**: This repository is only an updated version of the work of Fernando Goncalves and Rajesh Kumar Ravi.

## Installation

Expand All @@ -18,17 +18,16 @@ All you need you do is drop the driver in your `plugins/` directory. You can gra
- Database Name: usually `default`
- Username: usually `token`
- Password: personal access token created in Databrick's dashboard
- Additional JDBC connection string options:
- Additional JDBC connection string options:
- SQL Warehouse (Endpoint): you can find it at `/sql/warehouses/` at the `Connection details` tab. It should have the following pattern: `;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/endpoints/<SQL WAREHOUSE ID>;UID=token;PWD=<ACCESS TOKEN>`
- Cluster Endpoint: you will find it at your cluster's details page. It should have the following pattern: `;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/<ORG ID>/<CLUSTER ID>;AuthMech=3;UID=token;PWD=<ACCESS TOKEN>`


## Building the driver (the fast way)

Use the `Dockerfile` on this repo:

```bash
curl -L "https://github.com/relferreira/metabase-sparksql-databricks-driver/releases/download/1.4.0/sparksql-databricks.metabase-driver.jar" -o sparksql-databricks.metabase-driver.jar
curl -L "https://github.com/relferreira/metabase-sparksql-databricks-driver/releases/download/1.5.0/sparksql-databricks.metabase-driver.jar" -o sparksql-databricks.metabase-driver.jar
mkdir target
mv sparksql-databricks.metabase-driver.jar target/
docker build -t metabase:metabase-head-databricks-1.3.0 .
Expand All @@ -37,7 +36,8 @@ docker build -t metabase:metabase-head-databricks-1.3.0 .
And you can deploy to some docker registry of your own and use the image!

Example of running:
- `docker run -d -p 3000:3000 --name metabase metabase:metabase-head-databricks-1.3.0`

- `docker run -d -p 3000:3000 --name metabase metabase:metabase-head-databricks-1.5.0`

And access `http://localhost:3000`.

Expand All @@ -56,7 +56,7 @@ cd /path/to/metabase/

```bash
# (In the sparksql-databricks driver directory)
clojure -X:build :project-dir "\"$(pwd)\""
clojure -X:build :project-dir "\"$(pwd)\""
```

### Copy it to your plugins dir and restart Metabase
Expand All @@ -67,7 +67,7 @@ cp target/sparksql-databricks.metabase-driver.jar /path/to/metabase/plugins/
jar -jar /path/to/metabase/metabase.jar
```

*or:*
_or:_

```bash
mkdir -p /path/to/metabase/plugins
Expand Down

0 comments on commit 69d73cf

Please sign in to comment.