> For the complete documentation index, see [llms.txt](https://dataflint.gitbook.io/dataflint-for-spark/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://dataflint.gitbook.io/dataflint-for-spark/getting-started/install-on-databricks.md).

# Install on DataBricks

### Install on DataBricks as a spark plugin (recommended)

{% hint style="warning" %}
DataBricks init scripts are not supported in DataBricks Community, in this case you will need to install it from a notebook
{% endhint %}

Add to you init script the following section

```bash
DATAFLINT_VERSION="0.9.9"
SPARK_DEFAULTS_FILE="/databricks/driver/conf/00-custom-spark-driver-defaults.conf"

mkdir -p /databricks/jars/

wget --quiet \
  -O /databricks/jars/spark_2.12-$DATAFLINT_VERSION.jar \
  https://repo1.maven.org/maven2/io/dataflint/spark_2.12/$DATAFLINT_VERSION/spark_2.12-$DATAFLINT_VERSION.jar

if [[ $DB_IS_DRIVER = "TRUE" ]]; then
  mkdir -p /mnt/driver-daemon/jars/
  cp /databricks/jars/spark_2.12-$DATAFLINT_VERSION.jar /mnt/driver-daemon/jars/spark_2.12-$DATAFLINT_VERSION.jar
  echo "[driver] {" >> $SPARK_DEFAULTS_FILE
  echo "  spark.plugins = io.dataflint.spark.SparkDataflintPlugin" >> $SPARK_DEFAULTS_FILE
  echo "}" >> $SPARK_DEFAULTS_FILE
fi
```

### How to create an init script

If you do not have an existing init script, this is how you create one in Databricks:

Go to workspace and press "Create -> File"

<figure><img src="/files/65L0PyYDdMOC0qOLopqy" alt=""><figcaption></figcaption></figure>

Paste the DataFlint snippet & save. Afterward in your cluster config, to to Advanced -> init scripts and add your newly created init script:

<figure><img src="/files/5W60iwGesx3nXegRvKWe" alt=""><figcaption></figcaption></figure>

### Install on DataBricks from a notebook

{% hint style="info" %}
This method supports both DataBricks Community and the DataBricks paid version
{% endhint %}

1. Go to your cluster to "library" tab and click on the "Install new" blue button:

<figure><img src="/files/1mOtAOIn0NN9RxvS4h2o" alt=""><figcaption></figcaption></figure>

2. Choose "Maven" and enter in the coordinates for dataflint

```
io.dataflint:spark_2.12:0.9.9
```

<figure><img src="/files/hcIWwNOiNy17jURqJh2t" alt=""><figcaption><p>image is using version 0.1.0, you should use the newest dataflint version :)</p></figcaption></figure>

3. In your notebook or app, run this 2 lines (if in python notebook, also add %scala)

```scala
%scala
import io.dataflint.spark.SparkDataflint
SparkDataflint.install(spark.sparkContext)
```

<figure><img src="/files/D1f5rVoxjFotkfNTs3AZ" alt=""><figcaption></figcaption></figure>

Now DataFlint is installed! now you can go to "Spark UI" tab and see a "dataflint" tab. It's highly recommended to use the "Open in new tab" link so you get the best experience.

<figure><img src="/files/BHwFbJSSKFUDW69ubzHN" alt=""><figcaption></figcaption></figure>

{% hint style="warning" %}
Dataflint is only supported when the cluster is running, after the cluster is running dataflint will no longer be available
{% endhint %}

### Spark 4 support on Databricks

Spark 4 is supported on Databricks.

Use this section when your cluster runs Databricks Runtime `17.3+`.

The Spark 4 artifact is:

```
io.dataflint:dataflint_spark4-databricks_2.13:0.9.9
```

#### Init script for Spark 4

```bash
DATAFLINT_VERSION="0.9.9"
SPARK_DEFAULTS_FILE="/databricks/driver/conf/00-custom-spark-driver-defaults.conf"

mkdir -p /databricks/jars/

wget --quiet \
  -O /databricks/jars/dataflint-spark4-databricks_2.13-$DATAFLINT_VERSION.jar \
  https://repo1.maven.org/maven2/io/dataflint/dataflint-spark4-databricks_2.13/$DATAFLINT_VERSION/dataflint-spark4-databricks_2.13-$DATAFLINT_VERSION.jar


if [[ $DB_IS_DRIVER = "TRUE" ]]; then
  mkdir -p /mnt/driver-daemon/jars/
  cp /databricks/jars/dataflint-spark4-databricks_2.13-$DATAFLINT_VERSION.jar /mnt/driver-daemon/jars/dataflint-spark4-databricks_2.13-$DATAFLINT_VERSION.jar
  echo "[driver] {" >> $SPARK_DEFAULTS_FILE
  echo "  spark.plugins = io.dataflint.spark.SparkDataflintPlugin" >> $SPARK_DEFAULTS_FILE
  echo "}" >> $SPARK_DEFAULTS_FILE
fi
```

#### Notebook installation for Spark 4

Install this Maven package in your cluster libraries:

```
io.dataflint:dataflint_spark4-databricks_2.13:0.9.9
```

Then run the same notebook setup:

```scala
%scala
import io.dataflint.spark.SparkDataflint
SparkDataflint.install(spark.sparkContext)
```