# Alerts

## Summary

One of the unique "selling points" of DataFlint is the alerting system. \
Where in traditional spark monitoring you get a bunch of metrics and needs to figure out what they mean, with DataFlint you have alerts that points you to what is wrong and suggest you on potential fixes.

## Alerts

### Reading Small Files

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FWuWhShaxMCvgiV1b8f4o%2Fimage.png?alt=media&#x26;token=84cc2217-6552-4325-8014-31f504e902b3" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FU4lzgp1QE15XHNMCd4w3%2Fimage.png?alt=media&#x26;token=e6ef5ee0-4c06-4545-9fa3-af10f2bd30c2" alt=""><figcaption></figcaption></figure>

\[Also works for Apache Iceberg tables]

### Writing Small Files



<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FMZcSzfDHKvXgBYcvizAZ%2Fimage.png?alt=media&#x26;token=e301e7eb-7908-4090-8c8f-af6c6c1ff886" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FZT2womUnbmcCwdaymek7%2Fimage.png?alt=media&#x26;token=a9d1b234-976f-4bd6-a6b7-4134deb16803" alt=""><figcaption></figcaption></figure>

### Apache iceberg - inefficient replace of data

### &#x20;![](https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2F3tp15bWcFmXlA2Te6DGm%2Fimage.png?alt=media\&token=d449d870-d81e-4fff-b496-b18eb29c9824)



### Partition Skew

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FRRCCMiI5301bQH6T5i4V%2Fimage.png?alt=media&#x26;token=eb46f4d7-0515-44d7-a362-34a3a7d7324e" alt=""><figcaption></figcaption></figure>

### Large Number Of Small Tasks

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FdWpUet4ZCEQWwUahnQ0f%2Fimage.png?alt=media&#x26;token=6458cfc7-6925-4414-bc38-4c21d46a65fc" alt="" width="374"><figcaption></figcaption></figure>

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2F0Okn3KZbY0ztZe6m03St%2Fimage.png?alt=media&#x26;token=2fb35712-34fb-43bf-a328-ab9efbcecb17" alt=""><figcaption></figcaption></figure>

### Memory Over-Provisioning

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2F2ry9hswzKug0EbIZ9C7b%2Fimage.png?alt=media&#x26;token=8ca67ced-a11f-4646-8f13-7b49391e56f0" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2Fx4syhsTwdLJqufSXM2eg%2Fimage.png?alt=media&#x26;token=3502dfde-2381-4cde-bacb-c19cc76b8bcf" alt=""><figcaption></figcaption></figure>

### Memory Under-Provisioning

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FxDbWE9V969Ke3zuovKMd%2Fimage.png?alt=media&#x26;token=605f8546-1f7d-4bae-9d13-b25d84265629" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FlA1zUZkIRQsZAQQZ1QXc%2Fimage.png?alt=media&#x26;token=685c1e75-8791-4b51-bca0-cd44c19387bd" alt=""><figcaption></figcaption></figure>

### High wasted cores rate



<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FcN5hPMfNeHlEkkYlePY7%2Fimage.png?alt=media&#x26;token=0b8dce33-d271-413a-a06b-3e0aadce3be6" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FZvM93tkX4yXCotYFMILe%2Fimage.png?alt=media&#x26;token=29f32e10-5c41-472f-b691-bcfa82f72952" alt=""><figcaption></figcaption></figure>

### Large Data Broadcast

### &#x20;![](https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FYHr0IDHWRsXohEjfDMSz%2Fimage.png?alt=media\&token=79ef9184-7ebd-4065-b514-de632a3e0bb6)&#x20;

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2Fhm3S83CLVw0o2WcN3G45%2Fimage.png?alt=media&#x26;token=d73d70f9-3bed-457b-976e-7868e24e5adf" alt=""><figcaption></figcaption></figure>

## Broadcast small table in Sort Merge Join

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FQ6ompYe4Y8VjLwMSpez6%2Fimage.png?alt=media&#x26;token=c21cb7de-ac6e-4631-9967-c5df8cc1f5bb" alt="" width="375"><figcaption></figcaption></figure>

## Large Cross Join Scan

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2F4OGpFc9NGOUVu0Svga89%2Fimage.png?alt=media&#x26;token=53d60249-afb9-4dd0-a5bb-26de2401e0fe" alt="" width="375"><figcaption></figcaption></figure>

## Large Partition Size

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FWbWAIju0NreuOATNFEdH%2Fimage.png?alt=media&#x26;token=77003fb4-b7d5-4525-ae01-2ad435468667" alt="" width="375"><figcaption></figcaption></figure>

## Long Filter Conditions

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FIhpfkzoVxKTCvDK4fgMB%2Fimage.png?alt=media&#x26;token=69bdf7c5-14a5-443d-8a1c-620a4c7634fc" alt="" width="375"><figcaption></figcaption></figure>

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2FWOWayfM68bU0EwJIcQHB%2Fimage.png?alt=media&#x26;token=ba8d863b-7126-4a76-973d-6f83fcea3fa7" alt=""><figcaption></figcaption></figure>

## Query Failures

Another type of "alert" is query failure. When hovering on the alert icon, DataFlint extract the error from the scary JVM stack trace and show it in the top of the message.

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2F8x1900xyzByJNyVtI05L%2Fimage.png?alt=media&#x26;token=462d0ef8-4afa-41dd-b9c9-e06d21459f9d" alt=""><figcaption></figcaption></figure>

When you press the query you can see the exact place on the logical plan the query failed, in our case it's in the stage relates to writing files in the end of the query plan

<figure><img src="https://2982210886-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcg8pTm3VgVaeMncRl8LP%2Fuploads%2F9sFVZbxdq2Jfx0s3Xy5o%2Fimage.png?alt=media&#x26;token=7d458c60-3559-4a72-ba03-1cc0e1bcfedd" alt=""><figcaption></figcaption></figure>

## Alerts roadmap

1. High task error rate
2. High executors error rate
3. High disk spill relative to input size and available memory.
4. repartition before write with low cardinality that causes lack of parallelism or huge files.
5. Executor memory overhead is too low and causes container failure

