✨Our Features
Last updated
Last updated
See metrics on the current SQL query being run, including node-level granularity at real time
Get performance summary of your run, including the option to check which query used the most resources.
SQL queries or performance metrics with performance alerts are being marked with a warning icon, just like a warning light in your car's dashboard.
Measure your spark resource usage with a graph of executors over time, and the relevant resource configuration for your allocation mode
In case of SQL that failed, hover on it to see the exception, click to copy the error to clipboard
In the summary screen you can choose a query to see a visualized plan, including alerts on specific query nodes
Press the blue down arrow on a node to see stage information such as distribution of tasks duration/input/output in the stage
See which part of your SQL is taking the longest, hover on precentage to see how much compute time the part took
This feature is WIP, and might not identify the node's right duration. It also not accurate with very fast SQL queries Some nodes are running at the same stage (such in the example filtering and selecting) so the same percentage is shown to both of them Please raise any issue/inconsistency with this feature
You can switch to advanced mode to see repartitions, sorting and broadcasting.
Or you can switch to "Only IO" mode to see only input, output and joins
Hover on long SQL expressing to see them with syntax highlighting
Get alerts on performance issues, including suggestions for fixes with specific config/code changes suggestions!
DataFlint knows to collect iceberg write metrics that does not exist in spark, and alert in cases of performance issues when using functionality like update/delete/merge operators
Require adding the iceberg metric reporter, see Apache Iceberg for more details
