optimiz 101

skills/tinybird-best-practices/rules/endpoint-optimization.md

@@ -10,7 +10,7 @@ Use this checklist when optimizing endpoints.
 ## Step 2: 5-Question Diagnostic
 
 1) Aggregations at query time?
-- Fix: Move to materialized views or ingestion-time transforms.
+- Fix: Move to materialized views when possible, to snapshots (copy pipes) or lambda architecture if MVs do not fit.
 
 2) Filters match sorting keys?
 - Fix: Include frequent filters in ENGINE_SORTING_KEY; order by selectivity.
@@ -36,6 +36,10 @@ Use this checklist when optimizing endpoints.
 - Track tinybird.pipe_stats_rt and tinybird.pipe_stats.
 - Success metrics: lower latency, lower read_bytes, improved read_bytes/write_bytes.
 
+## Query Explain
+
+- For more details, call the endpoint with explain=true parameter to understand the query plan. E.g: https://$TB_HOST/v0/pipes/endpoint_name?explain=true
+
 ## Templates
 
 Materialized view:

skills/tinybird-best-practices/rules/materialized-files.md

@@ -34,3 +34,10 @@ ENGINE "AggregatingMergeTree"
 ENGINE_PARTITION_KEY "toYYYYMM(date)"
 ENGINE_SORTING_KEY "date, dimension_1, dimension_2"
 ```
+
+## Usual gotchas
+- Materialized Views work as insert triggers, which means a delete or truncate operation on your original Data Source doesn't affect the related Materialized Views.
+
+- As transformation and ingestion in the Materialized View is done on each block of inserted data in the original Data Source, some operations such as GROUP BY, ORDER BY, DISTINCT and LIMIT might need a specific engine, such as AggregatingMergeTree or SummingMergeTree, which can handle data aggregations.
+
+- The Data Source resulting from a Materialized View generated using JOIN is automatically updated only if and when a new operation is performed over the Data Source in the FROM.