A company has built a data pipeline using Snowpipe to ingest files from an Amazon S3 bucket. Snowpipe is configured to load data into staging database tables. Then a task runs to load the data from the staging database tables into the reporting database tables.
The company is satisfied with the availability of the data in the reporting database tables, but the reporting tables are not pruning effectively. Currently, a size 4X-Large virtual warehouse is being used to query all of the tables in the reporting database.
What step can be taken to improve the pruning of the reporting tables?
Effective pruning in Snowflake relies on the organization of data within micro-partitions. By using an ORDER BY clause with clustering keys when loading data into the reporting tables, Snowflake can better organize the data within micro-partitions. This organization allows Snowflake to skip over irrelevant micro-partitions during a query, thus improving query performance and reducing the amount of data scanned12.
Reference =
* Snowflake Documentation on micro-partitions and data clustering2
* Community article on recognizing unsatisfactory pruning and improving it1
Chandra
9 months agoLatrice
9 months agoKrystina
9 months agoSalena
10 months agoBecky
10 months agoBarrett
10 months agoHubert
10 months agoMelissa
10 months agoAlisha
11 months agoReita
11 months agoCharisse
11 months agoMan
11 months agoAudry
11 months agoZona
11 months agoDominga
11 months agoPolly
2 years agoSina
2 years agoYolando
2 years agoCarissa
2 years agoGladys
2 years agoAnnalee
2 years agoCasey
2 years agoDenna
2 years agoVivan
2 years agoEmile
2 years agoRozella
2 years agoRolande
2 years agoMyong
2 years agoStaci
2 years agoMargurite
2 years agoOliva
2 years agoPolly
2 years agoJessenia
2 years agoAlica
2 years agoMohammad
2 years agoMitsue
2 years agoWei
2 years ago