Leveraging table partitioning for query performance and data archiving | POSETTE 2025
Video of a conference talk about leveraging table partitioning for query performance and data archiving. Table partitioning has become a known technique in the PostgreSQL world. There has been a talk about it at most of the conferences I visited last year. But often we talk about the technicalities of partitioning and not why we partition.
This talk will be about why table partitioning can be done. We use it for PII compliance, improving query performance by using enable_partitionwise_join and enable_partitionwise_aggregate and transparent data archival with respect to the application. Especially the data archival part is something I am really proud of.
All these techniques may sound easy, but there is a lot of ground to cover before it works to perfection. Think about finding your leading figure, denormalise tables and aligning partition boundaries. It can be a lot of work but we managed to improve query performance over 50 times for some queries!
And with all the partitioning in place we can start moving data out of our primary database into a secondary archival system without the application being aware of it. Now our mean PostgreSQL database becomes smaller, easier to manage and even faster.
Derk van Veen is a database specialist at Adyen. Adyen has some very large PostgreSQL databases under continuous high pressure. An important part of his job is to find creative solutions for tough problems. Whenever he, or his team, finds interesting solutions he tries to share the lessons learned with the PostgreSQL community.
► Video bookmarks:
⏩ 00:00 Introduction
⏩ 02:42 3 Reasons to Partition
⏩ 04:31 3 Types of Partitioning
⏩ 07:41 Date Range Partitioning
⏩ 10:32 Integer Partitioning
⏩ 13:28 Joining Tables
⏩ 16:17 Align your partitioning around a leading figure
⏩ 19:15 UUIDv7 (Derk’s new favorite data type)
⏩ 21:29 Data Lifecycle Management
⏩ 24:26 Summary
📕 Everything you need to know about POSETTE: An Event for Postgres 2025 can be fo
Watch on YouTube ↗
(saves to browser)
Sign in to unlock AI tutor explanation · ⚡30
Playlist
Uploads from Microsoft Developer · Microsoft Developer · 0 of 60
← Previous
Next →
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Prepare for the DP-300 exam & the Azure Database Administrator Associate cert | Data Exposed
Microsoft Developer
What I Wish I Knew ... about landing a job in tech
Microsoft Developer
Igniting Developer Innovation with Vector Search
Microsoft Developer
Combining the power of vector search with Azure OpenAI then revolutionize image search with vectors!
Microsoft Developer
What I Wish I Knew ... about finding your place in tech
Microsoft Developer
Fluent UI React Insights: Accessible by default
Microsoft Developer
Signing Container Images with Notary Project
Microsoft Developer
What I Wish I Knew ... about finding your place in tech
Microsoft Developer
What programming languages does GitHub Copilot support?
Microsoft Developer
What I Wish I Knew ... about how much your job can change
Microsoft Developer
What I Wish I Knew ... about how much your job can change
Microsoft Developer
How do I become more confident about AI?
Microsoft Developer
How do I become more confident about AI?
Microsoft Developer
Performance Demos of SQL’s Intelligent Query Processing Feedback capabilities | Data Exposed
Microsoft Developer
What I Wish I Knew ... about coming to Microsoft
Microsoft Developer
What I Wish I Knew ... about coming to Microsoft
Microsoft Developer
Revolutionizing Image Search with Vectors
Microsoft Developer
Igniting developer innovation with Vector search and Azure OpenAI
Microsoft Developer
Getting Started with Azure AI Studio's Prompt Flow - Part 2
Microsoft Developer
What I Wish I Knew ... about finding my career path
Microsoft Developer
What I Wish I Knew ... about finding my career path
Microsoft Developer
Windows Terminal's journey to Open Source
Microsoft Developer
Can I trust the code that GitHub Copilot generates?
Microsoft Developer
What I Wish I Knew ... about interviewing
Microsoft Developer
What I Wish I Knew ... about interviewing
Microsoft Developer
What is the Microsoft TechSpark Program?
Microsoft Developer
SQL Server 2022: Accelerate query performance while reducing query compile time - w/ no code changes
Microsoft Developer
What I Wish I Knew ... about discovering computer science
Microsoft Developer
What I Wish I Knew ... about discovering computer science
Microsoft Developer
Call center transcription and analysis using Azure AI
Microsoft Developer
How to use Text Analytics for health in Azure AI Language
Microsoft Developer
Azure OpenAI-powered summarization in Azure AI Language
Microsoft Developer
Accelerate data labeling using Azure OpenAI and Azure AI Language
Microsoft Developer
Building a Private ChatGPT with Azure OpenAI
Microsoft Developer
What I Wish I Knew ... about how to interview
Microsoft Developer
What I Wish I Knew ... about how to interview
Microsoft Developer
Getting Started with Azure AI Studio's Prompt Flow - Part 3
Microsoft Developer
Intelligent Apps with Azure Kubernetes Service (AKS)
Microsoft Developer
Getting Started with Azure Blob Storage | Data Exposed: MVP Edition
Microsoft Developer
Chat + Your Data + Plugins
Microsoft Developer
What I Wish I Knew ... about different career paths
Microsoft Developer
What I Wish I Knew ... about different career paths
Microsoft Developer
Advanced Dev Tunnels Features | OD122
Microsoft Developer
Learn Live - Manage performance and availability in Azure Cosmos DB for PostgreSQL
Microsoft Developer
Plan your SQL Migration to Azure with confidence | Data Exposed
Microsoft Developer
What I Wish I Knew ... about social skills in a tech career
Microsoft Developer
What I Wish I Knew ... about social skills in a tech career
Microsoft Developer
All About Vectors, Search, and Function Calling in Azure OpenAI - Labor Day Special
Microsoft Developer
Introduction to project ORAS
Microsoft Developer
What I Wish I Knew ... about finding the right major
Microsoft Developer
What I Wish I Knew ... about finding the right major
Microsoft Developer
What I Wish I Knew ... about how to approach programming
Microsoft Developer
What I Wish I Knew ... about how to approach programming
Microsoft Developer
Learn Live - Scale from a single node to multiple nodes with Azure Cosmos DB for PostgreSQL
Microsoft Developer
What I Wish I Knew ... about diversity in tech #1
Microsoft Developer
What I Wish I Knew ... about diversity in tech #1
Microsoft Developer
Get started with SQL Server AGs across Windows, Linux and Container Replicas | Data Exposed
Microsoft Developer
Writing LLM Apps with Azure AI and PromptFlow
Microsoft Developer
What I Wish I Knew ... about how cool working in tech could be
Microsoft Developer
Open Source foundation models in Azure Machine Learning & optimization techniques behind the scenes
Microsoft Developer
Related AI Lessons
⚡
⚡
⚡
⚡
Neobank Monzo Builds Governed Data Mesh Across 100 Teams and 12000 dbt Models
InfoQ AI/ML
The Complete History of Nasdaq Stock Market Returns
Dev.to AI
Operational Philosophy of Digital Entities
Medium · Data Science
Backtesting an ICT strategy at 184 speed: timezone-cache + bisect lookup
Dev.to · Marco Messina
🎓
Tutor Explanation
DeepCamp AI