AWS & Kubernetes: The Definitive Guide to Data Persistence with PV and PVC

Mervin Praison · Beginner ·☁️ DevOps & Cloud ·2y ago

Key Takeaways

This video provides a comprehensive guide to data persistence in Kubernetes using AWS, covering the concepts of Persistent Volume (PV) and Persistent Volume Claim (PVC), and demonstrating how to set up and manage persistent storage in a Kubernetes cluster.

Full Transcript

now we are going to see about persistent storage for kubernetes this is where persistent volume and persistent volume claim terminology are used let's get started when you see data persistence in kubernetes whenever an application runs and stores the data in the port it is not permanent as soon as the Pod get deleted the data gets deleted so how can we retain the data let's take an example MySQL data you are saving data to the database and you want those data to persist but when the bot get destroyed data gets destroyed so how can we solve this issue that's when this persistent volume comes let's see in this picture how the persistent volume and persistent volume claim works how can the Pod attain a storage that can be permanent firstly we need to create a storage in AWS it could be EFS or EBS EFS elastic file system generally used for multi-read multi-rut elastic Block store generally used for read write once which means one board can write only one but if it's EFS multi rights or multi-read that time a multiple ports can write the same time so we create either EFS or EBS in AWS then what is PV what is persistent volume persistent volume is something that to let kubernetes know there is a volume exist there in the cloud using persistent volume we inform kubernetes here is the volume using persistent volume claim the Pod request the persistent volume the way the Pod gets the volume is by submitting a request which is called persistent volume claim this is a quick overview now we are going to dive into the deep examples first step how we can create a elastic file store elastic file system in elastic file system just go to the dashboard and fill all those details at the end you will get a file system ID that's the first step second step is that in kubernetes you create a persistent volume in the persistent volume you mention the file system ID which we got on the previous step this is how you let kubernetes know that there is a volume exist there the third step is that you create a persistent volume claim using persistent volume claim the Pod requests or it binds to the persistent volume it claims the volume that it want to use if you see a Port you can clearly see that EFS claim you mentioned the name of the persistent volume claim there this is how everything is linked support links to a person volume claim which is EFS claim in the EFS claim is the name of the personal volume claim and this request is made to the person and the persistent volume will link back to the file system right this is how the link works this is the static provisioning but we want Dynamic what are the advantages of having a dynamic it helps us to Auto scale it helps us to reuse the storage so the way we can use the dynamic storage is that you need to define a storage class in this example we use storage class and same like the first step where we create the file system you will get the file system number as well you assign that to the storage class and you give a name to the storage class in this way whenever you want to claim for the persistent volume you can mention the storage class there in that way it's more Dynamic and you can request persistent volume claim multiple times to that file system if it is the static way you can claim only once to the file system but in this way you can claim multiple times multiple storage to the file system and the persistent volume claim can be multiple times assigned to the storage Clause report then links back to the persistent volume claim this is how everything is linked you can see EFS claim one and here also you can see efficient that's how everything is linked together so this is a quick overview of the different types of storage static provisioning and also Dynamic frame provisioning and the way the persistent volume claim on the personal volume Works in kubernetes thanks for watching

Original Description

🔥 Welcome to the Ultimate Guide on Persistent Storage in Kubernetes using AWS! 🔥 Are you tired of losing your valuable data every time a Kubernetes Pod gets terminated? Say goodbye to data loss and hello to data persistence! In this comprehensive tutorial, we dive deep into how to set up and manage Persistent Volumes (PV) and Persistent Volume Claims (PVC) in a Kubernetes cluster hosted on AWS. 👇 What You'll Learn 👇 The basics of data persistence in Kubernetes How to set up AWS Elastic File System (EFS) and Elastic Block Store (EBS) Creating and linking Persistent Volumes in Kubernetes to AWS storage How to use Persistent Volume Claims to bind your Pods to the right storage The benefits of dynamic provisioning through Storage Classes 🛠 Key Takeaways 🛠 Understand the difference between EFS and EBS in AWS Master the art of creating and managing PV and PVC in Kubernetes Learn how to dynamically provision storage for scalability and reusability 🔗 Useful Links 🔗 📌 Hashtags 📌 #Kubernetes #AWS #DataPersistence #PersistentVolumes #PersistentVolumeClaims #EFS #EBS #DevOps #CloudComputing #StorageSolutions 👍 Don't forget to like, share, and subscribe for more awesome tutorials on Kubernetes and AWS! 👍 Optimize your Kubernetes deployments by mastering data persistence with AWS. Click the video now to get started! 🎥 ⏰ Time Stamps: 00:00 - Introduction 00:08 - Basics of Data Persistence in Kubernetes 00:29 - Static Storage 00:39 - AWS Storage Options: EFS vs EBS 01:03 - Persistent Volume 01:14 - Persistent Volume Claim 01:27 - 1. Setting up AWS Elastic File System (EFS) 01:38 - 2. Setting up Persistent Volume 01:48 - 3. Setting up Persistent Volume Claim 01:59 - 4. Setting up Pod 02:24 - Dynamic Storage 02:30 - 1. Setting up Storage Class 02:46 - 2. Setting up Persistent Volume Claim 03:11 - 3. Setting up Pod 03:20 - Conclusion
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Mervin Praison · Mervin Praison · 5 of 60

1 Build GCP Infra using Pulumi in YAML format
Build GCP Infra using Pulumi in YAML format
Mervin Praison
2 How to Convert a Pulumi YAML File to Python Format
How to Convert a Pulumi YAML File to Python Format
Mervin Praison
3 Speed Up AWS EKS: A Complete Guide to Performance Tuning & Debugging!
Speed Up AWS EKS: A Complete Guide to Performance Tuning & Debugging!
Mervin Praison
4 Learn GCP GKE to AWS EKS Migration in Just 5 Minutes: Quick Guide
Learn GCP GKE to AWS EKS Migration in Just 5 Minutes: Quick Guide
Mervin Praison
AWS & Kubernetes: The Definitive Guide to Data Persistence with PV and PVC
AWS & Kubernetes: The Definitive Guide to Data Persistence with PV and PVC
Mervin Praison
6 ChatGPT Voice Conversation RELEASED! It's AMAZING!! (Demo)
ChatGPT Voice Conversation RELEASED! It's AMAZING!! (Demo)
Mervin Praison
7 How to Install Mistral 7B in Minutes: Quick & Easy Guide! ✅
How to Install Mistral 7B in Minutes: Quick & Easy Guide! ✅
Mervin Praison
8 Code Llama Install Locally: 🐍💻 Elevate Your Python Skills!
Code Llama Install Locally: 🐍💻 Elevate Your Python Skills!
Mervin Praison
9 Orca Mini: Your Ultimate Guide to Install and Test on Mac & Linux 💻
Orca Mini: Your Ultimate Guide to Install and Test on Mac & Linux 💻
Mervin Praison
10 Quick & Easy Vicuna Setup on Mac and Linux 💻
Quick & Easy Vicuna Setup on Mac and Linux 💻
Mervin Praison
11 Quick Guide: Llama2 Local Installation and ChatGPT with pip! Python🛠️
Quick Guide: Llama2 Local Installation and ChatGPT with pip! Python🛠️
Mervin Praison
12 Query PDFs Like a Pro with Local GPT: Full Setup Guide! 📜
Query PDFs Like a Pro with Local GPT: Full Setup Guide! 📜
Mervin Praison
13 LM Studio: EASIEST way to Run Large Language Models Locally!
LM Studio: EASIEST way to Run Large Language Models Locally!
Mervin Praison
14 AMAZING ChatGPT Vision is OUT! 🤯 14+ Examples (Step-by-Step) FULL Tutorial
AMAZING ChatGPT Vision is OUT! 🤯 14+ Examples (Step-by-Step) FULL Tutorial
Mervin Praison
15 Unbelievable! Build ANY App Instantly with Smol AI! 😲🔥
Unbelievable! Build ANY App Instantly with Smol AI! 😲🔥
Mervin Praison
16 Amazing! AutoGen Made Easy: A Step-by-Step Beginners Guide 📚
Amazing! AutoGen Made Easy: A Step-by-Step Beginners Guide 📚
Mervin Praison
17 How to Set Up LoLLMS and Run LLMs Locally! 🚀 Step-by-Step Tutorial
How to Set Up LoLLMS and Run LLMs Locally! 🚀 Step-by-Step Tutorial
Mervin Praison
18 GPT4All: INSANE Way to Run Large Language Models Locally! 😲 Step-By-Step Tutorial
GPT4All: INSANE Way to Run Large Language Models Locally! 😲 Step-By-Step Tutorial
Mervin Praison
19 Incredible AI-Powered NPCs in Unity Game Engine: Step by Step Tutorial!🤯
Incredible AI-Powered NPCs in Unity Game Engine: Step by Step Tutorial!🤯
Mervin Praison
20 MemGPT 🧠 LLM as Operating System. It's INSANE! Step-by-Step Tutorial 🤯
MemGPT 🧠 LLM as Operating System. It's INSANE! Step-by-Step Tutorial 🤯
Mervin Praison
21 Text Generation Web UI: MIND-BLOWING Way to Run LLM Locally! 🤯
Text Generation Web UI: MIND-BLOWING Way to Run LLM Locally! 🤯
Mervin Praison
22 Unlock the INSANE Power of OpenAI GPT-4 with C#/.NET! 😲
Unlock the INSANE Power of OpenAI GPT-4 with C#/.NET! 😲
Mervin Praison
23 Integrate Langchain and Ollama for Local AI Power 🤯 Indeed POWERFUL!
Integrate Langchain and Ollama for Local AI Power 🤯 Indeed POWERFUL!
Mervin Praison
24 ChatDev: INSANE Virtual AI Agents! Future of Software Development 😲
ChatDev: INSANE Virtual AI Agents! Future of Software Development 😲
Mervin Praison
25 Query PDFs Using Mistral: Unlock INSANE Power! 🤯
Query PDFs Using Mistral: Unlock INSANE Power! 🤯
Mervin Praison
26 AutoGen + Open-Source LLMs: UNBELIEVABLE! Step-by-Step Tutorial You Can't Miss! 🤯
AutoGen + Open-Source LLMs: UNBELIEVABLE! Step-by-Step Tutorial You Can't Miss! 🤯
Mervin Praison
27 AutoGen + Text Generation WebUI: Unbelievable 100% Local Private Setup 🤯
AutoGen + Text Generation WebUI: Unbelievable 100% Local Private Setup 🤯
Mervin Praison
28 MemGPT: Amazing! External Context for LLM #ai #llm #memgpt  #generativeai #mem #gpt #openai #chatgpt
MemGPT: Amazing! External Context for LLM #ai #llm #memgpt #generativeai #mem #gpt #openai #chatgpt
Mervin Praison
29 GeniA: Kubernetes + AI for MIND-BLOWING Operational Efficiency! 🤯 FULL Tutorial
GeniA: Kubernetes + AI for MIND-BLOWING Operational Efficiency! 🤯 FULL Tutorial
Mervin Praison
30 VertexAI Meets LangChain for Mind-Blowing AI Conversations! 😲 Step by Step Tutorial
VertexAI Meets LangChain for Mind-Blowing AI Conversations! 😲 Step by Step Tutorial
Mervin Praison
31 Simplified ChatGPT API Setup on Node.js for Newbies! 😍 Step by Step Tutorial
Simplified ChatGPT API Setup on Node.js for Newbies! 😍 Step by Step Tutorial
Mervin Praison
32 Autogen: Ollama integration 🤯 Step by Step Tutorial. Mind-blowing!
Autogen: Ollama integration 🤯 Step by Step Tutorial. Mind-blowing!
Mervin Praison
33 LiteLLM: One-Function Call to ANY Large Language Model! 🤯 UNBELIEVABLE!
LiteLLM: One-Function Call to ANY Large Language Model! 🤯 UNBELIEVABLE!
Mervin Praison
34 ChatGPT Chatbot in Less Time Than You Think! 🚀😎 Step-by-Step Tutorial
ChatGPT Chatbot in Less Time Than You Think! 🚀😎 Step-by-Step Tutorial
Mervin Praison
35 LiteLLM Chatbot: Build Your Own in MINUTES! INSANE! 🤖🔥
LiteLLM Chatbot: Build Your Own in MINUTES! INSANE! 🤖🔥
Mervin Praison
36 Create Chatbot: Turn ANY Open-Source LLM into a Conversation Pro! 🤖
Create Chatbot: Turn ANY Open-Source LLM into a Conversation Pro! 🤖
Mervin Praison
37 Create Chatbot: Ollama Integration Made UNBELIEVABLY Easy! 🎉
Create Chatbot: Ollama Integration Made UNBELIEVABLY Easy! 🎉
Mervin Praison
38 LlamaIndex + ChatGPT: Ingest Data and Experience UNBELIEVABLE Query Results! 🌟
LlamaIndex + ChatGPT: Ingest Data and Experience UNBELIEVABLE Query Results! 🌟
Mervin Praison
39 INSANE! OpenAgents: Automated Data Analysis with Kaggle 🤯
INSANE! OpenAgents: Automated Data Analysis with Kaggle 🤯
Mervin Praison
40 React.js LLM Agent for Next-Gen Coding using ChatGPT 🚀 Mind-Blowing 🤯
React.js LLM Agent for Next-Gen Coding using ChatGPT 🚀 Mind-Blowing 🤯
Mervin Praison
41 MemGPT + Any LLM 🚀 100% Local & Private Integration Unveiled! Unlimited Memory
MemGPT + Any LLM 🚀 100% Local & Private Integration Unveiled! Unlimited Memory
Mervin Praison
42 MemGPT  + AutoGen 🧠🤖 Unlimited Memory & Autonomous AI Agents! INSANE🤯
MemGPT + AutoGen 🧠🤖 Unlimited Memory & Autonomous AI Agents! INSANE🤯
Mervin Praison
43 AutoGen + Google's Palm LLM & More! Revolutionary AI Integration 🚀
AutoGen + Google's Palm LLM & More! Revolutionary AI Integration 🚀
Mervin Praison
44 MemGPT & LM Studio Integration Revealed! 🔥 Next-Level AI
MemGPT & LM Studio Integration Revealed! 🔥 Next-Level AI
Mervin Praison
45 🚀 AutoLLM: Unlock the Power of 100+ Language Models! Step-by-Step Tutorial
🚀 AutoLLM: Unlock the Power of 100+ Language Models! Step-by-Step Tutorial
Mervin Praison
46 AutoLLM & Gradio Integration You Won't Believe! 🤯 Mind-Blowing
AutoLLM & Gradio Integration You Won't Believe! 🤯 Mind-Blowing
Mervin Praison
47 AutoLLM & FastAPI Tutorial: Query 100+ Language Models! 😱
AutoLLM & FastAPI Tutorial: Query 100+ Language Models! 😱
Mervin Praison
48 Quivr: LLM's Second Brain - Transforming Data Management & Advanced Query with AI! 🤯
Quivr: LLM's Second Brain - Transforming Data Management & Advanced Query with AI! 🤯
Mervin Praison
49 AutoGen & MemGPT with Local LLM: A Complete Setup Tutorial! 🧠 AMAZING 🤯
AutoGen & MemGPT with Local LLM: A Complete Setup Tutorial! 🧠 AMAZING 🤯
Mervin Praison
50 LocalAI: Free, Open Source OpenAI Alternative 🚀 INSANE 🤯 Step-by-Step Tutorial
LocalAI: Free, Open Source OpenAI Alternative 🚀 INSANE 🤯 Step-by-Step Tutorial
Mervin Praison
51 Yarn Mistral 7B 128k LARGE context window, Small size 🤯 INSANE 🚀 Setup Tutorial!
Yarn Mistral 7B 128k LARGE context window, Small size 🤯 INSANE 🚀 Setup Tutorial!
Mervin Praison
52 Zephyr-7B: The Small and Mighty LLM 🤯 Step by Step Tutorial! 📘
Zephyr-7B: The Small and Mighty LLM 🤯 Step by Step Tutorial! 📘
Mervin Praison
53 Promptfoo: How to Test Your LLM ? 🚀  VERY EASY!
Promptfoo: How to Test Your LLM ? 🚀 VERY EASY!
Mervin Praison
54 Pydantic: How to Validate LLM Responses? 🚀 Quality Response. VERY EASY!!!!
Pydantic: How to Validate LLM Responses? 🚀 Quality Response. VERY EASY!!!!
Mervin Praison
55 Pydantic: FORCE Your AI to Respond Back in UPPERCASE! 🤯 Step-by-Step Tutorial 🔥
Pydantic: FORCE Your AI to Respond Back in UPPERCASE! 🤯 Step-by-Step Tutorial 🔥
Mervin Praison
56 Pydantic: How to use LLM to convert unstructured data to structured data?
Pydantic: How to use LLM to convert unstructured data to structured data?
Mervin Praison
57 AutoGen Function Calling: INSANE 🚀 Custom Integrations! Step-by-Step Tutorial 🤯
AutoGen Function Calling: INSANE 🚀 Custom Integrations! Step-by-Step Tutorial 🤯
Mervin Praison
58 OpenAI Assistants API + Python 🤖 How to get started? (FULL Tutorial) 🤯 INSANE
OpenAI Assistants API + Python 🤖 How to get started? (FULL Tutorial) 🤯 INSANE
Mervin Praison
59 GPT-4 Vision API 🤯 INSANE Video Recognition Powers! Step-by-Step Tutorial 🚀
GPT-4 Vision API 🤯 INSANE Video Recognition Powers! Step-by-Step Tutorial 🚀
Mervin Praison
60 GPT-4 Vision API 🚀 The Future of Image Recognition! 🤯 Step-by-Step Tutorial
GPT-4 Vision API 🚀 The Future of Image Recognition! 🤯 Step-by-Step Tutorial
Mervin Praison

This video teaches viewers how to set up and manage persistent storage in a Kubernetes cluster using AWS, covering the concepts of PV and PVC, and demonstrating dynamic and static provisioning methods.

Key Takeaways
  1. Create an EFS or EBS storage in AWS
  2. Create a Persistent Volume in Kubernetes
  3. Create a Persistent Volume Claim in Kubernetes
  4. Define a Storage Class for dynamic provisioning
  5. Claim a Persistent Volume using the Storage Class
💡 Using Persistent Volumes and Persistent Volume Claims allows for data persistence in Kubernetes, even when pods are terminated or recreated.

Related AI Lessons

Chapters (15)

Introduction
0:08 Basics of Data Persistence in Kubernetes
0:29 Static Storage
0:39 AWS Storage Options: EFS vs EBS
1:03 Persistent Volume
1:14 Persistent Volume Claim
1:27 1. Setting up AWS Elastic File System (EFS)
1:38 2. Setting up Persistent Volume
1:48 3. Setting up Persistent Volume Claim
1:59 4. Setting up Pod
2:24 Dynamic Storage
2:30 1. Setting up Storage Class
2:46 2. Setting up Persistent Volume Claim
3:11 3. Setting up Pod
3:20 Conclusion
Up next
Containers on Amazon ECS with Mama J
AWS Developers
Watch →