Castle On The Hill (Ed Sheeran) - AI Animation from Lyrics

Prompt Engineer · Intermediate ·🎨 Image & Video AI ·3y ago

Key Takeaways

The video showcases an AI animation of Ed Sheeran's song Castle On The Hill, created using Stable Diffusion, Deforum Diffusion, and Waifu2x.

Original Description

Leave a suggestion in the comments for my next song to animate! I’m releasing new videos every week again, subscribe if you’d like to see more. Attributions: I created all the visuals myself, but the audio is entirely attributed to Ed Sheeran. This would not have been possible without the amazing open-source project Stable Diffusion https://github.com/CompVis/stable-diffusion And another amazing open-source extension Deforum Diffusion https://github.com/HelixNGC7293/DeforumStableDiffusionLocal Upscaling was done with Waifu2x, specifically with this amazing GUI https://github.com/AaronFeng753/Waifu2x-Extension-GUI The font is “Caviar Dreams” by Lauren Thompson https://www.dafont.com/caviar-dreams.font #edsheeran #aiart #musicvideo
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Prompt Engineer · Prompt Engineer · 29 of 35

1 Lemon Tree - An AI's Dream
Lemon Tree - An AI's Dream
Prompt Engineer
2 The Emerald Sword - Unofficial music video made with Stable Diffusion AI
The Emerald Sword - Unofficial music video made with Stable Diffusion AI
Prompt Engineer
3 Brandy (You're a fine girl) - But every lyric is an AI generated image
Brandy (You're a fine girl) - But every lyric is an AI generated image
Prompt Engineer
4 Infinity - Jaymes Young (But every lyric is an AI animation) - 4K
Infinity - Jaymes Young (But every lyric is an AI animation) - 4K
Prompt Engineer
5 Come Fly With Me - But every lyric is an AI generated video
Come Fly With Me - But every lyric is an AI generated video
Prompt Engineer
6 Dancing in the Moonlight - Unofficial Music Video (AI Generated Animation)
Dancing in the Moonlight - Unofficial Music Video (AI Generated Animation)
Prompt Engineer
7 Ed Sheeran, Pokémon - Celestial [AI Animated Lyric Video]
Ed Sheeran, Pokémon - Celestial [AI Animated Lyric Video]
Prompt Engineer
8 Last Christmas - Unofficial Music Video (Stable Diffusion Animation)
Last Christmas - Unofficial Music Video (Stable Diffusion Animation)
Prompt Engineer
9 Without Me (Eminem) - Unofficial Animated Music Video
Without Me (Eminem) - Unofficial Animated Music Video
Prompt Engineer
10 INSANE - Unofficial Music video (Stable Diffusion Animation)
INSANE - Unofficial Music video (Stable Diffusion Animation)
Prompt Engineer
11 Wellerman (Sea Shanty) - Unofficial Animated Music Video
Wellerman (Sea Shanty) - Unofficial Animated Music Video
Prompt Engineer
12 The Evolution of Art
The Evolution of Art
Prompt Engineer
13 Believer (Imagine Dragons) - Unofficial Animated Music Video
Believer (Imagine Dragons) - Unofficial Animated Music Video
Prompt Engineer
14 Overwhelmed - Unofficial Animated Music Video
Overwhelmed - Unofficial Animated Music Video
Prompt Engineer
15 The Ultimate Showdown of Ultimate Destiny - AI Animation from Lyrics
The Ultimate Showdown of Ultimate Destiny - AI Animation from Lyrics
Prompt Engineer
16 Blur (Stellar) - AI Animation From Song Lyrics
Blur (Stellar) - AI Animation From Song Lyrics
Prompt Engineer
17 Darude - Sandstorm 10 Hour Seamless Loop (Stable Diffusion Animation)
Darude - Sandstorm 10 Hour Seamless Loop (Stable Diffusion Animation)
Prompt Engineer
18 Hell's Comin' With Me - Poor Man's Poison (Stable Diffusion Animation)
Hell's Comin' With Me - Poor Man's Poison (Stable Diffusion Animation)
Prompt Engineer
19 Let it Snow! (Dean Martin) - Stable Diffusion Animation
Let it Snow! (Dean Martin) - Stable Diffusion Animation
Prompt Engineer
20 Miracle - Caravan Palace (Stable Diffusion Animation)
Miracle - Caravan Palace (Stable Diffusion Animation)
Prompt Engineer
21 Everything Matters - Aurora (Stable Diffusion Animation)
Everything Matters - Aurora (Stable Diffusion Animation)
Prompt Engineer
22 Heat Waves -  AI Music Animation (Vaporwave Aesthetic)
Heat Waves - AI Music Animation (Vaporwave Aesthetic)
Prompt Engineer
23 All Star - Smashmouth (Stable Diffusion Animation)
All Star - Smashmouth (Stable Diffusion Animation)
Prompt Engineer
24 Ghost Division - Sabaton (AI Music Video Animation)
Ghost Division - Sabaton (AI Music Video Animation)
Prompt Engineer
25 Sunflowers - Pogo - Unofficial Music Video
Sunflowers - Pogo - Unofficial Music Video
Prompt Engineer
26 The Race For Space - Public Service Broadcasting (AI Music Video)
The Race For Space - Public Service Broadcasting (AI Music Video)
Prompt Engineer
27 Flowers - Miley Cyrus (Animated Music Video)
Flowers - Miley Cyrus (Animated Music Video)
Prompt Engineer
28 Crazy - Gnarls Barkley (AI Animation)
Crazy - Gnarls Barkley (AI Animation)
Prompt Engineer
Castle On The Hill (Ed Sheeran) - AI Animation from Lyrics
Castle On The Hill (Ed Sheeran) - AI Animation from Lyrics
Prompt Engineer
30 How Far I'll Go (From Moana) - AI Animated Music Video
How Far I'll Go (From Moana) - AI Animated Music Video
Prompt Engineer
31 Counting Stars (OneRepublic) - Animated Music Video
Counting Stars (OneRepublic) - Animated Music Video
Prompt Engineer
32 Viva La Vida (Coldplay) - AI Music Video
Viva La Vida (Coldplay) - AI Music Video
Prompt Engineer
33 Green Day - Boulevard of Broken Dreams [Animated]
Green Day - Boulevard of Broken Dreams [Animated]
Prompt Engineer
34 Baby I'm Yours [AI Animated]
Baby I'm Yours [AI Animated]
Prompt Engineer
35 Come Alive Diana (Everything Everything) - AI Music Video
Come Alive Diana (Everything Everything) - AI Music Video
Prompt Engineer

This video demonstrates how to create an AI animation from song lyrics using Stable Diffusion, Deforum Diffusion, and Waifu2x. The creator showcases their process and provides links to the open-source projects used.

Key Takeaways
  1. Choose a song with meaningful lyrics
  2. Use Stable Diffusion to generate images based on the lyrics
  3. Utilize Deforum Diffusion to create an animation
  4. Upscale the animation using Waifu2x
  5. Add text overlays using a font like Caviar Dreams
💡 The combination of Stable Diffusion, Deforum Diffusion, and Waifu2x can be used to create high-quality AI animations from song lyrics.

Related Reads

📰
6 Cinematic Image Prompts That Don’t Look AI-Generated
Learn to create cinematic image prompts that don't look AI-generated and improve your image generation skills
Medium · ChatGPT
📰
Building the Next Generation of Synthetic Media – OpenCV Live Ep. 218
Learn how generative AI is revolutionizing synthetic media creation, including digital humans and video content, and discover tools like Avatar SDK
OpenCV Blog
📰
Why Diffusion Transformers (DiTs) Are Replacing U-Nets in Generative AI
Learn why Diffusion Transformers are replacing U-Nets in Generative AI and how they improve image generation
Medium · AI
📰
Why Diffusion Transformers (DiTs) Are Replacing U-Nets in Generative AI
Learn why Diffusion Transformers are replacing U-Nets in Generative AI and how they improve image generation
Medium · Deep Learning
Up next
How To Use ChatGPT Image Generator - Easy Way to Get Best Results!
AI Andy
Watch →