Talking AGI with Sam Altman: A Deepfake Showcase

Key Takeaways

hi Fam thank you for joining me today to discuss the future of genetic Ai and its role in the development of artificial general intelligence how do you see genetic AI evolving in the coming years thanks for having me I think generative artificial intelligence will continue to improve in its ability to create realistic and novel outputs such as images music and language we'll also see more applications of generative artificial intelligence in areas like drug Discovery and Materials Science no that's exciting what do you see as the biggest challenges in developing artificial general intelligence one of the biggest challenges is ensuring that the AI systems we create are aligned with human values and goals we also need to address concerns around safety transparency and accountability thank you so much for your Insight Sam you know I really appreciate your perspective on it my pleasure it was great to chat with you today hello everyone welcome to AI anytime Channel today we are going to work on a very interesting project it's a weekend project and it's going to be very interesting we are going to do some super cool thing today we are going to create some sort of defect video okay so first we'll create a deep fake audio with help of custom text to speech models and then we will combine that with uh the sample video that we will download from YouTube or Internet uh and then we'll combine the defect audio on top of that video so we'll use something called wave to LEAP okay which is basically a gan model it's a generative adversarial Network model so we're going to combine all this technique and then try to create a simple defect video so sort of having a conversation with Sam Altman or Elon Musk for example we'll try to you know achieve the results uh so it's not going to be a complete defect video because we do not have that infrastructure right now okay uh in my machine at least so I'm just going to see that how we can combine all these different models so we have Gan models we have large language models you know we have mid Journey For example to generate images if you don't have a sample reference we can combine all these techniques or the models and then try to come up with the output to provide the experiences okay so we basically we're going to play with this technology so you can see I am currently on something called Toto hdds okay which is basically takes to speech model okay and they also have here on GitHub you can see it's it's very uh uh very famous by the way in the community it's it's almost you know uh more than a year old okay and you can see it says a multi-voice TTS system trained with an emphasis on quality so it helps you generate Krishna vices okay so uh it can help you synthesize the voice you know Microsoft has religious either of Vali where you you know you give uh two or three seconds of audio at least and it will create a similar audio okay having a similar voice okay so it looks at the physical properties of sound and then try to generate a similar audio for you so we're just going going to use this tortoise TTS they have they also have a collab notebook where you can you know use this but I'm just going to use replicate uh you can see I'm currently on replicate which also provides you to interact with the API you know suppose if you want to interact with the total htts API you can also do that but after creating a GitHub account you can link your GitHub account with replica by the way you say it says generate speech from text clone voices from MP3 files so we're not going to clone uh Sam Iceman's voice so if you don't know Sam Ashman he is the co-founder of open Ai and you can see I so what I have done I have first downloaded couple of videos of Sam Iceman interviews or podcasts from YouTube and then I extracted the audio out of it using ffmpeg you can also use any other tool a lot of free tools available you can see I am currently on this tool which is MP3 to web image to video there are a lot of tools available if your data is not confidential of course you can use this tool okay but if your data is confidential you know you don't want to upload your data on the cloud or you don't want to send this data to them you can use ffmpeg that I have already used over here okay so I'll give the command in the description that how you can extract an audio file from a video for a specific length as well you can trim it you can cut that so you can see this is the uh actual video that I had I cut it perfectly in the video I think you want to look for the intersection of what you're good at what you enjoy and what where is Sam Iceman and there's a 28 second video and then I what I did I also you know extracted the audio out of it so you can okay I think you want to look for the intersection of what yeah and I'm just going to upload this audio here on Tortoise TTS you can see I'm currently on Toto https on replicate by the way you can also use the collab notebook you can use uh qualif3 GPU to perform the same task it works by the uh this is the same way so this is what I'm going to tell you so what I did you know I used share GPT and chat GPT again is acting as a co-founder so I said I'm going to have a conversation with open AI co-founder Sam Altman on the topic future of generative Ai and its role in the artificial general intelligence can you please write a very short script for this so or chat GPT has written a script for me here you can read that okay I it's a kind of a simple conversation so one one I have done I have copied this Sam Altman's part the first response and I have uh I have pasted over here on input section that you see on the text so what let me just explain this so we have a text option text input where you can input your text and then you have a custom voice option now if you don't want a custom wires you just want to go ahead with the default voices there are a lot of default voice available so but I'm going with the custom work and in this custom wires you can see it says create a custom voice based on an MP3 file of a speaker audios will be at least 15 seconds only contains one speaker of course to avoid the uh you know redundancy there or the duplication because it gets confused if you have multiple speakers overwrite The Voice a input so it currently has a voice a here no and it override suppose if you have a default it will overwrite what I'm going to do I'm going to upload this you know from my folder which is under my project conversation with even by The Resort I have to rename the folder name and then I have to admin voice here so what I will do I'll upload this is voice you know and your samples would be at least 15 seconds so mine is 28 second which is more than 15 and I've upload it here and then you have Voice v as well if you suppose you want to go to Voice v you want to you can see it says create new voice from averaging the Left End okay so to find the latent speed it will create an average value of it and then it will uh do the voice mixing but I'm not going with that option I just need this custom wires of you know with uh Sam Aikman and then YC of course and then you have preset now preset in this case that you see it has four different classes Ultra fast to high quality this is basically for you to get uh the uh patterns do you want the fast speed do you want uh standard quality do you want to ultra fast I'll just go with fast which is also a default by the way and then see it for reproducibility I don't want to go ahead because I only need it once but if you want to uh also uh you can also have a look at reproducibility if you are generating the results again and again what I will do this is my take and this is a custom voice that I already uploaded now what I will do is just click submit after clicking on submit in the right hand side you will see the output it's starting so the model in the back end right it started running okay it will spin up and then it will take your custom wire sample you'll see that you know the all the locks will be available here by the way so let's wait for it it can say this can sometimes take around three to five minutes let it uh let it up the model is booting up what we will do after this guys you know after this we will use something called whiff to lip Okay so this is wave to lip a it's an accurately lip sync videos to any space now we if if you see that you're already generating this speech we'll just lip sync with the videos so we already have the video of you know uh Sam Ashman but you can also suppose if you don't have a video what you can do you can also generate an image okay using mid journey of no Sam Ashman provide a sample image and then you generate a image of Sam Iceman and then you convert it to a video of 5 or 10 seconds and then again you can pass it over here on web tool lip but I already have a sample video here you know Eric man cut that you see the fine limb I'll just upload this here and we'll use it so I'm not going to use this interactive demo I'm going to use their collab notebook it says web to lip uh D fake engine engineering dot ip1b okay your different English by the way I don't know what the meaning of Eng in this case it can be either English or engineering okay so I'm going to use this collab notebook to generate a deep fake video of it let's go back to tortoise TTS on replicate and see where are we with that it's still starting up it will take time you know it kind of generates you will see what I'm talking about so so here we are going to use dot OS web to lip and then we are going to uh get an output that will see how close we are with that so if I have to explain that what we are really doing so let me just uh draw it here for you okay so what we are going to do in this case excuse me we have a sample video first what are the things that we need we need a sample video we need an a sample audio as well these are the two requirements that we have Sample video and Sample audio you know in this case we have went ahead with you know we have went ahead with Sam Altman we have his uh video and we have extracted his audio using FFM mpic what now we'll do we'll first have this audio that we have and we pass it to tortoise TTS and it will provide a script text this is our script text it will generate this is basically this clones the voice right then so I'll write clone and generate the speech so this is what we are going to do in the first step so we have an audio and then we are passing it to total htts of course you have to give the script or the text that you have it will then clone and generate the speech so let's go back to Total HD you can see it says running predict it says so it using the predict function of course they will have a predict function and then creating wires from so the file name you know they are storing somewhere in a temporary folder okay I don't know where this backup I think it's maybe hugging face okay I have no idea of it right now okay maybe uh what the back end engine is okay so we are using our replica it says generating Auto regressive samples okay so it's still use Auto regressive air samples it's a generating text using voices you can see we are around 50 of it let me also go ahead and copy the I'll just copy the last one okay I just want to show you the technique now you can extend it further you can create a complete movie or complete podcast you know you create you go ahead create a conversation with Elon Musk or something Adela or anybody else you want to do you can create an entire movie out of it you know with the help of same technique you do a defect podcast why not right using the same technique so I'll just use uh the last one maybe I'll use this uh what do you see the biggest challenges in developing AGI I'll use this Sam admins response by the way uh we create a line with human value we also need to address on the object I'll just use still here I don't want to create a big audio right now at this moment you can see uh this they have uh come they see the logs it says Computing best candidates they might have created several candidates they know looking at the best candidate and then they're generating the speech so wait for it let me also connect this I'll also run this now so I'll just so this link will be given in the description of web 2 lip you know and you can use the default Google collab that the GPU which is taste like a80 I'm not sure which one is and now if you come here and use this install dependency that will install the dependencies download pre-trained models it will download here in the pre-trained models you can say cloning into F2 lit and you can see the web tool leak over here okay it's it's downloading this webtoolift underscore gang dot uh pth okay it's downloading that file over here the back end is by torch again okay it's storing all the weights here in this folder and we have to upload our sample data in this one I will tell you how let's go back to the and you can see we have generated uh 14 seconds audio uh thanks for having Let's uh let's hear this thanks for having me I think generative artificial intelligence will continue to improve in its ability to create realistic and novel outputs such as images music and language we'll also see more applications of generative artificial intelligence in areas like drug Discovery or Material Science this is fantastic also let me download this generative artificial intelligence will continue to improve in its ability to create realizing by the way okay so we have downloaded one audio let me just rename it what I'm going to do now I'm going to rename and I'm going to call it Sam this is the first one let's call this I'll copy this I'll copy this I'll put it in the same folder I will come here on this my projects uh conversation uh with paste I don't know why I I'll name this folder as conversation with Elon maybe I wanted to create first with clear most but then I thought okay I'll create for Sam I'll explain by the way it says cancel because I am opening this now I can rename it uh conversation with Sam yes so you can see now we have one section already we got it right from Total htts let's have one more okay so I'll come back here I will try to change this again and I'll paste it so I'll paste this one one of the biggest challenges accountability and I'll come back over here on this uh text version I'll have this one again I will upload this uh admin voice and I'll keep this you know standard by the way in this one okay let me keep standard and then let's generate and see if you are able to get the desired response it says starting again it will follow the same uh mechanism so it will first run the predict function it will use the auto regressive samples we generated it will find the best candidate and then it will generate the speed for you so now let's come back over here in the sample data what we have to do we have to upload this uh file I'll upload this file by the way the first one the Sam voice one and uh do we upload I think it's we have to first convert it to wave let's convert mp3 to wave so I'll convert this to converting it to web file because web 2 lead accepts web file okay as an audio file so I'll upload this and convert let's download is downloading I will just copy this one more option I'll come back here I'll paste it over here I'll say let's do open full movie by the way same way let's see thanks for having me I think generative artificial intelligence Yep this makes sense now I'll go back to web to leave and I'll upload it here again upload and this time I think yeah we have five so what we will do Sam voice wave here we have to upload the in this path so let's do Sam underscore wise underscore one that's done and then here on the input video dot MP4 we have to upload the video by the way so let's upload the video what I'm going to do I'm going to Altman cut this is the video that I'm going to upload which is an mp4 let me rename the file path or the file name your Altman cut dot MP4 so what we are doing here what it will do this this uh command that you see in collab notebook it's both inside this web 2 lip you can see we are CD into this web to leave so CD web to live if you click on this web tool lip you will say they have a file called inference dot Pi so we are running this file called python inference dot pi and we are passing the model where it's the checkpoint uh from the checkpoints you can see here are the checkpoints by the way and this is the downloaded model weight so we are passing this wave to lead underscore gain Dot pth and then we are you know passing the phase which is of course a video in this case and if the face should be visible you know I if you want to get a high quality videos using it it's better to have a side view you know when people are talking okay it generates better video in that case and your face and then your audio so audio that told you that we have used auto htts you know to clone the Sam Altman's voice and we have created a fake Voice by the way in that case so you know you can see that we are in this webtoolate because we are all already uploaded it so what I'm going to do now we have this man underscore cut.mp4 and the same wire now let's try to run this okay we'll run this I'm just going to run this and it will take little time let's go back here it's still generating for our second uh segment of text and this will be our last for this uh this video guys okay you can use the same techniques to create a long podcast which is completely deep or deep fake by the way completely fake so you can see in this case it's using Cuda for inference we are using GPU runtime in collab you know it's reading the video frames finding out the frame which are 672 frames you know and then it's doing the mail jumps okay looking at the chunks you know creating those uh sort of chunks of the video that you have the audio by the way in this case and it will take little time so come back here on replicate we are 88 percent done few more seconds and we'll have our second segment of text and then again we will follow the same mechanism to get the video out of it you can say Computing based candle okay we are Computing best candidate so if I come back over here on my screen card so what we are doing here guys okay if you see now I'll just use this so we have we have first have this clone audio we have this cloned voice I will write a rather okay and then after clone voice we use web to lip on Sample video and then we create a sort of defect video for this we have used Toto sdts so you can see just we are using uh so this is a gan model by the way we are using Gan we are combining with a Y synthesis model it takes to speech model and then we are creating this output right this is our output we have also used charge apt for the script we have used chat GPT to get the script which is again powered by an llm large language model is powered by EPT 3.5 for gpt4 by the way and you know you can see in this case play result video Let's play it these are still being generated if we play the result video for you can see thanks for having me I think generative artificial intelligence will continue to improve in its abilities this is so fantastic you can see some you know uh some drops in the frame there are some drops in the frames okay by the way you can if you understand uh signal processing if you understand computer vision by the way signal process is a very different uh topic altogether if you just understand the frames okay if you have good understanding you can easily find out that this is a deep fake by the way if you see there's a drop uh you can see the drop which is music and language I'll give all this script prompts the outputs in the description guys go ahead have a look we also got his second audio and this time you know we had uh standard at preset I don't know what's the exact clone wire here let's hear that okay one of the biggest challenges is ensuring that the AI systems we create are aligned with human values and goals we also need to address concerns around safety transparency and accountability I'm so happy with the we learned that I have received uh using a total htts so total htts have a look it's extremely powerful you know for text to speed you can also run it offline you don't have to use cloud cognitive Services you know to send your data and to do a text to speech we can also create an application or an app to do whatever you want to do with total https it's open source just have a look at their license uh if you have to cite please cite them you know you can do this you can do the sighting on wherever they have published the research paper please cite them and follow the licensing you can use it now we have generated this one so what we'll also do now will also generate this uh this second voice that we have downloaded right if you come over here on the downloads this is the voice that you have created but first we have to first let me rename this okay so I'll just rename this I'll say Sam voice 2 in this case it's the second cloned wire the second segment that we have for that conversation now again we have to convert this to wave again you can use ffmpeg I will give the command list in description but I don't want to use that because this video is already available on YouTube it's not a confidential data in this case for me I'll just again select file go to download use the same voice too and here I'll click on convert and once I click on convert I'll just download and here I'm going to download this okay so Sam voice two dot wave okay and let me just go here and click on you know copy I'll come back here in my folder and why I open this sorry I'll come here conversation with Sam I'll paste it over here I'll just uh okay now what we have to do again we have to go back to this folder and we have to upload this Sam voice to not from here from my projects conversation with Sam because we have to upload a web file in this case and we only have to make one change we have just to uh remove this ys1 to voice two you can see it has been uploaded and I'll just hit that again Ctrl shift to run the cell by the way and now you see it again follow it will follow the same mechanism you know it will read the frames and it will generate the uh video for UE okay it will just uh kind of perform the lip syncing accurate accuracy that's what they say and please cite their paper guys okay uh I have been using it for a long time okay you can also I'll give the link of this uh demo interactive demo site you can also use it but it's better to go programmatically use this you can also integrate in your application and use it so let's wait for it it's anything for us you can see it generating I'll close this not required image to video but if the more interesting thing will be to you know generate an image okay from mid Journey or any other uh diffuser models also if you want Dali or stable diffusion you you generate an image okay of Sam Altman convert that to a video and then you uh use that sample video rather than having the same video that I have downloaded from internet Okay so let's come back over here so you can see that we are not using you know multiple gpus or some high uh compute uh based infrastructure okay if if you have this infrastructure you know I can suggest some other models as well please reach out to me if you want to reach out you know I do have a different system where I have used several other Gan models like you know face to swap or fail a lot of other models that we have a deep phase to create actual realistic defect videos okay so let's run this so I'll run this and it will play the video and you can also download it that's one of the biggest challenges is ensuring that the AI systems this is fantastic let me now download this and I'll just come here I will click on download I'll just first rename it guys so let me rename this I'll say Sam video two and I'll click on copy I'll come back over here on conversation with Sam by the way and I'll just say paste common option paste same video too I think we haven't uh now uh I think this was a download first let me see you thanks for having me I think yes so let me click on some more and I'll click Sam was Sam 's voice one sorry fan video one by the fam Video One and yes I'll just copy this so we have this now I'll create a teaser out of it I'll show you what I'm talking about and yes paste so now you can see this is a video one I think general and this is the video too one of the biggest challenges is ensuring that the AI system now this is the can we do one more round guy for this uh thank you part so I can create a teaser video with uh me having conversation with Sam Edwin I'll just say okay let's create my pleasure it was great to chat with you okay let me I I close replicate okay I'll I'll open this replicate Auto https see if we can get uh this one as well I don't know uh let's see that now come over here I'll say ultimate Voice by the way by MP3 let's call it standard let's click on submit and we'll see so here I will change that because we are getting a new audio and I'll call it three and that this Remains the Same let's see if you're getting it from and it has a limit guys okay uh we can also put your cards okay so it depends I just wanted to show the capabilities that how you can you know link this uh multiple AI models learning models and create something pretty cool right to provide the experiences now you can use this in your promotions PR marketing you know you're working at some workplace you can create a complete you know virtual defect podcast with your CEO you know with your CTO or with your manager or your boss right by the way so you can also create a movie as I said in the beginning so it totally depends on you that how you want to utilize this Cutting Edge Technologies you want to use it for uh you know enabling or empowering you know the society or you want to use it for able purposes now it's completely in your hand okay so let's come back let's wait for this uh uh audio that we are getting the Clone uh voice of Sam Ashman this is going to be the last one okay and I'm going to create uh a movie also so I'll also uh create the other video other YouTube video that where I will have a complete uh movie that will be based on Mid journey llms and again these Gan models so please wait for that video as well now let's come back over here on replicate where are we okay and let's download this I'll just download it it was great to chat with you today okay and I'll just uh so in folder I'll call it Sam so more Sam voice three exactly we now have three segment I'll just do copy let me do a copy of it and I will come on this desktop my projects conversation with Sam I'll just do a paste and here I'm going to use MP3 to have again and this guy will say that uh so many MP3 to app converter that I have converted using this tool and I'll just again upload it and I just download I can just download it because it's done let me just go ahead copy this so more option copy and I will paste it over here in this conversation with Sam folder which is the web file in this case and once it is done I will come back in this sample data folder on Google collab by the way and I will upload this web file I hopefully it's over five years now I'm uploading it once it is done now what I have to do I will just have to run this it says create web to live video using this model weights web tool lip underscore gang Dot pth and I'll just run this and it will start generating the video for me So based on two things the first thing is your sample video now that I have already downloaded it from YouTube you can also use air to AI image generator to generate an image and then convert it to video and upload it over here with that I'm also you know passing the cloned voice of Sam management I don't know if Sam Ashman is watching this okay so I don't have you know uh too many subscribers so this video might not reach to Sam Iceman okay but anyway at least the people who are watching this video you guys can you know utilize this technique so technology is by the way to you know create super cool stuff right so now let's come back over here and then we have play result video my pleasure it was great to chat with you today yeah done it's it's super cool I I really love the response I really loved what I just you know worked on today guys I have used it previously at my workplace you know I have done a lot of things with this uh let me just do Sam voice I'll just do copy and come back in this folder conversation with Sam and now we have three videos so I hope you liked it okay because this was the you know agenda of this video I just wanted to you know do end to end we first use total https to create a clone voice we cloned the Sam admin voice with them 28 second audio that audio I got it from internet and using total htts I cloned the audio samples then I converted that to a web file I already have a video file so I use web to lip you know this uh this model the Gan model and I created the video out of it and you have already seen the result in the you know also in the beginning of the video and also now that three segment 3D video that we have created this is what I wanted to you know do guys today in this video and please let me know you know what what you what you are doing with this you know techniques or Technologies if you want to extend it further please share your result with me you can reach out to me via my email address you know you can find that email address on my channel description please reach out to me you know if you want to work together on some research project you know in in these areas like Gan or large language models I do find some interest in those areas please reach out to me we can work together uh on those things if you have any thoughts feedback for me you know please drop uh that in the comment box and you can also reach out to me personally I hope you like this video guys you know you uh if you like this video if you if you are liking the content that I am creating you know please subscribe to the channel if you haven't subscribed yet and please share the channel you know with your friends and to peer thank you so much for watching this video guys see you in the next video