Model Tradeoffs and the Future of Computer Vision

Key Takeaways

hey there it's joseph from rebel flow today's discussion we're going to focus on evaluating different model architectures why is it you choose one model over another what sort of trade-offs do you have to consider um i mean you want to open this up with what is it that you should be thinking about when comparing one machine learning model versus another yeah so i think there are just kind of a few broad categories um and then we'll kind of dive in and start to take apart each one of these categories in isolation but the main things you need to be considering are uh how fast is your model going to trade how fast is it going to inference how well is your model going to perform um and then also after you've kind of weighed all of those in in place you need to start thinking about deployment so what's the model size of your your model like in terms of just raw storage um and then also where can you deploy your model so in computer vision that becomes really relevant where uh you're thinking do you need real-time object detection or real-time detection or or you uh it's some latency okay can you do it on the server side where can you actually put your model once you're done so those i think are kind of the main components that you're weighing as you're thinking about different model architectures yeah yeah i think it makes a ton of sense and the components there that are interrelated is really like model performance um in terms of its speed and its accuracy and model size and how that determines as you're mentioning deployment environment and otherwise right so if you have a large model you would in terms of number of parameters and therefore in terms of size of that model in terms of the number of megabytes that it takes up you would generally expect that model to be more accurate right because it has greater granularity it can be fine-tuned to be a bit more performant but also likely slower performing right inferences takes a bit longer and it takes more computational power and in that that side of things i mean you're probably going to be considering server-side deployment i don't know like large models that you're going to be able to get away with deploying on device um but maybe i'm wrong i mean like if we're thinking about that as one of the the trade-offs like where is the line like what which architectures might fall into one corner versus another is this gonna change over time like how do you think about that yeah certainly and i mean it's definitely an exciting time to uh to be in computer division from that standpoint and that the compute capabilities um of different uh different compute engines like gpus and mobidius bpus and uh there are tpus and there's all the different use now that you can be um uh deploying things to you know it's just uh vastly increasing so the size of these models that we're able to actually deploy to the edge um is is ever increasing and not only on the hardware side but also on the software side there are frameworks like openvino and uh nvidia's tensor rt which are speeding up uh the the speed of computations on these things so um you're actually starting to get even more performant models that are larger that you can uh deploy at faster speeds um so that's definitely kind of changing the game and reawakening architectures that previously were kind of uh monolithic and were unable to be to be wielded but certainly the dichotomies here that that joseph has been elaborating on on the fact that a larger model is going to perform better and it's going to infer slower is just kind of always going to be a brief fact and state of the art for a lot of these tasks will always be uh with these extremely large models that are trained on you know many many tpus yeah i mean efficient debt comes to mind there right i mean that's why efficient release d0 to d7 um and yeah i mean the of course if you have the resources of a google it's just fundamentally different than um than elsewhere um i'm gonna ask sort of a question so like if you think about this and you extend out um 10 20 30 years um what does vision look like as a result of these considerations like does that mean that like you're right like as as models become more performant as size becomes less of an issue as we can start to do real time with higher quality um i mean today that looks like the warring frameworks if you will of you know tensor rt and openvino and whatnot from nvidia and intel respectively but you extend it over like ignore those details paper over those just imagine that we're 10 years 20 years 30 years in the future what do you think like yeah state of vision looks like yeah so i mean i think i'll just start with kind of reflecting on the present day um just in the in the few years um you know that that i've seen this these sort of things evolving and i think it's just incredible today that there's uh gpu resources available on say like google collab um for only ten dollars a month you can be accessing a tesla v100 you know and that's just given you by default now and that that's insane you know that uh that that kind of compute is out there and available and i think naturally that's going to kind of only increase as time goes by so the size of the models that are going to be able to deploy the speed that they're going to be able to train and their their performance will just rapidly increase say in 20 to 30 years so that means that the amount of data that is required to learn certain tasks and the you know the size of the data set will be able to shrink you know to be able to achieve these tasks so you'll be able to bootstrap a lot quicker to new areas and new domains um which i think is exciting and you know certain certain problems that we thought were um unreachable before like self-driving cars for example will be um you know for reality we'll be able to achieve those things i don't know what what you think yeah i mean i think those are that's entirely accurate around the the what like i mean the cost of training you're continuing to fall the amount of data that's collected and stored what is it doubles every every couple years the but the implications for that i mean given current circumstances um surrounding the rise in inevitability of remote work not just for you know office jobs but for like remote sensing jobs or like sending drones forward or like i mean in ten ten years we'll probably have uh our first broad scale augmented reality use cases i mean the vision isn't just like if you think about it like right now it's kind of like vision is is is a separate component part it's like okay i gotta get my phone and i start to do vision stuff but like in a very short period of time it'll just be ingrained in an enhancing part of the way that we experience the world and the way that businesses and products etc can experience the world and so the real-time understanding of the real world around us as well as a computer cam with the context that a computer can provide is just going to be um in a lot of ways like trying to predict uber when there first was you know a backlink in the 90s right like when you had arpanet who would have thought about uber as like the same way that we're talking about vision now versus now versus what's capable and it makes for some really really exciting capabilities that um really excited basically be a part of building the future yeah it's it's really an amazing uh amazing time to be be part of things awesome so i mean from evaluating model trade-offs to the future of vision um i really enjoyed this one thanks so much yeah thanks for being here guys uh don't forget to like and subscribe below for more videos like this and uh we'll see you on the next video