Can Open Source Collaboration Train AI?

Key Takeaways

the current Paradigm for training models requires that all of the gpus that train the model these you know these computers that do the training they all have to be like in the same room and like that seems very counterintuitive the way the technology stack is grown it it assumes that all of the uh little brains that are training the model can talk to each other very fast that's like the most the simplest way to do it and that's okay if you put it all into a warehouse and if you're a single entity who can just put it into one data center but we and sort of the the collective we of the open source AR movement aren't one entity so how can we cooperate to actually train a a state-of-the-art AI that we all own and there were a bunch of technical problems that ultimately LED down to the fact that the bandwidth on the Internet is much smaller than the bandwidth between these gpus that they train in these data centers