Throughput Is Not All You Need and more

In this meetup, Neha led our discussion of the paper, Throughput Is Not All You Need, and other related works. Our Meetup: https://www.meetup.com/East-Bay-Tri-Valley-Machine-Learning-Meetup/ *Content* 00:00 Intro 02:25 Basic Concepts 11:47 Throughput vs Goodput 18:24 Prefill 32:41 Colocate 35:45 KV cache 42:40 Related papers 50:30 LMCache ============================ 😊About Us West Coast Machine Learning is a channel dedicated to exploring the exciting world of machine learning! Our group of techies is passionate about deep learning, neural networks, computer vision, tiny ML, and other c…