AI Engineering: A Realistic Roadmap for Beginners

Key Takeaways

I spent the last few weeks diving into Chip Huan's 500page AI engineering book so you don't have to. This video is the road map for learning what actually matters in AI engineering for complete beginners and it's based on Chip's book packed with over,200 references and I went through all of them to pull out the most useful and most importantly free ones for you. I also have it written out [clears throat] in a PDF so you can download it on my website for free. link will be in the description. Now, before we jump in, let me make one thing clear. This guide is for you. Even if you're not a PhD researcher or math wiz, AI isn't just for machine learning scientists anymore. If you want to work in AI, whether as a builder or in roles like AI product, AI strategy, safety, research ops, or even recruiting, you don't need to go learn multivariable calculus or train giant models from scratch. So if you've been thinking that I want to be a part of this but I don't know where to start, this video is for you and let me explain why. To start, we need to first understand what even is AI engineering and how is it different from machine learning engineering. AI engineering blew up recently for two simple reasons. One, AI got way better and it got way easier to use. Old school machine learning engineers built models from scratch. They spent months training algorithms, tuning parameters, and this often requires deep understanding of math. But AI engineers are building with what already exists. Instead of starting from zero, they work with GPT4 or even open-source models like LMA. They don't need to train the models themselves, but they need to be able to build products that actually solve problems for real people. And that is why AI engineers move faster. They're not reinventing the model, but they are wiring it into something useful. And most importantly, it's not a math heavy role like machine learning engineers. So, it's very beginner friendly. If you want a deeper breakdown of how AI engineers are different from machine learning engineers, you can check out my video on that. I'll link it somewhere here. Let's first start with the foundation, the models that started it all. They began as simple language models that learned to predict the next word. Then came the big breakthrough, selfs supervision. Models learned from massive amounts of text without humans labeling every example. So they're basically teaching themselves by reading the internet. Then they got bigger and smarter and started handling not just text but images and also videos. And these are called foundation models because everything else is built on top of them. They now power tools like GitHub, Copilot, image generators, and even video tools like Sora, which maybe you have been playing around with. To be a good AI engineer, you want to at least understand what's happening under the hood, at least conceptually, like what data was the model trained on. Some models are multilingual, others specialize in code or science or conversations. Model size matters, too. Bigger models do harder things, but they might cost more and run slower. So, you want to be able to think about the tradeoffs. There are also post-training where humans show the model good examples so it learns what people actually prefer. If you want to deepen your understanding of models, there are three key papers to start with. The gopher from deep mind paper is where the bigger is better era started to happen. Instruct GPT from open AAI is the paper that taught models to follow instructions and lama from meta especially the section on synthetic data generation and verification is a must readad. The links are all in the road map. Sampling is how a model chooses its next word. It's not deterministic like traditional code meaning in traditional code when the input is X output is always Y. But in AI, the same prompt can produce different answers. And this is just inherent to how AI is designed. If you want to really understand this, read Chip Puan's post on sampling for text generation is the clearest breakdown of how models decide what to say next. Once you understand models, you hit the first major challenge. How do you know if your AI is actually good? Like we discussed earlier, in traditional software, testing is binary. A software either works or it doesn't. But AI is not binary. So how do you evaluate it? It's not just measuring correctness, yes or no. So there are metrics like entropy and perplexity. And we can also use AI as a judge which is models developed to score other models responses. If you want to go deeper on these concepts, read challenges in evaluating AI systems by anthropic. And for AI as a judge, read judging LLM as a judge with MTB bench and chopout arena. That's the paper behind modern leaderboard systems. Prompt engineering is how you communicate with the model to get the output you want. In context learning means giving examples inside your prompts so the model runs them on the fly called fshot. Zero shot means you give no examples and rely on the model's previous training. System prompts set the overall behavior and user prompts control the specific task. You also need to think about context lengths, how much information fits in the model all at once. Good prompts include context, break problems down, and ask the model to think step by step. You also need prompt security to prevent prompt injection or jailbreaking. And if you want to level up on your prompting, read Meta's prompt engineering guide. It focuses on reliability and not hacks. There's also context engineering which has been pretty hot lately shaping what the model sees before it responds. So now this is becoming more important than prompting itself. So for this read and anthropics effective contents engineering for AI agents. This concept wasn't really big at the time of the writing of Chip Huan's AI engineering book. So here's additional resources. RAG stands for retrieval augmented generation. This is where you stop expecting the model to know everything from the past training. Like if you ever use chat GPT when it first came out, it didn't have any new information or be able to access the internet or it was kind of stuck in the past with certain information. But modern AI can query external resources. And this could be the internet databases or your own documents because you're not relying on old data from past training anymore. This way you get a much more relevant response. The hard part is the retrieval choosing the right type of information to find. A great overview on this is retrieval augmented generation for large language models a survey. And for production best practices, read Antropics. Introducing contextual retrieval. Agents are AI systems that don't just answer, but they can do stuff for you. So they can plan, use tools, take multiple step actions over time. Imagine asking an AI to plan a trip, and it books flights, find hotels, and builds an itinerary for you. I mean, AI generating text or videos is cool, but making decisions and executing them for you is the next level. But as of now, AI agents still have challenges too, like they might loop or forget context or misuse tools. So if you want to learn more about AI agents, check out the react paper from 2022. It introduced the core loop behind modern agents. How they think, decide, act, absorb, and repeat. Sometimes prompting and retrieval are not enough, and you need to actually change the model itself. That's what's called fine-tuning. customizing the model's behavior with new training. But this is a more advanced topic and it's completely optional if you're a beginner. So feel free to skip the deep dive here, but you should just know that there's a technique called Laura which makes fine-tuning affordable by updating only a small part of the model. And here's a paper on it if you want to learn more. Data is the backbone of all of this. Models are only as good as what they're trained on. You should learn about data curation, data augmentation, synthesis, and data processing. Synthetic data is now a major part of improving models. So, if there is one paper I would recommend, read the Deep Minds best practices and lessons learned on synthetic data for language models. Once your model works, you still need to make it fast and cheap enough to run in production. This is called inference optimization. Reducing latency and compute without sacrificing results. A great overview on this is Nvidia's mastering LLM techniques inference optimization. This covers the major techniques used to make large language models faster and cheaper to deploy in production. Finally, AI engineering isn't just about models, but it's about systems. Real world AI means retrieval pipelines, guard rails, routing, caching, monitoring, and feedback loops. For all these topics, a great resource is Microsoft Skyline for human AI interaction. And that is AI engineering, the full arc for models to systems. And if you want to explore what other jobs are there other than AI engineering, you want to check out this video. and I'll see you there.