Transforming API Output Quality with Assertions // Vikram Rangnekar // MLOps Podcast #259 clip

Key Takeaways

I didn't quite understand the idea of these assertions and building out the API basically being able to validate the type yes and how you said all right well one abstraction is what they were doing with DSP but another abstraction is saying this can now be an API and so when we give whatever output to someone it comes in inherent with the idea of it's going to be in the type that you specify is that what I understand yes so there are two types of assertions one is you put them um you assert a type essentially you say you know while you're creating the function signature you just say this output type is string or string array or number or Boolean or whatever or you know and and the second type of assertion is like a custom function where you you want to like like actually do something with the output you might just you might actually call another llm if you really want you be like hey is this is this good enough if not throw like a error you know essentially and then the first LM can correct itself right so it's it's it's a function called on this input output value and uh and this can be done either when the whole value is available you can wait till the end of value you might have like multiple key values that are outputed so you know in which case then you know you might want to look at the whole value or you can do it as the value streaming out because it's not Json it if you can somehow validate it maybe you know you're looking for some prefix or something before so you can chop up every line that's coming wait for a line to come in then then kind of look at it with the regx and saying you know sorry you know you're messing up sort of thing yeah and this early failure is is great because now firstly the assertions concept makes it easy to code cuz now you have all of this like captured within your uh dpy program and when you initially put in the examples and you run the bootstrap all of this gets engaged right so the error correction all that's so your outputs all the traces are high quality in between because the assertions have ensured so in production less of the stuff happens right so cuz youve already fine-tuned it down to the correct what you need in terms of code you're not saying please add a dash in front of you know like you have an assertion it's explained that so you got high quality examples you just save those in there and the LM knows oh I know what this guy needs [Music] TR