Evals Expanding to Third Party Models & Agent Testing

Key Takeaways

But one of the things that we launched today with evals too is ability to use like thirdparty models as well and kind of bringing that in to one place. And so I think definitely kind of see where the ecosystem is at which is you know using multimodels and kind of having >> third party models as in non nonopen models. >> Yeah. Yeah. It'll work with eval starting today. >> Yeah. >> Okay. Got it. Uh we have a really cool setup with open router where we're working with them and then you can bring your open router set up and then with that you can actually you know you write your evals using our data sets tool or use our data set tool to create a bunch of evals and you'd actually be able to hit a bunch of different model providers um you know take your pick from wherever even like open source ones on together and see the see the results uh in our