The Chronicles of RiDiC: Generating Datasets with Controlled Popularity Distribution for Long-form Factuality Evaluation

1. 1Utilize Wikipedia and Wikidata as data sources to generate entities with specified characteristics
2. 2Configure the pipeline to control popularity distribution and other characteristics
3. 3Generate multilingual datasets for evaluating LLMs' long-form factuality
4. 4Apply the RiDiC dataset as an example for evaluating LLMs' performance

NLP engineers and researchers on a team benefit from this pipeline as it helps evaluate the factuality of LLMs' long-form generation, while data scientists and ML researchers can utilize the generated datasets for model training and testing

💡 The RiDiC pipeline enables controlled generation of datasets for evaluating LLMs' long-form factuality, complementing short-form QA datasets

📊 Introducing RiDiC: a pipeline for generating datasets to evaluate LLMs' long-form factuality #LLMs #NLP