LangFuzz: Redteaming for Language Models

LangChain · Intermediate ·🧠 Large Language Models ·1y ago
One of the hardest parts of testing your LLM application is coming with a good dataset of edge cases to benchmark it on `langfuzz` is a new experimental library that helps with that ❓How? It uses a fuzz testing technique similar to metamorphic testing. It generates pairs of similar questions, and then runs them through the LLM application. If the answers are drastically different - then one of them must be wrong! From there, you can add both, one, or neither of the questions to a LangSmith dataset so you can iterate on your app and continue benchmarking on these datapoints. GitHub: https:…
Watch on YouTube ↗ (saves to browser)

Playlist

Uploads from LangChain · LangChain · 0 of 60

← Previous Next →
1 Chat With Your Documents Using LangChain + JavaScript
Chat With Your Documents Using LangChain + JavaScript
LangChain
 2 LangChain SQL Webinar
LangChain SQL Webinar
LangChain
 3 LangChain Document Question-Answering Webinar
LangChain Document Question-Answering Webinar
LangChain
 4 LangChain Output Parsing and Extraction Webinar
LangChain Output Parsing and Extraction Webinar
LangChain
 5 LangChain Agents Webinar
LangChain Agents Webinar
LangChain
 6 LangChain Prompt Injection Webinar
LangChain Prompt Injection Webinar
LangChain
 7 LangChain Low-Code/No-Code Webinar
LangChain Low-Code/No-Code Webinar
LangChain
 8 LangChain "Agents in Production" Webinar
LangChain "Agents in Production" Webinar
LangChain
 9 LangChain "Hallucinations in Document Question-Answering" Webinar
LangChain "Hallucinations in Document Question-Answering" Webinar
LangChain
 10 LangChain Retrieval Webinar
LangChain Retrieval Webinar
LangChain
 11 LangChain "in Education" Webinar
LangChain "in Education" Webinar
LangChain
 12 LangChain "Quivr Deepdive" Webinar
LangChain "Quivr Deepdive" Webinar
LangChain
 13 LangChain "OpenSource LLMs" Webinar
LangChain "OpenSource LLMs" Webinar
LangChain
 14 LangChain "OpenAI functions" Webinar
LangChain "OpenAI functions" Webinar
LangChain
 15 LangChain "Chains vs Agents" Webinar
LangChain "Chains vs Agents" Webinar
LangChain
 16 LangChain "Characters" Webinar
LangChain "Characters" Webinar
LangChain
 17 LangSmith Launch
LangSmith Launch
LangChain
 18 LangChain x Pinecone: Supercharging Llama-2 with RAG
LangChain x Pinecone: Supercharging Llama-2 with RAG
LangChain
 19 LangChain Expression Language
LangChain Expression Language
LangChain
 20 LangChain "Advanced Retrieval" Webinar
LangChain "Advanced Retrieval" Webinar
LangChain
 21 Building LLM applications with LangChain with Lance
Building LLM applications with LangChain with Lance
LangChain
 22 Benchmarking Question/Answering Over CSV Data
Benchmarking Question/Answering Over CSV Data
LangChain
 23 LangChain "Production Ingestion" Webinar
LangChain "Production Ingestion" Webinar
LangChain
 24 LangChain "RAG Evaluation" Webinar
LangChain "RAG Evaluation" Webinar
LangChain
 25 Fine-tuning in Your Voice Webinar
Fine-tuning in Your Voice Webinar
LangChain
 26 Tabular Data Retrieval
Tabular Data Retrieval
LangChain
 27 Building an LLM Application with Audio by AssemblyAI
Building an LLM Application with Audio by AssemblyAI
LangChain
 28 Superagent Deepdive Webinar
Superagent Deepdive Webinar
LangChain
 29 Lessons from Deploying LLMs with LangSmith
Lessons from Deploying LLMs with LangSmith
LangChain
 30 Shortwave Assistant Deepdive Webinar
Shortwave Assistant Deepdive Webinar
LangChain
 31 Cognitive Architectures for Language Agents
Cognitive Architectures for Language Agents
LangChain
 32 Effectively Building with LLMs in the Browser with Jacob
Effectively Building with LLMs in the Browser with Jacob
LangChain
 33 Data Privacy for LLMs
Data Privacy for LLMs
LangChain
 34 "Theory of Mind" Webinar with Plastic Labs
"Theory of Mind" Webinar with Plastic Labs
LangChain
 35 LangChain Templates
LangChain Templates
LangChain
 36 Using Natural Language to Query Postgres with Jacob
Using Natural Language to Query Postgres with Jacob
LangChain
 37 LangServe and LangChain Templates Webinar
LangServe and LangChain Templates Webinar
LangChain
 38 Building a Research Assistant from Scratch
Building a Research Assistant from Scratch
LangChain
 39 Benchmarking RAG over LangChain Docs
Benchmarking RAG over LangChain Docs
LangChain
 40 Skeleton-of-Thought: Building a New Template from Scratch
Skeleton-of-Thought: Building a New Template from Scratch
LangChain
 41 Benchmarking Methods for Semi-Structured RAG
Benchmarking Methods for Semi-Structured RAG
LangChain
 42 LangSmith Highlights: Getting Started
LangSmith Highlights: Getting Started
LangChain
 43 LangSmith Highlights: Debugging
LangSmith Highlights: Debugging
LangChain
 44 LangSmith Highlights: Datasets
LangSmith Highlights: Datasets
LangChain
 45 LangSmith Highlights: Evaluation
LangSmith Highlights: Evaluation
LangChain
 46 LangSmith Highlights: Human Annotation
LangSmith Highlights: Human Annotation
LangChain
 47 LangSmith Highlights: Monitoring
LangSmith Highlights: Monitoring
LangChain
 48 LangSmith Highlights: Hub
LangSmith Highlights: Hub
LangChain
 49 SQL Research Assistant
SQL Research Assistant
LangChain
 50 Getting Started with Multi-Modal LLMs
Getting Started with Multi-Modal LLMs
LangChain
 51 Build a Full Stack RAG App With TypeScript
Build a Full Stack RAG App With TypeScript
LangChain
 52 Auto-Prompt Builder (with Hosted LangServe)
Auto-Prompt Builder (with Hosted LangServe)
LangChain
 53 LangChain v0.1.0 Launch: Introduction
LangChain v0.1.0 Launch: Introduction
LangChain
 54 LangChain v0.1.0 Launch: Observability
LangChain v0.1.0 Launch: Observability
LangChain
 55 LangChain v0.1.0 Launch: Integrations
LangChain v0.1.0 Launch: Integrations
LangChain
 56 LangChain v0.1.0 Launch: Composability
LangChain v0.1.0 Launch: Composability
LangChain
 57 LangChain v0.1.0 Launch: Streaming
LangChain v0.1.0 Launch: Streaming
LangChain
 58 LangChain v0.1.0 Launch: Output Parsing
LangChain v0.1.0 Launch: Output Parsing
LangChain
 59 LangChain v0.1.0 Launch: Retrieval
LangChain v0.1.0 Launch: Retrieval
LangChain
 60 LangChain v0.1.0 Launch: Agents
LangChain v0.1.0 Launch: Agents
LangChain
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Next Up
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)