Reverse-Engineering SEC EDGAR's Full-Text Search API (One Undocumented GET Request)
📰 Dev.to · Max
Learn to reverse-engineer and use the undocumented SEC EDGAR full-text search API to fetch clean JSON data without a key, using Python
Action Steps
- Send a GET request to efts.sec.gov using Python's requests library to test the API
- Use query parameters to customize the search query and fetch specific data
- Implement pagination to handle large result sets and avoid rate limiting
- Handle rate limit blocks by implementing exponential backoff or other retry mechanisms
- Parse the returned JSON data to extract relevant information
Who Needs to Know This
Developers and data scientists on a team can benefit from this knowledge to build applications that leverage the EDGAR database, while also understanding how to navigate undocumented APIs and rate limiting
Key Insight
💡 The EDGAR full-text search API can be queried using a simple GET request to efts.sec.gov, returning clean JSON data without requiring an API key
Share This
🚀 Reverse-engineer the SEC EDGAR full-text search API to fetch clean JSON data without a key! 🚀
Key Takeaways
Learn to reverse-engineer and use the undocumented SEC EDGAR full-text search API to fetch clean JSON data without a key, using Python
Full Article
The EDGAR full-text search runs on one undocumented GET to efts.sec.gov that returns clean JSON. No key. How to query it from Python, paginate, and dodge rate-limit blocks.
DeepCamp AI