Classify Failures Into Categories
Beginner
The problem, from first principles
A hundred failed agent runs, each with its own raw error message, doesn't tell anyone what to fix. "17 timeouts, 12 auth errors, 3 rate limits, 68 uncategorized" does -- it turns a pile of unstructured text into a short list of where the real problems are.
The task
Write classify_failures(failures, categories), where categories maps a category name to a list of keywords. For each failure message, find the first category (in the order categories is given) whose keyword list has a match as a substring of the message, and count it there; a message matching nothing counts under "UNCATEGORIZED". Return counts for every category plus "UNCATEGORIZED", including zeros.
Keywords
agentic-systemsagent-observabilityevaluation
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.