Gorilla: Detecting Hallucinated APIs
Beginner
The problem, from first principles
Models can invent APIs that sound plausible. Gorilla's evaluation counts such hallucinations separately, because a made-up function fails at runtime even when the call looks well formed.
From theory to code
Implement is_hallucinated(call_name, api_list), returning True when the called API is not known.
Constraints
- Match names exactly.
Hints
Hint 1
Check whether the name is absent from the list of known APIs.
Keywords
research-papersagentstool-useapis
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.