The Zapier NLA API solves a major problem for large language models—the ability to interact with real systems. Rather than a developer integrate with every possible service, they can integrate once with Zapier and run every “Action” the user has authorized using natural language instructions.
One thing I noticed after testing it out is that the underlying action materially impacts the quality of results. For example, the Slack API is fairly straightforward to parse the instructions into the correct action (e.g. post this message to this channel). Whereas something more complicated (with an API design that is not great or very abstract) like the HubSpot API fails on the simplest instructions (e.g. “find XYZ” to the Find Company action).
The primary issue I see here is that a lot of control is deferred to the NLA instructions which the developer has no control over. This is going to lead to really odd hacks where you need to wrap the instructions to the NLA API with another language model to augment the instructions to something more explicit creating a bad interop problem (calling APIs directly would be better at that point).
Links to this note
-
Zapier NLA Is Bad at Generating API Parameters
I tried out Zapier Natural Language Actions API and found that it’s not particularly good for the one thing I needed it to be good at glueing together other APIs with natural language. API endpoints that are simple and straightforward are easy for large language models to generate the right inputs but more complicated (and poorly designed) like HubSpot are unusable.