I kept getting an opaque rust compile errors when writing an axum API handler.
Like this:
the trait bound `fn(axum::extract::State<Arc<std::sync::RwLock<AppState>>>, axum::Json<ChatRequest>) -> impl Future<Output = axum::Json<ChatResponse>> {chat_handler}: Handler<_, _>` is not satisfied
And:
the trait `Handler<_, _>` is not implemented for fn item `fn(State<Arc<…>>, …) -> … {chat_handler}` = help: the following other types implement trait `Handler<T, S>`: `MethodRouter<S>` implements `Handler<(), S>` `axum::handler::Layered<L, H, T, S>` implements `Handler<T, S>`
Turns out my handler code utilized dynamic dispatch using trait objects and that was incompatible with Axum’s async implementation.
See if you can spot the issue:
let note_search_tool = NoteSearchTool::default();
let tools: Option<Vec<Box<dyn ToolCall>>> = Some(vec![Box::new(note_search_tool)]);
let mut history = vec![];
chat(&mut history, &tools).await;
The fix is actually straightforward, but the opaque error messages made this difficult to track down. By default, a trait object is not thread-safe. I needed to add some additional trait bounds to the usage of dynamic dispatch.
let note_search_tool = NoteSearchTool::default();
let tools: Option<Vec<Box<dyn ToolCall + Send + Sync + 'static>>> = Some(vec![Box::new(note_search_tool)]);
Adding a type alias cleans up these type signatures.
type BoxedToolCall = Box<dyn ToolCall + Send + Sync + 'static>;
let note_search_tool = NoteSearchTool::default();
let tools: Option<Vec<BoxedToolCall>>> = Some(vec![BoxedToolCall::new(note_search_tool)]);