The agentic framework ecosystem is rapidly evolving 🚀, with AutoGen, MetaGPT, LlamaIndex, LangGraph, CrewAI and most recently, smolagents. Over the past six months, I've experimented with each of these to varying degrees. While I've seen strong results with publicly hosted models from Anthropic and OpenAI, local Ollama integration 🛠️, particularly for tool-calling, has presented some challenges. Successfully transitioning these systems to Ollama and locally hosted models often necessitates more careful design, prompt engineering 💡and a careful understanding of supported features, especially when using frameworks that describe Ollama support as "experimental."
As agentic systems advance, they'll inevitably utilise a combination of publicly and privately hosted models 🌐. Therefore, robust support for heterogeneous deployments, without requiring model parameter adjustments, is crucial.
Heterogeneous Systems: What are your experiences integrating agentic frameworks with diverse model deployments (public/private)? What challenges have you encountered, and what solutions have you found?
Ollama Maturity: Which frameworks have you found to be the most mature and reliable when working with Ollama, particularly for tool-calling?
Open Source Tool-Calling Models: What are your go-to open-source models for reliable tool-calling, and why?
