Ollama vs. vLLM: A Power-user's Perspective on LLM Serving

For the past 18 months, Ollama has been my go-to tool for running Large Language Models (LLMs). Its reliability and minimalist design, focusing on user experience, are truly commendable. Installation, configuration and model management are incredibly straightforward. However, the rise of …

more ...