I've been experimenting with different LLM gateways lately, looking for something simple, flexible, and easy to manage. Here's how it went.
My first attempt was LLM Gateway. Setting it up with Docker was a bit challenging, but I managed to get it running. Then I discovered that Dynamic Routes is a business feature, which requires a paid plan. Time to move on!
Next, I checked out OpenRouter. Before investing time in setting it up, I explored what the free account offers. The 50-request limit was a dealbreaker for my experiments, so I moved on.
Finally, I gave LiteLLM a try, and this one looks promising!
The Docker quick-start link seems outdated, so I'm preparing a PR to fix it. Fortunately, finding the updated Docker Compose file wasn't difficult. Installation and setup were very straightforward.
I added two models and configured an Auto Router. It's still in beta, but I think I might have finally found what I was looking for!
I'll share more about my experience as I continue experimenting.