Tags: #cloud-deployment
LLM Serving Framework
Docker
12.3k
bentoml/OpenLLM
A framework for easily self-hosting and serving any open-source Large Language Models as OpenAI-compatible API endpoints in the cloud.
Replaces:
Details A framework for easily self-hosting and serving any open-source Large Language Models as OpenAI-compatible API endpoints in the cloud.