Docker Compose Tip #60: Declaring LLMs with the models section
LLMs are now first-class citizens in Compose. The models top-level key lets you declare which models your application needs and wire them into your services, all in the same Compose file. Basic usage Declare a model at the top level, reference it from a service: models: smollm: model: ai/smollm2 services: app: image: myapp models: - smollm When the stack starts, Compose ensures the model is available locally and connects the app service to it. The container receives endpoint information via environment variables. ...