@orchestra-research/serving-llms-vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
No skills found in this collection.
SKILL.md
Loading...
Select a file to preview
Analyzing security...
Checking scan reports and verification data.
Bill of Materials
Everything this skill can do — files, network, commands, and more.
No demos yet. To add one, ask your AI agent: "Submit a demo for @orchestra-research/serving-llms-vllm" — or upload via the API.
To add a demo, ask your AI agent: "Submit a demo for @orchestra-research/serving-llms-vllm"
No eval data yet. Eval results are published via the SkillSafe API.