Package · PyPI

nextai-vllm

PyPI · v0.0.7 · Apache 2.0

A high-throughput and memory-efficient inference and serving engine for LLMs

Tentative categories were assigned with moderate confidence (0.5 to 0.75): they are shown, but they do not count toward a category’s momentum. How categories work

Install

pip install nextai-vllm

Adoption

Adoption
Not published by PyPI
Not published
Popularity
Not published by PyPI
Not published
Dependents
Not published by PyPI
Not published
Maintenance
0.16
DerivedRelease cadence and recency, 0 to 1
Releases per year
26
6 versions over its life

Growth by period

PyPI publishes no adoption figure, so there is no growth to derive. Nothing is estimated in its place.

7 days
Not published by PyPI
Not published
30 days
Not published by PyPI
Not published
90 days
Not published by PyPI
Not published
1 year
Not published by PyPI
Not published

Adoption history

Adoption history

PyPI publishes no adoption count, so there is no history to chart. That is a fact about the registry, not a gap in ours: the package’s releases and maintenance are measured instead.

Facts from the registry

Latest version
v0.0.7
6 versions
First published
JAN 30, 2024
Last release
APR 25, 2024
2y ago
Licence
Apache 2.0

Repository

In the Crest catalog

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Stars
93.1K
Forks
22.9K
Language
Python
Last push
22d ago
Last release
32d ago

Linked by the package’s own registry metadata.GitHub (opens in a new tab)

Dependencies latest version · v0.0.7

Runtime 20

  • cmake>=3.21
  • fastapiAny version
  • ninjaAny version
  • numpyAny version
  • outlines==0.0.34
  • prometheus-client>=0.18.0
  • psutilAny version
  • py-cpuinfoAny version
  • pydantic>=2.0
  • pynvml==11.5.0
  • ray>=2.9
  • requestsAny version
  • sentencepieceAny version
  • tiktoken==0.6.0
  • torch==2.1.2
  • transformers>=4.39.1
  • triton>=2.1.0
  • uvicornAny version
  • vllm-nccl-cu12<2.19,>=2.18
  • xformers==0.0.23.post1

Related news