Package · PyPI
PyPI · v0.1.6.0 · MIT License
To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.
Tentative categories were assigned with moderate confidence (0.5 to 0.75): they are shown, but they do not count toward a category’s momentum. How categories work
pip install minferencePyPI publishes no adoption figure, so there is no growth to derive. Nothing is estimated in its place.
Adoption history
PyPI publishes no adoption count, so there is no history to chart. That is a fact about the registry, not a gap in ours: the package’s releases and maintenance are measured instead.
In the Crest catalog
[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.
Linked by the package’s own registry metadata.GitHub (opens in a new tab)
>=4.37.0No published news event mentions this package or its repository yet.