🦙 Python Bindings for llama.cpp

Documentation PyPI PyPI - Python Version PyPI - License PyPI - Downloads

Simple Python bindings for @ggerganov's llama.cpp library. This package provides:

  • Low-level access to C API via ctypes interface.
  • High-level Python API for text completion
    • OpenAI-like API
    • LangChain compatibility

Installation

Install from PyPI:

pip install llama-cpp-python

Usage

>>> from llama_cpp import Llama
>>> llm = Llama(model_path="models/7B/...")
>>> output = llm("Q: Name the planets in the solar system? A: ", max_tokens=32, stop=["Q:", "\n"], echo=True)
>>> print(output)
{
  "id": "cmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "text_completion",
  "created": 1679561337,
  "model": "models/7B/...",
  "choices": [
    {
      "text": "Q: Name the planets in the solar system? A: Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, Neptune and Pluto.",
      "index": 0,
      "logprobs": None,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 14,
    "completion_tokens": 28,
    "total_tokens": 42
  }
}

License

This project is licensed under the terms of the MIT license.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.3.34)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for abetlen from gravatar.com abetlen

Credits

Author: Andrei Betlen

License

MIT

Requires

Python >=3.7

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

llama_cpp_python-0.1.13.tar.gz (520.8 kB view details)

Uploaded Source

File details

Details for the file llama_cpp_python-0.1.13.tar.gz.

File metadata

  • Download URL: llama_cpp_python-0.1.13.tar.gz
  • Upload date:
  • Size: 520.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.8.10

File hashes

Hashes for llama_cpp_python-0.1.13.tar.gz
Algorithm Hash digest
SHA256 038cc2ad332a16bc85c4e5d8231bee28143c018179e1bb540c0eb81a4a0ea411
MD5 5f8f79540c2934895080ca480bf8be94
BLAKE2b-256 638f1bb0a901a1be8c243e741a17ece1588615a1c5c4b9578ce80f12ce809d14

See more details on using hashes here.

Release history Release notifications | RSS feed

0.3.34

0.3.33

0.3.32

0.3.31

0.3.30

0.3.29

0.3.28

0.3.27

0.3.26

0.3.25

0.3.24

0.3.23

0.3.22

0.3.21

0.3.20

0.3.19

0.3.18

0.3.17

0.3.16

0.3.15

0.3.14

0.3.13

0.3.12

0.3.11

0.3.10

0.3.9

0.3.8

0.3.7

0.3.6

0.3.5

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.90

0.2.89

0.2.88

0.2.87

0.2.86

0.2.85

0.2.84

0.2.83

0.2.82

0.2.81

0.2.80

0.2.79

0.2.78

0.2.77

0.2.76

0.2.75

0.2.74

0.2.73

0.2.72

0.2.71

0.2.70

0.2.69

0.2.68

0.2.67

0.2.66

0.2.65

0.2.64

0.2.63

0.2.62

0.2.61

0.2.60

0.2.59

0.2.58

0.2.57

0.2.56

0.2.55

0.2.54

0.2.53

0.2.52

0.2.51

0.2.50

0.2.49

0.2.48

0.2.47

0.2.46

0.2.45

0.2.44

0.2.43

0.2.42

0.2.41

0.2.40

0.2.39

0.2.38

0.2.37

0.2.36

0.2.35

0.2.34

0.2.33

0.2.32

0.2.31

0.2.30

0.2.29

0.2.28

0.2.27

0.2.26

0.2.25

0.2.24

0.2.23

0.2.22

0.2.20

0.2.19

0.2.18

0.2.17

0.2.16

0.2.15

0.2.14

0.2.13

0.2.12

0.2.11

0.2.10

0.2.9

0.2.8 yanked

Reason this release was yanked:

Broken build

0.2.7

0.2.6

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.85

0.1.84

0.1.83

0.1.82

0.1.81

0.1.80

0.1.79

0.1.78

0.1.77

0.1.76

0.1.74

0.1.73

0.1.72

0.1.71

0.1.70

0.1.69

0.1.68

0.1.67

0.1.66

0.1.65

0.1.64

0.1.63

0.1.62

0.1.61

0.1.59

0.1.57

0.1.56

0.1.55

0.1.54

0.1.53

0.1.52

0.1.51

0.1.50

0.1.49

0.1.48

0.1.47

0.1.46

0.1.45

0.1.44

0.1.43

0.1.42

0.1.41

0.1.40

0.1.39

0.1.38

0.1.37

0.1.36

0.1.35

0.1.34

0.1.33

0.1.32

0.1.31

0.1.30

0.1.29

0.1.28

0.1.27

0.1.26

0.1.25

0.1.24

0.1.23

0.1.22

0.1.21

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

This release

0.1.13

0.1.12

0.1.11

0.1.10

0.1.9

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page