Commit Graph

4 Commits (bd4b46d6ba504b99c936f43fc014529adffb6048)

Author SHA1 Message Date
Mack Straight 074bea2eb1
sentencepiece bpe compatible tokenizer (#252)
* potential out of bounds read

* fix quantize

* style

* Update convert-pth-to-ggml.py

* mild cleanup

* don't need the space-prefixing here rn since main.cpp already does it

* new file magic + version header field

* readme notice

* missing newlines

Co-authored-by: slaren <2141330+slaren@users.noreply.github.com>
2 years ago
Thomas Klausner 41be0a3b3d
Add NetBSD support. (#90) 2 years ago
Georgi Gerganov 7211862c94
Update Makefile var + add comment 2 years ago
Georgi Gerganov 26c0846629
Initial release 2 years ago