AI/ML Library
2.0k 2026-05-01
withcatai/node-llama-cpp
A Node.js binding for llama.cpp, enabling local execution of large language models with advanced features like JSON schema enforcement and function calling.
Core Features
Run LLMs locally on your machine with Metal, CUDA, and Vulkan GPU acceleration.
Provides pre-built binaries and automatically adapts to your hardware for easy setup.
Enforce JSON schema on model output and support function calling for structured responses.
Includes embedding and reranking capabilities for advanced AI applications.
Offers a great developer experience with full TypeScript support and comprehensive documentation.
Quick Start
npm install node-llama-cppDetailed Introduction
node-llama-cpp is a powerful Node.js library that provides bindings for `llama.cpp`, allowing developers to run large language models (LLMs) directly on their local machines. This project eliminates the need for cloud-based LLM APIs, offering enhanced privacy, reduced costs, and offline capabilities. It stands out with features like automatic hardware adaptation, GPU acceleration (Metal, CUDA, Vulkan), and advanced control over model output, including JSON schema enforcement and function calling, making it ideal for building robust, locally-powered AI applications.