LLM

Run open-source large language models on your own computer or server.

Overview

Large Language Models (LLMs) power AI assistants that can write, summarise, translate and answer questions. Besides cloud services, there are many open-source and open-weight models that you can run on your own hardware, keeping your data private.

Tools such as Ollama and llama.cpp make it simple to download a model and use it from the command line, a chat interface or your own application.

Where it is used

Private AI assistants
Document summarisation and search
Code assistance
Learning and experimenting with AI

Try a local model with Ollama

After installing Ollama from its official website, run a model from the terminal (the model name is an example - pick one that fits your hardware):

ollama run llama3.2

LLM

Run open-source large language models on your own computer or server.

Talk to us

What you will cover

The key topics and features of LLM

Local Models

Run models such as Llama, Mistral, Gemma or Qwen on your own machine.

Ollama & llama.cpp

Simple tools to download and serve models.

Chat Interfaces

Use a web UI such as Open WebUI with local models.

API Access

Call your local model from Java, Python or JavaScript.

Private Documents

Build question-answering over your own files (RAG).

Data Privacy

Your prompts and data stay with you.

LLM