Facilitator(s)
Nishad Thalhath (@nishad)
Abstract
Local large language models (LLMs) have moved on since the first edition of this workshop in 2024. The privacy and cost arguments still apply, but many capable models now run on an ordinary laptop, come with permissive licenses, and a few are fully open, with weights, training data, and training code all open. For libraries, this makes provenance, reproducibility, and vendor independence practical concerns. This third edition is built around Apertus, a fully open multilingual model family from the Swiss AI Initiative (EPFL, ETH Zurich, CSCS).
Part 1 - Fundamentals
How LLMs work (parameters, tokens, quantization, context); the difference between open source, open weights, and fully open models. Installing a local runtime (Ollama or LM Studio). Choosing a model size for your hardware and writing effective prompts. No programming experience required for this part.
Part 2 - Working with your data
Working with LLMs from Python. Generating structured output and validating it. Embeddings and semantic search over controlled vocabularies. Retrieval-augmented generation (RAG) over your own documents. Basics of tool use, such as querying a SPARQL endpoint or an authority file, with a human in the loop.
This is a hands-on workshop. Participants can follow along with the graphical tools only, run the notebooks, or work through the materials afterward. The workshop runs for four hours in two halves. Part 1 (first two hours) is for beginners and needs no prior experience; Part 2 (second two hours) assumes you can already run a local model and builds on it with Python. Each half stands on its own, so you can attend only Part 1 and leave at the break, or join at the break for Part 2. Exact times will be posted with the programme.
Prerequisites:
A computer running Windows, macOS, or Linux with 20 GB of free storage. 8 GB of RAM is sufficient with a small model; 16 GB is recommended for a larger model. A stable internet connection is needed to download models, or you can download them in advance (instructions will be posted about a week before). For Part 2, a working Python environment (3.12 or newer) is needed to follow along.
The Zoom room opens 30 minutes early for installation help.
To register your participation in this workshop click on the “Going” button above. You will then receive an email notification as soon as facilitators post an update. Watch out to not register for two parallel workshops.