Facilitator(s)
Nishad Thalhath (@nishad)
Abstract
Local large language models (LLMs) are continuing to gain traction in 2025 due to their privacy-first, secure, and cost-effective nature. Running LLMs on personal hardware ensures that data remains local (without relying on external servers) and eliminates ongoing subscription or cloud usage fees. This updated workshop guides participants through the process of setting up and operating LLMs on their own systems using the latest open-source tools and models. We will cover recent developments that have made local deployment significantly more accessible, from models that support complex instructions to intuitive interfaces that simplify setup and usage.
Part 1 – Fundamentals: An introduction to LLM concepts and architecture, including how to prepare a local environment, choosing suitable open-source models and tools, and the basics of writing effective instructions (prompts) for the model.
Part 2 – Intermediate Topics: Automating and scripting interactions with local LLMs; understanding vector embeddings and semantic search to enable retrieval of relevant information; an introduction to retrieval-augmented generation (RAG), which integrates external data with local model outputs.
This is a hands-on workshop, where participants will be able to follow along and configure a local LLM environment during the session (or afterward using the provided instructions). It is designed for both newcomers and those with prior exposure, ensuring that all attendees can build a solid foundation and deepen their understanding through practical activities. By the end of the session, participants will be equipped to run and use LLMs locally for a variety of personal or professional tasks.
Prerequisites:
Participants should have a computer (running Windows, macOS, or Linux) with at least 8 GB of RAM and 20 GB of free storage space for model and library installation. A stable and reasonably fast internet connection is recommended for downloading the required resources. For Part 2, some familiarity with Python and access to a working Python environment (with Jupyter Notebook support) will help run example code and exercises.
To register your participation in this workshop click on the “Going” button above. You will then receive an email notification as soon as facilitators post an update. Watch out to not register for two parallel workshops.
Zoom Meeting Information
Generative AI in a Nutshell — 
What Is ChatGPT Doing and Why Does It Work? —
Hands-on Tools for the Workshop
Preparation Tips
Early Help Session