We ❤️ Open Source
A community education resource
Getting started with Jan: A private, open source alternative to ChatGPT
Your fast-start guide to Jan: Local GPT models, private AI, and quick prototyping.
Are you looking for a locally hosted, open source GPT that protects your privacy while letting you tap into the power of modern AI models? Then you should check out Jan.
Jan is a ChatGPT alternative that runs entirely offline on your desktop. It’s open source under the Apache 2.0 license, easy to install on Linux, macOS, and Windows, and supported by excellent documentation.
The goal of Jan is simple: Make it easy for anyone, whether you’re a developer or just AI-curious, to run large language models (LLMs) with full control and complete data privacy. Everything is stored locally, so your data stays on your machine. Internet access is optional. You can choose your own AI models, including both local and cloud-based options, without worrying about data collection or third-party tracking.
Jan is powered by Llama.cpp, a fast local AI engine that also supports an OpenAI-compatible API. This means you can use Jan in many of the same ways you’d use ChatGPT or other hosted models, right from your laptop.
Read more: Getting started with AI on a budget
Getting started with Jan
I installed Jan on Linux Mint 22.1 using the .deb package. On my M3 MacBook Air, I used the macOS version. If you’re on a non-Debian-based Linux distro like Fedora, there’s an AppImage version available. The setup process was smooth thanks to the project’s clear and thorough documentation.
Following the setup guide, I downloaded and installed the jan-nano-128K model to start experimenting. I wanted to see how well it handled simple programming requests, so I asked it to write a small Python web app that converts Fahrenheit to Celsius using Flask.
After about 15 seconds, Jan generated the following code, which I copied into VSCodium and saved as a Python file:
from flask import Flask, request, render_template
app = Flask(__name__)
@app.route('/', methods=['GET', 'POST'])
def convert_temp():
result = None
if request.method == 'POST':
fahrenheit = float(request.form['fahrenheit'])
celsius = (fahrenheit - 32) * 5/9
result = celsius
return render_template('convert.html', result=result)
if __name__ == '__main__':
app.run(debug=True)It also generated an HTML file to go along with it:
<!DOCTYPE html>
<html>
<head>
<title>Fahrenheit to Celsius Converter</title>
</head>
<body>
<h1>Fahrenheit to Celsius Converter</h1>
<form method="post">
<label for="fahrenheit">Enter Fahrenheit:</label>
<input type="text" id="fahrenheit" name="fahrenheit" required>
<button type="submit">Convert</button>
</form>
{% if result is not none %}
<h2>Result: {{ result }}°C</h2>
{% endif %}
</body>
</html>Read more: 15 overlooked rules of AI
To test the app, I created a Python virtual environment:
python3 -m venv temperatureI saved both files in the environment, then launched the app:
python3 temperature.pyOpening a browser at https://127.0.0.1:5000, I was greeted with the web interface and successfully converted Fahrenheit to Celsius using the Jan-generated code.
More to explore
Jan’s documentation is a must-read, and the community is very active across GitHub, Discord, and LinkedIn. There’s also a regularly updated blog with additional tutorials and examples.
If you’re looking to experiment with LLMs in a privacy-conscious, open source way, Jan is a fantastic place to start. I hope this fast-start guide gets you up and running quickly.
More from We Love Open Source
- 15 overlooked rules of AI
- Getting started with AI on a budget
- What if your AI agent could actually help?
- Build better with AI: Lessons from real-world GenAI projects
- How I used Pinokio to run OpenAudio and clone a voice in seconds
This article is adapted from “Introducing Jan: A privacy-focused, locally hosted open source GPT for AI enthusiasts” by Don Watkins, and is republished with permission from the author.
The opinions expressed on this website are those of each author, not of the author's employer or All Things Open/We Love Open Source.