Skip to main content

MarvinOS Local AI Stack: Fully Self-Hosted AI Experimentation

  



MarvinOS Local AI Stack: Fully Self-Hosted AI Experimentation

Imagine having a powerful, fully self-hosted AI experimentation platform that allows you to explore the possibilities of artificial intelligence without relying on cloud dependencies or compromising on security. That's exactly what we're excited to introduce today — the MarvinOS Local AI Stack!

This innovative tool is designed for local AI experimentation, secure internal LLM access, GPU-accelerated inference, and even supports offline or air-gapped environments. It also includes Retrieval-Augmented Generation (RAG) with local document embeddings, enabling long-term knowledge bases without leaving your machine.

Key Features & Benefits

MarvinOS Local AI Stack offers a wide range of features that make it an ideal choice for those who value autonomy and control over their AI experiments:

  • Fully local LLM inference: No cloud dependency means you're in complete control.

  • RAG with Qdrant vector database: Create local knowledge bases and retrieve relevant context with citations.

  • GPU-accelerated (NVIDIA): Leverage the power of your NVIDIA GPU to accelerate your experimentation.

  • HTTPS ingress with Nginx: Ensure secure access to your AI experiments through a trusted interface.

  • Web-based chat UI (Open WebUI): A seamless user experience for interacting with your LLM models.

  • Private web search grounding (SearXNG): Protect sensitive data by keeping it internal and not exposing it to the public internet.

  • Local OpenAI-compatible TTS: Generate speech using OpenAI's technology, but without relying on cloud services.

  • Stable Diffusion-based image creation: Unleash your creativity with our local GPU-accelerated image generation capabilities.

Getting Started

Getting started is easy! Simply follow these steps:

git clone git@github.com:MarvinOS-online/fitlab-core.git
cd fitlab-core
docker compose up --build -d

By doing so, you'll have a fully functional AI experimentation platform that's tailored to your needs, including LLMs, RAG, TTS, Web Search, and Image Generation.

Prerequisites

Before diving in, please ensure that:

  • You're running on a Linux host (recommended)

  • Your Docker version is at least 24+

  • You have Docker Compose v2 installed

  • NVIDIA drivers and Container Toolkit are installed

  • Verify GPU support:

docker run --rm --gpus all nvidia/cuda:12.2.0-base nvidia-smi

Common Operations & Security

For a seamless experience:

  • Restart the stack:

docker compose restart

  • Stop the stack:

docker compose down

  • View logs in real-time:

docker compose logs -f open-webui
docker compose logs -f ollama
docker compose logs -f qdrant
docker compose logs -f ingress

Security notes:

  • Self-signed SSL certificates are used by default

  • Open WebUI authentication is enabled

  • Replace placeholder secrets before exposure

  • SearXNG is intended for internal use only

Licensing & Attribution

MarvinOS Local AI Stack is proudly open source. Components are released under the following licenses:

  • Ollama — Apache 2.0

  • Open WebUI — MIT

  • Nginx — BSD-like

  • SearXNG — AGPLv3

  • Automatic1111 Stable Diffusion — GPL-compatible

This orchestration layer is designed to be freely used, modified, and adapted to your needs.

Support & Donate

If you find MarvinOS Local AI Stack useful and want to help support ongoing development, consider contributing:

  • Donations help cover server costs, development time, and model hosting

  • Any contribution — big or small — keeps the project sustainable and improves future features like RAG, TTS, and GPU performance tuning

Your support ensures that MarvinOS remains free, open, and cutting-edge for everyone in the AI community.



Conclusion

MarvinOS Local AI Stack represents a significant step forward in empowering the AI community. With its self-hosted, GPU-accelerated design and local RAG capabilities, it's an ideal choice for those seeking control, security, and flexibility.

If you're ready to explore the vast potential of AI without compromising your values, join us today by trying MarvinOS Local AI Stack — and consider supporting the project to keep it growing!

Sources & Learn More:
https://marvinos.online/about


Comments

Post a Comment

Popular posts from this blog

How I Built an AI App to Help Students Pass the New Jersey Driver’s Test — and Why Personalized Learning Matters

  How I Built an AI App to Help Students Pass the New Jersey Driver’s Test — and Why Personalized Learning Matters Studying for the New Jersey Driver’s Test can be stressful. The manual is long, the rules are specific, and traditional study methods usually take a one-size-fits-all approach: read the book, take the same practice tests as everyone else, and hope it sticks. I built a Dify-powered AI app to change that. The goal was simple: help people study smarter, not harder — by using AI to adapt to each individual student , instead of forcing everyone through the same cookie-cutter experience. 👉 You can try the app here: https://marvinos.online:8093/chat/VkJxPJhtWC6vQc0W The Problem with Traditional Test Prep Most driver’s test prep tools work the same way: Static practice questions Generic explanations No awareness of what you already know or struggle with But not all students are the same. Some people struggle with: Road signs Right-of-way rules DUI and point system questions ...
Post a Comment
Read more

How I Built an AI App to Help People Pass the USCIS 2025 Civics Test — and Why Personalized Learning Matters

  How I Built an AI App to Help People Pass the USCIS 2025 Civics Test — and Why Personalized Learning Matters Preparing for the USCIS Civics Test is one of the most important steps on the path to U.S. citizenship. It’s not just a test of memorization — it’s about understanding American history, government, and civic values. For many applicants, studying can be intimidating. The material may be unfamiliar, English may not be a first language, and traditional study tools often assume everyone learns the same way. That’s why I built a Dify-powered AI app designed specifically to help people prepare for the USCIS 2025 Civics Test — with a focus on clarity, personalization, and confidence. 👉 Try the app here: https://marvinos.online:8093/chat/eaJZnnTyehL6eaqC The Challenge with Traditional Civics Test Prep Most civics prep resources rely on: Static question lists Memorization-heavy flashcards One-size-fits-all explanations But USCIS applicants come from different backgrounds, culture...
Post a Comment
Read more