As I delved into the world of artificial intelligence (AI) and language models, my curiosity grew about self-hosting these powerful tools on my own machine. With colleagues at XDA sharing their experiences with specialized LLMs, I decided to give it a try.
I began by exploring qwen2.5-coder, a dedicated LLM for coding tasks. This model is trained on an enormous dataset of code and provides a deep understanding of over 40 programming languages. I chose the 7B model, which proved perfect for writing new code, fixing bugs, and repairing broken code.
Next, I discovered wizard-math, a specialized version of the WizardLM family designed to excel in complex mathematical problems, logical reasoning, and solving puzzles. This model helped me test my solutions and explore new ways to approach difficult problems.
For converting HTML content into markdown files, I self-hosted reader-lm. While it struggles with large or messy HTML code, it works well enough most of the time and makes a huge difference in productivity.
To ensure safe interactions with LLMs, I set up llama-guard3, a powerful model that acts as a dedicated content moderation tool for all my local LLMs. It checks prompts against 13 different safety categories and provides feedback on their safety.
I also self-hosted Gemma 3, which offers a premium experience with the flexibility of running locally. This model is built on the same research as Gemini and can handle massive context windows, process text and images, and understand over 140 languages.
While self-hosting these AI assistants has been incredibly productive, I’ve learned that relying solely on local models can lead to speed and reliability issues for complex or important tasks. A hybrid approach, combining the benefits of both local and cloud-based services, offers the most flexibility and reliability. By leveraging the privacy of local models while still accessing cutting-edge capabilities, I’ve found a balance that enhances my productivity and works best for me.
Source: https://www.xda-developers.com/self-hosted-llms-i-use-for-specific-tasks