5 Self-Hosted LLMs We Use for Specific Tasks

As active contributors to the Magisk Modules Repository, we are constantly seeking innovative solutions to enhance our workflow and improve the quality of our contributions. We’ve found that large language models (LLMs) can be invaluable tools, but relying on third-party services often raises concerns about data privacy and control. That’s why we’ve explored the world of self-hosted LLMs, and we’re excited to share five models that have significantly improved our efficiency in specific areas. These aren’t just general-purpose models; they’re finely tuned for coding, math, web content conversion, safety analysis, and creative text generation. By self-hosting, we maintain complete control over our data, ensuring confidentiality and compliance with our internal policies.

1. Code Generation and Debugging with StarCoder

Why StarCoder is Our Go-To for Code Tasks

For code generation, debugging, and documentation, we rely heavily on StarCoder. This LLM, developed by BigCode, excels at understanding and generating code in various programming languages. Its training dataset is massive, encompassing code from GitHub repositories, allowing it to learn diverse coding styles and conventions. What sets StarCoder apart is its ability to handle complex coding tasks with a high degree of accuracy. We found that when used correctly, it vastly reduces the amount of time needed to test our modules within the Magisk Modules Repository.

Practical Applications in Magisk Module Development

We primarily use StarCoder for:

• Generating boilerplate code: When creating new Magisk modules, StarCoder helps us quickly generate the basic structure and essential functions, saving valuable time. For example, we can input a description of the desired module functionality (e.g., “a module that disables battery charging when the battery level reaches 80%”) and StarCoder will generate the initial code framework.

• Debugging existing code: StarCoder’s ability to analyze code and identify potential errors is remarkable. We’ve used it to pinpoint bugs in complex scripts, often providing suggested fixes that are remarkably accurate. We can give StarCoder a code snippet and it can analyze it, give feedback or provide fixes on the code.

• Adding comments and documentation: Maintaining clear and concise documentation is crucial for any software project. StarCoder can automatically generate comments and documentation for our code, ensuring that it’s easy to understand and maintain. The documentation is much more verbose, and helps new developers easily grasp the intention and code structure.

Setting Up StarCoder Locally

Running StarCoder locally requires significant computational resources, including a powerful GPU and sufficient RAM. We use a dedicated server with an NVIDIA RTX 3090 to ensure optimal performance. The setup process involves:

1. Installing the necessary dependencies: This includes Python, PyTorch, and the Transformers library from Hugging Face.

2. Downloading the StarCoder model: The model can be downloaded from the Hugging Face Model Hub.

3. Configuring the inference pipeline: We use the Transformers library to create an inference pipeline that allows us to interact with the model.

4. Fine-tuning (optional): For specific tasks, we can fine-tune StarCoder on our own codebase to improve its performance. This requires preparing a dataset of code examples and training the model using PyTorch.

2. Mathematical Calculations and Problem Solving with MathGPT

Precision and Accuracy in Mathematical Tasks

MathGPT is our preferred choice for handling mathematical calculations and problem-solving tasks within the Magisk Modules Repository. Unlike general-purpose LLMs, MathGPT is specifically trained on a massive dataset of mathematical equations, theorems, and proofs. This specialized training enables it to perform complex calculations with a high degree of accuracy and solve intricate mathematical problems with ease. General-purpose LLMs sometimes give the wrong math answer, and MathGPT avoids that problem.

How We Leverage MathGPT in Our Workflow

We utilize MathGPT for:

• Validating numerical results: When analyzing performance data or conducting simulations, we use MathGPT to verify the accuracy of our numerical results.

• Solving optimization problems: Many Magisk modules involve optimization problems, such as finding the optimal configuration for a particular setting. MathGPT helps us formulate and solve these problems efficiently.

• Generating mathematical expressions: For documentation and explanations, MathGPT can generate clear and concise mathematical expressions, improving the readability of our materials.

Local Deployment and Usage of MathGPT

Deploying MathGPT locally is similar to deploying other LLMs, but it may require specific libraries for handling mathematical expressions. Our setup involves:

1. Installing the required libraries: This includes NumPy, SciPy, and SymPy, in addition to the standard deep learning libraries.

2. Downloading the MathGPT model: The model can be obtained from the Hugging Face Model Hub or other repositories.

3. Creating a custom inference pipeline: We develop a custom pipeline that allows us to input mathematical problems and receive accurate solutions.

3. Converting Web Content to Markdown with LLM-WebConverter

Streamlining Content Creation and Management

Converting web content into Markdown format is a frequent task for us. Maintaining the Magisk Module Repository involves converting HTML pages, blog posts, and other online resources into Markdown for documentation and module descriptions. LLM-WebConverter is a specialized LLM that excels at this conversion process. It accurately translates HTML elements, including headings, lists, tables, and images, into corresponding Markdown syntax. This saves us considerable time and effort compared to manual conversion.

Real-World Applications in Our Workflow

We employ LLM-WebConverter for:

• Creating module documentation: We automatically convert web pages containing module documentation into Markdown files for inclusion in our repository.
• Archiving online resources: We convert important online resources into Markdown format to preserve them for future reference.

Setting up LLM-WebConverter Locally

This process is relatively straightforward:

1. Ensure prerequisites are met: As with most LLMs, this includes Python and associated libraries like Transformers and BeautifulSoup4 (for initial HTML parsing).

2. Download and deploy the model: Obtain the LLM-WebConverter model from its source (often Hugging Face).

3. Develop an HTML to Markdown conversion script: This script uses the model to process HTML input and generate Markdown output.

4. Content Safety Analysis with Detoxify

Ensuring a Positive and Inclusive Community

Maintaining a safe and respectful community within the Magisk Modules Repository is of utmost importance to us. We use Detoxify, a powerful LLM designed for content safety analysis. Detoxify can identify toxic language, hate speech, and other harmful content in text. By integrating Detoxify into our moderation workflow, we can proactively identify and address potential issues, ensuring a positive and inclusive environment for all users.

How We Use Detoxify to Moderate Content

We integrate Detoxify into our:

• Comment moderation system: Detoxify automatically scans user comments and flags potentially toxic content for review.

• Module description analysis: We use Detoxify to ensure that module descriptions are free from harmful language.

Implementing Detoxify for Local Content Analysis

1. Install Detoxify: Using pip, install the Detoxify library.

2. Load the model: Instantiate the Detoxify model.

3. Analyze text: Feed text through the model to get toxicity scores.

4. Set thresholds: Define toxicity thresholds for flagging content.

5. Creative Text Generation with GPT-2

Unleashing Creativity for Module Descriptions and Blog Posts

While models like GPT-3 and GPT-4 are renowned for creative text generation, running them locally can be challenging due to their size and computational requirements. We’ve found that GPT-2, a smaller but still capable LLM, is well-suited for our needs. We use GPT-2 to generate creative text for module descriptions, blog posts, and other content, often using its generated text as a starting point before manually fine-tuning it.

Practical Applications of GPT-2 in Content Creation

We leverage GPT-2 for:

• Generating module descriptions: GPT-2 helps us create engaging and informative module descriptions that attract users to our repository.

• Writing blog posts: GPT-2 can generate initial drafts of blog posts, saving us time and effort.

• Brainstorming new ideas: We use GPT-2 to generate random ideas and explore new possibilities for our projects.

Running GPT-2 Locally

1. Install the Transformers library: The Transformers library from Hugging Face provides a convenient way to access and use GPT-2.

2. Download the GPT-2 model: Download the pre-trained GPT-2 model from the Hugging Face Model Hub.

3. Generate text: Use the Transformers library to generate text based on a given prompt.

Conclusion: The Power of Self-Hosted LLMs

Self-hosted LLMs offer a powerful way to enhance productivity, maintain data privacy, and customize AI models for specific tasks. By leveraging models like StarCoder, MathGPT, LLM-WebConverter, Detoxify, and GPT-2, we’ve significantly improved our workflow within the Magisk Modules Repository. While setting up and managing these models requires technical expertise and computational resources, the benefits of control, privacy, and customization are well worth the effort. These tools are not just about automation; they are about empowering us to be more creative, efficient, and effective in our work. By exploring the world of self-hosted LLMs, you too can unlock new possibilities and achieve remarkable results.