Implement LLM Code Generation with Magicoder

Special LLM for any fully open-source coding task

Large Language Model (LLM) has recently become big and has changed how we work. I don’t even remember when I didn’t have a GPT model to help me write an article.

Implementation of LLM already exists in many tasks, for example, answering questions, translating language, planning businesses, and debugging code.

With many tasks that LLM could tackle, research is needed to solve the requirements. For example, I have previously written about Gorilla LLM, an LLM for any open-source API.

In this newsletter, I want to introduce another SOTA LLM research for various code tasks called Magicoder. It would be important in the future as I imagine this model could help streamline any coding tasks.

So, how does it work? Let’s get into it.

For my paid subscribers, you can read the next section anytime. For the free subscriber, the post would be unlocked in 3 days.

Magicoder

Magicoder is a 7B-LLM model developed by Wei et al. (2023) that trained on 75K synthetic data generated by OpenAI models using OSS-Instruct. The OSS-Instruct is a new approach proposed by the Magicoder researcher. The new method is used to mitigate LLMs' inherent bias and improve code-generation tasks via direct learning from the open source.

The overall OSS-Instruct LLM building and solution are shown in the image below.

OSS-Instruct Methodology (Wei et al. (2023))

Trained with the massive open-source codebase, Magicoder allows the user to access any requirement for your coding tasks. Performance-wise, the Magicoder model could outperform many of the other LLMs.

Magicoder evaluation performance benchmark (Wei et al. (2023))

That’s the simple introduction of Magicoder. Let’s try out the Magicoder LLM to understand how the model works.

Magicoder Tutorial

Installing the Transformers and Torch Python packages in your environment is requisite for using this tutorial, so make sure they are there. With everything in place, we could experiment with the Magicoder.

First, we would establish the Transformers pipeline for Magicoder.

from transformers import pipeline
import torch

generator = pipeline(
    model="ise-uiuc/Magicoder-S-DS-6.7B",
    task="text-generation",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

Next, we would try to acquire Python code for specific tasks. For example, we would try the code-generation task to provide high-level API for the TODO list application. We would set the prompt similar to the image below.

MAGICODER_PROMPT = """You are an exceptionally intelligent coding assistant that consistently delivers accurate and reliable responses to user instructions.

@@ Instruction
{instruction}

@@ Response
"""

instruction = "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."
prompt = MAGICODER_PROMPT.format(instruction=instruction)

Once the prompt is there, we will try to generate the code with the following code.

result = generator(prompt, max_length=2048, num_return_sequences=1, temperature=0.0)
print(result[0]["generated_text"])

The code above uses temperature parameters to control the output randomness or creativity. As we set it to 0, the result would always be the same.

Overall, the result should be similar to the code below.

@@ Instruction
Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception.

@@ Response
Here is a simple implementation of a TODO list API in Python. This API allows for the following operations:

- `add_task(task)`: Adds a task to the list.
- `remove_task(task)`: Removes a task from the list.
- `mark_task_as_done(task)`: Marks a task as done.
- `get_tasks()`: Returns the current list of tasks.

```python
class TodoList:
    def __init__(self):
        self.tasks = []

    def add_task(self, task):
        if not isinstance(task, str):
            raise ValueError("Task must be a string")
        self.tasks.append({"task": task, "done": False})

    def remove_task(self, task):
        if task not in [t["task"] for t in self.tasks]:
            raise ValueError("Task not found")
        self.tasks = [t for t in self.tasks if t["task"]!= task]

    def mark_task_as_done(self, task):
        if task not in [t["task"] for t in self.tasks]:
            raise ValueError("Task not found")
        for t in self.tasks:
            if t["task"] == task:
                t["done"] = True

    def get_tasks(self):
        return [t for t in self.tasks if not t["done"]]
```

You can use this API like this:

```python
todo = TodoList()
todo.add_task("Buy milk")
todo.add_task("Clean the house")
todo.mark_task_as_done("Buy milk")
print(todo.get_tasks())  # Output: ['Clean the house']
todo.remove_task("Buy milk")
print(todo.get_tasks())  # Output: ['Clean the house']
```

This API is very basic and does not include any persistence (i.e., it will not remember the tasks across different sessions). If you want to persist the tasks, you would need to add some kind of storage mechanism, such as a database.

For another example, we would try to write a prompt to generate code to develop a simple LLM application with Streamlit and Langchain.

MAGICODER_PROMPT = """You are an exceptionally intelligent coding assistant that consistently delivers accurate and reliable responses to user instructions.

@@ Instruction
{instruction}

@@ Response
"""

instruction = "Implement a simple Python source code for simple LLM application with Streamlit and Langchain"

prompt = MAGICODER_PROMPT.format(instruction=instruction)

The prompt was passed to our previous pipeline, but I tweaked the parameter slightly because I expected some changes.

result = generator(prompt, max_length=2048, num_return_sequences=1, temperature=0.5)
print(result[0]["generated_text"])

The result is shown in the code output below.

You are an exceptionally intelligent coding assistant that consistently delivers accurate and reliable responses to user instructions.

@@ Instruction
Implement a simple Python source code for simple LLM application with Streamlit and Langchain

@@ Response
Here is a simple Python source code for a simple LLM application with Streamlit and Langchain.

```python
import streamlit as st
from langchain.llms import OpenAI
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate

# Set up the OpenAI LLM
llm = OpenAI(temperature=0.9)

# Set up the prompt template
prompt_template = """
You are a helpful assistant.

{input}
"""
prompt = PromptTemplate(template=prompt_template, input_variables=["input"])

# Set up the LLMChain
llm_chain = LLMChain(llm=llm, prompt=prompt)

# Set up the Streamlit app
st.title("Simple LLM Application")

# Get user input
user_input = st.text_input("Enter your question:")

# If user input is not empty, generate response
if user_input:
    response = llm_chain.run(input=user_input)
    st.write("Response: ", response)
```

This code creates a simple Streamlit application that takes user input, uses Langchain to generate a response, and then displays the response.

Please note that you need to install the necessary libraries (streamlit, langchain, openai) before running the code. You can install them using pip:

```bash
pip install streamlit langchain openai
```

Also, you need to set up your OpenAI API key in the environment variable `OPENAI_API_KEY`.

This is a very basic example and does not include any error handling or other advanced features. For a more complex application, you would likely want to use a more sophisticated prompt template, chain, or other components.

The output is concise yet valuable, as the information we need is also there to implement the code. This would prove helpful to avoid any errors in the future.

You could experiment further with the code to understand Magicode even further. For the Notebook experiment, I would put it here.

I hope it helps!

---

Thank you, everyone, for subscribing to my newsletter. If you have something you want me to write or discuss, please comment or directly message me through my social media!