How to Run a Locally Hosted AI Coding Assistant in VSCode

In this article, we will explore how to connect your locally installed Large Language Model (LLM) to VSCode using the Continue extension, allowing you to optimize your coding sessions. This process can be completed in under 5 minutes, and we will guide you through it step-by-step.

Introduction to Locally Hosted AI Coding Assistants

Introduction to the video, explaining the purpose of the tutorial The video begins by introducing the concept of locally hosted AI coding assistants and their benefits. The speaker explains that they will show how to connect a locally installed LLM to VSCode, making it possible to optimize coding sessions.

Requirements for the Tutorial

Requirements for following the tutorial, including VSCode and LLM installation To follow this tutorial, you need to have VSCode installed on your machine. Additionally, you should have a locally installed LLM, such as Alama, and have it running. If you don't know how to install Alama and the LLM, the speaker refers to a previous video that explains the process in 10 minutes.

Installing the Continue Extension

Installing the Continue extension in VSCode After ensuring you have VSCode and a locally installed LLM, you need to install the Continue extension. This extension allows you to connect your LLM to VSCode. The speaker guides you through the process of searching for and installing the Continue extension.

Configuring the Continue Extension

Configuring the Continue extension to work with your locally installed LLM Once the Continue extension is installed, you need to configure it to work with your locally installed LLM. The speaker explains how to open the config.js file of the Continue extension and add your LLM settings. This includes specifying the models you have installed on your LLM server and configuring the provider, IP address, and port number.

Adding Your LLM Settings

Adding your LLM settings to the config.js file The speaker provides a detailed explanation of how to add your LLM settings to the config.js file. This includes adding the model names, system messages, and other configurations necessary for the Continue extension to work with your LLM.

Testing the Configuration

Testing the configuration to ensure it works as expected After configuring the Continue extension, the speaker tests the configuration by asking the LLM to generate a piece of Python code. The LLM successfully generates the code, demonstrating that the configuration is correct.

Auto Completion and Code Generation

Auto completion and code generation using the LLM The speaker also demonstrates the auto completion feature of the LLM, which can suggest code completions as you type. Additionally, the LLM can generate code based on a prompt, making it a powerful tool for coding.

Conclusion and Next Steps

Conclusion and next steps for using the LLM with VSCode The speaker concludes the tutorial by summarizing the steps taken to connect a locally installed LLM to VSCode using the Continue extension. They also provide next steps for using the LLM with VSCode, including exploring the auto completion feature and generating code.

Final Thoughts and Future Plans

Final thoughts and future plans for using the LLM with VSCode The speaker shares their final thoughts on the potential of using a locally installed LLM with VSCode, including the possibility of adding a separate rack system. They also invite viewers to share their thoughts and suggestions for future tutorials.

Conclusion and Call to Action

Conclusion and call to action, including links to additional resources The speaker concludes the video by thanking viewers for watching and inviting them to like, subscribe, and comment on the video. They also provide links to additional resources, including their GitHub repository and social media channels.