From the course: Responsible Generative AI and Local LLMs

Unlock this course with a free trial

Join today to access over 24,700 courses taught by industry experts.

The chat loop with StarCoder

The chat loop with StarCoder

From the course: Responsible Generative AI and Local LLMs

Start my 1-month free trial Buy for my team

The chat loop with StarCoder

“

- [Instructor] One of the more exciting things you can do with chat loops using large language models is build your own coding assistant. With HuggingFace StarCoder, it has been able to outperform even earlier models from OpenAI or competing solutions. And if we take a look at this particular example here, here's how you could actually dive into and build your own custom conversational AI. So if you look at a prompt here, we would want to make some thoughtful response. So you would call out to a StarCoder endpoint. This could be hosted on a GPU instance, let's say AWS G5. And then the async request from the client would allow non-blocking asynchronous requests, so the chatbot would be able to respond very easily. Also, you'd be able to append each message pair to the conversation history to provide context for the AI. And then the Serde JSON crate would allow you to do easy serialization using the JSON format. And we could leverage Rust's strong typing system and memory safety so that…

Contents