AI/ML

Set Up Qwen 2.5 in Docker with Ollama on Azure VM - Complete Guide

Name: OneClick IT Consultancy P Limited
Address: 407-412, President Plaza Opp. Titanium Square Thaltej, Ahmedabad, Gujarat, 380054, India
Telephone: +1(802) 684-0486
Price range: $$$

Qwen 2.5 Model for your Business?

Cost Efficiency (Open Source)
Lower Long Term costs
Customised data control
Pre-trained model

Get Your Qwen 2.5 AI Model Running in a Day

Need technical help?

Our experts will get back to you within 24 hours.

Free Installation Guide - Step by Step Instructions Inside!

Introduction

Qwen 2.5 is a robust AI model built for advanced text processing and natural language understanding. Deploying it within a Docker container on an Azure Virtual Machine (VM) provides a streamlined, scalable and isolated execution environment.

Prerequisites

Before proceeding, make sure you have:

An Azure VM (Ubuntu recommended) with Docker installed.
SSH access to the instance.

Step 1: Connect to Your Azure VM

Access your VM by executing the following command in your terminal:

ssh -i your-private-key.pem azure-user@your-vm-ip

Once connected, update the package lists:

sudo apt update && sudo apt upgrade -y

Step 2: Deploy the Ollama Container

Launch an Ollama container to manage the model execution:

docker run -d --name ollama -v ollama:/root/.ollama -p 11434:11434 ollama/ollama

Step 3: Enter the Ollama Container

After the container is up and running, gain shell access to it:

docker exec -it ollama /bin/bash

Step 4: Download the Qwen 2.5 Model

Retrieve the Qwen 2.5 model inside the Ollama container:

ollama pull qwen:2.5

This command ensures all dependencies are properly installed.

Step 5: Start the Model

Start the model with the following command:

ollama run qwen:2.5

Test it by entering a prompt:

>>> Describe the history of artificial intelligence.

Step 6: Enable Web-Based Interaction

To interact with Qwen 2.5 through a browser, deploy the Open WebUI tool:

docker run -d -p 3000:8080 -e OLLAMA_BASE_URL=http://<YOUR-VM-IP>:11434 -vopen-webui:/app/backend/data --name open-webui --restart alwaysghcr.io/open-webui/open-webui:main

Now, open http://<YOUR-VM-IP>:3000 in a browser to start using Qwen 2.5 via a web interface.

Conclusion

By running Qwen 2.5 on an Azure Virtual Machine with Docker and Ollama, you achieve an efficient and scalable AI deployment. This approach simplifies model execution while maintaining an isolated and reusable setup for various workloads.

Ready to elevate your business with cutting edge AI and ML solutions? Contact us today to harness the power of our expert technology services and drive innovation.