AI/ML
Deploying Qwen7B in Docker with Ollama - A Complete Setup Guide
Need technical help?
Our experts will get back to you within 24 hours.
Problem
You want to run Qwen7B inside a Docker container for portability and isolation but are unsure about setting up Ollama within the container.
Solution
- Create a Dockerfile to install Ollama inside a container.
- Pull and run Qwen7B inside the container.
- Allow the container to interact with the host system for efficient execution.
1. Create a Dockerfile
Define a Dockerfile to install Ollama and set up Qwen7B inside a container.
# Use an Ubuntu base imageFROM ubuntu:22.04 # Set environment variables to prevent interactive promptsENV DEBIAN_FRONTEND=noninteractive # Install dependenciesRUN apt update && apt install -y curl # Install OllamaRUN curl -fsSL https://ollama.com/install.sh | sh # Download and preload the Qwen 7B modelRUN ollama pull qwen:7b # Expose the port for API access (optional)EXPOSE 11434 # Run Ollama as the default processCMD ["ollama", "serve"] 2. Build the Docker Image
Navigate to the directory where the Dockerfile is saved and build the image:
docker build -t qwen-7b-container . This will create a Docker image named qwen-7b-container.
3. Run the Container
Run the container and start Ollama:
docker run --rm -it -p 11434:11434 qwen-7b-container
This starts the container and exposes Ollama’s API on port 11434.
4. Interact with Qwen from the Host
Now that the container is running, you can execute prompts from outside the container using:
ollama run qwen:7b "Tell me about black holes."
OR, send a request via curl :
curl http://localhost:11434/api/generate -d '{ "model": "qwen:7b", "prompt": "Explain quantum mechanics in simple terms"}' 5. Additional
There are servral model sizes, including 0.5B, 1.8B, 4B (default), 7B, 14B, 32B (new) and 72B. If you want to test another model replace tag 7b in the Dockerfile with the required model as mentioned here. Also see the size of model in below table.
- 0.5B-395MB
- 1.8B-1.1GB
- 4B -2.3GB
- 7B -4.5GB
- 14B -8.2GB
- 32B-18GB
- 72B-41GB
Conclusion
Running Qwen 7B in a Docker container provides portability, isolation, and easy deployment across multiple environments. You can now interact with the AI model using local commands or API requests.
Ready to transform your business with our technology solutions? Contact Us today to Leverage Our AI/ML Expertise.
0
