ollama安裝和運行llama3.1 8b

conda create -n ollama python=3.11 -y
conda activate ollama
curl -fsSL https://ollama.com/install.sh | sh
ollama run songfy/llama3.1:8b

就這么簡單就能運行起來了.

我們可以在命令行中與他交互.

當然我們也可以用接口訪問:

curl http://localhost:11434/api/generate -d '{
  "model": "songfy/llama3.1:8b",
  "prompt":"Why is the sky blue?"
}'

curl http://localhost:11434/api/chat -d '{
  "model": "songfy/llama3.1:8b",
  "messages": [
    { "role": "user", "content": "why is the sky blue?" }
  ]
}'

安裝open-webui

vim /etc/systemd/system/ollama.service, 增加Environment

vim /etc/systemd/system/ollama.service

########## 內容 ###########################################################
[Unit]
Description=Ollama Service
After=network-online.target

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=/root/anaconda3/envs/ollama/bin:/root/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin"

[Install]
WantedBy=default.target

systemctl daemon-reload
systemctl enable ollama
systemctl restart ollama

docker run -d -p 8801:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

這個8801是我們開放的端口之一.

啟動以后, 我們就可以用: ip: 對應外網端口訪問.

我們會看到一個注冊頁面:

因為是私服, 直接隨便注冊一個, 登錄進去:

然后就可以使用了:

就這樣, 也沒啥大用.

生成openai兼容的api

端口轉發:

cat > /etc/yum.repos.d/nux-misc.repo << EOF
[nux-misc]
name=Nux Misc
baseurl=http://li.nux.ro/download/nux/misc/el7/x86_64/
enabled=0
gpgcheck=1
gpgkey=http://li.nux.ro/download/nux/RPM-GPG-KEY-nux.ro
EOF
yum -y --enablerepo=nux-misc install redir

redir --lport=8802 --caddr=0.0.0.0 --cport=11434

這樣就可以使用python進行調用

from openai import OpenAI

client = OpenAI(
    base_url='http://{ip}:{port}/v1/',
    api_key='ollama',  # 此處的api_key為必填項，但在ollama中會被忽略
)

completion = client.chat.completions.create(
model="songfy/llama3.1:8b",
messages=[
{"role": "user", "content": "寫一個c++快速排序代碼"}
])

print(completion.choices[0].message.content)

返回:

?```cpp
#include <iostream>

void swap(int &a, int &b) {
    int temp = a;
    a = b;
    b = temp;
}

void quickSort(int arr[], int left, int right) {
    if (left < right) {
        int pivotIndex = partition(arr, left, right);
        
        // Recursively sort subarrays
        quickSort(arr, left, pivotIndex - 1);
        quickSort(arr, pivotIndex + 1, right);
    }
}

int partition(int arr[], int left, int right) {
    int pivot = arr[right]; 
    int i = left - 1;

    for (int j = left; j < right; j++) {
        if (arr[j] <= pivot) {
            i++;
            swap(arr[i], arr[j]);
        }
    }

    swap(arr[i + 1], arr[right]);

    return i + 1;
}

void printArray(int arr[], int size) {
    for (int i = 0; i < size; i++)
        std::cout << arr[i] << " ";
    std::cout << "\n";
}

// Example usage
int main() {
    int arr[] = {5, 2, 8, 1, 9};
    int n = sizeof(arr) / sizeof(arr[0]);

    quickSort(arr, 0, n - 1);

    printArray(arr, n);
    return 0;
}
?```
輸出:
 `1 2 5 8 9`

快速排序是基于兩個下降數組 partitioned，然后遞歸地對每個子數組進行相似的操作的算法。`partition()`函數根據列表中最小或最大值來劃分數據，并在每次重復時，將列表分割為左邊較大的值和右邊較小的值。