3 분 소요

로컬에서 LLM을 사용하기

환경 구성

개발 환경 : Retina 5K, 27-inch, 2019 iMac

PyCharm 2024.1.4 (Professional Edition)

Poetry 설치

Ollama 설치

llama.cpp 설치

Postman 설치

Source

전처리 및 Dirctory 구조

HuggingfaceHub 가상 환경 구성후 llama.cpp를 내부에 clone.

poetry package 설치

(huggingfacehub-py3.11) snoopy_kr@iMac HuggingfaceHub % poetry install

Directory 구조

(huggingfacehub-py3.11) snoopy_kr@iMac HuggingfaceHub %  ls -al
total 31417576
drwxr-xr-x   15 snoopy_kr  staff          480 Jul 19 17:10 .
drwxr-xr-x   12 snoopy_kr  staff          384 Jul 19 13:53 ..
-rw-r--r--@   1 snoopy_kr  staff         6148 Jul 19 14:48 .DS_Store
drwxr-xr-x   12 snoopy_kr  staff          384 Jul 19 17:16 .git
-rw-r--r--    1 snoopy_kr  staff          583 Jul 19 17:10 .gitignore
drwxr-xr-x    9 snoopy_kr  staff          288 Jul 19 17:15 .idea
-rw-r--r--@   1 snoopy_kr  staff          597 Jul 19 14:51 Modelfile
-rw-r--r--    1 snoopy_kr  staff          235 Jul 19 13:55 download.py
drwxr-xr-x  143 snoopy_kr  staff         4576 Jul 19 17:07 llama.cpp
-rw-r--r--    1 snoopy_kr  staff       125506 Jul 19 14:42 poetry.lock
-rw-r--r--    1 snoopy_kr  staff          907 Jul 19 14:42 pyproject.toml

Downloading a HuggingFace model

download.py

from huggingface_hub import snapshot_download
model_id="MLP-KTLim/llama-3-Korean-Bllossom-8B"
snapshot_download(repo_id=model_id, local_dir="llama-3-Korean-Bllossom-8B",
                  local_dir_use_symlinks=False, revision="main")

실행

(huggingfacehub-py3.11) snoopy_kr@iMac HuggingfaceHub %  python download.py

llama-3-Korean-Bllossom-8B 디렉토리 생성 확인

(huggingfacehub-py3.11) snoopy_kr@iMac HuggingfaceHub %  ls -al
total 31417576
drwxr-xr-x   15 snoopy_kr  staff          480 Jul 19 17:10 .
drwxr-xr-x   12 snoopy_kr  staff          384 Jul 19 13:53 ..
-rw-r--r--@   1 snoopy_kr  staff         6148 Jul 19 14:48 .DS_Store
drwxr-xr-x   12 snoopy_kr  staff          384 Jul 19 17:16 .git
-rw-r--r--    1 snoopy_kr  staff          583 Jul 19 17:10 .gitignore
drwxr-xr-x    9 snoopy_kr  staff          288 Jul 19 17:15 .idea
-rw-r--r--@   1 snoopy_kr  staff          597 Jul 19 14:51 Modelfile
-rw-r--r--    1 snoopy_kr  staff          235 Jul 19 13:55 download.py
drwxr-xr-x   15 snoopy_kr  staff          480 Jul 19 13:59 llama-3-Korean-Bllossom-8B
drwxr-xr-x  143 snoopy_kr  staff         4576 Jul 19 17:07 llama.cpp
-rw-r--r--    1 snoopy_kr  staff       125506 Jul 19 14:42 poetry.lock
-rw-r--r--    1 snoopy_kr  staff          907 Jul 19 14:42 pyproject.toml

Converting the model

실행

(huggingfacehub-py3.11) snoopy_kr@iMac HuggingfaceHub % cd llama.cpp 

(huggingfacehub-py3.11) snoopy_kr@iMac llama.cpp % python convert-hf-to-gguf.py ../llama-3-Korean-Bllossom-8B --outfile ..llama-3-Korean-Bllossom-8B.gguf

llama-3-Korean-Bllossom-8B.gguf 파일 생성 확인

(huggingfacehub-py3.11) snoopy_kr@iMac HuggingfaceHub %  ls -al
total 31417576
drwxr-xr-x   15 snoopy_kr  staff          480 Jul 19 17:10 .
drwxr-xr-x   12 snoopy_kr  staff          384 Jul 19 13:53 ..
-rw-r--r--@   1 snoopy_kr  staff         6148 Jul 19 14:48 .DS_Store
drwxr-xr-x   12 snoopy_kr  staff          384 Jul 19 17:16 .git
-rw-r--r--    1 snoopy_kr  staff          583 Jul 19 17:10 .gitignore
drwxr-xr-x    9 snoopy_kr  staff          288 Jul 19 17:15 .idea
-rw-r--r--@   1 snoopy_kr  staff          597 Jul 19 14:51 Modelfile
-rw-r--r--    1 snoopy_kr  staff          235 Jul 19 13:55 download.py
drwxr-xr-x   15 snoopy_kr  staff          480 Jul 19 13:59 llama-3-Korean-Bllossom-8B
-rw-r--r--    1 snoopy_kr  staff  16068891008 Jul 19 14:49 llama-3-Korean-Bllossom-8B.gguf
drwxr-xr-x  143 snoopy_kr  staff         4576 Jul 19 17:07 llama.cpp
-rw-r--r--    1 snoopy_kr  staff       125506 Jul 19 14:42 poetry.lock
-rw-r--r--    1 snoopy_kr  staff          907 Jul 19 14:42 pyproject.toml

Add Model

Modelfile

FROM  llama-3-Korean-Bllossom-8B.gguf

SYSTEM """당신은 유용한 AI 어시스턴트입니다. 사용자의 질의에 대해 친절하고 정확하게 답변해야 합니다. You are a helpful AI assistant, you'll need to answer users' queries in a friendly and accurate manner. 모든 대답은 한국어(Korean)으로 대답해주세요."""


TEMPLATE """
<s></s>
<s>Human:
</s>
<s>Assistant:
"""

PARAMETER temperature 0.6
PARAMETER num_predict 3000
PARAMETER num_ctx 4096
PARAMETER stop <s>
PARAMETER stop </s>
PARAMETER stop <|eot_id|>

실행

(huggingfacehub-py3.11) snoopy_kr@iMac llama.cpp % cd ..

(huggingfacehub-py3.11) snoopy_kr@iMac HuggingfaceHub % ollama create llama-korea -f Modelfile

획인

(huggingfacehub-py3.11) snoopy_kr@iMac HuggingfaceHub % ollama list
NAME                    ID              SIZE    MODIFIED    
llama-korea:latest      f8909476a010    16 GB   3 hours ago     
mistral:latest          2ae6f6dd7a3d    4.1 GB  2 weeks ago     
llama3:latest           365c0bd3c000    4.7 GB  2 weeks ago

Postman 작업

실행

URL

  • http://localhost:11434/api/generate POST 방식

Headers

  • Content-Type : application/json

Body

  • raw 
    {
"model": "llama-korea",
"prompt": "서울의 유명한 관광 코스를 만들어줄래",
"stream": false
}

결과

{
    "model": "llama-korea",
    "created_at": "2024-07-19T06:07:42.006458Z",
    "response": "안녕하세요! 서울을 방문하시는 동안 즐길 수 있는 다양한 관광 코스 중 몇 가지를 추천해드릴게요.
    각 코스는 서울의 대표적인 명소와 함께 역사, 문화, 그리고 현대적 매력을 느낄 수 있도록 구성했습니다.

    1. **경복궁과 인사동 코스**
      - 경복궁: 조선 시대의 주요 궁궐 중 하나로, 아름다운 건축물과 정원이 어우러져 있습니다.
      - 인사동: 전통 문화와 현대 예술이 조화를 이루는 거리입니다. 다양한 전통 소품 가게와 갤러리, 그리고 맛집도 즐길 수 있습니다.

    2. **남산타워와 한강 코스**
      - 남산타워: 서울의 상징적인 랜드마크로, 날씨를 감상할 수 있는 전망대가 있습니다.
      - 한강: 걷기 또는 자전거 타기를 즐길 수 있으며, 여름에는 폭죽 축제나 야간 음악회 등 다양한 이벤트가 열립니다.

    3. **동대문과 홍대 코스**
      - 동대문 디자인 플라자(DDP): 현대적인 건축물로 유명하며, 다양한 전시와 공연이 열리는 곳입니다.
      - 홍대: 젊은이들이 즐겨 찾는 거리로, 독특한 카페와 레스토랑, 그리고 라이브 음악장소가 많습니다.

    4. **북촌 한옥마을과 창덕궁 코스**
      - 북촌 한옥마을: 전통 한옥이 잘 보존된 마을로, 한국의 전통 건축과 생활상을 느낄 수 있습니다.
      - 창덕궁: 조선 시대의 궁궐로, 정원과 함께 역사적 유물을 감상할 수 있습니다.

    이 외에도 서울에는 더 많은 매력 있는 장소가 많으니, 방문하시는 동안 즐거운 여행 되시길 바랍니다",
    "done": true,
    "done_reason": "stop",
    "context": [
        198,
        45147,
        29,

        == 생략 ==

        30426,
        106103,
        122419
    ],
    "total_duration": 275172921227,
    "load_duration": 8205175912,
    "prompt_eval_count": 93,
    "prompt_eval_duration": 6222256000,
    "eval_count": 422,
    "eval_duration": 260743871000
}

댓글남기기

참고