(sLM-6b) Prompt Engineering 실습 2

Prompt Engineering 실습 2

1. LLaMA-3-8B (Multi-turn PE, Few-shot PE)
2. Mistral-7B (CoT PE, Zero-shot PE)
3. Phi-3-3.8B (Multi-turn PE, Generated Knowledge PE)
4. Gemma 7B (Few-shot PE, Self-Ask PE, Chaining)

Contents

1. 모델 설정
2. CoT PE
3. Zero-Shot PE

관련 패키지 설치 및 환경 설정

!pip install bitsandbytes==0.43.1
!pip install accelerate==0.30.1
!pip install transformers==4.39.3
!pip install gradio==4.29.0

1. 모델 설정

import torch
from transformers import (
    AutoTokenizer,
    AutoModelForCausalLM,
    BitsAndBytesConfig
)

a) 모델 설정: Mistral-7B-Instruct

model_id = "mistralai/Mistral-7B-Instruct-v0.2"

b) Tokenizer 불러오기

tokenizer = AutoTokenizer.from_pretrained(model_id)

c) Model 불러오기

Quantization 설정

config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16
)

(Quantize해서) 불러오기

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16, 
    device_map="auto",          
    quantization_config=config  
)

2. CoT PE

a) Prompt (message)

• CoT PE에 맞는 prompt 구성하기

messages = [
    {"role": "user", "content": "Can you help me solve this quadratic equation: x^2 - 3x - 4 = 0?"},
    {"role": "assistant", "content": """
    Sure, to solve the quadratic equation x^2 - 3x - 4 = 0,
    we can use the quadratic formula, x = [-b ± sqrt(b² - 4ac)] / (2a).
    In this equation, a=1, b=-3, and c=-4.
    First, let's calculate the value under the square root,
    b² - 4ac, which is (-3)² - 4*1*(-4).
    The value is 9 + 16, which equals to 25. Next, we need to compute the values of x using the quadratic formula.
    So, x = [3 ± sqrt(25)] / 2. The roots of the equation are x = 4 and x = -1.
    """
    },

    {"role": "user", "content": "Can you help me solve this quadratic equation: x^2 - 6x + 9 = 0?"}
]

for message in messages:
    message["content"] = message["content"].replace("\n", "")

b) Tokenize

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

c) Generate

outputs = model.generate(
    input_ids,
    max_new_tokens=1024,
    do_sample=True,
    temperature=0.3,       
    top_p=0.9,             
)

d) Result

response = outputs[0][input_ids.shape[-1]:]
print("response : ", tokenizer.decode(response, skip_special_tokens=True))

Yes, I can help you solve the quadratic equation x^2 - 6x + 9 = 0. This is called a quadratic equation because it is in the form ax^2 + bx + c = 0.

To find the solutions for x, we can use the quadratic formula: x = (-b ± √(b^2 - 4ac)) / (2a). In this equation, a = 1, b = -6, and c = 9.

First, we calculate the value under the square root, which is called the discriminant: b^2 - 4ac = (-6)^2 - 4(1)(9) = 36 - 36 = 0.

Since the discriminant is zero, this equation has only one solution. We can find it by using the value of the square root as zero:

x = (-b + 0) / (2a) = (-(-6)) / (2*1) = 6 / 2 = 3

So, the quadratic equation x^2 - 6x + 9 = 0 has only one solution, which is x = 3.

3. Zero-Shot PE

a) Prompt (message)

• Zero-shot PE에 맞는 prompt 구성하기

messages = [
    {"role": "user", "content": "Could you write a Python code that loads GPT-2 from Hugging Face and performs inference?"},
]

for message in messages:
    message["content"] = message["content"].replace("\n", "")

b) Tokenize

위와 동일

c) Generate

위와 동일

d) Result

위와 동일

Reference

• [패스트캠퍼스] 8개의 sLM모델로 끝내는 sLM 파인튜닝