AI/ML Deep Dive: Prompt Engineering

It's like writing unit tests: drop a few "input → output" examples into the prompt and the model generalizes to new inputs by analogy — no retraining, just an ad-hoc lesson delivered through examples.

Traditional ML for a new task ("is this comment sarcastic?") meant labeling thousands of examples and training a model. Few-shot prompting collapses that pipeline into "write 3-5 examples in the prompt." Zero-shot goes further still — give zero examples, just describe the task. Both leverage the pattern-completion capacity from pre-training, shifting "training" from a separate step to runtime.

The model does one thing: continue text. Zero-shot phrases the task as a natural-language instruction; few-shot constructs a "Q-and-A" pattern that the model then extends. The order, format, and wording of examples noticeably shape the output — that's the core idea that "prompt form is model behavior."

# Zero-shot: only the task description
"Classify sentiment (positive/negative): 'The service was terrible' →"

# Few-shot: show 3 examples first, the model continues the pattern
"Input: Weather is lovely today → positive
Input: Battery dies in a day → negative
Input: Support patiently solved my problem → positive
Input: The service was terrible →"

from anthropic import Anthropic
client = Anthropic()

# Few-shot: teach the model "extract company name" by example
prompt = """Extract company names from sentences. Output the company name only.

Sentence: Apple launched a new iPhone today.
Company: Apple

Sentence: Tesla's stock surged last night.
Company: Tesla

Sentence: NVIDIA is partnering with TSMC to build a new fab.
Company:"""

resp = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=50,
    messages=[{"role": "user", "content": prompt}]
)
print(resp.content[0].text)  # Expected: NVIDIA, TSMC

• Prompt Engineering Guide — systematic, example-heavy, bilingual
• Anthropic's official Prompt Engineering docs

Like enforcing "write the comment explaining your approach before the code" in code review — make the model "think out loud" before giving the final answer, and accuracy jumps a tier.

For word problems, multi-step reasoning, and logic tasks, asking the model for the answer directly often fails. The reason: per-token compute is constant, and "compute the final answer in one step" is too tight a budget for hard problems. CoT's insight: letting the model emit more intermediate tokens effectively gives it more "thinking time." The model breaks reasoning into small steps, each built on the previous result, and error rates drop sharply.

Triggering it is absurdly simple: add "Let's think step by step." at the end of the prompt, or include reasoning in your few-shot examples. The model writes out reasoning first, then the answer.

# Without CoT
"Mike has 5 apples. He eats 2, then buys 3 times as many as he ate. How many does he have now?"
→ "9"  (wrong!)

# With CoT
"... Let's think step by step."
→ "Mike started with 5, ate 2 leaving 3;
   bought 3 × 2 = 6 more;
   total = 3 + 6 = 9.
   Wait — recheck: 3 × (apples eaten = 2) = 6, plus remaining 3 = 9. Answer: 9."  (correct reasoning made explicit)

Analogy: your brain does 23 × 47 by mentally splitting into 20 × 47 + 3 × 47 first. CoT forces the model to do that explicitly.

from anthropic import Anthropic
client = Anthropic()

question = "A car drives 2.5 hours at 60 km/h, then 1.5 hours at 80 km/h. What's the average speed?"

# Trigger CoT: write "reason first, answer second" into the system prompt
resp = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=500,
    system="Write your reasoning steps inside <thinking> tags, then give the final number inside <answer> tags.",
    messages=[{"role": "user", "content": question}]
)
print(resp.content[0].text)
# Model will compute total distance = 60×2.5 + 80×1.5 = 270, then the average

• Original paper (Wei et al., 2022): Chain-of-Thought Prompting — clear in 4 pages
• Prompting Guide: CoT chapter with many examples

Like REPL debugging: instead of emitting the full answer at once, the model "thinks one step → calls a tool → reads the result → thinks another step," looping until done. CoT is pure mental reasoning; ReAct is reasoning plus actually going out to fetch data or execute commands.

Pure CoT reasons only over what the model already knows. Ask "what's today's stock price?" or "how many stars does this GitHub repo have?" and it will hallucinate. ReAct weaves reasoning with action: the model first thinks about what information it lacks, then acts by calling a tool, observes the result, and resumes thinking — looping toward the answer. This is the conceptual backbone of nearly every modern agent framework.

It's a Thought → Action → Observation → Thought ... loop. Each round the model emits a short segment; an outer runtime parses the Action portion, executes the real tool, then splices the result back into the prompt as Observation so the model can continue.

# Typical ReAct conversation format
User: How many stars does BigCat's ai-ml-daily GitHub repo have?

Thought: I don't have real-time data; I need the GitHub API.
Action:  http_get("https://api.github.com/repos/cissy0802/ai-ml-daily")
Observation: {"stargazers_count": 42, ...}
Thought: I have the data now, I can answer.
Answer:  42 stars.

Essentially: ReAct = CoT + tool use + loop. Day 6 covers Function Calling, which is ReAct's industrialized form.

from langchain.agents import create_react_agent, AgentExecutor
from langchain_anthropic import ChatAnthropic
from langchain.tools import Tool

# 1) Define tools — the things the model can actually do
tools = [
    Tool(name="calculator",
         func=lambda x: str(eval(x)),
         description="Evaluate a math expression"),
    Tool(name="search",
         func=my_search_api,           # your search endpoint
         description="Search real-time web information"),
]

# 2) Let LangChain assemble the ReAct prompt + parsing loop
agent = create_react_agent(ChatAnthropic(model="claude-opus-4-7"),
                           tools, prompt=react_prompt_template)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

executor.invoke({"input": "Today's Shanghai Composite move × my 500k position = ?"})
# You'll see the full Thought → Action → Observation trace

• Original paper ReAct: Synergizing Reasoning and Acting (Yao et al., 2022)
• LangChain Agent tutorial — run it to get the intuition

Like asking several engineers to solve the same problem independently and then voting: sample the model multiple times at non-zero temperature, run CoT each time, take the most frequent answer. The majority is usually right. It's a statistical "wisdom of crowds" that trades compute for accuracy.

A single CoT chain can still slip on one step, and one mistake cascades. Self-Consistency's insight: correct reasoning paths tend to converge to the same answer, while errors scatter. Sample N times (say N=10), majority-vote the answers, and robustness climbs steeply. On math and logic tasks this often adds 10-20 accuracy points.

Three steps: (1) sample N times at temperature > 0 (e.g. 0.7) so the same prompt produces different CoT chains; (2) extract the final answer from each chain; (3) majority-vote.

# Pseudocode: Self-Consistency
question = "If 3x + 7 = 22, x = ?"
answers = []
for _ in range(10):
    cot = llm(question, temperature=0.7)   # different chain each time
    answers.append(extract_final(cot))      # pull out the final number
final = mode(answers)                       # vote: [5,5,5,5,4,5,5,6,5,5] → 5

Analogy: a single CoT is one student doing the problem; Self-Consistency hands the problem to ten students independently and tallies their answers. The price is ~10x tokens.

from anthropic import Anthropic
from collections import Counter
import re

client = Anthropic()
question = "Anna has 3 apples. Mom gives her twice as many more, then she shares half with her brother. How many does she have left?"

answers = []
for _ in range(10):                          # sample 10 independent chains
    resp = client.messages.create(
        model="claude-opus-4-7",
        max_tokens=300,
        temperature=0.7,                       # key: temperature for diversity
        messages=[{"role": "user",
                   "content": question + "\nReason step by step, then give the number."}]
    )
    # Take the last number as the answer
    nums = re.findall(r"\d+", resp.content[0].text)
    if nums: answers.append(nums[-1])

print(Counter(answers).most_common(1))      # the most frequent answer wins

• Original paper Self-Consistency Improves CoT (Wang et al., 2022)
• Prompting Guide: Self-Consistency section