Using the OpenAI API and Anthropic API for Custom Prompt Applications

Once you have outgrown chat UIs, the natural next step is calling these models from code. The OpenAI and Anthropic APIs let you build features — summarisers, classifiers, agents, search front-ends — into your own apps. The good news: if you can write a strong prompt, you are already 80% of the way there.

Note: API surfaces change. SDK names, endpoint paths and parameter defaults shift; check the current OpenAI and Anthropic documentation before you ship.

1. Introduction

The chat apps you have been using are wrappers around the same APIs you can call yourself. Calling them directly gives you three powerful things: repeatability (the exact same prompt every time, no UI drift), composition (you can chain calls, transform output, and embed the model inside a larger system), and control (parameters, system prompts, tool use, and structured output that the consumer chat apps hide).

This tutorial focuses on the prompt-engineering side: the messages format, system prompts in code, parameters worth knowing, JSON-mode for reliable structured output, and tool/function calling patterns. We use illustrative code, not platform-specific syntax — concepts that hold up as the SDKs evolve.

2. The Messages Format (Both Vendors)

Both OpenAI and Anthropic accept a list of messages with roles. The roles are nearly identical, even if the JSON shapes differ slightly.

{
  "model": "[a chat model id]",
  "system": "You are a concise UK legal plain-English explainer. ...",
  "messages": [
    { "role": "user", "content": "Explain this clause to a small-business owner: …" },
    { "role": "assistant", "content": "Sure — here's what that clause means in plain English. …" },
    { "role": "user", "content": "Now turn that into a 3-bullet summary." }
  ],
  "max_tokens": 600,
  "temperature": 0.3
}

The pattern is the same across vendors: a system prompt sets persona and rules; a sequence of user and assistant turns simulates the conversation. Sending only the latest user message loses prior context — you have to manage the conversation history yourself.

3. The System Prompt in Code

Everything you learned about system prompts in Custom GPTs and Claude Projects applies here — only now you control it programmatically. Two habits worth adopting from day one:

• Version your prompts. Store system prompts in source control. Treat changes like code changes — review and roll back as needed.
• Templatise variables. Use placeholders ({{user_name}}, {{tone}}, {{tenant_brand_voice}}) and inject them at call time. Don't bake user-specific text into the system prompt.

You are the support assistant for {{product_name}}.

VOICE
{{brand_voice}}

CAPABILITIES
- Answer questions using only the knowledge base supplied
  by the tool `kb_search`.
- Escalate to a human if confidence is low.

REFUSE
- Do not promise specific dates.
- Do not discuss pricing changes.
- Do not invent product features that are not in the KB.

OUTPUT
- 2–4 short paragraphs.
- End with: "Was that helpful? Reply yes/no."

4. Parameters Worth Knowing

The same parameters you saw in the Foundations section show up here as request fields. The ones you'll actually use day to day:

• temperature — 0–1+ on most APIs. Lower (0–0.3) for classification, extraction, and anywhere you want deterministic answers. Higher (0.7–1.0) for creative writing and brainstorming.
• max_tokens — hard cap on output. Set this for cost control even when you expect short replies; runaway generations are a real failure mode.
• top_p — nucleus sampling. Usually leave at default and tune temperature instead; tune one at a time.
• stop — sequences that terminate generation. Useful for keeping outputs inside a template.
• response_format / JSON mode — forces valid JSON output. Game-changing for any feature that processes the response programmatically.
• seed — improves reproducibility (best-effort, not guaranteed). Useful in evals.

5. JSON Mode and Structured Outputs

One of the highest-leverage features on both vendors is structured output. Instead of asking the model to "return JSON" and hoping, you provide a schema and the API guarantees valid output.

SYSTEM:
You extract structured information from customer-support emails.

USER:
Extract the following from the email below.

Email:
"""
Hi — order #20458 hasn't arrived. It's been two weeks since
"shipped" status. Tried the tracking number, nothing updates.
Please cancel and refund. — Priya
"""

Return JSON matching this schema:
{
  "order_id": "string",
  "issue_type": "shipping | billing | product | other",
  "urgency": "low | medium | high",
  "requested_action": "refund | resend | reply | escalate",
  "sentiment": "positive | neutral | negative"
}

With structured outputs enabled, your downstream code can rely on the shape. No regex parsing of natural language — and no failures from a stray "Sure! Here's your JSON:" preamble.

6. Tool / Function Calling

Tool calling lets the model trigger code in your application — a database lookup, an API call, a calculation — instead of inventing the answer. You describe each tool's name, purpose, and JSON schema for arguments; the model decides when to call one and emits the structured arguments.

tools = [
  {
    "name": "kb_search",
    "description": "Search the support knowledge base for an answer.",
    "input_schema": {
      "type": "object",
      "properties": {
        "query": { "type": "string" },
        "top_k": { "type": "integer", "default": 3 }
      },
      "required": ["query"]
    }
  },
  {
    "name": "create_ticket",
    "description": "Open a support ticket when the assistant cannot answer.",
    "input_schema": {
      "type": "object",
      "properties": {
        "summary":   { "type": "string" },
        "priority":  { "type": "string", "enum": ["low","medium","high"] }
      },
      "required": ["summary", "priority"]
    }
  }
]

Two patterns that consistently work well with tool calling:

• Describe tools precisely. Each description is read by the model. Vague descriptions cause wrong tool choices. Be concrete about when the tool should and should not be called.
• Keep the loop tight. Model emits a tool call → your code runs it → you append the tool result to the messages → model produces the final answer. Limit the loop to a few iterations to prevent runaway behaviour.

7. Practice Exercises

8. Key Takeaways