NanoGPT API Tutorial: Python Guide with Code Examples

if you can use OpenAI's Python SDK, you can use NanoGPT. the API is identical. i switched my whole workflow over in about 10 minutes. here's everything you need to know, with working code you can copy-paste right now.

tl;dr: NanoGPT's API is identical to OpenAI's. same SDK, same request format, just change the base URL. this tutorial covers basic usage, streaming, function calling, error handling, and practical examples you can copy-paste right now.

Key Takeaways:

NanoGPT API uses the exact same OpenAI Python SDK. just change base_url to api.nano-gpt.com/v1
streaming, function calling, and model switching work identically to OpenAI with zero code changes
always set max_tokens for routine tasks to avoid paying for 4000 tokens when you only needed 200

👉 Get NanoGPT with 5% discount - you'll need an API key for this tutorial.

setup

install the OpenAI SDK

short answer: pip install openai. the official OpenAI SDK works directly with NanoGPT's API format.

pip install openai

that's the only dependency. NanoGPT uses the OpenAI API format, so the official SDK works directly.

basic configuration

short answer: set base_url to api.nano-gpt.com/v1 and use your NanoGPT API key. always use environment variables, never hardcode keys.

import openai

client = openai.OpenAI(
    base_url="https://api.nano-gpt.com/v1",
    api_key="your-nanogpt-api-key"
)

important: never hardcode your API key in source code. use environment variables:

import os
import openai

client = openai.OpenAI(
    base_url="https://api.nano-gpt.com/v1",
    api_key=os.environ.get("NANOGPT_API_KEY")
)

set the environment variable:

export NANOGPT_API_KEY="your-key-here"

see our API key setup guide for getting your key.

basic usage

simple chat completion

short answer: create a client with NanoGPT base URL, send messages with model name, print the response. identical to OpenAI.

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "Explain Python list comprehensions in 2 sentences."}
    ]
)

print(response.choices[0].message.content)

with system prompt

short answer: add a system message to set the model's behavior. works exactly like OpenAI's system prompts.

response = client.chat.completions.create(
    model="claude-3-5-sonnet",
    messages=[
        {"role": "system", "content": "You are a Python expert. Be concise."},
        {"role": "user", "content": "What's the difference between a list and a tuple?"}
    ]
)

print(response.choices[0].message.content)

multi-turn conversation

short answer: append assistant and user messages to maintain conversation history across multiple API calls.

messages = [
    {"role": "system", "content": "You are a helpful coding assistant."},
    {"role": "user", "content": "Write a function to reverse a string."},
]

# first response
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages
)
assistant_msg = response.choices[0].message.content
print(assistant_msg)

# follow-up
messages.append({"role": "assistant", "content": assistant_msg})
messages.append({"role": "user", "content": "Now make it handle None values."})

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages
)
print(response.choices[0].message.content)

streaming

streaming gives you token-by-token output. essential for chat applications.

basic streaming

short answer: set stream=True and iterate over chunks. print content as it arrives for real-time response display.

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a short story about a robot."}],
    stream=True
)

for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

print()  # newline after streaming

streaming with collection

short answer: stream and collect the full response simultaneously. useful for displaying and saving the complete output.

def stream_response(client, model, messages):
    """Stream response and return full text."""
    stream = client.chat.completions.create(
        model=model,
        messages=messages,
        stream=True
    )
    
    full_response = ""
    for chunk in stream:
        content = chunk.choices[0].delta.content
        if content:
            full_response += content
            print(content, end="", flush=True)
    
    print()
    return full_response

# usage
response_text = stream_response(client, "gpt-4o", [
    {"role": "user", "content": "Explain async/await in Python."}
])

advanced features

function calling (tools)

short answer: define tools in the same format as OpenAI. the model calls functions and you send results back. works with GPT-4o, Claude, Gemini.

import json

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "City name"
                    }
                },
                "required": ["city"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
    tool_choice="auto"
)

# check if model wants to call a function
if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    function_name = tool_call.function.name
    arguments = json.loads(tool_call.function.arguments)
    
    print(f"Model wants to call: {function_name}")
    print(f"Arguments: {arguments}")
    
    # simulate function result
    weather_result = {"temp": "22°C", "condition": "sunny"}
    
    # send result back to model
    messages = [
        {"role": "user", "content": "What's the weather in Tokyo?"},
        response.choices[0].message,
        {
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(weather_result)
        }
    ]
    
    final_response = client.chat.completions.create(
        model="gpt-4o",
        messages=messages
    )
    print(final_response.choices[0].message.content)

controlling output

short answer: set max_tokens to limit response length and save money. temperature controls creativity. top_p controls sampling.

control how the model responds:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a haiku about Python."}],
    max_tokens=100,        # limit response length (saves money)
    temperature=0.9,       # higher = more creative (0-2)
    top_p=0.95,            # nucleus sampling
    frequency_penalty=0.1, # reduce repetition
    presence_penalty=0.1   # encourage new topics
)

print(response.choices[0].message.content)

tip: always set max_tokens for routine tasks. without it, the model might generate 4000 tokens when you only needed 200. that's wasted money.

model switching

short answer: switch models by changing one parameter. ask the same question to GPT-4o, Claude, and DeepSeek with one function.

def ask_any_model(model, question):
    """Ask any NanoGPT model a question."""
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": question}]
    )
    return response.choices[0].message.content

# same question, different models
question = "What is recursion?"

gpt4_answer = ask_any_model("gpt-4o", question)
claude_answer = ask_any_model("claude-3-5-sonnet", question)
deepseek_answer = ask_any_model("deepseek-v3", question)

print("GPT-4o:", gpt4_answer)
print("Claude:", claude_answer)
print("DeepSeek:", deepseek_answer)

error handling

common errors and fixes

short answer: 401: invalid key. 429: rate limited, add backoff. 500: server error, retry. timeout: increase timeout or try smaller model.

from openai import (
    APIError,
    RateLimitError,
    AuthenticationError,
    APITimeoutError
)

def safe_api_call(client, model, messages, max_retries=3):
    """API call with retry logic."""
    for attempt in range(max_retries):
        try:
            response = client.chat.completions.create(
                model=model,
                messages=messages,
                timeout=30  # 30 second timeout
            )
            return response.choices[0].message.content
            
        except AuthenticationError:
            print("Invalid API key. Check your NANOGPT_API_KEY.")
            return None
            
        except RateLimitError:
            wait_time = 2 ** attempt
            print(f"Rate limited. Waiting {wait_time}s...")
            time.sleep(wait_time)
            
        except APITimeoutError:
            print(f"Timeout on attempt {attempt + 1}/{max_retries}")
            if attempt == max_retries - 1:
                return None
                
        except APIError as e:
            print(f"API error: {e}")
            if attempt == max_retries - 1:
                return None
            time.sleep(1)
    
    return None

error reference

short answer: 401 Unauthorized: check key. 429 Rate Limited: add retry with backoff. 500: retry after delay. timeout: increase timeout.

common errors you'll encounter and how to fix them:

Error	Cause	Fix
401 Unauthorized	Invalid API key	Check key in dashboard
429 Rate Limited	Too many requests	Add retry with backoff
500 Internal Error	NanoGPT server issue	Retry after delay
502/503	Service unavailable	Wait and retry
Timeout	Slow response	Increase timeout, try smaller model

practical examples

example 1: code reviewer

short answer: send code to Claude 3.5 Sonnet with a review system prompt. returns specific, actionable feedback.

def review_code(code, language="python"):
    """Get AI code review."""
    response = client.chat.completions.create(
        model="claude-3-5-sonnet",
        messages=[
            {"role": "system", "content": "You are a senior code reviewer. Be direct and specific."},
            {"role": "user", "content": f"Review this {language} code:\n\n```{language}\n{code}\n```"}
        ]
    )
    return response.choices[0].message.content

# usage
code_to_review = """
def add(a, b):
    return a + b
"""
print(review_code(code_to_review))

example 2: multi-model comparison

short answer: ask the same question to GPT-4o, Claude 3.5, and DeepSeek V3. compare responses side by side.

def compare_models(question, models=None):
    """Ask the same question to multiple models."""
    if models is None:
        models = ["gpt-4o", "claude-3-5-sonnet", "deepseek-v3"]
    
    results = {}
    for model in models:
        response = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": question}],
            max_tokens=500
        )
        results[model] = response.choices[0].message.content
    
    return results

# usage
answers = compare_models("What's the best Python web framework?")
for model, answer in answers.items():
    print(f"\n--- {model} ---")
    print(answer[:200] + "...")

example 3: document summarizer

short answer: use Gemini 1.5 Pro for long document summarization with its massive context window.

def summarize_document(text, max_length=200):
    """Summarize a long document."""
    response = client.chat.completions.create(
        model="gemini-1.5-pro",  # good for long contexts
        messages=[
            {"role": "system", "content": f"Summarize in under {max_length} words."},
            {"role": "user", "content": text}
        ]
    )
    return response.choices[0].message.content

# usage
with open("long_document.txt", "r") as f:
    document = f.read()

summary = summarize_document(document)
print(summary)

example 4: conversation memory

short answer: ChatBot class maintains message history across turns. the model remembers previous context.

class ChatBot:
    """Simple chatbot with conversation memory."""
    
    def __init__(self, model="gpt-4o", system_prompt="You are a helpful assistant."):
        self.client = openai.OpenAI(
            base_url="https://api.nano-gpt.com/v1",
            api_key=os.environ.get("NANOGPT_API_KEY")
        )
        self.model = model
        self.messages = [{"role": "system", "content": system_prompt}]
    
    def chat(self, user_message):
        self.messages.append({"role": "user", "content": user_message})
        response = self.client.chat.completions.create(
            model=self.model,
            messages=self.messages
        )
        assistant_message = response.choices[0].message.content
        self.messages.append({"role": "assistant", "content": assistant_message})
        return assistant_message

# usage
bot = ChatBot(model="claude-3-5-sonnet", system_prompt="You are a Python tutor.")
print(bot.chat("What is a decorator?"))
print(bot.chat("Show me an example."))  # remembers the previous context

example 5: batch processing

short answer: process multiple tasks with rate limiting. add delays between requests to avoid 429 errors.

import time

def batch_process(tasks, model="gpt-4o-mini", delay=0.5):
    """Process multiple tasks with rate limiting."""
    results = []
    for i, task in enumerate(tasks):
        response = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": task}]
        )
        results.append(response.choices[0].message.content)
        print(f"Processed {i+1}/{len(tasks)}")
        time.sleep(delay)  # rate limit protection
    return results

# usage
tasks = [
    "Summarize: Python is a programming language...",
    "Translate to Spanish: Hello, how are you?",
    "Classify sentiment: This product is amazing!",
]

results = batch_process(tasks)

checking available models

# list all available models
models = client.models.list()
for model in models.data:
    print(model.id)

see our full model list for details on each model.

next steps

get your API key set up
explore available models
learn about cost optimization
connect to Open WebUI or LibreChat

👉 Get started with NanoGPT

Last updated: July 2026

NanoGPT API Key Setup - get your key
Best Models for Coding - model recommendations
NanoGPT for Developers - developer workflow
NanoGPT Pricing - cost breakdown
All NanoGPT Models - complete list
NanoGPT + Open WebUI - GUI setup

Disclosure: This article contains affiliate links. If you sign up through our referral link, you get a 5% discount and we earn a small commission. This doesn't affect our reviews - we pay for all services ourselves.