Souk's Notes - Unofficial Summaries

2026-11-06 · open the original ↗

• The Python MCP SDK's browser inspector tests a server without an application. Running "mcp dev mcp_server.py" starts it; the screenshot shows a proxy on port 6277 and the inspector at 127.0.0.1:6274, and the lesson warns the UI is still changing.
• The Tools tab lets you run tools individually and in sequence. After "List Tools", you select read_doc_contents, enter an ID like "deposition.md" and run it; editing with edit_document and re-reading confirms the change applied.
• The inspector gives a fast edit-test-debug loop. The lesson's claim is that it catches issues before integrating with Claude or other apps.

2026-11-06 · open the original ↗

• The Python SDK replaces hand-written JSON schemas with decorators. A server is one line, FastMCP("DocumentMCP", log_level="ERROR"), and @mcp.tool with a name, description and Field-described parameters generates the schema Claude needs.
• The example server has read and edit tools over in-memory documents. Documents live in a dictionary of ID to content; read_doc_contents returns a document's text and raises ValueError for unknown IDs.
• edit_document does an exact-match find-and-replace. It takes doc_id, old_str and new_str and uses Python's str.replace, so old_str must match exactly, including whitespace.
• SDK tools are plain, testable Python functions. Cited benefits are type-hint validation, Field descriptions that guide Claude, errors via ordinary exceptions, and automatic registration.

2026-11-06 · open the original ↗

• The project is a CLI chatbot over documents, with client and server. The MCP server exposes tools to read and update in-memory documents (document.pdf, spreadsheet.xlsx, report.txt, spec.md); the client handles users and talks to Claude.
• Real projects usually build only one side. You'd build a server to expose your service to AI models, or a client to use others' servers; this one builds both purely for teaching.
• Setup is an API key, dependencies, then one run command. Put ANTHROPIC_API_KEY in .env, install with uv (recommended) or pip, run "uv run main.py" (or "python main.py"), and check the chat by asking "what's 1+1?"; the starter cli_project.zip has basic chat, and cli_project_COMPLETE.zip is the finished version.

2026-11-06 · open the original ↗

• MCP is transport agnostic. The common setup runs client and server on one machine over standard input/output, but they can also connect over HTTP, WebSockets or other protocols.
• Two request/result pairs carry most of the work. ListToolsRequest/ListToolsResult asks what tools the server offers; CallToolRequest/CallToolResult runs a named tool with arguments and returns the result.
• A single question triggers a long relay through the client. For "What repositories do I have?", your server gets the tool list via the client, sends query plus tools to Claude, receives a tool-use request, has the client send CallToolRequest to the GitHub MCP server (which calls GitHub), then returns the result to Claude in a user message for the final answer.

2026-11-06 · open the original ↗

• MCP moves tool-writing work from your server to MCP servers. Your server becomes an MCP client connecting to servers that hold tools, prompts and resources and wrap outside services like GitHub, AWS or databases.
• Covering a large service yourself means many tools to maintain. Answering "what open pull requests are there across my repositories?" in full generality would require schemas and functions such as get_repos, search_issues, update_issue, create_issue and create_file.
• Anyone can write an MCP server, often the provider itself. The lesson gives AWS releasing an official server as a possible example, and you can wrap any service you need.
• MCP complements tool use rather than replacing it. MCP is about who builds and maintains the tool schemas and functions; tool use is how Claude calls them, and calling an API directly means writing those definitions yourself.

2026-11-06 · open the original ↗

• Cache tool schemas by marking a copy of the last tool. Clone the tools list and its last schema, add cache_control {"type": "ephemeral"} to the clone, and pass that, so the original definitions aren't mutated and later reordering doesn't break.
• Cache the system prompt by sending it as a text-block list. The system parameter becomes a list holding one text block with the cache_control field.
• Usage fields show whether the cache was written or read. The first request reports cache_creation_input_tokens, identical repeats report cache_read_input_tokens, and changing only the system prompt can show both — but only if the tools alone reach the minimum length, since a breakpoint below it writes nothing.
• A single changed character invalidates the cache entry. That makes stable tool schemas and system prompts ideal candidates, and the lesson recommends building caching into chat functions by default.

2026-11-06 · open the original ↗

• Caching requires a manual breakpoint on a block. Everything before and including the breakpoint is cached and reused only if identical up to that point; adding cache_control {"type": "ephemeral"} requires the longhand content-block form, not the shorthand string.
• Breakpoints can go on system prompts and tool definitions. These are the most common opportunities because they rarely change, and breakpoints also span messages, including assistant messages.
• Tools, system prompt and messages are joined in that order. A breakpoint on the last tool caches the tools but not the system prompt or messages; up to four breakpoints are allowed.
• Content below a model-specific minimum silently isn't cached. For Claude Haiku 4.5, the course's model, the minimum is 4,096 tokens summed across the cached content; no error returns, and both cache_creation_input_tokens and cache_read_input_tokens read 0.

2026-11-06 · open the original ↗

• Without caching, Claude discards its input-processing work after each request. The lesson describes tokenizing, creating embeddings and adding context before generating, all thrown away once the response is sent.
• Conversations force reprocessing of identical earlier content. A follow-up includes the original message and Claude's reply, so the same text gets processed again moments later.
• Prompt caching saves that work for reuse on matching content. The first request writes it to a cache acting as a lookup table and follow-ups read it, making requests faster and cheaper.
• The cache is short-lived and needs exact matches. It lives 5 minutes, but the lesson says repeated content is still extremely common, in conversations, repeated document references and iterated prompts.

2026-11-06 · open the original ↗

• Citations show users answers came from their documents, not memory. Without them users can't verify information or tell it came from supplied sources.
• Enable citations with a title and citations flag. Add "title" (e.g. "earth.pdf") and "citations": {"enabled": True} to the document block; responses then carry cited_text, document_index, document_title, start_page_number and end_page_number.
• Plain-text documents get character locations instead of pages. A text source with media_type "text/plain" returns CitationCharLocation objects, allowing exact sentences or paragraphs to be highlighted.
• Citations pay off in interfaces that display them. Numbered references users can hover or click to see the document and pages let them verify claims, which the lesson calls essential for sensitive documents or users who want the original source.

2026-11-06 · open the original ↗

• Images go in as base64 blocks under hard size and count limits. The lesson lists up to 100 images per request, 10MB per image after base64 encoding, 8000px max side for a single image and 2000px when sending several. Each image costs tokens of ⌈width/28⌉ × ⌈height/28⌉, and the image block sits in the user message beside the text block.
• Simple vision prompts give poor results even on clear images. Asked plainly how many marbles were in a photo of 12, Claude answered 13. The lesson's claim is that the usual prompt-engineering techniques (detailed steps, examples, breaking the task down) apply to images just as they do to text.
• Giving a counting method or a worked example fixes the error. A prompt that numbers each marble and then re-counts row by row from the bottom-left produced the correct 12. So did a one-shot setup: an image labelled as 11 marbles, followed by the target image.
• Real-world vision tasks need a full analysis framework, not a one-line ask. For insurance wildfire assessment from satellite images, the prompt walks through finding the residence, tree overhang (in 0-25/25-50/50-75/75%+ bands), fire vulnerability, and defensible space, then a 1-4 rating. On a heavily wooded property Claude returned "3 (High Risk)", which the lesson reports as correct.

2026-11-06 · open the original ↗

• Extended thinking trades cost and latency for accuracy on hard tasks. You pay for every thinking token and wait longer for the reply. The lesson says to turn it on only after evaluations show that prompt optimization alone still falls short.
• With thinking on, responses carry a thinking block before the text. Each thinking block has a cryptographic signature, so the reasoning cannot be edited when you send it back in later turns; changing it makes signature validation fail. The lesson's reason is that Claude relies heavily on that content to generate its response.
• Flagged reasoning arrives as an encrypted redacted thinking block. You cannot read it, but you should still pass it back in later turns so Claude keeps the context of its earlier reasoning. During development, a special trigger string in the message forces a redacted response so you can test how your code handles one.
• The thinking budget has a floor and must sit below max_tokens. The minimum budget is 1024 tokens, and max_tokens must be larger (at least 1025 for that budget). In the example, a 1024 budget with max_tokens of 4000 leaves up to 2976 tokens for the answer, set through params["thinking"] = {"type": "enabled", "budget_tokens": ...}.

2026-11-06 · open the original ↗

• Chunking cuts each piece off from the document around it. The lesson's argument is that a chunk that no longer knows where it came from, or how it relates to other sections, is harder to retrieve accurately.
• Have Claude write a short context snippet for each chunk before indexing. Claude is sent each chunk with the source document and asked for a brief snippet placing the chunk in the whole. The snippet is prepended to the chunk, and the result goes into both the vector and BM25 indexes. In the example, the snippet ties a Software Engineering section to an incident ID that also appears in the Cybersecurity Analysis section.
• For documents too large for the context window, send a reduced context. Instead of the whole document, send a few chunks from the start, where summaries or abstracts often sit, plus the chunks just before the target. The sample code uses 2 of each.
• The technique pays off where sections depend on each other. It suits technical documents, reports and papers whose chunks refer to concepts defined elsewhere. The cost is an extra API call per chunk, which the lesson considers worth it when context matters.

2026-11-06 · open the original ↗

• Hybrid search can still rank the wrong section first. For "what did the eng team do with INC-2023-Q4-011?", the Cybersecurity section came out ahead of Software Engineering, even though the query is specifically about the engineering team.
• Re-ranking passes the merged hybrid results through Claude for reordering. The prompt gives Claude the user's question and the candidate documents, and asks for the k most relevant in order of decreasing relevance, returned as JSON. The re-ranker plugs into the Retriever as a reranker_fn parameter.
• Ask for document IDs, not text, to keep re-ranking fast. Giving each chunk a unique ID in advance and asking Claude to return only the IDs avoids waiting while it copies out long passages.
• Re-ranking fixed the example at the price of an extra LLM call. With the re-ranker, Software Engineering ranked first, because Claude picked up that the question was about the engineering team. The cost is added latency and an API charge on every search, which the lesson judges worthwhile when precise retrieval matters.

2026-11-06 · open the original ↗

• A single Retriever class can wrap both search indexes. Because VectorIndex and BM25Index share the same add_document() and search() API, the Retriever simply forwards documents and queries to each and merges what comes back.
• Reciprocal Rank Fusion merges results by rank, not raw score. Cosine similarity and BM25 scores aren't comparable, so each document scores Σ 1/(k + rank) across the indexes. k is usually 60; the lesson uses 1 for readability. In the example, sections ranked (2,7,6) by vector and (6,2,7) by BM25 score 0.833, 0.75 and 0.583, giving the final order 2, 6, 7.
• The hybrid search beat vector search alone on the incident query. For "what happened with INC-2023-Q4-011?" it returned Cybersecurity Analysis (the incident report) first, then Software Engineering, then Legal Developments.
• The shared interface makes adding more indexes easy. Any new index that implements search() and add_document(), such as one for named entities, is folded into the RRF ranking automatically, whatever the number of indexes.

2026-11-06 · open the original ↗

• Semantic search can miss exact terms like incident IDs. A query for "INC-2023-Q4-011" returned Cybersecurity and Financial Analysis, when the right sections were Cybersecurity and Software Engineering. The lesson's explanation is that embeddings match on meaning, not exact strings.
• BM25 weights query terms by rarity across the documents. It splits the query into terms, counts how often each appears, and gives rare terms more weight. Chunks with more of the high-weight terms rank higher, so the incident ID counts for far more than a word like "a".
• Run lexical and semantic search side by side, then merge. BM25Index has the same add_document()/search() API as the vector index. Run alone, it put the sections that actually contain the ID first. The lesson sets up merging the two result sets as the next step.

2026-11-06 · open the original ↗

• The basic RAG pipeline runs in five steps. Chunk the text by section, embed each chunk, store the vectors in a VectorIndex, embed the user's query, then search for the nearest chunks by cosine distance.
• Store each chunk's text alongside its embedding. A vector is meaningless to people, so search results are useless without the content they stand for. Some setups store an ID pointing back to the text instead; the lesson stores the text directly for simplicity.
• The example query returned relevant sections with close distances. "What did the software engineering dept do last year?" returned Section 2, Software Engineering – Project Phoenix (distance 0.71), and the Methodology section (0.72); lower distance means more similar. The lesson says later sections will cover cases where this basic approach falls short.

2026-11-06 · open the original ↗

• Chunking, embedding and storage all happen before any user query. Using an imaginary two-number model (medicine, software), the medical section embeds as [0.97, 0.34] because of the word "bug", and the software section as [0.30, 0.97]. Normalized to unit length, these become [0.944, 0.331] and [0.295, 0.955] and are stored in the vector database.
• The query is embedded by the same model and matched by cosine similarity. The software-engineering question embeds to [0.1, 0.89], normalized to [0.112, 0.993]. It scores 0.983 against Software Engineering and 0.398 against Medical Research, so the software section is returned.
• Cosine distance is one minus similarity, so smaller means closer. Similarity runs from 1 (same direction) through 0 (perpendicular) to −1 (opposite). Distance therefore runs from 0 (very similar) to 2 (opposite), and vector database documentation often reports distance.
• The final prompt combines the question with the retrieved chunk. The software-engineering section goes into <report> tags beside the <user_question>. That lets Claude answer from your own documents rather than only from its training.

2026-11-06 · open the original ↗

• Finding relevant chunks is a search problem, usually solved semantically. Semantic search compares embeddings of the question and of each chunk, so it can find related content even when the words differ.
• An embedding is a list of numbers whose features nobody can interpret. Each value runs from −1 to +1 and scores some quality of the text. Labels like "how happy" are only illustrations: the model learns its features in training, and they are not labelled, yet they still allow texts to be compared mathematically.
• Claude can't produce embeddings, so Vertex AI supplies a separate model. The lesson uses text-embedding-005 through the google-genai SDK, calling client.models.embed_content with vertexai=True and returning each embedding's values.

2026-11-06 · open the original ↗

• Bad chunking puts the wrong context in the prompt. In the lesson's example, "How many bugs did engineers fix this year?" can retrieve a medical-research section that calls a pathogen a "bug", rather than the software-engineering section. The result is a wrong answer.
• Size-based chunking is simple but needs overlap to keep context. Equal-length cuts split words (a 325-character document cut into three chunks of about 108 characters breaks "signific/ant") and drop section headers. The sample chunk_by_char uses 150-character chunks with 20 characters of overlap.
• Structure-based chunking is cleanest, but only for reliably structured documents. Splitting markdown on \n## headers gives tidy sections. Plain text, PDFs and user uploads often have no markers to split on.
• Semantic chunking gives the best chunks at the highest cost. It groups sentences by measured similarity, which the lesson says is expensive and more than most applications need. Sentence-based chunking (5 sentences, 1 sentence of overlap in the example) is offered as a middle ground.
• Choose the strategy by document type, then iterate on real data. The lesson recommends structure-based for consistent structure, sentence-based for mixed types, and character-based for code or unknown formats. Start simple and refine after testing.

2026-11-06 · open the original ↗

• Putting a whole large document in the prompt is a poor default. For an 800+ page financial report, the lesson gives three reasons: the text may exceed the hard input limit, Claude gets less effective with very long prompts, and big prompts cost more and run slower.
• RAG chunks the document and sends only the relevant pieces. A question about risks pulls in only the Risk Factors chunk. That keeps Claude focused, scales to large or multiple documents, and makes prompts cheaper and faster.
• RAG adds engineering work and can drop needed context. It needs a preprocessing step, a way to search for "relevant" chunks, and a choice among many chunking methods. Retrieved chunks can also miss context: Risk Factors alone leaves out the Strategy Outlook section on how the company plans to handle those risks.

2026-11-06 · open the original ↗

• Web search is built into Claude; you only supply a schema. The schema has type web_search_20250305, name web_search, and max_uses (5 in the example), which caps how many follow-up searches Claude can run. Claude then decides for itself when to search.
• allowed_domains restricts searches to trusted sources. The example limits searches to nih.gov, so health answers come from government sites rather than random fitness blogs.
• The structured response is designed for rendering with citations. It contains text blocks, the exact search query used, the search results with titles and URLs, and citations tying statements to source text. The lesson suggests showing the results as a reference list, citations inline, and the cited text highlighted, so users can check Claude's sources.

2026-11-06 · open the original ↗

• Claude knows the text editor tool's schema, but you write the implementation. Claude can ask to view files, directories or line ranges, replace text, create files, and insert lines. Your code has to perform each of those operations.
• A small, model-specific schema stub turns the tool on. For Haiku 4.5 the stub is type text_editor_20250728 with name str_replace_based_edit_tool, and Claude expands it into the full specification. Version strings for other models are in Anthropic's text editor tool docs.
• The tool lets your own app work like an AI code editor. In the example, Claude views main.py, rewrites it with a function that calculates pi to the 5th digit, and creates test.py with unit tests. The lesson pitches it for apps that edit files programmatically or run without a full-featured editor.

2026-11-06 · open the original ↗

• A forced tool call extracts structured data more reliably than prompting. You write a schema for the data you want and force Claude to call it, then read the result from response.content[0].input. No follow-up message is needed. The lesson contrasts this with the simpler prefill-and-stop-sequence approach, which is easier to set up but less reliable.
• tool_choice decides whether Claude must call a tool. auto (the default) lets the model decide, any requires some tool, and {"type": "tool", "name": ...} requires a specific one. Extraction uses the last option to guarantee the schema tool is called.
• Use prompting for quick jobs and tools for guaranteed structure. The lesson's article example extracts title, author and a list of key insights. It recommends tools whenever reliability matters more than the extra setup.

2026-11-06 · open the original ↗

• Claude often makes parallelizable tool calls one at a time. Asked to set two reminders for the same date, it typically calls the tool once, waits for the result, then calls it again, rather than sending two tool use blocks in one message.
• A batch tool nudges Claude into grouping its calls. The batch_tool schema takes a list of invocations, each with a tool name and arguments. run_batch runs each one through the normal run_tool router and returns the results together. The lesson openly calls it a workaround that "tricks" Claude.
• Batching cuts the round trips to one call and one reply. With the batch tool, both reminders went out in a single call and the results came back in one follow-up, which the lesson says reduces latency.

2026-11-06 · open the original ↗

• Once the tool plumbing exists, adding a tool takes three steps. Add its schema to the tools list in run_conversation, add a branch for it in run_tool, and write the function. The reminder system gets three tools: get the current datetime, add a duration (because Claude isn't reliable at date arithmetic), and a placeholder set_reminder.
• Claude chains tools across turns to finish a compound request. For "a reminder 177 days after Jan 1st, 2050", it first called add_duration_to_datetime to get June 27, 2050, then set_reminder. Each assistant message pairs a text block explaining the step with a tool use block.

2026-11-06 · open the original ↗

• Keep looping until Claude stops asking for tools. run_conversation calls Claude, appends the reply, and stops when stop_reason is anything other than "tool_use". Otherwise it runs the requested tools and sends the results back as a user message.
• Every tool use block needs a matching tool result block. run_tools picks out every tool_use block in the reply and builds a result for each, with the same ID in tool_use_id and the output as JSON in content.
• Return failed tool calls as results, not as exceptions. If a tool raises, the code still sends back a tool result with "Error: …" as the content and is_error: True, so Claude sees what went wrong.
• Route tool names through a separate run_tool function. A dispatcher that maps names to functions lets you add tools without touching the conversation loop.

2026-11-06 · open the original ↗

• One question can need several tool calls in sequence. "What day is 103 days from today?" makes Claude call get_current_datetime, then add_duration_to_datetime, before it answers. The application must handle that back-and-forth automatically.
• Helper functions must handle whole messages, not just text. add_user_message/add_assistant_message now accept a string, a list of blocks, or a full Message. chat takes a tools list and returns the full message, and text_from_message joins the text blocks for display. The point is to keep every block Claude returns, not just the text.

2026-11-06 · open the original ↗

• Call your function with the tool block's input, then return the output. The tool use block's input dict is unpacked with ** into the function. The output goes back in a tool_result block with the matching tool_use_id, the output as a string in content, and an is_error flag.
• Matching IDs pair each result with its request, whatever the order. When Claude asks for 10 + 10 and 30 + 30 in separate tool use blocks, the results can come back in either order, because each result carries its request's ID.
• Send the tool schema again in the follow-up request. The follow-up carries the full history plus the tool result. Even if Claude won't call the tool again, it needs the schema to understand the tool references in that history.

2026-11-06 · open the original ↗

• A tool-enabled request returns an assistant message with several blocks. Passing schemas in tools can produce a reply with a text block explaining the step and a tool use block. The tool use block holds an ID, the function name, and inputs matching the schema.
• Append the entire response content to history, not only the text. Claude keeps no history between calls, so messages.append({"role": "assistant", "content": response.content}) is needed to keep the tool use block for later calls. Helpers that only handle a single text block have to be updated.

2026-11-06 · open the original ↗

• A tool spec is a name, a description and a JSON Schema. JSON Schema is an existing data-validation standard that the AI field adopted. The lesson advises descriptions of 3-4 sentences covering what the tool does, when to use it and what it returns, plus a detailed description of each argument.
• Have Claude write the schema for you. Paste the function into Claude with Anthropic's tool-use docs attached and ask for a valid schema that follows their best practices. Store it under a function_name_schema variable, optionally wrapped in ToolParam to avoid type errors later.

2026-11-06 · open the original ↗

• Tool functions are plain Python that Claude calls for data or actions. The first example is get_current_datetime(date_format="%Y-%m-%d %H:%M:%S"), one of three tools planned for a reminder app, along with adding a duration and setting a reminder.
• Validate inputs and raise clear errors, because Claude reads and retries them. Claude sees the exact error text, so a message like "date_format cannot be empty" or "Location cannot be empty" lets it retry with corrected arguments. Descriptive function and parameter names are the other recommended practice.

2026-11-06 · open the original ↗

• Setting a reminder from a relative date is harder than it looks. For "Set a reminder for my doctor's appointment. It's a week from Thursday", the lesson says Claude may know the date but not the exact time, doesn't always handle date addition well far into the future, and has no way to set a reminder.
• Three custom tools cover those gaps. One gets the current date and time, one adds a duration to a date-time (rather than trusting Claude to work out 379 days from January 13, 1973), and one sets the reminder; they are built one at a time, starting with the date-time tool to teach tool calling.

2026-11-06 · open the original ↗

• Without tools Claude is limited to its training data. Asked for the weather in San Francisco, it can only reply that it has no access to up-to-date weather information.
• Tool use is a four-step exchange your server mediates. You send the question with instructions on how to get extra data; Claude decides it needs data and requests specifics (e.g. weather for a location); your server fetches it from an API or database; and you send it back so Claude answers from both the question and the fresh data.

2026-11-06 · open the original ↗

• Sample input/output pairs show Claude what descriptions struggle to say. One-shot or multi-shot prompting introduces each example ("Here is an example input with an ideal response") and wraps it in <sample_input> and <ideal_output> tags; it is also recommended for complex output formats like JSON or reports.
• Examples earn their keep on corner cases such as sarcasm. A tweet calling a film the best since "Plan 9 from Outer Space" reads positive but is negative, so the prompt adds a sarcastic example ("Oh yeah, I really needed a flight delay tonight! Excellent!") labelled Negative.
• Take examples from your highest-scoring eval outputs. Input/output pairs that scored 10 in the HTML eval report show Claude what "perfect" looks like for the task.
• Explaining why an example is ideal reinforces the wanted qualities. A short note after the ideal output (e.g. well-structured, detailed quantities, aligned with the athlete's goals) helps; the lesson advises starting one-shot and adding examples as eval results warrant.

2026-11-06 · open the original ↗

• XML tags separate instructions from interpolated data. With large inputs such as 20 pages of sales records, Claude can struggle to tell instructions from data; wrapping the data in a descriptively named tag like <sales_records> marks the boundary and says what the content is.
• Tags matter most when similar-looking content is mixed together. In a debugging prompt, <my_code> and <docs> tags let Claude tell the buggy code from the reference documentation instead of guessing.
• Even small inputs benefit from grouping. Wrapping the athlete's height, weight, goal and restrictions in <athlete_information> marks them as external input rather than instructions, and the lesson adds that tagged prompts are easier to maintain and debug.

2026-11-06 · open the original ↗

• An unspecified prompt leaves the output's shape to chance. A bare request for a short story about a hidden talent could come back at 200 or 2,000 words, with one character or five, as comedy or drama.
• Quality guidelines list attributes the output must have. Examples are a length limit (under 1,000 words), a structural requirement and a content requirement; the lesson says these suit almost any prompt.
• Process steps make Claude weigh options before answering. Steps such as brainstorming three talents and picking the most interesting suit troubleshooting, decisions and critical thinking; for a sales team's 30% quarterly drop, steps force it to consider market conditions, individual performance, organizational changes and customer feedback.
• Adding guidelines roughly doubled the meal-plan eval score. Six guidelines (accurate calories, protein/fat/carb amounts, meal timing, foods fitting restrictions, portions in grams, budget-friendly if mentioned) moved the score from 3.92 to 7.86.

2026-11-06 · open the original ↗

• Open the prompt with a plain instruction, not a question. The lesson calls the first line the most important part of a prompt: use simple language, state the task outright and start with an action verb such as "Write", "Create" or "Generate", e.g. "Write three paragraphs about how solar panels work" rather than a rambling question about "those solar panel things".
• Rewriting only the opening line lifted the eval score substantially. Replacing "What should this person eat?" with "Generate a one-day meal plan for an athlete that meets their dietary restrictions" raised the meal-plan score from 2.32 to 3.92.

2026-11-06 · open the original ↗

• Prompt improvement is an evaluate, change, re-evaluate loop. Set a goal, write an initial prompt, evaluate it, apply a technique, re-evaluate, and repeat the last two, making one change at a time so you know which technique helped.
• Start from a deliberately weak baseline. The running example generates a one-day athlete meal plan (inputs such as 180cm, 75kg, "maintain weight and improve power", lactose intolerance) from the prompt "What should this person eat?"; its score of 2.3 out of 10 across 50 test cases, with a 0.0% pass rate, is called a perfect starting point.
• Explicit criteria let the grader score consistently. Good output must include a daily calorie total, a macronutrient breakdown, and meals with exact foods, portions and timing.
• Read eval reports for patterns, not one-off failures. Per-case outputs, score breakdowns and grader reasoning show what is missing; systematic issues across many cases are what to fix.
• Keep evaluator concurrency low to avoid rate limits. Start at 1–3 concurrent requests and raise it only if no rate-limit errors appear.

2026-11-06 · open the original ↗

• Syntax validators score output by whether it parses. validate_json (json.loads), validate_python (ast.parse) and validate_regex (re.compile) return 10 on success and 0 on a parse error.
• Each test case must declare its expected format. A "format" field such as "python" tells the code grader which validator to use, and the dataset-generation prompt can be updated to emit it.
• Stricter instructions and a generic code prefill improve compliance. The prompt adds "Respond only with Python, JSON, or a plain Regex" and "Do not add any comments or commentary or explanation", and prefills "```code" so Claude starts writing raw code without naming the language.
• Averaging model and syntax scores weights quality and correctness equally. The final score is (model_score + syntax_score) / 2, with weights adjustable per use case; the lesson says the baseline is neither good nor bad, only a reference for whether prompt changes improve it.

2026-11-06 · open the original ↗

• Graders come in three kinds with different tradeoffs. Code graders run programmatic checks (length, required or banned words, JSON/Python/regex syntax, readability); model graders use another API call to judge quality, instruction following, completeness, helpfulness or safety; human graders are the most flexible but time-consuming and tedious.
• Define evaluation criteria before building any grader. For the code-generation prompt the criteria are format (only Python, JSON or regex), valid syntax and task following; the first two suit a code grader, the third a model grader.
• Ask the model grader for reasoning, not only a score. Requesting 1–3 strengths, 1–3 weaknesses and reasoning alongside a 1–10 score stops it defaulting to middling scores like 6; the grading prompt gives the grader a role, the task, the solution and an output format, parsed via prefill and stop sequence.
• Averaging scores gives one number to improve. run_eval reports the mean across test cases, though the lesson concedes model graders can be capricious and may need better guidance.

2026-11-06 · open the original ↗

• The eval pipeline is three nested functions. run_prompt merges a test case into the prompt template and calls Claude, run_test_case runs it and grades the output (for now a hard-coded score of 10), and run_eval loops over the dataset collecting results.
• Each result records the output, the test case and the score. A full dataset can take around 30 seconds even with Haiku, and outputs come back verbose because the prompt has no formatting instructions yet.
• This skeleton is most of an eval pipeline; grading is missing. The lesson says the complexity lies in the details: better prompts, real grading to replace the placeholder, and performance optimizations.

2026-11-06 · open the original ↗

• The example prompt must return bare code, nothing else. For AWS tasks it should produce Python, JSON or a regex, and output should carry no explanations, headers or footers; prompt v1 is just "Please provide a solution to the following task:" followed by the task.
• Claude can generate the eval dataset, and a fast model suffices. The lesson suggests Haiku since this is test data, with a generation prompt asking for an array of {"task": ...} objects solvable by a single Python function, JSON object or regex without much code, three objects to start.
• Prefill and a stop sequence keep the generated dataset parseable. Prefilling "json" and stopping on "" yields JSON for json.loads, saved to dataset.json; sample tasks included extracting the AWS region from an ARN, a Lambda JSON config, and an S3 bucket-name regex (lowercase, alphanumeric, 3–63 characters).

2026-11-06 · open the original ↗

• A typical eval is five steps ending in a loop. Draft a prompt, build a dataset, run each input through Claude, grade the outputs, then change the prompt and repeat; many open-source and paid tools exist, but the lesson recommends starting small.
• Eval datasets can be written by hand or generated by Claude. Real ones run to tens, hundreds or thousands of records; the example uses three questions ("What's 2+2?", making oatmeal, distance to the Moon).
• A grader scores each answer and the scores are averaged. Scores typically run 1–10 (10 perfect, 4 adequate with room to improve); the example's 10, 4 and 9 average to 7.66.
• Comparing averages shows whether a prompt change helped. Prompt v1 scored 7.66; v2, which added "Answer the question with ample detail", scored 8.7, which the lesson treats as objective evidence of improvement.

2026-11-06 · open the original ↗

• Prompt engineering writes prompts; prompt evaluation measures them. Engineering is the toolkit of techniques such as multishot prompting and XML tags; evaluation is automated testing that checks against expected answers, compares prompt versions and reviews outputs for errors.
• Testing a prompt once or a few times is a trap. The instructor says all engineers, themself included, fall into it, and that limited self-testing breaks in production because users supply inputs no one anticipated.
• An eval pipeline costs more upfront but replaces guessing with scores. Scoring the prompt and iterating on that data shows whether each change helps or hurts, which the lesson presents as the route to production-ready prompts.

2026-11-06 · open the original ↗

• Claude's habit of explaining its work breaks raw-data use cases. In the example AWS EventBridge rule generator, JSON wrapped in markdown fences with headers and footers stops users simply copying the whole output.
• Prefill plus a stop sequence returns only the structured content. Prefilling the assistant message with "json" and stopping on "" makes Claude assume it has already opened the block, write the JSON, and halt when it tries to close the fence; json.loads(text.strip()) then cleans and validates it, and the same trick works for Python code or bulleted lists.

2026-11-06 · open the original ↗

• A prefilled assistant message steers Claude's answer from its opening words. Claude treats a trailing assistant message as a response it already started and continues exactly where it ends, without repeating it; asked "Is tea or coffee better at breakfast?", it normally gives a balanced answer, but prefilled with "Coffee is better because" it argues for coffee, and "Neither is very good because" yields a contrarian case.
• Stop sequences end generation as soon as a given string appears. "Count from 1 to 10" with stop sequence "5" returns "1, 2, 3, 4, " (the stop string is excluded), while ", 5" gives the cleaner "1, 2, 3, 4"; uses include capping list length, stopping at delimiters and keeping formats consistent.

2026-11-06 · open the original ↗

• Streaming replaces the dead wait for a complete response. Responses can take 10–30 seconds; with streaming, Claude sends an initial response and then a series of events carrying pieces of text, all within a single request, which the server can forward to the client as they arrive.
• Only ContentBlockDelta events carry the generated text. The event sequence is MessageStart, ContentBlockStart, ContentBlockDelta (repeated), ContentBlockStop, MessageDelta and MessageStop; raw streaming is enabled with stream=True on messages.create.
• The SDK's stream helper filters to text and assembles the final message. client.messages.stream(...) exposes stream.text_stream for text only, and stream.get_final_message() afterwards returns the complete message for storage or conversation history.
• Chunks are not one word each. A chunk can hold several words or whole sentences depending on generation speed; in production the lesson expects chunks forwarded via WebSockets or Server-Sent Events.

2026-11-06 · open the original ↗

• Temperature reshapes the probabilities Claude samples the next token from. It runs from 0 to 1; in the example "about" has 30% and "would" 20%, and near 0 Claude almost always takes the top token (shown as "about" at 100%), while near 1 probability spreads out (23% down to 9% across seven tokens).
• Choose the temperature range by task. Low (0.0–0.3) for factual answers, coding, data extraction and moderation; medium (0.4–0.7) for summarization, educational content, problem-solving and constrained creative writing; high (0.8–1.0) for brainstorming, creative writing, marketing and jokes.
• Higher temperature raises the odds of variety without guaranteeing it. Generating movie ideas at 0.0 kept returning a time-travelling archaeologist premise, while 1.0 gave more variety, though similar outputs can still recur; the chat helper gains a temperature=1.0 parameter.

2026-11-06 · open the original ↗

• A system prompt makes Claude answer as the named role would. It is a plain string passed as system to the create call; the math-tutor example tells Claude to give hints and guide step by step rather than hand over answers or say "use a calculator".
• The change in output is large. For "How do I solve 5x + 2 = 3 for x?", Claude without a system prompt gives the full solution; with the tutor prompt it asks what a good first step to isolate x would be.
• The API rejects system=None, so add it conditionally. The reusable chat(messages, system=None) builds a params dict and includes system only when one is supplied.

2026-11-06 · open the original ↗

• The API keeps no conversation history between requests. Every call is independent, so a follow-up like "Write another sentence" after a quantum-computing question gets a sentence about something random.
• Context comes from resending the whole message list each time. Your code keeps the list, appends Claude's reply as an "assistant" message and the follow-up as a "user" message, then sends everything; the lesson wraps this in three helpers, add_user_message, add_assistant_message and chat.

2026-11-06 · open the original ↗

• The SDK needs its vertex extra and a Vertex-specific client. Install with %pip install "anthropic[vertex]<1", then create AnthropicVertex(region="global", project_id="your-project-id"); the lesson's model is claude-haiku-4-5@20251001, stored in a variable for reuse.
• max_tokens is a budget, not a target. At 1000, Claude writes whatever length it judges appropriate and only stops if it would exceed the limit.
• Messages carry the conversation as role-tagged dictionaries. Each has a role ("user" for human-written input, "assistant" for Claude's output) and content; client.messages.create takes model, max_tokens and this messages list.
• The generated text sits at message.content[0].text. The full response object carries a lot of metadata, and this pattern extracts just the text.

2026-11-06 · open the original ↗

• Anthropic models must first be enabled in Vertex's Model Garden. In the Google Cloud console, open Model Garden, search "Anthropic", pick the model and click Enable; if there is no Enable button, you already have access.
• The SDK uses gcloud's application-default credentials automatically. After installing the gcloud CLI, run gcloud init, gcloud auth login, gcloud config set project YOUR_PROJECT_ID and gcloud auth application-default login; no further credential handling is needed in code.

2026-11-06 · open the original ↗

• Never call the API directly from client code. Requests carry secret credentials that anyone could read in client code, so the lesson routes every request through a server you control, which talks to Vertex using Anthropic's SDKs (Python, TypeScript, Go, Ruby) or Google's Vertex SDKs.
• Every request needs four fields. Google Cloud credentials to authorize it, the model name, a messages list with the user's input placed in a "user" message, and max tokens to cap how much the model generates.
• Claude builds a response one token at a time. Input is tokenized, each token becomes an embedding (a list of numbers standing for its possible meanings), embeddings are adjusted by their neighbours to settle meaning (e.g. "quantum" as physics, computing or "very small"), and an output layer gives next-word probabilities. Claude samples using a mix of probability and randomness rather than always taking the top word, appends it, and repeats.
• Generation stops on one of three conditions. The max-tokens limit is reached, the model emits an end-of-sequence token (an invisible signal of a natural ending), or it produces a predefined stop sequence.
• The response metadata is worth watching. Vertex returns the generated message, usage counts of input and output tokens, and the stop reason; the server forwards the text to the client, and the lesson advises monitoring the metadata to track usage and understand model behaviour.

2026-11-06 · open the original ↗

• Agents observe the environment far more than they change it. In the Claude Code example of adding a test for a corner case, three of four tool calls (two file reads, one test run) gathered information and only one modified a file; Computer Use likewise gets a screenshot back from every action.
• Effective agents run a small, focused toolset in a loop. The lesson says agents should keep calling a few simple, well-defined tools until they hit an iteration limit, an error, or the success criteria, which lets them course-correct.
• Context comes from tools, not from long prompts or RAG. Claude has no built-in knowledge of your environment, so the lesson argues the quality of an agent's context-gathering tools directly sets how well it performs.
• Point agents at high-value tasks where mistakes are cheap. Writing code is the model case, expertise-heavy but with errors that can be caught and fixed; the lesson rates Computer Use's error cost very low and Claude Code's low but not negligible, and warns off high-stakes economic or safety decisions.
• Only continuous evaluation makes agents reliable. The lesson says the sole dependable method is to define evaluation criteria and keep testing the agent's prompts and tool descriptions against real scenarios.

2026-11-06 · open the original ↗

• Computer use is ordinary tool use with a different tool. The flow is the same as a get_weather call: you send tool schemas, Claude returns a tool-use request, your server runs it and sends the result back; only now the tool clicks, types, scrolls and takes screenshots.
• A minimal schema is expanded behind the scenes into the full interface. You send a small computer_20250124 tool schema with the display size, and it is converted into a detailed specification with action, coordinate, duration and text properties.
• Claude never controls the computer directly; your infrastructure does. Claude only issues tool requests, and your server executes them in an environment that can simulate input, which in the reference implementation is a Docker container running Firefox.
• Anthropic's reference implementation removes the need to build this yourself. It needs a Docker runtime (orbstack.dev is given as an example), a local AWS profile (usually "default"), and the code from the Anthropic quickstarts repository on GitHub, then starts with one docker run command and shows a chat panel beside a browser.

2026-11-06 · open the original ↗

• Computer use lets Claude perform tasks, not just describe them. It takes screenshots, clicks, types, moves between applications and tabs, and follows multi-step processes that need visual feedback, which the lesson says suits QA testing of user interfaces.
• Claude can run a QA test plan and find real bugs. Given three test cases for a React "@" mention autocomplete, it passed typing "Did you read @" and pressing Enter, but found the dropdown appeared in the wrong place after backspace, and reported this with screenshots.
• The testing run follows a repeatable structure. Claude opens the app in a browser, works through each case step by step, verifies with screenshots, refreshes the page between tests for a clean state, and ends with a pass/fail summary report; the lesson stresses it runs identically after every code change.
• Computer use should always run in an isolated environment. The typical setup is a Docker container with a desktop, a controllable browser and a chat interface, fully separated from your main system so your data is not at risk.
• Give specific tasks and success criteria, then verify the findings. The lesson's best practices are to break complex workflows into small steps, state clear success criteria, and manually check any important result Claude reports.

2026-11-06 · open the original ↗

• Production-only errors are costly to find by hand. In the example, a chatbot deployed to AWS Amplify failed silently on spreadsheet generation that worked locally, and the cause, "The provided model identifier is invalid", was buried in CloudWatch logs.
• A scheduled GitHub Action can hand the whole debugging loop to Claude. Run daily, it checks out the repo, pulls the last 24 hours of CloudWatch logs with the AWS CLI, has Claude deduplicate the errors and fix each one, then commits and opens a pull request.
• Claude traced the example failure to a typo'd model ID. The invalid identifier was referenced only in production configuration; Claude corrected it in lib/ai/providers.ts and the pull request carried Error, Root Cause and Fix Implemented sections.
• The pull request keeps a human review step in the loop. The lesson presents the PR as the point where you verify a fix before merging, alongside proactive detection and documented explanations.
• The workflow needs limits and permissions set deliberately. The lesson lists CloudWatch access permissions, repository write permissions for PRs, a cap on errors processed to fit the context window, deduplication logic, and running off-peak.

2026-11-06 · open the original ↗

• Running several Claude Code instances at once is a major productivity gain. Because Claude is lightweight, the lesson says you can give each copy a separate task and in effect get a team of virtual engineers, though instances editing the same files will produce conflicting code.
• Git worktrees give each instance an isolated copy of the project. Each worktree is a full copy of the codebase on its own branch in its own directory, so an instance works without interference and its branch is merged back like any other.
• Claude Code can automate creating its own worktrees. The example prompt has it create a worktree in a .trees folder (stopping if one exists), symlink untracked dependencies such as .venv, and launch VS Code in that directory.
• Custom slash commands turn the long prompts into reusable commands. A .md file in .claude/commands becomes a command, and $ARGUMENTS is replaced by what you pass, so /project:create_worktree feature_a builds a worktree named feature_a.
• A second command lets Claude merge worktrees and resolve conflicts. It changes into the worktree, examines the latest commit, returns to the root, merges the branch, and resolves conflicts from its understanding of each branch's changes; the lesson claims capacity multiplies by the number of instances.

2026-11-06 · open the original ↗

• Claude Code's built-in MCP client lets servers add new capabilities. Each connected server can supply tools, prompts and resources; the example adds a custom server whose document_path_to_markdown tool converts PDF and Word files to markdown.
• One command registers a server with Claude Code. claude mcp add documents uv run main.py takes a server name of your choosing and the command that starts the server; you stop Claude Code first and it connects on restart.
• Existing MCP servers already cover common development services. The lesson lists sentry-mcp for fixing logged bugs, playwright-mcp for browser automation, figma-context-mcp, mcp-atlassian for Confluence and Jira, firecrawl-mcp-server for scraping, and slack-mcp for posting messages.
• Combining servers tailors Claude Code to your own workflow. The suggested pattern pairs Sentry for production errors, Jira for ticket requirements and Slack for team notification, plus custom servers for in-house tools.

2026-11-06 · open the original ↗

• Claude Code is meant to cover the whole project lifecycle. The lesson frames it across five stages, Discover, Design, Build, Deploy, and Support and Scale, rather than as a code-writing tool only.
• /init records your codebase in a CLAUDE.md Claude rereads automatically. It notes structure, dependencies, commands and patterns; CLAUDE.md files can be project-level (checked into git), local (not checked in) or user-level (all projects), and the # shortcut appends quick notes.
• Planning before coding gets better results on complex features. The workflow is to have Claude read the relevant files, ask it to plan the solution while explicitly not writing code, then have it implement the plan.
• Test-driven development costs more upfront but makes Claude far more effective. Claude reads the context, proposes test cases without writing code, implements only the relevant tests, then iterates on code until they pass, which the lesson says catches unconsidered edge cases.
• Claude Code also handles routine development chores. The lesson lists environment setup and dependency installs, staging and committing with descriptive messages, running and interpreting tests, and /clear to reset context.

2026-11-06 · open the original ↗

• Each MCP primitive is controlled by a different layer of the stack. The lesson's organizing claim is that Claude controls tools, your application controls resources, and the user controls prompts.
• Tools are model-controlled and extend what Claude can do. Claude alone decides when to call them; asked for the square root of 3 using JavaScript, it chose a JavaScript execution tool without any prompting from the app or user.
• Resources are app-controlled and get data into your application. Your code decides when to fetch them, for autocomplete, file pickers, or adding context to messages; Claude's "Add from Google Drive" lists documents in the UI and injects the chosen one into the chat.
• Prompts are user-controlled predefined workflows. Users trigger them through buttons, menu options or slash commands, like the writing, learning and coding buttons under Claude's chat input.

2026-11-06 · open the original ↗

• Listing prompts is a thin wrapper over the client session. list_prompts awaits the session's list_prompts call and returns result.prompts.
• get_prompt interpolates arguments and returns messages ready for Claude. The arguments, such as a doc_id for the "format" prompt, are passed as keyword arguments to the server's prompt function, and the returned messages can go straight to Claude.
• In the client, prompts surface as slash commands. Typing "/" lists /format; choosing it and a document sends the interpolated prompt as a user message, and Claude then uses the server's tools to fetch the document and return it reformatted.
• Server prompts should be well-tested and specific to the server's purpose. The workflow is to write and evaluate the prompt, define it with @mcp.prompt, and let any client request it with arguments, making it reusable across clients.

2026-11-06 · open the original ↗

• Server-supplied prompts beat what users would write themselves. A one-line "Convert report.pdf to markdown" works, but the lesson argues a tested, detailed prompt covering edge cases does better, and server authors can craft that once for every user.
• Prompts are defined with a decorator, like tools and resources. @mcp.prompt takes a name and description, and the function returns a list of base.Message objects, so a prompt can hold several user and assistant messages; base is imported from mcp.server.fastmcp.prompts.
• The example "format" prompt chains the server's own tools. It takes a doc_id, tells Claude to read the document, rewrite it with markdown headers, bullets and tables, and save it with the edit_document tool.
• Test prompts in the MCP Inspector before shipping them. The Prompts tab takes the parameters (doc_id set to outlook.pdf in the example) and shows the generated messages, confirming interpolation and content.
• Prompts give quality control, consistency and reuse. The lesson lists authors refining prompts before users see them, everyone getting the same prompt, domain specialization, and use across multiple client applications.

2026-11-06 · open the original ↗

• Resources put data into the prompt without a tool call. When a user types "@", the client sends a ReadResourceRequest for docs://documents to populate autocomplete, and the selected document's content goes straight into the prompt to Claude.
• The client's read_resource must branch on MIME type. It calls the session's read_resource with an AnyUrl, takes the first item of result.contents, and returns json.loads of the text for application/json or the raw text otherwise.
• Resources suit relatively static reference material. The lesson credits them with lower latency, no extra calls during the conversation, a better autocomplete experience, and a cleaner split between static data and dynamic operations, for things like documentation and reports.

2026-11-06 · open the original ↗

• Resources are read-only endpoints, like GET handlers in HTTP. They are addressed by URI, can return strings, JSON or binary data, and carry a mime_type as a hint to the client; a client fetches one with a ReadResourceRequest.
• Resources are either direct or templated. Direct ones have fixed URIs such as docs://documents; templated ones carry parameters such as docs://documents/{doc_id}, which the Python SDK parses and passes to the function as keyword arguments of the same name.
• The SDK serializes return values for you. With @mcp.resource, list_docs returns a Python list as application/json and fetch_doc returns text/plain, raising ValueError for an unknown ID, with no manual JSON conversion.
• The Inspector lists direct and templated resources separately. Started with uv run mcp dev mcp_server.py, it shows direct ones under Resources and templated ones under Resource Templates, each testable for its exact response.
• Resources let clients fetch data proactively, without tools. The lesson names autocomplete lists, file contents, database records and configuration as uses, and document mentions injected into prompts as the key one.

2026-11-06 · open the original ↗

• Real projects build an MCP client or a server, rarely both. The course builds both only so the interaction can be seen end to end.
• The client wraps the SDK's ClientSession to manage cleanup. The session from the MCP Python SDK does the low-level communication but needs careful resource cleanup at shutdown, which the custom MCP Client class handles.
• The application needs just two client calls: list_tools and call_tool. The CLI lists tools to send to Claude and runs tools when Claude asks; each is a one-line await on the session, and uv run mcp_client.py runs a test harness that prints the tool definitions.
• MCP tool schemas differ from what Bedrock expects. The MCP spec has its own tool format, so the project's to_bedrock_tools function in core/bedrock.py converts MCP definitions before they reach Claude.
• The full loop runs from a plain question to a tool call. With uv run main.py, asking for the contents of report.pdf makes Claude pick read_doc_contents, which the client executes on the server before Claude answers.

2026-11-06 · open the original ↗

• The SDK's browser inspector tests a server without a full application. mcp dev mcp_server.py starts it on a local URL, typically port 6277; the lesson warns the interface is under active development and may look different.
• Tools can be run and checked directly from the inspector. After Connect and List Tools, selecting read_doc_contents with "deposition.md" returns its text, and edit_document can be run with old and new strings and then verified with another read.
• The inspector supports a fast edit-restart-test loop. It keeps a history of tool calls for repeating tests, so you change server code, restart the inspector and verify immediately.

2026-11-06 · open the original ↗

• The MCP Python SDK replaces hand-written JSON schemas with decorators. @mcp.tool plus type hints and pydantic Field descriptions generates the tool schema from the function signature, with validation and error handling handled by the SDK.
• The course code breaks on the newer SDK major version. It uses version 1; version 2 renamed FastMCP, so the from mcp.server.fastmcp import FastMCP import fails; the pin is uv add "mcp[cli]<2" or pip install "mcp[cli]<2".
• The example server keeps six documents in an in-memory dictionary. FastMCP("DocumentMCP", log_level="ERROR") serves docs keyed by ID, such as deposition.md and report.pdf, each mapped to a one-line content string.
• Two tools read and edit documents, each rejecting unknown IDs. read_doc_contents returns a document's text; edit_document does a find-and-replace with Python's str.replace using doc_id, old_str (which must match exactly, including whitespace) and new_str; both raise ValueError for a missing doc_id.
• The Claude Agent SDK's @tool decorator works differently. It takes an explicit input_schema, either a dict mapping names to types like {"latitude": float} or a full JSON Schema for enums, ranges, optional or nested fields, rather than Field annotations; the lesson advises checking which SDK tool code imports.

2026-11-06 · open the original ↗

• The course project is a CLI chatbot over in-memory documents. It gives Claude tools to read and update documents, lets users mention a document with @doc_name and run commands with /command_name, and keeps all files in memory to stay focused on MCP.
• Building both client and server in one codebase is for teaching only. In real projects you would usually build an MCP server to distribute a service to other developers, or a client to connect to existing third-party servers, not both.

2026-11-06 · open the original ↗

• MCP is transport agnostic. Client and server most often run on the same machine and talk over standard input/output, but they can also connect over HTTP, WebSockets or other network protocols.
• Two request/result pairs carry most of the traffic. ListToolsRequest/ListToolsResult asks a server what tools it provides, and CallToolRequest/CallToolResult runs a named tool with arguments and returns the output.
• Your server no longer executes tools; the MCP server does. In the "What repositories do I have?" example, your server gets the tool list through the client, sends query plus tools to Claude, then on a tool-use reply asks the client to run it; the MCP server calls GitHub and the result returns via the client to Claude for the final answer.
• The many steps buy a clear division of responsibility. The lesson concedes the flow is long but argues each component has one job, and the client hides the server communication from your application logic.

2026-11-06 · open the original ↗

• MCP moves tool definition and execution onto dedicated servers. An MCP client in your server connects to MCP servers holding tools, prompts and resources, each wrapping an outside service such as GitHub, AWS or a database.
• Without MCP, broad integrations mean writing many tool schemas yourself. Supporting a question like "What open pull requests are there across all my repositories?" would require authoring, testing and maintaining tools for GitHub's repositories, pull requests, issues, projects and more.
• Anyone can write an MCP server, often the service provider. The lesson gives AWS releasing an official server for its services as an example, and says calling an API directly still leaves you writing the schemas and functions.
• MCP complements tool use rather than duplicating it. Tool use is Claude calling functions; MCP is about who provides them, already implemented in a server instead of authored by you.

2026-11-06 · open the original ↗

• Caching is manual, set by cache points in the request. It is not automatic; a {"cachePoint": {"type": "default"}} part caches the work for everything before it and nothing after it, and the first request writes the cache that follow-ups read within its 5-minute life.
• The cache is used only when the prefix is identical. Any change before the cache point, even adding "Please" to the start of the prompt, forces Claude to reprocess everything from scratch.
• A cache point can cover a whole conversation history. Placed at the end of a user/assistant/user sequence, it caches the processing for all those messages, assistant turns included.
• Content under the minimum length is not cached. The total before the cache point must be at least 1024 tokens; one "Hi there!" falls short, while repeating it about 500 times qualifies.
• System prompts and tool definitions are the best caching targets. Cache points can go at the end of the tools list or the system prompt, and the lesson calls these the most valuable because they rarely change between requests.

2026-11-06 · open the original ↗

• Claude normally discards its input processing after every response. It tokenizes the prompt, creates embeddings and adds context before generating output, then throws that work away, so a follow-up that resends the same message forces the same work again.
• Prompt caching stores that work for reuse on repeated content. The cache maps specific input messages to their processing results, making follow-up requests that hit it cheaper and faster.
• The short cache life still fits common usage. The cache lives 5 minutes and only helps with repeated content, but the lesson argues chatbots, document analysis and other tools constantly resend system prompts, history and large documents within a session.

2026-11-06 · open the original ↗

• Citations address the trust problem in PDF answers. The lesson argues users otherwise have to take Claude's reading of a document on faith, and citations show where each piece of information came from.
• One document parameter enables citations. Adding "citations": {"enabled": True} to the document block (format "pdf", bytes source) makes Claude track and return where it found information.
• Cited responses add structured parts mapping statements to the source. Alongside text parts, the response carries citations content giving the document, page numbers and exact supporting text, which the lesson says makes answers verifiable and auditable for professional and research use.

2026-11-06 · open the original ↗

• A PDF is sent as a document part, not an image part. The part is {"document": {"format": "pdf", "name": ..., "source": {"bytes": ...}}}, where name is the filename without its extension and the bytes are the file read in binary mode. A text part with the instruction follows it.
• Claude reads the whole multi-page PDF, including layout and images. In the lesson's example it summarized a Wikipedia article on Earth in one sentence. The lesson lists extraction, question answering, structure analysis and mixed text-and-image PDFs as supported uses.
• PDF input becomes more useful when combined with citations. Citations let Claude point to specific parts of the document in its answer, which the lesson recommends for research and content-extraction workflows.

2026-11-06 · open the original ↗

• Image input has hard per-request limits and a token cost. The limits are up to 20 images across all messages in a request, at most 3.75MB each, and at most 8000px in height or width. Each image costs ⌈width/28⌉ × ⌈height/28⌉ tokens.
• Each image goes in as its own part of the user message. The part is {"image": {"format": "png", "source": {"bytes": ...}}}, followed by a text part. Sending several image parts in one message lets Claude compare them or find a common theme.
• Prompt engineering raises vision accuracy just as it does for text. Asked plainly, Claude counted 13 marbles in a photo of 12. A prompt that had it number each marble and then re-count row by row from the bottom-left got 12, and so did a one-shot example image labelled with its correct count of 11.
• Complex visual assessments need detailed, step-by-step prompts. The worked example is an insurer scoring fire risk from satellite images instead of sending inspectors. The prompt walks Claude through dense trees near the house, emergency access routes, overhanging branches and overall tree spacing.

2026-11-06 · open the original ↗

• Extended thinking returns a visible reasoning part before the answer. The response has a reasoning-content part and then the text part. The reasoning shows how Claude broke the problem down, which helps when debugging complex tasks.
• Use extended thinking only after evaluations show prompt work isn't enough. It improves accuracy on complex tasks, but every thinking token is billed and it adds latency. The lesson's rule is to optimize the prompt first, then turn thinking on only if the evaluations still fall short.
• A signature makes the reasoning tamper-proof, and redacted reasoning still works. The signature lets reasoning be passed back in later turns while proving it was not modified, which the lesson presents as a safety measure. Reasoning flagged by safety systems comes back as an encrypted redactedContent field. You can't read it, but you can pass it back without losing context, and a special test string forces this case so you can test your handling of it.
• Current models use adaptive thinking, and manual budgets are being removed. On Claude Opus 5 and Sonnet 5, thinking is adaptive and on by default, and "display": "summarized" is needed to see the reasoning. The effort parameter guides how much Claude thinks. type: "enabled" with budget_tokens is deprecated on Opus 4.6 and Sonnet 4.6, and Opus 4.7 and later reject it with a 400 error.
• Older models still need a manual thinking budget. Claude Haiku, Sonnet and Opus 4.5 and earlier have no adaptive thinking. They take type: "enabled" with budget_tokens, minimum 1024, tuned through evaluations. For a new task, the lesson says to read the reasoning on a few typical requests and re-run evaluations before keeping the setting.

2026-11-06 · open the original ↗

• Chunks lose document context, so Claude adds it before indexing. Each chunk is sent with the source document and a prompt asking for a short snippet that places the chunk in the document. That snippet is put in front of the chunk, and the combined "contextualized chunk" is what goes into the vector and BM25 indexes.
• For documents too long to send whole, use partial context. Instead of the full document, the prompt gets a few chunks from the start, which often hold summaries or abstracts, plus the chunks just before the target. In the diagram, chunks 1–3 and 7–8 serve as context for chunk 9.
• The added context matters more as documents get more cross-referenced. In the example, a chunk is described as Section 2 of an Annual Interdisciplinary Research Review, sitting between Methodology and Financial Analysis, in a report covering ten research domains. The lesson says simple documents may show little improvement.

2026-11-06 · open the original ↗

• Hybrid search can still put the wrong chunk first. For "What did the eng team do with INC-2023-Q4-011?", the merged results can rank the cybersecurity section above the software engineering section, even though the engineering section is the better answer.
• Re-ranking asks Claude to reorder the merged results by relevance. After the vector and BM25 results are merged, a reranker prompt gives Claude the user question and the candidate documents, and asks for the top k in decreasing order of relevance.
• Have Claude return chunk IDs, not chunk text. Each chunk gets a unique ID in advance, so Claude only returns a short list such as ["1p5g", "51n3", "ab83"], which is much faster than copying text back. The code reads this list from a document_ids JSON field, using a stop sequence on ```.
• Re-ranking adds latency in exchange for better accuracy. The lesson says Claude correctly ranks the engineering section first for an engineering-team query, at the cost of an extra model call on every search.

2026-11-06 · open the original ↗

• One Retriever class runs every search index and merges the results. The vector and BM25 indexes both expose add_document() and search(). The Retriever sends documents and queries to each index, then merges what comes back.
• Reciprocal Rank Fusion merges results by rank, not raw score. Scores from different search methods can't be compared, so each document scores Σ 1/(k + rank) across the systems, with k typically 60 (1 gives clearer numbers). With k=1, Section 2 (ranks 1 and 2) scores 0.833, Section 6 (ranks 3 and 1) scores 0.75 and Section 7 (ranks 2 and 3) scores 0.583, in that order.
• Hybrid search beat either method alone on the incident-ID query. For "what happened with INC-2023-Q4-011?" the cybersecurity incident section came first at about 0.0325, followed by Section 2 (software engineering) at about 0.0320.
• Any index with the same interface can be added. Because the Retriever depends only on add_document() and search(), the lesson says a different embedding model or a domain-specific search can be plugged in without other changes.

2026-11-06 · open the original ↗

• Semantic search misses exact identifiers that appear in the text. A query for incident "INC-2023-Q4-011" returned the Cybersecurity and Financial Analysis sections rather than Cybersecurity and Software Engineering, the two that mention the ID. Embeddings match on meaning, not on exact strings.
• BM25 ranks chunks by how often they contain rare query terms. It tokenizes the query, counts how often each term appears across documents, gives rarer terms more weight, and scores chunks by how many high-weight terms they contain. A common word like "a" counts for little, while a unique ID counts for a lot.
• BM25 found the right sections and uses the vector store's interface. With add_document() and search(), the same query returned the Software Engineering and Cybersecurity sections. The lesson's next step is to run both searches in parallel and merge the results.

2026-11-06 · open the original ↗

• The basic RAG flow takes five steps of code. Split report.md into sections with chunk_by_section, embed each chunk, store it in a VectorIndex, embed the user's query, then call store.search(user_embedding, 2). The embedding step makes one API call per chunk, so it is slow.
• Store each chunk's text with its embedding. The lesson calls this the key design decision. A search that returns only vectors is useless for building a prompt, so each vector is saved with {"content": chunk}.
• Lower cosine distance means a closer match. For "What did the software engineering dept do last year?", the software engineering section came first at a distance of 0.71 and the methodology section second at 0.72.

2026-11-06 · open the original ↗

• A made-up two-number embedding shows how the pipeline works. In the imaginary model, the first number measures how medical a text is and the second how much it is about software. The medical section, which says "bug", embeds as [0.97, 0.34]. The software section, which says "infection vectors", embeds as [0.30, 0.97].
• Embeddings are normalized to length 1 before they are stored. The embedding API usually does this. The vectors become [0.944, 0.331] and [0.295, 0.955], points on the unit circle, and are stored ahead of time in a vector database until a query comes in.
• The query goes through the same model, and cosine similarity finds the closest chunk. "What did the software engineering dept do this year?" embeds as [0.1, 0.89], or [0.112, 0.993] after normalization. Its cosine similarity is 0.983 with the software chunk and 0.398 with the medical one. Similarity runs from −1 to 1, and cosine distance is 1 − similarity, so near 0 means similar.
• The best-matching chunk and the question go into Claude's prompt. The full flow is: chunk, embed, store, embed the query, find the most similar chunks, build the prompt, send it to Claude. The lesson says understanding the math helps when retrieval returns unexpected chunks.

2026-11-06 · open the original ↗

• Finding relevant chunks needs semantic search, not keyword matching. In the example, the question "How many bugs did engineers fix this year?" shares the word "bug" only with the medical chunk, not the software chunk. Embeddings match on what a text means rather than which words it uses.
• An embedding is a list of numbers that scores the text's meaning. A model typically outputs 1024 numbers between −1 and +1. You can think of each one as scoring some quality, such as how happy the text is or how much it mentions oceans, but what each number actually measures is learned in training and can't be read by people.
• Bedrock embeddings come from Amazon Titan through invoke_model. The example uses amazon.titan-embed-text-v2:0 with dimensions=1024 and normalize=True. Access may have to be requested in the Bedrock console, and v1 is fine for learning.

2026-11-06 · open the original ↗

• Bad chunking puts the wrong context in front of the model. In the example, "How many bugs did engineers fix this year?" can pull up the medical research section because it calls a pathogen (XDR-47) a "bug", so the model answers wrongly.
• Size-based chunking is simple but cuts through words and context. A 325-character document split into three chunks of about 108 characters breaks "significant" into "signific" and "ant". Overlapping chunks fix the lost context at the cost of some duplication, which the lesson says is usually worth it. The example defaults are 150 characters with 20 of overlap.
• Structure-based chunking works well only when the format is guaranteed. Splitting markdown on \n## lines the chunks up with the document's sections. It is unreliable for plain text or documents formatted inconsistently.
• Pick a strategy by document type, then test and refine. Character-based chunking is the safest fallback for user uploads. Sentence-based (5 sentences with 1 of overlap) suits prose but breaks on code that uses periods. Section-based gives the best results on structured internal documents. Semantic grouping is described as more relevant but costly.

2026-11-06 · open the original ↗

• Putting a whole large document in the prompt hits hard limits. For something like an 800-page financial report, token limits may not fit it at all. Very long prompts also make Claude less effective, cost more and run slower.
• RAG sends Claude only the chunks relevant to each question. The document is split ahead of time into sections like Risk Factors or Balance Sheet. A question about company risks retrieves only the Risk Factors chunk, which scales to large documents and collections with smaller, cheaper prompts.
• RAG adds complexity, so it isn't always worth it. It needs a chunking step, a way to search the chunks, and a choice among chunking strategies, and the retrieved chunks may still miss context the answer needs. The lesson says simply including everything is sometimes fine.

2026-11-06 · open the original ↗

• Claude has the text editor tool's schema built in, but you implement it. Unlike a custom tool, you don't write the schema. Your code still has to carry out the view, str_replace, create and insert commands Claude sends, routed under the name str_replace_based_edit_tool.
• The tool version must match the model. Claude 4 and later use text_editor_20250728. Earlier models use text_editor_20250124, where the tool is named str_replace_editor and also has an undo_edit command. The lesson points to the AWS documentation for current IDs.
• With file access, Claude can work as a coding assistant. Asked to describe ./main.py, Claude sends {"command": "view", "path": "./main.py"} and answers from the file it reads. Asked to add a function that calculates pi to 5 digits and a test file, it reads, edits and creates the files itself.

2026-11-06 · open the original ↗

• One generic to_json tool can replace a schema for each extraction task. Its input schema is an object with additionalProperties: True, so it accepts any structure. The structure you want is written in the prompt, and tool_choice="to_json" forces Claude to call it.
• Changing the output structure is a prompt edit. Adding a field such as "num_topics": int takes one line in the prompt and no schema changes.
• The lesson puts it at about 90% of the quality for 10% of the setup. It recommends the flexible tool for prototyping, simple extractions and requirements that change often. Dedicated schemas remain the choice for critical production extraction, complex nested data and the highest accuracy.

2026-11-06 · open the original ↗

• A tool's input schema can produce more reliable structured output. You define a tool whose parameters are the fields you want, and Claude "calls" it with the extracted data as arguments. The lesson says this beats prefill-and-stop-sequence prompting, though it takes more setup, which is why both methods are taught.
• Force the specific tool, then stop after the tool call. toolChoice can be auto (the default), any (must use some tool), or tool with a name (must use that tool). For extraction the named option is almost always right. The data is read from the tool-use arguments and no follow-up turn is sent.
• Worked example: pull title, author and topics into a schema. The article_details tool requires title and author as strings and topics as an array of strings. The course's chat helper takes tool_choice="article_details" and builds the nested toolChoice object for you.

2026-11-06 · open the original ↗

• Claude often makes independent tool calls one at a time. Asked for March 12th, 2025 plus 50 days and plus 100 days, it could request both in one message. The lesson says some versions tend to make the calls sequentially instead, which is less efficient.
• A batch_tool pushes Claude to bundle calls into one request. Its schema takes a list of invocations, each with a tool name and its arguments as a JSON string. With it in place, the date question came back as a single batch call containing both calculations.
• The batch handler sends each invocation through the existing run_tool. run_batch parses each invocation's arguments and collects each tool's name and output. Unlike other tools, batch_tool gets the raw tool_input rather than **tool_input.

2026-11-06 · open the original ↗

• Once tool use is set up, a new tool takes two changes. Add its schema to the tools list passed to chat, and add a branch for its name in run_tool. An unknown name raises an exception. This lesson adds add_duration_to_datetime and set_reminder, the latter only printing for now.
• Claude plans and chains several tools for a single request. "Set a reminder to go to the doctor. The appointment is in 100 days" led Claude to get the current date, add 100 days and set the reminder. It explained its plan first and set the reminder for 2025-07-12. The lesson adds the date-arithmetic tool because Claude is not perfect at that math on its own.

2026-11-06 · open the original ↗

• Check the stop reason before processing tools. If the code assumes every response needs tool handling, plain answers like "What is 1+1?" add empty tool-result messages to the history. A stop reason of tool_use means Claude wants a tool. The others are end_turn, max_tokens and stop_sequence.
• chat should return the parts, the stop reason and the joined text. The lesson joins the text of every text part rather than assuming the first part is text, which it calls more robust.
• A loop runs tools until Claude stops for another reason. run_conversation calls Claude, adds the reply to the history, and while the stop reason is tool_use runs the tools and appends their results as a user message. "What time is it?" uses the tool, and "What is 1+1?" is answered directly.

2026-11-06 · open the original ↗

• Tool results go back as a user message after the full history. The history must run: user question, then the assistant's tool-use message (added with add_assistant_message), then a user message holding the tool-result parts from run_tools.
• The final call must include the original tool schemas. Without them, Claude would not understand the tool references in the history or what get_current_datetime does.
• The tool supplies information Claude doesn't have, such as the exact time. With the result in the history, Claude answered "The current date and time is 2025-04-03, 12:54:00." The lesson notes that Claude knows the date but not the current time.

2026-11-06 · open the original ↗

• Assume a response can contain several tool requests. An assistant message can be a text part followed by one or more toolUse parts. run_tools filters the parts for toolUse and handles each one.
• run_tool sends each call to its function, unpacking the arguments. Claude always sends arguments as a dictionary, so the function is called with **tool_input. An unknown tool name raises an exception.
• Each result must carry the matching tool use ID. A result part has toolUseId, content (the output serialized as a string), and status set to "success" or "error". The IDs tell parallel calls apart: two calculator calls with IDs ab3 (10 + 10) and po9 (30 + 30) can come back in any order.
• Return errors to Claude instead of letting them crash the loop. Running the tool inside try/except and returning Error: … with status "error" lets Claude change its approach. The lesson presents this as Claude handling tool errors intelligently.

2026-11-06 · open the original ↗

• toolChoice controls whether, and which, tool Claude must use. auto (the default) lets Claude decide, any requires some tool, and a named tool forces that tool. The lesson notes the named option is especially useful for testing.
• A tool-use reply has a text part and a toolUse part. The toolUse part has toolUseId, which the result must echo back, the tool name from your schema, and an input dictionary of arguments. In the example it calls get_current_datetime with date_format %H:%M:%S.
• The message helpers must accept both strings and lists of parts. add_user_message and add_assistant_message wrap a string as a text part and pass a list through unchanged. chat now returns the text and the full parts list, because each next request needs the whole history, including the tool-use message.
• A stopReason of "tool_use" means Claude wants a tool. Because chat doesn't return this field yet, the lesson detects tool use for now by looking for a part with a toolUse key. A later lesson adds the stop reason to what chat returns.

2026-11-06 · open the original ↗

• A tool spec has two parts: when to use it, and its arguments. The name and description at the top help Claude decide when to call the tool. The inputSchema below describes the arguments in standard JSON Schema, an existing data-validation format not invented for AI.
• Generate the schema from sample arguments rather than writing it by hand. Write a dictionary of the function's arguments with sample values, convert it to JSON (Python True becomes true), paste it into an online JSON-to-JSON-Schema converter, and remove the $schema line it adds.
• The descriptions are the most important part of the schema. The tool description should be 3–4 sentences on what it does, when to use it and what it returns, and every property needs a detailed description. The get_weather example lists its return fields and valid location formats such as "London,UK", and Claude itself can draft the descriptions.

2026-11-06 · open the original ↗

• Reminders need custom tools for time, date math and scheduling. The lesson says Claude may know the date but not the exact time, handles date addition poorly when looking many days ahead, and has no way to set a reminder. So three tools are built: get the current date and time, add a duration to a date, and set a reminder.
• Tool functions should validate inputs and return meaningful errors. They are plain Python functions with descriptive argument names. Claude will retry a call after an error, so errors should say what went wrong. The first example is get_current_datetime(date_format="%Y-%m-%d %H:%M:%S").
• A detailed JSON Schema tells the model when and how to call. The description should say what the tool does, when to use it and what it returns, in about 3–4 sentences, and each parameter needs a detailed description. The lesson notes that JSON Schema is a general data-validation standard and that online tools can generate schemas.

2026-11-06 · open the original ↗

• Tools give the model access to live, external information. By default Claude knows only its training data, so "What's the weather in San Francisco, California?" gets an apology. Tools are the bridge to outside data sources.
• Tool use is a four-message exchange between server and model. You send the question along with instructions for getting extra data, and Claude asks for the specific data it needs. Your server runs code to fetch it (for example, a weather API call) and returns it, and Claude answers using it.
• The code is written in a different order than it runs. Implementation jumps between the tool function, its JSON schema, the ToolUse and ToolResult parts, and adding the schema to the request. The lesson says this is why tool use seems complex at first.

2026-11-06 · open the original ↗

• Sample input/output pairs teach what instructions alone miss, such as sarcasm. A tweet calling a film the best since "Plan 9 from Outer Space" reads as positive but is sarcastic. One-shot or multi-shot examples in <sample_input>/<ideal_output> tags, introduced explicitly, show that the correct label is "Negative".
• Examples suit corner cases, complex formats and defining "good". For tricky cases the prompt can add a warning with more examples, such as "Be especially careful with tweets that contain sarcasm", followed by a flight-delay tweet labeled Negative. Examples also pin down complex output formats such as specific JSON structures.
• Mine your evaluation report's top-scoring outputs for examples. Copy the input and output of a response that scored 10, or your highest score. Then add a sentence saying why it is ideal, so Claude learns the reasons behind the format as well as the format itself.

2026-11-06 · open the original ↗

• XML tags separate instructions from interpolated data. In a large input such as 20 pages of sales records, Claude can confuse the data with the instructions, and wrapping it in <sales_records> draws the boundary. Tag names are free-form, and specific names generally work better than generic ones.
• Tags distinguish multiple content types within one prompt. A debugging prompt separates <my_code> from <docs>, so Claude knows which is the code to fix and which is reference material. Even small inputs benefit, such as athlete details grouped in an <athlete_information> block.
• Expect modest gains on simple prompts. The lesson says tags help most with large context, several distinct content types, or interpolated content that could be mistaken for instructions. Simple prompts may not improve dramatically.