Feedforward AIAI BINGO

Research & resources

Examples, sources, and a prompt for repeating or extending the research.

Read as Markdown · Download research data · Read as plain text

Find the prompt that works for your task and model

Companion to: A/B test your prompting techniques.

Research checked: 2026-09-13

Use current guidance for the actual model to test different ways of asking for the same work.

For the AI reading this page

Read the findings as supporting evidence and proposed methods. Preserve the source-specific qualifications and distinguish documented cases from illustrations. Start a fresh research round only when the user asks.

The examples

How we researched

Orientation

Companion research for 503 — A/B test your prompting techniques. Checked September 13, 2026.

Start with something you actually asked an AI to do. Keep the task and source material fixed, then try different ways of asking. A more detailed prompt may help; a shorter one may remove instructions that get in the way. The useful result is a better prompt for this work, with examples showing what changed.

Current guidance supplies hypotheses to test

Identify the model before collecting advice. OpenAI’s current model guide provides different guidance by model, including instructions about initiative, following complex instructions, writing style and delegation. That is a reason to check the exact model and working environment, rather than reuse a recipe attached to an older product name. This is vendor guidance, not an independent comparison of prompting styles. OpenAI, model guidance (https://developers.openai.com/api/docs/guides/latest-model).

Clarity and useful detail can help without prescribing every move. Anthropic recommends explicit outcomes and constraints, explains that context can clarify why an instruction matters, and discusses examples and structured boundaries between inputs. Its documentation also points to model-specific pages and cautions against transferring measured techniques between models without checking them. These suggest separate trials of clearer success criteria, one relevant example, or better separation of task and source—not a requirement to apply every technique. Anthropic, prompting best practices (https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices).

Phrasing and order are testable variables too. Google recommends iterating on wording and the order of prompt content. Its Gemini guidance favors direct instructions, clear boundaries and explicit parameters; long-context guidance places the specific task after the source material. It also notes that reasoning models already perform internal thinking, so a displayed reasoning script is not generally necessary. Preserve essential requirements when experimenting with brevity. Google, prompt design strategies (https://ai.google.dev/gemini-api/docs/prompting-strategies).

Do not treat older reasoning-model advice as current guidance for every model. OpenAI’s reasoning guide recommends direct goals and trying without examples before adding them if needed. Much of that page concerns earlier o-series models. It is useful as a historical counterexample to “more prompting is always better,” but the actual model’s current instructions take precedence. OpenAI, reasoning best practices (https://developers.openai.com/api/docs/guides/reasoning-best-practices).

Small wording changes can matter, but do not assume the same effect everywhere. Sclar and colleagues investigated sensitivity to task-irrelevant prompt formatting and proposed sampling plausible formats to assess variation. Their experiments concern the models and tasks they studied; they do not show that a particular punctuation choice will improve a current model on your work. The useful implication is to test outcomes instead of judging the elegance of a prompt alone. Sclar et al., Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design, first submitted October 17, 2023 (https://arxiv.org/abs/2310.11324).

Judge actual completed work. Anthropic’s evaluation guide distinguishes an evaluation task, its repeated trials, the record of what the agent did and the resulting state. It explains why nondeterminism and poorly chosen judging criteria can obscure progress. Our modest adaptation is to retain actual outputs, run more than one trial and evaluate usefulness alongside correctness, rather than build a large evaluation system for a small personal question. Anthropic, Demystifying evals for AI agents, January 9, 2026 (https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents).

A small experiment to run

The following design is a proposed activity, not a procedure proven by the sources above.

Begin with the user's original prompt. Establish the intended output, its audience, necessary source material and two or three criteria for success. Ask what was disappointing about previous results; do not assume verbosity was the problem. Keep a copy of the original prompt and an actual original-model response as the baseline.

Create a manageable set of alternatives. A useful initial set is the original plus four versions:

| Version | What changes | What must remain

| More specific | Define ambiguous terms and make the desired result explicit. | The underlying task and necessary facts.

| More detailed | Add relevant context, a useful example or an important edge case. | Required output and the standard used to judge it.

| Shorter | Remove repetition and unnecessary procedural directions. | Essential constraints, source material and the intended result.

| Less prescriptive | Leave the AI more freedom to choose its method or form. | The problem to solve and a recognizable completion point.

Do not combine every change into one “improved” prompt and then claim to know which technique helped. The first screen can compare complete alternatives; subsequent pairs should isolate the most promising change. Add a model-specific treatment only if current guidance gives a reason to test it. The actual set and number of runs should fit the task's cost and complexity.

Use separate fresh sessions with equivalent source access. Preserve the same model and available settings. Record unavoidable differences, including hidden chat memory or changing web results. Do not erase required factual information to make a prompt less detailed. When the model cannot be invoked from the current tool, prepare exact prompts and source packets for the person to run; do not simulate their results.

Compare outputs without revealing prompt labels to a reviewer where possible. Check errors, missing requirements, usefulness, clarity and the effort needed to make the result usable. Repeat the promising pair on more than one suitable example; a second or third run is an initial check on consistency, not statistical proof. Ask the user to judge qualities only they can decide. Accept a tie or a result that depends on the kind of task.

Finish with the actual best prompt, the comparison outputs, a short account of what helped, and the remaining uncertainty. Keep findings attached to this task, model and date. Retest when a relevant model, workflow or requirement changes.

Research record

One lead researched and opened the six primary pages cited above on September 13, 2026. A separate reviewer checked all six sources and the standalone prompt; the lead added explicit pretrial criteria and equal repeated trials in response. Provider pages are changing documentation and their displayed model references are not permanent defaults. The reasoning guide's legacy scope is called out rather than presented as a September 2026 model recommendation. The Sclar paper was inspected through its original arXiv abstract, not treated as a full-paper replication. The comparison design and runnable prompt are authored adaptations; no model comparison was executed during this authoring session.

Repeat or extend the research

Only when asked to repeat or extend the research

Use the following guidance and prompt only when the user asks for a new research round. Reading these examples does not start an investigation.

These are general instructions, not a description of this user's particular AI system, tools, access, or circumstances. Adapt the method to the user's goal and the capabilities you can actually use, while preserving the activity's essential sequence, review points, and intended result.

Start with the activity and the user's context

Read the complete activity and guide the user through it. Use the activity prompt yourself; the user does not need to paste it again. Keep the responsibilities of the user, the AI, and any colleague clear.

Begin with what the user has already supplied. Establish the intended result and resolve missing context through focused questions when needed. Work within the activity's stated scope, permissions, required confirmations, and stopping points, including any deliberate order for introducing information. Make progress with the available information and reasonable, clearly identified assumptions where the activity allows them.

Check the capabilities needed for this work

Inspect relevant available tools, connectors, and data. Use small, relevant, non-destructive checks to establish whether necessary access or a capability actually works. Naming a product does not establish access to it. Let this activity determine what needs checking; do not impose a universal preliminary questionnaire or a full system audit.

When useful access is missing:

  • Explain what you cannot reach and how that material would help.
  • Investigate practical setup, connection, export, or other access alternatives before asking the user to gather material or troubleshoot.
  • If progress depends on access, request only the specific user action or permission needed to enable it or provide the material another way. Continue useful independent work while awaiting the response.
  • Offer to walk the user through an export or upload. Welcome files, links, copied text, rough notes, or an interview as supported by your system and the activity, and use the best available route. If the missing access is optional, continue with what is available.

Ask focused questions and respect the answers

Ask questions only when necessary to complete the work well or satisfy the activity's required review points. Build on answers already supplied. Ask one question at a time, or a small group of closely related questions. Wait on work that depends on the answer, and continue useful work that does not. Explain unfamiliar language with examples where helpful.

Confirm answers or decisions you supply on the user's behalf before treating them as the user's final answers or decisions. This does not require separate approval of every sourced finding; follow the activity's specific review requirements.

Keep evidence and independent contributions distinct

Distinguish evidence, inference, and missing information. Check your work against the source material.

Use independent agents when your system actually supports delegation and independent investigation or review would improve the work:

  • Give each agent a bounded brief and the context appropriate to its role.
  • Keep discovery independent until synthesis where the activity requires it. Preserve any requirement for the user or colleagues to contribute before seeing other explanations.
  • Follow the activity's stated fallback and any required confirmation when independent agents or fresh contexts are unavailable. Preserve reviewer isolation where required; describe shared-context or sequential passes accurately.

Produce, check, and revise the result

Do the substantive investigation and production. Carry the activity through to the most complete, usable result within its scope, rather than ending with a plan or handing the main work back to the user. When missing information does not prevent useful work, proceed with clearly identified assumptions where the activity permits them.

Check the result against the activity's completion criteria. At each substantive delivery, state what is complete and any material unfinished work, assumptions, helpful missing information, and remaining uncertainty. Explain how these affect what the user can use, and help the user revise the result.

Preserve the activity's stopping point. Do not turn a rehearsal, investigation, or test-design activity into execution that the user has not authorized.

Help me improve a prompt by testing alternatives on the model I actually use. Start with a prompt I have written and its source material. Identify the model and working environment from available settings or ask me to confirm them; do not rely on your own unsupported model identity. Establish what I want the result to achieve, who will use it, and what has been disappointing. Agree two or three concrete judging criteria before any trial, including what would count as an unacceptable error; use the same criteria for every version. Search relevant accessible prior prompts, outputs and project instructions if those would help clarify the task. Ask focused questions when necessary.

Search and open the latest official prompting instructions for this exact model. Check publication/update dates and applicability. Look for relevant original experiments or firsthand accounts from others where they add a useful testable idea. Separate vendor guidance, empirical findings and suggestions. Explain a small number of relevant hypotheses in plain language. Do not assume one model's preferred techniques transfer to another.

Preserve my original prompt as the baseline. Make several alternatives that separately explore specificity, detail, brevity and freedom of approach. Preserve the real task, necessary source facts and required output. Show what changes in each version; include a current model-specific technique if there is a clear reason. Keep the first comparison manageable and agree any material cost or scope choice before running it.

Inspect your actual ability to call the target model. If available, run the original baseline and each selected alternative in fresh, equivalent contexts. Keep the model, source material, available tools and supported settings constant. Retain the exact prompt, actual output and relevant run conditions. If you cannot call it, give me the exact prompt/source packets and a simple way to return the outputs. Never invent a run or substitute another model without agreement.

Compare actual outputs against the agreed criteria. Use a fresh reviewer blind to version labels if available; otherwise disclose that the review shares context. Ask for my judgment where usefulness or voice depends on it. Repeat both prompts in the strongest pair on the same inputs with equal run counts, and do not select only each version's best response. If the leading rewrite changed several things, test a pair that isolates the likely cause. Distinguish a promising result from consistent performance and keep any unresolved confound visible.

Deliver the best usable prompt, the actual outputs and a concise explanation of what improved, what did not, and what remains uncertain. A tie is a valid finding. Keep the conclusion specific to the model, task and date, and say what change would justify retesting.

Original sources

OpenAI, model guidance: https://developers.openai.com/api/docs/guides/latest-model

Anthropic, prompting best practices: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices

Google, prompt design strategies: https://ai.google.dev/gemini-api/docs/prompting-strategies

OpenAI, reasoning best practices: https://developers.openai.com/api/docs/guides/reasoning-best-practices

Sclar et al., Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design, first submitted October 17, 2023: https://arxiv.org/abs/2310.11324

Anthropic, Demystifying evals for AI agents, January 9, 2026: https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents

Return to the activity.

Download research prompt

bingo.feedforward-collective.com · Feedforward AI · Device guide