Improve prompts and test changes

Three tools, three scopes

Prompt Studio takes a task description, a baseline prompt, and a target model. It produces rewritten messages for you to review and copy into your application. Running Studio does not deploy those messages or change workspace defaults.

Experiments compares system prompts, models, or technique settings. A variant applies to test requests or application requests sent with that experiment's header. Creating an experiment does not enroll existing application traffic.

Techniques describes optional request processing and lets authorized users change workspace defaults for eligible requests. A workspace default can affect requests beyond a single experiment, so enabling it is a separate decision.

Use the overview

Open Build → Optimization in the sidebar. Select Overview, Prompt Studio, Experiments, or Techniques from its children. The parent label opens Overview; its chevron expands the menu. In the collapsed rail, the Optimization icon opens the destination menu.

Overview shows the latest three saved Studio runs, the total number of running experiments and their three newest entries, and enabled techniques among available implementations. In-progress Studio jobs remain managed in Studio. Overview is read-only: Refresh reloads activity, and each panel shows its update time. An unavailable or failed panel is not a count of zero; retry that panel or open its tool.

Set up a run

1. Describe the task, the intended audience, the required output, and important constraints. For example: Generate five distinct search keywords for a local healthcare clinic; include the service and location and avoid unsupported claims.
2. Provide a system prompt, a user prompt, or both. The system prompt contains the model's role and shared instructions. The user prompt contains the particular task or question. Keep representative constraints in your baseline so the optimizer has them to work with.
3. Use double-brace placeholders such as {{service}} and {{location}} for values supplied by your application. These placeholders survive optimization. You replace them with real values when using the result.
4. Choose the Target task model on which you intend to use the rewritten prompt. Studio identifies the separate model it uses to rewrite and refine candidates; that optimizer model is not your deployment target.
5. Set Candidate evaluations, from 4 to 100. This is the number of scored candidate evaluations, not a dollar limit. More evaluations can explore further and take longer or cost more. Begin with a smaller run while refining your task description.
6. Select Bill to API key and optionally enter a Run name to recognize it in History. Start the run and watch its progress in Studio.

Billing and availability

Optimization calls are billed to the selected API key and share its budget and request limits. Candidate evaluations does not cap spend in dollars. Check the selected key and available wallet balance before running. If no key is listed, create an active inference-capable key under API keys and return to Studio.

If Studio is unavailable, follow its availability message or contact support. A failed run is not a usable result. Read its error, check key access and balance where relevant, and retry after resolving the issue. Completed inference calls may still have incurred costs even if the overall optimization fails.

Review and use the messages

1. Read the rewritten system and user messages. Check instructions, output format, factual constraints, and the listed variables against your original task.
2. Treat initial, final, and improvement scores as evaluation results from this run. Validate the output on representative examples, including edge cases, before wider use. Do not read an improvement percentage as a reduction in your API bill.
3. Copy an individual message or the result's JSON payload. Replace placeholders with actual values, keep the target model intentional, and integrate the messages into your application's request construction. Copying alone does not deploy anything.
4. Use History to reopen recent saved results. Studio also tracks in-progress jobs there, while the overview shows saved runs only. Use Refresh to reload saved history. An empty history means no saved runs were returned; a history error means the read failed.

Test techniques with this prompt

Select Test techniques with this prompt to open a technique experiment setup. The handoff seeds the same optimized system prompt and target model into both Control and Treatment. It places the optimized user prompt in the experiment's test-request box after creation. Choose the technique differences before creating the experiment.

This handoff tests technique settings against a shared prompt. It does not create an original-versus-rewritten prompt comparison, and it does not create per-variant user messages. Replace any placeholders in the test request before sending it. Browse techniques opens the catalog if you need to understand the available settings first.

Create an experiment

1. Select New experiment. Name the comparison and use the description to record your hypothesis and what would count as success. The generated slug identifies the experiment in application headers.
2. Choose the comparison dimension: system prompt, model, technique settings, or a mixed comparison. For an interpretable first experiment, change one thing between variants.
3. Choose Sticky or Random assignment. Sticky uses X-CI-End-User to consistently assign an end user. Send a stable identifier for each user. Without it, assignment falls back to the account identity, so calls without user identifiers can all land on the same variant. Random makes a weighted choice for each request.
4. Define Control as your baseline and name each Treatment after its change. Set relative traffic weights: equal weights split traffic evenly in expectation; weights of 3 and 1 target a 75% and 25% split. Small samples can differ from the intended proportions.
5. For a system-prompt comparison, enter the variant instructions. A nonempty override replaces the first system message, or adds one if none exists. Leaving it empty inherits the incoming request's system prompt. The user message continues to come from the request.
6. For a model comparison, pin the desired model per variant or inherit the incoming request's model. Check that each chosen model supports the request features you plan to send.
7. Choose technique overrides and create the experiment. Running means ready to receive experiment traffic. Nothing is automatically added to your application requests.

Understand inherited technique settings

Inherit leaves the effective setting unchanged, including workspace defaults and supported per-request opt-ins. Force on or Force off sets an explicit override for that variant. These overrides affect experiment requests and do not save workspace defaults.

For an on-versus-off comparison, explicitly force the chosen technique off in Control and on in Treatment. An inherited Control is not necessarily off if the workspace already enables that technique or requests opt into it. Keep other overrides, prompts, and models aligned to isolate the change.

Test a variant in the dashboard

1. Open the experiment detail page and find Test a variant. Choose Control or a Treatment under Send to. This targets that variant directly, so it is useful for a smoke test rather than checking traffic weights.
2. Review the request model and enter a representative user message. A variant with a pinned model overrides the request model. If you arrived from Studio, review its seeded user prompt and substitute all variables.
3. Send the request and inspect the returned variant and output. Repeat the same task for the other variant so you can compare behavior. These are live inference calls using an active API key and its wallet balance.

Connect your application

Add X-CI-Experiment with the exact slug shown on the experiment detail page. For Sticky assignment, also send X-CI-End-User with your stable end-user identifier. Requests without the experiment header follow normal request processing and workspace settings.

The example below uses Chat Completions. Replace the API key, experiment slug, model, and messages with your own values. The -i flag includes response headers so you can inspect x-ci-experiment-variant and retain x-ci-request-id for feedback. Use the detail page's integration examples for your SDK.

1curl -i https://api.cheaperinference.com/v1/chat/completions \2  -H "Authorization: Bearer ci_live_YOUR_API_KEY" \3  -H "Content-Type: application/json" \4  -H "X-CI-Experiment: YOUR_EXPERIMENT_SLUG" \5  -H "X-CI-End-User: customer-42" \6  -d '{7    "model": "YOUR_MODEL_ID",8    "messages": [9      {"role": "system", "content": "Answer clearly and accurately."},10      {"role": "user", "content": "Suggest five search keywords for a dental clinic in Austin."}11    ]12  }'

Collect useful evidence

Review request counts, input and output tokens, and average cost for each variant. Use comparable tasks and enough representative requests to judge your workload. A single low-cost answer is not proof that the configuration will work for other inputs. Check output completeness and correctness, especially after context trimming, output caps, or model changes.

Submit a score from 1 to 5 and an optional comment for a completed request using its x-ci-request-id. Feedback is supplied by you or your application; experiment quality is not automatically scored. Use the same scoring rubric across variants. The detail page reports the average score and rating count, so consider how many requests were actually rated.

1curl https://api.cheaperinference.com/v1/feedback \2  -H "Authorization: Bearer ci_live_YOUR_API_KEY" \3  -H "Content-Type: application/json" \4  -d '{5    "request_id": "REQUEST_ID_FROM_RESPONSE",6    "score": 4,7    "comment": "Relevant keywords and correct format; one suggestion was too broad."8  }'

Pause, finish, or promote

Pause stops ordinary variant assignment; it does not stop API requests. Those requests continue through normal processing. Resume makes the experiment ready to assign variants again. End without a winner completes the comparison without applying a winning variant. A completed experiment cannot be restarted.

Promote a winner only after reviewing your evidence. The winning variant's prompt, model, and technique settings then apply to future requests carrying that experiment's header. Promotion does not change workspace defaults or requests that do not carry the header. Inherited fields continue to inherit rather than becoming a saved snapshot of every request setting.

To use the result more broadly, make a separate deployment decision: update your application's prompt or model, or have an authorized workspace owner or admin change defaults under Techniques. Review and monitor that wider change independently.

Review before enabling

1. Open Available techniques. Expand an item and read its mechanism, savings summary, quality risk, linked evidence, and implementation. An available technique can still do nothing for an ineligible request or an input that has nothing to change.
2. Use Help me choose if you want guidance. Safest defaults describes the current safe-compression preset, Save more points to context and output controls, and Run an experiment helps you explore changes to behavior. Read the proposed action before selecting it. Opening the help or choosing a guidance card does not itself enable the switches.
3. Test changes on representative inputs. The technique-experiment launcher opens experiment setup so you can keep a baseline Control and select Treatment overrides. It does not turn on workspace defaults or automatically choose a difference for every technique.
4. When ready for wider use, change the individual workspace switch, or explicitly apply the safe-compression preset. Personal workspaces can manage their own defaults. Organization defaults require an owner or admin; members can inspect the catalog and its permission guidance.
5. Send eligible requests and review their reported technique attribution and actual usage. Turn a default off if the tradeoff is unsuitable. Explicit experiment overrides and supported per-request opt-ins have their own scope, so also check those when investigating why a technique still applies.

Choose the appropriate scope

A switch on Techniques changes the shared workspace default for eligible future requests. A variant override changes requests assigned to that experiment variant. A Studio result changes neither. Keep these three scopes separate when interpreting activity or deciding how to deploy a result.

The enabled count includes only available implementations with an enabled effective workspace default. It does not count research items or imply that every request used every enabled technique. If either the registry or settings read fails, retry the read before interpreting the state; an error is not an empty registry.

Formatting and context

JSON Minification removes unnecessary formatting from supported JSON content. It is the technique currently included in the safe-compression preset. Validate any workflow that depends on exact text formatting even when the JSON values are preserved.

Whitespace Normalization reduces redundant spacing and blank lines. Whitespace-sensitive code, data, or Markdown can behave differently, so this technique is not in the safe-compression preset.

Duplicate Message Collapse removes eligible repeated messages. Test cases in which repetition expresses emphasis or conversational intent. Context Window Trim removes older eligible context to meet its context budget; test long conversations where an answer depends on earlier details.

Output and model behavior

Output Token Budget adds an output cap when the request has no existing positive max_tokens or max_completion_tokens cap. A cap can limit runaway generation but may also truncate a valid answer. Check whether the output is complete, not just cheaper.

Concise Output Instruction adds guidance for shorter answers. It can change answer style or omit desired explanation. Simple-Query Model Routing can route eligible simple requests to a configured cheaper model. Verify quality and the resulting model on your tasks.

Concise output and routing also support per-request opt-ins through X-CI-Concise: 1 and X-CI-Route: auto. An explicit experiment override takes precedence over these effective inherited settings. Leave these headers out of a baseline request unless they are part of the behavior you intend to compare.

Test cache behavior

Read the current eligibility details on the technique card. Enable caching at the intended scope, send an eligible request to populate the cache, then repeat the same request and inspect its attribution and billed usage. Changing request content or sending an ineligible request may require another provider call.

Use a realistic mix of new and repeated requests when evaluating savings. A warm repeated request does not represent every request in your workload. Disable the workspace default if reuse is unsuitable, and check experiment overrides when testing its off state.

Resolve common points of confusion

Studio completed but the application is unchanged: review and copy the result into your application, or use its technique-experiment handoff. Completion does not deploy a prompt.

Experiment is Running but has zero requests: send a dashboard test request or add its exact X-CI-Experiment slug to application requests. Check the response's variant attribution. Running only indicates readiness.

Sticky traffic appears to choose only one variant: confirm that different end users send different stable X-CI-End-User values. With no identifier, calls use the account fallback. A small sample can also be uneven.

A winner was promoted but workspace switches did not change: this is expected. Promotion affects future requests carrying that experiment's header. Changing shared defaults requires a separate action on Techniques.

An enabled technique reports no savings: inspect its request eligibility, the incoming content, any explicit variant overrides, and the kind of savings it provides. Output changes and model routing may affect downstream cost without reporting removed input tokens.

A workspace switch is unavailable: read the permission guidance and confirm that the technique is available, rather than research only. An organization owner or admin must manage shared defaults. For a failed activity or registry read, retry instead of treating the missing data as zero activity.