Hot AI News

Change an AI model’s temperature or prompt live, with automatic rollback

AWS AppConfig now stores model settings and prompts as versioned configuration, so teams can change them in production without redeploying and roll back on alarms.

Change an AI model’s temperature or prompt live, with automatic rollback
Image: Amazon

Amazon Web Services added two profile types to AWS AppConfig on November 19: one for AI model invocation settings and one for prompts. Teams can change both while an application runs, with gradual rollout and automatic rollback.

Two new profile types for model settings and prompts

According to AWS, an AI model invocation configuration bundles everything an application needs to call a model: which model to use, generation controls such as temperature and a cap on output length, and seed messages like a system prompt or sample dialogue turns. Options that only one vendor supports go into custom parameters, which AWS says lets a single schema serve several AI providers.

Change an AI model's temperature or prompt live, with automatic rollback
Image: Amazon

An AI prompt configuration is a separate item. Prompts are stored as their own versioned artifacts, so one can be tuned without touching the model configuration and reused across several applications.

Both go through AppConfig’s existing deployment machinery. AWS gives the example of exposing a change to 10% of traffic, then 20%, then 30%, while tracking the effect on system behavior, user experience and token consumption. If a monitoring alarm, whether from CloudWatch, Datadog, New Relic or another tool, trips during the rollout, AppConfig can return automatically to the prior state.

The profile types can be used in every AWS Region that offers AppConfig, and AWS does not charge extra for them, according to the announcement.

Advertisements

What the documentation adds about schema and retrieval

AWS’s user guide fills in details the announcement leaves out. The model profile type is AWS.AppConfig.AIModelInvocation, stored in AppConfig’s hosted configuration store and checked against a schema when it is created and again at deployment. Required fields are schemaVersion and model; inferenceParameters, customParameters and messages are optional. The inference parameters mirror the ones in Amazon Bedrock’s Converse API: temperature, topP, maxTokens and stopSequences.

The prompt type, AWS.AppConfig.AIPrompt, is plain text. The guide says any text format works, including Markdown, and it documents no placeholder or variable syntax, so the content is opaque to AppConfig.

AppConfig does not call the model. The guide states that it stores and delivers configuration data but does not call the AI provider for you. The application fetches the configuration from the AppConfig Agent on a local endpoint, then maps the fields onto the provider’s own request format. The guide’s example uses Bedrock’s Converse call. AppConfig checks that customParameters is valid JSON but not what is inside it, so a wrong provider-specific option would still pass validation.

Why rollout speed and bake time matter for prompts

This is the technical significance: a prompt change behaves like a code change but rarely gets treated like one. A reworded system prompt or a higher token cap can shift answer quality and spending at once, and neither shows up in a unit test. AppConfig’s existing deployment strategies were built for exactly that kind of slow-burn risk.

Its predefined AppConfig.Linear20PercentEvery6Minutes strategy, which AWS labels as recommended, reaches 20% of targets every six minutes over 30 minutes and then watches CloudWatch alarms for another 30. The “bake time” is the period after full rollout when alarms can still trigger a rollback. That matters because token-cost problems and quality regressions often surface only after traffic has built up. Rollback needs AppConfig to be permitted to read the alarms, per the guide.

Separating the prompt from the model settings also changes who can work on what. A prompt engineer can ship a new version without a model-configuration change, and a platform team can switch model parameters without touching prompts. Rollback of a prompt returns to the previous prompt version rather than the whole bundle, according to the guide.

Advertisements

The free label has limits, and some details are missing

“No additional cost” refers to the new profile types. AppConfig’s regular pay-as-you-go pricing still governs retrieval. According to the AWS pricing page, that is $0.0000002 per configuration request and $0.0008 per configuration received. AWS’s own example, 2,000 targets polling every two minutes with three updates a day, comes to $152.64 a month. AWS recommends the AppConfig Agent to keep such costs down.

Several things remain unestablished. The documentation lists no supported-provider list or quotas for the new types, and no variable substitution for prompts. The claim that the feature helps teams avoid unexpected behavior and token-cost surprises is AWS’s own and has not been independently tested. How the new types compare with prompt-management features inside Bedrock is not addressed in the announcement.

The release follows a run of AWS work around model operations. Earlier this week AWS added explicit prompt caching for open-weight models on Bedrock, another lever that affects both cost and latency. For teams already using AppConfig for feature flags, the change is that model and prompt settings can now sit in the same system, under the same alarms and rollout rules, rather than in code or a separate tool.

Sign Up for the Daily AI Pulse

One email a day. All the stories that matter.

Alex CarterAI News & Big Tech Correspondent

Alex Carter writes for Aiholics, keeping readers updated on the fast-paced world of AI and Big Tech.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may also like