Update Models and Prompts Without Code Changes

Why Use Deployments?

At heart, deployments are simply a way to manage some completion parameters (usually prompt and model) that would traditionally be committed and deployed with the codebase. With the speed at which LLMs are evolving, re-deploying code any time a prompt needs to be adjusted or a model needs to be changed prevents non-engineers from making updates and slows down iteration cycles.

There are three parts to the deployment process:

1. Initial Deployment Creation (requires engineering): Set up your code to use deployments instead of hardcoded parameters and deploy a first version of your agent.
2. Non-breaking Deployment Updates (Ongoing) (no code changes needed): Non-breaking changes to the agent's prompts, models, and parameters can be made through updating deployments, no code changes required.
3. Breaking Deployment Updates (Possible, periodic) (requires engineering): If the changes made are considered breaking changes, a new deployment will need to be created and your code will need to be updated to point to the new deployment.

Initial Deployment Setup (Requires Engineering)

This initial setup requires an AI coding agent with access to your codebase to modify your agent code.

Deploy the version of anotherai/agent/travel-assistant in my code to production. 
Create a deployment for this version and update to my code to match this version 
and reference the deployment_id.

Deploy anotherai/version/acf2635be31cbd89f9363bfd3b2c6abc to production. 
Create a deployment for this version and update to my code to match this version 
and reference the deployment_id.

Deploy anotherai/version/acf2635be31cbd89f9363bfd3b2c6abc to production. 
Create a deployment for this version and update to my code to match this version 
and reference the deployment_id.

When a new deployment is created, it is given a unique deployment_id that can be used to reference the deployment in your code. Optionally, you can request a specific deployment_id be set when creating the deployment. Otherwise, your AI coding agent will pick one for you automatically.

Non-breaking Deployment Updates (Ongoing)

Once your code is set up to use deployments, in many cases you can update your agent's behavior without any engineering involvement or changes to your code.

Benefits of Deployment Updates

Updating an existing deployment does not require any code changes. Because no code changes are required, updating an existing deployment is generally much faster than creating and releasing a new deployment.

What updates can be made to an existing deployment?

You can update an existing deployment if the new version is considered a non-breaking change.

Non-breaking Changes Examples

• Changing the model
• Adjusting temperature or other generation parameters
• Editing prompt wording while keeping the same variables

How to update an existing deployment

When your changes don't affect the input variables or output schema, you can update the existing deployment:

Update deployment anotherai/deployment/question-answering-agent:production#1 to use
anotherai/version/a9f1fc5ab11299a9fee5604e51fe7b6e 

Breaking Deployment Updates (Possible, periodic)

In some cases, the difference between two versions of your agent will require a new deployment to be created instead of simply updating the version connected to an existing deployment_id. In these cases, engineering involvement is required to update the code to point to the new deployment.

When do I have to create a new deployment instead of updating an existing one?

Creating a new deployment is required when the changes you are making are considered breaking changes.

Breaking Changes Examples

• Editing the input variables
  • Adding a new variable
  • Removing a variable
  • Changing the name or the type of an existing variable
• Editing the output schema
  • Adding a new field
  • Removing a field
  • Changing the name or the type of an existing field

Creating a new deployment_id for an already-deployed agent is the same process as creating that initial deployment. You can refer to the initial deployment setup section for the process.