> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ctrlplane.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Retry

> Learn how to configure automatic retry behavior for failed deployments.

**Retry rules** configure how Ctrlplane handles failed jobs. You can control the
number of retry attempts, which failure types trigger retries, and the backoff
strategy between attempts.

## Overview

```mermaid theme={null}
flowchart TD
    A[Job Runs] --> B{Success?}
    B -->|Yes| C[Complete]
    B -->|No| D{Retries Left?}
    D -->|Yes| E[Wait for Backoff]
    E --> A
    D -->|No| F[Failed]
```

## Why Use Retry Rules?

Retry rules help you:

* **Handle transient failures** - Automatically recover from temporary issues
* **Reduce manual intervention** - Let the system retry before alerting
* **Configure per-environment** - More retries in dev, fewer in production
* **Control retry behavior** - Set backoff strategies to avoid thundering herd

## Configuration

<Tabs>
  <Tab title="API">
    ```bash theme={null}
    curl -X POST https://api.ctrlplane.com/v1/workspaces/{workspaceId}/policies \
      -H "Authorization: Bearer $TOKEN" \
      -H "Content-Type: application/json" \
      -d '{
        "name": "Retry on Failure",
        "selector": "environment.name == '\''production'\''",
        "rules": [
          {
            "retry": {
              "maxRetries": 3
            }
          }
        ]
      }'
    ```
  </Tab>
</Tabs>

<Note>
  Retry rules are not yet supported in the Terraform provider. Use the REST API
  to configure retry behavior.
</Note>

## Properties

<ParamField path="retry.maxRetries" type="integer" required>
  Maximum retry attempts. `0` means no retries (1 attempt total), `3` means up
  to 4 attempts (1 initial + 3 retries).
</ParamField>

<ParamField path="retry.retryOnStatuses" type="array">
  Job statuses that trigger a retry. Defaults to `["failure",
      "invalidIntegration", "invalidJobAgent"]` when `maxRetries > 0`. When
  `maxRetries = 0`, also includes `"successful"` to enforce deploy-once
  semantics.
</ParamField>

<ParamField path="retry.backoffSeconds" type="integer" default="0">
  Seconds to wait between retry attempts. If not set, retries are allowed
  immediately after job completion.
</ParamField>

<ParamField path="retry.backoffStrategy" type="string" default="linear">
  Backoff strategy: `linear` (constant delay) or `exponential` (doubling delay
  with each retry using `backoffSeconds * 2^(attempt-1)`).
</ParamField>

<ParamField path="retry.maxBackoffSeconds" type="integer">
  Maximum backoff cap in seconds (for exponential backoff). If not set, no
  maximum is enforced.
</ParamField>

## Job Statuses

The following job statuses can be used in `retryOnStatuses`:

| Status               | Description                     |
| -------------------- | ------------------------------- |
| `failure`            | Job failed during execution     |
| `successful`         | Job completed successfully      |
| `cancelled`          | Job was manually cancelled      |
| `skipped`            | Job was skipped                 |
| `invalidIntegration` | Integration configuration error |
| `invalidJobAgent`    | Job agent configuration error   |

<Note>
  Cancelled and skipped jobs never count toward the retry limit by default,
  allowing redeployment after manual cancellation.
</Note>

## Common Patterns

### Basic Retry

Retry failed jobs up to 3 times:

```json theme={null}
{
  "retry": {
    "maxRetries": 3
  }
}
```

### Retry with Backoff

Wait between retry attempts:

```json theme={null}
{
  "retry": {
    "maxRetries": 3,
    "backoffSeconds": 30
  }
}
```

### Exponential Backoff

Increase wait time with each retry:

```json theme={null}
{
  "retry": {
    "maxRetries": 5,
    "backoffSeconds": 10,
    "backoffStrategy": "exponential",
    "maxBackoffSeconds": 300
  }
}
```

With exponential backoff, wait times are: 10s → 20s → 40s → 80s → 160s (capped
at 300s)

### No Retries (Deploy-Once)

Disable retries for critical deployments. When `maxRetries` is `0`, the default
`retryOnStatuses` also includes `"successful"`, enforcing deploy-once semantics:

```json theme={null}
{
  "name": "No Retry Production",
  "selector": "environment.name == 'production'",
  "rules": [
    {
      "retry": { "maxRetries": 0 }
    }
  ]
}
```

### Retry Specific Statuses

Only retry on specific failure types:

```json theme={null}
{
  "retry": {
    "maxRetries": 3,
    "retryOnStatuses": ["failure", "invalidIntegration"],
    "backoffSeconds": 60
  }
}
```

### Environment-Specific Retry

Different retry behavior per environment:

```json theme={null}
[
  {
    "name": "Dev Retry",
    "selector": "environment.name == 'development'",
    "rules": [
      { "retry": { "maxRetries": 5, "backoffSeconds": 5 } }
    ]
  },
  {
    "name": "Staging Retry",
    "selector": "environment.name == 'staging'",
    "rules": [
      { "retry": { "maxRetries": 3, "backoffSeconds": 30 } }
    ]
  },
  {
    "name": "Production Retry",
    "selector": "environment.name == 'production'",
    "rules": [
      {
        "retry": {
          "maxRetries": 2,
          "backoffSeconds": 60,
          "backoffStrategy": "exponential"
        }
      }
    ]
  }
]
```

## Backoff Strategies

### Linear Backoff

Constant wait time between retries:

```
Attempt 1: immediate
Attempt 2: wait 30s
Attempt 3: wait 30s
Attempt 4: wait 30s
```

### Exponential Backoff

Doubling wait time with each retry:

```
Attempt 1: immediate
Attempt 2: wait 10s  (10 * 2^0)
Attempt 3: wait 20s  (10 * 2^1)
Attempt 4: wait 40s  (10 * 2^2)
Attempt 5: wait 80s  (10 * 2^3)
```

Use `maxBackoffSeconds` to cap the maximum wait time.

## Retry Lifecycle

### 1. Job Fails

A job completes with a status in `retryOnStatuses`.

### 2. Retry Check

Ctrlplane checks if retries remain (`attempt < maxRetries + 1`).

### 3. Backoff Wait

If `backoffSeconds` is configured, Ctrlplane waits before the next attempt. The
`nextEvaluationTime` is set to indicate when the retry will be allowed.

### 4. Retry Attempt

A new job is created for the retry attempt.

### 5. Success or Exhausted

The process continues until success or all retries are exhausted.

## Best Practices

### Retry Guidelines

| Scenario             | Max Retries | Backoff | Strategy    |
| -------------------- | ----------- | ------- | ----------- |
| Transient network    | 3-5         | 10-30s  | exponential |
| Rate limiting        | 3           | 60s     | exponential |
| Resource contention  | 2-3         | 30s     | linear      |
| Critical production  | 1-2         | 60s     | linear      |
| Flaky tests (dev/qa) | 5           | 5s      | linear      |

### Recommendations

* ✅ Use exponential backoff for external service failures
* ✅ Set `maxBackoffSeconds` to avoid excessive wait times
* ✅ Use fewer retries in production than in development
* ✅ Monitor retry rates to identify systemic issues
* ✅ Combine with alerting on final failure

### Anti-Patterns

* ❌ Infinite retries (always set `maxRetries`)
* ❌ No backoff for rate-limited APIs
* ❌ Same retry config across all environments
* ❌ Retrying on non-transient failures

## Next Steps

* [Policies Overview](./overview) - Learn about policy structure
* [Environment Progression](./environment-progression) - Control promotion flow
* [Version Cooldown](./version-cooldown) - Batch frequent releases