{
  "name": "Throttle AI API calls to a rate limit with Split In Batches and Wait",
  "nodes": [
    {
      "parameters": {
        "content": "## Throttle AI API calls to stay under a rate limit\n\nPaces a run to your provider's requests-per-minute ceiling instead of firing everything at once and collecting 429s.\n\n### Who's it for\nAnyone on a tier with a real rate limit who has watched a bulk run fail halfway through. Cheaper and calmer than retrying into the same wall.\n\n### How it works\n- **One call at a time** walks the workload one item per iteration, so nothing runs in parallel by accident.\n- **Pace the call** is a token bucket. It works out how long this call has to wait to stay inside your limit, and returns that delay.\n- **Hold for the limit** sleeps for exactly that long. When you are under the limit the delay is zero and nothing is wasted.\n- **Rate limited?** catches a genuine 429 that got through anyway and sends it to **Queue for retry** with its reason, rather than dropping it.\n- **Summarise the run** reports what succeeded and what is still owed.\n\n### Setup\n1. Replace **Sample workload** with your real input.\n2. Open **Pace the call** and set the requests-per-minute your plan actually allows. Set it slightly below the published number, because the provider counts differently than you do.\n3. Open **Call the model** and attach your credential under Authentication, Generic, Header Auth.\n\n### Requirements\nAn API credential for any rate-limited endpoint. No community nodes.\n\n### How to customize\nToken-per-minute limits bite before request-per-minute limits on long prompts. If that is your ceiling, pace on estimated tokens instead of on call count.\n\nFull walkthrough: https://www.theagentecosystem.com/blog/n8n-rate-limit-ai-workflows",
        "height": 992,
        "width": 760,
        "color": 1
      },
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        -40,
        -742
      ],
      "name": "Note: Overview"
    },
    {
      "parameters": {
        "content": "## 1. Walk the workload\nOne item per iteration, so nothing fires in parallel by accident.",
        "height": 260,
        "width": 700,
        "color": 7
      },
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        -40,
        490
      ],
      "name": "Note: Walk the workload"
    },
    {
      "parameters": {
        "content": "## 2. Pace, call, catch the 429s\nA token bucket sets the delay; under the limit it is zero. A genuine 429 is queued with its reason.",
        "height": 260,
        "width": 900,
        "color": 7
      },
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        660,
        610
      ],
      "name": "Note: Pace, call, catch the 429s"
    },
    {
      "parameters": {
        "content": "## 3. Keep or queue\nResults kept, rate-limited items held for a second pass rather than dropped.",
        "height": 500,
        "width": 240,
        "color": 7
      },
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        1560,
        490
      ],
      "name": "Note: Keep or queue"
    },
    {
      "parameters": {
        "content": "## 4. Report what is still owed\nSucceeded and outstanding, so a partial run is recoverable.",
        "height": 260,
        "width": 480,
        "color": 7
      },
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        1800,
        370
      ],
      "name": "Note: Report what is still owed"
    },
    {
      "parameters": {},
      "id": "a1b2c3d4-0000-4000-8000-000000000003",
      "name": "Run the batch",
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [
        0,
        640
      ]
    },
    {
      "parameters": {
        "mode": "raw",
        "jsonOutput": "={{ JSON.stringify([\n  { endpoint: 'https://api.openai.com/v1/chat/completions', model: 'gpt-4o-mini', prompt: 'Summarise what a token bucket is in one sentence.' },\n  { endpoint: 'https://api.openai.com/v1/chat/completions', model: 'gpt-4o-mini', prompt: 'Name one thing a fixed Wait node cannot do that a pacer can.' },\n  { endpoint: 'https://api.openai.com/v1/chat/completions', model: 'gpt-4o-mini', prompt: 'Give one reason a 429 can survive three retries.' }\n]) }}",
        "options": {}
      },
      "id": "a1b2c3d4-0000-4000-8000-000000000004",
      "name": "Sample workload",
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        220,
        640
      ]
    },
    {
      "parameters": {
        "batchSize": 1,
        "options": {}
      },
      "id": "a1b2c3d4-0000-4000-8000-000000000005",
      "name": "One call at a time",
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 3,
      "position": [
        460,
        640
      ]
    },
    {
      "parameters": {
        "jsCode": "// Token-bucket pacer.\n//\n// Holds every call to the provider's requests-per-minute limit across the whole\n// run, and remembers the last slot between runs because the counter lives in\n// workflow static data rather than in the item.\n//\n// Set RPM to what your provider actually allows. This is the only number most\n// people need to change.\nconst RPM = 20;\nconst MIN_GAP_MS = Math.ceil(60000 / RPM);\n\nconst state = $getWorkflowStaticData('global');\nconst now = Date.now();\n\n// Claim the next free slot BEFORE waiting, so two items can never claim the\n// same one. A fixed Wait node cannot do this — it has no idea when the last\n// call actually went out.\nconst slotAt = Math.max(now, state.nextSlotAt || 0);\nconst waitMs = slotAt - now;\nstate.nextSlotAt = slotAt + MIN_GAP_MS;\n\nreturn $input.all().map((item) => ({\n  json: {\n    ...item.json,\n    rpm: RPM,\n    minGapMs: MIN_GAP_MS,\n    waitSeconds: Math.round((waitMs / 1000) * 100) / 100,\n  },\n}));"
      },
      "id": "a1b2c3d4-0000-4000-8000-000000000006",
      "name": "Pace the call",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        700,
        760
      ]
    },
    {
      "parameters": {
        "amount": "={{ $json.waitSeconds }}",
        "unit": "seconds"
      },
      "id": "a1b2c3d4-0000-4000-8000-000000000007",
      "name": "Hold for the limit",
      "type": "n8n-nodes-base.wait",
      "typeVersion": 1.1,
      "position": [
        920,
        760
      ]
    },
    {
      "parameters": {
        "method": "POST",
        "url": "={{ $json.endpoint }}",
        "sendBody": true,
        "specifyBody": "json",
        "jsonBody": "={{ JSON.stringify({ model: $json.model, messages: [{ role: 'user', content: $json.prompt }] }) }}",
        "options": {
          "response": {
            "response": {
              "fullResponse": true,
              "neverError": true
            }
          }
        }
      },
      "id": "a1b2c3d4-0000-4000-8000-000000000008",
      "name": "Call the model",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        1140,
        760
      ],
      "retryOnFail": true,
      "maxTries": 3,
      "waitBetweenTries": 5000,
      "onError": "continueRegularOutput"
    },
    {
      "parameters": {
        "conditions": {
          "options": {
            "caseSensitive": true,
            "leftValue": "",
            "typeValidation": "loose",
            "version": 2
          },
          "conditions": [
            {
              "id": "a1b2c3d4-0000-4000-8000-000000000010",
              "leftValue": "={{ $json.statusCode }}",
              "rightValue": 429,
              "operator": {
                "type": "number",
                "operation": "equals"
              }
            }
          ],
          "combinator": "and"
        },
        "options": {}
      },
      "id": "a1b2c3d4-0000-4000-8000-000000000009",
      "name": "Rate limited?",
      "type": "n8n-nodes-base.if",
      "typeVersion": 2.2,
      "position": [
        1360,
        760
      ]
    },
    {
      "parameters": {
        "jsCode": "// Genuinely over quota: the node already retried 3 times, 5 seconds apart,\n// before we got here. Retrying again in this run will not help, so collect the\n// item with its reason instead of dropping it. Re-run just these afterwards.\nconst retryAfter = $input.first().json.headers?.['retry-after'] ?? null;\n\nreturn $input.all().map((item) => ({\n  json: {\n    ok: false,\n    reason: 'rate limited after retries',\n    retryAfterSeconds: retryAfter ? Number(retryAfter) : null,\n    prompt: $('One call at a time').first().json.prompt,\n  },\n}));"
      },
      "id": "a1b2c3d4-0000-4000-8000-00000000000a",
      "name": "Queue for retry",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        1600,
        880
      ]
    },
    {
      "parameters": {
        "mode": "raw",
        "jsonOutput": "={{ JSON.stringify({ ok: $json.statusCode >= 200 && $json.statusCode < 300, statusCode: $json.statusCode, waitedSeconds: $('Pace the call').first().json.waitSeconds, prompt: $('One call at a time').first().json.prompt, reply: $json.body?.choices?.[0]?.message?.content ?? null }) }}",
        "options": {}
      },
      "id": "a1b2c3d4-0000-4000-8000-00000000000b",
      "name": "Keep the result",
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        1600,
        640
      ]
    },
    {
      "parameters": {
        "jsCode": "// Every item has been through the loop. Report what the pacing actually cost,\n// so you can tell a working rate limiter from one that is silently doing nothing.\nconst items = $input.all().map((i) => i.json);\nconst waited = items.reduce((sum, i) => sum + (i.waitedSeconds || 0), 0);\nconst ok = items.filter((i) => i.ok).length;\n\nreturn [{\n  json: {\n    total: items.length,\n    succeeded: ok,\n    rateLimited: items.length - ok,\n    totalWaitSeconds: Math.round(waited * 100) / 100,\n    note: waited === 0\n      ? 'Nothing waited. Either RPM is high enough for this batch, or the workflow was never saved so static data did not persist.'\n      : 'Pacing held. Lower RPM to space calls further apart.',\n  },\n}];"
      },
      "id": "a1b2c3d4-0000-4000-8000-00000000000c",
      "name": "Summarise the run",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        1840,
        520
      ]
    },
    {
      "parameters": {},
      "id": "a1b2c3d4-0000-4000-8000-00000000000d",
      "name": "Finished",
      "type": "n8n-nodes-base.noOp",
      "typeVersion": 1,
      "position": [
        2080,
        520
      ]
    }
  ],
  "connections": {
    "Run the batch": {
      "main": [
        [
          {
            "node": "Sample workload",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Sample workload": {
      "main": [
        [
          {
            "node": "One call at a time",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "One call at a time": {
      "main": [
        [
          {
            "node": "Summarise the run",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Pace the call",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Pace the call": {
      "main": [
        [
          {
            "node": "Hold for the limit",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Hold for the limit": {
      "main": [
        [
          {
            "node": "Call the model",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Call the model": {
      "main": [
        [
          {
            "node": "Rate limited?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Rate limited?": {
      "main": [
        [
          {
            "node": "Queue for retry",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Keep the result",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Queue for retry": {
      "main": [
        [
          {
            "node": "One call at a time",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Keep the result": {
      "main": [
        [
          {
            "node": "One call at a time",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Summarise the run": {
      "main": [
        [
          {
            "node": "Finished",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "settings": {
    "executionOrder": "v1"
  },
  "active": false,
  "tags": []
}