We currently measure API rate limits in terms of concurrent requests. Depending on your requests per minute, tokens per request, and model selection, one request can handle a range of traffic. For this reason, we try to be flexible with users to adjust their limit to accommodate their growth.
For API users in the non-commercial evaluation phase, we limit users to 1 concurrent request.
Once you sign our standard commercial contract, we automatically increase your API keys to serve up to 5 concurrent requests.
To increase your rate limit further, we require that you pass our Trust and Safety Review. You can request an increase by emailing email@example.com with your estimated requests per minute and your estimated tokens.