A rate rule restricts the flow of site traffic with the intention of:
Diverting malicious or inadvertent DDoS traffic.
Preventing an origin server (e.g., your web server) from being overloaded.
Apply one or more rate limit(s) to your traffic by adding the desired rate rules to a Security Application configuration and then defining how requests that exceed those rate limits will be handled. In addition to logging these violations, you can block, drop, redirect to another URL, or send a custom response.
A rate rule restricts the rate of traffic that may be directed to one or more web site(s). Requests that exceed a rate rule will not be honored.
Key information:
Our service inspects all traffic when determining whether a rate rule is applicable.
A rate rule may be applied to all requests, unique IP addresses, or unique user agents.
If you are using Edgio Premier, then you may also apply a rate rule by HTTP status codes, ASNs, JA3, JA4, cookies, query string parameters, and request headers. Additionally, you can combine up to two items. For example, you can apply a rate rule to unique user agents on a per ASN basis.
The set of traffic that will be rate limited is determined by two factors:
Security Application Configuration: Apply a rate limit to your traffic by assigning a rate rule to a Security Application configuration. This Security Application configuration determines the set of requests to which this rate rule will be applied.
Condition Groups: You may add one or more condition group(s) to a rate rule. Each condition group defines additional prerequisites (e.g., URL or user agent) that a request must meet before it will count towards the rate limit.
If one or more condition group(s) have been defined, then the request must also satisfy all of the conditions defined within at least one condition group.
A rate rule does not take into account whether content needs to be requested from an origin server or if a cached version may be served directly from our network.
A rate rule is enforced by each POP according to the approximate number of requests that it receives over the specified time interval (e.g., 1 second, 10 seconds, or 1 minute).
A rate rule always runs in production mode. Although you may not run it in audit mode, you may configure your Security Application configuration to only generate alerts when a rate limit is exceeded.
Requests that generate alerts count towards the rate limit for other rate rules defined within a Security Application configuration.
Setting up a rate rule involves defining a rate limit and determining how that rate limit will be applied. You may also specify additional critieria that identify the set of requests to which this rate rule will be applied.
Rate Limit: Define a maximum request rate before a predefined action is triggered.
To each unique client that exceeds the defined rate limit.
Identify a unique client by its user agent, IP address, or both. Edgio Premier customers may identify clients using up to two of the following criteria: IP address, user agent, HTTP status code, ASN, JA3, JA4, cookie, query string parameters, and request headers.
Condition Group: A request counts towards a rate limit when it satisfies all of the following criteria:
If one or more condition group(s) have been defined, then the request must also satisfy all of the conditions defined within at least one condition group.
Each condition identifies the type of requests that are eligible for rate limiting by URL path, request headers, IP address, file extension, and request method. Edgio Premier customers may also identify requests by ASN, country, request header, request URL path, JA3, JA4, query string, and cookie.
Action: Rate limit your traffic by adding one or more rate rule(s) to a Security Application configuration. This Security Application configuration determines the type of action that will be applied to requests that exceed the above rate limit.
A rate rule’s source determines how Edgio counts requests for the purpose of rate limiting your traffic. For example, you can configure Edgio to count all requests against your rate limit or you can configure it to count requests from each unique IP address. Define this behavior from within the Apply rate limit
to option. The available modes are described below.
Any request: Indicates that all requests will count towards the rate limit. Once the specified rate limit is exceeded, it will be enforced without taking into consideration which client submitted the request.
This mode is not recommended when there are malicious clients that are spoofing legitimate traffic. This type of configuration may potentially lead to a situation where spoofed traffic is honored while legitimate traffic is rate limited.
IP address: Indicates that the requests from each unique client, as determined by its IP address, will be tracked. The specified rate limit will only be enforced on the clients that exceed it.
Certain services and applications, such as VPNs, mask a client’s IP address. Specifically, they will report an IP address of their choosing instead of the client’s real IP address. As a result, multiple devices and perhaps even users may end up sharing the same IP address.
IP address and user agent: Edgio Security Business and below. Indicates that the requests from each unique client, as determined by each unique combination of IP address and user agent (e.g., web browser), will be tracked. The specified rate limit will only be enforced on the clients that exceed it.
All requests from a specific IP address that contain a blank or missing User-Agent header will be treated as a single client.
User agent: Requires Edgio Premier. Indicates that the requests from each unique user agent (e.g., web browser) will be tracked.
ASN: Requires Edgio Premier. Indicates that the requests from each unique autonomous system number (ASN) will be tracked.
Status Code: Requires Edgio Premier. Indicates that the requests from either a specific or a range of HTTP status codes will be tracked. For example, if you specify 403, then Edgio will track all requests that result in a 403 Forbidden response. Alternatively, if you specify 400-499, then Edgio will track all requests that result in a 4xx response.
An advantage of rate limiting by status code is that it allows you to define a more aggressive policy without increasing your risk of false positives. For example, clients do not typically generate many 403 Forbidden responses. This means that you can temporarily block IP addresses that generate too many 403 Forbidden responses (e.g., 5 per second).
JA3: Requires Edgio Premier. Indicates that the requests for each unique JA3 fingerprint will be tracked.
JA4: Requires Edgio Premier. Indicates that the requests for each unique JA4 fingerprint will be tracked.
Cookie: Requires Edgio Premier. Indicates that the requests for each unique value for the specified cookie will be tracked.
ARGS: Requires Edgio Premier. Indicates that the requests for each unique value for the specified query string parameter will be tracked.
Header: Requires Edgio Premier. Indicates that the requests for each unique value for the specified request header will be tracked.
Edgio Premier allows you to combine up to two of the above sources when defining unique clients.
The maximum rate at which requests will be honored before a predefined action is applied to it is known as the rate limit. A rate limit defines the number of requests over a given time period (e.g., 5 seconds, 10 seconds, or 1 minute). Define the desired rate limit through the Rate limit option.
Key information:
The source that will be rate limited should be taken into account when defining a rate limit.
A rate limit that applies across all requests should be significantly larger than a rate limit that applies to unique clients.
The specified rate limit is enforced on each POP based on the number of requests that it receives.
Edgio Premier customers may specify a duration longer than 5 minutes (i.e., 10, 15, and 30 minutes) when rate limiting by IP address. Adding another source (e.g., user agent, request header, or cookie) restricts the maximum duration to 5 minutes.
A condition group defines one or more prerequisites that must be met before a request will count towards the rate limit.
A request will only count towards the rate limit when it satisfies the host and URL path match conditions defined within a Security Application configuration. Additionally, if one or more condition group(s) have been defined, then the request must also satisfy all of the conditions defined within at least one condition group.
The types of prerequisites that may be defined are described below.
ASN: A request will count towards the rate limit when it originates from an autonomous system (AS) whose number (ASN) matches a value defined in the Value(s) option.
Example:
15133
Bot Score: Requires Bot Manager Advanced, Edgio Enterprise, or Edgio Premier. A request will count towards the rate limit according to the request’s bot score. Set the Value(s) option to the bot score threshold and then select the type of comparison that will be performed from within the Match Type option.
Edgio assigns a bot score according to the level of confidence that the request was generated by a bot. This score is calculated by analyzing the request and its behavior. The range for this score is 0 to 100.
Country: A request will count towards the rate limit when it originates from a country whose code matches a value defined in the Value(s) option.
File extension: A request will count towards the rate limit when the filename of the requested content contains a file extension that matches a value defined in the Value(s) option.
Syntax:
.<File Extension>
Example:
.htm
A request’s file extension is determined by the request’s relative path. If the query string determines the file extension for the response, then you should use request query instead. Create a regular expression that matches for the desired file extension(s) within the query string.
IP address: A request will count towards the rate limit when its IP address matches a value defined in the Value(s) option. Use standard IPv4 and CIDR notation when setting this option.
Specify a subnet by appending a slash (/) and the desired bit-length of the prefix (e.g., 11.22.33.0/22).
Request header: A request will count towards the rate limit when the value corresponding to the specified request header is an exact case-sensitive match for the one defined in the Value(s) option. This condition supports the following request headers:
Host: A request will count towards the rate limit when its Host header matches the specified hostname or IP address.
Syntax:
<Host>
<Host>:<Port>
Key information:
The entire Host header value will be compared against the specified value.
The CDN only accepts HTTP/HTTPS requests on standard ports (i.e., 80 and 443). Typically, a Host request header does not include port information for standard ports. However, the requesting user agent defines the Host request header submitted to the CDN.
For the purpose of this comparison, the hostname defined by this match condition will not be resolved to an IP address.
For the purpose of this comparison, an origin configuration’s HTTP Host Header option is irrelevant.
User-Agent: A request will count towards the rate limit when its User-Agent header is an exact match to a specified value.
User agent strings typically vary by type and version.
Referer: A request will count towards the rate limit when its Referer header is an exact match to a specified value.
Request method: A request will count towards the rate limit when the request’s HTTP method matches a value defined in the Value(s) option. Valid values are:
GET | POST | PUT | HEAD | DELETE | OPTIONS
Request URL path: A request will count towards the rate limit when its request URL contains a relative path that matches a value defined in the Value(s) option.
For the purposes of this option, specify a URL path pattern that starts directly after the hostname. Do not include a protocol or a hostname.
Sample values:
/marketing
/800001/mycustomerorigin
This type of match condition requires a Host condition within the same condition group.
Syntax:
/<Path>/<Asset>
Example:
/marketing/brochures/widget.htm
A partial match does not count towards the rate limit. For example, given the above sample configuration, the following request would not count towards the rate limit: http://cdn.example.com/ marketing/brochures/widget.html.
JA3: Requires Edgio Premier. A request will count towards the rate limit when its JA3 fingerprint matches the specified value.
JA4: Requires Edgio Premier. A request will count towards the rate limit when its JA4 fingerprint matches the specified value.
Request query: Requires Edgio Premier. A request will count towards the rate limit when its query string matches the specified value or pattern.
Request cookie: Requires Edgio Premier. A request will count towards the rate limit when it contains the specified cookie with the specified value.
Key information:
A single condition group may contain multiple conditions.
You may define multiple condition groups.
All conditions are case-sensitive.
A request will count towards the rate limit when it satisfies both of the following conditions:
A Security Application configuration’s hostname and URL path conditions.
If one or more condition group(s) have been defined, then the request must also satisfy all of the conditions defined within at least one condition group.
Each condition must contain a value through which requests will be identified.
Regular expressions or wildcards may not be used to identify requests.
Blank values are not allowed.
Add a value by typing it and then pressing ENTER.
Ensure that your rate limiting configuration complies with the following limits:
Condition groups per rate rule: 5
Conditions per condition group: 5
# of values per condition:
IP Address: 300
All Other Conditions: 100
This limit only applies when the Type option for a condition is set to Multiple Match.
You may define multiple rate rules within a Security Application configuration. This type of setup provides greater control when determining how requests will be rate limited.
Although most enforcement actions will cause Edgio to stop evaluating a request, this is not the case for alerts. Requests that generate alerts are still eligible for enforcement by other rate rules defined within a Security Application configuration.
Rate rules is an exception with regards to request evaluation after an alert has been triggered. For all other security rules, Edgio Security does not perform further evaluation of a request once enforcement is triggered.
Common use cases for multiple rules:
Define site-specific rules.
Define rules based on traffic profiles.
Define rules that track abnormal traffic patterns.
The order in which rules are listed is critical, since it determines which rule will be applied to a request.
It is recommended to order rules according to how they identify requests. Stricter rules that identify requests using multiple conditions should be placed closer to the top of the list, while catch-all rules should be placed closer to the bottom. This ensures that rules are applied to requests as intended.
Key information:
Rules are processed in the order in which they are listed (i.e., top to bottom) within a Security Application configuration.
If a Security Application configuration’s enforcement action for rate rules affects the response (e.g., drops the request or sends a custom response), then only the first rate rule that a request satisfies will be applied to it. Once a request satisfies a rate rule, all subsequent rate rules will be skipped.
Apply a rate rule to production traffic by adding it to a Security Application configuration and then determining how it will be enforced. Multiple Security Application configurations may use the same rate rule. Leverage this capability to tailor security screening by application or traffic profile.
Before adding a rate rule to a Security Application configuration, verify that the Security Application configuration’s hostname and URL path conditions do not conflict with your rate rule’s conditions.
This service inspects all traffic, regardless of platform, to determine whether it should be rate limited.
Sample scenario: This scenario assumes a rate rule with the following configuration:
Identify unique clients by IP address.
A rate limit of 100 requests per minute.
A Security Application configuration that matches all hostnames and URL paths.
No condition groups.
If a client makes 200 requests per minute, then it will be rate limited to 100 requests per minute. If another client makes 75 requests per minute, then the second client will not be rate limited.
It may take up to 2 minutes for an updated rate rule to be applied across our entire network.
To create a rate rule
Navigate to the Rate Rules page.
From the Edgio Console, select the desired organization.
From the Security section, click Rate Rules.
Click + New Rate Ruleset.
In the Name option, type the unique name by which this rate rule will be identified. This name should be sufficiently descriptive to identify it when setting up a Security Application configuration.
In the Apply rate limit to option, indicate whether the rate limit should be applied across all requests or to each unique client.
Edgio Premier: You may select up to two sources when defining a unique client. For example, you may select User agent and Cookie to identify each combination of a specific cookie and user agent as a unique client.
In the Rate limit option, define the maximum rate at which requests may flow to your origin server(s). Define this rate by indicating the maximum number of requests for the selected time interval (e.g., 1 second, 30 seconds, 1 minute, etc.).
Optional. Create a condition group to identify the types of requests that qualify for rate limiting.
Click the + New Condition Group label.
Optional. Rename Condition Group 1 to a brief name that describes the purpose of the condition group.
In the Matched by option, select the method by which requests will be identified.
If you set this option to Request header or Cookie, then you should also define the name of the desired request header or cookie within the Request header name or Request cookie name option, respectively.
If the Match type option is available, determine whether the Value(s) option will contain one or more exact
value(s) (Multiple exact match) or a regular expression (Regex).
Perform either of the following steps:
Values: In the Values option, type the value that must be satisfied before a
request will count towards the rate rule and then press ENTER. Repeat this step as needed.
If the Case sensitive option is available, use it to determine whether a case-sensitive comparison will be performed.
Value: In the Value option, type the desired regular expression pattern.
Choose whether this condition will be satisfied when a request matches or does not match a value defined in the Value(s) option.
Matches: Clear the Negative match option.
Does Not Match: Mark the Negative match option.
Optional. Add another condition to the current condition group by clicking + New condition and then repeating steps 6.3 - 6.6.
If a condition group has been defined, then a request must satisfy all of the conditions within at least one condition group in order to be eligible for rate limiting.
Optional. Create another condition group by following steps 6.1 - 6.7.
Multiple condition groups provide the means for identifying different types of requests for the purpose of rate limiting.
Optional. Enable a rule by toggling the Rule Status option to On.
Click Save.
To modify a rate rule
Navigate to the Rate Rules page.
From the Edgio Console, select the desired organization.
From the Security section, click Rate Rules.
Click on the desired rate rule.
Make the desired changes.
Click Save.
To delete a rate rule
You cannot delete a rate rule that is associated with a Security
Application configuration. Please either modify the Security
Application configuration to point to a different rate rule or
delete that Security Application configuration.
Check your Security Application configurations to verify
that the desired rate rule is not in use.
Navigate to the Rate Rules page.
From the Edgio Console, select the desired organization.
Requests will be rate limited according to this configuration:
Type
Setting
Value
Security Application
Hostname
Default configuration (Match all hostnames.)
Security Application
URL path(s)
Default configuration (Match all URL paths.)
Rate Rule
Apply rate limit to
Any request
This type of source applies a rate limit to all requests without taking into account the client that submitted each request.
Rate Rule
Rate limit
300 requests per minute
In the above scenario, the CDN received 1200 requests per minute for the following three files:
Index.html: 600 requests
Styles.css: 400 requests
Logo.png: 200 requests
The CDN handled the above requests in the following manner:
It honored the following 800 requests:
Index.html: 300 requests
Styles.css: 300 requests
Logo.png: 200 requests
More information:
A rate limit of 300 requests per minute is being enforced on a per POP basis. Only 200 requests were honored for Logo.png, since it was only requested 200 times within that minute.
Our CDN service applied a predefined rate limiting action (e.g., URL redirection) to the following 400 requests:
Requests will be rate limited according to this configuration:
Type
Setting
Value
Security Application
Hostname
Default configuration (Match all hostnames.)
Security Application
URL path(s)
Default configuration (Match all URL paths.)
Rate Rule
Apply rate limit to
IP address
This type of source applies a rate limit to all requests from unique IP addresses.
Rate Rule
Rate limit
50 requests per minute
The following diagram illustrates how this rate rule controls the total number of requests that may flow through our network.
In this scenario, only a single client (i.e., laptop) exceeded the rate limit. Specifically, our CDN service enforced rate limiting for that client’s requests for Index.html and Styles.css. Our CDN service
honored all other requests, since they do not exceed the defined rate limit (i.e., 50 requests per minute).
Aggregate traffic is not taken into account when rate limiting unique clients. For example, this rate rule would not be enforced on 2,000 clients if they only issued 5 requests per minute. Although aggregate traffic exceeded the rate rule, no single client exceeded 50 requests per minute.
A Security Application configuration may contain multiple rate rules that define a custom rate limit for different types of requests. These rate rules are processed in the order in which they are listed.
Once a rule is satisfied, it will be applied to the request and no additional rules will be processed.
It is recommended to order rate rules according to how they identify requests. Stricter rules that identify requests using multiple conditions should be placed closer to the top of the list, while
catch-all rules should be placed closer to the bottom. This ensures that your rate limits are applied to requests as intended.
In this scenario, your Security Application configuration applies the following rate rules to all requests.
Action: Approximately 200 requests will be honored, while approximately 100 requests will be dropped.
Why?
These requests do not meet the conditions defined for the first rule. However, they do satisfy the second rule. As a result, the subsequent rule (i.e., third rule) is skipped.
These requests do not meet the conditions defined for the first or second rule. However, the third rule is applicable, since it is a catch-all rule that applies to all requests.
Although the total number of requests from all clients exceeds 500 requests per minute, only the 450 requests submitted by this client are eligible for rate limiting by the third rule. Therefore, the rate limit defined in the third rule is not being exceeded.
Notice that a different rate rule is applied to these requests, even though they originate from the same client.
Compare a previous version of a configuration to the current version.
Edgio Security Premier and Business support a rolling window of up to 200 versions, while Edgio Security Essentials and Insights is restricted to a rolling window of up to 100 versions.
An advantage of using version control is that it allows you to quickly roll back to a previously vetted configuration. For example, if you notice that a new configuration has resulted in more false positives, then you can roll back to the previous version before analyzing the data.
To view, compare, and reactivate a previous configuration
Load the desired security configuration (e.g., access rule, rate rule, or custom rule).
Click Versions.
Click on the desired version to view it.
Optional. Compare the version selected in the previous step to the current version by clicking Diff. Differences between those two versions are highlighted in green (new or updated lines) and red (modified or deleted lines).
Optional. Reactivate the version selected in step 3 by clicking Reactivate. Click Reactivate this version to confirm that it will be reactivated.
If you are using Edgio Premier, then you may also apply a rate rule by HTTP status codes, ASNs, JA3, JA4, cookies, query string parameters, and request headers. Additionally, you can combine up to two items. For example, you can apply a rate rule to unique user agents on a per ASN basis.
