You are here: Webhelp > Object > URL Filter

URL Filtering

URL filtering controls the access to some certain websites and records log messages for the access actions. URL filtering helps you control the network behaviors in the following aspects:

  • Access control to certain category of websites, such as gambling and pornographic websites.
  • Access control to certain category of websites during the specified period. For example, forbid to access IM websites during the office hours.
  • Access control to the website whose URL contains the specified keywords. For example, forbid to access the URL that contains the keyword of game.

If IPv6 is enabled, you can configure URL and keyword for both IPv4 and IPv6 address. How to enable IPv6, see StoneOS_CLI_User_Guide_IPv6.

Configuring URL Filtering

Configuring URL filtering contains two parts:

  • Create a URL filtering rule
  • Bind a URL filtering rule to a security zone or policy rule

Part 1: Creating a URL filtering rule

  1. Select Object > URL Filtering>Profile.
  2. Click New.
  3. In the URL Category part to configure the URL category control type for URL filtering rules to control the access to some certain category of website.
  4. In the URL Keyword Category part to configure the URL keyword category control type for URL filtering rules to control the access to the website whose URL contains the specific keywords.
  5. Click OK to save the settings.
The control type of a URL filtering rule can configure both the URL category and the URL keyword category.

Part 2: Binding a URL filtering rule to a security zone or security policy rule

The URL filtering configurations are based on security zones or policies.

  • If a security zone is configured with the URL filtering function, system will perform detection on the traffic that is destined to the binding zone specified in the rule, and then do according to what you specified.
  • If a policy rule is configured with the URL filtering function, system will perform detection on the traffic that is destined to the policy rule you specified, and then respond.
  • The threat protection configurations in a policy rule are superior to that in a zone rule if they are specified at the same time, and the URL filtering configurations in a destination zone are superior to that in a source zone if they are specified at the same time.
  • To perform the URL filtering function on the HTTPS traffic, see the policy-based URL filtering.

To create the zone-based URL filtering, take the following steps:

  1. Create a zone. For more information about how to create this, refer to Security Zone.
  2. In the Zone Configuration dialog box, select the Threat Protection tab.
  3. Enable the threat protection that you need, and select the URL filtering rules from the profile drop-down list below; you can click Add Profile from the profile drop-down list below to create a URL filtering rule. For more information, see Part 1: Creating a URL filtering rule.
  4. Click OK to save the settings.

To create the policy-based URL filtering, take the following steps:

  1. Configure a security policy rule. For more information, see Configuring a Security Policy Rule.
  2. In the Protection tab, select the Enable check box of URL Filtering.
  3. From the Profile drop-down list, select a URL filtering rule. You can also click Add Profile to create a new URL filtering rule.
  4. To perform the URL filtering function on the HTTPS traffic, you need to enable the SSL proxy function for this security policy rule. System will decrypt the HTTPS traffic according to the SSL proxy profile and then perform the URL filtering function on the decrypted traffic.
  5. Click OK to save the settings.

If necessary, you can go on to configure the functions of Predefined URL DB, URL Lookup, and Warning Page.

Object Description
Predefined URL DB The predefined URL database includes dozens of categories and tens of millions of URLs and you can use it to specify the URL categories.
URL Lookup Use the URL lookup function to inquire URL information from the URL database, including the URL category and the category type.
Warning Page
  • Block warning: When your network access is blocked, a warning page will prompt in the Web browser.
  • Audit warning: When your network access is audited, a warning page will prompt in the Web browser.
  • Only after canceling the binding can you delete the URL filtering rule.
  • To get the latest URL categories, you are recommended to update the URL database first. For more information about URL database, see Predefined URL DB.

Cloning a URL filtering Rule

System supports the rapid clone of a URL filtering rule. You can clone and generate a new URL filtering rule by modifying some parameters of the one current URL filtering rule.

To clone a URL filtering rule, take the following steps:

  1. Select Object > URL Filtering.
  2. Select a URL filtering rule in the list.
  3. Click the Clone button above the list, and the Name configuration box will appear below the button. Then enter the name of the new URL filtering rule.
  4. The cloned URL filtering rule will be generated in the list.

Viewing Web Surfing Records

To view the Web surfing records, view URL Log. Before you view the Web surfing records, see Log Configuration to enable URL Log function.

Configuring URL Filtering Objects

When using URL filtering function, you need to configure the following objects:

Object Description
Predefined URL DB The predefined URL database includes dozens of categories and tens of millions of URLs and you can use it to specify the URL categories.
User-defined URL DB The user-defined URL database is defined by you and you can use it to specify the URL category.
URL Lookup Use the URL lookup function to inquire URL information from the URL database.
Keyword Category Use the keyword category function to customize the keyword categories.
Warning Page Enable or disable the warning page.
  • Block warning: When your network access is blocked, a warning page will prompt in the Web browser.
  • Audit warning: When your network access is audited, a warning page will prompt in the Web browser.

Predefined URL DB

System contains a predefined URL database.

The predefined URL database is controlled by a license . Only after a URL license is installed, the predefined URL database can be used.

The predefined URL database provides URL categories for the configurations of a URL filtering. It includes dozens of categories and tens of millions of URLs .

When identifying the URL category, the user-defined URL database has a higher priority than the predefined URL database.

Configuring Predefined URL Database Update Parameters

By default, system updates predefined URL database everyday. You can change the update parameters according to your own requirements. Currently, two default update servers are provided: https://update1.hillstonenet.com and https://update2.hillstonenet.com. Besides, you can update the predefined URL database from your local disk.

To change the update parameters, take the following steps:

  1. Select System > Upgrade Management > Signature Database Update.
  2. In the URL category database update section, you can view the current version of the database, perform the remote update, configure the remote update, and perform the local update.
  3. Click Enable button of Auto Update to enable the automatic update function and then continue to specify the frequency and time. Click OK to save your settings.
  4. Double click an entry of Update Server to configure the update server URL. Specify the URL or IP address of the update server, and select the virtual router that can connect to the server. To restore the URL settings to the default ones, click Restore Default.
  5. Double click an entry of Proxy Server, then enter the IP addresses and ports of the main proxy server and the backup proxy server. When the device accesses the Internet through a HTTP proxy server, you need to specify the IP address and the port number of the HTTP proxy server. With the HTTP proxy server specified, various signature databases can update normally.
  6. Click OK to save the settings.

Upgrading Predefined URL Database Online

To upgrade the URL database online, take the following steps:

  1. Select System > Upgrade Management > Signature Database Update.
  1. In the URL category database update section, click Update to update the predefined URL database.

Upgrading Predefined URL Database from Local

To upgrade the predefined URL database from local, take the following steps:

  1. System > Upgrade Management > Signature Database Update
  2. In the URL category database update section, click Browse to select the URL database file from your local disk.
  3. Click Upload to update the predefined URL database.
You can not upgrade the predefined URL database from local in non-root VSYS.

User-defined URL DB

Besides categories in predefined URL database, you can also create user-defined URL categories, which provides URL categories for the configurations of URL filtering. When identifying the URL category, the user-defined URL database has a higher priority than the predefined URL database.

System provides three predefined URL categories: custom1, custom2, custom3. You can import your own URL lists into one of the predefined URL categories.

You can not import your own URL lists into one of the predefined URL category in non-root VSYS.

Configuring User-defined URL DB

To configure a user-defined URL category, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, select Configuration > User-defined URL DB. The User-defined URL DB dialog box will appear.
  3. Click New. The URL Category dialog box will appear.
  4. Type the category name in the Category box. URL category name cannot only be a hyphen (-). And you can create at most 16 user-defined categories.
  5. Type a URL into the URL http(s):// box.
  6. Click Add to add the URL and its category to the table.
  7. To edit an existing one, select it and then click Edit. After editing it, click Add to save the changes.
  1. Click OK to save the settings.

Importing User-defined URL

System supports to batch imported user-defined URL lists into the predefined URL category named custom1/2/3. To import user-defined URL, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, select Configuration > User-defined URL DB. The User-defined URL DB dialog box will appear.
  3. Select one of the predefined URL category(custom1/2/3), and then click Import.
  4. In the Batch Import URL dialog box, click Browse button to select your local URL file. The file should be less than 1 M, and have at most 1000 URLs. Wildcard is supported to use once in the URL file, which should be located at the start of the address.
  5. Click OK to finish importing.

Clearing User-defined URL

In the predefined URL category named custom1/2/3, clear a user-defined URL, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, select Configuration > User-defined URL DB. The User-defined URL DB dialog box will appear.
  3. Select one of the predefined URL categories(custom1/2/3), and then click Clear. The URL in the custom 1/2/3 will be cleared from the system.

URL Lookup

You can inquire a URL to view the details by URL lookup, including the URL category and the category type.

Inquiring URL Information

To inquiry URL information, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, click Configuration > URL Lookup. The URL Lookup dialog box will appear.
  3. Type the URL into the Please enter the URL to inquire box.
  4. Click Inquire, and the results will be displayed at the bottom of the dialog box.

Configuring URL Lookup Servers

URL lookup server can classify an uncategorized URL (URL is neither in predefined URL database nor in user-defined URL database) you have accessed, and then add it to the URL database during database updating. Two default URL lookup servers are provided: url1.hillstonenet.com and url2.hillstonenet.com. By default, the URL lookup servers are enabled.

To configure a URL lookup server, take the following steps:

  1. Select Object > URL Filtering>Profile.
  2. At the top-right corner, Select Configuration > Predefined URL DB. The Predefined URL DB dialog box will appear.
  3. Click Inquiry Server Configuration. The Predefined URL DB Inquiry Server Configuration dialog box will appear.
  4. In the Inquiry server section, double-click the cell in the IP/Port/Virtual Router column of Server1/2 and type a new value.
  5. Select the check box in the Enable column to enable this URL lookup server.
  6. Click OK to save the settings.

Keyword Category

You can customize the keyword category and use it in the URL filtering function.

After configuring a URL filtering rule, system will scan traffic according to the configured keywords and calculate the trust value for the hit keywords. The calculating method is: adding up the results of times * trust value of each keyword that belongs to the category. Then system compares the sum with the threshold 100 and performs the following actions according to the comparison result:

  • If the sum is larger than or equal to category threshold (100), the configured category action will be triggered;
  • If more than one category action can be triggered and there is block action configured, the final action will be Block;
  • If more than one category action can be triggered and all the configured actions are Permit, the final action will be Permit.

For example, a URL filtering rule contains two keyword categories C1 with action block and C2 with action permit. Both of C1 and C2 contain the same keywords K1 and K2. Trust values of K1 and K2 in C1 are 20 and 40. Trust values of K1 and K2 in C2 are 30 and 80.

If system detects 1 occurrence of K1 and K2 each on a URL, then C1 trust value is 20*1+40*1=60<100, and C2 trust value is 30*1+80*1=110>100. As a result, the C2 action is triggered and the URL access is permitted.

If system detects 3 occurrences of K1 and 1 occurrence of K2 on a URL, then C1 trust value is 20*3+40*1=100, and C2 trust value C2 is 30*3+80*1=170>100. Conditions for both C1 and C2 are satisfied, but the block action for C1 is triggered, so the web page access is denied.

Configuring a Keyword Category

To configure a keyword category, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, select Configuration > Keyword Category. The Keyword Category dialog box will appear.
  3. Click New. The Keyword Category Configuration dialog box will appear.
  4. Type the category name.
  5. Click New. In the slide area, specify the keyword, character matching method (simple/regular expression), and trust value (100 by default).
  6. Click Add to add the keyword to the list below.
  7. Repeat the above steps to add more keywords.
  8. To delete a keyword, select the keyword you want to delete from the list and click Delete.
  9. Click OK to save your settings.

Warning Page

The warning page shows the user block information and user audit information. You can enable or disable the warning page as needed.

The warning page include predefined warning page and user-defined warning page.

  • Predefined warning page: Displays the predefined warning information content, including prompt information and warning reasons.
  • User-defined warning page: You can customize the warning page by custom warning information and pictures. For details, please refer to Warning Page Management..

Enabling/ Disabling the Block Warning

The block warning is disabled by default. If the internet behavior is blocked by the URL filtering function, the Internet access will be denied. The information of Access Denied will be shown in your browser, and some web surfing rules will be shown to you on the warning page at the same time. According to the different network behaviors, the predefined warning page includes the following two situations:

  • Visiting a certain type of URL.
  • Visiting the URL that contains a certain type of keyword category.

To enable or disable the block warning , take the following steps:

  1. Click Object > URL Filtering > Profile.
  2. At the top-right corner, select Configuration > Warning Page. The Warning Page dialog box will appear.
  3. In the Block Warning section, select Enable. To disable this function, unselect the Enable check box.
  4. Click OK to save the settings.

Enabling/ Disabling the Audit Warning

The audit warning function is disabled by default. After enabling the audit warning function, when your network behavior matches the configured URL filtering rule, your HTTP request will be redirected to a warning page where the audit and privacy protection information is displayed. See the picture below:

To enable or disable the audit warning function, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, select Configuration > Warning Page. The Warning Page dialog box will appear.
  3. In the Audit Warning section, select Enable.To disable this function, unselect the Enable check box.
    • If the user-defined warning page is not configured, the predefined warning page will be used.
    • If the user-defined warning page is configured and enabled, the user-defined warning page will be used.
    For details, please refer to Warning Page Management..
  4. Click OK to save the settings.

First Access of Uncategorized URL

For the uncategorized URL that you visit for the first time, that is, the URL which is neither in the system's predefined URL database nor in the user-defined URL database, system will continue to query the category of the URL in the cloud. Because the query may takes a litter while, system cannot process the uncategorized URL immediately until the query result is returned.

To solve the above problem, you can specify the waiting time of query and enable the block action when waiting times out. After the waiting time of query is exceeded, system will block the access to the uncategorized URL.

To configure related content of the first access of an uncategorized URL, take the following steps:

Select Object > URL Filtering > Profile.

At the top-right corner, select Configuration > First Access of Uncategorized URL. The First Access of Uncategorized URL dialog box will appear.

Type the waiting time value of query into the Waiting Time of Query text box. The range is 0 to 5000ms. The default value is 0, which means there is no wait time limit.

Select the Enable check box after Block after Waiting Timeout to enable the block action, after the waiting time of query is exceeded, system will block the access of uncategorized URL. After clearing the Enable check box, after the waiting time of query is exceeded, system will continue to perform URL filtering according to the configuration of URL filtering profile.

Click OK to save the settings.

Configuring the URL Blacklist/Whitelist

You can further control the access to some websites by configuring URL blacklists and whitelists.

  • After the URL blacklist is configured, when you send an access request to the specified URL in the blacklist, the system will block the request.
  • After the URL whitelist is configured, when you send an access request to the specified URL in the whitelist, system will not perform URL filtering for the access request and let the request pass
  • The URL blacklist, the URL whitelist and the URL filtering rule all configured with URL categories, the matching priority for URL category filtering is: the URL blacklist > the URL whitelist > the URL filtering rule.
  • An URL category can only be referenced by an object (URL blacklist, URL whitelist or URL filtering profile). For example, when the URL category "Advertisement" has been added to the URL blacklist, this URL category cannot be added to the URL whitelist, and it will not be referenced in the URL filtering profile.
  • Non-root VSYS does not support the URL blacklist\whitelist function, and the URL blacklist/whitelist configuration under root VSYS does not take effect and has no effect on non-root VSYS.

Configuring the URL Blacklist

To configure the URL blacklist, take the following steps:

  1. Select Object > URL Filtering > URL Blacklist/Whitelist.
  2. Select URL Blacklist tab to open the URL blacklist page, which displays all URL categories that have been added to the URL blacklist and the corresponding URL type and description.
  3. Click "+" , and select the add the URL category needed to add to the URL black list.
  4. The "URL category" on the left contains all URL categories that can be referenced (predefined URL DB and user-defined URL DB). You can also click to create a new URL category. For specific steps, see Configuring User-defined URL DB.
  5. If you need to delete the URL category entry in the URL blacklist, in the "URL blacklist" list on the right, select the URL category entry you want to delete and click .
  6. Click OK.

Configuring the URL Whitelist

To configure the URL whitelist, take the following steps:

  1. Select Object > URL Filtering > URL Blacklist/Whitelist.
  2. Select URL Whitelist tab to open the URL whitelist page, which displays all URL categories that have been added to the URL whitelist and the corresponding URL type and description.
  3. Click "+" , and select the add the URL category needed to add to the URL white list.
  4. The "URL category" on the left contains all URL categories that can be referenced (predefined URL DB and user-defined URL DB). You can also click to create a new URL category. For specific steps, see Configuring User-defined URL DB.
  5. If you need to delete the URL category entry in the URL whitelist, in the "URL whitelist" list on the right, select the URL category entry you want to delete and click .
  6. Click OK.