You are here: Webhelp 5.5R7 > Object > URL Filter

URL Filtering

URL filtering controls the access to some certain websites and records log messages for the access actions. URL filtering helps you control the network behaviors in the following aspects:

  • Access control to certain category of websites, such as gambling and pornographic websites.

  • Access control to certain category of websites during the specified period. For example, forbid to access IM websites during the office hours.

  • Access control to the website whose URL contains the specified keywords. For example, forbid to access the URL that contains the keyword of game.

If IPv6 is enabled, you can configure URL and keyword for both IPv4 and IPv6 address. How to enable IPv6, see StoneOS_CLI_User_Guide_IPv6.

Configuring URL Filtering

Configuring URL filtering contains two parts:

  • Create a URL filtering rule
  • Bind a URL filtering rule to a security zone or policy rule

Part 1: Creating a URL filtering rule

  1. Select Object > URL Filtering.
  2. Click New.

  3. Click URL Category tab to configure the URL category control type for URL filtering rules to control the access to some certain category of website.

  4. Click URL Keyword Category tab to configure the URL keyword category control type for URL filtering rules to control the access to the website whose URL contains the specific keywords.

  5. Click OK to save the settings.
The control type of a URL filtering rule can configure both the URL category and the URL keyword category.

Part 2: Binding a URL filtering rule to a security zone or security policy rule

The URL filtering configurations are based on security zones or policies.

  • If a security zone is configured with the URL filtering function, system will perform detection on the traffic that is destined to the binding zone specified in the rule, and then do according to what you specified.
  • If a policy rule is configured with the URL filtering function, system will perform detection on the traffic that is destined to the policy rule you specified, and then respond.
  • The threat protection configurations in a policy rule are superior to that in a zone rule if they are specified at the same time, and the URL filtering configurations in a destination zone are superior to that in a source zone if they are specified at the same time.
  • To perform the URL filtering function on the HTTPS traffic, see the policy-based URL filtering.

To create the zone-based URL filtering, take the following steps:

  1. Create a zone. For more information about how to create this, refer to Security Zone.

  2. In the Zone Configuration dialog box, select the Threat Protection tab.

  3. Enable the threat protection that you need, and select the URL filtering rules from the profile drop-down list below; you can click Add Profile from the profile drop-down list below to create a URL filtering rule. For more information, see Part 1: Creating a URL filtering rule.
  4. Click OK to save the settings.

To create the policy-based URL filtering, take the following steps:

  1. Configure a security policy rule. For more information, see Configuring a Security Policy Rule.
  2. In the Protection tab, select the Enable check box of URL Filtering.
  3. From the Profile drop-down list, select a URL filtering rule. You can also click Add Profile to create a new URL filtering rule.
  4. To perform the URL filtering function on the HTTPS traffic, you need to enable the SSL proxy function for this security policy rule. System will decrypt the HTTPS traffic according to the SSL proxy profile and then perform the URL filtering function on the decrypted traffic.
  5. Click OK to save the settings.

If necessary, you can go on to configure the functions of Predefined URL DB, URL Lookup, and Warning Page.

Object Description
Predefined URL DB

The predefined URL database includes dozens of categories and tens of millions of URLs and you can use it to specify the URL categories.

URL Lookup

Use the URL lookup function to inquire URL information from the URL database, including the URL category and the category type.

Warning Page
  • Block warning: When your network access is blocked, a warning page will prompt in the Web browser.

  • Audit warning: When your network access is audited, a warning page will prompt in the Web browser.
  • Only after canceling the binding can you delete the URL filtering rule.

  • To get the latest URL categories, you are recommended to update the URL database first. For more information about URL database, see Predefined URL DB.

Cloning a URL filtering Rule

System supports the rapid clone of a URL filtering rule. You can clone and generate a new URL filtering rule by modifying some parameters of the one current URL filtering rule.

To clone a URL filtering rule, take the following steps:

  1. Select Object > URL Filtering.
  2. Select a URL filtering rule in the list.
  3. Click the Clone button above the list, and the Name configuration box will appear below the button. Then enter the name of the new URL filtering rule.
  4. The cloned URL filtering rule will be generated in the list.

Viewing URL Hit Statistics

The URL access statistics includes the following parts:

  • Summary: The statistical information of the top 10 user/IPs, the top 10 URLs, and the top 10 URL categories during the specified period of time are displayed.
  • User/IP: The user/IP and detailed hit count are displayed.
  • URL: The URL and detailed hit count are displayed.
  • URL Category: The URL category and detailed hit count and traffic are displayed.

To view the URL hit statistics, see URL Hit in Monitor.

Viewing Web Surfing Records

To view the Web surfing records, view URL Logs. Before you view the Web surfing records, see Log Configuration to enable URL Log function.

Configuring URL Filtering Objects

When using URL filtering function, you need to configure the following objects:

Object Description
Predefined URL DB

The predefined URL database includes dozens of categories and tens of millions of URLs and you can use it to specify the URL categories.

User-defined URL DB

The user-defined URL database is defined by you and you can use it to specify the URL category.

URL Lookup

Use the URL lookup function to inquire URL information from the URL database.

Keyword Category

Use the keyword category function to customize the keyword categories.

Warning Page
  • Block warning: When your network access is blocked, a warning page will prompt in the Web browser.

  • Audit warning: When your network access is audited, a warning page will prompt in the Web browser.

Predefined URL DB

System contains a predefined URL database.

The predefined URL database is controlled by a license . Only after a URL license is installed, the predefined URL database can be used.

The predefined URL database provides URL categories for the configurations of a URL filtering. It includes dozens of categories and tens of millions of URLs .

When identifying the URL category, the user-defined URL database has a higher priority than the predefined URL database.

Configuring Predefined URL Database Update Parameters

By default, system updates predefined URL database everyday. You can change the update parameters according to your own requirements. Currently, two default update servers are provided: https://update1.hillstonenet.com and https://update2.hillstonenet.com. Besides, you can update the predefined URL database from your local disk.

To change the update parameters, take the following steps:

  1. Select System > Upgrade Management > Signature Database Update.
  2. In the URL category database update section, you can view the current version of the database, perform the remote update, configure the remote update, and perform the local update.
  3. Select Enable Auto Update to enable the automatic update function and then continue to specify the frequency and time. Click OK to save your settings.

  4. Click Configure Update Server to configure the update server URL. In the pop-up dialog box, specify the URL or IP address of the update server, and select the virtual router that can connect to the server. To restore the URL settings to the default ones, click Restore Default.
  5. Click Configure Proxy Server, then enter the IP addresses and ports of the main proxy server and the backup proxy server. When the device accesses the Internet through a HTTP proxy server, you need to specify the IP address and the port number of the HTTP proxy server. With the HTTP proxy server specified, various signature databases can update normally.
  6. Click OK to save the settings.

Upgrading Predefined URL Database Online

To upgrade the URL database online, take the following steps:

  1. Select System > Upgrade Management > Signature Database Update.
  1. In the URL category database update section, click Update to update the predefined URL database.

Upgrading Predefined URL Database from Local

To upgrade the predefined URL database from local, take the following steps:

  1. System > Upgrade Management > Signature Database Update
  2. In the URL category database update section, click Browse to select the URL database file from your local disk.
  3. Click Upload to update the predefined URL database.
You can not upgrade the predefined URL database from local in non-root VSYS.

User-defined URL DB

Besides categories in predefined URL database, you can also create user-defined URL categories, which provides URL categories for the configurations of URL filtering. When identifying the URL category, the user-defined URL database has a higher priority than the predefined URL database.

System provides three predefined URL categories: custom1, custom2, custom3. You can import your own URL lists into one of the predefined URL categories.

You can not import your own URL lists into one of the predefined URL category in non-root VSYS.

Configuring User-defined URL DB

To configure a user-defined URL category, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, select Configuration > User-defined URL DB. The User-defined URL DB dialog box will appear.
  3. Click New. The URL Category dialog box will appear.
  4. Type the category name in the Category box. URL category name cannot only be a hyphen (-). And you can create at most 16 user-defined categories.
  5. Type a URL into the URL http(s):// box.
  6. Click Add to add the URL and its category to the table.
  7. To edit an existing one, select it and then click Edit. After editing it, click Add to save the changes.
  1. Click OK to save the settings.

Importing User-defined URL

System supports to batch imported user-defined URL lists into the predefined URL category named custom1/2/3. To import user-defined URL, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, select Configuration > User-defined URL DB. The User-defined URL DB dialog box will appear.
  3. Select one of the predefined URL category(custom1/2/3), and then click Import.
  4. In the Batch Import URL dialog box, click Browse button to select your local URL file. The file should be less than 1 M, and have at most 1000 URLs. Wildcard is supported to use once in the URL file, which should be located at the start of the address.
  5. Click OK to finish importing.

Clearing User-defined URL

In the predefined URL category named custom1/2/3, clear a user-defined URL, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, select Configuration > User-defined URL DB. The User-defined URL DB dialog box will appear.
  3. Select one of the predefined URL categories(custom1/2/3), and then click Clear. The URL in the custom 1/2/3 will be cleared from the system.

URL Lookup

You can inquire a URL to view the details by URL lookup, including the URL category and the category type.

Inquiring URL Information

To inquiry URL information, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, click Configuration > URL Lookup. The URL Lookup dialog box will appear.
  3. Type the URL into the Please enter the URL to inquire box.

  4. Click Inquire, and the results will be displayed at the bottom of the dialog box.

Configuring URL Lookup Servers

URL lookup server can classify an uncategorized URL (URL is neither in predefined URL database nor in user-defined URL database) you have accessed, and then add it to the URL database during database updating. Two default URL lookup servers are provided: url1.hillstonenet.com and url2.hillstonenet.com. By default, the URL lookup servers are enabled.

To configure a URL lookup server, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, Select Configuration > Predefined URL DB. The Predefined URL DB dialog box will appear.
  3. Click Inquiry Server Configuration. The Predefined URL DB Inquiry Server Configuration dialog box will appear.
  4. In the Inquiry server section, double-click the cell in the IP/Port/Virtual Router column of Server1/2 and type a new value.

  5. Select the check box in the Enable column to enable this URL lookup server.
  6. Click OK to save the settings.

Keyword Category

You can customize the keyword category and use it in the URL filtering function.

After configuring a URL filtering rule, system will scan traffic according to the configured keywords and calculate the trust value for the hit keywords. The calculating method is: adding up the results of times * trust value of each keyword that belongs to the category. Then system compares the sum with the threshold 100 and performs the following actions according to the comparison result:

  • If the sum is larger than or equal to category threshold (100), the configured category action will be triggered;
  • If more than one category action can be triggered and there is block action configured, the final action will be Block;
  • If more than one category action can be triggered and all the configured actions are Permit, the final action will be Permit.

For example, a URL filtering rule contains two keyword categories C1 with action block and C2 with action permit. Both of C1 and C2 contain the same keywords K1 and K2. Trust values of K1 and K2 in C1 are 20 and 40. Trust values of K1 and K2 in C2 are 30 and 80.

If system detects 1 occurrence of K1 and K2 each on a URL, then C1 trust value is 20*1+40*1=60<100, and C2 trust value is 30*1+80*1=110>100. As a result, the C2 action is triggered and the URL access is permitted.

If system detects 3 occurrences of K1 and 1 occurrence of K2 on a URL, then C1 trust value is 20*3+40*1=100, and C2 trust value C2 is 30*3+80*1=170>100. Conditions for both C1 and C2 are satisfied, but the block action for C1 is triggered, so the web page access is denied.

Configuring a Keyword Category

To configure a keyword category, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, select Configuration > Keyword Category. The Keyword Category dialog box will appear.
  3. Click New. The Keyword Category Configuration dialog box will appear.
  4. Type the category name.
  5. Click New. In the slide area, specify the keyword, character matching method (simple/regular expression), and trust value (100 by default).
  6. Click Add to add the keyword to the list below.
  7. Repeat the above steps to add more keywords.

  8. To delete a keyword, select the keyword you want to delete from the list and click Delete.

  9. Click OK to save your settings.

Warning Page

The warning page shows the user block information and user audit information.

Configuring Block Warning

If the internet behavior is blocked by the URL filtering function, the Internet access will be denied. The information of Access Denied will be shown in your browser, and some web surfing rules will be shown to you on the warning page at the same time. According to the different network behaviors, the default block warning page includes the following two situations:

  • Visiting a certain type of URL.
  • Visiting the URL that contains a certain type of keyword category.

The block warning function is disabled by default. To configure the block warning function, take the following steps:

  1. Click Object > URL Filtering.
  2. At the top-right corner, select Configuration > Warning Page. The Warning Page dialog box will appear.
  3. In the Block Warning section, select Enable.
  4. Click OK to save the settings.

Configuring Audit Warning

After enabling the audit warning function, when your network behavior matches the configured URL filtering rule, your HTTP request will be redirected to a warning page where the audit and privacy protection information is displayed. See the picture below:

The audit warning function is disabled by default. To configure the audit warning function, take the following steps:

  1. Select Object > URL Filtering.
  2. At the top-right corner, select Configuration > Warning Page. The Warning Page dialog box will appear.
  3. In the Audit Warning section, select Enable.
  4. Click OK to save the settings.