IP-reputation feature

https://blog.norz.at/citrix-netscaler-ip-reputation-feature/

I recently had to protect a website using IP reputation feature. There is some good information about this feature, however I decided to glean information here.

Facts about this feature

IP reputation is a platinum feature. It is included in web application firewall (there are extra licenses for the WAF available, they also contain IP-reputation).

IP-reputation feature provides us with a constantly updated feed of “known” malicious IP addresses. This database maintained by webroot. This database is dynamically generated and updated every 5 minutes, so it will never be outdated. Webroot uses sensor networks for this fully automated process. You may use this database manually from here.

It is designed to check for the reputation of an IP address, so to find out if this address is a well known malicious one, or not. All IPs not found in this database are considered to be non-malicious.

IP reputation in admin partitions

IP reputation works fine in admin partition. The iprep.db database-file is shared across admin partitions. so only the root partition needs to be able to access the internet. From technical point of view the partition using ipreputation service neither needs internet access nor be able to resolve names.

Requirements

IP-reputation does HTTP call-outs to api.bcss.brightcloud.com on port 443. You therefore need to be able to:

have a valid license (i.e. a WAF license)
resolve api.bcss.brightcloud.com from your NetScaler
connect from NSIP to this IP via port 443
advanced feature Reputation has to be enabled (enable feature reputation)

Yes. that’s right: NSIP. NetScaler BSD system always uses NSIP, and IP-reputation is done in BSD, not inside the NetScaler subsystem.

In the end you’ll find the data base file at /var/nslog/iprep/iprep.db. This file will appear immediately after enabling the feature. This file is not human readable, it’s SQLite. So you have to mess with SQLite browser to dig into it.

NetScaler stores a copy of WebRoot’s database for off-line use (and to avoid undesired latency). It automatically checks for updates every 5 minutes.

During first start of the IP-reputation service Citrix NetScaler does an initial call-out to api.bcss.brightcloud.com from it’s NSIP via port 443 to fetch the database. This process is logged into /var/log/iprep.log

Oct 4 03:50:00 82e6de130138 iprep: iprep process started... Oct 4 03:50:00 82e6de130138 iprep: iprep_get_schema_version:134 current schema version:1.0 Oct 4 03:50:00 82e6de130138 iprep: iprep_check_db_upgrade:296 DB schema is not up-to-date. Oct 4 03:50:00 82e6de130138 iprep: iprep_upgrade_db:242 upgrading schema version from 1.0 to 1.1. Oct 4 03:50:01 82e6de130138 iprep: IPREP update versions: major version:1 minor version:1068 update version:231 total ips:1640158 last update time:1485009026 Oct 4 03:50:01 82e6de130138 iprep: Webroot credentials from PE. oem_id:Citrix device_id:450000 user_id:HE2H91SCZ6. Oct 4 03:50:01 82e6de130138 iprep: PE update versions: major version:0 minor version:0 update version:0 total ips:0 last update time:0 Oct 4 03:50:01 82e6de130138 iprep: outfile:/var/nslog/iprep/webroot_http_resp_1507089001.xml

This database is updated every fife minutes. Database updates also get logged:

Oct 4 04:15:21 <local2.info> 82e6de130138 iprep: File:update_1.1332_107.txt no of ips:0. Oct 4 04:15:21 <local2.info> 82e6de130138 iprep: WebRoot update versions: major version:1 minor version:1332 update version:107 total ips:2110941 last update time:1507090521 Oct 4 04:15:22 <local2.info> 82e6de130138 iprep: This update version doesn't have any new ip data. Oct 4 04:15:22 <local2.info> 82e6de130138 iprep: iprep_update_pe_cur_versions:430 updating PE with DB versions..
The database is stored in /var/nslog/iprep/iprep.db (the file mentioned in the logs is a temporary file and gets deleted immediately).

root@myNetScaler# ls -l total 88080 -rw-r--r-- 1 root wheel 90113024 Oct 4 04:20 iprep.db
So my reputation file is round about 90 MB in size. It’s a binary file, so there is no point in looking into it.

Thread categories

NetScaler has two built in functions:

IPREP_THREAT_CATEGORY(category)
IPREP_IS_MALICIOUS

While the later is a general one, the first one is very specific. There is a set of threat categories, and you have to specify the ones you’re interested in.

In a reverse proxy deployment you would filter malicious clients: CLIENT.IP.SRC. If you want to protect your clients from connecting to a malicious server you would rather filter potentially malicious server IPs: CLIENT.IP.DST

There are several thread categories (sources: Product documentation by BrightCloud, Citrix)

SPAM_SOURCES: The Spam Sources category includes tunneling spam messages through a proxy, anomalous SMTP activities, and forum spam activities.

WINDOWS_EXPLOITS: The Windows Exploits category includes active IP addresses offering or distributing malware, shell code, rootkits, worms or viruses.

WEB_ATTACKS: The Web Attacks category includes cross site scripting, iFrame injection, SQL injection, cross domain injection, or domain password brute force attack.

BOTNETS: The Botnets category includes Botnet C&C channels, and infected zombie machines controlled by Bot master.

SCANNERS: The Scanners category includes all reconnaissance such as probes, host scan, domain scan, and password brute force attack.

DOS: The Denial of Service category includes DOS, DDOS, anomalous sync flood, and anomalous traffic detection.

REPUTATION: The Reputation category denies access from IP addresses currently known to be infected with malware. This category also includes IPs with average low Webroot Reputation Index score. Enabling this category will prevent access from sources identified to contact malware distribution points.

PHISHING: The Phishing category includes IP addresses hosting phishing sites and other kinds of fraud activities such as ad click fraud or gaming fraud.

PROXY: The Proxy category includes IP addresses providing proxy services.

NETWORK: IPs providing proxy and anonymization services including The Onion Router aka TOR or darknet.

CLOUD_PROVIDERS: I didn’t find any information about this category. As far as I understood, this means, the IP belongs to a cloud provider like AWS, Azure, … So it does not indicate a malicious IP at all.

MOBILE_THREATS: I didn’t find any information about this category. It seems to be a collection of IPs harmful for mobile devices

How to use IP-Reputation service?

Usually I create responder policies with IP reputation feature.

Proxying outside

During proxying to outside I usually use responder policies redirecting to an error page, or respond with a predefined error message telling the user about the reason for blocking.

Action

add responder action res_act_block_malicious respondwith q{"HTTP/1.1 401 Requested URL not allowed\r\n\r\nYour URL had been blocked due to security concerns about target IP "+ CLIENT.IP.DST}

This action responds with a HTTP 401 (unauthorized) text telling the user about the problem. This will help both, user ans help desk, understanding what’s going on. I would not reset the connection (from technical point of view: send a TCP reset), as a user would not understand, what’s going on, nor would I drop the connection, which would be more or less the same, from perspective of a user, but by far slower (as there is no reply from the server).

Policy

add responder policy res_pol_block_malicious "CLIENT.IP.DST.IPREP_THREAT_CATEGORY(WINDOWS_EXPLOITS) || CLIENT.IP.DST.IPREP_THREAT_CATEGORY(PHISHING)" res_act_block_malicious

This policy checks if server’s IP is either known for spreading windows exploits or hosting sites used for phishing.

Proxying inside: a Reverse Proxy

if we proxy to inside (which is more common than load balancing outbound) we have several options: Responding with a HTML page (i.e. an error message), blocking or dropping.

Respond with an HTML page seems to be a good idea. My action would look simmilar to the action above. But it would inform the attacker about the reason why he is unable to connect and thereby what to do to connect even though (get an other IP).
Blocking in my opinion is stupid. What would you think if I would block you? My first guess would be: What can I do to be blocked no more? Maybe use a different IP? Use TOR network? So blocking does not seem to be the right thing!
Dropping is my choice. Using a well known bad IP my server would look like not up and running. Being a Black-Hat I’d give up (or invest a serious amount of time on examining the issue).

The policy

add responder policy res_pol_dropmalicious "CLIENT.IP.SRC.IPREP_THREAT_CATEGORY(DOS) || CLIENT.IP.SRC.IPREP_THREAT_CATEGORY(BOTNETS)" DROP -logAction log_drop_malicious

so this policy will drop a client request originating from either a well known source of DOS attacks, or from a well known bot network.

some concerns?

Services like IP reputation may tend to false positives. I’d strongly recommend to log, so you’ll be able to investigate issues. My policies therefore usually contain a logging policy.

But think of one of your main customers using an IP with bad reputation? You could ask WebRoot to change the reputation of this very IP address. However this does not work well and takes time. Webroot rates IPs for reason, so it’s very likely to reappear within some days. This may be an IP of a proxy also used by bad guys.

We could white list IPs by just combining the policy with
&& CLIENT.IP.SRC.IN_SUBNET(98.12.43.5/26).NOT

However this would lead to an endles list of exceptions, unreadable to humans, inefficient in NetScaler and unmanageable. So I would rather use a data set.

Data sets are lists of numbers, in this case: IP addresses. I use these lists to white list IP addresses.

add policy dataset Alowed_IP_List ipv4 bind policy dataset Alowed_IP_List 82.218.161.177 -index 1 ...
The policy would now look like that:
add responder policy res_pol_block_malicious "res_act_block_malicious((CLIENT.IP.DST.IPREP_THREAT_CATEGORY(WINDOWS_EXPLOITS) || CLIENT.IP.DST.IPREP_THREAT_CATEGORY(PHISHING)) && CLIENT.IP.SRC.TYPECAST_TEXT_T.CONTAINS_ANY(\"Alowed_IP_List\")).NOT"

So we now first check if the IP gets blocked by any of my IP reputation checks, and then if it’s not in my allow list.

===================== End