CVE-2019-19781: Critical Vulnerability in Citrix ADC and Gateway Sees Active Exploitation While Patches are Still Not Available

Following the release of exploit scripts for a critical flaw in Citrix Application Delivery Controller (ADC) and Gateway, attackers launch attacks against vulnerable hosts, while Citrix announces release date for patches

UPDATE 01/24/2020: This blog post has been updated to reflect the availability of patches released by Citrix.

Background

Attacks Increase After Exploit Scripts Released

On January 10, Tenable Security Response observed exploit scripts for CVE-2019-19781, a critical vulnerability in Citrix ADC and Gateway (formerly known as NetScaler ADC and NetScaler Gateway) had been published to GitHub. Soon after, reports of increased exploitation attempts against vulnerable hosts emerged.

#Citrix ADC Exploits are Public / Heavily Used (some attempt to install #backdoor). #cve201919781 https://t.co/5TN4mfpdTh pic.twitter.com/WDN6wUm4Km

— SANS ISC (@sans_isc) January 11, 2020

According to SANS Internet Storm Center, the released exploit scripts have been “heavily used,” as they observed a spike in exploitation attempts against their honeypots.

Citrix Updates Support Article with Patch Release Dates

On January 11, Citrix updated their support article for the vulnerability, announcing plans to release patches for Citrix ADC and Gateway near the end of January.

Thousands of Citrix ADC and Gateway Endpoints Remain Vulnerable

On January 12, Troy Mursch, chief research officer at Bad Packets published a blog with statistics showing that over 25,000 Citrix ADC and Gateway endpoints were vulnerable to CVE-2019-19781. Mursch used BinaryEdge to scan over 60,000 endpoints. Vulnerable endpoints included those in government agencies, education, healthcare, utilities, banking, and “numerous” Fortune 500 companies.

At that time, Mursch found that there were vulnerable hosts in over 122 countries, which were outlined in a graphic:

Source: Over 25,000 Citrix (Netscaler) Endpoints Vulnerable to CVE-2019-19781 (Troy Mursch)

Attackers Can Obtain LDAP Passwords, Cookies from Vulnerable Hosts

On January 13, Rich Warren, principal security consultant at NCC Group, identified additional avenues of exploitation for attackers targeting vulnerable ADC and Gateway hosts. According to Warren, attackers have the capability to read the /flash/nsconfig/ns.conf file, which could contain hashed Active Directory/Lightweight Directory Access Protocol (LDAP) credentials, including SHA512 passwords which are “easily crackable” using hashcat.

CVE-2019-19781 post-exploitation notes:

If you are seeing attackers reading your /flash/nsconfig/ns.conf file then you need to change all passwords. The SHA512 passwords are easily crackable with hashcat. pic.twitter.com/mNMaTT1oCE

— Rich Warren (@buffaloverflow) January 13, 2020

Additionally, Warren notes that attackers could access authenticated cookies from the path “/var/stmp/sess_*” which according to Warren can be reused by attackers.

If you see the attacker reading /var/nstmp/sess_* then they just stole authenticated cookies which can be re-used

— Rich Warren (@buffaloverflow) January 13, 2020

On January 14, security researcher dozer published a blog detailing how one could decrypt values obtained from the Citrix configuration files, along with providing a Python script to perform the decryption.

On the same day, hashcat was updated with support for cracking the SHA512 hashes obtained from Citrix Netscaler in version 6.0.0.

Support added to crack Citrix NetScaler (SHA512) hashes with hashcat 6.0.0: https://t.co/OtKRrGuzX4 pic.twitter.com/Qr9nc2Avy4

— hashcat (@hashcat) January 14, 2020

Dutch Cybersecurity Centre Advises Shutting Down ADC and Gateway Servers, As Mitigation Steps Ineffective In Some Instances

On January 14, the Dutch National Cybersecurity Centre (NCSC) warned that many Dutch Citrix servers were vulnerable to attacks. On January 16, they published a follow-up saying the measures recommended by Citrix are “not always effective” as some of the mitigation steps don’t appear to work for certain devices.

Doubling down, the NCSC stressed that until a patch is available “there is currently no good, guaranteed reliable solution for all Citrix ADC and Citrix Gateway servers.” As a result, the NCSC made the recommendation to consider “switching off the Citrix ADC and Gateway servers” if the impact is acceptable. If not, they advise close monitoring for “possible abuse.”

Vigilante Applies Mitigation to Vulnerable ADC and Gateway Hosts, Maintains Backdoor

On January 16, researchers at FireEye published a blog regarding a peculiar observation in exploitation attempts against vulnerable ADC and Gateway hosts. According to FireEye, they’ve identified a threat actor that is blocking attempts to exploit the vulnerability, cleaning up previous malware infections on affected hosts, while also deploying their own backdoor called NOTROBIN.

The blog mentions that during one engagement, FireEye observed multiple threat actors had launched successful attacks against a vulnerable host. However, once NOTROBIN was installed on the host, they identified “more than a dozen exploitation attempts were thwarted by NOTROBIN.” FireEye concludes that the actor behind NOTROBIN compromising these devices may be doing so as they “prepare for an upcoming campaign.”

Proof of concept

There are currently several exploit scripts available on GitHub from TrustedSec, ProjectZeroIndia, and mpgn, as well as a script to check for vulnerable hosts from the United States Cybersecurity and Infrastructure Security Agency (CISA).

Vendor response

Since the publication of exploit scripts, Citrix has updated their support article multiple times, providing a timeline for patches, as well as additional information about newly affected products and a bug in certain releases of ADC that impacts the mitigation steps.

Following an investigation, Citrix says that CVE-2019-19781 also affects “certain deployments of Citrix SDWAN,” specifically the Citrix SD-WAN WANOP because they package it with Citrix ADC “as a load balancer.”

In addition to the newly affected product, Citrix has identified a bug affecting version 12.1 of Citrix ADC and Gateway that prevents their mitigation steps from blocking exploit attempts. According to Citrix, version 12.1 builds prior to 51.16/51.19 and 50.31 contain a bug that “affects responder and rewrite policies bound to VPN virtual servers.” If packets match policy rules the bug will prevent these systems from processing the packets. For those customers running a build of 12.1 prior to 51.16/51.19 or 50.31, it is recommended to update to a newer build in order to apply the mitigation steps until the patch is released on January 27.

Solution

At the time this blog was published, patches were still not available for this vulnerability. However, Citrix says they plan to release patches for Citrix ADC, NetScaler Gateway and SD-WAN WANOP before the end of January 2020.

The following table contains expected release dates of “refresh builds” for Citrix ADC and Gateway:

Separately, the following table contains expected release dates of the NetScaler releases for Citrix SD-WAN WANOP:

Identifying affected systems

Tenable Research has released a direct check plugin (ID 132752) to identify vulnerable assets in addition to the our version check plugin (ID 132397), which can be found here. Note that the version check plugin (ID 132397) requires enabling 'paranoid mode'.

Get more information

• Tenable's Previous Blog on Availability of Exploit Scripts for CVE-2019-19781
• Citrix releases first patches - timeline accelerated
• Citrix releases second patches - timeline further accelerated
• Citrix Security Advisory for CVE-2019-19781 (CTX267027)
• Citrix Mitigation Steps for CVE-2019-19781 (CTX267679)
• Dozer blog on Decrypting Values from ADC and Gateway Configurations
• FireEye Blog on Threat Actor "NOTROBIN" Exploiting CVE-2019-19781

Join Tenable's Security Response Team on the Tenable Community.

Learn more about Tenable, the first Cyber Exposure platform for holistic management of your modern attack surface.

Get a free 30-day trial of Tenable.io Vulnerability Management.