VMware ESXi (6.0 before ESXi600-201711101-SG, 5.5 ESXi550-201709101-SG), Workstation (12.x before 12.5.8), and Fusion (8.x before 8.5.9) contain a vulnerability that could allow an authenticated VNC session to cause a stack overflow via a specific set of VNC packets. Successful exploitation of this issue could result in remote code execution in a virtual machine via the authenticated VNC session. Note: In order for exploitation to be possible in ESXi, VNC must be manually enabled in a virtual machine's .vmx configuration file. In addition, ESXi must be configured to allow VNC traffic through the built-in firewall.
https://www.vmware.com/security/advisories/VMSA-2017-0021.html