Understanding the performance impact of Spectre and Meltdown mitigations on Windows Systems

Last week the technology industry and many of our customers learned of new vulnerabilities in the hardware chips that power phones, PCs and servers. We (and others in the industry) had learned of this vulnerability under nondisclosure agreement several months ago and immediately began developing engineering mitigations and updating our cloud infrastructure. In this blog, Ill describe the discovered vulnerabilities as clearly as I can, discuss what customers can do to help keep themselves safe, and share what weve learned so far about performance impacts.

What Are the New Vulnerabilities?

On Wednesday, Jan. 3, security researchers publicly detailed three potential vulnerabilities named Meltdown and Spectre. Several blogs have tried to explain these vulnerabilities further a clear description can be found via Stratechery.

On a phone or a PC, this means malicious software could exploit the silicon vulnerability to access information in one software program from another. These attacks extend into browsers where malicious JavaScript deployed through a webpage or advertisement could access information (such as a legal document or financial information) across the system in another running software program or browser tab. In an environment where multiple servers are sharing capabilities (such as exists in some cloud services configurations), these vulnerabilities could mean it is possible for someone to access information in one virtual machine from another.

What Steps Should I Take to Help Protect My System?

Currently three exploits have been demonstrated as technically possible. In partnership with our silicon partners, we have mitigated those through changes to Windows and silicon microcode.

Exploited Vulnerability CVE Exploit
Name
Public Vulnerability Name Windows Changes Silicon Microcode Update ALSO Required on Host
Spectre 2017-5753 Variant 1 Bounds Check Bypass Compiler change; recompiled binaries now part of Windows Updates

Edge & IE11 hardened to prevent exploit from JavaScript

No
Spectre 2017-5715 Variant 2 Branch Target Injection Calling new CPU instructions to eliminate branch speculation in risky situations Yes
Meltdown 2017-5754 Variant 3 Rogue Data Cache Load Isolate kernel and user mode page tables No

 

Because Windows clients interact with untrusted code in many ways, including browsing webpages with advertisements and downloading apps, our recommendation is to protect all systems with Windows Updates and silicon microcode updates.

For Windows Server, administrators should ensure they have mitigations in place at the physical server level to ensure they can isolate virtualized workloads running on the server. For on-premises servers, this can be done by applying the appropriate microcode update to the physical server, and if you are running using Hyper-V updating it using our recent Windows Update release. If you are running on Azure, you do not need to take any steps to achieve virtualized isolation as we have already applied infrastructure updates to all servers in Azure that ensure your workloads are isolated from other customers running in our cloud. This means that other customers running on Azure cannot attack your VMs or applications using these vulnerabilities.

Windows Server customers, running either on-premises or in the cloud, also need to evaluate whether to apply additional security mitigations within each of their Windows Server VM guest or physical instances. These mitigations are needed when you are running untrusted code within your Windows Server instances (for example, you allow one of your customers to upload a binary or code snippet that you then run within your Windows Server instance) and you want to isolate the application binary or code to ensure it cant access memory within the Windows Server instance that it should not have access to. You do not need to apply these mitigations to isolate your Windows Server VMs from other VMs on a virtualized server, as they are instead only needed to isolate untrusted code running within a specific Windows Server instance.

We currently support 45 editions of Windows. Patches for 41 of them are available now through Windows Update. We expect the remaining editions to be patched soon. We are maintaining a table of editions and update schedule in our Windows customer guidance article.

Silicon microcode is distributed by the silicon vendor to the system OEM, which then decides to release it to customers. Some system OEMs use Windows Update to distribute such microcode, others use their own update systems. We are maintaining a table of system microcode update information here. Surface will be updated through Windows Update starting today.

 

Guidance on how to check and enable or disable these mitigations can be found here:

Performance

One of the questions for all these fixes is the impact they could have on the performance of both PCs and servers. It is important to note that many of the benchmarks published so far do not include both OS and silicon updates. Were performing our own sets of benchmarks and will publish them when complete, but I also want to note that we are simultaneously working on further refining our work to tune performance. In general, our experience is that Variant 1 and Variant 3 mitigations have minimal performance impact, while Variant 2 remediation, including OS and microcode, has a performance impact.

Here is the summary of what we have found so far:

  • With Windows 10 on newer silicon (2016-era PCs with Skylake, Kabylake or newer CPU), benchmarks show single-digit slowdowns, but we dont expect most users to notice a change because these percentages are reflected in milliseconds.
  • With Windows 10 on older silicon (2015-era PCs with Haswell or older CPU), some benchmarks show more significant slowdowns, and we expect that some users will notice a decrease in system performance.
  • With Windows 8 and Windows 7 on older silicon (2015-era PCs with Haswell or older CPU), we expect most users to notice a decrease in system performance.
  • Windows Server on any silicon, especially in any IO-intensive application, shows a more significant performance impact when you enable the mitigations to isolate untrusted code within a Windows Server instance. This is why you want to be careful to evaluate the risk of untrusted code for each Windows Server instance, and balance the security versus performance tradeoff for your environment.

For context, on newer CPUs such as on Skylake and beyond, Intel has refined the instructions used to disable branch speculation to be more specific to indirect branches, reducing the overall performance penalty of the Spectre mitigation. Older versions of Windows have a larger performance impact because Windows 7 and Windows 8 have more user-kernel transitions because of legacy design decisions, such as all font rendering taking place in the kernel. We will publish data on benchmark performance in the weeks ahead.

Conclusion

As you can tell, there is a lot to this topic of side-channel attack methods. A new exploit like this requires our entire industry to work together to find the best possible solutions for our customers. The security of the systems our customers depend upon and enjoy is a top priority for us. Were also committed to being as transparent and factual as possible to help our customers make the best possible decisions for their devices and the systems that run organizations around the world. Thats why weve chosen to provide more context and information today and why we released updates and remediations as quickly as we could on Jan. 3. Our commitment to delivering the technology you depend upon, and in optimizing performance where we can, continues around the clock and we will continue to communicate as we learn more.

-Terry

from Jenny Erie

Advertisements

Application fuzzing in the era of Machine Learning and AI

Proactively testing software for bugs is not new. The earliest examples date back to the 1950s with the term fuzzing. Fuzzing as we now refer to it is the injection of random inputs and commands into applications. It made its debut quite literally on a dark and stormy night in 1988. Since then, application fuzzing has become a staple of the secure software development lifecycle (SDLC), and according to Gartner*, security testing is growing faster than any other security market, as AST solutions adapt to new development methodologies and increased application complexity.

We believe there is good reason for this. The overall security risk profile of applications has grown in lockstep with accelerated software development and application complexity. Hackers are also aware of the increased vulnerabilities and, as the recent Equifax breach highlights, the application layer is highly targeted. Despite this, the security and development groups within organizations cannot find easy alignment to implement application fuzzing.

While DevOps is transforming the speed at which applications are created, tested, and integrated with IT, that same efficiency hampers the ability to mitigate identified security risks and vulnerabilities, without impacting business priorities. This is exactly the promise that machine learning, artificial intelligence (AI), and the use of deep neural networks (DNN) are expected to deliver on in evolved software vulnerability testing.

Most customers I talk to see AI as a natural next step given that most software testing for bugs and vulnerabilities is either manual or prone to false positives. With practically every security product claiming to be machine learning and AI-enabled, it can be hard to understand which offerings can deliver real value over current approaches.

Adoption of the latest techniques for application security testing doesnt mean CISOs must become experts in machine learning. Companies like Microsoft are using the on-demand storage and computing power of the cloud, combined with experience in software development and data science, to build security vulnerability mitigation tools that embed this expertise in existing systems for developing, testing, and releasing code. It is important, however, to understand your existing environment, application inventory, and testing methodologies to capture tangible savings in cost and time. For many organizations, application testing relies on tools that use business logic and common coding techniques. These are notoriously error-prone and devoid of security expertise. For this latter reason, some firms turn to penetration testing experts and professional services. This can be a costly, manual approach to mitigation that lengthens software shipping cycles.

Use cases

Modern application security testing that is continuous and integrated with DevOps and SecOps can be transformative for business agility and security risk management. Consider these key use cases and whether your organization has embedded application security testing for each:

  • Digital Transformation moving applications to the cloud creates the need to re-establish security controls and monitoring. Fuzzing can uncover errors and missed opportunities to shore up defenses. Automated and integrated fuzzing can further preserve expedited software shipping cycles and business agility.
  • Securing the Supply Chain Open Source Software (OSS) and 3rd party applications are a common vector of attack, as we saw with Petya, so a testing regimen is a core part of a plan to manage 3rd party risk.
  • Risk Detection whether building, maintaining, or refactoring applications on premises, the process and risk profile have become highly dynamic.Organizations need to be proactive to uncover bugs, holes and configuration errors on a continuous basis to meet both internal and regulatory risk management mandates.

Platform leverage

Of course, software development and testing are about more than just tools. The process to communicate risks to all stakeholders, and to act, is where the real benefit materializes. A barrier to effective application security testing is the highly siloed way that testing and remediation are conducted. Development waits for IT and security professionals to implement the changesslowing deployment and time to market. Legacy application security testing is ready for disruption and the built-in approach can deliver long-awaited efficiency in the development and deployment pipeline. Digital transformation, supply chain security, and risk detection all benefit from speed and agility. Lets consider the DevOps and SecOps workflows possible on a Microsoft-based application security testing framework:

  • DevOps Continuous fuzzing built into the DevOps pipeline identifies bugs and feeds them to the continuous integration and deployment environment (i.e. Visual Studio Team Services and Team Foundation Server). Developers and stakeholders are uniformly advised of risky code and provided the option of running additional Azure-based fuzzing techniques. For apps in production that are found to be running risky code, IT pros can mitigate risks by using PowerShell and Group Policy (GPO) to enable the features of Windows Defender Exploit Guard. While the apps continue to run, the attack surface can be reduced, and connection scenarios which increase risk are blocked. This gives teams time to develop and implement mitigations without having to take the applications entirely offline.
  • SecOps – Azure-hosted containers and VMs, as well as on-premise machines, are scanned for risky applications and code including OSS. The results inform Microsofts various desktop, mobile, and server threat protection regimes, including application whitelisting. Endpoints can be scanned for the presence of the risky code and administrators are informed through Azure Security Center. Mitigations can also be deployed to block those applications implicated and enforce conditional access through Azure Active Directory.

Cloud and AI

Machine learning and artificial intelligence are not new, but the relatively recent availability of graphics processing units (GPUs) have brought their potential to mainstream by enabling faster (parallel) processing of large amounts of data. Our recently announced Microsoft Risk Detection (MSRD) service is a showcase of the power of the cloud and AI to evolve fuzz testing. In fact, Microsofts award winning work in a specialized area of AI called constraint solving has been 10 years in the making and was used to produce the worlds first white-box fuzzer.

A key to effective application security testing is the inputs or seeds used to establish code paths and bring about crashes and bug discovery. These inputs can be static and predetermined, or in the case of MSRD, dynamic and mutated by training algorithms to generate relevant variations based on previous runs. While AI and constraint solving are used to tune the reasoning for finding bugs, Azure Resource Manager dynamically scales the required compute up or down creating a fuzzing lab that is right-sized for the customers requirement. The Azure based approach also gives customers choices in running multiple fuzzers, in addition to Microsofts own, so the customer gets value from several different methods of fuzzing.

The future

For Microsoft, application security testing is fundamental to a secure digital transformation. MSRD for Windows and Linux workloads is yet another example of our commitment to building security into every aspect of our platform. While our AI-based application fuzzing is unique, Microsoft Research is already upping the ante with a new project for neural fuzzing. Deep neural networks are an instantiation of machine learning that model the human brain. Their application can improve how MSRD identifies fuzzing locations and the strategies and parameters used. Integration with our security offerings is in the initial phases, and by folding in more capabilities over time we remove the walls between IT, developers, and security, making near real-time risk mitigation a reality. This is the kind of disruption that, as a platform company, Microsoft uniquely brings to application security testing for our customers and serves as further testament for the power of built-in.


* Gartner: Magic Quadrant for Application Security Testing published: 28 February 2017 ID: G00290926

from Jenny Erie

[SANS ISC] 2017, The Flood of CVEs

I published the following diary on isc.sans.org: “2017, The Flood of CVEs“:

2017 is almost done and it’s my last diary for this year. I made a quick review of my CVE database (I’m using a local cve-search instance). The first interesting number is the amount of CVE’s created this year. Do you remember when the format was CVE-YYYY-XXXX? The CVE ID format changed in 2014 to break the limit of 9999 entries per year. This was indeed a requirement when you see the number of entries for the last five years… [Read more]

[The post [SANS ISC] 2017, The Flood of CVEs has been first published on /dev/random]

from Xavier

Who’s That Bot?

If you own a website, you already know that servers are visited all day long by bots and crawlers with multiple intents, sometimes good but also sometimes bad. An interesting field in web server logs is the “user-agent”. The RFC 2616 describes the User-Agent field used in HTTP requests:

The User-Agent request-header field contains information about the user agent originating the request. This is for statistical purposes, the tracing of protocol violations, and automated recognition of user agents for the sake of tailoring responses to avoid particular user agent limitations. User agents SHOULD include this field with requests. The field can contain multiple product tokens (section 3.8) and comments identifying the agent and any subproducts which form a significant part of the user agent. By convention, the product tokens are listed in order of their significance for identifying the application.

It’s always interesting to keep an eye on the User-Agents found in your logs even if often they are spoofed. It can indeed contain almost anything. Note that many websites trust the User-Agent to display some content in different ways depending on the browser, the operating system. During an old pentest engagement, I also saw an authentication bypass via a specific User-Agent… which is bad! That’s why there exists a tool to stress test a website with multiple variations of User-Agent strings: ua-tester.

Most tools and browsers allow selecting the User-Agent via a configuration file or a plugin (for Browsers). The choice of a User-Agent string may vary depending on how your goal. During a penetration test, you’ll try to work below the radar by using a very common User-Agent (a well-known browser on a modern OS):

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) \
  Chrome/63.0.3239.84 Safari/537.36

Sometimes, you will change the User-Agent to detect behaviour changes (ex: to access the mobile version of a website).

Mozilla/5.0 (Linux; Android 4.4.4; SAMSUNG SM-G318H Build/KTU84P) AppleWebKit/537.36 \
  (KHTML, like Gecko) SamsungBrowser/2.0 Chrome/34.0.1847.76 Mobile Safari/537.36

Finally, sometimes it’s better to show clean hands!

For my researches and while hunting, I’m fetching a *lot* of data from websites. Sometimes, I’m accessing the same pages again and again. This behaviour can be seen as intrusive by the website owner. In this case, it’s always better to be polite and to present yourself. In my scripts, I’m always using the following User-Agent:

XmeBot/1.0 (https://blog.rootshell.be/bot/)

The URL is hidden on my blog and is available to provide more information about me and my intents (basically, why I’m fetching data). By keeping an eye on the page access statistics, it also helps me to learn who’s keeping an eye on their website logs 😉

[The post Who’s That Bot? has been first published on /dev/random]

from Xavier

Installing Python Modules on Air-Gapped Hosts

Who said that all computers are connected today? They are many classified environments where computers can simply never connect to the wild Internet. But sometimes, you need to install some pieces of software from online resources. The classic case is Python modules. Let’s take a practical example with the PyMISP which allows interacting with a MISP instance. Just forget to make a ‘pip install pymisp’ on an air-gapped computer!

The next challenge is to resolve all the dependencies. On an air-gapped host, to make PyMISP work properly, I had to follow this dependency tree (this might change depending on your Python environment):

pymisp -> dateutil -> six
       -> requests -> idna
                   -> urllib3
                   -> chardet
                   -> certify

If you need to download and transfer these packages manually from source and build them, you will probably face another issue: the air-gapped environment does not have all tools to build the package (sounds logical). There is a solution to solve this: “Wheel”. It’s a built, archive format that can greatly speed installation compared to building and installing from source archives. A wheel archive (.whl) can be installed with the pip tool:

C:\Temp> pip install .\requests‑2.18.4‑py2.py3‑none‑any.whl

I found a wonderful source of ready-to-use Wheel archives prepared by Christoph Gohlke from the University of California, Irvine. He maintains a huge library of common Python packages (2 & 3):

If you’re working in a restricted environment, you probably don’t have admin rights. Add the ‘–user’ argument to the pip commands to install the Python module in your $USER environment.

Keep in mind:

  • Some packages might contain executable code. Don’t break your local policy!
  • Always download material from trusted sources
  • Validate hashes
  • Build your own Wheel archives if in doubt and create your own repository

[The post Installing Python Modules on Air-Gapped Hosts has been first published on /dev/random]

from Xavier