With Windows 8 we set out to enable all applications to have the beautiful and high-performance graphics enabled by modern graphics hardware. This work builds on the well-established foundations of DirectX graphics, which have been providing an increasing breadth of APIs and capabilities. In Windows 7, we expanded the capabilities of DirectX to provide a common hardware-accelerated graphics platform for a broader range of applications. Whereas previously, DirectX mainly provided 3-D graphics, we added functionality for what we call “mainstream” graphics. Mainstream uses center on the typical desktop applications most people find themselves using every day, including web browsers, email, calendars, and productivity applications. Windows 7 added two new components to DirectX: Direct2D for two-dimensional graphics (shapes, bitmaps, etc.) and DirectWrite for handling text. Both of these additions not only focused on performance but also on delivering high-quality 2-D rendering. With these additions, DirectX became a hardware-accelerated graphics platform for all types of applications. Indeed, we showed what a typical application could achieve by using DirectX when Internet Explorer 9 brought hardware-accelerated graphics to the web. WinRT bring these capabilities to the full range of new Windows 8 applications. In this post, authored by Rob Copeland the group program manager on our Graphics team, we look at the details behind the scenes in enabling this new class of graphical application. --StevenIn computer graphics, high performance is a guiding principle. In the early days of personal computing, discrete, add-on graphics cards were mostly focused on specialized applications such as CAD/CAM and gaming. Even early on, there was a view that all of this graphics horsepower could be used for more: notably a better user interface and experience. One of the first graphics cards for a PC was called a “Windows Accelerator” from S3 Graphics, which focused on the user experience by moving windows around the screen faster. As graphics hardware evolved, so, too, did the methods that developers use to interact with that hardware.DirectX is the part of Windows that provides a common application programming interface, or API, that allows developers to use the graphics hardware in the PC to draw text, shapes, and three-dimensional scenes, and display them on the screen. DirectX has also evolved over time in both capabilities and performance characteristics. In the early years, DirectX was focused mainly on games. As applications evolved to provide richer and more graphically-intense user experiences, many of them started to use DirectX as a way to get better performance and richer visuals.[h=3]Enter Windows 8[/h]When we started to plan the work we’d undertake for graphics in Windows 8, we knew that we would be creating a new, visually rich way for users to interact with apps and with Windows itself. We also knew that we’d be building a new platform for creating Metro style apps, and that we’d be targeting a more diverse set of hardware than ever before. While we had a great graphics platform to start with, there was more work to do in order to support those efforts. We came up with four main goals:
Along with text, we also made dramatic performance improvements for 2D geometry rendering. Geometry rendering is the core graphics technology that is used to create things like tables, charts, graphs, diagrams, and user interface elements, as shown in the example below. For Windows 8, our improvements in this area have primarily focused on delivering high-performance implementations of HTML5 Canvas and SVG technologies for use in Metro style apps, and webpages viewed with Internet Explorer 10.
The Weather app in Windows 8 uses geometry to display a graph of historical temperature and precipitation dataWhen Direct2D draws geometry, it takes instructions from the app about what to draw in the form of 2D figures (e.g. rectangles, ellipses, and paths), the size and location of the figures, and specifics about the style of rendering, including brush color and stroke style. Then it converts those instructions into a set of triangles and commands that it sends to Direct3D to generate the desired output. We call this conversion process tessellation.
To improve geometry rendering performance in Windows 8, we focused on reducing the CPU cost associated with tessellation in two ways.
First, we optimized our implementation of tessellation when rendering simple geometries like rectangles, lines, rounded rectangles, and ellipses. Below is a chart showing the impact of these improvements.
Second, to improve performance when rendering irregular geometry (e.g. geographical borders on a map), we use a new graphics hardware feature called Target Independent Rasterization, or TIR.
TIR enables Direct2D to spend fewer CPU cycles on tessellation, so it can give drawing instructions to the GPU more quickly and efficiently, without sacrificing visual quality. TIR is available in new GPU hardware designed for Windows 8 that supports DirectX 11.1.
Below is a chart showing the performance improvement for rendering anti-aliased geometry from a variety of SVG files on a DirectX 11.1 GPU supporting TIR:
We worked closely with our graphics hardware partners to design TIR. Dramatic improvements were made possible because of that partnership. DirectX 11.1 hardware is already on the market today and we’re working with our partners to make sure more TIR-capable products will be broadly available.
[h=4]Rendering images[/h]Images are widely used in a variety of scenarios including displaying user interfaces, webpages, and other app content. Websites commonly use JPEGs for pictures and PNG and GIF files to efficiently store user interface elements such as button graphics.
Working with digital photographs is also a very common activity on Windows. The number of digital photographs that Windows customers view and manipulate on their PCs continues to grow at an incredible rate.
We’ve made several performance improvements for working with images and photographs using the JPEG, GIF, and PNG formats.
For JPEG, improvements include:
For PNG, improvements include:
- Faster image decoding by expanding SIMD usage on all CPU architectures
- Faster Huffman decoding and encoding
In addition, we’ve improved pixel format conversion as well as image scaling. This results in faster decoding and rendering of images for all apps.
- Faster image decoding by expanding SIMD usage on all CPU architectures
- Faster image encoding and decoding by optimizing our zlib implementation
The video below uses a test app to measure the decoding and rendering time for a set of images. Windows 8 takes 40% less time than Windows 7 to render 64images (4.38 seconds vs. 7.28 seconds)
[h=4]Rendering and displaying[/h]As we evolve DirectX to support more mainstream scenarios, another area we we’ve invested in is optimizing how apps render and display their content. There are some big differences in how a 3D game draws its content and how a mainstream app such as Internet Explorer draws its content. For example, consider the video of the game below. In games like this, the entire scene changes rapidly. As the “camera” moves around the vehicle, the clouds move across the sky, and smoke billows up from the engine, the app must redraw the entire scene in each frame in order to achieve a life-like and engaging experience.
Your browser doesn't support HTML5 video.Now consider the webpage below. It has both a text article and a video. While the video plays, the browser must update the portion of the window containing the video but not the text. Additionally, if the user scrolls the page up, then we only need to render the new text at the bottom of the page. The rest of the text has already been rendered and simply needs to be moved.
Download this video to view it in your favorite media player:
High quality MP4 | Lower quality MP4
To improve apps that don’t need to redraw the entire screen for each frame, we optimized how DirectX deals with redrawing just portions of the screen and how it scrolls. This work not only improves app efficiency and performance, but since it reduces redundant drawing and reduces the number of times graphics data needs to be copied in memory, it also reduces power consumption, thus increasing battery life.
[h=4]Making the entire platform great[/h]All of these changes help Windows render experiences very quickly and smoothly. While we’ve talked mostly about features in DirectX, the great thing is that all of this work contributes to making our entire platform hardware-accelerated by default. Since we built the Metro style platform on top of DirectX, all apps take full advantage of the graphics hardware on the system, regardless of the programming language and framework the developer chooses.
[h=3]Creating stunning visual experiences with Direct2D and Direct3D[/h][h=4]Direct2D Effects[/h]Stylistic effects applied to images are becoming more common in modern user experiences. They can help highlight an area of an app, draw your attention to a specific part of the screen, or just make things look better. As we planned the graphics capabilities for Windows 8, we wanted to make it really easy for developers to apply these types of effects in their apps. We looked at two main areas where image processing would be useful:
To enable these types of experiences, we added “Direct2D Effects,” a new set of APIs that enable high-quality, hardware-accelerated effects to be applied to any image. Direct2D Effects have the following benefits:
- User interface images
The Metro style experience uses dynamic visuals. We wanted to enable Metro style apps to do image processing in real-time. This can range from 3D transition effects to perspective transforms, blurs, and highlights on user interface elements.- Photos
Apps that deal with photographs often want a rich set of image processing features. Effects such as adjusting exposure, brightness, and contrast, applying vibrancy and clarity, working with advanced curves, and applying lens corrections all allow these apps to enhance your digital memories.
Direct2D Effects power some of the new user experiences in Windows 8. For example, when tapping on a tile on the Start screen, the tile uses the 3D perspective transform effect to “tilt” in the right direction. They also power the rest of our platform. For example, SVG filter effects and CSS 3D transforms are implemented using Direct2D Effects.
- They provide optimal-quality renderings of image effects to suit the needs of wide variety of apps.
- The effects are hardware-accelerated and work on a wide variety of graphics hardware.
- A simple API enables great effects with minimal programming.
- They provide many built-in effects.
- They support large image sizes and up to 32 bits per channel.
- Custom effects can be combined with built-in effects or other custom effects.
[h=4]Direct3D 11.1 as a common foundation[/h]While adding new features like Direct2D Effects is a great way to help developers deliver new experiences, we also looked at ways to make it easier to use existing DirectX features.
Over years of development, we've added various different features to DirectX. Hardware acceleration of video decoding came alongside programmable shaders in Direct3D 9. In Windows 7, we added Direct2D and built it on top of Direct3D 10. At that time, we also created DirectCompute, a new system for high-performance computation on the GPU that became part of Direct3D 11. One result of all these updates is that DirectX has a very comprehensive set of features around graphics and GPU computation, but as a side effect, it has also become increasingly difficult to create an app that uses video, 2D graphics, 3D graphics, text, and DirectCompute together.
In Windows 8, the new Direct3D 11.1 API is the foundation for hardware acceleration of 2D graphics and text, image processing, 3D graphics and computation, and video. The new API makes it much simpler to mix different types of content in a single scene because that single API now manages all of the GPU resources associated with rendering. This also reduces memory usage by eliminating the redundancy involved in creating multiple graphics device-management objects in app code. In addition, Direct3D 11.1 provides a uniform way for apps to access the various capabilities of different graphics hardware. It provides mechanisms for the app to determine what features are available, and then only uses those capabilities. This enables apps to make maximum use of the GPU’s capabilities, whether the GPU was designed for long battery life on a tablet, or high-end gaming on a desktop PC.
[h=3]Diverse graphics hardware[/h]Historically, the expectations for each successive release of Windows have been that both the graphics platform and the graphics hardware capabilities will become richer and higher in performance. This is still true, as the graphics hardware industry continues to develop faster, more powerful GPUs. But in Windows 7, we started to see an inflection point in these assumptions, as the diversity of the hardware broadened with the introduction of mobile, low-power devices.
With Windows 8, this trend towards diverse hardware types is continuing and accelerating, both with new, high-performance graphics cards, and with an increasingly wide range of low-power mobile devices. The diversity of the hardware for Windows 8 will span a broader range than ever before; from graphics hardware that consumes on the order of 1 watt in always-connected tablets all the way up to high-end systems with multiple graphics cards that use a total of 1,000 watts or more. This broadening diversity brings with it new design considerations.
Our goal remains to provide visually compelling, high-performance experiences. With highly mobile devices, the primary power source is a battery, so we also need to maximize battery life. To meet both the performance and power consumption requirements of these new form factors, many of our graphics hardware partners have employed new GPU architectures.
[h=4]Low-power systems[/h]One of the graphics architectures commonly used in low-power system designs to achieve performance along with great battery life is called “tile-based rendering.” The general concept of a tile-based rendering approach is to have a very high performance (but small) memory cache that the graphics engine uses for rendering. The GPU then renders the screen in sections (or tiles) by repeatedly processing the same set of commands on each tile, rather than the whole screen at once. The intent is to minimize operations that use memory off-chip, therefore keeping power consumption low and performance high. Repeatedly accessing memory off-chip is expensive both in terms of time and power consumption.
To increase the efficiency of these tile-based architectures, we added a number of flags, hints, and new APIs that can minimize the number of times the tiles are rendered. We have incorporated the use of these into the Metro style app development platform to ensure greater efficiency in apps running on graphics hardware that uses a tile-based rendering architecture.
Another way for graphics hardware to reduce power consumption while still achieving great performance is to perform graphics rendering calculations using fewer bits of precision. This allows the GPU to more efficiently structure its data so that it can process more data simultaneously, thus reducing the power needed. For Windows 8, we added new mechanisms for apps to specify the amount of precision needed in their graphical calculations. For example, when doing custom blending of multiple images where the image data is 8 bits per component, the blending computations could be done with 10 bits of precision rather than the default of 32 bits. The reduced precision doesn’t impact image quality, but does reduce power consumption.
[h=3]Great performance, smoothly rendered[/h]Your browser doesn't support HTML5 video.As you can see, we’ve done a lot of work to enable a very fast and smoothly animated user experience in Windows 8. From new ways to measure our progress, to optimizations for mainstream uses of our graphics platform, and new hardware features, we’ve created the best Windows graphics platform yet. And of course, we continue to push the envelope on immersive, three-dimensional gaming, with great performance and new features such as stereoscopic 3D.
Download this video to view it in your favorite media player:
High quality MP4 | Lower quality MP4
From high-end gaming rigs to light-weight, always-connected tablets, Windows 8 supports the broadest range of graphics hardware ever in a single operating system. We hope this post has helped explain some ways in which this work enables a whole new set of rich experiences.
- Rob Copeland
P.S. Thanks to Sriram Subramanian, Dan McLachlan, Kam VedBrat, Steve Lim, and Jianye Lu, for their substantial contributions to this blog post.
More...
