Nuno Filipe Godinho

MIX’09 - Reviewing – Deep Dive into Microsoft Silverlight Graphics

Seema Ramchandani

Overview

• < 1 year from Silverlight 2 release
• Scenarios:
  • Customer adoption blockers
  • Ease key customer pain points
  • Engage in the enterprise space
  • Continue media momentum
• Final Release will be at Summer 2009

In order to work in the “Customer adoption blockers” Microsoft defined 3 elements to be worked that were:

• Effects
• Projection
• Hardware (GPU usage)

How we draw or how is Silverlight is Drawing elements

• Threads
  • UI Thread
    • User defined Code
    • Control Code
    • Animations
    • Layout
  • Non-UI Thread
    • Frame Rasterization
    • Media Decoding
    • GPU work
• Draw loop
  • By default happens 60 time a second
  • Elements (in Loop)
    • Event Handlers (Muscles of the Application)
    • Ticks (By default happens every 33 msec)
    • Any Property changes?
    • Layout
      • Pool for dirty flags
      • This will be changed from Pool version to Push in the RTM version
      • 2 Passes like in WPF:
        • Measure
          • Check the size needed based on the children that are part of the control
        • Arrange
          • After having got the size arrange the layout
      • Design for nesting
        • In the situation where you don’t know the panel layout were you are going have your control placed, you can opt-in to Measure and Arrange or not.
    • Queue up rendering changes
    • Rasterize in back buffer
      • Analyzing all changes at a time
    • Hey Browser, Show me the Frame
  • Draw Steps (same as in WPF) in SL2
    1. Children
    2. Opacity Mask
    3. Opacity
    4. Clip
    5. RenderTransform
    6. Layout Offset (Internat RenderTransform)
    7. Parent Node
  • Draw Steps in SL3
    1. Children
    2. Hardware Cache
    3. OpacityMask
    4. Opacity
    5. Effect
    6. Clip
    7. Projection
    8. Render Transform
    9. Layout Offset
    10. ParentNode
• All three new concepts work on an Intermediate surface
  • In there the effects will be applied
  • Use the RenderToIntermediate

RenderToIntermediate

• Effects
  • Two types of effects
    • Built-in and Multi-pass
      • Blur Effect
      • DropShadow Effect
    • Custom Effect
      • Compiled HLSL pixel shader (2.0)
      • .NET Class invokes the shader
  • Effects can pull content from any brush
  • SL’s Shaders Engine
    • Consumes .ps files
    • Registers available:
      • 16 textures
      • 32 constants
  • Pixel Shaders 2.0
    • DX SDK http://bit.ly/DXsdk
      • fxc.exe myshader.fx /Tps_2_0
    • WPF Build Task: http://bit.ly/buildtask
  • Performance Overview
    • Blur and Dropshadow
      • When done, nullify UIElement.Effect
      • Smaple all pixels
      • BlurRadius impacts how wide the sample
    • Custom Effects
      • Math is easy
      • Pushing around memory is time-costly
    • Effects can pull content from any brush
      • Effects do not cache output
      • Change a pixel, run again
• Writable Bitmap
  • Way to replicate content
  • A cache of the rendering pipeline
  • In RTM it will be possible to save the output of the rendering
  • A way to draw whatever you like
• Projection
  • SLIdentity – Slide Builder tool in Silverlight that uses Projection
  • GlobalOffset Property is relative to the plane – Z axis
  • LocalOffset Property is relative to the projection
  • Layout Properties applied before:
    • Internal LayoutClip
  • Layout Properties at the end
    • Panel.Zindex
    • Internal Layout Offset
• GPU Caching
  • Also called Hardware Cache
  • Silverlight 3 GPU Acceleration
    • 3 Properties can be HW accelerated
      • Render Transforms
      • Rectangular Clipping
      • Blending
    • Platform avoid shuttling back and forth from the SW to the HW
    • Fallback to SW
      • Software-only path above the Cache point
  • Hardware Acceleration
    • Plugin Setting EnableGPUAcceleration
    • UIElement.CacheMode Property set to BitmapCache
      • API: “Cache” element as a bitmap
      • Behavior: GPU handles Transforms, Blends and Clips
      • Ex:GPU handles Media shrink or stretch
    • Support:
      • Windows: Browser & Fullscreen (all DX9.0) – don’t have support for Windows Server 2000
      • MAC: Fullscreen
  • Enhancing Media with GPU
    • Is the MediaElement doing any of these?
      • Streching
      • Rectangular clip
      • Blending via Opacity
    • If so, then:
      • Set the Properties
        • PlugIn: EnableGPUAcceleration=true
        • MediaElement: CacheMode=BitmapCache
        • Turn on EnableCacheVisualization
    • You can use XPerf to analyze the performance of the accelarations. Read how to use it in Siverlight
    • Test Results:
      • 360p Silverlight H.264 Video Stretched to FS
        • XP laptop 2004 with 1.3 GHz and Single Core
          • CPU w/ SW Stretch = 70% CPU
          • CPU w/ HW Stretch = 32% CPU
          • Difference = –55%
        • Vista desktop 2004 with 2.4 GHz and Single Core
          • CPU w/ SW Stretch = 88% CPU
          • CPU w/ HW Stretch = 44% CPU
          • Difference = –50%
        • Vista laptop 2007 with 2.3 GHz and Core 2 Duo
          • CPU w/ SW Stretch = 60% CPU
          • CPU w/ HW Stretch = 24% CPU
          • Difference = –60%
      • 540p Silverlight H.264 Video Stretched to FS
        • XP laptop 2004 with 1.3 GHz and Single Core
          • CPU w/ SW Stretch = 69% CPU
          • CPU w/ HW Stretch = 59% CPU
          • Difference = –15%
        • Vista laptop 2007 with 2.3 GHz and Core 2 Duo
          • CPU w/ SW Stretch = 71% CPU
          • CPU w/ HW Stretch = 37% CPU
          • Difference = –47%