Download Article

Download Deforming Terrain Through Touch [PDF 888KB]

Introduction

Dynamically deformable terrain has long been a desired feature of game developers. Ultrabook™ devices and Windows 8* tablets open new attractive opportunities for interactivity, where touch screens and other sensors can be used to influence the terrain in new ways.

This sample demonstrates a technique that enables users to dynamically deform terrain using a touch screen. The two main goals of the sample are to demonstrate how to leverage the complete Ultrabook platform to render fully dynamic textured terrain and how to exploit touch screens in Windows 8 to enrich the user experience. Figure 1 shows the sample screen shot.

Figure 1: Screen Shot of the Deformable Terrain with Touch Screen sample

This sample is based on the previously published SNB terrain sample. It exploits the same hierarchical quad tree data structure, triangulation algorithm, and compact triangulation encoding scheme. Please refer to the original sample for more information on these topics. The following features are new to this release:

• Touch screen is exploited for greater user interaction
• Asynchronous terrain update operations (height map displacement, re-triangulation, etc.)
• Layer-based texturing improves visual fidelity

These features are described in detail in the following sections.

Using the touch screen

Handling WM_TOUCH windows message

When the user touches the screen, a WM_TOUCH message is generated by the system. When this message is handled by the windows procedure, the low-order word of wParam contains the number of touch points associated with the message and lParam contains a touch input handle that can be used in a call to GetTouchInputInfo() to retrieve detailed information about the touch points. After the message has been processed, the handle must be freed by a call to CloseTouchInputHandle(). Please refer to MSDN for more detailed information. The following code snippet handles touch messages and adds the screen coordinates of the touch point to the list of pending points (see CTouchTerrainSample::CPUTHandleWindowsEvent()).

case WM_TOUCH:
    {
        UINT numInputs = LOWORD(wParam);
        static std::vector<TOUCHINPUT> pInputs;
        pInputs.resize(numInputs);
        if (GetTouchInputInfo((HTOUCHINPUT)lParam, numInputs, &pInputs[0], sizeof(TOUCHINPUT)))
        {
            for(int iTap=0; iTap<(int)numInputs; ++iTap)
            {
                // Convert the touch point into window coords
                POINT pt = { pInputs[iTap].x/100, pInputs[iTap].y/100 };
                MapWindowPoints(NULL, hWnd, &pt, 1);
                m_NewTapPositions.push_back( std::make_pair(pt.x, pt.y) );
            }
            CloseTouchInputHandle( (HTOUCHINPUT)lParam );
        }
    }
    break;

Reconstructing world space positions of touched points

To reconstruct the world space position of the point being touched, we have to transform the point’s projection space coordinates by the world-view-projection inverse matrix. Projection space x and y coordinates can be derived from the screen position as shown below:

PosPS.x = -1.f + 2.f * ((float)xPos + 0.5f) / (float)windowWidth;
PosPS.y =  1.f - 2.f * ((float)yPos + 0.5f) / (float)windowHeight;

To get the z coordinate, we need to access the depth buffer. Directly reading data from the GPU-residing depth buffer would require CPU-GPU synchronization, which would cause performance degradation. To eliminate this, the depths of points being touched are first copied to temporary 1×1 staging textures, one texture for each point. Texture cache is used to create and manage these auxiliary textures. On subsequent frames we attempt to map these textures using the D3D11_MAP_FLAG_DO_NOT_WAIT flag to eliminate stalls. If the map operation succeeds, the CTouchTerrainSample::ModifyTerrain() method is called to modify the terrain at the selected locations. Note that the world-view-projection matrix of the frame on which the touch event happened, not necessarily the matrix from the current frame, must be used to reconstruct world position. If the map operation fails, the map attempt is repeated on the next frame. CTouchTerrainSample::ProcessPendingModifications() method performs the steps described above. The process of reconstructing a touch point’s world coordinates is illustrated in Figure 2: Reconstructing touch point world coordinates.

Figure 2: Reconstructing touch point world coordinates

In DirectX* 11, only the whole depth buffer can be copied. As a result, we cannot directly copy the required depth value to a 1×1 texture. To get around this limitation, we first copy the depth buffer to a resource having the same dimensions and format but without the D3D11_BIND_DEPTH_STENCIL flag set, and then copy the required texel from this resource to a 1×1 staging texture.

Asynchronous terrain updates

As soon as we find at which location the terrain is to be deformed, we have to perform a number of operations: displace the height map, re-triangulate the affected region, update the data base, re-compute the normal map, etc. These tasks are time-consuming, especially re-triangulation, and performing all the required operations in the main thread would cause noticeable stalls. Thus it is reasonable to rebuild the triangulation asynchronously in order to not affect rendering performance. However, using the old triangulation for the modified region could be insufficient to accurately represent the new surface geometry. Imagine a flat area represented by a small number of large triangles and a small hole created through modification, which cannot be accurately modeled by large triangles. To solve this problem, we temporarily render modified patches using the full resolution triangulation until an updated triangulation is ready. Thus, when the terrain is modified, some tasks are done immediately to show the effect of deformation, while time-consuming operations are postponed to asynchronous threads:

• The main thread immediately updates the patch height map on the GPU by rendering to texture, assigns full-resolution triangulation to the patch, and starts the asynchronous thread.
• The asynchronous thread updates the height map on the CPU, rebuilds the triangulation and encodes it, and performs other time-consuming operations. As soon as the updated triangulation is ready, the main thread starts using it for rendering the patch.

Each node of the quad tree has a list of pending modifications. When the terrain is modified, the new modification is added to the list. The lists are then asynchronously processed by worker threads. The workflow of the terrain modification is depicted in Figure 3.

Figure 3: Updating terrain triangulation

Note that a modified patch could be located at any level of the quad tree, not necessarily at the finest resolution level. If the patch is located at a coarser level, all affected patches up to the finest resolution level are visited and updated.

To implement asynchronous execution, each node of the quad tree can be assigned one of the following tasks:

• Increase LOD task is executed when the model is about to be refined. It creates the patch’s children at level n+1
• Decrease LOD is executed when the model is about to be coarsened. It processes four sibling patches at level n+1 and replaces them with one coarse parent patch at level n. If some of the children are marked as modified, the task updates the parent patch (creates and encodes new triangulation, calculates new minimum/maximum patch height, updates bounding box, etc.). If no children are marked as modified, the task does nothing.
• Recompress task is executed when terrain is modified. It recursively traverses all the affected quad tree nodes starting from the modified patch and updates triangulation for each node. When the procedure reaches the finest resolution level, it also updates the height map and material mask.

The following rules are also enforced to ensure correct execution:

• A new task cannot be assigned to a node until the previous task at this node is completed.
• A decrease LOD task cannotbe created in the following cases:
  • There are pending modifications assigned to one or more child nodes which have not been processed yet.
  • There is an Increase LOD task executing for one or more child nodes.
  • There is a Recompress task executing for one or more child nodes.
  • If new modifications are assigned to some child nodes while a decrease LOD task is executing, the task is discarded after it is completed and is executed again after the modifications are processed.
  • An Increase LOD task is not assigned to a node until all modifications are processed by a Recompress task.

If there are new modifications while the Recompress task is running, they are added to the list and processed by a new Recompress task as soon as the previous task finishes.

Layer-based texturing

Terrain texturing in this sample is done using a well-known, tile-based approach where a number of materials such as sand, grass, rock, and sand are tiled across the surface and blended using weights given by a global mask texture. Material textures are usually tiled many times to provide high frequency details, while the mask texture has much lower resolution and is responsible for global terrain appearance. The material mask is managed in the same way as the height map. In addition to a height map and normal map, each terrain has a material mask covering the same terrain region. When terrain is modified, the material mask is updated along with the height map in the asynchronous thread.

In this sample each material is assigned diffuse and normal maps. A 4-channel material mask is used and the weight of a fifth (base) material is implicitly defined as a complement to 1. The normal map is used to alter world space normals. All blending is performed in the pixel shader, which provides high quality visual results. To implement smooth transitions between different levels of detail, the pixel shader implements morphing, as shown in Listing 1.

float3 RenderPatchPS(RenderPatchVS_Output In) : SV_Target
{
    float4 SurfaceColor = 0;

    // Load material weights
    float4 MtrlWeights = g_tex2DMtrlMap.Sample(samLinearClamp, In.NormalMapUV.xy);

    // Load material weights from parent patch to implement morph
    float4 ParentMtrlWeights = 
        g_tex2DParentMtrlMap.Sample( samLinearClamp, In.NormalMapUV.zw);
    MtrlWeights = lerp(MtrlWeights, ParentMtrlWeights, In.fMorphCoeff);

    // Normalize weights and compute base material weight
    MtrlWeights /= max( dot(MtrlWeights, float4(1,1,1,1)) , 1 );
    float BaseMaterialWeight = saturate(1 - dot(MtrlWeights, float4(1,1,1,1)));
    // Get diffuse color of the base material
    float4 BaseMaterialDiffuse = 
        g_tex2DTileTextures[0].Sample(samLinearWrap, In.TileTexUV.xy);
    float4x4 MaterialColors = (float4x4)0;

    // Get tangent space normal of the base material
    float3 BaseMaterialNormal = 
        g_tex2DTileNormalMaps[0].Sample(samLinearWrap, In.TileTexUV.xy);
    float4x3 MaterialNormals = (float4x3)0;

    // Load material colors and normals
    [unroll]for(int iTileTex = 1; iTileTex < NUM_TILE_TEXTURES; iTileTex++)
    {
        const float fThresholdWeight = 3.f/256.f;
        MaterialColors[iTileTex-1] = 
            MtrlWeights[iTileTex-1] > fThresholdWeight ? 
                g_tex2DTileTextures[iTileTex].Sample(samLinearWrap, In.TileTexUV.xy) : 0.f;

        MaterialNormals[iTileTex-1] = 
            MtrlWeights[iTileTex-1] > fThresholdWeight ? 
                g_tex2DTileNormalMaps[iTileTex].Sample(samLinearWrap, In.TileTexUV.xy) : 0.f;
    }
    // Blend materials and normals using the weights
    SurfaceColor = BaseMaterialDiffuse * BaseMaterialWeight + 
        mul(MtrlWeights, MaterialColors);

    float3 SurfaceNormalTS = BaseMaterialNormal * BaseMaterialWeight + 
        mul(MtrlWeights, MaterialNormals);
    SurfaceNormalTS = normalize(SurfaceNormalTS*2-1);

    // Load world-space normal
    float3 Normal; 
    Normal.xy = g_tex2DNormalMap.Sample(samLinearClamp, In.NormalMapUV.xy).xy;

    // Load parent patch world-space normal
    float2 ParentNormalXY = 
        g_tex2DParentNormalMap.Sample( samLinearClamp, In.NormalMapUV.zw).xy;
    // Belnd between child and parent patch normals
    Normal.xy = lerp(Normal.xy, ParentNormalXY.xy, In.fMorphCoeff);
    Normal.z = sqrt( max(1 - dot(Normal.xy,Normal.xy), 0.0) );
    Normal = normalize( Normal );

    // Transform tangent space normal
    float3 Tangent   = normalize(float3(1,0,In.HeightMapGradients.x));
    float3 Bitangent = normalize(float3(0,1,In.HeightMapGradients.y));
    Normal = normalize( mul(SurfaceNormalTS, float3x3(Tangent, Bitangent, Normal)) );

    // Compute diffuse illumination
    float DiffuseIllumination = 
        max(0, dot(Normal, g_LightAttribs.m_f4DirectionOnSun.xyz));
    float3 lightColor = g_LightAttribs.m_f4SunColorAndIntensityAtGround.rgb;

    SurfaceColor.rgb *= (DiffuseIllumination*lightColor + 
                         g_LightAttribs.m_f4AmbientLight.rgb);

    // Add fog
    float FogFactor = exp( -In.fDistToCamera * g_TerrainAttribs.m_fFogScale );
    SurfaceColor.rgb = SurfaceColor.rgb * FogFactor + 
                      (1-FogFactor)*g_TerrainAttribs.m_f4FogColor.rgb;

    return SurfaceColor;

Listing 1: Full code listing of terrain rendering pixel shader

Note that to improve rendering performance, diffuse and normal maps are accessed only if the corresponding weight is greater than some small threshold.

Sample architecture

The sample consists of a number of components as shown in Figure 4.

Figure 4: The sample architecture

The elevation (CElevationDataSource), material mask (CMtrlMaskSource), and triangulation (CTriangDataSource) data sources provide access to the corresponding type of data. Elevation and material mask sources have a Modify() method that updates the required data region. The triangulation data source provides a CreateAdaptiveTriangulation() method that constructs adaptive triangulation for the specified quad tree patch.

The core component of the sample is the block-based adaptive model (CBlockBasedAdaptiveModel). It performs all the tasks required to maintain and update the patch quad tree. It accesses the data sources to retrieve data and creates/destroys patches in the quad tree. It also manages all the asynchronous tasks. The object also maintains the list of optimal patches that are rendered by CAdaptiveModelDX11Render. This component is also responsible for rendering bounding boxes, the terrain minimap, and quad tree structure on the map. The object creates and manages all the required shaders, states, buffers, and all other resources.

The Terrain patch object (CTerrainPatch) contains all the resources associated with the patch, the height map, the material mask, and the adaptive triangulation. The raw texture management is done by the D3D texture cache. The terrain patch object also contains methods that modify the data on the GPU: the render techniques, which displace the height map and update material mask.

Controlling the sample

The following controls can be used to control the sample:

• Wireframe checkbox turns wireframe overlay on/off.
• Update model checkbox enables/disable the model update. When it is unmarked, the model is not adapting to the camera location and triangulation updates are not performed.
• Show bound boxes check box shows/hides bounding boxes of the quad tree patches.
• Adaptive triangulation checkbox enables/disables using adaptive triangulation for rendering a patch. When it is unmarked, the full-resolution triangulation is used to render each patch.
• Screen space threshold slider controls the accuracy of the constructed terrain model.
• Rock, White sand, Yellow sand, Gray sand, and Snow buttons select corresponding material.
• Displacement scale slider controls the magnitude and disposition of modification. The leftmost location corresponds to the maximum negative displacement. The rightmost location sets the maximum positive displacement. The middle location sets no displacement. Only the material mask is modified in this case.
• Size slider controls the area affected by the modification.
• Const size in screen space checkbox forces the modifications to have constant size in screen space (when possible).

Figure 5 shows some different materials applied to the terrain surface.

Figure 5: Different materials applied to the terrain surface

The right mouse button can also be used to modify terrain surface, and the middle mouse button rotates the light.

Rendering a fully textured terrain at 1600 x 900 resolution takes less than 11 ms on Intel® HD Graphics 4000 hardware.

Resources

1. P. Lindstrom, D. Koller, W. Ribarsky, L. F. Hodges, N. Faust, and G. A. Turner. Real-time, continuous level of detail rendering of height fields. In Proc. SIGGRAPH 96, pages 109-118. ACM SIGGRAPH, 1996.
2. Renato Pajarola. Large scale terrain visualization using the restricted quadtree triangulation. In Proceedings Visualization 98, pages 19-26 and 515. IEEE Computer Society Press, 1998.
3. Thatcher Ulrich. Rendering massive terrains using chunked level of detail control. SIGGRAPH Course Notes (2002). Volume 3, Issue 5.
4. SNB terrain sample
5. Touch for Windows Desktop sample
6. Egor Yusov. Real-Time Deformable Terrain Rendering with DirectX 11. GPU Pro3: advanced rendering techniques / edited by Wolfgang Engel, Chapter I.2. Boca Raton, FL: Taylor & Francis Group, LLC, 2012.

Related Links

• Touch Terrain code sample
• Win8 desktop code samples
• VC Source Game code samples

Intel, the Intel logo, and Ultrabook are trademarks of Intel Corporation in the US and/or other countries.

Copyright © 2013 Intel Corporation. All rights reserved.

*Other names and brands may be claimed as the property of others.

Download Article

Download Porting iOS* Apps to Windows* 8 - Overview [PDF 841KB]

Objective

Developers looking to port their existing iOS* apps to the new Windows* 8 platform face several challenges. In this article we will provide an overview of the differences between the two platforms and show developers how to go about porting their apps.  We will compare developer tools in both platforms and provide an overview of app framework differences.  We will discuss basic app creation, packaging, and app store differences. Fundamental app concepts like UI and input handling will be discussed. This article will serve as a foundation for other articles that delve into more advanced topics in porting iOS apps to Windows 8.

Table of Contents

1. Introduction
2. Development Environment and Tools
3. iOS Apps vs. Windows 8 Apps Overview
4. App UI and Input Handling
5. Summary

1. Introduction

Windows 8 is a major revision for one of the predominant desktop OSs in the industry. From a developer’s point of view, it brings major changes to the way Windows applications are developed, designed, and built. It is redesigned and reimagined from the ground up to support new user experiences and to take advantage of all the capabilities of modern mobile devices like tablets and UltraBook™ devices.

With the introduction of new form factor devices like smartphones and tablets, more and more software is being targeted for these platforms. Currently there are hundreds of thousands of apps in iOS and other mobile ecosystem stores. 

Developers who already have an existing iOS app might want to consider targeting the new Windows 8 OS. These two OSs, their apps, and development environments, one might argue, are radically different. But at a higher level they also have some common features like support for touch, sensors, mobility, and other hardware features. There are lot of benefits and opportunities in developing for Windows 8. In this article, we will give an overview of differences and cover the basics of porting iOS apps to Windows 8.

We can broadly classify the porting challenges into development environment and tools, app framework differences, and the overall OS/ecosystem differences.

Below are few basic snapshots of how iOS and Windows 8 OS look and differ:

Figure 1:iOS* “Home Screen” (Photo Source: www.apple.com/iphone)

Figure 2:Windows* 8 Home Screen (captured from Simulator)

As seen in these two screen shots, the two OSs differ dramatically. The new Windows 8 UI removes all non-app related notifications from the main screen (like the notification bar at the top and other icons). You will notice new Windows 8 apps focus on content and remove the chrome of traditional app UI as much as possible. Also, while iOS features application icons, Windows 8 introduced a unique UI feature called tiles where each app on the home screen represents a box or a window resembling tiles. The content of these tiles can be dynamically updated by the owning app with app-relevant data or information, and are hence referred to as “live tiles.”

We will go into other details shortly, but these are some of the basic and important UI differences.

2. Development Environment and Tools

When considering porting an app from one platform to another, one important area developers must consider is developer tools and programming environment. The current industry trend in this area is to provide an integrated development experience as much as possible. Companies try to provide tools for all stages of development—coding, debugging, testing, building, packing the apps, and even uploading the apps to stores.

Developers coming from the iOS platform and looking to port their apps to Windows 8 will find that the programming environment and development tools in Windows 8 are comprehensive and powerful. While Apple requires a purchased developer license to deploy applications written in the XCode* IDE, this is not the case with Windows 8.  Local packages can be created in the Visual Studio* 2012 IDE for device testing. But like Apple, Microsoft requires a purchased license for deploying apps to the Windows Store.  The following screen shot from the Visual Studio IDE shows the deployment options:

Figure 3:Windows* 8 App Deployment (captured from VS IDE)

In Windows 8, developers can choose their programming language of preference. Multiple programming languages are supported: C++, C#, Visual Basic*, and JavaScript*/HTML5. All of these programming languages are first-class citizens, meaning they all have access to full system capabilities and use the same common Windows 8 API (Windows Runtime).

Another important area is the integrated development environment (IDE). While iOS developers use XCode, Windows 8 developers use Visual Studio 2012, which has been updated to support the new Windows 8 platform. Blend for Visual Studio 2012 will assist in Window 8 app UI (XAML based) interactive design and testing.  Blend also supports HTML5 apps UI design and CSS3 styling. These tools can be downloaded from the website below:

http://msdn.microsoft.com/en-us/library/windows/apps/hh974577.aspx

Use Visual Studio 2012 with Windows 8 for Windows Store app development, as the new apps depend on the Windows 8 platform and APIs.

Similar to default project templates for apps available in iOS, Visual Studio 2012 comes with templates for different kinds of Windows 8 apps and programming languages.

The screen shots below show the templates for JavaScript and C++:

Figure 4:JavaScript* Project Templates (captured from Visual Studio* 2012)

Figure 5:Visual C++* Project Templates (captured from Visual Studio* 2012)

Visual C#* and Visual Basic have similar project templates too.

iOS developers wanting to fully rewrite their apps for Windows 8 might want to consider using one of the above project templates as a baseline for their apps.

Windows 8 has a powerful UI design tool—Blend for Visual Studio 2012. It is fully integrated into the Visual Studio IDE and makes the development life cycle as simple as possible. Blend can be directly invoked from inside the VS 2012 IDE. Blend has support for both XAML-based UI design as well as the HTML5/CSS3 apps.

Below is a screen shot of Blend showing the "Split App (XAML)" template choice for Visual C#:

Figure 6:XAML-based UI editing (captured from Blend for Visual Studio*)

Blend is a very powerful and comprehensive UI design tool. It supports interactive UI editing/design and quick testing. For example, in Blend for HTML5/CSS3 apps, we can edit the HTML DOM and CSS3 styling live and immediately see the result. This is very powerful for trying out different styling properties, layouts, or themes.

Below is a screen shot of Blend used in designing a HTML5/CSS3 app:

Figure 7:HTML5/CSS3 UI editing (captured from Blend for Visual Studio*)

Blend for Visual Studio 2012 has powerful testing tools for different display sizes and views.

The next screen shot illustrates how developers can test their Windows 8 apps for different views and screen sizes supported by the Windows 8 platform:

Figure 8:Different Device Views and Display Target Settings (captured from Blend for Visual Studio*)

Another developer tool that is common in modern app development is the emulator or simulator. iOS has the iOS device emulator with support for different configurations through XCode, such as whether an app supports both iPad* and iPhone* or just one. Windows 8 has a powerful simulator that is fully integrated with Visual Studio 2012 IDE and an app development life cycle. It simulates several of the common platform features like touch, geo-location, camera, device orientation, and different form factors and screen sizes.

Windows 8 simulator can be invoked directly from VS 2012 IDE. The screen shot below shows simulator capabilities and features:

Figure 9:Windows* 8 Simulator buttons and display options (captured from Windows 8 Simulator)

We also need a way to test our app on a real device. In XCode we can select "iOS Device" from the appropriate IDE dropdown menu. Windows 8 apps can be tested on remote devices using the Remote Debugging Tools for Visual Studio. We will need to install the remote tools on the target device and configure them before we can use it with Visual Studio remote debugging. This web site provides a comprehensive guide to running Windows Store apps on a remote machine:

http://msdn.microsoft.com/en-us/library/hh441469.aspx

In Visual Studio, we can choose our target of choice for debugging: Simulator, Local Machine, or Remote Machine. The next screen shot shows these Debug options in Visual Studio 2012:

Figure 10:Debug Target options (captured from Visual Studio* 2012)

Another important development environment topic is that of app building, packaging, and app stores. In iOS, XCode integrates and automates the process of building, signing, and packaging the apps by using the developer's trusted key certificate. Similarly, Windows 8 provides comprehensive support for the entire app development life cycle from developing, building, signing/certificates, packaging, and direct integration with the Windows Store.

The table below provides a summary of the different topics we discussed in this section (please note it is not comprehensive so it may not have all of the details).

3. iOS Apps vs. Windows 8 Apps Overview

To port apps from iOS to Windows 8, we need to study the differences between these two platforms. These two platforms have totally different application stacks—from the low level app interactions with the OS, to the highest abstractions like the UI.

Before we delve into the details, we need to note one important caveat—Windows 8 supports two types of applications: the new Windows 8 apps (called “Windows Store apps”) and the traditional Windows applications (“Desktop apps”).

Existing desktop applications continue to work and behave the same on Windows 8. However, existing desktop applications can now take advantage of new features like touch, sensors, and other new Windows 8 features. But some of the new Windows 8 features, like live tiles, the new app life cycle, security sandbox, etc., might not be available in desktop apps.

The web page below provides an overview of the new Windows Store apps:

http://msdn.microsoft.com/en-us/library/windows/apps/hh974576.aspx

At a higher level, iOS apps can be composed of three important pieces:

• Application components (Views, Services, Content Providers)
• Property list file (defines bundle configuration, XML structures, identifiers, etc.)
• Application resources (images, media, language strings, locale support, resources for different device types, etc.)

Windows Store apps also have similar pieces, though they may not map exactly to iOS pieces:

• Different application components like the main app component, app contracts, extensions, background tasks, notifications/tiles, etc.
• Package manifest for the app (defines package identity, properties, capabilities/permissions, extensions, visual elements, etc.)
• App resources and localization (Package Resource Index PRI file, all app resources are indexed and associated) 

The screen shot below shows the capabilities and available declarations in the Windows 8 Package Manifest of a sample app:

Figure 11:Package Manifest – capabilities (left) and declarations (right) (captured from Visual Studio* 2012)

Both Apple and Windows Store apps have a well-defined app life cycle, or different states the app goes through from start to end. The link below provides an overview of the app life cycle of Windows Store apps:

http://msdn.microsoft.com/en-us/library/windows/apps/hh464925.aspx

Similar to iOS, Windows 8 supports different views for the app—portrait mode, landscape mode, different screen sizes, etc. In addition to these, Windows 8 supports two more views not available on iOS: “Snap View” and “Filled View.”

The screen shot below shows a Windows 8 app in four different views: landscape, portrait, snapped view (bottom left, Windows Store app is in filled view), filled view (bottom right, Windows Store app is in snapped view).

Figure 12:Sample Windows* 8 App in different views (captured from Windows 8 Simulator)

For guidelines on how to design for snapped and fill views, please refer to:

http://msdn.microsoft.com/en-us/library/windows/apps/hh465371.aspx

Another important difference is the common app controls and UI surfaces. iOS has various bars for completing various tasks, including navigation bar, search bar, and so on.  For Windows 8, the app bar is the primary command interface for Windows Store apps. Similar to the “Navigation Bar” in iOS, we can use it for navigation and other app actions. By default the app bar is hidden in Windows 8 apps; it can be invoked by a Windows 8 swipe gesture. Its behavior can be customized as needed (e.g., always stay active, dismiss on touch, etc.). The next screen shot shows an app bar in action in a Windows 8 Internet Explorer* app:

Figure 13:App bar in Internet Explorer* 10 (captured from Windows* 8 Internet Explorer)

The app bar can be at the bottom, the top, or both. It can be invoked by a swipe gesture either from the top or bottom of the screen. With keyboard-mouse input, it can be invoked by right clicking inside the app.

Charms are a unique feature in Windows 8 apps. They are a specific, common set of buttons always shown in every app when charms are invoked (again, by the swipe gesture). The buttons are: Search, Share, Start button, Connect to Devices, and Settings.

The next screen shot shows the Charms bar, invoked inside a Windows Store app:

Figure 14:Charms bar in Store App (captured from Windows* 8 Store)

The Charms bar is usually positioned on the right-hand side of the screen and invoked by a swipe gesture from the right side of the device.

Another area of difference is obviously the programming languages and platform APIs. Since iOS apps utilize the Objective C programming language, porting apps to Windows 8 requires rewriting the app code using one of the Windows 8 programming languages.  Depending on the app features, requirements, and capabilities, Windows 8 may have equivalent platform APIs available that developers can use. For example, the touch and sensor functionality can be ported by using the equivalent Windows 8 APIs. Please visit this site for a comprehensive API reference for Windows Store apps:

http://msdn.microsoft.com/en-us/library/windows/apps/br211369

In this section, we provided an overview of different areas of porting challenges that iOS developers face when converting their apps to the Windows 8 platform. As we have seen, many high level app concepts overlap. Developers can map their app design and logic to corresponding or equivalent patterns in Windows 8. Sometimes the conversion might not exactly map because that feature is not available or it is implemented in a totally different pattern. For example, the iOS Storyboard allows developers to drag in a "DatePicker" (calendar) object into the app design.  While developers can drag in a similar object when developing Java apps for Windows 8, this isn’t the case for C# apps; developers would need to create a custom date picker.

 You can find a nice method for building a date picker for C# by referring to this blog: http://software.intel.com/en-us/blogs/2012/12/14/how-to-make-a-date-picker-calendar-for-c-sharp-in-windows-8

In these cases, a rewrite of that particular feature/component needs to be considered. Further, the apps can be enhanced to take advantage of the unique features of Windows 8 like live tiles, new kinds of app views, etc. This will help provide the best user experience for your app on Windows 8.

4. App UI and Input Handling

The previous section provided overall conceptual differences between iOS apps and Windows 8 apps. In this section, we will build on that and cover how to port one of the first and basic pieces of any app: UI and Input Handling.

In Windows 8, an app’s UI can be built in a few different ways:

• Using XAML with C++/C#/VB
• Using HTML5/CSS3 with JavaScript
• Direct graphics interface using DirectX/C++ (some XAML can be mixed here)
• Desktop apps UI

Developers should carefully consider different aspects of the target app when picking one of these choices. It depends on app requirements and planning for future extensibility and reuse. For example, if the existing app uses a lot of web-based UI content or already has significant HTML code, developers should probably use JavaScript/HTML5. On the other hand, if the app requires heavy graphics and full GPU control, consider DirectX/C++. As noted previously, all programming languages have the same access to all the system capabilities (via common Windows 8 Runtime API).

Apps that require desktop features, with several UI controls and chrome, may be more suitable for a Desktop app model.

Once we pick our programming environment of choice, we can proceed to look at other UI aspects like window management, views handling, event handling, styling, UI controls, animations, etc. These differ quite a bit depending on the underlying environment. For example, HTML5 apps have their own event handling patterns (as defined by HTML DOM standards and web practices) and styling capabilities (CSS3).

In iOS apps, the UI of a typical app consists of multiple "View" components, with "Segues" in between that host “View” and “GroupView” objects that implement the UI elements like buttons and layouts. The UI life cycle, back stack, input, and event handling all are taken care of by this framework. Windows 8 has a very powerful UI framework and programming mode that has common system-wide UI concepts:

• Common design style – typography, layouts, navigation, animations
• Touch interaction – well-defined, system-wide touch gestures, with new features like semantic zoom
• Snapping and scaling – support for new views and different screen layouts and sizes
• Common UI controls – e.g., buttons, list view, app bar, charms, settings, etc.
• Tiles and notifications

A comprehensive discussion about UI for Windows Store apps can be found at the link below:

http://msdn.microsoft.com/en-us/library/windows/apps/hh779072.aspx

It is strongly recommended for developers to take advantage of Blend for Visual Studio 2012 to design UI for Windows 8 apps. Blend supports both XAML-based UI design, as well as HTML5/CSS3 UI design. It has comprehensive tooling and design support and allows for interactive and quick UI verification/testing.

Developers will also need to make design choices for how to best take advantage of all the UI features and capabilities offered by Windows 8. The article below titled “Design case study: iPad* to Windows Store app” covers some of the UI design choices we need to consider when porting apps from other platforms:

http://msdn.microsoft.com/en-us/library/windows/apps/hh868262

For a full step-by-step dive into the details of Windows 8 touch code (C#), take a look at this article titled "Enabling Touch in Windows 8* Style UI Apps with C#":

http://software.intel.com/en-us/articles/enabling-touch-in-windows-8-metro-style-apps-with-c

The article starts with touch basics and also covers more complex touch scenarios (such as custom manipulations).  The article is suitable for all levels of platform porting.

5. Summary

Porting apps from iOS to Windows 8 takes a lot of planning and effort. The porting process can be simplified and more efficient, if we fully understand how iOS platform’s features and capabilities compare to the ones available in Windows 8. If developers conceptually map their app logic and design to equivalent features in Windows 8, they will be able to achieve the best possible port.

In this article we provided an overview of differences between the two platforms and discussed how iOS platform features compare or translate to Windows 8. We covered the basics like development tools, programming environment, platform and framework differences, and UI concepts.  We hope this article will serve as a foundation for future articles that delve into more advanced and in-depth topics.

*Other names and brands may be claimed as the property of others.

Copyright © 2012 Intel Corporation. All rights reserved.

OpenGL is a registered trademark and the OpenGL ES logo is a trademark of Silicon Graphics Inc. used by permission by Khronos.

Objective

Most mobile apps, regardless of the platform, have features like search, app settings, complex app life cycle, and state maintenance in them. In this article we will cover how to port these features from existing iOS apps, to the Windows* 8 platform. We will compare and contrast how search is implemented in iOS apps and Windows 8 apps. We will look at different states an app goes through and how to persist or recover state during these app life cycle states. Finally, we will discuss how app settings are implemented and integrated into the platform. Search and Settings contracts in Windows 8 will be discussed.

Table of Contents

1. Introduction
2. App Life Cycle in iOS and Windows 8 Store Apps
3. Porting App Settings from iOS to Windows 8 settings contract
4. Integrating App Search functionality -  iOS vs. Windows 8 Store Apps
5. Summary

Introduction

Applications need to support robust user experiences that include state and session maintenance, search, and settings.

Mobile apps have complex run time and life cycle states. Modern OS platforms, like iOS, enforce their own app life cycle that each app has to conform to. iOS developers are familiar with different states an app goes through in its life:

http://developer.apple.com/library/ios/#documentation/iphone/conceptual/iphoneosprogrammingguide/ManagingYourApplicationsFlow/ManagingYourApplicationsFlow.html

Windows 8 Store apps also have well-defined life cycle states. In this article we will discuss how to handle these state transitions to enable seamless user experience.

Apps that depend and manipulate large amounts of data, typically also have a way for users to search quickly for information or relevant data in the app. For example, the iOS mobile platform has support for UI controls like UISearchBar to provide a common look and feel for search. Most apps tend to use this common search experience on iOS platforms.

Developers looking to port their iOS app to Windows 8 will be glad to know Windows 8 has a well-integrated search feature at the platform level. In Windows 8, users have access to a common system-wide search interface that apps can plug-into, thereby providing a seamless experience.

Windows 8 provides this interface as part of its system-wide UI control called a charms bar. Please see the screen shot below.

***Figure 1: Charms bar (captured from a Windows* Store app)

The charms bar can be invoked by users anytime using a special gesture (touch or keyboard CTRL-C). Along with Search, it has other interfaces like settings, devices, start button, and share. In this article we will discuss how to implement search and settings in your mobile app and compare to iOS equivalents.

App Life Cycle in iOS and Windows 8 Store Apps

Apps on the  iOS platform are in one of these states: not running, inactive, active, background, or suspended. Which state and how long the app will be in a particular state are unpredictable. iOS apps depend on UIKit to handle states and to provide a seamless user experience for the app.

iOS developers are familiar with how UIKit can  help in state preservation and restoration of apps:

http://developer.apple.com/library/ios/#DOCUMENTATION/iPhone/Conceptual/iPhoneOSProgrammingGuide/StatePreservation/StatePreservation.html#//apple_ref/doc/uid/TP40007072-CH11-SW13

When porting an existing iOS app to Windows 8, developers can conceptually map the UI and its view states to corresponding UI screens in Windows Store apps. Similar kinds of state can be saved and restored for the apps.

Like the iOS platform, the Windows 8 platform also has its own app life cycle states. Similar high level concepts of saving state before the app goes into the suspend (and eventual termination or “not running”) state apply. This enables us to provide a good user experience for the app.

This article provides the Windows Store apps life cycle overview:

http://msdn.microsoft.com/en-us/library/windows/apps/hh464925.aspx

In addition to a typical app launch (e.g., from the home screen), a Windows Store app can be started via contracts (e.g., charms bar) or extensions.

The UI for Windows Store apps using C# is built using XAML. Please refer to articles on porting iOS UI to Windows Store apps for details on XAML and UI methodologies. The default C#/XAML project templates in Visual Studio* 2012 make it very easy to implement the app life cycle transitions. They include two base classes: LayoutAwarePage (from which all “pages,” or views in iOS, inherit from), and SuspensionManager (similar to iOS UIKit type functionality to persist state). The LayoutAwarePage base class depends on, and uses, the SuspensionManager class, so all our views automatically inherit this functionality.

Whenever your app is switched or sent to the background (when another app is invoked), the LayoutAwarePage will automatically handle the state transitions. It will call appropriate methods in the page to give an opportunity to persist state.

Let’s see some code. Typically, when you want to save/restore states in an XAML page (UI view), you modify or implement the “SaveState” function (which is a method of LayoutAwarePage), and its corresponding “LoadState” where you restore the state. By default, the LayoutAwarePage automatically saves the navigation stack. We only need to take care of the page-specific state. The sample code listing** below shows a Login UI page, where we save and restore the login username in case the app gets suspended (and may get terminated).

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using Windows.Foundation;
using Windows.Foundation.Collections;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Primitives;
using Windows.UI.Xaml.Data;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Navigation;
 
using PRApp.ViewModels;
 
// The Basic Page item template is documented at http://go.microsoft.com/fwlink/?LinkId=234237
 
namespace PRApp
{
    /// <summary>
    /// A basic page that provides characteristics common to most applications.
    /// </summary>
    public sealed partial class Login : PRApp.Common.LayoutAwarePage
    {
        public Login()
        {
            this.InitializeComponent();
        }
 
        /// <summary>
        /// Populates the page with content passed during navigation.  Any saved state is also
        /// provided when recreating a page from a prior session.
        /// </summary>
        /// <param name="navigationParameter">The parameter value passed to
        /// <see cref="Frame.Navigate(Type, Object)"/> when this page was initially requested.
        /// </param>
        /// <param name="pageState">A dictionary of state preserved by this page during an earlier
        /// session.  This will be null the first time a page is visited.</param>
        protected override void LoadState(Object navigationParameter, Dictionary<String, Object> pageState)
        {
            var uvm = UsersViewModel.GetDefaultUserViewModel();
            if (pageState != null && pageState.ContainsKey("username"))
                uvm.Login = (String)pageState["username"];
           
            this.DataContext = uvm;
        }
 
        /// <summary>
        /// Preserves state associated with this page in case the application is suspended or the
        /// page is discarded from the navigation cache.  Values must conform to the serialization
        /// requirements of <see cref="SuspensionManager.SessionState"/>.
        /// </summary>
        /// <param name="pageState">An empty dictionary to be populated with serializable state.</param>
        protected override void SaveState(Dictionary<String, Object> pageState)
        {
            pageState["username"] = usernameInput.Text;
        }
 
        private void Button_Click_1(object sender, RoutedEventArgs e)
        {
            var uvm = (UsersViewModel)this.DataContext;
            uvm.LogOut();
            if (uvm.LogIn()) this.Frame.Navigate(typeof(Patients));
        }
    }
}
 

In the SaveState method, we simply save the “Text” value of our “usernameInput” XAML text box UI control. We use the “pageState” dictionary that was passed into the method. Anything you put in this dictionary will automatically persist on suspension. It’s pretty simple and straightforward.

Please note, your app may have two kinds of data to persist:  app-specific data (often associated with data model or app settings) and UI session or state (a temporary, session, or UI-related state that can be discarded or purged with no impact on the app’s functionality).  It’s recommended we use pageState dictionary to persist the latter kind.

To persist the former kind, you will want to use a different mechanism. Windows Store apps can store their app-specific data on different kinds of data stores: local, roaming, and temporary stores.  If you would like your app data to sync between multiple devices where the app was installed, you would use roaming. If not, use local store. The temporary store is just that, temporary and can be removed any time. More detailed information can be found here:

http://msdn.microsoft.com/en-us/library/windows/apps/xaml/hh464917.aspx

Below, we will look at an example of how to use local settings data store. A similar technique can be applied to other data stores like the roaming settings.  LocalSettings is available in Windows.Storage namespace. It exposes a key-value store that we can use to save/restore objects. In our sample app, we want to persist user settings for displaying patients. We make this available in the settings flyout (discussed later in this article). To save this user setting across reboots, we can use LocalSettings data store. Please see the code listing** below for an example:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using PRApp.Common;
using PRApp.DataModel;
using PRApp.ViewModels;
using System.Collections.ObjectModel;
using Windows.Storage;
 
namespace PRApp.ViewModels
{
    class SessionSettingsViewModel : BindableBase
    {
        ApplicationDataContainer locsettings = null;
        public SessionSettingsViewModel()
        {
            locsettings = ApplicationData.Current.LocalSettings;
            if (!locsettings.Values.ContainsKey("showdeceased"))
            {
                locsettings.Values["showdeceased"] = false;
            }
 
            showdeceased = (bool)locsettings.Values["showdeceased"];
 
        }
        private UsersViewModel loginuser = null;
        public UsersViewModel Loginuser { get { return loginuser; } set { this.SetProperty(ref loginuser, value); } }
 
        private static SessionSettingsViewModel sessionsettings = null;
        public static SessionSettingsViewModel SessionSettings
        {
            get
            {
                if (sessionsettings == null) { sessionsettings = new SessionSettingsViewModel(); sessionsettings.Loginuser = UsersViewModel.GetDefaultUserViewModel(); }
                return sessionsettings;
            }
        }
 
        private bool showdeceased = false;
        public bool ShowDeceased { get { return showdeceased; } set { locsettings.Values["showdeceased"] = value; this.SetProperty(ref showdeceased, value); } }
    }   
}

In the above code snippet, we are persisting “showdeceased” user setting to the LocalSettings data store. Since we expose this variable as a bindable property (inherited from BindableBase), we can directly bind this variable to our UI controls. Please refer to the next section on how to bind this value to the app settings UI flyout. We can also persist new containers or composite objects. For more detailed information please refer to the link below:

http://msdn.microsoft.com/en-us/library/windows/apps/xaml/hh700361.aspx

In this section we looked at how iOS app life cycle states, state preservation, and restoration translate to Windows Store apps. We looked at a couple of examples on how to save persist View state and app-specific data.

Porting App Settings from iOS to Windows 8 settings contract

In iOS, we use the user defaults system or the new iCloud key-value store to implement our app settings. The iOS platform provides a way to expose app settings under system settings/preferences, thereby providing a seamless and consistent settings experience across all apps. iOS developers are also free to implement their own in-app settings screen for frequently used options (e.g., volume level).

Developers porting their iOS apps to Windows Store apps, will find the settings framework in Windows 8 is comprehensive. It even includes consistent UI guidelines, with size, color and general look-and-feel specifications. As we discussed in the introduction, the system-wide settings interface is exposed via a charms bar. This interface is available in any app and at system level too.

The screen shot below shows the charms bar in our sample app.

***Figure 2: Sample Patient Records App with Charms Bar (captured from simulator)

When the settings button is pressed, it brings up the standard settings flyout with default options (typically just “Permissions” by default). The app can choose to implement additional settings options in the settings flyout. Each option, in turn can open another flyout or may even direct to custom settings screen. The screen shot below shows our sample app with our own “PRApp Options” command added to the settings flyout.

***Figure 3: Default Settings flyout (captured from simulator)

Below are comprehensive guidelines for app settings in Windows Store apps :

http://msdn.microsoft.com/en-us/library/windows/apps/hh770544.aspx

To implement all of these guidelines from the ground up takes lot of effort. Unfortunately, XAML/C# has no standard or default Settings Flyout control (though, there is one available in HTML5/JavaScript*/WinJS framework).

C# has some 3^rd party libraries that we can use to get most of the functionality and comply with guidelines. If you would like to implement your own custom app settings flyout from scratch, please refer to the article below:

http://msdn.microsoft.com/en-us/library/windows/apps/xaml/hh872190.aspx

We can either programmatically add our settings control, or declare an XAML page that inherits it from UserControl and has all our custom UI pieces in there. Below** is an example of our sample settings flyout:

<UserControl
    x:Class="PRApp.PRSettingsControl"    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"    xmlns:local="using:PRApp"    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"    mc:Ignorable="d"    d:DesignHeight="300"    d:DesignWidth="400"    xmlns:c="using:PRApp.ViewModels">
    <UserControl.Resources>
        <c:SessionSettingsViewModel x:Key="myDataSource"/>
    </UserControl.Resources>
    <Grid>
        <StackPanel>
            <StackPanel Orientation="Horizontal" Margin="5">
                <TextBlock Text="{Binding Source={StaticResource myDataSource}, Path=SessionSettings.Loginuser.Login}" Margin="0,0,5,0" />
                <TextBlock Text="{Binding Source={StaticResource myDataSource}, Path=SessionSettings.Loginuser.Loginmsg}" />
            </StackPanel>
            <Button Content="Logout" Margin="5" Click="Button_Click_1" />
            <ToggleSwitch Header="Show Deceased Patients" Margin="5" IsOn="{Binding Mode=TwoWay, Source={StaticResource myDataSource}, Path=SessionSettings.ShowDeceased}"/>
        </StackPanel>
    </Grid>
</UserControl>

The code behind is pretty straightforward, we just implement the button click method for “Logout.” The “Show Deceased Patients” ToggleSwitch binds to the SessionSettingsViewModel directly.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using Windows.Foundation;
using Windows.Foundation.Collections;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Primitives;
using Windows.UI.Xaml.Data;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Navigation;
using PRApp.ViewModels;
 
// The User Control item template is documented at http://go.microsoft.com/fwlink/?LinkId=234236
 
namespace PRApp
{
    public sealed partial class PRSettingsControl : UserControl
    {
        public PRSettingsControl()
        {
            this.InitializeComponent();
        }
 
        private void Button_Click_1(object sender, RoutedEventArgs e)
        {
            SessionSettingsViewModel.SessionSettings.Loginuser.LogOut();
 
            var frame = Window.Current.Content as Frame;
            frame.Navigate(typeof(Login));
        }
    }
}

Settings UI guidelines require us to provide a consistent user experience. In XAML/C# we can embed the above** UI view (usercontrol) into a popup XAML control that can emulate the flyout and light-dismiss behavior.

Integrating App Search functionality - iOS vs. Windows Store Apps

Search has become an integral part of modern mobile apps, specifically for apps that have lots of data. Apps on the iOS platform implement search using the UISearchBar class, which provides a standard search experience for iOS users.

***Figure 4: UISearchBar in iOS* (captured from iOS iPad* emulator)

As we discussed previously, the search interface in Windows Store apps is part of the charms bar. Apps can register themselves to react to search queries at the system level, not just when the app is in the foreground or active. So apps with search contract implemented could be invoked (“activated”) directly from the search interface.

A detailed step-by-step search example is available in the Windows Store apps developer documentation:

http://msdn.microsoft.com/en-us/library/windows/apps/xaml/hh868180.aspx

The screen shot of our sample app below shows the Windows Store app search interface, which was invoked via the charms bar.

***Figure 5: Different components of a Search (captured from simulator)

To show the search results, a separate XAML page can be used and customized as needed. Developers can choose to implement user convenient features like search filters or suggestions, automatic searches invoked when users start typing (mainly useful on devices with a keyboard interface), and if needed, enable incremental search too.

Before we delve into implementing the search interface in our sample app, we first need to declare the app supports “Search” in the App manifest.

***Figure 6: App Manifest showing search declaration

If you use the Visual Studio 2012 search contract template, it will automatically add this to app manifest. Detailed information on the search contract template can be found here:

http://msdn.microsoft.com/EN-US/library/windows/apps/jj655408(v=vs.10).aspx

***Figure 7: Visual Studio* 2012 search contract template

In addition, the search contract template automatically modifies the main application object (App.xaml.cs) to enable search. It will add code stubs for “OnSearchActivated” method, which is a kind of trigger point for your app to respond to search activation of the app. The App.xaml.cs code** snippets below show the search-related code in the main application object :

/// Invoked when the application is activated to display search results.
        /// </summary>
        /// <param name="args">Details about the activation request.</param>
        protected async override void OnSearchActivated(Windows.ApplicationModel.Activation.SearchActivatedEventArgs args)
        {
            await InitMainPage(args);
            ShowSearch(args.QueryText);
        }       
 
        private void ShowSearch(String query)
        {
            var frame = Window.Current.Content as Frame;
 
            var loggedinuser = SessionSettingsViewModel.SessionSettings.Loginuser;
            if (loggedinuser == null || !loggedinuser.LoginState)
            {
                frame.Navigate(typeof(Login));
            }
            else if (query != "")
                frame.Navigate(typeof(SearchResults), query);
            else
                frame.Navigate(typeof(Patients));
        }
 
        private void OnQuerySubmitted(object sender, SearchPaneQuerySubmittedEventArgs args)
        {
            ShowSearch(args.QueryText);
        }
 
        private void OnQueryChanged(object sender, SearchPaneQueryChangedEventArgs args)
        {
            ShowSearch(args.QueryText);
        }
 
        protected override void OnWindowCreated(WindowCreatedEventArgs args)
        {
            // Register QuerySubmitted handler for the window at window creation time and only registered once
            // so that the app can receive user queries at any time.
            SearchPane.GetForCurrentView().QuerySubmitted += new TypedEventHandler<SearchPane, SearchPaneQuerySubmittedEventArgs>(OnQuerySubmitted);
 
            SearchPane.GetForCurrentView().QueryChanged += new TypedEventHandler<SearchPane, SearchPaneQueryChangedEventArgs>(OnQueryChanged);
           
            AppSettings.Current.ContentBackgroundBrush = new SolidColorBrush(Windows.UI.Colors.Green);
            AppSettings.Current.AddCommand<PRSettingsControl>("PRApp Options", SettingsFlyout.SettingsFlyoutWidth.Narrow);
 
        }

As seen above, it is recommended we handle all possible search scenarios including a blank search query. If you have login or session state(s), you might want to put in additional checks (e.g., require them to login) before you let users search an app’s data.

Windows Store apps recommended checklist and guidelines for search can be found here:

http://msdn.microsoft.com/en-us/library/windows/apps/xaml/hh465233.aspx

In our sample app, we used the Visual Studio 2012 to search contract templates. It automatically adds an XAML page for displaying the results in a grid view (or list view), which is very similar to other page templates. It also adds code behind with support for search filters. We just need to plug in our app data into these template hooks. Please see the code snippet below**, which shows the modifications done in our sample app:

protected override void LoadState(Object navigationParameter, Dictionary<String, Object> pageState)
        {
            var queryText = navigationParameter as String;
 
            Results = new Dictionary<string, IEnumerable<PatientsViewModel>>();
            var filterList = new List<Filter>();
 
            var ps = PatientsViewModel.GetPatientsCached();
            if (ps != null)
            {
                Results["Lastname"] = ps.Where(p => p.Lastname.Contains(queryText.ToUpper()));
                filterList.Add(new Filter("Lastname", Results["Lastname"].Count()));
                Results["Firstname"] = ps.Where(p => p.Firstname.Contains(queryText.ToUpper()));
                filterList.Add(new Filter("Firstname", Results["Firstname"].Count()));
                Results["All"] = Results["Firstname"].Union(Results["Lastname"]);
                filterList.Insert(0, new Filter("All", Results["All"].Count(), true));
            }
            // Communicate results through the view model
            this.DefaultViewModel["QueryText"] = 'u201c' + queryText + 'u201d';
            this.DefaultViewModel["Filters"] = filterList;
            this.DefaultViewModel["ShowFilters"] = filterList.Count > 1;
        }
 
        /// <summary>
        /// Invoked when a filter is selected using the ComboBox in snapped view state.
        /// </summary>
        /// <param name="sender">The ComboBox instance.</param>
        /// <param name="e">Event data describing how the selected filter was changed.</param>
        void Filter_SelectionChanged(object sender, SelectionChangedEventArgs e)
        {
            // Determine what filter was selected
            var selectedFilter = e.AddedItems.FirstOrDefault() as Filter;
            if (selectedFilter != null)
            {
                // Mirror the results into the corresponding Filter object to allow the
                // RadioButton representation used when not snapped to reflect the change
                selectedFilter.Active = true;
 
                if (Results.ContainsKey(selectedFilter.Name))
                    this.DefaultViewModel["Results"] =
                         new List<PatientsViewModel>(Results[selectedFilter.Name]);
 
                // Ensure results are found
……
 

We can also enable automatic search invocation when a user starts typing, using the “ShowOnKeyboardInput” property of the search pane. In the below example, we enable this feature only on the main patients screen (which could potentially have thousands of entries). So when the user is on this page and starts typing a patient’s name, it will automatically bring up the search interface.

protected override void LoadState(Object navigationParameter, Dictionary<String, Object> pageState)
        {
            // check login status
            var loggedinuser = SessionSettingsViewModel.SessionSettings.Loginuser;
            if (loggedinuser == null || !loggedinuser.LoginState)
            {
                var frame = Window.Current.Content as Frame;
                frame.Navigate(typeof(Login));
                return;
            }
 
            this.loginUser.Text = loggedinuser.Login + " (" + loggedinuser.Fullname + ")";
            this.DefaultViewModel["Patients"] = PatientsViewModel.GetPatients();
            // enable type to search.
            SearchPane.GetForCurrentView().ShowOnKeyboardInput = true;
        }
 
        protected override void SaveState(Dictionary<String, Object> pageState)
        {
            // disable type to search.
            SearchPane.GetForCurrentView().ShowOnKeyboardInput = false;
        }

This code** snippet is in the code behind file for the patients XAML page. Similarly, we can enable or disable this feature on a per-page basis depending on user requirements.

Summary

Lines-of-business apps tend to have search, settings, and complex app state maintenance in them. In this article we reviewed how iOS apps implement these features and compared and contrasted them to Windows Store apps. Using a sample app and source code, we discussed how to implement these features on Windows 8.

Notices

*Other names and brands may be claimed as the property of others

**This sample source code is released under the Intel OBL Sample Source Code License (MS-LPL Compatible) , Microsoft Limited Public License and Visual Studio 2012 License. Copyright© 2012 Intel Corporation. All rights reserved.

***This sample figure is released under the Intel OBL Sample Source Code License (MS-LPL Compatible) and Xcode License. Copyright© 2012 Intel Corporation. All rights reserved.

Download Article

Download Porting Advanced User Interfaces From iOS* To Windows 8* [PDF 620KB]

Abstract

This article discusses porting advanced user interface features from an iOS app to a Windows Store app. We use an electronic medical record (EMR) application for this case study.

1 Introduction

In recent years, tablets, as new forms of mobile computing platforms, have quickly moved from the consumer electronics spaces into the business and enterprise computing domains. After the release of Windows 8 operating systems earlier this year, we felt there was a need to provide some quick start tutorials for developers on how to port their existing apps from other platforms such as iOS to Windows 8, and start developing new apps on Intel Ultrabook™ devices, tablets, and other Intel architecture-based devices running Windows 8. This article serves this objective and focuses on the advanced user interface topics.

On iOS, natively Objective-C is the main development language. For Windows Store apps, you have multiple choices available, including Visual C#, HTML / JavaScript*, and others. In this case study, we use Visual C#* as the development language of choice.

1.1 From Xcode* 4 to Visual Studio* 2012

Like the Xcode tools package on OS X* for iOS application developers, Visual Studio 2012 provides an integrated development environment (IDE) for Windows Store app developers. Also like the Interface Building design tool on Xcode 4, which supports storyboarding (Figure 1), Visual Studio 2012 includes a XAML Designer tool (Figure 2).

Figure 1. The Interface Builder in Xcode 4

Figure 2. Visual Studio 2012 XAML Designer

1.2 The Case Study Project

This article is one of a series based on a case study project. In the project (link to the folder of other articles based on this project), we ported a medical record management application from iOS to Windows 8. The basic requirements of the application include:

• Show a list of patients
• Show the personal and medical information of a specific patient, which includes identity, billing, vitals, lab tests, medical images, etc.
• Display detailed graphs and images when selected

This article will cover the advanced UI features of the project.

2 High Level UI Design and Navigation Patterns

On iOS, we can use the split-view controller to present a master view and a detailed view on the screen. We can use table views or tab bar views to group different categories of information on the view. Figure 3 shows the split view along with the master table view and the detailed table view. The left pane of the split view shows the scrollable patient list. The right pane shows the medical records associated with the selected patient in the list. We use a table view to put the medical record categories on the same view. We can also use the tab bar view, with each tab view displaying a specific medical record category. Figure 4 shows how this view is created in Xcode 4 storyboard.

Figure 3. On iOS, a split view controller and its master table view and detailed table view

Figure 4. Use Xcode storyboard to create a split view and its master and detailed table views

In a Windows Store app, we can accommodate this design by following the Windows Store app hierarchical system of navigation pattern (Figure 5). The first level page shows a grid view that includes a tile for each patient (Figure 6). The second level page is a grouped item page that includes the medical records for the patient selected from the first level page (Figure 7). The third level page is a group detail page that shows the specific category of medical records selected from the second level page (Figure 8). We can also have a fourth level page that shows the item details, for example, the actual X-ray image selected from the third level page.

Figure 5. Windows Store app hierarchical system of navigation.

Figure 6. In the Windows Store app, the root level grid view includes tiles for the patient list.

Figure 7. In the Windows Store app, the second level page shows the medical records associated with the selected patient.

Figure 8. In the Windows Store app, the third level page shows the group selected from the second level page.

In Visual Studio 2012 projects, the UI page is defined with a “XAML” file, and a C# implementation file (.cs) associated with it. Because the transitions from one page to another page usually originate from user actions, for example, when a grid view item is pressed, naturally the event listeners are the places used to handle the navigations. Figure 9 shows our first level page PatientsListPage where we define an ItemsGridView_ItemClick event listener.

<common:LayoutAwarePage
    x:Name="pageRoot"
    x:Class="PRApp.PatientsListPage"
    DataContext="{Binding DefaultViewModel, RelativeSource={RelativeSource Self}}"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:PRApp"
    xmlns:common="using:PRApp.Common"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d">

…
<GridView
            x:Name="itemGridView"
            AutomationProperties.AutomationId="ItemsGridView"
            AutomationProperties.Name="Items"
            TabIndex="1"
            Grid.RowSpan="2"
            Padding="116,136,116,46"
            ItemsSource="{Binding Source={StaticResource itemsViewSource}}"
            ItemTemplate="{StaticResource PatientsItemStyle1}"
            SelectionMode="None"
            IsSwipeEnabled="false" IsItemClickEnabled="True" ItemClick="ItemsGridView_ItemClick"/>    

…
</common:LayoutAwarePage>

Figure 9 The GridView in PatientsListPage XAML file includes an ItemGsGridView_ItemClick event listener (**)

Figure 10 shows in PatientsListPage.xaml.cs, we implement the ItemsGridView_ItemClick method, which calls the Windows Runtime Frame.Navigate() method to create a second level page PatientGroupItemsPage object and show as the current page on screen. When we construct the PatientGroupItemsPage object, we pass in the clicked item, which is a PatientsViewModel object.

namespace PRApp
{
    /// <summary>
    /// A page that displays a collection of item previews. 
	/// In the Split Application this page
    /// is used to display and select one of the available groups.
/// </summary>

    public sealed partial class PatientsListPage :  PRApp.Common.LayoutAwarePage
    {        
…
        void ItemsGridView_ItemClick(object sender, ItemClickEventArgs e)
        {
            this.Frame.Navigate(typeof(PatientGroupItemsPage), ((PatientsViewModel)e.ClickedItem));
        }
…
    }
}

Figure 10 PatientsListPage.cs implements the ItemsGridView_ItemClick event listener (**)

In the LoadState method in PatientGroupItemsPage.xaml.cs (Figure 11), we retrieve the patient object from this parameter and construct the Groups collection for the grid view data model.

/// <summary>

/// Populates the page with content passed during navigation. Any saved state is also

/// provided when recreating a page from a prior session.

/// </summary>

/// <param Title="navigationParameter">The parameter value passed to

/// <see cref="Frame.Navigate(Type, Object)"/> when this page was initially requested.

/// </param>

/// <param Title="pageState">A dictionary of state preserved by this page during an earlier

/// session. This will be null the first time a page is visited.</param>

protected override void LoadState(Object navigationParameter, Dictionary<String, Object> pageState)

{
    // TODO: Assign a collection of bindable groups to this.DefaultViewModel["Groups"]

    PatientsViewModel patient = (PatientsViewModel)navigationParameter;
    this.DefaultViewModel["Groups"] = pgipvm.GetGroups(_patient.id);
}

Figure 11 The LoadState method of PatientGroupItemsPage.xaml.cs

3 Windows Project Templates and Data Binding

In Figure 7 and Figure 8, items are grouped and shown nicely in the grid views. Visual Studio 2012 Windows Store project templates provide a powerful basis to construct these user interface pages. The predefined project templates include grouped items page, group detail page, item detail page, etc. We use the X-rays group detail page as an example here.

In Visual Studio 2012’s Solution Explorer window, right click the project name and select “Add -> New Item…” from the pop-up menu. Select “Visual C#” on the left pane. On the center pane, we see the list of the predefined page templates. Among those templates, we select “Group Item Page.” A preview of the template is shown on the right pane. We also enter a name for the page in the text box at the bottom of the dialog (Figure 12), and press the “Add” button. Visual Studio 2012 now generates a file named “XRayImagesGroupDetailPage.xaml” file and a file named “XRayImagesGroupDetailPage.xaml.cs” in the project.

Figure 12 Add New Item dialog shows the Window Store project templates (**)

If we inspect the generated XRayImagesGroupDetailPage.xaml file, we can see this page is bound to “DefaultViewModel” data context and the grid view items in this page are bound to the “Items” as the collection view source (Figure 13).

<common:LayoutAwarePage
    x:Name="pageRoot"
    x:Class="PRApp.XRayImagesGroupDetailPage"
    DataContext="{Binding DefaultViewModel, RelativeSource={RelativeSource Self}}"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:PRApp"
    xmlns:common="using:PRApp.Common"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d">   
 <Page.Resources>

        <!-- Collection of items displayed by this page -->
        <CollectionViewSource
            x:Name="itemsViewSource"
            Source="{Binding Items}"/>
    </Page.Resources>
…

Figure 13 XRayImagesGroupDetailPage.xaml specifies the binding data source (**).

In XRayImagesGroupDetailPage.xaml.cs, we can see XRayImagesGroupDetailPage is derived from PRApp.Common.LayoutAwarePage (Figure 14).

namespace PRApp

{

/// <summary>

/// A page that displays an overview of a single group, including a preview of the items

/// within the group.

/// </summary>

public sealed partial class XRayImagesGroupDetailPage : PRApp.Common.LayoutAwarePage

{…

Figure 14 XRayImagesGroupDetailPage derives from LayoutAwarePage

In Visual Studio 2012, if we expand the “Common” folder generated under the project (Figure 15), we can see Visual Studio has generated a group of files under it. Among these files, LayoutAwarePage.cs contains the class that we derive the XRayImagesGroupDetailPage from.

Figure 15 The "Common" folder in the project.

The “Common” folder also includes the “BindableBase.cs” file. We derive a data model for the view from this class. Figure 16 provides the outlines of the XRayImagesGroupDetailPageViewModel class.

…

namespace PRApp.ViewModels
{
    public abstract class XRayImageDataCommon : BindableBase
    {
    }

    public class XRayImageDataItem : XRayImageDataCommon
    {
    }

    public class XRayImageDataGroup : XRayImageDataCommon
{
…
        private ObservableCollection<XRayImageDataItem> _items = new ObservableCollection<XRayImageDataItem>();
        public ObservableCollection<XRayImageDataItem> Items
        {
            get { return this._items; }
        }
    …

    }

    public sealed class XRayImagesGroupDetailPageViewModel
    {
        …
        public static XRayImageDataGroup GetGroup(string uniqueId)
        {
            …
        }
        …
    }

}
…

Figure 16 XRayImagesGroupDetailPageViewModel.cs, which the group and item classes derived from BindableBase class (**)

To connect the view and the data sources, in XRayImagesGroupDetailPage.xaml class’s LoadState method, we pass the PRApp.Common.BindableBase derived group and items object to DefaultViewModel (Figure 17).

…
        /// <summary>
        /// Populates the page with content passed during navigation.  Any saved state is also
        /// provided when recreating a page from a prior session.
        /// </summary>
        /// <param name="navigationParameter">The parameter value passed to
        /// <see cref="Frame.Navigate(Type, Object)"/> when this page was initially requested.
        /// </param>
        /// <param name="pageState">A dictionary of state preserved by this page during an earlier
        /// session.  This will be null the first time a page is visited.</param>
        protected override void LoadState(Object navigationParameter, Dictionary<String, Object> pageState)
        {
            // TODO: Assign a bindable group to this.DefaultViewModel["Group"]
            // TODO: Assign a collection of bindable items to this.DefaultViewModel["Items"]
            var group = XRayImagesGroupDetailPageViewModel.GetGroup((String) navigationParameter);
            this.DefaultViewModel["Group"] = group;
            this.DefaultViewModel["Items"] = group.Items;
        }
…

Figure 17. In XRayImagesGroupDetailPage.cs, the view items are connected with the data source in the LoadState method. (**)

4 Summary

In this article, we used an EMR app to investigate how to port some advanced UI features from iOS to the new Windows Store app platforms. From this case study, we can see Windows Runtime provides a rich set of features and development tools for developing impressive UI features.

About the Author

Miao Wei is a software engineer with Intel Corporation’s Software and Services Group. He currently works on the Intel Atom™ Scale and Enabling projects. He has over 15 years of experience in developing mobile platforms, web browsers, location-based applications and services, digital map databases, and other software products.

*Other names and brands may be claimed as the property of others.

Copyright © 2012 Intel Corporation. All rights reserved.

OpenGL is a registered trademark and the OpenGL ES logo is a trademark of Silicon Graphics Inc. used by permission by Khronos.

Download Article

Download Porting the User Interface from an iOS* Basic App to Windows* 8 [PDF 620KB]

Table of Contents

• 1. Introduction. 2
• 2. Basic UI Navigation and Layout 2
  • 2.1 Navigation Hierarchy. 2
  • 2.1.1. Storyboard design of iOS app. 2
  • 2.1.2. Hierarchy system of Windows store app. 4
  • 2.2 Basic UI Layout 6
  • 2.2.1. Choose a layout 6
  • 2.2.2. Provide adaptive layouts 8
• 3. Basic Component/Layout Design of a View/Page. 10
  • 3.1. Button Design. 10
  • 3.2. Text box. 10
  • 3.3. Grid View. 11
  • 3.4. Group Grid View. 12
  • 3.4.1. Xaml page. 12
  • 3.4.2. Use grouped data source. 12
• 4. Main Entry Point Design. 14
  • 4.1.Logo. 15
  • 4.2. Tile. 16
• 5. Summary. 17
• 6. About the Author 17

1. Introduction

Windows* 8 is a great platform for writing apps with increased productivity, easy deployment, and natural human interactions. This article shows you how to port the basic UI design of an application from iOS* to Windows 8 on devices with a variety of form-factors like tablets and Ultrabook™ devices. The content of the article will be based on a case study on a healthcare app created for clinic/hospital physicians with patient electronic medical records (EMR).

2. Basic UI Navigation and Layout

We use a healthcare app as a case study. The healthcare app allows physicians to view patient EMRs. Windows 8 provides designs with fluid and flexible user experiences on a variety of devices. We discuss the navigation hierarchy and the layout designs and use the case study to illustrate how to port from iOS* to Windows Store apps.

2.1. Navigation Hierarchy

The navigation design directly impacts users’ interactions with the app. On iOS*, the storyboard is recommended for navigation design. For Windows Store apps, XAML pages with navigation methods are recommended for C# developers. We discuss how to port the storyboard design to the XAML page designs.

2.1.1. Storyboard design of iOS* app

In our case study app, the storyboard is designed with a split-view with a master-view controller and a detail-view controller.

2.1.1.1. Screen shot

Figure 1 is a screen shot of the iOS* app in landscape view.

Figure 1: Screen shot of the main page of the healthcare iOS* app (Note that names and SSNs shown in the figure are not real.)

2.1.1.2. Storyboard design

To design the UI in Figure 1, the storyboard is laid out with split-views with a master-view controller and a detail-view controller. The master-view controller is connected to a navigation controller that can navigate to a table view with a list of patients. The detail view controller is connected to a navigation controller that can navigate to a detail view with static images, text boxes, and a container with static table views. Figure 2 shows the storyboard design for the iOS* app on an iPad*.

Figure 2: Storyboard design of the main page of the healthcare iOS* app***

2.1.2. Hierarchy system of Windows Store app

To provide a similar or better user experience on Windows* 8, we port the split-view design from iOS* to a hierarchy system on Windows* 8 with XAML pages. We also follow the Windows Store app design guidelines from Microsoft.

2.1.2.1. Hierarchical system

The two kinds of navigation designs for Windows Store apps are: the flat system and the hierarchical system. The flat system design is best usually when the core scenarios involve rapid switching between a small number of pages and all the pages belong to the same hierarchical level. The hierarchical system applies to the scenario of the separation of content into different sections and different levels of details. Thee hub page will have multiple group entries and each group includes pages with items that belong to this group. For some apps, the UI design could be a mixture of the hierarchical and flat systems.

To port the iOS* split-view storyboard design to Windows* 8, we chose the hierarchical system as the major navigation design for the Windows Store app. For the pages within one group, we use the flat system to navigate between the related pages. Figure 3 is the navigation design of the healthcare app, which is a mixture of both systems. The main design is the hierarchical system, and the design for pages within a group is a flat system. Because many Windows Store apps are designed with large tiles due to the larger area for displaying key information, we design the pages with tiles.

Figure 3: Navigation design of the Windows* Store healthcare app

2.1.2.2. Screen shot

Figure 3 is the screen shot of the first page with patient entries, which includes tiles of all the patients. Users also have a search option from the Charm menu to search for a particular patient. Figure 4 is the screen shot for a patient. The page includes the abstract information of the patient with groups. This is the hub page for the next level of pages with detailed items in each group.

Figure 3: Screen shot of the patient entry page of the healthcare app on Windows* 8

Figure 4: Screen shot of the main page of the healthcare iOS* app

2.2. Basic UI Layout

For a Windows Store app, we need to design a layout for each page in XAML format. In iOS*, components are utilized for the layout of a page. In Windows* 8, templates with specific layouts are typically used.

2.2.1. Choose a layout

Windows Store app developers have many options of layout templates to choose from for the most flexible and adaptable user interface. Flexible box layout, grid alignment, multi-column layout, and regions layout are available depending on the context and user needs of the app. For the healthcare app, we want to present the physician clear information for a specific patient with divided groups like personal profile, notes, lab results, vitals, and X-ray images. We choose the grid alignment layout as it can divide available space into columns and rows to show user information clearly. Multiple grid layout templates can be selected in Visual Studio.

Sample code 1 shows the grid alignment layout template, LayoutRootStyle, used in our case study.

<!--
        This grid acts as a root panel for the page that defines two rows:
        * Row 0 contains the back button and page title
        * Row 1 contains the rest of the page layout
    -->
    <Grid Style="{StaticResource LayoutRootStyle}">
        <Grid.RowDefinitions>
            <RowDefinition Height="140"/>
            <RowDefinition Height="*"/>
        </Grid.RowDefinitions>

        <!-- Horizontal scrolling grid used in most view states -->
        <GridView
            x:Name="itemGridView"
            AutomationProperties.AutomationId="ItemsGridView"
            AutomationProperties.Name="Items"
            TabIndex="1"
            Grid.RowSpan="2"
            Padding="116,136,116,46"
            ItemsSource="{Binding Source={StaticResource itemsViewSource}}"
            ItemTemplate="{StaticResource PatientsItemStyle1}"
            SelectionMode="None"
            IsSwipeEnabled="false" IsItemClickEnabled="True" ItemClick="ClickItem"/>

        <!-- Vertical scrolling list only used when snapped -->
        <ListView
            x:Name="itemListView"
            AutomationProperties.AutomationId="ItemsListView"
            AutomationProperties.Name="Items"
            TabIndex="1"
            Grid.Row="2"
            Visibility="Collapsed"
            Margin="0,-10,0,0"
            Padding="10,0,0,60"
            ItemsSource="{Binding Source={StaticResource itemsViewSource}}"
            ItemTemplate="{StaticResource Standard80ItemTemplate}"
            SelectionMode="None"
            IsSwipeEnabled="false" IsItemClickEnabled="True" ItemClick="ClickItem"/>

        <!-- Back button and page title -->
        <Grid>
            <Grid.ColumnDefinitions>
                <ColumnDefinition Width="Auto"/>
                <ColumnDefinition Width="*"/>
            </Grid.ColumnDefinitions>
            <Button x:Name="backButton" Click="GoBack" IsEnabled="{Binding Frame.CanGoBack, ElementName=pageRoot}" Style="{StaticResource BackButtonStyle}"/>
            <TextBlock x:Name="pageTitle" Grid.Column="1" Text="Patients" IsHitTestVisible="false" Style="{StaticResource PageHeaderTextStyle}"/>            
        </Grid>
        <StackPanel Margin="10" Grid.Row="0" Orientation="Horizontal" HorizontalAlignment="Right">
            <TextBlock Grid.Row="1" Text="LoginUser:"/>
            <TextBlock x:Name="loginUser" Grid.Row="1" Text="" Margin="5,0"/>
            
        </StackPanel>
</Grid>

Sample Code 1. Grid alignment layout in our case study**

2.2.2. Provide adaptive layouts

As users rotate their devices, Windows provides adaptive layouts for orientations and views. Figure 5 shows screen shots of a layout in different orientations and views.

Figure 5: Screenshots of layouts for the healthcare iOS* app

Sample code 2 is the XAML code of a page with different layouts in our case study. This is auto-generated code when you choose a template page to start with in Visual Studio.

<VisualStateManager.VisualStateGroups>
    <!-- Visual states reflect the application's view state -->
        <VisualStateGroup x:Name="ApplicationViewStates">
                <VisualState x:Name="FullScreenLandscape"/>
                <VisualState x:Name="Filled"/>

                <!-- The entire page respects the narrower 100-pixel 
                   margin convention for portrait -->
                <VisualState x:Name="FullScreenPortrait">
                    <Storyboard>
                        <ObjectAnimationUsingKeyFrames 
                         Storyboard.TargetName="backButton" 
                         Storyboard.TargetProperty="Style">
                            <DiscreteObjectKeyFrame KeyTime="0" 
                    Value="{StaticResource PortraitBackButtonStyle}"/>
                        </ObjectAnimationUsingKeyFrames>
                        <ObjectAnimationUsingKeyFrames 
                        Storyboard.TargetName="itemGridView" 
                        Storyboard.TargetProperty="Padding">
                            <DiscreteObjectKeyFrame KeyTime="0"                       
                     Value="96,136,86,56"/>
                        </ObjectAnimationUsingKeyFrames>
                    </Storyboard>
                </VisualState>

                <!--
                    The back button and title have different styles 
              when snapped, and the list representation is substituted
                    for the grid displayed in all other view states
                -->
                <VisualState x:Name="Snapped">
                    <Storyboard>
                        <ObjectAnimationUsingKeyFrames 
                        Storyboard.TargetName="backButton" 
                        Storyboard.TargetProperty="Style">
                            <DiscreteObjectKeyFrame KeyTime="0"  
                    Value="{StaticResource SnappedBackButtonStyle}"/>
                        </ObjectAnimationUsingKeyFrames>
                        <ObjectAnimationUsingKeyFrames 
                        Storyboard.TargetName="pageTitle"               
                        Storyboard.TargetProperty="Style">
                            <DiscreteObjectKeyFrame KeyTime="0" 
               Value="{StaticResource SnappedPageHeaderTextStyle}"/>
                        </ObjectAnimationUsingKeyFrames>

                        <ObjectAnimationUsingKeyFrames                
                         Storyboard.TargetName="itemListView" 
                         Storyboard.TargetProperty="Visibility">
                        <DiscreteObjectKeyFrame KeyTime="0" 
                          Value="Visible"/>
                        </ObjectAnimationUsingKeyFrames>
                        <ObjectAnimationUsingKeyFrames 
                          Storyboard.TargetName="itemGridView" 
                          Storyboard.TargetProperty="Visibility">
                            <DiscreteObjectKeyFrame KeyTime="0" 
                             Value="Collapsed"/>
                        </ObjectAnimationUsingKeyFrames>
                    </Storyboard>
                </VisualState>
            </VisualStateGroup>
        </VisualStateManager.VisualStateGroups>

Sample Code 2. Adaptive layout for different orientation and views **

3. Basic Component/Layout Design of a View/Page

In this section, we discuss how to port the basic component/layout of a page from iOS* to Windows* 8. On iOS*, we design a page with components chosen from an object library. For Windows Store apps, we have XAML format descriptions and templates for basic component/layout designs.

3.1. Button Design

Buttons are described in an XAML page with a format. Sample code 3 shows an example of a button design. The click is associated with a function named “GoBack,” which is already defined in LayoutAwarePage class. Whenever the button is clicked, the GoBack function in the class is executed. The style of the button follows the template as BackButtonStyle.

<Button x:Name="backButton" Click="GoBack" IsEnabled="{Binding Frame.CanGoBack, ElementName=pageRoot}"
Style="{StaticResource BackButtonStyle}"/>

Sample Code 3. Button design in a XAML page**

3.2. Text Box

Text boxes are described in an XAML page with a format. Sample code 4 shows an example of a text box design. It includes multiple properties with specific values. The style of the text box also follows a template as PageHeaderTextStyle.

<TextBlock x:Name="pageTitle" 
    Grid.Column="1"
    Text="Patient Medical Record"
    IsHitTestVisible="false"
    Style="{StaticResource PageHeaderTextStyle}"   
    Grid.ColumnSpan="2"/>

Sample Code 4. Text box design in a XAML page**

3.3. Grid View

Grid views describe the layout of an XAML page with grids. Sample code 5 is an example of a grid view in landscape layout. The data source is bound to the specified static resource patientgroupedItemsViewSourceList. Please refer to Section 3.1.2. The source list is defined in C# code for the page. The item grid template uses the Stand250x250ItemTemplate. Also, in the C# file for the page, we need to define the function ItemView_ItemClick so users can click on items in the grid.

<!—Horizontal scrolling grid used in most view states -->
 <GridView
      x:Name="itemGridView"
      AutomationProperties.AutomationId="ItemGridView"
      AutomationProperties.Name="Grouped Items"
      Grid.Row="1"
      Margin="0,-3,0,0"
      Padding="116,0,40,46"
      ItemsSource="{Binding Source=
      {StaticResource patientgroupedItemsViewSourceList}}"
      ItemTemplate="{StaticResource Standard250x250ItemTemplate}"
      SelectionMode="None"
      IsItemClickEnabled="True"
      ItemClick="ItemView_ItemClick">

      <GridView.ItemsPanel>
          <ItemsPanelTemplate>                        
             <VirtualizingStackPanel  Orientation="Horizontal"/>
          </ItemsPanelTemplate>
      </GridView.ItemsPanel>
      <GridView.GroupStyle>
          <GroupStyle>
             <GroupStyle.HeaderTemplate>
                 <DataTemplate>
                    <Grid Margin="1,0,0,6">
                       <Button
                        AutomationProperties.Name="Group Title"
                       Content="{Binding Title}"
                       Click="Header_Click"
                       Style="{StaticResource TextButtonStyle}"/>
                    </Grid>
                  </DataTemplate>
             </GroupStyle.HeaderTemplate>
             <GroupStyle.Panel>
                <ItemsPanelTemplate>
      <VariableSizedWrapGrid Orientation="Vertical" Margin="0,0,80,0"/>
                </ItemsPanelTemplate>
             </GroupStyle.Panel>
         </GroupStyle>
      </GridView.GroupStyle>
     </GridView>

Sample Code 5. Grid view design of the hub page for a patient**

3.4. Group Grid View

When we have multiple grids to be organized into different groups, we can group them into different categories.

3.4.1. XAML page

First, we need to specify the data source to bind for the grid view for groups in XAML file. The sample code 6 shows the binding of the data source in our case study. Here patientgroupedItemsViewSourceList is the group source for the grid view.

<Page.Resources>

        <!--
            Collection of grouped items displayed by this page, bound  
            to a subset of the complete item list because items in 
            groups cannot be virtualized
        -->
        
       <CollectionViewSource
            x:Name="patientgroupedItemsViewSource"
            Source="{Binding MedicalRecordGroups}"
            IsSourceGrouped="true"
           />    

        <CollectionViewSource   
            x:Name="patientgroupedItemsViewSourceList" 
            IsSourceGrouped="True" 
            ItemsPath="MedicalRecordItems"/>

    </Page.Resources>

Sample Code 6. Bind the data to the grid view in an XAML page**

3.4.2. Use grouped data source

Next we need to use the grouped data source for the grid view. Sample code 7 provides an example on how to provide the grouped data source. The function should be called when the page is loaded.

private void PopulateGroups(PatientsViewModel patient)
{
        List<MedicalRecordCategory> MedicalRecordCategories = new 
                          List<MedicalRecordCategory>();
            int count = 0;

            MedicalRecordCategory newMedicalRecordCategory = new 
                                  MedicalRecordCategory();
            newMedicalRecordCategory.Title = "Patient Profile";
            newMedicalRecordCategory.MedicalRecordItems.Add(new 
                MedicalRecordItem() {
                Title = "Patient Information",
                Complete = true,
                Subtitle = patient.Firstname + "" + patient.Lastname,
                Image = patient.Image,
                MedicalRecordCategoryId = 1,
                MedicalRecordItemId = 1
            });
            newMedicalRecordCategory.MedicalRecordItems.Add(new 
                MedicalRecordItem() {
                Title = "Family History ",
                Complete = true,
                Subtitle = patient.Insurance,
                Image = "Assets/history.jpg",
                MedicalRecordCategoryId = 1,
                MedicalRecordItemId = 2

            });
            newMedicalRecordCategory.MedicalRecordItems.Add(new  
                MedicalRecordItem() { 
                Title = "Billing ", Complete = false, Subtitle = 
                patient.Lastvisit,
                Image = "Assets/billing.jpg",
                MedicalRecordCategoryId = 1,
                MedicalRecordItemId = 3
            });
            MedicalRecordCategories.Add(newMedicalRecordCategory);

            if (patient_note_count == 0)
            {
                newMedicalRecordCategory = new MedicalRecordCategory();
                newMedicalRecordCategory.Title = "Notes";
                newMedicalRecordCategory.MedicalRecordItems.Add(new 
                MedicalRecordItem()
                {
                    Title = "No notes",
                    Complete = true,
                    Subtitle = "",
                    Image = "Assets/notes.jpg",
                    MedicalRecordCategoryId = 2,
                    MedicalRecordItemId = 0
                });
                MedicalRecordCategories.Add(newMedicalRecordCategory);
            }
            else 
            {
                newMedicalRecordCategory = new MedicalRecordCategory();
                newMedicalRecordCategory.Title = "Notes";
                for (count = 0; count < patient_note_count; count++)                               
                {

                newMedicalRecordCategory.MedicalRecordItems.Add(new 
                MedicalRecordItem()
                {
                    Title = 
                    PatientNotesArray[count].creatorid.ToString(),
                    Complete = true,
                    Subtitle = PatientNotesArray[count].note,
                    Image = "Assets/notes.jpg",
                    MedicalRecordCategoryId = 2,
                    MedicalRecordItemId = count+1
                });
                }                
                MedicalRecordCategories.Add(newMedicalRecordCategory);
            }……
            // Add source to the grid view groups
            patientgroupedItemsViewSourceList.Source = 
            MedicalRecordCategories;
        }

}

    public class MedicalRecordCategory
    {
        public MedicalRecordCategory()
        {
            MedicalRecordItems = new 
            ObservableCollection<MedicalRecordItem>();
        }
        public string Title { get; set; }
        public ObservableCollection<MedicalRecordItem> 
                MedicalRecordItems { get; private set; }      
    }

    public class MedicalRecordItem
    {
        public int MedicalRecordCategoryId { get; set; }
        public int MedicalRecordItemId { get; set; }
        public string Title { get; set; }
        public string Subtitle { get; set; }
        public string Image { get; set; }
        public bool Complete { get; set; }
        public string MedicalRecordCategory { get; set; }
    }

Sample Code 7. Provide the grouped data source**

The main hub page for a patient becomes a page with several categories. Each category includes multiple items if they exist. Please refer to Figure 4.

4. Main Entry Point Design

For iOS* apps, a logo is the main entry point of an app. Windows Store apps have two options: either a logo or a tile window with key messages to users.

4.1. Logo

The logo design is similar to iOS* apps. Windows Store apps have a manifest file for defining some properties of the app including the logo specification. A developer only needs to specify the logo images for the general logo, wide logo, and small logo with different required sizes. Figure 6 is the screen shot of the manifest values for logo entries.

Figure 6: Screen shot of manifest file for logo specification

We have specified the wide logo as shown in Figure 7.

Figure 7: Logo of the healthcare app

4.2. Tile

A unique UI feature of Windows Store apps is a tile with key messages for users as the main entry. Here, we discuss how to develop a tile for the healthcare app. We want to design the tile as shown in Figure 8. When not in use, the logo appears as shown in the left image. After a user logs in, the logo changes to show the users’ name and the number of appointments for the day. After a time interval, the tile switches to a peek message window with more detailed information. This new Windows UI gives users a way to quickly check key information.

Figure 8: Tile design of the healthcare app

Sample code 8 shows the implementation of the tile design. The UpdateTileWithImageText should be called in the page after the user is logged in. The function utilizes a template for the tile design. This template comes from the NotificationsExtensions library. Please refer to the website on how to use this library for different styles of tiles.

void UpdateTileWithImageText(int numofpatients)
      {
            // Note: This sample contains an additional project, 
             NotificationsExtensions.
            // NotificationsExtensions exposes an object model for 
            creating notifications, but you can also modify the xml
            // of the notification directly using 
             TileUpdateManager.GetTemplateContent(TileTemplateType)

            // Create notification content based on a visual template.
            ITileWidePeekImageAndText02 tileContent = 
            TileContentFactory.CreateTileWidePeekImageAndText02();

            tileContent.TextBody1.Text = loginUser.Text;
            tileContent.TextBody2.Text = "     5 appointments today";
            tileContent.TextBody3.Text = "" + 
                 numofpatients.ToString() + " patients in records";
            tileContent.TextBody4.Text = "" + "20 new patients";
            tileContent.TextBody5.Text = "" + 
                                   DateTime.Today.ToString();
            //tileContent.TextCaption2.Text =  " more patients";
            tileContent.Image.Src = "images/LogoBlueWide.png";
            tileContent.Image.Alt = "Logo image";

            // Users can resize tiles to square or wide.
            // Apps can choose to include only square assets (meaning 
            //   the app's tile can never be wide), or
            // include both wide and square assets (the user can resize 
            //   the tile to square or wide).
            // Apps should not include only wide assets.

            // Apps that support being wide should include square tile 
            //    notifications since users
            // determine the size of the tile.

            // create the square template and attach it to the wide 
            // template
            ITileSquareImage squareContent = 
            TileContentFactory.CreateTileSquareImage();
            squareContent.Image.Src = "images/graySquare.png";
            squareContent.Image.Alt = "Gray image";
            tileContent.SquareContent = squareContent;

            // Send the notification to the app's application tile.
            TileUpdateManager.CreateTileUpdaterForApplication()
                    .Update(tileContent.CreateNotification());

        }

        private void ClearTile()
        { 
           TileUpdateManager.CreateTileUpdaterForApplication().Clear();
        }

Sample Code 8. Update/clear tile**

5. Summary

In the article we discuss how to port the basic UI design from an iOS* app to a Windows Store app. We study a healthcare app and discuss some implementation experiences on the navigation and layout design, basic UI component design, and main entry point design. We expect our case study can give developers a quick start on porting apps from iOS* to Windows 8.

6. About the Author

Sushu Zhang is a software engineer in Intel's Software and Services Group. She is working on ISV scale enabling for Intel-based platforms with Android* / Windows OS. Most recently, Sushu has been involved with several energy-efficiency projects at Intel. Prior to joining Intel, Sushu worked at Microsoft on Windows energy efficiency. Sushu earned a PhD in Computer Science at the Arizona State University. Her research area was system-level power and thermal management.

Intel, the Intel logo, and Ultrabook are trademarks of Intel Corporation in the U.S. and other countries.

*Other names and brands may be claimed as the property of others

**This sample source code is released under the Intel OBL Sample Source Code License (MS-LPL Compatible) , Microsoft Limited Public License and Visual Studio 2012 License. Copyright© 2012 Intel Corporation. All rights reserved.

***This sample figure is released under the Intel OBL Sample Source Code License (MS-LPL Compatible) and Xcode* License. Copyright© 2012 Intel Corporation. All rights reserved.

Download Article

Download Software Occlusion Culling [PDF 1.1MB]

Abstract

This article details an algorithm and associated sample code for software occlusion culling which is available for download. The technique divides scene objects into occluders and occludees and culls occludees based on a depth comparison with the occluders that are software rasterized to the depth buffer. The sample code uses frustum culling and is optimized with Streaming SIMD Extensions (SSE) instruction set and multi-threading to achieve up to 8X performance speedup compared to a non-culled display of the sample scene.

Introduction

Any realistic 3D scene contains objects that are at least partially occluded. For example, buildings and the environment typically occlude a large number of objects in the scene. The z-buffer is the last stage in the rendering pipeline where object fragments can be identified as occluded. However, occluded objects are still sent to the GPU for rendering, to no visual effect. If we can lower the computational cost of occluded objects, we can improve the application’s performance with no impact to the scene quality.

Using the GPU-generated depth buffer on the CPU is one technique for early detection of occluded objects. However, doing occlusion culling in this manner may cause latency and overhead issues and introduce visual artifacts since the GPU may be several frames behind the CPU. To avoid these problems, we propose software rasterization of the depth buffer on the CPU.

In this approach, we use the CPU to rasterize the depth buffer. Then, we use axis-aligned bounding box tests to determine if an object is occluded. Occluded objects are removed from the rendering pipeline to reduce overhead. We do not cull visible and partially visible objects; we submit them to the GPU for rendering. The sample code associated with the Software Occlusion Culling sample implements this technique. Additionally, in this sample, the software rasterizer is vectorized using SSE and multi-threaded to improve performance.

Key Words

Occluders: Objects that are sufficiently large to hide or occlude other objects in the scene.

Occludees: Objects that are hidden, or occluded, by other objects in the scene.

Software Occlusion Culling has two steps: depth buffer rasterization and depth test culling. The next sections provide details about these steps.

Figure 1: Screenshot of the Software Occlusion Culling sample

Depth Buffer Rasterization

The occluders in the scene are rasterized to the depth buffer on the CPU. Figure 1 shows a screenshot of the Software Occlusion Culling sample. The castle walls and the ground plane are considered as occluders in the scene. The castle walls along with the attached tiny wooden trim and decorations are used as occluders to avoid special pre-processing of the art assets. Ideally, it would have been better to hand pick just the large castle walls as occluders, but we wanted to make sure that the software occlusion culling algorithm worked with a realistic scene without the need for expensive content changes.

Our rasterizer divides the frame buffer into tiles and bins occluder triangles based on the intersection with the tiles. The rasterizer bins triangles into multiple tiles when the triangles cross tile boundaries. When processing a tile, a thread steps through the binned triangles and uses bounding box traversal to rasterize the triangles to the depth buffer. The rasterizer checks if the pixel under consideration is inside the triangle, and if so, it uses barycentric coordinates to interpolate the depth value at the pixel location. The depth buffer rasterization process updates the depth buffer at the pixel position if the newly computed depth is closer to the observer as compared to the already existing depth at the same pixel location.

Once the depth buffer has been rasterized on the CPU, all the occludees in the scene have to be depth tested against the depth buffer to determine which occludees are visible and which can be culled.

Depth Test Culling

We use object space axis-aligned bounding boxes (AABBs) to depth test the occludees in the scene against the CPU-generated depth buffer. The depth test algorithm treats all objects in the scene including the occluders (castle walls and ground plane) as occludees. Using AABB makes the depth tests more conservative. If the AABB is occluded, then the object contained inside it is also occluded and can be culled. If the AABB is visible, then the assumption is that the object contained inside might also be visible. However, as the bounding box is conservative, this assumption may not always be true, and we may have some false positives.

Ideally, if the camera is inside the AABB, the bounding box should be clipped and the portion of the object in front of the camera should be processed. However, we do not have a clipper stage implemented in the sample. Therefore, if any of the occludee bounding boxes is being clipped by the near clip plane, then we trivially accept the occludee as visible and send it to the GPU for rendering.

To determine if an object is clipped by the near clip plane, we could use the homogeneous coordinate w, of the vertices that make up the bounding box. The near clip plane is set to 1.0 in the sample. Objects that are in front of the camera have a w> 1.0. So if any of the vertices that make up the bounding box has a w < 1.0, then the object is being clipped by the near clip plane. 

However, we store 1/w instead of the homogenous coordinate w to avoid divide by w at multiple locations in the rasterizer.

So the bounding box of an occludee is being clipped if 1/w > 1.0 or 1/w < 0.0 for any vertex of the bounding box in which case we trivially accept the occludee as visible.

When frustum culling is enabled, the AABB depth test algorithm processes occludees that are completely inside the view frustum or are clipped by the view frustum. This means occludees that are behind the camera are frustum culled and not rendered. The sample allows enabling depth buffer culling even when frustum culling is disabled.  This is not useful in practice, but we provided this feature to measure the performance gain obtained by frustum culling alone. We would like to note that we have a known bug in this configuration; we trivially-accept all objects behind the camera. Hence the number of draw calls increases when frustum culling is disabled and depth test culling is enabled.

When an object is visible and the first pixel of its bounding box passes the depth test, our depth test algorithm marks the object as visible and sends it to the GPU for rendering. However, when an object and the bounding box are completely occluded, the depth test algorithm tests all the pixels of the bounding box. In short, the depth test algorithm spends the least amount of time when an object is submitted to the GPU for rendering and performs the most amount of work when an object is culled.

Optimizations

There were several optimizations implemented in the Software Occlusion Culling sample.

• Binning:

  As detailed in the section on Depth Buffer Rasterization, a binning pre-pass is performed on the triangles before rasterizing occluder triangles to the depth buffer. The frame buffer is divided into tiles and the occluder triangles are binned based on the intersection with the tiles. Each tile is individually worked on, and only the pixels inside the tile are considered when rasterizing the triangle to the depth buffer. We use binning to prevent two or more threads from operating on the same pixel at the same time. Binning may also help cache coherency as the thread interacts repeatedly with the same small tile.

• Frustum Culling:

  When frustum culling is enabled only objects that are inside the view frustum are processed.

• Vectorizing using SSE:

  The Software Occlusion Culling sample is optimized using SSE intrinsics.  Wherever possible, the code processes 4 triangles or 4 pixels a time.

• Tasking:

  Intel® Threading Building Blocks (Intel® TBB) is used to multi-thread the Software Occlusion Culling sample. Intel TBB is a library that allows developers to distribute and load balance a set of tasks across all available physical cores of a CPU. The sample uses a tasking system created by (Minadakis, 2011) for multi-tasking the depth buffer rasterization and the depth testing algorithms. The task manager is used because it helpfully abstracts and simplifies the task creation and synchronization process performed by Intel TBB.

• Occluder Size Threshold :

  We use the occluder size threshold to reduce the number of occluders rasterized to the depth buffer. Occluders that have a screen space size greater than the threshold value may be big enough to occlude other objects in the scene, and we rasterize them to the depth buffer. The depth buffer rasterizer does not rasterize all other occluders in the scene. This improves the depth buffer rasterization time.

• Occludee Size Threshold :

  We use the occludee threshold size to reduce the number of occludees that are sent to the GPU for drawing. Occludees, whose screen space size is smaller than the threshold value may be significantly small (occupies only a few pixels); we do not draw them even if they are visible.

Running the Software Occlusion Culling Sample

Figure 1 shows a screenshot of the Software Occlusion Culling sample.  The sample has several user interface elements that allow you to inspect and change the behavior of the sample.

Use the Rasterizer technique drop-down list to choose between scalar and SSE versions of the algorithm.

Figure 3: Screenshot of the CPU rasterized depth buffer

The Occluders section in the GUI provides information about the occluders in the scene. 

• Number of Models: The number of occluder models in the scene.
• Depth rasterized models: The number of occluder models rasterized on the CPU to the depth buffer.
• Number of tris: The number of triangles in the occluder models.
• Depth rasterized tris: The number of occluder triangles CPU rasterized to the depth buffer.
• Depth rasterizer time: The time taken to CPU rasterize the occluder models to the depth buffer.
• Occluder Size Threshold: Vary this slider to determine which occluder models are rasterized to the depth buffer. If the screen space size of the occluder is greater than the threshold, the occluder is big enough to hide other occludees in the scene and is rasterized to the depth buffer.

The Occludees section in the GUI provides information about the occludees in the scene.

• Number of Models: The number of occludee models in the scene.
• Models culled: The number of occludee models that are culled as they are hidden by occluders.
• Models visible: The number of occludee models that are visible in the scene.
• Number of tris: The number of triangles in the occludee models.
• Culled tris: The number of occludee triangles culled as they are occluded in the scene.
• Visible tris: The number of occludee triangles that are visible in the scene.
• Depth test time: The time taken to depth test the AABBs of the occludee models against the CPU-generated depth buffer.
• Occludee Size Threshold: Vary this slider to determine which occludee models are sent to the GPU for drawing. If the screen space size of the occludee is significantly small (maybe occupies just a few pixels), we avoid drawing the occludee even if it is visible.

The Depth Test Culling checkbox enables / disables software occlusion culling in the sample.

The Frustum Culling checkbox enables / disables frustum culling in the sample. When frustum culling is enabled, only occluders and occludees inside the view frustum are processed in the sample.

The View Depth Buffer checkbox enables /disables viewing of the CPU rasterized depth buffer. When enabled the FP32 depth value in the depth buffer is interpreted as DXGI_FORMAT_R8G8B8A8_UNORM color and displayed on the screen. Figure 2 shows the CPU rasterized depth buffer.

The View Bounding Box checkbox enables / disables the drawing of AABBs for the occluders and occludees.

The Multi Tasking checkbox enables / disables multi-threading in the sample.

The Vsync checkbox enables / disables frame rate clamping in the sample. If enabled, the sample frame rate is clamped to 60 FPS.

The Number of draw calls shows the number of draw calls issued to complete drawing the scene.

Use the Depth Test Tasks slider to vary the number of tasks created by the task manager to depth test the occludees in the scene.

Performance

The performance for the Software Occlusion Culling sample was measured on a 2.3 GHz 3rd gen Intel® Core™ processor (Ivy Bridge) system with 4 core / 8 threads and Intel® HD Graphics 3000. We set the rasterizer technique to SSE, the occluder size threshold to 1.5, the occludee size threshold to 0.01, and the number of depth test tasks to 20. We enabled frustum culling and multi-tasking and disabled vsync. 

The castle scene has 1628 occluder models and ~1.05 million occluder triangles. It has 26897 occludee models (occluders are treated as occludees) and ~1.9 million occludee triangles.

The time taken to rasterize the occluders to the depth buffer on the CPU was ~3.77 milliseconds, and the time taken to depth test the occludees was ~2.23 milliseconds

Table 1: Performance for SSE version. Frame rate is frames per second (fps) and frame time is in milliseconds (ms).

Table 1 shows the SSE performance numbers. When multi-threading and frustum culling are enabled, we get ~3X performance gain over just having multi-threading enabled. Similarly when multi-threading, frustum culling, and depth test culling are enabled, we get ~8X performance gain over just having multithreading enabled and ~3X performance gain over having multithreading and frustum culling enabled.

Future Work

We would like to further optimize the Software Occlusion Culling sample as follows:

• Use AVX2 to optimize the rasterizer code. Since the rasterizer implements fixed point math and AVX2 supports integer math, AX2 can be used to optimize the rasterizer.
• Implement pipelining so that the GPU does not have to wait for the CPU to complete executing the software occlusion culling algorithm to determine which objects have to be sent to the GPU for rendering.   
• Implement a non-binned version of depth buffer rasterizer and compare its performance with the binned version.
• Experiment with smaller depth buffers and see how that affects performance. The CPU depth buffer size is currently set to 1280x720 pixels, and it is at the same resolution as the frame buffer. Note that the depth buffer size can be changed before the sample is launched, by modifying the SCREENW and SCREENH values.
• Vary the number of tiles, as the current depth buffer is divided into 32 tiles for the sake of binning, and test how that influences performance.

Release Notes

Here are some release notes for the sample:

• Running the sample in Release mode with the debugger attached (<F5>) will result in long load times. Running the sample without the debugger attached (<Ctrl> <F5>) will dramatically reduce the load times.
• The sample intentionally loads fewer models in Debug mode to improve performance and interactivity.

References

Anderson, J. (2009). Parallel Graphics in Frostbite-Current and Future. Siggraph, (pp. http://s09.idav.ucdavis.edu/talks/04-JAndersson-ParallelFrostbite-Siggraph09.pdf).

Intel Threading Building Blocks. (n.d.). Retrieved from http://threadingbuildingblocks.org/

Kas. (2011, Feb). Software Occlusion Culling. http://fatkas.blogspot.com/2011/02/software-occlusion-culling.html.

KriS. (2010, Sep). Software Occlusion Culling. http://kriscg.blogspot.com/2010/09/software-occlusion-culling.html.

Marschner, P. S. (July 21, 2009). Fundamentals of Computer Graphics [Hardcover] (3rd ed.).

Minadakis, Y. (2011, March). Using Tasking to Scale Game Engine Systems. Retrieved from http://software.intel.com/en-us/articles/using-tasking-to-scale-game-engine-systems

Intel, the Intel logo, and Core are trademarks of Intel Corporation in the US and/or other countries.

Copyright © 2012 Intel Corporation. All rights reserved.

*Other names and brands may be claimed as the property of others.

Overview

This article takes a look at the various ways custom graphs and charts are used in iOS* to create rich enterprise healthcare applications and how these same custom graphics can be created in an equivalent Windows* 8 Store app experience. An example written in C# and XAML will be used to illustrate the steps needed to create a chart in a Windows 8 Store app.

Contents

1. Charts and Graphs
2. iOS Charts and Graphs
3. Quartz 2D and Core Graphics
4. Core Plot
5. Windows 8 Store UI Charts and Graphs
6. XAML Charts and Graphs
7. DirectX* Charts and Graphs
8. Conclusion
9. About the Author

Charts and Graphs

One common element in applications is the use of charts and graphs to convey information in a compelling, visually rich medium instead of just rows and columns of numbers.  The primary content focus of this article is healthcare applications but the lessons learned here can easily be applied to other contexts.

The following screen shots are some real-life examples of how healthcare apps are making use of charts and graphs to convey information.

Figure 1 : Screen shot of application E.M.R. by K.M. Medical Software Limited

Figure 2 : Screen shot of application Wand* by Allscripts

Figure 3 : Screen shot of application AirStrip - Patient Monitoring by AirStrip Technologies, LLC

These are just a small set of the different charts and graphs found in applications on the iOS platform. The charts sometimes take center stage within an app and at other times they are used in a supporting role. In either case, the charts and graphs provide a rich visual experience for looking at data and are important in various applications.

iOS Charts and Graphs

Charts and graphs on iOS can be created in a variety of ways. Using a library is one easy way to quickly integrate complex graphics into your application. For total control of the look and feel, using the drawing primitives of the platform is perhaps the way to go.

Quartz 2D and Core Graphics

The Cocoa* framework provides a powerful and lightweight 2D drawing engine named Quartz 2D. This API exposes all the drawing mechanisms you need to draw custom charts and graphs from scratch. In addition to Quartz 2D, the Core Animation framework provides the ability to add animations or handle real-time data charts with features that improve performance.

A closer look at programming techniques used to build graphs and charts this way will not be discussed, but the following examples and articles are great references for getting started down this path:

• Overview of Quartz 2D         
• Introduction to Core Animation Programming Guide
• Sample: AccelerometerGraph

Core Plot

One of the quickest ways to create custom graphs and charts is to use a library. Fortunately, when developing applications for iOS, a mature framework is available that allows you to easily create rich charts and graphs. This library is Core Plot and is an open source project that works on both iOS and Mac OS X platforms. The following section goes into detail about how you might incorporate Core Plot SDK into your application and also discusses the type of customizations available.

The Core Plot framework is found at http://code.google.com/p/core-plot/ and is available for download under the New BSD License.

The first step in the configuration is to integrate Core Plot into your project. This is not an extremely simple task and may take a few trials to set up your environment correctly. When finished, depending on how you incorporate the project, the Project Navigator should show a similar nested CorePlot-CocoaTouch project as in Figure 4.  In addition, the QuartzCore.framework needs to be manually added to your project.

Figure 4: Project Navigator view of a sample project from within Xcode*

Creating a graph using Core Plot is a straightforward exercise and with a minimum amount of effort a functional and nice looking graph can be created. Figure 5 is a simple example of a line graph with two sets of data, two axes with labels, and an easy to read legend.

Figure 5 : Screen shot of sample blood pressure chart created with Core Plot

The Objective-C* code for this graph is not very complex, and a few of the highlights are shown below.

First, initialize the graph and ready the current view to display the graph.

graph = [[CPTXYGraph alloc] initWithFrame: self.view.bounds];
CPTGraphHostingView *hostView = (CPTGraphHostingView *)self.view;
hostView.hostedGraph = graph;

Sample Code 1**

Set up a background color and padding from the edge of the view.

hostView.backgroundColor = [UIColor whiteColor];
graph.paddingLeft = 60.0;
graph.paddingTop = 20.0;

Sample Code 2**

Next, set up this graph to be a XY scatter plot and configure the axes. The example below shows the setup of the Y-axis. The last line configures the graph to allow touch-based pan and zoom within the graph.

CPTXYPlotSpace *plot = (CPTXYPlotSpace *)graph.defaultPlotSpace;
plot.yRange = [CPTPlotRange plotRangeWithLocation:CPTDecimalFromFloat(0.0)
length:CPTDecimalFromFloat(300.0)];
plot.allowsUserInteraction = YES;

Sample Code 3**

Next, the axis and tick style are formatted with a custom width black line.

CPTMutableLineStyle *tickStyle = [CPTMutableLineStyle lineStyle];
tickStyle.lineColor = [CPTColor blackColor];
tickStyle.lineWidth = 1.0f;

CPTXYAxisSet *axes = (CPTXYAxisSet *)graph.axisSet;
CPTXYAxis *y = axes.yAxis;
y.orthogonalCoordinateDecimal = CPTDecimalFromFloat(0);
y.majorIntervalLength = CPTDecimalFromString(@"20");
y.minorTicksPerInterval = 5;
y.majorTickLineStyle = tickStyle;
y.majorTickLength = 8;
y.minorTickLineStyle = tickStyle;

Sample Code 4**

Format and add the plot line. Notice that the data source is set to our self in this case.

CPTScatterPlot *systolicSourceLinePlot = [[CPTScatterPlot alloc] init];
systolicSourceLinePlot.identifier = @"Systolic";

CPTMutableLineStyle* lineStyle = [systolicSourceLinePlot.dataLineStyle
mutableCopy];
lineStyle.lineWidth = 3.f;
lineStyle.lineColor = [CPTColor greenColor];
systolicSourceLinePlot.dataLineStyle = lineStyle;
systolicSourceLinePlot.dataSource = self;

[graph addPlot:systolicSourceLinePlot];

Sample Code 5**

For a nice visual touch, a graph legend can be added with a few lines of code.

CPTLegend* myLegend = [CPTLegend legendWithGraph:graph];
myLegend.textStyle = x.titleTextStyle;
myLegend.fill = [CPTFill fillWithColor:[CPTColor lightGrayColor]];
myLegend.borderLineStyle = x.axisLineStyle;
myLegend.cornerRadius = 5.0f;
myLegend.swatchSize = CGSizeMake(50.0,50.0);
myLegend.anchorPoint = CGPointMake(0.0, 0.0);
graph.legend = myLegend;

Sample Code 6**

Last, we add some data to an NSArray, and then within the UIViewController methods the data points are provided to the framework.

// populate array with data
NSArray *plotSystolicData;
	
-(NSNumber *)numberForPlot:(CPTPlot *)thePlot
              field:(NSUInteger)theField
              recordIndex:(NSUInteger)idx
{    
   if ( plot.identifier == @"Systolic" )
   {
     NSDecimalNumber *value = [plotSystolicData objectAtIndex:idx];
     return value;
   }
}    

Sample Code 7**

Although this might look like quite a bit of code, most of it just defines the look and feel of the graph, and a large amount of customization is possible. In addition to customizing, a variety of plot styles are available for use (e.g., CPTBarPlot.)

Some additional resources to assist you in integrating and using the Core Plot framework in your project:

• Using Core Plot Within an Application - http://code.google.com/p/core-plot/wiki/UsingCorePlotInApplications
• How To Draw Graphs with Core Plot, Part 1 - http://www.raywenderlich.com/13269/how-to-draw-graphs-with-core-plot-part-1
• Using Core Plot in an iPhone* Application - http://www.switchonthecode.com/tutorials/using-core-plot-in-an-iphone-application

Windows 8 Store UI Charts and Graphs

Now that we have looked at possible solutions for iOS, the question is what does it take to get an equivalent graph running and looking as nice in Windows 8 Store UI?

Some concepts will be unfamiliar to an iOS developer when thinking about porting or designing a Windows 8 Store app. Some of these items are choosing one or multiple programming languages, familiarizing oneself with new UI paradigms such as App Bars and Snap View, and learning to use new development tools and a completely new IDE.  

Here are some articles that go into greater details about the basics of Windows 8 Store apps from an iOS developer’s perspective along with some more general learning resources.

• Porting from iOS to Windows* 8 - Overview
• Porting from iOS to Windows* 8 - App Life Cycle, Settings and Search features
• Porting from iOS to Windows* 8 - Basic User Interface
• Porting from iOS to Windows* 8 - Advanced User Interfaces
• Resources for iOS developers
• Porting iOS apps to Windows Store apps has some good general guidelines for UI design throughout.  Case studies (Windows Store apps) (Windows)
• Learn to build Windows Store apps

There are obviously other ways to accomplish creating graphs and charts, and some other methods will be mentioned in the sections below. Figure 6 shows a screen shot of a similar blood pressure graph created using WinRT XAML Toolkit.

Figure 6: Screen shot of sample blood pressure chart created with WinRT XAML Toolkit

XAML Charts and Graphs

Free or Open Source options for graphing libraries that support the Windows 8 Store app platform are not as robust and plentiful as with iOS, but one option stands out as having all the features needed to create compelling graphs.  WinRT XAML Toolkit available under the MIT License provides a variety of customizable chart controls, including line, pie, scatter, and bar charts.

Let’s take a look at the C# code and XAML source used to create this graph. As in iOS, you first need to set up the project to reference the graphing library.

Figure 7: Solution Explorer view of sample graph project in Visual Studio* 2012

The WinRT XAML Toolkit solution has several projects, but only two projects are needed to build the graph and are highlighted in Figure 7. This figure also shows the outcome of configuring the references from the main user application project, PRApp in this case, located under the References folder.

The approach taken to implement the graph was to use a combination of both C# code and XAML to configure the graph. The example source below is obviously not the only way to code the graph.

The first thing needed is a graph component defined in XAML. To simplify things, the page I used inherits from LayoutAwarePage. Within the definition tag, the proper WinRTXamlToolkit namespace is pulled in and the <charting:Chart> tag starts the definition of the graph.

<common:LayoutAwarePage
   x:Class="PRApp.VitalsGraph"
   xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
   xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
   xmlns:local="using:PRApp"
   xmlns:common="using:PRApp.Common"
   xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
   xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
   xmlns:charting="using:WinRTXamlToolkit.Controls.DataVisualization.Charting"
   mc:Ignorable="d">
…
<charting:Chart
    x:Name="MyLineChart"
    Margin="301,70,299,70"
    Padding="10" BorderThickness="5" BorderBrush="#FF233083" 
    Background="#FF406295" Title="Blood Pressure">
   <charting:LineSeries 
DependentValueBinding="{Binding BloodPressureSystolic}" 
      	IndependentValueBinding="{Binding CreateDayTime}" 
      	IsSelectionEnabled="True" 
       Title="Systolic"/>
   <charting:LineSeries 
DependentValueBinding="{Binding BloodPressureDiastolic}" 
IndependentValueBinding="{Binding CreateDayTime}" 
      	IsSelectionEnabled="True" 
       Title="Diastolic"/>
</charting:Chart>

Sample Code 8**

This defines an object with the name MyLineChart that can easily be used from within C# code. The interesting XAML code above defines two LineSeries to be used to plot the data. In this case, the data will use a data binding to a collection object that contains the named elements BloodPressureSystolic, BloodPressureDiastolic, and CreateDayTime.  

Within C# code, the first thing to do is to set up the connection between the data and the graph.  Using data binding and the VitalsViewModel, this can be accomplished with a minimum amount of code as seen below:

// retrieve the view model that will supply the data for our graph
VitalsViewModel vvm = new VitalsViewModel();
ObservableCollection<VitalsViewModel> vitals = vvm.GetVitals(((PatientsViewModel)navigationParameter).Id);

// assign view model as the source of the data
foreach (LineSeries series in MyLineChart.Series)
{
series.ItemsSource = vitals;
}

Sample Code 9**

Some of the hidden magic is contained within the VitalsViewModel class. This handy method takes a parameter, an ID to a Patient DB record in this case, and returns an enumerable collection of information about the patient’s vitals.  This collection is then assigned directly to the ItemsSource property of the line series. The XAML code shown previously then defines which VitalsViewModel property is used as the source for the data points. In this case we will use CreateDayTime, BloodPressureSystolic, and BloodPressureDiastolic.

This next section of code sets up the look and feel of the axes.

// setup the y-axis
LinearAxis yAxis = new LinearAxis();
yAxis.ShowGridLines = true;
yAxis.Orientation = AxisOrientation.Y;
yAxis.Minimum = 0;
yAxis.Maximum = 225;
yAxis.Interval = 25;
yAxis.MajorTickMarkStyle = (Style)Resources["MajorYTickStyle"];

// setup the x-axis
DateTimeAxis xAxis = new DateTimeAxis();
xAxis.Orientation = AxisOrientation.X;
xAxis.IntervalType = DateTimeIntervalType.Months;
xAxis.Interval = 2;
xAxis.MajorTickMarkStyle = (Style)Resources["MajorXTickStyle"];
xAxis.AxisLabelStyle = (Style)Resources["ShortDateTimeAxisLabelStyle"];

Sample Code 10**

The tick marks and date axis label have an additional level of customization that is accomplished with XAML and is pulled in with the code by specifying a resource name (e.g., “ShortDateTimeAxisLabelStyle”). Here is what the customization looks like in the XAML code:

<Style TargetType="Line" x:Key="MajorXTickStyle">
    <Setter
                Property="Stroke"
                Value="Gray" />
    <Setter
                Property="Y2"
                Value="6" />
    <Setter 
            Property="StrokeThickness"
            Value="4"
        />
</Style>

<Style x:Key="ShortDateTimeAxisLabelStyle" 
        TargetType="charting:DateTimeAxisLabel">
    <Setter
            Property="MonthsIntervalStringFormat"
            Value="{}{0:M/yy}" />
    <Setter
            Property="RenderTransform">
        <Setter.Value>
            <RotateTransform Angle="30"/>
        </Setter.Value>
    </Setter>
</Style>

Sample Code 11**

The style modifications for the tick mark increase the thickness and length of the line slightly and also update the color of the mark.  The modifications to the date axis are a little more interesting. A rotation transform is applied to the label values with the RenderTransform element. In addition, a shortened format for the date is created by overriding the MonthsIntervalStringFormat element.  

The code below continues some style optimizations and then applies the axis formatting to each series in the graph.

// increase the font of the axis and legend labels
MyLineChart.FontSize = 24;

// use a custom style for the title of the chart
MyLineChart.TitleStyle = (Style)Resources["TitleStyle"];

// apply the axis formatting to each series in the chart
foreach (LineSeries series in MyLineChart.Series)
{
    series.IndependentAxis = xAxis;
    series.DependentRangeAxis = yAxis;
}

Sample Code 11**

The chart font size only seems to update the axis and legend labels so an additional custom style for the Title is needed.

<Style TargetType="datavisual:Title" x:Key="TitleStyle">
    <Setter
                Property="FontSize"
                Value="34" />
    <Setter
                Property="HorizontalAlignment"
                Value="Center" />
    <Setter
                Property="Margin"
                Value="0,10,0,10" />
</Style>

Sample Code 13**

That’s it! The graph is ready to go with a level of customization that is easily equal to what was accomplished in the iOS example.

DirectX* Charts and Graphs

Other ways to create graphical charts in Windows 8 Store apps include using the powerful Direct2D engine found in DirectX. Although DirectX is a C++ only component, there are ways to combine the ease of XAML and C# with the power of Direct2D graphics.

The primary way of using DirectX is to create a C++ Windows 8 Store app. XAML can still be used, and one of the template projects in Visual Studio* 2012 provides a great starting point for an application built with this model (see Figure 8).  The preview image at the bottom right gives you an idea of what to expect. One string is drawn with XAML, the other with DirectX.

Figure 8: Screenshot Visual Studio* 2012

Although all the functionality for building a Windows 8 Store app is available with the previous method, a more interesting way of integrating Direct2D graphics is to use a hybrid approach where you combine a C# Store app with C++ code from a DirectX Windows Runtime component. First, a  C++ component based on SurfaceImageSource is created to perform the DirectX work. Next, you simply set the ImageSource of a XAML component to this SurfaceImageSource object and you now have DirectX graphics displayed in your C# application.

For a more in-depth look at the strategies discussed, including details on using the additional method, VirtualSurfaceImageSource, check out the article Combining XAML and DirectX. To see this method in action, the Microsoft provided sample XAML SurfaceImageSource DirectX interop sample is an excellent place to start.

Conclusion

Hopefully with some of the methods outlined here, creating a visually rich Windows 8 Store app is a little easier. The variety of programming languages, UI drawing options, and kits available to you, all contribute in making the port of an existing iOS as easy as possible.

About the Author

Nathan Totura is an application engineer in Intel's Software and Services Group. Currently working on the Intel® Atom™ processor-enabling team, he helps connect software developers with Intel technology and resources. Primarily these technologies include tablets and handsets on the Android*, Windows 8, and iOS platforms. 

**This sample source code is released under the Intel OBL Sample Source Code License (MS-LPL Compatible) , Microsoft Limited Public License and Visual Studio 2012 License. Copyright© 2012 Intel Corporation. All rights reserved.

Download Article

Download Handling touch input in Windows* 8 Applications [PDF 745KB]

With the growing number of devices supporting touch, handling the touch interaction method in your applications is more and more important.

Windows* 8 standard controls and templates already handle touch perfectly, so if you only use them, you don’t really need to know what is under the hood.

But if you have a desktop application or if you want to build some custom controls (for a game, for example), then you need to know how to handle touch properly.

Applications that worked on Windows 7 will survive:

• If your application only handles mouse events, touch events will fall back to these (you will only miss the hover event)
• If your application handles Windows 7 touch events (WM_TOUCH / WM_GESTURE APIs) that’s fine, these APIs are still available.

In both cases, you should consider improving your application by using the new Windows 8 input APIs, as they will help you provide consistency with other applications and a better overall user experience.

If your application was not designed to handle touch, you may also have to make modifications to the user interface for it to be more touch friendly: larger controls, less clutter, etc. You can find more advice on touch-compatible interface design here: Designing for Ultrabook™ Devices and Touch-enabled Desktop Applications

Developing for touch without a touch-enabled device

If you don’t have any touch-enabled devices, you can still try your desktop or Windows Store application inside the Windows 8 simulator shipped with Visual Studio* 2012 on Windows 8.

That tool can be found in “C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Simulator\11.0\ Microsoft.Windows.Simulator.exe.”

You can launch any application inside it, like from your regular session:

Windows 8 Store apps can also directly be launched inside the simulator from Visual Studio:

Then you can choose the interaction mode on the right side of the simulator. The default is mouse, but the other three are the interesting ones: single touch, pinch to zoom, and rotation modes.

Dual touch modes work using mouse left click + wheel to zoom or rotate.

Windows 8 OS touch interactions

To provide a good user experience, your app needs to at least be consistent with OS-wide gestures:

source: //build/ 2011 - APP-391T

Pointer input concept

Overview

Handling more than one input method in applications can be tedious. Fortunately, Microsoft introduced the unified “Pointer” input with Windows 8:

The Pointer input abstracts/includes the Mouse/Pen/Touch input methods. Coding for Pointers allows you to code only once to handle all these input methods.

Pointer events are the most basic thing you will be able to get. You can get these on any Windows 8 XAML UI Element as well as on ICoreWindow. On the HTML5 side, these events are available, but with slightly different names, prefixed with MS.: MSPointerDown, MSPointerMove, MSPointerUp.

The Win32* equivalents of pointer events are WM_POINTERXXX messages, you can receive these in your Win32 window callback function. For Win32 applications, WM_POINTERXXX messages don’t include mouse messages by default. You first need to call EnableMouseInPointer(true) to get the completely unified pointer messages.

Pointer Events (XAML / JS / Win32)

• PointerPressed / MSPointerDown / WM_POINTERDOWN
• PointerMoved / MSPointerMove / WM_POINTERUPDATE
• PointerReleased / MSPointerUp / WM_POINTERUP
• PointerEntered / MSPointerHover / WM_POINTERENTER: hover/over state started.
• PointerExited / MSPointerOut / WM_POINTERLEAVE: hover/over state ended.
• PointerCanceled / MSPointerCancel /: Pointer lost (touch canceled by pen coming into range, too many contacts, user log off…).
• PointerCaptureLost / MSLostPointerCapture / WM_POINTERCAPTURECHANGED
• PointerWheelChanged / - / WM_POINTER(H)WHEEL:  Standard or horizontal wheel state changed of at least one detent since last event.

Higher level objects like XAML UI elements directly provide Gesture and Manipulation events:

Gesture Events

• Tapped
• DoubleTapped
• RightTapped: Long Tap, equivalent to a right click
• Holding / MSGestureHold: Fired before Pointer release.

Manipulation Events

• ManipulationStarting
• ManipulationStarted
• ManipulationDelta
• ManipulationCompleted
• ManipulationInertiaStarting

Manipulation events can provide information on zooming, rotating, and panning, and they also provide inertia. They can be configured using ManipulationMode to toggle inertia or allow only some interactions/add constraints (like rails for translation on X/Y).

In Windows 8 Store apps using HTML5/JavaScript*, you can use a WinRT GestureRecognizer to get these events.

If you want your application to also work inside IE10 (i.e., without WinJS), you can use an MSGesture object. It will trigger these events that are equivalent to manipulation events: MSGestureStart, MSGestureEnd, MSInertiaStart, and MSManipulationStateChanged and these gesture events: MSGestureTap, MSGestureHold.

Note: Manipulation events are also available for C#/WPF 4.x desktop apps like they were under Windows 7, without ManipulationMode.

Windows 8 APIs overview

If the object you are dealing with doesn’t trigger gesture events, you can send the pointer events to a GestureRecognizer. The GestureRecognizer will trigger selected gestures and manipulations events as well as dragging and cross-slidingevents.

InteractionContext is the Win32 equivalent of the GestureRecognizer from the Windows Runtime API. The Interaction Context object triggers INTERACTION_CONTEXT_OUTPUT_CALLBACK associated with the different gestures/manipulations.

Something else you can integrate in your touch-enabled application is an InkRecognizer. It allows you to recognize handwriting from desktop apps as well as from Windows 8 Store apps.

You can also inject touch events from desktop applications using the Touch Injection API.

Summary of ways to integrate touch within your application

Here is a wrap-up of the ways to handle touch input depending on the objects / application types you are dealing with:

Sample codes

Win32* touch/Interaction Context sample– Intel

Windows* 8 gestures C#/JS sample– MSDN

DirectX* Touch input C++ sample– MSDN

InkRecognizer JS sample– MSDN

Intel, the Intel logo, and Ultrabook are trademarks of Intel Corporation in the US and/or other countries.

Copyright © 2012 Intel Corporation. All rights reserved.

*Other names and brands may be claimed as the property of others.

Download Article

Download Case Study: Blue Innovations Embraces Touch and Intel® Smart Connect Technology to Upgrade Its MoneyBag App for the Intel® Ultimate Coder: Ultrabook™ Challenge [PDF 973KB]

Blue Innovations recently had an opportunity to upgrade its financial tracking app, MoneyBag, to optimize it for Ultrabook™ devices and Windows 8. Prior to this upgrade, MoneyBag was a basic app, so the development team considered many areas for additional functionality. The team performed the upgrade as part of the Intel® Ultimate Coder: Ultrabook™ Challenge, which took place over just six weeks in August and September 2012.

For the Challenge, the team focused on optimizing for the touch interface, using sensors as much as possible, creating a fluid design, and taking advantage of Intel® Smart Connect Technology. This case study describes the development opportunities and decisions that the Blue Innovations team made as they essentially redesigned MoneyBag. Key opportunities included:

• Ensuring that MoneyBag worked well with Windows
• Using Windows Desktop for the primary app and creating a compatible mobile version
• Making the app as touch optimized as possible, yet also allowing for high performance with keyboard and mouse input
• Understanding typical user patterns and designing the user interface (UI) for ease of use
• Designing the app to be completely fluid, adapting to any screen resolution and size
• Incorporating Intel Smart Connect Technology
• Making good use of Ultrabook sensors

Company

Blue Innovations was started by two childhood friends, George Christopher and Sureshkumar Gunasekaran, who shared a vision to create applications that make life easier and every-day processes simpler. As Intel® Black Belts and winners of the Most Valuable Developer and several Development Challenge Awards, they build software that "touches people's lives." The company is customer-centric and applies strategic thinking, design insight, and practicality to every project. Blue Innovations boasts a wide range of skills and knowledge and strives to deliver solutions and approaches that large companies cannot match. In addition to MoneyBag, the company's apps include Tasksbox, Habits, Bluebirds, iAlarm, and iContacts.

Product

MoneyBag is a straightforward, intuitive money manager that allows users to easily enter transactions and create custom categories and reports. Users can set a budget for a particular category and track it continuously. Financial data entry is simplified, with reminders and the ability to set up repeat transactions. MoneyBag uses savings and goal-setting functions to help users focus on getting out of debt and moving toward financial freedom. The app can be synchronized across devices and systems using Dropbox and provides an advanced tool to help people build a strong relationship with money (see Figure 1).

Figure 1. Screen shot of the MoneyBag interface for Ultrabook™

Development Process for Windows 8

The Blue Innovations team knew upgrading for Windows 8 would be critical for the evolution of MoneyBag and were able to integrate Windows 8 upgrades as part of the Intel® Ultimate Coder: Ultrabook™ Challenge. They wanted to ensure that their application could take advantage of the features the combination of Ultrabook and Windows 8 offered, especially touch, which was integral to the updated version. Christopher says, "We knew that both Ultrabook and Windows 8 had huge potential and were going to be adopted by a lot of people, so we wanted to make sure that our app would work seamlessly on an Ultrabook with the Windows 8 platform."

The team decided to make this version of MoneyBag a Desktop application rather than a Windows Store app. Christopher explains: "We are a desktop application because we are a productivity application, which requires a lot of interaction from the user through the keyboard and mouse. A lot of data has to be keyed in when you are dealing with financial transactions. Because it is a creation mode application, not a consumption mode application—like many games and media apps—we believed it would fit the Desktop model better than the Store model."

The team thought carefully about this decision and ultimately arrived at it based on the belief that if they made a Windows Store application, they would have to compromise on too many key features. However, they did decide to create a mobile companion application that allows users to enter transactions on the go. Christopher offers an example: "If you go to Starbucks, you can pick up your phone and enter the transaction using our MoneyBag companion app on your mobile device. It then syncs with the MoneyBag application on your desktop. You can perform functions that don't transfer well to the mobile environment using the desktop app." Such functions include analysis, reports, budgeting, and configuration (see Figure 2).

Figure 2. MoneyBag reporting function

Christopher notes that, from a developer perspective, the team also wanted as many users as possible to have access to MoneyBag. Keeping it as a desktop application enabled them to do so. He notes, "We wanted to make it platform-independent so we could target people who are using all types of desktop devices, including Ultrabooks, desktops, and Macs."

Development Process for Ultrabook™

In considering how to optimize MoneyBag for Ultrabook, a major factor was that the Ultrabook was more powerful and had more features than devices the team had previously worked with. Christopher remarks, "We recognized that these features would enhance interactivity with users. We took them, along with features made possible by Windows 8, into account as we started to port our application and configure it for this platform." Yet the team had to pick and choose to enable only the features that made the most sense for their app. They considered the sensors, gyroscope, and accelerometer as well as Intel Smart Connect Technology.

MoneyBag Improvements

As a result of optimizing for Ultrabook, the MoneyBag user experience was improved. Christopher states, "The app became easier to use. Users will choose either the keyboard and mouse or touch or a combination; users will find their own way of interacting with the app, and there are a variety of options. Using myself as an example, when I first started using MoneyBag with Ultrabook, when I was in Desktop mode, I found myself doing most of the operations by touch. But, when I had to enter something in as text or I had to do some specific operations, I'd find myself using the keyboard and mouse."

In their usability testing, the team found users working with the keyboard and mouse, then unconsciously switching over to a touch mode, doing a few operations in touch, and then switching back automatically to keyboard and mouse mode. Christopher comments, "I think this is the way things will happen in the future with the advent of devices that are enabled with both a touchscreen and a keyboard and mouse. This strategy has helped us to take MoneyBag to users who appreciate both ways of interacting. We are catering to both desktop and touch users."

Touch Capability

The team understood that, with a touchscreen, users will expect to perform touch-based operations. Christopher says, "For that, we had to make sure that that option was available. But we also knew that some users would prefer to use a keyboard and mouse. We wanted to provide options so that users who prefer to could use the keyboard and mouse, and others could use touch. Here we encountered a challenge in how to provide both sets of controls."

The team wanted to offer a hybrid of both types of input. However, notes Christopher, "The major problem is when you have a keyboard and mouse application, the touch buttons can be small; but when you have a touch-based application, the buttons have to be large. We weren't sure what to do to keep from changing the entire look and feel of the application. We ended up using a hidden menu (see Figure 3)—a menu exclusively for touch—just behind the main application. Users can enable it using a single touch. This option lights up the application, and users can use touch functionality as they would on an Apple iPad*."

Figure 3. MoneyBag hidden menu

In addition to button size, the team updated the button layout to better suit the way testers wanted to use MoneyBag on Ultrabook and optimized the menu to be easy to use. All the menus are placed at the bottom of the screen, but users can also use the keyboard and mouse. They can go back and forth based on their preference. Christopher says, "Most apps are either designed for touch or a keyboard and mouse, but not for both. MoneyBag is unique in that it caters to both of them effortlessly, and it switches between them seamlessly."

Touchable, Touch-enabled, and Touch-optimized

Because Windows 8 is designed for touch and many Ultrabook devices use touch, the team was conscious of how touch-friendly their app was. Christopher notes, "The app has to be creative. It has to support touch and different resolutions. So, the real challenge is from a design perspective. The application must be fluid, yet at the same time have a structure and a beautiful UI."

The Blue Innovations team was impressed with the touch functionality of Ultrabook and the number of touch-enabled options available, which they got to know better as they developed their app for touch interaction. They wanted to make the app as touch-optimized as possible. Christopher comments, "Touchable wouldn't do. Touch-enabled wouldn't do. We wanted to make sure it had as many touch-optimized features as possible. That involved a lot of consideration and thinking on the features that were already in place. Sometimes, we went so far as to go back and change a feature." Through the development process, the team considered the three levels of touch-friendliness: touchable, touch-enabled, and touch-optimized.

Touchable

Christopher explains: "The first category is touchable, meaning that the developer doesn't have to do anything special for touch to work because the device and the platform take care of it. Any application is touchable the moment you put it on a touch-enabled device. However, the biggest concern with a touchable app is the buttons. On a desktop, the mouse is sharp and able to select even the smallest buttons. But, when you put such an application in a touch environment, it is not efficient because the small buttons are more difficult to navigate using a touch interface. Other examples are drop-down menus and text entry, which also do not work well in a touch environment."

Touch-enabled

The next category is touch-enabled, in which the application is enabled to receive touch gestures. The developer makes the buttons slightly larger and properly spaces the layout, with a menu. Many layout and design actions can be taken to ensure the application is touch-enabled. A touch-enabled app doesn't ask the user for a lot of data; most data is auto-populated.

Initially, MoneyBag was a touchable application. Christopher describes what came next: "We took that design and installed it in our touch-enabled Ultrabook. We saw that the options were working, but the design was not efficient. For example, we had a tough time trying to open the drop-down menus, display the lists, and select options. So we eliminated all the drop-downs in the application as well as other navigation tools that were difficult for users and replaced them with check boxes or radio buttons. We tried to create a uniform experience for the user between the touch and keyboard/mouse interfaces."

The next step in moving MoneyBag from touchable to touch-enabled was to ensure that all the buttons were at least 40 × 40 pixels (see Figure 5). There were some areas where the team felt the button size could not be increased without spoiling the UI. But the hidden menu was touch-enabled, so the team was confident that touch users would find it easy to navigate.

Figure 5. MoneyBag with large buttons

Touch-optimized

The third category is touch-optimized. In this case, the app is completely optimized for a touch-enabled device. From a design perspective, touch-optimized considerations must be thought through carefully. A touch-optimized app should be more forgiving when users accidentally touch something. For example, with a touch-enabled device, when a user touches a delete function, many times it's accidental. So the user should be prompted with a message saying, "Do you really want to delete this?"

Making the app touch-optimized was difficult because it took the greatest effort on the part of the design team. Christopher says, "When we say touch-optimized, the key is to make sure the most important options are available in the main menu. The user should not be looking around; all options and features should be available to the user immediately. In MoneyBag, there are no submenus. There are no menus that require a user to navigate several levels to go to a feature. Anywhere in the application you want to go, you can go to in a single touch."

Another important consideration was gesture. Christopher explains: "A user who is using a mouse and keyboard will click to go to the next page, but a user with a touchscreen will swipe to go to the next screen. We wanted to make sure those gestures were adequately captured. So a user can quickly scroll through a long list of transactions using a flick gesture, similar to how one would use a reader."

Overall, says Christopher, "The layout was designed to be easy for users to use. More than 90 percent of frequently used buttons are available in the most accessible part of Ultrabook."

Fluid Screen Size

Another consideration was ensuring that the display was fluid on various types of devices (see Figure 6). Christopher says, "We aimed to make the application completely fluid, meaning it would resize itself to adapt to any screen resolution without loss of quality. We believe that anyone developing for Ultrabook should not focus on one particular fixed layout. If you develop on an iPad, it has a particular fixed resolution, and that's it. This is the only resolution that will be on users' screens, so your design can be based on that particular fixed layout. But Ultrabook resolutions can range from more than 600 or 700 pixels in height. Developers must make the app fluid enough to clearly fit in each screen in the same layout with the same elegance and the same quality."

Figure 6. Fluid layout ensures high-quality interaction on any device.

To address this issue, the team tried different design concepts. Christopher remarks, "Before we finalized the layout of the application, we had tried out more than 250 design concepts. That's the kind of effort and the work that have to be done to make sure the app is fluid and supports all resolutions. Coming up with a fluid layout, and implementing it was a huge challenge. But it was important because Ultrabook is available in different resolutions and various screen sizes."

Intel® Smart Connect Technology

Blue Innovations found the Intel Smart Connect Technology associated with Ultrabook useful. Christopher says, "Intel Smart Connect Technology is a huge fascination for us."

This feature can be used to set up a reminder, which is perfect for a financial application such as MoneyBag, in which users need to be reminded of things like upcoming bills. George states, "Even if the Ultrabook is in a sleep state, Intel Smart Connect Technology enables it to wake up at intervals and connect to the Internet. That was a huge thing for us because even if the Ultrabook is sleeping, MoneyBag can still use that window and send reminders to users."

Another use for Intel Smart Connect Technology is ensuring that transactions users record on the go with their mobile devices can be recorded on the version of MoneyBag housed on the Ultrabook.

Sensors

The MoneyBag development team wanted to make full use of Ultrabook technology but knew that not all of the sensors available were relevant to their app. For example, they did not need the gyroscope or accelerometer, which are more useful for games.

One sensor that did prove to be useful was the location Global Positioning System (GPS) feature. Christopher explains: "When a user enters a transaction, the device uses this sensor to find the location and automatically adds that location to the app. The next time the user makes a purchase at the same location, it is auto-categorized. This is a transactional parameter that would otherwise have to be entered manually."

Not all users want this functionality, but it is particularly useful for travelers, who can later easily determine expenses for each location traveled to. Another helpful component of the feature is that once the app "knows" what type of store is at a particular location, it can auto-populate other aspects, such as the store type (grocery, electronics, etc.). Christopher notes that this feature will become even more helpful as Ultrabook becomes more mobile.

The sensors were both a big advantage and a big challenge. Christopher explains: "As developers, we had to write some integration code to make the sensors operate properly. So, there is that extra step to make full use of the sensors." At the same time, he observes, "We don't use all of them. The accelerometer and gyroscope, for example, are not relevant to MoneyBag. But we really love them because they would fit in a game application."

Ultimate Coder Challenge Experience

The Blue Innovations team learned several lessons during the Challenge:

• Never use a fixed layout.Christopher says, "Doing a fixed layout application is easy because the constraints are fixed for you. The x and y and the rectangle in the box in which you have to work is fixed. But the app won't scale for different resolutions or for different devices and screens. So we learned that we always have to be fluid, even though it takes a lot of effort. But once you fix the framework for a fluid layout, you don't have to worry about various screen resolutions because it will automatically work across all resolutions. If you have a fixed layout for each screen type, you have to work on it, and that is a problem."

• Expect how users will use the application.Christopher explains: "As developers, we need to know what the usability pattern will be so we can enhance the application for users. We did a lot of research from usability, touch, layout, and design perspectives, and we made sure those observations were brought into the application. Now we are hearing from customers that they love the UI. This would not have been possible if we hadn't done that homework."

• Understand how to leverage the power of the underlying device platform.Christopher notes, "Ultrabook as a platform and as a device has a lot of advantages, a lot of features. We learned how we can leverage that to our app's advantage so that we get more buzz and more revenue."

  The team learned other lessons as well—so many that they wrote an e-book about their process to help other developers learn from their mistakes—and created an Ultrabook development website.

  All the hard work has paid off. In the two to three months following the Challenge, the team had more than 10,000 downloads of MoneyBag from their website. As of late November 2012, they expected the app to be in the Mac App Store* by December 2012.

  After the Intel Ultimate Coder: Ultrabook Challenge, the team continued to make Ultrabook-specific modifications to MoneyBag. Christopher says, "We are further fine-tuning it. We added a lot of features, made sure they were stable. Since optimizing it for Ultrabook, not only Ultrabook users but other device users as well, really want to use it because the UI looks good, with a fluid layout."

  For details on the winning entry in the challenge, visit the Ultimate Coder Winner announcement.

  Summary

  Blue Innovations participated in the Intel Ultimate Coder: Ultrabook Challenge, updating its money management app, MoneyBag, for use with Windows 8 and Ultrabook. Because MoneyBag is a productivity app rather than a consumption app, the team decided to use the Windows Desktop platform rather than Windows Store. Optimizing for touch was a big part of the redesign, and the team worked its way through touchable, touch-enabled, and touch-optimized features such as larger buttons, removal of difficult drop-down lists, and hand gestures. However, they still wanted to provide keyboard and mouse functionality and included a hidden menu for touch to accommodate both interactivity options. Creating a fluid screen size was another item the team considered essential for optimal functionality on a variety of screen sizes and resolutions. They also made use of sensors, particularly the GPS sensor, to help users categorize purchases by location as well as Intel Smart Connect Technology to record transactions and send reminder messages. Each upgrade presented its own obstacles, but the Blue Innovations team came out of the Challenge with a greatly improved app and having learned many valuable development lessons.

  About the Author

  Karen Marcus, M.A., is a technology marketing writer with 15 years of experience. She has developed case studies, brochures, white papers, data sheets, solution briefs, articles, website copy, and other documents for such companies as HP, Samsung, IBM, Microsoft, Amazon Web Services, Intel, and EMC. Karen is familiar with a variety of current technologies, including cloud computing, enterprise computing, video displays, operating systems, and personal computing.

  Any software source code reprinted in this document is furnished under a software license and may only be used or copied in accordance with the terms of that license.

  Intel, the Intel logo, and Ultrabook are trademarks of Intel Corporation in the US and/or other countries.

  Copyright © 2012 Intel Corporation. All rights reserved.

  *Other names and brands may be claimed as the property of others.

Abstract

SIMD instruction sets can be used to boost performance in Windows 8 Store applications. This document focuses on how to create a SIMD library that can be consumed by any Windows 8 Store application programmed in any supported language. This paper shows how to begin making a SIMD library programmed in C++/CX, and three similar applications that use libraries written in C++/CX and XAML, C# and XAML, as well as JavaScript and HTML5. The SIMD library will aid in speeding up the applications.

Download Source Code

SIMDSampleApps.zip

License

Intel sample sources are provided to users under the Intel Sample Source Code License Agreement.

Download Article

Optimizing Windows* 8 Applications for Connected Standby [PDF 1.4MB]

Abstract

This white paper describes how to validate and analyze the behavior of Windows 8* applications during connected standby which is one of the Microsoft* WHQL requirement for Windows 8 [1]. It explains how to identify applications that drain the battery excessively during connected standby and the steps necessary to mitigate that problem. This document is intended for software developers, Original Equipment Manufacturers, and technical consumers.

Introduction

The connected standby feature enables the system to stay up-to-date and reachable whenever network connectivity is available. Much like how a phone maintains connectivity to the cellular network while the screen is off, Windows 8 applications written for connected standby are able to deliver an up-to-date experience immediately after returning from a low power state. More information on connected standby on PCs can be obtained from Microsoft [1].

When the display is turned off on connected standby capable systems, all running software (including applications and operating system software) become subject to a new set of activity restrictions. Windows Desktop Activity Moderator (DAM) suppresses legacy app execution in a manner similar to the Sleep state. It does this by suspending all applications in the user session and throttling all 3^rd party services to create predictable power usage over the period of idle time. This enables systems that support connected standby to deliver minimized resource usage and long, consistent battery life while enabling Windows 8 Modern UI apps to deliver the connected experiences they promise. In addition, as hardware power states become more sensitive, software services must be well behaved at connected standby so they don’t needlessly wake/throttle the system, which would limit battery life.

The rest of this paper details tools and techniques for understanding system behavior during connected standby, and then presents two case studies of applications that can improve their behavior during connected standby.

Tools

We used two easily available developer tools to understand the behavior of applications running in connected standby, as described in this section.

Windows PowerCfg

Windows Powercfg is a command-line utility used to control power settings. It uses Event Tracing for Windows (ETW) to profile systems. Users can use Windows Powercfg to view and modify power plans and settings such as standby time, wake timer, and power schemes. Running Powercfg with the “-energy” option analyzes common energy-efficiency and battery life problems, such as platform timer-tick setting changes, changes to timers by application processes or DLLs, and processor utilization per process. With this setting, it will also validate if the system supports connected standby and will report power-management settings in hardware and in the OS. Administrator permission is required to run Windows Powercfg.

Two command line options are used to validate and find information on behavior during connected standby:

Powercfg –a: This option reports the available sleep states of the system. To try it, open a command window from Windows. At the prompt type: % powercfg –a

A system that supports connected standby will report the supported sleep states available on the system and list Connected Standby as a supported state. Figure 1 shows output of powercfg –a on connected standby system.

Figure 1: Powercfg -a output

Powercfg -batteryreport

The “-batteryreport” option provides information on connected standby support and other related information. It creates an HTML report of the system battery life statistics by collecting a profile based on always-running, built-in system tracing. The report provides a summary of the battery installed, BIOS version, connected standby support, recent usage, and a battery life estimate based on actual use of the system including connected standby usage. Figure 2 shows sample output of “-batteryreport” when run on a PC that supports connected standby.

Figure 2: Battery Report with Connected Standby Support

The report also provides battery usage when the system was in active, suspended and connected standby states, as shown in Figure 3.

Figure 3: Battery usages in different states

More information on the use of Windows Powercfg can be found at this Microsoft website [2].

Microsoft Windows Performance Analyzer

Windows Performance Analyzer (WPA), also known as xperf, is a set of performance monitoring tools used to produce in-depth performance and power profiles of Microsoft Windows and of applications. WPA is useful to troubleshoot power hygiene problems.

Before we go into the case study, you should understand the terminology of WPA. These are the definitions of key terms and column names in WPA, taken from the System Internals documentation at [3]:

• Ready Thread: Thread in the ready state is waiting to execute or ready to be in-swapped after completing a wait. When looking for a thread to execute, the dispatcher considers only the pool of threads in the ready state.
• Standby: A thread in the standby state has been selected to run next on a particular processor. When the correct conditions exist, the dispatcher performs a context switch to this thread. Only one thread can be in the standby state for each processor on the system. Note that a thread can be preempted out of the standby state before it ever executes (if, for example, a higher priority thread becomes runnable before the standby thread begins execution).
• Waiting: A thread can enter the waiting state in several ways: a thread can voluntarily wait for an object to synchronize its execution, the operating system can wait on the thread’s behalf (such as to resolve a paging I/O), or an environment subsystem can direct the thread to suspend itself. When the thread’s wait ends, depending on the priority, the thread either begins running immediately or is moved back to the ready state.
• CPU Precise: The CPU Usage (Precise) graph records information associated with context switch events. Each row represents a collection of data associated with a single context switch, when a thread started running.
• % CPU Usage: The CPU usage of the new thread after it is switched in expressed as a percentage of total CPU time over the currently visible time range.
• Count: The number of context switches represented by the row (always 1 for individual rows).
• NewThreadId: The thread ID of the new thread.
• NewThreadStack: The stack of the new thread when it is switched in.
• ReadyingProcess: The process owning the readying thread.
• SwitchInTime(s): The time when the new thread was switched in.
• LastSwitchOutTime (s): The time when the new thread was last switched out.
• TimeSinceLast (s): SwitchInTime(s) - LastSwitchOutTime (s)

Figure 4 shows the key column names in WPA UI.

Figure 4 WPA Overview

Generic Events: User provided events are populated to analyze kernel trace data.

• OneShotTimer : This can be part of an always on timer at connected standby. The OS fires OneShotTimer every 30 seconds. Applications can create a timer by calling SetTimer or SetEvent.
• PeriodicTimer: These timers fire after the specified amount of time has elapsed, and then reset themselves to fire again.

Periodic timers are application specific and can cause kernel-mode transition while OneShotTimers are operating system specific during connected standby.

Developer should run a minimum of two tests – baseline (without apps installed) and target (with app installed) to isolate the impact of the application.

How to collect the trace

• Run powercfg.exe –a to confirm that your system supports connected standby.
• Install Windows Performance Analyzer from Windows ADK [4].
• Start the trace collection by creating the batch file using following command line:
  • xperf -on PROC_THREAD+LOADER+INTERRUPT+DPC+CSWITCH+IDLE_STATES+POWER+TIMER+CLOCKINT+IPI+DISPATCHER+DISK_IO -stackwalk TimerSetPeriodic+TimerSetOneShot -clocktype perfcounter -buffering -buffersize 1024 -MinBuffers 128 -MaxBuffers 128

• PROC_THREAD+LOADER: Provides information on device interrupts and timer.
• INTERRUPT: Useful for break event analysis. Provides information related to HW interrupts.
• DPC: Useful for break sources analysis. Provides information related to DPC logs.
• CSWITCH: Useful for break sources analysis. Provides information related to context switches.
• IPI: Provides information related to inter-processor interrupts.
• TimerSetPeriodic+TimerSetOneShot: Required stacks for timer analysis and device interrupt analysis.
• Let the system enter connected standby state (e.g. by pressing power button)
  • Wait while xperf collects the trace for a minimum of 4 hours. Long durations provide better understanding of software activity at connected standby.
  • Wake the system from connected standby (e.g. by pressing power button).

• Stop the trace.

xperf -flush xperf -stop xperf -merge \kernel.etl MyTrace.etl

Once the trace is complete, it will generate a Mytrace.etl file in the current directory.

Post processing the trace

Run this command to post process the trace file with wakeup information:

xperf -symbols -i mytrace1.etl -o cleanCS_diag.csv -a energydiag –verbose

You can post process a selected region of the trace by adding range

Xperf –symbols –I mytrace1.etl –o cleanCS_diag.csv –a energygdiag –range T1 T2

e.g: xperf -symbols -i -o EnergyDiag.csv -a energydiag -verbose -range 1000000 15000000000

Figure 5 shows files generated after post processing.

cleanCS_diag: Contains all the events and system wakeup activity.

MyTrace1: Contains the raw information of the trace.

Figure 5: Trace Output Example

cleanCS_diag:

Post processing the collected trace generates a log including the number of interrupts from devices, the tick of timers and it buckets the results for each CPU. It also includes the frequency of device and timer wakeup activities. This post processing can also be done on traces taken during idle and active power analysis. Post processing the script helps you find the software activity impact on battery life.

Figure 6: Post Processing Script Output

The total number of device interrupts as shown in Figure 6 is the sum of total interrupt counts for all device modules in the collected trace. Total timer expiration is the subset of the total interrupts that are due to timers. During connected standby, timer expiration includes system timers, events, oneshottimer and periodictimer which are caused due to throttling.

The next step is to find the system busyness during connected standby. You can scroll down the report until you find the histogram of Busy Enter/Exit Time - Group "All CPUs". Busy Percent provides a good understanding of total platform activity in connected standby. This gives the total system busyness. The higher the busyness factor is relative to the baseline, the higher the impact is on platform power. Figure 7 shows a trace of baseline Total busy percent without the test apps running. Figure 8 shows a trace collected with multiple applications running, as well as a background service. Comparison between Figure 7 and Figure 8 shows an increase in activity by a factor of 150x due to wakeups triggered by those applications and background service.

Figure 7: Baseline Output

Figure 8: Trace Output with Apps Installed

Analyzing the raw traces:

You can also inspect the trace file directly with Windows Performance Analyzer. Figure 9 shows the Graph Explorer in WPA.

Figure 9: WPA Window after opening a trace file

Figure 10 shows computation data in the analysis tab. You can zoom into the narrow bands of activity to see the wakeup activities from Processes and System. Figure 10 shows how OneShotTimer from the System aligns with the Process activities.

Figure 10: High-level view System during connected standby

To verify OneShotTimer calls from the system, drag and drop generic events from the system activity group into the Analysis tab window. Load the symbols from Microsoft server or from an application symbols directory by using “Load Symbols” from the Trace menu. Figure 11 shows the item enabled in the Trace menu.

Figure 11: Symbols Loading

You can enable graph and table for stack walk and process/thread decoding in WPA by clicking the first block on the right corner of WPA Graph, as shown in Figure 12.

Figure 12: WPA Graph and Table

The next step is to enable columns as shown in Figure 12 to get the stack walk on OneShotTimer.

Arrange the columns of the analysis table to find the wakeup activities from system or application services. Figure 13 shows process – “System” with threadID – “68” triggers OneShotTimer 36 times over the visible duration. Wakeup is happening every 30 seconds from the system process.

Figure 13 WPA showing the stack walk of OneShotTimer

Good behavior vs. Bad behavior:

Differentiating between good and bad behavior is important when optimizing applications for better connected standby battery life. Activities like storage access or network access by software updates are examples of things that can cause wakeup if it happens outside system wakeup.

Good behavior: An application service happens within “System” process. E.g. Application service goes to sleep before “System” process enters sleep state. This helps to meet the Microsoft Windows 8 connected standby WHQL requirement of 5% battery loss during 16 hours of system in connected standby.

Bad behavior: An application activity happens independent of “System” process or enters sleep state after “System” enters sleep state. Wake-ups which are not aligned can cause battery degradation in connected standby and can fail the Microsoft WHQL requirement.

Figure 14 shows good vs. bad behavior in connected standby.

Figure 14: Good/Bad behavior in connected standby

Case Study 1: Storage Access

A need to access local storage is very common for software services, such as anti-virus or software update services. When these services are running in connected standby, the local storage access should be delayed until the System process wakes up. Figure 15 shows a scenario of storage access for ~65 seconds in connected standby. The application wakes up when “System” process (marked in orange) enters active sleep state. “ProcessX.exe” starts the storage access in “System32” which prevents the system from entering connected standby. The application can be optimized by removing the long storage access. If the application needs access storage in connected standby, it can be done by coalescing with system activity and going into the suspend state by broadcasting a power state transition notification.

Figure 15: Storage access by an App service in connected standby

Once that change is made, Figure 16 shows storage and “System” process coalesced in connected standby. This shows good behavior where the application is not impacting system power in connected standby.

Figure 16: Optimized Storage access in connected standby

Case Study 2: Application threads wakeup

Optimizing an application wakeup caused by the OS is tricky to analyze. You need to understand CPU Precise and Generic events to find if OneShotTimer is happening within the “System” process wakeup. Figure 16 shows a wakeup by the application thread when the “System” process is in a sleep state. This is bad way of writing the process services which keeps the system awake unnecessarily. ProcessX.exe (ID: 2440) creates several threads. The table in Figure 16 shows two threads are not aligned to the “System” ready process. Using the generic events table, you can map the threadID to setTimer and clock interrupts. As shown in Figure 16, there are Timer Set Thread tasks which need to be investigated (Thread ID 3432 and Thread ID 1824). The next step is to map the thread ID identified in the previous step (Thread ID 3432 and Thread ID 1824) to CPU Usage (Precise) table to find the activity associated with the threads. It can be related to either Timer Set or to thread schedule or to I/O activity. You can plot different charts in one view to visualize the issue.

Figure 17: App threads are keeping system active during a sleep state

The SetTimer function can be used to modify the thread timer in an application.

UINT_PTR WINAPI SetTimer(
  _In_opt_  HWND hWnd,
  _In_      UINT_PTR nIDEvent,
  _In_      UINT uElapse,
  _In_opt_  TIMERPROC lpTimerFunc
);

The application window (HWND) is used to handle notification through the window procedure which been called after “uElapse” microseconds causing wakeup even after the “System” process has entered into connected standby state.

To fix this, if your application has a window (HWND) and you want to handle these notifications through the window procedure, callRegisterSuspendResumeNotification to register for these messages (orUnregisterSuspendResumeNotification to unregister). You can use DEVICE_NOTIFY_WINDOW_HANDLE in the Flags parameter, and pass your window’s HWND in as the Recipient parameter. The message received is the WM_POWERBROADCAST message.

If your application does not have a HWND handler or if you want a direct callback, callPowerRegisterSuspendResumeNotification to register for these messages (or PowerUnregisterSuspendResumeNotification to unregister). You can use DEVICE_NOTIFY_WINDOW_HANDLE in the Flags parameter and pass a value of type PDEVICE_NOTIFY_SUBSCRIBE_PARAMETERS in the Recipient parameter.

Conclusion

Enabling applications for connected standby is important for battery life. Systems that support connected standby must meet Connected Standby Windows Hardware Certification (WHCK) requirements for battery life. This requirement specifies that all connected standby systems MUST drain less than 5% of system battery capacity over a 16 hour idle period in the default shipping configuration. A certification test can be found in the Microsoft WHCK.

About the Author

Manuj Sabharwal is a software engineer in the Software Solutions Group at Intel. Manuj has been involved in exploring power enhancement opportunities for idle and active software workloads. He has significant research experience in power efficiency and has delivered tutorials and technical sessions in the industry. He also works on enabling client platforms through software optimization techniques.

References

[1] Microsoft WHCK: http://msdn.microsoft.com/en-US/library/windows/hardware/jj128256

[2] PowerCfg: http://technet.microsoft.com/en-us/library/cc748940(WS.10).aspx

[3] Windows Internals: http://technet.microsoft.com/en-us/sysinternals/bb963901.aspx

[4] Windows Assessment Toolkit: http://www.microsoft.com/en-us/download/details.aspx?id=30652

*Other names and brands may be claimed as the property of others.

Copyright ©2013 Intel Corporation.

Intel® Perceptual Computing SDK - My First C++ Application

Intel® Perceptual Computing SDK - How to Capture Raw Image

Intel® Perceptual Computing SDK - How to Use the Face Analysis Module

Intel® Perceptual Computing SDK - How to Use the Finger Tracking Module

Intel® Perceptual Computing SDK - How To Use the SDK with the Unity* Game Engine

Table of Content

1. Instroduction
2. Steps on Porting
3. Creating a HTML5 App with Visual Studio 2012*
4. Required Files
5. Specifying Device Features Needed by the HTML5 Sample
6. Directory Structure and Testing HTML5 Samples with Visual Studio 2012
7. Issues Encountered

Instroduction

This article is intended for beginner HTML5 developers to port the HTML5 samples provided by the Intel(R) HTML5 Development Environment tool. You can sign up the beta at http://software.intel.com/en-us/html5.
We have ported HTML5 samples to Windows 8* Store:

• Counting Beads
• RPN Calculator
• Towers of Hanoi
• jQM Springboard Nav

At the time of the writting PhoneGap Build* does not support Windows 8 Store yet.

Steps for Porting

Porting a HTML5 app to Windows 8* store app is not difficult if the app has been ported to Internet Explorer* 10 (IE10) already. It could be a time consuming task for getting the app working on all browsers. The steps for porting the HTML5 samples we used are:

1. Porting the sample to IE10 first
   Check what features are supported by IE10 by browsing to "http://html5test.com/" in IE10 and understand where your app might be facing issues.
   Please see the Tips section on what we have encountered.
2. Creating a Blank JavaScript App with Visual Studio 2012*
   We used Visual Studio 2012 express edition on a Windows 8 machine. Please refer to Microsoft MSDN Get the tools (Windows Store apps) for detailed information about the tools.
3. Adding the files to the Blank JavaScript Project
4. Setting the correct device features
5. Testing

Creating a HTML5 App with Visual Studio 2012

Here are the steps for creating a HTML5 sample app on Windows 8:

1. Create an new JavaScript "BlankApp" project
   It creates a JavaScript project with empty content and lots of default files.
2. Rename "default.html" to "index.html"
   "default.html" is the entry page created by default. Changing the name with right click on the file and select "rename".
3. Set "start page" to "index.html" for the package
   Double click on file "package.appxmanifest" and it opens with manifest editor; change the "Start page" from "default.html" to "index.hmtl":
   
4. Rename folder "images" to "win8-images" and update the following images with the HTML5 sample's images
   • logo.png
   • smalllogo.png
   • storelogo.png
   • splashscreen.png
5. Remove the files not needed: default.css & default.js
6. Copy the files from HTML5 sample to the project root directory
7. Add the files from HTML5 sample:
   This takes time because all the folders and files have to be added in order to be packaged.
8. Click on run on "Local Machine" or "Simulator" to test

This MSDN page Create your first Windows Store app using JavaScript might be useful to you.

Required Files

Building the Windows 8 store app has certain requirements. The full checklist for submitting an app is located on this msdn page.

During developing and testing time the must-have files are the 4 logo image files:

But if you start from the project types provided by Visual Studio 2012, it creates those image files with the required size by default.

Specifying Device Features Needed by the HTML5 Sample

Device features for the Windows 8 Store app are declared in "package.appxmanifest". This file is created by default with Visual Studio 2012. The device feature categories of the Windows 8 store app are different comparing to PhoneGap's feature categories. So you need to carefully map them from PhoneGap's "config.xml" to "package.appxmanifest".
The following screen shot shows the device feature list in Visual Studio 2012:

The following is all the device features used in PhoneGap's "config.xml":

<!-- If you do not want any permissions to be added to your app, add the
    following tag to your config.xml; you will still have the INTERNET
    permission on your app, which PhoneGap requires. -->
<preference name="permissions" value="none"/>

<!-- to enable individual permissions use the following examples -->
<feature name="http://api.phonegap.com/1.0/battery"/>
<feature name="http://api.phonegap.com/1.0/camera"/>
<feature name="http://api.phonegap.com/1.0/contacts"/>
<feature name="http://api.phonegap.com/1.0/file"/>
<feature name="http://api.phonegap.com/1.0/geolocation"/>
<feature name="http://api.phonegap.com/1.0/media"/>
<feature name="http://api.phonegap.com/1.0/network"/>
<feature name="http://api.phonegap.com/1.0/notification"/>

For example, the HTML5 audio sample that records and plays back, the "package.appxmanifest" should contain:

• Microphone
• Music Library

For PhoneGap "config.xml" only following is needed:

• <feature name="http://api.phonegap.com/1.0/media"/>

Directory Structure and Testing HTML5 Samples with Visual Studio 2012

Because it takes time to add all the folders and files to the projects. So we have included the Visual Studio 2012* project files for your convinience in those HTML5 Samples that are ported to Windows 8 Store. The directory structure is like below:

To build it with Visual Studio 2012 for Windows 8 Store:

1. Copy the project files under win8-proj to the root folder
2. Load *.jsproj with Visual Studio 2012, and run from the IDE

Issues Encountered

1. During our porting to IE10, most issues we encountered are css gradient related. i.e. adding "-ms-linear-gradient" in following case:
   

       background: -moz-linear-gradient(top,#7A987E 0%,#000000);
       background: -webkit-gradient(linear, left top, left bottom, from(#7A987E),to(#000000));
       background: -ms-linear-gradient(top, #7A987E, #000000);  /* IE10 */
   
   

2. The next issue is specific to jQuery-1.8.x.js. If using the normal jQuery from "http://jquery.com/download/", you will see the error below from Visual Studio 2012:
   HTML1701: Unable to add dynamic content .... A script attempted to inject dynamic content, or elements previously modified dynamically, that might be unsafe. For example, using the innerHTML property to add script or malformed HTML will generate this exception. Use the toStaticHTML method to filter dynamic content, or explicitly create elements and attributes with a method such as createElement. For more information, see "http://go.microsoft.com/fwlink/?LinkID=247104".
   
   We encountered this issue with "jqm-springboard-nav" sample. The version of jQuery to use for Windows 8 Store app can be downloaded from "https://github.com/appendto/jquery-win8". In "jqm-springboard-nav" sample we used the following code:
   

   <script>
       function includeJavaScript( url ){
           var tag = document.createElement( 'script' );
           tag.type = "text/javascript";
           tag.src = url;
   
           // Insert it to the <head>
           document.getElementsByTagName("head")[0].appendChild( tag );
       }
       </script>
   
       <!-- note: use jquery-1.8.0.min.js & jquery.mobile-1.1.1.min.js for production -->
       <script>
   	if (deviceCheck.windows8) {
   	    includeJavaScript("vendor/jquery/jquery-1.8.2-win8-1.0.js");
   	}
   	else {
   	    includeJavaScript("vendor/jquery/jquery-1.8.0.js");
   	}
       setTimeout(function(){
       	includeJavaScript("vendor/jquery.mobile/jquery.mobile-1.1.1.js");
   		includeJavaScript("app/springboard.js");
           }, 100);
       </script>
   
   

But when porting the Counting Beads HTML5 sample, there are more issues because the app originally only works on Chrome. You can see all the porting details of Counting Beads sample on this article.

Safeguard Sensitive Information with Intel® Identity Protection Technology ( Intel® IPT )

Guarding personal identities and online accounts has become a major concern for consumers, business, government and institution as the threat from hackers and malware grows.  Creating a simple, strong and secure process to authenticate the identities of users logging into online accounts or conducting online transactions is key to safeguarding personal identities and online accounts.  A hardware based security technology is what many experts say is the key to providing strong identity protection and online account protection.  That’s why Intel has developed a built-in, hardware based security technology, Intel® Identity Protection Technology, into all Ultrabooks™ and the latest PC’s using the Intel® Core™ vPro™ processors.  Read the Technology Brief: Safeguard Sensitive Information with Intel® Identity Protection Technology to get a brief overview of how this Intel® technology is providing the simple, strong and secure protection.

By Paul Ferrill

Download Article

Download Developing Desktop Apps for Ultrabook™ Devices in Windows* 8: Adapting Existing Apps [PDF 752KB]

Microsoft introduced the Extensible Application Markup Language (XAML) in conjunction with the release of version 3.0 of the Microsoft .NET Framework. XAML represents a totally different way of creating applications and was architected to support both traditional desktop applications and web-based apps using Microsoft Silverlight*. XAML is a significant departure from the Windows Forms approach in that it uses a more declarative approach to creating user interfaces (UIs).

Support for building XAML-based applications was somewhat limited in the beginning and required a more web design mentality, as much of the syntax was similar to building web applications using HTML. Microsoft Visual Studio* 2012 provides a rich set of design-time tools that makes this process much easier and is essentially on par with what you would expect if you’ve been developing Windows* desktop apps for any length of time.

The big news here is that existing XAML-based applications lend themselves well to adaptation for the new functionality found in the Ultrabook platform. This article looks at how you can quickly and easily add touch functionality to any XAML app with a minimal amount of code. We’ll start with a simple existing app and get it running on the Windows 8 desktop. Then, we’ll walk through the steps for adding new features to take advantage of what the Ultrabook has to offer.

Step 1: Compile and Run

Part 1 of this series looked at the steps required to convert projects from older versions of Visual Studio and get them to run with Visual Studio 2012. This step is only necessary if you have any references to libraries or DLLs that may not convert over properly. Examples include third-party libraries compiled for a specific operating system or version of the Microsoft .NET Framework.

Step 2: Evaluate the UI for Adaptation

Whether you’re building from scratch or adapting an application for touch, you need to consider a few design principles that Microsoft has outlined in Windows Touch Guide (see "For More Informationn" for a link). Basically, you need to think about the size and placement of any object you want a user to touch. Think about the application from the perspective of a touch-enabled device to look for ways to add new functionality.

Some user interactions typically accomplished with a right mouse click could just as easily be done using touch. Other candidate changes include using gestures to close or advance through a list of items. All of these tasks should be evaluated in the context of your specific application. XAML provides direct support for many of these types of interactions with little code. If your app is not XAML based, you’ll have to look at alternative methods that may require significantly more lines of code.

Step 3: Add Sensor Interaction to Enhance the Experience

One way to improve the look of an application is to change the color scheme based on available lighting. In part 1 of this series, you created a simple application to display the value from the ambient light sensor. Adapting this app for use in a production environment requires a few more lines of code to allow functions such as averaging the value over a period of time to keep from making adjustments too quickly. In Microsoft Visual Basic* .NET, this is easy to do using a timer control. Listing 1 shows the original code from part 1 modified to add a timer and average the value over 10 seconds.

Listing 1. Source code for a Visual Basic program to average the light sensor value over time

Imports Windows.Devices.Sensors

Public Class Form1
    Public avgvalue As Integer = 0
    Public counter As Integer = 0
    Public Sub New()
        ' This call is required by the designer.
        InitializeComponent()
        Timer1.Interval = 1000
        Timer1.Enabled = False
    End Sub
    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        If Timer1.Enabled = True Then
            Button1.Text = "Enable Timer"
            Timer1.Enabled = False
        Else
            Button1.Text = "Disable Timer"
            Timer1.Enabled = True
        End If
    End Sub
    Private Sub Timer1_Tick(sender As Object, e As EventArgs) Handles Timer1.Tick
        Dim mylight As LightSensor
        mylight = LightSensor.GetDefault
        Dim lightlevel As LightSensorReading = mylight.GetCurrentReading()
        counter += 1
        If lightlevel IsNot Nothing Then
            avgvalue += lightlevel.IlluminanceInLux
        End If

        If counter Mod 10 = 0 Then
            TextBox1.Text = (avgvalue / 10).ToString
            avgvalue = 0
        End If
    End Sub
End Class

The first step is to initialize the timer in Sub New() to fire once per second. You must set the timer interval to the number of milliseconds between events—hence the value 1000. In the Timer1_Tick routine, you use a counter to average over 10 ticks, or 10 seconds. There’s no magic in the value of 10 seconds, and you might want to change the interval depending on your application. In the case of this demo program, you start with the timer disabled until the user clicks a button. When the button is clicked, the label is changed to Disable Timer, and the timer is enabled.

When the timer fires, you get the current sensor light level and add it to the avgvalue variable. If you’ve reached 10 ticks, you divide avgvalue by 10 and load the result into TextBox1. Resetting the value for the avgvalue variable ensures that you get a new average each time. This demonstrates one simple way to add a sensor interaction to any application.

In part 1 of this series, you went through the steps to create a new project named tstproj that required a manual editing step of the XML-based project file. In a nutshell, you simply create a new Visual Basic Windows Forms project, save it, and then exit Visual Studio 2012. Next, you need to use Microsoft Notepad to edit one of the files that Visual Studio created. With Notepad, you must add the following three lines to the tstproj.vbproj file to have access to the sensor information:

<PropertyGroup>
    <TargetPlatformVersion>8.0</TargetPlatformVersion>
</PropertyGroup>

Save these changes and exit Notepad. Launch Visual Studio 2012 and open the tstproj project. With the changes to the tstproj.vbproj file, you should be able to add a reference to the Core Windows application programming interface (API) functions by clicking Project > Add Reference, expanding the Windows in the navigation pane, and then selecting the Windows check box. The heading of this section should have the text "Targeting: Windows 8.0."

Step 4: Add New Features That Take Advantage of the Platform

Having a touch screen on a full-featured Ultrabook brings a whole new world of possibilities to applications. This next sample shows how an existing XAML-based application can use touch with a minimal amount of coding. The basic application fully supports multifinger touch functions, including pan, zoom, and rotation. You can zoom in or out using your thumb and forefinger or one finger and both hands. Figure 1 shows what the app looks like.

 Figure 1.Example of an XAML touch application

The application has a single canvas with an image loaded (Common_Jezebel_Delias.jpg). From there, you set the IsManipulationEnabled property to True along with references to the code that will handle the actual manipulation.

Listing 2 shows the XAML code.

Listing 2. XAML code for the image rotation by touch example

<Window x:Class="Manipulations.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="Rotate and Zoom" WindowState="Maximized" Height="600" Width="800">
 <Canvas x:Name="_canvas" 
            ManipulationStarting="_canvas_ManipulationStarting"
            ManipulationDelta="_canvas_ManipulationDelta">
 <Image IsManipulationEnabled="True" Width="200" Source="Common_Jezebel_Delias.jpg">
	 <Image.RenderTransform>
		 <MatrixTransform Matrix="1 0 0 1 200 200" />
	 </Image.RenderTransform>
 </Image>
</Canvas>
</Window>

The Visual Basic code behind the XAML is not complicated at all. Listing 3 shows what it looks like.

Listing 3. Visual Basic code behind the image rotation by touch example

Namespace Manipulations
    '''<summary>
    ''' Interaction logic for MainWindow.xaml
    '''</summary>
    Partial Public Class MainWindow
        Inherits Window
        Public Sub New()
            InitializeComponent()
        End Sub
        Private Sub _canvas_ManipulationStarting(sender As Object, _
                                                 e As ManipulationStartingEventArgs)
            e.ManipulationContainer = _canvas
            e.Handled = True
        End Sub
        Private Sub _canvas_ManipulationDelta(sender As Object, _
                                              e As ManipulationDeltaEventArgs)
	    Dim element = TryCast(e.OriginalSource, UIElement)
	    Dim transform = TryCast(element.RenderTransform, MatrixTransform)
	    Dim matrix__1 = If(transform Is Nothing, Matrix.Identity, transform.Matrix)
	    matrix__1.ScaleAt(e.DeltaManipulation.Scale.X, e.DeltaManipulation.Scale.Y, _
					  e.ManipulationOrigin.X, e.ManipulationOrigin.Y)
	    matrix__1.RotateAt(e.DeltaManipulation.Rotation, e.ManipulationOrigin.X, _
					   e.ManipulationOrigin.Y)
	    matrix__1.Translate(e.DeltaManipulation.Translation.X, _
						e.DeltaManipulation.Translation.Y)
	    element.RenderTransform = New MatrixTransform(matrix__1)
	    e.Handled = True
        End Sub
    End Class
End Namespace

Microsoft makes several sample applications available that demonstrate the manipulation piece. It’s basically a matrix transformation using the inherent capabilities of the platform. The two subroutines ManipulationStarting and ManipulationDelta are declared in the XAML and serviced in the Visual Basic code.

Step 5: Make an Example Graphing App Touch Capable

The final example adds touch functionality to an existing XAML-based graphing app. The basic starting point is an app that displays sales information by region in a pie chart. Figure 2 shows what the initial app looks like.

 Figure 2.Example pie chart application

One place you can add functionality is in the event handlers of the XAML object. To get to them, simply click the Lightning Bolt icon in the Properties dialog box (see Figure 3). Many events are available to code against should you so choose.

 Figure 3.Event handlers for the XAML chart object

Entering text in any of these boxes creates a stub event handler to which you can add any code you like. In Figure 3, you can see the two names TouchDown and TouchMove entered next to their respective events. These events are distinct from mouse events, although they do follow some of the same processing steps.

Inside your processing code, you have access to a wide range of methods and properties associated with the pieChart. Microsoft IntelliSense* is a great way to explore what’s available if you’re not familiar with the entire list. Figure 4 shows an example of what you should see after typing pieChart followed by a period.

 Figure 4.IntelliSense for methods and properties of the XAML chart object

If you’re unfamiliar with how a particular object behaves at runtime, you can use a trick to set a breakpoint in your code, and then use the Locals dialog box in Visual Studio 2012 to see the names and values of the various properties. To demonstrate this technique, create a TouchDown event handler, and then add a Debug.Writeline statement to give you a place to set a breakpoint. Figure 5 shows what that looks like.

 Figure 5.Locals dialog box from Visual Studio 2012

All of these variables are available to the code behind the XAML page and in event-handling routines such as TouchDown. It’s up to you to decide the kind of functionality to add to your application to give it that extra bit of pizzazz. Some possibilities include a pop-up window in which to view the underlying data or even change it. You could also add the ability to save the image to a file. Both of these options are relatively simple to implement.

Wrapping Up

If you have an existing XAML-based application, you’re way ahead of the curve in moving to the desktop in Windows 8. If your application is Windows Forms based, you still have access to the sensor data, as shown in the light level sample. Basic touch functions will work just like mouse clicks for things like list boxes or to scroll through a grid. Adding multitouch features is much simpler with XAML apps, so if you need that, you’ll want to consider moving to a XAML-based design.

The key thing to take away from these articles is how easy it is to add functionality to your application to take advantage of the new Ultrabook capabilities. It really doesn’t take that much code to get full multitouch functionality, as shown in the picture sample. Take the time to read through the Microsoft documentation and become familiar with the new API functions. Understanding how to add the code and what you need to implement your new features will make the process much easier.

For More Information

• Windows Touch Guide at http://archive.msdn.microsoft.com/wintouchguide
• MSDN Sensors Reference at http://msdn.microsoft.com/en-us/library/windows/apps/windows.devices.sensors.aspx
• Windows SDK for Windows 8 at http://msdn.microsoft.com/en-us/windows/hardware/hh852363.aspx
• Windows 8 API List at http://msdn.microsoft.com/en-us/library/windows/desktop/hh920511%28v=vs.85%29.aspx
• Windows 8 Touch Guidance at http://download.microsoft.com/download/8/A/6/8A652B51-AF09-4A5A-888C-A0465D00FE5E/Windows%208%20Touch%20Guidance.pdf

Intel, the Intel logo, and Ultrabook are trademarks of Intel Corporation in the US and/or other countries.

Copyright © 2013 Intel Corporation. All rights reserved.

*Other names and brands may be claimed as the property of others.

Abstract

Connected Standby is a new feature introduced by Microsoft in Windows 8*. The use case on the tablet/mobile systems is similar to that on phones like Instant ON and Always Connected. Intel® dual-core Atom™ Z2760 (code name CloverTrail) is the first Intel platform to support Connected Standby. Interesting applications can be developed for Connected Standby to further showcase the platform capabilities like push email, multiplayer games, etc.

In this sample application, we showcase the use of Connected Standby and demonstrate the use of the API to register the background task with the timer for periodic execution in the background, so when the system enters standby, the task executes and generates an execution log. It is a simple interface and application for any ISV to use inside their apps.

Article

Windows* 8 Desktop App - Connected Standby Whitepaper

Download Source Code

AOAC.zip

License

Intel sample sources are provided to users under the Intel Sample Source Code License Agreement