Xamarin: Speech Recognition in iOS 10

Speech is increasingly becoming a big part of building modern mobile applications. Users expect to be able to interact with apps through speech, so much so that speech is developing into a user interface itself. iOS contains multiple ways for users to interact with their mobile device through speech, mainly via Siri and Keyboard Dictation. […]

The post Speech Recognition in iOS 10 appeared first on Xamarin Blog.

Details

Tomasz Cielecki: Improving layout performance on Android

I’ve been working on improving performance of some of my Xamarin Android apps recently. One of the things I’ve been hunting down and improving on is GPU overdraw. What this means is how many times the same pixels on the screen are drawn per frame. Minimising this improves drawing performance on Android and in the end means smoother scrolling, faster drawing of views and generally makes your app perform more smoothly. The other thing is to hunt down nested layouts and flatten them to improve performance of the view laying itself out on the screen.

Now there are actually quite a lot of things you can do to your app to improve on, to reduce GPU overdraw and how long it takes to layout your views. I will try to cover some of them in this blog post.

Flattening your Layouts

In order to improve how fast your views are laid out on the screen on the device, you can do a very important thing. Flattening your layout. What does this mean? Let me show you an example!
Consider the following page layout, which is very much made out of nested LinearLayouts.

The problem with the nested layout above is the amount of measure passes which has to be done in order to lay it out. For each child with a layout_weight tag, it needs to measure itself twice. Other layouts need to measure themselves too. Imagine having a more complex layout, with a lot of nesting; would resolve in excessive measure passes and make displaying of your layout slow. Especially in cases where you use the layout is used as a row layout. This would hit the app performance quite a lot and there would be noticeable slowdowns in your app.
The layout above, can be flattened using RelativeLayout. You will notice, that layout_weight is quite powerful for layout out equal sized views, so some tricks have to be used to achieve the same with a RelativeLayout. The performance in the end is much better though.

As you see. The above layout employs two additional layouts, which are used to center the other views. An alternative to RelativeLayout to make percentage sized views is PercentRelativeLayout from the Android.Support.Percent package. With it you can set aspect ratios, use percents for widths and heights and more. I would recommend keeping your layouts as simple as possible.

If you simply wish to stack views on top of each other, you can use FrameLayout, which is a real good performer as well.

You can read more about optimizing layouts in the official Android documentation, which also shows how to use the Hierarchy viewer to inspect slow layouts.

GPU Overdraw

GPU overdraw is another problem you may encounter. What is this overdraw? It tells us about how many times the same pixel gets drawn on during a frame. Why is this bad? The more times the same pixels get drawn on, there more time we are wasting. In order to get fluid animations and scrolling etc. you need to have as high a frame rate as possible. A good goal is to try to hit 60 FPS (Frames Per Second) all the time. In order to do this, we need to spend as little as possible time drawing a frame and below 16ms. That is not a long time! Let’s explore some things you can do as a developer to improve on this.

Enable showing GPU overdraw

You can enable an overlay on your device or emulator, which will show you the GPU overdraw done by your app. You can find it in developer settings.

 This will give you a funny looking screen with colors laid on top of it. Now these colors actually mean something.
Overdraw chart from Android Documentation

The purple-ish blue means pixels have been over drawn once, green means twice, light red means thrice and dark red 4 times or more. You will also see stuff showing in its original color, this means that the pixels have not been overdrawn. The aim is to have no overdraw at all. However, this can be very hard to accomplish, unless you just have a background drawn on the screen.

Removing backgrounds from views

 A simple thing to reduce overdraw is to just remove backgrounds from views. Let us consider the first layout I showed you, now with everything having a background.

This will gives us this when showing it with GPU overdraw debugging enabled.

Our layout is RED just because we added backgrounds to our layouts. Removing these backgrounds reduce overdraw significantly and in turn improves performance of your app.

Just removing the outermost background reduces overdraw and in this layout the change won’t be visible anyways.

The two nested LinearLayouts use the same color, what if we use that as our theme background and remove the color from the layouts?

Again less overdraw. Here is the view without GPU overdraw enabled.

In this case, since the buttons themselves have a background, there will be some overdraw and that can we sometimes cannot do anything to prevent. However, simply reducing a layout from being red all over to green or light blue, means a lot towards performance. Especially in cases where the layout is used in a ListView or RecyclerView or similar Adapter type of view as less time is used to draw the row.

Hence, try to avoid using backgrounds, especially if you can’t see them at all. Also a good idea is instead of adding a background to each layout, add that background to your theme, which is pretty simple.

You can also opt to actively remove backgrounds from views with android:background:”@null” for views you don’t really care about their background.

As for shadows, borders and the like, which you could do as a background. If you really have to have them use 9-patches with transparency in the areas you don’t show anyways. Android will optimize the drawing of these for you and will not overdraw here.

Reducing overdraw in custom views

You might have views that override the OnDraw method where you draw stuff to the Canvas it provides. Here overdraw matters as well. Using OnDraw is what normal views essentially end up using in the end when they draw them selves on the screen. So you have to be careful here as well.

One way to eliminate all overdraw is to draw to a Bitmap first and then draw that to the canvas. This is normally know as double buffering. Be careful with using this, it gives the overhead of first drawing to the Bitmap then to the canvas which draws it to the screen.

The above code shows a simplified version of it. It does indeed result in just 1x overdraw if draw on a background. However, if your draw code is slow, you may encounter flickering if you need to redraw your view a lot.
SurfaceView in Android does it a bit differently. It does all the buffered drawing on a separate thread. You could do this as well. Then call Invalidate() or PostInvalidate() when you need the buffer to be shown on the screen

The technique I ended up using is a modified version, where I delay the drawing until everything is drawn on the Bitmap. Then I signal with PostInvalidate(). It looks something like this the code below.

Now this could probably made a bit simpler. However, what I achieve with this is, whenever I manually signal with Refresh() I cancel any current drawing operations to the buffer as I am not interested in what it provides as it is old data… PostInvalidate() triggers the OnDraw method whenever the GPU is ready. In here I kick off a new Task which draws to the buffer. When that Task is done it calls PostInvalidate() to signal that the buffer has changed and that will be drawn. It is a variation of double buffering, which allows draw operations to take a long time. This has resulted in smooth scrolling of the RecyclerView I am using with these graphs as rows and no overdraw.

Maybe you can use some of these techniques in your app. Let me know what you find out in your application.
In general you want to

  • Flatten your layouts to reduce measure calls
    • Use RelativeLayout instead of LinearLayout with weights
    • Even better FrameLayout for stacked views
  • Remove backgrounds that are not shown anyways
    • Use theme background where applicable
    • android:background:”@null”
  • Use 9-patch for borders and shadows
  • Reduce overdraw in your own OnDraw calls

Resources

https://medium.com/@elifbon/android-application-performance-step-1-rendering-ba820653ad3
https://www.hackerearth.com/practice/notes/rendering-performance-in-android-overdraw/
http://www.xenomachina.com/2011/05/androids-2d-canvas-rendering-pipeline.html
http://developer.android.com/tools/performance/debug-gpu-overdraw/index.html
http://developer.android.com/reference/android/graphics/Canvas.html
https://www.udacity.com/course/android-performance–ud825

Details

Greg Shackles: Creating Slack Slash Commands With Azure Functions

One of the nice features of Slack is how easy they make it to add custom slash commands simply by providing an HTTP endpoint for them to call. This is the kind of scenario where a “serverless” architecture shines, allowing you to create these HTTP endpoints without having to maintain any actual infrastructure for it. In this post I’ll show how easy it is to write a custom Slack slash command that is backed by Azure Functions. The command will allow the user to say /logo {query} to look up the logo for a given company.

Create The Function

To start off we’ll implement this function in C#. There’s not much code, so I’ll just include it all at once:

#r "Newtonsoft.Json"
using System.Net;  
using System.Net.Http.Formatting;  
using System.Collections.Generic;  
using Newtonsoft.Json;

public class Company  
{
    public string Logo { get; set; }
    public string Name { get; set; }
}

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)  
{
    var text = req.GetQueryNameValuePairs()
        .FirstOrDefault(q => string.Compare(q.Key, "text", true) == 0)
        .Value;

    using (var client = new HttpClient()) 
    {
        var json = await client.GetStringAsync($"https://autocomplete.clearbit.com/v1/companies/suggest?query={Uri.EscapeUriString(text)}");
        var companies = JsonConvert.DeserializeObject<IList<Company>>(json);
        var company = companies.First();

        var output = new 
        {
            text = $"Here's the logo for *{company.Name}*:",
            attachments = new[] 
            {
                new { image_url = company.Logo, text = company.Logo }
            }
        };

        return req.CreateResponse(HttpStatusCode.OK, output, JsonMediaTypeFormatter.DefaultMediaType);
    }
}

Using the string passed through from Slack we query Clearbit’s Autocomplete API, pick the first company returned, and construct a message in the format Slack expects it, and returns that as a JSON response. We also get to use all the nice things we’re used to in C# development like async/await, HttpClient, and Json.NET.

Finally, we’ll need to update function.json to set the HTTP input and output:

{
  "bindings": [
    {
      "type": "httpTrigger",
      "name": "req",
      "authLevel": "anonymous",
      "direction": "in"
    },
    {
      "type": "http",
      "name": "res",
      "direction": "out"
    }
  ],
  "disabled": false
}

That’s everything needed for the function, which should now be fully operational.

Create the Slash Command

In your Slack settings, choose to create a new slash command, set it to use a GET, and point it at your Azure Function:

Slash Command Config

Save that and you should be good to go! Let’s try it out:

/logo olo

Olo Logo

/logo microsoft

Microsoft Logo

Easy!

Make That Function More Functional

For extra credit, let’s recreate this function in F# because F# is awesome. First we’ll need a project.json file to bring in some dependencies:

{
    "frameworks": {
        "net46": {
            "dependencies": {
                "FSharp.Data": "2.3.2",
                "Newtonsoft.Json": "9.0.1"
            }
        }
    }
}

This function can use the same function.json settings as the C# version. Finally, the code:

open System.Net  
open System.Net.Http.Formatting  
open System.Text  
open FSharp.Data  
open Newtonsoft.Json  

type Company = { Name: string; Logo: string; }  
type Attachment = { image_url: string; text: string; }  
type Message = { text: string; attachments: Attachment[]; }

let getSearchQuery (req: HttpRequestMessage) =  
    req.GetQueryNameValuePairs() 
    |> Seq.find (fun pair -> pair.Key.ToLowerInvariant() = "text")
    |> fun pair -> pair.Value

let getCompany query =  
    Uri.EscapeUriString query
    |> sprintf "https://autocomplete.clearbit.com/v1/companies/suggest?query=%s"
    |> Http.RequestString
    |> JsonConvert.DeserializeObject<Company[]>
    |> Array.head

let Run (req: HttpRequestMessage) =  
    getSearchQuery req
    |> getCompany
    |> fun company -> { text = (sprintf "Here's the logo for *%s*:" company.Name)
                        attachments = [| { image_url = company.Logo; text = company.Logo } |] }
    |> JsonConvert.SerializeObject
    |> fun json -> new HttpResponseMessage(HttpStatusCode.OK, Content = new StringContent(json, Encoding.UTF8, "application/json"))    

Now our function is nice and functional.

Details

Gerald Versluis: Pro-tip: copy/paste URL to the iOS simulator

Just a quick tip if you need to copy and paste text to your iOS simulator.

For an app we are developing we send out verification links through e-mail with a hash. After finding out that cmd + C & cmd + V weren’t working if actually did type if over more then once..

For whatever reason I decided to try and drag-and-drop the text onto the Simulator, and what do you know?! It worked! For URLs that is.. Plain text doesn’t seem to be picked up unfortunately.

via GIPHY

Details

Wallace B. (Wally) McClure: Realities of Cross-Platform Development: How Platform-Specific Can You Go? – Part 1

My personal beliefs on cross-platform development were formed in November 1993. I worked at The Coca-Cola Company at the time, and a few colleagues and I were discussing how to provide Mac users with the same set of applications that we were building on Windows 3.1 with PowerBuilder.

The discussion centered around the UI. The concern was with providing a Windows-centric interface to Mac users. I remember one of the great lines from the meeting: “If you provide Windows help to a user expecting Mac balloon help, you are going to have users that hate you.” After this and some personal experiments, I came to agree with this viewpoint. If developers want to make users as happy as possible, they need to call the native APIs of the platform.

Fast-forward to 2009, when I fell in love with the Xamarin iOS product from the first announcement. Xamarin had created a way for C# developers to use the Microsoft .NET Framework and to call into the native platform APIs. Xamarin Android operated the same way. While I would often discuss the need for a cross-platform UI with Xamarin management, I never forgot the lessons of cross-platform from many years ago. At the same time, I knew that there was a need for a cross-platform toolset to create an application. I had talked to enough people to understand the pain and agony. Quite honestly, I was a fence sitter in this area. I liked the idea of XF, but I was never quite ready to make the jump and, honestly, try XF for anything more than some personal experiments and helping a couple of customers.

Url: Platform Specific with Xamarin-Forms

Details