Executing asynchronous calls in Pivotal

December 2. 2013 0 Comments

Posted in:
.net
pivotal

In the previous post (How to show a progress bar for a long operation in Pivotal) I talked about how to improve the user experience when executing long running operations. I explained how to show a progress bar to provide feedback to users that an operation is in progress and they should (patiently) wait. The user experience changes a lot when we provide feedback to users.

In this post I will talk about an alternative for long running operations that basically removes the wait time from the user. In this case, the long running operation will be executing on the server while the user gets back the control of the system. An example that shows a good use of an asynchronous operation is the following: suppose that when the user submits a new service ticket there are a bunch of emails being sent (to the supervisor, to the service queue, to the user acknowledging the submission, etc.) All these emails take time to be sent. If we do this on the server task associated to the service ticket, then the user will have to wait a (possibly) long time until all emails are being sent. All the user wants is to submit the incident, he/she doesn’t care if the emails are being sent at submission time or later.

There are many ways to do this. You can set up an email queue and then have a background process (e.g. the scheduler script) to read the queue and send emails. In this case the submission task is responsible to submit the email jobs to the queue. In this post I will talk of a simpler way: a generic asynchronous task that executes server task method in the background.

For illustration purposes, let’s use a simple task to simulate our long running operation. The same example for our previous post will do:

[TaskExecute]
public virtual void LongRunningOperation(Id recordId)
{
    Trace.WriteLine(String.Format("Starting LongRunningOperation with recordId '{0}'", recordId));
    System.Threading.Thread.Sleep(20000);
    Trace.WriteLine(String.Format("Finishing LongRunningOperation with recordId '{0}'", recordId));
}

The client task makes a call to this method:

[ClientTaskCommand]
public virtual void LongRunningTask()
{
    try
    {
        Globals.Execute(this.DataTemplate, "LongRunningOperation", new object[] { this.RecordId });
        PivotalMessageBox.Show("Done", System.Windows.Forms.MessageBoxButtons.OK, System.Windows.Forms.MessageBoxIcon.Information);
    }
    catch (Exception e)
    {
        Globals.HandleException(e.InnerException ?? e, true);
    }
}

As it is right now, the user will have to wait 20 seconds for the task to execute, and during this time the client is freezing doing nothing. Let’s see how we can change this and make all this asynchronous. The client will remain the same, we don’t to change the client side code. The server side code, however, will suffer a little change:

[TaskExecute]
public virtual void LongRunningOperationSync(Id recordId)
{
    Trace.WriteLine(String.Format("Starting LongRunningOperation with recordId '{0}'", recordId));
    System.Threading.Thread.Sleep(20000);
    Trace.WriteLine(String.Format("Finishing LongRunningOperation with recordId '{0}'", recordId));
}

[TaskExecute]
public virtual void LongRunningOperation(Id recordId)
{
    TransitionPointParameter t = new TransitionPointParameter();
    t.SetUserDefinedParameter(1, "Ticket Service");
    t.SetUserDefinedParameter(2, "LongRunningOperationSync");
    t.SetUserDefinedParameter(3, recordId);

    this.SystemServer.ExecuteServerTask("Async Service", "ExecuteApp", t.ParameterList, false);
}

We have split our long running operation in two: the LongRunningOperation method now makes a call to a new service task called “Async Service”, executing the method “ExecuteApp”, and passing some transition point parameters. These parameters are: the name of the service task (in this example I’m assuming our service task is called “Ticket Service”), and the method that actually does the hard work, and the parameters for such method. Our second method, the LongRunningOperationSync, is the one that actually does the job. The body of this method is the same as we had before.

The Async Service task is a generic service task that executes another service task asynchronously. Let’s see how to implement our async service:

[TaskExecute]
public virtual void ExecuteApp(ParameterList parameterList)
{
    string serverName = TypeConvert.ToString(TransitionPointParameter.GetUserDefinedParameter(1));
    string methodName = TypeConvert.ToString(TransitionPointParameter.GetUserDefinedParameter(2));

    object[] parameters = new object[] { };
    if (this.TransitionPointParameter.UserDefinedParametersNumber > 2) // get the rest of parameters
    {
        List<object> passedParameters = new List<object>();
        for (int i = 3; i <= this.TransitionPointParameter.UserDefinedParametersNumber; i++)
        {
            object parameter = this.TransitionPointParameter.GetUserDefinedParameter(i);
            if (String.IsNullOrWhiteSpace(TypeConvert.ToString(parameter)))
                passedParameters.Add(null);
            else
                passedParameters.Add(parameter);
        }
        parameters = passedParameters.ToArray();
    }

    ExecuteApp(serverName, methodName, parameters);
}

/// <summary>
/// Executes an application server task method
/// </summary>
/// <param name="appName">The name of the application server task</param>
/// <param name="methodName">The name of the server task method</param>
/// <param name="methodParameters">An array of parameters to be passed to the server taks</param>
private bool ExecuteApp(string appName, string methodName, object[] methodParameters)
{
    try
    {
        XElement xmlCommand = BuildExecuteServiceTaskCommand(appName, methodName, methodParameters == null ? new object[] {} : methodParameters);
        SendCommandAsync(xmlCommand);

        return true;
    }
    catch (Exception e)
    {
        // log the error
        throw;
    }
}

The service exposes only one public method. This method is responsible to retrieve the required parameters: the service task and method name, and the parameters for such method. It parses the transition point parameters for such information and then it calls the second method, which is more interesting. This method calls two helper methods: BuildExecuteServiceTask (to build an XML document) and SendCommandAsync (to do the async call). Let’s see these methods are implemented.

The BuildExecuteServiceTaskCommand job is to build an XML document according to the PBS XML specification (see the Pivotal Composite API Reference). This document will be sent to the PBS XML interface.

private const string PivotalNamespace = "urn:schemas-pivotal-com/LifecycleServer60";

private XElement BuildExecuteServiceTaskCommand(string serverRuleName, string methodName, params object[] parameters)
{
    XElement commandXml = new XElement(PivotalNamespace + "executeAppServerRule",
        new XElement(PivotalNamespace + "appServerRuleName", serverRuleName),
        new XElement(PivotalNamespace + "appServerRuleMethod", methodName));

    XElement commandXmlDocument = new XElement(PivotalNamespace + "command",
        new XElement(PivotalNamespace + "systemName", SystemServer.SystemInformation.SystemName),
        new XElement(PivotalNamespace + "loginType", "Client"));

    commandXmlDocument.Add(commandXml);

    if (parameters != null)
    {
        XElement commandParametersXml = new XElement(PivotalNamespace + "commandParameters");
        commandXml.Add(commandParametersXml);

        for (int i = 0; i < 6; i++) // the first six parameters must be empty
            commandParametersXml.Add(new XElement(PivotalNamespace + "emptyParameter"));

        foreach (object parameter in parameters)
        {
            string type, value;
            GetParameterData(parameter, out type, out value);
            commandParametersXml.Add(new XElement(PivotalNamespace + type, value));
        }
    }

    return commandXmlDocument;
}

For our example, the resulting XML document will be something like this:

<command xmlns="urn:schemas-pivotal-com/LifecycleServer60">
    <systemName>CRM</systemName>
    <loginType>Client</loginType>
    <executeAppServerRule>
        <appServerRuleName>Async Service</appServerRuleName>
        <appServerRuleMethod>ExecuteApp</appServerRuleMethod>
        <commandParameters>
            <emptyParameter/>
            <emptyParameter/>
            <emptyParameter/>
            <emptyParameter/>
            <emptyParameter/>
            <emptyParameter/>            
            <stringParameter>Ticket Service</stringParameter>
            <stringParameter>LongRunningOperation</stringParameter>
            <binaryParameter>0000000000000A81</binaryParameter>
        </commandParameters>
    </executeAppServerRule>
    <collectStatistics/>
</command>

This XML will be sent using a .Net HttpRequest object in the SendCommandAsync method. There is one method that requires attention in the above code, the GetParameterData method. This method converts a parameter to the Pivotal equivalent using the following strings:

//binaryParameter
//booleanParameter
//stringParameter
//stringParameter
//floatParameter
//dateParameter
//floatParameter
//binaryParameter
//binaryParameter
//integerParameter
//binaryParameter
//stringParameter
//timeInstantParameter
//timeInstantParameter

Since this is just a matter of querying the parameter type and then decide which string of the above to use, I will leave this out in sake of space, since the implementation is more related to .Net than to Pivotal and is not difficult to do. Let’s us see how the XML document is sent asynchronously to the PBS.

In order to do an async operation we will use the BeginInvoke method defined by .Net delegates. Our delegate needs the XML document to send, and the URI to send to:

private delegate void SendAsyncDelegate(XElement xmlDocument, Uri url);
private const string PivotalUrl = "http://localhost/ePower/XMLServices.asp";

The PBS endpoint for XML documents is XMLServices.asp, located in the root of the ePower virtual directory (this endpoint is old, but still works and is used by many Pivotal services such as the EMS, the scheduled script, ePartner/eService, etc.).

And this is how we send the document:

private void SendCommandAsync(XElement xmlDocument)
{
    SendAsyncDelegate sendAsyc = new SendAsyncDelegate(SendXml2UrlAsyncDelegate);
    IAsyncResult result = sendAsyc.BeginInvoke(xmlDocument, new Uri(PivotalUrl), null, null);
}

protected void SendXml2UrlAsyncDelegate(XElement xmlDocument, Uri url)
{
    try
    {
        HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create(url);
        httpRequest.Method = "POST";
        httpRequest.ContentType = "text/xml";
        httpRequest.Credentials = CredentialCache.DefaultCredentials;

        StreamWriter writer = new StreamWriter(httpRequest.GetRequestStream());
        writer.Write(xmlDocument.ToString());
        writer.Flush();
        writer.Close();

        HttpWebResponse httpResponse = (HttpWebResponse)httpRequest.GetResponse();
        XmlTextReader reader = new XmlTextReader(httpResponse.GetResponseStream());
        XElement xmlResponse = XElement.Load(reader);

        reader.Close();
        httpResponse.Close();
    }
    catch (WebException e)
    {
        // handle error
    }
}

There are some interesting things here: notice how we create the HTTP request using the default network credentials, which will use NTLM credentials to call the endpoint. Also notice that we don’t care about the response (it’s an async method). Line 4 contains the most important part of our implementation: the asynchronous call.

This is all we need to improve the user experience when executing a long running operation: submit an async call to the server task method and return control to the user as soon as possible. Of courser there are some things that need to be considered before we can actually put this into a production system. Here is a list of things that need to be done:

Error handling: in the code above you’ve noticed that I’ve not done any error handling.
Security: the PBS endpoint might be configured in HTTPS. Also the default credentials might not be what we want. In this case we could do user impersonation to do the actual call
Logging: the async service doesn’t care about the result of the operation, but the user might. In this case we would need some kind of status report on the async tasks.
Form task methods: the example above works for service task, but sometimes we want to call methods in a form task.

All the above considerations are not difficult to do. Personally I prefer to use a solution that gives me all this and more: I personally use the Pivotal Development Tools created by Grupo Lanka, which include an async service with all that’s required for a good implementation of async tasks.

For now on, there is no excuse to leave the user wondering what happened with their application, you now have two tools to improve the user experience: progress bars and asynchronous calls. I hope you implement them!