September 25, 2015

Top 8 errors made by Dynamics CRM developers - Part 3 - Querying for all columns

This is the third post in a 8 part series describing the most common errors made by Dynamics CRM developers. It will cover:

Issue #3 - Querying for all columns


This issue is very obvious, but unfortunately still quite common. Whoever has any SQL experience knows that SELECT * is (almost) never a good idea. Why query for data you don't need?
Exactly the same rule applies for CRM. Do not retrieve data you don't need. If you need 3 fields from an entity to perform some calculation, retrieve those 3 field. Not one more. The bigger the number of columns queries, the slower the query will perform. How much slower? Significantly.

Real life example:
"Large" entity with ~200 fields and ~100.000 records. Connection over internet.

Retrieve top 5000 records with all columns - 59 seconds
Retrieve top 5000 records with just 1 column - 1.2 second!!!

Yes, the difference is almost ... fifty times.
If this doesn't convince someone - I don't know what will ;)

There are off course some exceptions, like when you build a custom auditing or duplicate detection solution. But even then it's not always needed to retrieve all columns.

To summarize:





How to spot this?


Whenever you see:
  • select 'theObject' in LINQ queries
  • new ColumnSet(true) in QueryExpression
  • <all-attributes/> in Fetch XML
... you can sense that something is wrong.

September 24, 2015

Top 8 errors made by Dynamics CRM developers - Part 2 - Not choosing the correct query method

This is the second post in a 8 part series describing the most common errors made by Dynamics CRM developers. This time I would like to focus on:

Issue #2 - Not choosing the correct query method


Dynamics CRM offers 3 basic SDK based methods to retrieve data (I will not cover direct SQL access). In historical order those are: FetchXML, QueryExpression and LINQ. 

Below is an example on how using each of those methods retrieve the first and last names of contacts which city starts with the letter A.

IOrganizationService orgService = new OrganizationService("CRM");

// ----------
// Parameters
// ----------   
int topCount = 10;
string letterToSearchFor = "a";

// ---------------
// Using Fetch XML
// ---------------
string fetchXml = String.Format(@"<fetch version='1.0' output-format='xml-platform' mapping='logical'
                                            distinct='false' page='1' count='{0}'>
                                    <entity name='contact'>                                                
                                    <attribute name='firstname' />                                               
                                    <attribute name='lastname' />    
                                    <order attribute='firstname' descending='false' />
                                    <filter type='and'>
                                        <condition attribute='address1_city' operator='like' value='{1}%' />
                                    </filter>
                                    </entity>
                                </fetch>", topCount, letterToSearchFor);

EntityCollection ecollFetch = orgService.RetrieveMultiple(new FetchExpression(fetchXml));

foreach (Entity contact in ecollFetch.Entities)
{
    string firstName = contact.Attributes.ContainsKey("firstname") ? (string)contact["firstname"] ?? "" : "";
    string lastName = contact.Attributes.ContainsKey("lastname") ? (string)contact["lastname"] ?? "" : "";

    Console.WriteLine("First name: {0}   Last name: {1}", firstName, lastName);
}

// ---------------------
// Using QueryExpression
// ---------------------
QueryExpression query = new QueryExpression("contact");
query.ColumnSet = new ColumnSet("firstname", "lastname");
query.Criteria.AddCondition("address1_city", ConditionOperator.BeginsWith, letterToSearchFor);
query.AddOrder("firstname", OrderType.Ascending);
query.PageInfo = new PagingInfo() { PageNumber = 1, Count = topCount };

EntityCollection ecollQueryExpression = orgService.RetrieveMultiple(query);

foreach (Entity contact in ecollQueryExpression.Entities)
{
    string firstName = contact.Attributes.ContainsKey("firstname") ? (string)contact["firstname"] ?? "" : "";
    string lastName = contact.Attributes.ContainsKey("lastname") ? (string)contact["lastname"] ?? "" : "";

    Console.WriteLine("First name: {0}   Last name: {1}", firstName, lastName);
}

// ----------
// Using LINQ
// ----------
DataContext data = new DataContext(orgService);

var contacts = (from c in data.ContactSet
                where c.Address1_City.StartsWith(letterToSearchFor)
                orderby c.FirstName
                select new
                {
                    c.FirstName,
                    c.LastName
                }).Take(topCount).ToList();

foreach (var contact in contacts)
{
    Console.WriteLine("First name: {0}   Last name: {1}", contact.FirstName, contact.LastName);
}

Each next version is more readable and less error prone. Why use bloated XML when you can build a QueryExpression? Why use QueryExpression if you can use, the more readable, type safe and with Intellisense support, LINQ?

Off course LINQ is not always the answer. Although possible it gets quite messy when using dynamic queries. If you don't know the attribute names during build - QueryExpression seems to be the best choice. Another reason to use QueryExpression is wanting to get passed the 5000 (or else defined) query limit by using paging.
When to use FetchXML? I know only one reason you would want to do that - when building aggregate queries, described here - https://msdn.microsoft.com/en-us/library/gg309565.aspx. If you want to retrieve a count of records or total of some field, use FetchXML aggregates. It will be much better performance wise, because the aggregation is done directly in SQL instead of retrieving all the data and calculating it later. Any other reason? Cannot think of any.

To summarize the following rules should be used:

  • FetchXML should only be used when utilizing aggregate queries, else:
  • QueryExpression should be used when:
    • The query has dynamic attribute names
    • Paging is required, in most cases when the expected number of results is greater than the limit (5000 by default).
  • In all other cases type safe LINQ queries should be used.

How to spot this?


  • Fetch XML is used, without it being an aggregate query
  • QueryExpression is used with well known attribute names

September 22, 2015

Top 8 errors made by Dynamics CRM developers - Part 1 - Not facilitating the pre-event plugins


Having spend some time working as a CRM focused developer I often see the same development issues reappearing again and again. The root of the problem is that CRM programming is kind of special and previous experience from working with SQL databases or Web Development, although very precious and desired, can cause certain aspects of the code to be developed not exactly how it should have been.
In this series of posts I’ll try to summarize the most common mistakes or misconceptions I often see. As a matter of fact, because of my previous SQL and Web experience, I made some of them myself when starting my work with CRM several years ago.
To keep this a blog I have decided to split it up in a series of posts. The subjects I intend to cover are:
  1. Not facilitating the pre-event plugins (this post)
  2. Choosing the right query method: FetchXML vs QueryExpression vs LINQ
  3. Querying for all columns
  4. No or invalid plugin filters
  5. Unsupported DOM manipulation
  6. Getting the true value of an attribute inside a plugin
  7. Not understanding how solutions work
  8. Putting business logic client side 

Issue #1 – Not facilitating the pre-event plugins


This is probably the most common issue I see, and also the one causing the most problems when it comes to performance.

Simple rule is:
If you want to update the record that triggered the plugin, update the values directly in the target entity, using a pre-operation plugin. Do not use the .Update() method of the CRM web service.

A good example would be filling in a “full name” based on the first and last names. There are off course several ways of doing this, but let’s assume we use a plugin (which is probably also the best choice – see Issue #8).

Let’s see a short reminder on how the Dynamic CRM plugin execution pipeline looks like:
image
Fig 1 – Dynamics CRM plugin execution pipeline

As we can see it start with an event, like a certain field being updated or a record being created.
Next we have the place to register pre-validation plugins. Pre-validation means that even if we, for example, decide to limit an integer field to the 0-100 range and a user types in 1000, this plugin will still get executed. We can correct the value in code, without the user seeing an error message.
Right after the pre-validation plugins the “platform” performs the validation, like throwing errors when values are not in expected range.

Now the pre-operation plugins get executed, and this is where it gets interesting. We are still before the core database operation, so nothing has been stored yet. On the other hand we have access to the data that will be passed over to SQL and can freely modify it. This is the place where all changes to the current record should be done. Whatever we change here will modify the state of the object passed to SQL.

In the above example we fill in the full name. See how the value we filled in is then later passed to SQL and stored in the DB. This is the correct way of doing it. No further plugins will be triggered.

The anti-pattern looks like this:
image
Fig 2 – Update anti-pattern

What happens here is that we execute and completely unnecessary update, causing the whole pipeline to execute two times instead of once. In the worse case scenario we have setup our plugins to trigger on change of all attributes and then check the execution depth to avoid infinite loops… That is a no no no…

Some code examples


Let’s say we want to have a plugin that changes the last name of a contact to upper case. This should be performed on each update of the last name.

This is how it should not be done:
// Post-update plugin
// Registered on update of LastName
public class LastNameToUpperPlugin : IPlugin
{
    public void Execute(IServiceProvider serviceProvider)
    {
        // Get context
        IPluginExecutionContext context = (IPluginExecutionContext)
            serviceProvider.GetService(typeof(IPluginExecutionContext));

        // AVOID INFINITE LOOP
        if(context.Depth > 1)
        {
            return;
        }
           
        // Get target contact
        Entity entity = (Entity)context.InputParameters["Target"];
        Contact targetContact = entity.ToEntity<Contact>();

        // Get organization service
        IOrganizationServiceFactory serviceFactory = (IOrganizationServiceFactory)
            serviceProvider.GetService(typeof(IOrganizationServiceFactory));
        IOrganizationService orgService = serviceFactory.CreateOrganizationService(context.UserId);

        // Business logic
        string lastNameUpper = (targetContact.LastName ?? "").ToUpper(CultureInfo.InvariantCulture);

        Contact updateContact = new Contact()
        {
            ContactId = targetContact.Id,
            LastName = lastNameUpper
        };

        // !! Here we call the .Update()
        // This will cause an INFINITE LOOP
        orgService.Update(updateContact);
    }
}
In general, whenever you see checking the execution depth, with 99,99% probability there is something wrong with the code.

What would be a much better solution is:
// Pre-update plugin
// Registered on update of LastName
public class LastNameToUpperPlugin : IPlugin
{
    public void Execute(IServiceProvider serviceProvider)
    {
        // Get context
        IPluginExecutionContext context = (IPluginExecutionContext)
            serviceProvider.GetService(typeof(IPluginExecutionContext));


        // Get target contact
        Entity entity = (Entity)context.InputParameters["Target"];
        Contact targetContact = entity.ToEntity<Contact>();

        // Get organization service
        IOrganizationServiceFactory serviceFactory = (IOrganizationServiceFactory)
            serviceProvider.GetService(typeof(IOrganizationServiceFactory));
        IOrganizationService orgService = serviceFactory.CreateOrganizationService(context.UserId);

        // Business logic
        string lastNameUpper = (targetContact.LastName ?? "").ToUpper(CultureInfo.InvariantCulture);

        targetContact.LastName = lastNameUpper;           
    }
}

How to spot this?


Whenever you see the execution context depth checked there is probably something wrong...