Statically Compiled LINQ Queries Are Broken In .NET 4.0

Written by William Roush on January 19, 2014 at 5:31 pm

Diving into how a minor change in error handling in .NET 4.0 has broken using compiled LINQ queries as per the MSDN documentation.

Query was compiled for a different mapping source than the one associated with the specified DataContext.

When working on high performing LINQ code this error can cause a massive amount of headaches. This StackOverflow post blames the problem on using multiple LINQ mappings (which the same mappings from different DataContexts will count as "different mappings"). In the example below, we’re going to use the same mapping, but different instances which is extremely common for short-lived DataContexts (and reusing DataContexts come with a long list of problematic side-effects).

namespace ConsoleApplication1
{
    using System;
    using System.Data.Linq;
    using System.Linq;

    class Program
    {
        protected static Func<MyContext, Guid, IQueryable<Post>> Query =
            CompiledQuery.Compile<MyContext, Guid, IQueryable<Post>>(
                (dc, id) =>
                    dc.Posts
                        .Where(p => p.AuthorID == id)
            );

        static void Main(string[] args)
        {
            Guid id = new Guid("340d5914-9d5c-485b-bb8b-9fb97d42be95");
            Guid id2 = new Guid("2453b616-739f-458f-b2e5-54ec7d028785");

            using (var dc = new MyContext("Database.sdf"))
            {
                Console.WriteLine("{0} = {1}", id, Query(dc, id).Count());
            }

            using (var dc = new MyContext("Database.sdf"))
            {
                Console.WriteLine("{0} = {1}", id2, Query(dc, id2).Count());
            }

            Console.WriteLine("Done");
            Console.ReadKey();
        }
    }
}

This example follows MSDN’s examples, yet I’ve seen people recommending you do this to resolve the changes in .NET 4.0:

protected static Func<MyContext, string, IQueryable<Post>> Query
{
    get
    {
        return
            CompiledQuery.Compile<MyContext, string, IQueryable<Post>>(
                 (dc, id) =>
                    dc.Posts
                        .Where(p => p.AuthorID == id)
            );
    }
}

Wait a second! I’m recompiling on every get, right? I’ve seen claims it doesn’t. However peeking at the IL code doesn’t hint at that, the process is as follows:

  • Check if the query is assignable from ITable, if so let the Lambda function compile it.
  • Create a new CompiledQuery object (just stores the Lambda function as a local variable called “query”).
  • Compile the query using the provider specified by the DataContext (always arg0).

At no point is there a cache check, the only place a cache could be placed is in the provider (which SqlProvider doesn’t have one), and that would be a complete maintenance mess if it was done that way.

Using a test application (code is available at https://bitbucket.org/StrangeWill/blog-csharp-static-compiled-linq-errors/, use the db.sql file to generate the database, please use a local installation of MSSQL server to give the best speed possible so that we can evaluate query compilation times), we’re going to force invoking the CompiledQuery.Compile method on every iteration (10,000 by default) by passing in delegates as opposed to passing in the resulting compiled query.

QueryCompiled Average: 0.5639ms
QueryCompiledGet Average: 1.709ms
Individual Queries Average: 2.1312ms
QueryCompiled Different Context (.NET 3.5 only) Average: 0.6051ms
QueryCompiledGet Different Context Average: 1.7518ms
Individual Queries Different Context Average: 2.0723ms

We’re no longer seeing the 1/4 the runtime you get with the compiled query. The primary problem lies in this block of code found in CompiledQuery:

if (context.Mapping.MappingSource != this.mappingSource)
{
	throw Error.QueryWasCompiledForDifferentMappingSource();
}

This is where the CompiledQuery will check and enforce that you’re using the same mapper, the problem is that System.Data.Linq.Mapping.AttributeMappingSource doesn’t provide an Equals override! So it’s just comparing whether or not they’re the same instance of an object, as opposed to them being equal.

There are a few fixes for this:

  • Use the getter method, and understand that performance benefits will mainly be seen where the result from the property is cached and reused in the same context.
  • Implement your own version of the CompiledQuery class.
  • Reuse DataContexts (typically not recommended! You really shouldn’t…).
  • Stick with .NET 3.5 (ick).
  • Update: RyanF below details sharing a MappingSource below in the comments. This is by far the best solution.

3 thoughts on “Statically Compiled LINQ Queries Are Broken In .NET 4.0

  1. RyanF

    I ended up going a different route because the custom CompiledQuery route would be complicated due to the internals it references. My solution was to augment the code-generated DataContext with the ability to cache the first MappingSource and then use that mapping in subsequent contexts that leverage compiled queries. Since the code-generated context is a partial class, I used a separate partial class file to add the functionality. Example:

    public partial class MyContext
    {
        public MyContext(string connectionString, MappingSource mapping)
            : base(connectionString, mapping)
        {
            OnCreated();
        }
    
        public static MappingSource SharedMappingSource { get; set; }
    
        public class MappingSourceWrapper : MappingSource
        {
            private MetaModel _model;
    
            public MappingSourceWrapper(MetaModel model)
            {
                _model = model;
            }
    
            #region MappingSource
    
            protected override MetaModel CreateModel(Type dataContextType)
            {
                if (_model.ContextType == dataContextType)
                    return _model;
    
                throw new InvalidOperationException("Data context type not supported by this mapping source.");
            }
    
            #endregion
        }
    } 
    

    Then in my initialization logic, which ensures the DB is present, etc. I cache the mapping source. Example from a unit test project:

    [AssemblyInitialize]
    public static void Initialize(TestContext context)
    {
        // initialize the DB
        using (var ctx = new MyContext(MyContext.ConnectionString))
        {
            ctx.CreateIfNotExists();
    
            // set the shared mapping source so compiled queries work across contexts
            MyContext.SharedMappingSource = new MyContext.MappingSourceWrapper(ctx.Mapping);
        }
    } 
    

    Then, when using compiled queries create the context using the shared mapping. Example:

    using (var context = new MyContext(MyContext.ConnectionString, MyContext.SharedMappingSource))
    {
        // execute a compiled query
    }
    

    So far so good. The implementation could certainly be hardened. Hopefully this works for others.

    Regards,
    Ryan

    Reply
  2. William Roush Post author

    Hey Ryan,
    Glad this helped put you on the right track.

    However, I really do like your approach. It allows you to use Microsoft’s code, avoid reflection, and makes it work as per Microsoft’s intended behavior. As long as there are no side-effects to reusing a MappingSource like that it’ll work better than the reflection method.

    Reply

Leave a Reply to RyanF Cancel reply

Your email address will not be published.

Time limit is exhausted. Please reload CAPTCHA.