Core Data – efficient fetching of portions of Entities

by

NOTE:  This is reported not to work on iOS 8+ (see comments).  This post is also fairly old at this point and I haven’t re-verified that it works on newer versions of OS X, though the relevant comment in NSFetchRequest.h seems to still be present in the 10.12 OS X SDK.

Quick write up of an active conversation on twitter and the results of some research.

Brent Simmons posted about something I’d started looking into today: Efficient Core Data fetching when you only need a few fields of a database Entity.  The name of setPropertiesToFetch: in NSFetchRequest looks promising however the documentation says:

This value is only used if resultType is set to NSDictionaryResultType.

Well that’s a bummer because we’d really like our nice subclass of NSManagedObject to be used by our view controller (say) or we only need to access one property of the complex object.  Here’s something odd I noticed though: the comment in the NSFetchRequest.h file (OS X 10.8 SDK & 10.9 SDK) for setPropertiesToFetch: says this:

If NSManagedObjectResultType is set, then NSExpressionDescription cannot be used, and the results are managed object faults partially pre-populated with the named properties

Well hey! That’s exactly what we want it to do for efficiency when we only need a couple of properties out of many for a given entity.

Thanks to some twitter dialog with @pgor and others, and especially a reminder from @JimRoepcke about setting -com.apple.CoreData.SQLDebug 1 to show the sql logging for Core Data when talking to an sqlite backend, I was able to verify that this does work the way we want it to, at least on OS X running under 10.8.4:

If you use code like the following:


NSFetchRequest * request = [NSFetchRequest fetchRequestWithEntityName: @"AppEntry"];
[request setPredicate: [NSPredicate predicateWithFormat: @"app_id in %@", inAppIDs]];
[request setIncludesSubentities: NO];
[request setPropertiesToFetch: @[ @"app_id", @"fooprop"]];
// [request setIncludesPropertyValues: NO]; // you need YES, which is the default
[request setResultType: NSManagedObjectResultType];
[request setReturnsObjectsAsFaults: NO]; // maybe not needed.  It will still be a fault,
                                         // but the properties are preloaded
NSError * error = nil;
NSArray * fetchedArray = [self.managedObjectContext executeFetchRequest: request error: &error ];
if ( fetchedArray != nil )
{
  if ( [fetchedArray count] > 0 )
  {
    AppEntry * entry = [fetchedArray firstObject];
    NSLog(@" isFault? %@", entry.isFault ? @"YES" : @"NO" );
    NSNumber * appID = entry.app_id;
    NSLog(@" appID: %@", appID );
    NSLog(@" isFault? %@", entry.isFault ? @"YES" : @"NO" );
      // accessing property not in setPropertiesToFetch:
      // causes fault to fire & load whole object
    NSString * otherProperty = entry.otherProperty;
    NSLog(@" isFault? %@", entry.isFault ? @"YES" : @"NO" );
    NSLog(@"otherProperty: %@", otherProperty );
  }
  else
    NSLog(@"no resuts");
}

(code changed to mask client project details so typos are because of that)

and the output is:

// the fetch does this - only the two properties we indicated are fetched:
 CoreData: sql: SELECT t0.Z_ENT, t0.Z_PK, t0.ZAPP_ID, t0.ZFOO FROM 
                  ZAPPENTRY t0 WHERE t0.ZAPP_ID IN (?,?,?)
 isFault? YES
 appID: 281940292
 isFault? YES
 CoreData: sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZAPP_ID, t0.ZOTHERPROPERTY, 
                   t0.ZDATESTAMP, t0.ZNAME, t0.ZFOO, t0.ZTITLE FROM ZAPPENTRY t0 
                   WHERE t0.Z_PK = ?
 CoreData: annotation: sql connection fetch time: 0.0025s
 CoreData: annotation: total fetch execution time: 0.0030s for 1 rows.
 CoreData: annotation: fault fulfilled from database for : 0x100150590 ...
 isFault? NO
 otherProperty: it works!

So you can see that the main fetch request only grabbed the two properties that were in setPropertiesToFetch:. The result object is a faulted NSManagedObject (see where we checked isFault) but the property values for the properties listed in setPropertiesToFetch: are available without faulting the object and doing another database fetch. You’ll see we accessed appID to check this.

However, if you then access a property that was not in the setPropertiesToFetch: list (otherProperty in the above code) you can see that another sql call is made and the object is faulted and fully loaded.  The next check for isFault returns NO because it is no longer a fault.

So to me this indicates that setPropertiesToFetch: IS useful exactly as we’d like even when you’re not getting the NSDictionaryResultType and so is ideal for exactly the situation Brent was asking about for his timeline view.    I’ve filed a radar on the documentation error rdar://16073227 so hopefully that’ll get updated sooner than later.

Update:

I was curious if one could modify one of these limited properties without faulting the entire object and all of it’s properties (which seemed unlikely) and so I did an additional test and the answer is: nope.  Modifying a prefetched property on the object causes a fault and the whole object and all it’s properties are loaded into memory and isFault returns NO.

Update #2: 

So this technique above turns out to work and be handy when you only need to fetch an item in a one-to-many property/relation off of an object with lots of properties to avoid loading all those properties into memory.  In our sample data an AppEntry has a bunch of properties including a relation to a list of prices and we’d like the most recent price (just any price for this sample code) but we don’t need to load in the rest of the data in the App Entry entity.  Works like this:

NSFetchRequest * request = [NSFetchRequest fetchRequestWithEntityName: @"AppEntry"];
[request setPredicate: [NSPredicate predicateWithFormat: @"app_id == %@", inAppID]];
[request setIncludesSubentities: NO];
[request setReturnsObjectsAsFaults: NO];
[request setPropertiesToFetch: @[ @"app_id" ]];
[request setRelationshipKeyPathsForPrefetching: @[ @"prices" ]];
NSError * error = nil;
NSArray * fetchedArray = [self.managedObjectContext executeFetchRequest: request error: &error ];
if ( fetchedArray != nil )
{
  if ( [fetchedArray count] > 0 )
  {
    AppEntry * appData = [fetchedArray firstObject];
    if ( appData != nil )
    {
      PriceEntry * price = [appData.prices anyObject];
      NSLog(@"a price is: %@", price.price );
    }
  }
}

and the logging looks like this:

CoreData: sql: SELECT t0.Z_ENT, t0.Z_PK, t0.ZAPP_ID FROM ZAPPENTRY t0 WHERE t0.ZAPP_ID = ?
CoreData: annotation: sql connection fetch time: 0.0005s
CoreData: sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZDATESTAMP, t0.ZPRICE, t0.ZAPP_ID
   FROM ZPRICEENTRY t0 WHERE t0.ZAPP_ID IN (?) ORDER BY t0.ZAPP_ID
CoreData: annotation: sql connection fetch time: 0.0004s
CoreData: annotation: total fetch execution time: 0.0007s for 1 rows.
CoreData: annotation: Prefetching with key 'prices'. Got 1 rows.
CoreData: annotation: total fetch execution time: 0.0022s for 1 rows.
a price is: 0

So you can see that it only loads the app_id property from the AppEntry object but still you can get to the prices relation.  Unfortunately I’m not seeing a way to cascade the setPropertiesToFetch: down into the related entities and so all the fields in the PriceEntity in this example get fetched.  I’d love to find a way to do that so if you figure that out please let me know – thanks!

Update #3: 

(I know, I know! What’s with all these updates?! – I wanted to get the basics out there and then I kept thinking of other angles/aspects of this topic).

So one thing that might be helpful is to show how to use the -com.apple.CoreData.SQLDebug 1 setting to examine what Core Data is doing when you’re using it. Setting this was critical to understanding what was happening in the above research.  You want to pass this as an argument to your app. So if you’re launching your app from the terminal you’d pass -com.apple.CoreData.SQLDebug 1 as a command line argument as usual. If you’re running from within Xcode then you set the arguments to pass to your app in the Scheme settings which you access via Edit Scheme… or Manage Schemes… + Edit Scheme….  Here’s a screenshot of the Edit Scheme sheet where you can see how to enter this flag:

Set Command Line Agrument

Advertisements

6 Responses to “Core Data – efficient fetching of portions of Entities”

  1. John Says:

    Just a heads-up – setting propertiesToFetch with NSManagedObjectResult when using NSPrivateQueueConcurrencyType in iOS causes a core data exception. Have not discovered a workaround yet.

  2. indiekiduk Says:

    I’d appreciate a reference to this being broken since iOS 8. On 9.2 I can’t get the behaviour described in this article in my private queue setting propertiesToFetch doesn’t crash but it has no effect a faulted object is returned as normal without the specified properties pre-fetched.

  3. Karan Says:

    Does “setPropertiesToFetch” also works with NSFetchedResultController?

    • Dad Says:

      Good question. I haven’t tried that so don’t know. You do set the NSFetchRequest on the NSFetchResultController so if you set up the NSFetchRequest as indicated then it might work. Use the SQL debugging flag (and an SQL based store) noted above and you can see what it’s going on to verify.

      Be curious what you discover if you try this. Thanks.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: