This is called the Offline Address Book and is being generated on the Exchange server.
Due to various synching schedules, it can take up to 48 hours before you could actually see a change in Outlook after a modification has been made on the Exchange server or in Active Directory.
Use the Update-Global Address List cmdlet to update the recipients included in a global address list (GAL).
For information about the parameter sets in the Syntax section below, see Syntax.
Synchronizing your Outlook (cached) address book with the Global Address book may fix these discrepancies.
Although all parameters for this cmdlet are listed in this topic, you may not have access to some parameters if they're not included in the permissions assigned to you.
When changes occur within the main Global Address Book (GAL) on the Exchange server, computers running Outlook 2007, 2010, and 2013 may not automatically receive the updates.
If this is happening, then you may notice missing, non-existent and/or incorrect entries.
Please visit the document on How to turn on or off Cached Exchange Mode for (Outlook 2007, Outlook 2010, and Outlook 2013).
Outlook 2011 does not use Exchange Cached mode and directly connects to the Global Address Book (GAL) via LDAP connection.In that case the whole OAB is being downloaded again.