Hey Mac – I don’t appreciate you spying on me! (Hidden Downloads Log in OS X)
A few months ago I switched to Mac, tired of the back-pain I had from carrying a heavy (and ugly) Lenovo laptop that had 40% the computing power of my current MacBook Air. So lately I’ve been learning the ways of the mac, and being the cygwin user that I was, I felt at home with the OS X’s Terminal (I use iTerm).
Interesting things that you find while cruising the web searching for OS X tips and tricks. Today, I came across this post from OSXDaily discussing a Mac OS X (10.5+) feature where Launch Services keeps track of your downloads history, excluding few applications that Apple deems ‘safe’ (App Store, Outlook?!)… regardless of your privacy settings or the application mode (e.g. Chrome incognito, Safari Private browsing).
As it seems, it is the responsibility of the application to report the files it downloads, so that the OS will mark the files with an extended attribute as potentially unsafe and present the below alert.
“Example is an application downloaded from the internet. Are you sure you want to open it?”
Apparently, marking the file with an extended attribute wasn’t enough for this ‘security’ feature, so Apple decided they should also keep a log, dating back all these downloaded files. That is a SQLite DB that OS X use to show the download URL in the alert window when the file is executed. And it keeps information about downloads by all kinds of applications (skype, cyberduck, email attachments…), not just browsers. This is a mechanism supposed to protect us users form unintentional execution of malware. Apple didn’t seem to mind that the DB will continue to collect data even after file quarantine was turned off for files and apps.
Apparently this is no news [1, 2, *3*, *4*] in the Mac world, what’s ‘disturbing’ here is the lack of any official user-facing documentation form Apple on the topic (not even in the Apple link “About file quarantine in OS X”), and the lack of a [user-facing] feature to disable this behavior. Seems like a complete disregard of the fact that this can become a serious privacy issue if the computer was indeed exposed to any spyware/malware… Oh the irony :)
View the following file in your favorite XML viewer: /System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/Exceptions.plist and you’ll learn that OS X also blacklist applications (forcing to LSFileQuarantineEnabled). More importantly, OS X doesn’t purge that log, even after you’ve cleared the logs and cache from the applications, so the DB just keeps on growing, forever. - So much for being the application’s responsibility to report.
IMHO, that is a serious privacy violation by Apple. It’s not just that the user is not made aware of the tracking, and not even that there is no user-facing way of canceling it, it’s that it is totally not secure (I could access the file even without a sudo), and that the file is never purged (why would OS X need that download info forever?)
Tracking All Your Downloads…
All the commands below refer to OS X 10.8.2.
Little SQLite and bash: The command below will print all the tracked downloads and the apps that downloaded them ordered by download date (grep at the end for little coloring of the separator):
sqlite3 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV* 'select LSQuarantineAgentName, LSQuarantineDataURLString, date(LSQuarantineTimeStamp + 978307200, "unixepoch") as downloadedDate from LSQuarantineEvent order by LSQuarantineTimeStamp' | sort | grep '|' --color
Pipe an additional grep, and you have a handy filter for downloads per application, per domain, search for specific file names… (yes, you can do that filter using WHERE inside the SQL, but grep will also color the search terms!).
sqlite3 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV* 'select LSQuarantineAgentName, LSQuarantineDataURLString, date(LSQuarantineTimeStamp + 978307200, "unixepoch") as downloadedDate from LSQuarantineEvent order by LSQuarantineTimeStamp' | sort | grep -i 'kaltura.org' | grep '|' --color
Had enough? To clean the file, run a delete query and rebuild the DB to confirm full deletion -
sqlite3 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV* 'delete from LSQuarantineEvent' sqlite3 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV* 'vacuum'
The SQLite table
But before you make it go forever, now that you know of its existence, that file can also be used for some retroactive self-discovery… Below you’ll find all the fields in the DB table and their meaning.
sqlite3 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV* '.schema LSQuarantineEvent'
And investigating the source file: /System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Headers/LSQuarantine.h
- LSQuarantineEventIdentifier – Unique Id for the record
- LSQuarantineTimeStamp – The time (in seconds since 1/1/2001 12am)
- LSQuarantineAgentBundleIdentifier – the downloading application bundle name
- LSQuarantineAgentName – The application name
- LSQuarantineDataURLString – The URL the file was downloaded from
- LSQuarantineSenderName – The name of the person who sent you the email from which you downloaded the attachment
- LSQuarantineSenderAddress - The email of the person who sent you the email from which you downloaded the attachment
- LSQuarantineTypeNumber - Enum identifying the type of application that downloaded the file. The value is one of the following:
- kLSQuarantineTypeWebDownload=0 – if the URL scheme was http(s) the download will be set to this, otherwise to 1.
- kLSQuarantineTypeOtherDownload=1 – anything that wasn’t identified.
- kLSQuarantineTypeEmailAttachment=2 – supposedly an email attachement (I wasn’t able to reproduce this)
- kLSQuarantineTypeInstantMessageAttachment=3 - supposedly a download from instant messaging app (I wasn’t able to reproduce this, Skype download were logged as kLSQuarantineTypeOtherDownload for me)
- kLSQuarantineTypeCalendarEventAttachment=4 – an ical file via email? (I wasn’t able to reproduce this)
- kLSQuarantineTypeOtherAttachment=5 - I wasn’t able to reproduce this
- LSQuarantineOriginTitle – No info, and seemed to be always empty in my log.
- LSQuarantineOriginURLString – “The URL of the resource originally hosting the quarantined item, from the user’s point of view. For web downloads, this property is the URL of the web page on which the user initiated the download. For attachments, this property is the URL of the resource to which the quarantined item was attached (e.g. the email message, calendar event, etc.). The origin URL may be a file URL for local resources, or a custom URL to which the quarantining application will respond when asked to open it. The quarantining application should respond by displaying the resource to the user. Note: The origin URL should not be set to the data URL, or the quarantining application may start downloading the file again if the user choses to view the origin URL while resolving a quarantine warning.”
- LSQuarantineOriginAlias - No info, and seemed to be always empty in my log.
I only had my mac for a few months now, but it’s already showing interesting stories about my personal downloads trail. Maybe I’ll followup on that post with some statistics and analytics of the sites, file types, sources, etc. Till the next time I find time to write.
Did you discover surprising personal downloads trends?