Building a history of search terms
14 January 2016
The keywords a user chooses in order to search the Internet are one of the most valuable pieces of information that can be retrieved from browser history. The chosen keywords often give an insight into the exact content the user was searching for and provide some context to the remainder of the Internet records stored. Often typing errors made by the user while entering search terms will also provide evidence that a user was physically sat at the keyboard typing (if there is any dispute as to how the keyword came to be generated and recorded).
So where can we find this search history? The files on disk that hold records of these searches are largely dependent on what web browser was used by the user.
If Chrome is being used then searches carried out using a number of major online services will be explicitly stored within the 'keyword_search_terms
' table of the 'History
' SQLite database file. Within this table Chrome records the keywords the user searched for and the URL associated with that search.
If Firefox is being used then any searches made via the Search Bar
are stored within the 'moz_formhistory
' table of the 'formhistory.sqlite
' database file. This table is typically used for storing text entered into forms on web pages. We can identify searches made via the Search Bar by querying for records where the 'fieldname
' value is 'searchbar-history
'. As the moz_formhistory table is only populated for the purpose of autocomplete options, the associated URL is not stored in the same table.
We can also extract search history from a number of other sources regardless of the web browser being used. Many websites execute searches using a HTTP GET request and pass the search terms within the URL. A common approach to this is to include the search terms within the query string part of the URL. For example, here is the URL for searching for the term 'dfir
' on Twitter:https://twitter.com/search?q=dfir&src=typd
The query string is the part of the URL following the '?' character, and can contain a number of name-value pairs separated by '&' characters. Knowing the structure of the Twitter search URL allows us to parse out the search terms. To be certain that the query string parameter 'q
' represents the search terms we can only consider URLs where the path preceding it is '/search
'. Many other websites include search terms in a similar manner, such as Bing, eBay, Facebook, Yahoo and YouTube.
One notable exception to this is the Google search engine which often includes search terms within the URL fragment (which is at the end of the URL after the '#' character). For example, here is the URL for searching for the term 'digital forensics
' on Google:
Some websites use cleaner looking URLs where the search terms are included in the path of the URL. For example here is the URL for searching for the term 'England
' on Google Maps:
Therefore, any URL that appears in browser history may contain keywords that have been searched for on a website. This means the potential sources for building a history of searches can include Website Visits, Session Data, Cache Records and Favicons.
Browser History Examiner
(BHE) makes this process much simpler by automatically extracting search terms from multiple sources for popular sites such as Google, Bing, Yahoo, Google Maps, YouTube, Facebook, Twitter and eBay. Where possible the date of the search is included, along with the search terms, the search engine and the original URL from which this data was extracted.
Please get in touch if you have suggestions for further search history relating to specific websites that you'd also like to see parsed in BHE.