In the recent 2020 Cyber Threat Intelligence Report, 34% of OSINT practitioners reported that they had no prior experience with OSINT collection and 85% have received little or no training in OSINT techniques and risks. In accordance with this – we at Media Sonar thought it might be helpful to cover some OSINT investigation concepts that are beneficial to keep in mind as you collect open source intelligence.
Definition of Open Source Intelligence
OSINT is publicly produced and publicly available data that can be collected and shared without breaking laws or policies, needing a warrant, or participating in what would be commonly considered shady practices. The U.S. Department of Defense/U.S. Director of National Intelligence, as well as security researcher Mark M. Lowenthal, cover most of the principles in their definitions of OSINT.
US Department of Defense / US Director of National Intelligence
“[OSINT is intelligence] produced from publicly available information that is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence requirement.”
Mark M. Lowenthal
Former Assistant Director of Central Intelligence for Analysis and Production, US Central Intelligence Agency defines OSINT as, “any and all information that can be derived from overt collection: all types of media, government reports and other documents, scientific research and reports, commercial vendors of information, the Internet, and so on. The main qualifiers to open-source information are that it does not require any type of clandestine collection techniques to obtain it and that it must be obtained through means that entirely meet the copyright and commercial requirements of the vendors where applicable.”
OSINT Investigation Best Practices
Civil Rights and Liberties
We’re going to point to the elephant in the room right out of the gate: regardless of the legality of OSINT collection, especially as it pertains to social media – the entire OSINT industry is bound to and driven by civil rights and liberties. The public’s opinion of the use of this data – the optics that surround it – matters. Direly. As a result – it should be of ultimate concern for OSINT investigators. As practitioners and investigators – integrity and a high-level of ethics should be front of mind as an investigation proceeds.
To meet the demands of the security landscape – whether that be corporate, law enforcement, government, intelligence, or military – a comprehensive set of tactics, techniques, and procedures (TTPs) must be employed. To that point – OSINT is no magic bullet. It is but one facet of intelligence one could gather to be added to a bigger pool of investigative data from other sources in order to best complete the mission at hand. OSINT should not – ever – be the only tool in your toolbox.
The Law and Warrants
A question we often get asked – “Is OSINT Legal?”. While maybe not directly clear to corporate teams – as it is to judicially-bound teams – why consideration of the law is important, that consideration drives some best practices for an OSINT investigation.
For law enforcement this topic is easy: don’t do anything that procedurally endangers the usefulness/prosecution of an investigation, and if you feel you need or definitely need a warrant – get one.
For corporate security teams: this connection to the law can become a bit more vague. The obvious association or importance of adhering to laws and investigation frameworks is if your organization wishes to pursue legal action against your findings. In this case – anything that would sully a law enforcement investigation in criminal court will likely do the same to your investigation in civil or criminal courts. Keeping the topics of ethics, probable cause, and a general sense of “am I doing something shady right now,” etc. in mind will go a long way to maintaining the standing of your investigations for future possible legal use.
An important concept for both parties to consider: it can be easy to get lost in the amount of OSINT data out there, and the lines of how and where you find that data can start to blur. Perfect example: public Facebook data. Information and public posts on Facebook are technically “OSINT” – except that they aren’t. There’s an important part of Mark Lowenthal’s definition of OSINT to recall:
“The main qualifiers to open-source information are that it does not require any type of clandestine collection techniques to obtain it and that it must be obtained through means that entirely meet the copyright and commercial requirements of the vendors where applicable.”
In reality – Facebook Terms of Service policies on public data scraping are that it is not allowed. Period. If you’re Law Enforcement and you need access to Facebook posts, you really should be getting a warrant and engaging with Facebook directly. If you’re a corporate team, there isn’t really going to be a tool out there that will give you access to scraping that data. That’s not to say you cannot manually view public content on Facebook – but the scraping of the data, or creating false profiles in order to view private content (read: “something shady”) breaks Facebook policy, and therefore can put an investigation at risk.
The point here is that it’s not just the traditional “Law” that needs to be considered – but even corporate “laws” and policies. In order for OSINT to remain OSINT, the collection and use of the data need to be legal and legit.
An OSINT investigation is simply an OSINT investigation
OSINT investigations are just that – an investigation that collects OSINT data and that alone. It generally shouldn’t be associated with hacking, intrusion testing, physical security testing, undercover operations or any other security-related offerings. If your OSINT investigation starts to require the above, it might be a good time to recall a statement from the previous section: it might be time to get a warrant.
Getting started with OSINT can be intimidating. We’ve put together a report to help you understand How to Use OSINT for Security POI Investigations where you’ll learn how you can gather, consolidate, analyze, and organize open intelligence on Points of Interest such as People, Organizations, and Domains.
“Masters” and Learning
This section is less related specifically to OSINT or the landscape of security, but to more to all human experience-based endeavors: always be learning, and be cautious of claiming, or anyone that claims, to be a master in the field. Realistically, across all areas of knowledge and skill, the most seasoned practitioners and teachers will admit that for all they know, there is still much to be learned about the unknown.
In short – there is always room to grow and learn something new. Challenge yourself, keep an open mind, and always seek new and conflicting information to compare against your previously held beliefs about OSINT and its practices. Never stifle yourself by falling into the fallacy of believing that you’ve “mastered” any craft to finality.
The vast landscape of OSINT data and its usefulness in enhancing your security posture is undeniable. With the increase in its demand and the evolution of its applications across the security industry, it’s easy to sprint to the finish line in terms of OSINT data – and miss important concepts and practices that will better ensure the success of the mission. By keeping ethics and civil liberties at the top of mind, and incorporating OSINT data into a larger collection of information – you maximize the effectiveness of your investigation. By eliminating personal bias, committing to continual learning, and applying your knowledge creatively – you maximize your growth and success as a practitioner.
OSINT can be extremely powerful when used effectively and ethically. A cliché – but applicable – quote to leave on: “With great power comes great responsibility.”