278 reads

Your OSINT Study Plan (Part 1): Sharpening Up The Basics

by Security by AccidentJanuary 11th, 2024

Too Long; Didn't Read

Part One of my series that teaches the art and craft of OSINT.

featured image - Your OSINT Study Plan (Part 1): Sharpening Up The Basics

Open Source Intelligence (OSINT) has been one of the most useful skills I have learned. Many of the mindsets, methods, and tools it uses can be applied to a variety of tasks, like job hunting, pen-testing, or research for writing tasks. There are many great resources out there that help in learning and developing this skill, and this blog post aims to provide a repository and a structured format to guide you on your journey to learn this amazing skill.

How To Read?

This post should be read in full without clicking on all the cool links or trying out the stuff I am talking about. This will allow you to absorb the message of this post. Afterward, you can briefly go over it again, click all the links, and try all the stuff.

OSINT?

I will not spend a long time providing a definition of what OSINT is. There are plenty of discussions about this out there, and I do not think a perfect definition is of much value to someone starting to learn this skill. For the purpose of this series of blog posts, OSINT will be the activity of collecting and analyzing open-source data to answer specific questions. There is much more to this definition than meets the eye, but we will leave it at that in the meantime and come back to it later.

The OSINT Mindset

I hear you screaming, "Booh, give us some tools and tricks," and I will soon do so. I just want to briefly mention that a proper mindset is the foundation of being a good OSINTer. But it is the same as in martial arts: while a proper mindset is key, they still start by teaching you how to stand and throw punches before discussing when and why you want to and don’t want to hit someone. Therefore, I will show you some OSINT punches first, but I ask you to not neglect the importance of learning about an “OSINT Mindset” and hope you come back to it once you feel confident enough to throw some punches.

Searching For Stuff

If you want to analyze open source information to answer certain questions (aka. OSINT), you obviously have to know where to find all the information you need to answer your questions.

Disclaimer: Many people view OSINT as an activity that is only about finding information. While it is crucial for OSINT just sucking up information does not do justice to the INTelligence part of OSINT.

The ways of finding information are just countless, and if you follow some OSINT creators on X (which you should), you will receive new sources, hacks, and dorks almost on a daily basis.

The first thing you should learn is how to google it, but do it the right way. This is so-called Google Dorking. I will not do another guide about it because there are plenty of them out there, but you can check out the great blog written by HackTheBox: https://www.hackthebox.com/blog/What-Is-Google-Dorking, and you can use the information in it to find some more guides that will help you develop this skill. Many of the search operators detailed in this blog post work on other search engines like Yahoo, Bing, Yandex, and other specialized search engines.

And that is the next point we want to talk about: Specialized Search Engines.

Google is great and all, but sometimes you have some special thing you want to search for. At this point, it is handy to have some more niche search engines at hand, like shodan.io or netlas.io for internet-connected devices, or grep.app for source code, or intelx.io for data leaks and breaches. This amazing GitHub profile, https://github.com/cipher387, lists a lot of search engines and some more amazing stuff, so you should browse through it and take notes.

But you don’t always need a specialized search engine; the site search operator can do wonders. Sure, a specialized GitHub, Amazon S3 Bucket, or paste search engine is fun, but do not forget the good old site:github.com, site:amazonaws.com, or site:pastebin.com search queries. Sometimes, they have better results than custom search engines.

I feel like you have heard a lot of theories. It's time to give you a juicy OSINT snack. One of my favorite Google Dorks:

site:amazonaws.com “Confidential”

You go ahead and figure out what it does, and as a homework assignment, find some domains for the other cloud providers that you can use this dork with.

Taking Notes

Now, we can find quite a bit of information, but we have to keep it organized somehow. Here, you need three things: something to take notes, something to do mind maps, and (maybe) something to do free-hand sketches.

Notes are obviously to save all the nice tidbits of information you have collected. You should have a certain kind of structure that works for you. Sometimes, I like to dump stuff in a large document and move it around and be a bit messy; some like their stuff in a nice and clean structure. You have to test and find out what works for you.

Mind maps are great for investigations because investigations are about connecting the dots, and well, that is what mind maps do. They allow you to show how information is connected and what path your investigation follows.

Free-hand sketches do not have to follow any structure; you are only limited to the things you can produce with a pen. This is really great for times when you need ideas to flow, like at the beginning of an investigation or at a pivotal point in your investigation.

The question now is: What tool should I use?

Technically, OneNote can do all of those things. Some people really love Notion or Obsidian. I will not recommend any tool because this is a lot about individual preferences and styles. I will just give you some considerations when choosing an application:

Notebook format: For your main notes application, you have to use something that allows you to organize your notes in different sections and categories. So using Word or txt files is a terrible idea.
Sensitivity of data: If you work on high-profile cases and/or sensitive information, you might want to consider a solution with local notes rather than something that relies on cloud storage.
All-in-one vs. tailor-made: While applications like OneNote could do all the things you need, they might not do all of them well. So, you have to consider the trade-off between convenience and functionality.
Cross-platform/device: If you have that one special investigation device, you do not need to think about it. If you like to work from multiple locations on multiple devices with varying operating systems, you need to have something that syncs your notes wherever you are. See also the consideration of data sensitivity.
AI and other special features: AI just recently emerged, but plenty of applications now allow you to interact with and enhance your notes via AI. Consider if these features are beneficial to you or if your data might be too sensitive for them.

Pivoting

In OSINT knowing how to pivot is (caution: bad joke) pivotal. What does that mean? You need to be able to use information from source A to get better information from source B. You also need the ability to get from something like a social media profile to information like a physical address. You need to be able to correlate everything you do and uncover during your investigation. This is what real analysts do. I even wrote a whole blog about it, and you can check it out here: https://security-by-accident.com/unearting-sources-osint/ . This concept can only be fully understood and learned by practicing your skills.

Asking the right questions

This is something I was referring to in the section about “The OSINT Mindset,” and I did not talk more about it because I knew you were not ready for it. Even though every OSINT process MUST BEGIN with a well-defined question, the OSINT learning journey must begin with uncovering some hidden stuff, or else you will not be able to appreciate the fascinating power of OSINT.

But why are questions so important in OSINT? Imagine this: You are a professional OSINT investigator. A client gives you an assignment. Your target is “Peter James van Doe.” The client sends you his LinkedIn profile. Immediately, you start investigating. You know about this great hack someone showed you that allows you to get the email he used to create his LinkedIn account. This email is amazing; he uses it everywhere, where you find passwords, social media accounts, forum posts, and all that stuff. You are having a blast, but you end up OSINTing nonstop till it is way after midnight. You send your client an email telling him how well the investigation is going.

The next morning, your client calls. She says, “And? Should I hire this guy as an accountant?”. "Oh, f***ing hell.”, you think. You’ve had a fun evening by wasting some time because you did not know why you were doing all this OSINT. Sure, you have some rough idea of whether that guy is trustworthy or not. But can you give your client a high-confidence assessment? Can you bill your client for all those hours you spent?

Sure, the investigation was fun, and doing OSINT investigations for fun and just pulling all the tricks and tools you have is great for sharpening your OSINT skills, but it IS NOT a PROFESSIONAL OSINT investigation.

So before you start your investigation, think about the “WHY?” and “WHAT?” before you think about anything else, and only start investigating after you have clearly written down these points. And afterward you can concern yourself with the HOW? and then you may start investigating.

I could write a lot more about this point, but it was done before by one of OSINT’s finest, and you can check out his article here: https://medium.com/secjuice/osint-as-a-mindset-7d42ad72113d. Please do give it a good read.

Where tools?

You read this article because you wanted some cool OSINT tools, right? Well, I am not going to give them to you YET. If you have read the article in the previous section, you will know why. But stay tuned for the next part of this series. There will be tools, I promise.

And now?

Well, you do not really have tools, but you have something far better. You are equipped with the foundational skills and concepts of a good analyst. Now, go ahead and ask some questions you want to learn more about. Use the concepts and skills from this article and answer them. On this journey, you will become a better analyst (if you do it right), and you will automatically find some tools you can use.

And as a little treat, I will share with you one of my other posts that talks about one of the most fun OSINT topics that you can explore: https://security-by-accident.com/leaks-breaches-osint/ .

But seriously, please do some real investigating with questions, notes, mind maps, and stuff.

Also, follow me on X for some OSINT and security knowledge: https://twitter.com/Secbyaccident and wherever you have read this blog post.

Homework:

Follow OSINT people on social media.
Join some OSINT discord servers and make some connections in the OSINT community.
Understand why you want to learn OSINT. This will help you understand what skills you will need to pick up.