Week in OSINT #2020–16
This weeks topics range from mapping beers and military bases, to scripts to find bad people or scrape the Internet Archive.
I really thought I heard someone say yesterday "do you also notice there's less OSINT links being shared online?"… Well, I seriously thought that would be the case a few weeks ago, but I seriously struggle to keep up with everything! Because of the pandemic people are not only creating innovative news feeds (see below), but they also finally take the time to write down things or develop new techniques and tools. So keep those links coming, I'm more than willing to keep track of them!
- Corona News Feed
- Offensive OSINT
- Website Investigation Workflow
- Mapping Russian Bases
- UntappdScraper Webified
- Trendsmap
- Finding OSINT Eggs in Mobile Apps
- Scraping The Internet Archive
News: Corona News Feed
The website FiftyOne has been bringing you vetted news about cryptocurrency for some time, but they've now changed their default feed to the topic of Covid-19. This website scrapes the news from Twitter, and then uses a sort of 'ranking' system giving points to each share by an expert in the field who shared the link. Only articles with enough points are being posted and they also show how many of them shared it, where they rank the top shared article on top. Thanks sharing this Camille Besse!
Link: https://fifty.one/
Article: Offensive OSINT
Twitter user the_wojciech has written a few blog posts where he touches a more technical side of OSINT. This article is not suitable for everybody, since the techniques are more used by malware researchers, digital forensic specialists, pentesters and the likes. But it is still a really good read if you want to see an example on how several different skills work together to complete a puzzle. The article talks about how making 'obfuscated JavaScript' code readable again leads to some clever online sharing mechanisms by bad actors.
Tip: Website Investigation Workflow
Twitter user https://twitter.com/sinwindie shared with the world a nice flowchart, or as he calls it: "General Website Attack Surface for OSINT Investigations". I must say it is a fairly complete overview of the most important parts you will need when investigating websites. So click that link, like his tweet and follow him to stay up to date when he shares new content!
I decided to finally document one of my usual #OSINT workflows.
— Sinwindie (@sinwindie) April 18, 2020
This chart shows the many points of #exploitation I look for when #investigating a target website.
Keep in mind, this is more for identifying who owns a website rather than collecting pentesting information. pic.twitter.com/6Mjv66Rfga
Tweet: https://twitter.com/sinwindie/status/1251320697370095617
Article: Mapping Russian Bases
User 'Status-6' has been busy mapping things lately, and to be more precise the location of Russian military bases. Over 600 locations were indexed, mapped, described and uploaded for the world to use.
Download link can be found here (tested and safe): https://mega.nz/file/o34z3aZY#nnA8_AioA35c-Jtq3dY0nqr52aH0PuU218Oa8ocoBJY
Original Twitter thread: https://twitter.com/Archer83Able/status/1251197117466644480
Site: UntappdScraper Webified
Micah Hoffman created the script untappdScraper some time ago, that searches for beer drinking lovers on the website Untappd. The script doesn't just give insight into the drinking habits of the people (lol) but is mainly interesting to track its users all around the globe, while they log their beverage consuming moments. With the help of Brendan Evans and others he now has the script online, with recent activity, visited locations and even a map if you want to!
Link: https://webbreacher.github.io/untappdScraper/
GitHub: https://twitter.com/WebBreacher/status/125146790182094438
Site: Trendsmap
OSINTtechniques shared the link to 'Trendsmap' this weekend, which is a website that visualises the hashtags and users that are most active in a certain region. The site has some interesting options, but most of that is locked behind a subscription. If you do want extensive Twitter analysis on specific hashtags, keywords or locations, it might be interesting to look at. But for me the map where the most popular hashtags and users are plotted are already interesting. And I can imagine that journalists would love to see upcoming trends in their own region, so they can start investigating stories right when they happen.
Link: https://www.trendsmap.com/
Article: Finding OSINT Eggs in Mobile Apps
This article by @BTF117 might get somewhat technical for some of my readers, but if you want to find new techniques or hidden gems, then this is absolutely worth a read. It shows how to use the Developer Tools to investigate links, but also how to use a proxy to track the different URL's that mobile apps are communicating with. And that has also given me the idea to maybe start writing some articles on OSINTCurious, touching these kind of topics, maybe let me know if you'd like that and what you would like to learn. Anyway, nice read, and thanks for the share Kris G!
Link: https://link.medium.com/xHhCV8TBL5
Tutorial: Scraping The Internet Archive (Fr)
This article by OpenFacto is in French, and I have to say that the word 'scraping' is used with respect (do read the article to the end!). It was written by Hpiedcoq just yesterday and despite it is only in French, I do recommend you to read it, and Google Translate actually creates readable English content. How many times have you seen loads of useful information on a website, that was archived by the Wayback Machine, but it took too much time to click on things? Well, that is where this article comes in, teaching you how to patiently scrape archived content and download it in bulk so you can use it in an investigation.
Link: https://openfacto.fr/2020/04/19/recuperer-des-fichiers-pdf-en-masse-sur-archive-org/
| FirstDraft Course
First Draft has updated some course materials and offers a new course for journalists, to help battle the disinformation about the Coronavirus. When you are on the website, also check out their other resources, like the flashcards, their basic toolkit on Start.me or their essential guides. Even though First Draft is aimed at journalists, I do recommend people with investigative jobs to have a browse through their content. Because learning from another can help you get better at your own job!
Course: https://covid.firstdraftnews.org/
| Class Central Courses
Classcentral is another place I recommend you to have a look if you are searching for free courses. Whether it's a course about law, IT security, starting out with Python of even learning Dutch 😉 you can find it all there. And even better is the fact they have links to all kinds of online training centers that offer free classes!
Link: https://www.classcentral.com/report/free-online-learning-coronavirus/
| OSINT Bibliography
During the OSINTCurious webcast with Arno Reuser, that was recorded yesterday, Arno shared a link where he keeps a bibliography. It's possible to order books online, but since you're stuck at home and might want to read up on some OSINT related topics, here's something to keep you busy for the months to come! Thanks for sharing that with us!
Link: http://bibliography.opensourceintelligence.biz/
Have a good day and have a good search!
