none028

유출 계정 덤프에서 빠르게 유효한 이메일 계정 정보만 체크하기 위해 작성한 스크립트입니다.

Code Link: github.com/none028/CHECKER

Input: test.txt

none028@gmail.com,1234
1234,213114
000@gmail.com,123141
!@#!@#@gmail.com,123124

Run: python validEmail.py -t test.txt -l 1 -o "my_data"

Output: 2020-09-12_my_data.csv

none028@gmail.com,1234

Cyber Threat Intelligence에서 가장 핵심적인 요소인 Threat Actor에 대해서 기존에 있는 자료들을 수집했다. (그냥 자료 모으기용 포스트)

1. Malpedia - Malware Wikipedia (?) [link]

Threat Actor와 관련된 간략한 정보와 함께 최근 뉴스들을 확인할 수 있다.

2. Threat Group Cards: A Threat Actor Encyclopedia by ThaiCert [link]

해당 보고서는 다른 자료들과는 다르게 Threat Actor별로 중요한 요소들을 한번에 요점정리한 느낌이였다.

3. MITRE ATT&CK [link]

MITRE ATT&CK® is a globally-accessible knowledge base of adversary tactics and techniques based on real-world observations. The ATT&CK knowledge base is used as a foundation for the development of specific threat models and methodologies in the private sector, in government, and in the cybersecurity product and service community.

With the creation of ATT&CK, MITRE is fulfilling its mission to solve problems for a safer world — by bringing communities together to develop more effective cybersecurity. ATT&CK is open and available to any person or organization for use at no charge.

MITER ATT & CK®는 실제로 일어나는 적의 전술과 기술에 대한 전 세계적으로 액세스 가능한 자료이다. ATT & CK 자료는 민간 부문, 정부 및 사이버 보안 제품 및 서비스 커뮤니티에서 특정 위협 모델 및 방법론 개발을 위한 기반으로 사용된다.

ATT & CK의 설립으로 MITER는 보다 효과적인 사이버 보안을 만들어 가기 위해 다함께 커뮤니티에 모아서 보다 안전한 세상을 위한 문제를 해결하는 사명을 완수하고 있다. ATT & CK는 모든 사람이나 조직이 무료로 사용할 수 있도록 개방되어 있다.

MITRE ATT&CK는 앞서 언급된 자료보다 더 디테일하게 정리되어 있었다. 위에서 ThaiCert 보고서에서 나온 APT37 관련 자료를 MITRE에서 찾아보면 아래와 같이 나온다.

그리고, APT37을 클릭하면 해당 TA (e.g. Threat Actor)에 대한 간단한 정보와 이들이 사용하는 테크닉에 대해서 상세하게 정의되어 있다.

[공유] Peerlyst - Some useful Open Source Intelligence (OSINT) tools by David Dunmore

해당 포스트는 Peerlyst - Some useful Open Source Intelligence (OSINT) tools by David Dunmore 포스트에 있는 OSINT 도구들에 대한 내용만 발췌한 포스트입니다.

Maltego

The aim of Maltego is to analyze real world relationships between people, web pages and sites, groups, domains, network infrastructure, Social Networks, and indeed anything else that is discoverable on the internet. Maltego can then present these results in a variety of graphical formats.

It is mainly used in Open Source Intelligence, and digital forensics

If you want to see the differences between the various versions of Maltego, have a look at this page

The Community edition can be downloaded from Paterva's downloads page, and is available for Windows, Linux (.deb and .rpm, and .zip) and for Mac. The downloads page detects your OS and offers the relevant format for the download.

Here’s Maltego’s UI from my installation (On PCLinuxOS using the rpm packaged installation file), showing the default transforms, and a couple of useful free (As in free Beer) that I’ve found to be useful.

I wrote a short series of two posts about Maltego’s free Community edition a while ago called ‘How to use Maltego’ Part 1 and Part 2.

Shodan

On the Home page, Shodan describes itself as ‘The search engine for the Internet Of Things’. It not a free Open Source resource, rather Shodan has several subscription plans, which are on a monthly rolling basis, so you can subscribe for just one month to evaluate its usefulness. The main difference between them is the number of IP addresses per month you can access.

Shodan also has an API, use of which is included with all the subscription options, but you have to register separately to access the API. The documentation for Shodan’s REST API is available here.

NOTE: All the API methods are limited to 1 request per second. This may or may not be a limitation you can live with. For development purposes it’s unlikely to be an issue, but I can see this being a bottleneck in some investigative scenarios.

Shodan can be integrated with a number of application, including Maltego, Nmap, Metasploit and many others here’s the full list.

Google Dorks

Despite the overtly nerdy, techy name, there’s really nothing mysterious going on here. Simply put, Google Dorks are just smarter / more advanced ways to search using Google for more specific results. Once you’re familiar with some of these techniques, you will (genuinely) wonder how you searched previously.

Here are a few that I find useful, here is a list of more of them.

“Double quotes” - (“”) search specifically for the exact words inside the double quotes

Being a British Railway nerd, I searched for “Bulleid Pacific Clan Line” which returned 102 results specific to that particular locomotive which was built in the 1940s for British Railways Southern Region.

Camera:£350 This one does not give exactly the result you might expect. It does return one result for cameras under £350, but also several results for cameras that feature the number 350 in their names. Under the default ‘All’ search, search using the ‘shopping’ tab does indeed return a large number of results for cameras under £350, including some IP cameras and CCTV ones as well.

To be a bit more flexible, try camera £100..£300 for cameras in that price range. Again for best results, choose the ‘shopping’ tab.

You can use AND. OR to combine results. e.g. camera AND film to exclude digital cameras.

To search social media, use the @ operator e.g. @Facebook or @twitter

intext:Mars Rover Finds web pages that include ‘Mars Rover’ in their text (57.5 million results when I tried this one).

Site:autotrader.co.uk Limits the search to the specified site

for more, there’s the link above, but there are many more, just search for ‘Google Dorks’ and be amazed!

While not Dorks per se, Google have a number of other useful resources, including images and maps. One I have found to be very handy is Google Correlate

Here’s a simple example, I used the term ‘Diet and Exercise’

CheckUserNames

This is potentially a very useful site, the aim is to see whether a given username is available on any of more than 160 social networking sites. The homepage is Check User Names, If you enter a username, the site will highlight which sites have the name available. It’s a handy way of seeing whether a given username is registered on social media. The site also has a link to their new site knowem.com, which claims to check more than 500 social media sites I tried some of my social media usernames, and the site (CheckUserNames) did correctly identify the sites where the names have been registered.

The FOCA

Fingerprinting Organizations with Collected Archives to expand on the (possibly devised to fit the tool’s functionality) acronym.

The main purpose of this tool is to find metadata and other hidden data contained in a variety of documents and some image files and to show relationships, for example, that some documents may have been produced by the same person or team.

FOCA is a Windows application that requires an instance of SQL Server to be available for it to store its search results, which may produce a large volume of data.

The FOCA is open source, released under the GNU Public License (GPL), and is available for download from this GitHub page.

Eleven Paths, who are the authors, also have a FOCA market page where you can download several plugins to expand on FOCA’s functionality.

There are several video Tutorials on YouTube, this link will find them.

While thinking about what useful information can be found in metadata, another very useful tool is

Metagoofil,

written by Christian Martorella of Edge Security, here’s Metagoofil’s page It’s written for Linux and can be downloaded as a .tar.gz file from Google Code Archive here. It’s not been updated since Feb 10 2013, but that may not matter, provided the functionality is correct. Metagoofil ins included in Buscador below, where it can be found in Software → Domains, which opens a small box titled ‘Domains: Choose tool, which contains radio buttons to choose a tool:

Clicking OK opens another small message with an input field for a URL to search. enter a URL, and

Metagoofil will go off and do its thing. Returning its findings in the form of an html page in its own folder.

Metagoofil is also included in Kali Linux.

Spiderfoot

Is an automated OSINT tool that gathers freely available information from more than 100 public sources, the type of information that Spiderfoot can gather includes IP addresses, Email addresses, names, and quite a lot more. It can present the information that it finds in a variery of graphical formats

Spiderfoot is available for Linux,BSD, Solaris and Windows. The Windows version is a freestanding executable (.exe file) that appears to be portable as the website says that it comes pre-packaged will all dependencies.

If you don’t want to compile Spiderfoot for Linux, it is also available packaged for Docker.

For further information on Spiderfoot, here’s a link: Spiderfoot's home page

If you use Kali Linux, you’ll know that Spiderfoot is not included in the default Kali distribution, but it can be installed Here's a link to a tutorial blog post. Note that some browsers may flag this as a site that has been reported as containing harmful software. That’s because it contains links to download Spiderfoot.

There are quite a few video tutorials on YouTube, this link will find them.

Buscador Investigative Operating System

This is a Linux distro loaded with OSINT tools, available as a Virtual Machine (VM)image for both VirtualBox and VMWare. It is developed and maintained by David Westcott and Michael Bazzell, and hosed on Google Drive The link for the download, and a list of included OSINT applications are available from IntelTechniques on their Buscador page, which also has a helpful list of installation notes, and some helpful notes on using the image in a virtual machine.

This may be the subject of a future post (or short series of posts) in some detail – watch this space!

There are several video tutorials on YouTube, this link will find them.

Kali Linux

Although this is intended primarily as a Penetration tester’s toolkit, it aims to be the pentester’s ‘Swiss Army Knife), Kali does contain several tools of interest to the OSINT investigator, including Metagoofil and The Harvester, which is a tool used to gather email accounts and sub-domains from publicly available sources, and a number of other tools. If you've not considered Kali for its OSINT tools, do have a kook, there’s a good amount of video tutorials on YouTube, this link will find them.

There are a good number of Video tutorials for metagoofil on YouTube, this link will find them.

There are also a similarly good number of video tutorials covering The Harvester on YouTube, this link will find them

Search Engines

And let’s not overlook the obvious start point for most OSINT (or indeed any other intelligence gathering) – Search engines, and Google in particular, have become so embedded in our consciousness that they may not register as the valuable tools that they undoubtedly are. If you doubt that, imagine trying to find a supplier of cheap watches in China or Japan (If you’re in Europe or the USA) without internet access and a search engine.)

Some (like Google, Bing and Yahoo) will track your searches, others, primarily DuckDuckGo make a point of NOT tracking your searches.

Some of the others, Ask for example have changed their underlying technology. Ask, at one time ‘Ask Jeeves) was for a while effectively a re branded Yahoo search (as is Swagbucks search currently), now however, Ask.com appears to be it’s own thing, and any association with another search engine is not readily apparent. Tracking these changes of ownership might be an interesting OSINT exercise.

Please feel free to do this exercise, and post your results in the comments section under this post.

Some old (And I thought long defunct) search engines are still going like Dog Pile, which used to search several other engines ad return aggregated results. This site lists several other search engines, and a number of other resources that are useful for OSINT investigations.

Have I Been Pwned

This is a very useful site, just enter an email address and the site will tell you whether the email address has been pwned (found among others in one or more breaches) try it here: Have I Been Pwned.

For a more comprehensive overview of some other OSINT tools, have a look at E Investigator's OSINT tools page.

Virus Total

If you need to check a file for known viruses, then Virus Total is the place to go, here’s their upload page where you can upload a suspect file for checking. Virus Total also have a number of API scripts allowing developers to use the Virus Checking functionality within their applications.

A variety of languages are supported, as are Maltego, in the form of a couple of transforms.

They also have and app for Android in the Play Store, desktop applications for Windows and Mac, but not specifically for Linux, although the Mac application (which uses Qt) can be compiled for you own distribution and browser extensions for Chrome, Firefox and Internet Explorer, but apparently not for Edge yet.

Forums (Fora?)

There’s any number of forums dedicated to most interests (including some that are illegal in most jurisdictions). The best way to find forums, if they’re relevant to your research, is probably to use a search engine (I prefer DuckDuckGo as it doesn’t track your activity).

It’s worth using mire than one search, using slightly different terms, as they can sometimes give differing results.

Blogs

there are a number of sites than claim to help you find blogs on any subject, but the one that works for me is The Blog Search Engine which along with blogs on the chosen subject, will also return YouTube videos and other related sites.

Useful Documents

Hiding from the internet book workbook new. Source: IntelTechniques

OSINT tools – Useful Links - new