STRLCPY/opensquat

release 1.96
Andre Tenreiro committed 4 years ago

8dbd0cef

1 parent 075f3cc5

Revision indexing in progress... (symbol navigation in revisions will be accurate after indexed)

■ ■ ■ ■ ■ ■

CHANGELOG

	1	+	v1.96 (2020-10-11)
	2	+	+ Bandwidth saving - only download the latest feeds if it has changed
	3	+	+ added generic.txt containing generic keywords used for phishing
	4	+
1	5		v1.95 (2020-09-25)
2	6		+ Added --portcheck flag to validate if flagged domains have port 80/443 opened
3	7
		skipped 60 lines

■ ■ ■ ■ ■ ■

README.md

1	1		openSquat
2	2		====
	3	+	Your phishing and domain squatting watchdog :dog:
3	4
4	5		[![Codacy Badge](https://api.codacy.com/project/badge/Grade/9231646e8ddf4efc9fb1f62f628df34a)](https://www.codacy.com/manual/atenreiro/opensquat?utm_source=github.com&utm_medium=referral&utm_content=atenreiro/opensquat&utm_campaign=Badge_Grade)
5	6		[![Hits](https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2Fatenreiro%2Fopensquat&count_bg=%2379C83D&title_bg=%23555555&icon=&icon_color=%23E7E7E7&title=hits&edge_flat=false)](https://hits.seeyoufarm.com)
		skipped 3 lines
9	10		What is openSquat
10	11		-------------
11	12
12		-	openSquat is an opensource Intelligence (OSINT) security project to identify cyber squatting threats to specific companies or domains, such as:
	13	+	openSquat is an opensource Intelligence (OSINT) security tool to identify cyber squatting threats to specific companies or domains, such as:
13	14
14	15		* Phishing campaigns
15	16		* Domain squatting
		skipped 17 lines
33	34
34	35		This is an opensource project so everyone's welcomed to contribute.
35	36
36		-	Installation
	37	+	Web Demo
	38	+	------------
	39	+	Please check [Phishy Domains](https://phishydomains.com) for a simple version of the openSquat.
	40	+
	41	+	How to Install
37	42		------------
38	43
39	44		```bash
40	45		git clone https://github.com/atenreiro/opensquat
41	46		pip install -r requirements.txt
42	47		```
	48	+	Make sure you have Python 3.6+ and pip3 in your environment
43	49
44		-	Make sure you have Python 3.6+ and pip3 in your environment
	50	+	How to Update
	51	+	------------
	52	+	To update your current version, just type the following commands inside the openSquat directory:
	53	+	```bash
	54	+	git pull
	55	+	pip install -r requirements.txt
	56	+	```
	57	+	The "pip install" is just to make sure no new libs very added with the new upgrade.
	58	+
45	59
46	60		Usage Examples
47	61		------------
48	62		[Demo Video](https://asciinema.org/a/361931)
	63	+
	64	+	Edit the "keywords.txt" with your customised keywords to hunt.
	65	+
49	66		```bash
50	67		# Lazy run with default options
51	68		python opensquat.py
52	69
53	70		# for all the options
54	71		python opensquat.py -h
	72	+
	73	+	# Search for generic terms used in phishing campaigns (can lead to false positives)
	74	+	python opensquat.py -k generic.txt
55	75
56	76		# With DNS validation (quad9)
57	77		python opensquat.py --dns
		skipped 34 lines
92	112		* ~~Homograph detection~~ done
93	113		* ~~Improve code quality from B to A grade (codacy)~~
94	114		* ~~PEP8 compliance~~
95		-	* Integration with PulseDive
96	115		* Add documentation
97	116
98	117		Feature Request
		skipped 13 lines
112	131		Contributors
113	132		* Please check the contributors page on GitHub
114	133
	134	+	How to help
	135	+	-------------
115	136		You can help this project in many ways:
116	137		* Providing your time and coding skills to enhance the project
117	138		* Build a decent but simple [project webpage](https://opensquat.com)
		skipped 6 lines

■ ■ ■ ■ ■ ■

keywords.txt

1 1 #This is a comment
2 - google
3 - facebook
4 - amazon
2 + #google
3 + #facebook
4 + #amazon
5 5 paypal
6 - microsoft
6 + #microsoft

All occurrences
■ ■ ■ ■ ■ ■

main.py

1 - print('\nRun file has moved to "opensquat.py" - please run this one!\n')
2 - exit(0)
3 -

All occurrences
■ ■ ■ ■ ■ ■

opensquat/__init__.py

1 1 # -*- coding: utf-8 -*-
2 2 # Module: __init__.py
3 3 """openSquat Version."""
4 - __VERSION__ = "1.95"
4 + __VERSION__ = "1.96"
5 5

All occurrences

■ ■ ■ ■ ■ ■

opensquat/app.py

		skipped 11 lines
12	12		"""
13	13		import requests
14	14		import bisect
	15	+	import hashlib
	16	+	import os
15	17		from opensquat import __VERSION__
16	18
17	19		from colorama import Fore, Style
		skipped 2 lines
20	22
21	23
22	24		class Domain:
23		-
24	25		"""
25	26		The Domain class with handle all the functions related to
26	27		the domain verifications.
		skipped 96 lines
123	124		):
124	125		self.list_file_keywords.append(line)
125	126
	127	+	def check_latest_feeds(self):
	128	+
	129	+	URL = self.URL + self.URL_file + ".md5"
	130	+
	131	+	print("[*] Checking for the latest feeds...")
	132	+
	133	+	# User-Agent
	134	+	ver = "openSquat-" + __VERSION__
	135	+	headers = {'User-Agent': ver}
	136	+
	137	+	try:
	138	+	response = requests.get(URL, headers=headers)
	139	+	except requests.exceptions.RequestException:
	140	+	return False
	141	+
	142	+	if (response.status_code != 200):
	143	+	return False
	144	+
	145	+	latest_checksum = response.content.decode('utf-8')
	146	+	latest_checksum = latest_checksum.replace("\n", "")
	147	+	latest_checksum = latest_checksum.strip()
	148	+
	149	+	response.close()
	150	+
	151	+	# Compare if local file is the latest
	152	+	if os.path.exists(self.domain_filename):
	153	+	with open(self.domain_filename, "rb") as f:
	154	+	try:
	155	+	local_checksum = hashlib.md5(f.read()).hexdigest()
	156	+
	157	+	if latest_checksum == local_checksum:
	158	+	print("[*] You have the latest feeds\n")
	159	+	# print("-> ", latest_checksum, " ", local_checksum)
	160	+	return True
	161	+	else:
	162	+	# print("-> ", latest_checksum, " ", local_checksum)
	163	+	return False
	164	+
	165	+	except:
	166	+	return False
	167	+	return False
	168	+
126	169		def download(self):
127	170		"""
128	171		Download the latest newly registered domains and save locally.
		skipped 4 lines
133	176		Return:
134	177		none
135	178		"""
136		-	if self.period == "day":
137		-	self.URL_file = "domain-names.txt"
138		-	elif self.period == "week":
139		-	self.URL_file = "domain-names-week.txt"
140		-	elif self.period == "month":
141		-	self.URL_file = "domain-names-month.txt"
142		-
143	179		URL = self.URL + self.URL_file
144	180
145		-	print("[*] Downloading fresh domain list from", URL)
	181	+	print("[*] Downloading fresh domain list:", self.URL_file)
146	182
147	183		# User-Agent
148	184		ver = "openSquat-" + __VERSION__
		skipped 37 lines
186	222
187	223		return True
188	224
	225	+	def set_domain_filename(self, domain_filename):
	226	+	"""
	227	+	Method to set the domain filename.
	228	+
	229	+	Args:
	230	+	domain_filename
	231	+
	232	+	Returns:
	233	+	none
	234	+	"""
	235	+	if domain_filename == "":
	236	+	self.domain_filename = "domain-names.txt"
	237	+	else:
	238	+	self.domain_filename = domain_filename
	239	+
189	240		def set_filename(self, filename):
190	241		"""
191	242		Method to set the filename.
		skipped 17 lines
209	260		none
210	261		"""
211	262		self.period = search_period
	263	+
	264	+	if self.period == "day":
	265	+	self.URL_file = "domain-names.txt"
	266	+	elif self.period == "week":
	267	+	self.URL_file = "domain-names-week.txt"
	268	+	elif self.period == "month":
	269	+	self.URL_file = "domain-names-month.txt"
212	270
213	271		def set_dns_validation(self, dns):
214	272		"""
		skipped 313 lines
528	586		self.method = method
529	587
530	588		if self.domain_filename == "":
531		-	self.download()
	589	+	self.domain_filename = self.URL_file
	590	+	if not self.check_latest_feeds():
	591	+	self.download()
532	592
533	593		self.count_files()
534	594		self.read_files()
		skipped 4 lines

■ ■ ■ ■ ■ ■

opensquat/dns_resolvers.py

1 1 # -*- coding: utf-8 -*-
2 2 # Module: dns_resolvers.py
3 3 """
4 - openSquat
4 + openSquat.
5 5
6 6 (c) CERT-MZ
7 7
skipped 71 lines

All occurrences
opensquat.py | 100644 /~icons-ver-BEF942F0F42935333EFA072090F4E956.svg#arrow3 100755

Content is identical
■ ■ ■ ■ ■ ■

requirements.txt

skipped 11 lines
12 12 flake8
13 13 beautifulsoup4
14 14 dnspython==1.16.0
15 + hashlib

All occurrences

1	1		#This is a comment
2		-	google
3		-	facebook
4		-	amazon
	2	+	#google
	3	+	#facebook
	4	+	#amazon
5	5		paypal
6		-	microsoft
	6	+	#microsoft

1	-	print('\nRun file has moved to "opensquat.py" - please run this one!\n')
2	-	exit(0)
3	-

1	1		# -- coding: utf-8 --
2	2		# Module: __init__.py
3	3		"""openSquat Version."""
4		-	__VERSION__ = "1.95"
	4	+	__VERSION__ = "1.96"
5	5

1	1		# -- coding: utf-8 --
2	2		# Module: dns_resolvers.py
3	3		"""
4		-	openSquat
	4	+	openSquat.
5	5
6	6		(c) CERT-MZ
7	7
		skipped 71 lines

		skipped 11 lines
12	12		flake8
13	13		beautifulsoup4
14	14		dnspython==1.16.0
	15	+	hashlib

release 1.96