🤬
  • ■ ■ ■ ■ ■ ■
    CHANGES.txt
    1  -v2.2.51/58 21.10.2022 -- Building upon the major update, a few errors have been fixed,
    2  - installation has been improved and stablized, and several
    3  - new features have been added, including:
    4  - - Added "has media" column to archive files.
    5  - - Added post URL to archive file.
    6  - - Additional investigation resources added to the README.
    7  - - Maximise edgelist compatability with Gephi.
    8  - - Additional support for multiple location lookups at once.
     1 +v2.3.0/2 25.10.2022 -- Significant update, including major efficiency tweaks.
    9 2   
    10  -v2.2.50 16.10.2022 -- Major update! Code refactoring, reply retrieval,
    11  - translated descriptions, speedier decryption of media, support for
    12  - analyzing private groups, bug fixes and much more!
     3 + Retrieval of message reactions is here! Gain insights on the
     4 + engagement in a chat with new analytics and see how many times
     5 + a message has been reacted to, how much it's been forwarded, and
     6 + how many replies it has. Message edit date/time is also now available.
    13 7   
     8 + In addition to that, more changes have been made:
     9 + - Engagement rate calculation based on reactions, comments and forwards.
     10 + - Speed improvements as part of the ongoing refactoring (up to two messages archived per second).
     11 + - Posts are now not translated by default but with the "-tr" flag, speeding up retrieval time.
     12 + - Improved location save files based on feedback from OS2INT.
     13 + - Addition of up to four alternative logins.
     14 + - Telepathy can now look up users by either ID or by username.
     15 + - Information about when a user was last online is now part of the user lookup.
     16 + - Bug fixes.
    14 17   
    15  -v2.1.10 25.08.2022 -- Bugs busted, solved initialization error, efficiency tweaks,
    16  - added automatic translations, fixed issue with media archiving,
    17  - multiple targets now work with user and location flags.
    18  - Added a flag for exporting list of chats the account is a
    19  - participant of.
     18 +v2.2.51/62 21.10.2022 -- Building upon the major update, a few errors have been fixed,
     19 + installation has been improved and stablized, and several
     20 + new features have been added, including:
     21 + 
     22 + - Added "has media" column to archive files.
     23 + - Added post URL to archive file.
     24 + - Additional investigation resources added to the README.
     25 + - Maximise edgelist compatability with Gephi.
     26 + 
     27 +v2.2.50 16.10.2022 -- Major update! Code refactoring, reply retrieval,
     28 + translated descriptions, speedier decryption of media, support for
     29 + analyzing private groups, bug fixes and much more!
     30 + 
     31 + 
     32 +v2.1.10 25.08.2022 -- Bugs busted, solved initialization error, efficiency tweaks,
     33 + added automatic translations, fixed issue with media archiving,
     34 + multiple targets now work with user and location flags.
     35 + Added a flag for exporting list of chats the account is a
     36 + participant of.
    20 37   
    21  -v2.1.9, 16.08.2022 -- Added JSON export flag.
     38 +v2.1.9, 16.08.2022 -- Added JSON export flag.
    22 39   
    23  -v2.1.4/8, 09.08.2022 -- Added support for multiple clients. Bug fixes.
     40 +v2.1.4/8, 09.08.2022 -- Added support for multiple clients. Bug fixes.
    24 41   
    25  -v2.1.0/3, 08.08.2022 -- Added user lookup, location lookup, support for multiple
    26  - targets, log file, bug fixes, and improved encoding.
     42 +v2.1.0/3, 08.08.2022 -- Added user lookup, location lookup, support for multiple
     43 + targets, log file, bug fixes, and improved encoding.
    27 44   
    28  -v2.0.0, 28.07.2022 -- Major update! Everything is simpler, more organized
    29  - and complies with updates to Telegram's API. Automated
    30  - statistics are included, as well as a few more toys,
    31  - with more to come!
     45 +v2.0.0, 28.07.2022 -- Major update! Everything is simpler, more organized
     46 + and complies with updates to Telegram's API. Automated
     47 + statistics are included, as well as a few more toys,
     48 + with more to come!
    32 49   
    33  -v1.1.15, 25.04.2022 -- Fixed locations module, added phone number retrieval to
    34  - the participants module, improved documentation.
     50 +v1.1.15, 25.04.2022 -- Fixed locations module, added phone number retrieval to
     51 + the participants module, improved documentation.
    35 52   
    36  -v1.1.13/14, 21.04.2022 -- Bug fixes.
     53 +v1.1.13/14, 21.04.2022-- Bug fixes.
    37 54   
    38  -v1.1.12, 21.04.2022 -- Fixed file management, updated advanced tools.
     55 +v1.1.12, 21.04.2022 -- Fixed file management, updated advanced tools.
    39 56   
    40  -v1.1.4/11, 16.04.2022 -- Bug fixes.
     57 +v1.1.4/11, 16.04.2022 -- Bug fixes.
    41 58   
    42  -v1.1.3, 16.04.2022 -- Added support for multiple -n flags.
     59 +v1.1.3, 16.04.2022 -- Added support for multiple -n flags.
    43 60   
    44  -v1.1.2, 15.04.2022 -- Bug fixes.
     61 +v1.1.2, 15.04.2022 -- Bug fixes.
    45 62   
    46  -v1.1.1, 15.04.2022 -- Updated directory handling.
     63 +v1.1.1, 15.04.2022 -- Updated directory handling.
    47 64   
    48  -v1.1.0, 15.04.2022 -- Initial release.
     65 +v1.1.0, 15.04.2022 -- Initial release.
    49 66   
  • ■ ■ ■ ■ ■ ■
    README.md
     1 +Telepathy: An OSINT toolkit for investigating Telegram chats. Developed by Jordan Wildon. Version 2.3.2.
    1 2   
     3 +Telepathy has been described as the "swiss army knife of Telegram tools," allowing OSINT analysts, researchers and digital investigators to archive Telegram chats (including replies, media content, comments and reactions), gather memberlists, lookup users by given location, analyze top posters in a chat, map forwarded messages, and more.
    2 4   
    3  -Telepathy: An OSINT toolkit for investigating Telegram chats. Developed by Jordan Wildon. Version 2.2.58.
     5 +The toolkit has already seen a wide variety of use cases, including but not limited to: in investigative and data journalism, by academic and research institutions, and for intelligence gathering and analysis.
     6 + 
     7 + 
     8 +## !! IMPORTANT:
     9 +With the update to 2.3.0, you will need to delete your login.txt file to prevent errors if using the alternative login feature. Upon first use, Telepathy will guide you through setup of the details once again. To work around this, instead of deleting and recreating the file, you can add a newline character to the end of your current API details to ensure Telepathy scans the file correctly.
     10 + 
     11 +A note on unique identifiers per account: You will notice that depending on which alternative account you use, the access hash will vary. The same will happen with User IDs, which are unique to each Telegram account accessing them. For deeper data analysis based on user IDs, this is important to bare in mind as users will have as many unique IDs as accounts you've used to access information. In future, Telepathy may include a feature to assign unique identifier per account found based on a hash of the available information, regardless of which account accessed the data.
    4 12   
    5 13   
     14 + 
    6 15  ## Installation
    7 16   
    8 17  ### Pip install (recommended)
    skipped 26 lines
    35 44  Options:
    36 45  - **'--target', '-t' [CHAT]**
    37 46   
    38  -this option will identify the target of the scan. The specified chat must be public. To get the chat name, look for the 't.me/chatname' link, and subtract the 't.me/'.
     47 +this option will identify the target of the scan. The specified chat must be public or have a private link. To get the chat name, look for the 't.me/chatname' link, and subtract the 't.me/'.
    39 48   
    40 49  For example:
    41 50   
    skipped 6 lines
    48 57   
    49 58  - **'--comprehensive', '-c'**
    50 59   
    51  -A comprehensive scan will offer the same information as the basic scan, but will also archive a chat's message history.
     60 +A comprehensive scan will offer the same information as the basic scan, but will also archive a chat's message history, gather the number of reactions, archive how many times a message has been forwarded, the number of replies to each message, and more.
     61 + 
     62 +Reaction lists are included in the archive file, including basic calculations of engagement rate. Only the most-common reactions are listed, with the total including all possible reactions. Currently, Telepathy calculates engagement rates based on forwards, comments and reactions seperately, with a calculation based on post views and one based on chat participant count. In future, Telepathy may include deeper analytics which can be cross-compared between chats based on a combination of these metrics, fixing for when comments, reactions or forwards are allowed or disallowed in a given chat.
    52 63   
    53 64  For example:
    54 65   
    skipped 4 lines
    59 70   
    60 71  - **'--forwards', '-f'**
    61 72   
    62  -This flag will create an edgelist based on messages forwarded into a chat. It can be used alongside either a default or comprehensive scan.
     73 +This flag will create an edgelist based on messages forwarded into a chat. It can be used alongside either a default or comprehensive scan. Since 2.3.0, Telepathy now formats these edgelists to maximize compatability with Gephi.
    63 74   
    64 75  For example:
    65 76   
    66 77  ```
    67 78  $ telepathy -t durov -f
     79 + 
     80 +$ telepathy -t durov -c -f
    68 81  ```
    69 82   
    70 83   
    skipped 1 lines
    72 85   
    73 86  Use this flag to include media archiving alongside a comprehensive scan. This makes the process take significantly longer and should also be used with caution: you'll download all media content from the target chat, and it's up to you to not store illegal files on your system.
    74 87   
    75  -Since 2.2.0, downloading all media files will also generate a CSV file listing the files' metadata.
    76  - 
    77  -For example, this will run a comprehensive scan, including media archiving:
     88 +To archive media, you must run a comprehensive scan:
    78 89   
    79 90  ```
    80 91  $ telepathy -t durov -c -m
    81 92  ```
    82 93   
     94 +Once files have downloaded, you can run exiftool on the associated media directory to gather deeper insights on the files, their metadata, and in some cases attribute who might be behind an anonymous channel. Further details are in the "bonus investigations tips" section of this README.
    83 95   
    84  -- **'--user', '-u' [USER]**
    85 96   
    86  -Looks up a specified user ID. This will only work if your account has "encountered" the user before (for example, after archiving a group), you can specify User ID or @nickname
     97 +- **'--user', '-u'**
     98 + 
     99 +Looks up a specified user. This will only work if your account has "encountered" the user before (for example, after archiving a group), you can specify User ID or @nickname. If looking up by username, it's not always necessary for your account to have already seen the user.
    87 100   
    88 101  ```
    89 102  $ telepathy -t 0123456789 -u
    skipped 2 lines
    92 105  ```
    93 106   
    94 107   
    95  -- **'--location', '-l' [COORDINATES]**
     108 +- **'--location', '-l']**
    96 109   
    97  -Finds users near to specified coordinates. Input should be longitude followed by latitude, seperated by a comma. This feature only works if your Telegram account has a profile image which is set to publicly viewable.
     110 +Finds users near to specified coordinates. Input should be longitude followed by latitude, seperated by a comma. This feature only works if your Telegram account has a profile image which is set to be publicly viewable.
     111 + 
     112 +While searches for multiple locations at once may work in some cases, Telegram appears to have a limit on how quickly an account can cycle through locations. At the time of writing, this appears to be at least ten minutes. Further location scanning support while using multiple accounts is being explored for a future release.
    98 113   
    99 114  ```
    100 115  $ telepathy -t 51.5032973,-0.1217424 -l
    101 116  ```
    102 117   
    103 118   
    104  -- **'--alt', '-a'**
     119 +- **'--alt', '-a' [NUMBER]**
    105 120   
    106  -Flag for running Telepathy from an alternative number. You can use the same API key and Hash but authenticate with a different phone number. Allows for running multiple scans at the same time.
     121 +Flag for running Telepathy from an alternative number or API details. You can use the same API key and Hash but authenticate with a different phone number. This allows for running multiple scans at the same time. Telepathy will default to the first details you offer, and up to four others can be added. Please see the notes at the top of this README for information regarding limitations with user IDs using this method.
    107 122   
    108 123  ```
    109  -$ telepathy -t Durov -c -a
     124 +$ telepathy -t Durov -c -a 1
    110 125  ```
    111 126   
    112 127   
    113 128  - **'--export', '-e'**
    114 129   
    115  -Exports all chats your account is part of to a CSV file. In a future release, this may assist with setting up multiple accounts following the same groups.
     130 +Exports all chats your account is part of to a CSV file. In a future release, this may assist with provisioning new accounts to automatically following the listed groups.
    116 131   
    117 132  ```
    118 133  $ telepathy -e
    skipped 2 lines
    121 136   
    122 137  - **'--reply', '-r'**
    123 138   
    124  -Flag for enable the reply in the channel, it will map users who replied in the channel and it will dump the full conversation chain
     139 +Flag for enabling channel reply retrieval, this will archive replies and list users who replied to messages in the target channel.
    125 140   
    126 141  ```
    127 142  $ telepathy -t [CHANNEL] -c -r
    128 143  ```
    129 144   
    130 145   
     146 +- **'--translate', '-tr'**
     147 + 
     148 +Flag for enabling auotmatic translation (currently only into English) during message retrieval.
     149 + 
     150 +```
     151 +$ telepathy -t [CHANNEL] -c -tr
     152 +```
     153 + 
     154 + 
    131 155  ## Bonus investigations tips:
    132 156   
    133 157   - Navigating to a media archive directory and running Exiftool may give you a whole host of useful information for further investigation. Telegram doesn't currently scrub metadata from PDF, DOCX, XLSX, MP4, MOV and some other filetypes, which offer creation and edit time metadata, often timezones, sometimes authors, and general technical information about the perosn or people who created a media file.
    skipped 1 lines
    135 159  $ cd ./telepathy/telepathy_files/CHATNAME/media
    136 160  $ exiftool * > metadata.txt
    137 161  ```
    138  - - Group and inferred channel memberlists offer a point of further investigation for usernames found. By using Maigret, you can look up where else a username has been used. While this is not accurate in all cases, it's been proven to be useful for handles that are often reused. In this case, remember to verify your findings to avoid false positives.
     162 + - Group and inferred channel memberlists offer a point of further investigation for usernames found. By using [Maigret](https://github.com/soxoj/maigret), you can look up where else a username has been used online. While this is not accurate in all cases, it's been proven to be helpful for identifying where a person has reused handles across platforms. In this case, remember to verify your findings to avoid false positives.
    139 163   
    140 164   
    141 165  ## A note on how Telegram works
    142 166   
    143  -Telegram chats are organised into three key types: Channels, Megagroups/Supergroups and Gigagroups. Each module works slightly differently depending on the chat type. Channels can have seemingly unlimited subscribers and are where an admin will broadcast messages to an audience, Megagroups can have up to 200,000 members, each of whom can participate (if not restricted), and Gigagroups sit somewhere between the two.
     167 +Telegram chats are organised into three key types: Channels, Megagroups/Supergroups and Gigagroups. Each option works slightly differently depending on the chat type. Channels can have seemingly unlimited subscribers and are where an admin will broadcast messages to an audience, Megagroups can have up to 200,000 members, each of whom can participate (if not restricted), and Gigagroups sit somewhere between the two.
    144 168   
    145 169   
    146 170  ## Upcoming changes
    skipped 2 lines
    149 173  Upcoming features include:
    150 174   
    151 175   - [ ] Adding a time specification flag to set archiving for specific period.
    152  - - [ ] The ability to gather the number of reactions to messages, including statistics on engagement rate.
     176 + - [x] The ability to gather the number of reactions to messages, including statistics on engagement rate.
    153 177   - [ ] Finding a method to once again gather complete memberlists (currently restricted by the API).
    154 178   - [ ] Improved statistics: including timestamp analysis for channels.
    155 179   - [ ] Generating an entirely automated complete report, including visualisation for some statistics.
    156 180   - [ ] Hate speech analytics.
    157  - - [x] Maximise compatibility of edgelists with Gephi.
    158 181   - [ ] Include sockpuppet account provisioning (creation of accounts from previous exported lists).
    159 182   - [ ] Listing who has group admin rights in memberlists.
    160  - - [ ] Media downloaded in the background to increase efficiency.
    161  - - [ ] When media archiving is flagged, the location of downloaded content will be added to the archive file.
    162  - - [ ] Exploring, and potentially integrating, media cross checks based on https://github.com/conflict-investigations/media-search-engine.
     183 + - [ ] Media downloaded in the background to increase efficiency or progress bars for media downloads to give a better estimation of runtime.
     184 + - [x] When media archiving is flagged, the location of downloaded content will be added to the archive file.
     185 + - [ ] Exploring, and potentially integrating, media cross-checks based on https://github.com/conflict-investigations/media-search-engine.
    163 186   - [ ] Ensuring inferred channel memberlists don't contain duplicate entries.
    164 187   - [ ] Introducing local chat retrival within the location lookup module.
    165  - - [ ] Adding trilateration option for location lookup to aid better location matching.
    166  - - [ ] Further code refactoring to ensure long-term maintainability.
    167  - - [ ] Progress bars for media downloads to give a better estimation of runtime.
    168  - - [ ] Adding additional alternative logins.
     188 + - [x] Further code refactoring to ensure long-term maintainability.
     189 + - [x] Adding additional alternative logins.
    169 190   - [ ] Improved language support.
    170  - - [ ] Ensure inferred channel memberlists (based on repliers) contains each account only once.
    171 191   - [ ] Correctly define destinction between reply (as in a chat) and comment (as in channel).
    172  - 
     192 + - [ ] Exploration of whether channel events can be included, such as name changes.
     193 + - [x] Including last seen on user lookup.
    173 194   
    174 195  ## feedback
    175 196   
    skipped 14 lines
  • ■ ■ ■ ■
    build/lib/telepathy/const.py
    1 1  __author__ = "Jordan Wildon (@jordanwildon)"
    2 2  __license__ = "MIT License"
    3  -__version__ = "2.2.58"
     3 +__version__ = "2.3.2"
    4 4  __maintainer__ = "Jordan Wildon"
    5 5  __email__ = "[email protected]"
    6 6  __status__ = "Development"
    7 7   
    8 8  user_agent = [
     9 + 
    9 10   # Chrome
    10 11   "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36",
    11 12   "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36",
    skipped 5 lines
    17 18   "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36",
    18 19   "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36",
    19 20   "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36",
     21 + 
    20 22   # Firefox
    21 23   "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1)",
    22 24   "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko",
    skipped 8 lines
    31 33   "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)",
    32 34   "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)",
    33 35   "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
     36 + 
    34 37   # Safari
    35 38   "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) RockMelt/0.9.50.549 Chrome/10.0.648.205 Safari/534.16"
    36 39   "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18"
    skipped 21 lines
  • ■ ■ ■ ■ ■ ■
    build/lib/telepathy/telepathy.py
    1 1  #!/usr/bin/python3
    2 2   
    3  - 
    4 3  """Telepathy cli interface:
    5 4   An OSINT toolkit for investigating Telegram chats.
    6 5  """
    7 6   
    8  -from tokenize import group
    9 7  import pandas as pd
    10 8  import datetime
    11  -import requests
    12  -import json
    13  -import random
    14  -import glob
    15  -import csv
    16 9  import os
    17 10  import getpass
    18 11  import click
    19 12  import re
    20  -import textwrap
    21 13  import time
    22  -import pprint
    23 14   
    24 15  from telepathy.utils import (
    25 16   print_banner,
    skipped 2 lines
    28 19   process_message,
    29 20   process_description,
    30 21   parse_tg_date,
    31  - parse_html_page
     22 + parse_html_page,
     23 + print_shell,
     24 + createPlaceholdeCls
    32 25  )
    33  -import telepathy.const as const
    34  - 
    35  -from colorama import Fore, Back, Style
    36 26   
    37 27  from telethon.errors import SessionPasswordNeededError, ChannelPrivateError
    38 28  from telethon.tl.types import (
    39 29   InputPeerEmpty,
    40 30   PeerUser,
     31 + User,
    41 32   PeerChat,
    42 33   PeerChannel,
    43 34   PeerLocated,
    skipped 4 lines
    48 39  from telethon import TelegramClient, functions, types, utils
    49 40  from telethon.utils import get_display_name, get_message_id
    50 41  from alive_progress import alive_bar
    51  -from bs4 import BeautifulSoup
    52  - 
     42 +from colorama import Fore, Style
    53 43   
    54 44  @click.command()
    55 45  @click.option(
    56 46   "--target",
    57 47   "-t",
    58 48   #default="",
    59  - multiple=True,
    60  - help="Specifies a chat to investigate.",
    61  -)
     49 + multiple = True,
     50 + help = "Specifies a chat to investigate.",
     51 + )
    62 52  @click.option(
    63 53   "--comprehensive",
    64 54   "-c",
    65  - is_flag=True,
    66  - help="Comprehensive scan, includes archiving.",
    67  -)
     55 + is_flag = True,
     56 + help = "Comprehensive scan, includes archiving.",
     57 + )
    68 58  @click.option(
    69  - "--media", "-m", is_flag=True, help="Archives media in the specified chat."
    70  -)
    71  -@click.option("--forwards", "-f", is_flag=True, help="Scrapes forwarded messages.")
    72  -@click.option("--user", "-u", is_flag=True, help="Looks up a specified user ID.")
     59 + "--media",
     60 + "-m",
     61 + is_flag = True,
     62 + help = "Archives media in the specified chat."
     63 + )
    73 64  @click.option(
    74  - "--location", "-l", is_flag=True, help="Finds users near to specified coordinates."
    75  -)
     65 + "--forwards",
     66 + "-f",
     67 + is_flag = True,
     68 + help = "Scrapes forwarded messages."
     69 + )
     70 +@click.option(
     71 + "--user",
     72 + "-u",
     73 + is_flag = True,
     74 + help = "Looks up a specified user ID."
     75 + )
     76 +@click.option(
     77 + "--location",
     78 + "-l",
     79 + is_flag = True,
     80 + help = "Finds users near to specified coordinates."
     81 + )
     82 +@click.option(
     83 + "--alt",
     84 + "-a",
     85 + default = 0,
     86 + help = "Uses an alternative login."
     87 + )
    76 88  @click.option(
    77  - "--alt", "-a", is_flag=True, default=False, help="Uses an alternative login."
    78  -)
    79  -@click.option("--json", "-j", is_flag=True, default=False, help="Export to JSON.")
     89 + "--json",
     90 + "-j",
     91 + is_flag = True,
     92 + default = False,
     93 + help = "Export to JSON."
     94 + )
    80 95  @click.option(
    81 96   "--export",
    82 97   "-e",
    83  - is_flag=True,
    84  - default=False,
    85  - help="Export a list of chats your account is part of.",
    86  -)
     98 + is_flag = True,
     99 + default = False,
     100 + help = "Export a list of chats your account is part of.",
     101 + )
    87 102  @click.option(
    88 103   "--replies",
    89 104   "-r",
    90  - is_flag=True,
    91  - default=False,
    92  - help="Enable replies analysis in channels.",
    93  -)
     105 + is_flag = True,
     106 + default = False,
     107 + help = "Enable replies analysis in channels.",
     108 + )
     109 +@click.option(
     110 + "--translate",
     111 + "-tr",
     112 + is_flag = True,
     113 + default = False,
     114 + help = "Enable translation of chat content.",
     115 + )
     116 + 
    94 117  def cli(
    95  - target, comprehensive, media, forwards, user, location, alt, json, export, replies
    96  -):
     118 + target,
     119 + comprehensive,
     120 + media,
     121 + forwards,
     122 + user,
     123 + location,
     124 + alt,
     125 + json,
     126 + export,
     127 + replies,
     128 + translate
     129 + ):
     130 + 
    97 131   print_banner()
    98  - telepathy_file = "./telepathy_files/"
    99  - try:
    100  - os.makedirs(telepathy_file)
    101  - except FileExistsError:
    102  - pass
    103 132   
    104 133   # Defining default values
    105  - basic = False
    106  - comp_check = False
    107  - media_archive = False
    108  - forwards_check = False
    109  - forward_verify = False
    110  - reply_analysis = False
    111  - user_check = False
    112  - location_check = False
    113  - last_date = None
    114  - chunk_size = 1000
    115  - filetime = datetime.datetime.now().strftime("%Y_%m_%d-%H_%M")
    116  - filetime_clean = str(filetime)
     134 + user_check = location_check = False
     135 + basic = True if target else False
     136 + reply_analysis = True if replies else False
     137 + forwards_check = True if forwards else False
     138 + comp_check = True if comprehensive else False
     139 + media_archive = True if media else False
     140 + json_check = True if json else False
     141 + translate_check = True if translate else False
     142 + last_date, chunk_size, user_language = None, 1000, 'en'
    117 143   
    118  - # Will add more languages later
    119  - user_language = "en"
    120  - 
    121  - if target:
    122  - basic = True
    123  - if replies:
    124  - reply_analysis = True
    125  - if forwards:
    126  - forwards_check = True
    127 144   if user:
    128  - user_check = True
    129  - basic = False
     145 + user_check, basic = True, False
    130 146   if location:
    131  - location_check = True
    132  - basic = False
    133  - if comprehensive:
    134  - comp_check = True
    135  - if media:
    136  - media_archive = True
     147 + location_check, basic = True, False
    137 148   if export:
    138 149   t = " "
    139  - if alt:
    140  - alt_check = True
    141  - else:
    142  - alt_check = False
     150 + 
     151 + filetime = datetime.datetime.now().strftime("%Y_%m_%d-%H_%M")
     152 + filetime_clean = str(filetime)
     153 + 
     154 + # Defining file values
     155 + telepathy_file = "./telepathy_files/"
     156 + json_file = telepathy_file + "json_files/"
     157 + login = telepathy_file + "login.txt"
     158 + log_file = telepathy_file + "log.csv"
     159 + export_file = telepathy_file + "export.csv"
     160 + 
     161 + # Creating core data file
     162 + if not os.path.exists(telepathy_file):
     163 + os.makedirs(telepathy_file)
    143 164  
    144  - if json:
    145  - json_check = True
    146  - json_file = telepathy_file + "json_files/"
    147  - try:
    148  - os.makedirs(json_file)
    149  - except FileExistsError:
    150  - pass
    151  - else:
    152  - json_check = False
     165 + '''Start of API details'''
    153 166  
    154  - if alt_check == True:
    155  - login = telepathy_file + "login_alt.txt"
     167 + def login_function():
     168 + api_id = input("Please enter your API ID:\n")
     169 + api_hash = input("Please enter your API Hash:\n")
     170 + phone_number = input("Please enter your phone number:\n")
     171 + return api_id, api_hash, phone_number
    156 172   
    157  - if os.path.isfile(login) == False:
    158  - api_id = input(" Please enter your API ID:\n")
    159  - api_hash = input(" Please enter your API Hash:\n")
    160  - phone_number = input(" Please enter your phone number:\n")
    161  - with open(login, "w+", encoding="utf-8") as f:
    162  - f.write(api_id + "," + api_hash + "," + phone_number)
    163  - else:
    164  - with open(login, encoding="utf-8") as f:
    165  - details = f.read()
     173 + if os.path.isfile(login) == False:
     174 + api_id, api_hash, phone_number = login_function()
     175 + with open(login, "w+", encoding="utf-8") as f:
     176 + f.write(api_id + "," + api_hash + "," + phone_number + "\n")
     177 + else:
     178 + with open(login, encoding="utf-8") as file:
     179 + content = file.readlines()
     180 + if alt == 0:
     181 + details = content[0]
    166 182   api_id, api_hash, phone_number = details.split(sep=",")
    167  - else:
    168  - login = telepathy_file + "login.txt"
     183 + elif alt == 1:
     184 + try:
     185 + if content[1]:
     186 + details = content[1]
     187 + api_id, api_hash, phone_number = details.split(sep=",")
     188 + except:
     189 + print("Setting up alt 1: ")
     190 + api_id, api_hash, phone_number = login_function()
     191 + with open(login, "a+", encoding="utf-8") as file:
     192 + file.write(api_id + "," + api_hash + "," + phone_number + "\n")
     193 + elif alt == 2:
     194 + try:
     195 + if content[2]:
     196 + details = content[2]
     197 + api_id, api_hash, phone_number = details.split(sep=",")
     198 + except:
     199 + print("Setting up alt 2: ")
     200 + api_id, api_hash, phone_number = login_function()
     201 + with open(login, "a+", encoding="utf-8") as file:
     202 + file.write(api_id + "," + api_hash + "," + phone_number + "\n")
     203 + elif alt == 3:
     204 + try:
     205 + if content[3]:
     206 + details = content[3]
     207 + api_id, api_hash, phone_number = details.split(sep=",")
     208 + except:
     209 + print("Setting up alt 3: ")
     210 + api_id, api_hash, phone_number = login_function()
     211 + with open(login, "a+", encoding="utf-8") as file:
     212 + file.write(api_id + "," + api_hash + "," + phone_number + "\n")
     213 + elif alt == 4:
     214 + try:
     215 + if content[4]:
     216 + details = content[4]
     217 + api_id, api_hash, phone_number = details.split(sep=",")
     218 + except:
     219 + print("Setting up alt 4: ")
     220 + api_id, api_hash, phone_number = login_function()
     221 + with open(login, "a+", encoding="utf-8") as file:
     222 + file.write(api_id + "," + api_hash + "," + phone_number + "\n")
    169 223   
    170  - if os.path.isfile(login) == False:
    171  - api_id = input(" Please enter your API ID:\n")
    172  - api_hash = input(" Please enter your API Hash:\n")
    173  - phone_number = input(" Please enter your phone number:\n")
    174  - with open(login, "w+", encoding="utf-8") as f:
    175  - f.write(api_id + "," + api_hash + "," + phone_number)
    176  - else:
    177  - with open(login, encoding="utf-8") as f:
    178  - details = f.read()
    179  - api_id, api_hash, phone_number = details.split(sep=",")
     224 + '''End of API details'''
    180 225   
    181 226   client = TelegramClient(phone_number, api_id, api_hash)
    182 227   
    183 228   async def main():
    184 229   
    185 230   await client.connect()
     231 + 
    186 232   if not await client.is_user_authorized():
    187 233   await client.send_code_request(phone_number)
    188  - await client.sign_in(phone_number)
    189 234   try:
    190  - await client.sign_in(code=input(" Enter code: "))
     235 + await client.sign_in(
     236 + phone = phone_number,
     237 + code=input("Enter code: "),
     238 + )
    191 239   except SessionPasswordNeededError:
    192 240   await client.sign_in(
    193  - password=getpass.getpass(prompt="Password: ", stream=None)
    194  - )
     241 + password=getpass.getpass(
     242 + prompt="Password: ",
     243 + stream=None,
     244 + )
     245 + )
     246 + 
    195 247   result = client(
    196 248   GetDialogsRequest(
    197 249   offset_date=last_date,
    skipped 1 lines
    199 251   offset_peer=InputPeerEmpty(),
    200 252   limit=chunk_size,
    201 253   hash=0,
     254 + )
    202 255   )
    203  - )
    204  - else:
    205 256   
     257 + else:
    206 258   if export == True:
    207  - export_file = telepathy_file + "export.csv"
    208 259   exports = []
    209  - 
    210 260   print("Exporting...")
    211 261   
    212  - # progress bar
    213  - 
    214 262   for Dialog in await client.get_dialogs():
    215 263   try:
    216 264   if Dialog.entity.username:
    skipped 4 lines
    221 269   group_description = web_req["group_description"]
    222 270   total_participants = web_req["total_participants"]
    223 271   
    224  - _desc = process_description(
    225  - group_description, user_language
    226  - )
    227  -
    228  - original_language = _desc[
    229  - "original_language"
    230  - ]
    231  - translated_description = _desc["translated_text"]
     272 + if translate_check == True:
     273 + _desc = process_description(
     274 + group_description, user_language
     275 + )
     276 + translated_description = _desc["translated_text"]
     277 + else:
     278 + translated_description = "N/A"
    232 279   
    233 280   if Dialog.entity.broadcast is True:
    234 281   chat_type = "Channel"
    skipped 56 lines
    291 338   )
    292 339   
    293 340   if not os.path.isfile(export_file):
    294  - export_df.to_csv(export_file, sep=";", index=False)
    295  - else:
    296 341   export_df.to_csv(
    297  - export_file, sep=";", mode="w", index=False
     342 + export_file,
     343 + sep=";",
     344 + index=False,
    298 345   )
     346 + else:
     347 + export_df.to_csv(
     348 + export_file,
     349 + sep=";",
     350 + mode="w",
     351 + index=False,
     352 + )
    299 353   
    300 354   except AttributeError:
    301 355   pass
    302 356   
    303 357   else:
    304  - 
    305 358   for t in target:
    306  - target_clean = t
    307 359   alphanumeric = ""
    308  -
    309  - 
    310  - for character in target_clean:
     360 + for character in t:
    311 361   if character.isalnum():
    312 362   alphanumeric += character
    313 363   
    314 364   if "https://t.me/+" in t:
    315 365   t = t.replace('https://t.me/+', 'https://t.me/joinchat/')
    316 366   
    317  - if basic is True or comp_check is True:
     367 + if basic == True or comp_check == True:
    318 368   save_directory = telepathy_file + alphanumeric
    319  - try:
     369 + if not os.path.exists(save_directory):
    320 370   os.makedirs(save_directory)
    321  - except FileExistsError:
    322  - pass
    323  - 
    324  - # Creating logfile
    325  - log_file = telepathy_file + "log.csv"
    326 371   
    327 372   if media_archive:
    328 373   media_directory = save_directory + "/media"
    329  - try:
     374 + if not os.path.exists(media_directory):
    330 375   os.makedirs(media_directory)
    331  - except FileExistsError:
    332  - pass
    333 376   
    334 377   if basic == True and comp_check == False:
    335 378   color_print_green(" [!] ", "Performing basic scan")
    336 379   elif comp_check == True:
    337 380   color_print_green(" [!] ", "Performing comprehensive scan")
     381 + 
    338 382   file_archive = (
    339 383   save_directory
    340 384   + "/"
    skipped 2 lines
    343 387   + filetime_clean
    344 388   + "_archive.csv"
    345 389   )
     390 + 
    346 391   reply_file_archive = (
    347 392   save_directory
    348 393   + "/"
    skipped 13 lines
    362 407   + filetime_clean
    363 408   + "_edgelist.csv"
    364 409   )
     410 + 
    365 411   forward_directory = save_directory + "/edgelists/"
    366  - 
    367  - try:
     412 + if not os.path.exists(forward_directory):
    368 413   os.makedirs(forward_directory)
    369  - except FileExistsError:
    370  - pass
    371 414   
    372 415   edgelist_file = (
    373  - forward_directory + "/" + alphanumeric + "_edgelist.csv"
     416 + forward_directory
     417 + + "/"
     418 + + alphanumeric
     419 + + "_edgelist.csv"
    374 420   )
    375 421   
    376 422   if basic is True or comp_check is True:
    377 423   
    378 424   color_print_green(" [-] ", "Fetching details for " + t + "...")
     425 + 
    379 426   memberlist_directory = save_directory + "/memberlists"
    380  - 
    381  - try:
     427 + if not os.path.exists(memberlist_directory):
    382 428   os.makedirs(memberlist_directory)
    383  - except FileExistsError:
    384  - pass
    385 429   
    386 430   memberlist_filename = (
    387  - memberlist_directory + "/" + alphanumeric + "_members.csv"
     431 + memberlist_directory
     432 + + "/"
     433 + + alphanumeric
     434 + + "_members.csv"
    388 435   )
     436 + 
    389 437   reply_memberlist_filename = (
    390 438   memberlist_directory
    391 439   + "/"
    skipped 2 lines
    394 442   )
    395 443   
    396 444   entity = await client.get_entity(t)
     445 +
    397 446   first_post = "Not found"
    398 447   
    399 448   async for message in client.iter_messages(t, reverse=True):
    skipped 4 lines
    404 453   break
    405 454   
    406 455   if entity.username:
    407  - name = entity.title
    408 456   group_url = "http://t.me/" + entity.username
    409 457   group_username = entity.username
    410 458   web_req = parse_html_page(group_url)
    skipped 2 lines
    413 461   web_req = parse_html_page(group_url)
    414 462   group_username = "Private group"
    415 463   else:
    416  - group_url = "Private group"
    417  - group_username = "Private group"
    418  - 
     464 + group_url, group_username = "Private group", "Private group"
    419 465   
    420 466   group_description = web_req["group_description"]
    421 467   total_participants = web_req["total_participants"]
    422 468   
    423  - _desc = process_description(
    424  - group_description, user_language
     469 + if translate_check == True:
     470 + _desc = process_description(
     471 + group_description, user_language
    425 472   )
    426 473   
    427  - original_language = _desc[
    428  - "original_language"
    429  - ]
     474 + original_language = _desc[
     475 + "original_language"
     476 + ]
     477 + translated_description = _desc["translated_text"]
     478 + else:
     479 + translated_description = "N/A"
    430 480   
    431  - translated_description = _desc["translated_text"]
     481 + group_description = ('"' + group_description + '"')
    432 482   
    433  - preferredWidth = 70
    434  - descript = Fore.GREEN + "Description: " + Style.RESET_ALL
    435  - prefix = descript
    436  - wrapper_d = textwrap.TextWrapper(
    437  - initial_indent=prefix,
    438  - width=preferredWidth,
    439  - subsequent_indent=" ",
    440  - )
    441  - 
    442  - trans_descript = Fore.GREEN + "Translated: " + Style.RESET_ALL
    443  - prefix = trans_descript
    444  - wrapper_td = textwrap.TextWrapper(
    445  - initial_indent=prefix,
    446  - width=preferredWidth,
    447  - subsequent_indent=" ",
    448  - )
    449  - 
    450  - group_description = ('"' + group_description + '"')
     483 + if(entity.__class__ == User):
     484 + color_print_green(" [!] ", "You can't search for users using flag -c, run Telepathy using the flag -u.")
     485 + exit(1)
    451 486   
    452 487   if entity.broadcast is True:
    453 488   chat_type = "Channel"
    skipped 9 lines
    463 498   if 1 in entity.restriction_reason:
    464 499   android_restriction = entity.restriction_reason[1]
    465 500   group_status = (
    466  - str(ios_restriction) + ", " + str(android_restriction)
     501 + str(ios_restriction)
     502 + + ", "
     503 + + str(android_restriction)
    467 504   )
    468 505   else:
    469 506   group_status = str(ios_restriction)
    470 507   else:
    471 508   group_status = "None"
    472 509   
    473  - restrict = Fore.GREEN + "Restrictions:" + Style.RESET_ALL
    474  - prefix = restrict + " "
    475  - preferredWidth = 70
    476  - wrapper_r = textwrap.TextWrapper(
    477  - initial_indent=prefix,
    478  - width=preferredWidth,
    479  - subsequent_indent=" ",
    480  - )
     510 + found_participants, found_percentage = 0, 0
    481 511   
    482 512   if chat_type != "Channel":
    483 513   members = []
    484  - all_participants = []
     514 + members_df = None
    485 515   all_participants = await client.get_participants(t, limit=5000)
    486 516   
    487  - members_df = None
    488 517   for user in all_participants:
    489 518   members_df = pd.DataFrame(
    490 519   members,
    skipped 14 lines
    505 534   members_df.to_csv(save_members, sep=";")
    506 535   
    507 536   if json_check == True:
     537 + if not os.path.exists(json_file):
     538 + os.makedirs(json_file)
     539 + 
    508 540   members_df.to_json(
    509 541   json_file + alphanumeric + "_memberlist.json",
    510 542   orient="records",
    skipped 1 lines
    512 544   lines=True,
    513 545   index=True,
    514 546   )
    515  - else:
    516  - pass
    517 547   
    518 548   found_participants = len(all_participants)
    519 549   found_participants = int(found_participants)
    skipped 6 lines
    526 556   if chat_type != "Channel":
    527 557   print("\n")
    528 558   color_print_green(" [+] Memberlist fetched", "")
    529  - else:
    530  - pass
    531 559  
    532  - color_print_green(" ┬ Chat details", "")
    533  - color_print_green(" ├ Title: ", str(entity.title))
    534  - color_print_green(" ├ ", wrapper_d.fill(group_description))
    535  - if translated_description != group_description:
    536  - color_print_green(" ├ ", wrapper_td.fill(translated_description))
    537  - color_print_green(
    538  - " ├ Total participants: ", str(total_participants)
    539  - )
     560 + setattr(entity, "group_description", group_description)
     561 + setattr(entity, "group_status", group_status)
     562 + setattr(entity, "group_username", group_username)
     563 + setattr(entity, "first_post", first_post)
     564 + setattr(entity, "group_url", group_url)
     565 + setattr(entity, "chat_type", chat_type)
     566 + setattr(entity, "translated_description", translated_description)
     567 + setattr(entity, "total_participants", total_participants)
    540 568   
    541 569   if chat_type != "Channel":
    542  - color_print_green(
    543  - " ├ Participants found: ",
    544  - str(found_participants)
    545  - + " ("
    546  - + str(format(found_percentage, ".2f"))
    547  - + "%)",
    548  - )
     570 + setattr(entity, "found_participants", found_participants)
     571 + setattr(entity, "found_percentage", found_percentage)
     572 + setattr(entity, "memberlist_filename", memberlist_filename)
    549 573   else:
    550  - found_participants = "N/A"
    551  - 
    552  - color_print_green(" ├ Username: ", str(group_username))
    553  - color_print_green(" ├ URL: ", str(group_url))
    554  - color_print_green(" ├ Chat type: ", str(chat_type))
    555  - color_print_green(" ├ Chat id: ", str(entity.id))
    556  - color_print_green(" ├ Access hash: ", str(entity.access_hash))
     574 + setattr(entity, "found_participants", found_participants)
     575 + print_flag = "group_recap"
    557 576   
    558 577   if chat_type == "Channel":
    559  - scam_status = str(entity.scam)
    560  - color_print_green(" ├ Scam: ", str(scam_status))
    561  - else:
    562  - scam_status = "N/A"
     578 + print_flag = "channel_recap"
    563 579   
    564  - color_print_green(" ├ First post date: ", str(first_post))
    565  - 
    566  - if chat_type != "Channel":
    567  - color_print_green(
    568  - " ├ Memberlist saved to: ", memberlist_filename
    569  - )
    570  - 
    571  - color_print_green(
    572  - " â”” ", wrapper_r.fill(group_status)
    573  - )
    574  - #print("\n")
     580 + print_shell(print_flag, entity)
    575 581   
    576 582   log.append(
    577 583   [
    skipped 8 lines
    586 592   chat_type,
    587 593   entity.id,
    588 594   entity.access_hash,
    589  - scam_status,
     595 + str(entity.scam),
    590 596   date,
    591 597   mtime,
    592 598   group_status,
    skipped 54 lines
    647 653   if message.forward is not None:
    648 654   forward_count += 1
    649 655   
    650  - #print("\n")
    651 656   color_print_green(" [-] ", "Fetching forwarded messages...")
    652 657   
    653 658   progress_bar = (
    skipped 17 lines
    671 676   
    672 677   substring = "PeerUser"
    673 678   string = str(f_from_id)
    674  - if substring in string:
    675  - user_id = re.sub("[^0-9]", "", string)
    676  - user_id = await client.get_entity(
    677  - PeerUser(int(user_id))
    678  - )
    679  - user_id = str(user_id)
    680  - result = (
    681  - "User: "
    682  - + str(ent.first_name)
    683  - + " / ID: "
    684  - + str(user_id.id)
    685  - )
     679 + if chat_type != "Channel":
     680 + if substring in string:
     681 + user_id = re.sub("[^0-9]", "", string)
     682 + user_id = await client.get_entity(
     683 + PeerUser(int(user_id))
     684 + )
     685 + user_id = str(user_id)
     686 + result = (
     687 + "User: "
     688 + + str(ent.first_name)
     689 + + " / ID: "
     690 + + str(user_id.id)
     691 + )
     692 + else:
     693 + result = str(ent.title)
    686 694   else:
    687 695   result = str(ent.title)
    688 696   
    skipped 41 lines
    730 738   lines=True,
    731 739   index=True,
    732 740   )
    733  - else:
    734  - pass
    735 741   
    736 742   if forward_count >= 15:
    737 743   forwards_found = forwards_df.Source.count()
    skipped 2 lines
    740 746   name="counts"
    741 747   )
    742 748   
    743  - top_forward_one = df01.iloc[0]["unique_values"]
    744  - top_value_one = df01.iloc[0]["counts"]
    745  - top_forward_two = df01.iloc[1]["unique_values"]
    746  - top_value_two = df01.iloc[1]["counts"]
    747  - top_forward_three = df01.iloc[2]["unique_values"]
    748  - top_value_three = df01.iloc[2]["counts"]
    749  - top_forward_four = df01.iloc[3]["unique_values"]
    750  - top_value_four = df01.iloc[3]["counts"]
    751  - top_forward_five = df01.iloc[4]["unique_values"]
    752  - top_value_five = df01.iloc[4]["counts"]
    753  - 
    754  - forward_one = (
    755  - str(top_forward_one)
     749 + report_forward = createPlaceholdeCls()
     750 + report_forward.forward_one = (
     751 + str(df01.iloc[0]["unique_values"])
    756 752   + ", "
    757  - + str(top_value_one)
     753 + + str(df01.iloc[0]["counts"])
    758 754   + " forwarded messages"
    759 755   )
    760  - forward_two = (
    761  - str(top_forward_two)
     756 + report_forward.forward_two = (
     757 + str(df01.iloc[1]["unique_values"])
    762 758   + ", "
    763  - + str(top_value_two)
     759 + + str(df01.iloc[1]["counts"])
    764 760   + " forwarded messages"
    765 761   )
    766  - forward_three = (
    767  - str(top_forward_three)
     762 + report_forward.forward_three = (
     763 + str(df01.iloc[2]["unique_values"])
    768 764   + ", "
    769  - + str(top_value_three)
     765 + + str(df01.iloc[2]["counts"])
    770 766   + " forwarded messages"
    771 767   )
    772  - forward_four = (
    773  - str(top_forward_four)
     768 + report_forward.forward_four = (
     769 + str(df01.iloc[3]["unique_values"])
    774 770   + ", "
    775  - + str(top_value_four)
     771 + + str(df01.iloc[3]["counts"])
    776 772   + " forwarded messages"
    777 773   )
    778  - forward_five = (
    779  - str(top_forward_five)
     774 + report_forward.forward_five = (
     775 + str(df01.iloc[4]["unique_values"])
    780 776   + ", "
    781  - + str(top_value_five)
     777 + + str(df01.iloc[4]["counts"])
    782 778   + " forwarded messages"
    783 779   )
    784 780   
    785 781   df02 = forwards_df.Source.unique()
    786  - unique_forwards = len(df02)
    787  - 
    788  - #print("\n")
    789  - color_print_green(" [+] Forward scrape complete", "")
    790  - color_print_green(" ┬ Statistics", "")
    791  - color_print_green(
    792  - " ├ Forwarded messages found: ", str(forward_count)
    793  - )
    794  - color_print_green(
    795  - " ├ Forwards from active public chats: ",
    796  - str(forwards_found),
    797  - )
    798  - color_print_green(
    799  - " ├ Unique forward sources: ", str(unique_forwards)
    800  - )
    801  - color_print_green(
    802  - " ├ Top forward source 1: ", str(forward_one)
    803  - )
    804  - color_print_green(
    805  - " ├ Top forward source 2: ", str(forward_two)
    806  - )
    807  - color_print_green(
    808  - " ├ Top forward source 3: ", str(forward_three)
    809  - )
    810  - color_print_green(
    811  - " ├ Top forward source 4: ", str(forward_four)
    812  - )
    813  - color_print_green(
    814  - " ├ Top forward source 5: ", str(forward_five)
    815  - )
    816  - color_print_green(" â”” Edgelist saved to: ", edgelist_file)
    817  - #print("\n")
    818  - 
     782 + report_forward.unique_forwards = len(df02)
     783 + report_forward.edgelist_file = edgelist_file
     784 + print_shell("forwarder_stat",report_forward)
    819 785   else:
    820 786   print(
    821 787   "\n"
    skipped 3 lines
    825 791   )
    826 792   
    827 793   else:
    828  - 
    829 794   if comp_check is True:
    830 795   
    831 796   messages = client.iter_messages(t)
    skipped 1 lines
    833 798   message_list = []
    834 799   forwards_list = []
    835 800   
    836  - user_reaction_list = []
    837  - 
    838 801   replies_list = []
    839 802   user_replier_list = []
    840 803   
    841  - timecount = []
    842  - 
    843  - forward_count = 0
    844  - private_count = 0
     804 + forward_count, private_count, message_count = 0, 0, 0
    845 805   
    846 806   if media_archive is True:
    847 807   files = []
    skipped 5 lines
    853 813   color_print_green(
    854 814   " [!] ", "Calculating number of messages..."
    855 815   )
    856  - 
    857  - message_count = 0
    858 816   
    859 817   async for message in messages:
    860 818   if message is not None:
    skipped 18 lines
    879 837   t, limit=None
    880 838   ):
    881 839   if message is not None:
    882  - 
    883 840   try:
    884  - 
    885 841   c_archive = pd.DataFrame(
    886 842   message_list,
    887 843   columns=[
    888 844   "To",
    889 845   "Message ID",
    890 846   "Display_name",
    891  - "ID",
     847 + "User ID",
    892 848   "Message_text",
    893 849   "Original_language",
    894 850   "Translated_text",
    895 851   "Translation_confidence",
    896 852   "Timestamp",
    897 853   "Has_media",
    898  - "Reply",
     854 + "Reply_to_ID",
     855 + "Replies",
     856 + "Forwards",
    899 857   "Views",
     858 + "Total_reactions",
     859 + "Reply_ER_reach",
     860 + "Reply_ER_impressions",
     861 + "Forwards_ER_reach",
     862 + "Forwards_ER_impressions",
     863 + "Reaction_ER_reach",
     864 + "Reactions_ER_impressions",
     865 + "Thumbs_up",
     866 + "Thumbs_down",
     867 + "Heart",
     868 + "Fire",
     869 + "Smile_with_hearts",
     870 + "Clap",
     871 + "Smile",
     872 + "Thinking",
     873 + "Exploding_head",
     874 + "Scream",
     875 + "Angry",
     876 + "Single_tear",
     877 + "Party",
     878 + "Starstruck",
     879 + "Vomit",
     880 + "Poop",
     881 + "Pray",
     882 + "Edit_date",
    900 883   "URL",
     884 + "Media save directory"
    901 885   ],
    902 886   )
    903 887   
    skipped 68 lines
    972 956   user_replier_list.append(
    973 957   userdet
    974 958   )
    975  - mss_txt = process_message(
    976  - repl.text, user_language
    977  - )
     959 + 
     960 + if translate_check == True:
     961 + mss_txt = process_message(
     962 + repl.text, user_language
     963 + )
     964 + original_language = mss_txt["original_language"],
     965 + translated_text = mss_txt["translated_text"],
     966 + translation_confidence = mss_txt["translation_confidence"],
     967 + reply_text = mss_txt["message_text"]
     968 + else:
     969 + original_language = "N/A"
     970 + translated_text = "N/A"
     971 + translation_confidence = "N/A"
     972 + reply_text = repl.text
     973 + 
    978 974   replies_list.append(
    979 975   [
    980 976   t,
    skipped 1 lines
    982 978   repl.id,
    983 979   userdet[1],
    984 980   userdet[2],
    985  - mss_txt["message_text"],
    986  - mss_txt[
    987  - "original_language"
    988  - ],
    989  - mss_txt[
    990  - "translated_text"
    991  - ],
    992  - mss_txt[
    993  - "translation_confidence"
    994  - ],
     981 + reply_text,
     982 + original_language,
     983 + translated_text,
     984 + translation_confidence,
    995 985   parse_tg_date(
    996 986   repl.date
    997 987   )["timestamp"],
    skipped 21 lines
    1019 1009   ]
    1020 1010   reply = message.reply_to_msg_id
    1021 1011   
    1022  - _mess = process_message(
    1023  - message.text, user_language
    1024  - )
    1025  - message_text = _mess["message_text"]
    1026  - original_language = _mess[
    1027  - "original_language"
    1028  - ]
    1029  - translated_text = _mess["translated_text"]
    1030  - translation_confidence = _mess[
    1031  - "translation_confidence"
    1032  - ]
     1012 + if translate_check == True:
     1013 + _mess = process_message(
     1014 + message.text, user_language
     1015 + )
     1016 + message_text = _mess["message_text"]
     1017 + original_language = _mess[
     1018 + "original_language"
     1019 + ]
     1020 + translated_text = _mess["translated_text"]
     1021 + translation_confidence = _mess[
     1022 + "translation_confidence"
     1023 + ]
     1024 + else:
     1025 + message_text = message.text
     1026 + original_language = "N/A"
     1027 + translated_text = "N/A"
     1028 + translation_confidence = "N/A"
    1033 1029   
    1034 1030   if message.forwards is not None:
    1035 1031   forwards = int(message.forwards)
    1036 1032   else:
    1037  - forwards = "None"
     1033 + forwards = "N/A"
    1038 1034   
    1039 1035   if message.views is not None:
    1040 1036   views = int(message.views)
    1041 1037   else:
    1042  - views = "Not found"
     1038 + views = 'N/A'
    1043 1039   
    1044  - #if message.reactions:
    1045  - #if message.reactions.can_see_list:
    1046  - #print(dir(message.reactions.results))
    1047  - #print("#### TODO: REACTIONS")
     1040 + if message.reactions:
     1041 + reactions = message.reactions.results
     1042 + total_reactions = 0
     1043 + i = range(len(reactions))
     1044 +
     1045 + for idx, i in enumerate(reactions):
     1046 + total_reactions = total_reactions + i.count
     1047 + thumbs_up = i.count if i.reaction == '👍' else 0
     1048 + thumbs_down = i.count if i.reaction == '👎' else 0
     1049 + heart = i.count if i.reaction == '❤️' else 0
     1050 + fire = i.count if i.reaction == '🔥' else 0
     1051 + smile_with_hearts = i.count if i.reaction == '🥰' else 0
     1052 + clap = i.count if i.reaction == '👏' else 0
     1053 + smile = i.count if i.reaction == '😁' else 0
     1054 + thinking = i.count if i.reaction == '🤔' else 0
     1055 + exploding_head = i.count if i.reaction == '🤯' else 0
     1056 + scream = i.count if i.reaction == '😱' else 0
     1057 + angry = i.count if i.reaction == '🤬' else 0
     1058 + single_tear = i.count if i.reaction == '😢' else 0
     1059 + party_popper = i.count if i.reaction == '🎉' else 0
     1060 + starstruck = i.count if i.reaction == '🤩' else 0
     1061 + vomiting = i.count if i.reaction == '🤮' else 0
     1062 + poop = i.count if i.reaction == '💩' else 0
     1063 + praying = i.count if i.reaction == '🙏' else 0
     1064 + else:
     1065 + total_reactions = 'N/A'
     1066 + thumbs_up = 'N/A'
     1067 + thumbs_down = 'N/A'
     1068 + heart = 'N/A'
     1069 + fire = 'N/A'
     1070 + smile_with_hearts = 'N/A'
     1071 + clap = 'N/A'
     1072 + smile = 'N/A'
     1073 + thinking = 'N/A'
     1074 + exploding_head = 'N/A'
     1075 + scream = 'N/A'
     1076 + angry = 'N/A'
     1077 + single_tear = 'N/A'
     1078 + party_popper = 'N/A'
     1079 + starstruck = 'N/A'
     1080 + vomiting = 'N/A'
     1081 + poop = 'N/A'
     1082 + praying = 'N/A'
    1048 1083   
    1049 1084   if media_archive == True:
    1050  - if message.media:
     1085 + if message.media is not None:
    1051 1086   path = await message.download_media(
    1052  - file=media_directory
     1087 + file = media_directory
    1053 1088   )
    1054 1089   files.append(path)
     1090 + media_file = path
    1055 1091   else:
    1056  - pass
     1092 + media_file = "N/A"
     1093 + else:
     1094 + media_file = "N/A"
    1057 1095  
    1058 1096   if message.media is not None:
    1059 1097   has_media = "TRUE"
    1060 1098   else:
    1061  - has_media = 'FALSE'
     1099 + has_media = "FALSE"
     1100 + 
     1101 + if message.replies:
     1102 + reply_count = int(message.replies.replies)
     1103 + else:
     1104 + reply_count = "N/A"
    1062 1105   
    1063  - post_url = "https://t.me/s/" + t + "/" + message.id
     1106 + if message.edit_date:
     1107 + edit_date = str(message.edit_date)
     1108 + else:
     1109 + edit_date = "None"
     1110 + 
     1111 + '''Need to find a way to calculate these in case these figures don't exist to make it
     1112 + comparable across channels for a total engagement number (e.g. if replies/reactions are off).
     1113 + If not N/A would cover if it's off, zero if it's none. Working on some better logic here.'''
     1114 + 
     1115 + if reply_count != 'N/A' and total_participants is not None:
     1116 + reply_reach_ER = (reply_count / int(total_participants)) * 100
     1117 + else:
     1118 + reply_reach_ER = 'N/A'
     1119 + 
     1120 + if reply_count != 'N/A' and views != 'N/A':
     1121 + reply_impressions_ER = (reply_count / int(views)) * 100
     1122 + else:
     1123 + reply_impressions_ER = 'N/A'
     1124 + 
     1125 + if forwards != 'N/A' and total_participants is not None:
     1126 + forwards_reach_ER = (forwards / int(total_participants)) * 100
     1127 + else:
     1128 + forwards_reach_ER = 'N/A'
     1129 + 
     1130 + if forwards != 'N/A' and views != 'N/A':
     1131 + forwards_impressions_ER = (forwards / int(views)) * 100
     1132 + else:
     1133 + forwards_impressions_ER = 'N/A'
     1134 + 
     1135 + if total_reactions != 'N/A' and total_participants is not None:
     1136 + reactions_reach_ER = (total_reactions / int(total_participants)) * 100
     1137 + else:
     1138 + reactions_reach_ER = 'N/A'
     1139 + 
     1140 + if total_reactions != 'N/A' and views != 'N/A':
     1141 + reactions_impressions_ER = (total_reactions / int(views)) * 100
     1142 + else:
     1143 + reactions_impressions_ER = 'N/A'
     1144 + 
     1145 + post_url = "https://t.me/s/" + t + "/" + str(message.id)
    1064 1146   
    1065 1147   message_list.append(
    1066 1148   [
    skipped 8 lines
    1075 1157   timestamp,
    1076 1158   has_media,
    1077 1159   reply,
     1160 + reply_count,
     1161 + forwards,
    1078 1162   views,
     1163 + total_reactions,
     1164 + reply_reach_ER,
     1165 + reply_impressions_ER,
     1166 + forwards_reach_ER,
     1167 + forwards_impressions_ER,
     1168 + reactions_reach_ER,
     1169 + reactions_impressions_ER,
     1170 + thumbs_up,
     1171 + thumbs_down,
     1172 + heart,
     1173 + fire,
     1174 + smile_with_hearts,
     1175 + clap,
     1176 + smile,
     1177 + thinking,
     1178 + exploding_head,
     1179 + scream,
     1180 + angry,
     1181 + single_tear,
     1182 + party_popper,
     1183 + starstruck,
     1184 + vomiting,
     1185 + poop,
     1186 + praying,
     1187 + edit_date,
    1079 1188   post_url,
     1189 + media_file,
    1080 1190   ]
    1081 1191   )
    1082 1192   
    1083 1193   if message.forward is not None:
    1084  - forward_verify = True
    1085 1194   try:
    1086 1195   forward_count += 1
    1087 1196   to_title = to_ent.title
    skipped 5 lines
    1093 1202   ent = await client.get_entity(
    1094 1203   f_from_id
    1095 1204   )
     1205 + 
    1096 1206   user_string = "user_id"
    1097 1207   channel_string = "broadcast"
    1098 1208   
    skipped 7 lines
    1106 1216   ent.broadcast
    1107 1217   is True
    1108 1218   ):
    1109  - ent_type = "Channel"
     1219 + ent_type = (
     1220 + "Channel"
     1221 + )
    1110 1222   elif (
    1111 1223   ent.megagroup
    1112 1224   is True
    skipped 89 lines
    1202 1314   "None",
    1203 1315   "None",
    1204 1316   "None",
     1317 + "None",
     1318 + "None",
     1319 + "None",
     1320 + "None",
     1321 + "None",
     1322 + "None",
     1323 + "None",
     1324 + "None",
     1325 + "None",
     1326 + "None",
     1327 + "None",
     1328 + "None",
     1329 + "None",
     1330 + "None",
     1331 + "None",
     1332 + "None",
     1333 + "None",
     1334 + "None",
     1335 + "None",
     1336 + "None",
     1337 + "None",
     1338 + "None",
     1339 + "None",
     1340 + "None",
     1341 + "None",
     1342 + "None",
     1343 + "None",
    1205 1344   ]
    1206 1345   )
    1207  - pass
    1208 1346   
    1209 1347   time.sleep(0.5)
    1210 1348   bar()
    skipped 18 lines
    1229 1367   
    1230 1368   if json_check == True:
    1231 1369   c_archive.to_json(
    1232  - json_file + alphanumeric + "_archive.json",
     1370 + json_file
     1371 + + alphanumeric
     1372 + + "_archive.json",
    1233 1373   orient="records",
    1234 1374   compression="infer",
    1235 1375   lines=True,
    1236 1376   index=True,
    1237 1377   )
    1238  - else:
    1239  - pass
    1240 1378   
    1241 1379   if forwards_check is True:
    1242 1380   with open(
    skipped 3 lines
    1246 1384   
    1247 1385   if json_check == True:
    1248 1386   c_forwards.to_json(
    1249  - json_file + alphanumeric + "_edgelist.json",
     1387 + json_file
     1388 + + alphanumeric
     1389 + + "_edgelist.json",
    1250 1390   orient="records",
    1251 1391   compression="infer",
    1252 1392   lines=True,
    1253 1393   index=True,
    1254 1394   )
    1255  - else:
    1256  - pass
    1257  - else:
    1258  - pass
    1259 1395   
    1260 1396   messages_found = int(c_archive.To.count()) - 1
    1261  - message_frequency_count = {}
    1262  - message_text = {}
    1263  - word_count = {}
    1264  - most_used_words = {}
    1265  - most_used_words_filtered = {}
    1266  - # message stats, top words
     1397 + report_obj = createPlaceholdeCls()
     1398 + report_obj.messages_found = messages_found
     1399 + report_obj.file_archive = file_archive
    1267 1400   
    1268  - if chat_type != "Channel":
    1269  - pcount = c_archive.Display_name.count()
     1401 + if chat_type == "Channel":
     1402 + print_shell("channel_stat", report_obj)
     1403 + else:
    1270 1404   pvalue_count = c_archive["Display_name"].value_counts()
    1271 1405   df03 = pvalue_count.rename_axis(
    1272 1406   "unique_values"
    1273 1407   ).reset_index(name="counts")
    1274 1408   
    1275  - top_poster_one = str(df03.iloc[0]["unique_values"])
    1276  - top_pvalue_one = df03.iloc[0]["counts"]
    1277  - top_poster_two = str(df03.iloc[1]["unique_values"])
    1278  - top_pvalue_two = df03.iloc[1]["counts"]
    1279  - top_poster_three = str(df03.iloc[2]["unique_values"])
    1280  - top_pvalue_three = df03.iloc[2]["counts"]
    1281  - top_poster_four = str(df03.iloc[3]["unique_values"])
    1282  - top_pvalue_four = df03.iloc[3]["counts"]
    1283  - top_poster_five = str(df03.iloc[4]["unique_values"])
    1284  - top_pvalue_five = df03.iloc[4]["counts"]
     1409 + '''
     1410 + message_frequency_count = {}
     1411 + message_text = {}
     1412 + word_count = {}
     1413 + most_used_words = {}
     1414 + most_used_words_filtered = {}
     1415 + '''
     1416 + #message stats, top words
    1285 1417   
    1286  - poster_one = (
    1287  - str(top_poster_one)
     1418 + report_obj.poster_one = (
     1419 + str(df03.iloc[0]["unique_values"])
    1288 1420   + ", "
    1289  - + str(top_pvalue_one)
     1421 + + str(df03.iloc[0]["counts"])
    1290 1422   + " messages"
    1291 1423   )
    1292  - poster_two = (
    1293  - str(top_poster_two)
     1424 + report_obj.poster_two = (
     1425 + str(df03.iloc[1]["unique_values"])
    1294 1426   + ", "
    1295  - + str(top_pvalue_two)
     1427 + + str(df03.iloc[1]["counts"])
    1296 1428   + " messages"
    1297 1429   )
    1298  - poster_three = (
    1299  - str(top_poster_three)
     1430 + report_obj.poster_three = (
     1431 + str(df03.iloc[2]["unique_values"])
    1300 1432   + ", "
    1301  - + str(top_pvalue_three)
     1433 + + str(df03.iloc[2]["counts"])
    1302 1434   + " messages"
    1303 1435   )
    1304  - poster_four = (
    1305  - str(top_poster_four)
     1436 + report_obj.poster_four = (
     1437 + str(df03.iloc[3]["unique_values"])
    1306 1438   + ", "
    1307  - + str(top_pvalue_four)
     1439 + + str(df03.iloc[3]["counts"])
    1308 1440   + " messages"
    1309 1441   )
    1310  - poster_five = (
    1311  - str(top_poster_five)
     1442 + report_obj.poster_four = (
     1443 + str(df03.iloc[4]["unique_values"])
    1312 1444   + ", "
    1313  - + str(top_pvalue_five)
     1445 + + str(df03.iloc[4]["counts"])
    1314 1446   + " messages"
    1315 1447   )
    1316 1448   
    1317 1449   df04 = c_archive.Display_name.unique()
    1318  - plength = len(df03)
    1319 1450   unique_active = len(df04)
     1451 + report_obj.unique_active = unique_active
     1452 + print_shell("group_stat", report_obj)
    1320 1453   
    1321  - elif reply_analysis is True:
     1454 + if reply_analysis is True:
    1322 1455   if len(replies_list) > 0:
    1323  - replier_count = c_repliers["User id"].count()
    1324  - replier_value_count = c_repliers["User id"].value_counts()
     1456 + replier_value_count = c_repliers["User ID"].value_counts()
    1325 1457   replier_df = replier_value_count.rename_axis(
    1326 1458   "unique_values"
    1327 1459   ).reset_index(name="counts")
    1328 1460   
    1329  - top_replier_one = str(replier_df.iloc[0]["unique_values"])
    1330  - top_replier_value_one = replier_df.iloc[0]["counts"]
    1331  - top_replier_two = str(replier_df.iloc[1]["unique_values"])
    1332  - top_replier_value_two = replier_df.iloc[1]["counts"]
    1333  - top_replier_three = str(replier_df.iloc[2]["unique_values"])
    1334  - top_replier_value_three = replier_df.iloc[2]["counts"]
    1335  - top_replier_four = str(replier_df.iloc[3]["unique_values"])
    1336  - top_replier_value_four = replier_df.iloc[3]["counts"]
    1337  - top_replier_five = str(replier_df.iloc[4]["unique_values"])
    1338  - top_replier_value_five = replier_df.iloc[4]["counts"]
    1339  - 
    1340  - replier_one = (
    1341  - str(top_replier_one)
     1461 + repliers = createPlaceholdeCls()
     1462 + repliers.replier_one = (
     1463 + str(replier_df.iloc[0]["unique_values"])
    1342 1464   + ", "
    1343  - + str(top_replier_value_one)
     1465 + + str(replier_df.iloc[0]["counts"])
    1344 1466   + " replies"
    1345 1467   )
    1346  - replier_two = (
    1347  - str(top_replier_two)
     1468 + repliers.replier_two = (
     1469 + str(replier_df.iloc[1]["unique_values"])
    1348 1470   + ", "
    1349  - + str(top_replier_value_two)
     1471 + + str(replier_df.iloc[1]["counts"])
    1350 1472   + " replies"
    1351 1473   )
    1352  - replier_three = (
    1353  - str(top_replier_three)
     1474 + repliers.replier_three = (
     1475 + str(replier_df.iloc[2]["unique_values"])
    1354 1476   + ", "
    1355  - + str(top_replier_value_three)
     1477 + + str(replier_df.iloc[2]["counts"])
    1356 1478   + " replies"
    1357 1479   )
    1358  - replier_four = (
    1359  - str(top_replier_four)
     1480 + repliers.replier_four = (
     1481 + str(replier_df.iloc[3]["unique_values"])
    1360 1482   + ", "
    1361  - + str(top_replier_value_four)
     1483 + + str(replier_df.iloc[3]["counts"])
    1362 1484   + " replies"
    1363 1485   )
    1364  - replier_five = (
    1365  - str(top_replier_five)
     1486 + repliers.replier_five = (
     1487 + str(replier_df.iloc[4]["unique_values"])
    1366 1488   + ", "
    1367  - + str(top_replier_value_five)
     1489 + + str(replier_df.iloc[4]["counts"])
    1368 1490   + " replies"
    1369 1491   )
    1370 1492   
    1371  - replier_count_df = c_repliers["User id"].unique()
    1372  - replier_length = len(replier_df)
     1493 + replier_count_df = c_repliers["User ID"].unique()
    1373 1494   replier_unique = len(replier_count_df)
    1374  - 
    1375  - else:
    1376  - pass
    1377  - 
    1378  - #print("\n")
    1379  - color_print_green(" [+] Chat archive saved", "")
    1380  - color_print_green(" ┬ Chat statistics", "")
    1381  - color_print_green(
    1382  - " ├ Number of messages found: ", str(messages_found)
    1383  - )
    1384  - 
    1385  - if chat_type != "Channel":
    1386  - color_print_green(
    1387  - " ├ Top poster 1: ", str(poster_one)
    1388  - )
    1389  - color_print_green(
    1390  - " ├ Top poster 2: ", str(poster_two)
    1391  - )
    1392  - color_print_green(
    1393  - " ├ Top poster 3: ", str(poster_three)
    1394  - )
    1395  - color_print_green(
    1396  - " ├ Top poster 4: ", str(poster_four)
    1397  - )
    1398  - color_print_green(
    1399  - " ├ Top poster 5: ", str(poster_five)
    1400  - )
    1401  - color_print_green(
    1402  - " ├ Total unique posters: ", str(unique_active)
    1403  - )
    1404  -
    1405  - else:
    1406  - pass
    1407  - # timestamp analysis
    1408  - # print(Fore.GREEN
    1409  - # + ' ├ Number of messages: '
    1410  - # + Style.RESET_ALL
    1411  - # + str(message_count))
    1412  - 
    1413  - color_print_green(
    1414  - " â”” Archive saved to: ", str(file_archive)
    1415  - )
    1416  - 
    1417  - if reply_analysis is True:
    1418  - if len(replies_list) > 0:
    1419  - middle_char = "├"
    1420  - if user_replier_list == 0:
    1421  - middle_char = "â””"
    1422  - 
    1423  - #print("\n")
    1424  - color_print_green(" [+] Replies analysis ", "")
    1425  - color_print_green(" ┬ Chat statistics", "")
    1426  - color_print_green(
    1427  - f" {middle_char} Archive of replies saved to: ",
    1428  - str(reply_file_archive),
    1429  - )
    1430  - if len(user_replier_list) > 0:
    1431  - color_print_green(
    1432  - " â”” Active members list who replied to messages, saved to: ",
    1433  - str(reply_memberlist_filename),
    1434  - )
    1435  - 
    1436  - color_print_green(
    1437  - " ├ Top replier 1: ", str(replier_one)
    1438  - )
    1439  - color_print_green(
    1440  - " ├ Top replier 2: ", str(replier_two)
    1441  - )
    1442  - color_print_green(
    1443  - " ├ Top replier 3: ", str(replier_three)
    1444  - )
    1445  - color_print_green(
    1446  - " ├ Top replier 4: ", str(replier_four)
    1447  - )
    1448  - color_print_green(
    1449  - " ├ Top replier 5: ", str(replier_five)
    1450  - )
    1451  - color_print_green(
    1452  - " ├ Total unique repliers: ", str(replier_unique)
    1453  - )
     1495 + repliers.user_replier_list_len = len(user_replier_list)
     1496 + repliers.reply_file_archive = str(reply_file_archive)
     1497 + repliers.reply_memberlist_filename = str(reply_memberlist_filename)
     1498 + repliers.replier_unique = str(replier_unique)
     1499 + print_shell("reply_stat", repliers)
    1454 1500   
    1455 1501   if forwards_check is True:
    1456 1502   if forward_count >= 15:
    skipped 3 lines
    1460 1506   "unique_values"
    1461 1507   ).reset_index(name="counts")
    1462 1508   
    1463  - top_forward_one = c_f_stats.iloc[0]["unique_values"]
    1464  - top_value_one = c_f_stats.iloc[0]["counts"]
    1465  - top_forward_two = c_f_stats.iloc[1]["unique_values"]
    1466  - top_value_two = c_f_stats.iloc[1]["counts"]
    1467  - top_forward_three = c_f_stats.iloc[2][
    1468  - "unique_values"
    1469  - ]
    1470  - top_value_three = c_f_stats.iloc[2]["counts"]
    1471  - top_forward_four = c_f_stats.iloc[3][
    1472  - "unique_values"
    1473  - ]
    1474  - top_value_four = c_f_stats.iloc[3]["counts"]
    1475  - top_forward_five = c_f_stats.iloc[4][
    1476  - "unique_values"
    1477  - ]
    1478  - top_value_five = c_f_stats.iloc[4]["counts"]
    1479  - 
    1480  - forward_one = (
    1481  - str(top_forward_one)
     1509 + report_forward = createPlaceholdeCls()
     1510 + report_forward.forward_one = (
     1511 + str(c_f_stats.iloc[0]["unique_values"])
    1482 1512   + ", "
    1483  - + str(top_value_one)
     1513 + + str(c_f_stats.iloc[0]["counts"])
    1484 1514   + " forwarded messages"
    1485 1515   )
    1486  - forward_two = (
    1487  - str(top_forward_two)
     1516 + report_forward.forward_two = (
     1517 + str(c_f_stats.iloc[1]["unique_values"])
    1488 1518   + ", "
    1489  - + str(top_value_two)
     1519 + + str(c_f_stats.iloc[1]["counts"])
    1490 1520   + " forwarded messages"
    1491 1521   )
    1492  - forward_three = (
    1493  - str(top_forward_three)
     1522 + report_forward.forward_three = (
     1523 + str(c_f_stats.iloc[2]["unique_values"])
    1494 1524   + ", "
    1495  - + str(top_value_three)
     1525 + + str(c_f_stats.iloc[2]["counts"])
    1496 1526   + " forwarded messages"
    1497 1527   )
    1498  - forward_four = (
    1499  - str(top_forward_four)
     1528 + report_forward.forward_four = (
     1529 + str(c_f_stats.iloc[3]["unique_values"])
    1500 1530   + ", "
    1501  - + str(top_value_four)
     1531 + + str(c_f_stats.iloc[3]["counts"])
    1502 1532   + " forwarded messages"
    1503 1533   )
    1504  - forward_five = (
    1505  - str(top_forward_five)
     1534 + report_forward.forward_five = (
     1535 + str(c_f_stats.iloc[4]["unique_values"])
    1506 1536   + ", "
    1507  - + str(top_value_five)
     1537 + + str(c_f_stats.iloc[4]["counts"])
    1508 1538   + " forwarded messages"
    1509 1539   )
    1510 1540   
    1511 1541   c_f_unique = c_forwards.Source.unique()
    1512  - unique_forwards = len(c_f_unique)
    1513 1542   
    1514  - #print("\n")
    1515  - color_print_green(" [+] Edgelist saved", "")
    1516  - color_print_green(
    1517  - " ┬ Forwarded message statistics", ""
    1518  - )
    1519  - color_print_green(
    1520  - " ├ Forwarded messages found: ",
    1521  - str(forward_count),
    1522  - )
    1523  - color_print_green(
    1524  - " ├ Forwards from active public chats: ",
    1525  - str(forwards_found),
    1526  - )
    1527  - color_print_green(
    1528  - " ├ Forwards from private (or now private) chats: ",
    1529  - str(private_count),
    1530  - )
    1531  - color_print_green(
    1532  - " ├ Unique forward sources: ",
    1533  - str(unique_forwards),
    1534  - )
    1535  - color_print_green(
    1536  - " ├ Top forward source 1: ", str(forward_one)
    1537  - )
    1538  - color_print_green(
    1539  - " ├ Top forward source 2: ", str(forward_two)
    1540  - )
    1541  - color_print_green(
    1542  - " ├ Top forward source 3: ",
    1543  - str(forward_three),
    1544  - )
    1545  - color_print_green(
    1546  - " ├ Top forward source 4: ", str(forward_four)
    1547  - )
    1548  - color_print_green(
    1549  - " ├ Top forward source 5: ", str(forward_five)
    1550  - )
    1551  - color_print_green(
    1552  - " â”” Edgelist saved to: ", edgelist_file
    1553  - )
    1554  - #print("\n")
     1543 + report_forward.unique_forwards = len(c_f_unique)
     1544 + report_forward.edgelist_file = edgelist_file
     1545 + report_forward.private_count = private_count
     1546 + print_shell("forwarder_stat", report_forward)
    1555 1547   
    1556 1548   else:
    1557  - #print("\n")
    1558 1549   color_print_green(
    1559 1550   " [!] Insufficient forwarded messages found",
    1560 1551   edgelist_file,
    1561 1552   )
    1562  - else:
    1563  - pass
    1564 1553   
    1565 1554   if user_check == True:
    1566 1555   my_user = None
    1567 1556   try:
    1568  - 
    1569  - user = int(t)
    1570  - my_user = await client.get_entity(PeerUser(int(user)))
     1557 + if "@" in t:
     1558 + my_user = await client.get_entity(t)
     1559 + else:
     1560 + user = int(t)
     1561 + my_user = await client.get_entity(PeerUser(int(user)))
    1571 1562   
    1572 1563   user_first_name = my_user.first_name
    1573 1564   user_last_name = my_user.last_name
    1574 1565   if user_last_name is not None:
    1575 1566   user_full_name = (
    1576  - str(user_first_name) + " " + str(user_last_name)
     1567 + str(user_first_name)
     1568 + + " "
     1569 + + str(user_last_name)
    1577 1570   )
    1578 1571   else:
    1579 1572   user_full_name = str(user_first_name)
    skipped 3 lines
    1583 1576   else:
    1584 1577   user_photo = "None"
    1585 1578   
     1579 + if my_user.status is not None:
     1580 + if "Empty" in str(my_user.status):
     1581 + user_status = "Last seen over a month ago"
     1582 + elif "Month" in str(my_user.status):
     1583 + user_status = "Between a week and a month"
     1584 + elif "Week" in str(my_user.status):
     1585 + user_status = "Between three and seven days"
     1586 + elif "Offline" in str(my_user.status):
     1587 + user_status = "Offline"
     1588 + elif "Online" in str(my_user.status):
     1589 + user_status = "Online"
     1590 + elif "Recently" in str(my_user.status):
     1591 + user_status = "Recently (within two days)"
     1592 + else:
     1593 + user_status = "Not found"
     1594 + 
    1586 1595   if my_user.restriction_reason is not None:
    1587 1596   ios_restriction = entity.restriction_reason[0]
    1588 1597   if 1 in entity.restriction_reason:
    skipped 8 lines
    1597 1606   else:
    1598 1607   user_restrictions = "None"
    1599 1608   
    1600  - color_print_green(" [+] ", "User details for " + t)
    1601  - color_print_green(" ├ Username: ", str(my_user.username))
    1602  - color_print_green(" ├ Name: ", str(user_full_name))
    1603  - color_print_green(" ├ Verification: ", str(my_user.verified))
    1604  - color_print_green(" ├ Photo ID: ", str(user_photo))
    1605  - color_print_green(" ├ Phone number: ", str(my_user.phone))
    1606  - color_print_green(
    1607  - " ├ Access hash: ", str(my_user.access_hash)
    1608  - )
    1609  - color_print_green(" ├ Language: ", str(my_user.lang_code))
    1610  - color_print_green(" ├ Bot: ", str(my_user.bot))
    1611  - color_print_green(" ├ Scam: ", str(my_user.scam))
    1612  - color_print_green(" â”” Restrictions: ", str(user_restrictions))
     1609 + setattr(my_user, "user_restrictions", str(user_restrictions))
     1610 + setattr(my_user, "user_full_name", str(user_full_name))
     1611 + setattr(my_user, "user_photo", str(user_photo))
     1612 + setattr(my_user, "user_status", str(user_status))
     1613 + setattr(my_user, "target", t)
     1614 + print_shell("user", my_user)
    1613 1615   
    1614 1616   except ValueError:
    1615 1617   pass
     1618 + 
    1616 1619   if my_user is None:
    1617 1620   print(
    1618 1621   Fore.GREEN
    skipped 4 lines
    1623 1626   
    1624 1627   if location_check == True:
    1625 1628   
    1626  - location = t
    1627  - 
    1628 1629   print(
    1629 1630   Fore.GREEN
    1630 1631   + " [!] "
    1631 1632   + Style.RESET_ALL
    1632 1633   + "Searching for users near "
    1633  - + location
     1634 + + t
    1634 1635   + "\n"
    1635 1636   )
    1636  - latitude, longitude = location.split(sep=",")
     1637 + 
     1638 + latitude, longitude = t.split(sep=",")
    1637 1639   
    1638 1640   locations_file = telepathy_file + "locations/"
    1639  - 
    1640  - try:
     1641 + if not os.path.exists(locations_file):
    1641 1642   os.makedirs(locations_file)
    1642  - except FileExistsError:
    1643  - pass
    1644 1643   
    1645 1644   save_file = (
    1646 1645   locations_file
    skipped 7 lines
    1654 1653   )
    1655 1654   
    1656 1655   locations_list = []
     1656 + l_save_list = []
     1657 + 
    1657 1658   result = await client(
    1658 1659   functions.contacts.GetLocatedRequest(
    1659 1660   geo_point=types.InputGeoPoint(
    skipped 4 lines
    1664 1665   self_expires=42,
    1665 1666   )
    1666 1667   )
    1667  - 
    1668  - # progress bar?
    1669 1668   
    1670 1669   for user in result.updates[0].peers:
    1671 1670   try:
    1672 1671   user_df = pd.DataFrame(
    1673  - locations_list, columns=["User_ID", "Distance"]
     1672 + locations_list, columns=[
     1673 + "User_ID",
     1674 + "Distance"]
     1675 + )
     1676 + 
     1677 + l_save_df = pd.DataFrame(
     1678 + l_save_list, columns=[
     1679 + "User_ID",
     1680 + "Distance",
     1681 + "Latitude",
     1682 + "Longitude",
     1683 + "Date_retrieved"
     1684 + ]
    1674 1685   )
     1686 + 
    1675 1687   if hasattr(user, "peer"):
    1676 1688   ID = user.peer.user_id
    1677  - else:
    1678  - pass
     1689 + 
    1679 1690   if hasattr(user, "distance"):
    1680 1691   distance = user.distance
    1681  - else:
    1682  - pass
    1683 1692   
    1684 1693   locations_list.append([ID, distance])
    1685  - 
     1694 + l_save_list.append(
     1695 + [
     1696 + ID,
     1697 + distance,
     1698 + latitude,
     1699 + longitude,
     1700 + filetime
     1701 + ]
     1702 + )
    1686 1703   except:
    1687 1704   pass
    1688 1705   
    1689  - d_500 = 0
    1690  - d_1000 = 0
    1691  - d_2000 = 0
    1692  - d_3000 = 0
     1706 + distance_obj = createPlaceholdeCls()
     1707 + distance_obj.d500 = 0
     1708 + distance_obj.d1000 = 0
     1709 + distance_obj.d2000 = 0
     1710 + distance_obj.d3000 = 0
    1693 1711   
    1694 1712   for account, distance in user_df.itertuples(index=False):
    1695 1713   account = int(account)
    1696 1714   my_user = await client.get_entity(PeerUser(account))
    1697 1715   user_id = my_user.id
    1698  - name = my_user.first_name
    1699 1716   distance = int(distance)
    1700 1717   
    1701 1718   if distance == 500:
    1702  - d_500 += 1
     1719 + distance_obj.d500 += 1
    1703 1720   elif distance == 1000:
    1704  - d_1000 += 1
     1721 + distance_obj.d1000 += 1
    1705 1722   elif distance == 2000:
    1706  - d_2000 += 1
     1723 + distance_obj.d2000 += 1
    1707 1724   elif distance == 3000:
    1708  - d_3000 += 1
     1725 + distance_obj.d3000 += 1
    1709 1726   
    1710  - with open(
    1711  - save_file, "w+", encoding="utf-8"
    1712  - ) as f:
    1713  - user_df.to_csv(f, sep=";", index=False)
    1714 1727   
    1715  - total = len(locations_list)
     1728 + with open(save_file, "w+", encoding="utf-8") as f:
     1729 + l_save_df.to_csv(f, sep=";", index=False)
    1716 1730   
    1717  - color_print_green(" [+] Users located", "")
    1718  - color_print_green(" ├ Users within 500m: ", str(d_500))
    1719  - color_print_green(" ├ Users within 1000m: ", str(d_1000))
    1720  - color_print_green(" ├ Users within 2000m: ", str(d_2000))
    1721  - color_print_green(" ├ Users within 3000m: ", str(d_3000))
    1722  - color_print_green(" ├ Total users found: ", str(total))
    1723  - color_print_green(" â”” Location list saved to: ", save_file)
     1731 + total = len(locations_list)
    1724 1732   
    1725  - user_df.iloc[0:0]
     1733 + distance_obj.save_file = save_file
     1734 + distance_obj.total = total
     1735 + print_shell("location_report",distance_obj)
     1736 + # can also do the same for channels with similar output file to users
     1737 + # may one day add trilateration to find users closest to exact point
    1726 1738  
    1727 1739   with client:
    1728 1740   client.loop.run_until_complete(main())
    1729 1741   
    1730 1742  if __name__ == "__main__":
    1731 1743   cli()
    1732  - 
  • ■ ■ ■ ■ ■ ■
    build/lib/telepathy/utils.py
    1  -from colorama import Fore, Back, Style
    2  -from googletrans import Translator, constants
    3  -from .const import __version__, user_agent
     1 +from colorama import Fore, Style
     2 +from googletrans import Translator
     3 +from telepathy.const import __version__, user_agent
    4 4  import requests
    5 5  import textwrap
    6 6  from bs4 import BeautifulSoup
    7 7  import random
     8 + 
     9 +def createPlaceholdeCls():
     10 + class Object(object):
     11 + pass
     12 + a = Object()
     13 + return a
    8 14   
    9 15  def print_banner():
    10 16   print(
    skipped 27 lines
    38 44   if user.username:
    39 45   username = user.username
    40 46   else:
    41  - username = "n/a"
     47 + username = "N/A"
    42 48   if user.first_name:
    43 49   first_name = user.first_name
    44 50   else:
    skipped 5 lines
    50 56   if user.phone:
    51 57   phone = user.phone
    52 58   else:
    53  - phone = "n/a"
     59 + phone = "N/A"
     60 + if user.id:
     61 + user_id = user.id
     62 + else:
     63 + user_id = "N/A"
    54 64   full_name = (first_name + " " + last_name).strip()
    55  - return [username, full_name, user.id, phone, group_or_chat]
     65 + return [username, full_name, user_id, phone, group_or_chat]
    56 66   
    57 67   
    58 68  def process_message(mess, user_lang):
    skipped 1 lines
    60 70   if mess is not None:
    61 71   mess_txt = '"' + mess + '"'
    62 72   else:
    63  - mess_txt = "none"
     73 + mess_txt = "None"
    64 74   
    65  - if mess_txt != "none":
     75 + if mess_txt != "None":
    66 76   translator = Translator()
    67 77   detection = translator.detect(mess_txt)
    68  - language_code = detection.lang
    69 78   translation_confidence = detection.confidence
    70 79   translation = translator.translate(mess_txt, dest=user_lang)
    71 80   original_language = translation.src
    72 81   translated_text = translation.text
    73 82   else:
    74 83   original_language = user_lang
    75  - translated_text = "n/a"
    76  - translation_confidence = "n/a"
     84 + translated_text = "N/A"
     85 + translation_confidence = "N/A"
    77 86   
    78 87   return {
    79 88   "original_language": original_language,
    skipped 6 lines
    86 95   if desc is not None:
    87 96   desc_txt = '"' + desc + '"'
    88 97   else:
    89  - desc_txt = "none"
     98 + desc_txt = "None"
    90 99   
    91  - if desc_txt != "none":
     100 + if desc_txt != "None":
    92 101   translator = Translator()
    93 102   detection = translator.detect(desc_txt)
    94  - language_code = detection.lang
    95 103   translation_confidence = detection.confidence
    96 104   translation = translator.translate(desc_txt, dest=user_lang)
    97 105   original_language = translation.src
    98 106   translated_text = translation.text
    99 107   else:
    100 108   original_language = user_lang
    101  - translated_text = "n/a"
    102  - translation_confidence = "n/a"
     109 + translated_text = "N/A"
     110 + translation_confidence = "N/A"
    103 111   
    104 112   return {
    105 113   "original_language": original_language,
    skipped 32 lines
    138 146   "div", {"class": ["tgme_page_description"]}
    139 147   ).text
    140 148   descript = Fore.GREEN + "Description: " + Style.RESET_ALL+ group_description
    141  - prefix = descript + " "
    142 149   except:
    143 150   group_description = "None"
    144 151   descript = Fore.GREEN + "Description: " + Style.RESET_ALL+ group_description
    145  - prefix = descript + " "
    146 152   
    147 153   try:
    148 154   group_participants = soup.find(
    skipped 8 lines
    157 163   .replace("member", "")
    158 164   )
    159 165   except:
    160  - total_participants = "Not found" # could be due to restriction, might need to mention
     166 + total_participants = "Not found"
    161 167   
    162 168   return {"name":name,"group_description":group_description, "total_participants":total_participants}
     169 + 
     170 + 
     171 +def generate_textwrap(text_string, size=70):
     172 + trans_descript = Fore.GREEN + f"{text_string} " + Style.RESET_ALL
     173 + prefix = trans_descript
     174 + return textwrap.TextWrapper(
     175 + initial_indent=prefix,
     176 + width=size,
     177 + subsequent_indent=" ",
     178 + )
     179 + 
     180 +def print_shell(type, obj):
     181 + if type == "user":
     182 + color_print_green(" [+] ", "User details for " + obj.target)
     183 + color_print_green(" ├ Username: ", str(obj.username))
     184 + color_print_green(" ├ Name: ", str(obj.user_full_name))
     185 + color_print_green(" ├ Verification: ", str(obj.verified))
     186 + color_print_green(" ├ Photo ID: ", str(obj.user_photo))
     187 + color_print_green(" ├ Phone number: ", str(obj.phone))
     188 + color_print_green(
     189 + " ├ Access hash: ", str(obj.access_hash)
     190 + )
     191 + color_print_green(" ├ Language: ", str(obj.lang_code))
     192 + color_print_green(" ├ Bot: ", str(obj.bot))
     193 + color_print_green(" ├ Scam: ", str(obj.scam))
     194 + color_print_green(" ├ Last seen: ", str(obj.user_status))
     195 + color_print_green(" └ Restrictions: ", str(obj.user_restrictions))
     196 + 
     197 + if type == "location_report":
     198 + color_print_green(" [+] Users located", "")
     199 + color_print_green(" ├ Users within 500m: ", str(obj.d500))
     200 + color_print_green(" ├ Users within 1000m: ", str(obj.d1000))
     201 + color_print_green(" ├ Users within 2000m: ", str(obj.d2000))
     202 + color_print_green(" ├ Users within 3000m: ", str(obj.d3000))
     203 + color_print_green(" ├ Total users found: ", str(obj.total))
     204 + color_print_green(" └ Location list saved to: ", obj.save_file)
     205 + 
     206 + if type == "channel_recap" or type == "group_recap":
     207 + 
     208 + d_wrapper = generate_textwrap("Description:")
     209 + td_wrapper = generate_textwrap("Translated Description:")
     210 + 
     211 + color_print_green(" ┬ Chat details", "")
     212 + color_print_green(" ├ Title: ", str(obj.title))
     213 + color_print_green(" ├ ", d_wrapper.fill(obj.group_description))
     214 + if obj.translated_description != obj.group_description:
     215 + color_print_green(" ├ ", td_wrapper.fill(obj.translated_description))
     216 + color_print_green(
     217 + " ├ Total participants: ", str(obj.total_participants)
     218 + )
     219 + 
     220 + if type == "group_recap":
     221 + color_print_green(
     222 + " ├ Participants found: ",
     223 + str(obj.found_participants)
     224 + + " ("
     225 + + str(format(obj.found_percentage, ".2f"))
     226 + + "%)",
     227 + )
     228 + 
     229 + color_print_green(" ├ Username: ", str(obj.group_username))
     230 + color_print_green(" ├ URL: ", str(obj.group_url))
     231 + color_print_green(" ├ Chat type: ", str(obj.chat_type))
     232 + color_print_green(" ├ Chat id: ", str(obj.id))
     233 + color_print_green(" ├ Access hash: ", str(obj.access_hash))
     234 + if type == "channel_recap":
     235 + scam_status = str(obj.scam)
     236 + color_print_green(" ├ Scam: ", str(scam_status))
     237 + color_print_green(" ├ First post date: ", str(obj.first_post))
     238 + if type == "group_recap":
     239 + color_print_green(
     240 + " ├ Memberlist saved to: ", obj.memberlist_filename
     241 + )
     242 + color_print_green(
     243 + " └ Restrictions: ", (str(obj.group_status))
     244 + )
     245 + 
     246 + if type == "group_stat":
     247 + color_print_green(" [+] Chat archive saved", "")
     248 + color_print_green(" ┬ Chat statistics", "")
     249 + color_print_green(
     250 + " ├ Number of messages found: ", str(obj.messages_found)
     251 + )
     252 + color_print_green(
     253 + " ├ Top poster 1: ", str(obj.poster_one)
     254 + )
     255 + color_print_green(
     256 + " ├ Top poster 2: ", str(obj.poster_two)
     257 + )
     258 + color_print_green(
     259 + " ├ Top poster 3: ", str(obj.poster_three)
     260 + )
     261 + color_print_green(
     262 + " ├ Top poster 4: ", str(obj.poster_four)
     263 + )
     264 + color_print_green(
     265 + " ├ Top poster 5: ", str(obj.poster_five)
     266 + )
     267 + color_print_green(
     268 + " ├ Total unique posters: ", str(obj.unique_active)
     269 + )
     270 + color_print_green(
     271 + " └ Archive saved to: ", str(obj.file_archive)
     272 + )
     273 + return
     274 + 
     275 + if type == "channel_stat":
     276 + color_print_green(" [+] Channel archive saved", "")
     277 + color_print_green(" ┬ Channel statistics", "")
     278 + color_print_green(
     279 + " ├ Number of messages found: ", str(obj.messages_found)
     280 + )
     281 + color_print_green(
     282 + " └ Archive saved to: ", str(obj.file_archive)
     283 + )
     284 + return
     285 + 
     286 + if type == "reply_stat":
     287 + middle_char = "├"
     288 + if obj.user_replier_list_len == 0:
     289 + middle_char = "└"
     290 + 
     291 + color_print_green(" [+] Replies analysis ", "")
     292 + color_print_green(" ┬ Chat statistics", "")
     293 + color_print_green(
     294 + f" {middle_char} Archive of replies saved to: ",
     295 + str(obj.reply_file_archive),
     296 + )
     297 + if obj.user_replier_list_len > 0:
     298 + color_print_green(
     299 + " └ Active members list who replied to messages, saved to: ",
     300 + str(obj.reply_memberlist_filename),
     301 + )
     302 + color_print_green(
     303 + " ┬ Top replier 1: ", str(obj.replier_one)
     304 + )
     305 + color_print_green(
     306 + " ├ Top replier 2: ", str(obj.replier_two)
     307 + )
     308 + color_print_green(
     309 + " ├ Top replier 3: ", str(obj.replier_three)
     310 + )
     311 + color_print_green(
     312 + " ├ Top replier 4: ", str(obj.replier_four)
     313 + )
     314 + color_print_green(
     315 + " ├ Top replier 5: ", str(obj.replier_five)
     316 + )
     317 + color_print_green(
     318 + " └ Total unique repliers: ", str(obj.replier_unique)
     319 + )
     320 + 
     321 + if type == "forwarder_stat":
     322 + color_print_green(" [+] Forward scrape complete", "")
     323 + color_print_green(" ┬ Statistics", "")
     324 + color_print_green(
     325 + " ├ Forwarded messages found: ", str(obj.forward_count)
     326 + )
     327 + color_print_green(
     328 + " ├ Forwards from active public chats: ",
     329 + str(obj.forwards_found),
     330 + )
     331 + if hasattr(object, "private_count"):
     332 + color_print_green(
     333 + " ├ Forwards from private (or now private) chats: ",
     334 + str(obj.private_count),
     335 + )
     336 + color_print_green(
     337 + " ├ Unique forward sources: ", str(obj.unique_forwards)
     338 + )
     339 + color_print_green(
     340 + " ├ Top forward source 1: ", str(obj.forward_one)
     341 + )
     342 + color_print_green(
     343 + " ├ Top forward source 2: ", str(obj.forward_two)
     344 + )
     345 + color_print_green(
     346 + " ├ Top forward source 3: ", str(obj.forward_three)
     347 + )
     348 + color_print_green(
     349 + " ├ Top forward source 4: ", str(obj.forward_four)
     350 + )
     351 + color_print_green(
     352 + " ├ Top forward source 5: ", str(obj.forward_five)
     353 + )
     354 + color_print_green(" └ Edgelist saved to: ", obj.edgelist_file)
  • dist/telepathy-2.3.2-py2.py3-none-any.whl
    Binary file.
  • dist/telepathy-2.3.2.tar.gz
    Binary file.
  • ■ ■ ■ ■
    setup.py
    skipped 1 lines
    2 2   
    3 3  setup(
    4 4   name="telepathy",
    5  - version='2.2.58',
     5 + version='2.3.2',
    6 6   author='Jordan Wildon',
    7 7   author_email='[email protected]',
    8 8   packages=['telepathy'],
    skipped 23 lines
  • src/telepathy/__pycache__/telepathy.cpython-310.pyc
    Binary file.
  • ■ ■ ■ ■
    src/telepathy/const.py
    1 1  __author__ = "Jordan Wildon (@jordanwildon)"
    2 2  __license__ = "MIT License"
    3  -__version__ = "2.2.58"
     3 +__version__ = "2.3.2"
    4 4  __maintainer__ = "Jordan Wildon"
    5 5  __email__ = "[email protected]"
    6 6  __status__ = "Development"
    7 7   
    8 8  user_agent = [
     9 + 
    9 10   # Chrome
    10 11   "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36",
    11 12   "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36",
    skipped 5 lines
    17 18   "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36",
    18 19   "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36",
    19 20   "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36",
     21 + 
    20 22   # Firefox
    21 23   "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1)",
    22 24   "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko",
    skipped 8 lines
    31 33   "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)",
    32 34   "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)",
    33 35   "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
     36 + 
    34 37   # Safari
    35 38   "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) RockMelt/0.9.50.549 Chrome/10.0.648.205 Safari/534.16"
    36 39   "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18"
    skipped 21 lines
  • ■ ■ ■ ■ ■ ■
    src/telepathy/telepathy.py
    1 1  #!/usr/bin/python3
    2 2   
    3  - 
    4 3  """Telepathy cli interface:
    5 4   An OSINT toolkit for investigating Telegram chats.
    6 5  """
    7 6   
    8  -from tokenize import group
    9 7  import pandas as pd
    10 8  import datetime
    11  -import requests
    12  -import json
    13  -import random
    14  -import glob
    15  -import csv
    16 9  import os
    17 10  import getpass
    18 11  import click
    19 12  import re
    20 13  import time
    21 14   
    22  - 
    23 15  from telepathy.utils import (
    24 16   print_banner,
    25 17   color_print_green,
    skipped 6 lines
    32 24   createPlaceholdeCls
    33 25  )
    34 26   
    35  -from colorama import Fore, Back, Style
    36 27  from telethon.errors import SessionPasswordNeededError, ChannelPrivateError
    37 28  from telethon.tl.types import (
    38 29   InputPeerEmpty,
    skipped 9 lines
    48 39  from telethon import TelegramClient, functions, types, utils
    49 40  from telethon.utils import get_display_name, get_message_id
    50 41  from alive_progress import alive_bar
     42 +from colorama import Fore, Style
    51 43   
    52 44  @click.command()
    53 45  @click.option(
    54 46   "--target",
    55 47   "-t",
    56 48   #default="",
    57  - multiple=True,
    58  - help="Specifies a chat to investigate.",
    59  -)
     49 + multiple = True,
     50 + help = "Specifies a chat to investigate.",
     51 + )
    60 52  @click.option(
    61 53   "--comprehensive",
    62 54   "-c",
    63  - is_flag=True,
    64  - help="Comprehensive scan, includes archiving.",
    65  -)
     55 + is_flag = True,
     56 + help = "Comprehensive scan, includes archiving.",
     57 + )
    66 58  @click.option(
    67  - "--media", "-m", is_flag=True, help="Archives media in the specified chat."
    68  -)
    69  -@click.option("--forwards", "-f", is_flag=True, help="Scrapes forwarded messages.")
    70  -@click.option("--user", "-u", is_flag=True, help="Looks up a specified user ID.")
     59 + "--media",
     60 + "-m",
     61 + is_flag = True,
     62 + help = "Archives media in the specified chat."
     63 + )
    71 64  @click.option(
    72  - "--location", "-l", is_flag=True, help="Finds users near to specified coordinates."
    73  -)
     65 + "--forwards",
     66 + "-f",
     67 + is_flag = True,
     68 + help = "Scrapes forwarded messages."
     69 + )
     70 +@click.option(
     71 + "--user",
     72 + "-u",
     73 + is_flag = True,
     74 + help = "Looks up a specified user ID."
     75 + )
     76 +@click.option(
     77 + "--location",
     78 + "-l",
     79 + is_flag = True,
     80 + help = "Finds users near to specified coordinates."
     81 + )
     82 +@click.option(
     83 + "--alt",
     84 + "-a",
     85 + default = 0,
     86 + help = "Uses an alternative login."
     87 + )
    74 88  @click.option(
    75  - "--alt", "-a", is_flag=True, default=False, help="Uses an alternative login."
    76  -)
    77  -@click.option("--json", "-j", is_flag=True, default=False, help="Export to JSON.")
     89 + "--json",
     90 + "-j",
     91 + is_flag = True,
     92 + default = False,
     93 + help = "Export to JSON."
     94 + )
    78 95  @click.option(
    79 96   "--export",
    80 97   "-e",
    81  - is_flag=True,
    82  - default=False,
    83  - help="Export a list of chats your account is part of.",
    84  -)
     98 + is_flag = True,
     99 + default = False,
     100 + help = "Export a list of chats your account is part of.",
     101 + )
    85 102  @click.option(
    86 103   "--replies",
    87 104   "-r",
    88  - is_flag=True,
    89  - default=False,
    90  - help="Enable replies analysis in channels.",
    91  -)
     105 + is_flag = True,
     106 + default = False,
     107 + help = "Enable replies analysis in channels.",
     108 + )
     109 +@click.option(
     110 + "--translate",
     111 + "-tr",
     112 + is_flag = True,
     113 + default = False,
     114 + help = "Enable translation of chat content.",
     115 + )
     116 + 
    92 117  def cli(
    93  - target, comprehensive, media, forwards, user, location, alt, json, export, replies
    94  -):
     118 + target,
     119 + comprehensive,
     120 + media,
     121 + forwards,
     122 + user,
     123 + location,
     124 + alt,
     125 + json,
     126 + export,
     127 + replies,
     128 + translate
     129 + ):
     130 + 
    95 131   print_banner()
    96  - telepathy_file = "./telepathy_files/"
    97  - try:
    98  - os.makedirs(telepathy_file)
    99  - except FileExistsError:
    100  - pass
    101 132   
    102 133   # Defining default values
    103  - basic = False
    104  - comp_check = False
    105  - media_archive = False
    106  - forwards_check = False
    107  - forward_verify = False
    108  - reply_analysis = False
    109  - user_check = False
    110  - location_check = False
    111  - last_date = None
    112  - chunk_size = 1000
    113  - filetime = datetime.datetime.now().strftime("%Y_%m_%d-%H_%M")
    114  - filetime_clean = str(filetime)
    115  - 
    116  - # Will add more languages later
    117  - user_language = "en"
     134 + user_check = location_check = False
     135 + basic = True if target else False
     136 + reply_analysis = True if replies else False
     137 + forwards_check = True if forwards else False
     138 + comp_check = True if comprehensive else False
     139 + media_archive = True if media else False
     140 + json_check = True if json else False
     141 + translate_check = True if translate else False
     142 + last_date, chunk_size, user_language = None, 1000, 'en'
    118 143   
    119  - if target:
    120  - basic = True
    121  - if replies:
    122  - reply_analysis = True
    123  - if forwards:
    124  - forwards_check = True
    125 144   if user:
    126  - user_check = True
    127  - basic = False
     145 + user_check, basic = True, False
    128 146   if location:
    129  - location_check = True
    130  - basic = False
    131  - if comprehensive:
    132  - comp_check = True
    133  - if media:
    134  - media_archive = True
     147 + location_check, basic = True, False
    135 148   if export:
    136 149   t = " "
    137  - if alt:
    138  - alt_check = True
    139  - else:
    140  - alt_check = False
     150 + 
     151 + filetime = datetime.datetime.now().strftime("%Y_%m_%d-%H_%M")
     152 + filetime_clean = str(filetime)
     153 + 
     154 + # Defining file values
     155 + telepathy_file = "./telepathy_files/"
     156 + json_file = telepathy_file + "json_files/"
     157 + login = telepathy_file + "login.txt"
     158 + log_file = telepathy_file + "log.csv"
     159 + export_file = telepathy_file + "export.csv"
     160 + 
     161 + # Creating core data file
     162 + if not os.path.exists(telepathy_file):
     163 + os.makedirs(telepathy_file)
    141 164  
    142  - if json:
    143  - json_check = True
    144  - json_file = telepathy_file + "json_files/"
    145  - try:
    146  - os.makedirs(json_file)
    147  - except FileExistsError:
    148  - pass
    149  - else:
    150  - json_check = False
     165 + '''Start of API details'''
    151 166  
    152  - if alt_check == True:
    153  - login = telepathy_file + "login_alt.txt"
     167 + def login_function():
     168 + api_id = input("Please enter your API ID:\n")
     169 + api_hash = input("Please enter your API Hash:\n")
     170 + phone_number = input("Please enter your phone number:\n")
     171 + return api_id, api_hash, phone_number
    154 172   
    155  - if os.path.isfile(login) == False:
    156  - api_id = input(" Please enter your API ID:\n")
    157  - api_hash = input(" Please enter your API Hash:\n")
    158  - phone_number = input(" Please enter your phone number:\n")
    159  - with open(login, "w+", encoding="utf-8") as f:
    160  - f.write(api_id + "," + api_hash + "," + phone_number)
    161  - else:
    162  - with open(login, encoding="utf-8") as f:
    163  - details = f.read()
     173 + if os.path.isfile(login) == False:
     174 + api_id, api_hash, phone_number = login_function()
     175 + with open(login, "w+", encoding="utf-8") as f:
     176 + f.write(api_id + "," + api_hash + "," + phone_number + "\n")
     177 + else:
     178 + with open(login, encoding="utf-8") as file:
     179 + content = file.readlines()
     180 + if alt == 0:
     181 + details = content[0]
    164 182   api_id, api_hash, phone_number = details.split(sep=",")
    165  - else:
    166  - login = telepathy_file + "login.txt"
     183 + elif alt == 1:
     184 + try:
     185 + if content[1]:
     186 + details = content[1]
     187 + api_id, api_hash, phone_number = details.split(sep=",")
     188 + except:
     189 + print("Setting up alt 1: ")
     190 + api_id, api_hash, phone_number = login_function()
     191 + with open(login, "a+", encoding="utf-8") as file:
     192 + file.write(api_id + "," + api_hash + "," + phone_number + "\n")
     193 + elif alt == 2:
     194 + try:
     195 + if content[2]:
     196 + details = content[2]
     197 + api_id, api_hash, phone_number = details.split(sep=",")
     198 + except:
     199 + print("Setting up alt 2: ")
     200 + api_id, api_hash, phone_number = login_function()
     201 + with open(login, "a+", encoding="utf-8") as file:
     202 + file.write(api_id + "," + api_hash + "," + phone_number + "\n")
     203 + elif alt == 3:
     204 + try:
     205 + if content[3]:
     206 + details = content[3]
     207 + api_id, api_hash, phone_number = details.split(sep=",")
     208 + except:
     209 + print("Setting up alt 3: ")
     210 + api_id, api_hash, phone_number = login_function()
     211 + with open(login, "a+", encoding="utf-8") as file:
     212 + file.write(api_id + "," + api_hash + "," + phone_number + "\n")
     213 + elif alt == 4:
     214 + try:
     215 + if content[4]:
     216 + details = content[4]
     217 + api_id, api_hash, phone_number = details.split(sep=",")
     218 + except:
     219 + print("Setting up alt 4: ")
     220 + api_id, api_hash, phone_number = login_function()
     221 + with open(login, "a+", encoding="utf-8") as file:
     222 + file.write(api_id + "," + api_hash + "," + phone_number + "\n")
    167 223   
    168  - if os.path.isfile(login) == False:
    169  - api_id = input(" Please enter your API ID:\n")
    170  - api_hash = input(" Please enter your API Hash:\n")
    171  - phone_number = input(" Please enter your phone number:\n")
    172  - with open(login, "w+", encoding="utf-8") as f:
    173  - f.write(api_id + "," + api_hash + "," + phone_number)
    174  - else:
    175  - with open(login, encoding="utf-8") as f:
    176  - details = f.read()
    177  - api_id, api_hash, phone_number = details.split(sep=",")
     224 + '''End of API details'''
    178 225   
    179 226   client = TelegramClient(phone_number, api_id, api_hash)
    180 227   
    181 228   async def main():
    182 229   
    183 230   await client.connect()
     231 + 
    184 232   if not await client.is_user_authorized():
    185 233   await client.send_code_request(phone_number)
    186  - await client.sign_in(phone_number)
    187 234   try:
    188  - await client.sign_in(code=input(" Enter code: "))
     235 + await client.sign_in(
     236 + phone = phone_number,
     237 + code=input("Enter code: "),
     238 + )
    189 239   except SessionPasswordNeededError:
    190 240   await client.sign_in(
    191  - password=getpass.getpass(prompt="Password: ", stream=None)
    192  - )
     241 + password=getpass.getpass(
     242 + prompt="Password: ",
     243 + stream=None,
     244 + )
     245 + )
     246 + 
    193 247   result = client(
    194 248   GetDialogsRequest(
    195 249   offset_date=last_date,
    skipped 1 lines
    197 251   offset_peer=InputPeerEmpty(),
    198 252   limit=chunk_size,
    199 253   hash=0,
     254 + )
    200 255   )
    201  - )
    202  - else:
    203 256   
     257 + else:
    204 258   if export == True:
    205  - export_file = telepathy_file + "export.csv"
    206 259   exports = []
    207  - 
    208 260   print("Exporting...")
    209  - # progress bar
     261 + 
    210 262   for Dialog in await client.get_dialogs():
    211 263   try:
    212 264   if Dialog.entity.username:
    skipped 4 lines
    217 269   group_description = web_req["group_description"]
    218 270   total_participants = web_req["total_participants"]
    219 271   
    220  - _desc = process_description(
    221  - group_description, user_language
    222  - )
    223  - translated_description = _desc["translated_text"]
     272 + if translate_check == True:
     273 + _desc = process_description(
     274 + group_description, user_language
     275 + )
     276 + translated_description = _desc["translated_text"]
     277 + else:
     278 + translated_description = "N/A"
    224 279   
    225 280   if Dialog.entity.broadcast is True:
    226 281   chat_type = "Channel"
    skipped 56 lines
    283 338   )
    284 339   
    285 340   if not os.path.isfile(export_file):
    286  - export_df.to_csv(export_file, sep=";", index=False)
     341 + export_df.to_csv(
     342 + export_file,
     343 + sep=";",
     344 + index=False,
     345 + )
    287 346   else:
    288 347   export_df.to_csv(
    289  - export_file, sep=";", mode="w", index=False
    290  - )
     348 + export_file,
     349 + sep=";",
     350 + mode="w",
     351 + index=False,
     352 + )
    291 353   
    292 354   except AttributeError:
    293 355   pass
     356 + 
    294 357   else:
    295 358   for t in target:
    296  - target_clean = t
    297 359   alphanumeric = ""
    298  - 
    299  - for character in target_clean:
     360 + for character in t:
    300 361   if character.isalnum():
    301 362   alphanumeric += character
    302 363   
    303 364   if "https://t.me/+" in t:
    304 365   t = t.replace('https://t.me/+', 'https://t.me/joinchat/')
    305 366   
    306  - if basic is True or comp_check is True:
     367 + if basic == True or comp_check == True:
    307 368   save_directory = telepathy_file + alphanumeric
    308  - try:
     369 + if not os.path.exists(save_directory):
    309 370   os.makedirs(save_directory)
    310  - except FileExistsError:
    311  - pass
    312  - 
    313  - # Creating logfile
    314  - log_file = telepathy_file + "log.csv"
    315 371   
    316 372   if media_archive:
    317 373   media_directory = save_directory + "/media"
    318  - try:
     374 + if not os.path.exists(media_directory):
    319 375   os.makedirs(media_directory)
    320  - except FileExistsError:
    321  - pass
    322 376   
    323 377   if basic == True and comp_check == False:
    324 378   color_print_green(" [!] ", "Performing basic scan")
    325 379   elif comp_check == True:
    326 380   color_print_green(" [!] ", "Performing comprehensive scan")
     381 + 
    327 382   file_archive = (
    328 383   save_directory
    329 384   + "/"
    skipped 2 lines
    332 387   + filetime_clean
    333 388   + "_archive.csv"
    334 389   )
     390 + 
    335 391   reply_file_archive = (
    336 392   save_directory
    337 393   + "/"
    skipped 13 lines
    351 407   + filetime_clean
    352 408   + "_edgelist.csv"
    353 409   )
     410 + 
    354 411   forward_directory = save_directory + "/edgelists/"
    355  - 
    356  - try:
     412 + if not os.path.exists(forward_directory):
    357 413   os.makedirs(forward_directory)
    358  - except FileExistsError:
    359  - pass
    360 414   
    361 415   edgelist_file = (
    362  - forward_directory + "/" + alphanumeric + "_edgelist.csv"
     416 + forward_directory
     417 + + "/"
     418 + + alphanumeric
     419 + + "_edgelist.csv"
    363 420   )
    364 421   
    365 422   if basic is True or comp_check is True:
    366 423   
    367 424   color_print_green(" [-] ", "Fetching details for " + t + "...")
    368  - memberlist_directory = save_directory + "/memberlists"
    369 425   
    370  - try:
     426 + memberlist_directory = save_directory + "/memberlists"
     427 + if not os.path.exists(memberlist_directory):
    371 428   os.makedirs(memberlist_directory)
    372  - except FileExistsError:
    373  - pass
    374 429   
    375 430   memberlist_filename = (
    376  - memberlist_directory + "/" + alphanumeric + "_members.csv"
     431 + memberlist_directory
     432 + + "/"
     433 + + alphanumeric
     434 + + "_members.csv"
    377 435   )
     436 + 
    378 437   reply_memberlist_filename = (
    379 438   memberlist_directory
    380 439   + "/"
    skipped 2 lines
    383 442   )
    384 443   
    385 444   entity = await client.get_entity(t)
     445 +
    386 446   first_post = "Not found"
    387 447   
    388 448   async for message in client.iter_messages(t, reverse=True):
    skipped 12 lines
    401 461   web_req = parse_html_page(group_url)
    402 462   group_username = "Private group"
    403 463   else:
    404  - group_url = "Private group"
    405  - group_username = "Private group"
    406  - 
     464 + group_url, group_username = "Private group", "Private group"
    407 465   
    408 466   group_description = web_req["group_description"]
    409 467   total_participants = web_req["total_participants"]
    410 468   
    411  - _desc = process_description(
    412  - group_description, user_language
     469 + if translate_check == True:
     470 + _desc = process_description(
     471 + group_description, user_language
    413 472   )
    414 473   
    415  - original_language = _desc[
    416  - "original_language"
    417  - ]
     474 + original_language = _desc[
     475 + "original_language"
     476 + ]
     477 + translated_description = _desc["translated_text"]
     478 + else:
     479 + translated_description = "N/A"
    418 480   
    419  - translated_description = _desc["translated_text"]
    420 481   group_description = ('"' + group_description + '"')
    421 482   
    422 483   if(entity.__class__ == User):
    423  - color_print_green(" [!] ", "U can't search user using flag -c, run Telepathy using the flag -u.")
     484 + color_print_green(" [!] ", "You can't search for users using flag -c, run Telepathy using the flag -u.")
    424 485   exit(1)
    425 486   
    426 487   if entity.broadcast is True:
    skipped 10 lines
    437 498   if 1 in entity.restriction_reason:
    438 499   android_restriction = entity.restriction_reason[1]
    439 500   group_status = (
    440  - str(ios_restriction) + ", " + str(android_restriction)
     501 + str(ios_restriction)
     502 + + ", "
     503 + + str(android_restriction)
    441 504   )
    442 505   else:
    443 506   group_status = str(ios_restriction)
    444 507   else:
    445 508   group_status = "None"
    446 509   
    447  - found_participants = 0
    448  - found_percentage = 0
     510 + found_participants, found_percentage = 0, 0
     511 + 
    449 512   if chat_type != "Channel":
    450 513   members = []
     514 + members_df = None
    451 515   all_participants = await client.get_participants(t, limit=5000)
    452  - members_df = None
     516 + 
    453 517   for user in all_participants:
    454 518   members_df = pd.DataFrame(
    455 519   members,
    skipped 14 lines
    470 534   members_df.to_csv(save_members, sep=";")
    471 535   
    472 536   if json_check == True:
     537 + if not os.path.exists(json_file):
     538 + os.makedirs(json_file)
     539 + 
    473 540   members_df.to_json(
    474 541   json_file + alphanumeric + "_memberlist.json",
    475 542   orient="records",
    skipped 1 lines
    477 544   lines=True,
    478 545   index=True,
    479 546   )
    480  - else:
    481  - pass
    482 547   
    483 548   found_participants = len(all_participants)
    484 549   found_participants = int(found_participants)
    skipped 6 lines
    491 556   if chat_type != "Channel":
    492 557   print("\n")
    493 558   color_print_green(" [+] Memberlist fetched", "")
    494  - else:
    495  - pass
    496 559  
    497 560   setattr(entity, "group_description", group_description)
    498 561   setattr(entity, "group_status", group_status)
    skipped 3 lines
    502 565   setattr(entity, "chat_type", chat_type)
    503 566   setattr(entity, "translated_description", translated_description)
    504 567   setattr(entity, "total_participants", total_participants)
     568 + 
    505 569   if chat_type != "Channel":
    506 570   setattr(entity, "found_participants", found_participants)
    507 571   setattr(entity, "found_percentage", found_percentage)
    skipped 1 lines
    509 573   else:
    510 574   setattr(entity, "found_participants", found_participants)
    511 575   print_flag = "group_recap"
     576 + 
    512 577   if chat_type == "Channel":
    513 578   print_flag = "channel_recap"
    514 579   
    skipped 73 lines
    588 653   if message.forward is not None:
    589 654   forward_count += 1
    590 655   
    591  - #print("\n")
    592 656   color_print_green(" [-] ", "Fetching forwarded messages...")
    593 657   
    594 658   progress_bar = (
    skipped 17 lines
    612 676   
    613 677   substring = "PeerUser"
    614 678   string = str(f_from_id)
    615  - if substring in string:
    616  - user_id = re.sub("[^0-9]", "", string)
    617  - user_id = await client.get_entity(
    618  - PeerUser(int(user_id))
    619  - )
    620  - user_id = str(user_id)
    621  - result = (
    622  - "User: "
    623  - + str(ent.first_name)
    624  - + " / ID: "
    625  - + str(user_id.id)
    626  - )
     679 + if chat_type != "Channel":
     680 + if substring in string:
     681 + user_id = re.sub("[^0-9]", "", string)
     682 + user_id = await client.get_entity(
     683 + PeerUser(int(user_id))
     684 + )
     685 + user_id = str(user_id)
     686 + result = (
     687 + "User: "
     688 + + str(ent.first_name)
     689 + + " / ID: "
     690 + + str(user_id.id)
     691 + )
     692 + else:
     693 + result = str(ent.title)
    627 694   else:
    628 695   result = str(ent.title)
    629 696   
    skipped 41 lines
    671 738   lines=True,
    672 739   index=True,
    673 740   )
    674  - else:
    675  - pass
    676 741   
    677 742   if forward_count >= 15:
    678 743   forwards_found = forwards_df.Source.count()
    skipped 1 lines
    680 745   df01 = value_count.rename_axis("unique_values").reset_index(
    681 746   name="counts"
    682 747   )
     748 + 
    683 749   report_forward = createPlaceholdeCls()
    684 750   report_forward.forward_one = (
    685 751   str(df01.iloc[0]["unique_values"])
    skipped 2 lines
    688 754   + " forwarded messages"
    689 755   )
    690 756   report_forward.forward_two = (
    691  - str(df01.iloc[1]["unique_values"])
    692  - + ", "
    693  - + str(df01.iloc[1]["counts"])
    694  - + " forwarded messages"
     757 + str(df01.iloc[1]["unique_values"])
     758 + + ", "
     759 + + str(df01.iloc[1]["counts"])
     760 + + " forwarded messages"
    695 761   )
    696 762   report_forward.forward_three = (
    697  - str(df01.iloc[2]["unique_values"])
    698  - + ", "
    699  - + str(df01.iloc[2]["counts"])
    700  - + " forwarded messages"
     763 + str(df01.iloc[2]["unique_values"])
     764 + + ", "
     765 + + str(df01.iloc[2]["counts"])
     766 + + " forwarded messages"
    701 767   )
    702 768   report_forward.forward_four = (
    703  - str(df01.iloc[3]["unique_values"])
    704  - + ", "
    705  - + str(df01.iloc[3]["counts"])
    706  - + " forwarded messages"
     769 + str(df01.iloc[3]["unique_values"])
     770 + + ", "
     771 + + str(df01.iloc[3]["counts"])
     772 + + " forwarded messages"
    707 773   )
    708 774   report_forward.forward_five = (
    709  - str(df01.iloc[4]["unique_values"])
    710  - + ", "
    711  - + str(df01.iloc[4]["counts"])
    712  - + " forwarded messages"
     775 + str(df01.iloc[4]["unique_values"])
     776 + + ", "
     777 + + str(df01.iloc[4]["counts"])
     778 + + " forwarded messages"
    713 779   )
     780 + 
    714 781   df02 = forwards_df.Source.unique()
    715 782   report_forward.unique_forwards = len(df02)
    716 783   report_forward.edgelist_file = edgelist_file
    717 784   print_shell("forwarder_stat",report_forward)
    718  - 
    719 785   else:
    720 786   print(
    721 787   "\n"
    skipped 3 lines
    725 791   )
    726 792   
    727 793   else:
    728  - 
    729 794   if comp_check is True:
    730 795   
    731 796   messages = client.iter_messages(t)
     797 + 
    732 798   message_list = []
    733 799   forwards_list = []
    734  - user_reaction_list = []
     800 + 
    735 801   replies_list = []
    736 802   user_replier_list = []
    737  - timecount = []
    738  - forward_count = 0
    739  - private_count = 0
     803 + 
     804 + forward_count, private_count, message_count = 0, 0, 0
    740 805   
    741 806   if media_archive is True:
    742 807   files = []
    skipped 5 lines
    748 813   color_print_green(
    749 814   " [!] ", "Calculating number of messages..."
    750 815   )
    751  - 
    752  - message_count = 0
    753 816   
    754 817   async for message in messages:
    755 818   if message is not None:
    skipped 18 lines
    774 837   t, limit=None
    775 838   ):
    776 839   if message is not None:
    777  - 
    778 840   try:
    779  - 
    780 841   c_archive = pd.DataFrame(
    781 842   message_list,
    782 843   columns=[
    783 844   "To",
    784 845   "Message ID",
    785 846   "Display_name",
    786  - "ID",
     847 + "User ID",
    787 848   "Message_text",
    788 849   "Original_language",
    789 850   "Translated_text",
    790 851   "Translation_confidence",
    791 852   "Timestamp",
    792 853   "Has_media",
    793  - "Reply",
     854 + "Reply_to_ID",
     855 + "Replies",
     856 + "Forwards",
    794 857   "Views",
     858 + "Total_reactions",
     859 + "Reply_ER_reach",
     860 + "Reply_ER_impressions",
     861 + "Forwards_ER_reach",
     862 + "Forwards_ER_impressions",
     863 + "Reaction_ER_reach",
     864 + "Reactions_ER_impressions",
     865 + "Thumbs_up",
     866 + "Thumbs_down",
     867 + "Heart",
     868 + "Fire",
     869 + "Smile_with_hearts",
     870 + "Clap",
     871 + "Smile",
     872 + "Thinking",
     873 + "Exploding_head",
     874 + "Scream",
     875 + "Angry",
     876 + "Single_tear",
     877 + "Party",
     878 + "Starstruck",
     879 + "Vomit",
     880 + "Poop",
     881 + "Pray",
     882 + "Edit_date",
    795 883   "URL",
     884 + "Media save directory"
    796 885   ],
    797 886   )
    798 887   
    skipped 46 lines
    845 934   "Message ID",
    846 935   "Reply ID",
    847 936   "Display_name",
    848  - "User ID",
     937 + "ID",
    849 938   "Message_text",
    850 939   "Original_language",
    851 940   "Translated_text",
    skipped 8 lines
    860 949   message.chat_id,
    861 950   reply_to=message.id,
    862 951   ):
     952 + 
    863 953   user = await client.get_entity(
    864 954   repl.from_id.user_id
    865 955   )
     956 + 
    866 957   userdet = populate_user(user, t)
    867 958   user_replier_list.append(
    868 959   userdet
    869 960   )
    870  - mss_txt = process_message(
    871  - repl.text, user_language
    872  - )
     961 + 
     962 + if translate_check == True:
     963 + mss_txt = process_message(
     964 + repl.text, user_language
     965 + )
     966 + original_language = mss_txt["original_language"],
     967 + translated_text = mss_txt["translated_text"],
     968 + translation_confidence = mss_txt["translation_confidence"],
     969 + reply_text = mss_txt["message_text"]
     970 + else:
     971 + original_language = "N/A"
     972 + translated_text = "N/A"
     973 + translation_confidence = "N/A"
     974 + reply_text = repl.text
     975 + 
    873 976   replies_list.append(
    874 977   [
    875 978   t,
    skipped 1 lines
    877 980   repl.id,
    878 981   userdet[1],
    879 982   userdet[2],
    880  - mss_txt["message_text"],
    881  - mss_txt[
    882  - "original_language"
    883  - ],
    884  - mss_txt[
    885  - "translated_text"
    886  - ],
    887  - mss_txt[
    888  - "translation_confidence"
    889  - ],
     983 + reply_text,
     984 + original_language,
     985 + translated_text,
     986 + translation_confidence,
    890 987   parse_tg_date(
    891 988   repl.date
    892 989   )["timestamp"],
    skipped 21 lines
    914 1011   ]
    915 1012   reply = message.reply_to_msg_id
    916 1013   
    917  - _mess = process_message(
    918  - message.text, user_language
    919  - )
    920  - message_text = _mess["message_text"]
    921  - original_language = _mess[
    922  - "original_language"
    923  - ]
    924  - translated_text = _mess["translated_text"]
    925  - translation_confidence = _mess[
    926  - "translation_confidence"
    927  - ]
     1014 + if translate_check == True:
     1015 + _mess = process_message(
     1016 + message.text, user_language
     1017 + )
     1018 + message_text = _mess["message_text"]
     1019 + original_language = _mess[
     1020 + "original_language"
     1021 + ]
     1022 + translated_text = _mess["translated_text"]
     1023 + translation_confidence = _mess[
     1024 + "translation_confidence"
     1025 + ]
     1026 + else:
     1027 + message_text = message.text
     1028 + original_language = "N/A"
     1029 + translated_text = "N/A"
     1030 + translation_confidence = "N/A"
    928 1031   
    929 1032   if message.forwards is not None:
    930 1033   forwards = int(message.forwards)
    931 1034   else:
    932  - forwards = "None"
     1035 + forwards = "N/A"
    933 1036   
    934 1037   if message.views is not None:
    935 1038   views = int(message.views)
    936 1039   else:
    937  - views = "Not found"
     1040 + views = 'N/A'
    938 1041   
    939  - #if message.reactions:
    940  - #if message.reactions.can_see_list:
    941  - #print(dir(message.reactions.results))
    942  - #print("#### TODO: REACTIONS")
     1042 + if message.reactions:
     1043 + reactions = message.reactions.results
     1044 + total_reactions = 0
     1045 + i = range(len(reactions))
     1046 +
     1047 + for idx, i in enumerate(reactions):
     1048 + total_reactions = total_reactions + i.count
     1049 + thumbs_up = i.count if i.reaction == '👍' else 0
     1050 + thumbs_down = i.count if i.reaction == '👎' else 0
     1051 + heart = i.count if i.reaction == '❤️' else 0
     1052 + fire = i.count if i.reaction == '🔥' else 0
     1053 + smile_with_hearts = i.count if i.reaction == '🥰' else 0
     1054 + clap = i.count if i.reaction == '👏' else 0
     1055 + smile = i.count if i.reaction == '😁' else 0
     1056 + thinking = i.count if i.reaction == '🤔' else 0
     1057 + exploding_head = i.count if i.reaction == '🤯' else 0
     1058 + scream = i.count if i.reaction == '😱' else 0
     1059 + angry = i.count if i.reaction == '🤬' else 0
     1060 + single_tear = i.count if i.reaction == '😢' else 0
     1061 + party_popper = i.count if i.reaction == '🎉' else 0
     1062 + starstruck = i.count if i.reaction == '🤩' else 0
     1063 + vomiting = i.count if i.reaction == '🤮' else 0
     1064 + poop = i.count if i.reaction == '💩' else 0
     1065 + praying = i.count if i.reaction == '🙏' else 0
     1066 + else:
     1067 + total_reactions = 'N/A'
     1068 + thumbs_up = 'N/A'
     1069 + thumbs_down = 'N/A'
     1070 + heart = 'N/A'
     1071 + fire = 'N/A'
     1072 + smile_with_hearts = 'N/A'
     1073 + clap = 'N/A'
     1074 + smile = 'N/A'
     1075 + thinking = 'N/A'
     1076 + exploding_head = 'N/A'
     1077 + scream = 'N/A'
     1078 + angry = 'N/A'
     1079 + single_tear = 'N/A'
     1080 + party_popper = 'N/A'
     1081 + starstruck = 'N/A'
     1082 + vomiting = 'N/A'
     1083 + poop = 'N/A'
     1084 + praying = 'N/A'
    943 1085   
    944 1086   if media_archive == True:
    945  - if message.media:
     1087 + if message.media is not None:
    946 1088   path = await message.download_media(
    947  - file=media_directory
     1089 + file = media_directory
    948 1090   )
    949 1091   files.append(path)
     1092 + media_file = path
    950 1093   else:
    951  - pass
     1094 + media_file = "N/A"
     1095 + else:
     1096 + media_file = "N/A"
    952 1097  
    953 1098   if message.media is not None:
    954 1099   has_media = "TRUE"
    955 1100   else:
    956  - has_media = 'FALSE'
     1101 + has_media = "FALSE"
     1102 + 
     1103 + if message.replies:
     1104 + reply_count = int(message.replies.replies)
     1105 + else:
     1106 + reply_count = "N/A"
     1107 + 
     1108 + if message.edit_date:
     1109 + edit_date = str(message.edit_date)
     1110 + else:
     1111 + edit_date = "None"
     1112 + 
     1113 + '''Need to find a way to calculate these in case these figures don't exist to make it
     1114 + comparable across channels for a total engagement number (e.g. if replies/reactions are off).
     1115 + If not N/A would cover if it's off, zero if it's none. Working on some better logic here.'''
     1116 + 
     1117 + if reply_count != 'N/A' and total_participants is not None:
     1118 + reply_reach_ER = (reply_count / int(total_participants)) * 100
     1119 + else:
     1120 + reply_reach_ER = 'N/A'
     1121 + 
     1122 + if reply_count != 'N/A' and views != 'N/A':
     1123 + reply_impressions_ER = (reply_count / int(views)) * 100
     1124 + else:
     1125 + reply_impressions_ER = 'N/A'
     1126 + 
     1127 + if forwards != 'N/A' and total_participants is not None:
     1128 + forwards_reach_ER = (forwards / int(total_participants)) * 100
     1129 + else:
     1130 + forwards_reach_ER = 'N/A'
     1131 + 
     1132 + if forwards != 'N/A' and views != 'N/A':
     1133 + forwards_impressions_ER = (forwards / int(views)) * 100
     1134 + else:
     1135 + forwards_impressions_ER = 'N/A'
     1136 + 
     1137 + if total_reactions != 'N/A' and total_participants is not None:
     1138 + reactions_reach_ER = (total_reactions / int(total_participants)) * 100
     1139 + else:
     1140 + reactions_reach_ER = 'N/A'
     1141 + 
     1142 + if total_reactions != 'N/A' and views != 'N/A':
     1143 + reactions_impressions_ER = (total_reactions / int(views)) * 100
     1144 + else:
     1145 + reactions_impressions_ER = 'N/A'
    957 1146   
    958 1147   post_url = "https://t.me/s/" + t + "/" + str(message.id)
    959 1148   
    skipped 10 lines
    970 1159   timestamp,
    971 1160   has_media,
    972 1161   reply,
     1162 + reply_count,
     1163 + forwards,
    973 1164   views,
     1165 + total_reactions,
     1166 + reply_reach_ER,
     1167 + reply_impressions_ER,
     1168 + forwards_reach_ER,
     1169 + forwards_impressions_ER,
     1170 + reactions_reach_ER,
     1171 + reactions_impressions_ER,
     1172 + thumbs_up,
     1173 + thumbs_down,
     1174 + heart,
     1175 + fire,
     1176 + smile_with_hearts,
     1177 + clap,
     1178 + smile,
     1179 + thinking,
     1180 + exploding_head,
     1181 + scream,
     1182 + angry,
     1183 + single_tear,
     1184 + party_popper,
     1185 + starstruck,
     1186 + vomiting,
     1187 + poop,
     1188 + praying,
     1189 + edit_date,
    974 1190   post_url,
     1191 + media_file,
    975 1192   ]
    976 1193   )
    977 1194   
    978 1195   if message.forward is not None:
    979  - forward_verify = True
    980 1196   try:
    981 1197   forward_count += 1
    982 1198   to_title = to_ent.title
    skipped 5 lines
    988 1204   ent = await client.get_entity(
    989 1205   f_from_id
    990 1206   )
     1207 + 
    991 1208   user_string = "user_id"
    992 1209   channel_string = "broadcast"
    993 1210   
    skipped 7 lines
    1001 1218   ent.broadcast
    1002 1219   is True
    1003 1220   ):
    1004  - ent_type = "Channel"
     1221 + ent_type = (
     1222 + "Channel"
     1223 + )
    1005 1224   elif (
    1006 1225   ent.megagroup
    1007 1226   is True
    skipped 89 lines
    1097 1316   "None",
    1098 1317   "None",
    1099 1318   "None",
     1319 + "None",
     1320 + "None",
     1321 + "None",
     1322 + "None",
     1323 + "None",
     1324 + "None",
     1325 + "None",
     1326 + "None",
     1327 + "None",
     1328 + "None",
     1329 + "None",
     1330 + "None",
     1331 + "None",
     1332 + "None",
     1333 + "None",
     1334 + "None",
     1335 + "None",
     1336 + "None",
     1337 + "None",
     1338 + "None",
     1339 + "None",
     1340 + "None",
     1341 + "None",
     1342 + "None",
     1343 + "None",
     1344 + "None",
     1345 + "None",
    1100 1346   ]
    1101 1347   )
    1102  - pass
    1103 1348   
    1104 1349   time.sleep(0.5)
    1105 1350   bar()
    skipped 18 lines
    1124 1369   
    1125 1370   if json_check == True:
    1126 1371   c_archive.to_json(
    1127  - json_file + alphanumeric + "_archive.json",
     1372 + json_file
     1373 + + alphanumeric
     1374 + + "_archive.json",
    1128 1375   orient="records",
    1129 1376   compression="infer",
    1130 1377   lines=True,
    1131 1378   index=True,
    1132 1379   )
    1133  - else:
    1134  - pass
    1135 1380   
    1136 1381   if forwards_check is True:
    1137 1382   with open(
    skipped 3 lines
    1141 1386   
    1142 1387   if json_check == True:
    1143 1388   c_forwards.to_json(
    1144  - json_file + alphanumeric + "_edgelist.json",
     1389 + json_file
     1390 + + alphanumeric
     1391 + + "_edgelist.json",
    1145 1392   orient="records",
    1146 1393   compression="infer",
    1147 1394   lines=True,
    1148 1395   index=True,
    1149 1396   )
    1150  - else:
    1151  - pass
    1152  - else:
    1153  - pass
    1154 1397   
    1155 1398   messages_found = int(c_archive.To.count()) - 1
    1156 1399   report_obj = createPlaceholdeCls()
    skipped 23 lines
    1180 1423   + str(df03.iloc[0]["counts"])
    1181 1424   + " messages"
    1182 1425   )
    1183  - 
    1184 1426   report_obj.poster_two = (
    1185 1427   str(df03.iloc[1]["unique_values"])
    1186 1428   + ", "
    1187  - + str(df03.iloc[2]["counts"])
     1429 + + str(df03.iloc[1]["counts"])
    1188 1430   + " messages"
    1189 1431   )
    1190  - 
    1191 1432   report_obj.poster_three = (
    1192 1433   str(df03.iloc[2]["unique_values"])
    1193 1434   + ", "
    1194 1435   + str(df03.iloc[2]["counts"])
    1195 1436   + " messages"
    1196 1437   )
    1197  - 
    1198 1438   report_obj.poster_four = (
    1199 1439   str(df03.iloc[3]["unique_values"])
    1200 1440   + ", "
    1201  - + df03.iloc[3]["counts"]
     1441 + + str(df03.iloc[3]["counts"])
    1202 1442   + " messages"
    1203 1443   )
    1204  - 
    1205  - report_obj.poster_five = (
     1444 + report_obj.poster_four = (
    1206 1445   str(df03.iloc[4]["unique_values"])
    1207 1446   + ", "
    1208 1447   + str(df03.iloc[4]["counts"])
    skipped 6 lines
    1215 1454   print_shell("group_stat", report_obj)
    1216 1455   
    1217 1456   if reply_analysis is True:
    1218  - 
    1219 1457   if len(replies_list) > 0:
    1220 1458   replier_value_count = c_repliers["User ID"].value_counts()
    1221 1459   replier_df = replier_value_count.rename_axis(
    skipped 7 lines
    1229 1467   + str(replier_df.iloc[0]["counts"])
    1230 1468   + " replies"
    1231 1469   )
    1232  - 
    1233 1470   repliers.replier_two = (
    1234 1471   str(replier_df.iloc[1]["unique_values"])
    1235 1472   + ", "
    1236 1473   + str(replier_df.iloc[1]["counts"])
    1237 1474   + " replies"
    1238 1475   )
    1239  - 
    1240 1476   repliers.replier_three = (
    1241 1477   str(replier_df.iloc[2]["unique_values"])
    1242 1478   + ", "
    1243 1479   + str(replier_df.iloc[2]["counts"])
    1244 1480   + " replies"
    1245 1481   )
    1246  - 
    1247 1482   repliers.replier_four = (
    1248 1483   str(replier_df.iloc[3]["unique_values"])
    1249 1484   + ", "
    1250 1485   + str(replier_df.iloc[3]["counts"])
    1251 1486   + " replies"
    1252 1487   )
    1253  - 
    1254 1488   repliers.replier_five = (
    1255  - str(replier_df.iloc[3]["counts"])
     1489 + str(replier_df.iloc[4]["unique_values"])
    1256 1490   + ", "
    1257 1491   + str(replier_df.iloc[4]["counts"])
    1258 1492   + " replies"
    skipped 1 lines
    1260 1494   
    1261 1495   replier_count_df = c_repliers["User ID"].unique()
    1262 1496   replier_unique = len(replier_count_df)
    1263  - 
    1264 1497   repliers.user_replier_list_len = len(user_replier_list)
    1265 1498   repliers.reply_file_archive = str(reply_file_archive)
    1266 1499   repliers.reply_memberlist_filename = str(reply_memberlist_filename)
    skipped 7 lines
    1274 1507   c_f_stats = value_count.rename_axis(
    1275 1508   "unique_values"
    1276 1509   ).reset_index(name="counts")
     1510 + 
    1277 1511   report_forward = createPlaceholdeCls()
    1278 1512   report_forward.forward_one = (
    1279  - str(c_f_stats.iloc[0]["unique_values"])
    1280  - + ", "
    1281  - + str(c_f_stats.iloc[0]["counts"])
    1282  - + " forwarded messages"
     1513 + str(c_f_stats.iloc[0]["unique_values"])
     1514 + + ", "
     1515 + + str(c_f_stats.iloc[0]["counts"])
     1516 + + " forwarded messages"
    1283 1517   )
    1284 1518   report_forward.forward_two = (
    1285  - str(c_f_stats.iloc[1]["unique_values"])
    1286  - + ", "
    1287  - + str(c_f_stats.iloc[1]["counts"])
    1288  - + " forwarded messages"
     1519 + str(c_f_stats.iloc[1]["unique_values"])
     1520 + + ", "
     1521 + + str(c_f_stats.iloc[1]["counts"])
     1522 + + " forwarded messages"
    1289 1523   )
    1290 1524   report_forward.forward_three = (
    1291  - str(c_f_stats.iloc[2]["unique_values"])
    1292  - + ", "
    1293  - + str(c_f_stats.iloc[2]["counts"])
    1294  - + " forwarded messages"
     1525 + str(c_f_stats.iloc[2]["unique_values"])
     1526 + + ", "
     1527 + + str(c_f_stats.iloc[2]["counts"])
     1528 + + " forwarded messages"
    1295 1529   )
    1296 1530   report_forward.forward_four = (
    1297  - str(c_f_stats.iloc[3]["unique_values"])
    1298  - + ", "
    1299  - + str(c_f_stats.iloc[3]["counts"])
    1300  - + " forwarded messages"
     1531 + str(c_f_stats.iloc[3]["unique_values"])
     1532 + + ", "
     1533 + + str(c_f_stats.iloc[3]["counts"])
     1534 + + " forwarded messages"
    1301 1535   )
    1302 1536   report_forward.forward_five = (
    1303  - str(c_f_stats.iloc[4]["unique_values"])
    1304  - + ", "
    1305  - + str(c_f_stats.iloc[4]["counts"])
    1306  - + " forwarded messages"
     1537 + str(c_f_stats.iloc[4]["unique_values"])
     1538 + + ", "
     1539 + + str(c_f_stats.iloc[4]["counts"])
     1540 + + " forwarded messages"
    1307 1541   )
    1308 1542   
    1309 1543   c_f_unique = c_forwards.Source.unique()
     1544 + 
    1310 1545   report_forward.unique_forwards = len(c_f_unique)
    1311 1546   report_forward.edgelist_file = edgelist_file
    1312 1547   report_forward.private_count = private_count
    1313 1548   print_shell("forwarder_stat", report_forward)
     1549 + 
    1314 1550   else:
    1315  - #print("\n")
    1316 1551   color_print_green(
    1317 1552   " [!] Insufficient forwarded messages found",
    1318 1553   edgelist_file,
    1319 1554   )
    1320  - else:
    1321  - pass
    1322 1555   
    1323 1556   if user_check == True:
    1324 1557   my_user = None
    skipped 8 lines
    1333 1566   user_last_name = my_user.last_name
    1334 1567   if user_last_name is not None:
    1335 1568   user_full_name = (
    1336  - str(user_first_name) + " " + str(user_last_name)
     1569 + str(user_first_name)
     1570 + + " "
     1571 + + str(user_last_name)
    1337 1572   )
    1338 1573   else:
    1339 1574   user_full_name = str(user_first_name)
    skipped 3 lines
    1343 1578   else:
    1344 1579   user_photo = "None"
    1345 1580   
     1581 + if my_user.status is not None:
     1582 + if "Empty" in str(my_user.status):
     1583 + user_status = "Last seen over a month ago"
     1584 + elif "Month" in str(my_user.status):
     1585 + user_status = "Between a week and a month"
     1586 + elif "Week" in str(my_user.status):
     1587 + user_status = "Between three and seven days"
     1588 + elif "Offline" in str(my_user.status):
     1589 + user_status = "Offline"
     1590 + elif "Online" in str(my_user.status):
     1591 + user_status = "Online"
     1592 + elif "Recently" in str(my_user.status):
     1593 + user_status = "Recently (within two days)"
     1594 + else:
     1595 + user_status = "Not found"
     1596 + 
    1346 1597   if my_user.restriction_reason is not None:
    1347 1598   ios_restriction = entity.restriction_reason[0]
    1348 1599   if 1 in entity.restriction_reason:
    skipped 7 lines
    1356 1607   user_restrictions = str(ios_restriction)
    1357 1608   else:
    1358 1609   user_restrictions = "None"
     1610 + 
    1359 1611   setattr(my_user, "user_restrictions", str(user_restrictions))
    1360 1612   setattr(my_user, "user_full_name", str(user_full_name))
    1361 1613   setattr(my_user, "user_photo", str(user_photo))
     1614 + setattr(my_user, "user_status", str(user_status))
    1362 1615   setattr(my_user, "target", t)
    1363 1616   print_shell("user", my_user)
    1364 1617   
    1365 1618   except ValueError:
    1366 1619   pass
     1620 + 
    1367 1621   if my_user is None:
    1368 1622   print(
    1369 1623   Fore.GREEN
    skipped 4 lines
    1374 1628   
    1375 1629   if location_check == True:
    1376 1630   
    1377  - location = t
    1378  - 
    1379 1631   print(
    1380 1632   Fore.GREEN
    1381 1633   + " [!] "
    1382 1634   + Style.RESET_ALL
    1383 1635   + "Searching for users near "
    1384  - + location
     1636 + + t
    1385 1637   + "\n"
    1386 1638   )
    1387  - latitude, longitude = location.split(sep=",")
    1388 1639   
    1389  - locations_file = telepathy_file + "locations/"
     1640 + latitude, longitude = t.split(sep=",")
    1390 1641   
    1391  - try:
     1642 + locations_file = telepathy_file + "locations/"
     1643 + if not os.path.exists(locations_file):
    1392 1644   os.makedirs(locations_file)
    1393  - except FileExistsError:
    1394  - pass
    1395 1645   
    1396 1646   save_file = (
    1397 1647   locations_file
    skipped 7 lines
    1405 1655   )
    1406 1656   
    1407 1657   locations_list = []
     1658 + l_save_list = []
     1659 + 
    1408 1660   result = await client(
    1409 1661   functions.contacts.GetLocatedRequest(
    1410 1662   geo_point=types.InputGeoPoint(
    skipped 5 lines
    1416 1668   )
    1417 1669   )
    1418 1670   
    1419  - # progress bar?
    1420  - 
    1421 1671   for user in result.updates[0].peers:
    1422 1672   try:
    1423 1673   user_df = pd.DataFrame(
    1424  - locations_list, columns=["User_ID", "Distance"]
     1674 + locations_list, columns=[
     1675 + "User_ID",
     1676 + "Distance"]
     1677 + )
     1678 + 
     1679 + l_save_df = pd.DataFrame(
     1680 + l_save_list, columns=[
     1681 + "User_ID",
     1682 + "Distance",
     1683 + "Latitude",
     1684 + "Longitude",
     1685 + "Date_retrieved"
     1686 + ]
    1425 1687   )
     1688 + 
    1426 1689   if hasattr(user, "peer"):
    1427 1690   ID = user.peer.user_id
    1428  - else:
    1429  - pass
     1691 + 
    1430 1692   if hasattr(user, "distance"):
    1431 1693   distance = user.distance
    1432  - else:
    1433  - pass
    1434 1694   
    1435 1695   locations_list.append([ID, distance])
    1436  - 
     1696 + l_save_list.append(
     1697 + [
     1698 + ID,
     1699 + distance,
     1700 + latitude,
     1701 + longitude,
     1702 + filetime
     1703 + ]
     1704 + )
    1437 1705   except:
    1438 1706   pass
    1439 1707   
    skipped 7 lines
    1447 1715   account = int(account)
    1448 1716   my_user = await client.get_entity(PeerUser(account))
    1449 1717   user_id = my_user.id
    1450  - name = my_user.first_name
    1451 1718   distance = int(distance)
    1452 1719   
    1453 1720   if distance == 500:
    skipped 5 lines
    1459 1726   elif distance == 3000:
    1460 1727   distance_obj.d3000 += 1
    1461 1728   
    1462  - with open(
    1463  - save_file, "w+", encoding="utf-8"
    1464  - ) as f: # could one day append, including access time to differentiate
    1465  - user_df.to_csv(f, sep=";", index=False)
     1729 + 
     1730 + with open(save_file, "w+", encoding="utf-8") as f:
     1731 + l_save_df.to_csv(f, sep=";", index=False)
    1466 1732   
    1467 1733   total = len(locations_list)
     1734 + 
    1468 1735   distance_obj.save_file = save_file
    1469 1736   distance_obj.total = total
    1470 1737   print_shell("location_report",distance_obj)
    skipped 5 lines
    1476 1743   
    1477 1744  if __name__ == "__main__":
    1478 1745   cli()
    1479  - 
  • ■ ■ ■ ■ ■ ■
    src/telepathy/utils.py
    1  -from colorama import Fore, Back, Style
    2  -from googletrans import Translator, constants
     1 +from colorama import Fore, Style
     2 +from googletrans import Translator
    3 3  from telepathy.const import __version__, user_agent
    4 4  import requests
    5 5  import textwrap
    skipped 38 lines
    44 44   if user.username:
    45 45   username = user.username
    46 46   else:
    47  - username = "n/a"
     47 + username = "N/A"
    48 48   if user.first_name:
    49 49   first_name = user.first_name
    50 50   else:
    skipped 5 lines
    56 56   if user.phone:
    57 57   phone = user.phone
    58 58   else:
    59  - phone = "n/a"
     59 + phone = "N/A"
     60 + if user.id:
     61 + user_id = user.id
     62 + else:
     63 + user_id = "N/A"
    60 64   full_name = (first_name + " " + last_name).strip()
    61  - return [username, full_name, user.id, phone, group_or_chat]
     65 + return [username, full_name, user_id, phone, group_or_chat]
    62 66   
    63 67   
    64 68  def process_message(mess, user_lang):
    skipped 1 lines
    66 70   if mess is not None:
    67 71   mess_txt = '"' + mess + '"'
    68 72   else:
    69  - mess_txt = "none"
     73 + mess_txt = "None"
    70 74   
    71  - if mess_txt != "none":
     75 + if mess_txt != "None":
    72 76   translator = Translator()
    73 77   detection = translator.detect(mess_txt)
    74  - language_code = detection.lang
    75 78   translation_confidence = detection.confidence
    76 79   translation = translator.translate(mess_txt, dest=user_lang)
    77 80   original_language = translation.src
    78 81   translated_text = translation.text
    79 82   else:
    80 83   original_language = user_lang
    81  - translated_text = "n/a"
    82  - translation_confidence = "n/a"
     84 + translated_text = "N/A"
     85 + translation_confidence = "N/A"
    83 86   
    84 87   return {
    85 88   "original_language": original_language,
    skipped 6 lines
    92 95   if desc is not None:
    93 96   desc_txt = '"' + desc + '"'
    94 97   else:
    95  - desc_txt = "none"
     98 + desc_txt = "None"
    96 99   
    97  - if desc_txt != "none":
     100 + if desc_txt != "None":
    98 101   translator = Translator()
    99 102   detection = translator.detect(desc_txt)
    100  - language_code = detection.lang
    101 103   translation_confidence = detection.confidence
    102 104   translation = translator.translate(desc_txt, dest=user_lang)
    103 105   original_language = translation.src
    104 106   translated_text = translation.text
    105 107   else:
    106 108   original_language = user_lang
    107  - translated_text = "n/a"
    108  - translation_confidence = "n/a"
     109 + translated_text = "N/A"
     110 + translation_confidence = "N/A"
    109 111   
    110 112   return {
    111 113   "original_language": original_language,
    skipped 32 lines
    144 146   "div", {"class": ["tgme_page_description"]}
    145 147   ).text
    146 148   descript = Fore.GREEN + "Description: " + Style.RESET_ALL+ group_description
    147  - prefix = descript + " "
    148 149   except:
    149 150   group_description = "None"
    150 151   descript = Fore.GREEN + "Description: " + Style.RESET_ALL+ group_description
    151  - prefix = descript + " "
    152 152   
    153 153   try:
    154 154   group_participants = soup.find(
    skipped 8 lines
    163 163   .replace("member", "")
    164 164   )
    165 165   except:
    166  - total_participants = "Not found" # could be due to restriction, might need to mention
     166 + total_participants = "Not found"
    167 167   
    168 168   return {"name":name,"group_description":group_description, "total_participants":total_participants}
    169 169   
    skipped 21 lines
    191 191   color_print_green(" ├ Language: ", str(obj.lang_code))
    192 192   color_print_green(" ├ Bot: ", str(obj.bot))
    193 193   color_print_green(" ├ Scam: ", str(obj.scam))
     194 + color_print_green(" ├ Last seen: ", str(obj.user_status))
    194 195   color_print_green(" └ Restrictions: ", str(obj.user_restrictions))
    195 196   
    196 197   if type == "location_report":
    skipped 7 lines
    204 205   
    205 206   if type == "channel_recap" or type == "group_recap":
    206 207   
    207  - d_wrapper = generate_textwrap("Description")
    208  - td_wrapper = generate_textwrap("Translated Description")
     208 + d_wrapper = generate_textwrap("Description:")
     209 + td_wrapper = generate_textwrap("Translated Description:")
    209 210   
    210 211   color_print_green(" ┬ Chat details", "")
    211 212   color_print_green(" ├ Title: ", str(obj.title))
    skipped 27 lines
    239 240   " ├ Memberlist saved to: ", obj.memberlist_filename
    240 241   )
    241 242   color_print_green(
    242  - " └ ", d_wrapper.fill(obj.group_status)
     243 + " └ Restrictions: ", (str(obj.group_status))
    243 244   )
    244 245   
    245 246   if type == "group_stat":
    skipped 53 lines
    299 300   str(obj.reply_memberlist_filename),
    300 301   )
    301 302   color_print_green(
    302  - " Top replier 1: ", str(obj.replier_one)
     303 + " Top replier 1: ", str(obj.replier_one)
    303 304   )
    304 305   color_print_green(
    305 306   " ├ Top replier 2: ", str(obj.replier_two)
    skipped 8 lines
    314 315   " ├ Top replier 5: ", str(obj.replier_five)
    315 316   )
    316 317   color_print_green(
    317  - " Total unique repliers: ", str(obj.replier_unique)
     318 + " Total unique repliers: ", str(obj.replier_unique)
    318 319   )
    319 320   
    320 321   if type == "forwarder_stat":
    skipped 33 lines
  • ■ ■ ■ ■ ■ ■
    src/telepathy.egg-info/PKG-INFO
    1 1  Metadata-Version: 2.1
    2 2  Name: telepathy
    3  -Version: 2.2.58
     3 +Version: 2.3.2
    4 4  Summary: An OSINT toolkit for investigating Telegram chats.
    5 5  Home-page: https://pypi.python.org/pypi/telepathy/
    6 6  Author: Jordan Wildon
    skipped 2 lines
    9 9  Description-Content-Type: text/markdown
    10 10  License-File: LICENSE.txt
    11 11   
     12 +Telepathy: An OSINT toolkit for investigating Telegram chats. Developed by Jordan Wildon. Version 2.3.2.
    12 13   
     14 +Telepathy has been described as the "swiss army knife of Telegram tools," allowing OSINT analysts, researchers and digital investigators to archive Telegram chats (including replies, media content, comments and reactions), gather memberlists, lookup users by given location, analyze top posters in a chat, map forwarded messages, and more.
     15 + 
     16 +The toolkit has already seen a wide variety of use cases, including but not limited to: in investigative and data journalism, by academic and research institutions, and for intelligence gathering and analysis.
    13 17   
    14  -Telepathy: An OSINT toolkit for investigating Telegram chats. Developed by Jordan Wildon. Version 2.2.58.
     18 + 
     19 +## !! IMPORTANT:
     20 +With the update to 2.3.0, you will need to delete your login.txt file to prevent errors if using the alternative login feature. Upon first use, Telepathy will guide you through setup of the details once again. To work around this, instead of deleting and recreating the file, you can add a newline character to the end of your current API details to ensure Telepathy scans the file correctly.
     21 + 
     22 +A note on unique identifiers per account: You will notice that depending on which alternative account you use, the access hash will vary. The same will happen with User IDs, which are unique to each Telegram account accessing them. For deeper data analysis based on user IDs, this is important to bare in mind as users will have as many unique IDs as accounts you've used to access information. In future, Telepathy may include a feature to assign unique identifier per account found based on a hash of the available information, regardless of which account accessed the data.
     23 + 
    15 24   
    16 25   
    17 26  ## Installation
    skipped 28 lines
    46 55  Options:
    47 56  - **'--target', '-t' [CHAT]**
    48 57   
    49  -this option will identify the target of the scan. The specified chat must be public. To get the chat name, look for the 't.me/chatname' link, and subtract the 't.me/'.
     58 +this option will identify the target of the scan. The specified chat must be public or have a private link. To get the chat name, look for the 't.me/chatname' link, and subtract the 't.me/'.
    50 59   
    51 60  For example:
    52 61   
    skipped 6 lines
    59 68   
    60 69  - **'--comprehensive', '-c'**
    61 70   
    62  -A comprehensive scan will offer the same information as the basic scan, but will also archive a chat's message history.
     71 +A comprehensive scan will offer the same information as the basic scan, but will also archive a chat's message history, gather the number of reactions, archive how many times a message has been forwarded, the number of replies to each message, and more.
     72 + 
     73 +Reaction lists are included in the archive file, including basic calculations of engagement rate. Only the most-common reactions are listed, with the total including all possible reactions. Currently, Telepathy calculates engagement rates based on forwards, comments and reactions seperately, with a calculation based on post views and one based on chat participant count. In future, Telepathy may include deeper analytics which can be cross-compared between chats based on a combination of these metrics, fixing for when comments, reactions or forwards are allowed or disallowed in a given chat.
    63 74   
    64 75  For example:
    65 76   
    skipped 4 lines
    70 81   
    71 82  - **'--forwards', '-f'**
    72 83   
    73  -This flag will create an edgelist based on messages forwarded into a chat. It can be used alongside either a default or comprehensive scan.
     84 +This flag will create an edgelist based on messages forwarded into a chat. It can be used alongside either a default or comprehensive scan. Since 2.3.0, Telepathy now formats these edgelists to maximize compatability with Gephi.
    74 85   
    75 86  For example:
    76 87   
    77 88  ```
    78 89  $ telepathy -t durov -f
     90 + 
     91 +$ telepathy -t durov -c -f
    79 92  ```
    80 93   
    81 94   
    skipped 1 lines
    83 96   
    84 97  Use this flag to include media archiving alongside a comprehensive scan. This makes the process take significantly longer and should also be used with caution: you'll download all media content from the target chat, and it's up to you to not store illegal files on your system.
    85 98   
    86  -Since 2.2.0, downloading all media files will also generate a CSV file listing the files' metadata.
    87  - 
    88  -For example, this will run a comprehensive scan, including media archiving:
     99 +To archive media, you must run a comprehensive scan:
    89 100   
    90 101  ```
    91 102  $ telepathy -t durov -c -m
    92 103  ```
    93 104   
     105 +Once files have downloaded, you can run exiftool on the associated media directory to gather deeper insights on the files, their metadata, and in some cases attribute who might be behind an anonymous channel. Further details are in the "bonus investigations tips" section of this README.
    94 106   
    95  -- **'--user', '-u' [USER]**
     107 + 
     108 +- **'--user', '-u'**
    96 109   
    97  -Looks up a specified user ID. This will only work if your account has "encountered" the user before (for example, after archiving a group).
     110 +Looks up a specified user. This will only work if your account has "encountered" the user before (for example, after archiving a group), you can specify User ID or @nickname. If looking up by username, it's not always necessary for your account to have already seen the user.
    98 111   
    99 112  ```
    100 113  $ telepathy -t 0123456789 -u
     114 + 
     115 +$ telepathy -t @test_user -u
    101 116  ```
    102 117   
    103 118   
    104  -- **'--location', '-l' [COORDINATES]**
     119 +- **'--location', '-l']**
    105 120   
    106  -Finds users near to specified coordinates. Input should be longitude followed by latitude, seperated by a comma. This feature only works if your Telegram account has a profile image which is set to publicly viewable.
     121 +Finds users near to specified coordinates. Input should be longitude followed by latitude, seperated by a comma. This feature only works if your Telegram account has a profile image which is set to be publicly viewable.
     122 + 
     123 +While searches for multiple locations at once may work in some cases, Telegram appears to have a limit on how quickly an account can cycle through locations. At the time of writing, this appears to be at least ten minutes. Further location scanning support while using multiple accounts is being explored for a future release.
    107 124   
    108 125  ```
    109 126  $ telepathy -t 51.5032973,-0.1217424 -l
    110 127  ```
    111 128   
    112 129   
    113  -- **'--alt', '-a'**
     130 +- **'--alt', '-a' [NUMBER]**
    114 131   
    115  -Flag for running Telepathy from an alternative number. You can use the same API key and Hash but authenticate with a different phone number. Allows for running multiple scans at the same time.
     132 +Flag for running Telepathy from an alternative number or API details. You can use the same API key and Hash but authenticate with a different phone number. This allows for running multiple scans at the same time. Telepathy will default to the first details you offer, and up to four others can be added. Please see the notes at the top of this README for information regarding limitations with user IDs using this method.
    116 133   
    117 134  ```
    118  -$ telepathy -t Durov -c -a
     135 +$ telepathy -t Durov -c -a 1
    119 136  ```
    120 137   
    121 138   
    122 139  - **'--export', '-e'**
    123 140   
    124  -Exports all chats your account is part of to a CSV file. In a future release, this may assist with setting up multiple accounts following the same groups.
     141 +Exports all chats your account is part of to a CSV file. In a future release, this may assist with provisioning new accounts to automatically following the listed groups.
    125 142   
    126 143  ```
    127 144  $ telepathy -e
    skipped 2 lines
    130 147   
    131 148  - **'--reply', '-r'**
    132 149   
    133  -Flag for enable the reply in the channel, it will map users who replied in the channel and it will dump the full conversation chain
     150 +Flag for enabling channel reply retrieval, this will archive replies and list users who replied to messages in the target channel.
    134 151   
    135 152  ```
    136 153  $ telepathy -t [CHANNEL] -c -r
    137 154  ```
    138 155   
    139 156   
     157 +- **'--translate', '-tr'**
     158 + 
     159 +Flag for enabling auotmatic translation (currently only into English) during message retrieval.
     160 + 
     161 +```
     162 +$ telepathy -t [CHANNEL] -c -tr
     163 +```
     164 + 
     165 + 
    140 166  ## Bonus investigations tips:
    141 167   
    142 168   - Navigating to a media archive directory and running Exiftool may give you a whole host of useful information for further investigation. Telegram doesn't currently scrub metadata from PDF, DOCX, XLSX, MP4, MOV and some other filetypes, which offer creation and edit time metadata, often timezones, sometimes authors, and general technical information about the perosn or people who created a media file.
    skipped 1 lines
    144 170  $ cd ./telepathy/telepathy_files/CHATNAME/media
    145 171  $ exiftool * > metadata.txt
    146 172  ```
    147  - - Group and inferred channel memberlists offer a point of further investigation for usernames found. By using Maigret, you can look up where else a username has been used. While this is not accurate in all cases, it's been proven to be useful for handles that are often reused. In this case, remember to verify your findings to avoid false positives.
     173 + - Group and inferred channel memberlists offer a point of further investigation for usernames found. By using [Maigret](https://github.com/soxoj/maigret), you can look up where else a username has been used online. While this is not accurate in all cases, it's been proven to be helpful for identifying where a person has reused handles across platforms. In this case, remember to verify your findings to avoid false positives.
    148 174   
    149 175   
    150 176  ## A note on how Telegram works
    151 177   
    152  -Telegram chats are organised into three key types: Channels, Megagroups/Supergroups and Gigagroups. Each module works slightly differently depending on the chat type. Channels can have seemingly unlimited subscribers and are where an admin will broadcast messages to an audience, Megagroups can have up to 200,000 members, each of whom can participate (if not restricted), and Gigagroups sit somewhere between the two.
     178 +Telegram chats are organised into three key types: Channels, Megagroups/Supergroups and Gigagroups. Each option works slightly differently depending on the chat type. Channels can have seemingly unlimited subscribers and are where an admin will broadcast messages to an audience, Megagroups can have up to 200,000 members, each of whom can participate (if not restricted), and Gigagroups sit somewhere between the two.
    153 179   
    154 180   
    155 181  ## Upcoming changes
    skipped 2 lines
    158 184  Upcoming features include:
    159 185   
    160 186   - [ ] Adding a time specification flag to set archiving for specific period.
    161  - - [ ] The ability to gather the number of reactions to messages, including statistics on engagement rate.
     187 + - [x] The ability to gather the number of reactions to messages, including statistics on engagement rate.
    162 188   - [ ] Finding a method to once again gather complete memberlists (currently restricted by the API).
    163 189   - [ ] Improved statistics: including timestamp analysis for channels.
    164 190   - [ ] Generating an entirely automated complete report, including visualisation for some statistics.
    165 191   - [ ] Hate speech analytics.
    166  - - [x] Maximise compatibility of edgelists with Gephi.
    167 192   - [ ] Include sockpuppet account provisioning (creation of accounts from previous exported lists).
    168 193   - [ ] Listing who has group admin rights in memberlists.
    169  - - [ ] Media downloaded in the background to increase efficiency.
    170  - - [ ] When media archiving is flagged, the location of downloaded content will be added to the archive file.
    171  - - [ ] Exploring, and potentially integrating, media cross checks based on https://github.com/conflict-investigations/media-search-engine.
     194 + - [ ] Media downloaded in the background to increase efficiency or progress bars for media downloads to give a better estimation of runtime.
     195 + - [x] When media archiving is flagged, the location of downloaded content will be added to the archive file.
     196 + - [ ] Exploring, and potentially integrating, media cross-checks based on https://github.com/conflict-investigations/media-search-engine.
    172 197   - [ ] Ensuring inferred channel memberlists don't contain duplicate entries.
    173 198   - [ ] Introducing local chat retrival within the location lookup module.
    174  - - [ ] Adding trilateration option for location lookup to aid better location matching.
    175  - - [ ] Further code refactoring to ensure long-term maintainability.
    176  - - [ ] Progress bars for media downloads to give a better estimation of runtime.
    177  - - [ ] Adding additional alternative logins.
     199 + - [x] Further code refactoring to ensure long-term maintainability.
     200 + - [x] Adding additional alternative logins.
    178 201   - [ ] Improved language support.
    179  - - [ ] Ensure inferred channel memberlists (based on repliers) contains each account only once.
    180 202   - [ ] Correctly define destinction between reply (as in a chat) and comment (as in channel).
    181  - 
     203 + - [ ] Exploration of whether channel events can be included, such as name changes.
     204 + - [x] Including last seen on user lookup.
    182 205   
    183 206  ## feedback
    184 207   
    skipped 14 lines
Please wait...
Page is in error, reload to recover