Projects STRLCPY maigret Commits 731a8e01
🤬
  • Added separate `no-extracing` flag to rule page parsing

  • Loading...
  • Soxoj committed 4 years ago
    731a8e01
    1 parent f7263c9b
Revision indexing in progress... (symbol navigation in revisions will be accurate after indexed)
  • ■ ■ ■ ■ ■ ■
    maigret/checking.py
    skipped 288 lines
    289 289   
    290 290   
    291 291  async def maigret(username, site_dict, query_notify, logger,
    292  - proxy=None, timeout=None, recursive_search=False,
     292 + proxy=None, timeout=None, is_parsing_enabled=False,
    293 293   id_type='username', debug=False, forced=False,
    294 294   max_connections=100, no_progressbar=False,
    295 295   cookies=None):
    skipped 11 lines
    307 307   proxy -- String indicating the proxy URL
    308 308   timeout -- Time in seconds to wait before timing out request.
    309 309   Default is no timeout.
    310  - recursive_search -- Search for other usernames in website pages & recursive search by them.
     310 + is_parsing_enabled -- Search for other usernames in website pages.
    311 311   
    312 312   Return Value:
    313 313   Dictionary containing results from report. Key of dictionary is the name
    skipped 50 lines
    364 364   
    365 365   # Record URL of main site and username
    366 366   results_site['username'] = username
    367  - results_site['parsing_enabled'] = recursive_search
     367 + results_site['parsing_enabled'] = is_parsing_enabled
    368 368   results_site['url_main'] = site.url_main
    369 369   results_site['cookies'] = cookie_jar and cookie_jar.filter_cookies(site.url_main) or None
    370 370   
    skipped 235 lines
  • ■ ■ ■ ■ ■
    maigret/maigret.py
    skipped 105 lines
    106 106   )
    107 107   parser.add_argument("--no-recursion",
    108 108   action="store_true", dest="disable_recursive_search", default=False,
    109  - help="Disable parsing pages for other usernames and recursive search by them."
     109 + help="Disable recursive search by additional data extracted from pages."
     110 + )
     111 + parser.add_argument("--no-extracting",
     112 + action="store_true", dest="disable_extracting", default=False,
     113 + help="Disable parsing pages for additional data and other usernames."
    110 114   )
    111 115   parser.add_argument("--self-check",
    112 116   action="store_true", default=False,
    skipped 90 lines
    203 207   and u not in args.ignore_ids_list
    204 208   }
    205 209   
     210 + parsing_enabled = not args.disable_extracting
    206 211   recursive_search_enabled = not args.disable_recursive_search
    207 212   
    208 213   # Make prompts
    skipped 115 lines
    324 329   query_notify,
    325 330   proxy=args.proxy,
    326 331   timeout=args.timeout,
    327  - recursive_search=recursive_search_enabled,
     332 + is_parsing_enabled=parsing_enabled,
    328 333   id_type=id_type,
    329 334   debug=args.verbose,
    330 335   logger=logger,
    skipped 81 lines
Please wait...
Page is in error, reload to recover