侧边栏壁纸
博主头像
海绵

正向的根基决定逆向的上限

  • 累计撰写 30 篇文章
  • 累计创建 28 个标签
  • 累计收到 19 条评论

fiddler抓requests请求

海绵
2022-02-24 / 0 评论 / 0 点赞 / 303 阅读 / 619 字
def scrape(url, headers, num=3):

    print(url)
    if num == 0: return
    # proxies = random_ip()
    # 设置此处 8888为fiddler端口
    proxies = {'http': 'http://localhost:8888', 'https':'http://localhost:8888'}
    urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
    try:
        sessions = requests.session()
        sessions.keep_alive = False
        ret = sessions.get(url, headers=headers, timeout=5,verify=False,proxies=proxies)
        if ret.status_code == requests.codes.ok:
            ret.encoding = 'utf-8'
            if '<title>405' in ret.text:
                print(ret.text)
                assert False
            return ret.text
        else:
            print(f'状态码为{ret.status_code}')
            assert False
                # assert False
    except Exception as e:
        print('出错了',e)
        num -= 1
        return scrape(url, headers, num)
0

评论区