• Index

网络爬虫

Reads: 525 Edit


with ThreadPoolExecutor() as pool:
    futures = {}
    for url, html in htmls:
        future = pool.submit(blog_spider.parse, html)
        futures[future] = url
    for future in as_completed(futures):
        print(futures[future], future.result())

参考资料

https://blog.csdn.net/sixteen_16/article/details/116176587?spm=1001.2101.3001.6650.4&utm_medium=distribute.pc_relevant.none-task-blog-2%7Edefault%7ECTRLIST%7ERate-4-116176587-blog-103244542.pc_relevant_3mothn_strategy_recovery&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2%7Edefault%7ECTRLIST%7ERate-4-116176587-blog-103244542.pc_relevant_3mothn_strategy_recovery&utm_relevant_index=5

Comments

Make a comment

www.ultrapower.com ,王硕的博客,专注于研究互联网产品和技术,提供中文精品教程。 本网站与其它任何公司及/或商标无任何形式关联或合作。
  • Index
aaaaa