fatsand/xhs_crawler
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #24 from jayeeliu/main

Browse Source 
去掉搜索结果列表中的推荐搜索词（mode_type in ['rec_query', 'hot_query']）
This commit is contained in:
Relakkes 2023-08-01 13:35:00 +08:00 committed by GitHub
commit defd8232da
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
media_platform/xhs/core.py
View File

@ -93,6 +93,7 @@ class XiaoHongShuCrawler(AbstractCrawler):
task_list = [ task_list = [
self.get_note_detail(post_item.get("id"), semaphore) self.get_note_detail(post_item.get("id"), semaphore)
for post_item in notes_res.get("items", {}) for post_item in notes_res.get("items", {})
if post_item.get('model_type') not in ('rec_query', 'hot_query')
] ]
note_details = await asyncio.gather(*task_list) note_details = await asyncio.gather(*task_list)
for note_detail in note_details: for note_detail in note_details: