p2uavm3kl
/
BilispiderRedis
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '25967dac58'
${ noResults }
BilispiderRedis/README.md

8 lines
727 B
Raw Blame History Escape

This file contains ambiguous Unicode characters!

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

# 项目概述
本项目基于selenium+scrapy-redis框架爬取bilibili网站的数据，通过研究网站使用的反爬虫技术，使用Selenium驱动浏览器模拟真人使用浏览器，在浏览器中加入cookie绕过登录并使用代理ip，爬取的数据使用mysql数据库进行存储，并对爬取的数据进行数据分析和可视化展示。
主要功能包括，爬取了bilibili网站的番剧信息，和包括番剧在内的所有视频的信息，包括播放量、收藏量和评论量等。并筛选出相对优秀的视频数据，进行可视化的展示。最后通过展示出的图表进行分析。
关键字：selenium；scrapy-redis；cookie；代理ip；数据可视化；mysql数据库
Reference in new issue View Git Blame Copy Permalink