You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
mmxwz6ahu
4d2c47d0b6
|
3 years ago | |
|---|---|---|
| README.md | 3 years ago | |
| Vul_db.py | 3 years ago | |
| __init__.py | 3 years ago | |
| get_data.py | 3 years ago | |
| items.py | 3 years ago | |
| middlewares.py | 3 years ago | |
| pipelines.py | 3 years ago | |
| seebug.py | 3 years ago | |
| settings.py | 3 years ago | |
| ui2.py | 3 years ago | |
| vulcrawl.py | 3 years ago | |
| vulcrawl2.py | 3 years ago | |
| 爬虫项目.zip | 3 years ago | |
README.md
VulCrawler
本项目是使用scrapy-redis框架实现分布式爬虫，由于头歌平台上传不了目录，只能是一个一个文件上传，具体的项目结构见图
VulCrawl - init.py - items.py - middlewares.py - pipelines.py - settings.py - Vul_db.py --spiders - init.py - vulcrawl.py - vulcrawl2.py