pf2umc5of
/
weibo-analysis-system-master
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '16fad0ad7d'
${ noResults }
weibo-analysis-system-master/scrapydserver/logs/bot/weibo_spider/1fdd76e86d8811e99ec1b46d83b...

511 lines
26 KiB
Raw Blame History Escape

This file contains ambiguous Unicode characters!

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

2019-05-03 17:45:11 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: bot)
2019-05-03 17:45:11 [scrapy.utils.log] INFO: Versions: lxml 4.3.3.0, libxml2 2.9.5, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 18.9.0, Python 3.6.3 |Anaconda, Inc.| (default, Oct 15 2017, 03:27:45) [MSC v.1900 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
2019-05-03 17:45:11 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'bot', 'DOWNLOAD_DELAY': 3, 'LOG_FILE': 'logs\\bot\\weibo_spider\\1fdd76e86d8811e99ec1b46d83b992e9.log', 'NEWSPIDER_MODULE': 'bot.spiders', 'SPIDER_MODULES': ['bot.spiders']}
2019-05-03 17:45:11 [scrapy.extensions.telnet] INFO: Telnet Password: eb4855904ab15093
2019-05-03 17:45:11 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.logstats.LogStats']
2019-05-03 17:45:11 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2019-05-03 17:45:11 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2019-05-03 17:45:11 [scrapy.middleware] INFO: Enabled item pipelines:
['bot.pipelines.BotPipeline']
2019-05-03 17:45:11 [scrapy.core.engine] INFO: Spider opened
2019-05-03 17:45:11 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2019-05-03 17:45:11 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2019-05-03 17:45:12 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://weibo.cn/6368301915/info> (referer: None)
2019-05-03 17:45:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://weibo.cn/u/6368301915> (referer: https://weibo.cn/6368301915/info)
2019-05-03 17:45:16 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/u/6368301915>
{'Image': 'http://tvax4.sinaimg.cn/crop.0.0.512.512.180/006WYKf1ly8g2ly0947iqj30e80e8aa6.jpg',
'_id': '6368301915',
'brief_introduction': '在微博有房',
'constellation': '狮子座',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 12, 850710),
'fans_num': 566,
'follows_num': 207,
'gender': '男',
'nick_name': '5cool',
'province': '北京',
'tweets_num': 43,
'vip_level': '1级'}
2019-05-03 17:45:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://weibo.cn/6368301915/profile?page=1> (referer: https://weibo.cn/u/6368301915)
2019-05-03 17:45:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=1>
{'_id': '6368301915_Hsnc0FibN',
'comment_num': 8,
'content': '上instagram ，facebook 以及想玩国外app的小朋友 可以找她，我用一年了 vp恩超级好用。',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 21, 492405),
'created_at': '2019-05-02 22:07',
'like_num': 13,
'repost_num': 3,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hsnc0FibN'}
2019-05-03 17:45:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=1>
{'_id': '6368301915_HsuNKsfNW',
'comment_num': 0,
'content': '哈哈谁特么啊走漏了风声',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 21, 557162),
'created_at': '2019-05-03 17:29:21',
'like_num': 1,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HsuNKsfNW'}
2019-05-03 17:45:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=1>
{'_id': '6368301915_HstVKiERz',
'comment_num': 2,
'content': '长得这么好看 主要得感谢我的父母 要不是他们给了我一张嘴 我也不会天天说这种屁话',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 21, 577191),
'created_at': '2019-05-03 15:16',
'like_num': 15,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HstVKiERz'}
2019-05-03 17:45:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=1>
{'_id': '6368301915_HstdRDJd3',
'comment_num': 3,
'content': '时候真的很羡慕长得好看的女孩，可以被我这样优秀的人喜欢',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 21, 607186),
'created_at': '2019-05-03 13:28',
'like_num': 10,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HstdRDJd3'}
2019-05-03 17:45:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=1>
{'_id': '6368301915_HssNOuOmu',
'comment_num': 1,
'content': '哈哈哈哈废物！',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 21, 617215),
'created_at': '2019-05-03 12:24',
'like_num': 11,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HssNOuOmu'}
2019-05-03 17:45:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=1>
{'_id': '6368301915_Hss35uC7I',
'comment_num': 4,
'content': '果你把刷微博的时间拿来复习考研 你会发现你还是考不上 很多事情不是投入时间就行的',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 21, 637165),
'created_at': '2019-05-03 10:28',
'like_num': 16,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hss35uC7I'}
2019-05-03 17:45:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=1>
{'_id': '6368301915_HsrPjeSsc',
'comment_num': 3,
'content': '个人都觉得前任有错，但每个人都是别人的前任，所以每个人都有错',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 21, 647188),
'created_at': '2019-05-03 09:54',
'like_num': 12,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HsrPjeSsc'}
2019-05-03 17:45:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=1>
{'_id': '6368301915_Hsr1Djg1l',
'comment_num': 0,
'content': '实自私的人幸福指数较高，过得好是他赚到；过得不好，跟他的付出比，也没亏',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 21, 669630),
'created_at': '2019-05-03 07:52',
'like_num': 12,
'repost_num': 1,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hsr1Djg1l'}
2019-05-03 17:45:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=1>
{'_id': '6368301915_Hsn0iEfZi',
'comment_num': 0,
'content': '发这个晚安，就当作有人跟你说晚安：晚安',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 21, 685621),
'created_at': '2019-05-02 21:38',
'like_num': 14,
'repost_num': 1,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hsn0iEfZi'}
2019-05-03 17:45:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=1>
{'_id': '6368301915_HsmbvCMoh',
'comment_num': 1,
'content': '努力，越幸运！你是不是对幸运有什么误解？真正的幸运是不需要努力',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 21, 702601),
'created_at': '2019-05-02 19:33',
'like_num': 18,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HsmbvCMoh'}
2019-05-03 17:45:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://weibo.cn/6368301915/profile?page=5> (referer: https://weibo.cn/6368301915/profile?page=1)
2019-05-03 17:45:24 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=5>
{'_id': '6368301915_Hrj88gB7p',
'comment_num': 86,
'content': '人ID 绝无广告 更不卖东西',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 24, 679589),
'created_at': '2019-04-25 21:56',
'like_num': 50,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hrj88gB7p'}
2019-05-03 17:45:24 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=5>
{'_id': '6368301915_Hrd61D5YR',
'comment_num': 0,
'content': '生感叹：赚钱可真不容易',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 24, 704546),
'created_at': '2019-04-25 06:34',
'like_num': 7,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hrd61D5YR'}
2019-05-03 17:45:24 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=5>
{'_id': '6368301915_Hqc0cyOwJ',
'comment_num': 3,
'content': '衣机男＝刚开始泡你，缠着你，和你如胶似漆，最后把你甩一边，晾着菠萝男＝用坚硬的外壳，包裹自己爱酸的内心，并且头上还很绿哈哈哈哈哈哈哈哈哈哈',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 24, 722565),
'created_at': '2019-04-18 13:57',
'like_num': 9,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hqc0cyOwJ'}
2019-05-03 17:45:24 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=5>
{'_id': '6368301915_HmeG0p3qv',
'comment_num': 8,
'content': '爱选我，我超甜 网恋酱的秒拍视频',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 24, 743526),
'created_at': '2019-03-23 13:24',
'like_num': 9,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HmeG0p3qv'}
2019-05-03 17:45:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://weibo.cn/6368301915/profile?page=4> (referer: https://weibo.cn/6368301915/profile?page=1)
2019-05-03 17:45:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=4>
{'_id': '6368301915_HrzlRjwBr',
'comment_num': 5,
'content': '来我才知道，有些女孩子让你摘星星，说的是劳斯莱斯的星空顶，百达翡丽的sky moon，LV的Galaxy',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 28, 842133),
'created_at': '2019-04-27 15:14',
'like_num': 33,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrzlRjwBr'}
2019-05-03 17:45:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=4>
{'_id': '6368301915_HryC2tJq7',
'comment_num': 43,
'content': '果你看了我的微博超过五条 你就得关注我',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 28, 857121),
'created_at': '2019-04-27 13:21',
'like_num': 42,
'repost_num': 1,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HryC2tJq7'}
2019-05-03 17:45:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=4>
{'_id': '6368301915_Hry2HpJyp',
'comment_num': 5,
'content': '雕网友真的很坚强，不管在现实中被生活折磨成什么样，在社交软件上永远笑的像个二傻子',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 28, 868116),
'created_at': '2019-04-27 11:54',
'like_num': 77,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hry2HpJyp'}
2019-05-03 17:45:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=4>
{'_id': '6368301915_Hrwu4kCsn',
'comment_num': 1,
'content': '恋爱真麻烦，以后还要分手，你一气之下决定直接单身',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 28, 878099),
'created_at': '2019-04-27 07:56',
'like_num': 12,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hrwu4kCsn'}
2019-05-03 17:45:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=4>
{'_id': '6368301915_HrsoYtg2O',
'comment_num': 3,
'content': '在的网恋也太抢手了叭',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 28, 888104),
'created_at': '2019-04-26 21:32',
'like_num': 18,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrsoYtg2O'}
2019-05-03 17:45:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=4>
{'_id': '6368301915_HrqrWcZOt',
'comment_num': 59,
'content': '对象，现在发展感情5.1奔现，来月经的不要，发现假照火车站对砍',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 28, 897100),
'created_at': '2019-04-26 16:34',
'like_num': 63,
'repost_num': 2,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrqrWcZOt'}
2019-05-03 17:45:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=4>
{'_id': '6368301915_HroNL9Uhq',
'comment_num': 6,
'content': '你个事 你到底知不知道我喜欢你',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 28, 906094),
'created_at': '2019-04-26 12:22',
'like_num': 11,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HroNL9Uhq'}
2019-05-03 17:45:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=4>
{'_id': '6368301915_HrnzW9AaX',
'comment_num': 3,
'content': '要是不赶时间的话，这辈子跟我在一起算了，有啥其他的事下辈子再说。',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 28, 917077),
'created_at': '2019-04-26 09:15',
'like_num': 8,
'repost_num': 2,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrnzW9AaX'}
2019-05-03 17:45:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=4>
{'_id': '6368301915_HrmJ5orxy',
'comment_num': 1,
'content': '一个人，不一定要得到。说这话的人，通常没得到。',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 28, 924089),
'created_at': '2019-04-26 07:05',
'like_num': 8,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrmJ5orxy'}
2019-05-03 17:45:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=4>
{'_id': '6368301915_HrjiA4BmQ',
'comment_num': 1,
'content': '果人类的本质是复读机，那为什么我说一句我爱你，你却不复读',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 28, 939095),
'created_at': '2019-04-25 22:22',
'like_num': 6,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrjiA4BmQ'}
2019-05-03 17:45:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://weibo.cn/6368301915/profile?page=3> (referer: https://weibo.cn/6368301915/profile?page=1)
2019-05-03 17:45:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=3>
{'_id': '6368301915_Hs1nEf7ZQ',
'comment_num': 3,
'content': '要让傻叉影响你的心情，除非你觉得它好笑',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 33, 516437),
'created_at': '2019-04-30 14:35',
'like_num': 17,
'repost_num': 1,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hs1nEf7ZQ'}
2019-05-03 17:45:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=3>
{'_id': '6368301915_HrUHZyJnZ',
'comment_num': 4,
'content': '工作直接跟老板谈，找对象直接跟我谈',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 33, 536453),
'created_at': '2019-04-29 21:36',
'like_num': 17,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrUHZyJnZ'}
2019-05-03 17:45:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=3>
{'_id': '6368301915_HrUpcsuvy',
'comment_num': 0,
'content': '不但喜欢你，我还喜欢你的钱，毕竟爱一个人就要爱他的全部',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 33, 551417),
'created_at': '2019-04-29 20:50',
'like_num': 10,
'repost_num': 1,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrUpcsuvy'}
2019-05-03 17:45:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=3>
{'_id': '6368301915_HrRgUtu4U',
'comment_num': 0,
'content': '代年轻人喜欢的生活：要么整天在外面浪，要么整天不出门',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 33, 564437),
'created_at': '2019-04-29 12:51',
'like_num': 18,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrRgUtu4U'}
2019-05-03 17:45:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=3>
{'_id': '6368301915_HrPicpDN5',
'comment_num': 3,
'content': '对一夜暴富的渴望相比，我对爱情的期待简直不值一提',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 33, 578422),
'created_at': '2019-04-29 07:49',
'like_num': 27,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrPicpDN5'}
2019-05-03 17:45:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=3>
{'_id': '6368301915_HrGXM4IK9',
'comment_num': 3,
'content': '你同时处在寻找和被寻找的位置上，正不断接近一场最美好的相逢',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 33, 587409),
'created_at': '2019-04-28 10:36',
'like_num': 17,
'repost_num': 1,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrGXM4IK9'}
2019-05-03 17:45:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=3>
{'_id': '6368301915_HrGbzpIPB',
'comment_num': 1,
'content': '到网友私信，让我装成相亲对象恶搞一下他的朋友…… 酷酷的滕的秒拍视频',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 33, 597411),
'created_at': '2019-04-28 08:37',
'like_num': 4,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrGbzpIPB'}
2019-05-03 17:45:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=3>
{'_id': '6368301915_HrBZZsRAV',
'comment_num': 4,
'content': '果道歉有用的话，要lamer，cpb，圣罗兰，纪梵希，古驰，香奈儿，迪奥，兰蔻……做什么？',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 33, 609383),
'created_at': '2019-04-27 21:58',
'like_num': 21,
'repost_num': 1,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrBZZsRAV'}
2019-05-03 17:45:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=3>
{'_id': '6368301915_HrAyA2Iu7',
'comment_num': 2,
'content': '的人真没素质，竟然不爱我',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 33, 658356),
'created_at': '2019-04-27 18:18',
'like_num': 13,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrAyA2Iu7'}
2019-05-03 17:45:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=3>
{'_id': '6368301915_HrA9isRPf',
'comment_num': 10,
'content': '头发是因为你太久没处对象，你的头自认为你出家了，所以开始脱落',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 33, 699342),
'created_at': '2019-04-27 17:15',
'like_num': 15,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HrA9isRPf'}
2019-05-03 17:45:36 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://weibo.cn/6368301915/profile?page=2> (referer: https://weibo.cn/6368301915/profile?page=1)
2019-05-03 17:45:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=2>
{'_id': '6368301915_HsknPBKPZ',
'comment_num': 2,
'content': '下一个人很难，放下手机更难',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 36, 702036),
'created_at': '2019-05-02 14:58',
'like_num': 18,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HsknPBKPZ'}
2019-05-03 17:45:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=2>
{'_id': '6368301915_Hsjp5ymn4',
'comment_num': 3,
'content': '华德· 史塔克的儿子降生了，随着孩子一天天的长大，他察觉到儿子和有过一面之缘的波茨先生越来越像，他有些隐隐的不安了起来',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 36, 724005),
'created_at': '2019-05-02 12:28',
'like_num': 18,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hsjp5ymn4'}
2019-05-03 17:45:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=2>
{'_id': '6368301915_HsiNHb8JK',
'comment_num': 7,
'content': '情提醒：不要因为别人分手就不相信爱情，你还没和我谈过恋爱',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 36, 741985),
'created_at': '2019-05-02 10:56',
'like_num': 22,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HsiNHb8JK'}
2019-05-03 17:45:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=2>
{'_id': '6368301915_Hsh8VubLe',
'comment_num': 7,
'content': '据能量守恒定律，不发脾气的人容易记仇',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 36, 759973),
'created_at': '2019-05-02 06:43',
'like_num': 21,
'repost_num': 4,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hsh8VubLe'}
2019-05-03 17:45:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=2>
{'_id': '6368301915_HscrOw1FG',
'comment_num': 1,
'content': '钱甚至能买到贫穷，不信你把所有钱都给我，我把贫穷卖给你',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 36, 774994),
'created_at': '2019-05-01 18:45',
'like_num': 17,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HscrOw1FG'}
2019-05-03 17:45:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=2>
{'_id': '6368301915_HsbpStPB3',
'comment_num': 2,
'content': '后有条件的话，我想把一个房间装修成牢房，我就在里面打电话上网吃火锅，像一个真正的大佬。',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 36, 792955),
'created_at': '2019-05-01 16:08',
'like_num': 19,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/HsbpStPB3'}
2019-05-03 17:45:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=2>
{'_id': '6368301915_Hs9Gf4Ow6',
'comment_num': 5,
'content': '刚醒来发现脖子上凉飕飕的，出于职业本能，我以为是有人把刀抵在我脖子上，后来发现是大金链子太凉',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 36, 807974),
'created_at': '2019-05-01 11:43',
'like_num': 18,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hs9Gf4Ow6'}
2019-05-03 17:45:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=2>
{'_id': '6368301915_Hs8jxmaTk',
'comment_num': 1,
'content': '他娘的真是恋爱鬼才啊哈哈哈哈哈哈我也好想要这样真挚的情感啊哈哈哈哈哈哈',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 36, 823944),
'created_at': '2019-05-01 08:14',
'like_num': 19,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hs8jxmaTk'}
2019-05-03 17:45:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=2>
{'_id': '6368301915_Hs3Hti2PE',
'comment_num': 16,
'content': '些人分手后很长时间内都没有新对象，不是因为对前任念念不忘，而是确实找不到新欢。运气这玩意用完就没有了',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 36, 838936),
'created_at': '2019-04-30 20:29',
'like_num': 50,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hs3Hti2PE'}
2019-05-03 17:45:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://weibo.cn/6368301915/profile?page=2>
{'_id': '6368301915_Hs2s4cRMd',
'comment_num': 8,
'content': '時行樂',
'crawl_time': datetime.datetime(2019, 5, 3, 17, 45, 36, 893904),
'created_at': '2019-04-30 17:19',
'like_num': 23,
'repost_num': 0,
'user_id': '6368301915',
'weibo_url': 'https://weibo.com/6368301915/Hs2s4cRMd'}
2019-05-03 17:45:36 [scrapy.core.engine] INFO: Closing spider (finished)
2019-05-03 17:45:36 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 6950,
'downloader/request_count': 7,
'downloader/request_method_count/GET': 7,
'downloader/response_bytes': 30928,
'downloader/response_count': 7,
'downloader/response_status_count/200': 7,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2019, 5, 3, 9, 45, 36, 919882),
'item_scraped_count': 45,
'log_count/DEBUG': 52,
'log_count/INFO': 9,
'request_depth_max': 3,
'response_received_count': 7,
'scheduler/dequeued': 7,
'scheduler/dequeued/memory': 7,
'scheduler/enqueued': 7,
'scheduler/enqueued/memory': 7,
'start_time': datetime.datetime(2019, 5, 3, 9, 45, 11, 686788)}
2019-05-03 17:45:36 [scrapy.core.engine] INFO: Spider closed (finished)
Reference in new issue View Git Blame Copy Permalink