http://www.sufeinet.com/plugin.php?id=keke_group

苏飞论坛

 找回密码
 马上注册

QQ登录

只需一步,快速开始

分布式系统框架(V2.0) 轻松承载百亿数据,千万流量!讨论专区 - 源码下载 - 官方教程

HttpHelper爬虫框架(V2.7-含.netcore) HttpHelper官方出品,爬虫框架讨论区 - 源码下载 - 在线测试和代码生成

HttpHelper爬虫类(V2.0) 开源的爬虫类,支持多种模式和属性 源码 - 代码生成器 - 讨论区 - 教程- 例子

查看: 7966|回复: 12

[求助] httpHelper能否解决我的问题

[复制链接]
发表于 2015-4-1 15:54:59 | 显示全部楼层 |阅读模式
本帖最后由 eeeeeee518 于 2015-4-1 15:58 编辑

Remote Address:23.48.16.81:443
Request URL:https://steamcommunity.com/market/buylisting/425929146580637275
Request MethodOST
Status Code:502 Bad Gateway
Request Headersview source
Accept:*/*
Accept-Encoding:gzip,deflate
Accept-Language:zh-CN,zh;q=0.8
Connection:keep-alive
Content-Length:81
Content-Type:application/x-www-form-urlencoded; charset=UTF-8
Cookie:steamMachineAuth76561198092536937=0B4829611068D4353D05670C0F8D7ED68A636524; Steam_Language=english; 765_6workshopQueueTime=1427696221; strInventoryLastContext=570_2; sessionid= XXXXXXXXXXXX; steamCountry=CN%7C674d420bea9fa0d59241481e88605223; steamLogin=76561198092536937%7C%7C1A53DDB6C094E8D38B8F49E8E54543E61BB8AFBD; steamLoginSecure=76561198092536937%7C%7CEC985E37D9CF42F0359C31C7E543495A7DCA7125; webTradeEligibility=%7B%22allowed%22%3A1%2C%22allowed_at_time%22%3A0%2C%22steamguard_required_days%22%3A15%2C%22sales_this_year%22%3A218%2C%22max_sales_per_year%22%3A-1%2C%22forms_requested%22%3A0%2C%22new_device_cooldown_days%22%3A7%7D; timezoneOffset=28800,0; __utma=268881843.752842853.1426557236.1427854361.1427856273.25; __utmb=268881843.0.10.1427856273; __utmc=268881843; __utmz=268881843.1427854361.24.22.utmcsr=store.steampowered.com|utmccn=(referral)|utmcmd=referral|utmcct=/
Host:steamcommunity.com
Origin:http://steamcommunity.com
Referer:http://steamcommunity.com/market/listings/570/Twisted%20Firekeeper
User-Agent:Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36
Form Dataview parsed
sessionid=xxxxxx¤cy=1&subtotal=1&fee=2&total=3&quantity=1
Response Headersview source
Access-Control-Allow-Credentials:true
Access-Control-Allow-MethodsOST, GET, OPTIONS
Access-Control-Allow-Origin:http://steamcommunity.com
Access-Control-Expose-Headers:true
Access-Control-Max-Age:604800
Cache-Control:no-cache
Connection:close
Content-Length:91
Content-Security-Policy:script-src 'self' 'unsafe-inline' 'unsafe-eval' https://steamcommunity-a.akamaihd.net/ https://api.steampowered.com/ http://www.google-analytics.com https://ssl.google-analytics.com https://www.google.com https://www.gstatic.com https://apis.google.com; object-src 'none'; connect-src 'self' https://steamcommunity.com http://steamcommunity.com https://api.steampowered.com/; frame-src 'self' http://store.steampowered.com/ https://store.steampowered.com/ http://www.youtube.com https://www.youtube.com https://www.google.com;
Content-Type:application/json; charset=utf-8
Date:Wed, 01 Apr 2015 02:51:16 GMT
Expires:Mon, 26 Jul 1997 05:00:00 GMT
Server:Apache



1. 开通SVIP会员,免费下载本站所有源码,不限次数据,不限时间
2. 加官方QQ群,加官方微信群获取更多资源和帮助
3. 找站长苏飞做网站、商城、CRM、小程序、App、爬虫相关、项目外包等点这里
 楼主| 发表于 2015-4-1 15:57:03 | 显示全部楼层
上面是我用360浏览器获取的值
我本来想用httpwebrequest来实现
但是他需要是在登陆状态下才能完成上面的请求
所以想到了用httpHelper
但是不知道怎么写,我根据教程谢了一些,但就是不好用
谁能帮帮我缕清思路
发表于 2015-4-1 17:01:11 | 显示全部楼层
这个只要安教程写参数就行了吧。
 楼主| 发表于 2015-4-1 21:56:08 | 显示全部楼层
站长苏飞 发表于 2015-4-1 17:01
这个只要安教程写参数就行了吧。

            DotNet4.Utilities.HttpHelper httpHelper = new DotNet4.Utilities.HttpHelper();
            DotNet4.Utilities.HttpItem httpItem = new DotNet4.Utilities.HttpItem();
            httpItem.URL = "https://steamcommunity.com/market/buylisting/425929146580637275";
            httpItem.Referer = "http://steamcommunity.com/market/listings/570/Bright%20Moon%20Mokdori";
            httpItem.Accept = "*/*";
            httpItem.Method = "POST";
            httpItem.ContentType = "application/x-www-form-urlencoded; charset=UTF-8";
            httpItem.UserAgent = "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.146 BIDUBrowser/6.x Safari/537.36";
            CookieCollection objcok = new CookieCollection();
            objcok.Add(new Cookie("Steam_Language", "english"));
            objcok.Add(new Cookie("765_6workshopQueueTime", "1427696221"));
            objcok.Add(new Cookie("strInventoryLastContext", "570_2"));
            objcok.Add(new Cookie("timezoneOffset", "28800%2C1"));
            objcok.Add(new Cookie("__utma", "268881843.752842853.1426557236.1427786478.1427791762.23"));
            objcok.Add(new Cookie("__utmz", "268881843.1427791762.23.21.utmcsr=store.steampowered.com|utmccn=(referral)|utmcmd=referral|utmcct=/"));
            objcok.Add(new Cookie("sessionid", "xxxxxxxxxxxxxxxxxxxxx"));
            objcok.Add(new Cookie("steamCountry", "CN%7C674d420bea9fa0d59241481e88605223"));
            objcok.Add(new Cookie("steamLogin", "76561198092536937%7C%7C1A53DDB6C094E8D38B8F49E8E54543E61BB8AFBD"));
            httpItem.CookieCollection = objcok;

            DotNet4.Utilities.HttpResult httpResult = httpHelper.GetHtml(httpItem);
            string result = httpResult.Html;

<HTML><HEAD>
<TITLE>Bad Request</TITLE>
</HEAD><BODY>
<H1>Bad Request</H1>
Your browser sent a request that this server could not understand.<P>
Reference&#32;&#35;7&#46;965732b8&#46;1427896475&#46;bdae0e37
</BODY>
</HTML>
 楼主| 发表于 2015-4-1 21:56:57 | 显示全部楼层
站长苏飞 发表于 2015-4-1 17:01
这个只要安教程写参数就行了吧。

这么写不对么
不会啊~~~
 楼主| 发表于 2015-4-2 00:11:59 | 显示全部楼层
            DotNet4.Utilities.HttpHelper httpHelper = new DotNet4.Utilities.HttpHelper();
            DotNet4.Utilities.HttpItem httpItem = new DotNet4.Utilities.HttpItem();
            httpItem.URL = "https://steamcommunity.com/market/buylisting/437187513962166084";
            httpItem.Referer = "http://steamcommunity.com/market/listings/570/The%20Fiend%20Cleaver%20Loading%20Screen";
            httpItem.Method = "POST";
            httpItem.Postdata = "sessionid=xxxxxxxxx&currency=1&subtotal=1&fee=2&total=3&quantity=1";
            httpItem.ContentType = "application/x-www-form-urlencoded; charset=UTF-8";
            httpItem.UserAgent = "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.146 BIDUBrowser/6.x Safari/537.36";
            httpItem.Cookie = "Steam_Language=english; timezoneOffset=28800,0; __utma=268881843.1437743313.1425902569.1427900954.1427904382.17; __utmz=268881843.1425907107.2.2.utmcsr=store.steampowered.com|utmccn=(referral)|utmcmd=referral|utmcct=/; sessionid=xxxxxx; steamCountry=CN%7C72f4abb00a4a141250cd1d1305f1c2b0; steamLogin=76561198092536937%7C%7C1A53DDB6C094E8D38B8F49E8E54543E61BB8AFBD; steamRememberLogin=76561198092536937%7C%7Ccac71636e4377af1546f33930da96bfe; webTradeEligibility=%7B%22allowed%22%3A1%2C%22allowed_at_time%22%3A0%2C%22steamguard_required_days%22%3A15%2C%22sales_this_year%22%3A217%2C%22max_sales_per_year%22%3A-1%2C%22forms_requested%22%3A0%2C%22new_device_cooldown_days%22%3A7%7D; __utmb=268881843.0.10.1427904382; __utmc=268881843";
            DotNet4.Utilities.HttpResult httpResult = httpHelper.GetHtml(httpItem);
            string result = httpResult.Html;

这么写之后倒是返回数据了,但是返回的数据是不对的
还有post数据怎么获取啊
cookie的话是一直不变么?
 楼主| 发表于 2015-4-2 00:12:43 | 显示全部楼层
跟证书会有关系么
 楼主| 发表于 2015-4-2 10:49:23 | 显示全部楼层
站长苏飞 发表于 2015-4-1 17:01
这个只要安教程写参数就行了吧。

站长,求你粗线
我这个是实现后台购买功能的
是不是还得先登录,然后获取cookie,再然后拿那个cookie请求啊
还是直接抓包抓到之前好用的cookie就可以了呢
 楼主| 发表于 2015-4-2 11:14:17 | 显示全部楼层
本帖最后由 eeeeeee518 于 2015-4-2 14:17 编辑

=。=
回复

使用道具 举报

发表于 2015-4-2 11:24:23 | 显示全部楼层
直接抓到登录后的Cookie使用就行
您需要登录后才可以回帖 登录 | 马上注册

本版积分规则

QQ|手机版|小黑屋|手机版|联系我们|关于我们|广告合作|苏飞论坛 ( 豫ICP备18043678号-2)

GMT+8, 2024-11-22 13:51

© 2014-2021

快速回复 返回顶部 返回列表