采集分页的BUG

cctv1233 · 发表于 2007-7-6 05:00:19

有分页采集时第一页大都会重复采集。如果从规则上过滤掉第一页的话，则第二页会采不到，后面的可以正常采下来！

比如下面这段分页的采集！
采集这篇文章
http://www.enet.com.cn/article/2007/0705/A20070705709679.shtml

<select name=dldhop onchange="javascript:window.location.href=document.all.dldhop.value" class="size14" style="width:250px">
第 1 页
<option value="./A20070705709679.shtml" selected >第1页</option>
第 2 页
<option value="./A20070705709679_2.shtml" >第2页</option>
第 3 页
<option value="./A20070705709679_3.shtml" >第3页</option>
第 4 页
<option value="./A20070705709679_4.shtml" >第4页</option>
</select>

复制代码

文章内容分页链接识别规则设成<option value="./

cctv1233 · 发表于 2007-7-6 05:09:23

还有哦，
文章内容分页链接URL补充前缀建议象DEDECMS一样设成程序自动匹配！不然的话前缀是动态的就不好采分页了！
还是上面那个网站！分页链接URL补充前缀是http://www.enet.com.cn/article/2007/0705 最后的0705是随日期变化的，所以文章内容分页链接URL补充前缀就不好填了，只能采一天改一天！

。这个技术上应该是没问题的吧？DEDE都做得到。SS应该也可以吧。

cctv1233 · 发表于 2007-7-6 05:23:14

还有，写了分页规则，如果碰到没有分页的就会显示处理分页链接失败！真失败！

茄子 · 发表于 2007-7-6 09:45:47

不能分页应该是采集规则错误吧
可以将您要采集的页面和采集规则发出来看看

magic535 · 发表于 2007-7-6 09:47:47

不能分页应该是采集规则错误吧

8qzone · 发表于 2007-7-6 09:48:36

是啊我也遇到这样的问题

cctv1233 · 发表于 2007-7-6 20:10:04

原帖由茄子于 2007-7-6 09:45 发表
不能分页应该是采集规则错误吧
可以将您要采集的页面和采集规则发出来看看

不是不能分页，而是分页很不完美！

采集页面

http://www.enet.com.cn/emobile/l ... e&articleType=6

# SupeSite Dump
# Version: SupeSite 5.5
# Time: 2007-07-06 20:08:25
# From: 紫龙娱乐网 ([url]http://spaces.zl06.net.ru[/url])
#
# This file was BASE64 encoded
#
# SupeSite: [url]http://www.supesite.com[/url]
# Please visit our website for latest news about SupeSite
# --------------------------------------------------------
YTozNzp7czo3OiJyb2JvdGlkIjtzOjI6IjcxIjtzOjQ6Im5hbW
UiO3M6MTI6ImVuZXTK1rv618rRtiI7czozOiJ1aWQiO3M6Mjoi
MjUiO3M6ODoiZGF0ZWxpbmUiO3M6MTA6IjExODM3MjMwNzMiO3
M6ODoibGFzdHRpbWUiO3M6MTA6IjExODM3MjM0NjMiO3M6ODoi
cm9ib3RudW0iO3M6MjoiMTkiO3M6MTE6Imxpc3R1cmx0eXBlIj
tzOjQ6ImF1dG8iO3M6NzoibGlzdHVybCI7czo4NjoiaHR0cDov
L3d3dy5lbmV0LmNvbS5jbi9lbW9iaWxlL2xpc3QvaW5kZXguan
NwP3BhZ2U9W3BhZ2VdJnNpdGVpZD1lbW9iaWxlJmFydGljbGVU
eXBlPTYiO3M6MTM6Imxpc3RwYWdlc3RhcnQiO3M6MToiMSI7cz
oxMToibGlzdHBhZ2VlbmQiO3M6MToiMyI7czo2OiJhbGxudW0i
O3M6MzoiMTAwIjtzOjY6InBlcm51bSI7czoxOiIxIjtzOjc6In
NhdmVwaWMiO3M6MToiMSI7czo2OiJlbmNvZGUiO3M6MDoiIjtz
OjEzOiJwaWN1cmxsaW5rcHJlIjtzOjA6IiI7czo5OiJzYXZlZm
xhc2giO3M6MToiMCI7czoxNDoic3ViamVjdHVybHJ1bGUiO3M6
MTIxOiI8c3BhbiBjbGFzcz0ib3JhbmdlMSI+yKuyvzwvc3Bhbj
48L3A+W2xpc3RdPHRhYmxlIHdpZHRoPSIxMDAlIiBib3JkZXI9
IjAiIGNlbGxzcGFjaW5nPSIwIiBjZWxscGFkZGluZz0iMCIgYW
xpZ249ImNlbnRlciI+IjtzOjE4OiJzdWJqZWN0dXJsbGlua3J1
bGUiO3M6MTU6IjxhIGhyZWY9Ilt1cmxdIiI7czoxNzoic3Viam
VjdHVybGxpbmtwcmUiO3M6MjI6Imh0dHA6Ly93d3cuZW5ldC5j
b20uY24iO3M6MTE6InN1YmplY3RydWxlIjtzOjE4OiI8aDE+W3
N1YmplY3RdPC9oMT4iO3M6MTM6InN1YmplY3RmaWx0ZXIiO3M6
MDoiIjtzOjE0OiJzdWJqZWN0cmVwbGFjZSI7czowOiIiO3M6MT
Y6InN1YmplY3RyZXBsYWNldG8iO3M6MDoiIjtzOjEwOiJzdWJq
ZWN0a2V5IjtzOjA6IiI7czoxODoic3ViamVjdGFsbG93cmVwZW
F0IjtzOjE6IjAiO3M6MTI6ImRhdGVsaW5lcnVsZSI7czowOiIi
O3M6ODoiZnJvbXJ1bGUiO3M6MDoiIjtzOjEwOiJhdXRob3JydW
xlIjtzOjA6IiI7czoxMToibWVzc2FnZXJ1bGUiO3M6Mzk6Ijxk
aXYgY2xhc3M9ImNvbnRlbnR0eHQiPlttZXNzYWdlXTwvZGl2Pi
I7czoxMzoibWVzc2FnZWZpbHRlciI7czoyMzQ6IjxhKj58PC9h
Pnw8QSo+fDwvQT58PGltZyBzcmM9J2h0dHA6Ly9pbWFnZXMuZW
5ldC5jb20uY24vZW5kLmdpZicgd2lkdGg9JzExJyBoZWlnaHQ9
JzExJyBib3JkZXI9JzAnIHRhcmdldD0nX2JsYW5rJyBhbGlnbj
0nYWJzbWlkZGxlJy8+fDxwIGFsaWduPSdyaWdodCc+PHNwYW4g
c3R5bGU9J2ZvbnQtZmFtaWx5OsvOzOU7Zm9udC1zaXplOjEyJz
6hvtTwyM6x4LytICqhvzwvc3Bhbj48L3A+fGJvcmRlcj0iMSIg
ICI7czoxNToibWVzc2FnZXBhZ2V0eXBlIjtzOjQ6Im5leHQiO3
M6MTU6Im1lc3NhZ2VwYWdlcnVsZSI7czoxNDQ6IjxzZWxlY3Qg
bmFtZT1kbGRob3Agb25jaGFuZ2U9ImphdmFzY3JpcHQ6d2luZG
93LmxvY2F0aW9uLmhyZWY9ZG9jdW1lbnQuYWxsLmRsZGhvcC52
YWx1ZSIgY2xhc3M9InNpemUxNCIgc3R5bGU9IndpZHRoOjI1MH
B4Ij5bcGFnZWFyZWFdPC9zZWxlY3Q+KiI7czoxODoibWVzc2Fn
ZXBhZ2V1cmxydWxlIjtzOjI2OiI8b3B0aW9uIHZhbHVlPSIuL1
twYWdlXSIgPiI7czoyMToibWVzc2FnZXBhZ2V1cmxsaW5rcHJl
IjtzOjQxOiJodHRwOi8vd3d3LmVuZXQuY29tLmNuL2FydGljbG
UvMjAwNy8wNzA2LyI7czoxNDoibWVzc2FnZXJlcGxhY2UiO3M6
MDoiIjtzOjE2OiJtZXNzYWdlcmVwbGFjZXRvIjtzOjA6IiI7cz
o3OiJ2ZXJzaW9uIjtzOjM6IjUuNSI7fQ==

复制代码

cctv1233 · 发表于 2007-7-7 01:20:57

这个搜狐的也是一样的情况！
http://yule.sohu.com/gangtaimingxing.shtml

# SupeSite Dump
# Version: SupeSite 5.5
# Time: 2007-07-07 01:15:41
# From: 紫龙娱乐网 ([url]http://spaces.zl06.net.ru[/url])
#
# This file was BASE64 encoded
#
# SupeSite: [url]http://www.supesite.com[/url]
# Please visit our website for latest news about SupeSite
# --------------------------------------------------------
YTozNzp7czo3OiJyb2JvdGlkIjtzOjI6IjY4IjtzOjQ6Im5hbW
UiO3M6MTM6InNvaHUtuNvMqMP30MciO3M6MzoidWlkIjtzOjI6
IjI1IjtzOjg6ImRhdGVsaW5lIjtzOjEwOiIxMTgzNzQyMTI0Ij
tzOjg6Imxhc3R0aW1lIjtzOjEwOiIxMTgzNzQxMTM3IjtzOjg6
InJvYm90bnVtIjtzOjE6IjciO3M6MTE6Imxpc3R1cmx0eXBlIj
tzOjY6Im1hbnVhbCI7czo3OiJsaXN0dXJsIjtzOjQyOiJodHRw
Oi8veXVsZS5zb2h1LmNvbS9nYW5ndGFpbWluZ3hpbmcuc2h0bW
wiO3M6MTM6Imxpc3RwYWdlc3RhcnQiO3M6MToiMCI7czoxMToi
bGlzdHBhZ2VlbmQiO3M6MToiMCI7czo2OiJhbGxudW0iO3M6ND
oiMTAwMCI7czo2OiJwZXJudW0iO3M6MToiMiI7czo3OiJzYXZl
cGljIjtzOjE6IjAiO3M6NjoiZW5jb2RlIjtzOjA6IiI7czoxMz
oicGljdXJsbGlua3ByZSI7czowOiIiO3M6OToic2F2ZWZsYXNo
IjtzOjE6IjAiO3M6MTQ6InN1YmplY3R1cmxydWxlIjtzOjkwOi
IoJ3BhZ2VuYXYnKS52YWx1ZSk7cmV0dXJuIGZhbHNlOyI+R288
L2E+W2xpc3RdPHRkIG5vd3JhcD48YSBocmVmPScvZ2FuZ3RhaW
1pbmd4aW5nLnNodG1sJz4iO3M6MTg6InN1YmplY3R1cmxsaW5r
cnVsZSI7czoyMToiPGxpPqGkPGEgaHJlZj0nW3VybF0nIjtzOj
E3OiJzdWJqZWN0dXJsbGlua3ByZSI7czowOiIiO3M6MTE6InN1
YmplY3RydWxlIjtzOjE4OiI8aDE+W3N1YmplY3RdPC9oMT4iO3
M6MTM6InN1YmplY3RmaWx0ZXIiO3M6MDoiIjtzOjE0OiJzdWJq
ZWN0cmVwbGFjZSI7czowOiIiO3M6MTY6InN1YmplY3RyZXBsYW
NldG8iO3M6MDoiIjtzOjEwOiJzdWJqZWN0a2V5IjtzOjA6IiI7
czoxODoic3ViamVjdGFsbG93cmVwZWF0IjtzOjE6IjAiO3M6MT
I6ImRhdGVsaW5lcnVsZSI7czowOiIiO3M6ODoiZnJvbXJ1bGUi
O3M6NDI6IjxkaXYgY2xhc3M9ImxlZnQgdGV4dCI+KsC01LSjul
tmcm9tXTwvZGl2PiI7czoxMDoiYXV0aG9ycnVsZSI7czowOiIi
O3M6MTE6Im1lc3NhZ2VydWxlIjtzOjgzOiI8ZGl2IGlkPSJzb2
h1X2NvbnRlbnQiIGNsYXNzPSJhcnRpY2xlIj5bbWVzc2FnZV08
ZGl2IGNsYXNzPSJlZGl0VXNyIiBhbGlnbj0icmlnaHQiPiI7cz
oxMzoibWVzc2FnZWZpbHRlciI7czo3MjI6IjxkaXYgc3R5bGU9
Im1hcmdpbi1ib3R0b206MTBweCI+KjwvZGl2Pnw8dHI+PHRkIG
FsaWduPSJjZW50ZXIiPlu147v3zbzGrL34yOvPwtK70rNdPC90
ZD48L3RyPnw8dGFibGUgYm9yZGVyPTAgY2VsbHNwYWNpbmc9MC
BjZWxscGFkZGluZz0wICBhbGlnbj0ibGVmdCI+KjwvaWZyYW1l
PjwvdGQ+PC90cj48L3RhYmxlPio8L3RkPjwvdHI+PC90YWJsZT
58PGlmcmFtZSo8L2lmcmFtZT58PFNQQU4gY2xhc3M9YXJ0aWNs
ZUxpbms+KDxBKkE+KTwvU1BBTj58PGEqPnw8L2E+fDxBKj58PC
9BPnw8c3R5bGU+Kjwvc3R5bGU+fDxkaXYgaWQ9ImNtczRfenV0
dV9uYXYxIj4qPC9kaXY+fDxUQUJMRSBzdHlsZT0iTUFSR0lOOi
A1cHgiIGNlbGxTcGFjaW5nPTEgY2VsbFBhZGRpbmc9OCB3aWR0
aD01MjAgYWxpZ249Y2VudGVyKjwvVEFCTEU+fDxUQUJMRSBjZW
xsUGFkZGluZz0yIHdpZHRoPSIxMDAlIiBiZ0NvbG9yPSNmZmFk
ODYgYm9yZGVyPTA+KjwvVEFCTEU+PC9URD48L1RSPjwvVEJPRF
k+PC9UQUJMRT48L1REPjwvVFI+PC9UQk9EWT48L1RBQkxFPnw8
dGFibGUgYm9yZGVyPTAgd2lkdGg9MTAwJSBjZWxscGFkZGluZz
0wIGNlbGxzcGFjaW5nPTA+KjwvdGFibGU+fDxTVFJPTkc+oaHL
0br80+nA1rbAvNK45bz+o6yw5sioy/nT0KOsKteq1NgqPC9TVF
JPTkc+fMvRuvzT6cDWtsC80rjlvP6jrLDmyKjX98a3o6wq16rU
2Hw8Rk9OVCBjb2xvcj0jZmYwMDAwPsrTxrWjuio8L0ZPTlQ+Ij
tzOjE1OiJtZXNzYWdlcGFnZXR5cGUiO3M6NDoicGFnZSI7czox
NToibWVzc2FnZXBhZ2VydWxlIjtzOjQ4OiI8ZGl2IHN0eWxlPS
JtYXJnaW4tYm90dG9tOjEwcHgiPltwYWdlYXJlYV08L2Rpdj4i
O3M6MTg6Im1lc3NhZ2VwYWdldXJscnVsZSI7czoyNDoiPG9wdG
lvbiB2YWx1ZT0iW3BhZ2VdIiA+IjtzOjIxOiJtZXNzYWdlcGFn
ZXVybGxpbmtwcmUiO3M6MDoiIjtzOjE0OiJtZXNzYWdlcmVwbG
FjZSI7czowOiIiO3M6MTY6Im1lc3NhZ2VyZXBsYWNldG8iO3M6
MDoiIjtzOjc6InZlcnNpb24iO3M6MzoiNS41Ijt9

复制代码

茄子用上面的规则采看看是不是第二分页采不到？

再把文章内容分页链接识别规则改成

option value=""

复制代码

看看！是不是第一页重复采集了！

好多网站都是这样的哦！汗，不知道是不是真的偶的规则有问题了@！

茄子 · 发表于 2007-7-9 16:07:25

针对分页是无序日期数字的情况，还不能完美支持采集
这个我们之后改进下看看

cctv1233 · 发表于 2007-7-9 17:20:26

还有那个分页第一页重复采集或者第二页丢失的问题呢？希望茄子也可以解决下！

。
就是8楼那个采集规则！

		自动登录	找回密码
密码			立即注册