Skip to content
Naibo Wang edited this page Apr 23, 2024 · 46 revisions

教程导航

Issues中的文档和示例(作为下面教程的补充)

https://github.com/NaiboWang/EasySpider/issues?q=label%3Adocumentation+

EasySpider介绍:中国地震台网采集案例

https://www.bilibili.com/video/BV1th411A7ey/

Docker运行示例

https://github.com/NaiboWang/EasySpider/wiki/Docker%E8%BF%90%E8%A1%8C%E7%A4%BA%E4%BE%8B

设置页面向下滚动

https://www.bilibili.com/video/BV1G14y1o7Qa/

如何爬取需要登录才能爬的网站

https://www.bilibili.com/video/BV1BN411t71C/

循环点击列表中每个链接进入详情页采集详情页内容+设计时动态调试+动态JS

https://www.bilibili.com/video/BV12V411D7RZ/

实战采集汽车网文章内容并下载文章内图片

https://www.bilibili.com/video/BV14u4y1x7S5/

Ebay样例博客

https://blog.csdn.net/ihero/article/details/130805504

多种选中子元素模式区别展示+将最后一次提取到的数据作为变量进行输入+定时执行任务展示

https://www.bilibili.com/video/BV1N94y1a7Lp/

【重要】自定义条件判断之使用循环项内的JS命令返回值

https://www.bilibili.com/video/BV18C4y1V7J7/

如何执行自己写的JS代码和系统代码 (自定义操作)

https://www.bilibili.com/video/BV1UH4y1f7BM/

如何自定义循环和判断条件

https://www.bilibili.com/video/BV18w411a77e/

如何对元素和网页截图及命令行执行指南

https://www.bilibili.com/video/BV1ch4y1E7cn/

OCR识别元素内容功能

https://www.bilibili.com/video/BV1GP411y7u4/

如何爬需要输入验证码的网站

https://www.bilibili.com/video/BV1Rw411C7Hs/

如何切换IP池和使用隧道IP - 打开详情页采集案

https://www.bilibili.com/video/BV1zw411w7BN/

流程图执行逻辑解析 - 58同城房源描述采集案例

https://www.bilibili.com/video/BV14N4y1o73Y/

MacOS系统设计和执行eBay网站爬虫任务教程

https://www.bilibili.com/video/BV1E34y137fT/

如何定时执行任务

https://github.com/NaiboWang/EasySpider/issues/22

如何通过API调用来获得任务执行ID

https://github.com/NaiboWang/EasySpider/wiki/API-Invoke-Example

如何使用命令行执行任务|Commandline Arguments Instruction

https://github.com/NaiboWang/EasySpider/issues/28

https://github.com/NaiboWang/EasySpider/wiki/Argument-Instruction

任务流程有多个提取数据操作时,软件的执行和输出逻辑

https://github.com/NaiboWang/EasySpider/issues/35

并行多开,同时执行多个任务教程

https://www.bilibili.com/video/BV1Dj411b77M/

当前循环项包含元素的XPATH的写法

https://github.com/NaiboWang/EasySpider/wiki/%E5%BD%93%E5%89%8D%E5%BE%AA%E7%8E%AF%E9%A1%B9%E5%8C%85%E5%90%AB%E5%85%83%E7%B4%A0%E7%9A%84XPATH%E7%9A%84%E5%86%99%E6%B3%95

Python代码运算后的结果作为文本框的输入

https://www.bilibili.com/video/BV1kF411R7VJ/

Cloudflare反爬处理(已失效)

https://www.bilibili.com/video/BV1Ph4y1E7R9/

MySQL数据库写入教程

https://www.bilibili.com/video/BV1os4y1679S/

Exec和Eval模式使用教程

https://github.com/NaiboWang/EasySpider/wiki/EXEC%E5%92%8CEVAL%E7%94%A8%E6%B3%95%E7%A4%BA%E4%BE%8B

执行JavaScript语句时用正则表达式示例

https://github.com/NaiboWang/EasySpider/wiki/%E6%89%A7%E8%A1%8CJavaScript%E8%AF%AD%E5%8F%A5%E6%97%B6%E7%94%A8%E6%AD%A3%E5%88%99%E8%A1%A8%E8%BE%BE%E5%BC%8F%E7%A4%BA%E4%BE%8B

调整浏览器窗口大小&最大化浏览器窗口

https://github.com/NaiboWang/EasySpider/wiki/%E8%B0%83%E6%95%B4%E6%B5%8F%E8%A7%88%E5%99%A8%E7%AA%97%E5%8F%A3%E5%A4%A7%E5%B0%8F&%E6%9C%80%E5%A4%A7%E5%8C%96%E6%B5%8F%E8%A7%88%E5%99%A8%E7%AA%97%E5%8F%A3

操作台挡住登录页面的处理办法

https://github.com/NaiboWang/EasySpider/wiki/%E6%93%8D%E4%BD%9C%E5%8F%B0%E6%8C%A1%E4%BD%8F%E7%99%BB%E5%BD%95%E9%A1%B5%E9%9D%A2%E7%9A%84%E5%A4%84%E7%90%86%E5%8A%9E%E6%B3%95

循环次数设定(包括无限循环)及检测到页面内容才提取数据

https://github.com/NaiboWang/EasySpider/wiki/%E5%BE%AA%E7%8E%AF%E6%AC%A1%E6%95%B0%E8%AE%BE%E5%AE%9A%EF%BC%88%E5%8C%85%E6%8B%AC%E6%97%A0%E9%99%90%E5%BE%AA%E7%8E%AF%EF%BC%89%E5%8F%8A%E6%A3%80%E6%B5%8B%E5%88%B0%E9%A1%B5%E9%9D%A2%E5%86%85%E5%AE%B9%E6%89%8D%E6%8F%90%E5%8F%96%E6%95%B0%E6%8D%AE

无下一页按钮只有具体页面按钮时的翻页方法(EXEC和EVAL教程)

https://github.com/NaiboWang/EasySpider/wiki/%E6%97%A0%E4%B8%8B%E4%B8%80%E9%A1%B5%E6%8C%89%E9%92%AE%E5%8F%AA%E6%9C%89%E5%85%B7%E4%BD%93%E9%A1%B5%E9%9D%A2%E6%8C%89%E9%92%AE%E6%97%B6%E7%9A%84%E7%BF%BB%E9%A1%B5%E6%96%B9%E6%B3%95%EF%BC%88EXEC%E5%92%8CEVAL%E6%95%99%E7%A8%8B%EF%BC%89

What is EasySpider

EasySpider is a no-code visual web crawler for no-coders and also experts to crawl data from the World Wide Web. It can design and execute web crawling tasks without coding via GUI. The workflow of a new web crawling task can be visually programmed by following EasySpider's visual wizard on the target webpages using an intuitive point-and-click interface. The generated crawler task can then be easily invoked locally or as a web service. Our EasySpider is cross-platform and flexible to adapt to different web-resources. It also supports advanced configuration for complicated tasks and extension. The whole system is open-sourced and transparent for free-access, which avoids possible privacy leakage.

Features of EasySpider

  • Interactive and user-friendly: it provides a user-friendly point-to-click GUI for quick and easy task design for non- programmers. Only a little basic knowledge, such as XPath, is needed to modify the task flow.

  • Flexible: it is able to adapt to various web resources and platforms, dynamic content support, such as collect web content generated by JavaScript (AJAX).

  • Configurable: it supports easy design of complicated crawling tasks with unlimited loops, if-conditions, and fields.

  • Cross-platform: supports on Windows, Linux, and MacOS.

  • Open-Source & Trustworthy: the source code is publicly available, making it a transparent and white-box software, with no privacy leakage risk.

  • Free: all features listed in this documentation are free of charge.

UI of EasySpider

  • UI of Crawler Task design

image

  • UI of Task Information

service_information

  • UI of Task Invocation

service_invoke

  • Example Collected Results

results

Clone this wiki locally