name

locale

Name_collectionQuery

screenName

description

langs

parent

children

projectTotal

CategoryProjects_query

after

categoryName

first

projects

edges

pageInfo

query Name_collectionQuery(
  $name: String
  $locale: String
) {
  ...CategoryProjects_query_30PAWs
  category(name: $name, locale: $locale) {
    name
    screenName
    description
    langs
    tags {
      name
      id
    }
    parent(locale: $locale) {
      name
      screenName
      id
    }
    children(locale: $locale) {
      name
      screenName
      children(locale: $locale) {
        name
        screenName
        id
      }
      projects(locale: $locale) {
        id
        name
        fullName
        sourceUrl
        starCount
        forkCount
        description
        remark
        lang
        homepageUrl
        screenshot
        descTrans
        tags {
          name
          id
        }
        user {
          name
          remark
          id
        }
      }
      id
    }
    projectTotal
    id
  }
}

fragment CategoryProjects_query_30PAWs on Query {
  projects(after: "0", first: 10, categoryName: $name, locale: $locale) {
    edges {
      node {
        id
        name
        fullName
        sourceUrl
        starCount
        forkCount
        description
        remark
        lang
        url
        homepageUrl
        screenshot
        category(locale: $locale) {
          name
          screenName
          id
        }
        descTrans
        tags {
          name
          id
        }
        wiki
        chinaFlag
        createAt
        updateAt
        __typename
      }
      cursor
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}


crawling

0fenbei

Airing

幸明老匹夫

R Max Espinoza

Colly

 brianway

Roy Binux

Scrapy project

该页面收集了与爬虫相关的框架和应用程序。

crawling-frameworks

spiders

awesome-spider

News, full-text, and article metadata extraction in Python 3. Advanced docs:

newspaper

Create agents that monitor and act on your behalf.  Your agents are standing by!

huginn

wechat-spider

A Powerful Spider(Web Crawler) System in Python.

pyspider

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

twint

scrapy-redis

DHT Spider + BitTorrent Client = P2P Spider

p2pspider

proxy_pool

A scalable web crawler framework for Java.

webmagic

Elegant Scraper and Crawler Framework for Golang

colly

webporter

bilibili-user

👾 Fast and simple video download library and CLI tool written in Go

Scrapy, a fast high-level web crawling & scraping framework for Python.

scrapy

一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )

网络爬虫该页面收集了与爬虫相关的框架和应用程序。

📂爬虫框架

📂爬虫应用

该分类下的开源项目

推荐标签

网络爬虫 该页面收集了与爬虫相关的框架和应用程序。

📂爬虫框架

📂爬虫应用

该分类下的开源项目

推荐标签

网络爬虫该页面收集了与爬虫相关的框架和应用程序。