name

view_collectionQuery

locale

screenName

langs

CategoryOverview_category

CategoryProjects_query

after

categoryName

first

projects

edges

pageInfo

children

parent

projectTotal

query view_collectionQuery(
  $name: String
) {
  ...CategoryProjects_query_3Rf3hg
  category(name: $name, locale: "zh") {
    name
    screenName
    langs
    ...CategoryOverview_category
    id
  }
}

fragment CategoryOverview_category on Category {
  children {
    name
    screenName
    children {
      name
      screenName
      id
    }
    projects {
      id
      name
      fullName
      sourceUrl
      starCount
      forkCount
      description
      lang
      homepageUrl
      screenshot
    }
    id
  }
  parent {
    name
    screenName
    id
  }
  projectTotal
}

fragment CategoryProjects_query_3Rf3hg on Query {
  projects(after: "0", first: 10, categoryName: $name) {
    edges {
      node {
        id
        name
        fullName
        sourceUrl
        starCount
        forkCount
        description
        lang
        homepageUrl
        screenshot
        __typename
      }
      cursor
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}


crawling

crawling-frameworks

spiders

wechat-spider

python爬虫框架。简单易上手，自带在线编程和任务管理界面

pyspider

基于Scrapy和Redis的分布式爬虫框架

scrapy-redis

一个种子嗅探器，它从 BitTorrent 网络获取人们下载音乐、电影、游戏、文档等等时所用的种子

📂爬虫框架

scrapy @scrapy
37.16 k

pyspider @binux
13.91 k

scrapy-redis @rmax
3.19 k

📂爬虫应用

SinaSpider @LiuXingMing
2.87 k

p2pspider @fanpei91
2.81 k

wechat-spider @bowenpay
2.46 k

webporter @brianway
2.09 k

📂爬虫框架

scrapy@scrapy37.16 k

pyspider@binux13.91 k

scrapy-redis@rmax3.19 k

📂爬虫应用

SinaSpider@LiuXingMing2.87 k

p2pspider@fanpei912.81 k

wechat-spider@bowenpay2.46 k