How to scrape the Y Combinator companies list

The Y Combinator directory at ycombinator.com/companies is the canonical list of YC-funded startups, and it is served to anyone without an account. Exporting it is mostly a question of knowing how the directory addresses its own data, and where its one hard ceiling sits.

What the YC directory exposes

The directory page is a shell that queries a search index with a public, read-only key embedded in the page — the same key every anonymous visitor's browser uses. Each company then has its own page with founders, jobs, news and launch posts, and each job has a page with the full description and the hiring manager. On 1 October 2026 the index held 6,267 companies, 221 of them tagged Latin America and 1,487 flagged as hiring.

Step 1 — choose one row type

One run of the Y Combinator Scraper produces one type of row, set by outputType:

Each row is charged one row event by depth, never two added together. The email pattern is a separate $0.01 event, charged at most once per domain and only when evidence backs it.

Step 2 — filter with YC's own labels

regions, batches, industries and tags take YC's own labels and are checked against the live directory before the run starts. Batches accept full names or codes — W24, S25, F24, and X25 for Spring. The default is Latin America; an empty regions means the whole directory.

{
  "outputType": "founders",
  "regions": [],
  "batches": ["Fall 2026"],
  "maxItems": 250
}

Two filters behave differently from the rest. stages is applied after the search, because the directory cannot filter by stage, so a stage-filtered run reads companies it then drops. And the jobs output always restricts to hiring companies, which in Latin America meant 18 of 221 on 1 October 2026.

The 1,000-result ceiling and the batch split

One directory query returns at most 1,000 results, whatever the total says. A whole-directory export with one query would silently stop at the first thousand. The Actor splits any search larger than that by YC batch, runs each slice, and deduplicates; measured that way it reached 6,267 of 6,267 companies.

What is filled in, measured

Measured on the Actor's own output on 1 October 2026: 80 Latin American companies with their pages read, 80 founders and 60 jobs. Rates for other regions were not measured field by field; a wider 310-company sample read from the source gives company LinkedIn 88.1% and team size 99.7%.

Numeric salary was present on 95% of Latin American jobs and 70% of Remote jobs in the same 60-job sample; the gap is companies that list no range. The Actor's own development record notes a correction in build 0.1.5: a range written with a suffix, such as / monthly or CAD, had produced salaryMax equal to salaryMin on 820 of 6,112 jobs in one platform run, and a salaryPeriod field was added. Why the site publishes corrections rather than quietly applying them is in publishing measured fill rates.

Founder email candidates, read carefully

91.1% of the 80 founders got an emailCandidate. That number is easy to misread. Only 6 of the 80 companies had published an email that let the Actor match a pattern to a name; for the rest, the candidate is the provider's most common format applied to the founder's name, marked emailCandidateInferred: true with a confidence of 0.05 to 0.10.

{
  "fullName": "Ana Example",
  "emailCandidate": "ana@acmepay.example.com",
  "emailCandidateConfidence": 0.75,
  "emailCandidateInferred": false
}

Filter on emailCandidateInferred: false before treating a candidate as anything more than a format. Neither case is verified: the Actor resolves MX records and never connects to a mail server. The difference between the layers is the subject of what is stated and what is inferred, and the obligations attached to using these addresses at all are in founder data under the GDPR and CCPA.

What a run costs

Measured on the Apify platform on 2 October 2026, at the default 512 MB and without a proxy:

The 500-company run paid for 38 evidence-backed patterns, 7.6% of domains. Across the 14 platform runs made that day, 0 of 2,177 requests were blocked or rate-limited.

robots.txt and YC's terms

ycombinator.com/robots.txt, fetched on 10 October 2026, disallows /companies?*, /library?* and /verify/* for all user agents, and allows everything else. The Actor requests /companies without a query string, company pages and job pages, and its URL guard rejects any query string on those paths. How to read a file like that is covered in robots.txt as a specification.

Y Combinator's Site Terms of Use go further than robots.txt. They prohibit data mining, robots and scraping; they ask a blocked visitor not to circumvent the block, for example with a proxy IP; and they prohibit harvesting contact information to send unsolicited email. The Actor connects directly by default and measured no blocks, and its README states that it switches to a fresh session if the site starts blocking. Those are the facts; whether a given use is acceptable under the terms is for the person running it to judge.

The same tension, with a different decision, is described for the Wellfound Actor in how to scrape Wellfound jobs.