0%
HomeBlogAnnouncements
Pipelines is now streaming in beta

Pipelines is now streaming in beta

Author: Vinny Mullin

Reviewed by Brady Werkheiser


Published on April 24, 20242 min read
alchemy pipelines beta announcement

Today, Alchemy is announcing Pipelines, the easiest way to stream real-time blockchain data into your infrastructure. If you’re interested, sign-up for beta access here.

Developers building on top of blockchain data currently have a few options:

  1. Use node APIs (i.e. Supernode).

  2. Use higher-level APIs (i.e. NFT API, Token API).

  3. Use managed indexers (i.e. Subgraphs).

For engineers with application-specific needs, ingesting data from node APIs gives you control over how you want to process and join data.

However, building and maintaining a data pipeline can take weeks of engineering time and expertise. You have to set up a performant backfill, a reliable ingestion service, and a mechanism for dealing with reorgs. On top of all of that, you have to check for data completeness. All of this is time wasted that could be spent on building product features.

Alchemy Pipelines solves all of this with a point-and-click interface for creating data pipelines.

Simply configure a data source (e.g. decoded logs) and a destination (e.g. Postgres), and data will start flowing through instantly — all in less than 5 minutes.

create and define your web3 data pipeline

Out of the box, Pipelines comes with the following features:

  • Easy-to-use interface: Set up your first pipeline in < 5 minutes using our intuitive web UI.

  • Fast historical backfills: Write data straight to your destination of choice 100x faster than with JSON RPCs.

  • Real-time ingestion of incoming data: Gain peace of mind with reliable data ingestion within seconds of it being published on-chain.

  • Automatic re-org handling: Automatically handle chain re-orgs with out-of-the-box data retraction.

  • ABI decoding: Save time by using our built-in log decoding.

If you’re ingesting a significant amount of data via Webhooks or eth_getLogs, Pipelines could save you both time and money over your in-house system. Here are a few ways our beta customers are using Pipelines:

User was looking to ingest all transfers of the Aave token. They set up a Pipeline in under 3 minutes, and the data (over 1.3M transactions) streamed into his database within 10 minutes.

Customer previously used an in-house eth_getLogs polling system to ingest events from their NFT collection, but was struggling with maintenance time and reorgs. Pipelines helped them set up a new ingestion system in less than 5 minutes.

If you’re interested in trying Pipelines, sign up for our beta waitlist!

The team is iterating quickly based off of user feedback. Interested in a custom setup or looking for a specific data source, chain, or destination? Email the Pipelines team.

Section background image

Build blockchain magic

Alchemy combines the most powerful web3 developer products and tools with resources, community and legendary support.

Get your API key