Webhook vs API: choosing how two systems talk

Webhooks vs polling vs request-time calls, side by side

A webhook is an API call in the other direction: the source sends the request and your system receives it. A comparison of API polling vs webhooks needs a third column, for the call you make at the moment of use. A change in one system reaches another in one of three ways.

• Webhook. The source calls your endpoint after a record changes.
• Polling. Your system asks the source, on a schedule, for records changed since the last run.
• Request-time call. Your system asks for one record at the moment a person or a process needs it.

A webhook can arrive without the record it refers to. Procore calls its webhook a notification that a change happened, and your service follows it with a GET request to read the changed resource. A webhook integration on Procore makes API calls from its first day, and those calls count against a rate limit.

How to decide in three questions

Take the questions in order, and stop at the first one that settles the design.

1. Does the source publish an event for the change you need? Read the vendor's event list before the design meeting. Without that event, polling is your only option. Check the plan: Docusign states that account-level Connect is available to specific plans, while envelope-level notifications work on all of them.

2. How long can the receiving team wait? Finance may need the signed contract in the ERP within a minute, and that calls for a webhook. If tomorrow morning is soon enough, poll. A poll runs on a timer from inside your network, so your team has no public endpoint to secure and no queue to operate. A person who must see the current value before acting needs a request-time call.

3. What does one missed change cost? A missed "envelope completed" event can leave a customer signed and unprovisioned for days. At that cost, run a reconciliation poll behind the webhook. A dashboard that shows yesterday's totals delays no order, so one mechanism is enough there.

When to use webhooks and polling together

Use both for any record where a gap costs money. The webhook delivers the change within seconds, and the poll catches what the webhook dropped.

Procore calls webhook delivery best-effort, with notifications that can arrive late or go missing during an incident, and it advises a periodic reconciliation sync against the REST API. On its rate limiting page, Procore tells you to keep polling as a lower-frequency reconciliation pass, because a short interval spends quota to confirm that nothing changed.

Let the webhook handle each change as it arrives, and run a poll every hour for records modified since the last successful run. Save the cursor after the run commits. Start the next run a few minutes before the cursor, because search indexes lag. HubSpot notes that new or updated records may take a few moments to appear in search results.

Add a request-time call for a value that must be current. Your system reads the customer's credit status from the ERP at the moment it generates the contract, because the copy from last night's poll is up to a day old.

What goes wrong in production

Five failures appear after launch, once volume grows and a server goes down for the first time.

The same event arrives twice

Senders retry after a slow or failed response, and the first attempt may have done its work. HubSpot states that it does not guarantee a single notification per event. Procore warns that a delivery for the same event can arrive more than once. Your handler then raises two invoices for one signed order.

Keep a record of processed events: store each event under a unique key and skip an event whose key you hold. The guide to idempotency keys covers how to build the key and how long to keep it.

Events fire while your receiver is down

Each sender retries for a fixed period and then stops.

A server that stays down from Friday evening to Tuesday morning outlasts Procore's 12 hours and HubSpot's 24.

Build the endpoint as a thin receiver that verifies the signature and writes the raw message to a durable queue before it answers. Docusign recommends that design, in which you add the message to a nonvolatile queue and then send the 200 response. Then run the reconciliation poll from the section above. After an outage longer than the retry period, the next run reads the changes the sender stopped sending. Docusign lets an administrator publish missed Connect messages by hand.

A burst hits a rate limit

A bulk import in the source produces thousands of events in a minute. HubSpot sends up to 10 concurrent requests per connected account, each holding up to 100 events. That puts 1,000 events in flight. A handler that calls HubSpot back once per event makes 1,000 API calls, and HubSpot allows 110 requests every 10 seconds per account for an app distributed through its marketplace. The API answers the excess with status 429, which RFC 6585 defines as too many requests in a given amount of time. A request-time call behind a busy screen reaches the same limit with no import.

Separate receipt from processing. Acknowledge the delivery and queue the events. A fixed number of workers then drains the queue, and each worker reads the rate limit headers and slows down before the remaining count reaches zero. Procore gives the same advice for webhook-driven integrations: store events in a database and process them once the limit refreshes. If the API offers a batch read, fetch the changed records in batches: HubSpot advises batch APIs and caching in its usage guidelines.

Events arrive out of order

A retry can deliver an old event after a newer one. HubSpot does not guarantee that notifications arrive in the order they occurred, and it stamps each one with an occurredAt time. Docusign states the rule for its older aggregate formats: messages go out in event order, except after a notification failure. Its JSON SIM page states no delivery order. Its retry page states that Connect processes each envelope on its own: while a message for envelope A waits for a retry, Connect sends the messages for envelope B. A handler that writes each payload over the record replaces the newer status with the older one, and a completed deal shows as "sent".

Compare before you write, with data the event carries. Store the time of the last event you applied to each record, and skip an older one. HubSpot's occurredAt and Procore's timestamp hold the time the event occurred. A Docusign message names the event that generated it, so rank the events and skip one that would move an envelope from completed back to sent. On HubSpot and Procore you can read the record's current state from the API before you write. On Docusign, skip that read: the polling rule below allows one status request per envelope per 15 minutes, and Docusign can send three events for one envelope inside that time.

The poll loses changes between runs

A poll reads the state at the moment it runs. A deal that moves to "contract sent" and back to "negotiation" between two runs shows no change. A poll can exceed the result limit of the API it calls. HubSpot's search endpoints return at most 10,000 results for one query, and a request for a page beyond that returns a 400 error. The first large import pushes a poll that runs once a night past that ceiling.

Use a webhook where your process depends on each transition. Split a large poll into time windows, each small enough to stay under the result limit.

On the platforms you connect

Docusign. Docusign Connect is the Docusign webhook service. It sends an HTTPS POST to your listener after a subscribed event occurs in an eSignature workflow. Docusign asks you to answer within five seconds. With the Require Acknowledgment option on, Connect records a failure if no 200 arrives within 100 seconds, and the retries begin. A message in the JSON SIM format names the event and the envelope:

{
  "event": "envelope-completed",
  "uri": "/restapi/{apiVersion}/accounts/{accountId}/envelopes/{envelopeId}",
  "retryCount": 0,
  "configurationId": 10242474,
  "apiVersion": "v2.1",
  "generatedDateTime": "2021-05-14T23:15:56.3570000Z",
  "data": {
    "accountId": "0000000-0000-0000-0000-000000000000",
    "envelopeId": "cc6803ce-xxxx-xxxx-xxxx-3c4f50733703"
  }
}

Docusign limits polling to one GET status request per envelope per 15 minutes for each app. A breaching request goes through, but Docusign flags it as a violation, which can fail the app's go-live review. On the same page, Docusign sets the account default at 3,000 requests per hour, with a burst limit of 500 calls per 30 seconds in production. A request-time status check needs a cache per envelope, and Docusign advises polling at 20-minute intervals rather than 15. The guide to Docusign API integration covers the calls.

NetSuite. To push from NetSuite, you can deploy a script, and Oracle documents its two parts. User event scripts run on the NetSuite server when someone creates or updates a record. The N/https module sends HTTPS calls to a third party. For polling, a scheduled script runs on a recurring schedule. NetSuite governs inbound calls by concurrency, the number of requests in flight at once. For contracts from June 2020, the base limit is 5 on the Standard tier, 15 on Premium and 20 on Enterprise and Ultimate, plus 10 for each SuiteCloud Plus licence. A service that writes forty signed envelopes to NetSuite in parallel exceeds a base limit of 5, so size the worker pool below the limit.

HubSpot. HubSpot publishes events for contacts, companies, deals, tickets, products and line items. In its Webhooks API guide, written for legacy public apps, HubSpot gives your endpoint five seconds to answer a batch and states that deliveries do not count against the API rate limit. A HubSpot private app gets 100 requests per 10 seconds on Free and Starter, and 190 on Professional and Enterprise. The search endpoints a poll would use have a lower ceiling: five requests per second per account and 10,000 results per query.

Procore. Procore sends an event that names the resource and its ID, and your service reads the record with a follow-up GET. Procore's request timeout is 5 seconds after the connection opens. The API has an hourly limit and a 10-second spike limit, and Procore counts failed requests against both, so a 404 consumes quota like a successful call. Procore tells you to read the X-Rate-Limit headers on each response and pace your requests from those values, because the limits can change.