Skip to content
PayloadPuffin.
Join the waitlist

Source transforms

A provider sends a form, an XML document, or JSON in its own shape, and you want events with your own type and only the fields you use. A source can link a transform of kind source: a short piece of JavaScript that receives each request the source stores and returns the event. Several sources can link the same one. Without one, a JSON body becomes the payload as received, and any other format becomes an unbuilt event.

What it receives and returns

A source transform defines async function handler(request, { type, channels }) and returns the event:

JavaScript
async function handler(request, { type, channels }) {
	return { type, channels, payload: await request.json() };
}
  • request is a standard Request with the method, body and stored headers from the provider's request. Its URL is the source's URL without the token. Credential headers are masked before storage: Authorization, Proxy-Authorization, Cookie, Set-Cookie, and headers whose names contain token, secret, api-key, apikey or password. The transform can't recover their original values. Use source signature verification to verify the provider before storage.
  • type and channels are the source's defaults.
  • Return an object with payload, any JSON value. PayloadPuffin serializes it with JSON.stringify, so a string payload becomes a JSON string.
  • Add type or channels to override the source's defaults. An omitted one keeps the default. The type follows the event type rules, and the channels are channel slugs.

A source transform returns the event, not a Response. A Response is what a destination's transform returns, to shape the request that destination receives.

Recipes

A JSON body in the provider's shape:

source-transform-json.js
// A JSON request: pick the fields you need and name the event after the provider's own type.
async function handler(request, { type, channels }) {
	const body = await request.json();
	return {
		type: `billing.${body.type}`, // for example billing.invoice.paid
		payload: { id: body.id, amount: body.data.amount },
	};
}

A urlencoded form:

source-transform-form.js
// A urlencoded form, as some providers send: every field becomes a member of the payload.
async function handler(request, { type, channels }) {
	const form = await request.formData();
	return { payload: Object.fromEntries(form) };
}

Plain text or XML:

source-transform-text.js
// Plain text or XML: the sandbox has no XML parser, so read the text and keep what you need.
async function handler(request, { type, channels }) {
	const text = await request.text();
	const id = /<id>([^<]*)<\/id>/.exec(text)?.[1] ?? null;
	return { channels: [...channels, 'xml'], payload: { id, body: text } };
}

Draft, preview and publish

A source transform has a saved draft and live code, like a destination transform. Only the live code builds events, so you can edit and try a draft without changing what arrives.

  1. Create it with Create a transform, sending kind: "source", a name and the first draft as code, and link it by sending its ID as transformId when you create or update a source.
  2. Run it on a sample request with Preview a transform: send request, with a method, a content type, headers and a text body. Add sourceId to run with that source's default type and channels. It answers with the event the code builds as event, or why it failed. Each preview spends one of the app's daily sandbox previews, shared with destination transforms.
  3. Make it live with Publish a transform, sending the draft's savedAt as you last read it. Every source linking it builds with it from then on.

PayloadPuffin selects the live code when it builds an event, including requests already accepted and waiting to be built. An unbuilt event you rerun also uses the live code. The source's default type and channels stay as captured when the request arrived.

Updating a source with transformId: null unlinks its transform. A source transform needs the sources:read scope to read and sources:write for everything else.

In the console, create source transforms on the Transforms page, then pick one in the Transform field when you create a source or under Edit source. The source's Transform section links to the transform's page, where Edit code opens the editor. Test in sandbox runs the editor's code on the sample request.

When it fails

When the code throws, returns something that isn't an event, returns a payload over 262,144 bytes (256 KiB) once serialized, or runs for more than 5 seconds, the request becomes an unbuilt event whose reason is the error. Fix the code, publish it, and rerun the event. A source transform runs in the same sandbox as a destination's, with the same limits: no network, no secrets, and 50 milliseconds of CPU. See Transforms.