Real-Time Serverless Data Transformations: PII Detection and Hashing

Modern data platforms, such as Tealium, enable developers to manipulate event data on the fly as it streams through the system. Tealium Functions provide this important capability to developers via a serverless environment where user-defined JavaScript code can be run to transform, clean, and enrich incoming data in real time. Because Tealium functions are serverless, you don’t need to set up your own server, you can inject custom logic directly into Tealium’s intelligent data pipeline. This unlocks flexibility for data handling while Tealium takes care of the infrastructure, scaling, and low-latency execution.

Some key benefits of using Tealium Functions include:

• Serverless execution: Your code runs on Tealium’s infrastructure, so no servers to manage or deploy.
• Real-time processing: Transformations occur immediately as data is collected, not hours later in a batch job.
• Flexibility: You can write custom JavaScript to meet any data requirement or business rule.
• Scalability: The function will automatically scale to handle high event volumes without performance tuning on your part.
• Cost-efficient: Transformation and data wrangling without the compute cost. 

Common use cases for Tealium Functions range from simple data cleanup to complex routing logic. For example, you can perform data normalization and enrichment, mask or anonymize PII (personally identifiable information), reformat data to standard conventions, filter or route events based on conditions, or even compute custom metrics on the fly. Essentially, if you have a custom rule or transformation that should happen to every incoming event (or to specific events), Tealium Functions might be a good way to handle it.

Types of Tealium Functions

Tealium currently supports three different types of Functions: Data Transformation Functions, Event Functions, and Visitor Functions. The table below compares each:

In this article, we will focus on Data Transformation Functions, which execute in real time immediately after an event is collected and before it is processed further. This makes them ideal for tasks like validating or modifying incoming event data, removing or hashing sensitive information, flattening nested structures, etc., right as the data enters the system. By handling it at ingestion time, you ensure that all downstream systems only see cleaned or anonymized data. This reduces risk and saves you from doing additional processing later. In fact, it’s considered best practice that any user identifiers containing PII (like emails or phone numbers) be hashed or obfuscated before they are used elsewhere, to protect user privacy. 

How Tealium’s Data Pipeline Works (and Where Functions Fit)

Tealium’s event data pipeline can be summarized in three stages:

1. Data Collection: Data is collected from various sources (web, mobile, servers, etc.) and sent to Tealium. This is the entry point – for instance, via Tealium Collect API or Tag Manager – where raw events arrive with their payload (often referred to as the event’s data layer or UDO, Universal Data Object).
2. Transformation: The incoming event data is then processed within Tealium. This is where Functions execute, applying any custom JavaScript transformations or enrichments you’ve defined. It’s the core real-time processing step where data can be modified, filtered, or augmented on the fly.
3. Data Activation: Finally, the (now transformed) data is passed along to various outputs: it could be stored or used to update user profiles, and is forwarded to downstream tools via connectors (APIs, email marketing, analytics, etc.) for real-time action or analysis.

Figure 1. A simplified Tealium Data Pipeline. 

In the above diagram, the green node indicates where a Data Transformation Function would fire during the event’s journey. Because the function runs so early in the pipeline, any changes it makes (such as masking a piece of data) will persist through the rest of the flow. Downstream systems and even Tealium’s own AudienceStream attributes will only see the modified values.

To summarize, Tealium Functions allow you to inject custom code into the server-side stream of data. This serverless approach means you get the power of real-time data handling (similar to writing code in a microservice or cloud function) but managed entirely within Tealium’s interface. Now, let’s look at a concrete example to make this more tangible.

Use Case: Detecting and Hashing PII (Email Addresses)

One common requirement is to prevent sensitive personal data from flowing through systems in plain text. For instance, you might want to detect any email addresses coming in as part of event data and replace them with a hashed value. Hashing is a one-way encryption that converts the email into an irreversible string (often using SHA-256 or similar algorithms). This protects user privacy while still allowing you to use the hashed value as an identifier for matching purposes (since the same email will always hash to the same string) (Best practices for selecting user identifiers | Guides | Tealium Docs).

In our scenario, we’ll create a Tealium Function that scans each incoming event for any values that look like email addresses. If found, it will replace those values with a SHA-256 hash. We’ll also set a flag in the event data to mark that potential PII was detected (which could be useful for auditing or conditional logic in connectors). We want this to happen in real time, before the data is stored or sent out of Tealium.

To achieve this, we’ll use a Data Transformation Function with a custom trigger condition and a bit of JavaScript code:

• Trigger Condition: “If any field in the event data looks like an email address.” We can define this using a JSONPath selector and a regex (regular expression) pattern to match email strings.
• Action (Function Code): Hash those email fields using a cryptographic hash (SHA-256) and mark the event accordingly.

Below, we’ll walk through setting up this function step by step.

Implementation: Building the Email-Hashing Function in Tealium

Follow these steps in your Tealium server-side Functions UI to create and test the PII hashing function:

1. Create a new Function: In Tealium’s Customer Data Hub interface, navigate to Server-Side > Functions and click on + Add Function. This will start the setup for a new server-side function.
2. Choose function type and name: When prompted, select Data Transformation as the function type (since we want to transform incoming event data). Give the function a descriptive name, e.g., “Email Hashing”. (You can use the same name for the function’s trigger as well for clarity.)
3. Define the trigger rule: After clicking Continue, you’ll configure a custom trigger condition for when this function should run. In the Trigger Name, enter something like “PII Email Detection” (or reuse the function name). For the condition: set the Selector to $.data – this JSONPath points to the entire event data object (Create a function | Manage functions | Tealium Docs). Then choose the operator “matches regex” and enter a regex pattern that matches email addresses. For example, use:

/\b[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}\b/im

1. This pattern will match typical email formats (user@domain.tld). Essentially, it looks for a string containing @ and a “dot” with valid characters around them. By setting the selector to the root of the data, this condition will be true if any field in the event data matches the regex. (Under the hood, Tealium will evaluate this regex against a string representation of the event data or its fields.)
   
   Figure: Configuring the custom trigger in Tealium’s Functions UI. Here, the selector is $.data and the operator is “matches regex” with an email pattern, meaning the function will trigger for events containing an email address in any field.
2. Proceed to code editor: Once your condition is set up, click Continue. Tealium will create the function and take you to the Code tab where you can write the JavaScript logic. (By default, some sample code or comments may be present — you can remove those.)
3. Write the transformation code: Copy and paste the following code into the function’s code editor:

import CryptoES from "crypto-es";

// If needed, specify any keys that should be exempt from hashing (whitelist)
const allowedPIIKeys = ["mid_email"]; // e.g., keys that you consider already safe or hashed

function findAndHashEmailKeys(jsonData, hashFunction) {
  const emailRegex = /^[^\s@]+@[^\s@]+\.[^\s@]+$/; // simple regex to validate an email string

  function traverse(obj) {
    if (typeof obj === "object" && obj !== null) {
      for (const key in obj) {
        const value = obj[key];
        if (
          typeof value === "string" &&
          emailRegex.test(value) &&
          !allowedPIIKeys.includes(key)
        ) {
          // Found a string that looks like an email and is not in the allowed list
          obj[key] = hashFunction(value).toString();
        } else if (typeof value === "object") {
          // Recurse into nested objects/arrays
          traverse(value);
        }
      }
    }
  }

  traverse(jsonData);
}

// The main transform function provided by Tealium - this runs for each event
transform((event) => {
  // Apply our email-hashing function to the event's data object (UDO)
  findAndHashEmailKeys(event.data.udo, CryptoES.SHA256);

  // (Optional) Log the transformed event data for debugging
  console.log(JSON.stringify(event.data));
  // console.log(JSON.stringify(event, null, 2)); // pretty-print, if needed

  // Set a flag in the event indicating PII was detected (and handled)
  event.data.udo["potential_pii_detected"] = true;
});

1. Let’s break down what this code does:
   • We import the crypto-es library (available in the Functions environment) to use its SHA-256 hashing (CryptoES.SHA256).
   • We define an array allowedPIIKeys. This is a whitelist of field names that we will not hash even if they match the regex. In this example, [“mid_email”] is listed – imagine this is a field that may contain an email but we intentionally allow it (perhaps it’s already hashed or it’s needed in raw form). You can adjust or leave this list empty as needed.
   • The function findAndHashEmailKeys(jsonData, hashFunction) is our recursive scanner. It takes an object (jsonData) and a hashing function. Inside, it uses emailRegex (a simpler regex ^[^\s@]+@[^\s@]+\.[^\s@]+$ to validate an email string exactly) to test values.
   • The traverse(obj) inner function walks through the object. For each key/value:
     • If the value is a string and matches the email pattern, and the key is not in the allowed list, we replace obj[key] with the hashed version of the string. We use hashFunction(value).toString() – given our call will pass CryptoES.SHA256, this produces a SHA-256 hash of the string.
     • If the value is an object (or array), we recursively call traverse on it to check nested fields.
   • After defining this, we call traverse(jsonData) on the provided data object.
   • Finally, in the Tealium-provided transform((event) => { … }) callback, we execute our function on event.data.udo. In Tealium, event.data.udo is where the event’s custom data layer lives (the key-value pairs sent from the source). By passing CryptoES.SHA256 as the hashFunction, our code will hash any detected email values.
   • We log the event.data to the console for debugging (this output can be viewed in the function test logs). We also set event.data.udo[“potential_pii_detected”] = true unconditionally, assuming that if the function ran, we found PII. (You could make this conditional inside findAndHashEmailKeys if you wanted to flag only when a hash replacement occurred.)
   • The end result: any email addresses in the event’s UDO should now be replaced by a 64-character SHA-256 hash string, and a flag is set in the data.
2. Test the function: It’s important to verify our function works as expected. Tealium provides a built-in Test tab for functions. Switch to the Test view in the function editor. Here you can input a sample event payload JSON. For example, use a payload where some fields contain an email address, e.g.:

/\b[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}\b/im

1. In this sample, customer_email should be hashed by our function (since it looks like an email and is not whitelisted), but mid_email should remain unchanged (since we whitelisted that key). Run the test (click Run Test). In the output, inspect event.data.udo: you should see the customer_email value now replaced with a long hexadecimal string (the SHA-256 hash), mid_email still showing the original, and potential_pii_detected: true added. You should also see the console log output with the transformed JSON. If something isn’t working, use the logs to debug (for instance, ensure your JSONPath selector and regex in the trigger match the test payload so that the function is actually fired). For more details on testing functions, see Tealium’s Test Functions documentation.
2. Enable and deploy: Once testing looks good, go to the Configuration tab of the function. Set the Status to ON (by default, new functions might be created as off or in testing mode). Click Done to confirm. Now, Save and Publish your changes (the publish step pushes the function live in the environment). After publishing, the function becomes active in your event pipeline – any new incoming events that meet the trigger condition will now invoke your code. (Be aware it may take a couple of minutes after publishing for the function to fully propagate and begin executing on incoming data (Create a function | Manage functions | Tealium Docs).)

That’s it! You have deployed a serverless data transformation that detects email addresses in real time and hashes them. You can monitor the function’s execution via Tealium’s function logs or observe the outgoing data to confirm the emails are hashed.

What’s Next?

At this point, we’ve demonstrated how Tealium Functions can be used to enforce privacy by anonymizing PII on the fly. This same pattern can be extended to other forms of sensitive data (phone numbers, credit card numbers, etc.) by adjusting the regex and using the appropriate hashing or masking logic. You might also consider using Functions for tasks like validating data formats, rejecting events that don’t meet certain criteria, or enriching events with computed fields (e.g., categorizing a free-text field into a taxonomy).

For further reading, check out the official Tealium documentation on Data Transformation Functions to understand more about their capabilities and limitations (for example, execution time limits). You may also explore Tealium’s Developer blog for articles on real-time data processing – the concepts of Events, Functions, and Attributes working together. And if you’re interested in more advanced use cases, Tealium’s Functions can even call external APIs or perform async tasks (with some considerations on execution time).

By leveraging Tealium Functions, you can support a range of different use cases including as we have shown here ensuring that your data is clean, privacy-compliant, and tailored to your needs before it’s activated in downstream systems. This leads to safer data practices and sets the stage for more powerful real-time data applications.