Sending Amazon s3 Events data to Madkudu: A Step-by-Step Guide

This article explains the project management and onboarding process for connecting an Amazon S3 bucket to Madkudu. It is designed for HG Insights customers, particularly data and RevOps teams, who will be coordinating with HG Insights Support and Engineering. The technical steps for integration are covered in separate articles linked throughout this document.

Phase 1: Preparation

In this phase, you will need to:

• Set up your Amazon S3 bucket

• Give Madkudu access: Giving Madkudu access to an S3 bucket

• Define the event records you want to send and their related properties

Responsibility: Customer

Typical Duration: 1–3 business days

Phase 2: Test File and Validation

Once the bucket has been created and access has been shared with HG Insights, you should upload a test file to your S3 bucket and validate its format and content. Alternatively, you can share such a file with our team through a support ticket if you’d like to validate the format without uploading to the bucket.

This step ensures that the data structure meets the requirements for integration: How to format and send data to Madkudu via S3

Note

All the columns you plan to send to MadKudu on a recurring basis need to be present in this test file.

Responsibility: Customer + Support

Typical Duration: 1–2 business days (if no formatting issue is encountered)

Phase 3: Finalizing Integration Parameters & setting up the recurring upload

Document your data ingestion requirements. This documentation will be critical for the final integration.

Once documented, set up a recurring dump (at least daily) of your fresh data to S3 and give us a green light to activate the recurring pull of your data.

Data types: Madkudu only ingest event data through Amazon s3 at this moment. For Contacts or Accounts datapoints we recommend you to go through our Bigquery or Snowflake integration.

Historical data scope: If you’d like HG Insights to consider historical data, how far back should the HG Insights team ingest data, and how will this historical data be provided?

Frequency: HG Insights will ingest 1 file per day (default setup), unless otherwise specified. If you require more frequent pulls, please signal it to your CSM or to the support team so that we can adjust the ingestion frequency for you.

Timing: Please document the time at which the upload occurs, so that HG Insights can schedule its ingestion accordingly (to enhance processing efficiency).

Typical Duration: 1-2 business days

Phase 4: Engineering Setup and Validation

Once the requirements are documented, Support will create an Engineering ticket. The HG Insights Engineering team will conduct the setup work required to:

• Optional - Pull your historical data (once)

• Required - Create the recurring pull & validate that the system is operational

Responsibility: HG Insights Engineering

Typical Duration: 1–3 business days

Phase 5: Go Live and Ongoing Maintenance

Finally, Support team will confirm the launch and monitor the connection.
Our clients must ensure that file uploads are conducted consistently, and any updates to the schema should be communicated in advance, as they may disrupt the ingestion process.

Phase 6: Use the s3 data in RGI Platform

Once the data is ingested, you’ll be able to:

1. Map the data pulled in RGI platform in the Event Mapping.

2. Start building an Engagement Score or Aggregations based on your behavioral data.

Estimated Total Timeline

For any questions or assistance regarding this process, please reach out to HG Insights Support.