Lazada · Shopee · TikTok · dbt
Marketplace Platform-Fee Pipeline
I built dbt Silver and Gold models in Snowflake from Airflow-ingested Bronze platform-fee data, standardizing records from three marketplaces into one effective-dated structure for downstream analytics.
Why the raw fee data could not stay platform-specific
Each marketplace produces fee records with its own source structure. Keeping those sources separate would push platform-specific logic downstream and make reporting harder to maintain.
The goal was to standardize the records once, keep platform identity where it matters, and expose one consistent model for downstream use.
The modeling choice: standardize first, preserve history second
The Bronze platform-fee data is ingested by Airflow. From there, I built dbt Silver models to standardize and deduplicate the marketplace records, then a Gold model to present the data as an effective-dated dimension.
How I made the Gold layer safer to consume
The Gold model uses SHA2-256 surrogate keys, effective-date logic, data-quality flags, and dbt tests. Those checks and modeling choices give downstream users a more predictable record structure instead of three separate fee schemas.
Result: one modeled fee structure across three marketplaces
Lazada, Shopee, and TikTok fee sources are standardized into one effective-dated Gold model for downstream analytics.
The value of the project is less about the number of transformations and more about removing repeated platform-specific logic from whatever comes next.