Explorar las categorías
Explora
Fiverr Pro
Español
$
USD
I will setup databricks unity catalog, medallion layers and pyspark etl pipelines
Data Engineer, Python Developer, AI Automation and AI Agents
Lead Data Architect and Senior Data Engineer with 6+ years of experience specializing in automotive data enrichment, including TecDoc integration and parts catalog systems, alongside broader work desi...
Revisado por el equipo de Fiverr Pro
El equipo de Fiverr Pro seleccionó a Hamza Anwar por su experiencia.
Revisado para
Ingeniería de datos
Acerca de este Servicio
Vetted Pro
Most data lake projects fail at Silver. The raw data lands in Bronze and just sits there messy, untrusted, unusable. I build the full pipeline from raw ingestion to a Gold layer your BI tools can actually query.
I'm a Python Data Engineer with hands-on Databricks experience covering the full lakehouse stack medallion architecture, PySpark pipelines, Delta Lake, Unity Catalog and Databricks Workflows. I also hold a Master's in Business Intelligence, so I understand what the data needs to look like at the Gold layer for reporting to actually work.
What I'll build for you:
- Medallion architecture (Bronze / Silver / Gold) designed around your data sources and business logic
- PySpark notebooks documented, tested, production-ready.
- Delta tables with proper partitioning, Z-ordering and vacuuming.
- Unity Catalog setup with schemas, catalogs and access policies.
- Databricks Workflows to schedule, monitor and retry your pipelines automatically.
- BI-ready Gold layer your team can query from day one.
Not sure what you need? Send me your data sources and your end goal I'll tell you exactly what makes sense to build.
Plataforma de almacenes:
Databricks
Tipo de proyecto:
Nueva creación
Clientes con los cuales he trabajado
Acuity Healthcare
Built an automated healthcare executive leads pipeline in Python that scrapes Indeed, enriches contacts via Apollo, anymailfinder, verifies emails through Million Verifier, and delivers 2,000 job-matched leads per batch to Excel.
mar 2026-may 2026
Mi porfolio
Otros servicios de Ingeniería de datos que ofrezco
FAQ
What is medallion architecture and do I need it?
Medallion is a layered approach to organising data in a lakehouse. Bronze holds raw data. Silver cleans and conforms it. Gold aggregates it into business-ready tables. If you have multiple data sources and need reliable, queryable data for reporting or ML, it's the right pattern.
Do I need an existing Databricks workspace?
Yes, you'll need a Databricks workspace set up on Azure, AWS or GCP. I work inside your environment so everything stays in your account. If you're not sure what to set up first, message me and I can point you in the right direction.
What data sources can you ingest into Bronze?
REST APIs, relational databases (PostgreSQL, MySQL, SQL Server), cloud storage files (CSV, JSON, Parquet, Avro on S3 or ADLS), streaming sources via Auto Loader, and third-party platforms. Tell me your sources and I'll confirm what's straightforward vs. what needs extra work.
What's Unity Catalog and why does it matter?
Unity Catalog is Databricks data governance layer. It lets you control who can access which tables, track data lineage, and manage schemas across workspaces in one place. For teams with multiple users or regulatory requirements it's worth setting up from the start.
Can the Gold layer connect to Power BI or Tableau?
Yes. Gold Delta tables connect natively to Power BI via the Databricks connector, and to Tableau and Looker Studio the same way. I structure the Gold layer so your BI tool can query it directly without further transformation.
