Senior Data Architect and Data Engineer
- Verfügbarkeit einsehen
- 0 Referenzen
- 65€/Stunde
- 14169 Steglitz-Zehlendorf
- auf Anfrage
- en | de
- 18.09.2025
- Contract ready
Kurzvorstellung
Geschäftsdaten
Qualifikationen
Projekt‐ & Berufserfahrung
9/2022 – 1/2025
Tätigkeitsbeschreibung
Engaged part-time for strategic enterprise customers, managed end-to-end data architecture and solution design, with focus on cloud migration, data platform modernization, transformation, and ML/AI analytics implementation. Provided strategic advice on cloud migration, performance tuning, and data engineering best practices.
• Private Cloud Data Platform Implementation: Architected and deployed private cloud data platform on VMware vSphere cluster tailored to customer’s infrastructure and AI/ML workloads requirements. Designed and implemented multi-tier reference architecture integrating Greenplum MPP data warehouse for large-scale analytical processing, Apache Kafka for real-time event streaming, Kubernetes for containerized data workloads orchestration, and Apache Solr for distributed text search and indexing. Developed real-time data ingestion and transformation pipelines using Kafka Connect and Schema Registry. Optimized Kafka on Kubernetes by tuning partitions, replication factors, and broker configurations. Implemented observability and monitoring by Prometheus/Grafana.
• Enterprise Data Warehouse Migration and Optimization: Led Oracle Exadata to Greenplum cluster migration, rearchitected data models and optimized storage for high-performance queries. Implemented RabbitMQ with Debezium for real-time change data capture (CDC) and streaming. Implemented VectorDB for Generative AI and large language models (LLMs) for advanced search and retrieval of data.
• Cloud Migration Proof of Concept: Designed and executed multi-cloud migration PoC, assessing AWS, Azure, and GCP for compatibility with enterprise data and analytics workloads. Defined success KPIs (e.g., data transfer throughput, query latency, storage performance, operational cost efficiency, and scalability benchmarks) to objectively assess each platform. Executed end-to-end data migration tests including bulk data transfer. Validated analytics and streaming/real-time processing for performance and integration with existing pipelines. Delivered architecture recommendations for full-scale adoption, multi-cloud integration patterns and data lake/data warehousing layer strategies.
• Data Platform Modernization and Advisory: Assessed legacy on-premises data infrastructure and designed modern, cloud-native data platforms using Greenplum MPP DWH and containerized microservices (Kubernetes). Advised on scalability, disaster recovery, and high-availability architectures.
Skills: Data Warehousing · Big Data · Data Modeling · Sales Presentations · VMware vSphere · RabbitMQ · Apache Kafka · Data Governance · Data Architecture · Data Warehouse Architecture · Data Migration · Data Engineering · Kubernetes · Pre-Sales Technical Consulting · Data Visualization · Data Strategies · Technical Presentations · Data Solution Architecture · Customer Support · Cloud Migration · Greenplum · Pre-Sales Support · Data Security · Data Streaming · Data Infrastructure · Post-Sales Support · Sales · Customer Engagement · PostgreSQL · Data Quality · Sales Engineering · Presales Technical Support
Apache Kafka, Data Vault, Datenbankentwicklung, Datenmodelierung, ETL, Greenplum, Microsoft SQL-Server (MS SQL), Oracle Data Guard, Oracle Database, Postgresql, Google Cloud, Microsoft Azure, Amazon Web Services (AWS), Data Warehousing
2/2021 – 9/2025
Tätigkeitsbeschreibung
Work with different customers to provide in-depth technical support by designing and implementing data and analytics solutions. Provided strategic advice and best practices on cloud data platforms, data warehouses and data analytics. Led technical assessments and architected solutions for cloud migrations and adoption, data architecture modernization and advanced analytics implementations.
• Enterprise Data Platform Modernization: Led AWS data platform modernization by architecting and developing centralized medallion data architecture with dimensional model and integrated data marts. Developed modular and fault-tolerant data pipelines using Airflow and dbt. Created data lineage for Tableau and Salesforce CRM to improve auditability and root-cause analysis of data issues. Optimized Snowflake by using clustering keys, result caching, and compute resource allocation.
• AWS Data Warehouse Migration: Migrated large Oracle data warehouse to AWS using Amazon Redshift and Informatica Cloud Data Integration. Designed cloud-native data model and architecture, integrated AWS S3 for data lake and AWS Lambda for automated processing. Implemented multi-region and multi-cluster data sharing for high availability and accessibility. Set up AWS SageMaker for ML/AI models and notebooks, integrated Data Lake and DWH.
• Azure Business Intelligence Platform: Architected and implemented data platform on Azure Databricks and PowerBI. Designed and developed data streaming pipelines using Debezium for CDC and Delta Live Tables (DLT) for Data Lake ingestion and transformation. Architected and developed centralized semantic model (tabular model) using Azure Analysis Service (AAS). Improved Spark Cluster performance by optimizing partitions, shuffles and cache.
• Data Vault Model Enhancement: Enhanced Data Vault model for efficient data ingestions and transformations, optimized load performance and query execution times. Tuned SQL queries, implemented materialized views, and optimized keys and hashing with binary storage. Architected and developed Point-in-Time (PIT) and Bridge tables, and reviewed business keys, links, and model granularity for accuracy and performance improvement.
• Master Data Management Migration: Migrated IBM InfoSphere MDM to Informatica Cloud Pak for Data (CP4D), ensuring master data consistency. Developed data mapping, transformation, and validation processes to improve data quality and accuracy. Performed benchmark for containerized environment and suggested improvements.
• Strategic Advisory and Client Engagement: Conducted technical assessments of customers data platforms and developed roadmaps for cloud transformation. Provided executive presentations and technical workshops. Collaborated with cross-functional teams, running complex program management, including cross-project planning.
Skills: Data Warehousing · Amazon Web Services (AWS) · Data Modeling · Azure Data Lake · Sales Presentations · Data Governance · Snowflake · Data Analytics · Data Architecture · Data Warehouse Architecture · Microsoft Azure · Data Migration · Pre-Sales Technical Consulting · Data Visualization · Microsoft SQL Server · Tableau · Data Strategies · Technical Presentations · Data Solution Architecture · Cloud Migration · Pre-Sales Support · Data Security · Amazon Redshift · Azure Databricks · Data Infrastructure · Post-Sales Support · PostgreSQL · Data Quality · Sales Engineering · Presales Technical Support · Data Build Tool (DBT) · Apache Airflow · Snowflake · Databricks
Apache Spark, Azure Synapse Analytics, Big Data, Databricks, Power Bi, Snowflake, Microsoft Azure, Amazon Web Services (AWS), BI-Spezialist Data Warehouse, Data Engineer, Data Warehousing, Database Manager, Solution Architekt
8/2020 – 2/2021
Tätigkeitsbeschreibung
Architected and implemented AWS based data warehouse, integrated real-time data streaming with Apache Kafka and AWS Aurora (PostgreSQL). Designed and implemented ETL pipelines using AWS Glue, Spark, and PySpark. Created data lake and data warehouse with Amazon Redshift/Spectrum and AWS S3. Created master data management and data dictionary using AWS Glue Catalog. Implemented data governance policies, security protocols, and access controls.
Skills: Amazon Web Services (AWS) · AWS Glue · Amazon Redshift · Amazon Aurora · Amazon S3 · Apache Kafka · Apache Spark · PySpark · Python
Apache Kafka, Apache Spark, Python, Amazon Web Services (AWS)
7/2019 – 8/2020
Tätigkeitsbeschreibung
Led architecture and implementation of data solutions, analytics infrastructure, and cloud transformation programs. Designed data strategies for data platform optimization and led AI/ML adoption for BI and predictive analytics. Provided thought leadership in big data engineering and cloud migration projects.
• Enterprise Data Transformation: Led modernization of on-premises legacy data platform to scalable, cloud-native architecture in AWS. Designed and set up high-volume and fault-tolerant ETL pipelines using AWS Glue, Lambda, Kinesis, and S3 for batch and real-time ingestion processing. Set up data lake architecture with partitioned storage and schema evolution to optimize query performance. Integrated Redshift and Athena for analytics. Optimized data availability and governance using Lake Formation, IAM policies, and data encryption.
• Telecom External Data Monetization: Designed and implemented data monetization architecture for secure external data sharing with customers and partners. Designed data pipelines for ingestion, transformation, and segmentation of large telecom datasets using AWS Glue, EMR, and S3. Applied privacy-by-design principles like data anonymization, tokenization, and differential privacy techniques to achieve GDPR compliance. Developed segmentation and benchmarking frameworks for aggregated insights.
• Centralized Data Warehouse: Architected and implemented enterprise data warehouse by consolidating several transactional, operational, and third-party data sources and databases into single platforms. Developed ETL/ELT pipelines using Azure Data Factory, Python and SQL for data ingestion and transformation. Optimized Snowflake storage and query performance by partitioning, clustering, and materialized views.
• IoT Architecture and Predictive Analytics: Architect IoT analytics platform in the AWS to process large-volume real-time sensor data streams. Implemented and deployed ingestion pipelines using Apache Kafka to ingest and stream data into data lake on S3. Created scalable data transformation pipelines using Apache Spark with gap-filling techniques for irregular time-series data. Created rule-based predictive models in Python and SQL.
• Data Governance, Security and Cloud Migration Strategy: Established and implemented enterprise data governance processes that ensured compliance with GDPR, CCPA, and ISO 27001 rules. Created data classification, lineage, cataloging, and retention policies. Led large-scale cloud migration projects, performing comparative assessments of AWS, Azure, and GCP.
• Leadership and Customers Engagement: Provided strategic guidance to C-level leaders on data modernization and AI/ML analytics. Delivered technical workshops and executive briefings. Led cross-functional teams.
Key Technologies: Azure, GCP, AWS, Redshift, AWS Glue, Snowflake, Spark, Kafka, Python, Data Governance
Skills: Data Warehousing · Amazon Web Services (AWS) · Data Modeling · Google Cloud Platform (GCP) · Sales Presentations · Data Governance · Data Analytics · Microsoft Azure · Pre-Sales Technical Consulting · Data Visualization · Data Strategies · Data Solution Architecture · Cloud Migration · Pre-Sales Support · Data Security · Data Infrastructure · Post-Sales Support · Data Quality · Presales Technical Support
BI-Spezialist Data Warehouse, Datamanager, Google Cloud, Data Engineer, Microsoft Azure, Amazon Web Services (AWS), Data Warehousing
4/2017 – 7/2019
Tätigkeitsbeschreibung
• Led the migration of 40PB of data from Hadoop HDFS/Hive and Teradata to AWS S3, Redshift, and Snowflake, ensuring seamless data transformation and optimization.
• Designed and implemented a cloud-based data lake and ETL pipelines using AWS Glue, Spark, and Lambda, improving data ingestion speed and analytics efficiency.
• Developed high-volume, real-time data pipelines for structured and unstructured data, enabling IoT analytics and predictive maintenance.
• Optimized Redshift/Snowflake performance implemented Workload Management and query optimization strategies.
• Provided technical leadership and guidance to cross-functional teams, ensuring best practices in data architecture, governance, and security.
• Collaborated with global teams in the US and India to align data strategies with business objectives.
• Led the migration of 40PB of data from Hadoop HDFS/Hive and Teradata to AWS S3, Redshift, and Snowflake, ensuring seamless data transformation and optimization. • Designed and implemented a cloud-based data lake and ETL pipelines using AWS Glue, Spark, and Lambda, improving data ingestion speed and analytics efficiency. • Developed high-volume, real-time data pipelines for structured and unstructured data, enabling IoT analytics and predictive maintenance. • Optimized Redshift/Snowflake performance implemented Workload Management and query optimization strategies. • Provided technical leadership and guidance to cross-functional teams, ensuring best practices in data architecture, governance, and security. • Collaborated with global teams in the US and India to align data strategies with business objectives.
Skills: Data Warehousing · Amazon Aurora · Amazon Web Services (AWS) · Data Modeling · Apache Kafka · Data Governance · Snowflake · Linux · Data Analytics · Data Architecture · Data Migration · Data Visualization · Data Strategies · Machine Learning · Cloud Migration · AWS Glue · Amazon Redshift · Apache Spark · Data Infrastructure · Data Quality
Data Warehousing
12/2016 – 4/2017
Tätigkeitsbeschreibung
Designed and implemented a scalable data warehouse and BI platform to support advanced analytics and reporting. Led ETL development and dimensional modeling, utilizing PostgreSQL and Talend to automate data ingestion and transformation from multiple sources. Optimized the database schema and indexing, improving query performance and BI report generation time. Automated data validation workflows.
Designed and implemented a scalable data warehouse and BI platform to support advanced analytics and reporting. Led ETL development and dimensional modeling, utilizing PostgreSQL and Talend to automate data ingestion and transformation from multiple sources. Optimized the database schema and indexing, improving query performance and BI report generation time. Automated data validation workflows.
Skills: Data Warehousing · Windows Server · Data Modeling · Snowflake · Linux · Business Intelligence (BI) · Data Analytics · Data Architecture · Data Warehouse Architecture · Data Migration · Data Visualization · Cloud Migration · Amazon Redshift · Data Infrastructure · PostgreSQL
Data Warehousing
9/2015 – 12/2016
Tätigkeitsbeschreibung
Led Data Engineering and infrastructure projects focused on scalable data management and BI delivery. Implemented a centralized data warehouse and BI platform, consolidated databases to optimize performance, and executed secure data migrations to partner data centers. Deployed cloud-based backup and disaster recovery using Amazon S3/Glacier. Defined technical standards and processes to improve integration, scalability, and system efficiency.
Led Data Engineering and infrastructure projects focused on scalable data management and BI delivery. Implemented a centralized data warehouse and BI platform, consolidated databases to optimize performance, and executed secure data migrations to partner data centers. Deployed cloud-based backup and disaster recovery using Amazon S3/Glacier. Defined technical standards and processes to improve integration, scalability, and system efficiency.
Skills: Data Warehousing · Amazon Web Services (AWS) · Windows Server · Data Modeling · SQL Server Analysis Services (SSAS) · MySQL · Linux · Data Analytics · SQL Server Integration Services (SSIS) · Oracle RAC · SQL Server Reporting Services (SSRS) · Microsoft SQL Server · Unix · Tableau · PL/SQL · Data Infrastructure · PostgreSQL · Oracle
Data Warehousing
10/2013 – 9/2015
Tätigkeitsbeschreibung
Managed over 100 high-transaction production databases (Oracle, MS SQL Server, PostgreSQL) for online gaming services. Performed extensive database tuning, backup management, and AWS cloud integration to reduce storage costs and improve query performance. Automated ETL workflows with Pentaho and built a BI reporting environment on Amazon Redshift and Tableau. Ensured high availability through PostgreSQL clustering and virtualized SQL Server instances, optimizing infrastructure and reducing latency.
Managed over 100 high-transaction production databases (Oracle, MS SQL Server, PostgreSQL) for online gaming services. Performed extensive database tuning, backup management, and AWS cloud integration to reduce storage costs and improve query performance. Automated ETL workflows with Pentaho and built a BI reporting environment on Amazon Redshift and Tableau. Ensured high availability through PostgreSQL clustering and virtualized SQL Server instances, optimizing infrastructure and reducing latency.
Skills: ETL · Data Warehousing · Amazon Web Services (AWS) · Windows Server · Data Modeling · MySQL · RabbitMQ · Linux · SQL Server Integration Services (SSIS) · Microsoft SQL Server · Unix · Tableau · PL/SQL · MySQL Adminstration · Data Infrastructure · SQL Database Administration · PostgreSQL · Oracle · Oracle Database Administration
Database Manager
4/2012 – 10/2013
Tätigkeitsbeschreibung
Led database operations for high-availability OLAP/OLTP systems supporting real-time betting and analytics. Successfully migrated core systems to SQL Server 2008 and reengineered ETL workflows, improving system performance and reducing latency. Designed automated backup and recovery solutions, enhancing disaster recovery readiness. Implemented real-time data replication for immediate odds updates, ensuring responsiveness during peak traffic. Enforced regulatory compliance (GDPR, PCI-DSS) through robust access controls. Provided ongoing support and tuning for business-critical applications, improving availability and operational efficiency.
Led database operations for high-availability OLAP/OLTP systems supporting real-time betting and analytics. Successfully migrated core systems to SQL Server 2008 and reengineered ETL workflows, improving system performance and reducing latency. Designed automated backup and recovery solutions, enhancing disaster recovery readiness. Implemented real-time data replication for immediate odds updates, ensuring responsiveness during peak traffic. Enforced regulatory compliance (GDPR, PCI-DSS) through robust access controls. Provided ongoing support and tuning for business-critical applications, improving availability and operational efficiency.
Skills: Windows Server · Data Modeling · Linux · SQL Server Integration Services (SSIS) · Oracle RAC · Microsoft SQL Server · Unix · Data Infrastructure · SQL Database Administration · Oracle · Oracle Database Administration
Database Manager
3/2011 – 4/2012
Tätigkeitsbeschreibung
Led development of IaaS cloud management software, focusing on database architecture, replication, and transactional integrity. Implemented cross-platform Oracle-to-non-Oracle replication using LogMiner, Oracle Streams, and Redo log parsing. Automated database deployment and configuration processes to enable rapid scaling. Optimized query performance, indexing, and partitioning strategies, improving data retrieval speeds. Enhanced system scalability and processing efficiency. Provided technical support for sales, including product demos and architecture consultations.
Led development of IaaS cloud management software, focusing on database architecture, replication, and transactional integrity. Implemented cross-platform Oracle-to-non-Oracle replication using LogMiner, Oracle Streams, and Redo log parsing. Automated database deployment and configuration processes to enable rapid scaling. Optimized query performance, indexing, and partitioning strategies, improving data retrieval speeds. Enhanced system scalability and processing efficiency. Provided technical support for sales, including product demos and architecture consultations.
Skills: Windows Server · Agile Methodologies · Linux · Java · Unix · PL/SQL · Oracle Streams · Data Infrastructure · Oracle · Oracle Database Administration
Database Manager
4/2006 – 3/2011
Tätigkeitsbeschreibung
Managed 24/7 mission-critical Oracle databases with a focus on high availability, performance, and disaster recovery. Optimized SQL and ETL workflows. Administered Oracle RAC, ASM, and Data Guard to enhance system resilience. Led database consolidation. Automated MS SQL to Oracle data integration and developed data pipelines for operational efficiency. Streamlined backup and replication using RMAN, Veritas, and EMC-BCV. Ensured SOX compliance, implemented security protocols, and automated infrastructure tasks. Provided technical leadership to DBA teams.
Managed 24/7 mission-critical Oracle databases with a focus on high availability, performance, and disaster recovery. Optimized SQL and ETL workflows. Administered Oracle RAC, ASM, and Data Guard to enhance system resilience. Led database consolidation. Automated MS SQL to Oracle data integration and developed data pipelines for operational efficiency. Streamlined backup and replication using RMAN, Veritas, and EMC-BCV. Ensured SOX compliance, implemented security protocols, and automated infrastructure tasks. Provided technical leadership to DBA teams.
Skills: Agile Methodologies · Linux · Oracle RAC · Unix · PL/SQL · Oracle Streams · Systems Engineering · Perl · Data Infrastructure · ITIL · Oracle · Oracle Database Administration
Oracle Database
5/2003 – 4/2006
Tätigkeitsbeschreibung
Managed high-volume Oracle databases for telecom billing systems, maintaining 99.99% uptime and peak performance. Led migrations, performance tuning, and disaster recovery initiatives. Developed PL/SQL and ETL workflows to optimize data processing and system efficiency. Executed data conversions and ensured integrity across ETL pipelines. Tuned database architecture, partitions, and indexing strategies, improving query runtime. Enforced security policies, access controls, and data encryption to meet compliance standards.
Managed high-volume Oracle databases for telecom billing systems, maintaining 99.99% uptime and peak performance. Led migrations, performance tuning, and disaster recovery initiatives. Developed PL/SQL and ETL workflows to optimize data processing and system efficiency. Executed data conversions and ensured integrity across ETL pipelines. Tuned database architecture, partitions, and indexing strategies, improving query runtime. Enforced security policies, access controls, and data encryption to meet compliance standards.
Skills: Agile Methodologies · Linux · Oracle RAC · Unix · PL/SQL · Oracle Streams · Perl · Data Infrastructure · ITIL · Oracle · Oracle Database Administration
Oracle Database
6/2000 – 5/2003
Tätigkeitsbeschreibung
Managed Oracle databases for enterprise IT projects, delivering high availability, strong performance, and secure operations. Executed seamless migrations and implemented efficient backup and recovery strategies to enhance system reliability. Automated routine tasks, reducing manual effort and improving operational efficiency.
Managed Oracle databases for enterprise IT projects, delivering high availability, strong performance, and secure operations. Executed seamless migrations and implemented efficient backup and recovery strategies to enhance system reliability. Automated routine tasks, reducing manual effort and improving operational efficiency.
Skills: ETL · Data Modeling · Agile Methodologies · Linux · Oracle RAC · Unix · PL/SQL · Oracle Streams · Perl · Data Infrastructure · ITIL · Oracle · Oracle Database Administration
Oracle Database
Zertifikate
Ausbildung
Hertfordshire
Beer-Sheva
Über mich
Weitere Kenntnisse
• Data Platforms: Snowflake, Azure Synapse, MS Fabric, Databricks, AWS Redshift, Greenplum
• Big Data and Processing: Apache Spark, Hadoop (HDFS, Hive), AWS EMR
• Streaming and Integration: Airflow, DBT, Kafka, AWS Kinesis, Debezium, RabbitMQ, AWS Glue, Pentaho, Talend, ADF
• Programming and Scripting: Python, PySpark, Java, Perl, PowerShell, Unix Shell, SQL, PL/SQL, PL/pgSQL
• Databases: Oracle, Oracle RAC, PostgreSQL, MS SQL Server, MySQL, AWS RDS, Azure SQL
• Operating Systems: Unix (HP-UX, Solaris, AIX), Linux (RHEL, Debian, Ubuntu), Windows Server
• DevOps and Methodologies: CI/CD, Docker, Helm, Terraform, Ansible, Agile/Scrum, ITIL Fundamentals
Persönliche Daten
- Englisch (Muttersprache)
- Deutsch (Fließend)
- Europäische Union
- Schweiz
- Vereinigte Staaten von Amerika
Kontaktdaten
Nur registrierte PREMIUM-Mitglieder von freelance.de können Kontaktdaten einsehen.
Jetzt Mitglied werden