Data Engineer

12/2021 – offen

• Building and maintaining ELT pipelines using Python and Airflow/ Prefect
• Writing quality and performance tests to ensure the accuracy and reliability of the data
• Deploying infrastructure on AWS, using tools such as Docker, CloudFormation and Terraform
• Transforming data using dbt
• Implementing, maintaining, and expanding the organization's data lake
• Building data applications using Streamlit
• Ensure data is GDPR compliant
• Conducting proof-of-concept projects to evaluate new technologies, such as the Amundsen Data Catalog and Metabase

Datenbankentwicklung, Datenmodellierung, SQL, Python, Softwarepaketierung / Verteilung (allg.)

7/2020 – 12/2021

• Building and Maintaining ETL/ ELT Pipelines and Streaming Data Pipelines with Python, SQL, PDI, Snowflake and AWS services
• Ingesting Data from External Data Sources (Databases, APIs, Web, Webhooks) into Data Lake and Data Warehouse
• Building, Documenting and Maintaining RESTful APIs with Python, Lambda and API Gateway
• Writing, executing and debugging Python, Shell and SQL scripts
• Process scheduling with Airflow, Cron, Snowflake Tasks
• Data Manipulation with Python, SQL (Oracle, SnowSQL, PostgreSQL, MySQL, MariaDB), NoSQL (MongoDB)
• Handling Structured and Unstructured Data
• Documentation and Version Control with Git

Datenbankadministration, Datenbankentwicklung, Datenmodellierung, SQL, Python, Softwareentwicklung (allg.)

1/2017 – 7/2020

• Data Visualization with Tableau and Jupyter Notebook
• Data Analysis with Python (Pandas, Numpy) and SQL (MySQL, Impala, Redshift)
• Building and Maintaining ETL pipelines
• Managing ETLs via AWS Lambda, Cron and Airflow
• Creating basic Time Series Forecasts with Python (statsmodels, scikit-learn)
• Process and Report Automation
• Explaining Insights and Processes to various stakeholders (with and without technical knowledge)

SQL, Python, Datenanalyse, Softwareentwicklung (allg.), Testautomatisierung