Big data engineer

Profil Foto
Verfügbarkeit einsehen
Europa
en  |  sl  |  no
100€/Stunde
0356 OSLO
31.10.2018

Kurzvorstellung

I am a software developer with years of experience in programming, databases and information systems. Although I have worked a lot on Big data architectures in the past, my career focus is on data science.
Over 4 years’ experience with open source te

Ich biete

IT, Entwicklung
  • Apache Hadoop
  • Python
  • Business Intelligence (BI)
  • Oracle Database
  • Java (allg.)

Projekt‐ & Berufserfahrung

Big data specialist
Kundenname anonymisiert, Oslo
11/2016 – 12/2017 (1 Jahr, 2 Monate)
Handel
Tätigkeitszeitraum

11/2016 – 12/2017

Tätigkeitsbeschreibung

Responsibilities:
• Introducing Hadoop stack to the organization
• Designing and building Hadoop and Spark clusters in AWS
• Develop and drive technical roadmap for data and development infrastructure
• Defining knowledge roadmaps for internal employees in the field of Hadoop
• Machine Learning (evolutionary algorithms, feature engineering, neural networks) using Python
• Testing new technologies

Details:
As a Big Data developer focus was on all levels of the stacks. I have built two clusters in AWS, one is a pure Hadoop cluster (HDP 2.6) and the other one is a Spark cluster with separate storage in S3. The latter one launches on demand with dynamic resources. My tasks are architecture, maintenance and upgrades of the clusters. Both clusters rely heavily on Spark as the computational engine where I am mostly using Scala (for data integration), Python (for data science - ML) and SparkSQL.
Hive is the data warehouse on top of HDFS to provide users the SQL API.
Tested new visualization tools (Zeppelin, Druid, re:dash, superset…) to find best possible stack.
Key technology terms: Hortonworks, Ambari, HDFS, MapReduce2, YARN, Zookeeper, Hive, Zeppelin, Spark, Storm, Ranger, Redis, Flume, Sqoop, Druid, scikit learn, Jupyter.
PyCharm and Jupyter were used for the data science work. Main focus was on feature engineering, machine learning, evolutionary algorithms and neural networks.

Eingesetzte Qualifikationen

Apache Hadoop, Python, Scala

Ausbildung

1998
(2004)
Jahr: 2004
Ort: University of Ljubljana

Qualifikationen

Hadoop, Hortonworks, YARN, Spark, Python, Machine learning, Neural network, Ambari, Apache, Scala, Java, Storm, NoSql, Redis

Über mich

I am a software developer with years of experience in programming, databases and information systems. Although I have worked a lot on Big data architectures in the past, my career focus is on data science.
Over 4 years’ experience with open source technologies (big data). Installation, administration and configuration of Hadoop ecosystems - Apache and Hortonworks distributions in the AWS. Building and configuring Spark clusters and writing Spark code (Scala, PySpark and SparkR).

Persönliche Daten

Sprache
  • Englisch (Fließend)
  • Slowenisch (Muttersprache)
  • Norwegisch (Fließend)
  • Deutsch (Grundkenntnisse)
  • Serbisch (Gut)
Reisebereitschaft
Europa
Arbeitserlaubnis
  • Europäische Union
Profilaufrufe
131
Alter
38
Berufserfahrung
18 Jahre und 5 Monate (seit 07/2000)
Projektleitung
17 Jahre

Kontaktdaten

Nur registrierte PREMIUM-Mitglieder von freelance.de können Kontaktdaten einsehen.

Jetzt Mitglied werden »