Hello, I'm Daniel 

Enterprise Data Engineer · Azure & Databricks · AI-Ready Data Platforms

About

Portrait of Daniel

I'm Daniel Conejo Sobrino, a Data Engineer based in Málaga, Spain. I work at Viewnext, part of IBM, assigned to Repsol as client on the ARiA data platform. My work covers building end-to-end data pipelines across the full data lifecycle, from multi-source ingestion into Azure Data Lake Storage to curated analytical layers on Azure Databricks and Synapse. My day-to-day covers Python, PySpark, Azure Data Factory, and the complete delivery cycle from development through production.

Before specialising in data engineering, I worked in software development and cloud infrastructure, building applications with .NET, TypeScript and Next.js. That background gives me a practical grounding in how data systems connect to the applications and services that consume them, and shapes how I approach platform delivery: with attention to reliability, observability, and the engineering discipline that makes data trustworthy at scale.

Outside of client work, I maintain a personal home lab where I experiment with agentic AI systems, testing how autonomous agents interact with data, what governance structures they require, and where current tooling falls short. Those experiments feed into the Research Notes on this site, which are my main channel for working through ideas at the intersection of data engineering and AI governance.

Experience

Viewnext · IBM Group

Data Engineer

2024 – Present

Málaga, Spain · Full Remote

  • Work as a Data Engineer at Viewnext, part of IBM, assigned full-time to Repsol as client on the ARiA data platform.
  • Implement end-to-end data pipelines across the full data lifecycle, from source extraction and ingestion to curated analytical layers in Azure Data Lake and downstream platform consumption.
  • Collaborate with Solution Architects to translate architecture designs into robust, scalable technical implementations, including scope documentation, feasibility validation and test planning.
  • Develop and configure batch and streaming ingestion processes across heterogeneous sources such as Oracle, Teradata, SQL Server, APIs, Salesforce, sFTP file sources, Event Hubs and industrial PI System signals.
  • Build data transformation and quality workflows using Python, PySpark, SQL, Azure Databricks, Azure Data Factory and platform components for business logic, joins, aggregations, schema validation and semantic normalization.
  • Support data modelling across Raw, Processed and Analytical layers, including curated datasets and Synapse Dedicated Pool models optimized for analytical consumption.
  • Contribute to SDLC delivery across Development, Test, Acceptance and Production environments, supporting UAT, production readiness and post-release stability.
  • Integrate data processes with observability, monitoring and orchestration capabilities to track ingestion health, dependencies and SLA compliance.
  • Work with governance and security processes including metadata cataloguing, access requests, ACLs, Azure Key Vault and Service Principals.
  • Participate in Agile delivery using ServiceNow and Azure DevOps, including backlog management, CI/CD pipelines, Git workflows and delivery KPIs.

BeoneBe

Data & Cloud Engineer

January 2024 – July 2024

Marbella, Spain

  • Built an Azure-backed document intelligence pipeline for automated metadata extraction and authenticity verification across large document corpora
  • Deployed processing services in Docker on Linux with TLS/SSL termination and reverse-proxy routing
  • Designed REST APIs exposing data pipeline outputs for downstream consumer applications
  • Established CI/CD pipelines for automated testing and deployment of data processing services

Education

Universidad Internacional de La Rioja (UNIR)

Higher Degree in Networked Computer Systems Administration

Sept 2024 - Jul 2026

CPIFP Alan Turing, Parque Tecnológico de Andalucía (PTA)

Professional Certificate in Frontend Environment, Professional Certificate Microsoft Azure

Sept 2023 - Mar 2024

ILERNA FP

Higher Degree in Web Applications Development

Mar 2022 - Jul 2024

Stack

Languages & Query

PythonPySparkSQLTypeScriptScala

Cloud Platform

Azure DatabricksAzure Data FactoryADLS Gen2Azure Synapse

Storage & Sources

Delta LakeAzure SQLOracleSQL Server

Delivery & DevOps

Azure DevOpsServiceNowDockerGit

Governance

Unity CatalogAzure Key VaultAzure PurviewService PrincipalsMicrosoft Entra ID

Software & Web

.NETNext.jsTypeScriptREST APIsPostmanLinuxShell Scripting

Interested in working together?