Data Pipelines

Data Pipelines are a series of data processing steps where the output of one step is the input to the next. Coursera's Data Pipelines skill catalogue teaches you how to design, build, and manage these processes for efficiently moving and transforming data from one system to another. You'll learn about different data pipeline architectures, the use of various tools and technologies such as SQL, Python, Apache Kafka, and Hadoop. You'll also understand how to handle real-time data processing, batch processing, data orchestration, and error handling within a pipeline. This skill is integral for roles like data engineers or data scientists, and anyone looking to manage large volumes of data effectively.
55credentials
226courses

Related roles

Gain the knowledge and skills you need to advance.

View all

  • Data Engineer

    This role has a £62,739 median salary ¹.

    Data Engineer

    description:

    A Data Engineer builds data pipelines for large datasets, optimizing systems and ensuring reliable data flow using tools like Hadoop and Spark.

    This role has a £62,739 median salary ¹.

    Offered by

    DeepLearning.AI_logo
    Amazon Web Services_logo
    Google Cloud_logo

  • Data Warehouse Developer

    This role has a £52,362 median salary ¹.

    Data Warehouse Developer

    description:

    A Data Warehouse Developer designs and optimizes data warehouses, ensuring efficient storage and retrieval for analytics using ETL processes.

    This role has a £52,362 median salary ¹.

    Offered by

    IBM_logo
    Vanderbilt University_logo
View all

Most popular

Trending now

New releases

Filter by

Skip to search results

Subject
Required

Language
Required

The language used throughout the course, in both instruction and assessments.

Learning Product
Required

Build job-relevant skills in under 2 hours with hands-on tutorials.
Learn from top instructors with graded assignments, videos, and discussion forums.
Learn a new tool or skill in an interactive, hands-on environment.
Get in-depth knowledge of a subject by completing a series of courses and projects.
Earn career credentials from industry leaders that demonstrate your expertise.

Level
Required

Duration
Required

Subtitles
Required

Educator
Required

Results for "data pipelines"

  • Status: Free Trial

    Google Cloud

    Skills you'll gain: Real Time Data, Digital Transformation, Google Cloud Platform, Cloud Storage, Data Warehousing, Looker (Software), Big Data, Data Transformation, Analytics, Business Intelligence, Dataflow, Data Management, Data Pipelines, Data-Driven Decision-Making, Data Analysis

    Beginner · Course · 1 - 3 Months

  • Status: Free Trial

    Google Cloud

    Skills you'll gain: Real Time Data, Digital Transformation, Google Cloud Platform, Data Governance, Big Data, Data Warehousing, Looker (Software), Data Transformation, Business Intelligence, Data Management, Data-Driven Decision-Making, Data Access, Data Pipelines, Data Migration

    Beginner · Course · 1 - 3 Months

  • Google Cloud

    Skills you'll gain: MySQL, Google Cloud Platform, Data Migration, Database Management, SQL, Big Data, Extract, Transform, Load, Data Pipelines, Data Management

    Beginner · Project · Less Than 2 Hours

  • Status: Free Trial

    Microsoft

    Skills you'll gain: Responsible AI, Data Analysis, Image Analysis, Pandas (Python Package), Data Ethics, Machine Learning, Tensorflow, Data Processing, Data Pipelines, Data Transformation, Cloud Management, Artificial Neural Networks, Information Privacy, Supervised Learning, Virtual Machines, Application Deployment, Continuous Monitoring, Data Security, Performance Tuning, Scalability

    4.2
    Rating, 4.2 out of 5 stars
    582 reviews

    Intermediate · Professional Certificate · 3 - 6 Months

  • Google Cloud

    Skills you'll gain: Git (Version Control System), Google Cloud Platform, Remote Access Systems, Data Processing, Cloud Storage, Infrastructure As A Service (IaaS), Data Pipelines, Data Sharing, Virtual Machines, Cloud Computing, Data Transformation, Security Software, Process Analysis

    Beginner · Project · Less Than 2 Hours

  • Status: Preview

    Google Cloud

    Skills you'll gain: Data Pipelines, Data Processing, Google Cloud Platform, Data Warehousing, Data Maintenance, Data Storage, Cloud Engineering, Data Infrastructure, Data Analysis, Data Architecture, Automation, Data Security

    Advanced · Course · 1 - 3 Months

  • Status: Preview

    Google Cloud

    Skills you'll gain: Data Pipelines, Google Cloud Platform, Data Processing, Data Storage, Data Infrastructure, Data Architecture, Data Warehousing, Data Migration, Data Analysis, Data Transformation, Data Management, Automation, Data Security

    Advanced · Course · 1 - 3 Months

  • Status: Free Trial

    Google Cloud

    Skills you'll gain: Dataflow, Google Cloud Platform, Data Pipelines, Real Time Data, Data Processing, Systems Design, Cloud Engineering, Data Lakes, Data Infrastructure, Big Data, Data Warehousing, Extract, Transform, Load, MLOps (Machine Learning Operations), Cloud Storage, Data Modeling, Applied Machine Learning, Data Storage, Looker (Software), Apache Hive, Data Integration

    4.7
    Rating, 4.7 out of 5 stars
    812 reviews

    Intermediate · Specialization · 3 - 6 Months

  • Status: Free Trial

    LearnKartS

    Skills you'll gain: Site Reliability Engineering, Google Cloud Platform, Kubernetes, Real Time Data, Big Data, Data Infrastructure, CI/CD, Performance Tuning, Data Pipelines, Databases, Containerization, Data Processing, DevOps, Scalability, Cloud Storage, System Monitoring

    Intermediate · Course · 1 - 4 Weeks

  • Status: Free Trial

    Duke University

    Skills you'll gain: Jupyter, Microservices, AWS SageMaker, Cloud Engineering, Command-Line Interface, Data Pipelines, Package and Software Management, Containerization, Applied Machine Learning, Python Programming, Application Deployment, Development Environment, Test Automation, Algorithms, Virtual Environment

    4.4
    Rating, 4.4 out of 5 stars
    39 reviews

    Intermediate · Course · 1 - 4 Weeks

  • Status: Free Trial

    Packt

    Skills you'll gain: Data Integration, Microsoft Azure, Cloud Storage, Disaster Recovery, Data Architecture, Database Design, Data Infrastructure, Data Storage, Data Management, Cloud Security, Data Pipelines, NoSQL, Data Modeling, Encryption, Relational Databases, Scalability, Performance Tuning, Database Management

    Intermediate · Course · 1 - 3 Months

  • Status: Free Trial

    Google Cloud

    Skills you'll gain: Data Warehousing, Google Cloud Platform, Big Data, Apache Spark, Database Management, Data Integration, Dataflow, SQL, Data Pipelines, Metadata Management, Data Management, Real Time Data, Tensorflow, Data Science, Command-Line Interface, Applied Machine Learning, Cloud-Based Integration, Apache Hadoop, Query Languages, Machine Learning

    4.3
    Rating, 4.3 out of 5 stars
    125 reviews

    Intermediate · Specialization · 1 - 4 Weeks

What brings you to Coursera today?

Leading partners

  • Google Cloud
  • IBM
  • EDUCBA
  • Whizlabs
  • Duke University
  • Packt
  • DeepLearning.AI
  • Microsoft