Data Pipelines

Data Pipelines are a series of data processing steps where the output of one step is the input to the next. Coursera's Data Pipelines skill catalogue teaches you how to design, build, and manage these processes for efficiently moving and transforming data from one system to another. You'll learn about different data pipeline architectures, the use of various tools and technologies such as SQL, Python, Apache Kafka, and Hadoop. You'll also understand how to handle real-time data processing, batch processing, data orchestration, and error handling within a pipeline. This skill is integral for roles like data engineers or data scientists, and anyone looking to manage large volumes of data effectively.
56credentials
224courses

Related roles

Gain the knowledge and skills you need to advance.

View all

  • Data Engineer

    This role has a £62,565 median salary ¹.

    Data Engineer

    description:

    A Data Engineer builds data pipelines for large datasets, optimizing systems and ensuring reliable data flow using tools like Hadoop and Spark.

    This role has a £62,565 median salary ¹.

    Offered by

    DeepLearning.AI_logo
    Amazon Web Services_logo
    Google Cloud_logo

  • Data Warehouse Developer

    This role has a £52,216 median salary ¹.

    Data Warehouse Developer

    description:

    A Data Warehouse Developer designs and optimizes data warehouses, ensuring efficient storage and retrieval for analytics using ETL processes.

    This role has a £52,216 median salary ¹.

    Offered by

    IBM_logo
    Vanderbilt University_logo
View all

Most popular

Trending now

New releases

Filter by

Skip to search results

Subject
Required

Language
Required

The language used throughout the course, in both instruction and assessments.

Learning Product
Required

Build job-relevant skills in under 2 hours with hands-on tutorials.
Learn from top instructors with graded assignments, videos, and discussion forums.
Learn a new tool or skill in an interactive, hands-on environment.
Get in-depth knowledge of a subject by completing a series of courses and projects.
Earn career credentials from industry leaders that demonstrate your expertise.
Earn a university-issued career credential in a flexible, interactive format.

Level
Required

Duration
Required

Subtitles
Required

Educator
Required

Results for "data pipelines"

  • Status: Free Trial

    Google Cloud

    Skills you'll gain: Dataflow, Google Cloud Platform, Data Pipelines, Real Time Data, Cloud Engineering, Data Lakes, Big Data, Dashboard, Cloud Infrastructure, MLOps (Machine Learning Operations), Tensorflow, Data Infrastructure, Data Warehousing, Extract, Transform, Load, Data Processing, Cloud Storage, Applied Machine Learning, Data Visualization Software, Data Storage, Apache Hive

    4.7
    Rating, 4.7 out of 5 stars
    812 reviews

    Intermediate · Specialization · 3 - 6 Months

  • Status: Free Trial

    Google Cloud

    Skills you'll gain: Data Pipelines, Data Cleansing, Data Governance, Data Transformation, Cloud Security, Data Visualization, Extract, Transform, Load, Data Integration, Data Processing, Data Management, Data Migration, Data Collection, Cloud Computing, Cloud Storage, Data Analysis, SQL

    4.5
    Rating, 4.5 out of 5 stars
    26 reviews

    Beginner · Course · 1 - 4 Weeks

  • Status: Free Trial

    Google Cloud

    Skills you'll gain: Dataflow, Google Cloud Platform, Data Pipelines, Looker (Software), Data Lakes, Real Time Data, Data Warehousing, Big Data, Cloud Infrastructure, Data Infrastructure, Unstructured Data, Tensorflow, Data Storage, Dashboard, Apache Airflow, Data Architecture, Data Processing, MLOps (Machine Learning Operations), Cloud Storage, Applied Machine Learning

    4.4
    Rating, 4.4 out of 5 stars
    58 reviews

    Intermediate · Specialization · 3 - 6 Months

  • Status: Free Trial

    Google Cloud

    Skills you'll gain: Dataflow, Google Cloud Platform, Data Pipelines, Data Lakes, Data Warehousing, Cloud Infrastructure, Real Time Data, Big Data, Extract, Transform, Load, Data Infrastructure, Cloud Storage, Looker (Software), Tensorflow, Data Architecture, Unstructured Data, Data Integration, MLOps (Machine Learning Operations), Data Processing, Data Transformation, Serverless Computing

    4.7
    Rating, 4.7 out of 5 stars
    124 reviews

    Intermediate · Specialization · 3 - 6 Months

  • Status: New
    Status: Free Trial

    EDUCBA

    Skills you'll gain: PySpark, Apache Spark, MySQL, Data Pipelines, Scala Programming, Extract, Transform, Load, Customer Analysis, Apache Hadoop, Classification And Regression Tree (CART), Predictive Modeling, Applied Machine Learning, Data Processing, Advanced Analytics, Big Data, Apache Maven, Statistical Machine Learning, Unsupervised Learning, SQL, Apache, Python Programming

    4
    Rating, 4 out of 5 stars
    6 reviews

    Beginner · Specialization · 1 - 3 Months

  • Status: Free Trial

    Google Cloud

    Skills you'll gain: Dataflow, Google Cloud Platform, Data Pipelines, Real Time Data, Data Processing, Dashboard, Tensorflow, Apache Spark, Big Data, Applied Machine Learning, Query Languages, Scripting Languages, PySpark, Cloud Storage, Feature Engineering, Apache Hadoop, MLOps (Machine Learning Operations), Data Storage, Apache Hive, Artificial Intelligence and Machine Learning (AI/ML)

    Intermediate · Specialization · 3 - 6 Months

  • Status: Free Trial

    IBM

    Skills you'll gain: Design Thinking, Data Science, Process Design, Strategic Thinking, Data Pipelines, Data Validation, Data Processing, Workflow Management, Data Quality, Business Priorities, Pandas (Python Package), NumPy, Machine Learning, Python Programming

    4.3
    Rating, 4.3 out of 5 stars
    183 reviews

    Intermediate · Course · 1 - 4 Weeks

  • Status: Free Trial

    Microsoft

    Skills you'll gain: Responsible AI, Microsoft Azure, Unsupervised Learning, Databricks, MLOps (Machine Learning Operations), Applied Machine Learning, Regression Analysis, Scikit Learn (Machine Learning Library), Predictive Modeling, Cloud Management, Machine Learning, Artificial Intelligence and Machine Learning (AI/ML), Supervised Learning, Virtual Machines, Application Deployment, Data Pipelines

    4.3
    Rating, 4.3 out of 5 stars
    175 reviews

    Intermediate · Course · 1 - 4 Weeks

  • Status: Free Trial

    Google Cloud

    Skills you'll gain: Dataflow, Google Cloud Platform, Data Pipelines, Data Lakes, Cloud Engineering, Big Data, Data Warehousing, Data Infrastructure, Cloud Storage, Unstructured Data, MLOps (Machine Learning Operations), Extract, Transform, Load, Data Visualization, Real Time Data, Tensorflow, Dashboard, Data Architecture, Analytics, Data Processing, Apache Spark

    4.4
    Rating, 4.4 out of 5 stars
    289 reviews

    Intermediate · Specialization · 3 - 6 Months

  • Status: Free Trial

    Whizlabs

    Skills you'll gain: Serverless Computing, Amazon Web Services, Cloud Computing Architecture, Solution Architecture, Containerization, Data Migration, Data Pipelines, Cloud Solutions, Data Processing

    Intermediate · Course · 1 - 4 Weeks

  • Amazon Web Services

    Skills you'll gain: Extract, Transform, Load, Data-Driven Decision-Making, Amazon Web Services, Data Warehousing, Big Data, Databases, Business Analytics, NoSQL, Data Analysis, Data Pipelines, Unstructured Data, Data Processing, Cloud API, SQL, Data Storage, Data Architecture, Relational Databases, Data Transformation, Data Visualization Software

    4.7
    Rating, 4.7 out of 5 stars
    54 reviews

    Beginner · Course · 1 - 4 Weeks

  • Status: Preview

    Talend

    Skills you'll gain: Metadata Management, Data Manipulation, Data Integration, Extract, Transform, Load, Database Management Systems, Data Transformation, Data Access, Data Migration, Data Warehousing, Data Pipelines, Data Validation, Version Control

    4.6
    Rating, 4.6 out of 5 stars
    68 reviews

    Intermediate · Course · 1 - 4 Weeks

What brings you to Coursera today?

Leading partners

  • Google Cloud
  • IBM
  • EDUCBA
  • Whizlabs
  • Duke University
  • Packt
  • DeepLearning.AI
  • Microsoft