Data Pipelines

Data Pipelines are a series of data processing steps where the output of one step is the input to the next. Coursera's Data Pipelines skill catalogue teaches you how to design, build, and manage these processes for efficiently moving and transforming data from one system to another. You'll learn about different data pipeline architectures, the use of various tools and technologies such as SQL, Python, Apache Kafka, and Hadoop. You'll also understand how to handle real-time data processing, batch processing, data orchestration, and error handling within a pipeline. This skill is integral for roles like data engineers or data scientists, and anyone looking to manage large volumes of data effectively.
55credentials
226courses

Related roles

Gain the knowledge and skills you need to advance.

View all

  • Data Engineer

    This role has a £62,739 median salary ¹.

    Data Engineer

    description:

    A Data Engineer builds data pipelines for large datasets, optimizing systems and ensuring reliable data flow using tools like Hadoop and Spark.

    This role has a £62,739 median salary ¹.

    Offered by

    DeepLearning.AI_logo
    Amazon Web Services_logo
    Google Cloud_logo

  • Data Warehouse Developer

    This role has a £52,362 median salary ¹.

    Data Warehouse Developer

    description:

    A Data Warehouse Developer designs and optimizes data warehouses, ensuring efficient storage and retrieval for analytics using ETL processes.

    This role has a £52,362 median salary ¹.

    Offered by

    IBM_logo
    Vanderbilt University_logo
View all

Most popular

Trending now

New releases

Filter by

Skip to search results

Subject
Required

Language
Required

The language used throughout the course, in both instruction and assessments.

Learning Product
Required

Build job-relevant skills in under 2 hours with hands-on tutorials.
Learn from top instructors with graded assignments, videos, and discussion forums.
Learn a new tool or skill in an interactive, hands-on environment.
Get in-depth knowledge of a subject by completing a series of courses and projects.
Earn career credentials from industry leaders that demonstrate your expertise.

Level
Required

Duration
Required

Subtitles
Required

Educator
Required

Results for "data pipelines"

  • Amazon Web Services

    Skills you'll gain: Extract, Transform, Load, Data-Driven Decision-Making, Amazon Web Services, Data Warehousing, Big Data, Databases, Business Analytics, NoSQL, Data Analysis, Data Pipelines, Unstructured Data, Data Processing, Cloud API, SQL, Data Storage, Data Architecture, Relational Databases, Data Transformation, Data Visualization Software

    4.7
    Rating, 4.7 out of 5 stars
    54 reviews

    Beginner · Course · 1 - 4 Weeks

  • Status: New

    Google Cloud

    Skills you'll gain: Data Sharing, Cloud Engineering, Data Pipelines, Data Migration, Google Cloud Platform, Data Infrastructure, Data Warehousing, Cloud Storage, Data Integration, Data Architecture, Data Lakes, Data Processing, Extract, Transform, Load, Data Storage, Big Data, Command-Line Interface, Cloud Solutions, Automation, Serverless Computing, Event-Driven Programming

    Beginner · Course · 1 - 3 Months

  • IBM

    Skills you'll gain: Apache Spark, PySpark, Applied Machine Learning, Big Data, Data Storage Technologies, Statistical Machine Learning, Data Pipelines, Machine Learning Algorithms, Machine Learning, Data Processing, Data Science, Statistical Analysis

    3.8
    Rating, 3.8 out of 5 stars
    1.3K reviews

    Intermediate · Course · 1 - 4 Weeks

  • Status: New

    Google Cloud

    Skills you'll gain: Data Pipelines, Google Cloud Platform, Data Processing, Data Storage, Cloud Storage, Cloud Engineering, Data Infrastructure, Data Management, Data Architecture, Data Warehousing, Data Integration, Data Analysis, Data Migration, Automation

    Advanced · Course · 1 - 3 Months

  • Status: Preview

    EDUCBA

    Skills you'll gain: Azure Synapse Analytics, Data Warehousing, Power BI, Data Integration, Data Architecture, Data Visualization Software, Microsoft Azure, Apache Spark, Database Management, Data Pipelines, Performance Tuning, Data Processing, Data Security, Scalability

    4.9
    Rating, 4.9 out of 5 stars
    55 reviews

    Beginner · Course · 1 - 3 Months

  • Status: Free Trial

    Google Cloud

    Skills you'll gain: Data Maintenance, Google Cloud Platform, Data Pipelines, Data Storage, Data Analysis, Data Processing, Data Infrastructure, Data Migration, Data Warehousing, Automation, Data Security, Test Planning

    4.6
    Rating, 4.6 out of 5 stars
    1K reviews

    Advanced · Course · 1 - 3 Months

  • Status: New
    Status: Free Trial

    EDUCBA

    Skills you'll gain: MongoDB, Data Transformation, Customer Analysis, Data Analysis, Back-End Web Development, Customer Data Management, Data Processing, NoSQL, Data Maintenance, Pandas (Python Package), Data Validation, Financial Data, Data Import/Export, Database Management, Data Cleansing, Databases, Data Pipelines, Database Design, Data Access, Query Languages

    4.6
    Rating, 4.6 out of 5 stars
    26 reviews

    Beginner · Specialization · 1 - 3 Months

  • Status: New
    Status: Preview

    Coursera

    Skills you'll gain: Data Integration, Data Import/Export, NumPy, Data Wrangling, Data Manipulation, Pandas (Python Package), Data Pipelines, Data Transformation, Extract, Transform, Load, Data Cleansing

    Beginner · Course · 1 - 4 Weeks

  • Status: New
    Status: Free Trial

    Whizlabs

    Skills you'll gain: Responsible AI, Statistical Modeling, Microsoft Azure, MLOps (Machine Learning Operations), Prompt Engineering, Data Management, Data Science, Artificial Intelligence and Machine Learning (AI/ML), Statistics, Cloud Computing, Application Deployment, Azure Synapse Analytics, Large Language Modeling, Data Pipelines, Generative AI, Data Processing, Machine Learning, Applied Machine Learning, Scalability, Continuous Monitoring

    Intermediate · Specialization · 1 - 3 Months

  • Status: Free Trial

    IBM

    Skills you'll gain: Professional Networking, Interviewing Skills, LinkedIn, Technical Communication, Professional Development, Data Strategy, Data Ethics, Data Infrastructure, Data Pipelines, Verbal Communication Skills, Communication Strategies

    4.7
    Rating, 4.7 out of 5 stars
    88 reviews

    Beginner · Course · 1 - 4 Weeks

  • Google Cloud

    Skills you'll gain: Metadata Management, Data Sharing, Data Pipelines, Extract, Transform, Load, Google Cloud Platform, Data Migration, Data Warehousing, Apache Airflow, Data Processing, Data Integration, Big Data, Data Storage, Data Management, Cloud Storage, Automation

    Beginner · Course · 1 - 3 Months

  • Coursera Project Network

    Skills you'll gain: Snowflake Schema, Applied Machine Learning, Machine Learning, Predictive Modeling, Feature Engineering, Data Pipelines, Data Transformation, Data Processing, Data Science, Python Programming, Regression Analysis

    Intermediate · Guided Project · Less Than 2 Hours

What brings you to Coursera today?

Leading partners

  • Google Cloud
  • IBM
  • EDUCBA
  • Whizlabs
  • Duke University
  • Packt
  • DeepLearning.AI
  • Microsoft