Data Pipelines

Data Pipelines are a series of data processing steps where the output of one step is the input to the next. Coursera's Data Pipelines skill catalogue teaches you how to design, build, and manage these processes for efficiently moving and transforming data from one system to another. You'll learn about different data pipeline architectures, the use of various tools and technologies such as SQL, Python, Apache Kafka, and Hadoop. You'll also understand how to handle real-time data processing, batch processing, data orchestration, and error handling within a pipeline. This skill is integral for roles like data engineers or data scientists, and anyone looking to manage large volumes of data effectively.
56credentials
227courses

Related roles

Gain the knowledge and skills you need to advance.

View all

  • Data Engineer

    This role has a £62,590 median salary ¹.

    Data Engineer

    description:

    A Data Engineer builds data pipelines for large datasets, optimizing systems and ensuring reliable data flow using tools like Hadoop and Spark.

    This role has a £62,590 median salary ¹.

    Offered by

    DeepLearning.AI_logo
    Amazon Web Services_logo
    Google Cloud_logo

  • Data Warehouse Developer

    This role has a £52,237 median salary ¹.

    Data Warehouse Developer

    description:

    A Data Warehouse Developer designs and optimizes data warehouses, ensuring efficient storage and retrieval for analytics using ETL processes.

    This role has a £52,237 median salary ¹.

    Offered by

    IBM_logo
    Vanderbilt University_logo
View all

Most popular

Trending now

New releases

Filter by

Skip to search results

Subject
Required

Language
Required

The language used throughout the course, in both instruction and assessments.

Learning Product
Required

Build job-relevant skills in under 2 hours with hands-on tutorials.
Learn from top instructors with graded assignments, videos, and discussion forums.
Learn a new tool or skill in an interactive, hands-on environment.
Get in-depth knowledge of a subject by completing a series of courses and projects.
Earn career credentials from industry leaders that demonstrate your expertise.
Earn a university-issued career credential in a flexible, interactive format.

Level
Required

Duration
Required

Subtitles
Required

Educator
Required

Results for "data pipelines"

  • Status: Free Trial

    IBM

    Skills you'll gain: NoSQL, Data Warehousing, SQL, Apache Hadoop, Extract, Transform, Load, Apache Airflow, Data Security, Linux Commands, Data Migration, Database Design, Data Governance, MySQL, Database Administration, Apache Spark, Data Pipelines, Apache Kafka, Database Management, Bash (Scripting Language), Data Store, Data Architecture

    4.6
    Rating, 4.6 out of 5 stars
    7K reviews

    Beginner · Professional Certificate · 3 - 6 Months

  • Status: Free Trial

    IBM

    Skills you'll gain: Web Scraping, Database Design, SQL, MySQL, Data Transformation, Data Store, IBM DB2, Extract, Transform, Load, Data Architecture, Data Pipelines, Big Data, Databases, Data Warehousing, Data Governance, Database Management Systems, Relational Databases, Stored Procedure, Data Import/Export, Programming Principles, Python Programming

    4.6
    Rating, 4.6 out of 5 stars
    58K reviews

    Beginner · Specialization · 3 - 6 Months

  • Status: Free Trial

    Multiple educators

    Skills you'll gain: Apache Airflow, Data Modeling, Data Pipelines, Data Storage, Data Architecture, Requirements Analysis, Data Processing, Data Warehousing, Query Languages, Apache Hadoop, Extract, Transform, Load, Data Lakes, Amazon Web Services, Apache Spark, Database Systems, Feature Engineering, SQL, Data Integration, Infrastructure as Code (IaC), Data Management

    4.7
    Rating, 4.7 out of 5 stars
    513 reviews

    Intermediate · Professional Certificate · 3 - 6 Months

  • Status: Free Trial

    University of Colorado Boulder

    Skills you'll gain: Data Mining, Unsupervised Learning, Data Warehousing, Data Pipelines, Data Processing, Data Integration, Data Modeling, Data Cleansing, Big Data, Supervised Learning, Data Transformation, Machine Learning Methods, Data Quality, Classification And Regression Tree (CART), Anomaly Detection, Data Science, Machine Learning Algorithms, Data Analysis, Data Presentation, Descriptive Analytics

    Build toward a degree

    4.1
    Rating, 4.1 out of 5 stars
    159 reviews

    Intermediate · Specialization · 1 - 3 Months

  • Status: Free Trial

    Google Cloud

    Skills you'll gain: Data Pipelines, Dataflow, Google Cloud Platform, Real Time Data, Data Maintenance, Data Lakes, Data Storage, MLOps (Machine Learning Operations), Data Analysis, Data Warehousing, Data Processing, Extract, Transform, Load, Cloud Engineering, Data Infrastructure, Cloud Infrastructure, Cloud Storage, Big Data, Tensorflow, Unstructured Data, Data Management

    4.6
    Rating, 4.6 out of 5 stars
    19K reviews

    Intermediate · Professional Certificate · 3 - 6 Months

  • Status: Free Trial
    Status: AI skills

    Google Cloud

    Skills you'll gain: Responsible AI, Data Lakes, Data Storytelling, Data Governance, Data Visualization, Data Presentation, Data Architecture, Data Pipelines, Data Visualization Software, Dashboard, Looker (Software), Cloud Infrastructure, Generative AI, Data Cleansing, Data Management, Cloud Storage, Data Transformation, Cloud Computing, Google Cloud Platform, Data Ethics

    4.4
    Rating, 4.4 out of 5 stars
    207 reviews

    Beginner · Professional Certificate · 3 - 6 Months

  • Status: Preview

    Google Cloud

    Skills you'll gain: CI/CD, Google Cloud Platform, Apache Airflow, MLOps (Machine Learning Operations), Data Pipelines, Tensorflow, Kubernetes, Metadata Management, Scikit Learn (Machine Learning Library), Containerization

    Advanced · Course · 1 - 3 Months

  • Google Cloud

    Skills you'll gain: Looker (Software), Big Data, SQL, Data Pipelines, Data Transformation, Extract, Transform, Load, Data Warehousing, Data Cleansing, Data Analysis, Data Visualization Software, Google Sheets, Google Cloud Platform, Data Import/Export, Data Integrity

    4.4
    Rating, 4.4 out of 5 stars
    69 reviews

    Intermediate · Course · 1 - 3 Months

  • Google Cloud

    Skills you'll gain: Google Cloud Platform, Tensorflow, Data Pipelines, MLOps (Machine Learning Operations), Kubernetes, Machine Learning

    Intermediate · Project · Less Than 2 Hours

  • Status: Preview

    Google Cloud

    Skills you'll gain: MLOps (Machine Learning Operations), CI/CD, Google Cloud Platform, Data Pipelines, Kubernetes, Tensorflow, Metadata Management, PyTorch (Machine Learning Library), Containerization

    Advanced · Course · 1 - 3 Months

  • Status: Free Trial

    IBM

    Skills you'll gain: NoSQL, Apache Hadoop, Apache Spark, MongoDB, PySpark, Apache Hive, Databases, Apache Cassandra, Big Data, Machine Learning, Generative AI, IBM Cloud, Applied Machine Learning, Kubernetes, Supervised Learning, Distributed Computing, Docker (Software), Database Management, Data Pipelines, Scalability

    4.5
    Rating, 4.5 out of 5 stars
    794 reviews

    Beginner · Specialization · 3 - 6 Months

  • Coursera Project Network

    Skills you'll gain: Interactive Data Visualization, Data Transformation, Data Presentation, Data Visualization Software, Business Intelligence, Data Analysis, Extract, Transform, Load, Exploratory Data Analysis, Pandas (Python Package), Data Collection, Data Pipelines, Jupyter, Virtual Environment, Python Programming

    Intermediate · Guided Project · Less Than 2 Hours

What brings you to Coursera today?

Leading partners

  • Google Cloud
  • IBM
  • EDUCBA
  • Whizlabs
  • Duke University
  • Packt
  • DeepLearning.AI
  • Microsoft