Data Engineering and Data Integration Tools

About Data Engineering and Data Integration Tools

A warm welcome to the

Data Engineering and Data Integration Tools

course by

Uplatz

. Extract, Transform, Load (ETL) or Extract, Load, Transform (ELT) concepts are core of any datawarehousing initiative. Data ingestion, integration, and processing form a critical task for consolidating the data silos across the departments in an organisation and ultimately to build a robust and flexible datawarehouse for enterprise reporting & analytics. One of such tools is Talend. Talend is an ETL tool/software for Data Integration. It delivers software resolutions for data groundwork, data quality, data integration, application integration, data management and big data. There exist separate products of these different solutions in Talend. Big Data products and data integration are broadly used in Talend. Data integration and data management solutions are offered by Talend, as an open source platform. Big data integration is a specialty of Talend. Other features provided by Talend are related to cloud, big data, enterprise application integration, master data management and data quality. It also provides a unified repository to store and reuse the Metadata. Talend is one of the finest tools for cloud computing and big data integration. The most common invention of Talend Studio is data integration and big data. Talend can smoothly arrange big data integration with graphical tools and wizards. This permits the group to generate a condition to easily work with Apache Hadoop, Spark, and NoSQL databases for cloud. Talend data integration software tool has an open, accessible architecture. It permits quicker response to business needs. The tool contracts to modify and arrange data integration jobs faster than hand coding. Talend integration cloud tool offers connectivity, built-in data quality, and native code generation. Talend is protected cloud integration platform which allows IT and business users to connect shared both could and on-premise. It solves the power of cloud design job as it can manage, monitor, and control in the cloud.

Uplatz

provides this end-to-end course on this leading Data Integration and ETL tool called Talend. With many organizations using Talend as their leading data warehousing and data integration software, there are huge career prospects by learning and mastering Talend. If you wish to become an ETL Architect or a Data Integration Engineer, then Talend course can be a complete game changer.

Talend - Course Curriculum

1. Role of Open Source ETL Technologies in Big Data

Overview on: TOS (Talend Open Studio) for Data Integration

ETL concepts

Data warehousing concepts

2. Talend

Why Talend?

Features

Advantages

Talend Installation/System Requirements

GUI layout (designer)

Understanding it's Basic Features

Comparison with other market leader tools in ETL domain

Important areas in Talend Architecture: Project

Workspace

Job

Metadata

Propagation

Linking components

3. Talend: Read & Write various Types of Source/Target System

Data Source Connection

File as Source

Create meta data

Database as source

Create metadata

Using MySQL database (create tables, Insert, Update Data from Talend)

Read and write into excel files, into multiple tabs

View data

How to capture log and navigate around basic errors

Role of tLogrow and how it makes developers life easy

4. Talend: How to Transform Your Business: Basic

Using Advanced components like: tMap, tJoin, tFilter, tSortRow, tAggregateRow, tReplicate, tSplit, Lookup, tRowGenerator

5. Talend: How to Transform Your Business: Advanced 1

Trigger (types) and Row Types

Context Variables (parameterization)

Functions (basic to advanced functions to transform business rules such as string, date, mathematical etc.)

Accessing job level / component level information within the job

6. Talend: How to Transform Your Business: Advanced 2

Type Casting (convert data types among source-target platforms)

Looping components (like tLoop, tFor)

tFileList

tRunJob

How to schedule and run talend DI jobs externally (not in GUI)

7. Working with Hierarchical File Structures

Read and Write an XML file, configure the schema and XPath expression to parse an XML file

Read and Write a JSON file, configure the schema and JSONPath expression to parse a JSON file

Read and write delimited, fixed width files.

8. Context Variables and Global Variables

Create context/global variables

Use context/global variables in the configuration of Talend components

Load context variables from a flow

9. Best practices

Working with databases and implementing data warehousing concepts

Working with files (excel, delimited, JSON, XML etc.)

10. Orchestration and Controlling Execution Flow

Files - Use components to list, archive, and delete files from a directory

Database – Controlling Commit and Rollback

COMMIT at end of job/ every x number of rows

Rollback on error

11.

Shared DB connection across jobs and subjobs

Use triggers to connect components and subJobs

Orchestrate several jobs in master jobs.

Handling Errors

Kill a Job on a component error

Implement a specific Job execution path on a component error

Configure the log level in the console