LDSCI7229 Advanced Data Engineering Course Descriptor

Course Overview

In this course, you will learn more about the principles and concepts of data engineering and big data architecture. Then you will learn how data engineers work with big data. You will also develop a better understanding of how to build, schedule and monitor data pipelines. This understanding will advance your knowledge of data storage architectures, including non-relational databases, and will comprehensively allow you to use statistical and mathematical foundations and advanced practical knowledge of AI and machine learning methodologies applied to complex datasets to meet business objectives. There is a particular focus on sustainable development.

Course Aims

The aims of the course are to: 

• Demonstrate an understanding of data engineering principles and concepts, and big data architecture.
• Develop a critical awareness of current issues and developments in data engineering.
• Demonstrate the ability to apply tools and techniques used to develop data solutions in business.

Learning Outcomes

On successful completion of the course, students will be able to:

Knowledge and Understanding

Subject Specific Skills

Transferable and Professional Skills

Teaching and Learning

This course has a dedicated Virtual Learning Environment (VLE) page with a syllabus and range of additional resources (e.g. readings, question prompts, tasks, assignment briefs, discussion boards) to orientate and engage you in your studies.

The scheduled teaching and learning activities for this course are:

Lectures/labs: Contact hours are typically a mix of weekly lectures and lab sessions, totalling up to 50 scheduled hours:

• Version 1: All sessions in the same sized group, or
• Version 2: most of the sessions in larger groups; some of the sessions in smaller groups

Faculty hold regular ‘office hours’, which are opportunities for students to drop in or sign up to explore ideas, raise questions, or seek targeted guidance or feedback, individually or in small groups.

Students are to attend and participate in all the scheduled teaching and learning activities for this course and to manage their directed learning and independent study.

Indicative total learning hours for this course: 300

Assessment

Both formative and summative assessment are used as part of this course, with formative opportunities typically embedded within interactive teaching activities delivered via the VLE.

Summative

Further information about the assessments can be found in the Course Syllabus.

Feedback

Students will receive formative and summative feedback in a variety of ways, written (e.g. marked up on assignments or via the VLE) or oral (e.g. as part of interactive teaching sessions or in office hours).

Indicative Reading

Note: Comprehensive and current reading lists for courses are produced annually in the Course Syllabus or other documentation provided to students; the indicative reading list provided below is used as part of the approval/modification process only.

Books

• Data Engineering with Python Work with Massive Datasets to Design Data Models and Automate Data Pipelines Using Python.
• Cassandra: The Definitive Guide, (Revised) Third Edition, 3rd Edition. (2022). Cassandra. O’Reilly Media, Inc.
• White, T. (2015). Hadoop : the definitive guide. 4th ed. ed. Sebastopol, California: O’Reilly.
• Practical Apache Spark : using the Scala API
• Apache Spark for data science cookbook : over 90 insightful recipes to get lightning-fast analytics with Apache Spark
• Apache Hive essentials : essential techniques to help you process, and get unique insights from, big data
• Data pipelines with Apache Airflow.

Indicative Topics

Note: Comprehensive and current topics for courses are produced annually in the Course Syllabus or other documentation provided to students. The indicative topics provided below are used as a general guide and part of the approval/modification process only.

• Data engineering and data pipelines
• Machine learning models development and operations (MLOps)
• Big data software products
• Data engineering software products
• Non-relational databases

Version History