《神经网络与深度学习》教学大纲

课程代码

045102711

课程名称

神经网络与深度学习

英文名称

Neural Networks and Deep Learning

课程类别

选修课

课程性质

选修

学时

总学时:32实验学时:实习学时:其他学时:

学分

2

开课学期

第六学期

开课单位

计算机科学与工程学院

适用专业

计算机科学与技术全英联合班,全英创新班(本硕、本博连读)

授课语言

英文授课

先修课程

概率论,离散数学,线性代数,数学分析, C++

课程对毕业要求的支撑

本课程对学生达到如下毕业要求有如下贡献:

 1、研究:能够基于科学原理并采用科学方法对与计算机相关复杂工程问题进行研究,包括设计实验、分析与解释数据、并通过信息综合得到合理有效的结论。

 2、使用现代工具:能够针对与计算机相关复杂工程问题,开发、选择与使用恰当的技术、资源、现代工程工具和信息技术工具,包括对复杂工程问题的预测与模拟,并能够理解其局限性。

 3、工程与社会:能够基于工程相关背景知识进行合理分析,评价计算机专业工程实践和复杂计算机工程问题解决方案对社会、健康、安全、法律以及文化的影响,并理解应承担的责任。

4、环境和可持续发展:能够理解和评价针对与计算机相关复杂工程问题的专业工程实践对环境、社会可持续发展的影响。

课程目标

完成课程后,学生将具备以下能力:

1)掌握神经网络的发展历史和设计思想,培养学生发现问题、解决问题的基本能力。

2)掌握几种主要的神经网络训练和优化算法,并具有设计模型的能力。

3)掌握几个基本深度学习模型及其训练算法,了解当今深度学习的前沿进展和成果应用,提高学生对某些具体问题的建模能力,激发学生对人工智能的学习兴趣。

课程简介

当前人工智能领域发展迅速,特别是以神经网络、深度学习为代表模型和方法在图像、视频、语音、文本等多个应用领域取得了巨大进展。本课程主要围绕神经网络的基础模型和最新的深度学习模型等知识点展开。主要内容包括神经网络基本原理与优化方法、感知机、逻辑回归、前馈型神经网络、Backpropagation (BP)算法、降维与重构、自编码器、卷积神经网络(CNN)、递归神经网络、长短期记忆网络(LSTM)、状态回声网络等,主要让学生熟悉主流神经网络模型的原理、设计思想,掌握几种主要的神经网络训练和优化算法,了解当今深度学习的前沿进展和成果应用,并初步了解人工智能最前沿的研究方法。

教学内容与学时分配

(一)课程思政教学、课程目的、意义与内容组织、学时安排介绍    1学时

教学要求:结合新一代人工智能发展规划,介绍我国人工智能面临的机遇与挑战,并理解本课程所学内容在人工智能方向的作用与任务,了解本课程要学习的基本内容架构。

(二)神经元与神经网络基础知识                                2学时

教学要求:了解人工智能的发展,特别是神经网络的发展历史,理解人工神经网络中的神经元模型,掌握神经网络的基本模型及参数。

重点:掌握神经元模型、神经网络发展历史及模型参数

难点:神经元模型,基本神经网络的模型参数

(三)感知器与逻辑回归                                2学时

教学要求:掌握感知器模型的设计与学习过程,掌握逻辑回归模型的原理和参数,理解感知器在人工神经网络中地位和逻辑回归在神经网络中的作用。

重点:掌握感知器模型和逻辑回归模型

难点:多层感知机,反向传播算法 (BP)

(四)深度前馈网络                                2学时

教学要求:掌握前馈网络设计与学习过程,理解前馈网络到深度前馈网络的发展过程,以及深度前馈网络成功的应用领域。

重点:理解深度前馈网络的结构

难点:深度前馈网络的设计及训练算法

(五)深度学习中的正则化                               4学时

教学要求:理解过拟合的概念,掌握正则化方法的定义、以及正则化方法在深度学习中的作用。

重点:理解过拟合的概念

难点:不同正则化方法在深度学习中的作用

(六)深度学习中的优化                              4学时

教学要求:理解优化的作用,掌握随机梯度下降算法。

重点:随机梯度下降算法

难点:随机梯度下降算法

(七)卷积神经网络(CNN)                              4学时

教学要求:掌握卷积神经网络的结构设计、训练方法,理解卷积操作和池化操作各自的作用,了解当前主要深度卷积神经网络结构及其应用。

 1)卷积神经网络的训练和流行架构                 2学时

 2)卷积神经网络的应用                           2学时

重点:掌握卷积和池化的概念和作用

难点:卷积和池化的设计

(八)递归神经网络(RNN)                              4学时

教学要求:掌握一般递归神经网络的结构、训练方法,理解递归神经网络与前馈神经的区别和联系,掌握长短期记忆能力网络(LSTM)设计,了解当前递归神经网络的应用。

 1)一般递归神经网络的训练和流行架构                     1学时

 2)长短期记忆网络(LSTM)的原理、设计与应用              2学时

 3)状态回声递归神经网络(ESN)的原理与实现                1学时

重点:递归神经网络的设计与训练,长短期记忆网络的结构与训练

难点:递归神经网络的动态特性

(九)主成分分析(PCA)与独立成分分析(ICA)                    4学时

教学要求:掌握降维概念、主成分分析方法以及独立成分分析。

 1)主成分分析                       2学时

 2)独立成分分析                     2学时

重点:理解降维的概念,掌握主成分分析过程,掌握独立成分分析过程。

难点:主成分分析过程和独立成分分析过程。

(十)自编码器                                2学时

教学要求:掌握自编码器的设计与训练。

重点:理解自编码器的概念与模型设计。

难点:自编码器的训练。

(十一)生成对抗网络(GAN)                           2学时

教学要求:掌握生成对抗网络的基本思想,理解生成对抗网络的训练及应用。

重点:掌握生成对抗网络的基本思想,生成器和判别器的设计。

难点:生成对抗网络的训练。

(十二)深度学习的最新进展及应用                           1学时

教学要求:了解当前深度学习的最新进展及应用。

重点:当前深度学习的最新进展及应用



实验教学(包括上机学时、实验学时、实践学时)

教学方法

课程教学以课堂教学、综合讨论、课程大作业以及授课教师的科研项目等共同实施。

考核方式

本课程注重过程考核,成绩比例为:

平时考勤和课堂表现:20%

课程大作业或项目:40%

期末考试(闭卷):40%

教材及参考书

教材:Ian Goodfellow,  Yoshua Bengio and Aaron Courville. Deep Learning[M], MIT Press, 2016.

主要参考资料:

 Simon Haykin. Neural Networks and Learning Machines [M]. 机械工业出版社, 2009.

周志华. 机器学习[M]. 清华大学出版社, 2016.

 Geoffrey Hinton, Neural Networks for Machine Learninghttps://www.coursera.org/learn/neural-networks

 Domingos, Pedro. A few useful things to know about machine learning [J]. Communications of the ACM 55.10, 2012: 78-87.



制定人及制定时间

马千里,2019413

 “Neural Networks and Deep Learning” Syllabus

Course Code

045102711

Course Title

Neural Networks and Deep Learning

Course Category

Specialty-related Course

Course Nature

Elective Course

Class Hours

32

Credits

2

Semester

6

Institute

School of Computer Science and Engineering

ProgramOriented

Computer Science and Technology Full English Creative Class (Bachelor-Master, Bachelor-Doctor Successive Program)

Teaching Language

English

Prerequisites

Linear Algebra & Analytic GeometryProbability & Mathematical StatisticsMathematics AnalysisDiscrete MathematicsAdvanced Language Programming (C++)

 Student Outcomes

 (Special Training Ability)

1. Research: An ability to conduct investigations of complex engineering problems based on scientific theories and adopting scientific methods including design of experiments, analysis and interpretation of data and synthesis of information to provide valid conclusions.

2. Applying Modern Tools: An ability to create, select and apply appropriate techniques, resources, and modern engineering and IT tools, including prediction and modeling, to complex engineering activities, with an understanding of the limitations.

3. Engineering and Society: An ability to apply reasoning informed by contextual knowledge to assess societal, health, safety, legal and cultural issues and the consequent responsibilities relevant to professional engineering practice.

4. Environment and Sustainable Development: An ability to understand and evaluate the impact of professional engineering solutions in environmental and societal contexts and demonstrate knowledge of and need for sustainable development.

Course Objectives

When students finish this course, they will know the history of neural networks, and current developments of deep learning. They will learn skills to design the task-specific deep neural networks, and use the appropriate optimization algorithms to train them. They will also learn skills to do cutting-edge research by implementing, training and debugging neural networks.

Course Description

This course will cover the basics about neural networks, as well as recent developments in deep learning including convolutional neural networks (CNN), recurrent neural networks (RNN), and long-short term memory (LSTM). We will study details of the deep learning architectures with a focus on learning end-to-end models for classification and prediction. Students will learn skills to do research by implementing, training and debugging neural networks.

Teaching Content and Class Hours Distribution

  1. Introduction of the course              1 Class Hour

Introduce the new generation ai development plan of China, the objectives and structures of this course.

  1. Neurons and Neural networks           2 Class Hours

Key points: Neurons model, Developments of neural networks

Difficulties: Neurons model, neural networks

  1. Perceptions and Logistic Regression      2 Class Hours

Key points: Multilayer Perception and Logistic Regression

Difficulties: Multilayer Perception, Back propagation (BP)

  1. Deep Feedforward Networks            2 Class Hours

Key points: The structure of deep feedforword networks

Difficulties: The design of deep feedforword networks and gradient-based learning

  1. Regularization for Deep Learning         4 Class Hours

Key points: Understanding the concept of overfitting

Difficulties: Regularization methods

  1. Optimization for Training Deep Models    4 Class Hours

Key points: stochastic gradient descent

Difficulties: stochastic gradient descent

  1. Convolutional Networks                 4 Class Hours

 (1) Convolutional Networks: Training and Modern Architectures   2 Class Hours

(2) Convolutional Networks: Applications                     2 Class Hours

Key points: Convolution, Pooling

Difficulties: Convolution, Pooling

  1. Recurrent and Recursive Networks                 4 Class Hours

(1) General recurrent networks                    1 Class Hours

(2) Long short-term memory (LSTM)               2 Class Hours

(3) Echo state networks (ESNs)                    1 Class Hours

 Key points: Long short-term memory (LSTM), Echo state networks (ESNs)

Difficulties: Long short-term memory (LSTM)

  1. Principal components analysis (PCA) and independent component analysis ( ICA)                                        4 Class Hours

(1) Principal components analysis (PCA)            2Class Hours

(2) Independent component analysis ( ICA)          2 Class Hours

Key points: PCA, ICA

Difficulties: PCA, ICA

  1. Autoencoders                                   2 Class Hours

Key points: AE, DAE

Difficulties: AE, DAE

  1. Generative Adversarial Networks (GAN)            2 Class Hours

Key points: the concept and design of GAN

Difficulties: design of GAN

  1. Current developments and applications of deep learning    1 Class Hours

Key points: the applications of deep learning



Experimental Teaching

No

Teaching Method

Classroom TeachingDiscussionProjects

Examination Method

Attendance and classroom participation: 20%

Final project: 40%

Final examination: 40%

Teaching Materials and Reference Books

Text BookIan Goodfellow,  Yoshua Bengio and Aaron Courville. Deep Learning[M], MIT Press, 2016.

Reading List

Simon Haykin. Neural Networks and Learning Machines [M]. 2009.

Zhihua Zhou. Machine learning [M], 2016.

Geoffrey Hinton, Neural Networks for Machine Learninghttps://www.coursera.org/learn/neural-networks

Domingos, Pedro. A few useful things to know about machine learning [J]. Communications of the ACM 55.10, 2012: 78-87.

Prepared by Whom and When

MA Qianli   2019.4.13