Advanced Deep Learning — SS 2025

Lecture Advanced Deep Learning, Summer Semester 2025

Lectures:

• Monday 14:30–16:00, Room 2611.HS 6E
• Wednesday 8:30–10:00, Room 2301.HS 3C

Exercise Class: Tuesday 12:30–14:00, Room 2522.00.82

Assistant: Pawel Batorski (pawel.batorski@hhu.de)

Tutors: Malte Günnewig (malte.guennewig@hhu.de), Abtin Pourhadi (abtin.pourhadi@hhu.de)

Secretary: Annette Peitsch (annette.peitsch@hhu.de)

Remote access: Lecture streamed via Webex at hhu.webex.com/meet/pswoboda

Announcements

• Exam inspection (Klausureinsicht): Monday, August 4th, 4pm–6pm, Room 25.13.02.028
• Online lecture on Wednesday, May 21st
• No lecture on Monday, May 19th
• First exercise on Wednesday, April 15th

Topics

• GPT2 model/inference, training, data parallelism
• Flash attention, fine-tuning
• RL/PPO for RLHF
• RoPE & GQA
• RNN, LSTM, SSM/Mamba
• ViT/Axial Attention/Swin/MLP-Mixer
• MoE, Parallelism, Large-Scale training
• LLM Prompting/Reasoning
• Transformer architectures

Assessment: Exercise sheets on Ilias.