Pflichtseminar Deep Learning (Bachelor) — WS 2026/27

Pflichtseminar Deep Learning, Winter Semester 2026/27

Seminar for Bachelor students in Computer Science, offered as group 10 of the module Pflichtseminar (5 ECTS).

Schedule: Wednesday, 10:30–12:00

Location: Room 2533.00.41

First meeting: Wednesday, October 14th, 2026

Participants: up to 15. Places are not booked directly: students are assigned to the Pflichtseminar groups centrally at the beginning of October.

Format

Each participant works on one research paper:

  • Talk: about 30 minutes, presenting the paper’s question, method and main results.
  • Reproduction: a strongly reduced version of one central experiment — a small model, a toy problem or a handful of runs is enough. The point is to check a claim of the paper yourself, not to match its scale.

Topics are assigned in the first meeting. The topics on large language models need no model training, but they do need either API access to an LLM or a small open model running locally (e.g. via Ollama); please clarify this with us early in the semester.

Topics

Training dynamics

Mathematical papers on what gradient descent actually does — with elementary tools: linear algebra, basic analysis and simple ODEs.

  1. Saxe, McClelland & Ganguli 2013: Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
  2. Soudry et al. 2017: The Implicit Bias of Gradient Descent on Separable Data
  3. Arora, Cohen & Hazan 2018: On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
  4. Belkin et al. 2018: Reconciling modern machine learning practice and the bias-variance trade-off
  5. Lee et al. 2019: Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent
  6. Cohen et al. 2021: Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability

LLMs and mathematical tools

  1. Requena et al. 2026: A Minimal Agent for Automated Theorem Proving (ICML 2026)
  2. Xia, Gomes, Selman & Szeider 2026: Agentic Neurosymbolic Collaboration for Mathematical Discovery: A Case Study in Combinatorial Design
  3. Hasan et al. 2026: Model Context Protocol (MCP) Tool Descriptions Are Smelly!

Multi-agent systems

  1. Yuan et al. 2026: Rethinking Multi-Agent Collaboration: When More Is Less
  2. Li 2026: When Single-Agent with Skills Replace Multi-Agent Systems and When They Fail
  3. Shi, Zhang & Yang 2026: Emergent Collusion in Long-Horizon LLM Agent Interaction

Working with coding agents

  1. Fan et al. 2026: An Empirical Study of Harness Design for Coding Agents
  2. Ehsani et al. 2026: Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub (MSR 2026)

Automated research

  1. Zhang et al. 2025: MLRC-Bench: Can Language Agents Solve Machine Learning Research Challenges? (NeurIPS 2025)
  2. Huang et al. 2026: Reward Hacking Challenges Oversight of Autonomous Research Agents

LLMs as test subjects

  1. Schröder et al. 2025: Large Language Models Do Not Simulate Human Psychology

LLMs in games

  1. Kolasani et al. 2025: LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chess
  2. Doerschuk-Tiberi et al. 2026: Game Arena: Strategic LLM Evaluation in Competitive Environments
  3. Lin et al. 2026: How Far Are LLMs from Professional Poker Players? Revisiting Game-Theoretic Reasoning with Agentic Tool Use (ICLR 2026)
  4. Duffy et al. 2025: Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacy
  5. Liang et al. 2025: HardcoreLogic: Challenging Large Reasoning Models with Long-tail Logic Puzzle Games