Information Theory for Complex Systems

Written by
  1. News
  2. Course information (overview, literature, teachers)
  3. Schedule and lecture plan
  4. Problem sessions and solutions
  5. Homework
  6. Projects
  7. Exam, grading, old exams with solutions

News

2 May 2018

ANNOUNCEMENT: PhD student position open in "information-theoretic methods for deep learning” at the Department of Electrical Engoneering; see web page

16 March 2018

Solutions to today's exam can be found here:

Thanks to all students for participating in the course. When you get a chance, please fill out the course evaluation form.

6 March 2018

Tomorrow is the last examples class. We will solve the exam from March 2016 (you'll find it in the zip file with old exams -- the zip file was updated today to include it).

1 March 2018

Solutions to 8.1, 8.2, 8.7 are posted below.

19 February 2018

Some solutions to Chapter 5 problems are posted below.

Homework 4 and Homework 5 are posted below.

16 February 2018

The plan for the examples classes is updated.

14 February 2018

The microscopically reversible Ising dynamics paper discussed today is available as an open access paper.

Note that the lecture room on Friday is EC.

9 February 2018

Solutions to 4.6, 4.7, 4.8, 4.9 are posted below.

8 February 2018

The plan for the examples classes is updated.

Homework 3 is posted. Please find the file below.

31 January 2018

Deadline for Homework 2 is postponed by one week, until 9 February.

26 January 2018

Under the heading Problem sessions and solutions you will now find

  • Some solutions to problems from Chapter 2 and
  • a plan for the problem session on Monday 29 January.

25 January 2018

On the Friday lecture the 26th, we will discuss the Egg dropping puzzle. Thinking about coding may help you to solve this problem.

23 January 2018

On the Wednesday lecture the 24th, we will discuss the Monks Puzzle. What information is provided by (or results from) the Guru's statement?

Homework 2 is now posted. Please find the file below.

17 January 2018

Exam date now corrected. It is March 16, at 14.00-18.00.

11 January 2018

Homework 1 is posted. Also made a small change in the lecture plan.

10 January 2018

Lecture plan, instructions for projects, and old exams are now posted. Please see links below.

4 January 2018

The course starts on Monday, January 15, at 15.15 in MC with an introductory lecture.

Course schedule is available in TimeEdit.

As an introductory problem we will discuss a balance measurement puzzle from an information perspective during the first week.

The course will be based on the freely available Lecture Notes: Information theory for complex systems.

General course information (overview, literature, teachers)

Overview

The course provides an understanding of fundamental concepts used to describe complex systems, in particular dynamical systems such as chaotic low-dimensional systems, self-organizing systems, and simple spatially extended systems such as cellular automata. Many of the concepts are based in information theory.

  • Basic concepts of information theory: Shannon entropy, complexity measures.
  • Information theory and statistical mechanics.
  • Geometric information theory -- randomness and complexity in spatially extended systems.
  • Information flow. The relation between microscopic and macroscopic levels.
  • Statistical models, in particular hidden Markov models.
  • Cellular automata.
  • Applications in nonlinear dynamics, computational biology, chemical self-organizing systems, and statistical mechanics.

Literature

The lectures will follow the presentation in:
K. Lindgren, Information theory for complex systems — An information perspective on complexity in dynamical systems, physics, and chemistry. (Chalmers, 2014.)

If you want to learn more: T. M. Cover and J. A. Thomas, Elements of information theory (Wiley, 1991).

See also: David MacKay, Information theory, Inference, and Learning (2003).

Teachers

Kristian Lindgren (lecturer, examiner). Email: kristian.lindgren [at] chalmers.se

Rasmus Einarsson (examples classes, projects) Email: rasmus.einarsson [at] chalmers.se

Schedule and lecture plan

The schedule is in TimeEdit.

Further details are given the lecture plan (pdf).

Problem sessions and solutions

This section contains some details on past and upcoming problem sessions.

19 January 2018 We solved some problems from Chapter 2: 2.2, 2.4, 2.6, 2.8, 2.9. Also discussed the voluntary projects -- please let us know if you would like to work on a project.

29 January 2018 We will discuss Homework 1, and work together on problems from Ch 2 and Ch3. Please prepare by looking at 2.16, 3.2, 3.3. We will also look at 3.7 and/or 3.8 which might be helpful as you are solving Homework 2.

5 February 2018 We finished 3.3 and 3.7 from last time and then solved 3.5 and 4.1

9 February 2018 We will continue solving exercises from Chapter 4. We'll at least work on 4.2, 4.4, 4.6. If there is time we might work also on 4.5, 4.7, and/or 4.8.

16 February 2018 We will solve exercises from Chapter 5: We will at least work on 5.1, 5.4, 5.6, 5.8. If there is time: 5.2, 5.3.

28 February 2018 Chapter 8: 8.1, 8.2, 8.7.

7 March 2018 We will solve the exam from 2016-03-18. (Find it among the old exams below.)

Solutions

Homework

Five optional homework problems will be given. Each one gives up to two (2) extra points for the exam. Late submissions will normally not be graded. Hand-written solutions are fine, but please take care to make them legible.

Hand in your solutions in one of these ways:

  • on paper at the lecture
  • by email as a PDF file named [yourcid].pdf (e.g., rasmuse.pdf) to Rasmus (address: rasmus.einarsson [at] chalmers.se)

Downloads:

Projects

Optional project work can be done in groups of 1-3 students. The project work is awarded up to 10 extra points for the exam. Further instructions are given in this file:

Project ideas and instructions for projects (pdf)

Exam, grading, old exams with solutions

The exam is given on March 16, afternoon. A sheet with relevant equations etc is attached to the exam problems.

The course is graded based on the exam score including extra points from homework (max 10 points) and projects (max 10 points). The exam gives up to 50p. Grade limits (Chalmers/ECTS): 25p for 3/E, 28p for 3/D, 34p for 4/C, 38p for 4/B, 42p for 5/A. To pass, a minimum of 20p on the written exam is required, regardless of additional points.

Old exams with solutions

Old exams, some of them with solutions (zip file with pdfs, about 17 MB) [updated 6 March 2018]

Kristian Lindgren

Kristian Lindgren is professor in complex systems. He has a background in engineering physics, but since his graduate studies in the 1980's he has been working with complex systems in a variety of different disciplines. Some main areas are (i) information theory for complex and self-organizing systems, (ii) game theory for evolutionary systems, and (iii) agent-based modeling of economic systems. Since the mid 1990's Lindgren has also been working in the area of energy systems with development of models of regional and global energy systems in a climate change perspective.

Lindgren is Director of the Graduate school (PhD) for Complex Systems at Chalmers, and he is teacher and examiner in the International Masters Programme in Complex Adaptive Systems.

 

More in this category: Stochastic Optimization Algorithms »