Information Theory for Complex Systems

Written by
  1. News
  2. Course information (overview, literature, teachers)
  3. Schedule and lecture plan
  4. Problem sessions and solutions
  5. Homework
  6. Projects
  7. Exam, grading
  8. Exams with solutions

News

12 April 2019

Solution sketches for this year's exam 2019-03-22 are available here.

The exams have been corrected, but there will be a delay until they show up in Ladok. Sorry for that!

 

3 April 2019

Two open PhD student positions in Complex systems are announced (as part of a broader call for applications), see:

Two PhD student positions in Energy, Environment & Complex Systems

 

13 March 2019

Solutions to exam 2016-03-18 are now included in the zip file with old exams below.

20 February 2019

The following course representatives have accepted: Fanny, Gabriella, Henrik, and Wilhelm. We had a short meeting, and a request that was brought up is that I post PDFs of slides that I use in the lectures. So, as a start, I provide the slides of today's lecture in the following summary.   

In the lecture today, we discussed an Ising dynamics model (microscopically reversible and energy conserving) and how the approach to equilibrium can be understood. The slides are available here, and for further reading you may download the paper "The approach towards equilibrium in an Ising dynamics model".

18 February 2019

The solution to problem 5.8 in the PDF below is updated with a minor addition regarding the criterion for the parameter α.

11 February 2019

Note that the exam is on Friday, March 22, in the afternoon. (Previously, the last year's date was posted.)

2 February 2019

In the third week we will bring up the 100 Floors Egg Dropping Puzzle. Thinking about coding may guide you to a solution of this problem.

1 February 2019

We have posted a preliminary list of problems to be solved in the problem sessions below.

27 January 2019

The second week we will introduce information theory for symbol sequences, which will serve as a key theoretical basis for several applications throughout the course.

An interesting problem will be discussed on Wednesday: The Monks Spots Puzzle. Try to solve that. Consider also the situation in an information perspective (which is indeed very tricky).

21 January 2019

The course starts with an introductory lecture on Monday 21 January at 15.15 in MC.

All necessary information about the course is available below, including link to TimeEdit that contains the schedule, the lecture plan, the lecture nores in pdf, etc.

You may think about the balance problem puzzle that will be discussed in the first week.

 

General course information (overview, literature, teachers)

Overview

The course provides an understanding of fundamental concepts used to describe complex systems, in particular dynamical systems such as chaotic low-dimensional systems, self-organizing systems, and simple spatially extended systems such as cellular automata. Many of the concepts are based in information theory.

  • Basic concepts of information theory: Shannon entropy, complexity measures.
  • Information theory and statistical mechanics.
  • Geometric information theory -- randomness and complexity in spatially extended systems.
  • Information flow. The relation between microscopic and macroscopic levels.
  • Statistical models, in particular hidden Markov models.
  • Cellular automata.
  • Applications in nonlinear dynamics, computational biology, chemical self-organizing systems, and statistical mechanics.

Literature

The lectures will follow the presentation in:
K. Lindgren, Information theory for complex systems — An information perspective on complexity in dynamical systems, physics, and chemistry. (Chalmers, 2014.)

If you want to learn more: T. M. Cover and J. A. Thomas, Elements of information theory (Wiley, 1991).

See also: David MacKay, Information theory, Inference, and Learning (2003).

Teachers

Kristian Lindgren (lecturer, examiner). Email: kristian.lindgren [at] chalmers.se

Susanne Pettersson (examples classes, projects) Email: susannep [at] chalmers.se

Rasmus Einarsson (examples classes, projects) Email: rasmus.einarsson [at] chalmers.se

Schedule and lecture plan

The schedule is in TimeEdit.

Further details are given the lecture plan (pdf).

Problem sessions and solutions

Preliminary list of problems to be solved in the problem sessions

  • 25 January: 2.2, 2.4, 2.6, 2.8, 2.9, 2.16
  • 1 February: 3.2, 3.3, 3.5, 3.7, 3.8 (possibly leaving one or two for the next session)
  • 13 and 15 February: 4.1, 4.2, 4.4, 4.5, 4.6, 4.7, 4.8, 4.9
  • 22 February: 5.1, 5.2, 5.3, 5.4, 5.6, 5.8
  • 6 March: 8.1, 8.2, 8.7
  • 13 March: solving an old exam (which one to be determined)

Solutions

Homework

Five optional homework problems are given below. Each one gives up to two (2) extra points for the exam. Late submissions will normally not be graded. Hand-written solutions are fine, but please take care to make them legible.

Hand in your solutions in one of these ways:

  • on paper at the lecture
  • by email as a PDF file named yourcid.pdf (e.g., rasmuse.pdf) to Rasmus (address: rasmus.einarsson [at] chalmers.se)

The deadlines are:

  • Homework 1: Friday 1 February 2019, 13.15
  • Homework 2: Friday 15 February 2019, 13.15
  • Homework 3: Friday 22 February 2019, 13.15
  • Homework 4: Wednesday 6 March 2019 10.00
  • Homework 5: Wednesday 13 March 2019 10.00

Downloads:

Projects

Optional project work can be done in groups of 1-3 students. The project work is awarded up to 10 extra points for the exam. Further instructions are given in this file:

Project ideas and instructions for projects (pdf)

Exam, grading

The exam is given on March 22, afternoon. A sheet with relevant equations etc is attached to the exam problems.

The course is graded based on the exam score including extra points from homework (max 10 points) and projects (max 10 points). The exam gives up to 50p. Grade limits (Chalmers/ECTS): 25p for 3/E, 28p for 3/D, 34p for 4/C, 38p for 4/B, 42p for 5/A. To pass, a minimum of 20p on the written exam is required, regardless of additional points.

Exams and solutions

Old exams, some of them with solutions (zip file with pdfs, about 19 MB) [updated 13 March 2010]

Exam 2018-03-16 problems (pdf) and solutions (pdf)

Kristian Lindgren

Kristian Lindgren is professor in complex systems. He has a background in engineering physics, but since his graduate studies in the 1980's he has been working with complex systems in a variety of different disciplines. Some main areas are (i) information theory for complex and self-organizing systems, (ii) game theory for evolutionary systems, and (iii) agent-based modeling of economic systems. Since the mid 1990's Lindgren has also been working in the area of energy systems with development of models of regional and global energy systems in a climate change perspective.

Lindgren is Director of the Graduate school (PhD) for Complex Systems at Chalmers, and he is teacher and examiner in the International Masters Programme in Complex Adaptive Systems.

 

More in this category: Stochastic Optimization Algorithms »