Information Theory for Complex Systems

Written by
  1. News
  2. Course information (overview, literature, teachers)
  3. Schedule and lecture plan
  4. Homework
  5. Projects
  6. Exam, grading, old exams with solutions

News

18 March 2017

The exam was held on March 17, and sketches to solutions on the exam problems are now available. 

7 March 2017

On the Monday lecture (6th), Kristian presented some results from previous and ongoing research related to the course content, including "A microscopically reversible dynamic spins system model" (forthcoming), "Expressing the entropy of lattice systems as sums of conditional entropies" (J. Stat. Phys.), and "On the equivalence between stochastic baker's maps and two-dimensional spin systems" (Europhysics Letters).

1 March 2017

Homework 5 is published (see below).

14 February 2017

Homework 4 is published (see below).

10 February 2017

The Egg dropping puzzle will be discussed on the Monday (13th) lecture; there are information/coding ideas that can guide a solution.

1 February 2017

Homework 3 is published (see below).

A pdf file with instructions and suggestions for the optional projects is published. See below. Please note that you must email Kristian and Rasmus no later than Friday 10 February if you wish to have supervision meetings.

26 January 2017

If you wish to submit assignments by e-mail, please do so to rasmus.einarsson [at] chalmers.se

24 January 2017

Homework 2 is published (see below).

23 January 2017

The first homework assignment was unclear regarding the probabilities of different color combinations. We have updated the file with a clarification.

On Wednesday 25th, we will discuss the Monks Spots Puzzle, and especially what kind of information was provided by the Guru!

We started the course a discussion on a balance measurement puzzle, from an information perspective.

 

General course information (overview, literature, teachers)

Overview

The course provides an understanding of fundamental concepts used to describe complex systems, in particular dynamical systems such as chaotic low-dimensional systems, self-organizing systems, and simple spatially extended systems such as cellular automata. Many of the concepts are based in information theory.

  • Basic concepts of information theory: Shannon entropy, complexity measures.
  • Information theory and statistical mechanics.
  • Geometric information theory -- randomness and complexity in spatially extended systems.
  • Information flow. The relation between microscopic and macroscopic levels.
  • Statistical models, in particular hidden Markov models.
  • Cellular automata.
  • Applications in nonlinear dynamics, computational biology, chemical self-organizing systems, and statistical mechanics.

Literature

The lectures will follow the presentation in:
K. Lindgren, Information theory for complex systems — An information perspective on complexity in dynamical systems, physics, and chemistry. (Chalmers, 2014.)

If you want to learn more: T. M. Cover and J. A. Thomas, Elements of information theory (Wiley, 1991).

See also: David MacKay, Information theory, Inference, and Learning (2003).

Teachers

Kristian Lindgren (lecturer, examiner). Email: kristian.lindgren [at] chalmers.se

Rasmus Einarsson (examples classes, projects) Email: rasmus.einarsson [at] chalmers.se

Schedule and lecture plan

The schedule is in TimeEdit.

Further details are given the lecture plan (pdf).

Homework

Five optional homework problems will be given. Each one gives up to two (2) extra points for the exam.

Hand in your solutions on paper at the lecture, or by email to Rasmus (address: rasmus.einarsson [at] chalmers.se). Late assignments will not be corrected.

Homework 1 (pdf). Deadline 27 January 2017, 13:15. Updated 23 January
Homework 2 (pdf). Deadline 3 February 2017, 13:15.
Homework 3 (pdf). Deadline 10 February 2017, 13:15.
Homework 4 (pdf). Deadline 3 March 2017, 13:15.
Homework 5 (pdf). Deadline 8 March 2017, 10:00.

Projects

Optional project work can be done in groups of 1-3 students. The project work is awarded up to 10 extra points for the exam. Further instructions are given in this file:

Project ideas and instructions for projects (pdf)

Exam, grading, old exams with solutions

The exam is given on March 17, afternoon. A sheet with relevant equations etc is attached to the exam problems.

The course is graded based on the exam score including extra points from homework (max 10 points) and projects (max 10 points). The exam gives up to 50p. Grade limits (Chalmers/ECTS): 25p for 3/E, 28p for 3/D, 34p for 4/C, 38p for 4/B, 42p for 5/A. To pass, a minimum of 20p on the written exam is required, regardless of additional points.

Old exams with solutions

Old exams, some of them with solutions (zip file with pdfs, about 14 MB)

Kristian Lindgren

Kristian Lindgren is professor in complex systems. He has a background in engineering physics, but since his graduate studies in the 1980's he has been working with complex systems in a variety of different disciplines. Some main areas are (i) information theory for complex and self-organizing systems, (ii) game theory for evolutionary systems, and (iii) agent-based modeling of economic systems. Since the mid 1990's Lindgren has also been working in the area of energy systems with development of models of regional and global energy systems in a climate change perspective.

Lindgren is Director of the Graduate school (PhD) for Complex Systems at Chalmers, and he is teacher and examiner in the International Masters Programme in Complex Adaptive Systems.

 

More in this category: Stochastic Optimization Algorithms »