Thursday, 3 January 2013

Information flow


When I was a kid we had this book about electricity. There was a tutorial how to create a small engine using paperclips and battery. I couldn't do it but I was fascinated how can you create a more complex thing from something basic. (I know it's not entirely true.) For many years I thought making complex things what really motivates me. Then I realized it's information that is amazing.


I guess many learned in school about information theory. It sucked, right? Never understood why you learn something you won't use for a decade at least. Now I get it.

I'm a developer. What I do is I'm crafting, manipulating and transforming information. Well, to be really precise, I create high level rules. Why is it really important? Because you only get programming through information awareness. Whenever you develop a system you work with data and guide that data through various control structures. You need to know what is happening with all variables at all states of your program. Optionally you try to keep consistency - at the end that makes sure you don't break your code.

When I do code I often think of how the information flows though my functions, classes and other parts of the project. I think one of the biggest skill in coding is to maintain a mental abstraction stack in your brain. (And on purpose I'm not trying to emphasize code readability now.) Data structures are more effective when they can represent data by behavior. And control structures are acting as (de)coders in the process. When you hold these transformations in mind and try to change the behavior of a tier you need to transfer that change to the upper tiers. And yeah - good luck if it's not transitive or you have to encounter with information loss or redundancy.

It's really interesting when I see other developers handling information badly. I know - if it works sometimes it's enough. But I like the straight way that targets and keeps the entropy (I know I have to use this work very carefully - so please tell me if you disagree with using here).

Other interesting thing about information awareness is that when you actually take attention to it - you know the critical points of your code, where possible bugs can occur. Bugs - if you think about it - happen when the expected information differs from the reality. It's like playing chess. You try to guess all possible future states and make a decision on that. If you fail to do so - you'll more likely fail on the game.

Design patterns - imho - the queens are information theory. They explain gorgeous abstraction layers of information flows. To be honest - I don't know all of them by heart - I believe after some practice you learn the essence of it and can use without labeling them. Information states can be imagined as a directed graphs. Design patterns are the rules to make those directions. When you chose a pattern it's potentially the lowest energy level and maximum throughput option that serves the best.

And in general thinking in graphs I found a really good practice. It's an awesome representation and visualize nicely how to structure your system. What are the independent islands, connections, relations, cardinalities, etc.

I love information. It's fucking amazing. There was a post on StackOverflow about complex computation and energy. Information is not just a theoretical thing. It's everything. Information is the base of everything. You have it in your DNA. Environment have it in states in atomic level. Information and its state that defines everything that will happen and happened already. And information as a state needs energy to change. Energy is measurable. So the guy on that SO post calculated how much energy that certain algorithm would require. It's amazing. You realize you need energy to play games or watch tv or drive your car or run. But it's all information in the background. And think about evolution and future. How much energy the next technological breakthrough requires? How can we make it more effective by grades? Otherwise we will use all our resources up soon.
So please, don't handle your variables loosely ;)

---

If you know about any cool blog or article about that topic, please, share. I'm really interested to read them and I'm sure other visitors too.

Peter

4 comments:

  1. I really liked reading "The Information: A History, A Theory, A Flood" by James Gleick, if you haven't yet read it, check it out.
    http://www.amazon.com/Information-History-Theory-Flood/dp/1400096235

    ReplyDelete
  2. Hi Kristof :) Thank you, just got it. I'll write a review when read it. Thank you so much!

    ReplyDelete
  3. I really enjoyed your idea: "information is everything". My informatics/physics teacher (who is really interested in theoretical physics, and not-so-little-bit Taoist) said once that God may be thought as an infinite set of bits, ie. information beyond anything and everything. Holding a degree as a religious teacher myself, but I couldn't say he was wrong. When we continued that conversation, we had to agree that we, humans are just like any other beings: cars, cats, galaxies, etc.: smaller or bigger sets of bits. But neither of us: neither humans, cars, cats, nor even galaxies may know (ie.: contain) everything about everything, since it's simply impossible because of basic rules of information theory. As we're developing our precious selves, we're making ourselves bigger and fatter sets of bits – but none of us, human or any other beings besides God may became God, ever, since He's the only one who is made up by every bits of all available information about everything, because nobody and nothing else is capable of being so.

    PS. I'm really surprised how a guy just like me (a coder/developer) may have such great range of vision, even about topics that are only semi-related his daily work. Keep up this good habit! :)

    ReplyDelete
  4. Hi Laszlo, thank you so much :) I wouldn't consider the idea of God as part of the information world - not as a contradict but mostly because it's more personal and depending on the definition. However what you've said about the complexity and self-awareness is really interesting. I kinda agree with it and it's cool to realize that a close system - like us, even if it's growing - cannot overgrow itself :) Which indeed is sortof by definition. If I want to go crazy I would even ask - what it means to know? But that I'm afraid is rather a theoretical question. Unless it can be defined in any quantitative way. Oh man, it was a trap :D
    Thanks Laszlo!

    ReplyDelete