Feedback Loops

The idea of this blog is learning by writing. Essentially, my first attempt at applying the Feynman method. Beyond that, this is also my attempt at processing an Elon Musk quote I heard on a podcast a few years ago, which states that understanding requires constructing a knowledge tree so that new concepts can be connected back to a strong trunk of existing understanding.

“It is important to view knowledge as sort of a semantic tree — make sure you understand the fundamental principles, i.e. the trunk and big branches, before you get into the leaves/ details or there is nothing for them to hang on to.”
― Elon Musk

With this blog I am trying to figure out how exactly I could put this into practice. The big question being, of course, how do I form such a trunk. So my attempt is to identify an initial area of study and trace back concepts until I cannot find a more foundational concept. So I chose LLMs as my first area of interest and tried to reverse-engineer the semantic tree. That led me back through transformers, deep learning, machine learning, gradient descent, all the way down to what I would consider the first foundational concept to form my trunk - feedback loops.

Feedback loops occur when the output of a process is looped back as an input that becomes a cause of its future behavior.

While that may sound abstract, feedback loops govern a surprisingly wide range of phenomena in the world. They generally come in two main flavors:

Negative Feedback Loops: These are arguably the more commonly discussed. In this scenario, the behavior of the system is pushed against a deviation from some sort of reference value. A frequent example is a thermostat. It controls the temperature of a room by measuring the current temperature and comparing that against a defined target value. If the room is above target, the thermostat triggers the air conditioning, causing the room temperature to fall in the direction of the target. If it is below target, it turns off the A/C, causing the air to become warmer again, moving towards the target. The resulting behavior is that the room settles somewhere around the target value.

Positive Feedback Loops: In this case, the influence on the behavior of the process is the opposite. Instead of settling, the behavior of a process is amplified to an extreme. An example of this might be social media recommendation engines. If I see an interesting post about excessively configuring my own nvim setup, I interact with it and the platform will know to recommend more similar content to me. I continue to view the recommended content and before I know it, I have been pushed deep into an insane vim rabbit hole.

Both of these very different scenarios share the same mechanism of an output being fed back into the process to influence its future behavior. The distinction between positive and negative feedback loops defines whether the process is amplified or corrected towards some sort of target.

Since they require less explicit design, positive feedback loops can sometimes be the result of an accident, simply because they do not require the additional step of defining a reference value.

A common example of positive feedback loops easily running out of control is predictive policing. If a police department chooses to use software to schedule patrol car presence based on crime statistics, they may quickly find themselves in such a loop. The model determines that a specific neighborhood has higher crime rates than others; the police department uses this data to dispatch more patrol cars to that neighborhood; the higher presence of police makes it more likely for crimes to be discovered; this raises the crime statistics for the neighborhood further; this increases the confidence in the model’s high-crime prediction for this neighborhood; this increases police presence further; and so on until something breaks the cycle.

I found another example of an inadvertent positive loop in the great Farnam Street article on feedback loops. When safer helmets were introduced in American Football in the hopes of lowering head injuries, the outcome was actually the opposite. Safer helmets caused players to feel more protected; this led them to take part in riskier confrontations; this increased the frequency of injuries; prompting helmets to be further improved with harder shells and more padding; causing players to feel more protected; and so on.

How is this relevant for machine learning?

In the beginning of this post I said I chose feedback loops as my foundation for the semantic tree, hopefully leading me to understanding frontier AI research concepts. Where am I getting that idea?

The core underlying mechanism in machine learning, gradient descent, is essentially nothing more than a negative feedback loop. The output of a model is measured against a reference value (labeled data), the deviation is determined (loss function), and the behavior of the model is pushed in the opposite direction of this deviation (updated weights).

We have our trunk. Now we will just have to see whether it will hold up the daunting semantic tree of deep learning.