The Unfamiliar World - Predictions for the Future
On Predicting the Exponential Future
Heuristics for Predicting the Future
There is a heuristic that indicates that the best prediction of the future durability of anything is equal to its historical duration. For example, the best prediction for how long Chopin will be popular in the future (at least among a certain group) is precisely equal to how long Chopin has already existed. Certain classics have a better probability of future longevity than current pop. This makes sense in a non-exponential world that largely operates on concepts of survivor-ship bias. Naturally there are exceptions to this heuristic because new things do, in fact, come into existence that have staying power, but as a general heuristic of the likelihood of staying power it is useful.
We are currently, however, in an exponential world -- a world that most people have neither experienced in their lifetime nor are well adapted to understand biologically. We have little intuition for exponential growth. This is a new experience.
To be clear, exponential worlds are not worlds of butterflies flapping wings (e.g. chaos theory) -- these are more worlds of unpredictability
Exponential worlds are worlds of first principles and networks effects. They are worlds of low to zero marginal costs of information, energy, materials, etc. They are typically worlds based information and its transfer but increasingly are moving into the physical domain.
Our current exponential world has been sparked by the "information age". Prior to this, most forces that moved society were based on physical resources and those resources were derived from both natural sources and what could be created through relatively local human innovation. As an example, one pinnacle of these forces is exemplified in the aggregated accomplishment of going to the moon -- a massive and focused aggregation of capital and human potential -- and the early part of an exponential curve that began at Kitty Hawk.
This transition has been accelerating at a pace that is difficult to manage. In the 20th century, for example, Douglass MacArthur, a distinguished US general, operated under circumstances where the primary mode of transportation was a horse to overseeing a nuclear arsenal that could destroy the world many times over. This, in the span of a single career.
The fundamental features of exponential worlds are two fold: (1) the slope of the technology curve as a percentage of a human life span and (2) the surface area of contact between the general population and advancing technologies. Going from Kitty Hawk to the moon was a significant slope (relatively speaking), but the contact with the general population was not very high at least for some time – air travel took several decades to become mainstream even after we visited the moon.
The exponential world we are in now is accelerating on both fronts. Advances in machine learning and artificial intelligence are compounding exponentially and their surface area of contact with the general population is growing at the same pace. And this will happen across all aspects of life, whether we know it or not. It may be invisible to most, but will be embedded in everything.
Moreover, the world of information will increasingly accelerate the world of atoms. This will be seen not just in creative endeavors, such as with generative AI, but increasingly with the intersection of robotics and manufacturing based applications, the effects will be compounded. We are already seeing AI based algorithms training novel AI based algorithms.
Operating Frequencies for the Future
Exponential worlds create challenges for predicting the future. Even just 3 years ago, the protein folding problem was considered at least a 50 year grand challenge in biology. We considered content generation and the creative arts to still be largely the bastion of the human experience. We have seen super exponential declines in genetic sequencing costs arriving shortly at the $100 genome, reusable rockets going to space at rapidly decreasing costs, patients among us with precisely genetically engineered immune cells to fight cancer, autonomous vehicles, and instant access to the world’s information in our pockets. And underneath it all, there are currents that are nearly invisible that are creating new forces that define the future. For example, one key strategy for most technology companies is to move “down the stack” – in a world where information is the dominant currency, entities that control the communication infrastructure become the gatekeepers. This is not incidentally why companies (such as Meta) that did not develop distributed hardware or operating system infrastructure (both of which are lower frequency efforts), will struggle to control their own destiny. The same logic applies to AI companies that do not control the production of their own data. While information is virtual and at the center of exponential growth, its specific acquisition remains bound to the physical world.
In the past, the frequencies of change were not as much of a concern for individuals because they were on the order of (or longer than) the lifespan of any one person. As it relates to most of human history, seasonal transitions may have been the dominant measure of change but were generally predictable and cyclical, global events like wars and politics were longer but less frequent, and information ebbed and flowed at a rate that was largely inconsequential for daily life
We live in a fundamentally different world today. The average person today will likely consume more novel information in a single day, than a person a century ago would consume in a year (if not longer). The heuristics of longevity still apply (i.e. that the best prediction of future longevity is historical precedent), we just need to apply them to new categories of concepts - and these are largely based on first principles.
The exercise of doing this is a distillation of any idea or concept into its constituent parts. Take for example, a song. A song is composed of notes, lyrics, rhythm, beats, and other compositional elements. It is also composed of (or influences) emotions and sentiments. It may be topical, for example about a specific event, or it may tell a story.
A distillation of a song would then consider the different frequencies that these features operate on. For example, specific sounds may be correlated to the rate of change of the use of different instruments or particular cultural trends which may change frequently (especially with digital music and even more with generative AI). However, the underlying attractiveness of music is often tied closely with aspects of human biology, such as our brain’s resonance with certain frequency patterns or rhythms. These may change on a much slower frequency than the availability of specific sounds. Similarly, the emotional or sensory aspects of music that are tied to human experience, such as love, anger, sadness, or otherwise, are low frequency element of any creative art. As such we may observe music trends to change over time but the duration of their underlying features changes at their respective frequencies. We can image that love songs and perfect fifth intervals will still exist in the future, but any given instrument or story line may not. Pop songs are a dime a dozen, but the underlying 2-5-1 chord progression has staying power.
This frequency based approach is likely the only heuristic that is suitable for exponential worlds – and perhaps even more specific, its first principles reduction becomes the dominant feature for predicting most events that are on the order of one’s lifetime.
In short, it means that in an exponential world, the dominant prediction factors become the most basic. For example, we generally cannot predict the political or cultural news even a day, month or year in the future, but we can predict that human psychology is likely to persist with in-group/out-group thinking over long periods of time. We can not change our biology at a high frequency (at least not yet).
Yet, this concept of the future, or the rate of change in time comes in direct conflict with our own biological evolution. Humans are notoriously poor long term planners and when confronted with so much high frequency signal (e.g. your daily news feed), it can be very difficult to see the signal for the noise. This becomes particularly problematic when the magnitude of even the most profound low frequency signals can be world altering. This is precisely what we observed with the “fake news” challenges on social media, particularly among demographic groups that were not attuned to the frequency of the underlying shifts in content generation, micro-targeting, virality, and so forth and misunderstood the context in which they were receiving information, considering it as equal in quality to information they had received in the past. This is a fundamental mismatch caused by an increasing slope of technology change vs human experience (e.g. changes in the biology of psychology operate at a much lower frequency that even profound fundamental changes in technology) — which we are bound to encounter even more severely in the future of an exponential world. Do we know what ChatGPT will really mean?
A frequency based heuristic also applies to technology itself where it may in fact be even more critical because investments in new technology largely operate on relatively long time scales. As an exponential world is diminishing even the low frequency time scales of major developments, the high frequency of technology (for example, the rate of new publications) can be overwhelming. Making good predictions about the future of technology, despite the technologies becoming increasingly sophisticated, then becomes an exercise in simplification. Oddly, this is also almost the exact opposite of what is rewarded in academic settings where incrementalism is often the norm. That may be, in part, why recent reports have indicated that the frequency of high impact publications has been declining over the past several decades.
To be tangible about the frequency of technology, below is an example from the biological sciences:
Low frequency – physics/chemistry/AI advances in core macromolecular and cellular read/write technologies and discoveries.
Example: underlying mechanisms of immunotherapy, precision eukaryotic genome engineering
Mid frequency – most instrumentation improvements for research, therapeutics or diagnostics. Some new therapeutics or therapeutic modalities.
Example: CAR-T cells as a category, long read sequencing tools (excepting perhaps nanopores)
High frequency – The vast majority of new publications.
Example: Multi-functional iterations of CAR-T cells, bi-specific antibodies, etc…
Examples from the AI/ML space:
Low frequency – physical computational infrastructure
Examples: GPUs, quantum computers, and associated core infrastructure
Mid frequency – specific machine learning architectures
Examples: transformers, CNNs, RNNs, GNNs, LSTMs, etc.
High frequency – fine tuned LLMs for specific applications
Examples: many
Revisiting Future Prediction
Each frequency has its time and place in the world, however, filtering noise at higher frequencies becomes increasingly difficult. In an exponential world, what might be considered low frequency today will increasingly be accelerated. At the extreme, some have described this as the “singularity” which is not specifically the conclusion of this article. As a brief contrast to the exponential runaway perspective, it has been countered that exponential progress creates exponential friction. This is likely the case within any specific technology domain but as we have seen, domains are increasingly breached. We may, at some point, encounter some fundamental limit at the edges of information theory, entropy or some other foundation truth, but we are not there yet. For the foreseeable future, my primary prediction is that the process of making accurate predictions over shorter and shorter time scales in an increasingly complex world will, paradoxically, require increasingly simplified frameworks. We won’t know the details, but we’ll have a good sense of direction.
Perhaps ironically, my framework for predicting the future is to be as simple as possible.
Image credit: https://sums.org.uk/app/uploads/2020/10/Crystal-Ball-scaled.jpg