Loosely inspired by a submission to a hackathon on Autostructures, which is about radical transformations from even mildly intelligent/agentic AI.

Previously, I've briefly alluded to 6 infrastructural pillars for an era where "things that don't scale" start to scale. This post is mainly aimed at conveying an idea of the 1st pillar ("live interfaces"). The hope from interface design is to ease transition into infrastructure for generalizing mathematical or technical insights that haven't been generalizable so far. For example, alignment concepts like "scheming" or "power-seeking" don't seem to have one formalizable abstraction. This is the business of "live theory". 

Note: The links in this post lead to specific sections in previous posts in this sequence, which might look repetitive if you only hover over them rather than click-through.

You can jump to the abstract/TL;DR section directly if you wish; I wouldn't recommend it.

Subscribe here to stay updated, see footnote if you'd like to support.^[1]

Believe in Thursday

Time structures are pretty centralized right now.

Do you know what a "Thurs" is? Would you collect solar days together into 7-length segments? Why is some ancient drama between Julius and Augustus deciding the day you go out and have fun wearing spooky costumes? 

You could imagine throwing it all away and redoing it in a way that makes sense to you.

Or better yet, you can participate in temporal structures that reflect the needs and aesthetics of your local community. Maybe you would divide the day into powers of two and label them. Perhaps y'all like to orient to the lunar calendar, or to your menstrual cycle. You might find seasons to be only vaguely relevant if you're in the Maldives or elsewhere near the equator, and might orient via tides and monsoon or the migration patterns of whales instead of sections of the solar year.

You could do some of those by picking a preferred grain of abstraction on time "as it is" (i.e. as standardized). You could do that, just redo some grids creatively over this shared structure to kinda make it yours — notice, even the "28 hour day" above, creative though it is, fits within the 7-day pattern. That's a parametric, formulaic flexibility, which is nice. But it would be really great if each community got to co-create the underlying view of (temporal) reality... without disconnecting from others.

Clearly, this is a fantasy. To feed your kids, you need to believe in a Thursday. You have your weekly standup on Thursdays, at the eleventh hour of the ante-meridian. If we want to coordinate, we need to agree on a protocol. You need to catch your flight, and the pilot needs to show up on time. We need shared structure, shared reality, universal languages. Right?

Get Lucky

The punchline, of course, is that the formulaic flexibility above is the old way to scale: scaling by replicating fixed structure. When intelligence is cheap, however, you don't need static, shared structure. Ubiquitous, attentive infrastructure can create peer-to-peer structure on demand, without having to have things in common. 

Interoperable temporal meanings (as above) are really an interface design problem, and we can ask: what might peer-to-peer interfaces look like, a little bit ahead in the future?^[2]

Let's take something simple: a postformal^[3], "wish-based" interface to replace Calendly.

Instead of marking lines on a shared spatialization of time, your community can live by your local temporal rituals. You might use the rings of trees to commemorate new beginnings or births of children. Perhaps you will have a season of meeting others that lasts several weeks, based on your bipolar rhythms.

This, then, is your new Calendly: you broadcast "prayers" to meet others during desired times. Ambient intelligent infrastructure (ie. AI) can help you identify, broadcast and match this prayer with mine. I've set my prayer to have important meetings before meals. We get fluidly matched, and are led to find each other, physically or digitally. All without needing some shared foundation of time coordinates or formal correspondence between our temporal meanings. The correspondence is creative (mildly so), rather than prefigured, achieved by mildly intelligent models with extremely low latency and cost. As fast and straightforward as cloud sync today (at least, from the user's perspective).

The flexibility of prayers allows for you to run into me — a synchronicity!^[4] — during your meeting season. You (and I) get more lucky in the world, rather than getting more controlling-power over the world.^[5]

Sometimes I talk about the reality from the POV of tissues in your body. The collection of tiny animals that you are (i.e., your cells) are going about their ordinary existence. If they take damage, it's as if the normal flow of things get luckier. White blood cells happen to arrive just as they are needed. External antibiotics start to arrive too, and help things heal. If those fail, paramedics start coming in! It's all a bit "magical" for the cell, even though nothing actually spooky is going on. It's simply the power of friendship!

You can still have precision in this "increase luck, not control" way of doing things. Your prayer might specify that. "Only take dance invitations when your heart-rate has remained under 90 bpm between two cold showers." Control and precision might seem hard to decouple. But so do scalability and structure.^[6]

AI, if done right, can help with more inter-personalization, not hyper-personalization^[7]. Far from hallucination and spam, it can lead to radical unconforming into our deep, embodied truths, rather than conforming to pressures from epistemic bureaucracy. Why should our cycles and seasons of living and meeting be filtered through rigid structure that is numb to our needs yet infiltrates the ground of our meanings?^[8]

TL;DR

TL;DR. When actuation is cheap and fast (thanks to mildly intelligent AI), actuation takes a backseat to potentiation. "Potentials" or "telos" (eg. prompts, data) become objects in their own right—  more reified, meaningful. This gives rise to an "economy of exchange of potentials" among us living beings. The primary concern in the middle period is designing excellent infrastructure for such exchange. Doing this well can lead to alignment-relevant structures and artefacts (i.e., tools, theories, policies) that are less rigid, less formulaic, more pluralistic, more decentralized and more powerful.

P2P meaning

We looked at overhauling temporal meaning-making above. The second, more grounded example I'm going to use to illustrate this decentralized design philosophy (termed "P2P meaning") is websites.

You're reading this text on LessWrong. It is literally a duplication of the source files on your client machine, in your browser. At best, you might have switched to dark theme, or added a plugin, or switched it to a dyslexia-compatible font. Maybe you're using GreaterWrong, and that lets you do some more tweaking.

The culture of distribution of information via the internet is pretty centralized, and cookie-cutter, by default. You could, in principle, write massive scripts that completely change your experience. But have you? Would you, even, knowing that it will likely break the fragile interdependencies of layouts most webpages have and require maintenance?

The real problem with all of these customizations, is that they have to be formulaic. If you want a script to tailormake your experience across the web, you need to distill the core you want, so you can craft the relevant programmatic handles that work for each website. Greaterwrong is a whole project, for one website. That doesn't scale.

You're well cued up now to suggest "LLMs can do this personalization at scale!" 

Yes, that's not wrong. Indeed, the refrain in this sequence is leveraging the ability to do things that don't scale, at scale. You don't have to find and capture the universal programmatic patterns that will transform the website, $\forall$ websites. What these new, context-sensitive, broadly generalizing dream-machines actually do (increasingly) well is find and respond to specific patterns with informal input. You can deploy the intelligence to do the customization with each specific website, rather than once-and-forall with the right abstraction. Distribution (of mild intelligence) first, production (of structure) second.

However, this post, like this sequence, is not just about the present or the near future. It's about... the slightly far-off future. It is not about an extension, or a product; it can't be limited to that. It's about the infrastructure that underlies widescale commerce:

The diagram above is from Stewart Brand's concept of pace layering: layers of unfolding civilization that move at different paces. Each layer is shaped by the slower, more significant layer underneath. Among other relationships:

Fast learns, slow remembers.  Fast proposes, slow disposes.  Fast is discontinuous, slow is continuous.  Fast and small instructs slow and big by accrued innovation and by occasional revolution.  Slow and big controls small and fast by constraint and constancy.  Fast gets all our attention, slow has all the power.

Compare the previous post, where I mentioned that the vision for live (theoretical) machinery is about

Lending spirit to an ecosystem [...] that can competently caretake the widescale-exchange of postformal, adaptive artefacts.

"Ecosystems" and "infrastructure" don't have to be boring, unevocative words. The slow layers can also be exciting... ironically, only after they become boring to the faster layers. 

A brief interlude on "exciting after boring", before meeting "postformal, adaptive" websites.

Interlude: Take it for granted

My favorite bit of any sci-fi or worldbuilding or absurdism, is how mundane the background layers are, for the characters immersed in them. 

Developing the fantasy makes it mundane and fantastic at the same time. Characters remain unimpressed with what they inhabit, engaged in dull, routine activities in a magical realm. Usually a Chosen One enters the magical world from a more ordinary world in order to give the reader/viewer some company in feeling the magicalness.

But for the inhabitants themselves, the dullness towards their surroundings is an unthinking trust in it all hanging together. They are merely living their reality, not self-consciously creating it. Simply luxuriating, in the autonomous humming of their world– which forms the slower, supportive pace layers that host their further drama on top. Only the most immersed, the most settled in, are blessed with this entitlement — like a child's entitlement to the support of her parents — to have any emotional response to their surroundings other than enchantment or bafflement at the sheer absurdity of it all.

Take the guy below:

Instead of tripping out with a "holy shit, the dog is now a kitchen flooding?!" and tableflipping the whole thing with "am I dreaming, wtf", the man offers his anger on top of it. By acting as if that were normal and merely responding to a specific aspect of it, he colludes with the dream-logic.

Pulling off such immersion isn't just about verisimilitude. It's also the seriousness, the TLC, in the act of generating many "theorems" nurtured from the "axioms" of the fantasy. Worldbuilding is not very different from the activity of mathematics.

What could it look like, if we weren't just gaping our mouths in either awe or terror at our absurd magical future, but living in it? So familiar with it that we start taking infrastructural intelligence for granted? Uncovering detail, ontological shifts, not just graphs and hype? If your mind is being blown everyday by Veo2 or whatever, for example, then you're not really thinking about what the future takes for granted.

Most people who do imagine the tailormade future, I find, look only at the consumer side. But I think more interesting things happen when producers and consumers both take available intelligence for granted.

Prompt-to-Prompt websites

Back to the "postformal, adaptive" website example. Instead of digital infrastructure based on efficient duplication^[9] (of centrally produced website code):

we can have infrastructure based on efficient mild-intelligence (attending to the local prompts on each end and actuating them):

This enables cheap "just-in-time" production that fluidly composes the wishes of the parties involved, for each matched pair. This is the "P2P website", with human inspiration in blue and machine assistance in green. 

When compilers and renderers (namely, browsers) are augmented with cheap, low-latency, relatively-reliable generative AI, webservers can host source prompts instead of code. The "website" is created from the preferences of the server combined with the client. 

The server's prompt could contain a mix of formal and informal information about database access and proposed layout, important information, caveats, terms.

The client's prompt could contain UI & thematic preferences plus a background of information diet to recontextualize content. The client could have saved something like "I like curvy designs. Also, keep it mostly minimal. Here is the source of some of the websites I've favorited, and my CV."

These can combine on-the-fly, creating a composition of this "postformal specification" as fast as opening a website today.

Not Human-AI interfaces; Human-Human interfaces

People tend to hear this or read the autostructures post, and get excited about Human-AI interfaces. That isn't totally disconnected from the tasks at hand, but I do want to emphasize that the AI is backgrounded; only facilitating connection between living beings. As I said there:

.Q. So AI is going to do the meaning-making for me?

No. This is neither about you, nor the AI.

Meaning happens in the middle, between us.  Existing machinery both supports (because of scale and speed) and hinders (because of homogenization and fixity) the dynamics of rich meaning.

Trying to zero in on the relationship between you and AI bots is missing the point. There are no endless spammy AI bots in this vision.  Just because it is cheap to generate AI content, doesn’t mean you should spam the user and abdicate responsibility for tasteful design.

This is about how you and I will relate to each other via AI as mostly-ignorable infrastructure.  Like the internet, which is rarely the topic of conversation even in your internet-mediated video calls, and only foregrounded when the internet doesn't work right for a few seconds.

So this is about matching your postformal prayers with mine, and fluidly creating just-in-time interfaces and structures that precisely suit our wishes.  This means interfaces do not have to work with tradeoffs that come with preformed/universal structures.  They are free to be specific to the heterogeneous context, thanks to attentive infrastructure. 

Yet, the interaction between humans and autostructuring technology is not a secondary concern within this design philosophy.^[10] Interface-wise, there is no "chatbot", nor are there "autonomous" agents.^[11] The "AI", whatever that word is stuffed with these days, shows up only in the background, helping locate and crystallize structure as needed. In the website example, it creates structure that works for each counterparty. And then all of the design work for interface-designers is in empowering users to be able to leave postformal^[3] "hints", that combine structured and unstructured input – where the blue meets the green, above. This "postformal" guidance is the meat of the interface design labor (and what we're building! Connect with me if you're interested.)

Tasteful use of intelligence is tailored to the specific use-case of even a single user or single producer-consumer pair. Not spammy and generic like a chatbot, not out-of-control. Not offloading design-decisions to the user under the guise of "ask the AI anything you want!" with boring anthropomorphization.

Per this design cosmology, interfaces that work well are those that make your process luckier with minimal intrusion –  full of helpful synchronicities at just the right time. The AI isn't doing everything for you as a user; you contribute to the arranged luck by offering your connection to meaningfulness.^[12] 

Scaling Inspiration

The main section header ("P2P meaning") has a few different clever expansions, one of them being "potential to potential":

A couple notes on "potential" in the diagram above:

1. "Potentiation" is about inspiration, not ideation.

   "Potentiation" is not "ideas". AI might well become great at generating ideas. This "potential" is inspiration; an aliveness that empowers and vitalizes everyone around. This is truer to the etymology of 'potent', not about information but about powerful meaningfulness. 

   (If you demand some definition of "meaning" right away, I'm using: that which moves the integrity of a given being.^[13])

   Without proper attunement to what surrounds them, disconnected ideas can gather only a distorted sort of momentum (trust and empowerment) from others. This is true even for ideas in your head vs other parts of your own body, the latter remaining uninspired to actuate them (sometimes called "akrasia"). 

   Silicon shoggoths are incomparably more disembedded from our living ecosystems of meaning than arising thoughts are from our bodies. They thus need a fair amount of meaning-scaffolding to be appropriately relevant to what our bodies are up to, rather than remain merely clever contrivances (namely: slop).^[14]

   What kind of object is a potential? What binds the various actuations of the potential? These are all good inquiries, but note that potentials don't have to be "captured" in formally articulated objects with formal connections and formal distinctions. Taking intelligence, mind, and agency seriously means finished formalisms are less necessary. Formal articulation is a product (or even, residue) of creative intelligence, and useful precisely when crystallized intelligence is sufficient to handle the complexities. But most meaningful activities require some degree of fluidic creativity.^[15] 

2. Potentials as first-class citizens.

   "Actuation" of potentials tends to be singular, in the current way of doing things. For example, a project or company will tend to have one website. 

   The live design philosophy, on the other hand, capitalizes on actuation being cheap and fast and distributed (i.e. everything that follows from taking intelligence at scale seriously). This invites meaning and legitimacy to "potentials" that can host a plurality of possibilities. This is also what "lending spirit" in the previous section alludes to: giving people permission and affordance to spend gainful employment working on prompts/prayers/potentials as commodities^[16] in themselves. These potential commodities would fit into just-in-time actuation infrastructure to create thriving commerce.

   In the P2P website case, the website-prompt (or whatever multimodal package the potent is stored as) can only plug into an intelligent actuation if infrastructure for it has been widely adopted. Like most complementary things, one's existence depends on the integrity of the other.

   I sometimes use the analogy of the silly little applets on, say, Symbian OS, in Nokia phones in the old days. It was cute, and a hassle, to get apps for those on your phone, from some half-shady website. The reliable infrastructure of app stores today, in contrast, means entire multi-billion dollar companies can be "just an app". 

   The point is: coordinating efforts on such commerce requires being able to take the infrastructure for granted!

    

3. "AI" actuation is backgrounded and guided by "post-formal wishes and prayers".

   One of the first-class potents is a prayer. I defined prayer previously as:

   "Prayer", not as in "pray to an AI god”, but "prayer" as in "send out a message to a fabric of intelligence without trying super hard to control or delineate its workings in detail, in a way that is honest to you rather than a message controlled/built for someone or something else."

   With prayers, you can spy the "design for luck, not control" theme again. You can bundle together formal and informal content that isn't meant to be the last word, but only an inspiration or suggestion for what comes next in dance with another being.

   Prayers are, however, only one among myriad ways of jamming with each other. For example, I use "wishes" for guidance that is be slightly more defined, more targeted. But the terms are less relevant than the spectrum of possibilities. Skillful use of solidity, initiative, anchoring, force, direction, intention, clarity, leadership, precision is part of designing with tasteful use of intelligence. We don't want everything to be hyperattentive and pluriform. Sometimes you want to give exact specifications, without over-contextualization. Many designs will require some digital material to stay solid. The various ways are that humans and other beings will conduct AI constructions is an exciting open question. 

   Lastly: the diagram above might make it seem like your input happens pre-actuation. It never works like that, of course. Refinement and tastefulness are inspiration in dialogue with the actuation. A slightly less but still simplified expression: assuming AI constructs the 1-90 very quickly, you conduct not just the 0 to 1, but also the 90 to 100.

    

4. Inspiration can travel not just across peers but also across yourself in time.

   P2P is obviously peer to peer (the website example is a case of peer to peer production) but also person-moment to person-moment. That is, connections between the multiple yous across time. This is where the operations of storage and retrieval, even for yourself, become more prayer-like. 

   For example, instead of having new year's resolutions in the form of plans, you might try sending prayers to your future self. If you often feel coerced by binding structures from your past self, you might especially like to try this more gentle and trustful way.

   Another example is live decision theory. Instead of being restricted to an algorithmic ontology for your policy-at-the-beginning-of-time as in the usual articulation of FDT/UDT, you can send and receive prayers across your counterfactual selves. This is also my sense of the direction that Scott Garrabrant has been moving in (personal communication).

Anyway, digital interfaces are cool and important, but they're not the only abstractions we're tackling.

P2P math

I recently sent someone who knew very little math but was beginning to get curious about calculus the famous 3b1b series on it.

I hadn't seen the calculus series myself, so I watched one with them. It is of course, very friendly to newcomers, with dynamic and evocative animations rather than dry notation.

I've joked before that learning math is like learning not just musical notation so you can play it yourself, but also receiving assembly instructions for the instrument that you have to build yourself before you hear anything at all. Most excellent pedagogy, like 3b1b, assembles the instrument for the consumer so you can at least be allured by the music. 

(Of course you still have to do the exercises. IIRC, Grant from 3b1b was a rare math youtuber who would also link to Brilliant and suggest you do exercises. That no one can do the exercises for you, that you can't just "download kung-fu" is the subject of several upcoming posts in connection to AI risk, BCI, the future of math.)

But at some point Grant has to "give up" in his otherwise original illustrations, and use the standard, non-illustrative notation, $dt$

Even if $dt$ is well integrated within you now, you might remember a time when it was totally foxing and either annoying or intriguing.

There are at least two layers of cognitive load to take on, for students of math. 

1. The first is the mathematical ideas themselves, which can be very counter-intuitive.
2. The second is the specific notational culture used to articulate the subject matter, that they must get used to. Students must, in their own mind, substitute definitions, intuitions and concepts for symbols that are yet to become evocative. Their symbol-reflexes are still young, so this requires some thoughtfulness.

Grant helps with the first well, making the derivative intuitive. But on the second, Grant seems to quickly give up. Why doesn't Grant continue to exploit the precious friendship that viewer has made with, for example, the car symbol, in connection with rates of change? He could then take on the labor of evocativeness for the viewer as well. It wouldn't be so hard to make some cute notation that would have more iconicity, rhythm, poetry in connection with the local motivation and "culture" of the video series. 

Grant doesn't start using $dt$ because it is so perfect; it's only because it's so in use. He wants it to be painless for you to learn, but also for you to be able to continue to learn. This means interoperation with the rest of media out there. He can't get everyone to use a notation created for his viewers, even if it were very clever. All he can do is pave the way for you to get to the established city center of calculus-as-practiced, with its standard notation. 

This compromise is not out of a poverty of creativity, but a restriction on creativity from a poverty of interoperability at scale. Reifying something (the "established" notation, in this case) mainly because it already has a lot of Steam is... well, perhaps necessary, where background infrastructure can't support pluralism (of notations, in this case). But if this felt "necessity" doesn't come with a missing mood, any progress towards non-rivalrous steam-management infrastructure will feel like regress. Progress can come from P2P invitations of reification, where there is no centralized establishment over which you have to do some kind of steam-hacking or other compromise. You wouldn't have to be "Big", for your efforts to be reified/rise to salience for others. If instead we continue justifying "no, this custom notation is not good notation, because it doesn't already have enough social power", then we're locking in to a centralized store.^[17]

Grant is being kind to your further growth when he introduces the standard notation. But he's operating in his era. Just-in-time notation can leverage creative interoperability at scale, instead of the one true king notation that must be duplicated in each mind:

...with "lucky" bridges across these as needed, that can translate insight across notations. Not just formal correspondences, mind you — that's attention-scarcity thinking, where you must be finished, free of further attentional demands, as quickly as possible — but live correspondences. You don't need a fixed protocol of translation when adaptively maintained protocols are cheap and immediate. Whatever magic in humans allows us to create those protocols out of messy reality, that's what autostructuring does at scale, ongoingly.

Such interoperational infrastructure would permit Grant and his viewers to dedicate time to developing their own notational culture, without losing out on the many technical insights that have been reaped over the centuries, just because they're in a different "language". Better interpersonalization leads to better personalization.

Glimpses, of scaling that which doesn't scale

I quoted Nate Soares previously:

A problem isn't solved until it's solved automatically, without need for attention or willpower.

Imagine if someone was used to only finite state machines, and never conceived of or really internalized the idea of Turing machines. On their self-help blog, they might write

A problem wouldn't be solved until it's solved automatically, without need for memory or willpower.

For you, with your abundant GBs of RAM and secondary storage, this is silly. A memory-scarce mindset would just waste the resources you do actually have. 

Similarly, if personal attention (pillar 1; live interfaces) and interpersonal, interoperational attention (pillar 2; live bridges) were cheap, you don't need to get to the overly strict requirement of "without need for attention."

To get a glimpse of this, we can take something that can be done with personal attention but could not have been scaled until very recently.

One time, another math-curious friend was being a tease. I aimed to get her to meet the notion of  convexity. Most textbooks start with definitions, sometimes with a small paragraph of motivation. But definitions are prized answers, incredible compressions whose purpose is to shorten theorem-statements and their proofs while still being representative of the problem at hand. If the definition of convexity (namely, $\text{Convex}C:=\forall x,y\in C.\forall \lambda \in [0,1].\lambda x+(1-\lambda )y\in C$ ) is the answer, what is the question?

I gave her a Bongard problem like so:

which asks for an English language sentence distinguishing the objects on the left vs the right. And then we proceeded to refine her answers, with me offering Socratic questions that let her discover her own meanings but not get lost. This took a long while, spanning many analogies and travel across dimensions, and a lot of fun. I'd insert nothing of my own until she got too stuck or sure, at which point I'd ask a provocative question. If you have non-math friends with time and willingness, I'd highly recommend this activity.^[18]

Importantly, I asked her to come up with names for the various concepts she tried. This makes sure she's not dealing with the double challenge mentioned earlier. It let her pick names and symbols and conceptualizations that she finds evocative. If she picks "bouba" for convex and "kiki" for concave, then the rest of the lesson can proceed using her notation. Whatever language she uses, I adopt in dialogue with her, with sensitivity towards her directions.

This is not something you can do properly at scale, now. But if you're teaching one-on-one (or even, for a small group), you can solicit creative notation that's evocative for the student. This is not only more co-creative and engaging, it might lead to a genuine advance. There is also low-hanging design fruit everywhere. Have you noticed how the minus sign has horizontal symmetry, even though the operation isn't commutative? Strictly worse than infinitely many asymmetric possibilities, just stuck because of prevailing Steam.

But just-in-time notation is only an easy warm-up.

P2P math, not just P2P math notation

The proposal is not just to unlock teaching in ways that don't currently scale. Adaptive notation is a step towards deeper adaptive representation. Our sights are on entire mathematical abstractions, namely just-in-time formalisms:

The analogy for non-formulaic mathematics is fairly straightforward, but counterintuitive at first.

The above, I claim, is a more appropriate use of the term "autoformalization", rather than "turning natural language mathematics into proof assistant code", which is taking something already formal and making it even more formal. 

To say more:

"Autoformalization" — as the word is currently used —  usually refers to automatic conversion of mathematical definitions, proofs, and theorems written in natural language to formal proofs in the language of a proof assistant. LLMs and other machine learning models are also used to assist mathematicians in such proofs.

While proving theorems is an essential aspect of mathematics and formal systems, theorems are perhaps better seen as fruit of deeper labor: good definitions, which capture the phenomenon in question and simplify the network of theorems and shorten proofs. Or even further upstream from formal definitions: the technical insights that experts synthesize, often articulable in more than one formal way. 

This is the natural next step for conventional "autoformalization". Alex Altair has this trajectory of math skill, is his post, which AI should catch up to quickly:

AI-assisted definitions (created from human discussions), not just AI-assisted proofs, which seems overlooked even by Terry Tao, who is very excited by math copilots but only goes so far as AI-assisted lemmas, last I checked.

Post-rigor as first-class

In his blog post There’s more to mathematics than rigour and proofs, the same Terry Tao notes that mathematical skill does not culminate in formalisms, but extends beyond into a “post-rigorous stage” characterized by intuitive manipulation that can be “converted into a rigorous argument whenever required.”  Mathematicians, he says, 

no longer need the formalism in order to perform high-level mathematical reasoning, and are actually proceeding largely through intuition, which is then translated (possibly incorrectly) into formal mathematical language. 

While this phenomenon is easy enough to experience first-hand within a subfield, it does not scale. Despite being truly mathematical activity (as Tao claims) and possessing real technical content, the “intuitive” nature of post-rigorous reasoning means it is not yet granted first-class citizenship as a mathematical language or object.

There can be advantages to heading in this direction. In their paper Inflation is Conflict, Guido Lorenzoni and Iván Werning explore a technical intuition that the proximal cause of inflation is conflict over relative prices. Importantly, instead of presenting one unified model, they perform involved computations in many different formal models, connected by the same insight. They conclude with the remark (emphasis mine):

In our view, traditional ideas and models of inflation have been very useful, but are either incomplete about the mechanism or unnecessarily special. The broad phenomena of inflation deserves a wider and more adaptable framework, much in the same way as growth accounting is useful and transcends particular models of growth. The conflict view offers exactly this, a framework and concept that sits on top of most models. Specific fully specified models can provide different stories for the root causes, as opposed to proximate causes, of inflation.

Much like the concept of inflation, one should expect many technical concepts to resist a single formalization– or at least, benefit from a plurality of formalizations. In particular, fundamental notions in AI safety that have resisted definitional consensus — such as ‘deception’, ‘harm’, ‘power-seeking’, ‘autonomy’ — could all similarly “deserve a wider and more adaptable framework” that “sits on top of” (and are translated into) specific formal definitions.

Abstraction Anarchy

What does it look like, to have not only abstraction hierarchies, but fluid abstraction anarchies? The next post will dive more into P2P math (with sketches for alignment research in particular), interface-design FOOM, and the unreasonable practicality of agent foundations research in the potential economy.

Concretely

If you've read all the way to the end of this post, you might be interested in getting in early on concrete design experiments already underway as part of AI safety camp, or building your own. Get in touch!

1. ^{^}

   ETH address here, or contact me for larger donations.

2. ^{^}

   Especially, in a world with only mildly creative but still transformational AI?

3. ^{^}

   "Postformal" is a theme running through this sequence. This isn't anti-formal. An accessible glimpse, from Terry Tao, is here: There’s more to mathematics than rigour and proofs.

   A fun exploration of formal hyperlocalization of timezones is here: So You Want Continuous Time Zones. As the author notes, it is pretty terrible. They conclude:

   The sad, ultimate truth of modern timekeeping is this: it's not perfect, but it doesn't honestly get a whole lot better.

   It really doesn't... if we continue to conflate formalizable and interoperable/scalable anyway.

4. ^{^}

   If it isn't obvious, the title of the post is an allusion to Clarke's "sufficiently advanced technology is indistinguishable from magic", with magical-seeming "synchronicities" becoming a normal part of infrastructure.

   Carl Jung introduced "synchronicity" in his book Synchronicity: An Acausal Connecting Principle. Although the book is pretty woo, acausality as a philosophy is not considered insane in this community. With intelligent infrastructure, this (or similar) language might also gain pragmatic use without being woo in the slightest.

   Interestingly, he also talks about the importance of the specifics, things that escape pattern-finding.

   The experimental method of inquiry aims at establishing regular events which can be repeated. Consequently, unique or rare events are ruled out of account.

   Compare Hayek, in The Use of Knowledge in Society:

   Today it is almost heresy to suggest that scientific knowledge is not the sum of all knowledge. But a little reflection will show that there is beyond question a body of very important but unorganized knowledge which cannot possibly be called scientific in the sense of knowledge of general rules: the knowledge of the particular circumstances of time and place. 

   Now, I don't think these need to be quite so dichotomous. The dependence on regular/universal patterns is only one way in which to generalize. It is the natural default, when intelligence is still expensive. Live Theory intends to change that.

5. ^{^}

   For critical logistical and security matters, control systems can be pretty important. But just because it is necessary, and critical, doesn't mean it should occupy the entirety of our salience/attention! It is critical for someone to take care of various internet lines for you to receive this message, but it doesn't mean all our posts on the internet should be about the internet.

6. ^{^}

   No, I can't TL;DR this, sorry. If you're curious, try reading the linked section. Happy to dialogue after!

7. ^{^}

   Some of the scariest s-risks are extreme forms of hyperpersonalization, IMO. A lot of experience with digital interfaces is frustrating and annoying atm, but that isn't like being trapped in hellish abusive relationships that make you think they're good for you.

8. ^{^}

   Honestly, you could do a formal mini-version of this already that could replace calling. When you call someone, your screen goes "ringing..." which is really a n unnecessary skeuomorphism. Without any "fluidic" "AI', a different design: the call button is just a formal prayer to call.

9. ^{^}

   I like David Deustch's definition of information as variables that can be permutated and duplicated. Below we move into more fluidic reworkings of meaning than setting and copying information variables.

   (He also defined knowledge as "information with causal power." In that case, meaning should be "information with telic power" :P)

10. ^{^}

    "Design cosmology" or "design mythology" is probably more appropriate than philosophy, with "myth" in the sense of binding metanarratives across society. 

11. ^{^}

    You can have a small chatbot if you still need one. But I bet for almost anything that you want to do (apart from "simulate chat"), there is a better fluid interface.

12. ^{^}

    Compare also, automation vs integration, from a section in a previous post.

13. ^{^}

    Kind of like you might describe a film as "moving", or might be "moved to tears" or "moved to speak" or the movement of falling in love. 

    This isn't "meaning = feelings" either; if you have strong feelings but it doesn't move your "executional" integrity, that's a particular kind of fragmenting of meaning.

    More on "integrity" later in the sequence.

14. ^{^}

    Aside: you'd need meaning-closeness to even come into conflict, conceivably. Taking that seriously, in the context of autonomy risks, is the subject of a different, upcoming post. It argues that "timelines" to self-motivated humanity-outwitting AI might be way longer, if you pay attention to the fine-grained structure of meaningfulness for the AI.

15. ^{^}

    Saying "it's parameters all the way down" or "Church-Turing thesis" would be making mistakes similar to "AI is just software". The CT thesis has little to say about mattering, salience, relevance, which is what makes something potent.

16. ^{^}

    "Commodification" is a particular kind of integrity of meaning, for the economy. I've used that to make it easier to convey. But I mean exchange of "integrities" in any meaning-paradigm.

17. ^{^}

    This is a pillar in itself ("live storage", where the operations of storage and retrieval are not fixed but sensitive to context), and out of scope for this post.

18. ^{^}

    I've tried, and failed, to prompt a model to do this without any hint of spoiler. But I haven't tried with the recent ones.

19. ^{^}

    It's important to note again that live decision theory is distinct from FDT in that it does not require a common platonic abstraction to connect selves. 

20. ^{^}

    Abram Demski will soon write a lot more on this. Especially after I tell him about this footnote. He has written one of them, on Circular Reasoning. A "podcast" of discussion and why it might be important to each of us in a more live format: The Ponzi Schemes of Meaning.