#theorycrafting #situation #thinking All our endeavors are within context. One can not escape the fact that he is bound to a situation. His decisions will ultimately be tested against that background. Theorycrafting is the process of finding out what is optimal to achieve a given *effect* in particular situation. # Formulation and Simulation are Tools to Reason about Situations Coming up with efficient behavior can be done in two ways. (1) figure out what's going on and what to do about it. Or, (2) try and discover new efficient ways of doing things. In the first case, we formulate a theory of the situation. We establish the relationships between the agents that both make up and exist within the situation. Then, elaborate on the emerging dynamic from entertaining these constraints. We define a surface on which we can act, taking advantage of these constraints. In the second case, we have a rich context. Any attempt simplify its makeup would result into an inconsistent partial description. In such case, the exercise of our intuition within well crafted thought experiments yields a better result. Playout few scenarios to derive enough partial understanding to reverse engineer our answers. The very reason you want to do Theorycrafting is that you want to be able to find answers to "What to do in this situation?" on your own. Whether you use formulation or simulation, looking for what is optimal is the best way to efficiently deploy your assets. The understanding of what is optimal enable you to measure in which ways you and others might be inefficient at dealing with a situation. Moreover, this kind of work might lead you to explore alternative ways to produce similar effects. # Models are not the World The methods you use to answer questions will affect the answers you can find. Both formulations and simulations are modelling techniques. They depend greatly on our ability to both absorb and reflect your understanding of the situation. To some extent, all models are wrong, only some are useful. ![[Theorycrafting is Physics if we Knew Better 2024-03-25 17.32.56.excalidraw.png]] Thus, we try to use the methods and tools that are the least bad for our particular situation. After all, the goal of analysis is to provide insights that impact the decisions we make. Demanding more than that from a model is unfair, unless the extra comes in naturally. # Accuracy and Precision are not the Same #accuracy #precision Models are meant to model. They are by definition low-resolution representations of the world. The result of their predictions are determined by the amount of simplification and its affinity with the initial purpose of the model. The quality of a model is measured by (1) its **accuracy**, namely, how close are the model's predictions to the correct realistic answer. And, (2), its **precision**, which is a measure of how consistent are the predictions each time you run the same conditions. ![[Theorycrafting is Meta-Situated-Thinking 2024-03-26 03.25.55.excalidraw.png|center]] One can optimize for either and rarely for both at the same time. To model accurately and precisely the world, you will make a copy of it. That is simply too much to just try and answer few questions. If the best way to know the right answer is to spawn a copy of the world and let it run to the situation of interest. In that world, there would be the same person trying to do the same. And so on recursively, at no point he who seeks answer will find one, as he is reflexively waiting for himself at a deeper level. # A Question of Absolutes and Relatives When optimizing our model for precision, it will be possible to tell which parameter is responsible for which change. The ability to produce differentiable results is better suited to answer relative questions. Questions of the type, "Is THIS better than THAT?" seek to *characterize* the difference between THIS and THAT. Read this twice, slowly. Going for high-accuracy we prioritize having the right answers. What is the value of THIS? A question of absolute character is better answered in a framework capable of producing "the right" facts about THIS. # So Computations or Scenarios? A formulation is deterministic modeling that is aimed at generating answers through computation. It naturally optimizes for PRECISION. Thus formulation is naturally good at answering relative questions. If you have a single relative question then it is good to formulate a model. If you have a new question then you might need to reformulate. The amount of work with deterministic modelling scales linearly with the amount and nature of questions we want to answer at once. A simulation is probabilistic modeling that is aimed at statistically finding the answer by running scenarios. It optimizes for ACCURACY, which as you know makes it better at answering absolute questions. The only problem is that you have to account for all variable at all times, so effort-wise there is a lot of initial overhead. Once done, asking new questions rarely requires remodeling. ![[Theorycrafting is Physics if we Knew Better 2024-03-25 18.12.10.excalidraw.png|center]] Clearly the probabilistic approach has a faster truth convergence. While the deterministic approach requires more effort for every question we seek to answer. The depiction above reflects how modeling is inherently flawed. We either need infinite effort or the ability to account for every intricacy in the world we want to simulate. Both are unattainable. # Producing Right Answers is Hard To be right about things is more of a fantasy than something you can achieve. The only hope is that you are on the extreme of being "the least" wrong. To be right sometimes is less desirable in my opinion than to be consistently off the mark. Correcting the aim is better than just shooting and hoping it would hit. This logic translates to me rotating the previous quadrant as follows, ![[Theorycrafting is Physics if we Knew Better 2024-03-25 16.55.38.excalidraw.png|center]] It would make much more sense to optimize for precision in a way that you make it easier for you to answer relative questions then add second order accuracy to adjust and be able to answer absolute questions. ![[Theorycrafting is Physics if we Knew Better 2024-03-25 17.46.34.excalidraw.png|center]] # Your Analysis is as Good as your Ability to Implement it Modeling is not enough. We need to act within situations. Building theories that lead to no applicable conclusion is of no interest to any rational agent. The value of these constructions is mainly derived from the potency of their insights. These insights are derived from both the model and our understanding of the ways it is flawed within the situation of interest Mitigating the effect of these flaws in our reasonings leads to more actionable answers. --- This essay has been strongly inspired by the video essay of [Cautex on the Analytical Methodology: The Methods of the Madness](https://youtu.be/ImccFzDJfqA?list=PLhF6IUCVVVmcuiwn8bs7BWvQ4OJw3Uehd). I couldn't help but see a generalizable framework for thinking about situations. I appreciate greatly the inspiration.