Showing posts with label mathematics. Show all posts
Showing posts with label mathematics. Show all posts

Small steps add up

Mike's Notes

A great book about how to use a "systems approach" to grow a healthy, mentally strong attitude.

1 per cent better every day is also my gameplan for Pipi 9. It compounds.

Resources

References

  • Atomic Habits by James Clear. Penguin.
  • The Atomic Habits Workbook by James Clear, published by Penguin, $45.

Repository

  • Home > Ajabbi Research > Library > Subscriptions > 3-2-1 Newsletter
  • Home > Handbook > 

Last Updated

01/03/2026

Small steps add up

By: James Clear
Stuff: 12/01/2026

I’ve been writing at JamesClear.com about habits, decision making, and continuous improvement since 2012. I’m the author of the #1 New York Times bestseller, Atomic Habits, which has sold more than 25 million copies worldwide and has been translated into more than 60 languages. I'm also known for my popular 3-2-1 newsletter, which is sent out each week to more than 3 million subscribers.

At the start of a new year, it’s tempting to aim for big, dramatic changes that promise overnight transformation and shiny, instant results. But according to James Clear, the best-selling author of Atomic Habits, lasting success rarely arrives in a single heroic jolt. Instead, it’s built quietly, steadily, one small choice at a time.

In a new workbook, he explains the power of consistent improvements based around the idea that if you get just 1 per cent better each day, those little gains compound into remarkable results. Small habits are easy to overlook in the moment, but they’re the very building blocks of long-term change.

So, as we step into a fresh year and all the possibility it holds, this feels like the perfect reminder: you don’t need to overhaul your life to make progress.

You don’t need huge motivation or massive willpower. You just need to start small, stay consistent and trust that every tiny step is moving you somewhere bigger.

1 per cent better every day

The typical approach to self-improvement is to set a large goal, then try to take big leaps in order to accomplish it in as little time as possible. Too often, we convince ourselves that change is meaningful only if there is some large, visible outcome associated with it. Whether it is getting stronger, building a business, travelling the world or any number of goals, we put pressure on ourselves to make some earth-shattering improvement that will awe everyone around us.

While this may sound good in theory, it often ends in burnout, frustration and failure. And yet, while improving by just 1 per cent every day isn’t notable (and sometimes isn’t even noticeable), it can be just as meaningful, especially in the long run.

It is so easy to dismiss the value of making slightly better decisions on a daily basis. Sticking with the fundamentals is not impressive. Falling in love with boredom is not exciting. Getting 1 per cent better isn’t going to make headlines.

There is one thing about it though – it works.

In the beginning, there is basically no difference between making a choice that is 1 per cent better or not. (In other words, it won’t impact you very much today.) But as time goes on, these small improvements compound, and you suddenly find a very big gap between people who make slightly better decisions on a daily basis and those who don’t.

Here’s the punch line: if you get 1 per cent better each day for one year, you’ll end up 37 times better by the time you’re done. That’s probably a more massive result than you would ever expect, even from a one-time heroic leap, and yet it’s achievable through just one tiny change a day.

This is why small choices don’t make much of a difference at the time but add up over the long term.

[IMG]

Photo: Edited extract from The Atomic Habits Workbook by James Clear, published by Penguin, $45.

But here’s the thing: if positive compounding is true, then so is the inverse. If you get 1 per cent worse each day for one year, you’ll decline nearly down to zero. The lesson is that what starts as a small win, or a minor setback, grows into something much greater. This is why the first important concept when it comes to behaviour change is the key role of continuous self-improvement. Just one tiny shift can change everything.

If you want to predict where you’ll end up in life, all you have to do is follow the curve of tiny gains and losses and see how your daily choices will compound 10 or 20 years down the line.

This is why it doesn’t matter how successful or unsuccessful you are right now. What matters is whether your habits are putting you on the path toward success. Focus on your current trajectory, not your current results. It’s a much better indicator of where you’re headed.

So stop obsessing over the big and start focusing on the small – it’s the key to building the life you want. Success is the product of daily habits – not once-in-a-lifetime transformations.

Edited extract from The Atomic Habits Workbook by James Clear, published by Penguin, $45.

Biological Brains Inspire a New Building Block for Artificial Neural Networks

Mike's Notes

Back propagation is based on a flawed model of how the brain works. This model is based on a more current understanding of how the brain works.

I'm impressed by the work of the Flatiron Institute in New York. It would be great for Ajabbi Research to collaborate with.

Resources

References

  • A Logical Calculus of the ideas Imminent in Nervous Activity. By Warren McCulloch, Walter Pitts and Walter Pitts. 1943. University of Illinois at Chicago.
  • On Computable Numbers by Alan Turing. 1936. Proceedings of the London Mathematical Society.

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Announcements From the Simons Foundation
  • Home > Ajabbi Research > Library > Authors > Alan Turing
  • Home > Ajabbi Research > Library > Authors > John von Newmann
  • Home > Handbook > 

Last Updated

14/02/2026

Biological Brains Inspire a New Building Block for Artificial Neural Networks

By: 
Simons Foundation: 26/01/2026

.

While artificial intelligence systems have advanced tremendously in recent years, they still lag behind the performance of real brains in reliability and efficiency. A new type of computational unit developed at the Flatiron Institute could help close that gap.

New research is exploring how to improve neural networks using components more like those in real brains. Alex Eben Meyer for Simons Foundation

While artificial neural networks are revolutionizing technology and besting humans in tasks ranging from chess to protein folding, they still fall short of their biological counterparts in many key areas, particularly reliability and efficiency.

The solution to these shortcomings could be for AI to act more like a real brain. Computational neuroscientists at the Simons Foundation’s Flatiron Institute in New York City have drawn lessons from neurobiology to enhance artificial systems using a new type of computational component that is more akin to those found in real brains. The researchers presented their work at the annual conference of the Association for the Advancement of Artificial Intelligence (AAAI) in Singapore on January 23.

“Artificial intelligence systems like ChatGPT — amazing as they are — are, in several respects, inferior to the human brain,” says Dmitri “Mitya” Chklovskii, a group leader in the Center for Computational Neuroscience (CCN) at the Flatiron Institute. “They’re very energy- and data-hungry. They hallucinate, and they can’t do simple things that we take for granted, like reasoning or planning,” he says. Each of these individual issues may trace back to one larger problem, he says: The foundations of these systems differ significantly from “the foundations on which the brain is built.”

The current building blocks of artificial neural networks are deeply rooted in a previous era. During that time, “the people who wanted to understand how the brain works and the people who wanted to build artificial brains or artificial intelligence were either the same people or close colleagues and collaborators,” Chklovskii says. “Then, sometime in the ’60s and ’70s, those two fields divorced and basically became fields of their own,” he says. That divergence has also led to artificial networks that are based on an outdated understanding of how biological brains function.

In the new work, Chklovskii and his colleagues revisit the fundamentals of artificial neural network architecture. For more than 10 years, Chklovskii had been on a quest for an alternative to the decades-old neural network building blocks used in machine learning. Through years of research, learning from real animal brains and innovation, Chklovskii and his team cracked the problem and found the solution he’d been dreaming of, one rooted in our modern understanding of the brain.

He and his team built a biologically inspired multilayer neural network made up of a new type of fundamental computational unit called rectified spectral units, or ReSUs. These ReSUs extract the features of the recent past that are most predictive of the near future. The ReSUs are self-supervised, meaning they control their own training of how they process data based on the information they receive, rather than relying on external instructions. ReSUs are designed to learn from constantly changing data, just as our brains learn from the real world.

This is in stark contrast to the current standard units, which are called rectified linear units (ReLUs). ReLUs, which have roots in a 1943 paper, were popularized about 15 years ago. In that paper, researchers presented “a very simple, but very primitive, model of a neuron,” Chklovskii says.

Building on that earlier model, researchers developed ReLU-based networks, which are commonly trained using a concept known as error backpropagation. This method calculates the contribution to past mistakes of each individual neuron in an artificial network, enabling the network to adjust and perform more accurately in the future. “But standard error backpropagation, as used in deep learning, is widely viewed as biologically implausible, and there is no evidence that the brain implements it in that form,” Chklovskii says.

Unlike the ReLUs, the novel ReSUs “actually care about the history of the input” they receive, says Shanshan Qin, a former CCN research scientist who is now an assistant professor of computational neuroscience and biophysics at Shanghai Jiao Tong University in China and lead author of the article that accompanied the AAAI presentation. That alternative setup, which doesn’t involve backpropagation, means ReSU networks are far closer analogs of what actually happens in the brain, he says.

The team’s ReSU neural network succeeded in a proof-of-principle test. The researchers created videos comprised of photographic images that drift in different directions, which were then used to train the network. “Imagine you are sitting on a train looking out the window. The trees, mountains, and houses outside appear to ‘slide’ horizontally across your vision. That sliding movement is a ‘translation,’” Qin says.

They demonstrated that a network trained on these videos exhibited learned two key features that resemble components of the fruit fly (Drosophila) visual system. The first feature is temporal filters, which sift through the input history that real or artificial neurons receive. These filters select certain signals to emphasize and others to ignore based on when the signals were received and other patterns that emerge within the system. Motion-selective units are the second key feature. These units only fire when movement occurs in a certain direction.

Instead of the researchers needing to directly instruct the system through coded rules, “we gave the network a blank slate,” Qin says. “We showed it the ‘train window’ videos (translating scenes). The network realized on its own: ‘To make sense of this data, I must remember what happened a split-second ago (temporal filters), and compare neighbor to neighbor (motion selection),” he says.

If the approach can be successfully scaled up, it could perform more complex computational tasks using rules similar to those that govern how neighboring neurons learn together. The approach may also excel in situations where the program lacks supervision and is using raw data that hasn’t been labeled or given additional context, Qin says.

The work not only brings AI closer to biology, but it also helps explain how biological systems operate, Qin says. “We can explain a lot of existing experimental data in fruit fly visual systems using this architecture,” he adds.

In the future, Chklovskii, Qin and colleagues hope to build on this work by developing ReSU-based neural networks based on different sensory systems — such as those responsible for smell and hearing — in animals ranging from fruit flies to humans. Such work would help reveal how those systems operate in nature and could reveal new ways of designing neural networks, Qin says.

Building the Reasoning Engine at Axiom

Mike's Notes

This is a very worthy aim. I hope they succeed. I discovered this article from reading This Week In AI Research from Dr Ashish Bamania.

"AxiomProver is an autonomous multi-agent ensemble theorem prover for Lean 4.21, developed by Axiom Math.

It autonomously and fully solved 12 out of 12 problems in Putnam 2025, the world’s hardest college-level math test, using the formal verification language Lean, 8 of which within the exam time.

A repository containing the solutions generated by AxiomProver can be found using this link.

A technical report will follow in the coming days, as per the team." - This Week In AI Research

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > This Week In AI Research
  • Home > Handbook > 

Last Updated

13/01/2026

Building the Reasoning Engine at Axiom

By: Axiom
Axiom: 24/05/2025

.

How hierarchical planning, verification, and self-play converge to mathematical superintelligence.

For centuries, mathematics has been humanity's most powerful sandbox – a place where we construct logical frameworks to understand reality's most complex systems. Why do grid cells fire in a hexagonal pattern? Why do quantum energy levels align with the spacing of primes?

Yet mathematical progress has always been shackled by a fundamental cruel bottleneck: the scarcity of extraordinary minds. When Évariste Galois revolutionized algebra in a fevered night before his fatal duel at twenty, he left behind ideas so far ahead of their time that decades passed before his contemporaries could grasp them. Srinivasa Ramanujan, before he succumbed to malnutrition and ill health, channeled thousands of formulas from his dreams into notebooks – results so profound that mathematicians spent nearly a century proving what he had intuited.

Even when breakthroughs occur as mathematicians' legacy, the extreme fragmentation of modern mathematics means experts in different subfields often cannot understand each other's work, causing vital connections between domains to remain hidden for decades. This combination of scarce genius and siloed knowledge creates an extraordinarily long pipeline from discovery to application – fundamental theorems discovered today might take generations before their full implications reshape technology and society – a delay so long that Hardy didn't anticipate in A Mathematician's Apology.

We're on a moonshot mission to change that. Axiom is building a reasoning engine capable of mathematical discoveries at scales and speed previously unimaginable – an AI mathematician.

The timing

Three trends are colliding for the first time in history.

First, neural networks have stepped beyond pattern matching into scalable reasoning, improving as compute, model size, and data grow.

Second, mathematical formalization has come of age through languages like Lean: by the Curry–Howard correspondence, proofs become executable programs, and programming languages are no longer just tools for producing outputs but instruments for certifying properties of abstract objects.

And lastly, LLMs have crossed a critical threshold in code generation, reliably producing high-quality code across many languages – including formal specification languages – and serving as a strong prior over the otherwise infinite action space of mathematics.

This synergy creates an unprecedented opportunity: reasoning engines that can conjecture and prove infinite theorems with zero human involvement.

The Convergence

Autoformalization is the Natural Language Compiler

Our data factory

Imagine you were a programmer in the 1950s. Your day to day was punching machine code into cards.

In the 1970s, you were wrestling with FORTRAN. By the 1990s, maybe C++. Today? You're basically just talking to the computer in English. Turing's childhood dream is now your reality with coding agents.

Each generation got to think a little less about the machine and a little more about the fun problem they were actually trying to solve.

Modern compilers are one-way DAGs. They take Python and transform it down the stack through multiple representations until it becomes machine code. There's some upward flow - as you type in your IDE, a partial compilation happens via Language Server Protocol, feeding information back up to create those squiggly lines and suggestions. But compilers never go fully back up the abstraction ladder. Disassembly exists, but it doesn't reconstruct your original high-level intent.

Mathematics needs something compilers never achieved: a true bidirectional cycle. For thousands of years, mathematicians think in high-level, intuitive leaps, not formal logic. Yet, math is undergoing a modernization. With proofs now spanning hundreds of pages, they are often riddled with hidden errors. In fact, every time proofs dogmatically resist being formally proven, the informal human source was wrong – just recently, mistakes were fixed during the formalization effort of Fermat's Last Theorem. The bottom formal layer catches what high-level intuition misses.

Meanwhile, autoformalization - the process of converting natural language proofs to Lean code - is a form of hierarchical planning, bridging between the abstraction layers.

Going down the stack: Autoformalization translates intuitive mathematical reasoning into formal proof – the compiler's traditional direction.

Going up the stack: Autoinformalization translates formal proofs back into human intuition – something compilers never truly do.

When combined, these create an interactive prover in natural language, freeing mathematicians to explore dozens of strategies and rapidly prototype ideas while the machine handles formalization.

But here's the most powerful part: machine discoveries at the formal level feed back up. The machine finds patterns, lemmas, and connections in the formal space that humans never intuited, then surfaces them as new high-level insights. The mathematician's intuition becomes augmented by discoveries happening in a space they couldn't naturally explore.

The compiler revolution took decades; the mathematical compiler revolution is happening now.

Formal Verification Guides Mathematical World Modeling

Our algorithmic core

You are a gold prospector in 1849. Everyone brings you shiny rocks claiming they've struck it rich.

Experienced prospectors examine them: "Looks like gold to me."

But when confronted with exotic ores, even experts disagree. Their pattern matching fails on things they've never seen.

Then someone brings an assayer's scale. The metal either dissolves in acid or it doesn't. Binary truth.

When you write a proof, it's either correct or it's not. Formal verifiers like Lean provide perfect ground truth while model judges are pattern matchers that fail when being pushed on generating genuinely novel proofs. From an engineering angle, verification gives us efficiently scalable rewards for learning.

And here's the philosophical perspective of why we need formal verification: Our self-consistent, observer-supporting universe follows rules that can be captured mathematically – from laws of physics to probability theory. Mathematics is the consistent language of our consistent universe and formal languages like Lean let us consistently operate in the mathematical world model.

We are training systems in mathematics as reality's minimal simulation – by learning to navigate the world of mathematics one grounded step at a time, the hope is that the AI has learned some fundamental rules that our reality has to follow. Video generation models learn physics too. Sometimes one ponders … where do abstract reasoning and spatial reasoning join?

Conjecturing-Proving Loop Realizes Self-Improving AI

Our discovery engine

While able to test if gold is real, finding new veins is harder: working in concert with verification, we enter the chapter of scientific discovery. Imagine you're in the middle of an ocean. Sailing towards new lands, of course, you start daydreaming about mathematics:

Your map is the Knowledge Base – showing where you've been. The entire corpus of mathematics indexed into a knowledge graph: definitions, theorems, and proofs. Formalized mathematics as a searchable, Internet-scale dataset.

Your ship is the Conjecturer – navigating uncharted territories. It spots distant landmasses through fog: "something valuable three days west." Built for open-ended exploration beyond known results, it samples out of distribution and generalizes with leaps guided by intrinsic motivations.

But when you spot an unknown island on the horizon, how do you know if it's India or the West Indies? The shape looks right, the distance seems plausible, but educated guess isn't certainty. You ask the experienced captain for wisdom that you trust – that is the Prover. Successful proofs extend the knowledge base. Failed attempts provide signals for improving both the Conjecturer and Prover. While formal verification turns "might be true" into "is true," counterexample construction shows "is false." Both grow the library.

The loop is self-reinforcing. More verified theorems mean a richer knowledge base. A richer knowledge base enables more sophisticated conjectures. More proof attempts (successful and failed) train better models. Better models generate more interesting conjectures and find proofs faster.

Axiom is building the AlphaGo for mathematics, but with infinite branching.

The Path Forward

The implications extend far beyond pure mathematics. Every complex system humans want to understand – from protein folding to quantum field theory and economic models – ultimately reduces to mathematical structures. A reasoning engine that can autonomously explore mathematical space and generate new theories doesn't just solve math problems; it provides a general-purpose tool for understanding reality.

Our founding team brings together precisely the expertise needed for this revolution. We were among the first to apply AI to compilers, bringing deep experience in programming languages and compiler technology. Our work spans from AI for mathematical discovery to pioneering self-improving systems. We're building reasoning engines that can operate in the mathematical world model at superhuman scale to tackle our most complex challenges.

The mathematical renaissance isn't coming. It's here.


AxiomProver at Putnam 2025

Putnam 2025, the world's hardest college-level math test, ended December 6th. By the end of the competition, AxiomProver had solved 8 out of 12 problems. In the following days, it solved the remaining 4. AxiomProver is an autonomous multi-agent ensemble theorem prover for Lean 4.21.0, developed by Axiom Math.

This repository contains the solutions generated by AxiomProver. Asterisk denotes solutions found after the competition.

  1. 2025 A1: [source], [graph]. Prover: 110 minutes, 7M tokens. Proof: 652 lines, 23 theorems, 561 tactics.
  2. 2025 A2: [source], [graph]. Prover: 185 minutes, 6M tokens. Proof: 556 lines, 26 theorems, 581 tactics.
  3. 2025 A3: [source], [graph]. Prover: 165 minutes, 8M tokens. Proof: 1,333 lines, 78 theorems, 1,701 tactics.
  4. 2025 A4: [source], [graph]. Prover: 107 minutes, 8M tokens. Proof: 960 lines, 32 theorems, 1,107 tactics.
  5. 2025 A5*: [source], [graph]. Prover: 518 minutes, 9.1M tokens. Proof: 2,054 lines, 52 theorems, 3,074 tactics.
  6. 2025 A6*: [source], [graph]. Prover: 259 minutes, 16M tokens. Proof: 588 lines, 28 theorems, 670 tactics.
  7. 2025 B1: [source], [graph]. Prover: 270 minutes, 7M tokens. Proof: 1,386 lines, 49 theorems, 1,841 tactics.
  8. 2025 B2: [source], [graph]. Prover: 65 minutes, 2M tokens. Proof: 417 lines, 28 theorems, 325 tactics.
  9. 2025 B3: [source], [graph]. Prover: 43 minutes, 2.9M tokens. Proof: 340 lines, 11 theorems, 422 tactics.
  10. 2025 B4*: [source], [graph]. Prover: 112 minutes, 249K tokens. Proof: 1,061 lines, 23 theorems, 1,433 tactics.
  11. 2025 B5: [source], [graph]. Prover: 354 minutes, 18M tokens. Proof: 1,495 lines, 66 theorems, 1,967 tactics.
  12. 2025 B6*: [source], [graph]. Prover: 494 minutes, 21M tokens. Proof: 1,019 lines, 30 theorems, 1,052 tactics.

Bayesian Statistics

Mike's Notes

A great introduction to Bayesian Statistics. With example workbooks. All free, including an Excel Resource Pack. The Real Statistics website by Charles Zaiontz is a fantastic resource.

Update

The YouTube lesson by Vivek Vinushanth Christopher is superb.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

24/01/2026

Bayesian Statistics

By: Charles Zaiontz
Real Statistics: 01/05/2021

Dr. Charles Zaiontz has a PhD in mathematics from Purdue University and has taught as an Assistant Professor at the University of South Florida as well as at Cattolica University (Milan and Piacenza) and St. Xavier College (Milan).

Bayesian statistics uses an approach whereby beliefs are updated based on data that has been collected. This can be an iterative process, whereby a prior belief is replaced by a posterior belief based on additional data, after which the posterior belief becomes a new prior belief to be refined based on even more data. The initial prior belief in this series may be based on intuition, previous studies, experience, etc.

In inferential statistics, we commonly test hypotheses, estimate parameters, and make predictions. In the traditional approach to statistics, commonly called the frequentist approach, parameters are constants whose values we aim to discern. Bayesian statistics uses a different approach: we treat these parameters as variables that have a probability distribution.

Topics

References

  1. Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., Rubin, D. B. (2014) Bayesian data analysis, 3rd Ed. CRC Press
    https://statisticalsupportandresearch.files.wordpress.com/2017/11/bayesian_data_analysis.pdf
  2. Marin, J-M and Robert, C. R.  (2014) Bayesian essentials with R. 2nd Ed. Springer
    https://www.springer.com/gp/book/9781461486862
  3. Jordan, M. (2010) Bayesian modeling and inference. Course notes
    https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/lectures/index.html
  4. Lee, P. M. (2012) Bayesian statistics an introduction. 4th Ed. Wiley
    https://www.wiley.com/en-us/Bayesian+Statistics%3A+An+Introduction%2C+4th+Edition-p-9781118332573

A Practical Guide to Monte Carlo Simulation

Mike's Notes

A great introduction to using Microsoft Excel for Monte Carlo simulation by Jon Wittwer. The 4-part series of articles is combined into a single article here. Vetex42 is an excellent free resource for learning to use Excel.

I recently discovered that Pipi has used Markov chain Monte Carlo (MCMC) since version 6, but the method was unnamed, as is much of Pipi's internal functioning (A disadvantage of working visually 😇 and finding ideas from anywhere to solve a problem)

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

15/02/2026

A Practical Guide to Monte Carlo Simulation

By: Jon Wittwer
Vertex42: 1/06/2004

Jon Wittwer started Vertex42 in 2003 while working on a PhD in mechanical engineering. After finishing his degree, he worked for Sandia National Laboratories where he kept Vertex42.com running on the side. In 2008, he left the labs to work on Vertex42 full time. In addition to his expertise with Excel, Dr. Wittwer is respected in multiple fields for his development of financial tools such as the Debt Reduction Calculator, project management tools like the Gantt Chart Template, statistical tools such as the Monte Carlo Simulator, and a large collection of business productivity and time management tools.

A Monte Carlo method is a technique that involves using random numbers and probability to solve problems. The term Monte Carlo Method was coined by S. Ulam and Nicholas Metropolis in reference to games of chance, a popular attraction in Monte Carlo, Monaco (Hoffman, 1998; Metropolis and Ulam, 1949).

Computer simulation has to do with using computer models to imitate real life or make predictions. When you create a model with a spreadsheet like Excel, you have a certain number of input parameters and a few equations that use those inputs to give you a set of outputs (or response variables).

This type of model is usually deterministic, meaning that you get the same results no matter how many times you re-calculate.

Example 1: A Deterministic Model for Compound Interest

Deterministic Model

Figure 1: A parametric deterministic model maps a set of input variables to a set of output variables.

Monte Carlo simulation is a method for iteratively evaluating a deterministic model using sets of random numbers as inputs. This method is often used when the model is complex, nonlinear, or involves more than just a couple uncertain parameters. A simulation can typically involve over 10,000 evaluations of the model, a task which in the past was only practical using super computers.

By using random inputs, you are essentially turning the deterministic model into a stochastic model. Example 2 demonstrates this concept with a very simple problem.

Example 2: A Stochastic Model for a Hinge Assembly

In Example 2, we used simple uniform random numbers as the inputs to the model. However, a uniform distribution is not the only way to represent uncertainty. Before describing the steps of the general MC simulation in detail, a little word about uncertainty propagation:

The Monte Carlo method is just one of many methods for analyzing uncertainty propagation, where the goal is to determine how random variation, lack of knowledge, or error affects the sensitivity, performance, or reliability of the system that is being modeled.

Monte Carlo simulation is categorized as a sampling method because the inputs are randomly generated from probability distributions to simulate the process of sampling from an actual population. So, we try to choose a distribution for the inputs that most closely matches data we already have, or best represents our current state of knowledge. The data generated from the simulation can be represented as probability distributions (or histograms) or converted to error bars, reliability predictions, tolerance zones, and confidence intervals. (See Figure 2).

Uncertainty Propagation

Monte Carlo Analysis

Figure 2: Schematic showing the principal of stochastic uncertainty propagation. (The basic principle behind Monte Carlo simulation.)

If you have made it this far, congratulations! Now for the fun part! The steps in Monte Carlo simulation corresponding to the uncertainty propagation shown in Figure 2 are fairly simple, and can be easily implemented in Excel for simple models. All we need to do is follow the five simple steps listed below:

  • Step 1: Create a parametric model, y = f(x1, x2, ..., xq).
  • Step 2: Generate a set of random inputs, xi1, xi2, ..., xiq.
  • Step 3: Evaluate the model and store the results as yi.
  • Step 4: Repeat steps 2 and 3 for i = 1 to n.
  • Step 5: Analyze the results using histograms, summary statistics, confidence intervals, etc.

On to an example problem ...

REFERENCES:

  • Hoffman, P., 1998, The Man Who Loved Only Numbers: The Story of Paul Erdos and the Search for Mathematical Truth. New York: Hyperion, pp. 238-239.
  • Metropolis, N. and Ulam, S., 1949, "The Monte Carlo Method." J. Amer. Stat. Assoc. 44, 335-341.
  • Eric W. Weisstein. "Monte Carlo Method." From MathWorld--A Wolfram Web Resource.
  • Paul Coddington. "Monte Carlo Simulation for Statistical Physics." Northeast Parallel Architectures Center at Syracuse University. http://www.npac.syr.edu/users/paulc/lectures/montecarlo/p_montecarlo.html
  • Decisioneering.com. "What is Monte Carlo Simulation?"" Part of: Risk Analysis Overview - http://www.decisioneering.com/risk-analysis-start.html

Our example of Monte Carlo simulation in Excel will be a simplified sales forecast model. Each step of the analysis will be described in detail.

The Scenario: Company XYZ wants to know how profitable it will be to market their new gadget, realizing there are many uncertainties associated with market size, expenses, and revenue.

The Method: Use a Monte Carlo Simulation to estimate profit and evaluate risk.

You can download the example spreadsheet by following the instructions below. You will probably want to refer to the spreadsheet occasionally as we proceed with this example.

Download the Sales Forecast Example

Step 1: Creating the Model

We are going to use a top-down approach to create the sales forecast model, starting with:

Profit = Income - Expenses

Both income and expenses are uncertain parameters, but we aren't going to stop here, because one of the purposes of developing a model is to try to break the problem down into more fundamental quantities. Ideally, we want all the inputs to be independent. Does income depend on expenses? If so, our model needs to take this into account somehow.

We'll say that Income comes solely from the number of sales (S) multiplied by the profit per sale (P) resulting from an individual purchase of a gadget, so Income = S*P. The profit per sale takes into account the sale price, the initial cost to manufacturer or purchase the product wholesale, and other transaction fees (credit cards, shipping, etc.). For our purposes, we'll say the P may fluctuate between $47 and $53.

We could just leave the number of sales as one of the primary variables, but for this example, Company XYZ generates sales through purchasing leads. The number of sales per month is the number of leads per month (L) multiplied by the conversion rate (R) (the percentage of leads that result in sales). So our final equation for Income is:

Income = L*R*P

We'll consider the Expenses to be a combination of fixed overhead (H) plus the total cost of the leads. For this model, the cost of a single lead (C) varies between $0.20 and $0.80. Based upon some market research, Company XYZ expects the number of leads per month (L) to vary between 1200 and 1800. Our final model for Company XYZ's sales forecast is:

Profit = L*R*P - (H + L*C)
Y = Profits
X1 = L
X2 = C
X3 = R
X4 = P

Notice that H is also part of the equation, but we are going to treat it as a constant in this example. The inputs to the Monte Carlo simulation are just the uncertain parameters (Xi).

This is not a comprehensive treatment of modeling methods, but I used this example to demonstrate an important concept in uncertainty propagation, namely correlation. After breaking Income and Expenses down into more fundamental and measurable quantities, we found that the number of leads (L) affected both income and expenses. Therefore, income and expenses are not independent. We could probably break the problem down even further, but we won't in this example. We'll assume that L, R, P, H, and C are all independent.

Note: In my opinion, it is easier to decompose a model into independent variables (when possible) than to try to mess with correlation between random inputs.

Step 2: Generating Random Inputs

The key to Monte Carlo simulation is generating the set of random inputs. As with any modeling and prediction method, the "garbage in equals garbage out" principle applies. For now, I am going to avoid the questions "How do I know what distribution to use for my inputs?" and "How do I make sure I am using a good random number generator?" and get right to the details of how to implement the method in Excel.

For this example, we're going to use a Uniform Distribution to represent the four uncertain parameters. The inputs are summarized in the table shown below. (If you haven't already, Download the example spreadsheet).

Sales Forecast Input Table

Figure 1: Screen capture from the example sales forecast spreadsheet.

The table above uses "Min" and "Max" to indicate the uncertainty in L, C, R, and P. To generate a random number between "Min" and "Max", we use the following formula in Excel (Replacing "min" and "max" with cell references):

= min + RAND()*(max-min)

You can also use the Random Number Generation tool in Excel's Analysis ToolPak Add-In to kick out a bunch of static random numbers for a few distributions. However, in this example we are going to make use of Excel's RAND() formula so that every time the worksheet recalculates, a new random number is generated.

Let's say we want to run n=5000 evaluations of our model. This is a fairly low number when it comes to Monte Carlo simulation, and you will see why once we begin to analyze the results.

A very convenient way to organize the data in Excel is to make a column for each variable as shown in the screen capture below.

Random Inputs in Column Format

Figure 2: Screen capture from the example sales forecast spreadsheet.

Cell A2 contains the formula:

=Model!$F$14+RAND()*(Model!$G$14-Model!$F$14)

Note that the reference Model!$F$14 refers to the corresponding Min value for the variable L on the Model worksheet, as shown in Figure 1. (Hopefully you have downloaded the example spreadsheet and are following along.)

To generate 5000 random numbers for L, you simply copy the formula down 5000 rows. You repeat the process for the other variables (except for H, which is constant).

Step 3: Evaluating the Model

Our model is very simple, so to evaluate the output of our model (the Profit) for each run of the simulation, we just put the equation in another column next to the inputs, as shown in Figure 2.

Cell G2 contains the formula:

=A2*C2*D2-(E2+A2*B2)

Step 4: Running the Simulation

To iteratively evaluate our model, we don't need to write a fancy macro for this example. We simply copy the formula for profit down 5000 rows, making sure that we use relative references in the formula (no $ signs). Each row represents a single evaluation of the model, with columns A-E as inputs and the Profit as the output.

Re-run the Simulation: F9

Although we still need to analyze the data, we have essentially completed a Monte Carlo simulation. We have used the volatile RAND() function. So, to re-run the entire simulation all we have to do is recalculate the worksheet (F9 is the shortcut).

This may seem like a strange way to implement Monte Carlo simulation, but think about what is going on behind the scenes every time the Worksheet recalculates: (1) 5000 sets of random inputs are generated (2) The model is evaluated for all 5000 sets. Excel is handling all of the iteration.

If your model is not simple enough to include in a single formula, you can create your own custom Excel function (see my article on user-defined functions), or you can create a macro to iteratively evaluate your model and dump the data into a worksheet in a similar format to this example (Update 9/8/2014: See my new Monte Carlo Simulation template).

In practice, it is usually more convenient to buy an add-on for Excel than to do a Monte Carlo analysis from scratch every time. But not everyone has the money to spend, and hopefully the skills you will learn from this example will aid in future data analysis and modeling.

A Few Other Distributions

My new Monte Carlo Simulation template includes a worksheet that calculates inputs sampled from a variety of distributions. Some of the formulas are listed below.

Normal (Gaussian) distribution

To generate a random number from a Normal distribution you would use the following formula in Excel:

=NORMINV(rand(),mean,standard_dev)
Ex: =NORMINV(RAND(),$D$4,$D$5)
Excel 2010+: =NORM.INV(RAND(),$D$4,$D$5)

Lognormal distribution

To generate a random number from a Lognormal distribution with median = exp(meanlog), and shape = sdlog, you would use the following formula in Excel:

=LOGINV(RAND(),meanlog,sdlog)
Ex: =LOGINV(RAND(),$D$6,$D$5)
Excel 2010+: =LOGNORM.INV(RAND(),$D$4,$D$5)

Weibull distribution

There isn't an inverse Weibull function in Excel, but the formula is quite simple, so to generate a random number from a (2-parameter) Weibull distribution with scale = c, and shape = m, you would use the following formula in Excel:

=c*(-LN(1-RAND()))^(1/m)
Ex: $C$5*(-LN(1-RAND()))^(1/$C$6)

Beta distribution

This distribution can be used for variables with finite bounds (A,B). It uses two shape parameters, alpha and beta. When alpha=beta=1, you get a Uniform distribution. When alpha=beta=2, you get a dome-shaped distribution which is often used in place of the Triangular distribution. When alpha=beta=5 (or higher), you get a bell-shaped distribution. When alpha<>beta (not equal), you get a variety of skewed shapes.

Excel 2010+: =BETA.INV(RAND(),alpha,beta,A,B)

MORE Distribution Functions: Dr. Roger Myerson provides a free downloadable Excel add-in, Simtools.xla, that includes many other distribution functions for generating random numbers in Excel.

Creating a histogram is an essential part of doing a statistical analysis because it provides a visual representation of data.

In Part 3 of this Monte Carlo Simulation example, we iteratively ran a stochastic sales forecast model to end up with 5000 possible values (observations) for our single response variable, profit. If you have not already, download the Sales Forecast Example Spreadsheet.

The last step is to analyze the results to figure out how much the profit might be expected to vary based on our uncertainty in the values used as inputs for our model. We will start off by creating a histogram in Excel. The image below shows the end result. Keep reading below to learn how to make the histogram.

Histogram With Excel

Figure 1: A Histogram in Excel for the response variable Profit, created using a Bar Chart.

(From a Monte Carlo simulation using n = 5000 points and 40 bins).

We can glean a lot of information from this histogram:

  • It looks like profit will be positive, most of the time.
  • The uncertainty is quite large, varying between -1000 to 3400.
  • The distribution does not look like a perfect Normal distribution.
  • There doesn't appear to be outliers, truncation, multiple modes, etc.

The histogram tells a good story, but in many cases, we want to estimate the probability of being below or above some value, or between a set of specification limits. To skip ahead to the next step in our analysis, move on to Summary Statistics, or continue reading below to learn how to create the histogram in Excel.

Creating a Histogram in Excel

Update 7/2/15: A Histogram chart is one of the new built-in chart types in Excel 2016, finally! (Read about it).

Method 1: Using the Histogram Tool in the Analysis Tool-Pak.

This is probably the easiest method, but you have to re-run the tool each to you do a new simulation. AND, you still need to create an array of bins (which will be discussed below).

Method 2: Using the FREQUENCY function in Excel.

This is the method used in the spreadsheet for the sales forecast example. One of the reasons I like this method is that you can make the histogram dynamic, meaning that every time you re-run the MC simulation, the chart will automatically update. This is how you do it:

Step 1: Create an array of bins

The figure below shows how to easily create a dynamic array of bins. This is a basic technique for creating an array of N evenly spaced numbers.

To create the dynamic array, enter the following formulas:

  B6 = $B$2
  B7 = B6+($B$3-$B$2)/5

Then, copy cell B7 down to B11

Array of Bins in Excel

Figure 2: A dynamic array of 5 bins.

After you create the array of bins, you can go ahead and use the Histogram tool, or you can proceed with the next step.

Step 2: Use Excel's FREQUENCY formula

The next figure is a screen shot from the example Monte Carlo simulation. I'm not going to explain the FREQUENCY function in detail since you can look it up in the Excel's help file. But, one thing to remember is that it is an array function, and after you enter the formula, you will need to press Ctrl+Shift+Enter. Note that the simulation results (Profit) are in column G and there are 5000 data points ( Points: J5=COUNT(G:G) ).

The Formula for the Count column:

  FREQUENCY(data_array,bins_array)

  a) Select cells J8:J48
  b) Enter the array formula: =FREQUENCY(G:G,I8:I48)
  c) Press Ctrl+Shift+Enter

Layout for Creating a Scaled Histogram

Figure 3: Layout in Excel for Creating a Dynamic Scaled Histogram.

Creating a Scaled Histogram

If you want to compare your histogram with a probability distribution, you will need to scale the histogram so that the area under the curve is equal to 1 (one of the properties of probability distributions). Histograms normally include the count of the data points that fall into each bin on the y-axis, but after scaling, the y-axis will be the frequency (a not-so-easy-to-interpret number that in all practicality you can just not worry about). The frequency doesn't represent probability!

To scale the histogram, use the following method:

  Scaled = (Count/Points) / (BinSize)

  a) K8 = (J8/$J$5)/($I$9-$I$8)
  b) Copy cell K8 down to K48
  c) Press F9 to force a recalculation (may take a while)

Step 3: Create the Histogram Chart

Bar Chart, Line Chart, or Area Chart:

To create the histogram, just create a bar chart using the Bins column for the Labels and the Count or Scaled column as the Values. Tip: To reduce the spacing between the bars, right-click on the bars and select "Format Data Series...". Then go to the Options tab and reduce the Gap. Figure 1 above was created this way.

A More Flexible Histogram Chart

One of the problems with using bar charts and area charts is that the numbers on the x-axis are just labels. This can make it very difficult to overlay data that uses a different number of points or to show the proper scale when bins are not all the same size. However, you CAN use a scatter plot to create a histogram. After creating a line using the Bins column for the X Values and Count or Scaled column for the Y Values, add Y Error Bars to the line that extend down to the x-axis (by setting the Percentage to 100%). You can right-click on these error bars to change the line widths, color, etc.

Histogram Via Error Bars

Figure 4: Example Histogram Created Using a Scatter Plot and Error Bars.

REFERENCES:

CITE THIS PAGE AS:

Wittwer, J.W., "Creating a Histogram In Excel" From Vertex42.com, June 1, 2004