How to Learn the Math Needed for Machine Learning

Mike's Notes

An outline of what I need to study.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Towards Data Science
  • Home > Handbook > 

Last Updated

31/05/2025

How to Learn the Math Needed for Machine Learning

By: Edgor Howell
Towards Data Science: 15/05/2025

A breakdown of the three fundamental math fields required for machine learning: statistics, linear algebra, and calculus.

Maths can be a scary topic for people.

Many of you want to work in machine learning, but the maths skills needed may seem overwhelming.

I am here to tell you that it’s nowhere as intimidating as you may think and to give you a roadmap, resources, and advice on how to learn math effectively.

Let’s get into it!

Do you need maths for machine learning?

I often get asked:

Do you need to know maths to work in machine learning?

The short answer is generally yes, but the depth and extent of maths you need to know depends on the type of role you are going for.

A research-based role like:

  • Research Engineer — Engineer who runs experiments based on research ideas.
  • Research Scientist — A full-time researcher on cutting edge models.
  • Applied Research Scientist — Somewhere between research and industry.

You will particularly need strong maths skills.

It also depends on what company you work for. If you are a machine learning engineer or data scientist or any tech role at:

  • Deepmind
  • Microsoft AI
  • Meta Research
  • Google Research

You will also need strong maths skills because you are working in a research lab, akin to a university or college research lab.

In fact, most machine learning and AI research is done at large corporations rather than universities due to the financial costs of running models on massive data, which can be millions of pounds.

For these roles and positions I have mentioned, your maths skills will need to be a minimum of a bachelor’s degree in a subject such as math, physics, computer science, statistics, or engineering.

However, ideally, you will have a master’s or PhD in one of those subjects, as these degrees teach the research skills needed for these research-based roles or companies.

This may sound heartening to some of you, but this is just the truth from the statistics.

According to a notebook from the 2021 Kaggle Machine Learning & Data Science Survey, the research scientist role is highly popular among PhD and doctorates.

...

And in general, the higher your education the more money you will earn, which will correlate with maths knowledge.

...

However, if you want to work in the industry on production projects, the math skills needed are considerably less. Many people I know working as machine learning engineers and data scientists don’t have a “target” background.

This is because industry is not so “research” intensive. It’s often about determining the optimal business strategy or decision and then implementing that into a machine-learning model.

Sometimes, a simple decision engine is only required, and machine learning would be overkill.

High school maths knowledge is usually sufficient for these roles. Still, you may need to brush up on key areas, particularly for interviews or specific specialisms like reinforcement learning or time series, which are quite maths-intensive.

To be honest, the majority of roles are in industry, so the maths skills needed for most people will not be at the PhD or master’s level. 

But I would be lying if I said these qualifications do not give you an advantage.

What maths do you need to know?

There are three core areas you need to know:

  • Statistics
  • Calculus
  • Linear Algebra

Statistics

I may be slightly biased, but statistics is the most important area you should know and put the most effort into understanding.

Most machine learning originated from statistical learning theory, so learning statistics will mean you will inherently learn machine learning or its basics.

These are the areas you should study:

  • Descriptive Statistics — This is useful for general analysis and diagnosing your models. This is all about summarising and portraying your data in the best way.
    • Averages: Mean, Median, Mode
    • Spread: Standard Deviation, Variance, Covariance
    • Plots: Bar, Line, Pie, Histograms, Error Bars
  • Probability Distributions — This is the heart of statistics as it defines the shape of the probability of events. There are many, and I mean many, distributions, but you certainly don’t need to learn all of them.

    • Normal
    • Binomial
    • Gamma
    • Log-normal
    • Poisson
    • Geometric
  • Probability Theory — As I said earlier, machine learning is based on statistical learning, which comes from understanding how probability works. The most important concepts are

    • Maximum likelihood estimation
    • Central limit theorem
    • Bayesian statistics
  • Hypothesis Testing —Most real-world use cases of data and machine learning revolve around testing. You will test your models in production or carry out an A/B test for your customers; therefore, understanding how to run hypothesis tests is very important.

    • Significance Level
    • Z-Test
    • T-Test
    • Chi-Square Test
    • Sampling
  • Modelling & Inference —Models like linear regression, logistic regression, polynomial regression, and any regression algorithm originally came from statistics, not machine learning.

    • Linear Regression
    • Logistic Regression
    • Polynomial Regression
    • Model Residuals
    • Model Uncertainty
    • Generalised Linear Models

Calculus

Most machine learning algorithms learn from gradient descent in one way or another. And, gradient descent has its roots in calculus.

There are two main areas in calculus you should cover:

  • Differentiation
    • What is a derivative?
    • Derivatives of common functions.
    • Turning point, maxima, minima and saddle points.
    • Partial derivatives and multivariable calculus.
    • Chain and product rules.
    • Convex vs non-convex differentiable functions.
  • Integration

    • What is integration?
    • Integration by parts and substitution.
    • The integral of common functions.
    • Integration of areas and volumes.

Linear Algebra

Linear algebra is used everywhere in machine learning, and a lot in deep learning. Most models represent data and features as matrices and vectors.

  • Vectors 
    • What are vectors
    • Magnitude, direction
    • Dot product
    • Vector product
    • Vector operations (addition, subtraction, etc)
  • Matrices 
    • What is a matrix
    • Trace
    • Inverse
    • Transpose
    • Determinants
    • Dot product
    • Matrix decomposition
  • Eigenvalues & Eigenvectors 
    • Finding eigenvectors
    • Eigenvalue decomposition
    • Spectrum analysis

Best Resources

There are loads of resources, and it really comes down to your learning style.

If you are after textbooks, then you can’t go wrong with the following and is pretty much all you need:

  • Practical Statistics For Data Scientist — I recommend this book all the time and for good reason. This is the only textbook you realistically need to learn the statistics for Data Science and machine learning.
  • Mathematics for Machine Learning — As the name implies, this textbook will teach the maths for machine learning. A lot of the information in this book may be overkill, but your maths skills will be excellent if you study everything.

If you want some online courses, I have heard good things about the following ones.

  • Mathematics for Machine Learning and Data Science Specialisation — This course is by DeepLearning.AI, the same people who made the Machine Learning Specialisation, arguably the best machine learning course.

Learning Advice

The amount of maths content you need to learn may seem overwhelming, but don’t worry.

The main thing is to break it down step by step.

Pick one of the three: statistics, Linear Algebra or calculus.

Look at the things I wrote above you need to know and choose one resource. It doesn’t have to be any of the ones I recommended above.

That’s the initial work done. Don’t overcomplicate by looking for the “best resource” because such a thing doesn’t exist.

Now, start working through the resources, but don’t just blindly read or watch the videos.

Actively take notes and document your understanding. I personally write blog posts, which essentially employ the Feynman technique, as I am, in a way, “teaching” others what I know.

Writing blogs may be too much for some people, so just make sure you have good notes, either physically or digitally, that are in your own words and that you can reference later.

The learning process is generally quite simple, and there have been studies done on how to do it effectively. The general gist is:

  • Do a little bit every day
  • Review old concepts frequently (spaced repetition)
  • Document your learning
  • It’s all about the process; follow it, and you will learn!

How to be a great thinker

Mike's Notes

This says it all.

Resources

References

  • Vienna: How the City of Ideas Created the Modern World by Dr Richard Crockett

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

30/05/2025

How to be a great thinker

By: Simon Kuper
FT: 22/05/2025

Simon Kuper is a British, and naturalized French, author and journalist, best known for his work at the Financial Times and as a football writer.

Born in Uganda to South African parents, Kuper spent most of his childhood in the Netherlands and lives in Paris. After studies at Oxford, Harvard University and the Technische Universität Berlin, Kuper started his career in journalism at the FT in 1994, where he today writes about a wide range of topics, such as politics, society, culture, sports and urban planning. - Wikipedia

Most people are getting dumber. Largely because of the smartphone, we’re in an era of declining attention spans, reading skills, numeracy and verbal reasoning. How to buck the trend? I’ve charted seven intellectual habits of the best thinkers. True, these people exist in a different league from the rest of us. To use an analogy from computing, their high processing power allows them to crunch vast amounts of data from multiple domains. In other words, they have intellectual overcapacity. Still, we can learn from their methods. These can sound obvious, but few people live by them.

Read books.

A book is still the best technology to convey the nuanced complexity of the world. That complexity is a check on pure ideology. People who want to simplify the world will prefer online conspiracy theories. 

Don’t use screens much.

That frees time for books and creates more interstitial moments when the mind is left unoccupied, has freedom to roam and makes new connections. Darwin, Nietzsche and Kant experienced these moments on walks. The biochemist Jennifer Doudna says she gets insights when “out weeding my tomato plants” or while asleep.  

Do your own work, not the world’s.

The best thinkers don’t waste much time maximising their income or climbing hierarchies. Doudna left Berkeley to lead discovery research at biotech company Genentech. She lasted two months there. Needing full scientific freedom, she returned to Berkeley, where she ended up winning the chemistry Nobel Prize for co-inventing the gene-editing tool Crispr.

Be multidisciplinary.

Prewar Vienna produced thinkers including Freud, Hayek, Kurt Gödel and the irreducible polymath John von Neumann. The structure of the city’s university helped. Most subjects were taught within the faculties of either law or philosophy. That blurred boundaries between disciplines, writes Richard Cockett in Vienna: How the City of Ideas Created the Modern World. “There were no arbitrary divisions between ‘science’ and ‘humanities’ — all was ‘philosophy’, in its purest sense, the study of fundamental questions.” 

Hayek, for instance, “trained at home as a botanist to a quasi-professional level; he then graduated in law, received a doctorate in political science from the university, but . . . spent most of his time there studying psychology, all before becoming a revered economist.”

Breaking through silos goes against the set-up of modern academia. It also requires unprecedented processing power, given how much knowledge has accumulated in each field. But insights from one discipline can still revolutionise another. The psychologist Daniel Kahneman won the Nobel Prize for economics for his findings on human irrationality. 

Be an empiricist who values ideas.

During the second world war, Isaiah Berlin was first secretary at the British embassy in Washington. His weekly reports on the American political situation were brilliant empirical accounts of the world as it was. They mesmerised Winston Churchill, who was desperate to meet Berlin. (Due to a mix-up, Churchill invited Irving Berlin for lunch instead. The composer was baffled to be asked by Churchill himself, “When do you think the European war will end?”)

In March 1944, Isaiah Berlin returned from Washington to London on a bomber plane. He had to wear an oxygen mask all flight, wasn’t allowed to sleep for fear he would suffocate, and couldn’t read as there was no light. “One was therefore reduced to a most terrible thing,” he recalled, “to having to think — and I had to think for about seven or eight hours in this bomber.” During this long interstitial moment, Berlin decided to become an historian of ideas. He ended up writing the classic essays The Hedgehog and the Fox and Two Concepts of Liberty. 

Always assume you might be wrong.

Mediocre thinkers prefer to confirm their initial assumptions. This “confirmation bias” stops them reaching new or deeper insights. By contrast, Darwin was always composing arguments against his own theories. 

Keep learning from everyone.

Only mediocrities boast as adults about where they went to university aged 18. They imagine that intelligence is innate and static. In fact, people become more or less intelligent through life, depending on how hard they think. The best thinkers are always learning from others, no matter how young or low-status. I remember being at a dinner table where the two people who talked least and listened hardest were the two Nobel laureates.

Measuring IT Complexity

Mike's Notes

Here is a small extract of an excellent paper written by Roger Sessions in 2009. It describes the maths he uses to describe IT complexity.

Sadly, his websites have not been updated for years, and many of the links to his excellent papers are now broken.

Usually, I use a webcrawler to archive excellent websites, but I didn't add ObjectWatch. I am now actively scouring the web to create an electronic archive of his work before it disappears.

His work was brilliant and deserves a wider audience.

Resources

References

  • The IT Complexity Crisis: Danger and Opportunity. A White Paper by Roger Sessions. October 22, 2009
  • Fact and Fallacies About Software Engineering. By Robert Glass
  • Woodfield, Scott N. 1979. “An Experiment on Unit Increase in Problem Complexity.” IEEE Transactions on Software Engineering, (Mar.) p76-79 Scott N. Woodfield:

Repository

  • Home > Ajabbi Research > Library > Authors > Roger Sessions
  • Home > Handbook > 

Last Updated

30/05/2025

Measuring IT Complexity

By: Roger Sessions
ObjectWatch 22/10/2009

Roger Sessions, a tireless advocate for efficiency through simplicity, is the CTO of ObjectWatch, a company he founded thirteen years ago. He has written seven books including his most recent, Simple Architectures for Complex Enterprises, and dozens of articles. He assists both public and private sector organizations in reducing IT complexity by blending existing architectural methodologies and SIP. In addition, Sessions provides architectural reviews and analysis. Sessions holds multiple patents in software and architectural methodology. He is a Fellow of the International Association of Software Architects (IASA), Editor-in-Chief of the IASA Perspectives Journal, and a Microsoft recognized MVP in Enterprise Architecture. A frequent keynote speaker, Sessions has presented in countries around the world on the topics of IT Complexity and Enterprise Architecture. Sessions has a Masters Degree in Computer Science from the University of Pennsylvania. He lives in Chappell Hill, Texas.

Roger loves feedback (and a good, stirring debate or two!) Join Roger in his crusade against complexity. His blog is SimpleArchitectures.blogspot.com and his Twitter ID is @RSessions.

...

Measuring IT complexity is easier than you might think. It all starts with Glass’s Law [06], that for every 25% increase in the complexity of the problem space, there is a 100% increase in the complexity of the solution space. In IT systems, there are two contributors to the complexity of the problem space. The first is the number of business functions in the system. The second is the number of connections that system has to other systems.

We need to start with a standard complexity unit (SCU). I’ll define one SCU as the amount of complexity that a system has which contains only one business function and no connections to other systems. Based on Glass’s Law, this is the least complex system possible.

Okay, we start with a system with one business function and no connections. By definition, this has 1 SCU. Now let’s start adding more business functions into the system. As we add more functions into the system, the number of SCUs goes up a predictable amount. This amount is calculated using Bird’s Formula. Bird is Chris Bird, who showed me how to rewrite Glass’s Law as a mathematical formula.

Let’s say a system S has bf number of business functions, and no connections to other systems. Then the number of SCUs in that system is given by

S = 10 raised to the power of ((log(2)/log(1.25) X log (bf))

The log(2) and the log(1.25) are both constants, and can thus be combined, giving

S = 10 raised to the power of (3.1 X log(bf))

or, more simply,

S = 10 3.1 log(bf)

Similarly, we can calculate the complexity of a system with one business function and cn connections to other systems. Note that I am simplifying the analysis by assuming that a new connection adds about the same amount of complexity as a new business function, which follows my experience. If you don’t agree, it is easy enough to modify the equation accordingly. But given my assumption, the complexity because of connections is as follows:

S = 10 3.1 log(cn)

Most systems have both multiple business functions and multiple connections, so we need to add the two together:

S = 10 3.1 log(bf) + 10 3.1 log(cn)

Most systems are not made of a single system, but a number of smaller systems, each of which has a complexity described by the above equations. An SOA, for example, would have multiple services, each of which has a complexity rating.

If we assume that our system is an SOA, then the complexity of the SOA as a whole is the summation of the complexity of each of the individual services. This is expressed as what I describe as Sessions’s Summation of Bird’s Formulation of Glass’s Law. It looks like this:


Now while Sessions’s Summation looks rather ugly, it is in fact easily calculated using a straightforward spreadsheet.

Sessions’s Summation gives us an easy way to compare two different architectures with respect to their complexity. Just plug both into Sessions’s Summation and read the resulting number. That number is the complexity in SCUs of the architecture. If one architecture has a total SCU of 1,000 and another a total SCU of 500, then the first is twice as complex as the second. It is also twice as likely to fail.

Notice that while functionality is a factor in complexity, it is not the major factor. Much larger factors are how many services there are, how many functions there are in each of the services and how those services are connected to each other.

...

[06] Glass’s Law comes from Robert Glass’s Fact and Fallacies About Software Engineering. He did not discover the law. He actually described it from a paper by Scott Woodfield, but Glass did more than anybody to publicize the law.

Markus Covert: How to build a computer model of a cell

Mike's Notes

Another video interview with Marcus Covert. Pipi 6 was built by fusing Pipi with Covert's open-source cellular simulation software.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Authors > Markus Covert
  • Home > Handbook > 

Last Updated

05/06/2025

Markus Covert: How to build a computer model of a cell

By: Stanford Engineering Staff
The Future of Everything podcast: 20/10/2020

When Stanford bioengineer Markus Covert first decided to create a computer model able to simulate the behavior of a single cell, he was held back by more than an incomplete understanding of how a cell functions, but also by a lack of computer power.

His early models would take more than 10 hours to churn through a single simulation and that was when using a supercomputer capable of billions of calculations per second.

Nevertheless, in his quest toward what had been deemed “a grand challenge of the 21st century,” Covert pressed on and eventually published a paper announcing his success in building a model of just one microbe: E. coli, a popular subject in biological research. The model would allow researchers to run experiments not on living bacteria in a lab, but on a simulated cell on a computer.

After all was said and done, however, the greatest takeaway for Covert was that a cell is a very, very complex thing. There were fits and starts and at least one transcendent conceptual leap — which Covert has dubbed “deep curation” — needed to make it all happen, but he found a way. As Covert points out, no model is perfect, but some are useful. And that is how usefulness, not perfection, became the goal of his work, as he tells fellow bioengineer Russ Altman in this episode of Stanford Engineering’s The Future of Everything podcast.

UX Copy Sizes: Long, Short, and Micro

Mike's Notes

This definition of copy sizes from NNGroup is great. I am adding it to the Pipi Content Management System Engine (cms) internal schema.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > NNGroup
  • Home > pipiWiki > Engines > CMS Engine
  • Home > Repository > Schema > NNGroup > UX Copy Sizes

Last Updated

30/05/2025

UX Copy Sizes: Long, Short, and Micro

By: Taylor Dykes
NNGroup: 16/05/2025

Summary:

Better target user needs by understanding the three sizes of copy: long-form, short-form, and microcopy.

Content terms like long-form, short-form, and microcopy are often used interchangeably, but are ill-defined and don’t mean the same thing. Knowing the differences between these types of copy and when and how to design them effectively will create written information that better supports user needs.

Content vs. Copy: What’s the Difference?

Before discussing different copy lengths, let’s define the difference between content and copy.

  • Content is everything that goes into a digital interface, regardless of media format.
  • Copy is a subcategory of content encompassing all the user-interface text written by the organization (not by the user).

Diagram of digital content types—Copy, Audio, Video, User-generated, and Images—inside a large circle. Film, Print Media, and Television appear outside.


Content is an all-encompassing term that includes copy and all the other elements of digital interfaces, such as videos, audio, and images.

While it might be easy to distinguish between copy and content along media formats, it can be harder to distinguish between user-generated content and copy. Essentially, any content not designed by the publishing platform for that platform is user-generated content.

In this definition, reviews, comments, and social posts all count as user-generated content. For example, an Amazon review is not copy because it is not created by the platform; instead, someone from outside — the user — created this content.

Now that we’ve established the differences between copy and content, let’s review how copy length impacts the user experience.

Long-Form Copy

Long-form copy is 3 or more paragraphs that form a coherent and continuous unit.

Writers should use long-form copy when additional detail, complexity, or context needs to be communicated to the user. When used correctly, the expanded word count allows writers to elaborate and include every detail users might need to know to accomplish their tasks.

When people think of long-form copy, they often think of using it for blog posts, news, or educational articles. However, long-form copy can also be used for:

  • Policy descriptions
  • Product or technical documentation
  • Help and support pages
  • Reports or case studies
  • Product pages
  • About us pages
  • Proposals and grants

Long-form content is becoming increasingly rare online, due to an ever-decreasing user attention span. But it will never go extinct, as it is the only copy type capable of delivering complex, detail-rich information on topics like multistep processes or troubleshooting advanced technical problems. In addition, users sometimes read for a more comprehensive understanding when the topic is significant to them (also called the commitment pattern). Giving bite-sized information in such a situation might create distrust.

Another reason why long-form copy will likely always have a place online is its ability to aid search-engine optimization (SEO). Long-form copy can naturally hold many keywords, increase user engagement time, and assist internal linking, which help improve a site’s SEO.

A multi paragraph page describing a hospital system's history of treating orthopedic conditions, what sets them apart, and their continued research.


This landing page for a hospital system’s orthopedic offerings uses long-form copy to insert keywords (such as Maryland, Washington D.C., and Virginia) to increase its  ranking on the search-engine results page.

Long-form copy doesn’t easily grab user attention. Unlike the other copy lengths, long-form copy requires time, attention, and mental energy to read thoroughly. This factor has caused long-form copy to become increasingly uncommon outside of articles.

To help users quickly get the gist from long-form copy, good content designers and writers include short-form copy and microcopy to format, structure, and break up text. For example, the long-form copy might include microcopy like headings and subheadings, or short-form copy in an accordion’s answer to a frequently asked question. While long-form copy often comprises short-form and microcopy, it still reads as a cohesive unit to users.

Short-Form Copy

Short-form copy is 2–3 paragraphs focused on communicating one main idea.

Short-form copy is used when a single idea or main point needs to be conveyed quickly, often in a way that grabs the user's attention. Writers use short-form copy to help users find information or quickly understand important ideas and messages that, if buried in long-form copy, might be missed.

Some examples of short-form copy include:

  1. Onboarding tutorials
  2. Longer summaries
  3. Product descriptions
  4. Detailed mission statements

Short-form copy has become the default way for communicating information to audiences. Since reading long-form copy requires too much user effort and attention, writers must consider how they might fit detailed information into a short-form format. Strategically breaking up text or organizing it innovatively can communicate a lot of information in scanning-friendly short-form copy.

Page showing the different types of blood donation. Under the headings for two types is a single paragraph and short blurbs of information.


The American Red Cross used short-form copy within cards to structure information that had likely been presented as a long-form article in the past.

Short-form copy is the middle ground between long-form and microcopy; it’s short enough to scan but long enough to convey a whole idea. It won’t overwhelm users but may provide enough information to help them find something or make an informed decision, so they won’t need to read long-form copy on the same topic.

However, UX writers should avoid prematurely defaulting to this happy medium. Even if users are more likely to scan or even read the short-form copy in its entirety, short-form copy won’t help them comprehend the information if the topic’s complexity is better suited for long-form.

Short-form copy is also not a replacement for microcopy, even if it can create a more complete picture of a topic. Users want specific takeaways instantly, and that’s better suited for one-to-two sentence microcopy.

Microcopy

Microcopy is the smallest copy size: fewer than 3 sentences.

Microcopy is used when a writer needs to quickly inform, influence, or encourage interaction for the user’s next step. Because of its size, microcopy is the copy that is most easily processed by users (through scanning or screen-reader voice-over). Good microcopy will prevent errors, encourage clicks, and educate users.

Examples of microcopy include:

  1. Link and button labels
  2. Form-field instructions
  3. Input-control labels
  4. Page titles and meta descriptions
  5. Error messages
  6. Tooltips

Microcopy often makes up most of the written information in the experience. Designers favor microcopy because it allows them to guide users efficiently without disrupting the flow of an interaction. Users appreciate microcopy because it’s easy to skim and scan as they navigate, helping them quickly understand what to do without being overwhelmed by large amounts of text.

The homepage of IBM. There are taglines, buttons, link labels and summaries on this page.


IBM.com:  Each text snippet in this screenshot is an example of microcopy.

While much of the copy in an interface is microcopy, microcopy alone cannot create a complete experience. Microcopy needs other UI elements, images, videos, input controls, and a mix of short- and long-form to create an effective experience. Microcopy isn’t meant to be the primary focus of a website; it’s there to guide, support, and influence users toward the main content or action.

Copy Sizes: In Brief

Long-Form Copy

Definition

  • 3+ paragraphs that form a coherent and continuous unit

When to Use

  • For detailed or complex information
  • When users want more thorough knowledge
  • To aid SEO

Examples

  • Policy descriptions
  • Product or technical documentation
  • Help and support pages
  • Reports or case studies
  • Product pages
  • About us pages
  • Proposals and grants 

Short-Form Copy

Definition

  • 2–3 paragraphs that communicate one main idea 

When to Use

  • For succinctly communicating important and relatively detailed information
  • For breaking down long copy into scannable units

Examples

  • Onboarding tutorials
  • Longer summaries
  • Product descriptions
  • Detailed mission statements

Microcopy

Definition

  • Fewer than 3 sentences

When to Use

  • To guide users in an interface
  • To quickly communicate a critical point

Examples

  • Link and button labels
  • Form field instructions
  • Input control labels
  • Page titles and meta descriptions
  • Error messages
  • Tooltips

Conclusion

There are so many types of text in digital interfaces that it’s easy to confuse their definitions or forget how they differ. Taking the time to learn or refresh the scopes of UX copy can help writers choose the best approach for the experiences they’re designing.

A study in project failure

Mike's Notes

Here is another study into the cause of IT waste.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

26/05/2025

A study in project failure

By: Dr John McManus and Dr Trevor Wood-Harper
British Computer Society: 06/09/2008

Research highlights that only one in eight information technology projects can be considered truly successful (failure being described as those projects that do not meet the original time, cost and (quality) requirements criteria).

Despite such failures, huge sums continue to be invested in information systems projects and written off. For example the cost of project failure across the European Union was €142 billion in 2004.

The research looked at 214 information systems (IS) projects at the same time, interviews were conducted with a selective number of project managers to follow up issues or clarify points of interest. The period of analysis covered 1998-2005 the number of information systems projects examined across the European Union.

Number of IS projects examined within European Union

Rank Sector No. of projects examined
1 Manufacturing 43
2 Retail 36
3 Financial services 33
4 Transport 27
5 Health 18
6 Education 17
7 Defence 13
8 Construction 12
9 Logistics 9
10 Agriculture 6
Total   214

Project value in millions of Euros

Value range in millions (€) Number of projects Percentage(%) Accumulative (%)
0 – 1 51 23.831 23.831
1 – 2 20 9.346 33.177
2 - 3 11 5.140 38.317
3 - 5 33 15.421 53.738
5 - 10 4 1.869 55.607
10 - 20 87 40.654 96.261
20 - 50 6 2.804 99.065
50 - 80 2 0.935 100.000
Totals 214 100.00 100.00

At what stage in the project lifecycle are projects cancelled (or abandoned as failures)?

Prior research by the authors in 2002 identified that 7 out of 10 software projects undertaken in the UK adopted the waterfall method for software development and delivery. Results from the analysis of cases indicates that almost one in four of the projects examined were abandoned after the feasibility stage of those projects completed approximately one in three were schedule and budget overruns.

Project completions, cancellations and overruns

Waterfall method lifecycle stage Number of projects cancelled Number of projects completed Number of projects overrun (schedule and/or cost)
Feasibility None 214 None
Requirements analysis 3 211 None
Design 28 183 32
Code 15 168 57
Testing 4 164 57
Implementation 1 163 69
Handover None 163 69
Percentages 23.8% 76.2%  

Of the initial 214 projects studied 51 (23.8 per cent were cancelled) - a summary of the principal reasons why projects were cancelled is given below. Our earlier research elaborated on the symptoms of information systems project failure in three specific areas: frequent requests by users to change the system; insufficient communication between the different members of the team working on the project and the end users (stakeholders); and no clear requirements definitions. Whilst communication between team and end users was still perceived as an issue within some projects; the top three issues from this study are: business process alignment; requirements management; and overspends.

One notable causal factor in these abandonment's was the lack of due diligence at the requirements phase, an important factor here was the level of skill in design and poor management judgement in selecting software engineers with the right skill sets. Equally the authors found some evidence in poor tool set selection in that end users found it difficult to sign-off design work - in that they could not relate process and data model output with their reality and practical knowledge of the business processes.

Key reasons why projects get cancelled

  • Business reasons for project failure
  • Business strategy superseded;
  • Business processes change (poor alignment);
  • Poor requirements management;
  • Business benefits not clearly communicated or overstated;
  • Failure of parent company to deliver;
  • Governance issues within the contract;
  • Higher cost of capital;
  • Inability to provide investment capital;
  • Inappropriate disaster recovery;
  • Misuse of financial resources;
  • Overspends in excess of agreed budgets;
  • Poor project board composition;
  • Take-over of client firm;
  • Too big a project portfolio.

Management reasons

  • Ability to adapt to new resource combinations;
  • Differences between management and client;
  • Insufficient risk management;
  • Insufficient end-user management;
  • Insufficient domain knowledge;
  • Insufficient software metrics;
  • Insufficient training of users;
  • Inappropriate procedures and routines;
  • Lack of management judgement;
  • Lack of software development metrics;
  • Loss of key personnel;
  • Managing legacy replacement;
  • Poor vendor management
  • Poor software productivity;
  • Poor communication between stakeholders;
  • Poor contract management;
  • Poor financial management;
  • Project management capability;
  • Poor delegation and decision making;
  • Unfilled promises to users and other stakeholders.

Technical reasons

  • Inappropriate architecture;
  • Insufficient reuse of existing technical objects;
  • Inappropriate testing tools;
  • Inappropriate coding language;
  • Inappropriate technical methodologies;
  • Lack of formal technical standards;
  • Lack of technical innovation (obsolescence);
  • Misstatement of technical risk;
  • Obsolescence of technology;
  • Poor interface specifications;
  • Poor quality code;
  • Poor systems testing;
  • Poor data migration;
  • Poor systems integration;
  • Poor configuration management;
  • Poor change management procedures;
  • Poor technical judgement.

What is the average schedule and budget overrun?

In examining the cases it was noted that the average duration of a project was just over 26 months (115 weeks) and the average budget was approximate 6 million Euros, (Table 5). In many instances information on a project being over schedule and over budget will force senior management to act, however, the search for the underlying factors should begin else where in the projects history.

The pattern that emerges from a synthesis of case data is complex and multifaceted. In a few of the of cases examined the project commentary and history was ambiguous; however, once a decision had been made to support a project which was over schedule or over budget the ends usually justified the means irrespective of the viewpoints of individual project managers or stakeholders.

Cost and schedule overruns (N=69)

Projects From Sample 2 (2) 11 (13) 19 (32) 25 (57) 12 (69)
Schedule Overrun  11 weeks 29 weeks 46 weeks 80 weeks 103 weeks
Range Average  Budget + 10% Average  Budget + 25% Average Budget + 40% Average Budget + 70% Average Budget + 90%
Cost Overrun €600,000 €1,500,000 €2,400,000 €4,200,000 €5,400,000

What are the major causal factors contributing to project failure?

Judgements by project stakeholders about the relative success or failure of projects tend to be made early in the projects life cycle. On examination of the project stage reports it became apparent that many project managers plan for failure rather than success. 

If we consider the inherent complexity of risk associated with software project delivery it is not too surprising that only a small number of projects are delivered to the original time, cost, and quality requirements.

Our evidence suggests that the culture within many organisation's is often such that leadership, stakeholder and risk management issues are not factored into projects early on and in many instances cannot formally be written down for political reasons and are rarely discussed openly at project board or steering group meetings although they may be discussed at length behind closed doors.

Despite attempts to make software development and project delivery more rigorous, a considerable proportion of delivery effort results in systems that do not meet user expectations and are subsequently cancelled. In our view this is attributed to the fact that very few organisation's have the infrastructure, education, training, or management discipline to bring projects to successful completion.

One of the major weaknesses uncovered during the analysis was the total reliance placed on project and development methodologies. One explanation for the reliance on methodology is the absence of leadership within the delivery process. Processes alone are far from enough to cover the complexity and human aspects of many large projects subject to multiple stakeholders, resource and ethical constraints.

Although our understanding of the importance of project failure has increased, the underlying reasons still remain an issue and a point of contention for both practitioners and academics alike. Without doubt there is still a lot to learn from studying project failure.

Going back to the research undertaken there is little evidence that the issues of project failure have been fully addressed within information systems project management. Based on this research project failure requires recognition of the influence multiple stakeholders have on projects, and a broad based view of project leadership and stakeholder management.

Developing an alternative methodology for project management founded on a leadership, stakeholder and risk management should lead to a better understanding of the management issues that may contribute to the successful delivery of information systems projects.

Worldwide cost of IT failure (revisited): $3 trillion

Mike's Notes

Nine years ago, I came across the writings of Roger Sessions about the cause of IT failure waste. I then discovered the writings of Gene Kim et al from IT Revolution. The annual figure for IT failures exceeds US$3 trillion. 

I will locate those references and republish them here.

This waste has become the problem that Pipi was built to help solve.

Resources

References

  • IT Complexity White Paper  by Roger Sessions

Repository

  • Home > Ajabbi Research > Library > Authors > Roger Sessions
  • Home > Handbook > 

Last Updated

25/05/2025

Worldwide cost of IT failure (revisited): $3 trillion

By: Michael Krigsman
ZDNET: 09/04/2012

Michael Krigsman is an industry analyst and the host of CxOTalk, which tells stories of innovation and opportunity with the world's top business and technology leaders. He is a frequent speaker and panel moderator at technology conferences and advises the most successful enterprise companies on marketing and communications strategy. Michael's work has been referenced in the media over 1,000 times and in more than 50 books..

Calculating the global impact of IT failures on an annual basis is a worthy goal. However, aside from the challenge of collecting data, failure itself has no clear definition, subjecting the entire effort to assumptions and guesses. Still, several years ago, one analyst attempted to quantify the data in a noble, yet ultimately flawed, effort.

Also read: Worldwide cost of IT failure: $6.2 trillion Critique: $6.2 trillion global IT failure stats

Related: British Computer Society:A study in project failure

Nonetheless, the challenge of quantification remains. For this reason, I invited two qualified experts to re-assess the worldwide economic impact of IT failure. Gene Kim was the founder and former CTO of Tripwire, Inc., co-author of The Visible Ops Handbook, and a co-author of an upcoming book called When IT Fails: The Novel; his colleague, Mike Orzen wrote the book Lean IT and consults on IT operations and business transformation. They are currently co-authoring a new book called The DevOps Cookbook.

The two experts calculated the global impact of IT failure as being $3 trillion annually. They supplied the following text to explain their logic:

For just the Standard & Poor 500 companies, aggregate 2012 revenue is estimated to be $10 trillion. If 5 percent of aggregate revenue is spent on IT, and conservatively, 20 percent of that spending creates no value for the end customer - that is $100 billion of waste!

(The 20 percent assumption is an extremely conservative number when you consider that when we analyze the value streams of almost all processes across all industries, we discover over 80% of the effort creates no value in terms of benefit to the customer. Mike Orzen's work value stream mapping in many of the largest IT groups around the world consistently shows the same results.)

Here's another approach at calculating the total waste in IT worldwide: Both IDC and Gartner projected that in 2011, five percent of the worldwide gross domestic product will be spent on IT (hardware, services and telecom). In other words, in 2011, approximately $5.6 trillion was spent on IT.

There are two components to IT spend: capital projects and operations/maintenance.  If we assume conservatively that 30 percent of IT spending is for capitalized projects, and that 30 percent of those projects will fail, that's $252 billion of waste!

But IT is like a free puppy -- the lifecycle cost of the puppy is dominated by the "operate/maintain" costs, not the initial acquisition costs. If we conservatively estimate that 50 percent of global IT spend is on "operate/maintain" activities, and that at least 35 percent of that work is urgent, unplanned work or rework, that's $980 billion worldwide of waste!

What reward can we expect through better management, operational excellence and governance of IT?  If we halve the amount of waste, and instead convert it into 5x of value, that would be (50 percent * $1.2 trillion waste * 5x). That's $3 trillion of potential value that we're letting slip through our fingers!

That's a staggering amount of value, 4.7 percent of global GDP, or more than the entire economic output of Germany.

My take: These are the most reasonable numbers I have seen on the global economic impact of IT failures. Unlike previous estimates, which considered complicated matters such as lost opportunity costs, this formula is simple and credible.

Micro frontend

Mike's Notes

Today, I learned this new term from Neo Kim's newsletter. Pipi 9's frontend can be described as a "Micro frontend."

Pipi 6 was designed in 2017 with hundreds of self-contained, domain-driven micro apps.

Pipi 8 refactoring broke the frontend and self-documentation.

Pipi 9 refactoring is slowly restoring self-documentation and the front end.

  • The Render Engine (rnd) pre-assembles each webpage without needing any JavaScript.
  • Using micro-apps greatly simplifies automatic production by the server.
  • The file size of each web page is about 1% of the size of the JavaScript equivalent.
Below is an article from 2019 I found on Martin Fowler's Bliki.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > System Design Newsletter
  • Home > Handbook > 

Last Updated

21/05/2025

Micro frontends

By: Cam Jackson
Martinfowler.com: 19/06/2019

Good frontend development is hard. Scaling frontend development so that many teams can work simultaneously on a large and complex product is even harder. In this article we'll describe a recent trend of breaking up frontend monoliths into many smaller, more manageable pieces, and how this architecture can increase the effectiveness and efficiency of teams working on frontend code. As well as talking about the various benefits and costs, we'll cover some of the implementation options that are available, and we'll dive deep into a full example application that demonstrates the technique.

In recent years, microservices have exploded in popularity, with many organisations using this architectural style to avoid the limitations of large, monolithic backends. While much has been written about this style of building server-side software, many companies continue to struggle with monolithic frontend codebases.

Perhaps you want to build a progressive or responsive web application, but can't find an easy place to start integrating these features into the existing code. Perhaps you want to start using new JavaScript language features (or one of the myriad languages that can compile to JavaScript), but you can't fit the necessary build tools into your existing build process. Or maybe you just want to scale your development so that multiple teams can work on a single product simultaneously, but the coupling and complexity in the existing monolith means that everyone is stepping on each other's toes. These are all real problems that can all negatively affect your ability to efficiently deliver high quality experiences to your customers.

Lately we are seeing more and more attention being paid to the overall architecture and organisational structures that are necessary for complex, modern web development. In particular, we're seeing patterns emerge for decomposing frontend monoliths into smaller, simpler chunks that can be developed, tested and deployed independently, while still appearing to customers as a single cohesive product. We call this technique micro frontends, which we define as:

“An architectural style where independently deliverable frontend applications are composed into a greater whole”

In the November 2016 issue of the Thoughtworks technology radar, we listed micro frontends as a technique that organisations should Assess. We later promoted it into Trial, and finally into Adopt, which means that we see it as a proven approach that you should be using when it makes sense to do so.

A screenshot of micro frontends on the   Thoughtworks tech radar


Figure 1: Micro frontends has appeared on the tech radar several times.

  • Some of the key benefits that we've seen from micro frontends are:
  • smaller, more cohesive and maintainable codebases
  • more scalable organisations with decoupled, autonomous teams
  • the ability to upgrade, update, or even rewrite parts of the frontend in a more incremental fashion than was previously possible
  • It is no coincidence that these headlining advantages are some of the same ones that microservices can provide.

Of course, there are no free lunches when it comes to software architecture - everything comes with a cost. Some micro frontend implementations can lead to duplication of dependencies, increasing the number of bytes our users must download. In addition, the dramatic increase in team autonomy can cause fragmentation in the way your teams work. Nonetheless, we believe that these risks can be managed, and that the benefits of micro frontends often outweigh the costs.

Benefits

Rather than defining micro frontends in terms of specific technical approaches or implementation details, we instead place emphasis on the attributes that emerge and the benefits they give.

Incremental upgrades

For many organisations this is the beginning of their micro frontends journey. The old, large, frontend monolith is being held back by yesteryear's tech stack, or by code written under delivery pressure, and it's getting to the point where a total rewrite is tempting. In order to avoid the perils of a full rewrite, we'd much prefer to strangle the old application piece by piece, and in the meantime continue to deliver new features to our customers without being weighed down by the monolith.

This often leads towards a micro frontends architecture. Once one team has had the experience of getting a feature all the way to production with little modification to the old world, other teams will want to join the new world as well. The existing code still needs to be maintained, and in some cases it may make sense to continue to add new features to it, but now the choice is available.

The endgame here is that we're afforded more freedom to make case-by-case decisions on individual parts of our product, and to make incremental upgrades to our architecture, our dependencies, and our user experience. If there is a major breaking change in our main framework, each micro frontend can be upgraded whenever it makes sense, rather than being forced to stop the world and upgrade everything at once. If we want to experiment with new technology, or new modes of interaction, we can do it in a more isolated fashion than we could before.

Simple, decoupled codebases

The source code for each individual micro frontend will by definition be much smaller than the source code of a single monolithic frontend. These smaller codebases tend to be simpler and easier for developers to work with. In particular, we avoid the complexity arising from unintentional and inappropriate coupling between components that should not know about each other. By drawing thicker lines around the bounded contexts of the application, we make it harder for such accidental coupling to arise.

Of course, a single, high-level architectural decision (i.e. “let's do micro frontends”), is not a substitute for good old fashioned clean code. We're not trying to exempt ourselves from thinking about our code and putting effort into its quality. Instead, we're trying to set ourselves up to fall into the pit of success by making bad decisions hard, and good ones easy. For example, sharing domain models across bounded contexts becomes more difficult, so developers are less likely to do so. Similarly, micro frontends push you to be explicit and deliberate about how data and events flow between different parts of the application, which is something that we should have been doing anyway!

Independent deployment

Just as with microservices, independent deployability of micro frontends is key. This reduces the scope of any given deployment, which in turn reduces the associated risk. Regardless of how or where your frontend code is hosted, each micro frontend should have its own continuous delivery pipeline, which builds, tests and deploys it all the way to production. We should be able to deploy each micro frontend with very little thought given to the current state of other codebases or pipelines. It shouldn't matter if the old monolith is on a fixed, manual, quarterly release cycle, or if the team next door has pushed a half-finished or broken feature into their master branch. If a given micro frontend is ready to go to production, it should be able to do so, and that decision should be up to the team who build and maintain it.

A diagram showing 3 applications independently going from source control, through build, test and deployment to production

Figure 2: Each micro frontend is deployed to production independently

Autonomous teams

As a higher-order benefit of decoupling both our codebases and our release cycles, we get a long way towards having fully independent teams, who can own a section of a product from ideation through to production and beyond. Teams can have full ownership of everything they need to deliver value to customers, which enables them to move quickly and effectively. For this to work, our teams need to be formed around vertical slices of business functionality, rather than around technical capabilities. An easy way to do this is to carve up the product based on what end users will see, so each micro frontend encapsulates a single page of the application, and is owned end-to-end by a single team. This brings higher cohesiveness of the teams' work than if teams were formed around technical or “horizontal” concerns like styling, forms, or validation.

A diagram showing teams formed around 3 applications, and warning against forming a 'styling' team

Figure 3: Each application should be owned by a single team

In a nutshell

In short, micro frontends are all about slicing up big and scary things into smaller, more manageable pieces, and then being explicit about the dependencies between them. Our technology choices, our codebases, our teams, and our release processes should all be able to operate and evolve independently of each other, without excessive coordination.

The example

Imagine a website where customers can order food for delivery. On the surface it's a fairly simple concept, but there's a surprising amount of detail if you want to do it well:

  • There should be a landing page where customers can browse and search for restaurants. The restaurants should be searchable and filterable by any number of attributes including price, cuisine, or what a customer has ordered previously
  • Each restaurant needs its own page that shows its menu items, and allows a customer to choose what they want to eat, with discounts, meal deals, and special requests
  • Customers should have a profile page where they can see their order history, track delivery, and customise their payment options

A wireframe of a food delivery website

Figure 4: A food delivery website may have several reasonably complex pages

There is enough complexity in each page that we could easily justify a dedicated team for each one, and each of those teams should be able to work on their page independently of all the other teams. They should be able to develop, test, deploy, and maintain their code without worrying about conflicts or coordination with other teams. Our customers, however, should still see a single, seamless website.

Throughout the rest of this article, we'll be using this example application wherever we need example code or scenarios.

Integration approaches

Given the fairly loose definition above, there are many approaches that could reasonably be called micro frontends. In this section we'll show some examples and discuss their tradeoffs. There is a fairly natural architecture that emerges across all of the approaches - generally there is a micro frontend for each page in the application, and there is a single container application, which:

  • renders common page elements such as headers and footers
  • addresses cross-cutting concerns like authentication and navigation
  • brings the various micro frontends together onto the page, and tells each micro frontend when and where to render itself

A web page with boxes drawn around different sections. One box wraps the whole page, labelling it as the 'container application'. Another box wraps the main content (but not the global page title and navigation), labelling it as the 'browse micro frontend'

Figure 5: You can usually derive your architecture from the visual structure of the page

Server-side template composition

We start with a decidedly un-novel approach to frontend development - rendering HTML on the server out of multiple templates or fragments. We have an index.html which contains any common page elements, and then uses server-side includes to plug in page-specific content from fragment HTML files:

<html lang="en" dir="ltr">
  <head>
    <meta charset="utf-8">
    <title>Feed me</title>
  </head>
  <body>
    <h1>🍽 Feed me</h1>
    <!--# include file="$PAGE.html" -->
  </body>
</html>

We serve this file using Nginx, configuring the $PAGE variable by matching against the URL that is being requested:

server {
    listen 8080;
    server_name localhost;
    root /usr/share/nginx/html;
    index index.html;
    ssi on;
    # Redirect / to /browse
    rewrite ^/$ http://localhost:8080/browse redirect;
    # Decide which HTML fragment to insert based on the URL
    location /browse {
      set $PAGE 'browse';
    }
    location /order {
      set $PAGE 'order';
    }
    location /profile {
      set $PAGE 'profile'
    }
    # All locations should render through index.html
    error_page 404 /index.html;
}

This is fairly standard server-side composition. The reason we could justifiably call this micro frontends is that we've split up our code in such a way that each piece represents a self-contained domain concept that can be delivered by an independent team. What's not shown here is how those various HTML files end up on the web server, but the assumption is that they each have their own deployment pipeline, which allows us to deploy changes to one page without affecting or thinking about any other page.

For even greater independence, there could be a separate server responsible for rendering and serving each micro frontend, with one server out the front that makes requests to the others. With careful caching of responses, this could be done without impacting latency.

A flow diagram showing a browser making a request to a 'container app server', which then makes requests to one of either a 'browse micro frontend server' or a 'order micro frontend server'

Figure 6: Each of these servers can be built and deployed to independently

This example shows how micro frontends is not necessarily a new technique, and does not have to be complicated. As long as we're careful about how our design decisions affect the autonomy of our codebases and our teams, we can achieve many of the same benefits regardless of our tech stack.

Build-time integration

One approach that we sometimes see is to publish each micro frontend as a package, and have the container application include them all as library dependencies. Here is how the container's package.json might look for our example app:

{
  "name": "@feed-me/container",
  "version": "1.0.0",
  "description": "A food delivery web app",
  "dependencies": {
    "@feed-me/browse-restaurants": "^1.2.3",
    "@feed-me/order-food": "^4.5.6",
    "@feed-me/user-profile": "^7.8.9"
  }
}

At first this seems to make sense. It produces a single deployable Javascript bundle, as is usual, allowing us to de-duplicate common dependencies from our various applications. However, this approach means that we have to re-compile and release every single micro frontend in order to release a change to any individual part of the product. Just as with microservices, we've seen enough pain caused by such a lockstep release process that we would recommend strongly against this kind of approach to micro frontends.

Having gone to all of the trouble of dividing our application into discrete codebases that can be developed and tested independently, let's not re-introduce all of that coupling at the release stage. We should find a way to integrate our micro frontends at run-time, rather than at build-time.

Run-time integration via iframes

One of the simplest approaches to composing applications together in the browser is the humble iframe. By their nature, iframes make it easy to build a page out of independent sub-pages. They also offer a good degree of isolation in terms of styling and global variables not interfering with each other.

<html>
  <head>
    <title>Feed me!</title>
  </head>
  <body>
    <h1>Welcome to Feed me!</h1>
    <iframe id="micro-frontend-container"></iframe>
    <script type="text/javascript">
      const microFrontendsByRoute = {
        '/': 'https://browse.example.com/index.html',
        '/order-food': 'https://order.example.com/index.html',
        '/user-profile': 'https://profile.example.com/index.html',
      };
      const iframe = document.getElementById('micro-frontend-container');
      iframe.src = microFrontendsByRoute[window.location.pathname];
    </script>
  </body>
</html>

Just as with the server-side includes option, building a page out of iframes is not a new technique and perhaps does not seem that exciting. But if we revisit the chief benefits of micro frontends listed earlier, iframes mostly fit the bill, as long as we're careful about how we slice up the application and structure our teams.

We often see a lot of reluctance to choose iframes. While some of that reluctance does seem to be driven by a gut feel that iframes are a bit “yuck”, there are some good reasons that people avoid them. The easy isolation mentioned above does tend to make them less flexible than other options. It can be difficult to build integrations between different parts of the application, so they make routing, history, and deep-linking more complicated, and they present some extra challenges to making your page fully responsive.

Run-time integration via JavaScript

The next approach that we'll describe is probably the most flexible one, and the one that we see teams adopting most frequently. Each micro frontend is included onto the page using a <script> tag, and upon load exposes a global function as its entry-point. The container application then determines which micro frontend should be mounted, and calls the relevant function to tell a micro frontend when and where to render itself.

<html>
  <head>
    <title>Feed me!</title>
  </head>
  <body>
    <h1>Welcome to Feed me!</h1>
    <!-- These scripts don't render anything immediately -->
    <!-- Instead they attach entry-point functions to `window` -->
    <script src="https://browse.example.com/bundle.js"></script>
    <script src="https://order.example.com/bundle.js"></script>
    <script src="https://profile.example.com/bundle.js"></script>
    <div id="micro-frontend-root"></div>
    <script type="text/javascript">
      // These global functions are attached to window by the above scripts
      const microFrontendsByRoute = {
        '/': window.renderBrowseRestaurants,
        '/order-food': window.renderOrderFood,
        '/user-profile': window.renderUserProfile,
      };
      const renderFunction = microFrontendsByRoute[window.location.pathname];
      // Having determined the entry-point function, we now call it,
      // giving it the ID of the element where it should render itself
      renderFunction('micro-frontend-root');
    </script>
  </body>
</html>

The above is obviously a primitive example, but it demonstrates the basic technique. Unlike with build-time integration, we can deploy each of the bundle.js files independently. And unlike with iframes, we have full flexibility to build integrations between our micro frontends however we like. We could extend the above code in many ways, for example to only download each JavaScript bundle as needed, or to pass data in and out when rendering a micro frontend.

The flexibility of this approach, combined with the independent deployability, makes it our default choice, and the one that we've seen in the wild most often. We'll explore it in more detail when we get into the full example.

Run-time integration via Web Components

One variation to the previous approach is for each micro frontend to define an HTML custom element for the container to instantiate, instead of defining a global function for the container to call.

<html>
  <head>
    <title>Feed me!</title>
  </head>
  <body>
    <h1>Welcome to Feed me!</h1>
    <!-- These scripts don't render anything immediately -->
    <!-- Instead they each define a custom element type -->
    <script src="https://browse.example.com/bundle.js"></script>
    <script src="https://order.example.com/bundle.js"></script>
    <script src="https://profile.example.com/bundle.js"></script>
    <div id="micro-frontend-root"></div>
    <script type="text/javascript">
      // These element types are defined by the above scripts
      const webComponentsByRoute = {
        '/': 'micro-frontend-browse-restaurants',
        '/order-food': 'micro-frontend-order-food',
        '/user-profile': 'micro-frontend-user-profile',
      };
      const webComponentType = webComponentsByRoute[window.location.pathname];
      // Having determined the right web component custom element type,
      // we now create an instance of it and attach it to the document
      const root = document.getElementById('micro-frontend-root');
      const webComponent = document.createElement(webComponentType);
      root.appendChild(webComponent);
    </script>
  </body>
</html>

The end result here is quite similar to the previous example, the main difference being that you are opting in to doing things 'the web component way'. If you like the web component spec, and you like the idea of using capabilities that the browser provides, then this is a good option. If you prefer to define your own interface between the container application and micro frontends, then you might prefer the previous example instead.

Styling

CSS as a language is inherently global, inheriting, and cascading, traditionally with no module system, namespacing or encapsulation. Some of those features do exist now, but browser support is often lacking. In a micro frontends landscape, many of these problems are exacerbated. For example, if one team's micro frontend has a stylesheet that says h2 { color: black; }, and another one says h2 { color: blue; }, and both these selectors are attached to the same page, then someone is going to be disappointed! This is not a new problem, but it's made worse by the fact that these selectors were written by different teams at different times, and the code is probably split across separate repositories, making it more difficult to discover.

Over the years, many approaches have been invented to make CSS more manageable. Some choose to use a strict naming convention, such as BEM, to ensure selectors only apply where intended. Others, preferring not to rely on developer discipline alone, use a pre-processor such as SASS, whose selector nesting can be used as a form of namespacing. A newer approach is to apply all styles programatically with CSS modules or one of the various CSS-in-JS libraries, which ensures that styles are directly applied only in the places the developer intends. Or for a more platform-based approach, shadow DOM also offers style isolation.

The approach that you pick does not matter all that much, as long as you find a way to ensure that developers can write their styles independently of each other, and have confidence that their code will behave predictably when composed together into a single application.

Shared component libraries

We mentioned above that visual consistency across micro frontends is important, and one approach to this is to develop a library of shared, re-usable UI components. In general we believe that this a good idea, although it is difficult to do well. The main benefits of creating such a library are reduced effort through re-use of code, and visual consistency. In addition, your component library can serve as a living styleguide, and it can be a great point of collaboration between developers and designers.

One of the easiest things to get wrong is to create too many of these components, too early. It is tempting to create a Foundation Platform, with all of the common visuals that will be needed across all applications. However, experience tells us that it's difficult, if not impossible, to guess what the components' APIs should be before you have real-world usage of them, which results in a lot of churn in the early life of a component. For that reason, we prefer to let teams create their own components within their codebases as they need them, even if that causes some duplication initially. Allow the patterns to emerge naturally, and once the component's API has become obvious, you can harvest the duplicate code into a shared library and be confident that you have something proven.

The most obvious candidates for sharing are “dumb” visual primitives such as icons, labels, and buttons. We can also share more complex components which might contain a significant amount of UI logic, such as an auto-completing, drop-down search field. Or a sortable, filterable, paginated table. However, be careful to ensure that your shared components contain only UI logic, and no business or domain logic. When domain logic is put into a shared library it creates a high degree of coupling across applications, and increases the difficulty of change. So, for example, you usually should not try to share a ProductTable, which would contain all sorts of assumptions about what exactly a “product” is and how one should behave. Such domain modelling and business logic belongs in the application code of the micro frontends, rather than in a shared library.

As with any shared internal library, there are some tricky questions around its ownership and governance. One model is to say that as a shared asset, “everyone” owns it, though in practice this usually means that no one owns it. It can quickly become a hodge-podge of inconsistent code with no clear conventions or technical vision. At the other extreme, if development of the shared library is completely centralised, there will be a big disconnect between the people who create the components and the people who consume them. The best models that we've seen are ones where anyone can contribute to the library, but there is a custodian (a person or a team) who is responsible for ensuring the quality, consistency, and validity of those contributions. The job of maintaining the shared library requires strong technical skills, but also the people skills necessary to cultivate collaboration across many teams.

Cross-application communication

One of the most common questions regarding micro frontends is how to let them talk to each other. In general, we recommend having them communicate as little as possible, as it often reintroduces the sort of inappropriate coupling that we're seeking to avoid in the first place.

That said, some level of cross-app communication is often needed. Custom events allow micro frontends to communicate indirectly, which is a good way to minimise direct coupling, though it does make it harder to determine and enforce the contract that exists between micro frontends. Alternatively, the React model of passing callbacks and data downwards (in this case downwards from the container application to the micro frontends) is also a good solution that makes the contract more explicit. A third alternative is to use the address bar as a communication mechanism, which we'll explore in more detail later.

If you are using redux, the usual approach is to have a single, global, shared store for the entire application. However, if each micro frontend is supposed to be its own self-contained application, then it makes sense for each one to have its own redux store. The redux docs even mention “isolating a Redux app as a component in a bigger application” as a valid reason to have multiple stores.

Whatever approach we choose, we want our micro frontends to communicate by sending messages or events to each other, and avoid having any shared state. Just like sharing a database across microservices, as soon as we share our data structures and domain models, we create massive amounts of coupling, and it becomes extremely difficult to make changes.

As with styling, there are several different approaches that can work well here. The most important thing is to think long and hard about what sort of coupling you're introducing, and how you'll maintain that contract over time. Just as with integration between microservices, you won't be able to make breaking changes to your integrations without having a coordinated upgrade process across different applications and teams.

You should also think about how you'll automatically verify that the integration does not break. Functional testing is one approach, but we prefer to limit the number of functional tests we write due to the cost of implementing and maintaining them. Alternatively you could implement some form of consumer-driven contracts, so that each micro frontend can specify what it requires of other micro frontends, without needing to actually integrate and run them all in a browser together.

Backend communication

If we have separate teams working independently on frontend applications, what about backend development? We believe strongly in the value of full-stack teams, who own their application's development from visual code all the way through to API development, and database and infrastructure code. One pattern that helps here is the BFF pattern, where each frontend application has a corresponding backend whose purpose is solely to serve the needs of that frontend. While the BFF pattern might originally have meant dedicated backends for each frontend channel (web, mobile, etc), it can easily be extended to mean a backend for each micro frontend.

There are a lot of variables to account for here. The BFF might be self contained with its own business logic and database, or it might just be an aggregator of downstream services. If there are downstream services, it may or may not make sense for the team that owns the micro frontend and its BFF, to also own some of those services. If the micro frontend has only one API that it talks to, and that API is fairly stable, then there may not be much value in building a BFF at all. The guiding principle here is that the team building a particular micro frontend shouldn't have to wait for other teams to build things for them. So if every new feature added to a micro frontend also requires backend changes, that's a strong case for a BFF, owned by the same team.

A diagram showing three pairs of frontends / backends. The first backend talks only to its own database. The other two backends talk to shared downstream services. Both approaches are valid.

Figure 7: There are many different ways to structure your frontend/backend relationships

Another common question is, how should the user of a micro frontend application be authenticated and authorised with the server? Obviously our customers should only have to authenticate themselves once, so auth usually falls firmly in the category of cross-cutting concerns that should be owned by the container application. The container probably has some sort of login form, through which we obtain some sort of token. That token would be owned by the container, and can be injected into each micro frontend on initialisation. Finally, the micro frontend can send the token with any request that it makes to the server, and the server can do whatever validation is required.

Testing

We don't see much difference between monolithic frontends and micro frontends when it comes to testing. In general, whatever strategies you are using to test a monolithic frontend can be reproduced across each individual micro frontend. That is, each micro frontend should have its own comprehensive suite of automated tests that ensure the quality and correctness of the code.

The obvious gap would then be integration testing of the various micro frontends with the container application. This can be done using your preferred choice of functional/end-to-end testing tool (such as Selenium or Cypress), but don't take things too far; functional tests should only cover aspects that cannot be tested at a lower level of the Test Pyramid. By that we mean, use unit tests to cover your low-level business logic and rendering logic, and then use functional tests just to validate that the page is assembled correctly. For example, you might load up the fully-integrated application at a particular URL, and assert that the hard-coded title of the relevant micro frontend is present on the page.

If there are user journeys that span across micro frontends, then you could use functional testing to cover those, but keep the functional tests focussed on validating the integration of the frontends, and not the internal business logic of each micro frontend, which should have already been covered by unit tests. As mentioned above, consumer-driven contracts can help to directly specify the interactions that occur between micro frontends without the flakiness of integration environments and functional testing.

The example in detail

Most of the rest of this article will be a detailed explanation of just one way that our example application can be implemented. We'll focus mostly on how the container application and the micro frontends integrate together using JavaScript, as that's probably the most interesting and complex part. You can see the end result deployed live at https://demo.microfrontends.com, and the full source code can be seen on Github.

A screenshot of the 'browse' landing page of the full micro frontends demo application

Figure 8: The 'browse' landing page of the full micro frontends demo application

The demo is all built using React.js, so it's worth calling out that React does not have a monopoly on this architecture. Micro frontends can be implemented with many different tools or frameworks. We chose React here because of its popularity and because of our own familiarity with it.

The container

We'll start with the container, as it's the entry point for our customers. Let's see what we can learn about it from its package.json:

{
  "name": "@micro-frontends-demo/container",
  "description": "Entry point and container for a micro frontends demo",
  "scripts": {
    "start": "PORT=3000 react-app-rewired start",
    "build": "react-app-rewired build",
    "test": "react-app-rewired test"
  },
  "dependencies": {
    "react": "^16.4.0",
    "react-dom": "^16.4.0",
    "react-router-dom": "^4.2.2",
    "react-scripts": "^2.1.8"
  },
  "devDependencies": {
    "enzyme": "^3.3.0",
    "enzyme-adapter-react-16": "^1.1.1",
    "jest-enzyme": "^6.0.2",
    "react-app-rewire-micro-frontends": "^0.0.1",
    "react-app-rewired": "^2.1.1"
  },
  "config-overrides-path": "node_modules/react-app-rewire-micro-frontends"
}

In version 1 of react-scripts it was possible to have multiple applications coexist on a single page without conflicts, but version 2 uses some webpack features that cause errors when two or more apps try to render themselves on the one page. For this reason we use react-app-rewired to override some of the internal webpack config of react-scripts. This fixes those errors, and lets us keep relying on react-scripts to manage our build tooling for us.

From the dependencies on react and react-scripts, we can conclude that it's a React.js application created with create-react-app. More interesting is what's not there: any mention of the micro frontends that we're going to compose together to form our final application. If we were to specify them here as library dependencies, we'd be heading down the path of build-time integration, which as mentioned previously tends to cause problematic coupling in our release cycles.

To see how we select and display a micro frontend, let's look at App.js. We use React Router to match the current URL against a predefined list of routes, and render a corresponding component:

<Switch>
  <Route exact path="/" component={Browse} />
  <Route exact path="/restaurant/:id" component={Restaurant} />
  <Route exact path="/random" render={Random} />
</Switch>

The Random component is not that interesting - it just redirects the page to a randomly selected restaurant URL. The Browse and Restaurant components look like this:

const Browse = ({ history }) => (
  <MicroFrontend history={history} name="Browse" host={browseHost} />
);
const Restaurant = ({ history }) => (
  <MicroFrontend history={history} name="Restaurant" host={restaurantHost} />
);

In both cases, we render a MicroFrontend component. Aside from the history object (which will become important later), we specify the unique name of the application, and the host from which its bundle can be downloaded. This config-driven URL will be something like http://localhost:3001 when running locally, or https://browse.demo.microfrontends.com in production.

Having selected a micro frontend in App.js, now we'll render it in MicroFrontend.js, which is just another React component:

class MicroFrontend extends React.Component {
  render() {
    return <main id={`${this.props.name}-container`} />;
  }
}

This is not the entire class, we'll be seeing more of its methods soon.

When rendering, all we do is put a container element on the page, with an ID that's unique to the micro frontend. This is where we'll tell our micro frontend to render itself. We use React's componentDidMount as the trigger for downloading and mounting the micro frontend:

componentDidMount is a lifecycle method of React components, which is called by the framework just after an instance of our component has been 'mounted' into the DOM for the first time.

class MicroFrontend…
  componentDidMount() {
    const { name, host } = this.props;
    const scriptId = `micro-frontend-script-${name}`;
    if (document.getElementById(scriptId)) {
      this.renderMicroFrontend();
      return;
    }
    fetch(`${host}/asset-manifest.json`)
      .then(res => res.json())
      .then(manifest => {
        const script = document.createElement('script');
        script.id = scriptId;
        script.src = `${host}${manifest['main.js']}`;
        script.onload = this.renderMicroFrontend;
        document.head.appendChild(script);
      });
  }

We have to fetch the script's URL from an asset manifest file, because react-scripts outputs compiled JavaScript files that have hashes in their filename to facilitate caching.

First we check if the relevant script, which has a unique ID, has already been downloaded, in which case we can just render it immediately. If not, we fetch the asset-manifest.json file from the appropriate host, in order to look up the full URL of the main script asset. Once we've set the script's URL, all that's left is to attach it to the document, with an onload handler that renders the micro frontend:

class MicroFrontend…
  renderMicroFrontend = () => {
    const { name, history } = this.props;
    window[`render${name}`](`${name}-container`, history);
    // E.g.: window.renderBrowse('browse-container', history);
  };

In the above code we're calling a global function called something like window.renderBrowse, which was put there by the script that we just downloaded. We pass it the ID of the <main> element where the micro frontend should render itself, and a history object, which we'll explain soon. The signature of this global function is the key contract between the container application and the micro frontends. This is where any communication or integration should happen, so keeping it fairly lightweight makes it easy to maintain, and to add new micro frontends in the future. Whenever we want to do something that would require a change to this code, we should think long and hard about what it means for the coupling of our codebases, and the maintenance of the contract.

There's one final piece, which is handling clean-up. When our MicroFrontend component un-mounts (is removed from the DOM), we want to un-mount the relevant micro frontend too. There is a corresponding global function defined by each micro frontend for this purpose, which we call from the appropriate React lifecycle method:

class MicroFrontend…
  componentWillUnmount() {
    const { name } = this.props;
    window[`unmount${name}`](`${name}-container`);
  }

In terms of its own content, all that the container renders directly is the top-level header and navigation bar of the site, as those are constant across all pages. The CSS for those elements has been written carefully to ensure that it will only style elements within the header, so it shouldn't conflict with any styling code within the micro frontends.

And that's the end of the container application! It's fairly rudimentary, but this gives us a shell that can dynamically download our micro frontends at runtime, and glue them together into something cohesive on a single page. Those micro frontends can be independently deployed all the way to production, without ever making a change to any other micro frontend, or to the container itself.

The micro frontends

The logical place to continue this story is with the global render function we keep referring to. The home page of our application is a filterable list of restaurants, whose entry point looks like this:

import React from 'react';
import ReactDOM from 'react-dom';
import App from './App';
import registerServiceWorker from './registerServiceWorker';
window.renderBrowse = (containerId, history) => {
  ReactDOM.render(<App history={history} />, document.getElementById(containerId));
  registerServiceWorker();
};
window.unmountBrowse = containerId => {
  ReactDOM.unmountComponentAtNode(document.getElementById(containerId));
};

Usually in React.js applications, the call to ReactDOM.render would be at the top-level scope, meaning that as soon as this script file is loaded, it immediately begins rendering into a hard-coded DOM element. For this application, we need to be able control both when and where the rendering happens, so we wrap it in a function that receives the DOM element's ID as a parameter, and we attach that function to the global window object. We can also see the corresponding un-mounting function that is used for clean-up.

While we've already seen how this function is called when the micro frontend is integrated into the whole container application, one of the biggest criteria for success here is that we can develop and run the micro frontends independently. So each micro frontend also has its own index.html with an inline script to render the application in a “standalone” mode, outside of the container:

<html lang="en">
  <head>
    <title>Restaurant order</title>
  </head>
  <body>
    <main id="container"></main>
    <script type="text/javascript">
      window.onload = () => {
        window.renderRestaurant('container');
      };
    </script>
  </body>
</html>

A screenshot of the 'order' page running as a standalone application outside of the container

Figure 9: Each micro frontend can be run as a standalone application outside of the container.

From this point onwards, the micro frontends are mostly just plain old React apps. The 'browse' application fetches the list of restaurants from the backend, provides <input> elements for searching and filtering the restaurants, and renders React Router <Link> elements, which navigate to a specific restaurant. At that point we would switch over to the second, 'order' micro frontend, which renders a single restaurant with its menu.

An architecture diagram that shows the sequence of steps for navigation, as described above

Figure 10: These micro frontends interact only via route changes, not directly

The final thing worth mentioning about our micro frontends is that they both use styled-components for all of their styling. This CSS-in-JS library makes it easy to associate styles with specific components, so we are guaranteed that a micro frontend's styles will not leak out and effect the container, or another micro frontend.

Cross-application communication via routing

We mentioned earlier that cross-application communication should be kept to a minimum. In this example, the only requirement we have is that the browsing page needs to tell the restaurant page which restaurant to load. Here we will see how we can use client-side routing to solve this problem.

All three React applications involved here are using React Router for declarative routing, but initialised in two slightly different ways. For the container application, we create a <BrowserRouter>, which internally will instantiate a history object. This is the same history object that we've been glossing over previously. We use this object to manipulate the client-side history, and we can also use it to link multiple React Routers together. Inside our micro frontends, we initialise the Router like this:

<Router history={this.props.history}>

In this case, rather than letting React Router instantiate another history object, we provide it with the instance that was passed in by the container application. All of the <Router> instances are now connected, so route changes triggered in any of them will be reflected in all of them. This gives us an easy way to pass “parameters” from one micro frontend to another, via the URL. For example in the browse micro frontend, we have a link like this:

<Link to={`/restaurant/${restaurant.id}`}>

When this link is clicked, the route will be updated in the container, which will see the new URL and determine that the restaurant micro frontend should be mounted and rendered. That micro frontend's own routing logic will then extract the restaurant ID from the URL and render the right information.

Hopefully this example flow shows the flexibility and power of the humble URL. Aside from being useful for sharing and bookmarking, in this particular architecture it can be a useful way to communicate intent across micro frontends. Using the page URL for this purpose ticks many boxes:

  • Its structure is a well-defined, open standard
  • It's globally accessible to any code on the page
  • Its limited size encourages sending only a small amount of data
  • It's user-facing, which encourages a structure that models the domain faithfully
  • It's declarative, not imperative. I.e. “this is where we are”, rather than “please do this thing”
  • It forces micro frontends to communicate indirectly, and not know about or depend on each other directly

When using routing as our mode of communication between micro frontends, the routes that we choose constitute a contract. In this case, we've set in stone the idea that a restaurant can be viewed at /restaurant/:restaurantId, and we can't change that route without updating all applications that refer to it. Given the importance of this contract, we should have automated tests that check that the contract is being adhered to.

Common content

While we want our teams and our micro frontends to be as independent as possible, there are some things that should be common. We wrote earlier about how shared component libraries can help with consistency across micro frontends, but for this small demo a component library would be overkill. So instead, we have a small repository of common content, including images, JSON data, and CSS, which are served over the network to all micro frontends.

There's another thing that we can choose to share across micro frontends: library dependencies. As we will describe shortly, duplication of dependencies is a common drawback of micro frontends. Even though sharing those dependencies across applications comes with its own set of difficulties, for this demo application it's worth talking about how it can be done.

The first step is to choose which dependencies to share. A quick analysis of our compiled code showed that about 50% of the bundles was contributed by react and react-dom. In addition to their size, these two libraries are our most 'core' dependencies, so we know that all micro frontends can benefit from having them extracted. Finally, these are stable, mature libraries, which usually introduce breaking changes across two major versions, so cross-application upgrade efforts should not be too difficult.

As for the actual extraction, all we need to do is mark the libraries as externals in our webpack config, which we can do with a rewiring similar to the one described earlier.

module.exports = (config, env) => {
  config.externals = {
    react: 'React',
    'react-dom': 'ReactDOM'
  }
  return config;
};

Then we add a couple of script tags to each index.html file, to fetch the two libraries from our shared content server.

<body>
  <noscript>
    You need to enable JavaScript to run this app.
  </noscript>
  <div id="root"></div>
  <script src="%REACT_APP_CONTENT_HOST%/react.prod-16.8.6.min.js"></script>
  <script src="%REACT_APP_CONTENT_HOST%/react-dom.prod-16.8.6.min.js"></script>
</body>

Sharing code across teams is always a tricky thing to do well. We need to ensure that we only share things that we genuinely want to be common, and that we want to change in multiple places at once. However, if we're careful about what we do share and what we don't, then there are real benefits to be gained.

Infrastructure

The application is hosted on AWS, with core infrastructure (S3 buckets, CloudFront distributions, domains, certificates, etc), provisioned all at once using a centralised repository of Terraform code. Each micro frontend then has its own source repository with its own continuous deployment pipeline on Travis CI, which builds, tests, and deploys its static assets into those S3 buckets. This balances the convenience of centralised infrastructure management with the flexibility of independent deployability.

Note that each micro frontend (and the container) gets its own bucket. This means that it has free reign over what goes in there, and we don't need to worry about object name collisions, or conflicting access management rules, from another team or application.

Downsides

At the start of this article, we mentioned that there are tradeoffs with micro frontends, as there are with any architecture. The benefits that we've mentioned do come with a cost, which we'll cover here.

Payload size

Independently-built JavaScript bundles can cause duplication of common dependencies, increasing the number of bytes we have to send over the network to our end users. For example, if every micro frontend includes its own copy of React, then we're forcing our customers to download React n times. There is a direct relationship between page performance and user engagement/conversion, and much of the world runs on internet infrastructure much slower than those in highly-developed cities are used to, so we have many reasons to care about download sizes.

This issue is not easy to solve. There is an inherent tension between our desire to let teams compile their applications independently so that they can work autonomously, and our desire to build our applications in such a way that they can share common dependencies. One approach is to externalise common dependencies from our compiled bundles, as we described for the demo application. As soon as we go down this path though, we've re-introduced some build-time coupling to our micro frontends. Now there is an implicit contract between them which says, “we all must use these exact versions of these dependencies”. If there is a breaking change in a dependency, we might end up needing a big coordinated upgrade effort and a one-off lockstep release event. This is everything we were trying to avoid with micro frontends in the first place!

This inherent tension is a difficult one, but it's not all bad news. Firstly, even if we choose to do nothing about duplicate dependencies, it's possible that each individual page will still load faster than if we had built a single monolithic frontend. The reason is that by compiling each page independently, we have effectively implemented our own form of code splitting. In classic monoliths, when any page in the application is loaded, we often download the source code and dependencies of every page all at once. By building independently, any single page-load will only download the source and dependencies of that page. This may result in faster initial page-loads, but slower subsequent navigation as users are forced to re-download the same dependencies on each page. If we are disciplined in not bloating our micro frontends with unnecessary dependencies, or if we know that users generally stick to just one or two pages within the application, we may well achieve a net gain in performance terms, even with duplicated dependencies.

There are lots of “may’s” and “possibly’s” in the previous paragraph, which highlights the fact that every application will always have its own unique performance characteristics. If you want to know for sure what the performance impacts will be of a particular change, there is no substitute for taking real-world measurements, ideally in production. We've seen teams agonise over a few extra kilobytes of JavaScript, only to go and download many megabytes of high-resolution images, or run expensive queries against a very slow database. So while it's important to consider the performance impacts of every architectural decision, be sure that you know where the real bottlenecks are.

Environment differences

We should be able to develop a single micro frontend without needing to think about all of the other micro frontends being developed by other teams. We may even be able to run our micro frontend in a “standalone” mode, on a blank page, rather than inside the container application that will house it in production. This can make development a lot simpler, especially when the real container is a complex, legacy codebase, which is often the case when we're using micro frontends to do a gradual migration from old world to new. However, there are risks associated with developing in an environment that is quite different to production. If our development-time container behaves differently than the production one, then we may find that our micro frontend is broken, or behaves differently when we deploy to production. Of particular concern are global styles that may be brought along by the container, or by other micro frontends.

The solution here is not that different to any other situation where we have to worry about environmental differences. If we're developing locally in an environment that is not production-like, we need to ensure that we regularly integrate and deploy our micro frontend to environments that are like production, and we should do testing (manual and automated) in these environments to catch integration issues as early as possible. This will not completely solve the problem, but ultimately it's another tradeoff that we have to weigh up: is the productivity boost of a simplified development environment worth the risk of integration issues? The answer will depend on the project!

Operational and governance complexity

The final downside is one with a direct parallel to microservices. As a more distributed architecture, micro frontends will inevitably lead to having more stuff to manage - more repositories, more tools, more build/deploy pipelines, more servers, more domains, etc. So before adopting such an architecture there are a few questions you should consider:

  • Do you have enough automation in place to feasibly provision and manage the additional required infrastructure?
  • Will your frontend development, testing, and release processes scale to many applications?
  • Are you comfortable with decisions around tooling and development practices becoming more decentralised and less controllable?
  • How will you ensure a minimum level of quality, consistency, or governance across your many independent frontend codebases?

We could probably fill another entire article discussing these topics. The main point we wish to make is that when you choose micro frontends, by definition you are opting to create many small things rather than one large thing. You should consider whether you have the technical and organisational maturity required to adopt such an approach without creating chaos.

Conclusion

As frontend codebases continue to get more complex over the years, we see a growing need for more scalable architectures. We need to be able to draw clear boundaries that establish the right levels of coupling and cohesion between technical and domain entities. We should be able to scale software delivery across independent, autonomous teams.

While far from the only approach, we have seen many real-world cases where micro frontends deliver these benefits, and we've been able to apply the technique gradually over time to legacy codebases as well as new ones. Whether micro frontends are the right approach for you and your organiation or not, we can only hope that this will be part of a continuing trend where frontend engineering and architecture is treated with the seriousness that we know it deserves.

Acknowledgements

  • Huge thanks to Charles Korn, Andy Marks, and Willem Van Ketwich for their thorough reviews and detailed feedback.
  • Thanks also to Bill Codding, Michael Strasser, and Shirish Padalkar for their input given on the Thoughtworks internal mailing list.
  • Thanks to Martin Fowler for his feedback as well, and for giving this this article a home here on his website.
  • And finally, thanks to Evan Bottcher and Liauw Fendy for their encouragement and support.