What We Know About the NPM Supply Chain Attack

Mike's Notes

Lesson: don't rely on anything. This article was recommended in Amazing CTO. Thanks, Stephan.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Amazing CTO
  • Home > Handbook > 

Last Updated

28/10/2025

What We Know About the NPM Supply Chain Attack

By: Jeffrey Francis Bonaobra, Joshua Aquino
Trend Micro: 18/09/2025

Mike is the inventor and architect of Pipi and the founder of Ajabbi.

Key takeaways

  • Attackers reportedly launched a targeted phishing campaign to compromise Node Package Manager (NPM) maintainer accounts and inject malicious code into widely used JavaScript packages.
  • Certain malicious packages covertly diverted cryptocurrency assets by hijacking web APIs and manipulating network traffic.
  • One of the attack’s payloads is the Shai-hulud worm, which is delivered through compromised packages, steals cloud service tokens, deploys secret-scanning tools, and spreads to additional accounts.
  • Based on Trend Micro’s telemetry, organizations across North America and Europe have been among the most affected by one of the payloads, Cryptohijacker. There have been no detections of the Shai-Hulud worm so far.
  • Trend Vision One™ detects and blocks the indicators of compromise (IOCs) outlined in this blog, and provides customers with tailored threat hunting queries, threat insights, and intelligence reports.
  • On September 15, the Node Package Manager (NPM) repository experienced an ongoing supply chain attack, in which the attackers executed a highly targeted phishing campaign to compromise the account of an NPM package maintainer. With privileged access, the attackers injected malicious code into widely used JavaScript packages, threatening the entire software ecosystem. Notably, the attack has disrupted several key NPM packages, including those integral to application development and cryptography.

According to StepSecurity, the malicious actors behind this incident used similar techniques with the Nx supply chain attack last month. As of September 16, researchers at Socket have already identified close to 500 impacted NPM packages.

In this blog entry, Trend™ Research details an overview of the recent NPM ecosystem compromises, what SOC teams need to know, and security recommendations to avoid this threat.

What types of packages are at risk

The malicious modifications were made to critical JavaScript libraries, including those supporting development frameworks and cryptographic functions. Packages impacted by this attack are those with extremely high global download rates – over 2.6 billion per week – affecting a vast ecosystem of web applications and dependent projects.

Attackers stole cryptocurrency assets

The attackers hijacked web APIs and manipulated network traffic as a means of covertly diverting funds from legitimate channels to wallets they controlled, targeting both organizations and end-users interacting with compromised packages.

Shai-hulud attack chain analysis

One of the payloads is a self-replicating worm, dubbed Shai-hulud after the sandworm in Dune, that was detected in the NPM registry.  Trend Research provides analysis of Shai Hulud, its operational mechanics, and its implications for organizations relying on NPM.

Shai-Hulud stands out for its autonomous replication capability. Instead of a mere infection, Shai-Hulud introduces worm-like propagation, continuously seeking out and compromising additional packages and environments.

Attack chain

The Shai-Hulud attack chain began with a phishing email disguised as an NPM security alert, tricking a developer into revealing credentials (Figure 1). Attackers compromised the developer’s NPM account and uploaded a malicious package. When installed, this package executed JavaScript and embedded Unix shell scripts to establish persistence and start stealing information.

Using stolen GitHub access tokens, the malware authenticated to the GitHub API, checked user permissions, and listed all repositories the victim could access – including private ones. It cloned private repositories to attacker accounts, created a new branch in each, and deployed a malicious workflow to automate data theft.

Next, the malware downloaded and installed TruffleHog to scan for and harvest more secrets from files. It made all stolen repositories public and mirrored their entire history. Sensitive data was then exfiltrated to the attacker using automated web requests.

This chain shows how a single compromised account can lead to the spread of malicious code, credential theft, and mass data leakage across an organization’s entire development environment.

Figure 1. Observed attack chain and deployment of Shai-Hulud from compromised NPM package

The widespread exposure of this threat means that hundreds of packages could have been compromised before initial detection, undermining organizations’ trust in adopting open-source dependencies. The scalability of the attack, enabled by automation, significantly increases both technical and business risks, requiring minimal effort from the attacker once deployed. 

What makes Shai-Hulud distinctive?

Traditional software supply chain threats typically involve single-use payloads or targeted credential theft. Shai-Hulud distinguishes itself through its ability to self-replicate within the NPM ecosystem, using available functionality in post-install scripts to establish secondary and tertiary infections. Once a compromised package is installed, the worm automatically attempts to spread to new targets, creating a multiplying threat that does not rely on human actor intervention after initial deployment.

Key traits:

  • Self-propagation - Shai-Hulud behaves as a worm, automatically infecting additional NPM packages and projects by leveraging existing trust relationships in the open-source community.
  • Autonomy -  The malware runs without direct ongoing operator input, making it more persistent and difficult to contain.
  • Environmental impact - By embedding itself deeply within development and CI/CD environments, Shai-Hulud gains potential access to further credentials, tokens, and sensitive build secrets.

Technical methodology

  • Post-install abuse - The core propagation mechanism centers around malicious post-install scripts. When an infected package is deployed, arbitrary code executes, which may download further payloads or inject malicious scripts into other projects and dependencies.
  • Network activity - The worm can communicate with remote servers to exfiltrate data or receive updates, thus evolving even after initial deployment.
  • Recursive threat vector - Shai-Hulud is engineered for persistence—not just compromising a project once, but remaining a latent risk as dependencies update.

Risk to NPM and open source

The core strength and risk of NPM lies in its vast network of community-driven packages. Shai- Hulud's self-replicating worm design specifically targets this community trust, highlighting how quickly a single malicious actor can impact a disproportionately large segment of developers and software projects.

Shai-Hulud analyis

Malicious workflow injection analysis

The Shai-Hulud worm utilizes an advanced technique by injecting malicious GitHub Actions workflows into targeted repositories, enabling automated propagation and secret exfiltration across an organization’s development environment.

Upon execution, Shai-Hulud prepares the following:

  • Assigns a branch name such as shai-hulud to maintain consistency and help track infections across repositories.
  • Targets .github/workflows/shai-hulud-workflow.yml for the placement of its malicious workflow file.
  • Generates or fetches a YAML workflow file containing the malicious automation payload.

The primary function of the injected workflow, as shown in Figure 2, is to systematically collect and exfiltrate repository secrets:

  • The workflow enumerates all secrets exposed during its CI/CD runtime environment.
  • It packages these secrets into a payload formatted for transmission.
  • Secrets are sent via HTTP(S) requests to attacker-controlled webhook endpoints.

Figure 2. Secret exfiltration mechanism

Shai-Hulud also leverages GitHub’s REST API to automate its lateral movement and establish persistence (Figure 3). The worm checks the validity and permissions of available GitHub authentication tokens to confirm the ability to interact with the API.

Figure 3. GitHub API exploitation

By issuing API requests such as

/user/repos?affiliation=owner,collaborator,organization_member&since=2025-01-01T00:00:00Z&per_page=100

the worm identifies repositories where the compromised account has adequate privileges, filtering by owner, collaborator, or organization member roles and focusing on recent activity (Figure 4).

Figure 4. Github repository discovery

For each eligible repository, the Shai-Hulud worm carries out:

  • Branch creation. It creates a uniquely named branch (e.g., shai-hulud) in the repository to house the injected workflow and isolate malicious changes (Figure 5).

Figure 5. Automated branch creation

  • Workflow file upload. The worm uploads the malicious YAML file to the new branch, setting up ongoing automated secret exfiltration whenever workflows are triggered (Figure 6).

Figure 6. Automated workflow file upload

GitHub repository cloning analysis

Shai-Hulud’s attack chain features an automated process for cloning, migrating, and exposing private GitHub repositories from an organization to an attacker’s infrastructure. The following section outlines the programmatic stages of this cloning activity.

The main orchestration logic coordinates the full cloning cycle – from initialization through repository creation and exposure (Figure 7).

Figure 7. Main function logic for repository cloning

The worm iterates through all identified private repositories within a target organization, utilizing internal logic to ensure each repository is analyzed and handled (Figure 8).

Figure 8. Processing discovered private repositories

Initial checks confirm the presence and validity of required inputs – such as organization name, target username, and GitHub authentication token – to ensure both API compliance and workflow reliability (Figure 9).

Figure 9. Input validation and variable initialization

API interactions are abstracted behind a standardized communication wrapper, responsible for managing authentication (via bearer tokens or OAuth apps) and handling HTTP GET, POST, PUT, and PATCH methods for robust error handling (Figure 10).

Figure 10. Standard API communication wrapper

The process targets only private or internal repositories to maximize stealth and impact. API pagination is implemented to enumerate all repositories within large organizations efficiently (Figure 11).

Figure 11. Repository discovery

For every discovered repository, the worm creates a corresponding destination repository in the attacker’s account – embedding an identifier in the repository description such as “Shai-Hulud Migration” for tracking (Figure 12).

Figure 12. Repository creation under attacker control

Once created, what was a private repository in the victim’s organization is made public under the attacker’s control, facilitating mass data exposure and fingerprinting (Figure 13).

Figure 13. Converting stolen repository to public

To maximize the value of the theft, the worm performs a full mirror clone, capturing not just code contents but also the entire commit and branch history for later exploitation or secondary attacks (Figure 14).

Figure 14. Complete mirror cloning

Through these automated mechanisms, Shai-Hulud rapidly exfiltrates high-sensitivity intellectual property and source code from private repositories, weaponizing it for further data exposure, ransom, or downstream supply chain threats.

Credential harvesting via TruffleHog

As part of its post-compromise activities, Shai-Hulud leverages TruffleHog to further automate credential and secret discovery on compromised environments. The workflow begins by obtaining the latest release of the TruffleHog binary, programmatically retrieving the most recent version available for download (Figure 15).

Figure 15. Retrieving the latest TruffleHog release

Once the appropriate TruffleHog file is identified, the worm downloads the binary, automatically detecting and extracting the correct version based on the operating system present on the victim’s machine (Figure 16).

Figure 16. Downloading and extracting the TruffleHog binary

After extraction, TruffleHog is installed or placed into the environment, making it readily available for use by the malicious workflow (Figures 17 and 18).

Figure 17. TruffleHog installation and environment preparation along with automated secrets scanning and cleanup

Figure 18. TruffleHog installation and environment preparation along with automated secrets scanning and cleanup

The malware then spawns a child process, invoking TruffleHog to scan the local filesystem or target repository contents for high-entropy strings, keys, and other sensitive secrets. This process is conducted in-memory or within a runtime context to evade persistent detection. Once scanning is complete, the TruffleHog binary is deleted to cover tracks and minimize forensic artifacts.

By integrating TruffleHog in this automated fashion, Shai-Hulud markedly increases the volume and quality of exfiltrated secrets, while maintaining operational stealth throughout its attack lifecycle.

Who has been affected so far

Based on Trend's telemetry, attacks involving the Cryptohijacker payload have been reported across various countries, but primarily in North America and Europe. Organizations and developers that depend on widely adopted JavaScript libraries are among those most impacted. However, there have been no detections of the Shai-Hulud worm so far.

Security recommendations

To safeguard their development workflows and sensitive assets from the risks stemming from the ongoing NPM supply chain attack, organizations should prioritize a proactive security stance through the following best practices: 

  • Audit dependencies, focusing on recently updated packages. Review all dependencies, especially those recently modified, and remove or roll back any that appear compromised.
  • Revoke and rotate credentials, especially for NPM accounts. Immediately revoke and replace any credentials or API keys that may have been exposed, prioritizing sensitive accounts.
  • Monitor for evidence of Trufflehog and similar scanning tools in use. Check logs for any anomalous repository scanning activity and proactively scan your own codebase for exposed secrets.
  • Stay updated with advisories from the official NPM registry and trusted sources. Regularly monitor official advisories to apply the latest fixes and recommended actions promptly.
  • Tighten access and security policies. For example, apply the principle of least privilege for all accounts impacting repositories and automation. In addition, enforce multi-factor authentication (MFA) on all developer and CI/CD access points.

Workspace MVP

Mike's Notes

A progress report. It's going rather well.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

27/10/2025

Workspace MVP

By: Mike Peters
On a Sandy Beach: 27/10/2025

Mike is the inventor and architect of Pipi and the founder of Ajabbi.

The workspace UI experimentation and learning have been rapid. As a result, each workspace mockup is replaced with a new, improved version most days.

Static mockup

The HTML table-based mockup is static, incomplete, lacks CSS, and contains made-up information (an MVP). But it has been good enough to learn very fast. This version uses iFrames and is in English.

Additional versions will address accessibility and device/screen size requirements.

URL naming pattern

Previous On a Sandy Beach posts discussed and then described the URL naming pattern. That has worked out.

That has meant naming files and directories using rules.

That is working, with more to figure out.

Deep nesting

The Workspaces contain modules that can be reused, renamed and deeply nested. This needs to happen automatically without causing name collisions. So repeatable and unique URLs. So far, I have gone 3 module levels deep.

Will try a 7-8-level-deep experiment to make sure it's safe.

Groundhog day

3,700 web pages are uploaded every day, replacing the previous 3,700 HTML pages as errors are corrected and features are added.

Names

The names and descriptions of workspaces and modules have been reworked and made consistent across the documentation and workspaces.

URL consistency

Pattern-driven URL links must always work across workspaces, modules, settings, help, menus, how-to-guides, tutorials, and reference.

More work is needed to name and structure tutorials and reference information at Ajabbi Learn. URL link stability will take time to establish.

What's next

More of the above till all the problems disappear and it becomes 100% reliable. 

Then build a demo workspace for a fictitious organisation. eg "West Klingon Hospital", "Valhalla Rail Corp", "Red Baron Airport", "Gotham City Water, "Minas Tirith Power". And users like "Dr Who", "Kermit", "Jabba the Hutt", "Galadrial", etc.

Lots of testing and feedback while having some fun.

Then make the workspaces "live" and usable for those with accounts.

Thinking Like a Data Engineer

Mike's Notes

Great read here. It takes humility to write this. Thanks, Ananth.

Data Engineering Weekly is an excellent newsletter to subscribe to.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Data Engineering Weekly
  • Home > Handbook > 

Last Updated

30/10/2025

Thinking Like a Data Engineer

By: Ananth Packkildurai
Data Engineering Weekly: 23/10/2025

Ananth Packkildurai is a data engineering leader, writer, and author of Data Engineering Weekly, sharing insights on modern data platforms, large-scale pipelines, and AI-driven architectures.

I thought becoming a data engineer meant mastering tools. Instead, it meant learning how to see. I thought the hardest part would be learning the tools — Hadoop, Spark, SQL optimization, and distributed processing. Over time, I realized the real challenge wasn’t technical. It was learning how to think.

Learning to think like a data engineer — to see patterns in chaos, to connect systems to human behavior, to balance simplicity and scale — is a slow process of unlearning, observing, and reimagining. I didn’t get there through courses or certifications. I got there through people.

Four mentors, in four different moments of my life, unknowingly gave me lessons that shaped how I approach engineering, leadership, and even life. Each taught me something not about data, but about thinking systems.

What follows isn’t a tutorial. It’s a map of how four people — and their lessons — rewired how I think.

#1. “Chasing Knowledge.”

One of my friends recently asked why you are constantly reading and writing. It all started with an internship. A family friend of mine helped me find an internship. He was the person who taught me Java — patiently explaining not just syntax, but how to think through logic, abstraction, and design.

When I called him after getting my first full-time job, I expected congratulations or career advice. Instead, he said something that I only understood years later:

Don’t chase money. Chase knowledge. Money will follow.”

The advice struck a chord with me forever. In technology, everything changes — languages, frameworks, stacks, even paradigms. But curiosity compounds. The more you learn, the faster you learn. The more you focus on mastering fundamentals, the easier it becomes to adapt when the next wave arrives.

That advice became a quiet compass throughout my career. Every time I faced a decision — whether to take a higher-paying role or a role that stretched my skills — I thought back to his words. And every single time I chose learning, the money eventually caught up.

It taught me that data engineering is not a sprint to expertise, but a lifelong apprenticeship in curiosity.

#2. “Modeling the World.”

It was in my early days of career as a data engineer that I met another mentor — an industry veteran, family friend, and one of the most grounded data people I’ve ever known.

One day, I asked him a question I thought was simple:

“How do I become a data engineer?”

He didn’t answer directly. He just said, “Go to Walmart and tell me what you see.”

I didn’t get it. But I went.

I walked through aisles, looked at shelves, and bought a few things — shampoo, snacks, and batteries. I came back and said, “Okay, I went. I bought x, y, z. Now what?”

He smiled and said, “Now tell me — how would Walmart model this data?”

That’s when I stopped seeing stores — and started seeing systems. I started describing the data model — a shelves table, a products table, a users table, and an events table. I started seeing not a store, but a database. Every product placement was a joint. Every checkout was an event stream.

That exercise reshaped how I saw the world. Data engineering wasn’t about ETL jobs or pipelines. It was about modeling reality.

When you can take something as ordinary as a store and translate it into entities, relationships, and flows, you start to think like a data engineer.

That mental model helped me immensely later. During one of my interviews at Slack, I was asked to design a data warehouse for Netflix. I didn’t start from schemas or metrics. I started by thinking: what’s the store? What are the products? What are the customers?

That grounding in observation and modeling helped me move fast and think clearly.

That day at Walmart was my real introduction to data modeling — not through theory, but through the physical world.

#3. “Thinking in Systems.”

When I started my career in the U.S., I worked on a loyalty analytics system — one of my first end-to-end data design projects.

I spent weeks sketching, refining, changing, and rebuilding the system. Every week, it looked different. One day, as I stepped into the elevator, my boss turned to me and said,

“I saw your design. You changed it completely from last week. That’s good. It means you’re thinking. Keep it going.”

That short elevator ride changed the way I viewed iteration. Until then, I thought redesigning meant I’d made a mistake. But his words reframed it — evolution is the process of engineering.

I started seeing every design review not as a checkpoint, but as a conversation.

Every new diagram was not a failure of the previous one, but an iteration toward clarity.

As we worked together more, he became my go-to mentor — not just for technical guidance, but for how to think. One day, over coffee, I asked him a question that had been on my mind:

“How do you cultivate architectural thinking?”

He smiled and gave me an unusual exercise.

He said, “Every day you walk from Montgomery Street to the Caltrain. It’s what — fifteen minutes? During that walk, observe how the city works. Watch how people who don’t know each other coordinate. Notice how traffic lights, signs, and signals function without central control. You’ll start to see the same patterns in our system design from emergent behaviour to the leader-follower model.”

It sounded odd, but I did it.

Each evening, I walked through downtown San Francisco — watching the rhythm of people, cars, and crosswalks. I noticed how one signal turning green in one intersection triggered waves of motion down the street. I saw bottlenecks at corners, retries when people crossed late, failovers when lights malfunctioned, and officers took control.

And suddenly, distributed systems weren’t abstract anymore. They were everywhere.

Humans, too, were part of a system — loosely coordinated, mostly independent, but connected through protocols, feedback, and flow.

That was the moment I truly understood system thinking — the ability to look beyond components and see interactions, dependencies, and evolution.

It also taught me humility: a good architect doesn’t impose order; they design for emergence. They build for change, not for perfection.

To this day, when I review a data platform design or a data flow diagram, I still imagine that walk from Montgomery Street to Caltrain. Systems are just cities — made of processes instead of people. Once you see it, you can’t unsee it.

#4. “Believing You Belong”

After I moved to the USA, I spent more time doubting myself than writing code.

Every meeting felt like an exam I hadn’t studied for. Every question felt like a test of belonging. I carried a quiet imposter syndrome — that whisper in your head that says you just got lucky.

One afternoon, my boss noticed my hesitation during a design review. After the meeting, he stopped me and said something simple that stayed with me forever:

“If you’re sitting here, you’re already good enough.”

It was a short sentence, but it landed deeply. There was no lecture, no performance review, no pep talk — just a fact. If you’re in the room, it’s because you earned it.

That moment changed how I carried myself. I realized confidence doesn’t come from knowing everything; it comes from recognizing that you’ve earned your seat — and you can grow from there.

I’ve repeated that same line to dozens of people since then — junior engineers, interns, even peers who were struggling to see their worth. Because the truth is, the data world can be intimidating. There’s always a new framework, a new paper, a new “modern” stack. You’ll never feel fully caught up.

But you don’t need to.

You need to keep showing up, keep learning, and keep thinking.

That sentence became the foundation for something else — it taught me that data engineering, at its core, is a confidence game. You’re constantly making decisions under uncertainty: what schema to use, how to partition data, how to handle scale. Doubt will paralyze you faster than a bad design.

Good engineers are not fearless; they just keep thinking despite fear.

Looking Back

When I look back now, I realize that everything I learned about data engineering started with a mindset shift, not a technical one.

  • Don’t chase money — chase knowledge. → Focus on curiosity; mastery compounds faster than rewards.
  • You’re thinking — keep it going. → Iterate relentlessly and observe systems beyond code.
  • Go to Walmart. → Learn to model the world in data.
  • If you’re sitting here, you’re good enough. → Believe in your ability to learn and grow.

The tools, frameworks, and clouds will keep changing. The mental models won’t.

Those mentors taught me that thinking like a data engineer is not about syntax or pipelines; it’s about curiosity, observation, and humility.

It’s about realizing that the best engineers don’t just build systems — they listen to them.

They watch how systems behave, evolve, and sometimes break — and they design with that reality in mind.

These lessons stayed with me — not as quotes to remember, but as ways of seeing the world.

So, if you’re early in your career or doubting your place in the data world, remember this:

You don’t need to know everything to think like a data engineer. You just need to keep observing, iterating, and thinking.

Because in the end, data engineering is less about moving data — and more about understanding how the world moves.

Closing Thoughts

If I had to summarize years of learning into one sentence, it would be this:

“Thinking like a data engineer is not about data. It’s about systems — human, technical, and everything in between.”

The world around you is the best classroom. Every store, street, and signal is a distributed system waiting to be understood. Every mistake is a feedback loop waiting to be improved. Every mentor is a node in your network of thought.

Keep learning. Keep thinking. And most importantly — keep observing.

Once you start seeing the world as data, you realize you've always been engineering it.

Structured Procrastination

Mike's Notes

Bravo!

This is kind of familiar. Go visit John's blog, it's cool.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Amazing CTO
  • Home > Handbook > 

Last Updated

25/10/2025

Structured Procrastination

By: John Perry
Structured Procrastination: ?

John Perry is an emeritus professor of philosophy at Stanford. His office is Cordura 127 at CSLI, Stanford University, where he conducts several research projects. He is also a professor of philosophy (half-time) at the University of California, Riverside, on leave 2012–13.


Author practices jumping rope with seaweed while work awaits.

``. . . anyone can do any amount of work, provided it isn't the work he is supposed to be doing at that moment." -- Robert Benchley, in Chips off the Old Benchley, 1949

I have been intending to write this essay for months. Why am I finally doing it? Because I finally found some uncommitted time? Wrong. I have papers to grade, textbook orders to fill out, an NSF proposal to referee, dissertation drafts to read. I am working on this essay as a way of not doing all of those things. This is the essence of what I call structured procrastination, an amazing strategy I have discovered that converts procrastinators into effective human beings, respected and admired for all that they can accomplish and the good use they make of time. All procrastinators put off things they have to do. Structured procrastination is the art of making this bad trait work for you. The key idea is that procrastinating does not mean doing absolutely nothing. Procrastinators seldom do absolutely nothing; they do marginally useful things, like gardening or sharpening pencils or making a diagram of how they will reorganize their files when they get around to it. Why does the procrastinator do these things? Because they are a way of not doing something more important. If all the procrastinator had left to do was to sharpen some pencils, no force on earth could get him do it. However, the procrastinator can be motivated to do difficult, timely and important tasks, as long as these tasks are a way of not doing something more important.

Structured procrastination means shaping the structure of the tasks one has to do in a way that exploits this fact. The list of tasks one has in mind will be ordered by importance. Tasks that seem most urgent and important are on top. But there are also worthwhile tasks to perform lower down on the list. Doing these tasks becomes a way of not doing the things higher up on the list. With this sort of appropriate task structure, the procrastinator becomes a useful citizen. Indeed, the procrastinator can even acquire, as I have, a reputation for getting a lot done.

The most perfect situation for structured procrastination that I ever had was when my wife and I served as Resident Fellows in Soto House, a Stanford dormitory. In the evening, faced with papers to grade, lectures to prepare, committee work to be done, I would leave our cottage next to the dorm and go over to the lounge and play ping-pong with the residents, or talk over things with them in their rooms, or just sit there and read the paper. I got a reputation for being a terrific Resident Fellow, and one of the rare profs on campus who spent time with undergraduates and got to know them. What a set up: play ping pong as a way of not doing more important things, and get a reputation as Mr. Chips.

Procrastinators often follow exactly the wrong tack. They try to minimize their commitments, assuming that if they have only a few things to do, they will quit procrastinating and get them done. But this goes contrary to the basic nature of the procrastinator and destroys his most important source of motivation. The few tasks on his list will be by definition the most important, and the only way to avoid doing them will be to do nothing. This is a way to become a couch potato, not an effective human being.

At this point you may be asking, "How about the important tasks at the top of the list, that one never does?" Admittedly, there is a potential problem here.

The trick is to pick the right sorts of projects for the top of the list. The ideal sorts of things have two characteristics, First, they seem to have clear deadlines (but really don't). Second, they seem awfully important (but really aren't). Luckily, life abounds with such tasks. In universities the vast majority of tasks fall into this category, and I'm sure the same is true for most other large institutions. Take for example the item right at the top of my list right now. This is finishing an essay for a volume in the philosophy of language. It was supposed to be done eleven months ago. I have accomplished an enormous number of important things as a way of not working on it. A couple of months ago, bothered by guilt, I wrote a letter to the editor saying how sorry I was to be so late and expressing my good intentions to get to work. Writing the letter was, of course, a way of not working on the article. It turned out that I really wasn't much further behind schedule than anyone else. And how important is this article anyway? Not so important that at some point something that seems more important won't come along. Then I'll get to work on it.

Another example is book order forms. I write this in June. In October, I will teach a class on Epistemology. The book order forms are already overdue at the book store. It is easy to take this as an important task with a pressing deadline (for you non-procrastinators, I will observe that deadlines really start to press a week or two after they pass.) I get almost daily reminders from the department secretary, students sometimes ask me what we will be reading, and the unfilled order form sits right in the middle of my desk, right under the wrapping from the sandwich I ate last Wednesday. This task is near the top of my list; it bothers me, and motivates me to do other useful but superficially less important things. But in fact, the book store is plenty busy with forms already filed by non-procrastinators. I can get mine in mid-Summer and things will be fine. I just need to order popular well-known books from efficient publishers. I will accept some other, apparently more important, task sometime between now and, say, August 1st. Then my psyche will feel comfortable about filling out the order forms as a way of not doing this new task.

The observant reader may feel at this point that structured procrastination requires a certain amount of self-deception, since one is in effect constantly perpetrating a pyramid scheme on oneself. Exactly. One needs to be able to recognize and commit oneself to tasks with inflated importance and unreal deadlines, while making oneself feel that they are important and urgent. This is not a problem, because virtually all procrastinators have excellent self-deceptive skills also. And what could be more noble than using one character flaw to offset the bad effects of another?

ReBaz + Carpentries + RSE Conference

Mike's Notes

Plenty of opportunities in NZ to share research and upskill. I need to learn Python, and this might be a way to do so. I previously used Python with ESRI GIS, but that was 15 years ago.

Pipi 10 will have a Python interface.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

24/10/2025

ReBaz + Carpentries + RSE Conference

By: Mike Peters
On a Sandy Beach: 24/10/2025

Mike is the inventor and architect of Pipi and the founder of Ajabbi.

Research Bazaar

"The annual ResBaz (Research Bazaar) event – an amazing free series of online webinars and workshops that bring together researchers from around the country to develop skills in digital tools and research practices."

The next ResBaz Aotearoa 2026: 29 June - 3 July.

The Carpentries

" ..The Carpentries is a non-profit organisation that teaches foundational coding and data science skills to researchers worldwide. There are numerous courses freely available, including an introduction to Python for library and information workers."

HPC Carpentries

"HPC Carpentry teaches HPC-oriented coding, and data science skills to researchers. We want to work towards bringing High Performance Computing under the Carpentries umbrella."

New Zealand Research Software Engineering Conference

"Within the research sector, there is a growing number of people who combine expertise in programming with an intricate understanding of research. Although this combination of skills is extremely valuable, these people lack a formal place in the academic system. This means there is no easy way to recognise their contribution, to reward them, or to represent their views.  

A community-driven event 

In June 2020, NeSI decided to rebrand its successful Science Coding Conference to be named the NZ Research Software Engineers (RSE) Conference. Motivation for the change is two-fold:

  • to include people from all research communities who work on the cusp of technical and research domains, and
  •  to more fully align with the goals of the Australia / New Zealand RSE community. 

Initiated in the UK, the RSE movement is a global phenomenon with many associations now set up around the world. In line with that global trend, New Zealand’s community of research software users and developers has steadily grown, with more roles and numbers of people at the intersection of software and research." - NZRSE

Putting a wrapper around Pipi

Mike's Notes

I had an epiphany while building out the new workspace UI yesterday.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

26/10/2025

Putting a wrapper around Pipi

By: Mike Peters
On a Sandy Beach: 23/10/2025

Mike is the inventor and architect of Pipi and the founder of Ajabbi.

The new workspace applications smell like a wrapper around Pipi 9.

A GUI is a graphical user interface—windows, menus, tools, etc.— that you can click with a mouse. A wrapper is a piece of software that wraps an existing piece of software, like Pipi 9, and provides its interface (UI).

Headless

Pipi has been a "headless" working system since Pipi 8 in 2021. Getting Pipi to self-generate a UI has been a major task due to the number of rules and configuration standards imported. I wanted to solve some hard usability problems for users, which have taken time to address.

  • Accessibility
  • Localisation
  • Usability
  • Personalisation
  • Reliability
  • Security
  • Learning material

Documentation

I have also had to find a way to document a system with a novel architecture and subsystems that no one else uses. And then Pipi uses evolutionary algorithms rather than LLM, plus a lot of other stuff in the mix.

  • How to name things that are breaking conventions?
  • How to describe processes that are indeterminate yet highly reliable?
  • The UI will be a very thin skin on top of a multi-layer system that is at least 28 architecture layers deep.
  • There is no version control. Pipi is both the IDE and the source control. It's also the manager.

UI wrapper

It turns out that the new workspace Ui is simple, lightweight and very fast. It has a tiny attack surface, if at all. The UI looks, functions and smells like a wrapper. I figured that out only now while building the UI.

Discovery

I wonder what else will be discovered in this stretch. I have yet to start on the Templates, which are workspace customised configurations.

Example

  • Health
    • Modules
      • Booking
      • Facility
        • Cafe
        • Clean
        • Kitchen
        • Security
        • Ward
          • Room
          • Bed
      • Health Record
      • Health Worker
      • Imaging
      • Medication
      • Patient
      • Procedure
      • Transport
    • Settings
      • Billing
      • Dashboard
      • Plugin
      • Users
    • Templates
      • Allied
      • Animal Vet
      • Emergency
      • Family Doctor Clinic
      • Hospital
      • Public Health System
      • Patient

Thank you

I want to thank the early adopters who are helping me test and provide valuable feedback.

No Science, No Startups: The Innovation Engine We're Switching Off

Mike's Notes

A great description of how science works in the US.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Steve Blank
  • Home > Handbook > 

Last Updated

22/10/2025

No Science, No Startups: The Innovation Engine We're Switching Off

By: Steve Blank
On a Sandy Beach: 13/10/2025

Blank created the Lean Launchpad class and I-Corps curriculum which became the standard for science commercialization for the National Science Foundation, the National Institutes of Health and the U.S. Department of Energy. As of 2023, more than 3,051 teams and 1,300 startups have employed Blank’s methodologies..

Blank is co-creator of the U.S. Department of Defense's[6] Hacking for Defense program, and served on the Defense Business Board and the U.S. Navy’s Science and Technological Board. He is co-creator of the Gordian Knot Center for National Security Innovation at Stanford University.

Tons of words have been written about the Trump Administrations war on Science in Universities. But few people have asked what, exactly, is science? How does it work? Who are the scientists? What do they do? And more importantly, why should anyone (outside of universities) care?

(Unfortunately, you won’t see answers to these questions in the general press – it’s not clickbait enough. Nor will you read about it in the science journals– it’s not technical enough. You won’t hear a succinct description from any of the universities under fire, either – they’ve long lost the ability to connect the value of their work to the day-to-day life of the general public.)

In this post I’m going to describe how science works, how science and engineering have worked together to build innovative startups and companies in the U.S.—and why you should care.

(In a previous post I described how the U.S. built a science and technology ecosystem and why investment in science is directly correlated with a country’s national power. I suggest you read it first.)

How Science Works

I was older than I care to admit when I finally understood the difference between a scientist, an engineer, an entrepreneur and a venture capitalist; and the role that each played in the creation of advancements that made our economy thrive, our defense strong and America great.

Scientists

Scientists (sometimes called researchers) are the people who ask lots of questions about why and how things work. They don’t know the answers. Scientists are driven by curiosity, willing to make educated guesses (the fancy word is hypotheses) and run experiments to test their guesses. Most of the time their hypotheses are wrong. But every time they’re right they move the human race forward. We get new medicines, cures for diseases, new consumer goods, better and cheaper foods, etc.

Scientists tend to specialize in one area – biology, medical research, physics, agriculture, computer science, materials, math, etc. — although a few move between areas. The U.S. government has supported scientific research at scale (read billions of $s) since 1940.

Scientists tend to fall into two categories: Theorists and Experimentalists.

Theorists

Theorists develop mathematical models, abstract frameworks, and hypotheses for how the universe works. They don’t run experiments themselves—instead, they propose new ideas or principles, explain existing experimental results, predict phenomena that haven’t been observed yet. Theorists help define what reality might be.

Theorists can be found in different fields of science. For example:

 Science Study 
Physics  Quantum field theory, string theory, quantum mechanics
Biology Neuroscience and cognition, Systems Biology, gene regulation
Chemistry  Molecular dynamics, Quantum chemistry
Computer Science Design algorithms, prove limits of computation
Economics  Build models of markets or decision-making
Mathematics  Causal inference, Bayesian networks, Deep Learning

The best-known 20th-century theorist was Albert Einstein. His tools were a chalkboard and his brain. in 1905 he wrote an equation E=MC2 which told the world that a small amount of mass can be converted into a tremendous amount of energy. When he wrote it down, it was just theory. Other theorists in the 1930s and ’40s took Einstein’s theory and provided the impetus for building the atomic bomb. (Leo Szilard conceived neutron chain reaction idea, Hans Bethe led the Theoretical Division at Los Alamos, Edward Teller developed hydrogen bomb theory.) Einstein’s theory was demonstrably proved correct over Hiroshima and Nagasaki.

Experimentalists

In addition to theorists, other scientists – called experimentalists – design and run experiments in a lab. The pictures you see of scientists in lab coats in front of microscopes, test tubes, particle accelerators or NASA spacecraft are likely experimentalists. They test hypotheses by developing and performing experiments. An example of this would be NASA’s James Webb telescope or the LIGO Gravitational-Wave Observatory experiment. (As we’ll see later, often it’s engineers who build the devices the experimentalists use.)

Some of these experimentalists focus on Basic Science, working to get knowledge for its own sake and understand fundamental principles of nature with no immediate practical use in mind.

Other experimentalists work in Applied Science, which uses the findings and theories derived from Basic Science to design, innovate, and improve products and processes.

Applied scientists solve practical problems oriented toward real-world applications. (Scientists at Los Alamos weretrying to understand the critical mass of U-235 (the minimum amount that would explode.) Basic science lays the groundwork for breakthroughs in applied science. For instance: Quantum mechanics (basic science) led to semiconductors which led to computers (applied science). Germ theory (basic science) led to antibiotics and vaccines (applied science). In the 20th century Applied scientists did not start the companies that make end products. Engineers and entrepreneurs did this. (In the 21st century more Applied Scientists, particularly in life sciences, have also spun out companies from their labs.)

Scientists

Where is Science in the U.S. Done?

America’s unique insight that has allowed it to dominate Science and invention, is that after WWII we gave Research and Development money to universities, rather than only funding government laboratories. No other country did this at scale.

Corporate Research Centers

In the 20th century, U.S. companies put their excess profits into corporate research labs. Basic research in the U.S. was done in at Dupont, Bell Labs, IBM, AT&T, Xerox, Kodak, GE, et al.

This changed in 1982, when the Securities and Exchange Commission ruled that it was legal for companies to buy their own stock (reducing the number of shares available to the public and inflating their stock price.) Very quickly Basic Science in corporate research all but disappeared. Companies focused on Applied Research to maximize shareholder value. In its place, Theory and Basic research is now done in research universities.

Research Universities

From the outside (or if you’re an undergraduate) universities look like a place where students take classes and get a degree. However, in a research university there is something equally important going on. Science faculty in these schools not only teach, but they are expected to produce new knowledge—through experiments, publications, patents, or creative work. Professors get grants and contracts from federal agencies (e.g., NSF, NIH, DoD), foundations, and industry. And the university builds Labs, centers, libraries, and advanced computing facilities that support these activities.

In the U.S. there are 542 research universities, ranked by the Carnegie Classification into three categories.

R1: 187 Universities – Very High Research Activity

  • Conduct extensive research and award many doctoral degrees.
  • Examples: Stanford, UC Berkeley, Harvard, MIT, Michigan, Texas A&M …

R2: 139 Universities – High Research Activity

  • Substantial but smaller research scale.
  • Examples: Baylor, Wake Forest, UC Santa Cruz, …

R3: 216 Research Colleges/Universities

  • Limited research focus; more teaching-oriented doctoral programs.
  • Smaller state universities

Why Universities Matter to Science

U.S. universities perform about 50% of all basic science research (physics, chemistry, biology, social sciences, etc.) because they are training grounds for graduate students and postdocs. Universities spend ~$109 billion a year on research. ~$60 billion of that $109 billion comes from the National Institutes for Health (NIH) for biomedical research, National Science Foundation (NSF) for basic science, Department of War (DoW), Department of Energy (DOE), for energy/physics/nuclear, DARPA, NASA. (Companies tend to invest in applied research and development, that leads directly to saleable products.)

Professors (especially in Science, Technology, Engineering and Math) run labs that function like mini startups. They ask research questions, then hire grad students, postdocs, and staff and write grant proposals to fund their work, often spending 30–50% of their time writing and managing grants. When they get a grant the lead researcher (typically a faculty member/head of the lab) is called the Principal Investigator (PI).

The Labs are both workplaces and classrooms. Graduate students and Postdocs do the day-to-day science work as part of their training (often for a Ph.D.). Postdocs are full-time researchers gaining further specialization. Undergraduates may also assist in research, especially at top-tier schools.

(Up until 2025, U.S. science was deeply international with ~40–50% of U.S. basic research done by foreign-born researchers (graduate students, postdocs, and faculty). Immigration and student visas were a critical part of American research capacity.)

The results of this research are shared with the agencies that funded it, published in journals, presented at conferences and often patented or spun off into startups via technology transfer offices. A lot of commercial tech—from Google search to CRISPR—started in university labs.

Universities support their science researchers with basic administrative staff (for compliance, purchasing, and safety) but uniquely in the U.S., by providing the best research facilities (labs, cleanrooms, telescopes), and core scientific services: DNA sequencing centers, electron microscopes, access to cloud, data analysis hubs, etc. These were the best in the world – until the sweeping cuts in 2025.

Engineers Build on the Work of Scientists

Engineers design and build things on top of the discoveries of scientists. For example, seven years after scientists split the atom, it took 10s of thousands of engineers to build an atomic bomb. From the outset, the engineers knew what they wanted to build because of the basic and applied scientific research that came before them.

Scientists Versus Engineers

Engineers create plans, use software to test their designs, then… cut sheet metal, build rocket engines, construct buildings and bridges, design chips, build equipment for experimentalists, design cars, etc.

As an example, at Nvidia their GPU chips are built in a chip factory (TSMC) using the Applied science done by companies like Applied Materials which in turn is based on Basic science of semiconductor researchers. And the massive data centers OpenAI, Microsoft, Google, et al that use Nvidia chips are being built by mechanical and other types of engineers.

My favorite example is that the reusable SpaceX rocket landings are made possible by the Applied Science research on Convex Optimization frameworks and algorithms by Steven Boyd of Stanford. And Boyd’s work was based on the Basic science mathematical field of convex analysis (SpaceX, NASA, JPL, Blue Origin, Rocket Lab all use variations of Convex Optimization for guidance, control, and landing.)

Startup Entrepreneurs Build Iteratively and Incrementally

Entrepreneurs build companies to bring new products to market. They hire engineers to build, test and refine products.

Engineers and entrepreneurs operate with very different mindsets, goals, and tolerances for risk and failure. (Many great entrepreneurs start as engineers e.g., Musk, Gates, Page/Brin). An engineer’s goal is to design and deliver a solution to a known problem with a given set of specifications.

In contrast, entrepreneurs start with a series of unknowns about who are the customers, what are the wanted product features, pricing, etc. They retire each of these risks by building an iterative series of minimum viable products to find product/market fit and customer adoption. They pivot their solution as needed when they discover their initial assumptions are incorrect. (Treating each business unknown as a hypothesis is the entrepreneurs’ version of the Scientific Method.)

Venture Capitalists Fund Entrepreneurs

Venture capitalists (VCs) are the people who fund entrepreneurs who work with engineers who build things that applied scientists have proven from basic researchers.

Unlike banks which will give out loans for projects that have known specifications and outcomes, VCs invest in a portfolio of much riskier investments. While banks make money on the interest they charge on each loan, VCs take part ownership (equity) in the companies they invest in. While most VC investments fail, the ones that succeed make up for that.

Most VCs are not scientists. Few are engineers, some have been entrepreneurs. The best VCs understand technical trends and their investments help shape the future. VCs do not invest in science/researchers. VCs want to minimize the risk of their investment, so they mostly want to take engineering and manufacturing risk, but less so on applied science risk and rarely on basic research risk. Hence the role of government and Universities.

VCs invest in projects that can take advantage of science and deliver products within the time horizon of their funds (3–7 years). Science often needs decades before a killer app is visible.

As the flow of science-based technologies dries up, the opportunities for U.S. venture capital based on deep tech will decline, with its future in countries that are investing in science – China or Europe.

Why Have Scientists? Why Not Just a Country of Engineers, Entrepreneurs and VCs (or AI)?

If you’ve read so far, you might be scratching your head and asking, “Why do we have scientists at all? Why pay for people to sit around and think? Why spend money on people who run experiments when most of those experiments fail? Can’t we replace them with AI?”

The output of this university-industry-government science partnership became the foundation of Silicon Valley, the aerospace sector, the biotechnology industry, Quantum and AI. These investments gave us rockets, cures for cancer, medical devices, the Internet, Chat GPT, AI and more.

Investment in science is directly correlated with national power. Weaken science, you weaken the long-term growth of the economy, and national defense.

Tech firms’ investments of $100s of billions in AI data centers is greater than the federal government’s R&D expenditures. But these investments are in engineering not in science. The goal of making scientists redundant using artificial general intelligence misses the point that AI will (and is) making scientists more productive – not replacing them.

Countries that neglect science become dependent on those that don’t. U.S. post-WWII dominance came from basic science investments (OSRD, NSF, NIH, DOE labs). After WWII ended, the UK slashed science investment which allowed the U.S. to commercialize the British inventions made during the war.

The Soviet Union’s collapse partly reflected failure to convert science into sustained innovation, during the same time that U.S. universities, startups and venture capital created Silicon Valley. Long-term military and economic advantage (nuclear weapons, GPS, AI) trace back to scientific research ecosystems.

Lessons Learned

  • Scientists come in two categories
    • Theorists and experimentalists
    • Two types of experimentalists; Basic science (learn new things) or applied science (practical applications of the science)
    • Scientists train talent, create patentable inventions and solutions for national defense
  • Engineers design and build things on top of the discoveries of scientists
  • Entrepreneurs test and push the boundaries of what products could be built
  • Venture Capital provides the money to startups
  • Scientists, engineers, entrepreneurs – these roles are complementary
    • Remove one and the system degrades
  • Science won’t stop
    • Cut U.S. funding, then science will happen in other countries that understand its relationship to making a nation great – like China.
    • National power is derived from investments in Science
    • Reducing investment in basic and applied science makes America weak

 Appendix – How Does Science Work? – The Scientific Method

Whether you were a theorist or experimentalist, for the last 500 years the way to test science was by using the scientific method. This method starts by a scientist wondering and asking, “Here’s how I think this should work, let’s test the idea.”

The goal of the scientific method is to turn a guess (in science called a hypothesis) into actual evidence. Scientists do this by first designing an experiment to test their guess/hypothesis. They then run the experiment and collect and analyze the result and ask, “Did the result validate, invalidate the hypothesis? Or did it give us completely new ideas?” Scientists build instruments and run experiments not because of what they know, but because of what they don’t know.

These experiments can be simple ones costing thousands of dollars that can be run in a university biology lab while others may require billions of dollars to build a satellite, particle accelerator or telescope. (The U.S. took the lead in Science after WWII when the government realised that funding scientists was good for the American economy and defence.)

Good science is reproducible. Scientists just don’t publish their results, but they also publish the details of how they ran their experiment. That allows other scientists to run the same experiment and see if they get the same result for themselves. That makes the scientific method self-correcting (you or others can see mistakes).

One other benefit of the scientific method is that scientists (and the people who fund them) expect most of the experiments to fail, but the failures are part of learning and discovery. They teach us what works and what doesn’t. Failure in science testing unknowns means learning and discovery.