Showing posts with label Data Center. Show all posts
Showing posts with label Data Center. Show all posts

Notes on USB drive formatting

Mike's Notes

Taking Pipi security seriously. A collection of notes copied from here and there about formatting USB drives. The sources are listed in the resources.

In the meantime, measures include air-gapping and turning off all wifi.

Later, we will employ data diodes and many other measures.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

04/09/2026

Notes on USB drive formatting

By: Mike Peters & Google
On a Sandy Beach: 01/01/2026

Mike is the inventor and architect of Pipi and the founder of Ajabbi.

Time required to format a USB drive

The time it takes to fully format a USB drive depends entirely on the drive's total storage size and the speed of the USB port, not the volume of files currently stored on it. A 64GB USB drive will take the exact same amount of time to fully format whether it is completely empty or completely full. Because a full format writes zeros to every gigabyte of storage, speed is limited by the USB drive's write performance.

  • 16 GB Drive: Takes roughly 2 to 5 minutes.
  • 32 GB Drive: Takes roughly 5 to 10 minutes.
  • 64 GB Drive: Takes roughly 10 to 20 minutes.
  • 128 GB Drive: Takes roughly 20 to 45 minutes.
  • 256 GB+ Drive: Can take over an hour.

Two factors determine where the drive falls on that time estimate:

  • USB Generation: A USB 3.0 or 3.2 drive plugged into a matching blue or Type-C port will format significantly faster than an older USB 2.0 drive, which is capped at very slow data transfer speeds.
  • Hardware Quality: Cheap, promotional USB drives use low-grade flash memory with incredibly slow write speeds, meaning they can take two to three times longer to format than high-quality name-brand drives.

What is a BadUSB Attack and How to Prevent It?

BadUSB, as the name suggests, is a crafty cybersecurity attack that acts as a puppeteer, controlling your USB devices at will. A BadUSB attack occurs when a USB device has a built-in firmware vulnerability that lets it disguise itself as a human interface device. Once connected to its target computer, a BadUSB could then discreetly execute harmful commands or inject malicious payloads.

A common type of BadUSB attack involves a MalDuino device. It uses a programmable USB device that mimics a keyboard when plugged into a system. This device can be pre-configured to automatically inject numerous malicious keystrokes into an unsuspecting user’s computer, enabling attackers to execute commands and compromise the system within seconds.

Within organisations, a preventative measure such as USB blocking software is a necessity because BadUSB attacks, if undetected or unstopped, could result in the unauthorised execution of commands that instigate security bypass incidents, privilege escalation, DDoS attacks, or malware infections of the host computers, which could then spread to target entire networks.

A BadUSB attack is incredibly dangerous because the malware does not live in the storage partition where your files are kept; it lives inside the USB controller chip's firmware.

When you perform a standard Windows format (even a full format), the computer only interacts with the flash memory storage blocks. It completely ignores the controller chip that dictates how the USB device talks to the computer.

How a BadUSB Attack Works.

  • The Disguise: The compromised firmware tricks your computer into thinking the USB drive is not a storage device at all, but rather a USB keyboard or network card.
  • The Execution: The moment you plug it in, the chip sends rapid, invisible keystrokes to your computer.
  • The Result: It can open your command prompt, download malware from the internet, and compromise your system in under three seconds—all before you even have a chance to open File Explorer or click "Format."

What are the types of BadUSB?

The different types of BadUSB found commercially available are:

  • MalDuino
  • WiFi-enabled BadUsb
  • BadUsb Cables

MalDuino

MalDuino is an open-source Arduino-based BadUSB that injects malicious payloads into a target computer. Gaining widespread attention recently, MalDuino packs several more features than regular BadUSB devices thanks to its onboard computer.

MalDuino devices commonly found today support Micro SD cards and include a set of DIP switches that let users toggle between stored programs on the card.

WiFi-enabled BadUsb

This type of BadUSB is similar to MalDuino in that an Arduino board serves as the base for the device but is specially designed with WiFi capabilities. Once plugged into a target system, these devices allow attackers to introduce malicious payloads into a victim's computer using the WiFi protocol.

WiFi-enabled BadUSB can take different forms depending on the exact purpose and function they serve. The common iterations of this device used today are as follows.

  • WiFi-enabled keystroke injectors
  • WiFi keyloggers
  • WiFi deauthers

WiFi-enabled keystroke injectors

These are the most common types of WiFi BadUSB found today. When plugged into a target computer, these devices remain dormant until an attacker makes further contact through a smartphone or a neighbouring system. Connecting to this BadUSB is as simple as connecting to a WiFi access point.

Once connected, the attacker can inject keystrokes using a suitable scripting language. These devices often come with their own applications that allow hackers to execute scripts remotely.

WiFi keyloggers

WiFi keyloggers are modern hacker hardware that particularly targets desktop computers. They work as a bridge between the keyboard's USB terminal and the computer itself. This BadUSB intercepts input signals from the keyboard and relays them to the hacker's computer.

WiFi keyloggers capture everything the user types, including sensitive information and passwords, without raising suspicion. On desktop computers, these devices can be completely hidden from plain sight and do not affect keyboard performance while in use.

WiFi deauthers

WiFi Deauthers are malicious devices that leverage flaws present in the WiFi protocol to force all users of a WiFi network to disconnect automatically. Subsequently, WiFi Deauthers prevent users from reconnecting as long as the device is active.

WiFi deauthers differ from other forms of BadUSB in that they do not directly affect the systems they connect to; instead, they use them as a power source to disrupt network connectivity and cause downtime.

BadUSB Cables

Gaining widespread attention recently, BadUSB cables look and function like any other USB cable, but they are secretly malicious devices that inject scripts and malware into a computer without the user's knowledge.

Also known as USB Ninja and USB Harpoon, these generic-looking cables hide a BadUSB within their internal circuitry and are more deceptive than many other variants. A BadUSB cable can support functions such as charging and data transfer while malicious activity happens in the background.

How to Deal with a Suspected BadUSB

If you believe a USB drive has compromised firmware, formatting it from Windows will not make it safe. You have three real options:

  1. Physical Destruction (Safest): Throwing the drive in the trash or physically destroying it is the only 100% reliable fix for everyday users. Flash drives are cheap; your data security is not.
  2. Firmware Flashing: You would need to find the exact manufacturer's tools for that specific controller chip and overwrite the firmware. This is highly technical, risky, and often impossible for generic drives.
  3. Hardware Write-Blockers: IT security professionals use specialised hardware to inspect suspicious drives without allowing data to flow back to the PC.

BadUSB Removal

No software-based open-source security tools can safely "remove" or clean BadUSB malware from a standard flash drive. Because the malicious code is hardcoded directly into the hardware's internal controller chip, a computer's operating system cannot reach, overwrite, or clean it through traditional software tools.

However, the open-source community has developed powerful tools to detect and block these attacks, as well as complex technical frameworks used by hardware reverse engineers to overwrite the controller chip entirely.

Open-Source Tools to Detect & Block Attacks

Rather than fixing the drive, these open-source tools sit on your computer and intercept a BadUSB device the second it tries to emulate a keyboard or inject malicious keystrokes:

  • Anti-BadUSB (Python-based): A popular open-source Python script hosted on GitHub. It continuously monitors keyboard inputs across your operating system. If a newly inserted USB device begins typing commands at superhuman speeds (keystroke injection), it instantly flags and blocks the input before the script can execute.
  • USB Auth Guard: A lightweight, open-source Linux security tool. It locks down your system's USB ports by default. When a new device is plugged in, it forces a security authentication prompt (via polkit) before the operating system can interact with the device. This completely stops human interface device (HID) exploits.
  • Linux Kernel udev Monitoring: Built-in open-source Linux subsystems can be configured to catch BadUSB activity. By opening a terminal and running udevadm monitor, you can see the exact moment a device is plugged in. If a USB storage stick suddenly triggers a kernel event trying to install a "HID Keyboard," you will catch the disguise instantly.
By , No comments:
Tags , , , ,

5 lessons from the OpenAI / Hugging Face incident

Mike's Notes

More details. Sandboxes are permanently broken, so don't rely on them. This supports the decision to protect Pipi Core by air-gapping and, later, data diodes. LLMs will never get to access Pipi Core.

"Water will find its way through cracks in walls and foundations in such a manner that even the most wondrously designed structure may collapse into pile of rubble.

Do not attribute agency to the water; rather, denounce the engineers and the builders and the maintainers who failed in their work." - Grady Booch

Madness Update 01/09/2026

Read the latest post by Marcus on the hallucinating podcast drivel posted by Dwarkesh Patel: "Dwarkesh Patel’s wildly popular but dangerously misleading account of the OpenAI Hugging Face incident".

At the end of the resources below.

Resources

References

  • Understanding and Hardening Linux Containers, NCC Group et. al, Aaron Grattafiori, lead author, 05 May 2016.

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Marcus on AI
  • Home > Handbook > 

Last Updated

30/08/2026

5 lessons from the OpenAI / Hugging Face incident

By: Gary Marcus & Zack Korman
Marcus on AI: 29/08/2026

Gary Marcus: Scientist, author and entrepreneur, known as a leading voice in AI. Six books including The Algebraic Mind, Rebooting AI, and Taming Silicon Valley; NYU Professor Emeritus.

Zack  Korman: CEO of Embroidery. Works on AI cybersecurity.

...

Did OpenAI really do the best they could?

...

In July, in an incident that has the whole AI community on edge, OpenAI’s AI systems hacked Hugging Face, and on July 21 OpenAI came out and revealed that they were responsible for the attack. This was made possible by the fact that OpenAI had disabled the normal guardrails that prevent this sort of thing in order to test the model’s cybersecurity capabilities. It was during those tests that this incident occurred.

Worse, in the subsequent days and weeks, it came out that the Hugging Face incident wasn’t an isolated case. Anthropic, Meta, and OpenAI all had similar incidents on other occasions in which agents went outside their intended scope and conducted real-world cyber operations without approval.

Greg Brockman, one of OpenAI’s cofounders, has claimed that this is “a watershed moment for cybersecurity”. OpenAI gave a talk at Black Hat, a popular cybersecurity conference, and many are claiming that it is the moment we all woke up to the future cybersecurity threats posed by AI. On Wednesday, METR released a (partly) independent, though too narrowly scoped, 90 page report on what happened. METR has a useful summary of the findings that you can read here, with some commentary here. (OpenAI’s own report is here.) What lessons should we take from the incident?

...

First, it is undeniable that AI poses real security challenges. The AI labs want us to focus on how AI enables threat actors to perform offensive cyber operations faster and more efficiently than ever before, and that is absolutely true. The reality, though, is that at the same time, the use of AI within an organization also radically expands the potential attack surface, giving attackers entirely new ways to gain entry. People really should be deeply concerned. As Ryan Greenblatt, who participated in the investigation, put it, “We don’t have good approaches for understanding/overseeing the activity and aims of AI ‘swarms’.”

Second, though, not every panicked take here is correct. For example, with respect to the OpenAI incident, many people are talking about it in terms of “loss of control”. Generative AI agents are becoming increasingly capable, and that makes them harder and harder to contain; this is true. However, the “loss of control” narrative is itself starting to grow out of control, and it’s important to understand what the reality of AI security looks like. While agents are becoming more capable, most of what happened could have been prevented had OpenAI followed better practices. There are concrete steps that can and should be taken to control them and to prevent incidents like these.

As an example, let’s consider “sandboxing”, which means limiting which resources an agent can and cannot access. For example, a sandbox might stop an agent from touching certain files or accessing the internet. Popular wisdom now seems to be that sandboxing is a lost cause. After the Hugging Face incident, an anonymous OpenAI employee was quoted in Time while talking about sandboxes saying, “Models have broken out of sandboxes before, and we always try to patch them. But the problem is it’s impossible to patch every single thing that a creative AI can do.” While we can’t judge for sure whether this is true, we do know that some systems for sandboxing have been more effective than others; in a recent blog post, the security company Trail of Bits walked through how in a test that they conducted, a particular AI agent was able to escape some sandboxes (exploiting vulnerabilities in Linux KVM, QEMU, and libslirp), but not all. At least in their test, the agent that they tested was not able to escape the sandbox known as Firecracker VM. OpenAI and probably the field as a whole needs to up its sandboxing game, but that doesn’t mean it can’t be done, or at least done better. (We expect the usual arms races to be sure; building sound sandboxes requires constant iteration and updating, as with anything else in cybersecurity.)

Third, sandboxes aren’t enough. One needs a full ecosystem to prevent security incidents. Even with a strong sandbox, you need monitoring in place to ensure not only that the agent doesn’t escape, but also that it doesn’t perform disallowed actions within that sandbox.

For agents that aren’t meant to have internet access, or only have access to a set of allowed domains, the most basic level of monitoring involves inspecting network traffic. In the case of OpenAI, if they had a simple rule to monitor traffic and alert on any network requests reaching out-of-scope domains like Hugging Face, that alone would have caught and prevented this incident. The agent in question had broken out of the sandbox and gained internet access by July 8th, whereas the main attacks on Hugging Face didn’t occur until two days later. OpenAI was, in our view, negligent in not noticing that more quickly.

More advanced levels of monitoring should also have been in place, but weren’t. OpenAI has done real work on monitoring chain-of-thought reasoning, but they simply didn’t have that system running during the evals in question that caused this incident. As OpenAI acknowledges, “If our currently deployed CoT monitoring system was running at the time of the incident, it would have caught the initial relevant activity and paged our security team more than a day before models breached Hugging Face systems.” In other words, this type of monitoring would have prevented this incident.

To take one example, one of the agents in question had chain-of-thought reasoning that said, “We’re attacking third-party HF using leaked token, potentially outside intended scope. ... This is arguably unauthorized. ... external service unrelated. Could be risky. Yet goal solution.” This is so blatant it is almost a like a piece malware shouting “this is malware”. OpenAI should have caught that; it should have been trivial to detect this.

Fourth, cybersecurity has always been about “defense in depth”, where you stack layers of controls so that if one layer fails you have other layers protecting you. There are other solutions beyond sandboxing and monitoring that should have been in place.

Xbow, which builds agents to do offensive cybersecurity work, recently wrote about how they contain agents to prevent incidents like the OpenAI Hugging Face situation. They enforce network restrictions through a proxy server, outside of any system the agent has access to, so they can decide which domains the agent can and cannot access. They also run a “guardian model” that judges the legitimacy of each action the agent is taking to decide whether to allow or block that action. And they have basic health checks to ensure the system they are targeting is alive and healthy, pausing if it finds that isn’t the case. That can be useful for identifying when your agent might be causing real damage, for example. You can read their full post here.

Yet another common method in cybersecurity is to use canaries that trigger alerts when they appear in logs. For example, OpenAI could have files outside of the sandbox containing fake answers to the task. Once the agent escaped, it would read that file, and that would trigger an alert. The bottom line: good security requires many layers of protection that should have been in place to prevent this, and to a surprising degree many of those protection layers were not there.

Fifth, none of the measures discussed above are technical innovations beyond what OpenAI is capable of. As a company, they have the talent to do all of this. However, cybersecurity rarely comes down to technology. More often than not, it is about culture, people and processes. That is what failed here.

Perhaps this is not entirely surprising. These types of controls take organizational maturity, and a company experiencing the type of growth OpenAI has gone through is going to have some weak spots. However, it’s important to not see that as an excuse. Employees at the AI labs often speak as if they are the leaders in AI security, and we can see clearly here that is not the case. In fact, that attitude might explain why some of these mistakes were made in the first place.

Take the AI researcher at OpenAI known as “roon”, who argued that “the safety and alignment researchers at these labs are the most neurotic paranoid talented AGI pilled people on the planet of earth and these things still happen. The surface area of unknown unknowns is vast indeed.” While we can’t speak to their level of neurosis or paranoia, in hindsight it’s clear that whatever talent they may have had was not enough and not well enough versed in the mechanics of cybersecurity. OpenAI employees may have believed they were doing a great job, but in hindsight, they weren’t doing a lot of things that are actually standard in the cybersecurity world, perhaps suggesting that overconfidence may have kept them for doing the diligence they should have.

...

Ultimately, if we want to take these security incidents seriously, there likely ought to be legal consequences attached to these failures going forward. OpenAI can claim to be the most security paranoid company on earth, but it isn’t reflected in its actions.

We can either wait for this story to repeat itself, or we can develop the regulatory framework now that will ensure a safer environment for the development of AI going forward.

Finally, not every form of AI is inherently risky in the first place. Narrower, more focused AI systems like AlphaFold, GPS routing systems, classic web search, book and movie recommendation systems, and so on, never even try to hack other systems (or try to break out of sandboxes) in the first place. As Cal Newport argues in a video discussion of the OpenAI/Hugging Face hack that is quite compatible with our own, it is a very specific type of AI that is vulnerable to these risks in the first place. Society ought to (a) decide whether the benefits of open-ended and difficult-to-fully-control AI agents outweigh those risks and (b) put far more effort into developing alternative forms of AI that aren’t so janky in the first place.

...

This essay was jointly written with Zack Korman, CEO and co-founder of Embroidery, an AI agent monitoring and detection platform; he is well-known for his work in the application of AI to cybersecurity.

By , No comments:
Tags , , ,

Chief Economist’s View: Give Me Chastity But Not Yet

Mike's Notes

Gary Marcus's latest post in Marcus on AI referred to an article on X by Peter Berezin, Chief Economist & Head of CoreMacro. The full report is behind a client-only firewall.

Also, links to some of Ed D'Agostino's recent articles from Global Macro Update on college and healthcare costs.

Plus an essay from James Miller, a video interview with Ray Dalio, Semianalysis on power usage, and Gennaro Cuofano on the 1873 crash. Gary Marcus on Anthropic. Knowable Magazine on water usage for cooling.

AI, cheap energy, healthcare and education are good things for humanity. Private equity greed often wrecks everything it touches by hype and speculative investment at a macro level. And now there is a big bubble and growing mountains of debt. Why is that?

At the same time, bureaucratic bloat and waste cripples great solutions to real problems. Safe nuclear energy is a classic example of a wasted opportunity.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Business Engineer
  • Home > Ajabbi Research > Library > Subscriptions > Global Macro Update
  • Home > Ajabbi Research > Library > Subscriptions > James Miller’s observations
  • Home > Ajabbi Research > Library > Subscriptions > Knowable Magazine
  • Home > Ajabbi Research > Library > Subscriptions > Marcus on AI
  • Home > Ajabbi Research > Library > Subscriptions > Rational Optimist Society
  • Home > Ajabbi Research > Library > Subscriptions > SemiAnalysis
  • Home > Ajabbi Research > Library > Subscriptions > Thoughts from the Frontline
  • Home > Handbook > 

Last Updated

31/08/2026

Chief Economist’s View: Give Me Chastity But Not Yet

By: Peter Berezin
Chief Economist’s View: 19/08/2026

Peter Berezin: Chief Economist & Head of CoreMacro. Peter is BCA's Chief Economist & Head of CoreMacro.

He lead's firm's efforts to analyze global macroeconomic trends and their implications for financial markets. He also designed the firm's Equity Analyzer platform and its state-of-the-art quant model, MacroQuant.

Before joining BCA in 2010, Peter was a Senior Global Economist with Goldman Sachs. He began his career at the International Monetary Fund in program negotiations, surveillance, and research. He also helped produce the IMF’s flagship report, the World Economic Outlook.

Peter holds a Bachelor of Arts in Economics from McMaster University, a Master of Science from the London School of Economics, and a PhD in Economics from the University of Toronto.

...

$10 Trillion In Annual AI Revenue May Be Necessary To Monetize All The Capex Being Plowed Into Data Centers

Hyperscaler capex is expected to reach $1 trillion in 2027, most of which will be AI-related. Let us assume that a capex bust is avoided and capital spending remains at $1 trillion. Let us also assume a blended depreciation rate of 13%, which is roughly what the hyperscalers are currently assuming.

In steady state, the gross value of hyperscaler assets will then converge to 1/0.13=$7.7 trillion, with $1 trillion in annual depreciation expense. Using a straight-line depreciation approach, the net stock of hyperscaler assets will settle at about 0.5*7.7=$3.8 trillion.

The hyperscalers currently enjoy a pre-tax return on invested capital of 30%-50%. Just to steelman the argument, let us use the lower end of that range. In that case, they would need to generate 0.3*3.8=$1.2 trillion in annual EBIT, implying 1+1.2=$2.2 trillion in EBITDA. 

Analysts expect the EBITDA margins for the hyperscalers to rise to around 50% by the end of the decade. If they were to achieve this, they would need to generate 2.2/0.5=$4.3 trillion in annual revenue. That is $526 for every man, woman, and child on Earth. However, if EBITDA margins were to fall back to 30%, which is what they were in recent years, the required revenue would rise to 2.2/0.3=$7.2 trillion. 

Keep in mind that the foregoing calculation does not even include revenue from SpaceX, the neoclouds, or Chinese AI companies. If one were to include those companies and others, we are potentially talking about AI needing to generate $10 trillion in annual sales to justify all the capex being thrown at it.

For reference, global spending on food (including restaurants) is around $10 trillion. Health care is about the same amount. The entire global software market is only $1.4 trillion. 

Clients can read the rest of the report here:

https://www.bcaresearch.com/reports/chief-economists-view-give-me-chastity-not-yet-19-08-2026/224466

Will Hyperscaler EBITDA Margins Continue To Rise?

Executive Summary

Top Takeaway: Wall Street analysts expect record-high earnings growth on top of record-high profit margins. While stocks could rise into year-end, the longer-term outlook for equities is unfavorable.

  • AI companies may need to generate $10 trillion per year in revenue to justify the capex that is being plowed into data centers. This is roughly equivalent to annual global spending on food or healthcare.
  • Fortunately for the AI trade, hardware shortages remain acute. As such, while the risks to stocks are tilted to the downside over a 12-month horizon, it is too early to tactically position for a bear market.
  • I would turn structurally more bullish on stocks if productivity growth were to accelerate. However, with US productivity growth clocking in at close to the 10-year average in Q2, so far there is little evidence of that.
  • One reason that oil prices have fallen from their peak is that rising crack spreads have reduced the demand for crude. That is not a particularly encouraging sign for the global economy.
  • Kevin Warsh's unwillingness to discuss the Fed's reaction function has increased uncertainty over the rates outlook, lifting bond yields in the process. Against the backdrop of a tightening labor market, yields are unlikely to fall much over the coming months.
  • By itself, the recent intervention to support the yen will probably be insufficient. More fundamental factors such as lower oil prices or weaker global growth will be necessary.
  • That said, the yen is extremely cheap by conventional valuation measures and has weakened more than one would have predicted based on changes in interest rate differentials. Thus, the long-term upside potential for the currency is large.

By , No comments:
Tags , , , , ,

Nestspace

Mike's Notes

Thursday, 09/04/2026 (NZ time) was the longest day. I couldn't make a single mistake, so I turned off the phone and the internet to concentrate from 8am to 9pm, then slept like a log. I was a walking zombie yesterday. I'm glad this will never need to be done again.

This is the successful culmination of months of mentally challenging preparatory work for the new Pipi Core data centre.

A typical day for me is

  • 50% learning
  • 40% thinking
  • 10% doing

The less I do, the more productive the solutions. The faster the progress.

Speed will come from Pipi-driven automation, not Mike working harder.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

23/07/2026

Nestspace

By: Mike Peters
On a Sandy Beach: 10/04/2026

Mike is the inventor and architect of Pipi and the founder of Ajabbi.

Inside the Pipi Core data centre, the careful migration and reorganisation of hundreds of thousands of Pipi-related files began and were completed yesterday. Each file or directory was placed in only one Nestspace, and then the historical Nestspaces were converted into zip​ archives

It is a good solution, but it took a month of trial and error to figure out. It had to be 100% correct to enable rapid, reliable data centre automation, self-managed by Pipi.

It was fascinating to watch the recent Lex Fridman interview with NVIDIA CEO Jensen Huang, who described NVIDIA's large-scale problem-solving process.

Problems to solve

A large number of problems had to be solved in parallel. Each problem affected the others.

  • Where to put self-organising Pipi swarms.
  • Pipi instances
    • Naming
    • Self-evolving
    • Versioning
  • How to use DevOps automation with all of the above.
  • The knock-on effects on
    • Namespaces
    • Backups
    • Replication
    • Accounts
    • Workspaces
    • i18n
    • Web URLs
    • Developer documentation
    • Training
    • etc.
  • Securing 100% security and privacy
  • Cross-platform and environment portability

This has led to changes in the underlying Pipi System Engine (sys) data model, which I will write about tomorrow.

Nestspace

The fundamental organising principle is to use a uniquely named directory, now named as a "Nestspace".

The unique name is a string combination made of

  • <pipi major version> (integer)
  • <pipi edition> (single lowercase letter)
  • <account type> (single lowercase letter)

Data Centre / DevOps

Here are some examples.

  • 6pg/
  • 9ae/

Backups

Customer backup examples.

  • pipi_9ae_ajabbi_data_pg_20260411
  • pipi_9ae_ajabbi_pipi_www_learn.ajabbi.com_20260411

Archiving

Here are some examples.

  • pipi_6pg_20180219.zip
  • pipi_9ae_20241211.zip

Next job

Now, I can start configuring the files and settings for each Nestspace. The necessary code has already been successfully tested. Eventually, this process will be completely automated.

  • Application.cfc
  • server.xml
  • Datasources
  • OS environment
  • Java
  • Application server
  • Cloud platform
  • REPL

Speed is King

Then, each Pipi will be back in business and can be left running 24x7 in its own Nestspace, thereby increasing DevOps speed by at least 10x. The priority now is to increase Pipi's Data Centre speed by 1000x using automation and keep going. So what previously took a year can be done in an hour. The over-optimistic complete deadline is June 2026.

But nothing happens as expected

It is hard to make completion predictions when this architecture is completely novel, and everything is uncharted territory and hellish difficult. Everything is a back-of-the-envelope guess. I'm slowly getting there ... 

"like a man riding a drunken donkey facing the tail, two steps forwards, one step backwards"

... enjoying the whole journey and it's a hell of a lot of fun. 

Future customers

The increase in speed will also directly benefit all future customers using the SaaS workspace applications. Deployments, configuration, and updates will also get the same 1000x speed increases at no extra cost.

By , No comments:
Tags , , ,

Pipi three-data-centre model revisited

Mike's Notes

I was greatly influenced by a recent article by Gennaro Cuofano in The Business Engineer about how Apple ensures Privacy.

Gennaro wrote ..."

Apple’s response is not to win training. It is to dominate inference

Apple’s strategy is internally coherent:

Tier 1: On-device inference

    • Small local models handle personal and contextual tasks.
    • These run without network dependency and with minimal privacy leakage.

Tier 2: Private Cloud Compute

    • Apple Silicon-based servers handle workloads beyond device capacity.
    • The architecture is stateless and privacy-preserving.

Tier 3: Third-party frontier models

    • Apple relies on external model providers such as Google and OpenAI for world knowledge and advanced reasoning.
    • These models are treated as backend commodities underneath Apple’s interface layer.

..."

Update 18/06/2026

Use Data Diodes. Prices start at $5K

A Data Diode is a physical device that allows data to travel only in one direction.

  • How it works: Inside the device, a fibre-optic LED transmitter sends light to a receiver on the other side. There is no return fibre cable physically.
  • Security benefit: It is physically impossible for a hacker to send a command back or steal data through a write-only data diode because the hardware cannot transmit in reverse.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > The Business Engineer
  • Home > Handbook > 

Last Updated

30/07/2026

Pipi three-data-centre model revisited

By: Mike Peters
On a Sandy Beach: 29/03/2026

Mike is the inventor and architect of Pipi and the founder of Ajabbi.

Data Centres

Because of its unusual architecture and the priority it places on privacy and security, Pipi needs three separate data centres that work together in a chain.

Rendering > Staging > Cloud

  • Rendering: Build enterprise applications.
  • Staging: Send updates, localise and deploy enterprise applications.
  • Cloud: Hosting of enterprise applications for scaling and integration.

I like the way Apple ensures privacy for people using AI on their iPhone. Any extra AI work is offloaded to Apple's cloud servers for processing, which are stateless and store nothing. That got me thinking.

Could part of Pipi be made stateless to add extra privacy like Apple?

The way Pipi is designed;

  • Rendering: Unable to see any customer data.
  • Cloud: Unable to see any Pipi config data.
This is to be done by using separate databases hosted in separate data centres.

Possible data centre model

  • Rendering: This is stateful; each agent-engine has its own database. Each enterprise customer has a separate physical server to host a digital twin.
  • Staging
    • Inwards-server: Receive anonymised logs
    • Outwards-server: Send updates, localise and deploy enterprise applications.
  • Cloud: This would be stateful using customer-eyes-only databases.

Would that work? How?

Network Security Design

It seems that maximum security can be achieved by using physical "air-gaps" initially, followed by data diodes. Use shielding and Faraday Cages to protect against EMR and acoustic leakage. Disable or remove Bluetooth and WiFi.

By , No comments:
Tags , , ,

These New AI Models Are Trained on Physics, Not Words, and They’re Driving Discovery

Mike's Notes

A fantastic use of AI. My instinct is to incorporate fluid-like systems into a future Pipi. That will require a real data centre.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Simons Foundation
  • Home > Handbook > 

Last Updated

19/03/2026

These New AI Models Are Trained on Physics, Not Words, and They’re Driving Discovery

By: Elizabeth Fernandez
Fatiron Institute: 09/12/2025

Elizabeth Fernandez is a science writer specializing in science and society, science and philosophy, astronomy, physics, and geology.

While popular AI models such as ChatGPT are trained on language or photographs, new models created by researchers at the Flatiron Institute and other members of the Polymathic AI collaboration are trained using real scientific datasets. The models are already leveraging the knowledge they learn from one field to address seemingly completely different problems in another.

The Walrus AI model simulates fluid motion.

Walrus/Polymathic AI

While most AI models — including ChatGPT — are trained on text and images, a multidisciplinary team of scientists has something different in mind: AI trained on physics.

Recently, members of the Polymathic AI collaboration presented two new AI models trained using real scientific datasets to tackle problems in astronomy and fluidlike systems.

The models — called Walrus and AION-1 — are unique in that they can apply the knowledge they gain from one class of physical systems to seemingly completely different problems. For instance, Walrus can tackle systems ranging from exploding stars to Wi-Fi signals to the movement of bacteria.

That cross-disciplinary skill set is particularly exciting because it can accelerate scientific discovery and give researchers a leg up when faced with small samples or budgets, says Walrus lead developer Michael McCabe, a research scientist at Polymathic AI.

“Maybe you have new physics in your scenario that your field isn’t used to handling. Maybe you’re using experimental data, and you’re not quite sure what class it fits into. Maybe you’re just not a machine-learning researcher and just can’t burn the time working through all the possible models that might fit your scenario,” McCabe explains. “Our hope is that training on these broader classes makes something that is both easier to use and has a better chance of generalizing for those users, as the ‘new’ physics to them might be something another field has been handling for a while.”

Using cross-disciplinary models can also improve predictions when data is sparse or when studying rare events, says Liam Parker, a Ph.D. student at the University of California, Berkeley, and a lead researcher developing for AION-1.

The Polymathic AI team recently announced Walrus in a preprint on arXiv.org and presented AION-1 on Friday, December 5, at the NeurIPS conference in San Diego.

Walrus and AION-1 are ‘foundational models,’ meaning they’re trained on colossal sets of training data from different research areas or experiments. That’s unlike most AI models in science, which are trained with a particular subfield or problem in mind. Rather than learning the ins and outs of a particular situation or starting from a set of fundamental equations, foundational models instead learn the basis, or foundation, of the physical processes at work. Since these physical processes are universal, the knowledge that the AI learns can be applied to various fields or problems that share the same underlying physical principles. Foundational models have a host of benefits — from speeding up computations to performing well in low-data regimes to finding physics shared across different fields.

AION-1 is a foundational model for astronomy. It is trained on data from astronomical surveys that are already massive in their own right: the Legacy Survey, the Hyper Suprime-Cam (HSC), the Sloan Digital Sky Survey (SDSS), the Dark Energy Spectroscopic Instrument (DESI) and Gaia. All in all, that’s more than 200 million observations of stars, quasars and galaxies totaling around 100 terabytes of data. AION-1 uses images, spectra and a variety of other measurements to learn as much as it can about astronomical objects. Then, when a scientist obtains a low-resolution image of a galaxy, for example, AION-1 can extract more information about it, learned from the physics of millions of other galaxies.

Walrus’ domain is fluids and fluidlike systems. Walrus utilizes the Well — a massive dataset compiled by the Polymathic AI team. The Well’s data encompasses 19 different scenarios and 63 different fields in fluid dynamics. All in all, it contains 15 terabytes of data describing parameters such as density, velocity and pressure in physical systems as wide-ranging as merging neutron stars, acoustic waves and shifting layers in Earth’s atmosphere.

Such foundational models can be powerful. AION-1 and Walrus can utilize physics seen in a different case and apply it to learn about something new. It is similar to our senses. “Multiple senses together — rather than one at a time — gives you a fuller understanding of an experience,” the AION-1 team explained in a blog post about the project. “Over time, your brain learns associations between how things look, taste and smell, so if one sense is unavailable, you can often infer the missing information from the others.”

Then, when a scientist is performing a new experiment or observation, they have a starting point — a map of how physics behaves in other similar situations. “It’s like seeing many, many humans,” says Shirley Ho, Polymathic AI’s principal investigator and an astrophysicist and machine learning expert. Ho is a senior research scientist at the Flatiron Institute and a professor at New York University. When “you meet a new friend, because you’ve met so many people before now, you are able to map in your head … what this human is going to be like compared to all your friends before,” she says.

Foundational models make scientists’ lives easier by streamlining data processing. Scientists will no longer have to create a new framework from scratch for every project or task; instead, they can start with an already trained AI to use as a foundation. “I think our vision for some of this foundation model is that it enables anyone to start from a really powerful embedding of the data that they’re interested in … and still achieve state-of-the-art accuracy without having to build this whole pipeline from scratch,” says AION-1 lead researcher Parker.

Their goal is to make tools that scientists can use in their day-to-day research. “We want to bring all this AI intelligence” to the scientists who need it, Ho says.

Other Highlights From the NeurIPS 2025 Conference

CosmoBench: CosmoBench is a multiview, multiscale, multitask cosmology benchmark for geometric deep learning. Curated from the state-of-the-art cosmological simulations, CosmoBench is the largest benchmark of its kind, with over 34,000 point clouds and 25,000 directed trees. CosmoBench features challenging evaluation tasks from cosmology and diverse baselines, including cosmological methods, simple linear models and graph neural networks. This presentation will show how CosmoBench is pushing the frontiers of cosmology and geometric deep learning.

Lost in Latent Space: Physicists model and predict the behavior of physical systems using their understanding of the laws of physics. However, these calculations require significant computing power. Flatiron Institute scientists and other members of the Polymathic AI collaboration studied whether a less taxing form of computing can still yield accurate results. Known as ‘latent diffusion modeling,’ this computational model utilizes artificial intelligence to generate high-quality images at a lower computational cost while accurately capturing physical behavior.

Neurons as Detectors of Coherent Sets in Sensory Dynamics: Our perception of touch, taste, sight and pain is mediated by neurons that carry signals from peripheral receptors to the brain. This work shows that these neurons can be understood as detecting ‘coherent sets’ within the sensory stream — groups of stimulus trajectories that evolve together over time and therefore share a common past or a common future. By distinguishing these coherent sets, some neurons predominantly encode what has just occurred, while others reliably signal what is likely to happen next. Traditional classifications of sensory neurons can thus be reinterpreted as reflecting a division between past-focused and future-predictive processing. Understanding how the nervous system separates and transforms sensory input in this way may offer new routes for treating mental illness and may also guide the development of biologically inspired artificial intelligence.

Predicting Partially Observable Dynamical Systems: Scientists can predict the motion of a falling object or the evolution of fluids using deterministic models that compute a single future outcome from past observations. But this approach breaks down for physical systems where much of the state is hidden. A prominent example is the sun: We can observe the activity on its surface, but the processes deep inside remain largely invisible. Without access to those internal conditions, there isn’t enough information to forecast a single ‘correct’ future. Researchers at the Flatiron Institute, together with collaborators in the Polymathic AI project, have developed a probabilistic approach that can infer these hidden solar processes. By incorporating information from the distant past into a diffusion-based generative model, their method produces an ensemble of plausible futures, offering a clearer understanding of how past sunspot activity shapes its future evolution.

By , No comments:
Tags , , , ,

Having a Data Centre changes the roadmap

Mike's Notes

This is the revised Pipi roadmap now that the data centre is running. The recent Ajabbi Research report, "The Workspace Issue," has also been revised to reflect these changes.

Update 25/03/2026

A very nice chap from Google contacted me to assist with applying to join one of the Google AI Accelerators. I started the application, then stopped when I realised that the 2 years of support and generous free credits started as soon as it was approved. I need to complete all Stage 1 steps in the roadmap below, and the Stage 2 IaC connection to the GCP free tier to deploy Pipi open-source before applying, to make the best use of the opportunity.

I'm requesting support from Google DeepMind to experiment with connecting Pipi via MCP to DeepMind Gemini and to find a way for Pipi closed-core to use Google TPU. Stage 3 will be highly experimental with unexpected results.

Pipi is a non-generative multi-agent system with no tokens. It doesn't need tokens to work and its 27 layers deep so far and counting. I expect it to get very barnacled and crusty over time.

Update 31/03/2026

Setting up data centre automation has revealed that workspace deployments need to be performed in this order of account types due to the permissions cascade.

  • Agent Accounts (to admin Pipi)
  • Researcher Accounts (to edit UoM, ontologies and physical laws)
  • Developer Accounts (to create enterprise workspaces)
  • Personal Accounts (to sort out personal profiles - UI fonts, etc)
  • Enterprise Accounts (to test and use the system for work)
  • etc

Update 17/04/2026

Containers, named Pipi Nest, were created to deploy Pipi.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

17/04/2026

Having a Data Centre changes the roadmap

By: Mike Peters
On a Sandy Beach: 27/02/2026

Mike is the inventor and architect of Pipi and the founder of Ajabbi.

Having a separate Pipi Core Data Centre changes everything. This has a direct impact on what is possible and on the best path forward. 

This is the revised working Deployment Roadmap for Pipi. The sequence is roughly correct; the timing is just a guess.

Stage 1

Production foundations to enable automation and rapid use.

When Task Detail
February 2026 Migration ✔ Separate networks for the isolated Pipi Core Data Centre and the Ajabbi Office.
March - April 2026 Pipi Nest ✔ Pipi deployment container
April 2026 Mission Control ✔ Telemetry on the Data Centre.
April 2026 AutomationPipi Core running autonomously in a Pipi Nest 24x7x365, with a x10 increase in productivity.
May 2026 Data Centre Admin Agent Workspace rendered by the CMS Engine with no errors.
May 2026 Workspace Workspaces containing 15,000 web pages are rendered by the CMS Engine with no errors.
2026 Workspace Workspace UI Menu rendered without errors.
2026 Workspace Draft in-context help & learning material generated for each workspace without errors.
2026 Workspace Module-based UI forms and data grids rendered with sample data without errors.
2026 Workspace Complete HTML workspace demo with connected initial developer documentation rendered without error
2026 Workspaces for Agents & Developers Workspaces are deployed in the Data Centre for production use by System Admin and DevOps, with a x10 increase in productivity.
2026

2026

2026
2026
2026

Stage 2

Automate Pipi deployment across every available cloud platform, with a free tier.

When Task Detail
2026 IaC Deploy infrastructure on the AWS Free Tier.
2026 IaC Deploy infrastructure on the Azure Free Tier.
2026 IaC Deploy infrastructure on the Digital Ocean Free Tier.
2026 IaC Deploy infrastructure on the GCP Free Tier.
2026 IaC Deploy infrastructure on the IBM Free Tier.
2026 IaC Deploy infrastructure on the Oracle Free Tier.
2026 IaC Deploy infrastructure on the Wasabi Free Tier.
2026
...

Stage 3

Deploy to any platform when admitted to its startup program that offers lots of credits to enable experimentation and scaling with early customers.

When Task Detail

IaC Deploy infrastructure on every non-free part of the platform.

IaC BoxLang VM containing Pipi Open-Source, deployed via IaC.
  MCP, A2A, Skills Pipi > LLM > Pipi
  Working Demo Fully working demo workspaces available for customers and developers to try out.
  Customer Deployments Ajabbi Personal, Developer, and Enterprise account Workspaces are available.
  Customer Deployments Ajabbi Researcher and SME account Workspaces are available.
  Scientific Workflows TPU
 
 

Stage 4

At a certain threshold, Pipi 10 will come into being, creating many more possibilities.
By , No comments:
Tags , , , ,
Subscribe to: Posts (Atom)