Showing posts with label API. Show all posts
Showing posts with label API. Show all posts

AWS and Google Cloud Preview Secure Multicloud Networking

Mike's Notes

The Connection Coordinator API Specification may be essential to use if it becomes widely adopted. It depends a bit on the pricing model.

Resources

References

  • Connection Coordinator API Specification

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

13/03/2026

AWS and Google Cloud Preview Secure Multicloud Networking

By: Renato Losio
InfoQ: 25/12/2025

Renato has extensive experience as a cloud architect, tech lead, and cloud services specialist. Currently, he lives in Berlin and works remotely as a principal cloud architect. His primary areas of interest include cloud services and relational databases. He is an editor at InfoQ and a recognized AWS Data Hero. You can connect with him on LinkedIn.

In a surprising move, AWS and Google Cloud have recently partnered to simplify multicloud networking, introducing a common standard and leveraging "AWS Interconnect - Multicloud" and "Google Cloud's Cross-Cloud Interconnect". The new option makes it easier for organizations to manage and secure workloads across both clouds, with Azure expected to join in 2026.

Currently in preview, the solution combines AWS Interconnect – Multicloud with Google Cloud’s Cross-Cloud Interconnect and defines an open interoperability specification that other cloud providers can also adopt. Designed to avoid managing circuits, routers, and routing configurations, the new option is intended to simplify the deployment of secure multicloud workloads, enabling customers to establish private, high-speed connectivity between Google Cloud and AWS.

Available on GitHub, the Connection Coordinator API Specification describes the OpenAPI 3.0 specification for the symmetric API used to coordinate managed L3 connectivity. Rob Enns, VP of cloud networking at Google Cloud, and Robert Kennedy, VP of network services at AWS, write:

"Previously, to connect cloud service providers, customers had to manually set up complex networking components including physical connections and equipment (...) This could take weeks or even months. AWS had a vision for developing this capability as a unified specification that could be adopted by any cloud service provider, and collaborated with Google Cloud to bring it to market."

The new solution targets customers who want to run workloads across distributed regions, have low-bandwidth needs, and want cross-cloud connectivity without managing physical infrastructure. One of the common concerns among practitioners is the pricing of the solution, which has not been disclosed yet, with Corey Quinn, chief cloud economist at The Duckbill Group, writing:

"This is either transformative or a waste of everyone's time, and it's impossible to tell which because the one thing that matters most to settling that question is "what's the price." They aren't disclosing it yet, so at the moment it occupies a superposition of "excellent/crap." Please collapse the waveform so we know which one it is."

According to AWS documentation, the managed private connectivity service enables customers to define direct 1 Gbps connections between AWS VPCs and Google Cloud VPCs at no cost during the preview. Tyler Batts, senior customer ops engineer at Second Front, comments:

"It’s not in GovCloud yet, but the direction is obvious: AWS is baking multicloud into the platform instead of leaving teams to piece it together themselves (...) If you run serious workloads in the cloud, this is one of those updates worth paying attention to!"

All connections between the AWS and Google Cloud network devices are encrypted by default, and hardware is configured to transmit customer traffic only when the encryption session is active. Enns and Kennedy add:

Both providers engage in continuous monitoring to proactively detect and resolve issues. And this solution is built on a foundation of trust, utilizing MACsec encryption between the Google Cloud and AWS edge routers.

The preview is currently free and supports five AWS and Google Cloud regions in the US and Europe, including Northern Virginia, Oregon, and Frankfurt.

gRPC Clearly Explained

Mike's Notes

Good big picture explanation of gRPC. Pipi will have a dedicated gRPC Engine (grpc).

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Level Up Coding System Design
  • Home > Handbook > 

Last Updated

22/02/2026

gRPC Clearly Explained

By: Nikki Siapno
Level Up Coding System Design: 18/01/2026

Founder LUC | Eng Manager | ex-Canva | 400k+ audience | Helping you become a great engineer and leader.

What gRPC actually is, and the real reasons teams adopt it beyond “it’s faster”.

gRPC: What It Is and Why Teams Use It

Most teams reach for REST by habit.

It works, it’s everywhere, and every tool knows how to talk HTTP+JSON.

But once you have dozens of microservices calling each other thousands of times per request, REST can quietly become the bottleneck.

That’s where gRPC shows up with a very different model: “call a function on another service as if it were local,” and make it fast.

What gRPC (and RPC) really is

Before gRPC, there’s RPC.

Remote Procedure Call (RPC) is a model where you call a function that runs on another machine, but it feels local to the caller.

You call a method, pass arguments, and get a result back. The network exists, but it’s intentionally hidden so developers can think in terms of functions instead of sockets and packets.

gRPC is a modern, open-source RPC framework released by Google in 2015. It takes the RPC idea and standardizes how services define methods, exchange data, and communicate efficiently over the network.

Two building blocks drive almost everything you feel in practice:

  • HTTP/2 → The transport protocol. It keeps a long-lived connection and supports multiplexed streams.
  • Protocol Buffers (Protobuf) → The data format. It’s a compact binary serialization with a schema (a defined shape).

Together, these turn “call a service” into a strongly typed operation instead of a loosely structured document exchange.

How gRPC actually works

gRPC starts with a service definition. You describe your API in a .proto file by defining services (methods) and messages (request and response shapes).

This file is the contract. Both client and server are built from it.

From that contract, gRPC generates code:

  • Client stubs → Methods you call like local functions.
  • Server interfaces → Methods you implement with your business logic.

When a client calls a gRPC method, the flow is straightforward:

  1. Serialize → The request is encoded into Protobuf’s binary format.
  2. Send → The message travels over an existing HTTP/2 stream.
  3. Dispatch → The server routes it to the correct method.
  4. Execute → Your code runs.
  5. Respond → The result is serialized and streamed back.

Because gRPC uses HTTP/2, many calls share one connection and run in parallel. A slow response doesn’t block faster ones behind it, which keeps tail latency under control.

Streaming uses the same mechanism:

  • Server-streaming → One request, many responses over time.
  • Client-streaming → Many requests, one final response.
  • Bidirectional streaming → Both sides send messages independently.

Backpressure is built in. If one side slows down, gRPC slows the stream instead of piling up memory or threads.

The core idea is simple: define a strict contract, generate code from it, and move typed messages efficiently over a shared connection.

Where gRPC pays off

gRPC is a strong fit when you control both ends and you care about efficiency.

  • High call volume microservices → Lower per-call overhead adds up fast at scale.
  • Latency-sensitive graphs → Multiplexing + smaller payloads reduces tail latency pressure.
  • Polyglot stacks → One .proto contract generates stubs across languages, reducing “JSON drift.”
  • Service mesh environments → gRPC routes cleanly through modern proxies and is common in mesh control-plane protocols.

Tradeoffs you feel immediately

gRPC’s downsides are predictable, and they usually show up early on.

  • Browser calls are not native → You often need gRPC-Web or a REST/JSON gateway for front-end use.
  • Debugging is less “curl-friendly” → Binary payloads require tooling like grpcurl or GUI clients with schema access.
  • Contracts tighten coupling → Clients must update generated code as schemas evolve, so versioning discipline matters.
  • Infra must support HTTP/2 well → Some proxies and firewalls need explicit support or configuration.

How to Decide: Is gRPC a Fit for Your System?

Use gRPC when most of the following are true:

  • You control both client and server → Internal microservices inside the same organization.
  • You’re performance-sensitive → Many small calls per request, or very high QPS between services.
  • You’re polyglot → Multiple languages across teams and services.
  • You need streaming → Real-time updates, telemetry, chat, or continuous feeds.
  • You want strict contracts → You care about compile-time guarantees and explicit schemas.

Stick to REST (or layer a REST gateway in front of gRPC) when:

  • You expose public APIs to unknown clients.
  • You want easy browser and curl-based experimentation.
  • Your main pain is clarity and discoverability, not raw latency or throughput.

In practice, most teams don’t choose one or the other; they split the responsibility.

  • gRPC inside your network → Service-to-service, behind an API gateway or service mesh.
  • REST/JSON at the edge → For browsers, partners, and mobile apps that prefer HTTP+JSON.

Recap

gRPC is not “REST but faster.”

It’s a different model: remote procedure calls over HTTP/2, using Protobuf contracts, with first-class support for streaming and strong typing.

That makes it excellent for internal microservices, high-performance backends, and real-time systems, as long as you’re willing to invest in schemas, tooling, and a slightly steeper learning curve.

If you’re hitting the limits of REST inside your system (too many chatty JSON calls, tricky real-time updates, or a messy polyglot codebase) gRPC is worth a serious look.

Using Google Blogger API v3

Mike's Notes

On a Sandy Beach is a publication of Ajabbi Research.

Changes needed

Enable other people at Ajabbi Research to also contribute to On a Sandy Beach using the Workspaces for Research UI.

It looks very straightforward. This job will be done once the Workspaces for Research are available for researchers to use.

Preparation

Last year, all existing blog posts were reformatted.

New setup

The Pipi CMS Engine (cms) will format, store in a database, and export content to On a Sandy Beach, hosted on Google Blogger, using the Blogger API ver 3.

Either XML/Atom or JSON can be used.

Steps

  1. Import the existing blog posts into the CMS Engine via the Blogger API
  2. Reformat every post (see 4th resource link below)
  3. Store in the CMS database
  4. Render posts
  5. Export back to Blogger via the Blogger API.
  6. All future posts will be created by a human using a workspace, transferred to the CMS Engine, processed, and then published to Google Blogger.

Notes

Here are some initial notes taken from the Google Blogger API documentation and other sites, tweaked using Gemini and rewritten by Grammarly.

These notes will be presented using slides at tomorrow night's online Open Research Group meeting.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

09/02/2026

Using Google Blogger API v3

By: Mike Peters
On a Sandy Beach: 07/02/2026

Mike is the inventor and architect of Pipi and the founder of Ajabbi.

Blogger supports manual import and export of blog content via the Blogger dashboard or the Blogger API. The export file format for both methods is Atom, an XML-based format that includes all posts and comments. 

Blogger API for Import/Export

Developers can use the Blogger API (v3 is the current version) to programmatically manage blog content. The export file generated via the manual process uses the same Atom format as the API's feed requests. 

Key Points for API Use

  • Authentication: All operations for private data (including import/export) require authentication, typically using OAuth 2.0.
  • Format: The API uses REST APIs and JSON for standard operations. The specific import/export functions rely on the raw XML/Atom format described in the developer guides.
  • Functionality: The API allows retrieving, creating, updating, and deleting posts, comments, and pages, enabling the creation of custom import/export tools. For example, you can use API clients for Python, Java, or Node.js to manage content programmatically.
  • Custom Import Scripts: Developers can write scripts (e.g., in PHP or Python) to import content from other sources into Blogger via the API, which involves parsing the source data and making POST requests to Blogger API endpoints. 

For detailed API documentation, refer to the Blogger API Developer site.

REST

The Blogger API is a RESTful interface provided by Google that allows developers to integrate Blogger content and functionality into their own applications. It enables programmatic access to resources such as blogs, posts, comments, pages, and users via HTTP requests and JSON. 

Key Features and Concepts

  • Resources: The API revolves around five core resource types:
    • Blogs: The central container for all content and metadata.
    • Posts: The primary published content items, meant to be timely.
    • Comments: User reactions to specific posts.
    • Pages: Static content (e.g., about me, contact info).
    • Users: Represents a non-anonymous person interacting with Blogger, as an author, admin, or reader.
  • Operations: Developers can perform various operations on these resources, including list, get, insert (create), update, patch, and delete.
  • Authentication:
    • Public Data: Requests for public data (e.g., retrieving a public blog post) only require an API key.
    • Private Data: Operations involving private user data (e.g., creating a post, editing a comment) must be authorised using OAuth 2.0 tokens.
  • API Version: The latest recommended version is Blogger API v3. Support for the older v2.0 API ended on September 30, 2024, so applications must be updated to continue functioning. 

How to Access the API

Google recommends using their client libraries, which handle much of the authorisation and request/response processing for you. Libraries are available for a variety of programming languages: 

  • Go
  • Java (get started with the Java client library)
  • JavaScript
  • .NET (install the Google APIs NuGet package)
  • Node.js
  • PHP
  • Python (install the client library using pip install google-api-python-client)
  • Ruby 

Alternatively, you can interact with the API directly using RESTful HTTP requests. The Google APIs Explorer tool lets you test API calls in your browser. 

  • Pipi API Engine (api)  using CFML
  • BoxLang API using BX

For full documentation and developer guides, visit the official Blogger API documentation site on Google for Developers. 

Older Data API (v1/v2) Format: Atom XML 

The previous versions of the Blogger API relied heavily on the Atom Publishing Protocol (AtomPub) and Google Data API feeds for managing blog content. The API used standard HTTP methods (GET, POST, PUT, DELETE) to transport Atom-formatted XML payloads. 

Important: Support for the v2.0 Google Data API ended on September 30th, 2024, so applications must use the latest version to continue functioning. 

Exporting and Accessing Feeds via XML

Blogger still uses Atom XML for blog syndication and content backup: 

  • Public Blog Feed: Every Blogger blog has a public Atom feed, typically at an address like yourblogname.blogspot.com/feeds/posts/default or://yourblogname.blogspot.com.
  • Content Backup: Users can back up their blog's posts and comments as a single .xml file from the Blogger dashboard.

    • Sign in to Blogger.
    • Select your blog.
    • Go to Settings > Manage Blog.
    • Click Back up content and Download XML file. 
    • This downloaded file is in a specific Atom format for import and export. 

REST in the Blogger API

The supported Blogger operations map directly to REST HTTP verbs, as described in Blogger API operations.

The specific format for Blogger API URIs are:

  • https://www.googleapis.com/blogger/v3/users/userId
  • https://www.googleapis.com/blogger/v3/users/self
  • https://www.googleapis.com/blogger/v3/users/userId/blogs
  • https://www.googleapis.com/blogger/v3/users/self/blogs
  • https://www.googleapis.com/blogger/v3/blogs/blogId
  • https://www.googleapis.com/blogger/v3/blogs/byurl
  • https://www.googleapis.com/blogger/v3/blogs/blogId/posts
  • https://www.googleapis.com/blogger/v3/blogs/blogId/posts/bypath
  • https://www.googleapis.com/blogger/v3/blogs/blogId/posts/search
  • https://www.googleapis.com/blogger/v3/blogs/blogId/posts/postId
  • https://www.googleapis.com/blogger/v3/blogs/blogId/posts/postId/comments
  • https://www.googleapis.com/blogger/v3/blogs/blogId/posts/postId/comments/commentId
  • https://www.googleapis.com/blogger/v3/blogs/blogId/pages
  • https://www.googleapis.com/blogger/v3/blogs/blogId/pages/pageId

The full explanation of the URIs used and the results for each supported operation in the API is summarised in the Blogger API Reference document.

Examples

List the blogs that the authenticated user has access rights to:

  • GET https://www.googleapis.com/blogger/v3/users/self/blogs?key=YOUR-API-KEY

Get the posts on the code.blogger.com blog, which has blog ID 3213900:

  • GET https://www.googleapis.com/blogger/v3/blogs/3213900?key=YOUR-API-KEY

REST from JavaScript

You can invoke the Blogger API from JavaScript using the callback query parameter and a callback function. When the browser loads the script, the callback function is executed, and the response is passed to it. This approach allows you to write rich applications that display Blogger data without requiring server-side code.

The following example retrieves a post from the code.blogger.com blog, after you replace YOUR-API-KEY with your API key.

<html>
  <head>
    <title>Blogger API Example</title>
  </head>
  <body>
    <div id="content"></div>
    <script>
      function handleResponse(response) {
        document.getElementById("content").innerHTML += "<h1>" + response.title + "</h1>" + response.content;
      }
    </script>
    <script     src="https://www.googleapis.com/blogger/v3/blogs/3213900/posts/8398240586497962757?callback=handleResponse&key=YOUR-API-KEY"></script>
  </body>
</html>

Data format

JSON (JavaScript Object Notation) is a common, language-independent data format that provides a simple text representation of arbitrary data structures. For more information, see json.org.

Blogger API operations

You can invoke a number of different methods on collections and resources in the Blogger API, as described in the following table.

Operation Description REST HTTP mappings
list Lists all resources within a collection. GET on a collection URI.
get Gets a specific resource. GET on a resource URI.
getByUrl Gets a resource, looking it up by URL. GET with the URL passed in as a parameter.
getByPath Gets a resource by looking it up by its path. GET with the Path passed in as a parameter.
listByUser Lists resources owned by a User. GET on a user owned collection.
search Search for resources, based on a query parameter. GET on a Search URL, with the query passed in as a parameter.
insert Create a resource in a collection. POST on a collection URI.
delete Deletes a resource. DELETE on a resource URI.
patch Update a resource, using Patch semantics. PATCH on a resource URI.
update Update a resource. PUT on a resource URI.

The table below shows which methods are supported by each resource type. All list and get operations on private blogs require authentication.

Resource Type Supported Methods
list get getByUrl getByPath listByUser search insert delete patch update
Blogs N Y Y N Y N N N N N
Posts Y Y N Y N Y Y Y Y Y
Comments Y Y N N N N N N N N
Pages Y Y N N N N N N N N
Users N Y N N N N N N N N

A Critique of Iceberg REST Catalog: A Classic Case of Why Semantic Spec Fails

Mike's Notes

I like the specification and hope to enable its use via a future Pipi plugin. Here is an article by Ananth Packkildurai about some flaws that need fixing. His weekly newsletter is worth subscribing to.

Resources

References

  • Iceberg REST Catalog Specification.
  • Designing Data-Intensive Applications, by Martin Kleppmann.

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Data Engineering Weekly
  • Home > Handbook > 

Last Updated

19/01/2026

A Critique of Iceberg REST Catalog: A Classic Case of Why Semantic Spec Fails

By: Ananth Packkildurai
Data Engineering Weekly: 9/01/2026

Ananth Packkildurai is a data engineering leader, writer, and author of Data Engineering Weekly, sharing insights on modern data platforms, large-scale pipelines, and AI-driven architectures..

How a Semantically Correct API Becomes Operationally Unreliable at Scale.

“Latency is not just a performance characteristic; it is a fundamental part of correctness.” — Designing Data-Intensive Applications

In Designing Data-Intensive Applications, Martin Kleppmann makes a subtle but critical point: the CAP theorem omits latency, yet in real systems, latency often determines whether a system is usable at all. A system that is correct but slow is, in practice, incorrect.

This observation is directly applicable to the Apache Iceberg REST Catalog specification. While the specification achieves semantic clarity, it fails to define the operational realities that enable distributed systems to remain predictable at scale. The result is a standard that is formally correct, yet operationally fragile.

Semantic Interoperability Without Predictability

Over the past two years, the Iceberg REST Catalog specification has emerged as the de facto standard for metadata access in the Iceberg ecosystem. We have seen the outburst of the catalog war around the REST spec. It promises a universal interface that allows engines such as Trino, Spark, Flink, and StarRocks to interact with Iceberg tables via a common REST abstraction, independent of the underlying catalog implementation.

At the semantic level, this promise largely holds. The specification rigorously defines metadata structures: tables, schemas, snapshots, and namespace operations. A LoadTable or CreateNamespace request looks identical across implementations. This semantic interoperability has been critical to Iceberg’s rapid ecosystem adoption.

However, semantic interoperability alone is insufficient. The specification defines what metadata operations mean, but it avoids specifying how they must behave in real-world conditions, such as concurrency, latency sensitivity, and cross-catalog synchronization.

This gap—between semantic interoperability and operational interoperability—is where systems begin to fail in production.

The Core Problem: No Operational SLA, No Predictability

The Iceberg REST Catalog specification is intentionally silent on performance guarantees. There are no latency expectations, no throughput baselines, and no service-level objectives. While this flexibility lowers the barrier to implementation, it creates an ecosystem where:

  • Two catalogs can both be “compliant” yet differ by orders of magnitude in response time.
  • Clients cannot reason about metadata latency during query planning.
  • Synchronization behavior across catalogs becomes unpredictable.

In distributed data systems, predictability matters more than raw performance. Without a strict operational SLA—or at least defined behavioral constraints—clients are forced into defensive, retry-heavy designs that amplify load and increase tail latency.

The “List Tables” Problem: Cross-Catalog Sync Failure

The ListTables endpoint (GET /v1/namespaces/{namespace}/tables) is semantically straightforward. It allows clients to enumerate tables within a namespace and supports pagination through pageSize and pageToken.

The primary issue is not pagination itself. The real failure emerges when the same Iceberg tables are registered in multiple catalogs, a pattern that is increasingly common in hybrid and multi-platform deployments.

A Realistic Scenario

  • An Iceberg table is registered in Catalog A and Catalog B
  • Both catalogs point to the same underlying metadata and object storage.
  • One catalog is used by ingestion and streaming workloads.
  • Analytics engines or BI tools use the other.

The Sync Pathology

When a client connects to Catalog B and issues a metadata discovery operation—such as listing tables or syncing namespace state—the catalog must:

  1. Enumerate all tables
  2. Resolve metadata pointers
  3. Validate access permissions
  4. Reconcile the state with the underlying storage.

Because the REST specification defines no operational expectations:

  • There is no SLA for how long this sync should take
  • There is no distinction between a “lightweight” listing and a fully validated listing.
  • There is no mechanism to express intent (e.g., names only, no ACL validation)

As table counts grow into the tens of thousands, synchronization latency grows non-linearly. In practice, sync operations can take minutes—or fail—causing engines to stall, time out, or repeatedly retry.

The result is not merely slow metadata access. It is system-wide unpredictability. Query engines cannot determine whether a delay is transient, systemic, or catastrophic.

Latency Is Treated as an Implementation Detail—But It Is a Contract

The REST Catalog specification implicitly treats latency as an implementation concern. From a standards perspective, this is understandable. But in data-intensive systems, latency is part of the correctness contract.

The specification does not define:

  • Upper bounds on metadata retrieval latency
  • Maximum metadata payload sizes
  • Limits on metadata fan-out operations
  • The number of round-trip required to plan a query

As a result, a compliant catalog may require megabytes of JSON metadata and dozens of HTTP calls just to validate a single query plan. Engines appear slow and unstable, even though the root cause lies in an underspecified protocol.

This is precisely the class of problem Kleppmann warns about: correctness without latency guarantees is operationally meaningless.

Commit Semantics Under Contention: Undefined and Unfair

Iceberg relies on optimistic concurrency control. When multiple writers attempt to commit simultaneously, conflicts are expected and resolved through retries.

The REST specification defines the 409 Conflict response, but stops there. It does not define:

  • Backoff expectations
  • Retry fairness
  • Starvation prevention

In a multi-engine environment, this creates asymmetric outcomes. A high-frequency streaming writer with aggressive retries can permanently starve batch compaction jobs that follow conservative retry policies. Over time, table health degrades due to file explosion and unbounded metadata growth.

Once again, the issue is not semantic correctness. It is the absence of operational guarantees.

Caching Without a Freshness Model

While HTTP caching is permitted, it is not part of the correctness model. Support for conditional requests, ETags, or freshness validation is optional.

This forces clients into a pessimistic stance: always re-fetch, always revalidate, always assume staleness. The REST protocol degenerates into a chatty, high-latency control plane that negates its own architectural benefits.

Without a standardized freshness contract, caching becomes a gamble rather than a reliability tool.

Behavioral Conformance Is Missing

The Iceberg ecosystem has strong conformance testing for table formats. It lacks an equivalent for catalog behavior.

Today, “REST Catalog compliant” means:

  • The endpoints exist
  • The JSON schema is correct.
  • The happy path works.

It does not mean:

  • Predictable latency under load
  • Stable pagination during concurrent updates
  • Graceful overload signaling
  • Bounded retry amplification

Without behavioral conformance tests, compliance guarantees syntax, not operability.

Underspecification Is Still a Design Decision

The absence of operational constraints is not accidental. It reflects a deliberate choice to prioritize adoption and flexibility.

However, in distributed systems, underspecification pushes complexity downstream. It burdens clients, operators, and platform teams with the need to implement compensating logic. As Iceberg becomes core infrastructure rather than experimental tooling, this trade-off increasingly limits its reliability.

Semantic agreement without behavioral agreement leads to fragile systems.

Toward Operational Interoperability

Operational interoperability does not require rigid SLAs or centralized control. It requires acknowledging that latency, retries, and fairness are part of the interface.

Concrete improvements could include:

  • Defined operational profiles with minimum latency and concurrency expectations
  • Lightweight metadata views to avoid synchronization amplification
  • Standardized retry and backoff semantics for conflict scenarios
  • Explicit freshness and caching contracts

Semantic interoperability enabled Iceberg’s success. Operational interoperability will determine whether it remains dependable at scale.

Until then, the Iceberg REST Catalog remains a textbook example of why semantic specifications alone are not enough.

What is Diátaxis and should you be using it with your documentation?

Mike's Notes

I rediscovered this article by Technical Writer Tom Johnson while figuring out how to get the Pipi 9 CMS Engine (cms) to successfully integrate support material and learning documentation into the workspaces.

The actual Pipi 9 code base has no comments. Pipi writes the code and adds a metadata stamp, so why would it need comments? All documentation sits in parallel structures. This closes some doors and opens others.

Pipi 9 uses a structured combination of;

  • Read the Docs
  • Learning Objects
  • Diátaxis
  • The Good Docs Project

Information Pattern

Inspired by Read the Docs.

The whole is composed of Learning Objects in composable hierarchical structures. Each Learning Object

  • Is of one Diátaxis type
  • Uses one Good Docs Template

Learning material is presented in this default order because apparently it works for most people.

Tutorials > How-to guides > Technical reference > Explanation

Users can, however, explore learning material in any way that works for them and set it as a learning preference.

Show, not tell 

In my case, I like to see a diagram, then watch a talk with slides because I'm curious, then read the reference so that I can reverse-engineer to build a working model in my head, then a demo for the ahaha moment to confirm the model. Then an explanation to fill in the gaps, then a how-to guide and finally a tutorial to find the start button.😊 Worked examples are essential, because it gives me a pattern to start with and then change.

We all learn differently.

Ajabbi will need an excellent Technical Writer to improve what I have written, providing top-notch documentation and reducing complexity for readers.

Resources

References

  • Documenting APIs: A guide for technical writers and engineers. By Tom Johnson.

Repository

  • Home > Ajabbi Research > Library > Subscriptions > I'd rather be writing
  • Home > Ajabbi Research > Library > Learning > Learn API Docs
  • Home > Handbook > 

Last Updated

08/12/2025

What is Diátaxis and should you be using it with your documentation?

By: Tom Johnson
I'd rather be writing: 18/10/2023

I'm an API technical writer based in the Seattle area. On this blog, I write about topics related to technical writing and communication — such as software documentation, API documentation, AI, information architecture, content strategy, writing processes, plain language, tech comm careers, and more. Check out my API documentation course if you're looking for more info about documenting APIs. Or see my posts on AI and AI course section for more on the latest in AI and tech comm.

Summary

The Diátaxis approach to documentation organizes technical documentation into four types — tutorials, how-tos, reference, and explanation. In this post, I compare Diátaxis to DITA, Information Mapping, and the Good Docs Project, explaining similarities and differences. I also point out why identifying information patterns can be so worthwhile as a technical writer, and how identifying these patterns not only grounds our practice in the larger practice of rhetoric but also gives us useful patterns to use with AI tools.

Introduction

One topic I keep seeing surface in various places online is Diátaxis. Here are a few places I’ve seen it surface:

  • Upcoming webinars — see Introducing the Diátaxis Approach to Technical Documentation on the Content Wrangler’s BrightTALK
  • PyCon talks — see What nobody tells you about documentation
  • Reddit threads — see Diátaxis, a pragmatic system for technical documentation writing
  • Tutorials that mention it — see API tutorials beyond OpenAPI
  • Discussions among Python doc groups — see Adopting the Diátaxis framework for Python documentation, and more.
  • You can read more about it here: Diátaxis. It seems Diátaxis is gaining some traction in the technical writing community.

I knew Diátaxis involves organizing docs into four specific content types: explanation, reference, tutorials, and tasks. But I was fuzzy on most other specifics. The constant references to Diátaxis made me wonder, what am I missing? Is there something new here? Is this how I should be approaching the structure of my own documentation? And why is Diátaxis growing in popularity? What’s unique about the approach, and how does it differ from DITA?

Diataxis diagram

What is Diátaxis?

The Diátaxis approach divides documentation into four distinct content types:

  • Tutorials - Lessons that provide a learning experience, taking users step-by-step through hands-on exercises to build skills and familiarity.
  • How-To Guides - Practical guides focused on providing the steps to solve real-world problems.
  • Reference - Technical descriptions and factual information about the system, APIs, parameters, etc.
  • Explanation - Background information and conceptual discussions that provide context and illuminate topics more broadly.

The key premise of Diátaxis is that each content type serves a different user need and has a distinct purpose. Keeping them separated allows the content to be tailored and structured appropriately for that specific goal.

What does “Diátaxis” mean?

The name Diátaxis derives from Ancient Greek roots meaning “arrangement” or “layout.” It combines the Greek prefix dia, meaning “across”, with taxis, meaning “arrangement.”

This linguistic root relates to the organizational nature of the Diátaxis documentation approach. At its core, Diátaxis provides guidance for thoughtfully structuring documentation content into different categories based on user needs. The Diátaxis approach takes a jumble of disparate content and systematically arranges it into a meaningful, structured information architecture optimized around user goals.

Why follow the Diátaxis approach?

Here are some key benefits and promises of the Diátaxis approach:

  • Helps users find what they need more easily since content is organized by their goals.
  • Allows writers to focus on the type of content instead of wrestling with how to fit it into a less structured documentation system.
  • Serves both beginners and experts more effectively by separating learning content from reference.
  • Prevents muddling of content types, such as conceptual explanations in a how-to guide or instructions within a reference doc.
  • Provides a consistent, structured model for organizing documentation across products.
  • Promotes better quality content overall by keeping each type focused on its primary user need.

Who is Daniele Procida?

Daniele Procida developed the Diátaxis approach to documentation. Although he has a background in philosophy, Procida has been in the tech industry for a decade now, and has been involved in open-source software for even longer. He is currently an Engineering Director at Canonical.

Comparing Diátaxis with other approaches

As an approach for documentation, Diátaxis shares some similarities with other models.

How does Diátaxis compare with DITA?

DITA users will see quick comparisons with task, concept, and reference, but beyond these overlaps, DITA and Diátaxis actually have a lot of differences:

  • DITA is a formal XML-based standard maintained by the OASIS standards organization. It consists of XML schemas that content must validate against. Diátaxis isn’t an XML schema and doesn’t require validation.
  • DITA has a strong ecosystem of CMS tools and publishing systems built around its XML architecture. Diátaxis is not tied to specific tooling.
  • DITA defines a few more core topic types than Diátaxis, including troubleshooting, glossary, and generic topics. Diátaxis focuses on just four main types: tutorial, how-to, reference, and explanation.
  • DITA allows creating custom topic types through a specialization mechanism. Diátaxis doesn’t have extensibility for new types, as it’s not an XML schema.

Diátaxis has a more prescriptive recommended information architecture based on inherent user needs. DITA is more agnostic about optimal structures. For example, you can assemble the tasks, concepts, and reference types into whatever arrangements you want.

DITA emphasizes content reusability through topic-based authoring. The core idea is that componetizing information into these building blocks allows for efficient reuse across different deliverables (PDF, web, and more). Diátaxis isn’t concerned about content reusability and re-use across multiple outputs.

How does Diátaxis compare with Information Mapping?

DITA also shares some origins with Information Mapping, a technique developed by Robert Horn in the late 1960s. Although many details about Information Mapping are restricted by a paywall, here’s a brief comparison of Diátaxis and information mapping based on the information from Iva Cheung’s Introduction to Information Mapping post.

Cheung says Information Mapping identifies the following information types:

  • procedure—e.g., instructions on how to do something
  • process—e.g., description of how something works
  • principle—e.g., description of a standard or a convention
  • concept—e.g., description of a new idea or object
  • structure—e.g., description of an object’s components
  • fact—e.g., empirical information

You can see similarities here with other approaches. Diátaxis’ explanation is similar to principles and concepts. Reference might related to structure. How-to guides might relate to procedures and processes. I’m not sure what “fact” is, but perhaps it shares intent with tutorials.

Cheung says information management uses these three principles:

  • Chunking: group information into small, manageable chunks.
  • Relevance: limit each group or “unit of information” to a single topic, purpose, or idea.
  • Labelling: give each unit of information a meaningful name.

Here Information Mapping seems more similar to DITA than Diátaxis, in that information is broken down into small chunks with single ideas. The labels, however, seem unique to Information Mapping and interestingly seem to share commonality with embedding techniques used when preparing information for LLMs.

How does Diátaxis compare to The Good Docs project?

Both the Diátaxis approach and The Good Docs Project are focused on identifying best practice patterns and templates for technical documentation, but they approach it differently.

Diátaxis defines 4 core content types — tutorial, how-to, reference, and explanation. In contrast, The Good Docs Project has developed a set of templates mapped to various technical doc types and needs, including:

  • API quickstart
  • API reference
  • Code of conduct
  • Explanation
  • How-to
  • Installation guide
  • Logging
  • Our Team template
  • Overview
  • Quickstart
  • Style guide
  • Release notes
  • Troubleshooting
  • Tutorial

While Diátaxis focuses on the high-level IA, Good Docs provides more tactical templates and writing guides tailored to each type.

Good Docs templates help authors avoid “blank page anxiety” and provide examples with embedded best practices. In contrast, Diátaxis provides more general principles to help guide what should go in each content type.

Why is Diátaxis so popular?

I believe Diataxis is gaining popularity in part because it reveals discernible information patterns within documentation. The approach defines distinct information types, which helps writers recognize structures for organizing content tailored to specific user goals. By defining various information patterns, the Diátaxis helps writers shape and organize content.

The concept of information patterns extends far beyond documentation. For example:

  • Blog posts often follow story patterns.
  • Academic papers have a standard IMRaD pattern (intro, methods, results, discussion).
  • White papers use problem-solution patterns.
  • Knowledge base articles need clear question-focused patterns.
  • Speeches rely on patterns like the three-part list.
  • Legal documents leverage definitions, clauses, sections.
  • Marketing emails use subject lines, preview text, calls-to-action.

Seeing these patterns is like glimpsing the matrix behind communication. Understanding and applying patterns is at the core of rhetoric and document design.

The ability to identify and leverage information patterns in communication is central to the study of rhetoric. Rhetoric is often misunderstood as meaning language intended to manipulate or persuade. But its original definition is using language and information structuring techniques to fit a particular purpose, audience, or situation.

This is why so many technical communication academic programs are housed in rhetoric departments. The rhetorical tradition is fundamentally concerned with patterns of communication — how to shape content to achieve goals like persuading, informing, or motivating an audience.

Technical communicators are modern practitioners of rhetoric. Our job is fitting content to the expected discourse for optimal communication, comprehension, and usability.

Breaking down the Diátaxis information patterns

Let’s briefly look at the information patterns that Procida makes explicit. We’ll look at each information type and describe the salient characteristics.

Diátaxis information patterns

Information patterns in tutorials

In the Diátaxis approach to documentation, a tutorial is “a lesson, safely in the hands of an instructor” that provides a guided, hands-on learning experience.

Whereas DITA focuses on tasks/procedures, tutorials have a different rhetorical shape tailored for education rather than solving problems. Some key elements of a tutorial pattern are as follows:

  • Introduction/Learning goals - States what the user will accomplish in the tutorial.
  • Prerequisites - Lists required knowledge/tools.
  • Step-by-step instructions - Provides hands-on exercises and activities.
  • Recap - Summarizes key lessons and takeaways.
  • Assessment - Questions or exercises to check understanding.
  • Next steps - Points to additional resources for more learning.

This clear rhetorical pattern serves the unique goal of building skills interactively. The lesson shape provides scaffolding and direction ideal for beginners.

(By the way, I’m not sure why “tutorials” isn’t an information type in DITA. The tutorial fills an important niche for learning content in technical communication. Maybe the DITA specification group felt that tasks include tutorials.)

Information patterns in explanation content

In Diátaxis, explanation content provides background and context to illuminate a concept. The goal is deeper understanding rather than problem solving. The following are some common elements of the explanation pattern:

  • Introduction - States the purpose and previews the discussion.
  • Definition - Formal definition of the concept.
  • Background - Provides history and context.
  • Details - Elaborates on various aspects of the concept.
  • Visuals - Diagrams, illustrations, examples that clarify.
  • Relationships - Connects concepts to other ideas and approaches.
  • Implications - Explores effects, outcomes, and significance.
  • Summary - Recaps the key points about the concept.
  • Further Reading - Additional resources to learn more.

This rhetorical structure moves from overview to specifics in order to paint a rich picture of a topic. The goal is synthesizing disparate details into a cohesive narrative about the concept.

Information patterns in reference

In the Diátaxis approach, reference content follows its own distinct pattern serving a dedicated purpose. Procida states, “Reference guides are technical descriptions of the machinery and how to operate it.”

Whereas tutorials focus on learning, reference focuses on factual details about a product or system. Some hallmarks of the reference pattern are:

  • Overviews - High-level summaries of a component’s purpose and capabilities.
  • Specifications - Precise technical details of inputs, outputs, configurations, etc.
  • Options - Charts comparing different options or versions.
  • Parameters - Lists defining all available parameters and their usage.
  • Code samples - Snippets demonstrating implementation and usage.
  • Visuals - Diagrams illustrating technical concepts and relationships.

The reference pattern emphasizes comprehensive, accurate information presentation. The goal is precise and authoritative descriptions users can consult to understand the machinery while working with it.

Information patterns in how-to guides

In Diataxis, how-to guides follow a recipe-like shape focused on solving real-world problems. How-to guides help readers work their way through a problem-field.

Whereas reference provides technical details, how-to guides show practical application. For example, see this topic from the Django documentation: How to configure and use logging.

Some common elements of the how-to pattern include:

  • Prerequisites - Required knowledge or conditions to solve the problem.
  • Intro - If needed, a more detailed description of what the guide will cover.
  • Ordered steps - Sequential instructions to solve the problem, to the extent that’s possible.
  • Visuals - Diagrams and illustrations supporting the steps.
  • Examples - Specific use cases demonstrating the problem and solution.
  • Variations - How the steps may differ under certain conditions.
  • Recap - Summary of what was accomplished.
  • Related links - Pointers to other related procedures.

This structured pattern provides problem-solving assistance distinct from conceptual information. The goal is unambiguous guidance users can follow to solve a particular problem.

Objections about separating content by type

In learning about Diátaxis, I was concerned about the separation of content into distinct groups. Siloing documentation into tutorials, how-tos, reference, and explanation seemed overly opinionated and arbitrary as an information model. What research was this information model based on?

I reached out to Daniele on the #diataxis WTD Slack channel, and he clarified that Diátaxis isn’t meant to impose four rigid buckets that content must squeeze into. Rather, it’s an analytical approach that emerges from identifying four core user needs. In other words, when you identify the user’s information needs, these four fundamental types of documentation arise from those needs: tutorials, explanation, references, and how-to guides.

He acknowledges the critique that people don’t strictly separate these modes, but says that documentation itself should still be clear about its purpose and stabilize around meeting specific user needs.

So the core argument is:

  • Users have different needs from docs.
  • These needs can be mapped to 4 categories.
  • If you write according to those needs, patterns will start to emerge, and …

Thus, 4 types of documentation naturally form to serve those needs.

This model may oversimplify things, but the 4 Diataxis types are still useful as an abstract approach for thinking about docs, even if divisions aren’t absolute in practice.

Experiments at work

In my documentation at work, I recently separated out some concepts by type. For example, when working on reference materials for map data concepts, I initially had conceptual explanations scattered throughout the docs. This made it hard for users to find the conceptual information they needed.

So I decided to group all the conceptual topics into a central “Map Data Concepts” section — like a Wikipedia for map terms and concepts. This created a reliable one-stop shop for reference on key concepts. I now have a space to continue adding more concepts, and I don’t have to worry about reusing similar concepts in different API overviews.

Although I was initially reluctant to separate out concepts from the API overviews, this turned out to be a good move. Both conceptual definitions and overviews probably fit under “Explanation,” but even more granular separation of content by different explanation types seemed helpful.


All-in-one platform to create, share, and manage your documentation

Clear technical documentation, complex translation, and content operations

Will information organization still be necessary when users interface with docs through AI tools?

At this point, note that I’m steering away from explanations of Diátaxis and introducing my own ideas.

With the rise of AI, I think the information architecture of help content may become less critical in the future. Most users will likely interface with documentation primarily through chatbots and other AI tools rather than navigating a complex help system. I wrote about this in AI chat interfaces could become the primary user interface to read documentation

In a way, this reduces the need to obsess over the perfect organizational schema. ChatGPT, Claude, and other AI agents can deliver hyper-personalized help on demand, without requiring the user to know where some piece of information lives in the docs. The AI interface essentially abstracts away the information architecture or pattern and just surfaces the most relevant content dynamically. This makes the documentation’s organization less important from a user perspective.

That said, the actual documentation source still needs to be well-structured for the AI to “learn” from it effectively. But the end user probably won’t care as much, if the AI delivers the right information to them directly.

How can information patterns be used with AI prompting techniques?

In the context of AI, there’s another major benefit to the Diátaxis information model. You can more easily create structured prompts that ask an AI to sort and arrange unstructured information into specific information patterns. I wrote about this in Use cases for AI: Arrange content into information type patterns.

Let’s say that you gather a large body of content about the Widget API from internal documents, code, threads, and more. You can then supply this corpus of unstructured content to an AI and ask it to arrange the relevant information into an information template like this:

{Intro}

{Prerequisites}

{Problem to solve}

{Ordered steps}

{Substeps}

{Examples}

{Expected outcome}

{Related links}

You could even include descriptions of each template section. The AI tool will then pick out the relevant information from the large body of material and arrange it into the pattern you defined. This can significantly speed time to a first draft.

Conclusion

In conclusion, should you be using Diátaxis? Even if you don’t separate content by type, defining and shaping content into these four content types might help improve your documentation. It’s an easy-to-understand approach to documentation that draws power and appeal from this simplicity. You can learn more at https://diataxis.fr.

Related resources

See this BrightTALK video Introducing the Diátaxis Approach to Technical Documentation.

I wrote this post with some AI assistance.