Using Health Terminology Standards

Mikes Notes

Several duplicating international efforts are underway to standardise health Terminology. The health industry applications built on Pipi will first be based on SNOWMED CT and then made as interoperable with many other standards as possible using an open API.

These applications will be made freely available on GitHub Repositories.

Over time, this situation will change as standards change. It would be great for human health if, one day, health systems in every country used one international terminology system.

Ajabbi now has an account with SNOWMED CT in NZ.

Resources

As Wikipedia describes, these are some of the standards used today and some of the organisations involved.

Rendered by pipi

Mikes Notes

The bottom of each page or file contains a timestamp showing when Pipi rendered it. This is formatted according to the Pipi Render Naming  Convention.

Resources

Structure

  • Rendered:
  • 15:41 UTC, 16 June 2024
  • by Pipi 9.1.0.328

Rendered:

This means the web page or file was automatically created by a machine.

15:41 UTC, 16 June 2024

This is the Date and Time in UTC format.

by pipi 9.1.0.328

This is made up of "Pipi" and after one space "<major version>.<minor version>.<patch>.<build no>" .

Notes

The private pipi core instance name can be added when the platform core uses it.

Examples

Public

Rendered: 15:41 UTC, 16 June 2024 by Pipi 9.1.0.328

Private

Rendered: 15:41 UTC, 16 June 2024 by Pipi Freya 9.1.0.328

Designing a help system with Diátaxis

Mikes Notes

This month's job is to design and build a working help system for users. I'm using the Diátaxis approach.

Resources

What is Diátaxis?

"The Diátaxis approach divides documentation into four distinct content types:

  • Tutorials - Lessons that provide a learning experience, taking users step-by-step through hands-on exercises to build skills and familiarity.
  • How-To Guides - Practical guides focused on providing the steps to solve real-world problems.
  • Reference - Technical descriptions and factual information about the system, APIs, parameters, etc.
  • Explanation - Background information and conceptual discussions that provide context and illuminate topics more broadly.

The key premise of Diátaxis is that each content type serves a different user need and has a distinct purpose. Keeping them separated allows the content to be tailored and structured appropriately for that specific goal." - I'd rather be writing Blog

How-To-Guides

"A how-to guide is a recipe that shows the reader how to achieve something.

When the user is getting something done they are at work, applying their knowledge to a real task or problem. Now they need to be shown the key steps to take. What they need is a how-to guide.

Unlike a tutorial, a how-to guide is not responsible for providing a learning experience. It has no obligation to teach. It can assume that the reader has already acquired competence and familiarity with the machinery, its operation, the language used to talk about it and so on. Its obligation is to show a user how to get something done, step-by-step. It’s the obligation of the user to know what they want to get done, and to be able to apply the guide to the needs of their particular situation." - Daniele Procida

Example Outline by Daniele Procida

Overviews and introductory text

The content of a landing page itself should read like an overview.

That is, it should not simply present lists of other content, it should introduce them. Remember that you are always authoring for a human user, not fulfilling the demands of a scheme.

Headings and snippets of introductory text catch the eye and provide context; for example, a how-to landing page:

How to guides
=============

Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Installation guides
-------------------

Pellentesque malesuada, ipsum ac mollis pellentesque, risus
nunc ornare odio, et imperdiet dui mi et dui. Phasellus vel
porta turpis. In feugiat ultricies ipsum.

* Local installation      |
* Docker                        |  links to
* Virtual machines       |  the guides
* Linux containers       |

Deployment and scaling
-----------------------

Morbi sed scelerisque ligula. In dictum lacus quis felis
facilisisvulputate. Quisque lacinia condimentum ipsum
laoreet tempus.

* Deploy an instance        |  links to
* Scale your application   |  the guides

Examples of How-to-guides

robots.txt

Mikes Notes

The robots.txt goes to the root of every website. e.g.

  • www.example.com/robots.txt
  • https://example.com:8181/robots.txt
  • ftp://example.com/robots.txt

Resources

Description

"robots.txt is the filename used for implementing the Robots Exclusion Protocol, a standard used by websites to indicate to visiting web crawlers and other web robots which portions of the website they are allowed to visit.

The standard, developed in 1994, relies on voluntary compliance. Malicious bots can use the file as a directory of which pages to visit, though standards bodies discourage countering this with security through obscurity. Some archival sites ignore robots.txt. The standard was used in the 1990s to mitigate server overload. In the 2020s many websites began denying bots that collect information for generative artificial intelligence.

The "robots.txt" file can be used in conjunction with sitemaps, another robot inclusion standard for websites." - Wikipedia

Maximum size of a robots.txt file

The Robots Exclusion Protocol requires crawlers to parse at least 500 kibibytes (KiB) of robots.txt files, which Google maintains as a 500 kibibyte file size restriction for robots.txt files .

Examples

wildcard * stands for all robots
This example tells all robots to stay out of a website.
User-agent: *
Disallow: /:
This example tells all robots they can visit all files.
User-agent: *
Allow: /
This example tells all robots not to enter three directories.
User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /junk/
This example tells all robots to stay away from one specific file.
User-agent: *
Disallow: /directory/file.html
If you don't want crawlers to access sections of your site, you can create a robots.txt file with appropriate rules. A robots.txt file is a simple text file containing rules about which crawlers may access which parts of a site. For example, the robots.txt file for example.com may look like this.
# This robots.txt file controls crawling of URLs under https://example.com.
# All crawlers are disallowed to crawl files in the "includes" directory, such
# as .css, .js, but Google needs them for rendering, so Googlebot is allowed
# to crawl them.
User-agent: *
Disallow: /includes/

User-agent: Googlebot
Allow: /includes/

Sitemap: https://example.com/sitemap.xml

Data Model documentation options

Mikes Notes

Pipi has hundreds of autonomous databases, some with up to 60 tables. The existing Pipi entity engine stores every model, entity, relationship, and column Pipi uses. Developers need a web-based data model explorer to understand the databases they are using. The problem is how to design the user interface. This note examines how other modelling tools present information to develop something familiar and useful for Pipi.

Thanks, Alex, for sending me a list of more potential tools.

The experimental toy prototypes are here. (The web URLs will likely change in the future)

Resources

Requirements

A data model explorer needs to auto-generate documentation.

These objects are a minimum.
  • Database
  • Schema
  • E/R Diagram
  • Data Type
  • Table
  • Column
  • Data Domain
  • Relationship
  • Primary Keys (PK)
  • Foreign Keys (FK)
  • Indexes
And be able to provide these model views
  • Conceptual
  • Logical
  • Physical
Other
  • Data Dictionary including i18n
Linked data
The model objects need to automatically link to other documentation
  • DAO (Data Access Objects)
  • Ontologies
  • Use in workflows
  • Variables
  • "JavaDocs" style code packages
  • i18n object names

Examples

Database Workbench

DBeaver

dbDiagram

dbSchema



Drawio


DrawSQL



Enterprise Architect


ERDPlus


E/R Studio Data Architect

erwin Data Modeller


MS SQL Server Management Studio

MetaBase


Model Right

Navicat

QuickDBD

Software Ideas Modeller


SQLFlow

SQuirreL

SQLyog


Vertabelo Data Modeller




Fundamentals of Data Visualization

"As objective as data might be, there’s a human factor that is easily overseen when it comes to creating visualizations that accurately reflect it: bias and misunderstandings. Having worked with students and postdocs on thousands of data visualizations over the years, Claus O. Wilke, Professor of Integrative Biology, knows from experience that the same issues arise over and over when it comes to visualizing data."

"In his book Fundamentals of Data Visualization, he collected his accumulated knowledge from these interactions to help everyone create clear, attractive, and convincing data visualizations. You can read the complete manuscript for free on the author’s website." - Smashing Magazine.



Resources

Financial Times Visual Vocabulary

Mikes Notes

Smashing Magazine wrote about the Financial Times Visual Vocabulary, available on GitHub as a chart and a website in multiple languages.

"Violins, doughnuts, pies, slopes — data can be visualized in many ways. But which type of chart should you pick? To help you select the optimal visualization type for your data, the Financial Times Visual Journalism Team published the Financial Times Visual Vocabulary." - Smashing Magazine.

Resources

The material below is copied from the vocabulary.

Financial Times Visual Vocabulary

A poster (available in English, Japanese, traditional Chinese and simplified Chinese) and web site to assist designers and journalists to select the optimal symbology for data visualisations, by the Financial Times Visual Journalism Team.

The FT Visual Vocabulary is at the core of a newsroom-wide training session aimed at improving chart literacy. This learning resource is inspired by the Graphic Continuum by Jon Schwabish and Severino Ribecca. This is not an attempt to teach everyone how to make charts, but how to recognise the opportunities to use them effectively alongside words.

Read the Chart Doctor feature column for full background on why we made this: Simple techniques for bridging the graphics language gap

For D3 templates for producing many of these chart types in FT style, see our Visual Vocabulary repo.

Related reading

The full content of the poster, along with links to related material, including research and examples of best practice. This is a work in progress.

General

  • National Geographic: Taking data visualisation from eye candy to efficiency
  • William S. Cleveland and Robert McGill: Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods
  • Hadley Wickham: A Layered Grammar of Graphics
  • Tracey L. Weissgerber et al: Beyond Bar and Line Graphs: Time for a New Data Presentation Paradigm
  • Numeroteca: Uses and abuses of data visualisations in mass media
  • Andy Cotgreave: The inevitability of data visualization criticism
  • Alberto Cairo: "Our reader" won't understand something as complicated as that!
  • Alberto Cairo: Visualization's expanding vocabulary

Deviation

Emphasise variations (+/-) from a fixed reference point. Typically the reference point is zero but it can also be a target or a long-term average. Can also be used to show sentiment (positive/neutral/negative). Example FT uses: Trade surplus/deficit, climate change

Diverging bar

A simple standard bar chart that can handle both negative and positive magnitude values.

  • Chart Doctor: How the FT explained Brexit

Diverging stacked bar

Perfect for presenting survey results which involve sentiment (eg disagree/neutral/agree).

Spine chart

Splits a single value into 2 contrasting components (eg Male/Female)

Surplus/deficit filled line

The shaded area of these charts allows a balance to be shown – either against a baseline or between two series.

Correlation

Show the relationship between two or more variables. Be mindful that, unless you tell them otherwise, many readers will assume the relationships you show them to be causal (i.e. one causes the other). Example FT uses: Inflation & unemployment, income & life expectancy

  • Chart Doctor: The German election and the trouble with correlation

Scatterplot

The standard way to show the relationship between two continuous variables, each of which has its own axis.

  • Chart Doctor: The storytelling genius of unveiling truths through charts
  • Maarten Lambrechts: 7 reasons you should use dot graphs
  • Tim Brock: Too Big Data: Coping with Overplotting
  • Sara Kehaulani Goo: The art and science of the scatterplot
  • Chart Doctor: The storytelling genius of unveiling truths through charts
  • Examples: FT

Line + Column

A good way of showing the relationship between an amount (columns) and a rate (line)

  • Data Revelations: Be Careful with Dual Axis Charts
  • DataHero: The Do’s and Don’ts of Dual Axis Charts
  • Harvard Business Review: Beware Spurious Correlations

Connected scatterplot

Usually used to show how the relationship between two variables has changed over time.

  • Robert Kosara: The Connected Scatterplot for Presenting Paired Time Series
  • Data Revelations: Be Careful with Dual Axis Charts
  • Examples: Washington Post

Bubble

Like a scatterplot, but adds additional detail by sizing the circles according to a third variable

  • Chart Doctor: The storytelling genius of unveiling truths through charts
  • Examples: FT

XY heatmap

A good way of showing the patterns between 2 categories of data, less good at showing fine differences in amounts.

  • Chart Doctor: Use fewer maps to illustrate data better

Ranking

Use where an item’s position in an ordered list is more important than its absolute or relative value. Don’t be afraid to highlight the points of interest. Example FT uses: Wealth, deprivation, league tables, constituency election results

Ordered bar

Standard bar charts display the ranks of values much more easily when sorted into order

Ordered column

See above.

Ordered proportional symbol

Use when there are big variations between values and/or seeing fine differences between data is not so important.

Dot strip plot

Dots placed in order on a strip are a space-efficient method of laying out ranks across multiple categories.

Slope

Perfect for showing how ranks have changed over time or vary between categories.

Lollipop chart

Lollipops draw more attention to the data value than standard bar/column and can also show rank and value effectively.

Distribution

Show values in a dataset and how often they occur. The shape (or ‘skew’) of a distribution can be a memorable way of highlighting the lack of uniformity or equality in the data. Example FT uses: Income distribution, population (age/sex) distribution

  • Joey Cherdarchuk: Visualising distributions

Histogram

The standard way to show a statistical distribution - keep the gaps between columns small to highlight the ‘shape’ of the data

  • Aran Lunzer and Amelia McNamara: Exploring histograms

Boxplot

Summarise multiple distributions by showing the median (centre) and range of the data

Violin plot

Similar to a box plot but more effective with complex distributions (data that cannot be summarised with simple average).

Population pyramid

A standard way for showing the age and sex breakdown of a population distribution; effectively, back to back histograms.

Dot strip plot

Good for showing individual values in a distribution, can be a problem when too many dots have the same value.

Dot plot

A simple way of showing the change or range (min/max) of data across multiple categories.

Barcode plot

Like dot strip plots, good for displaying all the data in a table,they work best when highlighting individual values.

  • Maarten Lambrechts: Interactive strip plots for visualizing demographics

Cumulative curve

A good way of showing how unequal a distribution is: y axis is always cumulative frequency, x axis is always a measure.

Change over Time

Give emphasis to changing trends. These can be short (intra-day) movements or extended series traversing decades or centuries. Choosing the correct time period is important to provide suitable context for the reader. Example FT uses: Share price movements, economic time series

  • Flowing Data: 11 Ways to Visualize Changes Over Time – A Guide

Line

  • The standard way to show a changing time series. If data are irregular, consider markers to represent data points
  • Chart Doctor: A chart’s ability to mislead is off the scale
  • Office for National Statistics: Does the axis have to start at zero? (Part 1 – line charts)
  • Quartz: It's OK not to start your y-axis at zero
  • Vox: Shut up about the y-axis. It should't always start at zeroEmily Schuch: How to Make a Line Chart that Doesn't Lie

Column

Columns work well for showing change over time - but usually best with only one series of data at a time.

  • Chart Doctor: A chart’s ability to mislead is off the scale
  • Office for National Statistics: Does the axis have to start at zero? (Part 2 – bar charts)

Line + column

A good way of showing the relationship over time between an amount (columns) and a rate (line)

Stock price

Usually focused on day-to-day activity, these charts show opening/closing and hi/low points of each day

Slope

Good for showing changing data as long as the data can be simplified into 2 or 3 points without missing a key part of story

Area chart

Use with care – these are good at showing changes to total, but seeing change in components can be very difficult

Fan chart (projection)

Use to show the uncertainty in future projections - usually this grows the further forward to projection

Connected scatterplot

A good way of showing changing data for two variables whenever there is a relatively clear pattern of progression.

Calendar heatmap

A great way of showing temporal patterns (daily, weekly, monthly) – at the expense of showing precision in quantity.

Priestley timeline

Great when date and duration are key elements of the story in the data.

  • Chart Doctor: Communicating with data: Timelines
  • Examples: FT

Circle timeline

Good for showing discrete values of varying size across multiple categories (eg earthquakes by contintent).

Seismogram

Another alternative to the circle timeline for showing series where there are big variations in the data.

Part-to-whole

Show how a single entity can bebroken down into its component elements. If the reader’s interest issolely in the size of the components,consider a magnitude-type chartinstead. Example FT uses: Fiscal budgets, company structures,national election results

  • Flowing Data: 9 Ways to Visualize Proportions – A Guide

Stacked column

A simple way of showing part-to-whole relationships but can be difficult to read with more than a few components.

  • Robert Kosara: Stacked bars are the worst

Proportional stacked bar

A good way of showing the size and proportion of data at the same time – as long as the data are not too complicated.

  • Chart Doctor: How to apply Marimekko to data

Pie

A common way of showing part-to-whole data – but be aware that it’s difficult to accurately compare the size of the segments.

  • Robert Kosara: Ye olde pie chart debate
  • Robert Kosara: Pie Charts – Unloved, Unstudied, and Misunderstood
  • Robert Kosara: An Illustrated Tour of the Pie Chart Study Results
  • David Robinson: How to replace a pie chart
  • Office for National Statistics: The humble pie chart: part 1
  • Office for National Statistics: The humble pie chart: part 2
  • Ian Spence: No humble pie: The origins and usage of a statistical chart
  • Jeff Clark: In defense of pie charts
  • Stephen Few: Save the Pies for Dessert

Donut

Similar to a pie chart – but the centre can be a good way of making space to include more information about the data (eg. total)

Treemap

Use for hierarchical part-to-whole relationships; can be difficult to read when there are many small segments.

Voronoi

A way of turning points into areas – any point within each area is closer to the central point than any other centroid.

Arc

A hemicycle, often used for visualising political results in parliaments.

Gridplot

Good for showing % information, they work best when used on whole numbers and work well in multiple layout form.

Venn

Generally only used for schematic representation

Waterfall

Can be useful for showing part-to-whole relationships where some of the components are negative.

Magnitude

Show size comparisons. These can berelative (just being able to seelarger/bigger) or absolute (need tosee fine differences). Usually theseshow a ‘counted’ number (for example, barrels, dollars or people) rather thana calculated rate or per cent. Example FT uses: Commodity production, marketcapitalisation

Column

The standard way to compare the size of things. Must always start at 0 on the axis

Bar

See above. Good when the data are not time series and labels have long category names.

Paired column

As per standard column but allows for multiple series. Can become tricky to read with more than 2 series.

Paired bar

See above.

Proportional stacked bar

A good way of showing the size and proportion of data at the same time – as long as the data are not too complicated.

  • Chart Doctor: How to apply Marimekko to data

Proportional symbol

Use when there are big variations between values and/or seeing fine differences between data is not so important.

Isotype (pictogram)

Excellent solution in some instances – use only with whole numbers (do not slice off an arm to represent a decimal).

Lollipop chart

Lollipop charts draw more attention to the data value than standard bar/column – does not HAVE to start at zero (but preferable).

Radar chart

A space-efficient way of showing value pf multiple variables– but make sure they are organised in a way that makes sense to reader.

Parallel coordinates

An alternative to radar charts – again, the arrngement of the variables is important. Usually benefits from highlighting values.

Spatial

Used only when precise locations orgeographical patterns in data aremore important to the reader thananything else. Example FT uses: Locator maps, population density,natural resource locations, naturaldisaster risk/impact, catchment areas, variation in election results

  • Chart Doctor: Use fewer maps to illustrate data better
  • Matthew Ericson: When Maps Shouldn’t Be Maps
  • Mapbox: 7 data visualization techniques for location

Basic choropleth (rate/ratio)

The standard approach for putting data on a map – should always be rates rather than totals and use a sensible base geography

  • Vox: The bad map we see every presidential election
  • Vox: This “bad” election map? It’s not so bad.
  • UX•Blog: Telling the truth

Proportional symbol (count/magnitde)

Use for totals rather than rates – be wary that small differences in data will be hard to see.

  • Stephen Few: What Can’t Be Built with Bricks?

Flow map

For showing unambiguous movement across a map.

Contour map

For showing areas of equal value on a map. Can use deviation colour schemes for showing +/- values

Equalised cartogram

Converting each unit on a map to a regular and equally-sized shape – good for representing voting regions with equal value.

  • Chart Doctor: How the FT explained Brexit
  • 5W Blog: The power of cartograms and creating them easily

Scaled cartogram (value)

Stretching and shrinking a map so that each area is sized according to a particular value.

  • Chart Doctor: The search for a better US election map
  • 5W Blog: The power of cartograms and creating them easily
  • Vox: The bad map we see every presidential election

Dot density

Used to show the location of individual events/locations – make sure to annotate any patterns the reader should see.

  • Chart Doctor: The search for a better US election map

Heat map

Grid-based data values mapped with an intensity colour scale. As choropleth map – but not snapped to an admin/political unit.

  • 5W Blog: The power of cartograms and creating them easily

Flow

Show the reader volumes or intensity of movement between two or more states or conditions. These might belogical sequences or geographical locations. Example FT uses: Movement of funds, trade, migrants, lawsuits, information; relationship graphs.

  • RJ Andrews: Picturing the Great Migration

Sankey (aka river plot)

Shows changes in flows from one condition to at least one other; good for tracing the eventual outcome of a complex process.

  • Chart Doctor: Data visualisation: it is not all about technology

Waterfall

Designed to show the sequencing of data through a flow process, typically budgets. Can include +/- components.

Chord

A complex but powerful diagram which can illustrate 2-way flows (and net winner) in a matrix.

Network

Used for showing the strength and inter-connectedness of relationships of varying types.

Todo:

Uncertainty

  • Scientific American: Visualising uncertain weather
  • Oli Hawkins: Animating uncertainty

Animation

  • Chart Doctor: The storytelling genius of unveiling truths through charts
  • Evan Sinar: Use Animation to Supercharge Data Visualization

Interactivity

  • Chart Doctor: Why the FT creates so few clickable graphics
  • Gregor Aisch: In defense of interactive graphics
  • Zan Armstrong: Why choose? Scrollytelling and steppers)

Map projections

Colour